Transport API Reference

Log transport for LogEverything.

Network transports that ship structured log records from application processes to the dashboard or any central collector. Each transport is a logging.Handler subclass — just add it to any logger’s handler list.

Available transports: - HTTPTransportHandler (stdlib urllib, zero external deps) - TCPTransportHandler (persistent TCP, newline-delimited JSON) - UDPTransportHandler (fire-and-forget UDP datagrams)

LogBuffer

Shared batching / retry buffer for LogEverything transports.

Provides a thread-safe queue with a background flush thread, batching, exponential-backoff retry, and configurable back-pressure policy. Follows the same daemon-thread pattern as AsyncQueueHandler.

class logeverything.transport.buffer.LogBuffer(send_batch, max_buffer_size=10000, batch_size=100, flush_interval=5.0, max_retries=3, backpressure='drop')[source]

Bases: object

Thread-safe log buffer with background batched flushing.

Parameters:

  • send_batch (Callable[[List[Dict[str, Any]]], None]) – Callable that accepts a List[Dict] and ships them. Must raise on failure so the buffer can retry.

  • max_buffer_size (int) – Maximum number of records to hold in memory.

  • batch_size (int) – Number of records per batch sent to send_batch.

  • flush_interval (float) – Seconds between automatic flush attempts.

  • max_retries (int) – How many times to retry a failed batch (with backoff).

  • backpressure (Literal['drop', 'block']) – What to do when the buffer is full — "drop" silently discards the oldest records; "block" waits for space.

close()[source]

Flush remaining records and stop the background thread.

Return type:

None

flush()[source]

Immediately drain the buffer and send all batches.

Return type:

None

put(record)[source]

Add a serialised log record to the buffer.

Return type:

None

HTTP Transport

HTTP Transport Handler for LogEverything.

Ships structured log records to a dashboard or collector via HTTP POST using only the standard library (urllib.request). Zero external deps.

Usage:

from logeverything.transport.http import HTTPTransportHandler

handler = HTTPTransportHandler("http://dashboard:8999/api/ingest/logs")
logger.addHandler(handler)
class logeverything.transport.http.HTTPTransportHandler(endpoint, api_key=None, batch_size=50, flush_interval=2.0, source_name=None, max_retries=3, timeout=10.0, level=0)[source]

Bases: Handler

Logging handler that batches records and POSTs them as JSON to an HTTP endpoint.

Parameters:

  • endpoint (str) – URL to POST batched logs to (e.g. http://localhost:8999/api/ingest/logs).

  • api_key (Optional[str]) – Optional API key sent as Authorization: Bearer <key>.

  • batch_size (int) – Records per HTTP request.

  • flush_interval (float) – Seconds between automatic flushes.

  • source_name (Optional[str]) – Identifier for this application / process (defaults to PID).

  • max_retries (int) – Retry count for failed HTTP POSTs.

  • timeout (float) – HTTP request timeout in seconds.

close()[source]

Flush remaining records and release resources.

Return type:

None

emit(record)[source]

Serialize the record and queue it for batched delivery.

Return type:

None

flush()[source]

Flush the internal buffer immediately.

Return type:

None

TCP Transport

TCP Transport Handler for LogEverything.

Ships newline-delimited JSON over a persistent TCP socket. Suitable for high-reliability use within a private network.

Usage:

from logeverything.transport.tcp import TCPTransportHandler

handler = TCPTransportHandler("collector.internal", 5140)
logger.addHandler(handler)
class logeverything.transport.tcp.TCPTransportHandler(host, port, batch_size=100, flush_interval=5.0, source_name=None, max_retries=3, connect_timeout=5.0, level=0)[source]

Bases: Handler

Logging handler that sends newline-delimited JSON over a persistent TCP connection with automatic reconnection.

Parameters:

  • host (str) – Target host.

  • port (int) – Target port.

  • batch_size (int) – Records per write batch.

  • flush_interval (float) – Seconds between automatic flushes.

  • source_name (Optional[str]) – Identifier for this process.

  • max_retries (int) – Retry count for failed sends.

  • connect_timeout (float) – TCP connect timeout in seconds.

close()[source]

Tidy up any resources used by the handler.

This version removes the handler from an internal map of handlers, _handlers, which is used for handler lookup by name. Subclasses should ensure that this gets called from overridden close() methods.

Return type:

None

emit(record)[source]

Do whatever it takes to actually log the specified logging record.

This version is intended to be implemented by subclasses and so raises a NotImplementedError.

Return type:

None

flush()[source]

Ensure all logging output has been flushed.

This version does nothing and is intended to be implemented by subclasses.

Return type:

None

UDP Transport

UDP Transport Handler for LogEverything.

Fire-and-forget JSON datagrams over UDP. Suitable for high-throughput scenarios where occasional loss is acceptable.

Usage:

from logeverything.transport.udp import UDPTransportHandler

handler = UDPTransportHandler("collector.internal", 5141)
logger.addHandler(handler)
class logeverything.transport.udp.UDPTransportHandler(host, port, batch_size=100, flush_interval=2.0, source_name=None, max_packet_size=65000, level=0)[source]

Bases: Handler

Logging handler that sends JSON-encoded log records as UDP datagrams.

Each record is sent as a single datagram (up to ~64 KB). Batching is still used to amortise serialisation overhead, but each record in the batch is sent as its own datagram so loss is bounded.

Parameters:

  • host (str) – Target host.

  • port (int) – Target port.

  • batch_size (int) – Records to dequeue at once (each still sent individually).

  • flush_interval (float) – Seconds between automatic flushes.

  • source_name (Optional[str]) – Identifier for this process.

  • max_packet_size (int) – Maximum UDP payload size in bytes. Records larger than this are silently dropped.

close()[source]

Tidy up any resources used by the handler.

This version removes the handler from an internal map of handlers, _handlers, which is used for handler lookup by name. Subclasses should ensure that this gets called from overridden close() methods.

Return type:

None

emit(record)[source]

Do whatever it takes to actually log the specified logging record.

This version is intended to be implemented by subclasses and so raises a NotImplementedError.

Return type:

None

flush()[source]

Ensure all logging output has been flushed.

This version does nothing and is intended to be implemented by subclasses.

Return type:

None