Performance¶
structlog
’s default configuration tries to be as unsurprising to new developers as possible.
Some of the choices made come with an avoidable performance price tag – although its impact is debatable.
Here are a few hints how to get most out of structlog
in production:
Use a specific wrapper class instead of the generic one.
structlog
comes with ones for the Standard Library Logging and for Twisted:configure(wrapper_class=structlog.stdlib.BoundLogger)
structlog
also comes with native log levels that are based on the ones from the standard library (read: we’ve copy and pasted them), but don’t involvelogging
’s dynamic machinery. That makes them much faster. You can usestructlog.make_filtering_bound_logger()
to create one.Writing own wrapper classes is straightforward too.
Avoid (frequently) calling log methods on loggers you get back from
structlog.wrap_logger()
andstructlog.get_logger()
. Since those functions are usually called in module scope and thus before you are able to configure them, they return a proxy that assembles the correct logger on demand.Create a local logger if you expect to log frequently without binding:
logger = structlog.get_logger() def f(): log = logger.bind() for i in range(1000000000): log.info("iterated", i=i)
Set the cache_logger_on_first_use option to
True
so the aforementioned on-demand loggers will be assembled only once and cached for future uses:configure(cache_logger_on_first_use=True)
This has the only drawback is that later calls on
configure()
don’t have any effect on already cached loggers – that shouldn’t matter outside of testing though.Avoid sending your log entries through the standard library if you can: its dynamic nature and flexibiliy make it a major bottleneck. Instead use
structlog.PrintLoggerFactory
or – if your serializer returns bytes (e.g. orjson) –structlog.BytesLoggerFactory
.You can still configure
logging
for packages that you don’t control, but avoid it for your own log entries.Use a faster JSON serializer than the standard library. Possible alternatives are among others are orjson or RapidJSON.
Example¶
Here’s an example for a production-ready non-asyncio structlog
configuration that’s as fast as it gets:
import logging
import structlog
structlog.configure(
cache_logger_on_first_use=True,
wrapper_class=structlog.make_filtering_bound_logger(logging.INFO),
processors=[
structlog.threadlocal.merge_threadlocal_context,
structlog.processors.add_log_level,
structlog.processors.format_exc_info,
structlog.processors.TimeStamper(fmt="iso", utc=False),
structlog.processors.JSONRenderer(serializer=orjson.dumps),
],
logger_factory=structlog.BytesLoggerFactory(),
)
It has the following properties:
Caches all loggers on first use.
Filters all log entries below the
info
log level very efficiently. Thedebug
method literally consists ofreturn None
.Supports Thread Local Context.
Adds the log level name.
Renders exceptions.
Adds an ISO 8601 timestamp under the
timestamp
key in the UTC timezone.Renders the log entries as JSON using orjson which is faster than plain logging in
logging
.Uses
BytesLoggerFactory
because orjson returns bytes. That saves encoding ping-pong.
Therefore a log entry might look like this:
{"event":"hello","timestamp":"2020-11-17T09:54:11.900066Z"}
If you need standard library support for external projects, you can either just use a JSON formatter like python-json-logger, or pipe them through structlog
as documented in Standard Library Logging.