blob: 683d6ed5e8ba525739db836bb6f6aeca13ee8770 [file] [log] [blame]
Vinay Sajipc63619b2010-12-19 12:56:57 +00001:mod:`logging.config` --- Logging configuration
2===============================================
3
4.. module:: logging.config
5 :synopsis: Configuration of the logging module.
6
Vinay Sajipc63619b2010-12-19 12:56:57 +00007.. moduleauthor:: Vinay Sajip <vinay_sajip@red-dove.com>
8.. sectionauthor:: Vinay Sajip <vinay_sajip@red-dove.com>
9
Terry Jan Reedyfa089b92016-06-11 15:02:54 -040010**Source code:** :source:`Lib/logging/config.py`
11
Vinay Sajip01094e12010-12-19 13:41:26 +000012.. sidebar:: Important
13
14 This page contains only reference information. For tutorials,
15 please see
16
17 * :ref:`Basic Tutorial <logging-basic-tutorial>`
18 * :ref:`Advanced Tutorial <logging-advanced-tutorial>`
19 * :ref:`Logging Cookbook <logging-cookbook>`
20
Vinay Sajip31b862d2013-09-05 23:01:07 +010021--------------
22
Vinay Sajip01094e12010-12-19 13:41:26 +000023This section describes the API for configuring the logging module.
Vinay Sajipc63619b2010-12-19 12:56:57 +000024
25.. _logging-config-api:
26
27Configuration functions
28^^^^^^^^^^^^^^^^^^^^^^^
29
30The following functions configure the logging module. They are located in the
31:mod:`logging.config` module. Their use is optional --- you can configure the
32logging module using these functions or by making calls to the main API (defined
33in :mod:`logging` itself) and defining handlers which are declared either in
34:mod:`logging` or :mod:`logging.handlers`.
35
36.. function:: dictConfig(config)
37
38 Takes the logging configuration from a dictionary. The contents of
39 this dictionary are described in :ref:`logging-config-dictschema`
40 below.
41
42 If an error is encountered during configuration, this function will
43 raise a :exc:`ValueError`, :exc:`TypeError`, :exc:`AttributeError`
44 or :exc:`ImportError` with a suitably descriptive message. The
45 following is a (possibly incomplete) list of conditions which will
46 raise an error:
47
48 * A ``level`` which is not a string or which is a string not
49 corresponding to an actual logging level.
50 * A ``propagate`` value which is not a boolean.
51 * An id which does not have a corresponding destination.
52 * A non-existent handler id found during an incremental call.
53 * An invalid logger name.
54 * Inability to resolve to an internal or external object.
55
56 Parsing is performed by the :class:`DictConfigurator` class, whose
57 constructor is passed the dictionary used for configuration, and
58 has a :meth:`configure` method. The :mod:`logging.config` module
59 has a callable attribute :attr:`dictConfigClass`
60 which is initially set to :class:`DictConfigurator`.
61 You can replace the value of :attr:`dictConfigClass` with a
62 suitable implementation of your own.
63
64 :func:`dictConfig` calls :attr:`dictConfigClass` passing
65 the specified dictionary, and then calls the :meth:`configure` method on
66 the returned object to put the configuration into effect::
67
68 def dictConfig(config):
69 dictConfigClass(config).configure()
70
71 For example, a subclass of :class:`DictConfigurator` could call
72 ``DictConfigurator.__init__()`` in its own :meth:`__init__()`, then
73 set up custom prefixes which would be usable in the subsequent
74 :meth:`configure` call. :attr:`dictConfigClass` would be bound to
75 this new subclass, and then :func:`dictConfig` could be called exactly as
76 in the default, uncustomized state.
77
Vinay Sajip1a90f9c2011-01-27 19:14:16 +000078 .. versionadded:: 3.2
79
Vinay Sajipf97255f2011-04-19 13:56:39 +010080.. function:: fileConfig(fname, defaults=None, disable_existing_loggers=True)
Vinay Sajipc63619b2010-12-19 12:56:57 +000081
Vinay Sajipd8b24d72014-02-04 16:42:04 +000082 Reads the logging configuration from a :mod:`configparser`\-format file. The
83 format of the file should be as described in
84 :ref:`logging-config-fileformat`.
Vinay Sajipcf9e2f22012-10-09 09:06:03 +010085 This function can be called several times from an application, allowing an
86 end user to select from various pre-canned configurations (if the developer
87 provides a mechanism to present the choices and load the chosen
88 configuration).
89
90 :param fname: A filename, or a file-like object, or an instance derived
91 from :class:`~configparser.RawConfigParser`. If a
92 ``RawConfigParser``-derived instance is passed, it is used as
93 is. Otherwise, a :class:`~configparser.Configparser` is
94 instantiated, and the configuration read by it from the
95 object passed in ``fname``. If that has a :meth:`readline`
96 method, it is assumed to be a file-like object and read using
97 :meth:`~configparser.ConfigParser.read_file`; otherwise,
98 it is assumed to be a filename and passed to
99 :meth:`~configparser.ConfigParser.read`.
100
Vinay Sajipf97255f2011-04-19 13:56:39 +0100101
102 :param defaults: Defaults to be passed to the ConfigParser can be specified
103 in this argument.
104
105 :param disable_existing_loggers: If specified as ``False``, loggers which
106 exist when this call is made are left
Vinay Sajip12d9ddf2014-08-05 10:34:16 +0100107 enabled. The default is ``True`` because this
Martin Panter8f137832017-01-14 08:24:20 +0000108 enables old behaviour in a
109 backward-compatible way. This behaviour is to
Géry Ogamf0c74362019-01-23 08:12:39 +0100110 disable any existing non-root loggers unless
111 they or their ancestors are explicitly named
112 in the logging configuration.
Vinay Sajipc63619b2010-12-19 12:56:57 +0000113
Vinay Sajipcf9e2f22012-10-09 09:06:03 +0100114 .. versionchanged:: 3.4
115 An instance of a subclass of :class:`~configparser.RawConfigParser` is
116 now accepted as a value for ``fname``. This facilitates:
117
118 * Use of a configuration file where logging configuration is just part
119 of the overall application configuration.
120 * Use of a configuration read from a file, and then modified by the using
121 application (e.g. based on command-line parameters or other aspects
122 of the runtime environment) before being passed to ``fileConfig``.
Vinay Sajipc63619b2010-12-19 12:56:57 +0000123
Vinay Sajip4ded5512012-10-02 15:56:16 +0100124.. function:: listen(port=DEFAULT_LOGGING_CONFIG_PORT, verify=None)
Vinay Sajipc63619b2010-12-19 12:56:57 +0000125
126 Starts up a socket server on the specified port, and listens for new
127 configurations. If no port is specified, the module's default
128 :const:`DEFAULT_LOGGING_CONFIG_PORT` is used. Logging configurations will be
Vinay Sajip3d639b22016-09-11 13:15:06 +0100129 sent as a file suitable for processing by :func:`dictConfig` or
130 :func:`fileConfig`. Returns a :class:`~threading.Thread` instance on which
131 you can call :meth:`~threading.Thread.start` to start the server, and which
132 you can :meth:`~threading.Thread.join` when appropriate. To stop the server,
Vinay Sajipc63619b2010-12-19 12:56:57 +0000133 call :func:`stopListening`.
134
Vinay Sajip4ded5512012-10-02 15:56:16 +0100135 The ``verify`` argument, if specified, should be a callable which should
136 verify whether bytes received across the socket are valid and should be
137 processed. This could be done by encrypting and/or signing what is sent
138 across the socket, such that the ``verify`` callable can perform
139 signature verification and/or decryption. The ``verify`` callable is called
140 with a single argument - the bytes received across the socket - and should
Serhiy Storchakaecf41da2016-10-19 16:29:26 +0300141 return the bytes to be processed, or ``None`` to indicate that the bytes should
Vinay Sajip4ded5512012-10-02 15:56:16 +0100142 be discarded. The returned bytes could be the same as the passed in bytes
143 (e.g. when only verification is done), or they could be completely different
144 (perhaps if decryption were performed).
145
Vinay Sajipc63619b2010-12-19 12:56:57 +0000146 To send a configuration to the socket, read in the configuration file and
R David Murray48de2822016-08-23 20:43:56 -0400147 send it to the socket as a sequence of bytes preceded by a four-byte length
Vinay Sajipc63619b2010-12-19 12:56:57 +0000148 string packed in binary using ``struct.pack('>L', n)``.
149
Larry Hastings3732ed22014-03-15 21:13:56 -0700150 .. note::
151
152 Because portions of the configuration are passed through
Vinay Sajip393e23f2012-07-25 19:19:25 +0100153 :func:`eval`, use of this function may open its users to a security risk.
154 While the function only binds to a socket on ``localhost``, and so does
155 not accept connections from remote machines, there are scenarios where
156 untrusted code could be run under the account of the process which calls
157 :func:`listen`. Specifically, if the process calling :func:`listen` runs
158 on a multi-user machine where users cannot trust each other, then a
159 malicious user could arrange to run essentially arbitrary code in a
160 victim user's process, simply by connecting to the victim's
161 :func:`listen` socket and sending a configuration which runs whatever
162 code the attacker wants to have executed in the victim's process. This is
163 especially easy to do if the default port is used, but not hard even if a
Vinay Sajip4ded5512012-10-02 15:56:16 +0100164 different port is used). To avoid the risk of this happening, use the
165 ``verify`` argument to :func:`listen` to prevent unrecognised
166 configurations from being applied.
167
Vinay Sajip3d639b22016-09-11 13:15:06 +0100168 .. versionchanged:: 3.4
Vinay Sajip4ded5512012-10-02 15:56:16 +0100169 The ``verify`` argument was added.
Vinay Sajipc63619b2010-12-19 12:56:57 +0000170
Vinay Sajip3d639b22016-09-11 13:15:06 +0100171 .. note::
172
173 If you want to send configurations to the listener which don't
174 disable existing loggers, you will need to use a JSON format for
175 the configuration, which will use :func:`dictConfig` for configuration.
176 This method allows you to specify ``disable_existing_loggers`` as
177 ``False`` in the configuration you send.
178
179
Vinay Sajipc63619b2010-12-19 12:56:57 +0000180.. function:: stopListening()
181
182 Stops the listening server which was created with a call to :func:`listen`.
183 This is typically called before calling :meth:`join` on the return value from
184 :func:`listen`.
185
186
187.. _logging-config-dictschema:
188
189Configuration dictionary schema
190^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
191
192Describing a logging configuration requires listing the various
193objects to create and the connections between them; for example, you
194may create a handler named 'console' and then say that the logger
195named 'startup' will send its messages to the 'console' handler.
196These objects aren't limited to those provided by the :mod:`logging`
197module because you might write your own formatter or handler class.
198The parameters to these classes may also need to include external
199objects such as ``sys.stderr``. The syntax for describing these
200objects and connections is defined in :ref:`logging-config-dict-connections`
201below.
202
203Dictionary Schema Details
204"""""""""""""""""""""""""
205
206The dictionary passed to :func:`dictConfig` must contain the following
207keys:
208
209* *version* - to be set to an integer value representing the schema
210 version. The only valid value at present is 1, but having this key
211 allows the schema to evolve while still preserving backwards
212 compatibility.
213
214All other keys are optional, but if present they will be interpreted
215as described below. In all cases below where a 'configuring dict' is
216mentioned, it will be checked for the special ``'()'`` key to see if a
217custom instantiation is required. If so, the mechanism described in
218:ref:`logging-config-dict-userdef` below is used to create an instance;
219otherwise, the context is used to determine what to instantiate.
220
221* *formatters* - the corresponding value will be a dict in which each
222 key is a formatter id and each value is a dict describing how to
Vinay Sajip67f39772013-08-17 00:39:42 +0100223 configure the corresponding :class:`~logging.Formatter` instance.
Vinay Sajipc63619b2010-12-19 12:56:57 +0000224
225 The configuring dict is searched for keys ``format`` and ``datefmt``
226 (with defaults of ``None``) and these are used to construct a
Vinay Sajip67f39772013-08-17 00:39:42 +0100227 :class:`~logging.Formatter` instance.
Vinay Sajipc63619b2010-12-19 12:56:57 +0000228
BNMetrics18fb1fb2018-10-15 19:41:36 +0100229 .. versionchanged:: 3.8
230 a ``validate`` key (with default of ``True``) can be added into
231 the ``formatters`` section of the configuring dict, this is to
232 validate the format.
233
Vinay Sajipc63619b2010-12-19 12:56:57 +0000234* *filters* - the corresponding value will be a dict in which each key
235 is a filter id and each value is a dict describing how to configure
236 the corresponding Filter instance.
237
238 The configuring dict is searched for the key ``name`` (defaulting to the
239 empty string) and this is used to construct a :class:`logging.Filter`
240 instance.
241
242* *handlers* - the corresponding value will be a dict in which each
243 key is a handler id and each value is a dict describing how to
244 configure the corresponding Handler instance.
245
246 The configuring dict is searched for the following keys:
247
248 * ``class`` (mandatory). This is the fully qualified name of the
249 handler class.
250
251 * ``level`` (optional). The level of the handler.
252
253 * ``formatter`` (optional). The id of the formatter for this
254 handler.
255
256 * ``filters`` (optional). A list of ids of the filters for this
257 handler.
258
259 All *other* keys are passed through as keyword arguments to the
Martin Panter1050d2d2016-07-26 11:18:21 +0200260 handler's constructor. For example, given the snippet:
261
262 .. code-block:: yaml
Vinay Sajipc63619b2010-12-19 12:56:57 +0000263
264 handlers:
265 console:
266 class : logging.StreamHandler
267 formatter: brief
268 level : INFO
269 filters: [allow_foo]
270 stream : ext://sys.stdout
271 file:
272 class : logging.handlers.RotatingFileHandler
273 formatter: precise
274 filename: logconfig.log
275 maxBytes: 1024
276 backupCount: 3
277
278 the handler with id ``console`` is instantiated as a
279 :class:`logging.StreamHandler`, using ``sys.stdout`` as the underlying
280 stream. The handler with id ``file`` is instantiated as a
281 :class:`logging.handlers.RotatingFileHandler` with the keyword arguments
282 ``filename='logconfig.log', maxBytes=1024, backupCount=3``.
283
284* *loggers* - the corresponding value will be a dict in which each key
285 is a logger name and each value is a dict describing how to
286 configure the corresponding Logger instance.
287
288 The configuring dict is searched for the following keys:
289
290 * ``level`` (optional). The level of the logger.
291
292 * ``propagate`` (optional). The propagation setting of the logger.
293
294 * ``filters`` (optional). A list of ids of the filters for this
295 logger.
296
297 * ``handlers`` (optional). A list of ids of the handlers for this
298 logger.
299
300 The specified loggers will be configured according to the level,
301 propagation, filters and handlers specified.
302
303* *root* - this will be the configuration for the root logger.
304 Processing of the configuration will be as for any logger, except
305 that the ``propagate`` setting will not be applicable.
306
307* *incremental* - whether the configuration is to be interpreted as
308 incremental to the existing configuration. This value defaults to
309 ``False``, which means that the specified configuration replaces the
310 existing configuration with the same semantics as used by the
311 existing :func:`fileConfig` API.
312
313 If the specified value is ``True``, the configuration is processed
314 as described in the section on :ref:`logging-config-dict-incremental`.
315
Géry Ogamf0c74362019-01-23 08:12:39 +0100316* *disable_existing_loggers* - whether any existing non-root loggers are
317 to be disabled. This setting mirrors the parameter of the same name in
Vinay Sajipc63619b2010-12-19 12:56:57 +0000318 :func:`fileConfig`. If absent, this parameter defaults to ``True``.
319 This value is ignored if *incremental* is ``True``.
320
321.. _logging-config-dict-incremental:
322
323Incremental Configuration
324"""""""""""""""""""""""""
325
326It is difficult to provide complete flexibility for incremental
327configuration. For example, because objects such as filters
328and formatters are anonymous, once a configuration is set up, it is
329not possible to refer to such anonymous objects when augmenting a
330configuration.
331
332Furthermore, there is not a compelling case for arbitrarily altering
333the object graph of loggers, handlers, filters, formatters at
334run-time, once a configuration is set up; the verbosity of loggers and
335handlers can be controlled just by setting levels (and, in the case of
336loggers, propagation flags). Changing the object graph arbitrarily in
337a safe way is problematic in a multi-threaded environment; while not
338impossible, the benefits are not worth the complexity it adds to the
339implementation.
340
341Thus, when the ``incremental`` key of a configuration dict is present
342and is ``True``, the system will completely ignore any ``formatters`` and
343``filters`` entries, and process only the ``level``
344settings in the ``handlers`` entries, and the ``level`` and
345``propagate`` settings in the ``loggers`` and ``root`` entries.
346
347Using a value in the configuration dict lets configurations to be sent
348over the wire as pickled dicts to a socket listener. Thus, the logging
349verbosity of a long-running application can be altered over time with
350no need to stop and restart the application.
351
352.. _logging-config-dict-connections:
353
354Object connections
355""""""""""""""""""
356
357The schema describes a set of logging objects - loggers,
358handlers, formatters, filters - which are connected to each other in
359an object graph. Thus, the schema needs to represent connections
360between the objects. For example, say that, once configured, a
361particular logger has attached to it a particular handler. For the
362purposes of this discussion, we can say that the logger represents the
363source, and the handler the destination, of a connection between the
364two. Of course in the configured objects this is represented by the
365logger holding a reference to the handler. In the configuration dict,
366this is done by giving each destination object an id which identifies
367it unambiguously, and then using the id in the source object's
368configuration to indicate that a connection exists between the source
369and the destination object with that id.
370
Martin Panter1050d2d2016-07-26 11:18:21 +0200371So, for example, consider the following YAML snippet:
372
373.. code-block:: yaml
Vinay Sajipc63619b2010-12-19 12:56:57 +0000374
375 formatters:
376 brief:
377 # configuration for formatter with id 'brief' goes here
378 precise:
379 # configuration for formatter with id 'precise' goes here
380 handlers:
381 h1: #This is an id
382 # configuration of handler with id 'h1' goes here
383 formatter: brief
384 h2: #This is another id
385 # configuration of handler with id 'h2' goes here
386 formatter: precise
387 loggers:
388 foo.bar.baz:
389 # other configuration for logger 'foo.bar.baz'
390 handlers: [h1, h2]
391
392(Note: YAML used here because it's a little more readable than the
393equivalent Python source form for the dictionary.)
394
395The ids for loggers are the logger names which would be used
396programmatically to obtain a reference to those loggers, e.g.
397``foo.bar.baz``. The ids for Formatters and Filters can be any string
398value (such as ``brief``, ``precise`` above) and they are transient,
399in that they are only meaningful for processing the configuration
400dictionary and used to determine connections between objects, and are
401not persisted anywhere when the configuration call is complete.
402
403The above snippet indicates that logger named ``foo.bar.baz`` should
404have two handlers attached to it, which are described by the handler
405ids ``h1`` and ``h2``. The formatter for ``h1`` is that described by id
406``brief``, and the formatter for ``h2`` is that described by id
407``precise``.
408
409
410.. _logging-config-dict-userdef:
411
412User-defined objects
413""""""""""""""""""""
414
415The schema supports user-defined objects for handlers, filters and
416formatters. (Loggers do not need to have different types for
417different instances, so there is no support in this configuration
418schema for user-defined logger classes.)
419
420Objects to be configured are described by dictionaries
421which detail their configuration. In some places, the logging system
422will be able to infer from the context how an object is to be
423instantiated, but when a user-defined object is to be instantiated,
424the system will not know how to do this. In order to provide complete
425flexibility for user-defined object instantiation, the user needs
426to provide a 'factory' - a callable which is called with a
427configuration dictionary and which returns the instantiated object.
428This is signalled by an absolute import path to the factory being
429made available under the special key ``'()'``. Here's a concrete
Martin Panter1050d2d2016-07-26 11:18:21 +0200430example:
431
432.. code-block:: yaml
Vinay Sajipc63619b2010-12-19 12:56:57 +0000433
434 formatters:
435 brief:
436 format: '%(message)s'
437 default:
438 format: '%(asctime)s %(levelname)-8s %(name)-15s %(message)s'
439 datefmt: '%Y-%m-%d %H:%M:%S'
440 custom:
441 (): my.package.customFormatterFactory
442 bar: baz
443 spam: 99.9
444 answer: 42
445
446The above YAML snippet defines three formatters. The first, with id
447``brief``, is a standard :class:`logging.Formatter` instance with the
448specified format string. The second, with id ``default``, has a
449longer format and also defines the time format explicitly, and will
450result in a :class:`logging.Formatter` initialized with those two format
451strings. Shown in Python source form, the ``brief`` and ``default``
452formatters have configuration sub-dictionaries::
453
454 {
455 'format' : '%(message)s'
456 }
457
458and::
459
460 {
461 'format' : '%(asctime)s %(levelname)-8s %(name)-15s %(message)s',
462 'datefmt' : '%Y-%m-%d %H:%M:%S'
463 }
464
465respectively, and as these dictionaries do not contain the special key
466``'()'``, the instantiation is inferred from the context: as a result,
467standard :class:`logging.Formatter` instances are created. The
468configuration sub-dictionary for the third formatter, with id
469``custom``, is::
470
471 {
472 '()' : 'my.package.customFormatterFactory',
473 'bar' : 'baz',
474 'spam' : 99.9,
475 'answer' : 42
476 }
477
478and this contains the special key ``'()'``, which means that
479user-defined instantiation is wanted. In this case, the specified
480factory callable will be used. If it is an actual callable it will be
481used directly - otherwise, if you specify a string (as in the example)
482the actual callable will be located using normal import mechanisms.
483The callable will be called with the **remaining** items in the
484configuration sub-dictionary as keyword arguments. In the above
485example, the formatter with id ``custom`` will be assumed to be
486returned by the call::
487
488 my.package.customFormatterFactory(bar='baz', spam=99.9, answer=42)
489
490The key ``'()'`` has been used as the special key because it is not a
491valid keyword parameter name, and so will not clash with the names of
492the keyword arguments used in the call. The ``'()'`` also serves as a
493mnemonic that the corresponding value is a callable.
494
495
496.. _logging-config-dict-externalobj:
497
498Access to external objects
499""""""""""""""""""""""""""
500
501There are times where a configuration needs to refer to objects
502external to the configuration, for example ``sys.stderr``. If the
503configuration dict is constructed using Python code, this is
504straightforward, but a problem arises when the configuration is
505provided via a text file (e.g. JSON, YAML). In a text file, there is
506no standard way to distinguish ``sys.stderr`` from the literal string
507``'sys.stderr'``. To facilitate this distinction, the configuration
508system looks for certain special prefixes in string values and
509treat them specially. For example, if the literal string
510``'ext://sys.stderr'`` is provided as a value in the configuration,
511then the ``ext://`` will be stripped off and the remainder of the
512value processed using normal import mechanisms.
513
514The handling of such prefixes is done in a way analogous to protocol
515handling: there is a generic mechanism to look for prefixes which
516match the regular expression ``^(?P<prefix>[a-z]+)://(?P<suffix>.*)$``
517whereby, if the ``prefix`` is recognised, the ``suffix`` is processed
518in a prefix-dependent manner and the result of the processing replaces
519the string value. If the prefix is not recognised, then the string
520value will be left as-is.
521
522
523.. _logging-config-dict-internalobj:
524
525Access to internal objects
526""""""""""""""""""""""""""
527
528As well as external objects, there is sometimes also a need to refer
529to objects in the configuration. This will be done implicitly by the
530configuration system for things that it knows about. For example, the
531string value ``'DEBUG'`` for a ``level`` in a logger or handler will
532automatically be converted to the value ``logging.DEBUG``, and the
533``handlers``, ``filters`` and ``formatter`` entries will take an
534object id and resolve to the appropriate destination object.
535
536However, a more generic mechanism is needed for user-defined
537objects which are not known to the :mod:`logging` module. For
538example, consider :class:`logging.handlers.MemoryHandler`, which takes
539a ``target`` argument which is another handler to delegate to. Since
540the system already knows about this class, then in the configuration,
541the given ``target`` just needs to be the object id of the relevant
542target handler, and the system will resolve to the handler from the
543id. If, however, a user defines a ``my.package.MyHandler`` which has
544an ``alternate`` handler, the configuration system would not know that
545the ``alternate`` referred to a handler. To cater for this, a generic
Serhiy Storchaka46936d52018-04-08 19:18:04 +0300546resolution system allows the user to specify:
547
548.. code-block:: yaml
Vinay Sajipc63619b2010-12-19 12:56:57 +0000549
550 handlers:
551 file:
552 # configuration of file handler goes here
553
554 custom:
555 (): my.package.MyHandler
556 alternate: cfg://handlers.file
557
558The literal string ``'cfg://handlers.file'`` will be resolved in an
559analogous way to strings with the ``ext://`` prefix, but looking
560in the configuration itself rather than the import namespace. The
561mechanism allows access by dot or by index, in a similar way to
Serhiy Storchaka46936d52018-04-08 19:18:04 +0300562that provided by ``str.format``. Thus, given the following snippet:
563
564.. code-block:: yaml
Vinay Sajipc63619b2010-12-19 12:56:57 +0000565
566 handlers:
567 email:
568 class: logging.handlers.SMTPHandler
569 mailhost: localhost
570 fromaddr: my_app@domain.tld
571 toaddrs:
572 - support_team@domain.tld
573 - dev_team@domain.tld
574 subject: Houston, we have a problem.
575
576in the configuration, the string ``'cfg://handlers'`` would resolve to
577the dict with key ``handlers``, the string ``'cfg://handlers.email``
578would resolve to the dict with key ``email`` in the ``handlers`` dict,
579and so on. The string ``'cfg://handlers.email.toaddrs[1]`` would
580resolve to ``'dev_team.domain.tld'`` and the string
581``'cfg://handlers.email.toaddrs[0]'`` would resolve to the value
582``'support_team@domain.tld'``. The ``subject`` value could be accessed
583using either ``'cfg://handlers.email.subject'`` or, equivalently,
584``'cfg://handlers.email[subject]'``. The latter form only needs to be
585used if the key contains spaces or non-alphanumeric characters. If an
586index value consists only of decimal digits, access will be attempted
587using the corresponding integer value, falling back to the string
588value if needed.
589
590Given a string ``cfg://handlers.myhandler.mykey.123``, this will
591resolve to ``config_dict['handlers']['myhandler']['mykey']['123']``.
592If the string is specified as ``cfg://handlers.myhandler.mykey[123]``,
593the system will attempt to retrieve the value from
594``config_dict['handlers']['myhandler']['mykey'][123]``, and fall back
595to ``config_dict['handlers']['myhandler']['mykey']['123']`` if that
596fails.
597
Vinay Sajip817495a2011-08-11 13:45:48 +0100598
599.. _logging-import-resolution:
600
601Import resolution and custom importers
602""""""""""""""""""""""""""""""""""""""
603
604Import resolution, by default, uses the builtin :func:`__import__` function
605to do its importing. You may want to replace this with your own importing
606mechanism: if so, you can replace the :attr:`importer` attribute of the
607:class:`DictConfigurator` or its superclass, the
608:class:`BaseConfigurator` class. However, you need to be
609careful because of the way functions are accessed from classes via
610descriptors. If you are using a Python callable to do your imports, and you
611want to define it at class level rather than instance level, you need to wrap
612it with :func:`staticmethod`. For example::
613
614 from importlib import import_module
615 from logging.config import BaseConfigurator
616
617 BaseConfigurator.importer = staticmethod(import_module)
618
619You don't need to wrap with :func:`staticmethod` if you're setting the import
620callable on a configurator *instance*.
621
622
Vinay Sajipc63619b2010-12-19 12:56:57 +0000623.. _logging-config-fileformat:
624
625Configuration file format
626^^^^^^^^^^^^^^^^^^^^^^^^^
627
628The configuration file format understood by :func:`fileConfig` is based on
629:mod:`configparser` functionality. The file must contain sections called
630``[loggers]``, ``[handlers]`` and ``[formatters]`` which identify by name the
631entities of each type which are defined in the file. For each such entity, there
632is a separate section which identifies how that entity is configured. Thus, for
633a logger named ``log01`` in the ``[loggers]`` section, the relevant
634configuration details are held in a section ``[logger_log01]``. Similarly, a
635handler called ``hand01`` in the ``[handlers]`` section will have its
636configuration held in a section called ``[handler_hand01]``, while a formatter
637called ``form01`` in the ``[formatters]`` section will have its configuration
638specified in a section called ``[formatter_form01]``. The root logger
639configuration must be specified in a section called ``[logger_root]``.
640
Vinay Sajip2d965c22015-04-18 13:14:10 +0100641.. note::
642
643 The :func:`fileConfig` API is older than the :func:`dictConfig` API and does
644 not provide functionality to cover certain aspects of logging. For example,
645 you cannot configure :class:`~logging.Filter` objects, which provide for
646 filtering of messages beyond simple integer levels, using :func:`fileConfig`.
647 If you need to have instances of :class:`~logging.Filter` in your logging
648 configuration, you will need to use :func:`dictConfig`. Note that future
649 enhancements to configuration functionality will be added to
650 :func:`dictConfig`, so it's worth considering transitioning to this newer
651 API when it's convenient to do so.
652
Martin Panter1050d2d2016-07-26 11:18:21 +0200653Examples of these sections in the file are given below.
654
655.. code-block:: ini
Vinay Sajipc63619b2010-12-19 12:56:57 +0000656
657 [loggers]
658 keys=root,log02,log03,log04,log05,log06,log07
659
660 [handlers]
661 keys=hand01,hand02,hand03,hand04,hand05,hand06,hand07,hand08,hand09
662
663 [formatters]
664 keys=form01,form02,form03,form04,form05,form06,form07,form08,form09
665
666The root logger must specify a level and a list of handlers. An example of a
Martin Panter1050d2d2016-07-26 11:18:21 +0200667root logger section is given below.
668
669.. code-block:: ini
Vinay Sajipc63619b2010-12-19 12:56:57 +0000670
671 [logger_root]
672 level=NOTSET
673 handlers=hand01
674
675The ``level`` entry can be one of ``DEBUG, INFO, WARNING, ERROR, CRITICAL`` or
676``NOTSET``. For the root logger only, ``NOTSET`` means that all messages will be
677logged. Level values are :func:`eval`\ uated in the context of the ``logging``
678package's namespace.
679
680The ``handlers`` entry is a comma-separated list of handler names, which must
681appear in the ``[handlers]`` section. These names must appear in the
682``[handlers]`` section and have corresponding sections in the configuration
683file.
684
685For loggers other than the root logger, some additional information is required.
Martin Panter1050d2d2016-07-26 11:18:21 +0200686This is illustrated by the following example.
687
688.. code-block:: ini
Vinay Sajipc63619b2010-12-19 12:56:57 +0000689
690 [logger_parser]
691 level=DEBUG
692 handlers=hand01
693 propagate=1
694 qualname=compiler.parser
695
696The ``level`` and ``handlers`` entries are interpreted as for the root logger,
697except that if a non-root logger's level is specified as ``NOTSET``, the system
698consults loggers higher up the hierarchy to determine the effective level of the
699logger. The ``propagate`` entry is set to 1 to indicate that messages must
700propagate to handlers higher up the logger hierarchy from this logger, or 0 to
701indicate that messages are **not** propagated to handlers up the hierarchy. The
702``qualname`` entry is the hierarchical channel name of the logger, that is to
703say the name used by the application to get the logger.
704
705Sections which specify handler configuration are exemplified by the following.
Martin Panter1050d2d2016-07-26 11:18:21 +0200706
707.. code-block:: ini
Vinay Sajipc63619b2010-12-19 12:56:57 +0000708
709 [handler_hand01]
710 class=StreamHandler
711 level=NOTSET
712 formatter=form01
713 args=(sys.stdout,)
714
715The ``class`` entry indicates the handler's class (as determined by :func:`eval`
716in the ``logging`` package's namespace). The ``level`` is interpreted as for
717loggers, and ``NOTSET`` is taken to mean 'log everything'.
718
719The ``formatter`` entry indicates the key name of the formatter for this
720handler. If blank, a default formatter (``logging._defaultFormatter``) is used.
721If a name is specified, it must appear in the ``[formatters]`` section and have
722a corresponding section in the configuration file.
723
724The ``args`` entry, when :func:`eval`\ uated in the context of the ``logging``
725package's namespace, is the list of arguments to the constructor for the handler
726class. Refer to the constructors for the relevant handlers, or to the examples
Preston Landers6ea56d22017-08-02 15:44:28 -0500727below, to see how typical entries are constructed. If not provided, it defaults
728to ``()``.
729
730The optional ``kwargs`` entry, when :func:`eval`\ uated in the context of the
731``logging`` package's namespace, is the keyword argument dict to the constructor
732for the handler class. If not provided, it defaults to ``{}``.
Martin Panter1050d2d2016-07-26 11:18:21 +0200733
734.. code-block:: ini
Vinay Sajipc63619b2010-12-19 12:56:57 +0000735
736 [handler_hand02]
737 class=FileHandler
738 level=DEBUG
739 formatter=form02
740 args=('python.log', 'w')
741
742 [handler_hand03]
743 class=handlers.SocketHandler
744 level=INFO
745 formatter=form03
746 args=('localhost', handlers.DEFAULT_TCP_LOGGING_PORT)
747
748 [handler_hand04]
749 class=handlers.DatagramHandler
750 level=WARN
751 formatter=form04
752 args=('localhost', handlers.DEFAULT_UDP_LOGGING_PORT)
753
754 [handler_hand05]
755 class=handlers.SysLogHandler
756 level=ERROR
757 formatter=form05
758 args=(('localhost', handlers.SYSLOG_UDP_PORT), handlers.SysLogHandler.LOG_USER)
759
760 [handler_hand06]
761 class=handlers.NTEventLogHandler
762 level=CRITICAL
763 formatter=form06
764 args=('Python Application', '', 'Application')
765
766 [handler_hand07]
767 class=handlers.SMTPHandler
768 level=WARN
769 formatter=form07
770 args=('localhost', 'from@abc', ['user1@abc', 'user2@xyz'], 'Logger Subject')
Preston Landers6ea56d22017-08-02 15:44:28 -0500771 kwargs={'timeout': 10.0}
Vinay Sajipc63619b2010-12-19 12:56:57 +0000772
773 [handler_hand08]
774 class=handlers.MemoryHandler
775 level=NOTSET
776 formatter=form08
777 target=
778 args=(10, ERROR)
779
780 [handler_hand09]
781 class=handlers.HTTPHandler
782 level=NOTSET
783 formatter=form09
784 args=('localhost:9022', '/log', 'GET')
Preston Landers6ea56d22017-08-02 15:44:28 -0500785 kwargs={'secure': True}
Vinay Sajipc63619b2010-12-19 12:56:57 +0000786
Martin Panter1050d2d2016-07-26 11:18:21 +0200787Sections which specify formatter configuration are typified by the following.
788
789.. code-block:: ini
Vinay Sajipc63619b2010-12-19 12:56:57 +0000790
791 [formatter_form01]
792 format=F1 %(asctime)s %(levelname)s %(message)s
793 datefmt=
794 class=logging.Formatter
795
796The ``format`` entry is the overall format string, and the ``datefmt`` entry is
797the :func:`strftime`\ -compatible date/time format string. If empty, the
Vinay Sajip23cee802018-06-01 10:09:21 +0100798package substitutes something which is almost equivalent to specifying the date
799format string ``'%Y-%m-%d %H:%M:%S'``. This format also specifies milliseconds,
800which are appended to the result of using the above format string, with a comma
801separator. An example time in this format is ``2003-01-23 00:29:50,411``.
Vinay Sajipc63619b2010-12-19 12:56:57 +0000802
803The ``class`` entry is optional. It indicates the name of the formatter's class
804(as a dotted module and class name.) This option is useful for instantiating a
Vinay Sajip67f39772013-08-17 00:39:42 +0100805:class:`~logging.Formatter` subclass. Subclasses of
806:class:`~logging.Formatter` can present exception tracebacks in an expanded or
807condensed format.
Vinay Sajipc63619b2010-12-19 12:56:57 +0000808
Larry Hastings3732ed22014-03-15 21:13:56 -0700809.. note::
810
811 Due to the use of :func:`eval` as described above, there are
Vinay Sajip393e23f2012-07-25 19:19:25 +0100812 potential security risks which result from using the :func:`listen` to send
813 and receive configurations via sockets. The risks are limited to where
814 multiple users with no mutual trust run code on the same machine; see the
815 :func:`listen` documentation for more information.
816
Vinay Sajipc63619b2010-12-19 12:56:57 +0000817.. seealso::
818
819 Module :mod:`logging`
820 API reference for the logging module.
821
822 Module :mod:`logging.handlers`
823 Useful handlers included with the logging module.