| Vinay Sajip | 5dbca9c | 2011-04-08 11:40:38 +0100 | [diff] [blame] | 1 | :mod:`logging.config` --- Logging configuration | 
|  | 2 | =============================================== | 
|  | 3 |  | 
|  | 4 | .. module:: logging.config | 
|  | 5 | :synopsis: Configuration of the logging module. | 
|  | 6 |  | 
|  | 7 |  | 
|  | 8 | .. moduleauthor:: Vinay Sajip <vinay_sajip@red-dove.com> | 
|  | 9 | .. sectionauthor:: Vinay Sajip <vinay_sajip@red-dove.com> | 
|  | 10 |  | 
|  | 11 | .. sidebar:: Important | 
|  | 12 |  | 
|  | 13 | This page contains only reference information. For tutorials, | 
|  | 14 | please see | 
|  | 15 |  | 
|  | 16 | * :ref:`Basic Tutorial <logging-basic-tutorial>` | 
|  | 17 | * :ref:`Advanced Tutorial <logging-advanced-tutorial>` | 
|  | 18 | * :ref:`Logging Cookbook <logging-cookbook>` | 
|  | 19 |  | 
|  | 20 | This section describes the API for configuring the logging module. | 
|  | 21 |  | 
|  | 22 | .. _logging-config-api: | 
|  | 23 |  | 
|  | 24 | Configuration functions | 
|  | 25 | ^^^^^^^^^^^^^^^^^^^^^^^ | 
|  | 26 |  | 
|  | 27 | The following functions configure the logging module. They are located in the | 
|  | 28 | :mod:`logging.config` module.  Their use is optional --- you can configure the | 
|  | 29 | logging module using these functions or by making calls to the main API (defined | 
|  | 30 | in :mod:`logging` itself) and defining handlers which are declared either in | 
|  | 31 | :mod:`logging` or :mod:`logging.handlers`. | 
|  | 32 |  | 
|  | 33 | .. function:: dictConfig(config) | 
|  | 34 |  | 
|  | 35 | Takes the logging configuration from a dictionary.  The contents of | 
|  | 36 | this dictionary are described in :ref:`logging-config-dictschema` | 
|  | 37 | below. | 
|  | 38 |  | 
|  | 39 | If an error is encountered during configuration, this function will | 
|  | 40 | raise a :exc:`ValueError`, :exc:`TypeError`, :exc:`AttributeError` | 
|  | 41 | or :exc:`ImportError` with a suitably descriptive message.  The | 
|  | 42 | following is a (possibly incomplete) list of conditions which will | 
|  | 43 | raise an error: | 
|  | 44 |  | 
|  | 45 | * A ``level`` which is not a string or which is a string not | 
|  | 46 | corresponding to an actual logging level. | 
|  | 47 | * A ``propagate`` value which is not a boolean. | 
|  | 48 | * An id which does not have a corresponding destination. | 
|  | 49 | * A non-existent handler id found during an incremental call. | 
|  | 50 | * An invalid logger name. | 
|  | 51 | * Inability to resolve to an internal or external object. | 
|  | 52 |  | 
|  | 53 | Parsing is performed by the :class:`DictConfigurator` class, whose | 
|  | 54 | constructor is passed the dictionary used for configuration, and | 
|  | 55 | has a :meth:`configure` method.  The :mod:`logging.config` module | 
|  | 56 | has a callable attribute :attr:`dictConfigClass` | 
|  | 57 | which is initially set to :class:`DictConfigurator`. | 
|  | 58 | You can replace the value of :attr:`dictConfigClass` with a | 
|  | 59 | suitable implementation of your own. | 
|  | 60 |  | 
|  | 61 | :func:`dictConfig` calls :attr:`dictConfigClass` passing | 
|  | 62 | the specified dictionary, and then calls the :meth:`configure` method on | 
|  | 63 | the returned object to put the configuration into effect:: | 
|  | 64 |  | 
|  | 65 | def dictConfig(config): | 
|  | 66 | dictConfigClass(config).configure() | 
|  | 67 |  | 
|  | 68 | For example, a subclass of :class:`DictConfigurator` could call | 
|  | 69 | ``DictConfigurator.__init__()`` in its own :meth:`__init__()`, then | 
|  | 70 | set up custom prefixes which would be usable in the subsequent | 
|  | 71 | :meth:`configure` call. :attr:`dictConfigClass` would be bound to | 
|  | 72 | this new subclass, and then :func:`dictConfig` could be called exactly as | 
|  | 73 | in the default, uncustomized state. | 
|  | 74 |  | 
|  | 75 | .. versionadded:: 2.7 | 
|  | 76 |  | 
| Vinay Sajip | 0513275 | 2011-04-19 13:47:23 +0100 | [diff] [blame] | 77 | .. function:: fileConfig(fname, defaults=None, disable_existing_loggers=True) | 
| Vinay Sajip | 5dbca9c | 2011-04-08 11:40:38 +0100 | [diff] [blame] | 78 |  | 
| Vinay Sajip | 0513275 | 2011-04-19 13:47:23 +0100 | [diff] [blame] | 79 | Reads the logging configuration from a :mod:`configparser`\-format file | 
|  | 80 | named *fname*. This function can be called several times from an | 
|  | 81 | application, allowing an end user to select from various pre-canned | 
| Vinay Sajip | 5dbca9c | 2011-04-08 11:40:38 +0100 | [diff] [blame] | 82 | configurations (if the developer provides a mechanism to present the choices | 
| Vinay Sajip | 0513275 | 2011-04-19 13:47:23 +0100 | [diff] [blame] | 83 | and load the chosen configuration). | 
| Vinay Sajip | 5dbca9c | 2011-04-08 11:40:38 +0100 | [diff] [blame] | 84 |  | 
| Vinay Sajip | 0513275 | 2011-04-19 13:47:23 +0100 | [diff] [blame] | 85 | :param defaults: Defaults to be passed to the ConfigParser can be specified | 
|  | 86 | in this argument. | 
|  | 87 |  | 
|  | 88 | :param disable_existing_loggers: If specified as ``False``, loggers which | 
|  | 89 | exist when this call is made are left | 
|  | 90 | alone. The default is ``True`` because this | 
|  | 91 | enables old behaviour in a backward- | 
|  | 92 | compatible way. This behaviour is to | 
|  | 93 | disable any existing loggers unless they or | 
|  | 94 | their ancestors are explicitly named in the | 
|  | 95 | logging configuration. | 
|  | 96 |  | 
|  | 97 | .. versionchanged:: 2.6 | 
|  | 98 | The ``disable_existing_loggers`` keyword argument was added. Previously, | 
|  | 99 | existing loggers were *always* disabled. | 
| Vinay Sajip | 5dbca9c | 2011-04-08 11:40:38 +0100 | [diff] [blame] | 100 |  | 
|  | 101 | .. function:: listen(port=DEFAULT_LOGGING_CONFIG_PORT) | 
|  | 102 |  | 
|  | 103 | Starts up a socket server on the specified port, and listens for new | 
|  | 104 | configurations. If no port is specified, the module's default | 
|  | 105 | :const:`DEFAULT_LOGGING_CONFIG_PORT` is used. Logging configurations will be | 
|  | 106 | sent as a file suitable for processing by :func:`fileConfig`. Returns a | 
|  | 107 | :class:`Thread` instance on which you can call :meth:`start` to start the | 
|  | 108 | server, and which you can :meth:`join` when appropriate. To stop the server, | 
|  | 109 | call :func:`stopListening`. | 
|  | 110 |  | 
|  | 111 | To send a configuration to the socket, read in the configuration file and | 
|  | 112 | send it to the socket as a string of bytes preceded by a four-byte length | 
|  | 113 | string packed in binary using ``struct.pack('>L', n)``. | 
|  | 114 |  | 
| Vinay Sajip | 34e992d | 2012-07-25 19:12:35 +0100 | [diff] [blame] | 115 | .. note:: Because portions of the configuration are passed through | 
|  | 116 | :func:`eval`, use of this function may open its users to a security risk. | 
|  | 117 | While the function only binds to a socket on ``localhost``, and so does | 
|  | 118 | not accept connections from remote machines, there are scenarios where | 
|  | 119 | untrusted code could be run under the account of the process which calls | 
|  | 120 | :func:`listen`. Specifically, if the process calling :func:`listen` runs | 
|  | 121 | on a multi-user machine where users cannot trust each other, then a | 
|  | 122 | malicious user could arrange to run essentially arbitrary code in a | 
|  | 123 | victim user's process, simply by connecting to the victim's | 
|  | 124 | :func:`listen` socket and sending a configuration which runs whatever | 
|  | 125 | code the attacker wants to have executed in the victim's process. This is | 
|  | 126 | especially easy to do if the default port is used, but not hard even if a | 
|  | 127 | different port is used). | 
| Vinay Sajip | 5dbca9c | 2011-04-08 11:40:38 +0100 | [diff] [blame] | 128 |  | 
|  | 129 | .. function:: stopListening() | 
|  | 130 |  | 
|  | 131 | Stops the listening server which was created with a call to :func:`listen`. | 
|  | 132 | This is typically called before calling :meth:`join` on the return value from | 
|  | 133 | :func:`listen`. | 
|  | 134 |  | 
|  | 135 |  | 
|  | 136 | .. _logging-config-dictschema: | 
|  | 137 |  | 
|  | 138 | Configuration dictionary schema | 
|  | 139 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | 
|  | 140 |  | 
|  | 141 | Describing a logging configuration requires listing the various | 
|  | 142 | objects to create and the connections between them; for example, you | 
|  | 143 | may create a handler named 'console' and then say that the logger | 
|  | 144 | named 'startup' will send its messages to the 'console' handler. | 
|  | 145 | These objects aren't limited to those provided by the :mod:`logging` | 
|  | 146 | module because you might write your own formatter or handler class. | 
|  | 147 | The parameters to these classes may also need to include external | 
|  | 148 | objects such as ``sys.stderr``.  The syntax for describing these | 
|  | 149 | objects and connections is defined in :ref:`logging-config-dict-connections` | 
|  | 150 | below. | 
|  | 151 |  | 
|  | 152 | Dictionary Schema Details | 
|  | 153 | """"""""""""""""""""""""" | 
|  | 154 |  | 
|  | 155 | The dictionary passed to :func:`dictConfig` must contain the following | 
|  | 156 | keys: | 
|  | 157 |  | 
|  | 158 | * *version* - to be set to an integer value representing the schema | 
|  | 159 | version.  The only valid value at present is 1, but having this key | 
|  | 160 | allows the schema to evolve while still preserving backwards | 
|  | 161 | compatibility. | 
|  | 162 |  | 
|  | 163 | All other keys are optional, but if present they will be interpreted | 
|  | 164 | as described below.  In all cases below where a 'configuring dict' is | 
|  | 165 | mentioned, it will be checked for the special ``'()'`` key to see if a | 
|  | 166 | custom instantiation is required.  If so, the mechanism described in | 
|  | 167 | :ref:`logging-config-dict-userdef` below is used to create an instance; | 
|  | 168 | otherwise, the context is used to determine what to instantiate. | 
|  | 169 |  | 
|  | 170 | * *formatters* - the corresponding value will be a dict in which each | 
|  | 171 | key is a formatter id and each value is a dict describing how to | 
|  | 172 | configure the corresponding Formatter instance. | 
|  | 173 |  | 
|  | 174 | The configuring dict is searched for keys ``format`` and ``datefmt`` | 
|  | 175 | (with defaults of ``None``) and these are used to construct a | 
|  | 176 | :class:`logging.Formatter` instance. | 
|  | 177 |  | 
|  | 178 | * *filters* - the corresponding value will be a dict in which each key | 
|  | 179 | is a filter id and each value is a dict describing how to configure | 
|  | 180 | the corresponding Filter instance. | 
|  | 181 |  | 
|  | 182 | The configuring dict is searched for the key ``name`` (defaulting to the | 
|  | 183 | empty string) and this is used to construct a :class:`logging.Filter` | 
|  | 184 | instance. | 
|  | 185 |  | 
|  | 186 | * *handlers* - the corresponding value will be a dict in which each | 
|  | 187 | key is a handler id and each value is a dict describing how to | 
|  | 188 | configure the corresponding Handler instance. | 
|  | 189 |  | 
|  | 190 | The configuring dict is searched for the following keys: | 
|  | 191 |  | 
|  | 192 | * ``class`` (mandatory).  This is the fully qualified name of the | 
|  | 193 | handler class. | 
|  | 194 |  | 
|  | 195 | * ``level`` (optional).  The level of the handler. | 
|  | 196 |  | 
|  | 197 | * ``formatter`` (optional).  The id of the formatter for this | 
|  | 198 | handler. | 
|  | 199 |  | 
|  | 200 | * ``filters`` (optional).  A list of ids of the filters for this | 
|  | 201 | handler. | 
|  | 202 |  | 
|  | 203 | All *other* keys are passed through as keyword arguments to the | 
|  | 204 | handler's constructor.  For example, given the snippet:: | 
|  | 205 |  | 
|  | 206 | handlers: | 
|  | 207 | console: | 
|  | 208 | class : logging.StreamHandler | 
|  | 209 | formatter: brief | 
|  | 210 | level   : INFO | 
|  | 211 | filters: [allow_foo] | 
|  | 212 | stream  : ext://sys.stdout | 
|  | 213 | file: | 
|  | 214 | class : logging.handlers.RotatingFileHandler | 
|  | 215 | formatter: precise | 
|  | 216 | filename: logconfig.log | 
|  | 217 | maxBytes: 1024 | 
|  | 218 | backupCount: 3 | 
|  | 219 |  | 
|  | 220 | the handler with id ``console`` is instantiated as a | 
|  | 221 | :class:`logging.StreamHandler`, using ``sys.stdout`` as the underlying | 
|  | 222 | stream.  The handler with id ``file`` is instantiated as a | 
|  | 223 | :class:`logging.handlers.RotatingFileHandler` with the keyword arguments | 
|  | 224 | ``filename='logconfig.log', maxBytes=1024, backupCount=3``. | 
|  | 225 |  | 
|  | 226 | * *loggers* - the corresponding value will be a dict in which each key | 
|  | 227 | is a logger name and each value is a dict describing how to | 
|  | 228 | configure the corresponding Logger instance. | 
|  | 229 |  | 
|  | 230 | The configuring dict is searched for the following keys: | 
|  | 231 |  | 
|  | 232 | * ``level`` (optional).  The level of the logger. | 
|  | 233 |  | 
|  | 234 | * ``propagate`` (optional).  The propagation setting of the logger. | 
|  | 235 |  | 
|  | 236 | * ``filters`` (optional).  A list of ids of the filters for this | 
|  | 237 | logger. | 
|  | 238 |  | 
|  | 239 | * ``handlers`` (optional).  A list of ids of the handlers for this | 
|  | 240 | logger. | 
|  | 241 |  | 
|  | 242 | The specified loggers will be configured according to the level, | 
|  | 243 | propagation, filters and handlers specified. | 
|  | 244 |  | 
|  | 245 | * *root* - this will be the configuration for the root logger. | 
|  | 246 | Processing of the configuration will be as for any logger, except | 
|  | 247 | that the ``propagate`` setting will not be applicable. | 
|  | 248 |  | 
|  | 249 | * *incremental* - whether the configuration is to be interpreted as | 
|  | 250 | incremental to the existing configuration.  This value defaults to | 
|  | 251 | ``False``, which means that the specified configuration replaces the | 
|  | 252 | existing configuration with the same semantics as used by the | 
|  | 253 | existing :func:`fileConfig` API. | 
|  | 254 |  | 
|  | 255 | If the specified value is ``True``, the configuration is processed | 
|  | 256 | as described in the section on :ref:`logging-config-dict-incremental`. | 
|  | 257 |  | 
|  | 258 | * *disable_existing_loggers* - whether any existing loggers are to be | 
|  | 259 | disabled. This setting mirrors the parameter of the same name in | 
|  | 260 | :func:`fileConfig`. If absent, this parameter defaults to ``True``. | 
|  | 261 | This value is ignored if *incremental* is ``True``. | 
|  | 262 |  | 
|  | 263 | .. _logging-config-dict-incremental: | 
|  | 264 |  | 
|  | 265 | Incremental Configuration | 
|  | 266 | """"""""""""""""""""""""" | 
|  | 267 |  | 
|  | 268 | It is difficult to provide complete flexibility for incremental | 
|  | 269 | configuration.  For example, because objects such as filters | 
|  | 270 | and formatters are anonymous, once a configuration is set up, it is | 
|  | 271 | not possible to refer to such anonymous objects when augmenting a | 
|  | 272 | configuration. | 
|  | 273 |  | 
|  | 274 | Furthermore, there is not a compelling case for arbitrarily altering | 
|  | 275 | the object graph of loggers, handlers, filters, formatters at | 
|  | 276 | run-time, once a configuration is set up; the verbosity of loggers and | 
|  | 277 | handlers can be controlled just by setting levels (and, in the case of | 
|  | 278 | loggers, propagation flags).  Changing the object graph arbitrarily in | 
|  | 279 | a safe way is problematic in a multi-threaded environment; while not | 
|  | 280 | impossible, the benefits are not worth the complexity it adds to the | 
|  | 281 | implementation. | 
|  | 282 |  | 
|  | 283 | Thus, when the ``incremental`` key of a configuration dict is present | 
|  | 284 | and is ``True``, the system will completely ignore any ``formatters`` and | 
|  | 285 | ``filters`` entries, and process only the ``level`` | 
|  | 286 | settings in the ``handlers`` entries, and the ``level`` and | 
|  | 287 | ``propagate`` settings in the ``loggers`` and ``root`` entries. | 
|  | 288 |  | 
|  | 289 | Using a value in the configuration dict lets configurations to be sent | 
|  | 290 | over the wire as pickled dicts to a socket listener. Thus, the logging | 
|  | 291 | verbosity of a long-running application can be altered over time with | 
|  | 292 | no need to stop and restart the application. | 
|  | 293 |  | 
|  | 294 | .. _logging-config-dict-connections: | 
|  | 295 |  | 
|  | 296 | Object connections | 
|  | 297 | """""""""""""""""" | 
|  | 298 |  | 
|  | 299 | The schema describes a set of logging objects - loggers, | 
|  | 300 | handlers, formatters, filters - which are connected to each other in | 
|  | 301 | an object graph.  Thus, the schema needs to represent connections | 
|  | 302 | between the objects.  For example, say that, once configured, a | 
|  | 303 | particular logger has attached to it a particular handler.  For the | 
|  | 304 | purposes of this discussion, we can say that the logger represents the | 
|  | 305 | source, and the handler the destination, of a connection between the | 
|  | 306 | two.  Of course in the configured objects this is represented by the | 
|  | 307 | logger holding a reference to the handler.  In the configuration dict, | 
|  | 308 | this is done by giving each destination object an id which identifies | 
|  | 309 | it unambiguously, and then using the id in the source object's | 
|  | 310 | configuration to indicate that a connection exists between the source | 
|  | 311 | and the destination object with that id. | 
|  | 312 |  | 
|  | 313 | So, for example, consider the following YAML snippet:: | 
|  | 314 |  | 
|  | 315 | formatters: | 
|  | 316 | brief: | 
|  | 317 | # configuration for formatter with id 'brief' goes here | 
|  | 318 | precise: | 
|  | 319 | # configuration for formatter with id 'precise' goes here | 
|  | 320 | handlers: | 
|  | 321 | h1: #This is an id | 
|  | 322 | # configuration of handler with id 'h1' goes here | 
|  | 323 | formatter: brief | 
|  | 324 | h2: #This is another id | 
|  | 325 | # configuration of handler with id 'h2' goes here | 
|  | 326 | formatter: precise | 
|  | 327 | loggers: | 
|  | 328 | foo.bar.baz: | 
|  | 329 | # other configuration for logger 'foo.bar.baz' | 
|  | 330 | handlers: [h1, h2] | 
|  | 331 |  | 
|  | 332 | (Note: YAML used here because it's a little more readable than the | 
|  | 333 | equivalent Python source form for the dictionary.) | 
|  | 334 |  | 
|  | 335 | The ids for loggers are the logger names which would be used | 
|  | 336 | programmatically to obtain a reference to those loggers, e.g. | 
|  | 337 | ``foo.bar.baz``.  The ids for Formatters and Filters can be any string | 
|  | 338 | value (such as ``brief``, ``precise`` above) and they are transient, | 
|  | 339 | in that they are only meaningful for processing the configuration | 
|  | 340 | dictionary and used to determine connections between objects, and are | 
|  | 341 | not persisted anywhere when the configuration call is complete. | 
|  | 342 |  | 
|  | 343 | The above snippet indicates that logger named ``foo.bar.baz`` should | 
|  | 344 | have two handlers attached to it, which are described by the handler | 
|  | 345 | ids ``h1`` and ``h2``. The formatter for ``h1`` is that described by id | 
|  | 346 | ``brief``, and the formatter for ``h2`` is that described by id | 
|  | 347 | ``precise``. | 
|  | 348 |  | 
|  | 349 |  | 
|  | 350 | .. _logging-config-dict-userdef: | 
|  | 351 |  | 
|  | 352 | User-defined objects | 
|  | 353 | """""""""""""""""""" | 
|  | 354 |  | 
|  | 355 | The schema supports user-defined objects for handlers, filters and | 
|  | 356 | formatters.  (Loggers do not need to have different types for | 
|  | 357 | different instances, so there is no support in this configuration | 
|  | 358 | schema for user-defined logger classes.) | 
|  | 359 |  | 
|  | 360 | Objects to be configured are described by dictionaries | 
|  | 361 | which detail their configuration.  In some places, the logging system | 
|  | 362 | will be able to infer from the context how an object is to be | 
|  | 363 | instantiated, but when a user-defined object is to be instantiated, | 
|  | 364 | the system will not know how to do this.  In order to provide complete | 
|  | 365 | flexibility for user-defined object instantiation, the user needs | 
|  | 366 | to provide a 'factory' - a callable which is called with a | 
|  | 367 | configuration dictionary and which returns the instantiated object. | 
|  | 368 | This is signalled by an absolute import path to the factory being | 
|  | 369 | made available under the special key ``'()'``.  Here's a concrete | 
|  | 370 | example:: | 
|  | 371 |  | 
|  | 372 | formatters: | 
|  | 373 | brief: | 
|  | 374 | format: '%(message)s' | 
|  | 375 | default: | 
|  | 376 | format: '%(asctime)s %(levelname)-8s %(name)-15s %(message)s' | 
|  | 377 | datefmt: '%Y-%m-%d %H:%M:%S' | 
|  | 378 | custom: | 
|  | 379 | (): my.package.customFormatterFactory | 
|  | 380 | bar: baz | 
|  | 381 | spam: 99.9 | 
|  | 382 | answer: 42 | 
|  | 383 |  | 
|  | 384 | The above YAML snippet defines three formatters.  The first, with id | 
|  | 385 | ``brief``, is a standard :class:`logging.Formatter` instance with the | 
|  | 386 | specified format string.  The second, with id ``default``, has a | 
|  | 387 | longer format and also defines the time format explicitly, and will | 
|  | 388 | result in a :class:`logging.Formatter` initialized with those two format | 
|  | 389 | strings.  Shown in Python source form, the ``brief`` and ``default`` | 
|  | 390 | formatters have configuration sub-dictionaries:: | 
|  | 391 |  | 
|  | 392 | { | 
|  | 393 | 'format' : '%(message)s' | 
|  | 394 | } | 
|  | 395 |  | 
|  | 396 | and:: | 
|  | 397 |  | 
|  | 398 | { | 
|  | 399 | 'format' : '%(asctime)s %(levelname)-8s %(name)-15s %(message)s', | 
|  | 400 | 'datefmt' : '%Y-%m-%d %H:%M:%S' | 
|  | 401 | } | 
|  | 402 |  | 
|  | 403 | respectively, and as these dictionaries do not contain the special key | 
|  | 404 | ``'()'``, the instantiation is inferred from the context: as a result, | 
|  | 405 | standard :class:`logging.Formatter` instances are created.  The | 
|  | 406 | configuration sub-dictionary for the third formatter, with id | 
|  | 407 | ``custom``, is:: | 
|  | 408 |  | 
|  | 409 | { | 
|  | 410 | '()' : 'my.package.customFormatterFactory', | 
|  | 411 | 'bar' : 'baz', | 
|  | 412 | 'spam' : 99.9, | 
|  | 413 | 'answer' : 42 | 
|  | 414 | } | 
|  | 415 |  | 
|  | 416 | and this contains the special key ``'()'``, which means that | 
|  | 417 | user-defined instantiation is wanted.  In this case, the specified | 
|  | 418 | factory callable will be used. If it is an actual callable it will be | 
|  | 419 | used directly - otherwise, if you specify a string (as in the example) | 
|  | 420 | the actual callable will be located using normal import mechanisms. | 
|  | 421 | The callable will be called with the **remaining** items in the | 
|  | 422 | configuration sub-dictionary as keyword arguments.  In the above | 
|  | 423 | example, the formatter with id ``custom`` will be assumed to be | 
|  | 424 | returned by the call:: | 
|  | 425 |  | 
|  | 426 | my.package.customFormatterFactory(bar='baz', spam=99.9, answer=42) | 
|  | 427 |  | 
|  | 428 | The key ``'()'`` has been used as the special key because it is not a | 
|  | 429 | valid keyword parameter name, and so will not clash with the names of | 
|  | 430 | the keyword arguments used in the call.  The ``'()'`` also serves as a | 
|  | 431 | mnemonic that the corresponding value is a callable. | 
|  | 432 |  | 
|  | 433 |  | 
|  | 434 | .. _logging-config-dict-externalobj: | 
|  | 435 |  | 
|  | 436 | Access to external objects | 
|  | 437 | """""""""""""""""""""""""" | 
|  | 438 |  | 
|  | 439 | There are times where a configuration needs to refer to objects | 
|  | 440 | external to the configuration, for example ``sys.stderr``.  If the | 
|  | 441 | configuration dict is constructed using Python code, this is | 
|  | 442 | straightforward, but a problem arises when the configuration is | 
|  | 443 | provided via a text file (e.g. JSON, YAML).  In a text file, there is | 
|  | 444 | no standard way to distinguish ``sys.stderr`` from the literal string | 
|  | 445 | ``'sys.stderr'``.  To facilitate this distinction, the configuration | 
|  | 446 | system looks for certain special prefixes in string values and | 
|  | 447 | treat them specially.  For example, if the literal string | 
|  | 448 | ``'ext://sys.stderr'`` is provided as a value in the configuration, | 
|  | 449 | then the ``ext://`` will be stripped off and the remainder of the | 
|  | 450 | value processed using normal import mechanisms. | 
|  | 451 |  | 
|  | 452 | The handling of such prefixes is done in a way analogous to protocol | 
|  | 453 | handling: there is a generic mechanism to look for prefixes which | 
|  | 454 | match the regular expression ``^(?P<prefix>[a-z]+)://(?P<suffix>.*)$`` | 
|  | 455 | whereby, if the ``prefix`` is recognised, the ``suffix`` is processed | 
|  | 456 | in a prefix-dependent manner and the result of the processing replaces | 
|  | 457 | the string value.  If the prefix is not recognised, then the string | 
|  | 458 | value will be left as-is. | 
|  | 459 |  | 
|  | 460 |  | 
|  | 461 | .. _logging-config-dict-internalobj: | 
|  | 462 |  | 
|  | 463 | Access to internal objects | 
|  | 464 | """""""""""""""""""""""""" | 
|  | 465 |  | 
|  | 466 | As well as external objects, there is sometimes also a need to refer | 
|  | 467 | to objects in the configuration.  This will be done implicitly by the | 
|  | 468 | configuration system for things that it knows about.  For example, the | 
|  | 469 | string value ``'DEBUG'`` for a ``level`` in a logger or handler will | 
|  | 470 | automatically be converted to the value ``logging.DEBUG``, and the | 
|  | 471 | ``handlers``, ``filters`` and ``formatter`` entries will take an | 
|  | 472 | object id and resolve to the appropriate destination object. | 
|  | 473 |  | 
|  | 474 | However, a more generic mechanism is needed for user-defined | 
|  | 475 | objects which are not known to the :mod:`logging` module.  For | 
|  | 476 | example, consider :class:`logging.handlers.MemoryHandler`, which takes | 
|  | 477 | a ``target`` argument which is another handler to delegate to. Since | 
|  | 478 | the system already knows about this class, then in the configuration, | 
|  | 479 | the given ``target`` just needs to be the object id of the relevant | 
|  | 480 | target handler, and the system will resolve to the handler from the | 
|  | 481 | id.  If, however, a user defines a ``my.package.MyHandler`` which has | 
|  | 482 | an ``alternate`` handler, the configuration system would not know that | 
|  | 483 | the ``alternate`` referred to a handler.  To cater for this, a generic | 
|  | 484 | resolution system allows the user to specify:: | 
|  | 485 |  | 
|  | 486 | handlers: | 
|  | 487 | file: | 
|  | 488 | # configuration of file handler goes here | 
|  | 489 |  | 
|  | 490 | custom: | 
|  | 491 | (): my.package.MyHandler | 
|  | 492 | alternate: cfg://handlers.file | 
|  | 493 |  | 
|  | 494 | The literal string ``'cfg://handlers.file'`` will be resolved in an | 
|  | 495 | analogous way to strings with the ``ext://`` prefix, but looking | 
|  | 496 | in the configuration itself rather than the import namespace.  The | 
|  | 497 | mechanism allows access by dot or by index, in a similar way to | 
|  | 498 | that provided by ``str.format``.  Thus, given the following snippet:: | 
|  | 499 |  | 
|  | 500 | handlers: | 
|  | 501 | email: | 
|  | 502 | class: logging.handlers.SMTPHandler | 
|  | 503 | mailhost: localhost | 
|  | 504 | fromaddr: my_app@domain.tld | 
|  | 505 | toaddrs: | 
|  | 506 | - support_team@domain.tld | 
|  | 507 | - dev_team@domain.tld | 
|  | 508 | subject: Houston, we have a problem. | 
|  | 509 |  | 
|  | 510 | in the configuration, the string ``'cfg://handlers'`` would resolve to | 
|  | 511 | the dict with key ``handlers``, the string ``'cfg://handlers.email`` | 
|  | 512 | would resolve to the dict with key ``email`` in the ``handlers`` dict, | 
|  | 513 | and so on.  The string ``'cfg://handlers.email.toaddrs[1]`` would | 
|  | 514 | resolve to ``'dev_team.domain.tld'`` and the string | 
|  | 515 | ``'cfg://handlers.email.toaddrs[0]'`` would resolve to the value | 
|  | 516 | ``'support_team@domain.tld'``. The ``subject`` value could be accessed | 
|  | 517 | using either ``'cfg://handlers.email.subject'`` or, equivalently, | 
|  | 518 | ``'cfg://handlers.email[subject]'``.  The latter form only needs to be | 
|  | 519 | used if the key contains spaces or non-alphanumeric characters.  If an | 
|  | 520 | index value consists only of decimal digits, access will be attempted | 
|  | 521 | using the corresponding integer value, falling back to the string | 
|  | 522 | value if needed. | 
|  | 523 |  | 
|  | 524 | Given a string ``cfg://handlers.myhandler.mykey.123``, this will | 
|  | 525 | resolve to ``config_dict['handlers']['myhandler']['mykey']['123']``. | 
|  | 526 | If the string is specified as ``cfg://handlers.myhandler.mykey[123]``, | 
|  | 527 | the system will attempt to retrieve the value from | 
|  | 528 | ``config_dict['handlers']['myhandler']['mykey'][123]``, and fall back | 
|  | 529 | to ``config_dict['handlers']['myhandler']['mykey']['123']`` if that | 
|  | 530 | fails. | 
|  | 531 |  | 
| Vinay Sajip | 5088b50 | 2011-08-11 13:39:52 +0100 | [diff] [blame] | 532 |  | 
|  | 533 | .. _logging-import-resolution: | 
|  | 534 |  | 
|  | 535 | Import resolution and custom importers | 
|  | 536 | """""""""""""""""""""""""""""""""""""" | 
|  | 537 |  | 
|  | 538 | Import resolution, by default, uses the builtin :func:`__import__` function | 
|  | 539 | to do its importing. You may want to replace this with your own importing | 
|  | 540 | mechanism: if so, you can replace the :attr:`importer` attribute of the | 
|  | 541 | :class:`DictConfigurator` or its superclass, the | 
|  | 542 | :class:`BaseConfigurator` class. However, you need to be | 
|  | 543 | careful because of the way functions are accessed from classes via | 
|  | 544 | descriptors. If you are using a Python callable to do your imports, and you | 
|  | 545 | want to define it at class level rather than instance level, you need to wrap | 
|  | 546 | it with :func:`staticmethod`. For example:: | 
|  | 547 |  | 
|  | 548 | from importlib import import_module | 
|  | 549 | from logging.config import BaseConfigurator | 
|  | 550 |  | 
|  | 551 | BaseConfigurator.importer = staticmethod(import_module) | 
|  | 552 |  | 
|  | 553 | You don't need to wrap with :func:`staticmethod` if you're setting the import | 
|  | 554 | callable on a configurator *instance*. | 
|  | 555 |  | 
|  | 556 |  | 
| Vinay Sajip | 5dbca9c | 2011-04-08 11:40:38 +0100 | [diff] [blame] | 557 | .. _logging-config-fileformat: | 
|  | 558 |  | 
|  | 559 | Configuration file format | 
|  | 560 | ^^^^^^^^^^^^^^^^^^^^^^^^^ | 
|  | 561 |  | 
|  | 562 | The configuration file format understood by :func:`fileConfig` is based on | 
|  | 563 | :mod:`configparser` functionality. The file must contain sections called | 
|  | 564 | ``[loggers]``, ``[handlers]`` and ``[formatters]`` which identify by name the | 
|  | 565 | entities of each type which are defined in the file. For each such entity, there | 
|  | 566 | is a separate section which identifies how that entity is configured.  Thus, for | 
|  | 567 | a logger named ``log01`` in the ``[loggers]`` section, the relevant | 
|  | 568 | configuration details are held in a section ``[logger_log01]``. Similarly, a | 
|  | 569 | handler called ``hand01`` in the ``[handlers]`` section will have its | 
|  | 570 | configuration held in a section called ``[handler_hand01]``, while a formatter | 
|  | 571 | called ``form01`` in the ``[formatters]`` section will have its configuration | 
|  | 572 | specified in a section called ``[formatter_form01]``. The root logger | 
|  | 573 | configuration must be specified in a section called ``[logger_root]``. | 
|  | 574 |  | 
|  | 575 | Examples of these sections in the file are given below. :: | 
|  | 576 |  | 
|  | 577 | [loggers] | 
|  | 578 | keys=root,log02,log03,log04,log05,log06,log07 | 
|  | 579 |  | 
|  | 580 | [handlers] | 
|  | 581 | keys=hand01,hand02,hand03,hand04,hand05,hand06,hand07,hand08,hand09 | 
|  | 582 |  | 
|  | 583 | [formatters] | 
|  | 584 | keys=form01,form02,form03,form04,form05,form06,form07,form08,form09 | 
|  | 585 |  | 
|  | 586 | The root logger must specify a level and a list of handlers. An example of a | 
|  | 587 | root logger section is given below. :: | 
|  | 588 |  | 
|  | 589 | [logger_root] | 
|  | 590 | level=NOTSET | 
|  | 591 | handlers=hand01 | 
|  | 592 |  | 
|  | 593 | The ``level`` entry can be one of ``DEBUG, INFO, WARNING, ERROR, CRITICAL`` or | 
|  | 594 | ``NOTSET``. For the root logger only, ``NOTSET`` means that all messages will be | 
|  | 595 | logged. Level values are :func:`eval`\ uated in the context of the ``logging`` | 
|  | 596 | package's namespace. | 
|  | 597 |  | 
|  | 598 | The ``handlers`` entry is a comma-separated list of handler names, which must | 
|  | 599 | appear in the ``[handlers]`` section. These names must appear in the | 
|  | 600 | ``[handlers]`` section and have corresponding sections in the configuration | 
|  | 601 | file. | 
|  | 602 |  | 
|  | 603 | For loggers other than the root logger, some additional information is required. | 
|  | 604 | This is illustrated by the following example. :: | 
|  | 605 |  | 
|  | 606 | [logger_parser] | 
|  | 607 | level=DEBUG | 
|  | 608 | handlers=hand01 | 
|  | 609 | propagate=1 | 
|  | 610 | qualname=compiler.parser | 
|  | 611 |  | 
|  | 612 | The ``level`` and ``handlers`` entries are interpreted as for the root logger, | 
|  | 613 | except that if a non-root logger's level is specified as ``NOTSET``, the system | 
|  | 614 | consults loggers higher up the hierarchy to determine the effective level of the | 
|  | 615 | logger. The ``propagate`` entry is set to 1 to indicate that messages must | 
|  | 616 | propagate to handlers higher up the logger hierarchy from this logger, or 0 to | 
|  | 617 | indicate that messages are **not** propagated to handlers up the hierarchy. The | 
|  | 618 | ``qualname`` entry is the hierarchical channel name of the logger, that is to | 
|  | 619 | say the name used by the application to get the logger. | 
|  | 620 |  | 
|  | 621 | Sections which specify handler configuration are exemplified by the following. | 
|  | 622 | :: | 
|  | 623 |  | 
|  | 624 | [handler_hand01] | 
|  | 625 | class=StreamHandler | 
|  | 626 | level=NOTSET | 
|  | 627 | formatter=form01 | 
|  | 628 | args=(sys.stdout,) | 
|  | 629 |  | 
|  | 630 | The ``class`` entry indicates the handler's class (as determined by :func:`eval` | 
|  | 631 | in the ``logging`` package's namespace). The ``level`` is interpreted as for | 
|  | 632 | loggers, and ``NOTSET`` is taken to mean 'log everything'. | 
|  | 633 |  | 
|  | 634 | .. versionchanged:: 2.6 | 
|  | 635 | Added support for resolving the handler’s class as a dotted module and | 
|  | 636 | class name. | 
|  | 637 |  | 
|  | 638 | The ``formatter`` entry indicates the key name of the formatter for this | 
|  | 639 | handler. If blank, a default formatter (``logging._defaultFormatter``) is used. | 
|  | 640 | If a name is specified, it must appear in the ``[formatters]`` section and have | 
|  | 641 | a corresponding section in the configuration file. | 
|  | 642 |  | 
|  | 643 | The ``args`` entry, when :func:`eval`\ uated in the context of the ``logging`` | 
|  | 644 | package's namespace, is the list of arguments to the constructor for the handler | 
|  | 645 | class. Refer to the constructors for the relevant handlers, or to the examples | 
|  | 646 | below, to see how typical entries are constructed. :: | 
|  | 647 |  | 
|  | 648 | [handler_hand02] | 
|  | 649 | class=FileHandler | 
|  | 650 | level=DEBUG | 
|  | 651 | formatter=form02 | 
|  | 652 | args=('python.log', 'w') | 
|  | 653 |  | 
|  | 654 | [handler_hand03] | 
|  | 655 | class=handlers.SocketHandler | 
|  | 656 | level=INFO | 
|  | 657 | formatter=form03 | 
|  | 658 | args=('localhost', handlers.DEFAULT_TCP_LOGGING_PORT) | 
|  | 659 |  | 
|  | 660 | [handler_hand04] | 
|  | 661 | class=handlers.DatagramHandler | 
|  | 662 | level=WARN | 
|  | 663 | formatter=form04 | 
|  | 664 | args=('localhost', handlers.DEFAULT_UDP_LOGGING_PORT) | 
|  | 665 |  | 
|  | 666 | [handler_hand05] | 
|  | 667 | class=handlers.SysLogHandler | 
|  | 668 | level=ERROR | 
|  | 669 | formatter=form05 | 
|  | 670 | args=(('localhost', handlers.SYSLOG_UDP_PORT), handlers.SysLogHandler.LOG_USER) | 
|  | 671 |  | 
|  | 672 | [handler_hand06] | 
|  | 673 | class=handlers.NTEventLogHandler | 
|  | 674 | level=CRITICAL | 
|  | 675 | formatter=form06 | 
|  | 676 | args=('Python Application', '', 'Application') | 
|  | 677 |  | 
|  | 678 | [handler_hand07] | 
|  | 679 | class=handlers.SMTPHandler | 
|  | 680 | level=WARN | 
|  | 681 | formatter=form07 | 
|  | 682 | args=('localhost', 'from@abc', ['user1@abc', 'user2@xyz'], 'Logger Subject') | 
|  | 683 |  | 
|  | 684 | [handler_hand08] | 
|  | 685 | class=handlers.MemoryHandler | 
|  | 686 | level=NOTSET | 
|  | 687 | formatter=form08 | 
|  | 688 | target= | 
|  | 689 | args=(10, ERROR) | 
|  | 690 |  | 
|  | 691 | [handler_hand09] | 
|  | 692 | class=handlers.HTTPHandler | 
|  | 693 | level=NOTSET | 
|  | 694 | formatter=form09 | 
|  | 695 | args=('localhost:9022', '/log', 'GET') | 
|  | 696 |  | 
|  | 697 | Sections which specify formatter configuration are typified by the following. :: | 
|  | 698 |  | 
|  | 699 | [formatter_form01] | 
|  | 700 | format=F1 %(asctime)s %(levelname)s %(message)s | 
|  | 701 | datefmt= | 
|  | 702 | class=logging.Formatter | 
|  | 703 |  | 
|  | 704 | The ``format`` entry is the overall format string, and the ``datefmt`` entry is | 
|  | 705 | the :func:`strftime`\ -compatible date/time format string.  If empty, the | 
|  | 706 | package substitutes ISO8601 format date/times, which is almost equivalent to | 
|  | 707 | specifying the date format string ``'%Y-%m-%d %H:%M:%S'``.  The ISO8601 format | 
|  | 708 | also specifies milliseconds, which are appended to the result of using the above | 
|  | 709 | format string, with a comma separator.  An example time in ISO8601 format is | 
|  | 710 | ``2003-01-23 00:29:50,411``. | 
|  | 711 |  | 
|  | 712 | The ``class`` entry is optional.  It indicates the name of the formatter's class | 
|  | 713 | (as a dotted module and class name.)  This option is useful for instantiating a | 
|  | 714 | :class:`Formatter` subclass.  Subclasses of :class:`Formatter` can present | 
|  | 715 | exception tracebacks in an expanded or condensed format. | 
|  | 716 |  | 
| Vinay Sajip | 34e992d | 2012-07-25 19:12:35 +0100 | [diff] [blame] | 717 | .. note:: Due to the use of :func:`eval` as described above, there are | 
|  | 718 | potential security risks which result from using the :func:`listen` to send | 
|  | 719 | and receive configurations via sockets. The risks are limited to where | 
|  | 720 | multiple users with no mutual trust run code on the same machine; see the | 
|  | 721 | :func:`listen` documentation for more information. | 
|  | 722 |  | 
| Vinay Sajip | 5dbca9c | 2011-04-08 11:40:38 +0100 | [diff] [blame] | 723 | .. seealso:: | 
|  | 724 |  | 
|  | 725 | Module :mod:`logging` | 
|  | 726 | API reference for the logging module. | 
|  | 727 |  | 
|  | 728 | Module :mod:`logging.handlers` | 
|  | 729 | Useful handlers included with the logging module. | 
|  | 730 |  | 
|  | 731 |  |