#18891: Complete new provisional email API.
This adds EmailMessage and, MIMEPart subclasses of Message
with new API methods, and a ContentManager class used by
the new methods. Also a new policy setting, content_manager.
Patch was reviewed by Stephen J. Turnbull and Serhiy Storchaka,
and reflects their feedback.
I will ideally add some examples of using the new API to the
documentation before the final release.
diff --git a/Doc/library/email.contentmanager.rst b/Doc/library/email.contentmanager.rst
new file mode 100644
index 0000000..f3ba794
--- /dev/null
+++ b/Doc/library/email.contentmanager.rst
@@ -0,0 +1,427 @@
+:mod:`email.contentmanager`: Managing MIME Content
+--------------------------------------------------
+
+.. module:: email.contentmanager
+ :synopsis: Storing and Retrieving Content from MIME Parts
+
+.. moduleauthor:: R. David Murray <rdmurray@bitdance.com>
+.. sectionauthor:: R. David Murray <rdmurray@bitdance.com>
+
+
+.. note::
+
+ The contentmanager module has been included in the standard library on a
+ :term:`provisional basis <provisional package>`. Backwards incompatible
+ changes (up to and including removal of the module) may occur if deemed
+ necessary by the core developers.
+
+.. versionadded:: 3.4
+ as a :term:`provisional module <provisional package>`.
+
+The :mod:`~email.message` module provides a class that can represent an
+arbitrary email message. That basic message model has a useful and flexible
+API, but it provides only a lower-level API for interacting with the generic
+parts of a message (the headers, generic header parameters, and the payload,
+which may be a list of sub-parts). This module provides classes and tools
+that provide an enhanced and extensible API for dealing with various specific
+types of content, including the ability to retrieve the content of the message
+as a specialized object type rather than as a simple bytes object. The module
+automatically takes care of the RFC-specified MIME details (required headers
+and parameters, etc.) for the certain common content types content properties,
+and support for additional types can be added by an application using the
+extension mechanisms.
+
+This module defines the eponymous "Content Manager" classes. The base
+:class:`.ContentManager` class defines an API for registering content
+management functions which extract data from ``Message`` objects or insert data
+and headers into ``Message`` objects, thus providing a way of converting
+between ``Message`` objects containing data and other representations of that
+data (Python data types, specialized Python objects, external files, etc). The
+module also defines one concrete content manager: :data:`raw_data_manager`
+converts between MIME content types and ``str`` or ``bytes`` data. It also
+provides a convenient API for managing the MIME parameters when inserting
+content into ``Message``\ s. It also handles inserting and extracting
+``Message`` objects when dealing with the ``message/rfc822`` content type.
+
+Another part of the enhanced interface is subclasses of
+:class:`~email.message.Message` that provide new convenience API functions,
+including convenience methods for calling the Content Managers derived from
+this module.
+
+.. note::
+
+ Although :class:`.EmailMessage` and :class:`.MIMEPart` are currently
+ documented in this module because of the provisional nature of the code, the
+ implementation lives in the :mod:`email.message` module.
+
+
+.. class:: EmailMessage(policy=default)
+
+ If *policy* is specified (it must be an instance of a :mod:`~email.policy`
+ class) use the rules it specifies to udpate and serialize the representation
+ of the message. If *policy* is not set, use the
+ :class:`~email.policy.default` policy, which follows the rules of the email
+ RFCs except for line endings (instead of the RFC mandated ``\r\n``, it uses
+ the Python standard ``\n`` line endings). For more information see the
+ :mod:`~email.policy` documentation.
+
+ This class is a subclass of :class:`~email.message.Message`. It adds
+ the following methods:
+
+
+ .. attribute:: is_attachment
+
+ Set to ``True`` if there is a :mailheader:`Content-Disposition` header
+ and its (case insensitive) value is ``attachment``, ``False`` otherwise.
+
+
+ .. method:: get_body(preferencelist=('related', 'html', 'plain'))
+
+ Return the MIME part that is the best candidate to be the "body" of the
+ message.
+
+ *preferencelist* must be a sequence of strings from the set ``related``,
+ ``html``, and ``plain``, and indicates the order of preference for the
+ content type of the part returned.
+
+ Start looking for candidate matches with the object on which the
+ ``get_body`` method is called.
+
+ If ``related`` is not included in *preferencelist*, consider the root
+ part (or subpart of the root part) of any related encountered as a
+ candidate if the (sub-)part matches a preference.
+
+ When encountering a ``multipart/related``, check the ``start`` parameter
+ and if a part with a matching :mailheader:`Content-ID` is found, consider
+ only it when looking for candidate matches. Otherwise consider only the
+ first (default root) part of the ``multipart/related``.
+
+ If a part has a :mailheader:``Content-Disposition`` header, only consider
+ the part a candidate match if the value of the header is ``inline``.
+
+ If none of the candidates matches any of the preferences in
+ *preferneclist*, return ``None``.
+
+ Notes: (1) For most applications the only *preferencelist* combinations
+ that really make sense are ``('plain',)``, ``('html', 'plain')``, and the
+ default, ``('related', 'html', 'plain')``. (2) Because matching starts
+ with the object on which ``get_body`` is called, calling ``get_body`` on
+ a ``multipart/related`` will return the object itself unless
+ *preferencelist* has a non-default value. (3) Messages (or message parts)
+ that do not specify a :mailheader:`Content-Type` or whose
+ :mailheader:`Content-Type` header is invalid will be treated as if they
+ are of type ``text/plain``, which may occasionally cause ``get_body`` to
+ return unexpected results.
+
+
+ .. method:: iter_attachments()
+
+ Return an iterator over all of the parts of the message that are not
+ candidate "body" parts. That is, skip the first occurrence of each of
+ ``text/plain``, ``text/html``, ``multipart/related``, or
+ ``multipart/alternative`` (unless they are explicitly marked as
+ attachments via :mailheader:`Content-Disposition: attachment`), and
+ return all remaining parts. When applied directly to a
+ ``multipart/related``, return an iterator over the all the related parts
+ except the root part (ie: the part pointed to by the ``start`` parameter,
+ or the first part if there is no ``start`` parameter or the ``start``
+ parameter doesn't match the :mailheader:`Content-ID` of any of the
+ parts). When applied directly to a ``multipart/alternative`` or a
+ non-``multipart``, return an empty iterator.
+
+
+ .. method:: iter_parts()
+
+ Return an iterator over all of the immediate sub-parts of the message,
+ which will be empty for a non-``multipart``. (See also
+ :meth:``~email.message.walk``.)
+
+
+ .. method:: get_content(*args, content_manager=None, **kw)
+
+ Call the ``get_content`` method of the *content_manager*, passing self
+ as the message object, and passing along any other arguments or keywords
+ as additional arguments. If *content_manager* is not specified, use
+ the ``content_manager`` specified by the current :mod:`~email.policy`.
+
+
+ .. method:: set_content(*args, content_manager=None, **kw)
+
+ Call the ``set_content`` method of the *content_manager*, passing self
+ as the message object, and passing along any other arguments or keywords
+ as additional arguments. If *content_manager* is not specified, use
+ the ``content_manager`` specified by the current :mod:`~email.policy`.
+
+
+ .. method:: make_related(boundary=None)
+
+ Convert a non-``multipart`` message into a ``multipart/related`` message,
+ moving any existing :mailheader:`Content-` headers and payload into a
+ (new) first part of the ``multipart``. If *boundary* is specified, use
+ it as the boundary string in the multipart, otherwise leave the boundary
+ to be automatically created when it is needed (for example, when the
+ message is serialized).
+
+
+ .. method:: make_alternative(boundary=None)
+
+ Convert a non-``multipart`` or a ``multipart/related`` into a
+ ``multipart/alternative``, moving any existing :mailheader:`Content-`
+ headers and payload into a (new) first part of the ``multipart``. If
+ *boundary* is specified, use it as the boundary string in the multipart,
+ otherwise leave the boundary to be automatically created when it is
+ needed (for example, when the message is serialized).
+
+
+ .. method:: make_mixed(boundary=None)
+
+ Convert a non-``multipart``, a ``multipart/related``, or a
+ ``multipart-alternative`` into a ``multipart/mixed``, moving any existing
+ :mailheader:`Content-` headers and payload into a (new) first part of the
+ ``multipart``. If *boundary* is specified, use it as the boundary string
+ in the multipart, otherwise leave the boundary to be automatically
+ created when it is needed (for example, when the message is serialized).
+
+
+ .. method:: add_related(*args, content_manager=None, **kw)
+
+ If the message is a ``multipart/related``, create a new message
+ object, pass all of the arguments to its :meth:`set_content` method,
+ and :meth:`~email.message.Message.attach` it to the ``multipart``. If
+ the message is a non-``multipart``, call :meth:`make_related` and then
+ proceed as above. If the message is any other type of ``multipart``,
+ raise a :exc:`TypeError`. If *content_manager* is not specified, use
+ the ``content_manager`` specified by the current :mod:`~email.policy`.
+ If the added part has no :mailheader:`Content-Disposition` header,
+ add one with the value ``inline``.
+
+
+ .. method:: add_alternative(*args, content_manager=None, **kw)
+
+ If the message is a ``multipart/alternative``, create a new message
+ object, pass all of the arguments to its :meth:`set_content` method, and
+ :meth:`~email.message.Message.attach` it to the ``multipart``. If the
+ message is a non-``multipart`` or ``multipart/related``, call
+ :meth:`make_alternative` and then proceed as above. If the message is
+ any other type of ``multipart``, raise a :exc:`TypeError`. If
+ *content_manager* is not specified, use the ``content_manager`` specified
+ by the current :mod:`~email.policy`.
+
+
+ .. method:: add_attachment(*args, content_manager=None, **kw)
+
+ If the message is a ``multipart/mixed``, create a new message object,
+ pass all of the arguments to its :meth:`set_content` method, and
+ :meth:`~email.message.Message.attach` it to the ``multipart``. If the
+ message is a non-``multipart``, ``multipart/related``, or
+ ``multipart/alternative``, call :meth:`make_mixed` and then proceed as
+ above. If *content_manager* is not specified, use the ``content_manager``
+ specified by the current :mod:`~email.policy`. If the added part
+ has no :mailheader:`Content-Disposition` header, add one with the value
+ ``attachment``. This method can be used both for explicit attachments
+ (:mailheader:`Content-Disposition: attachment` and ``inline`` attachments
+ (:mailheader:`Content-Disposition: inline`), by passing appropriate
+ options to the ``content_manager``.
+
+
+ .. method:: clear()
+
+ Remove the payload and all of the headers.
+
+
+ .. method:: clear_content()
+
+ Remove the payload and all of the :exc:`Content-` headers, leaving
+ all other headers intact and in their original order.
+
+
+.. class:: ContentManager()
+
+ Base class for content managers. Provides the standard registry mechanisms
+ to register converters between MIME content and other representations, as
+ well as the ``get_content`` and ``set_content`` dispatch methods.
+
+
+ .. method:: get_content(msg, *args, **kw)
+
+ Look up a handler function based on the ``mimetype`` of *msg* (see next
+ paragraph), call it, passing through all arguments, and return the result
+ of the call. The expectation is that the handler will extract the
+ payload from *msg* and return an object that encodes information about
+ the extracted data.
+
+ To find the handler, look for the following keys in the registry,
+ stopping with the first one found:
+
+ * the string representing the full MIME type (``maintype/subtype``)
+ * the string representing the ``maintype``
+ * the empty string
+
+ If none of these keys produce a handler, raise a :exc:`KeyError` for the
+ full MIME type.
+
+
+ .. method:: set_content(msg, obj, *args, **kw)
+
+ If the ``maintype`` is ``multipart``, raise a :exc:`TypeError`; otherwise
+ look up a handler function based on the type of *obj* (see next
+ paragraph), call :meth:`~email.message.EmailMessage.clear_content` on the
+ *msg*, and call the handler function, passing through all arguments. The
+ expectation is that the handler will transform and store *obj* into
+ *msg*, possibly making other changes to *msg* as well, such as adding
+ various MIME headers to encode information needed to interpret the stored
+ data.
+
+ To find the handler, obtain the type of *obj* (``typ = type(obj)``), and
+ look for the following keys in the registry, stopping with the first one
+ found:
+
+ * the type itself (``typ``)
+ * the type's fully qualified name (``typ.__module__ + '.' +
+ typ.__qualname__``).
+ * the type's qualname (``typ.__qualname__``)
+ * the type's name (``typ.__name__``).
+
+ If none of the above match, repeat all of the checks above for each of
+ the types in the :term:`MRO` (``typ.__mro__``). Finally, if no other key
+ yields a handler, check for a handler for the key ``None``. If there is
+ no handler for ``None``, raise a :exc:`KeyError` for the fully
+ qualified name of the type.
+
+ Also add a :mailheader:`MIME-Version` header if one is not present (see
+ also :class:`.MIMEPart`).
+
+
+ .. method:: add_get_handler(key, handler)
+
+ Record the function *handler* as the handler for *key*. For the possible
+ values of *key*, see :meth:`get_content`.
+
+
+ .. method:: add_set_handler(typekey, handler)
+
+ Record *handler* as the function to call when an object of a type
+ matching *typekey* is passed to :meth:`set_content`. For the possible
+ values of *typekey*, see :meth:`set_content`.
+
+
+.. class:: MIMEPart(policy=default)
+
+ This class represents a subpart of a MIME message. It is identical to
+ :class:`EmailMessage`, except that no :mailheader:`MIME-Version` headers are
+ added when :meth:`~EmailMessage.set_content` is called, since sub-parts do
+ not need their own :mailheader:`MIME-Version` headers.
+
+
+Content Manager Instances
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Currently the email package provides only one concrete content manager,
+:data:`raw_data_manager`, although more may be added in the future.
+:data:`raw_data_manager` is the
+:attr:`~email.policy.EmailPolicy.content_manager` provided by
+:attr:`~email.policy.EmailPolicy` and its derivatives.
+
+
+.. data:: raw_data_manager
+
+ This content manager provides only a minimum interface beyond that provided
+ by :class:`~email.message.Message` itself: it deals only with text, raw
+ byte strings, and :class:`~email.message.Message` objects. Nevertheless, it
+ provides significant advantages compared to the base API: ``get_content`` on
+ a text part will return a unicode string without the application needing to
+ manually decode it, ``set_content`` provides a rich set of options for
+ controlling the headers added to a part and controlling the content transfer
+ encoding, and it enables the use of the various ``add_`` methods, thereby
+ simplifying the creation of multipart messages.
+
+ .. method:: get_content(msg, errors='replace')
+
+ Return the payload of the part as either a string (for ``text`` parts), a
+ :class:`~email.message.EmailMessage` object (for ``message/rfc822``
+ parts), or a ``bytes`` object (for all other non-multipart types). Raise
+ a :exc:`KeyError` if called on a ``multipart``. If the part is a
+ ``text`` part and *errors* is specified, use it as the error handler when
+ decoding the payload to unicode. The default error handler is
+ ``replace``.
+
+ .. method:: set_content(msg, <'str'>, subtype="plain", charset='utf-8' \
+ cte=None, \
+ disposition=None, filename=None, cid=None, \
+ params=None, headers=None)
+ set_content(msg, <'bytes'>, maintype, subtype, cte="base64", \
+ disposition=None, filename=None, cid=None, \
+ params=None, headers=None)
+ set_content(msg, <'Message'>, cte=None, \
+ disposition=None, filename=None, cid=None, \
+ params=None, headers=None)
+ set_content(msg, <'list'>, subtype='mixed', \
+ disposition=None, filename=None, cid=None, \
+ params=None, headers=None)
+
+ Add headers and payload to *msg*:
+
+ Add a :mailheader:`Content-Type` header with a ``maintype/subtype``
+ value.
+
+ * For ``str``, set the MIME ``maintype`` to ``text``, and set the
+ subtype to *subtype* if it is specified, or ``plain`` if it is not.
+ * For ``bytes``, use the specified *maintype* and *subtype*, or
+ raise a :exc:`TypeError` if they are not specified.
+ * For :class:`~email.message.Message` objects, set the maintype to
+ ``message``, and set the subtype to *subtype* if it is specified
+ or ``rfc822`` if it is not. If *subtype* is ``partial``, raise an
+ error (``bytes`` objects must be used to construct
+ ``message/partial`` parts).
+ * For *<'list'>*, which should be a list of
+ :class:`~email.message.Message` objects, set the ``maintype`` to
+ ``multipart``, and the ``subtype`` to *subtype* if it is
+ specified, and ``mixed`` if it is not. If the message parts in
+ the *<'list'>* have :mailheader:`MIME-Version` headers, remove
+ them.
+
+ If *charset* is provided (which is valid only for ``str``), encode the
+ string to bytes using the specified character set. The default is
+ ``utf-8``. If the specified *charset* is a known alias for a standard
+ MIME charset name, use the standard charset instead.
+
+ If *cte* is set, encode the payload using the specified content transfer
+ encoding, and set the :mailheader:`Content-Transfer-Endcoding` header to
+ that value. For ``str`` objects, if it is not set use heuristics to
+ determine the most compact encoding. Possible values for *cte* are
+ ``quoted-printable``, ``base64``, ``7bit``, ``8bit``, and ``binary``.
+ If the input cannot be encoded in the specified encoding (eg: ``7bit``),
+ raise a :exc:`ValueError`. For :class:`~email.message.Message`, per
+ :rfc:`2046`, raise an error if a *cte* of ``quoted-printable`` or
+ ``base64`` is requested for *subtype* ``rfc822``, and for any *cte*
+ other than ``7bit`` for *subtype* ``external-body``. For
+ ``message/rfc822``, use ``8bit`` if *cte* is not specified. For all
+ other values of *subtype*, use ``7bit``.
+
+ .. note:: A *cte* of ``binary`` does not actually work correctly yet.
+ The ``Message`` object as modified by ``set_content`` is correct, but
+ :class:`~email.generator.BytesGenerator` does not serialize it
+ correctly.
+
+ If *disposition* is set, use it as the value of the
+ :mailheader:`Content-Disposition` header. If not specified, and
+ *filename* is specified, add the header with the value ``attachment``.
+ If it is not specified and *filename* is also not specified, do not add
+ the header. The only valid values for *disposition* are ``attachment``
+ and ``inline``.
+
+ If *filename* is specified, use it as the value of the ``filename``
+ parameter of the :mailheader:`Content-Disposition` header. There is no
+ default.
+
+ If *cid* is specified, add a :mailheader:`Content-ID` header with
+ *cid* as its value.
+
+ If *params* is specified, iterate its ``items`` method and use the
+ resulting ``(key, value)`` pairs to set additional paramters on the
+ :mailheader:`Content-Type` header.
+
+ If *headers* is specified and is a list of strings of the form
+ ``headername: headervalue`` or a list of ``header`` objects
+ (distinguised from strings by having a ``name`` attribute), add the
+ headers to *msg*.