Fred Drake | 295da24 | 1998-08-10 19:42:37 +0000 | [diff] [blame] | 1 | \section{\module{multifile} --- |
Fred Drake | 812860e | 1999-04-23 14:46:18 +0000 | [diff] [blame] | 2 | Support for files containing distinct parts} |
Fred Drake | b91e934 | 1998-07-23 17:59:49 +0000 | [diff] [blame] | 3 | |
Fred Drake | 812860e | 1999-04-23 14:46:18 +0000 | [diff] [blame] | 4 | \declaremodule{standard}{multifile} |
Fred Drake | d795c5c | 1998-08-07 15:55:14 +0000 | [diff] [blame] | 5 | \modulesynopsis{Support for reading files which contain distinct |
Fred Drake | 812860e | 1999-04-23 14:46:18 +0000 | [diff] [blame] | 6 | parts, such as some MIME data.} |
| 7 | \sectionauthor{Eric S. Raymond}{esr@snark.thyrsus.com} |
Fred Drake | b91e934 | 1998-07-23 17:59:49 +0000 | [diff] [blame] | 8 | |
Guido van Rossum | 8668e8e | 1998-06-28 17:55:53 +0000 | [diff] [blame] | 9 | |
Fred Drake | 1717ba4 | 1998-07-02 19:36:50 +0000 | [diff] [blame] | 10 | The \class{MultiFile} object enables you to treat sections of a text |
| 11 | file as file-like input objects, with \code{''} being returned by |
| 12 | \method{readline()} when a given delimiter pattern is encountered. The |
Guido van Rossum | 8668e8e | 1998-06-28 17:55:53 +0000 | [diff] [blame] | 13 | defaults of this class are designed to make it useful for parsing |
| 14 | MIME multipart messages, but by subclassing it and overriding methods |
| 15 | it can be easily adapted for more general use. |
| 16 | |
Fred Drake | 1717ba4 | 1998-07-02 19:36:50 +0000 | [diff] [blame] | 17 | \begin{classdesc}{MultiFile}{fp\optional{, seekable}} |
Guido van Rossum | 8668e8e | 1998-06-28 17:55:53 +0000 | [diff] [blame] | 18 | Create a multi-file. You must instantiate this class with an input |
Fred Drake | 1717ba4 | 1998-07-02 19:36:50 +0000 | [diff] [blame] | 19 | object argument for the \class{MultiFile} instance to get lines from, |
Raymond Hettinger | 999b57c | 2003-08-25 04:28:05 +0000 | [diff] [blame] | 20 | such as a file object returned by \function{open()}. |
Guido van Rossum | 8668e8e | 1998-06-28 17:55:53 +0000 | [diff] [blame] | 21 | |
Fred Drake | 1717ba4 | 1998-07-02 19:36:50 +0000 | [diff] [blame] | 22 | \class{MultiFile} only ever looks at the input object's |
| 23 | \method{readline()}, \method{seek()} and \method{tell()} methods, and |
| 24 | the latter two are only needed if you want random access to the |
| 25 | individual MIME parts. To use \class{MultiFile} on a non-seekable |
| 26 | stream object, set the optional \var{seekable} argument to false; this |
| 27 | will prevent using the input object's \method{seek()} and |
| 28 | \method{tell()} methods. |
Guido van Rossum | 8668e8e | 1998-06-28 17:55:53 +0000 | [diff] [blame] | 29 | \end{classdesc} |
| 30 | |
Fred Drake | 1717ba4 | 1998-07-02 19:36:50 +0000 | [diff] [blame] | 31 | It will be useful to know that in \class{MultiFile}'s view of the world, text |
Guido van Rossum | 8668e8e | 1998-06-28 17:55:53 +0000 | [diff] [blame] | 32 | is composed of three kinds of lines: data, section-dividers, and |
| 33 | end-markers. MultiFile is designed to support parsing of |
| 34 | messages that may have multiple nested message parts, each with its |
| 35 | own pattern for section-divider and end-marker lines. |
| 36 | |
Fred Drake | 2d3c03d | 2002-08-06 21:26:01 +0000 | [diff] [blame] | 37 | \begin{seealso} |
| 38 | \seemodule{email}{Comprehensive email handling package; supercedes |
| 39 | the \module{multifile} module.} |
| 40 | \end{seealso} |
| 41 | |
Fred Drake | d795c5c | 1998-08-07 15:55:14 +0000 | [diff] [blame] | 42 | |
| 43 | \subsection{MultiFile Objects \label{MultiFile-objects}} |
Guido van Rossum | 8668e8e | 1998-06-28 17:55:53 +0000 | [diff] [blame] | 44 | |
| 45 | A \class{MultiFile} instance has the following methods: |
| 46 | |
Guido van Rossum | 8668e8e | 1998-06-28 17:55:53 +0000 | [diff] [blame] | 47 | \begin{methoddesc}{readline}{str} |
| 48 | Read a line. If the line is data (not a section-divider or end-marker |
| 49 | or real EOF) return it. If the line matches the most-recently-stacked |
Guido van Rossum | 8ec619f | 1998-06-30 16:35:25 +0000 | [diff] [blame] | 50 | boundary, return \code{''} and set \code{self.last} to 1 or 0 according as |
Guido van Rossum | 8668e8e | 1998-06-28 17:55:53 +0000 | [diff] [blame] | 51 | the match is or is not an end-marker. If the line matches any other |
Fred Drake | 1717ba4 | 1998-07-02 19:36:50 +0000 | [diff] [blame] | 52 | stacked boundary, raise an error. On encountering end-of-file on the |
| 53 | underlying stream object, the method raises \exception{Error} unless |
| 54 | all boundaries have been popped. |
Guido van Rossum | 8668e8e | 1998-06-28 17:55:53 +0000 | [diff] [blame] | 55 | \end{methoddesc} |
| 56 | |
| 57 | \begin{methoddesc}{readlines}{str} |
Fred Drake | 1717ba4 | 1998-07-02 19:36:50 +0000 | [diff] [blame] | 58 | Return all lines remaining in this part as a list of strings. |
Guido van Rossum | 8668e8e | 1998-06-28 17:55:53 +0000 | [diff] [blame] | 59 | \end{methoddesc} |
| 60 | |
Fred Drake | 1717ba4 | 1998-07-02 19:36:50 +0000 | [diff] [blame] | 61 | \begin{methoddesc}{read}{} |
Guido van Rossum | 8668e8e | 1998-06-28 17:55:53 +0000 | [diff] [blame] | 62 | Read all lines, up to the next section. Return them as a single |
| 63 | (multiline) string. Note that this doesn't take a size argument! |
| 64 | \end{methoddesc} |
| 65 | |
Fred Drake | 1717ba4 | 1998-07-02 19:36:50 +0000 | [diff] [blame] | 66 | \begin{methoddesc}{seek}{pos\optional{, whence}} |
Guido van Rossum | 8668e8e | 1998-06-28 17:55:53 +0000 | [diff] [blame] | 67 | Seek. Seek indices are relative to the start of the current section. |
Fred Drake | 1717ba4 | 1998-07-02 19:36:50 +0000 | [diff] [blame] | 68 | The \var{pos} and \var{whence} arguments are interpreted as for a file |
| 69 | seek. |
Guido van Rossum | 8668e8e | 1998-06-28 17:55:53 +0000 | [diff] [blame] | 70 | \end{methoddesc} |
| 71 | |
Fred Drake | 1717ba4 | 1998-07-02 19:36:50 +0000 | [diff] [blame] | 72 | \begin{methoddesc}{tell}{} |
| 73 | Return the file position relative to the start of the current section. |
Guido van Rossum | 8668e8e | 1998-06-28 17:55:53 +0000 | [diff] [blame] | 74 | \end{methoddesc} |
| 75 | |
Fred Drake | f0ebbe0 | 2001-03-08 22:46:41 +0000 | [diff] [blame] | 76 | \begin{methoddesc}{next}{} |
| 77 | Skip lines to the next section (that is, read lines until a |
| 78 | section-divider or end-marker has been consumed). Return true if |
| 79 | there is such a section, false if an end-marker is seen. Re-enable |
| 80 | the most-recently-pushed boundary. |
| 81 | \end{methoddesc} |
| 82 | |
Guido van Rossum | 8668e8e | 1998-06-28 17:55:53 +0000 | [diff] [blame] | 83 | \begin{methoddesc}{is_data}{str} |
Fred Drake | 1717ba4 | 1998-07-02 19:36:50 +0000 | [diff] [blame] | 84 | Return true if \var{str} is data and false if it might be a section |
Fred Drake | 812860e | 1999-04-23 14:46:18 +0000 | [diff] [blame] | 85 | boundary. As written, it tests for a prefix other than \code{'-}\code{-'} at |
Fred Drake | 1717ba4 | 1998-07-02 19:36:50 +0000 | [diff] [blame] | 86 | start of line (which all MIME boundaries have) but it is declared so |
| 87 | it can be overridden in derived classes. |
Guido van Rossum | 8668e8e | 1998-06-28 17:55:53 +0000 | [diff] [blame] | 88 | |
| 89 | Note that this test is used intended as a fast guard for the real |
Fred Drake | 1717ba4 | 1998-07-02 19:36:50 +0000 | [diff] [blame] | 90 | boundary tests; if it always returns false it will merely slow |
| 91 | processing, not cause it to fail. |
Guido van Rossum | 8668e8e | 1998-06-28 17:55:53 +0000 | [diff] [blame] | 92 | \end{methoddesc} |
| 93 | |
Fred Drake | f0ebbe0 | 2001-03-08 22:46:41 +0000 | [diff] [blame] | 94 | \begin{methoddesc}{push}{str} |
| 95 | Push a boundary string. When an appropriately decorated version of |
| 96 | this boundary is found as an input line, it will be interpreted as a |
| 97 | section-divider or end-marker. All subsequent |
| 98 | reads will return the empty string to indicate end-of-file, until a |
| 99 | call to \method{pop()} removes the boundary a or \method{next()} call |
| 100 | reenables it. |
| 101 | |
| 102 | It is possible to push more than one boundary. Encountering the |
| 103 | most-recently-pushed boundary will return EOF; encountering any other |
| 104 | boundary will raise an error. |
| 105 | \end{methoddesc} |
| 106 | |
| 107 | \begin{methoddesc}{pop}{} |
| 108 | Pop a section boundary. This boundary will no longer be interpreted |
| 109 | as EOF. |
| 110 | \end{methoddesc} |
| 111 | |
Guido van Rossum | 8668e8e | 1998-06-28 17:55:53 +0000 | [diff] [blame] | 112 | \begin{methoddesc}{section_divider}{str} |
| 113 | Turn a boundary into a section-divider line. By default, this |
Fred Drake | 812860e | 1999-04-23 14:46:18 +0000 | [diff] [blame] | 114 | method prepends \code{'-}\code{-'} (which MIME section boundaries have) but |
Fred Drake | 1717ba4 | 1998-07-02 19:36:50 +0000 | [diff] [blame] | 115 | it is declared so it can be overridden in derived classes. This |
| 116 | method need not append LF or CR-LF, as comparison with the result |
| 117 | ignores trailing whitespace. |
Guido van Rossum | 8668e8e | 1998-06-28 17:55:53 +0000 | [diff] [blame] | 118 | \end{methoddesc} |
| 119 | |
| 120 | \begin{methoddesc}{end_marker}{str} |
| 121 | Turn a boundary string into an end-marker line. By default, this |
Fred Drake | 812860e | 1999-04-23 14:46:18 +0000 | [diff] [blame] | 122 | method prepends \code{'-}\code{-'} and appends \code{'-}\code{-'} (like a |
Fred Drake | 1717ba4 | 1998-07-02 19:36:50 +0000 | [diff] [blame] | 123 | MIME-multipart end-of-message marker) but it is declared so it can be |
Raymond Hettinger | 7e43110 | 2003-09-22 15:00:55 +0000 | [diff] [blame] | 124 | overridden in derived classes. This method need not append LF or |
Fred Drake | 1717ba4 | 1998-07-02 19:36:50 +0000 | [diff] [blame] | 125 | CR-LF, as comparison with the result ignores trailing whitespace. |
Guido van Rossum | 8668e8e | 1998-06-28 17:55:53 +0000 | [diff] [blame] | 126 | \end{methoddesc} |
| 127 | |
| 128 | Finally, \class{MultiFile} instances have two public instance variables: |
| 129 | |
| 130 | \begin{memberdesc}{level} |
Fred Drake | 1717ba4 | 1998-07-02 19:36:50 +0000 | [diff] [blame] | 131 | Nesting depth of the current part. |
Guido van Rossum | 8668e8e | 1998-06-28 17:55:53 +0000 | [diff] [blame] | 132 | \end{memberdesc} |
| 133 | |
| 134 | \begin{memberdesc}{last} |
Fred Drake | 1717ba4 | 1998-07-02 19:36:50 +0000 | [diff] [blame] | 135 | True if the last end-of-file was for an end-of-message marker. |
Guido van Rossum | 8668e8e | 1998-06-28 17:55:53 +0000 | [diff] [blame] | 136 | \end{memberdesc} |
| 137 | |
Fred Drake | 1717ba4 | 1998-07-02 19:36:50 +0000 | [diff] [blame] | 138 | |
Fred Drake | d795c5c | 1998-08-07 15:55:14 +0000 | [diff] [blame] | 139 | \subsection{\class{MultiFile} Example \label{multifile-example}} |
Fred Drake | 9164f88 | 2000-04-08 04:53:29 +0000 | [diff] [blame] | 140 | \sectionauthor{Skip Montanaro}{skip@mojam.com} |
Guido van Rossum | 8668e8e | 1998-06-28 17:55:53 +0000 | [diff] [blame] | 141 | |
| 142 | \begin{verbatim} |
Fred Drake | c2c46c3 | 2000-04-07 16:09:59 +0000 | [diff] [blame] | 143 | import mimetools |
Martin v. Löwis | d15a942 | 2000-09-30 17:04:40 +0000 | [diff] [blame] | 144 | import multifile |
Fred Drake | c2c46c3 | 2000-04-07 16:09:59 +0000 | [diff] [blame] | 145 | import StringIO |
| 146 | |
| 147 | def extract_mime_part_matching(stream, mimetype): |
| 148 | """Return the first element in a multipart MIME message on stream |
| 149 | matching mimetype.""" |
| 150 | |
| 151 | msg = mimetools.Message(stream) |
| 152 | msgtype = msg.gettype() |
| 153 | params = msg.getplist() |
| 154 | |
| 155 | data = StringIO.StringIO() |
| 156 | if msgtype[:10] == "multipart/": |
| 157 | |
| 158 | file = multifile.MultiFile(stream) |
| 159 | file.push(msg.getparam("boundary")) |
| 160 | while file.next(): |
| 161 | submsg = mimetools.Message(file) |
| 162 | try: |
| 163 | data = StringIO.StringIO() |
| 164 | mimetools.decode(file, data, submsg.getencoding()) |
| 165 | except ValueError: |
| 166 | continue |
| 167 | if submsg.gettype() == mimetype: |
| 168 | break |
| 169 | file.pop() |
| 170 | return data.getvalue() |
Guido van Rossum | 8668e8e | 1998-06-28 17:55:53 +0000 | [diff] [blame] | 171 | \end{verbatim} |