blob: 637aa9177672ba58747180349285344cae08fb04 [file] [log] [blame]
Georg Brandl116aa622007-08-15 14:28:22 +00001
2:mod:`sgmllib` --- Simple SGML parser
3=====================================
4
5.. module:: sgmllib
6 :synopsis: Only as much of an SGML parser as needed to parse HTML.
7
8
9.. index:: single: SGML
10
11This module defines a class :class:`SGMLParser` which serves as the basis for
12parsing text files formatted in SGML (Standard Generalized Mark-up Language).
13In fact, it does not provide a full SGML parser --- it only parses SGML insofar
14as it is used by HTML, and the module only exists as a base for the
15:mod:`htmllib` module. Another HTML parser which supports XHTML and offers a
16somewhat different interface is available in the :mod:`HTMLParser` module.
17
18
19.. class:: SGMLParser()
20
21 The :class:`SGMLParser` class is instantiated without arguments. The parser is
22 hardcoded to recognize the following constructs:
23
24 * Opening and closing tags of the form ``<tag attr="value" ...>`` and
25 ``</tag>``, respectively.
26
27 * Numeric character references of the form ``&#name;``.
28
29 * Entity references of the form ``&name;``.
30
31 * SGML comments of the form ``<!--text-->``. Note that spaces, tabs, and
32 newlines are allowed between the trailing ``>`` and the immediately preceding
33 ``--``.
34
35A single exception is defined as well:
36
37
38.. exception:: SGMLParseError
39
40 Exception raised by the :class:`SGMLParser` class when it encounters an error
41 while parsing.
42
Georg Brandl116aa622007-08-15 14:28:22 +000043:class:`SGMLParser` instances have the following methods:
44
45
46.. method:: SGMLParser.reset()
47
48 Reset the instance. Loses all unprocessed data. This is called implicitly at
49 instantiation time.
50
51
52.. method:: SGMLParser.setnomoretags()
53
54 Stop processing tags. Treat all following input as literal input (CDATA).
55 (This is only provided so the HTML tag ``<PLAINTEXT>`` can be implemented.)
56
57
58.. method:: SGMLParser.setliteral()
59
60 Enter literal mode (CDATA mode).
61
62
63.. method:: SGMLParser.feed(data)
64
65 Feed some text to the parser. It is processed insofar as it consists of
66 complete elements; incomplete data is buffered until more data is fed or
67 :meth:`close` is called.
68
69
70.. method:: SGMLParser.close()
71
72 Force processing of all buffered data as if it were followed by an end-of-file
73 mark. This method may be redefined by a derived class to define additional
74 processing at the end of the input, but the redefined version should always call
75 :meth:`close`.
76
77
78.. method:: SGMLParser.get_starttag_text()
79
80 Return the text of the most recently opened start tag. This should not normally
81 be needed for structured processing, but may be useful in dealing with HTML "as
82 deployed" or for re-generating input with minimal changes (whitespace between
83 attributes can be preserved, etc.).
84
85
86.. method:: SGMLParser.handle_starttag(tag, method, attributes)
87
88 This method is called to handle start tags for which either a :meth:`start_tag`
89 or :meth:`do_tag` method has been defined. The *tag* argument is the name of
90 the tag converted to lower case, and the *method* argument is the bound method
91 which should be used to support semantic interpretation of the start tag. The
92 *attributes* argument is a list of ``(name, value)`` pairs containing the
93 attributes found inside the tag's ``<>`` brackets.
94
95 The *name* has been translated to lower case. Double quotes and backslashes in
96 the *value* have been interpreted, as well as known character references and
97 known entity references terminated by a semicolon (normally, entity references
98 can be terminated by any non-alphanumerical character, but this would break the
99 very common case of ``<A HREF="url?spam=1&eggs=2">`` when ``eggs`` is a valid
100 entity name).
101
102 For instance, for the tag ``<A HREF="http://www.cwi.nl/">``, this method would
103 be called as ``unknown_starttag('a', [('href', 'http://www.cwi.nl/')])``. The
104 base implementation simply calls *method* with *attributes* as the only
105 argument.
106
Georg Brandl116aa622007-08-15 14:28:22 +0000107
108.. method:: SGMLParser.handle_endtag(tag, method)
109
110 This method is called to handle endtags for which an :meth:`end_tag` method has
111 been defined. The *tag* argument is the name of the tag converted to lower
112 case, and the *method* argument is the bound method which should be used to
113 support semantic interpretation of the end tag. If no :meth:`end_tag` method is
114 defined for the closing element, this handler is not called. The base
115 implementation simply calls *method*.
116
117
118.. method:: SGMLParser.handle_data(data)
119
120 This method is called to process arbitrary data. It is intended to be
121 overridden by a derived class; the base class implementation does nothing.
122
123
124.. method:: SGMLParser.handle_charref(ref)
125
126 This method is called to process a character reference of the form ``&#ref;``.
127 The base implementation uses :meth:`convert_charref` to convert the reference to
128 a string. If that method returns a string, it is passed to :meth:`handle_data`,
129 otherwise ``unknown_charref(ref)`` is called to handle the error.
130
Georg Brandl116aa622007-08-15 14:28:22 +0000131
132.. method:: SGMLParser.convert_charref(ref)
133
134 Convert a character reference to a string, or ``None``. *ref* is the reference
135 passed in as a string. In the base implementation, *ref* must be a decimal
136 number in the range 0-255. It converts the code point found using the
137 :meth:`convert_codepoint` method. If *ref* is invalid or out of range, this
138 method returns ``None``. This method is called by the default
139 :meth:`handle_charref` implementation and by the attribute value parser.
140
Georg Brandl116aa622007-08-15 14:28:22 +0000141
142.. method:: SGMLParser.convert_codepoint(codepoint)
143
144 Convert a codepoint to a :class:`str` value. Encodings can be handled here if
145 appropriate, though the rest of :mod:`sgmllib` is oblivious on this matter.
146
Georg Brandl116aa622007-08-15 14:28:22 +0000147
148.. method:: SGMLParser.handle_entityref(ref)
149
150 This method is called to process a general entity reference of the form
151 ``&ref;`` where *ref* is an general entity reference. It converts *ref* by
152 passing it to :meth:`convert_entityref`. If a translation is returned, it calls
153 the method :meth:`handle_data` with the translation; otherwise, it calls the
154 method ``unknown_entityref(ref)``. The default :attr:`entitydefs` defines
155 translations for ``&amp;``, ``&apos``, ``&gt;``, ``&lt;``, and ``&quot;``.
156
Georg Brandl116aa622007-08-15 14:28:22 +0000157
158.. method:: SGMLParser.convert_entityref(ref)
159
160 Convert a named entity reference to a :class:`str` value, or ``None``. The
161 resulting value will not be parsed. *ref* will be only the name of the entity.
162 The default implementation looks for *ref* in the instance (or class) variable
163 :attr:`entitydefs` which should be a mapping from entity names to corresponding
164 translations. If no translation is available for *ref*, this method returns
165 ``None``. This method is called by the default :meth:`handle_entityref`
166 implementation and by the attribute value parser.
167
Georg Brandl116aa622007-08-15 14:28:22 +0000168
169.. method:: SGMLParser.handle_comment(comment)
170
171 This method is called when a comment is encountered. The *comment* argument is
172 a string containing the text between the ``<!--`` and ``-->`` delimiters, but
173 not the delimiters themselves. For example, the comment ``<!--text-->`` will
174 cause this method to be called with the argument ``'text'``. The default method
175 does nothing.
176
177
178.. method:: SGMLParser.handle_decl(data)
179
180 Method called when an SGML declaration is read by the parser. In practice, the
181 ``DOCTYPE`` declaration is the only thing observed in HTML, but the parser does
182 not discriminate among different (or broken) declarations. Internal subsets in
183 a ``DOCTYPE`` declaration are not supported. The *data* parameter will be the
184 entire contents of the declaration inside the ``<!``...\ ``>`` markup. The
185 default implementation does nothing.
186
187
188.. method:: SGMLParser.report_unbalanced(tag)
189
190 This method is called when an end tag is found which does not correspond to any
191 open element.
192
193
194.. method:: SGMLParser.unknown_starttag(tag, attributes)
195
196 This method is called to process an unknown start tag. It is intended to be
197 overridden by a derived class; the base class implementation does nothing.
198
199
200.. method:: SGMLParser.unknown_endtag(tag)
201
202 This method is called to process an unknown end tag. It is intended to be
203 overridden by a derived class; the base class implementation does nothing.
204
205
206.. method:: SGMLParser.unknown_charref(ref)
207
208 This method is called to process unresolvable numeric character references.
209 Refer to :meth:`handle_charref` to determine what is handled by default. It is
210 intended to be overridden by a derived class; the base class implementation does
211 nothing.
212
213
214.. method:: SGMLParser.unknown_entityref(ref)
215
216 This method is called to process an unknown entity reference. It is intended to
217 be overridden by a derived class; the base class implementation does nothing.
218
219Apart from overriding or extending the methods listed above, derived classes may
220also define methods of the following form to define processing of specific tags.
221Tag names in the input stream are case independent; the *tag* occurring in
222method names must be in lower case:
223
224
225.. method:: SGMLParser.start_tag(attributes)
226 :noindex:
227
228 This method is called to process an opening tag *tag*. It has preference over
229 :meth:`do_tag`. The *attributes* argument has the same meaning as described for
230 :meth:`handle_starttag` above.
231
232
233.. method:: SGMLParser.do_tag(attributes)
234 :noindex:
235
236 This method is called to process an opening tag *tag* for which no
237 :meth:`start_tag` method is defined. The *attributes* argument has the same
238 meaning as described for :meth:`handle_starttag` above.
239
240
241.. method:: SGMLParser.end_tag()
242 :noindex:
243
244 This method is called to process a closing tag *tag*.
245
246Note that the parser maintains a stack of open elements for which no end tag has
247been found yet. Only tags processed by :meth:`start_tag` are pushed on this
248stack. Definition of an :meth:`end_tag` method is optional for these tags. For
249tags processed by :meth:`do_tag` or by :meth:`unknown_tag`, no :meth:`end_tag`
250method must be defined; if defined, it will not be used. If both
251:meth:`start_tag` and :meth:`do_tag` methods exist for a tag, the
252:meth:`start_tag` method takes precedence.
253