blob: 9c322249824359d7ceb71adf9ffe6605220c12dc [file] [log] [blame]
Daniel Veillardcec50a62003-10-28 13:26:51 +00001124907 HTML parse buffer problem when parsing larse in-memory docs
Daniel Veillardcec50a62003-10-28 13:26:51 +00002124110 DTD validation && wrong namespace
Daniel Veillardcec50a62003-10-28 13:26:51 +00003123564 xmllint --html --format
Daniel Veillard01791d51998-07-24 19:24:09 +00004
Daniel Veillardad8f99d2000-01-15 14:20:03 +00005 TODO for the XML parser and stuff:
6 ==================================
Daniel Veillard01791d51998-07-24 19:24:09 +00007
Daniel Veillardc4f65ab2003-04-21 23:07:45 +00008 $Id$
Daniel Veillarde39a93d2001-04-28 14:35:02 +00009
10 this tend to be outdated :-\ ...
Daniel Veillard361d8452000-04-03 19:48:13 +000011
Daniel Veillard28ae6362001-07-14 16:44:32 +000012DOCS:
13=====
14
15- use case of using XInclude to load for example a description.
16 order document + product base -(XSLT)-> quote with XIncludes
17 |
18 HTML output with description of parts <---(XSLT)--
19
Daniel Veillard75c29212000-03-14 18:33:47 +000020TODO:
21=====
Daniel Veillard8edf1c52003-07-22 20:52:14 +000022- XInclude at the SAX level (libSRVG)
Daniel Veillard4ec885a2001-06-17 10:31:07 +000023- fix the C code prototype to bring back doc/libxml-undocumented.txt
24 to a reasonable level
Daniel Veillardbf432752000-11-12 15:56:56 +000025- Computation of base when HTTP redirect occurs, might affect HTTP
26 interfaces.
Daniel Veillardc4f65ab2003-04-21 23:07:45 +000027- Computation of base in XInclude. Relativization of URIs.
Daniel Veillarde0854c32000-08-27 21:12:29 +000028- listing all attributes in a node.
Daniel Veillard75c29212000-03-14 18:33:47 +000029- Better checking of external parsed entities TAG 1234
Daniel Veillard75c29212000-03-14 18:33:47 +000030- Go through erratas and do the cleanup.
Daniel Veillarde0854c32000-08-27 21:12:29 +000031 http://www.w3.org/XML/xml-19980210-errata ... started ...
Daniel Veillard46057e12000-09-24 18:49:59 +000032- jamesh suggestion: SAX like functions to save a document ie. call a
33 function to open a new element with given attributes, write character
34 data, close last element, etc
Daniel Veillarde19fc232002-04-22 16:01:24 +000035 + inversted SAX, initial patch in April 2002 archives.
Daniel Veillarda6d8eb62000-12-27 10:46:47 +000036- htmlParseDoc has parameter encoding which is not used.
37 Function htmlCreateDocParserCtxt ignore it.
Daniel Veillardc2f4df22001-01-04 14:06:39 +000038- fix realloc() usage.
Daniel Veillard8cf14d52001-06-23 16:32:46 +000039- Stricten the UTF8 conformance (Martin Duerst):
40 http://www.w3.org/2001/06/utf-8-test/.
41 The bad files are in http://www.w3.org/2001/06/utf-8-wrong/.
Daniel Veillard7bd8b4b2005-01-07 13:56:19 +000042- xml:id normalized value
Daniel Veillardad8f99d2000-01-15 14:20:03 +000043
44TODO:
45=====
46
Daniel Veillarde19fc232002-04-22 16:01:24 +000047- move all string manipulation functions (xmlStrdup, xmlStrlen, etc.) to
48 global.c. Bjorn noted that the following files depends on parser.o solely
49 because of these string functions: entities.o, global.o, hash.o, tree.o,
50 xmlIO.o, and xpath.o.
Daniel Veillardad8f99d2000-01-15 14:20:03 +000051
Daniel Veillard3dd82e72000-03-20 11:48:04 +000052- Optimization of tag strings allocation ?
Daniel Veillard75c29212000-03-14 18:33:47 +000053
54- maintain coherency of namespace when doing cut'n paste operations
55 => the functions are coded, but need testing
56
Daniel Veillard3dd82e72000-03-20 11:48:04 +000057- function to rebuild the ID table
58- functions to rebuild the DTD hash tables (after DTD changes).
Daniel Veillard75c29212000-03-14 18:33:47 +000059
Daniel Veillard75c29212000-03-14 18:33:47 +000060
61EXTENSIONS:
62===========
Daniel Veillard3dd82e72000-03-20 11:48:04 +000063
Daniel Veillard75c29212000-03-14 18:33:47 +000064- Tools to produce man pages from the SGML docs.
Daniel Veillardad8f99d2000-01-15 14:20:03 +000065
66- Add Xpointer recognition/API
67
68- Add Xlink recognition/API
69 => started adding an xlink.[ch] with a unified API for XML and HTML.
Daniel Veillard496a1cf2000-05-03 14:20:55 +000070 it's crap :-(
Daniel Veillardad8f99d2000-01-15 14:20:03 +000071
Daniel Veillardad8f99d2000-01-15 14:20:03 +000072- Implement XSchemas
Daniel Veillard496a1cf2000-05-03 14:20:55 +000073 => Really need to be done <grin/>
Daniel Veillardc4f65ab2003-04-21 23:07:45 +000074 - datatype are complete, but structure support is very limited.
Daniel Veillardad8f99d2000-01-15 14:20:03 +000075
Daniel Veillardad8f99d2000-01-15 14:20:03 +000076- extend the shell with:
77 - edit
78 - load/save
79 - mv (yum, yum, but it's harder because directories are ordered in
80 our case, mvup and mvdown would be required)
81
Daniel Veillardc4f65ab2003-04-21 23:07:45 +000082
83Done:
84=====
85
Daniel Veillardad8f99d2000-01-15 14:20:03 +000086- Add HTML validation using the XHTML DTD
87 - problem: do we want to keep and maintain the code for handling
88 DTD/System ID cache directly in libxml ?
Daniel Veillardc4f65ab2003-04-21 23:07:45 +000089 => not really done that way, but there are new APIs to check elements
90 or attributes. Otherwise XHTML validation directly ...
91
92- XML Schemas datatypes except Base64 and BinHex
93
94- Relax NG validation
95
96- XmlTextReader streaming API + validation
Daniel Veillardad8f99d2000-01-15 14:20:03 +000097
98- Add a DTD cache prefilled with xhtml DTDs and entities and a program to
99 manage them -> like the /usr/bin/install-catalog from SGML
100 right place seems $datadir/xmldtds
Daniel Veillard361d8452000-04-03 19:48:13 +0000101 Maybe this is better left to user apps
Daniel Veillardc4f65ab2003-04-21 23:07:45 +0000102 => use a catalog instead , and xhtml1-dtd package
Daniel Veillardad8f99d2000-01-15 14:20:03 +0000103
Daniel Veillardc4f65ab2003-04-21 23:07:45 +0000104- Add output to XHTML
105 => XML serializer automatically recognize the DTd and apply the specific
106 rules.
Daniel Veillardad8f99d2000-01-15 14:20:03 +0000107
Daniel Veillardc4f65ab2003-04-21 23:07:45 +0000108- Fix output of <tst val="x&#xA;y"/>
Daniel Veillarde3d88ef2000-01-24 13:55:06 +0000109
Daniel Veillardc4f65ab2003-04-21 23:07:45 +0000110- compliance to XML-Namespace checking, see section 6 of
111 http://www.w3.org/TR/REC-xml-names/
112
113- Correct standalone checking/emitting (hard)
114 2.9 Standalone Document Declaration
Daniel Veillardad8f99d2000-01-15 14:20:03 +0000115
Daniel Veillarde19fc232002-04-22 16:01:24 +0000116- Implement OASIS XML Catalog support
117 http://www.oasis-open.org/committees/entity/
118
119- Get OASIS testsuite to a more friendly result, check all the results
120 once stable. the check-xml-test-suite.py script does this
121
Daniel Veillard96ed5832001-06-15 22:22:04 +0000122- Implement XSLT
123 => libxslt
124
125- Finish XPath
126 => attributes addressing troubles
127 => defaulted attributes handling
128 => namespace axis ?
129 done as XSLT got debugged
130
Daniel Veillardd7e4d652001-06-15 14:57:10 +0000131- bug reported by Michael Meallin on validation problems
132 => Actually means I need to add support (and warn) for non-deterministic
133 content model.
134- Handle undefined namespaces in entity contents better ... at least
135 issue a warning
Daniel Veillard9403a042001-05-28 11:00:53 +0000136- DOM needs
137 int xmlPruneProp(xmlNodePtr node, xmlAtttrPtr attr);
138 => done it's actually xmlRemoveProp xmlUnsetProp xmlUnsetNsProp
139
Daniel Veillard1e851392000-10-15 10:02:56 +0000140- HTML: handling of Script and style data elements, need special code in
141 the parser and saving functions (handling of < > " ' ...):
142 http://www.w3.org/TR/html4/types.html#type-script
143 Attributes are no problems since entities are accepted.
Daniel Veillarde0854c32000-08-27 21:12:29 +0000144- DOM needs
145 xmlAttrPtr xmlNewDocProp(xmlDocPtr doc, const xmlChar *name, const xmlChar *value)
Daniel Veillard2f2bf412000-08-20 15:11:02 +0000146- problem when parsing hrefs with & with the HTML parser (IRC ac)
147- If the internal encoding is not UTF8 saving to a given encoding doesn't
148 work => fix to force UTF8 encoding ...
149 done, added documentation too
Daniel Veillard32bc74e2000-07-14 14:49:25 +0000150- Add an ASCII I/O encoder (asciiToUTF8 and UTF8Toascii)
151- Issue warning when using non-absolute namespaces URI.
152- the html parser should add <head> and <body> if they don't exist
153 started, not finished.
154 Done, the automatic closing is added and 3 testcases were inserted
Daniel Veillard3f6f7f62000-06-30 17:58:25 +0000155- Command to force the parser to stop parsing and ignore the rest of the file.
Daniel Veillard32bc74e2000-07-14 14:49:25 +0000156 xmlStopParser() should allow this, mostly untested
Daniel Veillardbe803962000-06-28 23:40:59 +0000157- support for HTML empty attributes like <hr noshade>
158- plugged iconv() in for support of a large set of encodings.
159- xmlSwitchToEncoding() rewrite done
Daniel Veillard496a1cf2000-05-03 14:20:55 +0000160- URI checkings (no fragments) rfc2396.txt
Daniel Veillard5d211f42000-04-07 17:00:24 +0000161- Added a clean mechanism for overload or added input methods:
162 xmlRegisterInputCallbacks()
Daniel Veillard361d8452000-04-03 19:48:13 +0000163- dynamically adapt the alloc entry point to use g_alloc()/g_free()
164 if the programmer wants it:
165 - use xmlMemSetup() to reset the routines used.
166- Check attribute normalization especially xmlGetProp()
167- Validity checking problems for NOTATIONS attributes
168- Validity checking problems for ENTITY ENTITIES attributes
169- Parsing of a well balanced chunk xmlParseBalancedChunkMemory()
170- URI module: validation, base, etc ... see uri.[ch]
171- turn tester into a generic program xmllint installed with libxml
Daniel Veillardec303412000-03-24 13:41:54 +0000172- extend validity checks to go through entities content instead of
173 just labelling them PCDATA
Daniel Veillard75c29212000-03-14 18:33:47 +0000174- Save Dtds using the children list instead of dumping the tables,
175 order is preserved as well as comments and PIs
176- Wrote a notice of changes requires to go from 1.x to 2.x
177- make sure that all SAX callbacks are disabled if a WF error is detected
178- checking/handling of newline normalization
179 http://localhost/www.xml.com/axml/target.html#sec-line-ends
180- correct checking of '&' '%' on entities content.
181- checking of PE/Nesting on entities declaration
182- checking/handling of xml:space
183 - checking done.
184 - handling done, not well tested
185- Language identification code, productions [33] to [38]
186 => done, the check has been added and report WFness errors
187- Conditional sections in DTDs [61] to [65]
188 => should this crap be really implemented ???
189 => Yep OASIS testsuite uses them
190- Allow parsed entities defined in the internal subset to override
191 the ones defined in the external subset (DtD customization).
192 => This mean that the entity content should be computed only at
193 use time, i.e. keep the orig string only at parse time and expand
194 only when referenced from the external subset :-(
195 Needed for complete use of most DTD from Eve Maler
196- Add regression tests for all WFC errors
197 => did some in test/WFC
198 => added OASIS testsuite routines
199 http://xmlsoft.org/conf/result.html
200
201- I18N: http://wap.trondheim.com/vaer/index.phtml is not XML and accepted
202 by the XML parser, UTF-8 should be checked when there is no "encoding"
203 declared !
204- Support for UTF-8 and UTF-16 encoding
205 => added some convertion routines provided by Martin Durst
206 patched them, got fixes from @@@
207 I plan to keep everything internally as UTF-8 (or ISO-Latin-X)
208 this is slightly more costly but more compact, and recent processors
209 efficiency is cache related. The key for good performances is keeping
210 the data set small, so will I.
211 => the new progressive reading routines call the detection code
212 is enabled, tested the ISO->UTF-8 stuff
Daniel Veillardad8f99d2000-01-15 14:20:03 +0000213- External entities loading:
214 - allow override by client code
215 - make sure it is alled for all external entities referenced
216 Done, client code should use xmlSetExternalEntityLoader() to set
217 the default loading routine. It will be called each time an external
218 entity entity resolution is triggered.
219- maintain ID coherency when removing/changing attributes
220 The function used to deallocate attributes now check for it being an
221 ID and removes it from the table.
222- push mode parsing i.e. non-blocking state based parser
223 done, both for XML and HTML parsers. Use xmlCreatePushParserCtxt()
224 and xmlParseChunk() and html counterparts.
225 The tester program now has a --push option to select that parser
226 front-end. Douplicated tests to use both and check results are similar.
227
228- Most of XPath, still see some troubles and occasionnal memleaks.
229- an XML shell, allowing to traverse/manipulate an XML document with
230 a shell like interface, and using XPath for the anming syntax
231 - use of readline and history added when available
232 - the shell interface has been cleanly separated and moved to debugXML.c
233- HTML parser, should be fairly stable now
234- API to search the lang of an attribute
235- Collect IDs at parsing and maintain a table.
236 PBM: maintain the table coherency
237 PBM: how to detect ID types in absence of DtD !
238- Use it for XPath ID support
239- Add validity checking
240 Should be finished now !
241- Add regression tests with entity substitutions
242
243- External Parsed entities, either XML or external Subset [78] and [79]
244 parsing the xmllang DtD now works, so it should be sufficient for
245 most cases !
246
247- progressive reading. The entity support is a first step toward
Daniel Veillard260a68f1998-08-13 03:39:55 +0000248 asbtraction of an input stream. A large part of the context is still
249 located on the stack, moving to a state machine and putting everyting
250 in the parsing context should provide an adequate solution.
Daniel Veillardad8f99d2000-01-15 14:20:03 +0000251 => Rather than progressive parsing, give more power to the SAX-like
252 interface. Currently the DOM-like representation is built but
253 => it should be possible to define that only as a set of SAX callbacks
254 and remove the tree creation from the parser code.
255 DONE
Daniel Veillard01791d51998-07-24 19:24:09 +0000256
Daniel Veillardad8f99d2000-01-15 14:20:03 +0000257- DOM support, instead of using a proprietary in memory
258 format for the document representation, the parser should
259 call a DOM API to actually build the resulting document.
260 Then the parser becomes independent of the in-memory
261 representation of the document. Even better using RPC's
262 the parser can actually build the document in another
263 program.
264 => Work started, now the internal representation is by default
265 very near a direct DOM implementation. The DOM glue is implemented
266 as a separate module. See the GNOME gdome module.
267
Daniel Veillard01791d51998-07-24 19:24:09 +0000268- C++ support : John Ehresman <jehresma@dsg.harvard.edu>
269- Updated code to follow more recent specs, added compatibility flag
Daniel Veillard260a68f1998-08-13 03:39:55 +0000270- Better error handling, use a dedicated, overridable error
271 handling function.
272- Support for CDATA.
273- Keep track of line numbers for better error reporting.
274- Support for PI (SAX one).
Daniel Veillardad8f99d2000-01-15 14:20:03 +0000275- Support for Comments (bad, should be in ASAP, they are parsed
276 but not stored), should be configurable.
277- Improve the support of entities on save (+SAX).
Daniel Veillard01791d51998-07-24 19:24:09 +0000278