TODO - fp2-dev/platform/external/libxml2 - Gitiles


            TODO for the XML parser and stuff:
 	   ==================================


 TODO:
 =====

 - Computation of base when HTTP redirect occurs, might affect HTTP
   interfaces.
 - performances: there is still improvements needed when parsing Docbook DTD
   a single function to optimize/avoid.
 - DOM needs
   int xmlPruneProp(xmlNodePtr node, xmlAtttrPtr attr);
 - listing all attributes in a node.
 - Correct standalone checking/emitting (hard)
   2.9 Standalone Document Declaration
 - Better checking of external parsed entities TAG 1234
 - Find way of representing PERefs in the Dtd so that %entity; can
   be saved back.
 - Go through erratas and do the cleanup.
   http://www.w3.org/XML/xml-19980210-errata ... started ...
 - Handle undefined namespaces in entity contents better ... at least
   issue a warning
 - fix --disable-corba configure switch handling, and use XML_WITHOUT_CORBA
   not WITHOUT_CORBA flag
 - reported by Michael, update of doc node when pasting on a new document
   There can be far more than simply the doc pointer which refers to
   the old document, for example namespace declarations or entities
   references can also be a nasty problem, far more than updating the
   doc values.
 - jamesh suggestion: SAX like functions to save a document ie. call a
   function to open a new element with given attributes, write character
   data, close last element, etc
 - htmlParseDoc has parameter encoding which is not used.
   Function htmlCreateDocParserCtxt ignore it.
 - bug reported by Michael Meallin on validation problems
 - solve the problems related to post-validating of XInclude results.

 TODO:
 =====

 - Get OASIS testsuite to a more friendly result, check all the results
   once stable. Current state at:
   http://xmlsoft.org/conf/result.html

 - Optimization of tag strings allocation ?

 - maintain coherency of namespace when doing cut'n paste operations
   => the functions are coded, but need testing

 - function to rebuild the ID table
 - functions to rebuild the DTD hash tables (after DTD changes).


 EXTENSIONS:
 ===========

 - Tools to produce man pages from the SGML docs.
 - Finish XPath
   => attributes addressing troubles
   => defaulted attributes handling
   => namespace axis ?

 - Add Xpointer recognition/API

 - Add Xlink recognition/API
   => started adding an xlink.[ch] with a unified API for XML and HTML.
      it's crap :-(

 - Implement XSLT
   => seems that someone volunteered ?!?

 - Implement XSchemas
   => Really need to be done <grin/>

 - O2K parsing;
   => this is a somewhat ugly mix of HTML and XML, adding a specific
      routine in the comment parsing code of HTML and plug the XML
      parsing one in-there should not be too hard. Key point is to get
      XSL to transform all this to something decent ...

 - extend the shell with:
    - edit
    - load/save
    - mv (yum, yum, but it's harder because directories are ordered in
      our case, mvup and mvdown would be required)

 - Add HTML validation using the XHTML DTD
   - problem: do we want to keep and maintain the code for handling
     DTD/System ID cache directly in libxml ?

 - Add a DTD cache prefilled with xhtml DTDs and entities and a program to
   manage them -> like the /usr/bin/install-catalog from SGML
   right place seems $datadir/xmldtds
   Maybe this is better left to user apps

 - Add output to XHTML in case of HTML documents.


 Done:
 =====

 - HTML: handling of Script and style data elements, need special code in
   the parser and saving functions (handling of < > " ' ...):
   http://www.w3.org/TR/html4/types.html#type-script
   Attributes are no problems since entities are accepted.
 - DOM needs
   xmlAttrPtr xmlNewDocProp(xmlDocPtr doc, const xmlChar *name, const xmlChar *value)
 - problem when parsing hrefs with & with the HTML parser (IRC ac)
 - If the internal encoding is not UTF8 saving to a given encoding doesn't
   work => fix to force UTF8 encoding ...
   done, added documentation too
 - Add an ASCII I/O encoder (asciiToUTF8 and UTF8Toascii)
 - Issue warning when using non-absolute namespaces URI.
 - the html parser should add <head> and <body> if they don't exist
   started, not finished.
   Done, the automatic closing is added and 3 testcases were inserted
 - Command to force the parser to stop parsing and ignore the rest of the file.
   xmlStopParser() should allow this, mostly untested
 - support for HTML empty attributes like <hr noshade>
 - plugged iconv() in for support of a large set of encodings.
 - xmlSwitchToEncoding() rewrite done
 - URI checkings (no fragments) rfc2396.txt
 - Added a clean mechanism for overload or added input methods:
   xmlRegisterInputCallbacks()
 - dynamically adapt the alloc entry point to use g_alloc()/g_free()
   if the programmer wants it:
     - use xmlMemSetup() to reset the routines used.
 - Check attribute normalization especially xmlGetProp()
 - Validity checking problems for NOTATIONS attributes
 - Validity checking problems for ENTITY ENTITIES attributes
 - Parsing of a well balanced chunk xmlParseBalancedChunkMemory()
 - URI module: validation, base, etc ... see uri.[ch]
 - turn tester into a generic program xmllint installed with libxml
 - extend validity checks to go through entities content instead of
   just labelling them PCDATA
 - Save Dtds using the children list instead of dumping the tables,
   order is preserved as well as comments and PIs
 - Wrote a notice of changes requires to go from 1.x to 2.x
 - make sure that all SAX callbacks are disabled if a WF error is detected
 - checking/handling of newline normalization
   http://localhost/www.xml.com/axml/target.html#sec-line-ends
 - correct checking of '&' '%' on entities content.
 - checking of PE/Nesting on entities declaration
 - checking/handling of xml:space
    - checking done.
    - handling done, not well tested
 - Language identification code, productions [33] to [38]
   => done, the check has been added and report WFness errors
 - Conditional sections in DTDs [61] to [65]
   => should this crap be really implemented ???
   => Yep OASIS testsuite uses them
 - Allow parsed entities defined in the internal subset to override
   the ones defined in the external subset (DtD customization).
   => This mean that the entity content should be computed only at
      use time, i.e. keep the orig string only at parse time and expand
      only when referenced from the external subset :-(
      Needed for complete use of most DTD from Eve Maler
 - Add regression tests for all WFC errors
   => did some in test/WFC
   => added OASIS testsuite routines
      http://xmlsoft.org/conf/result.html

 - I18N: http://wap.trondheim.com/vaer/index.phtml is not XML and accepted
   by the XML parser, UTF-8 should be checked when there is no "encoding"
   declared !
 - Support for UTF-8 and UTF-16 encoding
   => added some convertion routines provided by Martin Durst
      patched them, got fixes from @@@
      I plan to keep everything internally as UTF-8 (or ISO-Latin-X)
      this is slightly more costly but more compact, and recent processors
      efficiency is cache related. The key for good performances is keeping
      the data set small, so will I.
   => the new progressive reading routines call the detection code
      is enabled, tested the ISO->UTF-8 stuff
 - External entities loading:
    - allow override by client code
    - make sure it is alled for all external entities referenced
   Done, client code should use xmlSetExternalEntityLoader() to set
   the default loading routine. It will be called each time an external
   entity entity resolution is triggered.
 - maintain ID coherency when removing/changing attributes
   The function used to deallocate attributes now check for it being an
   ID and removes it from the table.
 - push mode parsing i.e. non-blocking state based parser
   done, both for XML and HTML parsers. Use xmlCreatePushParserCtxt()
   and xmlParseChunk() and html counterparts.
   The tester program now has a --push option to select that parser
   front-end. Douplicated tests to use both and check results are similar.

 - Most of XPath, still see some troubles and occasionnal memleaks.
 - an XML shell, allowing to traverse/manipulate an XML document with
   a shell like interface, and using XPath for the anming syntax
   - use of readline and history added when available
   - the shell interface has been cleanly separated and moved to debugXML.c
 - HTML parser, should be fairly stable now
 - API to search the lang of an attribute
 - Collect IDs at parsing and maintain a table.
    PBM: maintain the table coherency
    PBM: how to detect ID types in absence of DtD !
 - Use it for XPath ID support
 - Add validity checking
   Should be finished now !
 - Add regression tests with entity substitutions

 - External Parsed entities, either XML or external Subset [78] and [79]
   parsing the xmllang DtD now works, so it should be sufficient for
   most cases !

 - progressive reading. The entity support is a first step toward
   asbtraction of an input stream. A large part of the context is still
   located on the stack, moving to a state machine and putting everyting
   in the parsing context should provide an adequate solution.
   => Rather than progressive parsing, give more power to the SAX-like
      interface. Currently the DOM-like representation is built but
      => it should be possible to define that only as a set of SAX callbacks
 	and remove the tree creation from the parser code.
 	DONE

 - DOM support, instead of using a proprietary in memory
   format for the document representation, the parser should
   call a DOM API to actually build the resulting document.
   Then the parser becomes independent of the in-memory
   representation of the document. Even better using RPC's
   the parser can actually build the document in another
   program.
   => Work started, now the internal representation is by default
      very near a direct DOM implementation. The DOM glue is implemented
      as a separate module. See the GNOME gdome module.

 - C++ support : John Ehresman <jehresma@dsg.harvard.edu>
 - Updated code to follow more recent specs, added compatibility flag
 - Better error handling, use a dedicated, overridable error
   handling function.
 - Support for CDATA.
 - Keep track of line numbers for better error reporting.
 - Support for PI (SAX one).
 - Support for Comments (bad, should be in ASAP, they are parsed
   but not stored), should be configurable.
 - Improve the support of entities on save (+SAX).

	TODO for the XML parser and stuff:
	==================================


	TODO:
	=====

	- Computation of base when HTTP redirect occurs, might affect HTTP
	interfaces.
	- performances: there is still improvements needed when parsing Docbook DTD
	a single function to optimize/avoid.
	- DOM needs
	int xmlPruneProp(xmlNodePtr node, xmlAtttrPtr attr);
	- listing all attributes in a node.
	- Correct standalone checking/emitting (hard)
	2.9 Standalone Document Declaration
	- Better checking of external parsed entities TAG 1234
	- Find way of representing PERefs in the Dtd so that %entity; can
	be saved back.
	- Go through erratas and do the cleanup.
	http://www.w3.org/XML/xml-19980210-errata ... started ...
	- Handle undefined namespaces in entity contents better ... at least
	issue a warning
	- fix --disable-corba configure switch handling, and use XML_WITHOUT_CORBA
	not WITHOUT_CORBA flag
	- reported by Michael, update of doc node when pasting on a new document
	There can be far more than simply the doc pointer which refers to
	the old document, for example namespace declarations or entities
	references can also be a nasty problem, far more than updating the
	doc values.
	- jamesh suggestion: SAX like functions to save a document ie. call a
	function to open a new element with given attributes, write character
	data, close last element, etc
	- htmlParseDoc has parameter encoding which is not used.
	Function htmlCreateDocParserCtxt ignore it.
	- bug reported by Michael Meallin on validation problems
	- solve the problems related to post-validating of XInclude results.

	TODO:
	=====

	- Get OASIS testsuite to a more friendly result, check all the results
	once stable. Current state at:
	http://xmlsoft.org/conf/result.html

	- Optimization of tag strings allocation ?

	- maintain coherency of namespace when doing cut'n paste operations
	=> the functions are coded, but need testing

	- function to rebuild the ID table
	- functions to rebuild the DTD hash tables (after DTD changes).


	EXTENSIONS:
	===========

	- Tools to produce man pages from the SGML docs.
	- Finish XPath
	=> attributes addressing troubles
	=> defaulted attributes handling
	=> namespace axis ?

	- Add Xpointer recognition/API

	- Add Xlink recognition/API
	=> started adding an xlink.[ch] with a unified API for XML and HTML.
	it's crap :-(

	- Implement XSLT
	=> seems that someone volunteered ?!?

	- Implement XSchemas
	=> Really need to be done <grin/>

	- O2K parsing;
	=> this is a somewhat ugly mix of HTML and XML, adding a specific
	routine in the comment parsing code of HTML and plug the XML
	parsing one in-there should not be too hard. Key point is to get
	XSL to transform all this to something decent ...

	- extend the shell with:
	- edit
	- load/save
	- mv (yum, yum, but it's harder because directories are ordered in
	our case, mvup and mvdown would be required)

	- Add HTML validation using the XHTML DTD
	- problem: do we want to keep and maintain the code for handling
	DTD/System ID cache directly in libxml ?

	- Add a DTD cache prefilled with xhtml DTDs and entities and a program to
	manage them -> like the /usr/bin/install-catalog from SGML
	right place seems $datadir/xmldtds
	Maybe this is better left to user apps

	- Add output to XHTML in case of HTML documents.


	Done:
	=====

	- HTML: handling of Script and style data elements, need special code in
	the parser and saving functions (handling of < > " ' ...):
	http://www.w3.org/TR/html4/types.html#type-script
	Attributes are no problems since entities are accepted.
	- DOM needs
	xmlAttrPtr xmlNewDocProp(xmlDocPtr doc, const xmlChar name, const xmlChar value)
	- problem when parsing hrefs with & with the HTML parser (IRC ac)
	- If the internal encoding is not UTF8 saving to a given encoding doesn't
	work => fix to force UTF8 encoding ...
	done, added documentation too
	- Add an ASCII I/O encoder (asciiToUTF8 and UTF8Toascii)
	- Issue warning when using non-absolute namespaces URI.
	- the html parser should add <head> and <body> if they don't exist
	started, not finished.
	Done, the automatic closing is added and 3 testcases were inserted
	- Command to force the parser to stop parsing and ignore the rest of the file.
	xmlStopParser() should allow this, mostly untested
	- support for HTML empty attributes like <hr noshade>
	- plugged iconv() in for support of a large set of encodings.
	- xmlSwitchToEncoding() rewrite done
	- URI checkings (no fragments) rfc2396.txt
	- Added a clean mechanism for overload or added input methods:
	xmlRegisterInputCallbacks()
	- dynamically adapt the alloc entry point to use g_alloc()/g_free()
	if the programmer wants it:
	- use xmlMemSetup() to reset the routines used.
	- Check attribute normalization especially xmlGetProp()
	- Validity checking problems for NOTATIONS attributes
	- Validity checking problems for ENTITY ENTITIES attributes
	- Parsing of a well balanced chunk xmlParseBalancedChunkMemory()
	- URI module: validation, base, etc ... see uri.[ch]
	- turn tester into a generic program xmllint installed with libxml
	- extend validity checks to go through entities content instead of
	just labelling them PCDATA
	- Save Dtds using the children list instead of dumping the tables,
	order is preserved as well as comments and PIs
	- Wrote a notice of changes requires to go from 1.x to 2.x
	- make sure that all SAX callbacks are disabled if a WF error is detected
	- checking/handling of newline normalization
	http://localhost/www.xml.com/axml/target.html#sec-line-ends
	- correct checking of '&' '%' on entities content.
	- checking of PE/Nesting on entities declaration
	- checking/handling of xml:space
	- checking done.
	- handling done, not well tested
	- Language identification code, productions [33] to [38]
	=> done, the check has been added and report WFness errors
	- Conditional sections in DTDs [61] to [65]
	=> should this crap be really implemented ???
	=> Yep OASIS testsuite uses them
	- Allow parsed entities defined in the internal subset to override
	the ones defined in the external subset (DtD customization).
	=> This mean that the entity content should be computed only at
	use time, i.e. keep the orig string only at parse time and expand
	only when referenced from the external subset :-(
	Needed for complete use of most DTD from Eve Maler
	- Add regression tests for all WFC errors
	=> did some in test/WFC
	=> added OASIS testsuite routines
	http://xmlsoft.org/conf/result.html

	- I18N: http://wap.trondheim.com/vaer/index.phtml is not XML and accepted
	by the XML parser, UTF-8 should be checked when there is no "encoding"
	declared !
	- Support for UTF-8 and UTF-16 encoding
	=> added some convertion routines provided by Martin Durst
	patched them, got fixes from @@@
	I plan to keep everything internally as UTF-8 (or ISO-Latin-X)
	this is slightly more costly but more compact, and recent processors
	efficiency is cache related. The key for good performances is keeping
	the data set small, so will I.
	=> the new progressive reading routines call the detection code
	is enabled, tested the ISO->UTF-8 stuff
	- External entities loading:
	- allow override by client code
	- make sure it is alled for all external entities referenced
	Done, client code should use xmlSetExternalEntityLoader() to set
	the default loading routine. It will be called each time an external
	entity entity resolution is triggered.
	- maintain ID coherency when removing/changing attributes
	The function used to deallocate attributes now check for it being an
	ID and removes it from the table.
	- push mode parsing i.e. non-blocking state based parser
	done, both for XML and HTML parsers. Use xmlCreatePushParserCtxt()
	and xmlParseChunk() and html counterparts.
	The tester program now has a --push option to select that parser
	front-end. Douplicated tests to use both and check results are similar.

	- Most of XPath, still see some troubles and occasionnal memleaks.
	- an XML shell, allowing to traverse/manipulate an XML document with
	a shell like interface, and using XPath for the anming syntax
	- use of readline and history added when available
	- the shell interface has been cleanly separated and moved to debugXML.c
	- HTML parser, should be fairly stable now
	- API to search the lang of an attribute
	- Collect IDs at parsing and maintain a table.
	PBM: maintain the table coherency
	PBM: how to detect ID types in absence of DtD !
	- Use it for XPath ID support
	- Add validity checking
	Should be finished now !
	- Add regression tests with entity substitutions

	- External Parsed entities, either XML or external Subset [78] and [79]
	parsing the xmllang DtD now works, so it should be sufficient for
	most cases !

	- progressive reading. The entity support is a first step toward
	asbtraction of an input stream. A large part of the context is still
	located on the stack, moving to a state machine and putting everyting
	in the parsing context should provide an adequate solution.
	=> Rather than progressive parsing, give more power to the SAX-like
	interface. Currently the DOM-like representation is built but
	=> it should be possible to define that only as a set of SAX callbacks
	and remove the tree creation from the parser code.
	DONE

	- DOM support, instead of using a proprietary in memory
	format for the document representation, the parser should
	call a DOM API to actually build the resulting document.
	Then the parser becomes independent of the in-memory
	representation of the document. Even better using RPC's
	the parser can actually build the document in another
	program.
	=> Work started, now the internal representation is by default
	very near a direct DOM implementation. The DOM glue is implemented
	as a separate module. See the GNOME gdome module.

	- C++ support : John Ehresman <jehresma@dsg.harvard.edu>
	- Updated code to follow more recent specs, added compatibility flag
	- Better error handling, use a dedicated, overridable error
	handling function.
	- Support for CDATA.
	- Keep track of line numbers for better error reporting.
	- Support for PI (SAX one).
	- Support for Comments (bad, should be in ASAP, they are parsed
	but not stored), should be configurable.
	- Improve the support of entities on save (+SAX).