| <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" |
| "http://www.w3.org/TR/REC-html40/loose.dtd"> |
| <html> |
| <head> |
| <title>Libxml memory management</title> |
| <meta name="GENERATOR" content="amaya V3.2"> |
| <meta http-equiv="Content-Type" content="text/html"> |
| </head> |
| |
| <body bgcolor="#ffffff"> |
| <h1 align="center">Libxml memory management</h1> |
| |
| <p>Location: <a |
| href="http://xmlsoft.org/xmlmem.html">http://xmlsoft.org/xmlmem.html</a></p> |
| |
| <p>Libxml home page: <a href="http://xmlsoft.org/">http://xmlsoft.org/</a></p> |
| |
| <p>Mailing-list archive: <a |
| href="http://xmlsoft.org/messages/">http://xmlsoft.org/messages/</a></p> |
| |
| <p>Version: $Revision$</p> |
| |
| <p>Table of Content:</p> |
| <ol> |
| <li><a href="#General">General overview</a></li> |
| <li><a href="#setting">Setting libxml set of memory routines</a></li> |
| <li><a href="#cleanup">Cleaning up after parsing</a></li> |
| <li><a href="#Debugging">Debugging routines</a></li> |
| <li><a href="#General">General memory requirements</a></li> |
| </ol> |
| |
| <h2><a name="General">General overview</a></h2> |
| |
| <p>The module <code><a |
| href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlmemory.h</a></code> |
| provides the interfaces to the libxml memory system:</p> |
| <ul> |
| <li>libxml does not use the libc memory allocator directly but xmlFree(), |
| xmlMalloc() and xmlRealloc()</li> |
| <li>those routines can be reallocated to a specific set of routine, by |
| default the libc ones i.e. free(), malloc() and realloc()</li> |
| <li>the xmlmemory.c module includes a set of debugging routine</li> |
| </ul> |
| |
| <h2><a name="setting">Setting libxml set of memory routines</a></h2> |
| |
| <p>It is sometimes useful to not use the default memory allocator, either for |
| debugging, analysis or to implement a specific behaviour on memory management |
| (like on embedded systems). Two function calls are available to do so:</p> |
| <ul> |
| <li><a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMemGet |
| ()</a> which return the current set of functions in use by the parser</li> |
| <li><a |
| href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMemSetup()</a> |
| which allow to set up a new set of memory allocation functions</li> |
| </ul> |
| |
| <p>Of course a call to xmlMemSetup() should probably be done before calling |
| any other libxml routines (unless you are sure your allocations routines are |
| compatibles).</p> |
| |
| <h2><a name="cleanup">Cleaning up after parsing</a></h2> |
| |
| <p>Libxml is not stateless, there is a few set of memory structures needing |
| allocation before the parser is fully functionnal (some encoding structures |
| for example). This also mean that once parsing is finished there is a tiny |
| amount of memory (a few hundred bytes) which can be recollected if you don't |
| reuse the parser immediately:</p> |
| <ul> |
| <li><a href="http://xmlsoft.org/html/libxml-parser.html">xmlCleanupParser |
| ()</a> is a centralized routine to free the parsing states. Note that it |
| won't deallocate any produced tree if any (use the xmlFreeDoc() and |
| related routines for this).</li> |
| <li><a href="http://xmlsoft.org/html/libxml-parser.html">xmlInitParser |
| ()</a> is the dual routine allowing to preallocate the parsing state which |
| can be useful for example to avoid initialization reentrancy problems when |
| using libxml in multithreaded applications</li> |
| </ul> |
| |
| <p>Generally xmlCleanupParser() is safe, if needed the state will be rebuild |
| at the next invocation of parser routines, but be careful of the consequences |
| in multithreaded applications.</p> |
| |
| <h2><a name="Debugging">Debugging routines</a></h2> |
| |
| <p>When configured using --with-mem-debug flag (off by default), libxml uses a |
| set of memory allocation debugging routineskeeping track of all allocated |
| blocks and the location in the code where the routine was called. A couple of |
| other debugging routines allow to dump the memory allocated infos to a file or |
| call a specific routine when a given block number is allocated:</p> |
| <ul> |
| <li><a |
| href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMallocLoc()</a> |
| <a |
| href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlReallocLoc()</a> |
| and <a |
| href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMemStrdupLoc()</a> |
| are the memory debugging replacement allocation routines</li> |
| <li><a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMemoryDump |
| ()</a> dumps all the informations about the allocated memory block lefts |
| in the <code>.memdump</code> file</li> |
| </ul> |
| |
| <p> When developping libxml memory debug is enabled, the tests programs call |
| xmlMemoryDump () and the "make test" regression tests will check for any |
| memory leak during the full regression test sequence, this helps a lot |
| ensuring that libxml does not leak memory and bullet proof memory allocations |
| use (some libc implementations are known to be far too permissive resulting in |
| major portability problems!). </p> |
| |
| <p>If the .memdump reports a leak, it displays the allocation function and |
| also tries to give some informations about the content and structure of the |
| allocated blocks left. This is sufficient in most cases to find the culprit, |
| but not always. Assuming the allocation problem is reproductible, it is |
| possible to find more easilly:</p> |
| <ol> |
| <li>write down the block number xxxx not allocated</li> |
| <li>export the environement variable XML_MEM_BREAKPOINT=xxxx</li> |
| <li>run the program under a debugger and set a breakpoint on |
| xmlMallocBreakpoint() a specific function called when this precise block |
| is allocated</li> |
| <li>when the breakpoint is reached you can then do a fine analysis of the |
| allocation an step to see the condition resulting in the missing |
| deallocation.</li> |
| </ol> |
| |
| <p> I used to use a commercial tool to debug libxml memory problems but after |
| noticing that it was not detecting memory leaks that simple mechanism was used |
| and proved extremely efficient until now.</p> |
| |
| <h2><a name="General">General memory requirements</a></h2> |
| |
| <p>How much libxml memory require ? It's hard to tell in average it depends of |
| a number of things:</p> |
| <ul> |
| <li>the parser itself should work in a fixed amout of memory, except for |
| information maintained about the stacks of names and entities locations. |
| The I/O and encoding handlers will probably account for a few KBytes. This |
| is true for both the XML and HTML parser (though the HTML parser need more |
| state).</li> |
| <li>If you are generating the DOM tree then memory requirements will grow |
| nearly lineary with the size of the data. In general for a balanced |
| textual document the internal memory requirement is about 4 times the size |
| of the UTF8 serialization of this document (exmple the XML-1.0 |
| recommendation is a bit more of 150KBytes and takes 650KBytes of main |
| memory when parsed). Validation will add a amount of memory required for |
| maintaining the external Dtd state which should be linear with the |
| complexity of the content model defined by the Dtd </li> |
| <li>If you don't care about the advanced features of libxml like validation, |
| DOM, XPath or XPointer, but really need to work fixed memory requirements, |
| then the SAX interface should be used.</li> |
| </ul> |
| |
| <p></p> |
| |
| <p><a href="mailto:Daniel.Veillard@w3.org">Daniel Veillard</a></p> |
| |
| <p>$Id$</p> |
| </body> |
| </html> |