Gitiles
Code Review Sign In
gerrit-public.fairphone.software / fp2-dev / platform / external / libxml2 / 597f1c1f340a2d972ec47a9b4fc53f497da0208a / . / doc / xmlmem.html
blob: fd54b6e650340f99dc0d15d4151b8f23802dbef8 [file] [log] [blame]
Daniel Veillard1177ca42003-04-26 22:29:54 +0000[diff] [blame]1<?xml version="1.0" encoding="ISO-8859-1"?>
2<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
3<html xmlns="http://www.w3.org/1999/xhtml"><head><meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1" /><link rel="SHORTCUT ICON" href="/favicon.ico" /><style type="text/css">
Daniel Veillard373a4752002-02-21 14:46:29 +0000[diff] [blame]4TD {font-family: Verdana,Arial,Helvetica}
5BODY {font-family: Verdana,Arial,Helvetica; margin-top: 2em; margin-left: 0em; margin-right: 0em}
6H1 {font-family: Verdana,Arial,Helvetica}
7H2 {font-family: Verdana,Arial,Helvetica}
8H3 {font-family: Verdana,Arial,Helvetica}
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]9A:link, A:visited, A:active { text-decoration: underline }
Daniel Veillardd49370e2005-04-11 23:28:16 +0000[diff] [blame]10</style><title>Memory Management</title></head><body bgcolor="#8b7765" text="#000000" link="#a06060" vlink="#000000"><table border="0" width="100%" cellpadding="5" cellspacing="0" align="center"><tr><td width="120"><a href="http://swpat.ffii.org/"><img src="epatents.png" alt="Action against software patents" /></a></td><td width="180"><a href="http://www.gnome.org/"><img src="gnome2.png" alt="Gnome2 Logo" /></a><a href="http://www.w3.org/Status"><img src="w3c.png" alt="W3C Logo" /></a><a href="http://www.redhat.com/"><img src="redhat.gif" alt="Red Hat Logo" /></a><div align="left"><a href="http://xmlsoft.org/"><img src="Libxml2-Logo-180x168.gif" alt="Made with Libxml2 Logo" /></a></div></td><td><table border="0" width="90%" cellpadding="2" cellspacing="0" align="center" bgcolor="#000000"><tr><td><table width="100%" border="0" cellspacing="1" cellpadding="3" bgcolor="#fffacd"><tr><td align="center"><h1>The XML C parser and toolkit of Gnome</h1><h2>Memory Management</h2></td></tr></table></td></tr></table></td></tr></table><table border="0" cellpadding="4" cellspacing="0" width="100%" align="center"><tr><td bgcolor="#8b7765"><table border="0" cellspacing="0" cellpadding="2" width="100%"><tr><td valign="top" width="200" bgcolor="#8b7765"><table border="0" cellspacing="0" cellpadding="1" width="100%" bgcolor="#000000"><tr><td><table width="100%" border="0" cellspacing="1" cellpadding="3"><tr><td colspan="1" bgcolor="#eecfa1" align="center"><center><b>Developer Menu</b></center></td></tr><tr><td bgcolor="#fffacd"><form action="search.php" enctype="application/x-www-form-urlencoded" method="get"><input name="query" type="text" size="20" value="" /><input name="submit" type="submit" value="Search ..." /></form><ul><li><a href="index.html" style="font-weight:bold">Main Menu</a></li><li><a href="html/index.html" style="font-weight:bold">Reference Manual</a></li><li><a href="examples/index.html" style="font-weight:bold">Code Examples</a></li><li><a href="guidelines.html">XML Guidelines</a></li><li><a href="tutorial/index.html">Tutorial</a></li><li><a href="xmlreader.html">The Reader Interface</a></li><li><a href="ChangeLog.html">ChangeLog</a></li><li><a href="XSLT.html">XSLT</a></li><li><a href="python.html">Python and bindings</a></li><li><a href="architecture.html">libxml2 architecture</a></li><li><a href="tree.html">The tree output</a></li><li><a href="interface.html">The SAX interface</a></li><li><a href="xmlmem.html">Memory Management</a></li><li><a href="xmlio.html">I/O Interfaces</a></li><li><a href="library.html">The parser interfaces</a></li><li><a href="entities.html">Entities or no entities</a></li><li><a href="namespaces.html">Namespaces</a></li><li><a href="upgrade.html">Upgrading 1.x code</a></li><li><a href="threads.html">Thread safety</a></li><li><a href="DOM.html">DOM Principles</a></li><li><a href="example.html">A real example</a></li><li><a href="xml.html">flat page</a>, <a href="site.xsl">stylesheet</a></li></ul></td></tr></table><table width="100%" border="0" cellspacing="1" cellpadding="3"><tr><td colspan="1" bgcolor="#eecfa1" align="center"><center><b>API Indexes</b></center></td></tr><tr><td bgcolor="#fffacd"><ul><li><a href="APIchunk0.html">Alphabetic</a></li><li><a href="APIconstructors.html">Constructors</a></li><li><a href="APIfunctions.html">Functions/Types</a></li><li><a href="APIfiles.html">Modules</a></li><li><a href="APIsymbols.html">Symbols</a></li></ul></td></tr></table><table width="100%" border="0" cellspacing="1" cellpadding="3"><tr><td colspan="1" bgcolor="#eecfa1" align="center"><center><b>Related links</b></center></td></tr><tr><td bgcolor="#fffacd"><ul><li><a href="http://mail.gnome.org/archives/xml/">Mail archive</a></li><li><a href="http://xmlsoft.org/XSLT/">XSLT libxslt</a></li><li><a href="http://phd.cs.unibo.it/gdome2/">DOM gdome2</a></li><li><a href="http://www.aleksey.com/xmlsec/">XML-DSig xmlsec</a></li><li><a href="ftp://xmlsoft.org/">FTP</a></li><li><a href="http://www.zlatkovic.com/projects/libxml/">Windows binaries</a></li><li><a href="http://www.blastwave.org/packages.php/libxml2">Solaris binaries</a></li><li><a href="http://www.explain.com.au/oss/libxml2xslt.html">MacOsX binaries</a></li><li><a href="http://libxmlplusplus.sourceforge.net/">C++ bindings</a></li><li><a href="http://www.zend.com/php5/articles/php5-xmlphp.php#Heading4">PHP bindings</a></li><li><a href="http://sourceforge.net/projects/libxml2-pas/">Pascal bindings</a></li><li><a href="http://rubyforge.org/projects/xml-tools/">Ruby bindings</a></li><li><a href="http://tclxml.sourceforge.net/">Tcl bindings</a></li><li><a href="http://bugzilla.gnome.org/buglist.cgi?product=libxml2">Bug Tracker</a></li></ul></td></tr></table></td></tr></table></td><td valign="top" bgcolor="#8b7765"><table border="0" cellspacing="0" cellpadding="1" width="100%"><tr><td><table border="0" cellspacing="0" cellpadding="1" width="100%" bgcolor="#000000"><tr><td><table border="0" cellpadding="3" cellspacing="1" width="100%"><tr><td bgcolor="#fffacd"><p>Table of Content:</p><ol><li><a href="#General3">General overview</a></li>
Daniel Veillard8a469172003-06-12 16:05:07 +0000[diff] [blame]11 <li><a href="#setting">Setting libxml2 set of memory routines</a></li>
Daniel Veillard0b28e882002-07-24 23:47:05 +0000[diff] [blame]12 <li><a href="#cleanup">Cleaning up after parsing</a></li>
13 <li><a href="#Debugging">Debugging routines</a></li>
14 <li><a href="#General4">General memory requirements</a></li>
Daniel Veillard1177ca42003-04-26 22:29:54 +0000[diff] [blame]15</ol><h3><a name="General3" id="General3">General overview</a></h3><p>The module <code><a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlmemory.h</a></code>
Daniel Veillard8a469172003-06-12 16:05:07 +0000[diff] [blame]16provides the interfaces to the libxml2 memory system:</p><ul><li>libxml2 does not use the libc memory allocator directly but xmlFree(),
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]17 xmlMalloc() and xmlRealloc()</li>
Daniel Veillard0b28e882002-07-24 23:47:05 +0000[diff] [blame]18 <li>those routines can be reallocated to a specific set of routine, by
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]19 default the libc ones i.e. free(), malloc() and realloc()</li>
Daniel Veillard0b28e882002-07-24 23:47:05 +0000[diff] [blame]20 <li>the xmlmemory.c module includes a set of debugging routine</li>
Daniel Veillard8a469172003-06-12 16:05:07 +0000[diff] [blame]21</ul><h3><a name="setting" id="setting">Setting libxml2 set of memory routines</a></h3><p>It is sometimes useful to not use the default memory allocator, either for
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]22debugging, analysis or to implement a specific behaviour on memory management
Daniel Veillard1177ca42003-04-26 22:29:54 +0000[diff] [blame]23(like on embedded systems). Two function calls are available to do so:</p><ul><li><a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMemGet
Daniel Veillardaf43f632002-03-08 15:05:20 +0000[diff] [blame]24 ()</a> which return the current set of functions in use by the parser</li>
Daniel Veillard1177ca42003-04-26 22:29:54 +0000[diff] [blame]25 <li><a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMemSetup()</a>
Daniel Veillardaf43f632002-03-08 15:05:20 +0000[diff] [blame]26 which allow to set up a new set of memory allocation functions</li>
Daniel Veillard1177ca42003-04-26 22:29:54 +0000[diff] [blame]27</ul><p>Of course a call to xmlMemSetup() should probably be done before calling
Daniel Veillard8a469172003-06-12 16:05:07 +0000[diff] [blame]28any other libxml2 routines (unless you are sure your allocations routines are
29compatibles).</p><h3><a name="cleanup" id="cleanup">Cleaning up after parsing</a></h3><p>Libxml2 is not stateless, there is a few set of memory structures needing
Daniel Veillard63d83142002-05-20 06:51:05 +0000[diff] [blame]30allocation before the parser is fully functional (some encoding structures
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]31for example). This also mean that once parsing is finished there is a tiny
32amount of memory (a few hundred bytes) which can be recollected if you don't
Daniel Veillard1177ca42003-04-26 22:29:54 +0000[diff] [blame]33reuse the parser immediately:</p><ul><li><a href="http://xmlsoft.org/html/libxml-parser.html">xmlCleanupParser
Daniel Veillardaf43f632002-03-08 15:05:20 +0000[diff] [blame]34 ()</a> is a centralized routine to free the parsing states. Note that it
35 won't deallocate any produced tree if any (use the xmlFreeDoc() and
36 related routines for this).</li>
Daniel Veillard1177ca42003-04-26 22:29:54 +0000[diff] [blame]37 <li><a href="http://xmlsoft.org/html/libxml-parser.html">xmlInitParser
Daniel Veillardaf43f632002-03-08 15:05:20 +0000[diff] [blame]38 ()</a> is the dual routine allowing to preallocate the parsing state
39 which can be useful for example to avoid initialization reentrancy
Daniel Veillard8a469172003-06-12 16:05:07 +0000[diff] [blame]40 problems when using libxml2 in multithreaded applications</li>
Daniel Veillard1177ca42003-04-26 22:29:54 +0000[diff] [blame]41</ul><p>Generally xmlCleanupParser() is safe, if needed the state will be rebuild
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]42at the next invocation of parser routines, but be careful of the consequences
Daniel Veillard8a469172003-06-12 16:05:07 +0000[diff] [blame]43in multithreaded applications.</p><h3><a name="Debugging" id="Debugging">Debugging routines</a></h3><p>When configured using --with-mem-debug flag (off by default), libxml2 uses
Daniel Veillard63d83142002-05-20 06:51:05 +0000[diff] [blame]44a set of memory allocation debugging routines keeping track of all allocated
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]45blocks and the location in the code where the routine was called. A couple of
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]46other debugging routines allow to dump the memory allocated infos to a file
Daniel Veillard1177ca42003-04-26 22:29:54 +0000[diff] [blame]47or call a specific routine when a given block number is allocated:</p><ul><li><a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMallocLoc()</a>
Daniel Veillard0b28e882002-07-24 23:47:05 +0000[diff] [blame]48 <a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlReallocLoc()</a>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]49 and <a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMemStrdupLoc()</a>
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]50 are the memory debugging replacement allocation routines</li>
Daniel Veillard1177ca42003-04-26 22:29:54 +0000[diff] [blame]51 <li><a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMemoryDump
Daniel Veillardaf43f632002-03-08 15:05:20 +0000[diff] [blame]52 ()</a> dumps all the informations about the allocated memory block lefts
53 in the <code>.memdump</code> file</li>
Daniel Veillard8a469172003-06-12 16:05:07 +0000[diff] [blame]54</ul><p>When developing libxml2 memory debug is enabled, the tests programs call
Daniel Veillard024f1992003-12-10 16:43:49 +0000[diff] [blame]55xmlMemoryDump () and the "make test" regression tests will check for any
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]56memory leak during the full regression test sequence, this helps a lot
Daniel Veillard8a469172003-06-12 16:05:07 +0000[diff] [blame]57ensuring that libxml2 does not leak memory and bullet proof memory
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]58allocations use (some libc implementations are known to be far too permissive
Daniel Veillard1177ca42003-04-26 22:29:54 +0000[diff] [blame]59resulting in major portability problems!).</p><p>If the .memdump reports a leak, it displays the allocation function and
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]60also tries to give some informations about the content and structure of the
61allocated blocks left. This is sufficient in most cases to find the culprit,
Daniel Veillard63d83142002-05-20 06:51:05 +0000[diff] [blame]62but not always. Assuming the allocation problem is reproducible, it is
Daniel Veillard1177ca42003-04-26 22:29:54 +0000[diff] [blame]63possible to find more easily:</p><ol><li>write down the block number xxxx not allocated</li>
Daniel Veillard0b28e882002-07-24 23:47:05 +0000[diff] [blame]64 <li>export the environment variable XML_MEM_BREAKPOINT=xxxx , the easiest
Daniel Veillarda8a89fe2002-04-12 21:03:34 +0000[diff] [blame]65 when using GDB is to simply give the command
66 <p><code>set environment XML_MEM_BREAKPOINT xxxx</code></p>
Daniel Veillard0b28e882002-07-24 23:47:05 +0000[diff] [blame]67 <p>before running the program.</p>
68 </li>
69 <li>run the program under a debugger and set a breakpoint on
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]70 xmlMallocBreakpoint() a specific function called when this precise block
71 is allocated</li>
Daniel Veillard0b28e882002-07-24 23:47:05 +0000[diff] [blame]72 <li>when the breakpoint is reached you can then do a fine analysis of the
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]73 allocation an step to see the condition resulting in the missing
74 deallocation.</li>
Daniel Veillard8a469172003-06-12 16:05:07 +0000[diff] [blame]75</ol><p>I used to use a commercial tool to debug libxml2 memory problems but after
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]76noticing that it was not detecting memory leaks that simple mechanism was
Daniel Veillarda8a89fe2002-04-12 21:03:34 +0000[diff] [blame]77used and proved extremely efficient until now. Lately I have also used <a href="http://developer.kde.org/~sewardj/">valgrind</a> with quite some
78success, it is tied to the i386 architecture since it works by emulating the
79processor and instruction set, it is slow but extremely efficient, i.e. it
Daniel Veillard8a469172003-06-12 16:05:07 +0000[diff] [blame]80spot memory usage errors in a very precise way.</p><h3><a name="General4" id="General4">General memory requirements</a></h3><p>How much libxml2 memory require ? It's hard to tell in average it depends
Daniel Veillard1177ca42003-04-26 22:29:54 +0000[diff] [blame]81of a number of things:</p><ul><li>the parser itself should work in a fixed amount of memory, except for
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]82 information maintained about the stacks of names and entities locations.
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]83 The I/O and encoding handlers will probably account for a few KBytes.
84 This is true for both the XML and HTML parser (though the HTML parser
85 need more state).</li>
Daniel Veillard0b28e882002-07-24 23:47:05 +0000[diff] [blame]86 <li>If you are generating the DOM tree then memory requirements will grow
Daniel Veillard63d83142002-05-20 06:51:05 +0000[diff] [blame]87 nearly linear with the size of the data. In general for a balanced
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]88 textual document the internal memory requirement is about 4 times the
Daniel Veillard63d83142002-05-20 06:51:05 +0000[diff] [blame]89 size of the UTF8 serialization of this document (example the XML-1.0
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]90 recommendation is a bit more of 150KBytes and takes 650KBytes of main
91 memory when parsed). Validation will add a amount of memory required for
92 maintaining the external Dtd state which should be linear with the
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]93 complexity of the content model defined by the Dtd</li>
Daniel Veillardce192eb2003-04-16 15:58:05 +0000[diff] [blame]94 <li>If you need to work with fixed memory requirements or don't need the
95 full DOM tree then using the <a href="xmlreader.html">xmlReader
96 interface</a> is probably the best way to proceed, it still allows to
97 validate or operate on subset of the tree if needed.</li>
Daniel Veillard8a469172003-06-12 16:05:07 +0000[diff] [blame]98 <li>If you don't care about the advanced features of libxml2 like
Daniel Veillardce192eb2003-04-16 15:58:05 +0000[diff] [blame]99 validation, DOM, XPath or XPointer, don't use entities, need to work with
100 fixed memory requirements, and try to get the fastest parsing possible
101 then the SAX interface should be used, but it has known restrictions.</li>
Daniel Veillard1177ca42003-04-26 22:29:54 +0000[diff] [blame]102</ul><p></p><p><a href="bugs.html">Daniel Veillard</a></p></td></tr></table></td></tr></table></td></tr></table></td></tr></table></td></tr></table></body></html>