Gitiles
Code Review Sign In
gerrit-public.fairphone.software / platform / external / libxml2 / 69839ba1974ba8a7e996652bb2f7865de33b366f / . / doc / xmlmem.html
blob: 547a7edf06024c37a319b95d902d6ab210b7e1e4 [file] [log] [blame]
Daniel Veillard1177ca42003-04-26 22:29:54 +0000[diff] [blame]1<?xml version="1.0" encoding="ISO-8859-1"?>
2<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
3<html xmlns="http://www.w3.org/1999/xhtml"><head><meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1" /><link rel="SHORTCUT ICON" href="/favicon.ico" /><style type="text/css">
Daniel Veillard373a4752002-02-21 14:46:29 +0000[diff] [blame]4TD {font-family: Verdana,Arial,Helvetica}
5BODY {font-family: Verdana,Arial,Helvetica; margin-top: 2em; margin-left: 0em; margin-right: 0em}
6H1 {font-family: Verdana,Arial,Helvetica}
7H2 {font-family: Verdana,Arial,Helvetica}
8H3 {font-family: Verdana,Arial,Helvetica}
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]9A:link, A:visited, A:active { text-decoration: underline }
Daniel Veillardd463c992006-02-23 22:07:59 +0000[diff] [blame]10</style><title>Memory Management</title></head><body bgcolor="#8b7765" text="#000000" link="#a06060" vlink="#000000"><table border="0" width="100%" cellpadding="5" cellspacing="0" align="center"><tr><td width="120"><a href="http://swpat.ffii.org/"><img src="epatents.png" alt="Action against software patents" /></a></td><td width="180"><a href="http://www.gnome.org/"><img src="gnome2.png" alt="Gnome2 Logo" /></a><a href="http://www.w3.org/Status"><img src="w3c.png" alt="W3C Logo" /></a><a href="http://www.redhat.com/"><img src="redhat.gif" alt="Red Hat Logo" /></a><div align="left"><a href="http://xmlsoft.org/"><img src="Libxml2-Logo-180x168.gif" alt="Made with Libxml2 Logo" /></a></div></td><td><table border="0" width="90%" cellpadding="2" cellspacing="0" align="center" bgcolor="#000000"><tr><td><table width="100%" border="0" cellspacing="1" cellpadding="3" bgcolor="#fffacd"><tr><td align="center"><h1>The XML C parser and toolkit of Gnome</h1><h2>Memory Management</h2></td></tr></table></td></tr></table></td></tr></table><table border="0" cellpadding="4" cellspacing="0" width="100%" align="center"><tr><td bgcolor="#8b7765"><table border="0" cellspacing="0" cellpadding="2" width="100%"><tr><td valign="top" width="200" bgcolor="#8b7765"><table border="0" cellspacing="0" cellpadding="1" width="100%" bgcolor="#000000"><tr><td><table width="100%" border="0" cellspacing="1" cellpadding="3"><tr><td colspan="1" bgcolor="#eecfa1" align="center"><center><b>Developer Menu</b></center></td></tr><tr><td bgcolor="#fffacd"><form action="search.php" enctype="application/x-www-form-urlencoded" method="get"><input name="query" type="text" size="20" value="" /><input name="submit" type="submit" value="Search ..." /></form><ul><li><a href="index.html" style="font-weight:bold">Main Menu</a></li><li><a href="html/index.html" style="font-weight:bold">Reference Manual</a></li><li><a href="examples/index.html" style="font-weight:bold">Code Examples</a></li><li><a href="guidelines.html">XML Guidelines</a></li><li><a href="tutorial/index.html">Tutorial</a></li><li><a href="xmlreader.html">The Reader Interface</a></li><li><a href="ChangeLog.html">ChangeLog</a></li><li><a href="XSLT.html">XSLT</a></li><li><a href="python.html">Python and bindings</a></li><li><a href="architecture.html">libxml2 architecture</a></li><li><a href="tree.html">The tree output</a></li><li><a href="interface.html">The SAX interface</a></li><li><a href="xmlmem.html">Memory Management</a></li><li><a href="xmlio.html">I/O Interfaces</a></li><li><a href="library.html">The parser interfaces</a></li><li><a href="entities.html">Entities or no entities</a></li><li><a href="namespaces.html">Namespaces</a></li><li><a href="upgrade.html">Upgrading 1.x code</a></li><li><a href="threads.html">Thread safety</a></li><li><a href="DOM.html">DOM Principles</a></li><li><a href="example.html">A real example</a></li><li><a href="xml.html">flat page</a>, <a href="site.xsl">stylesheet</a></li></ul></td></tr></table><table width="100%" border="0" cellspacing="1" cellpadding="3"><tr><td colspan="1" bgcolor="#eecfa1" align="center"><center><b>API Indexes</b></center></td></tr><tr><td bgcolor="#fffacd"><ul><li><a href="APIchunk0.html">Alphabetic</a></li><li><a href="APIconstructors.html">Constructors</a></li><li><a href="APIfunctions.html">Functions/Types</a></li><li><a href="APIfiles.html">Modules</a></li><li><a href="APIsymbols.html">Symbols</a></li></ul></td></tr></table><table width="100%" border="0" cellspacing="1" cellpadding="3"><tr><td colspan="1" bgcolor="#eecfa1" align="center"><center><b>Related links</b></center></td></tr><tr><td bgcolor="#fffacd"><ul><li><a href="http://mail.gnome.org/archives/xml/">Mail archive</a></li><li><a href="http://xmlsoft.org/XSLT/">XSLT libxslt</a></li><li><a href="http://phd.cs.unibo.it/gdome2/">DOM gdome2</a></li><li><a href="http://www.aleksey.com/xmlsec/">XML-DSig xmlsec</a></li><li><a href="ftp://xmlsoft.org/">FTP</a></li><li><a href="http://www.zlatkovic.com/projects/libxml/">Windows binaries</a></li><li><a href="http://www.blastwave.org/packages.php/libxml2">Solaris binaries</a></li><li><a href="http://www.explain.com.au/oss/libxml2xslt.html">MacOsX binaries</a></li><li><a href="http://libxmlplusplus.sourceforge.net/">C++ bindings</a></li><li><a href="http://www.zend.com/php5/articles/php5-xmlphp.php#Heading4">PHP bindings</a></li><li><a href="http://sourceforge.net/projects/libxml2-pas/">Pascal bindings</a></li><li><a href="http://libxml.rubyforge.org/">Ruby bindings</a></li><li><a href="http://tclxml.sourceforge.net/">Tcl bindings</a></li><li><a href="http://bugzilla.gnome.org/buglist.cgi?product=libxml2">Bug Tracker</a></li></ul></td></tr></table></td></tr></table></td><td valign="top" bgcolor="#8b7765"><table border="0" cellspacing="0" cellpadding="1" width="100%"><tr><td><table border="0" cellspacing="0" cellpadding="1" width="100%" bgcolor="#000000"><tr><td><table border="0" cellpadding="3" cellspacing="1" width="100%"><tr><td bgcolor="#fffacd"><p>Table of Content:</p><ol><li><a href="#General3">General overview</a></li>
Daniel Veillard8a469172003-06-12 16:05:07 +0000[diff] [blame]11 <li><a href="#setting">Setting libxml2 set of memory routines</a></li>
Daniel Veillard0b28e882002-07-24 23:47:05 +0000[diff] [blame]12 <li><a href="#cleanup">Cleaning up after parsing</a></li>
13 <li><a href="#Debugging">Debugging routines</a></li>
14 <li><a href="#General4">General memory requirements</a></li>
Daniel Veillard69839ba2006-06-06 13:27:03 +0000[diff] [blame^]15</ol><h3><a name="General3" id="General3">General overview</a></h3><p>The module <code><a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlmemory.h</a></code>provides
16the interfaces to the libxml2 memory system:</p><ul><li>libxml2 does not use the libc memory allocator directly but
17 xmlFree(),xmlMalloc() and xmlRealloc()</li>
18 <li>those routines can be reallocated to a specific set of routine,
19 bydefault the libc ones i.e. free(), malloc() and realloc()</li>
Daniel Veillard0b28e882002-07-24 23:47:05 +0000[diff] [blame]20 <li>the xmlmemory.c module includes a set of debugging routine</li>
Daniel Veillard69839ba2006-06-06 13:27:03 +0000[diff] [blame^]21</ul><h3><a name="setting" id="setting">Setting libxml2 set of memory routines</a></h3><p>It is sometimes useful to not use the default memory allocator, either
22fordebugging, analysis or to implement a specific behaviour on memory
23management(like on embedded systems). Two function calls are available to do
24so:</p><ul><li><a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMemGet()</a>which
25 return the current set of functions in use by the parser</li>
26 <li><a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMemSetup()</a>which
27 allow to set up a new set of memory allocation functions</li>
28</ul><p>Of course a call to xmlMemSetup() should probably be done before
29callingany other libxml2 routines (unless you are sure your allocations
30routines arecompatibles).</p><h3><a name="cleanup" id="cleanup">Cleaning up after parsing</a></h3><p>Libxml2 is not stateless, there is a few set of memory structures
31needingallocation before the parser is fully functional (some encoding
32structuresfor example). This also mean that once parsing is finished there is
33a tinyamount of memory (a few hundred bytes) which can be recollected if you
34don'treuse the parser immediately:</p><ul><li><a href="http://xmlsoft.org/html/libxml-parser.html">xmlCleanupParser()</a>is
35 a centralized routine to free the parsing states. Note that itwon't
36 deallocate any produced tree if any (use the xmlFreeDoc() andrelated
37 routines for this).</li>
38 <li><a href="http://xmlsoft.org/html/libxml-parser.html">xmlInitParser()</a>is
39 the dual routine allowing to preallocate the parsing statewhich can be
40 useful for example to avoid initialization reentrancyproblems when using
41 libxml2 in multithreaded applications</li>
42</ul><p>Generally xmlCleanupParser() is safe, if needed the state will be
43rebuildat the next invocation of parser routines, but be careful of the
44consequencesin multithreaded applications.</p><h3><a name="Debugging" id="Debugging">Debugging routines</a></h3><p>When configured using --with-mem-debug flag (off by default), libxml2
45usesa set of memory allocation debugging routines keeping track of all
46allocatedblocks and the location in the code where the routine was called. A
47couple ofother debugging routines allow to dump the memory allocated infos to
48a fileor call a specific routine when a given block number is allocated:</p><ul><li><a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMallocLoc()</a><a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlReallocLoc()</a>and
49 <a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMemStrdupLoc()</a>are
50 the memory debugging replacement allocation routines</li>
51 <li><a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMemoryDump()</a>dumps
52 all the informations about the allocated memory block leftsin the
53 <code>.memdump</code>file</li>
54</ul><p>When developing libxml2 memory debug is enabled, the tests programs
55callxmlMemoryDump () and the "make test" regression tests will check for
56anymemory leak during the full regression test sequence, this helps a
57lotensuring that libxml2 does not leak memory and bullet proof
58memoryallocations use (some libc implementations are known to be far too
59permissiveresulting in major portability problems!).</p><p>If the .memdump reports a leak, it displays the allocation function
60andalso tries to give some informations about the content and structure of
61theallocated blocks left. This is sufficient in most cases to find the
62culprit,but not always. Assuming the allocation problem is reproducible, it
63ispossible to find more easily:</p><ol><li>write down the block number xxxx not allocated</li>
64 <li>export the environment variable XML_MEM_BREAKPOINT=xxxx , the
65 easiestwhen using GDB is to simply give the command
Daniel Veillarda8a89fe2002-04-12 21:03:34 +0000[diff] [blame]66 <p><code>set environment XML_MEM_BREAKPOINT xxxx</code></p>
Daniel Veillard0b28e882002-07-24 23:47:05 +0000[diff] [blame]67 <p>before running the program.</p>
68 </li>
Daniel Veillard69839ba2006-06-06 13:27:03 +0000[diff] [blame^]69 <li>run the program under a debugger and set a breakpoint
70 onxmlMallocBreakpoint() a specific function called when this precise
71 blockis allocated</li>
72 <li>when the breakpoint is reached you can then do a fine analysis of
73 theallocation an step to see the condition resulting in the
74 missingdeallocation.</li>
75</ol><p>I used to use a commercial tool to debug libxml2 memory problems but
76afternoticing that it was not detecting memory leaks that simple mechanism
77wasused and proved extremely efficient until now. Lately I have also used <a href="http://developer.kde.org/~sewardj/">valgrind</a>with quite somesuccess,
78it is tied to the i386 architecture since it works by emulating theprocessor
79and instruction set, it is slow but extremely efficient, i.e. itspot memory
80usage errors in a very precise way.</p><h3><a name="General4" id="General4">General memory requirements</a></h3><p>How much libxml2 memory require ? It's hard to tell in average it
81dependsof a number of things:</p><ul><li>the parser itself should work in a fixed amount of memory, except
82 forinformation maintained about the stacks of names and entities
83 locations.The I/O and encoding handlers will probably account for a few
84 KBytes.This is true for both the XML and HTML parser (though the HTML
85 parserneed more state).</li>
86 <li>If you are generating the DOM tree then memory requirements will
87 grownearly linear with the size of the data. In general for a
88 balancedtextual document the internal memory requirement is about 4 times
89 thesize of the UTF8 serialization of this document (example the
90 XML-1.0recommendation is a bit more of 150KBytes and takes 650KBytes of
91 mainmemory when parsed). Validation will add a amount of memory required
92 formaintaining the external Dtd state which should be linear with
93 thecomplexity of the content model defined by the Dtd</li>
94 <li>If you need to work with fixed memory requirements or don't need
95 thefull DOM tree then using the <a href="xmlreader.html">xmlReaderinterface</a>is probably the best way to
96 proceed, it still allows tovalidate or operate on subset of the tree if
97 needed.</li>
98 <li>If you don't care about the advanced features of libxml2
99 likevalidation, DOM, XPath or XPointer, don't use entities, need to work
100 withfixed memory requirements, and try to get the fastest parsing
101 possiblethen the SAX interface should be used, but it has known
102 restrictions.</li>
Daniel Veillard1177ca42003-04-26 22:29:54 +0000[diff] [blame]103</ul><p></p><p><a href="bugs.html">Daniel Veillard</a></p></td></tr></table></td></tr></table></td></tr></table></td></tr></table></td></tr></table></body></html>