Gitiles
Code Review Sign In
gerrit-public.fairphone.software / platform / external / libxml2 / 9ae4b7afa98d1904d786f5e10db49ada8a2945ad / . / doc / xmlmem.html
blob: 98d6f05f4dfbabd76fb95130c5ad800f34121a0a [file] [log] [blame]
Daniel Veillard43d3f612001-11-10 11:57:23 +0000[diff] [blame]1<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/1999/REC-html401-19991224/loose.dtd">
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]2<html>
3<head>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]4<meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type">
5<style type="text/css"><!--
6TD {font-size: 10pt; font-family: Verdana,Arial,Helvetica}
7BODY {font-size: 10pt; font-family: Verdana,Arial,Helvetica; margin-top: 5pt; margin-left: 0pt; margin-right: 0pt}
8H1 {font-size: 16pt; font-family: Verdana,Arial,Helvetica}
9H2 {font-size: 14pt; font-family: Verdana,Arial,Helvetica}
10H3 {font-size: 12pt; font-family: Verdana,Arial,Helvetica}
11A:link, A:visited, A:active { text-decoration: underline }
12--></style>
13<title>Memory Management</title>
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]14</head>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]15<body bgcolor="#8b7765" text="#000000" link="#000000" vlink="#000000">
16<table border="0" width="100%" cellpadding="5" cellspacing="0" align="center"><tr>
17<td width="180">
18<a href="http://www.gnome.org/"><img src="smallfootonly.gif" alt="Gnome Logo"></a><a href="http://www.w3.org/Status"><img src="w3c.png" alt="W3C Logo"></a><a href="http://www.redhat.com/"><img src="redhat.gif" alt="Red Hat Logo"></a>
19</td>
20<td><table border="0" width="90%" cellpadding="2" cellspacing="0" align="center" bgcolor="#000000"><tr><td><table width="100%" border="0" cellspacing="1" cellpadding="3" bgcolor="#fffacd"><tr><td align="center">
21<h1>The XML C library for Gnome</h1>
22<h2>Memory Management</h2>
23</td></tr></table></td></tr></table></td>
24</tr></table>
25<table border="0" cellpadding="4" cellspacing="0" width="100%" align="center"><tr><td bgcolor="#8b7765"><table border="0" cellspacing="0" cellpadding="2" width="100%"><tr>
26<td valign="top" width="200" bgcolor="#8b7765"><table border="0" cellspacing="0" cellpadding="1" width="100%" bgcolor="#000000"><tr><td>
27<table width="100%" border="0" cellspacing="1" cellpadding="3">
28<tr><td colspan="1" bgcolor="#eecfa1" align="center"><center><b>Main Menu</b></center></td></tr>
29<tr><td bgcolor="#fffacd"><ul style="margin-left: -2pt">
30<li><a href="index.html">Home</a></li>
31<li><a href="intro.html">Introduction</a></li>
32<li><a href="FAQ.html">FAQ</a></li>
33<li><a href="docs.html">Documentation</a></li>
34<li><a href="bugs.html">Reporting bugs and getting help</a></li>
35<li><a href="help.html">How to help</a></li>
36<li><a href="downloads.html">Downloads</a></li>
37<li><a href="news.html">News</a></li>
38<li><a href="XML.html">XML</a></li>
39<li><a href="XSLT.html">XSLT</a></li>
40<li><a href="architecture.html">libxml architecture</a></li>
41<li><a href="tree.html">The tree output</a></li>
42<li><a href="interface.html">The SAX interface</a></li>
43<li><a href="xmldtd.html">Validation &amp; DTDs</a></li>
44<li><a href="xmlmem.html">Memory Management</a></li>
45<li><a href="encoding.html">Encodings support</a></li>
46<li><a href="xmlio.html">I/O Interfaces</a></li>
47<li><a href="catalog.html">Catalog support</a></li>
48<li><a href="library.html">The parser interfaces</a></li>
49<li><a href="entities.html">Entities or no entities</a></li>
50<li><a href="namespaces.html">Namespaces</a></li>
51<li><a href="upgrade.html">Upgrading 1.x code</a></li>
Daniel Veillard52dcab32001-10-30 12:51:17 +0000[diff] [blame]52<li><a href="threads.html">Thread safety</a></li>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]53<li><a href="DOM.html">DOM Principles</a></li>
54<li><a href="example.html">A real example</a></li>
55<li><a href="contribs.html">Contributions</a></li>
56<li>
57<a href="xml.html">flat page</a>, <a href="site.xsl">stylesheet</a>
58</li>
59</ul></td></tr>
60</table>
61<table width="100%" border="0" cellspacing="1" cellpadding="3">
62<tr><td colspan="1" bgcolor="#eecfa1" align="center"><center><b>Related links</b></center></td></tr>
63<tr><td bgcolor="#fffacd"><ul style="margin-left: -2pt">
64<li><a href="http://mail.gnome.org/archives/xml/">Mail archive</a></li>
65<li><a href="http://xmlsoft.org/XSLT/">XSLT libxslt</a></li>
66<li><a href="http://www.cs.unibo.it/~casarini/gdome2/">DOM gdome2</a></li>
67<li><a href="ftp://xmlsoft.org/">FTP</a></li>
68<li><a href="http://www.fh-frankfurt.de/~igor/projects/libxml/">Windows binaries</a></li>
Daniel Veillarddb9dfd92001-11-26 17:25:02 +0000[diff] [blame]69<li><a href="http://garypennington.net/libxml2/">Solaris binaries</a></li>
Daniel Veillardc6271d22001-10-27 07:50:58 +0000[diff] [blame]70<li><a href="http://bugzilla.gnome.org/buglist.cgi?product=libxml">Bug Tracker</a></li>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]71</ul></td></tr>
72</table>
73</td></tr></table></td>
74<td valign="top" bgcolor="#8b7765"><table border="0" cellspacing="0" cellpadding="1" width="100%"><tr><td><table border="0" cellspacing="0" cellpadding="1" width="100%" bgcolor="#000000"><tr><td><table border="0" cellpadding="3" cellspacing="1" width="100%"><tr><td bgcolor="#fffacd">
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]75<p>Table of Content:</p>
76<ol>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]77<li><a href="#General3">General overview</a></li>
Daniel Veillard9c466822001-10-25 12:03:39 +0000[diff] [blame]78<li><a href="#setting">Setting libxml set of memory routines</a></li>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]79<li><a href="#cleanup">Cleaning up after parsing</a></li>
80<li><a href="#Debugging">Debugging routines</a></li>
81<li><a href="#General4">General memory requirements</a></li>
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]82</ol>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]83<h3><a name="General3">General overview</a></h3>
84<p>The module <code><a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlmemory.h</a></code>
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]85provides the interfaces to the libxml memory system:</p>
86<ul>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]87<li>libxml does not use the libc memory allocator directly but xmlFree(),
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]88 xmlMalloc() and xmlRealloc()</li>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]89<li>those routines can be reallocated to a specific set of routine, by
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]90 default the libc ones i.e. free(), malloc() and realloc()</li>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]91<li>the xmlmemory.c module includes a set of debugging routine</li>
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]92</ul>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]93<h3><a name="setting">Setting libxml set of memory routines</a></h3>
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]94<p>It is sometimes useful to not use the default memory allocator, either for
95debugging, analysis or to implement a specific behaviour on memory management
96(like on embedded systems). Two function calls are available to do so:</p>
97<ul>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]98<li>
99<a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMemGet ()</a>
100 which return the current set of functions in use by the parser</li>
101<li>
102<a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMemSetup()</a>
103 which allow to set up a new set of memory allocation functions</li>
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]104</ul>
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]105<p>Of course a call to xmlMemSetup() should probably be done before calling
106any other libxml routines (unless you are sure your allocations routines are
107compatibles).</p>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]108<h3><a name="cleanup">Cleaning up after parsing</a></h3>
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]109<p>Libxml is not stateless, there is a few set of memory structures needing
110allocation before the parser is fully functionnal (some encoding structures
111for example). This also mean that once parsing is finished there is a tiny
112amount of memory (a few hundred bytes) which can be recollected if you don't
113reuse the parser immediately:</p>
114<ul>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]115<li>
116<a href="http://xmlsoft.org/html/libxml-parser.html">xmlCleanupParser
117 ()</a>
118 is a centralized routine to free the parsing states. Note that it won't
119 deallocate any produced tree if any (use the xmlFreeDoc() and related
120 routines for this).</li>
121<li>
122<a href="http://xmlsoft.org/html/libxml-parser.html">xmlInitParser
123 ()</a>
124 is the dual routine allowing to preallocate the parsing state which can
125 be useful for example to avoid initialization reentrancy problems when
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]126 using libxml in multithreaded applications</li>
127</ul>
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]128<p>Generally xmlCleanupParser() is safe, if needed the state will be rebuild
129at the next invocation of parser routines, but be careful of the consequences
130in multithreaded applications.</p>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]131<h3><a name="Debugging">Debugging routines</a></h3>
132<p>When configured using --with-mem-debug flag (off by default), libxml uses
133a set of memory allocation debugging routineskeeping track of all allocated
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]134blocks and the location in the code where the routine was called. A couple of
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]135other debugging routines allow to dump the memory allocated infos to a file
136or call a specific routine when a given block number is allocated:</p>
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]137<ul>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]138<li>
139<a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMallocLoc()</a><a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlReallocLoc()</a>
140 and <a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMemStrdupLoc()</a>
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]141 are the memory debugging replacement allocation routines</li>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]142<li>
143<a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMemoryDump
144 ()</a>
145 dumps all the informations about the allocated memory block lefts in the
146 <code>.memdump</code> file</li>
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]147</ul>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]148<p>When developping libxml memory debug is enabled, the tests programs call
149xmlMemoryDump () and the &quot;make test&quot; regression tests will check for any
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]150memory leak during the full regression test sequence, this helps a lot
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]151ensuring that libxml does not leak memory and bullet proof memory
152allocations use (some libc implementations are known to be far too permissive
153resulting in major portability problems!).</p>
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]154<p>If the .memdump reports a leak, it displays the allocation function and
155also tries to give some informations about the content and structure of the
156allocated blocks left. This is sufficient in most cases to find the culprit,
157but not always. Assuming the allocation problem is reproductible, it is
158possible to find more easilly:</p>
159<ol>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]160<li>write down the block number xxxx not allocated</li>
161<li>export the environement variable XML_MEM_BREAKPOINT=xxxx</li>
162<li>run the program under a debugger and set a breakpoint on
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]163 xmlMallocBreakpoint() a specific function called when this precise block
164 is allocated</li>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]165<li>when the breakpoint is reached you can then do a fine analysis of the
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]166 allocation an step to see the condition resulting in the missing
167 deallocation.</li>
168</ol>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]169<p>I used to use a commercial tool to debug libxml memory problems but after
170noticing that it was not detecting memory leaks that simple mechanism was
171used and proved extremely efficient until now.</p>
172<h3><a name="General4">General memory requirements</a></h3>
173<p>How much libxml memory require ? It's hard to tell in average it depends
174of a number of things:</p>
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]175<ul>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]176<li>the parser itself should work in a fixed amout of memory, except for
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]177 information maintained about the stacks of names and entities locations.
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]178 The I/O and encoding handlers will probably account for a few KBytes.
179 This is true for both the XML and HTML parser (though the HTML parser
180 need more state).</li>
181<li>If you are generating the DOM tree then memory requirements will grow
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]182 nearly lineary with the size of the data. In general for a balanced
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]183 textual document the internal memory requirement is about 4 times the
184 size of the UTF8 serialization of this document (exmple the XML-1.0
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]185 recommendation is a bit more of 150KBytes and takes 650KBytes of main
186 memory when parsed). Validation will add a amount of memory required for
187 maintaining the external Dtd state which should be linear with the
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]188 complexity of the content model defined by the Dtd</li>
189<li>If you don't care about the advanced features of libxml like
190 validation, DOM, XPath or XPointer, but really need to work fixed memory
191 requirements, then the SAX interface should be used.</li>
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]192</ul>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]193<p>
Daniel Veillardc5d64342001-06-24 12:13:24 +0000[diff] [blame]194<p><a href="mailto:daniel@veillard.com">Daniel Veillard</a></p>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]195</td></tr></table></td></tr></table></td></tr></table></td>
196</tr></table></td></tr></table>
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]197</body>
198</html>