Gitiles
Code Review Sign In
gerrit-public.fairphone.software / platform / external / libxml2 / ce192eb8a698c87a223a87b8c01739580f008a52 / . / doc / xmlmem.html
blob: 7d0b3d4e36a3e50b220b70b871294a8116044e6f [file] [log] [blame]
Daniel Veillard43d3f612001-11-10 11:57:23 +0000[diff] [blame]1<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/1999/REC-html401-19991224/loose.dtd">
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]2<html>
3<head>
Daniel Veillard7216cfd2002-11-08 15:10:00 +0000[diff] [blame]4<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
Daniel Veillardc332dab2002-03-29 14:08:27 +0000[diff] [blame]5<link rel="SHORTCUT ICON" href="/favicon.ico">
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]6<style type="text/css"><!--
Daniel Veillard373a4752002-02-21 14:46:29 +0000[diff] [blame]7TD {font-family: Verdana,Arial,Helvetica}
8BODY {font-family: Verdana,Arial,Helvetica; margin-top: 2em; margin-left: 0em; margin-right: 0em}
9H1 {font-family: Verdana,Arial,Helvetica}
10H2 {font-family: Verdana,Arial,Helvetica}
11H3 {font-family: Verdana,Arial,Helvetica}
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]12A:link, A:visited, A:active { text-decoration: underline }
13--></style>
14<title>Memory Management</title>
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]15</head>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]16<body bgcolor="#8b7765" text="#000000" link="#000000" vlink="#000000">
17<table border="0" width="100%" cellpadding="5" cellspacing="0" align="center"><tr>
18<td width="180">
Daniel Veillard8f40f1e2002-08-28 21:18:45 +0000[diff] [blame]19<a href="http://www.gnome.org/"><img src="gnome2.png" alt="Gnome2 Logo"></a><a href="http://www.w3.org/Status"><img src="w3c.png" alt="W3C Logo"></a><a href="http://www.redhat.com/"><img src="redhat.gif" alt="Red Hat Logo"></a><div align="left"><a href="http://xmlsoft.org/"><img src="Libxml2-Logo-180x168.gif" alt="Made with Libxml2 Logo"></a></div>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]20</td>
21<td><table border="0" width="90%" cellpadding="2" cellspacing="0" align="center" bgcolor="#000000"><tr><td><table width="100%" border="0" cellspacing="1" cellpadding="3" bgcolor="#fffacd"><tr><td align="center">
22<h1>The XML C library for Gnome</h1>
23<h2>Memory Management</h2>
24</td></tr></table></td></tr></table></td>
25</tr></table>
26<table border="0" cellpadding="4" cellspacing="0" width="100%" align="center"><tr><td bgcolor="#8b7765"><table border="0" cellspacing="0" cellpadding="2" width="100%"><tr>
27<td valign="top" width="200" bgcolor="#8b7765"><table border="0" cellspacing="0" cellpadding="1" width="100%" bgcolor="#000000"><tr><td>
28<table width="100%" border="0" cellspacing="1" cellpadding="3">
29<tr><td colspan="1" bgcolor="#eecfa1" align="center"><center><b>Main Menu</b></center></td></tr>
Daniel Veillard4a603e42003-01-11 14:18:53 +0000[diff] [blame]30<tr><td bgcolor="#fffacd">
31<form action="search.php" enctype="application/x-www-form-urlencoded" method="GET">
32<input name="query" type="TEXT" size="20" value=""><input name="submit" type="submit" value="Search ...">
33</form>
34<ul>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]35<li><a href="index.html">Home</a></li>
36<li><a href="intro.html">Introduction</a></li>
37<li><a href="FAQ.html">FAQ</a></li>
38<li><a href="docs.html">Documentation</a></li>
39<li><a href="bugs.html">Reporting bugs and getting help</a></li>
40<li><a href="help.html">How to help</a></li>
41<li><a href="downloads.html">Downloads</a></li>
42<li><a href="news.html">News</a></li>
Daniel Veillard7b602b42002-01-08 13:26:00 +0000[diff] [blame]43<li><a href="XMLinfo.html">XML</a></li>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]44<li><a href="XSLT.html">XSLT</a></li>
Daniel Veillard6dbcaf82002-02-20 14:37:47 +0000[diff] [blame]45<li><a href="python.html">Python and bindings</a></li>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]46<li><a href="architecture.html">libxml architecture</a></li>
47<li><a href="tree.html">The tree output</a></li>
48<li><a href="interface.html">The SAX interface</a></li>
49<li><a href="xmldtd.html">Validation &amp; DTDs</a></li>
50<li><a href="xmlmem.html">Memory Management</a></li>
51<li><a href="encoding.html">Encodings support</a></li>
52<li><a href="xmlio.html">I/O Interfaces</a></li>
53<li><a href="catalog.html">Catalog support</a></li>
54<li><a href="library.html">The parser interfaces</a></li>
55<li><a href="entities.html">Entities or no entities</a></li>
56<li><a href="namespaces.html">Namespaces</a></li>
57<li><a href="upgrade.html">Upgrading 1.x code</a></li>
Daniel Veillard52dcab32001-10-30 12:51:17 +0000[diff] [blame]58<li><a href="threads.html">Thread safety</a></li>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]59<li><a href="DOM.html">DOM Principles</a></li>
60<li><a href="example.html">A real example</a></li>
61<li><a href="contribs.html">Contributions</a></li>
Daniel Veillard7b4b2f92003-01-06 13:11:20 +0000[diff] [blame]62<li><a href="xmlreader.html">The Reader Interface</a></li>
Daniel Veillardfc59c092002-06-05 14:48:26 +0000[diff] [blame]63<li><a href="tutorial/index.html">Tutorial</a></li>
Daniel Veillard7b4b2f92003-01-06 13:11:20 +0000[diff] [blame]64<li><a href="guidelines.html">XML Guidelines</a></li>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]65<li>
66<a href="xml.html">flat page</a>, <a href="site.xsl">stylesheet</a>
67</li>
Daniel Veillard5ede35e2002-10-01 11:37:35 +0000[diff] [blame]68</ul>
69</td></tr>
Daniel Veillard3bf65be2002-01-23 12:36:34 +0000[diff] [blame]70</table>
71<table width="100%" border="0" cellspacing="1" cellpadding="3">
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]72<tr><td colspan="1" bgcolor="#eecfa1" align="center"><center><b>Related links</b></center></td></tr>
Daniel Veillard8acca112002-01-21 09:52:27 +0000[diff] [blame]73<tr><td bgcolor="#fffacd"><ul>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]74<li><a href="http://mail.gnome.org/archives/xml/">Mail archive</a></li>
75<li><a href="http://xmlsoft.org/XSLT/">XSLT libxslt</a></li>
Daniel Veillard4a859202002-01-08 11:49:22 +0000[diff] [blame]76<li><a href="http://phd.cs.unibo.it/gdome2/">DOM gdome2</a></li>
Daniel Veillard2d347fa2002-03-17 10:34:11 +0000[diff] [blame]77<li><a href="http://www.aleksey.com/xmlsec/">XML-DSig xmlsec</a></li>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]78<li><a href="ftp://xmlsoft.org/">FTP</a></li>
Daniel Veillardc84f8b52002-12-19 22:12:47 +0000[diff] [blame]79<li><a href="http://www.zlatkovic.com/projects/libxml/">Windows binaries</a></li>
Daniel Veillarddb9dfd92001-11-26 17:25:02 +0000[diff] [blame]80<li><a href="http://garypennington.net/libxml2/">Solaris binaries</a></li>
Daniel Veillardcb7543b2002-09-09 10:54:06 +0000[diff] [blame]81<li><a href="http://www.zveno.com/open_source/libxml2xslt.html">MacOsX binaries</a></li>
Daniel Veillarde6d8e202002-05-02 06:11:10 +0000[diff] [blame]82<li><a href="http://sourceforge.net/projects/libxml2-pas/">Pascal bindings</a></li>
Daniel Veillard2d347fa2002-03-17 10:34:11 +0000[diff] [blame]83<li><a href="http://bugzilla.gnome.org/buglist.cgi?product=libxml&amp;product=libxml2">Bug Tracker</a></li>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]84</ul></td></tr>
85</table>
Daniel Veillard4a603e42003-01-11 14:18:53 +0000[diff] [blame]86<table width="100%" border="0" cellspacing="1" cellpadding="3">
87<tr><td colspan="1" bgcolor="#eecfa1" align="center"><center><b>API Indexes</b></center></td></tr>
88<tr><td bgcolor="#fffacd"><ul>
89<li><a href="APIchunk0.html">Alphabetic</a></li>
90<li><a href="APIconstructors.html">Constructors</a></li>
91<li><a href="APIfunctions.html">Functions/Types</a></li>
92<li><a href="APIfiles.html">Modules</a></li>
93<li><a href="APIsymbols.html">Symbols</a></li>
94</ul></td></tr>
95</table>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]96</td></tr></table></td>
97<td valign="top" bgcolor="#8b7765"><table border="0" cellspacing="0" cellpadding="1" width="100%"><tr><td><table border="0" cellspacing="0" cellpadding="1" width="100%" bgcolor="#000000"><tr><td><table border="0" cellpadding="3" cellspacing="1" width="100%"><tr><td bgcolor="#fffacd">
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]98<p>Table of Content:</p>
99<ol>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]100<li><a href="#General3">General overview</a></li>
Daniel Veillard0b28e882002-07-24 23:47:05 +0000[diff] [blame]101 <li><a href="#setting">Setting libxml set of memory routines</a></li>
102 <li><a href="#cleanup">Cleaning up after parsing</a></li>
103 <li><a href="#Debugging">Debugging routines</a></li>
104 <li><a href="#General4">General memory requirements</a></li>
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]105</ol>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]106<h3><a name="General3">General overview</a></h3>
107<p>The module <code><a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlmemory.h</a></code>
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]108provides the interfaces to the libxml memory system:</p>
109<ul>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]110<li>libxml does not use the libc memory allocator directly but xmlFree(),
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]111 xmlMalloc() and xmlRealloc()</li>
Daniel Veillard0b28e882002-07-24 23:47:05 +0000[diff] [blame]112 <li>those routines can be reallocated to a specific set of routine, by
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]113 default the libc ones i.e. free(), malloc() and realloc()</li>
Daniel Veillard0b28e882002-07-24 23:47:05 +0000[diff] [blame]114 <li>the xmlmemory.c module includes a set of debugging routine</li>
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]115</ul>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]116<h3><a name="setting">Setting libxml set of memory routines</a></h3>
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]117<p>It is sometimes useful to not use the default memory allocator, either for
118debugging, analysis or to implement a specific behaviour on memory management
119(like on embedded systems). Two function calls are available to do so:</p>
120<ul>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]121<li>
Daniel Veillardaf43f632002-03-08 15:05:20 +0000[diff] [blame]122<a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMemGet
123 ()</a> which return the current set of functions in use by the parser</li>
Daniel Veillard0b28e882002-07-24 23:47:05 +0000[diff] [blame]124 <li>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]125<a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMemSetup()</a>
Daniel Veillardaf43f632002-03-08 15:05:20 +0000[diff] [blame]126 which allow to set up a new set of memory allocation functions</li>
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]127</ul>
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]128<p>Of course a call to xmlMemSetup() should probably be done before calling
129any other libxml routines (unless you are sure your allocations routines are
130compatibles).</p>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]131<h3><a name="cleanup">Cleaning up after parsing</a></h3>
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]132<p>Libxml is not stateless, there is a few set of memory structures needing
Daniel Veillard63d83142002-05-20 06:51:05 +0000[diff] [blame]133allocation before the parser is fully functional (some encoding structures
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]134for example). This also mean that once parsing is finished there is a tiny
135amount of memory (a few hundred bytes) which can be recollected if you don't
136reuse the parser immediately:</p>
137<ul>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]138<li>
139<a href="http://xmlsoft.org/html/libxml-parser.html">xmlCleanupParser
Daniel Veillardaf43f632002-03-08 15:05:20 +0000[diff] [blame]140 ()</a> is a centralized routine to free the parsing states. Note that it
141 won't deallocate any produced tree if any (use the xmlFreeDoc() and
142 related routines for this).</li>
Daniel Veillard0b28e882002-07-24 23:47:05 +0000[diff] [blame]143 <li>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]144<a href="http://xmlsoft.org/html/libxml-parser.html">xmlInitParser
Daniel Veillardaf43f632002-03-08 15:05:20 +0000[diff] [blame]145 ()</a> is the dual routine allowing to preallocate the parsing state
146 which can be useful for example to avoid initialization reentrancy
147 problems when using libxml in multithreaded applications</li>
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]148</ul>
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]149<p>Generally xmlCleanupParser() is safe, if needed the state will be rebuild
150at the next invocation of parser routines, but be careful of the consequences
151in multithreaded applications.</p>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]152<h3><a name="Debugging">Debugging routines</a></h3>
153<p>When configured using --with-mem-debug flag (off by default), libxml uses
Daniel Veillard63d83142002-05-20 06:51:05 +0000[diff] [blame]154a set of memory allocation debugging routines keeping track of all allocated
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]155blocks and the location in the code where the routine was called. A couple of
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]156other debugging routines allow to dump the memory allocated infos to a file
157or call a specific routine when a given block number is allocated:</p>
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]158<ul>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]159<li>
Daniel Veillard0b28e882002-07-24 23:47:05 +0000[diff] [blame]160<a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMallocLoc()</a>
161 <a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlReallocLoc()</a>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]162 and <a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMemStrdupLoc()</a>
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]163 are the memory debugging replacement allocation routines</li>
Daniel Veillard0b28e882002-07-24 23:47:05 +0000[diff] [blame]164 <li>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]165<a href="http://xmlsoft.org/html/libxml-xmlmemory.html">xmlMemoryDump
Daniel Veillardaf43f632002-03-08 15:05:20 +0000[diff] [blame]166 ()</a> dumps all the informations about the allocated memory block lefts
167 in the <code>.memdump</code> file</li>
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]168</ul>
Daniel Veillard63d83142002-05-20 06:51:05 +0000[diff] [blame]169<p>When developing libxml memory debug is enabled, the tests programs call
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]170xmlMemoryDump () and the &quot;make test&quot; regression tests will check for any
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]171memory leak during the full regression test sequence, this helps a lot
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]172ensuring that libxml does not leak memory and bullet proof memory
173allocations use (some libc implementations are known to be far too permissive
174resulting in major portability problems!).</p>
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]175<p>If the .memdump reports a leak, it displays the allocation function and
176also tries to give some informations about the content and structure of the
177allocated blocks left. This is sufficient in most cases to find the culprit,
Daniel Veillard63d83142002-05-20 06:51:05 +0000[diff] [blame]178but not always. Assuming the allocation problem is reproducible, it is
179possible to find more easily:</p>
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]180<ol>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]181<li>write down the block number xxxx not allocated</li>
Daniel Veillard0b28e882002-07-24 23:47:05 +0000[diff] [blame]182 <li>export the environment variable XML_MEM_BREAKPOINT=xxxx , the easiest
Daniel Veillarda8a89fe2002-04-12 21:03:34 +0000[diff] [blame]183 when using GDB is to simply give the command
184 <p><code>set environment XML_MEM_BREAKPOINT xxxx</code></p>
Daniel Veillard0b28e882002-07-24 23:47:05 +0000[diff] [blame]185 <p>before running the program.</p>
186 </li>
187 <li>run the program under a debugger and set a breakpoint on
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]188 xmlMallocBreakpoint() a specific function called when this precise block
189 is allocated</li>
Daniel Veillard0b28e882002-07-24 23:47:05 +0000[diff] [blame]190 <li>when the breakpoint is reached you can then do a fine analysis of the
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]191 allocation an step to see the condition resulting in the missing
192 deallocation.</li>
193</ol>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]194<p>I used to use a commercial tool to debug libxml memory problems but after
195noticing that it was not detecting memory leaks that simple mechanism was
Daniel Veillarda8a89fe2002-04-12 21:03:34 +0000[diff] [blame]196used and proved extremely efficient until now. Lately I have also used <a href="http://developer.kde.org/~sewardj/">valgrind</a> with quite some
197success, it is tied to the i386 architecture since it works by emulating the
198processor and instruction set, it is slow but extremely efficient, i.e. it
199spot memory usage errors in a very precise way.</p>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]200<h3><a name="General4">General memory requirements</a></h3>
201<p>How much libxml memory require ? It's hard to tell in average it depends
202of a number of things:</p>
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]203<ul>
Daniel Veillard63d83142002-05-20 06:51:05 +0000[diff] [blame]204<li>the parser itself should work in a fixed amount of memory, except for
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]205 information maintained about the stacks of names and entities locations.
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]206 The I/O and encoding handlers will probably account for a few KBytes.
207 This is true for both the XML and HTML parser (though the HTML parser
208 need more state).</li>
Daniel Veillard0b28e882002-07-24 23:47:05 +0000[diff] [blame]209 <li>If you are generating the DOM tree then memory requirements will grow
Daniel Veillard63d83142002-05-20 06:51:05 +0000[diff] [blame]210 nearly linear with the size of the data. In general for a balanced
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]211 textual document the internal memory requirement is about 4 times the
Daniel Veillard63d83142002-05-20 06:51:05 +0000[diff] [blame]212 size of the UTF8 serialization of this document (example the XML-1.0
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]213 recommendation is a bit more of 150KBytes and takes 650KBytes of main
214 memory when parsed). Validation will add a amount of memory required for
215 maintaining the external Dtd state which should be linear with the
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]216 complexity of the content model defined by the Dtd</li>
Daniel Veillardce192eb2003-04-16 15:58:05 +0000[diff] [blame^]217 <li>If you need to work with fixed memory requirements or don't need the
218 full DOM tree then using the <a href="xmlreader.html">xmlReader
219 interface</a> is probably the best way to proceed, it still allows to
220 validate or operate on subset of the tree if needed.</li>
Daniel Veillard0b28e882002-07-24 23:47:05 +0000[diff] [blame]221 <li>If you don't care about the advanced features of libxml like
Daniel Veillardce192eb2003-04-16 15:58:05 +0000[diff] [blame^]222 validation, DOM, XPath or XPointer, don't use entities, need to work with
223 fixed memory requirements, and try to get the fastest parsing possible
224 then the SAX interface should be used, but it has known restrictions.</li>
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]225</ul>
Daniel Veillard6581e1b2003-02-09 22:21:43 +0000[diff] [blame]226<p></p>
Daniel Veillard3f4c40f2002-02-13 09:19:28 +0000[diff] [blame]227<p><a href="bugs.html">Daniel Veillard</a></p>
Daniel Veillardb8cfbd12001-10-25 10:53:28 +0000[diff] [blame]228</td></tr></table></td></tr></table></td></tr></table></td>
229</tr></table></td></tr></table>
Daniel Veillard189446d2000-10-13 10:23:06 +0000[diff] [blame]230</body>
231</html>