| <?xml version="1.0"?> <!-- -*- sgml -*- --> |
| <!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" |
| "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"> |
| |
| <chapter id="mc-manual" xreflabel="Memcheck: a heavyweight memory checker"> |
| <title>Memcheck: a heavyweight memory checker</title> |
| |
| <para>To use this tool, you must specify |
| <computeroutput>--tool=memcheck</computeroutput> on the Valgrind |
| command line.</para> |
| |
| |
| <sect1 id="mc-manual.bugs" |
| xreflabel="Kinds of bugs that Memcheck can find"> |
| <title>Kinds of bugs that Memcheck can find</title> |
| |
| <para>Memcheck is Valgrind-1.0.X's checking mechanism bundled up |
| into a tool. All reads and writes of memory are checked, and |
| calls to malloc/new/free/delete are intercepted. As a result, |
| memcheck can detect the following problems:</para> |
| |
| <itemizedlist> |
| <listitem> |
| <para>Use of uninitialised memory</para> |
| </listitem> |
| <listitem> |
| <para>Reading/writing memory after it has been free'd</para> |
| </listitem> |
| <listitem> |
| <para>Reading/writing off the end of malloc'd blocks</para> |
| </listitem> |
| <listitem> |
| <para>Reading/writing inappropriate areas on the stack</para> |
| </listitem> |
| <listitem> |
| <para>Memory leaks -- where pointers to malloc'd blocks are |
| lost forever</para> |
| </listitem> |
| <listitem> |
| <para>Mismatched use of malloc/new/new [] vs |
| free/delete/delete []</para> |
| </listitem> |
| <listitem> |
| <para>Overlapping <computeroutput>src</computeroutput> and |
| <computeroutput>dst</computeroutput> pointers in |
| <computeroutput>memcpy()</computeroutput> and related |
| functions</para> |
| </listitem> |
| <listitem> |
| <para>Some misuses of the POSIX pthreads API</para> |
| </listitem> |
| </itemizedlist> |
| |
| </sect1> |
| |
| |
| |
| <sect1 id="mc-manual.flags" |
| xreflabel="Command-line flags specific to memcheck"> |
| <title>Command-line flags specific to memcheck</title> |
| |
| <itemizedlist> |
| <listitem> |
| <para><computeroutput>--leak-check=no</computeroutput> |
| [default]</para> |
| <para><computeroutput>--leak-check=yes</computeroutput></para> |
| <para>When enabled, search for memory leaks when the client |
| program finishes. A memory leak means a malloc'd block, |
| which has not yet been free'd, but to which no pointer can be |
| found. Such a block can never be free'd by the program, |
| since no pointer to it exists. Leak checking is disabled by |
| default because it tends to generate dozens of error |
| messages.</para> |
| </listitem> |
| |
| <listitem> |
| <para><computeroutput>--show-reachable=no</computeroutput> |
| [default]</para> |
| <para><computeroutput>--show-reachable=yes</computeroutput></para> |
| <para>When disabled, the memory leak detector only shows |
| blocks for which it cannot find a pointer to at all, or it |
| can only find a pointer to the middle of. These blocks are |
| prime candidates for memory leaks. When enabled, the leak |
| detector also reports on blocks which it could find a pointer |
| to. Your program could, at least in principle, have freed |
| such blocks before exit. Contrast this to blocks for which |
| no pointer, or only an interior pointer could be found: they |
| are more likely to indicate memory leaks, because you do not |
| actually have a pointer to the start of the block which you |
| can hand to <computeroutput>free</computeroutput>, even if |
| you wanted to.</para> |
| </listitem> |
| |
| <listitem> |
| <para><computeroutput>--leak-resolution=low</computeroutput> |
| [default]</para> |
| <para><computeroutput>--leak-resolution=med</computeroutput></para> |
| <para><computeroutput>--leak-resolution=high</computeroutput></para> |
| <para>When doing leak checking, determines how willing |
| Memcheck is to consider different backtraces to be the same. |
| When set to <computeroutput>low</computeroutput>, the |
| default, only the first two entries need match. When |
| <computeroutput>med</computeroutput>, four entries have to |
| match. When <computeroutput>high</computeroutput>, all |
| entries need to match.</para> |
| <para>For hardcore leak debugging, you probably want to use |
| <computeroutput>--leak-resolution=high</computeroutput> |
| together with |
| <computeroutput>--num-callers=40</computeroutput> or some |
| such large number. Note however that this can give an |
| overwhelming amount of information, which is why the defaults |
| are 4 callers and low-resolution matching.</para> |
| <para>Note that the |
| <computeroutput>--leak-resolution=</computeroutput> setting |
| does not affect Memcheck's ability to find leaks. It only |
| changes how the results are presented.</para> |
| </listitem> |
| |
| <listitem> |
| <para><computeroutput>--freelist-vol=<number></computeroutput> |
| [default: 1000000]</para> |
| <para>When the client program releases memory using free (in |
| <literal>C</literal>) or delete (<literal>C++</literal>), |
| that memory is not immediately made available for |
| re-allocation. Instead it is marked inaccessible and placed |
| in a queue of freed blocks. The purpose is to delay the |
| point at which freed-up memory comes back into circulation. |
| This increases the chance that Memcheck will be able to |
| detect invalid accesses to blocks for some significant period |
| of time after they have been freed.</para> |
| <para>This flag specifies the maximum total size, in bytes, |
| of the blocks in the queue. The default value is one million |
| bytes. Increasing this increases the total amount of memory |
| used by Memcheck but may detect invalid uses of freed blocks |
| which would otherwise go undetected.</para> |
| </listitem> |
| |
| <listitem> |
| <para><computeroutput>--workaround-gcc296-bugs=no</computeroutput> |
| [default]</para> |
| <para><computeroutput>--workaround-gcc296-bugs=yes</computeroutput></para> |
| <para>When enabled, assume that reads and writes some small |
| distance below the stack pointer |
| <computeroutput>%esp</computeroutput> are due to bugs in gcc |
| 2.96, and does not report them. The "small distance" is 256 |
| bytes by default. Note that gcc 2.96 is the default compiler |
| on some popular Linux distributions (RedHat 7.X, Mandrake) |
| and so you may well need to use this flag. Do not use it if |
| you do not have to, as it can cause real errors to be |
| overlooked. Another option is to use a gcc/g++ which does |
| not generate accesses below the stack pointer. 2.95.3 seems |
| to be a good choice in this respect.</para> |
| <para>Unfortunately (27 Feb 02) it looks like g++ 3.0.4 has a |
| similar bug, so you may need to issue this flag if you use |
| 3.0.4. A while later (early Apr 02) this is confirmed as a |
| scheduling bug in g++-3.0.4.</para> |
| </listitem> |
| |
| <listitem> |
| <para><computeroutput>--partial-loads-ok=yes</computeroutput> |
| [the default]</para> |
| <para><computeroutput>--partial-loads-ok=no</computeroutput></para> |
| <para>Controls how Memcheck handles word (4-byte) loads from |
| addresses for which some bytes are addressible and others are |
| not. When <computeroutput>yes</computeroutput> (the |
| default), such loads do not elicit an address error. |
| Instead, the loaded V bytes corresponding to the illegal |
| addresses indicate undefined, and those corresponding to |
| legal addresses are loaded from shadow memory, as usual.</para> |
| <para>When <computeroutput>no</computeroutput>, loads from |
| partially invalid addresses are treated the same as loads |
| from completely invalid addresses: an illegal-address error |
| is issued, and the resulting V bytes indicate valid data.</para> |
| </listitem> |
| |
| <listitem> |
| <para><computeroutput>--cleanup=no</computeroutput></para> |
| <para><computeroutput>--cleanup=yes</computeroutput> [default]</para> |
| <para><command>This is a flag to help debug valgrind itself. |
| It is of no use to end-users.</command> When enabled, various |
| improvments are applied to the post-instrumented intermediate |
| code, aimed at removing redundant value checks.</para> |
| </listitem> |
| |
| </itemizedlist> |
| </sect1> |
| |
| |
| <sect1 id="mc-manual.errormsgs" |
| xreflabel="Explanation of error messages from Memcheck"> |
| <title>Explanation of error messages from Memcheck</title> |
| |
| <para>Despite considerable sophistication under the hood, |
| Memcheck can only really detect two kinds of errors, use of |
| illegal addresses, and use of undefined values. Nevertheless, |
| this is enough to help you discover all sorts of |
| memory-management nasties in your code. This section presents a |
| quick summary of what error messages mean. The precise behaviour |
| of the error-checking machinery is described in <xref |
| linkend="mc-manual.machine"/>.</para> |
| |
| |
| <sect2 id="mc-manual.badrw" |
| xreflabel="Illegal read / Illegal write errors"> |
| <title>Illegal read / Illegal write errors</title> |
| |
| <para>For example:</para> |
| <programlisting><![CDATA[ |
| Invalid read of size 4 |
| at 0x40F6BBCC: (within /usr/lib/libpng.so.2.1.0.9) |
| by 0x40F6B804: (within /usr/lib/libpng.so.2.1.0.9) |
| by 0x40B07FF4: read_png_image__FP8QImageIO (kernel/qpngio.cpp:326) |
| by 0x40AC751B: QImageIO::read() (kernel/qimage.cpp:3621) |
| Address 0xBFFFF0E0 is not stack'd, malloc'd or free'd |
| ]]></programlisting> |
| |
| <para>This happens when your program reads or writes memory at a |
| place which Memcheck reckons it shouldn't. In this example, the |
| program did a 4-byte read at address 0xBFFFF0E0, somewhere within |
| the system-supplied library libpng.so.2.1.0.9, which was called |
| from somewhere else in the same library, called from line 326 of |
| <filename>qpngio.cpp</filename>, and so on.</para> |
| |
| <para>Memcheck tries to establish what the illegal address might |
| relate to, since that's often useful. So, if it points into a |
| block of memory which has already been freed, you'll be informed |
| of this, and also where the block was free'd at. Likewise, if it |
| should turn out to be just off the end of a malloc'd block, a |
| common result of off-by-one-errors in array subscripting, you'll |
| be informed of this fact, and also where the block was |
| malloc'd.</para> |
| |
| <para>In this example, Memcheck can't identify the address. |
| Actually the address is on the stack, but, for some reason, this |
| is not a valid stack address -- it is below the stack pointer, |
| <literal>%esp</literal>, and that isn't allowed. In this |
| particular case it's probably caused by gcc generating invalid |
| code, a known bug in various flavours of gcc.</para> |
| |
| <para>Note that Memcheck only tells you that your program is |
| about to access memory at an illegal address. It can't stop the |
| access from happening. So, if your program makes an access which |
| normally would result in a segmentation fault, you program will |
| still suffer the same fate -- but you will get a message from |
| Memcheck immediately prior to this. In this particular example, |
| reading junk on the stack is non-fatal, and the program stays |
| alive.</para> |
| |
| </sect2> |
| |
| |
| |
| <sect2 id="mc-manual.uninitvals" |
| xreflabel="Use of uninitialised values"> |
| <title>Use of uninitialised values</title> |
| |
| <para>For example:</para> |
| <programlisting><![CDATA[ |
| Conditional jump or move depends on uninitialised value(s) |
| at 0x402DFA94: _IO_vfprintf (_itoa.h:49) |
| by 0x402E8476: _IO_printf (printf.c:36) |
| by 0x8048472: main (tests/manuel1.c:8) |
| ]]></programlisting> |
| |
| <para>An uninitialised-value use error is reported when your |
| program uses a value which hasn't been initialised -- in other |
| words, is undefined. Here, the undefined value is used somewhere |
| inside the printf() machinery of the C library. This error was |
| reported when running the following small program:</para> |
| <programlisting><![CDATA[ |
| int main() |
| { |
| int x; |
| printf ("x = %d\n", x); |
| }]]></programlisting> |
| |
| <para>It is important to understand that your program can copy |
| around junk (uninitialised) data to its heart's content. |
| Memcheck observes this and keeps track of the data, but does not |
| complain. A complaint is issued only when your program attempts |
| to make use of uninitialised data. In this example, x is |
| uninitialised. Memcheck observes the value being passed to |
| <literal>_IO_printf</literal> and thence to |
| <literal>_IO_vfprintf</literal>, but makes no comment. However, |
| _IO_vfprintf has to examine the value of x so it can turn it into |
| the corresponding ASCII string, and it is at this point that |
| Memcheck complains.</para> |
| |
| <para>Sources of uninitialised data tend to be:</para> |
| <itemizedlist> |
| <listitem> |
| <para>Local variables in procedures which have not been |
| initialised, as in the example above.</para> |
| </listitem> |
| <listitem> |
| <para>The contents of malloc'd blocks, before you write |
| something there. In C++, the new operator is a wrapper round |
| malloc, so if you create an object with new, its fields will |
| be uninitialised until you (or the constructor) fill them in, |
| which is only Right and Proper.</para> |
| </listitem> |
| </itemizedlist> |
| |
| </sect2> |
| |
| |
| |
| <sect2 id="mc-manual.badfrees" xreflabel="Illegal frees"> |
| <title>Illegal frees</title> |
| |
| <para>For example:</para> |
| <programlisting><![CDATA[ |
| Invalid free() |
| at 0x4004FFDF: free (vg_clientmalloc.c:577) |
| by 0x80484C7: main (tests/doublefree.c:10) |
| Address 0x3807F7B4 is 0 bytes inside a block of size 177 free'd |
| at 0x4004FFDF: free (vg_clientmalloc.c:577) |
| by 0x80484C7: main (tests/doublefree.c:10) |
| ]]></programlisting> |
| |
| <para>Memcheck keeps track of the blocks allocated by your |
| program with malloc/new, so it can know exactly whether or not |
| the argument to free/delete is legitimate or not. Here, this |
| test program has freed the same block twice. As with the illegal |
| read/write errors, Memcheck attempts to make sense of the address |
| free'd. If, as here, the address is one which has previously |
| been freed, you wil be told that -- making duplicate frees of the |
| same block easy to spot.</para> |
| |
| </sect2> |
| |
| |
| <sect2 id="mc-manual.rudefn" |
| xreflabel="When a block is freed with an inappropriate deallocation |
| function"> |
| <title>When a block is freed with an inappropriate deallocation |
| function</title> |
| |
| <para>In the following example, a block allocated with |
| <computeroutput>new[]</computeroutput> has wrongly been |
| deallocated with <computeroutput>free</computeroutput>:</para> |
| <programlisting><![CDATA[ |
| Mismatched free() / delete / delete [] |
| at 0x40043249: free (vg_clientfuncs.c:171) |
| by 0x4102BB4E: QGArray::~QGArray(void) (tools/qgarray.cpp:149) |
| by 0x4C261C41: PptDoc::~PptDoc(void) (include/qmemarray.h:60) |
| by 0x4C261F0E: PptXml::~PptXml(void) (pptxml.cc:44) |
| Address 0x4BB292A8 is 0 bytes inside a block of size 64 alloc'd |
| at 0x4004318C: __builtin_vec_new (vg_clientfuncs.c:152) |
| by 0x4C21BC15: KLaola::readSBStream(int) const (klaola.cc:314) |
| by 0x4C21C155: KLaola::stream(KLaola::OLENode const *) (klaola.cc:416) |
| by 0x4C21788F: OLEFilter::convert(QCString const &) (olefilter.cc:272) |
| ]]></programlisting> |
| |
| <para>The following was told to me be the KDE 3 developers. I |
| didn't know any of it myself. They also implemented the check |
| itself.</para> |
| |
| <para>In <literal>C++</literal> it's important to deallocate |
| memory in a way compatible with how it was allocated. The deal |
| is:</para> |
| <itemizedlist> |
| <listitem> |
| <para>If allocated with |
| <computeroutput>malloc</computeroutput>, |
| <computeroutput>calloc</computeroutput>, |
| <computeroutput>realloc</computeroutput>, |
| <computeroutput>valloc</computeroutput> or |
| <computeroutput>memalign</computeroutput>, you must |
| deallocate with <computeroutput>free</computeroutput>.</para> |
| </listitem> |
| <listitem> |
| <para>If allocated with |
| <computeroutput>new[]</computeroutput>, you must deallocate |
| with <computeroutput>delete[]</computeroutput>.</para> |
| </listitem> |
| <listitem> |
| <para>If allocated with <computeroutput>new</computeroutput>, |
| you must deallocate with |
| <computeroutput>delete</computeroutput>.</para> |
| </listitem> |
| </itemizedlist> |
| |
| <para>The worst thing is that on Linux apparently it doesn't |
| matter if you do muddle these up, and it all seems to work ok, |
| but the same program may then crash on a different platform, |
| Solaris for example. So it's best to fix it properly. According |
| to the KDE folks "it's amazing how many C++ programmers don't |
| know this".</para> |
| |
| <para>Pascal Massimino adds the following clarification: |
| <computeroutput>delete[]</computeroutput> must be called |
| associated with a <computeroutput>new[]</computeroutput> because |
| the compiler stores the size of the array and the |
| pointer-to-member to the destructor of the array's content just |
| before the pointer actually returned. This implies a |
| variable-sized overhead in what's returned by |
| <computeroutput>new</computeroutput> or |
| <computeroutput>new[]</computeroutput>.</para> |
| </sect2> |
| |
| |
| |
| <sect2 id="mc-manual.badperm" |
| xreflabel="Passing system call parameters with |
| inadequate read/write permissions"> |
| <title>Passing system call parameters with inadequate read/write |
| permissions</title> |
| |
| <para>Memcheck checks all parameters to system calls, i.e: |
| <itemizedlist> |
| <listitem><para>It checks all the direct parameters |
| themselves.</para></listitem> |
| <listitem><para>Also, if a system call needs to read from a buffer provided |
| by your program, Memcheck checks that the entire buffer is addressible and |
| has valid data, ie, it is readable.</para></listitem> |
| <listitem><para>Also, if the system call needs to write to a user-supplied |
| buffer, Memcheck checks that the buffer is addressible.</para></listitem> |
| </itemizedlist> |
| </para> |
| |
| <para>After the system call, Memcheck updates its tracked information to |
| precisely reflect any changes in memory permissions caused by the system call. |
| </para> |
| |
| <para>Here's an example of two system calls with invalid parameters:</para> |
| <programlisting><![CDATA[ |
| #include <stdlib.h> |
| #include <unistd.h> |
| int main( void ) |
| { |
| char* arr = malloc(10); |
| int* arr2 = malloc(sizeof(int)); |
| write( 1 /* stdout */, arr, 10 ); |
| exit(arr2[0]); |
| } |
| ]]></programlisting> |
| |
| <para>You get these complaints ...</para> |
| <programlisting><![CDATA[ |
| Syscall param write(buf) points to uninitialised byte(s) |
| at 0x25A48723: __write_nocancel (in /lib/tls/libc-2.3.3.so) |
| by 0x259AFAD3: __libc_start_main (in /lib/tls/libc-2.3.3.so) |
| by 0x8048348: (within /auto/homes/njn25/grind/head4/a.out) |
| Address 0x25AB8028 is 0 bytes inside a block of size 10 alloc'd |
| at 0x259852B0: malloc (vg_replace_malloc.c:130) |
| by 0x80483F1: main (a.c:5) |
| |
| Syscall param exit(error_code) contains uninitialised byte(s) |
| at 0x25A21B44: __GI__exit (in /lib/tls/libc-2.3.3.so) |
| by 0x8048426: main (a.c:8) |
| ]]></programlisting> |
| |
| <para>... because the program has (a) tried to write uninitialised junk from |
| the malloc'd block to the standard output, and (b) passed an uninitialised |
| value to <computeroutput>exit</computeroutput>. Note that the first error |
| refers to the memory pointed to by <computeroutput>buf</computeroutput> (not |
| <computeroutput>buf</computeroutput> itself), but the second error refers to |
| the argument <computeroutput>error_code</computeroutput> itself.</para> |
| |
| </sect2> |
| |
| |
| <sect2 id="mc-manual.overlap" |
| xreflabel="Overlapping source and destination blocks"> |
| <title>Overlapping source and destination blocks</title> |
| |
| <para>The following C library functions copy some data from one |
| memory block to another (or something similar): |
| <computeroutput>memcpy()</computeroutput>, |
| <computeroutput>strcpy()</computeroutput>, |
| <computeroutput>strncpy()</computeroutput>, |
| <computeroutput>strcat()</computeroutput>, |
| <computeroutput>strncat()</computeroutput>. |
| The blocks pointed to by their |
| <computeroutput>src</computeroutput> and |
| <computeroutput>dst</computeroutput> pointers aren't allowed to |
| overlap. Memcheck checks for this.</para> |
| |
| <para>For example:</para> |
| <programlisting><![CDATA[ |
| ==27492== Source and destination overlap in memcpy(0xbffff294, 0xbffff280, 21) |
| ==27492== at 0x40026CDC: memcpy (mc_replace_strmem.c:71) |
| ==27492== by 0x804865A: main (overlap.c:40) |
| ==27492== |
| ]]></programlisting> |
| |
| <para>You don't want the two blocks to overlap because one of |
| them could get partially trashed by the copying.</para> |
| |
| </sect2> |
| |
| |
| </sect1> |
| |
| |
| |
| <sect1 id="mc-manual.suppfiles" xreflabel="Writing suppressions files"> |
| <title>Writing suppressions files</title> |
| |
| <para>The basic suppression format is described in |
| <xref linkend="manual-core.suppress"/>.</para> |
| |
| <para>The suppression (2nd) line should have the form:</para> |
| <programlisting><![CDATA[ |
| Memcheck:suppression_type]]></programlisting> |
| |
| <para>Or, since some of the suppressions are shared with Addrcheck:</para> |
| <programlisting><![CDATA[ |
| Memcheck,Addrcheck:suppression_type]]></programlisting> |
| |
| <para>The Memcheck suppression types are as follows:</para> |
| |
| <itemizedlist> |
| <listitem> |
| <para><computeroutput>Value1</computeroutput>, |
| <computeroutput>Value2</computeroutput>, |
| <computeroutput>Value4</computeroutput>, |
| <computeroutput>Value8</computeroutput>, |
| <computeroutput>Value16</computeroutput>, |
| meaning an uninitialised-value error when |
| using a value of 1, 2, 4, 8 or 16 bytes.</para> |
| </listitem> |
| |
| <listitem> |
| <para>Or: <computeroutput>Cond</computeroutput> (or its old |
| name, <computeroutput>Value0</computeroutput>), meaning use |
| of an uninitialised CPU condition code.</para> |
| </listitem> |
| |
| <listitem> |
| <para>Or: <computeroutput>Addr1</computeroutput>, |
| <computeroutput>Addr2</computeroutput>, |
| <computeroutput>Addr4</computeroutput>, |
| <computeroutput>Addr8</computeroutput>, |
| <computeroutput>Addr16</computeroutput>, |
| meaning an invalid address during a |
| memory access of 1, 2, 4, 8 or 16 bytes respectively.</para> |
| </listitem> |
| |
| <listitem> |
| <para>Or: <computeroutput>Param</computeroutput>, meaning an |
| invalid system call parameter error.</para> |
| </listitem> |
| |
| <listitem> |
| <para>Or: <computeroutput>Free</computeroutput>, meaning an |
| invalid or mismatching free.</para> |
| </listitem> |
| |
| <listitem> |
| <para><computeroutput>Overlap</computeroutput>, meaning a |
| <computeroutput>src</computeroutput> / |
| <computeroutput>dst</computeroutput> overlap in |
| <computeroutput>memcpy() or a similar |
| function</computeroutput>.</para> |
| </listitem> |
| |
| <listitem> |
| <para>Last but not least, you can suppress leak reports with |
| <computeroutput>Leak</computeroutput>. Leak suppression was |
| added in valgrind-1.9.3, I believe.</para> |
| </listitem> |
| |
| </itemizedlist> |
| |
| <para>The extra information line: for Param errors, is the name |
| of the offending system call parameter. No other error kinds |
| have this extra line.</para> |
| |
| <para>The first line of the calling context: for Value and Addr |
| errors, it is either the name of the function in which the error |
| occurred, or, failing that, the full path of the .so file or |
| executable containing the error location. For Free errors, is |
| the name of the function doing the freeing (eg, |
| <computeroutput>free</computeroutput>, |
| <computeroutput>__builtin_vec_delete</computeroutput>, etc). For |
| Overlap errors, is the name of the function with the overlapping |
| arguments (eg. <computeroutput>memcpy()</computeroutput>, |
| <computeroutput>strcpy()</computeroutput>, etc).</para> |
| |
| <para>Lastly, there's the rest of the calling context.</para> |
| |
| </sect1> |
| |
| |
| |
| <sect1 id="mc-manual.machine" |
| xreflabel="Details of Memcheck's checking machinery"> |
| <title>Details of Memcheck's checking machinery</title> |
| |
| <para>Read this section if you want to know, in detail, exactly |
| what and how Memcheck is checking.</para> |
| |
| |
| <sect2 id="mc-manual.value" xreflabel="Valid-value (V) bit"> |
| <title>Valid-value (V) bits</title> |
| |
| <para>It is simplest to think of Memcheck implementing a |
| synthetic Intel x86 CPU which is identical to a real CPU, except |
| for one crucial detail. Every bit (literally) of data processed, |
| stored and handled by the real CPU has, in the synthetic CPU, an |
| associated "valid-value" bit, which says whether or not the |
| accompanying bit has a legitimate value. In the discussions |
| which follow, this bit is referred to as the V (valid-value) |
| bit.</para> |
| |
| <para>Each byte in the system therefore has a 8 V bits which |
| follow it wherever it goes. For example, when the CPU loads a |
| word-size item (4 bytes) from memory, it also loads the |
| corresponding 32 V bits from a bitmap which stores the V bits for |
| the process' entire address space. If the CPU should later write |
| the whole or some part of that value to memory at a different |
| address, the relevant V bits will be stored back in the V-bit |
| bitmap.</para> |
| |
| <para>In short, each bit in the system has an associated V bit, |
| which follows it around everywhere, even inside the CPU. Yes, |
| the CPU's (integer and <computeroutput>%eflags</computeroutput>) |
| registers have their own V bit vectors.</para> |
| |
| <para>Copying values around does not cause Memcheck to check for, |
| or report on, errors. However, when a value is used in a way |
| which might conceivably affect the outcome of your program's |
| computation, the associated V bits are immediately checked. If |
| any of these indicate that the value is undefined, an error is |
| reported.</para> |
| |
| <para>Here's an (admittedly nonsensical) example:</para> |
| <programlisting><![CDATA[ |
| int i, j; |
| int a[10], b[10]; |
| for ( i = 0; i < 10; i++ ) { |
| j = a[i]; |
| b[i] = j; |
| }]]></programlisting> |
| |
| <para>Memcheck emits no complaints about this, since it merely |
| copies uninitialised values from |
| <computeroutput>a[]</computeroutput> into |
| <computeroutput>b[]</computeroutput>, and doesn't use them in any |
| way. However, if the loop is changed to:</para> |
| <programlisting><![CDATA[ |
| for ( i = 0; i < 10; i++ ) { |
| j += a[i]; |
| } |
| if ( j == 77 ) |
| printf("hello there\n"); |
| ]]></programlisting> |
| |
| <para>then Valgrind will complain, at the |
| <computeroutput>if</computeroutput>, that the condition depends |
| on uninitialised values. Note that it <command>doesn't</command> |
| complain at the <computeroutput>j += a[i];</computeroutput>, |
| since at that point the undefinedness is not "observable". It's |
| only when a decision has to be made as to whether or not to do |
| the <computeroutput>printf</computeroutput> -- an observable |
| action of your program -- that Memcheck complains.</para> |
| |
| <para>Most low level operations, such as adds, cause Memcheck to |
| use the <literal>V bits</literal> for the operands to calculate |
| the V bits for the result. Even if the result is partially or |
| wholly undefined, it does not complain.</para> |
| |
| <para>Checks on definedness only occur in two places: when a |
| value is used to generate a memory address, and where control |
| flow decision needs to be made. Also, when a system call is |
| detected, valgrind checks definedness of parameters as |
| required.</para> |
| |
| <para>If a check should detect undefinedness, an error message is |
| issued. The resulting value is subsequently regarded as |
| well-defined. To do otherwise would give long chains of error |
| messages. In effect, we say that undefined values are |
| non-infectious.</para> |
| |
| <para>This sounds overcomplicated. Why not just check all reads |
| from memory, and complain if an undefined value is loaded into a |
| CPU register? Well, that doesn't work well, because perfectly |
| legitimate C programs routinely copy uninitialised values around |
| in memory, and we don't want endless complaints about that. |
| Here's the canonical example. Consider a struct like |
| this:</para> |
| <programlisting><![CDATA[ |
| struct S { int x; char c; }; |
| struct S s1, s2; |
| s1.x = 42; |
| s1.c = 'z'; |
| s2 = s1; |
| ]]></programlisting> |
| |
| <para>The question to ask is: how large is <computeroutput>struct |
| S</computeroutput>, in bytes? An |
| <computeroutput>int</computeroutput> is 4 bytes and a |
| <computeroutput>char</computeroutput> one byte, so perhaps a |
| <computeroutput>struct S</computeroutput> occupies 5 bytes? |
| Wrong. All (non-toy) compilers we know of will round the size of |
| <computeroutput>struct S</computeroutput> up to a whole number of |
| words, in this case 8 bytes. Not doing this forces compilers to |
| generate truly appalling code for subscripting arrays of |
| <computeroutput>struct S</computeroutput>'s.</para> |
| |
| <para>So <computeroutput>s1</computeroutput> occupies 8 bytes, |
| yet only 5 of them will be initialised. For the assignment |
| <computeroutput>s2 = s1</computeroutput>, gcc generates code to |
| copy all 8 bytes wholesale into |
| <computeroutput>s2</computeroutput> without regard for their |
| meaning. If Memcheck simply checked values as they came out of |
| memory, it would yelp every time a structure assignment like this |
| happened. So the more complicated semantics described above is |
| necessary. This allows <literal>gcc</literal> to copy |
| <computeroutput>s1</computeroutput> into |
| <computeroutput>s2</computeroutput> any way it likes, and a |
| warning will only be emitted if the uninitialised values are |
| later used.</para> |
| |
| <para>One final twist to this story. The above scheme allows |
| garbage to pass through the CPU's integer registers without |
| complaint. It does this by giving the integer registers |
| <literal>V</literal> tags, passing these around in the expected |
| way. This complicated and computationally expensive to do, but |
| is necessary. Memcheck is more simplistic about floating-point |
| loads and stores. In particular, <literal>V</literal> bits for |
| data read as a result of floating-point loads are checked at the |
| load instruction. So if your program uses the floating-point |
| registers to do memory-to-memory copies, you will get complaints |
| about uninitialised values. Fortunately, I have not yet |
| encountered a program which (ab)uses the floating-point registers |
| in this way.</para> |
| |
| </sect2> |
| |
| |
| <sect2 id="mc-manual.vaddress" xreflabel=" Valid-address (A) bits"> |
| <title>Valid-address (A) bits</title> |
| |
| <para>Notice that the previous subsection describes how the |
| validity of values is established and maintained without having |
| to say whether the program does or does not have the right to |
| access any particular memory location. We now consider the |
| latter issue.</para> |
| |
| <para>As described above, every bit in memory or in the CPU has |
| an associated valid-value (<literal>V</literal>) bit. In |
| addition, all bytes in memory, but not in the CPU, have an |
| associated valid-address (<literal>A</literal>) bit. This |
| indicates whether or not the program can legitimately read or |
| write that location. It does not give any indication of the |
| validity or the data at that location -- that's the job of the |
| <literal>V</literal> bits -- only whether or not the location may |
| be accessed.</para> |
| |
| <para>Every time your program reads or writes memory, Memcheck |
| checks the <literal>A</literal> bits associated with the address. |
| If any of them indicate an invalid address, an error is emitted. |
| Note that the reads and writes themselves do not change the A |
| bits, only consult them.</para> |
| |
| <para>So how do the <literal>A</literal> bits get set/cleared? |
| Like this:</para> |
| |
| <itemizedlist> |
| <listitem> |
| <para>When the program starts, all the global data areas are |
| marked as accessible.</para> |
| </listitem> |
| |
| <listitem> |
| <para>When the program does malloc/new, the A bits for |
| exactly the area allocated, and not a byte more, are marked |
| as accessible. Upon freeing the area the A bits are changed |
| to indicate inaccessibility.</para> |
| </listitem> |
| |
| <listitem> |
| |
| <para>When the stack pointer register |
| (<literal>%esp</literal>) moves up or down, |
| <literal>A</literal> bits are set. The rule is that the area |
| from <literal>%esp</literal> up to the base of the stack is |
| marked as accessible, and below <literal>%esp</literal> is |
| inaccessible. (If that sounds illogical, bear in mind that |
| the stack grows down, not up, on almost all Unix systems, |
| including GNU/Linux.) Tracking <literal>%esp</literal> like |
| this has the useful side-effect that the section of stack |
| used by a function for local variables etc is automatically |
| marked accessible on function entry and inaccessible on |
| exit.</para> |
| </listitem> |
| |
| <listitem> |
| <para>When doing system calls, A bits are changed |
| appropriately. For example, mmap() magically makes files |
| appear in the process's address space, so the A bits must be |
| updated if mmap() succeeds.</para> |
| </listitem> |
| |
| <listitem> |
| <para>Optionally, your program can tell Valgrind about such |
| changes explicitly, using the client request mechanism |
| described above.</para> |
| </listitem> |
| |
| </itemizedlist> |
| |
| </sect2> |
| |
| |
| <sect2 id="mc-manual.together" xreflabel="Putting it all together"> |
| <title>Putting it all together</title> |
| |
| <para>Memcheck's checking machinery can be summarised as |
| follows:</para> |
| |
| <itemizedlist> |
| <listitem> |
| <para>Each byte in memory has 8 associated |
| <literal>V</literal> (valid-value) bits, saying whether or |
| not the byte has a defined value, and a single |
| <literal>A</literal> (valid-address) bit, saying whether or |
| not the program currently has the right to read/write that |
| address.</para> |
| </listitem> |
| |
| <listitem> |
| <para>When memory is read or written, the relevant |
| <literal>A</literal> bits are consulted. If they indicate an |
| invalid address, Valgrind emits an Invalid read or Invalid |
| write error.</para> |
| </listitem> |
| |
| <listitem> |
| <para>When memory is read into the CPU's integer registers, |
| the relevant <literal>V</literal> bits are fetched from |
| memory and stored in the simulated CPU. They are not |
| consulted.</para> |
| </listitem> |
| |
| <listitem> |
| <para>When an integer register is written out to memory, the |
| <literal>V</literal> bits for that register are written back |
| to memory too.</para> |
| </listitem> |
| |
| <listitem> |
| <para>When memory is read into the CPU's floating point |
| registers, the relevant <literal>V</literal> bits are read |
| from memory and they are immediately checked. If any are |
| invalid, an uninitialised value error is emitted. This |
| precludes using the floating-point registers to copy |
| possibly-uninitialised memory, but simplifies Valgrind in |
| that it does not have to track the validity status of the |
| floating-point registers.</para> |
| </listitem> |
| |
| <listitem> |
| <para>As a result, when a floating-point register is written |
| to memory, the associated V bits are set to indicate a valid |
| value.</para> |
| </listitem> |
| |
| <listitem> |
| <para>When values in integer CPU registers are used to |
| generate a memory address, or to determine the outcome of a |
| conditional branch, the <literal>V</literal> bits for those |
| values are checked, and an error emitted if any of them are |
| undefined.</para> |
| </listitem> |
| |
| <listitem> |
| <para>When values in integer CPU registers are used for any |
| other purpose, Valgrind computes the V bits for the result, |
| but does not check them.</para> |
| </listitem> |
| |
| <listitem> |
| <para>One the <literal>V</literal> bits for a value in the |
| CPU have been checked, they are then set to indicate |
| validity. This avoids long chains of errors.</para> |
| </listitem> |
| |
| <listitem> |
| <para>When values are loaded from memory, valgrind checks the |
| A bits for that location and issues an illegal-address |
| warning if needed. In that case, the V bits loaded are |
| forced to indicate Valid, despite the location being invalid.</para> |
| <para>This apparently strange choice reduces the amount of |
| confusing information presented to the user. It avoids the |
| unpleasant phenomenon in which memory is read from a place |
| which is both unaddressible and contains invalid values, and, |
| as a result, you get not only an invalid-address (read/write) |
| error, but also a potentially large set of |
| uninitialised-value errors, one for every time the value is |
| used.</para> |
| <para>There is a hazy boundary case to do with multi-byte |
| loads from addresses which are partially valid and partially |
| invalid. See details of the flag |
| <computeroutput>--partial-loads-ok</computeroutput> for |
| details. </para> |
| </listitem> |
| |
| </itemizedlist> |
| |
| |
| <para>Memcheck intercepts calls to malloc, calloc, realloc, |
| valloc, memalign, free, new and delete. The behaviour you get |
| is:</para> |
| |
| <itemizedlist> |
| |
| <listitem> |
| <para>malloc/new: the returned memory is marked as |
| addressible but not having valid values. This means you have |
| to write on it before you can read it.</para> |
| </listitem> |
| |
| <listitem> |
| <para>calloc: returned memory is marked both addressible and |
| valid, since calloc() clears the area to zero.</para> |
| </listitem> |
| |
| <listitem> |
| <para>realloc: if the new size is larger than the old, the |
| new section is addressible but invalid, as with |
| malloc.</para> |
| </listitem> |
| |
| <listitem> |
| <para>If the new size is smaller, the dropped-off section is |
| marked as unaddressible. You may only pass to realloc a |
| pointer previously issued to you by malloc/calloc/realloc.</para> |
| </listitem> |
| |
| <listitem> |
| <para>free/delete: you may only pass to free a pointer |
| previously issued to you by malloc/calloc/realloc, or the |
| value NULL. Otherwise, Valgrind complains. If the pointer is |
| indeed valid, Valgrind marks the entire area it points at as |
| unaddressible, and places the block in the |
| freed-blocks-queue. The aim is to defer as long as possible |
| reallocation of this block. Until that happens, all attempts |
| to access it will elicit an invalid-address error, as you |
| would hope.</para> |
| </listitem> |
| |
| </itemizedlist> |
| |
| </sect2> |
| </sect1> |
| |
| |
| |
| <sect1 id="mc-manual.leaks" xreflabel="Memory leak detection"> |
| <title>Memory leak detection</title> |
| |
| <para>Memcheck keeps track of all memory blocks issued in |
| response to calls to malloc/calloc/realloc/new. So when the |
| program exits, it knows which blocks are still outstanding -- |
| have not been returned, in other words. Ideally, you want your |
| program to have no blocks still in use at exit. But many |
| programs do.</para> |
| |
| <para>For each such block, Memcheck scans the entire address |
| space of the process, looking for pointers to the block. One of |
| three situations may result:</para> |
| |
| <itemizedlist> |
| |
| <listitem> |
| <para>A pointer to the start of the block is found. This |
| usually indicates programming sloppiness; since the block is |
| still pointed at, the programmer could, at least in |
| principle, free'd it before program exit.</para> |
| </listitem> |
| |
| <listitem> |
| <para>A pointer to the interior of the block is found. The |
| pointer might originally have pointed to the start and have |
| been moved along, or it might be entirely unrelated. |
| Memcheck deems such a block as "dubious", that is, possibly |
| leaked, because it's unclear whether or not a pointer to it |
| still exists.</para> |
| </listitem> |
| |
| <listitem> |
| <para>The worst outcome is that no pointer to the block can |
| be found. The block is classified as "leaked", because the |
| programmer could not possibly have free'd it at program exit, |
| since no pointer to it exists. This might be a symptom of |
| having lost the pointer at some earlier point in the |
| program.</para> |
| </listitem> |
| |
| </itemizedlist> |
| |
| <para>Memcheck reports summaries about leaked and dubious blocks. |
| For each such block, it will also tell you where the block was |
| allocated. This should help you figure out why the pointer to it |
| has been lost. In general, you should attempt to ensure your |
| programs do not have any leaked or dubious blocks at exit.</para> |
| |
| <para>The precise area of memory in which Memcheck searches for |
| pointers is: all naturally-aligned 4-byte words for which all A |
| bits indicate addressibility and all V bits indicated that the |
| stored value is actually valid.</para> |
| |
| </sect1> |
| |
| |
| <sect1 id="mc-manual.clientreqs" xreflabel="Client requests"> |
| <title>Client Requests</title> |
| |
| <para>The following client requests are defined in |
| <filename>memcheck.h</filename>. They also work for Addrcheck. |
| See <filename>memcheck.h</filename> for exact details of their |
| arguments.</para> |
| |
| <itemizedlist> |
| |
| <listitem> |
| <para><computeroutput>VALGRIND_MAKE_NOACCESS</computeroutput>, |
| <computeroutput>VALGRIND_MAKE_WRITABLE</computeroutput> and |
| <computeroutput>VALGRIND_MAKE_READABLE</computeroutput>. |
| These mark address ranges as completely inaccessible, |
| accessible but containing undefined data, and accessible and |
| containing defined data, respectively. Subsequent errors may |
| have their faulting addresses described in terms of these |
| blocks. Returns a "block handle". Returns zero when not run |
| on Valgrind.</para> |
| </listitem> |
| |
| <listitem> |
| <para><computeroutput>VALGRIND_DISCARD</computeroutput>: At |
| some point you may want Valgrind to stop reporting errors in |
| terms of the blocks defined by the previous three macros. To |
| do this, the above macros return a small-integer "block |
| handle". You can pass this block handle to |
| <computeroutput>VALGRIND_DISCARD</computeroutput>. After |
| doing so, Valgrind will no longer be able to relate |
| addressing errors to the user-defined block associated with |
| the handle. The permissions settings associated with the |
| handle remain in place; this just affects how errors are |
| reported, not whether they are reported. Returns 1 for an |
| invalid handle and 0 for a valid handle (although passing |
| invalid handles is harmless). Always returns 0 when not run |
| on Valgrind.</para> |
| </listitem> |
| |
| <listitem> |
| <para><computeroutput>VALGRIND_CHECK_WRITABLE</computeroutput> |
| and <computeroutput>VALGRIND_CHECK_READABLE</computeroutput>: |
| check immediately whether or not the given address range has |
| the relevant property, and if not, print an error message. |
| Also, for the convenience of the client, returns zero if the |
| relevant property holds; otherwise, the returned value is the |
| address of the first byte for which the property is not true. |
| Always returns 0 when not run on Valgrind.</para> |
| </listitem> |
| |
| <listitem> |
| <para><computeroutput>VALGRIND_CHECK_DEFINED</computeroutput>: |
| a quick and easy way to find out whether Valgrind thinks a |
| particular variable (lvalue, to be precise) is addressible |
| and defined. Prints an error message if not. Returns no |
| value.</para> |
| </listitem> |
| |
| <listitem> |
| <para><computeroutput>VALGRIND_DO_LEAK_CHECK</computeroutput>: |
| run the memory leak detector right now. Returns no value. I |
| guess this could be used to incrementally check for leaks |
| between arbitrary places in the program's execution. |
| Warning: not properly tested!</para> |
| </listitem> |
| |
| <listitem> |
| <para><computeroutput>VALGRIND_COUNT_LEAKS</computeroutput>: |
| fills in the four arguments with the number of bytes of |
| memory found by the previous leak check to be leaked, |
| dubious, reachable and suppressed. Again, useful in test |
| harness code, after calling |
| <computeroutput>VALGRIND_DO_LEAK_CHECK</computeroutput>.</para> |
| </listitem> |
| |
| <listitem> |
| <para><computeroutput>VALGRIND_GET_VBITS</computeroutput> and |
| <computeroutput>VALGRIND_SET_VBITS</computeroutput>: allow |
| you to get and set the V (validity) bits for an address |
| range. You should probably only set V bits that you have got |
| with <computeroutput>VALGRIND_GET_VBITS</computeroutput>. |
| Only for those who really know what they are doing.</para> |
| </listitem> |
| |
| </itemizedlist> |
| |
| </sect1> |
| </chapter> |