| <html> |
| <head> |
| <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"> |
| <title>6. README_DEVELOPERS</title> |
| <link rel="stylesheet" type="text/css" href="vg_basic.css"> |
| <meta name="generator" content="DocBook XSL Stylesheets V1.79.1"> |
| <link rel="home" href="index.html" title="Valgrind Documentation"> |
| <link rel="up" href="dist.html" title="Valgrind Distribution Documents"> |
| <link rel="prev" href="dist.readme-missing.html" title="5. README_MISSING_SYSCALL_OR_IOCTL"> |
| <link rel="next" href="dist.readme-packagers.html" title="7. README_PACKAGERS"> |
| </head> |
| <body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"> |
| <div><table class="nav" width="100%" cellspacing="3" cellpadding="3" border="0" summary="Navigation header"><tr> |
| <td width="22px" align="center" valign="middle"><a accesskey="p" href="dist.readme-missing.html"><img src="images/prev.png" width="18" height="21" border="0" alt="Prev"></a></td> |
| <td width="25px" align="center" valign="middle"><a accesskey="u" href="dist.html"><img src="images/up.png" width="21" height="18" border="0" alt="Up"></a></td> |
| <td width="31px" align="center" valign="middle"><a accesskey="h" href="index.html"><img src="images/home.png" width="27" height="20" border="0" alt="Up"></a></td> |
| <th align="center" valign="middle">Valgrind Distribution Documents</th> |
| <td width="22px" align="center" valign="middle"><a accesskey="n" href="dist.readme-packagers.html"><img src="images/next.png" width="18" height="21" border="0" alt="Next"></a></td> |
| </tr></table></div> |
| <div class="chapter"> |
| <div class="titlepage"><div><div><h1 class="title"> |
| <a name="dist.readme-developers"></a>6. README_DEVELOPERS</h1></div></div></div> |
| <div class="literallayout"><p><br> |
| <br> |
| Building and not installing it<br> |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~<br> |
| To run Valgrind without having to install it, run coregrind/valgrind<br> |
| with the VALGRIND_LIB environment variable set, where <dir> is the root<br> |
| of the source tree (and must be an absolute path). Eg:<br> |
| <br> |
| VALGRIND_LIB=~/grind/head4/.in_place ~/grind/head4/coregrind/valgrind <br> |
| <br> |
| This allows you to compile and run with "make" instead of "make install",<br> |
| saving you time.<br> |
| <br> |
| Or, you can use the 'vg-in-place' script which does that for you.<br> |
| <br> |
| I recommend compiling with "make --quiet" to further reduce the amount of<br> |
| output spewed out during compilation, letting you actually see any errors,<br> |
| warnings, etc.<br> |
| <br> |
| <br> |
| Building a distribution tarball<br> |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~<br> |
| To build a distribution tarball from the valgrind sources:<br> |
| <br> |
| make dist<br> |
| <br> |
| In addition to compiling, linking and packaging everything up, the command<br> |
| will also attempt to build the documentation.<br> |
| <br> |
| If you only want to test whether the generated tarball is complete and runs<br> |
| regression tests successfully, building documentation is not needed.<br> |
| <br> |
| make dist BUILD_ALL_DOCS=no<br> |
| <br> |
| If you insist on building documentation some embarrassing instructions<br> |
| can be found in docs/README.<br> |
| <br> |
| <br> |
| Running the regression tests<br> |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~<br> |
| To build and run all the regression tests, run "make [--quiet] regtest".<br> |
| <br> |
| To run a subset of the regression tests, execute:<br> |
| <br> |
| perl tests/vg_regtest <name><br> |
| <br> |
| where <name> is a directory (all tests within will be run) or a single<br> |
| .vgtest test file, or the name of a program which has a like-named .vgtest<br> |
| file. Eg:<br> |
| <br> |
| perl tests/vg_regtest memcheck<br> |
| perl tests/vg_regtest memcheck/tests/badfree.vgtest<br> |
| perl tests/vg_regtest memcheck/tests/badfree<br> |
| <br> |
| <br> |
| Running the performance tests<br> |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~<br> |
| To build and run all the performance tests, run "make [--quiet] perf".<br> |
| <br> |
| To run a subset of the performance suite, execute:<br> |
| <br> |
| perl perf/vg_perf <name><br> |
| <br> |
| where <name> is a directory (all tests within will be run) or a single<br> |
| .vgperf test file, or the name of a program which has a like-named .vgperf<br> |
| file. Eg:<br> |
| <br> |
| perl perf/vg_perf perf/<br> |
| perl perf/vg_perf perf/bz2.vgperf<br> |
| perl perf/vg_perf perf/bz2<br> |
| <br> |
| To compare multiple versions of Valgrind, use the --vg= option multiple<br> |
| times. For example, if you have two Valgrinds next to each other, one in<br> |
| trunk1/ and one in trunk2/, from within either trunk1/ or trunk2/ do this to<br> |
| compare them on all the performance tests:<br> |
| <br> |
| perl perf/vg_perf --vg=../trunk1 --vg=../trunk2 perf/<br> |
| <br> |
| <br> |
| Debugging Valgrind with GDB<br> |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~<br> |
| To debug the valgrind launcher program (<prefix>/bin/valgrind) just<br> |
| run it under gdb in the normal way.<br> |
| <br> |
| Debugging the main body of the valgrind code (and/or the code for<br> |
| a particular tool) requires a bit more trickery but can be achieved<br> |
| without too much problem by following these steps:<br> |
| <br> |
| (1) Set VALGRIND_LAUNCHER to point to the valgrind executable. Eg:<br> |
| <br> |
| export VALGRIND_LAUNCHER=/usr/local/bin/valgrind<br> |
| <br> |
| or for an uninstalled version in a source directory $DIR:<br> |
| <br> |
| export VALGRIND_LAUNCHER=$DIR/coregrind/valgrind<br> |
| <br> |
| (2) Run gdb on the tool executable. Eg:<br> |
| <br> |
| gdb /usr/local/lib/valgrind/ppc32-linux/lackey<br> |
| <br> |
| or<br> |
| <br> |
| gdb $DIR/.in_place/x86-linux/memcheck<br> |
| <br> |
| (3) Do "handle SIGSEGV SIGILL nostop noprint" in GDB to prevent GDB from<br> |
| stopping on a SIGSEGV or SIGILL:<br> |
| <br> |
| (gdb) handle SIGILL SIGSEGV nostop noprint<br> |
| <br> |
| (4) Set any breakpoints you want and proceed as normal for gdb. The<br> |
| macro VG_(FUNC) is expanded to vgPlain_FUNC, so If you want to set<br> |
| a breakpoint VG_(do_exec), you could do like this in GDB:<br> |
| <br> |
| (gdb) b vgPlain_do_exec<br> |
| <br> |
| (5) Run the tool with required options (the --tool option is required<br> |
| for correct setup), e.g.<br> |
| <br> |
| (gdb) run --tool=lackey pwd<br> |
| <br> |
| Steps (1)--(3) can be put in a .gdbinit file, but any directory names must<br> |
| be fully expanded (ie. not an environment variable).<br> |
| <br> |
| A different and possibly easier way is as follows:<br> |
| <br> |
| (1) Run Valgrind as normal, but add the flag --wait-for-gdb=yes. This<br> |
| puts the tool executable into a wait loop soon after it gains<br> |
| control. This delays startup for a few seconds.<br> |
| <br> |
| (2) In a different shell, do "gdb /proc/<pid>/exe <pid>", where<br> |
| <pid> you read from the output printed by (1). This attaches<br> |
| GDB to the tool executable, which should be in the abovementioned<br> |
| wait loop.<br> |
| <br> |
| (3) Do "cont" to continue. After the loop finishes spinning, startup<br> |
| will continue as normal. Note that comment (3) above re passing<br> |
| signals applies here too.<br> |
| <br> |
| <br> |
| Self-hosting<br> |
| ~~~~~~~~~~~~<br> |
| This section explains :<br> |
| (A) How to configure Valgrind to run under Valgrind.<br> |
| Such a setup is called self hosting, or outer/inner setup.<br> |
| (B) How to run Valgrind regression tests in a 'self-hosting' mode,<br> |
| e.g. to verify Valgrind has no bugs such as memory leaks.<br> |
| (C) How to run Valgrind performance tests in a 'self-hosting' mode,<br> |
| to analyse and optimise the performance of Valgrind and its tools.<br> |
| <br> |
| (A) How to configure Valgrind to run under Valgrind:<br> |
| <br> |
| (1) Check out 2 trees, "Inner" and "Outer". Inner runs the app<br> |
| directly. Outer runs Inner.<br> |
| <br> |
| (2) Configure inner with --enable-inner and build/install as usual.<br> |
| <br> |
| (3) Configure Outer normally and build/install as usual.<br> |
| <br> |
| (4) Choose a very simple program (date) and try<br> |
| <br> |
| outer/.../bin/valgrind --sim-hints=enable-outer --trace-children=yes \<br> |
| --smc-check=all-non-file \<br> |
| --run-libc-freeres=no --tool=cachegrind -v \<br> |
| inner/.../bin/valgrind --vgdb-prefix=./inner --tool=none -v prog<br> |
| <br> |
| Note: You must use a "make install"-ed valgrind.<br> |
| Do *not* use vg-in-place for the outer valgrind.<br> |
| <br> |
| If you omit the --trace-children=yes, you'll only monitor Inner's launcher<br> |
| program, not its stage2. Outer needs --run-libc-freeres=no, as otherwise<br> |
| it will try to find and run __libc_freeres in the inner, while libc is not<br> |
| used by the inner. Inner needs --vgdb-prefix=./inner to avoid inner<br> |
| gdbserver colliding with outer gdbserver.<br> |
| Currently, inner does *not* use the client request <br> |
| VALGRIND_DISCARD_TRANSLATIONS for the JITted code or the code patched for<br> |
| translation chaining. So the outer needs --smc-check=all-non-file to<br> |
| detect the modified code.<br> |
| <br> |
| Debugging the whole thing might imply to use up to 3 GDB:<br> |
| * a GDB attached to the Outer valgrind, allowing<br> |
| to examine the state of Outer.<br> |
| * a GDB using Outer gdbserver, allowing to<br> |
| examine the state of Inner.<br> |
| * a GDB using Inner gdbserver, allowing to<br> |
| examine the state of prog.<br> |
| <br> |
| The whole thing is fragile, confusing and slow, but it does work well enough<br> |
| for you to get some useful performance data. Inner has most of<br> |
| its output (ie. those lines beginning with "==<pid>==") prefixed with a '>',<br> |
| which helps a lot. However, when running regression tests in an Outer/Inner<br> |
| setup, this prefix causes the reg test diff to fail. Give <br> |
| --sim-hints=no-inner-prefix to the Inner to disable the production<br> |
| of the prefix in the stdout/stderr output of Inner.<br> |
| <br> |
| The allocator (coregrind/m_mallocfree.c) is annotated with client requests<br> |
| so Memcheck can be used to find leaks and use after free in an Inner<br> |
| Valgrind.<br> |
| <br> |
| The Valgrind "big lock" is annotated with helgrind client requests<br> |
| so helgrind and drd can be used to find race conditions in an Inner<br> |
| Valgrind.<br> |
| <br> |
| All this has not been tested much, so don't be surprised if you hit problems.<br> |
| <br> |
| When using self-hosting with an outer Callgrind tool, use '--pop-on-jump'<br> |
| (on the outer). Otherwise, Callgrind has much higher memory requirements. <br> |
| <br> |
| (B) Regression tests in an outer/inner setup:<br> |
| <br> |
| To run all the regression tests with an outer memcheck, do :<br> |
| perl tests/vg_regtest --outer-valgrind=../outer/.../bin/valgrind \<br> |
| --all<br> |
| <br> |
| To run a specific regression tests with an outer memcheck, do:<br> |
| perl tests/vg_regtest --outer-valgrind=../outer/.../bin/valgrind \<br> |
| none/tests/args.vgtest<br> |
| <br> |
| To run regression tests with another outer tool:<br> |
| perl tests/vg_regtest --outer-valgrind=../outer/.../bin/valgrind \<br> |
| --outer-tool=helgrind --all<br> |
| <br> |
| --outer-args allows to give specific arguments to the outer tool,<br> |
| replacing the default one provided by vg_regtest.<br> |
| <br> |
| Note: --outer-valgrind must be a "make install"-ed valgrind.<br> |
| Do *not* use vg-in-place.<br> |
| <br> |
| When an outer valgrind runs an inner valgrind, a regression test<br> |
| produces one additional file <testname>.outer.log which contains the<br> |
| errors detected by the outer valgrind. E.g. for an outer memcheck, it<br> |
| contains the leaks found in the inner, for an outer helgrind or drd,<br> |
| it contains the detected race conditions.<br> |
| <br> |
| The file tests/outer_inner.supp contains suppressions for <br> |
| the irrelevant or benign errors found in the inner.<br> |
| <br> |
| An regression test running in the inner (e.g. memcheck/tests/badrw) will<br> |
| cause the inner to report an error, which is expected and checked<br> |
| as usual when running the regtests in an outer/inner setup.<br> |
| However, the outer will often also observe an error, e.g. a jump<br> |
| using uninitialised data, or a read/write outside the bounds of a heap<br> |
| block. When the outer reports such an error, it will output the<br> |
| inner host stacktrace. To this stacktrace, it will append the<br> |
| stacktrace of the inner guest program. For example, this is an error<br> |
| reported by the outer when the inner runs the badrw regtest:<br> |
| ==8119== Invalid read of size 2<br> |
| ==8119== at 0x7F2EFD7AF: ???<br> |
| ==8119== by 0x7F2C82EAF: ???<br> |
| ==8119== by 0x7F180867F: ???<br> |
| ==8119== by 0x40051D: main (badrw.c:5)<br> |
| ==8119== by 0x7F180867F: ???<br> |
| ==8119== by 0x1BFF: ???<br> |
| ==8119== by 0x3803B7F0: _______VVVVVVVV_appended_inner_guest_stack_VVVVVVVV_______ (m_execontext.c:332)<br> |
| ==8119== by 0x40055C: main (badrw.c:22)<br> |
| ==8119== Address 0x55cd03c is 4 bytes before a block of size 16 alloc'd<br> |
| ==8119== at 0x2804E26D: vgPlain_arena_malloc (m_mallocfree.c:1914)<br> |
| ==8119== by 0x2800BAB4: vgMemCheck_new_block (mc_malloc_wrappers.c:368)<br> |
| ==8119== by 0x2800BC87: vgMemCheck_malloc (mc_malloc_wrappers.c:403)<br> |
| ==8119== by 0x28097EAE: do_client_request (scheduler.c:1861)<br> |
| ==8119== by 0x28097EAE: vgPlain_scheduler (scheduler.c:1425)<br> |
| ==8119== by 0x280A7237: thread_wrapper (syswrap-linux.c:103)<br> |
| ==8119== by 0x280A7237: run_a_thread_NORETURN (syswrap-linux.c:156)<br> |
| ==8119== by 0x3803B7F0: _______VVVVVVVV_appended_inner_guest_stack_VVVVVVVV_______ (m_execontext.c:332)<br> |
| ==8119== by 0x4C294C4: malloc (vg_replace_malloc.c:298)<br> |
| ==8119== by 0x40051D: main (badrw.c:5)<br> |
| In the above, the first stacktrace starts with the inner host stacktrace,<br> |
| which in this case is some JITted code. Such code sometimes contains IPs<br> |
| that points in the inner guest code (0x40051D: main (badrw.c:5)).<br> |
| After the separator, we have the inner guest stacktrace.<br> |
| The second stacktrace gives the stacktrace where the heap block that was<br> |
| overrun was allocated. We see it was allocated by the inner valgrind<br> |
| in the client arena (first part of the stacktrace). The second part is<br> |
| the guest stacktrace that did the allocation.<br> |
| <br> |
| <br> |
| (C) Performance tests in an outer/inner setup:<br> |
| <br> |
| To run all the performance tests with an outer cachegrind, do :<br> |
| perl perf/vg_perf --outer-valgrind=../outer/.../bin/valgrind perf<br> |
| <br> |
| To run a specific perf test (e.g. bz2) in this setup, do :<br> |
| perl perf/vg_perf --outer-valgrind=../outer/.../bin/valgrind perf/bz2<br> |
| <br> |
| To run all the performance tests with an outer callgrind, do :<br> |
| perl perf/vg_perf --outer-valgrind=../outer/.../bin/valgrind \<br> |
| --outer-tool=callgrind perf<br> |
| <br> |
| Note: --outer-valgrind must be a "make install"-ed valgrind.<br> |
| Do *not* use vg-in-place.<br> |
| <br> |
| To compare the performance of multiple Valgrind versions, do :<br> |
| perl perf/vg_perf --outer-valgrind=../outer/.../bin/valgrind \<br> |
| --outer-tool=callgrind \<br> |
| --vg=../inner_xxxx --vg=../inner_yyyy perf<br> |
| (where inner_xxxx and inner_yyyy are the toplevel directories of<br> |
| the versions to compare).<br> |
| Cachegrind and cg_diff are particularly handy to obtain a delta<br> |
| between the two versions.<br> |
| <br> |
| When the outer tool is callgrind or cachegrind, the following<br> |
| output files will be created for each test:<br> |
| <outertoolname>.out.<inner_valgrind_dir>.<tt>.<perftestname>.<pid><br> |
| <outertoolname>.outer.log.<inner_valgrind_dir>.<tt>.<perftestname>.<pid><br> |
| (where tt is the two letters abbreviation for the inner tool(s) run).<br> |
| <br> |
| For example, the command<br> |
| perl perf/vg_perf \<br> |
| --outer-valgrind=../outer_trunk/install/bin/valgrind \<br> |
| --outer-tool=callgrind \<br> |
| --vg=../inner_tchain --vg=../inner_trunk perf/many-loss-records<br> |
| <br> |
| produces the files<br> |
| callgrind.out.inner_tchain.no.many-loss-records.18465<br> |
| callgrind.outer.log.inner_tchain.no.many-loss-records.18465<br> |
| callgrind.out.inner_tchain.me.many-loss-records.21899<br> |
| callgrind.outer.log.inner_tchain.me.many-loss-records.21899<br> |
| callgrind.out.inner_trunk.no.many-loss-records.21224<br> |
| callgrind.outer.log.inner_trunk.no.many-loss-records.21224<br> |
| callgrind.out.inner_trunk.me.many-loss-records.22916<br> |
| callgrind.outer.log.inner_trunk.me.many-loss-records.22916<br> |
| <br> |
| <br> |
| Printing out problematic blocks<br> |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~<br> |
| If you want to print out a disassembly of a particular block that<br> |
| causes a crash, do the following.<br> |
| <br> |
| Try running with "--vex-guest-chase-thresh=0 --trace-flags=10000000<br> |
| --trace-notbelow=999999". This should print one line for each block<br> |
| translated, and that includes the address.<br> |
| <br> |
| Then re-run with 999999 changed to the highest bb number shown.<br> |
| This will print the one line per block, and also will print a<br> |
| disassembly of the block in which the fault occurred.<br> |
| <br> |
| </p></div> |
| </div> |
| <div> |
| <br><table class="nav" width="100%" cellspacing="3" cellpadding="2" border="0" summary="Navigation footer"> |
| <tr> |
| <td rowspan="2" width="40%" align="left"> |
| <a accesskey="p" href="dist.readme-missing.html"><< 5. README_MISSING_SYSCALL_OR_IOCTL</a> </td> |
| <td width="20%" align="center"><a accesskey="u" href="dist.html">Up</a></td> |
| <td rowspan="2" width="40%" align="right"> <a accesskey="n" href="dist.readme-packagers.html">7. README_PACKAGERS >></a> |
| </td> |
| </tr> |
| <tr><td width="20%" align="center"><a accesskey="h" href="index.html">Home</a></td></tr> |
| </table> |
| </div> |
| </body> |
| </html> |