README_DEVELOPERS - platform/external/valgrind - Gitiles


 Building and not installing it
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 To run Valgrind without having to install it, run coregrind/valgrind
 with the VALGRIND_LIB environment variable set, where <dir> is the root
 of the source tree (and must be an absolute path).  Eg:

   VALGRIND_LIB=~/grind/head4/.in_place ~/grind/head4/coregrind/valgrind

 This allows you to compile and run with "make" instead of "make install",
 saving you time.

 Or, you can use the 'vg-in-place' script which does that for you.

 I recommend compiling with "make --quiet" to further reduce the amount of
 output spewed out during compilation, letting you actually see any errors,
 warnings, etc.


 Building a distribution tarball
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 To build a distribution tarball from the valgrind sources:

   make dist

 In addition to compiling, linking and packaging everything up, the command
 will also build the documentation. Even if all required tools for building the
 documentation are installed, this step may not succeed because of hidden
 dependencies. E.g. on Ubuntu you must have "docbook-xsl" installed.
 Additionally, specific tool versions maybe needed.

 If you only want to test whether the generated tarball is complete and runs
 regression tests successfully, building documentation is not needed.
 Edit docs/Makefile.am, search for BUILD_ALL_DOCS and follow instructions there.


 Running the regression tests
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 To build and run all the regression tests, run "make [--quiet] regtest".

 To run a subset of the regression tests, execute:

   perl tests/vg_regtest <name>

 where <name> is a directory (all tests within will be run) or a single
 .vgtest test file, or the name of a program which has a like-named .vgtest
 file.  Eg:

   perl tests/vg_regtest memcheck
   perl tests/vg_regtest memcheck/tests/badfree.vgtest
   perl tests/vg_regtest memcheck/tests/badfree


 Running the performance tests
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 To build and run all the performance tests, run "make [--quiet] perf".

 To run a subset of the performance suite, execute:

   perl perf/vg_perf <name>

 where <name> is a directory (all tests within will be run) or a single
 .vgperf test file, or the name of a program which has a like-named .vgperf
 file.  Eg:

   perl perf/vg_perf perf/
   perl perf/vg_perf perf/bz2.vgperf
   perl perf/vg_perf perf/bz2

 To compare multiple versions of Valgrind, use the --vg= option multiple
 times.  For example, if you have two Valgrinds next to each other, one in
 trunk1/ and one in trunk2/, from within either trunk1/ or trunk2/ do this to
 compare them on all the performance tests:

   perl perf/vg_perf --vg=../trunk1 --vg=../trunk2 perf/


 Debugging Valgrind with GDB
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
 To debug the valgrind launcher program (<prefix>/bin/valgrind) just
 run it under gdb in the normal way.

 Debugging the main body of the valgrind code (and/or the code for
 a particular tool) requires a bit more trickery but can be achieved
 without too much problem by following these steps:

 (1) Set VALGRIND_LAUNCHER to point to the valgrind executable.  Eg:

       export VALGRIND_LAUNCHER=/usr/local/bin/valgrind

     or for an uninstalled version in a source directory $DIR:

       export VALGRIND_LAUNCHER=$DIR/coregrind/valgrind

 (2) Run gdb on the tool executable.  Eg:

       gdb /usr/local/lib/valgrind/ppc32-linux/lackey

     or

       gdb $DIR/.in_place/x86-linux/memcheck

 (3) Do "handle SIGSEGV SIGILL nostop noprint" in GDB to prevent GDB from
     stopping on a SIGSEGV or SIGILL:

     (gdb) handle SIGILL SIGSEGV nostop noprint

 (4) Set any breakpoints you want and proceed as normal for gdb. The
     macro VG_(FUNC) is expanded to vgPlain_FUNC, so If you want to set
     a breakpoint VG_(do_exec), you could do like this in GDB:

     (gdb) b vgPlain_do_exec

 (5) Run the tool with required options:

     (gdb) run pwd

 Steps (1)--(3) can be put in a .gdbinit file, but any directory names must
 be fully expanded (ie. not an environment variable).

 A different and possibly easier way is as follows:

 (1) Run Valgrind as normal, but add the flag --wait-for-gdb=yes.  This
     puts the tool executable into a wait loop soon after it gains
     control.  This delays startup for a few seconds.

 (2) In a different shell, do "gdb /proc/<pid>/exe <pid>", where
     <pid> you read from the output printed by (1).  This attaches
     GDB to the tool executable, which should be in the abovementioned
     wait loop.

 (3) Do "cont" to continue.  After the loop finishes spinning, startup
     will continue as normal.  Note that comment (3) above re passing
     signals applies here too.


 Self-hosting
 ~~~~~~~~~~~~
 This section explains :
   (A) How to configure Valgrind to run under Valgrind.
       Such a setup is called self hosting, or outer/inner setup.
   (B) How to run Valgrind regression tests in a 'self-hosting' mode,
       e.g. to verify Valgrind has no bugs such as memory leaks.
   (C) How to run Valgrind performance tests in a 'self-hosting' mode,
       to analyse and optimise the performance of Valgrind and its tools.

 (A) How to configure Valgrind to run under Valgrind:

 (1) Check out 2 trees, "Inner" and "Outer".  Inner runs the app
     directly.  Outer runs Inner.

 (2) Configure inner with --enable-inner and build/install as usual.

 (3) Configure Outer normally and build/install as usual.

 (4) Choose a very simple program (date) and try

     outer/.../bin/valgrind --sim-hints=enable-outer --trace-children=yes  \
        --smc-check=all-non-file \
        --run-libc-freeres=no --tool=cachegrind -v \
        inner/.../bin/valgrind --vgdb-prefix=./inner --tool=none -v prog

 If you omit the --trace-children=yes, you'll only monitor Inner's launcher
 program, not its stage2. Outer needs --run-libc-freeres=no, as otherwise
 it will try to find and run __libc_freeres in the inner, while libc is not
 used by the inner. Inner needs --vgdb-prefix=./inner to avoid inner
 gdbserver colliding with outer gdbserver.
 Currently, inner does *not* use the client request
 VALGRIND_DISCARD_TRANSLATIONS for the JITted code or the code patched for
 translation chaining. So the outer needs --smc-check=all-non-file to
 detect the modified code.

 Debugging the whole thing might imply to use up to 3 GDB:
   * a GDB attached to the Outer valgrind, allowing
     to examine the state of Outer.
   * a GDB using Outer gdbserver, allowing to
     examine the state of Inner.
   * a GDB using Inner gdbserver, allowing to
     examine the state of prog.

 The whole thing is fragile, confusing and slow, but it does work well enough
 for you to get some useful performance data.  Inner has most of
 its output (ie. those lines beginning with "==<pid>==") prefixed with a '>',
 which helps a lot. However, when running regression tests in an Outer/Inner
 setup, this prefix causes the reg test diff to fail. Give
 --sim-hints=no-inner-prefix to the Inner to disable the production
 of the prefix in the stdout/stderr output of Inner.

 The allocator (coregrind/m_mallocfree.c) is annotated with client requests
 so Memcheck can be used to find leaks and use after free in an Inner
 Valgrind.

 The Valgrind "big lock" is annotated with helgrind client requests
 so helgrind and drd can be used to find race conditions in an Inner
 Valgrind.

 All this has not been tested much, so don't be surprised if you hit problems.

 When using self-hosting with an outer Callgrind tool, use '--pop-on-jump'
 (on the outer). Otherwise, Callgrind has much higher memory requirements.

 (B) Regression tests in an outer/inner setup:

  To run all the regression tests with an outer memcheck, do :
    perl tests/vg_regtest --outer-valgrind=../outer/.../bin/valgrind \
                          --all

  To run a specific regression tests with an outer memcheck, do:
    perl tests/vg_regtest --outer-valgrind=../outer/.../bin/valgrind \
                          none/tests/args.vgtest

  To run regression tests with another outer tool:
    perl tests/vg_regtest --outer-valgrind=../outer/.../bin/valgrind \
                          --outer-tool=helgrind --all

  --outer-args allows to give specific arguments to the outer tool,
  replacing the default one provided by vg_regtest.

 When an outer valgrind runs an inner valgrind, a regression test
 produces one additional file <testname>.outer.log which contains the
 errors detected by the outer valgrind.  E.g. for an outer memcheck, it
 contains the leaks found in the inner, for an outer helgrind or drd,
 it contains the detected race conditions.

 The file tests/outer_inner.supp contains suppressions for
 the irrelevant or benign errors found in the inner.

 (C) Performance tests in an outer/inner setup:

  To run all the performance tests with an outer cachegrind, do :
     perl perf/vg_perf --outer-valgrind=../outer/.../bin/valgrind perf

  To run a specific perf test (e.g. bz2) in this setup, do :
     perl perf/vg_perf --outer-valgrind=../outer/.../bin/valgrind perf/bz2

  To run all the performance tests with an outer callgrind, do :
     perl perf/vg_perf --outer-valgrind=../outer/.../bin/valgrind \
                       --outer-tool=callgrind perf

  To compare the performance of multiple Valgrind versions, do :
     perl perf/vg_perf --outer-valgrind=../outer/.../bin/valgrind \
       --vg=../inner_xxxx --vg=../inner_yyyy perf
   (where inner_xxxx and inner_yyyy are the versions to compare).
   Cachegrind and cg_diff are particularly handy to obtain a delta
   between the two versions.

 When the outer tool is callgrind or cachegrind, the following
 output files will be created for each test:
    <outertoolname>.out.<inner_valgrind_dir>.<tt>.<perftestname>.<pid>
    <outertoolname>.outer.log.<inner_valgrind_dir>.<tt>.<perftestname>.<pid>
  (where tt is the two letters abbreviation for the inner tool(s) run).

 For example, the command
     perl perf/vg_perf \
       --outer-valgrind=../outer_trunk/install/bin/valgrind \
       --outer-tool=callgrind \
       --vg=../inner_tchain --vg=../inner_trunk perf/many-loss-records

 produces the files
     callgrind.out.inner_tchain.no.many-loss-records.18465
     callgrind.outer.log.inner_tchain.no.many-loss-records.18465
     callgrind.out.inner_tchain.me.many-loss-records.21899
     callgrind.outer.log.inner_tchain.me.many-loss-records.21899
     callgrind.out.inner_trunk.no.many-loss-records.21224
     callgrind.outer.log.inner_trunk.no.many-loss-records.21224
     callgrind.out.inner_trunk.me.many-loss-records.22916
     callgrind.outer.log.inner_trunk.me.many-loss-records.22916


 Printing out problematic blocks
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 If you want to print out a disassembly of a particular block that
 causes a crash, do the following.

 Try running with "--vex-guest-chase-thresh=0 --trace-flags=10000000
 --trace-notbelow=999999".  This should print one line for each block
 translated, and that includes the address.

 Then re-run with 999999 changed to the highest bb number shown.
 This will print the one line per block, and also will print a
 disassembly of the block in which the fault occurred.

	Building and not installing it
	~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
	To run Valgrind without having to install it, run coregrind/valgrind
	with the VALGRIND_LIB environment variable set, where <dir> is the root
	of the source tree (and must be an absolute path). Eg:

	VALGRIND_LIB=~/grind/head4/.in_place ~/grind/head4/coregrind/valgrind

	This allows you to compile and run with "make" instead of "make install",
	saving you time.

	Or, you can use the 'vg-in-place' script which does that for you.

	I recommend compiling with "make --quiet" to further reduce the amount of
	output spewed out during compilation, letting you actually see any errors,
	warnings, etc.


	Building a distribution tarball
	~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
	To build a distribution tarball from the valgrind sources:

	make dist

	In addition to compiling, linking and packaging everything up, the command
	will also build the documentation. Even if all required tools for building the
	documentation are installed, this step may not succeed because of hidden
	dependencies. E.g. on Ubuntu you must have "docbook-xsl" installed.
	Additionally, specific tool versions maybe needed.

	If you only want to test whether the generated tarball is complete and runs
	regression tests successfully, building documentation is not needed.
	Edit docs/Makefile.am, search for BUILD_ALL_DOCS and follow instructions there.


	Running the regression tests
	~~~~~~~~~~~~~~~~~~~~~~~~~~~~
	To build and run all the regression tests, run "make [--quiet] regtest".

	To run a subset of the regression tests, execute:

	perl tests/vg_regtest <name>

	where <name> is a directory (all tests within will be run) or a single
	.vgtest test file, or the name of a program which has a like-named .vgtest
	file. Eg:

	perl tests/vg_regtest memcheck
	perl tests/vg_regtest memcheck/tests/badfree.vgtest
	perl tests/vg_regtest memcheck/tests/badfree


	Running the performance tests
	~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
	To build and run all the performance tests, run "make [--quiet] perf".

	To run a subset of the performance suite, execute:

	perl perf/vg_perf <name>

	where <name> is a directory (all tests within will be run) or a single
	.vgperf test file, or the name of a program which has a like-named .vgperf
	file. Eg:

	perl perf/vg_perf perf/
	perl perf/vg_perf perf/bz2.vgperf
	perl perf/vg_perf perf/bz2

	To compare multiple versions of Valgrind, use the --vg= option multiple
	times. For example, if you have two Valgrinds next to each other, one in
	trunk1/ and one in trunk2/, from within either trunk1/ or trunk2/ do this to
	compare them on all the performance tests:

	perl perf/vg_perf --vg=../trunk1 --vg=../trunk2 perf/


	Debugging Valgrind with GDB
	~~~~~~~~~~~~~~~~~~~~~~~~~~~
	To debug the valgrind launcher program (<prefix>/bin/valgrind) just
	run it under gdb in the normal way.

	Debugging the main body of the valgrind code (and/or the code for
	a particular tool) requires a bit more trickery but can be achieved
	without too much problem by following these steps:

	(1) Set VALGRIND_LAUNCHER to point to the valgrind executable. Eg:

	export VALGRIND_LAUNCHER=/usr/local/bin/valgrind

	or for an uninstalled version in a source directory $DIR:

	export VALGRIND_LAUNCHER=$DIR/coregrind/valgrind

	(2) Run gdb on the tool executable. Eg:

	gdb /usr/local/lib/valgrind/ppc32-linux/lackey

	or

	gdb $DIR/.in_place/x86-linux/memcheck

	(3) Do "handle SIGSEGV SIGILL nostop noprint" in GDB to prevent GDB from
	stopping on a SIGSEGV or SIGILL:

	(gdb) handle SIGILL SIGSEGV nostop noprint

	(4) Set any breakpoints you want and proceed as normal for gdb. The
	macro VG_(FUNC) is expanded to vgPlain_FUNC, so If you want to set
	a breakpoint VG_(do_exec), you could do like this in GDB:

	(gdb) b vgPlain_do_exec

	(5) Run the tool with required options:

	(gdb) run pwd

	Steps (1)--(3) can be put in a .gdbinit file, but any directory names must
	be fully expanded (ie. not an environment variable).

	A different and possibly easier way is as follows:

	(1) Run Valgrind as normal, but add the flag --wait-for-gdb=yes. This
	puts the tool executable into a wait loop soon after it gains
	control. This delays startup for a few seconds.

	(2) In a different shell, do "gdb /proc/<pid>/exe <pid>", where
	<pid> you read from the output printed by (1). This attaches
	GDB to the tool executable, which should be in the abovementioned
	wait loop.

	(3) Do "cont" to continue. After the loop finishes spinning, startup
	will continue as normal. Note that comment (3) above re passing
	signals applies here too.


	Self-hosting
	~~~~~~~~~~~~
	This section explains :
	(A) How to configure Valgrind to run under Valgrind.
	Such a setup is called self hosting, or outer/inner setup.
	(B) How to run Valgrind regression tests in a 'self-hosting' mode,
	e.g. to verify Valgrind has no bugs such as memory leaks.
	(C) How to run Valgrind performance tests in a 'self-hosting' mode,
	to analyse and optimise the performance of Valgrind and its tools.

	(A) How to configure Valgrind to run under Valgrind:

	(1) Check out 2 trees, "Inner" and "Outer". Inner runs the app
	directly. Outer runs Inner.

	(2) Configure inner with --enable-inner and build/install as usual.

	(3) Configure Outer normally and build/install as usual.

	(4) Choose a very simple program (date) and try

	outer/.../bin/valgrind --sim-hints=enable-outer --trace-children=yes \
	--smc-check=all-non-file \
	--run-libc-freeres=no --tool=cachegrind -v \
	inner/.../bin/valgrind --vgdb-prefix=./inner --tool=none -v prog

	If you omit the --trace-children=yes, you'll only monitor Inner's launcher
	program, not its stage2. Outer needs --run-libc-freeres=no, as otherwise
	it will try to find and run __libc_freeres in the inner, while libc is not
	used by the inner. Inner needs --vgdb-prefix=./inner to avoid inner
	gdbserver colliding with outer gdbserver.
	Currently, inner does not use the client request
	VALGRIND_DISCARD_TRANSLATIONS for the JITted code or the code patched for
	translation chaining. So the outer needs --smc-check=all-non-file to
	detect the modified code.

	Debugging the whole thing might imply to use up to 3 GDB:
	* a GDB attached to the Outer valgrind, allowing
	to examine the state of Outer.
	* a GDB using Outer gdbserver, allowing to
	examine the state of Inner.
	* a GDB using Inner gdbserver, allowing to
	examine the state of prog.

	The whole thing is fragile, confusing and slow, but it does work well enough
	for you to get some useful performance data. Inner has most of
	its output (ie. those lines beginning with "==<pid>==") prefixed with a '>',
	which helps a lot. However, when running regression tests in an Outer/Inner
	setup, this prefix causes the reg test diff to fail. Give
	--sim-hints=no-inner-prefix to the Inner to disable the production
	of the prefix in the stdout/stderr output of Inner.

	The allocator (coregrind/m_mallocfree.c) is annotated with client requests
	so Memcheck can be used to find leaks and use after free in an Inner
	Valgrind.

	The Valgrind "big lock" is annotated with helgrind client requests
	so helgrind and drd can be used to find race conditions in an Inner
	Valgrind.

	All this has not been tested much, so don't be surprised if you hit problems.

	When using self-hosting with an outer Callgrind tool, use '--pop-on-jump'
	(on the outer). Otherwise, Callgrind has much higher memory requirements.

	(B) Regression tests in an outer/inner setup:

	To run all the regression tests with an outer memcheck, do :
	perl tests/vg_regtest --outer-valgrind=../outer/.../bin/valgrind \
	--all

	To run a specific regression tests with an outer memcheck, do:
	perl tests/vg_regtest --outer-valgrind=../outer/.../bin/valgrind \
	none/tests/args.vgtest

	To run regression tests with another outer tool:
	perl tests/vg_regtest --outer-valgrind=../outer/.../bin/valgrind \
	--outer-tool=helgrind --all

	--outer-args allows to give specific arguments to the outer tool,
	replacing the default one provided by vg_regtest.

	When an outer valgrind runs an inner valgrind, a regression test
	produces one additional file <testname>.outer.log which contains the
	errors detected by the outer valgrind. E.g. for an outer memcheck, it
	contains the leaks found in the inner, for an outer helgrind or drd,
	it contains the detected race conditions.

	The file tests/outer_inner.supp contains suppressions for
	the irrelevant or benign errors found in the inner.

	(C) Performance tests in an outer/inner setup:

	To run all the performance tests with an outer cachegrind, do :
	perl perf/vg_perf --outer-valgrind=../outer/.../bin/valgrind perf

	To run a specific perf test (e.g. bz2) in this setup, do :
	perl perf/vg_perf --outer-valgrind=../outer/.../bin/valgrind perf/bz2

	To run all the performance tests with an outer callgrind, do :
	perl perf/vg_perf --outer-valgrind=../outer/.../bin/valgrind \
	--outer-tool=callgrind perf

	To compare the performance of multiple Valgrind versions, do :
	perl perf/vg_perf --outer-valgrind=../outer/.../bin/valgrind \
	--vg=../inner_xxxx --vg=../inner_yyyy perf
	(where inner_xxxx and inner_yyyy are the versions to compare).
	Cachegrind and cg_diff are particularly handy to obtain a delta
	between the two versions.

	When the outer tool is callgrind or cachegrind, the following
	output files will be created for each test:
	<outertoolname>.out.<inner_valgrind_dir>.<tt>.<perftestname>.<pid>
	<outertoolname>.outer.log.<inner_valgrind_dir>.<tt>.<perftestname>.<pid>
	(where tt is the two letters abbreviation for the inner tool(s) run).

	For example, the command
	perl perf/vg_perf \
	--outer-valgrind=../outer_trunk/install/bin/valgrind \
	--outer-tool=callgrind \
	--vg=../inner_tchain --vg=../inner_trunk perf/many-loss-records

	produces the files
	callgrind.out.inner_tchain.no.many-loss-records.18465
	callgrind.outer.log.inner_tchain.no.many-loss-records.18465
	callgrind.out.inner_tchain.me.many-loss-records.21899
	callgrind.outer.log.inner_tchain.me.many-loss-records.21899
	callgrind.out.inner_trunk.no.many-loss-records.21224
	callgrind.outer.log.inner_trunk.no.many-loss-records.21224
	callgrind.out.inner_trunk.me.many-loss-records.22916
	callgrind.outer.log.inner_trunk.me.many-loss-records.22916


	Printing out problematic blocks
	~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
	If you want to print out a disassembly of a particular block that
	causes a crash, do the following.

	Try running with "--vex-guest-chase-thresh=0 --trace-flags=10000000
	--trace-notbelow=999999". This should print one line for each block
	translated, and that includes the address.

	Then re-run with 999999 changed to the highest bb number shown.
	This will print the one line per block, and also will print a
	disassembly of the block in which the fault occurred.