Allows to run regression tests in an outer/inner setup.
A '3 lines how to':
perl tests/vg_regtest --outer-valgrind=../trunk_untouched/install/bin/valgrind --all
(the outer results for a test xxx is in xxx.outer.log)
To run with another tool (e.g. drd), add the argument --outer-tool=drd
Still to do/things to improve:
* Most (inner) tests are successful when running under an outer
memcheck. Need to analyse the reasons of remaining failures.
* The memcheck annotations in m_mallocfree.c can be improved:
- A superblock is marked 'undefined', it should rather be marked
'no access'.
- When a free block is splitted, the remaining free block is
not made 'no access'. Instead, it is made 'undefined'.
=> this decreases the chance to find bugs.
=> this is not very efficient (e.g. the rest of a superblock
is often marked undefined repetitively).
Similarly, the free block created by VG_(arena_memalign)
is marked 'undefined'. 'No access' would be preferrable.
- mkInuseBlock marks the new block as undefined. This is probably
not needed, as VALGRIND_MALLOCLIKE_BLOCK will do it already.
- VG_(arena_malloc) should give the requested size to
VALGRIND_MALLOCLIKE_BLOCK, not the malloc usable size,
as this decreases the chance to find buffer overrun bugs.
But giving the requested size is tricky (see comments in
the code).
* need to do memcheck annotations in m_poolalloc.c
so as to allow leak checking for pool allocated elements.
* vg_regtest.in
- should analyse the results of the outer and should
produce a separate result for the tests for which
the outer detects an error or a memory leak or ...
Changes done:
README_DEVELOPERS: document the new outer/inner features.
manual-core.xml: document the new sim-hint no-inner-prefix
tests/outer_inner.supp: new file, containing the suppressions for inner.
vg_regtest.in: implement new args --outer-valgrind, --outer-tool, --outer-args.
m_mallocfree.c: annotations for memcheck.
m_libcprint.c: handle the new sim-hint no-inner-prefix
m_main.c: do an (early) parse of --sim-hints
git-svn-id: svn://svn.valgrind.org/valgrind/trunk@12441 a5019735-40e9-0310-863c-91ae7b9d1cf9
diff --git a/README_DEVELOPERS b/README_DEVELOPERS
index a1a3503..d09917a 100644
--- a/README_DEVELOPERS
+++ b/README_DEVELOPERS
@@ -141,8 +141,7 @@
(1) Check out 2 trees, "Inner" and "Outer". Inner runs the app
directly. Outer runs Inner.
-(2) Configure inner with --enable-inner and build/install as
- usual.
+(2) Configure inner with --enable-inner and build/install as usual.
(3) Configure Outer normally and build/install as usual.
@@ -169,15 +168,48 @@
The whole thing is fragile, confusing and slow, but it does work well enough
for you to get some useful performance data. Inner has most of
its output (ie. those lines beginning with "==<pid>==") prefixed with a '>',
-which helps a lot.
+which helps a lot. However, when running regression tests in an Outer/Inner
+setup, this prefix causes the reg test diff to fail. Give
+--sim-hints=no-inner-prefix to the Inner to disable the production
+of the prefix in the stdout/stderr output of Inner.
-At the time of writing the allocator is not annotated with client requests
-so Memcheck is not as useful as it could be. It also has not been tested
-much, so don't be surprised if you hit problems.
+The allocator (coregrind/m_mallocfree.c) is annotated with client requests
+so Memcheck can be used to find leaks and use after free in an Inner
+Valgrind.
+
+The Valgrind "big lock" is annotated with helgrind client requests
+so helgrind and drd can be used to find race conditions in an Inner
+Valgrind.
+
+All this has not been tested much, so don't be surprised if you hit problems.
When using self-hosting with an outer Callgrind tool, use '--pop-on-jump'
(on the outer). Otherwise, Callgrind has much higher memory requirements.
+Regression tests in an outer/inner setup:
+ To run all the regression tests with an outer memcheck, do :
+ perl test/vg_regtest --outer-valgrind=../outer/.../bin/valgrind \
+ --all
+
+ To run a specific regression tests with an outer memcheck, do:
+ perl test/vg_regtest --outer-valgrind=../outer/.../bin/valgrind \
+ none/tests/args.vgtest
+
+ To run regression tests with another outer tool:
+ perl tests/vg_regtest --outer-valgrind=../outer/.../bin/valgrind \
+ --outer-tool=helgrind " --all
+
+ --outer-args allows to give specific arguments to the outer tool,
+ replacing the default one provided by vg_regtest.
+
+When an outer valgrind runs an inner valgrind, a regression test
+produces one additional file <testname>.outer.log which contains the
+errors detected by the outer valgrind. E.g. for an outer memcheck, it
+contains the leaks found in the inner, for an outer helgrind or drd,
+it contains the detected race conditions.
+
+The file tests/outer_inner.supp contains suppressions for
+the irrelevant or benign errors found in the inner.
Printing out problematic blocks
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~