Lucas De Marchi | 23e354b | 2012-02-06 16:50:54 -0200 | [diff] [blame] | 1 | testsuite |
| 2 | |
| 3 | OVERVIEW |
| 4 | ======== |
| 5 | |
| 6 | Kmod's testsuite was designed to automate the process of running tests with |
| 7 | different scenarios, configurations and architectures. The idea is that once we |
| 8 | received a bug report, we reproduce it using the testsuite so we avoid |
| 9 | recurring on the same bug in future. |
| 10 | |
| 11 | |
| 12 | FEATURES |
| 13 | ======== |
| 14 | |
| 15 | - Isolate each test by running them in separate processes; |
| 16 | - Exec a binary, so we can test the tools and not only the lib API |
| 17 | - Fake accesses to filesystem so we can provide a test rootfs with all the |
| 18 | configuration, indexes, etc that test needs to be executed. |
| 19 | - Fake calls to init_module(), delete_module() and uname(), so we don't have to |
| 20 | run tests as root and figure out how to deal with different architectures. |
| 21 | |
| 22 | HOW TO ADD A TEST |
| 23 | ================= |
| 24 | |
| 25 | The simplest way to add a test is to copy and paste one already there. Most of |
| 26 | the options you can have are covered by another tests. This is what you need to |
| 27 | pay attention when writing a test: |
| 28 | |
| 29 | 1 - Look at testsuite.h, struct test, to see all the options available. |
| 30 | |
Lucas De Marchi | 4083b09 | 2012-02-07 10:58:48 -0200 | [diff] [blame] | 31 | 2 - Use TESTSUITE_MAIN and DEFINE_TEST to add new tests. Don't forget to fill |
| 32 | its description. |
Lucas De Marchi | 23e354b | 2012-02-06 16:50:54 -0200 | [diff] [blame] | 33 | |
| 34 | 3 - If you want testsuite to compare the stdout/stderr of your tests in order |
Lucas De Marchi | 4083b09 | 2012-02-07 10:58:48 -0200 | [diff] [blame] | 35 | to check if it worked or not, fill in output.{stderr,stdout} the file with |
Caio Marcelo de Oliveira Filho | f4f408f | 2015-03-05 13:06:38 -0300 | [diff] [blame^] | 36 | the expected output. Bear in mind the same file is used for all |
Lucas De Marchi | 4083b09 | 2012-02-07 10:58:48 -0200 | [diff] [blame] | 37 | architectures, so don't print arch-dependent content if you are comparing |
| 38 | the output. |
Lucas De Marchi | 23e354b | 2012-02-06 16:50:54 -0200 | [diff] [blame] | 39 | |
| 40 | 4 - Fill in the config vector. Setting any of these configuration will make |
Lucas De Marchi | 4083b09 | 2012-02-07 10:58:48 -0200 | [diff] [blame] | 41 | testsuite to export LD_PRELOAD with the necessary override libs before |
| 42 | executing the test. If you are not exec'ing an external binary, you need to |
| 43 | pass "need_spawn = true" below, otherwise it will not work (LD_PRELOAD is |
| 44 | only applied when exec'ing a binary). See each config description in |
| 45 | testsuite.h |
Lucas De Marchi | 23e354b | 2012-02-06 16:50:54 -0200 | [diff] [blame] | 46 | |
| 47 | 5 - need_spawn: if testsuite will exec itself before running a test |
| 48 | |
| 49 | 6 - expected_fail: if that test is expected to fail, i.e. the return code is |
Lucas De Marchi | 4083b09 | 2012-02-07 10:58:48 -0200 | [diff] [blame] | 50 | expected not to be 0. |
Lucas De Marchi | 23e354b | 2012-02-06 16:50:54 -0200 | [diff] [blame] | 51 | |
Caio Marcelo de Oliveira Filho | f4f408f | 2015-03-05 13:06:38 -0300 | [diff] [blame^] | 52 | 7 - The rootfs is populated by copying the entire contents of rootfs-pristine |
| 53 | then running populate-modules.sh to copy generated modules from |
| 54 | module-playground. Update the latter script to include any modules your |
| 55 | test need. |
Lucas De Marchi | 23e354b | 2012-02-06 16:50:54 -0200 | [diff] [blame] | 56 | |
| 57 | 8 - Tests can be run individually, outside of 'make check'. strace and gdb work |
Lucas De Marchi | 4083b09 | 2012-02-07 10:58:48 -0200 | [diff] [blame] | 58 | too, as long as you tell them to operate on child process. |
Lucas De Marchi | 54c43df | 2012-02-06 19:50:17 -0200 | [diff] [blame] | 59 | |
| 60 | 9 - Make sure test passes when using "default" build flags, i.e. by running |
Lucas De Marchi | 4083b09 | 2012-02-07 10:58:48 -0200 | [diff] [blame] | 61 | bootstrap-configure instead of simpler bootstrap/autogen.sh |