| LTP Test Writing Guidelines |
| =========================== |
| |
| This document describes LTP guidelines and LTP test interface and is intended |
| for anybody who want to write or modify a LTP testcase. It's not a definitive |
| guide and it's not, by any means, a substitute for common sense. |
| |
| |
| 1. General Rules |
| ---------------- |
| |
| 1.1 Simplicity |
| ~~~~~~~~~~~~~~ |
| |
| For all it's worth keep the testcases simple or better as simple as possible. |
| The kernel and libc are tricky beasts and the complexity imposed by their |
| interfaces is quite high. Concentrate on the interface you want to test and |
| follow the UNIX philosophy. It's a good idea to make the test as |
| self-contained as possible too (it should not depend on tools or libraries |
| that are not widely available). |
| |
| Do not reinvent the wheel |
| |
| * Use LTP standard interface |
| * Do roll add custom PASS/FAIL reporting functions |
| * Do not write Makefiles from scratch |
| use LTP build system instead, etc. |
| * ... |
| |
| 1.2 Code duplication |
| ~~~~~~~~~~~~~~~~~~~~ |
| |
| Copy & paste is a good servant but very poor master. If you are about to copy a |
| large part of the code from one testcase to another, think what would happen if |
| you find bug in the code that has been copied all around the tree. What about |
| moving it to a library instead? |
| |
| The same goes for short but complicated parts, whenever you are about to copy & |
| paste a syscall wrapper that packs arguments accordingly to machine |
| architecture or similarly complicated code, put it into a header instead. |
| |
| 1.3 Coding style |
| ~~~~~~~~~~~~~~~~ |
| |
| 1.3.1 C coding style |
| ^^^^^^^^^^^^^^^^^^^^ |
| |
| LTP adopted Linux kernel coding style. If you aren't familiar with its rules |
| locate 'linux/Documentation/CodingStyle' in the kernel sources and read it, |
| it's a well written introduction. |
| |
| There is also a checkpatch (see 'linux/scripts/checkpatch.pl') script that can |
| be used to check your patches before the submission. |
| |
| NOTE: If checkpatch does not report any problems, the code still may be wrong |
| as the tool only looks for common mistakes. |
| |
| 1.3.2 Shell coding style |
| ^^^^^^^^^^^^^^^^^^^^^^^^ |
| |
| When writing testcases in shell write in portable shell only, it's a good idea |
| to try to run the test using alternative shell (alternative to bash, for |
| example dash) too. |
| |
| Here are some common sense style rules for shell |
| |
| * Keep lines under 80 chars |
| |
| * Use tabs for indentation |
| |
| * Keep things simple, avoid unnecessary shubshells |
| |
| * Don't do confusing things (i.e. don't name your functions like common shell |
| commands, etc.) |
| |
| * Quote variables |
| |
| * Be consistent |
| |
| 1.4 Commenting code |
| ~~~~~~~~~~~~~~~~~~~ |
| |
| Comments can sometimes save you day but they can easily do more harm than |
| good. There has been several cases where comments and actual implementation |
| were drifting slowly apart which yielded into API misuses and hard to find |
| bugs. Remember there is only one thing worse than no documentation, wrong |
| documentation. |
| |
| Generally everybody should write code that is obvious (which unfortunately |
| isn't always possible). If there is a code that needs to be commented keep it |
| short and to the point. Never ever comment the obvious. |
| |
| In case of LTP testcases it's customary to add a paragraph with highlevel test |
| description somewhere at the beginning of the file (usually right under the GPL |
| header). This helps other people to understand the overall goal of the test |
| before they dive into the technical details. |
| |
| 1.5 Backward compatibility |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| LTP test should be as backward compatible as possible. Think of an enterprise |
| distributions with long term support (more than five years since the initial |
| release) or of an embedded platform that needs to use several years old |
| toolchain supplied by the manufacturer. |
| |
| Therefore LTP test for more current features should be able to cope with older |
| systems. It should at least compile fine and if it's not appropriate for the |
| configuration it should return 'TCONF' (see test interface description below). |
| |
| There are several types of checks we use: |
| |
| The *configure script* is usually used to detect availability of a function |
| declarations in system headers. It's used to disable tests at compile time. |
| |
| The *tst_kvercmp()* which is runtime kernel version detection and comparison |
| and is used to disable tests at runtime. |
| |
| Checking the *errno* value is another type of runtime check. Most of the |
| syscalls returns either 'EINVAL' or 'ENOSYS' when syscall was not implemented |
| or was disabled upon kernel compilation. |
| |
| Sometimes it also makes sense to define a few macros instead of creating |
| configure test. One example are Linux specific POSIX clock ids in |
| 'include/lapi/posix_clocks.h'. |
| |
| 1.6 Dealing with messed up legacy code |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| LTP contains a lot of old and messy code and we are cleaning it up as fast as |
| we can but despite the efforts there is still a lot. If you start modifying |
| old or a messed up testcase and your changes are more complicated than simple |
| typo fixes you should do a cleanup first (in a separate patch). It's easier to |
| review the changes if you separate the formatting fixes from the changes that |
| affects the test behavior. |
| |
| The same goes for moving files. If you need a rename or move file do it in a |
| separate patch. |
| |
| 1.7 License |
| ~~~~~~~~~~~ |
| |
| Code contributed to LTP should be licensed under GPLv2+ (GNU GPL version 2 or |
| any later version). |
| |
| 2. Writing a testcase |
| --------------------- |
| |
| 2.1 LTP Structure |
| ~~~~~~~~~~~~~~~~~ |
| |
| The structure of LTP is quite simple. Each test is a binary written either in |
| portable shell (no bashishm) or C. The test gets a configuration via |
| environment variables and/or command line parameters, it prints additional |
| information into the stdout and reports overall success/failure via the exit |
| value. |
| |
| Tests are generally placed under the 'testcases/' directory. Everything that |
| is a syscall or (slightly confusingly) libc syscall wrapper goes under |
| 'testcases/kernel/syscalls/'. Then there is 'testcases/open_posix_testsuite' |
| which is a well maintained fork of the upstream project that has been dead |
| since 2005 and also a number of directories with tests for more specific |
| features. |
| |
| 2.1.1 Runtest Files |
| ^^^^^^^^^^^^^^^^^^^ |
| |
| The list of tests to be executed is stored in runtest files under the |
| 'runtest/' directory. The default set of runtest files to be executed is |
| stored in 'scenario_groups/default'. When you add a test you should add |
| corresponding entires into some runtest files as well. |
| |
| For syscall tests (these placed under 'testcases/kernel/syscalls/') use |
| 'runtest/syscalls' file, for kernel related tests for memory management we |
| have 'runtest/mm', etc. |
| |
| IMPORTANT: The runtest files should have one entry per a test. Creating a |
| wrapper that runs all your tests and adding it as a single test |
| into runtest file is strongly discouraged. |
| |
| 2.1.2 Datafiles |
| ^^^^^^^^^^^^^^^ |
| |
| If your test needs datafiles to work, these should be generally put into a |
| subdirectory named test name ('TCID' see below) and install it into the |
| 'testcases/data/' directory. The datafiles can be accessed as |
| '$LTPROOT/testcases/data/TCID/...'. See |
| 'testcases/network/rpc/basic_tests/rpc01/' for example. |
| |
| NOTE: There may be some convenience interface added later. |
| |
| 2.1.3 Subexecutables |
| ^^^^^^^^^^^^^^^^^^^^ |
| |
| If you test needs to execute a binary place it in the same directory as the |
| testcase and name the file starting with testname_ ('TCID_' see below). |
| Once the test is executed by the framework the path to the directory with all |
| LTP binaries is added to the '$PATH' and you can execute it just by it's name. |
| |
| TIP: If you need to execute such test from the LTP tree you can add path to |
| current directory to '$PATH' manually with: 'PATH="$PATH:$PWD" ./foo01'. |
| |
| 2.2 Writing a test in C |
| ~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| 2.2.1 Basic test structure |
| ^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| |
| Let's start with an example, following code is a simple test for a getenv(). |
| |
| [source,c] |
| ------------------------------------------------------------------------------- |
| /* |
| * This is test for basic functionality of getenv(). |
| * |
| * - create an env variable and verify that getenv() can get get it |
| * |
| * - call getenv() with nonexisting variable name, check that it returns NULL |
| */ |
| |
| #include "test.h" |
| #include "usctest.h" |
| |
| char *TCID = "getenv01"; |
| int TST_TOTAL = 2; |
| |
| #define TEST_ENV "LTP_TEST_ENV" |
| #define TEST_NE_ENV "LTP_TEST_THIS_DOES_NOT_EXIST" |
| #define TEST_ENV_VAL "val" |
| |
| static void setup(void) |
| { |
| if (setenv(TEST_ENV, TEST_ENV_VAL, 1)) |
| tst_brkm(TBROK | TERRNO, NULL, "setenv() failed"); |
| } |
| |
| static void test(void) |
| { |
| char *ret; |
| |
| ret = getenv(TEST_ENV); |
| |
| if (ret) { |
| if (!strcmp(ret, TEST_ENV_VAL)) |
| tst_resm(TPASS, "getenv(" TEST_ENV ") = '" |
| TEST_ENV_VAL "'"); |
| else |
| tst_resm(TFAIL, "getenv(" TEST_ENV "} = '%s', " |
| "expected '" TEST_ENV_VAL "'", ret); |
| } else { |
| tst_resm(TFAIL, "getenv(" TEST_ENV ") = NULL"); |
| } |
| |
| ret = getenv(TEST_NE_ENV); |
| |
| if (ret) |
| tst_resm(TFAIL, "getenv(" TEST_NE_ENV ") = '%s'", ret); |
| else |
| tst_resm(TPASS, "getenv(" TEST_NE_ENV ") = NULL"); |
| } |
| |
| int main(int argc, char *argv[]) |
| { |
| const char *msg; |
| int lc; |
| |
| if ((msg = parse_opts(argc, argv, NULL, NULL))) |
| tst_brkm(TBROK, NULL, "OPTION PARSING ERROR - %s", msg); |
| |
| setup(); |
| |
| for (lc = 0; TEST_LOOPING(lc); lc++) |
| test(); |
| |
| tst_exit(); |
| } |
| ------------------------------------------------------------------------------- |
| |
| Each test must define 'TCID' and 'TST_TOTAL'. |
| |
| 'TCID' defines test name (usually syscall/libcall name + number) and is used |
| as a base for temporary directory name (if 'tst_tmpdir()' is used). In most of |
| the cases the 'TCID' is the same as test filename (without the extension). |
| |
| 'TST_TOTAL' defines total number of tests. |
| |
| NOTE: The test should report 'TST_TOTAL' PASSES/FAILS on each iteration. |
| |
| The overall test initialization is usually done in a 'setup()' function and |
| the overall cleanup is done in a 'cleanup()' function. Here 'cleanup()' is |
| omitted as the test don't have to clean anything up. Usually it's called right |
| before the 'tst_exit()' to do the cleanup and passed to library functions that |
| can exit the test execution, for example 'tst_brkm()', so that the test can do |
| a cleanup before it exits. |
| |
| WARNING: Never pass a cleanup function to functions called from cleanup. |
| If you don't see why this is a problem you are not ready to write |
| a testcase yet. |
| |
| WARNING: Don't use 'tst_brkm()' in 'cleanup()' unless you really want to exit |
| the cleanup prematurely. See discussion below for further |
| information. |
| |
| The 'parse_opts()' parses the test command line arguments, it's important to |
| use it even if the test has no (other than default) parameters. |
| |
| The last important thing is the 'TEST_LOOPING()' macro, each test has standard |
| options so that it can be executed N times or for M seconds. |
| |
| NOTE: The 'test()' function must work correctly even if executed several times |
| from test main loop. |
| |
| A word about the cleanup() callback |
| +++++++++++++++++++++++++++++++++++ |
| |
| There are a few rules that needs to be followed in order to write correct |
| cleanup() callback. |
| |
| 1. Don't call callback() from within the callback(). |
| (This includes passing callback pointer to library 'tst_' functions.) |
| |
| 2. Make sure to free resources in the reverse order they were |
| initialized. (Some of the steps may not depend on others and everything |
| will work if there were swapped but let's keep it in order.) |
| |
| 3. Free only resources that were initialized. |
| |
| The third rule needs a bit more detailed discussion. Consider, for example, |
| test setup below which is example of a setup that prepares temporary |
| directory, two file descriptors and allocates a buffer. |
| |
| [source,c] |
| ------------------------------------------------------------------------------- |
| static int fd0, fd1; |
| static void *buf; |
| #define BUFSIZE (1024 * 1024) |
| |
| static void setup(void) |
| { |
| tst_require_root(); |
| tst_tmpdir(); |
| |
| fd0 = SAFE_OPEN(cleanup, "test_file1", O_CREAT | O_RDWR, 0666); |
| SAFE_UNLINK(cleanup, "test_file1"); |
| fd1 = SAFE_OPEN(cleanup, "test_file2", O_CREAT | O_RDWR, 0666); |
| SAFE_UNLINK(cleanup, "test_file2"); |
| |
| buf = SAFE_MALLOC(cleanup, BUFSIZE); |
| } |
| ------------------------------------------------------------------------------- |
| |
| In this case the 'cleanup()' function may be entered in five different states: |
| |
| * The first 'SAFE_OPEN()' has failed, temporary directory is created |
| no files are open and +fd0 == -1+, +fd1 == 0+ and +buf == NULL+. |
| |
| * The first 'SAFE_UNLINK()' has failed, +fd0+ holds file descriptor and |
| +fd1 == 0+. |
| |
| * The second 'SAFE_OPEN()' has failed, +fd0+ holds file descriptor and |
| +fd1 == -1+. |
| |
| * The second 'SAFE_UNLINK()' or 'SAFE_MALLOC()' has failed and both 'fd0' and |
| 'fd1' holds file descriptors, +buf+ is still +NULL+. |
| |
| * The 'cleanup()' was called at the end of the test, all +fd0+, +fd1+ and |
| +buf+ are initialized. |
| |
| The 'cleanup()' functions should be able to cope with all scenarios. In this |
| case following code will do: |
| |
| [source,c] |
| ------------------------------------------------------------------------------- |
| static void cleanup(void) |
| { |
| free(buf); |
| |
| if (fd1 > 0) |
| close(fd1); |
| |
| if (fd0 > 0) |
| close(fd0); |
| |
| tst_rmdir(); |
| } |
| ------------------------------------------------------------------------------- |
| |
| 2.2.2 Basic test interface |
| ^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| |
| [source,c] |
| ------------------------------------------------------------------------------- |
| void tst_resm(int ttype, char *arg_fmt, ...); |
| ------------------------------------------------------------------------------- |
| |
| Printf-like function to report test result, it's mostly used with ttype: |
| |
| |============================== |
| | 'TPASS' | Test has passed. |
| | 'TFAIL' | Test has failed. |
| | 'TINFO' | General message. |
| |============================== |
| |
| [source,c] |
| ------------------------------------------------------------------------------- |
| void tst_brkm(int ttype, void (*cleanup)(void), char *arg_fmt, ...); |
| ------------------------------------------------------------------------------- |
| |
| Printf-like function to report error and exit the test, it can be used with ttype: |
| |
| |============================================================ |
| | 'TBROK' | Something has failed in test preparation phase. |
| | 'TCONF' | test is not appropriate for current configuration |
| (syscall not implemented, unsupported arch, ...) |
| | 'TFAIL' | test has failed |
| |============================================================ |
| |
| |
| [source,c] |
| ------------------------------------------------------------------------------- |
| void tst_require_root(void (*cleanup)(void)); |
| ------------------------------------------------------------------------------- |
| |
| Abort the test if it's not executed with root privileges. If needed this should |
| be one of the first checks in the test 'setup()'. |
| |
| WARNING: The cleanup parameter is deprecated and should always be 'NULL'. |
| |
| |
| [source,c] |
| ------------------------------------------------------------------------------- |
| void tst_exit(void); |
| ------------------------------------------------------------------------------- |
| |
| Exits the tests, note that this function has no parameters, the PASSES/FAILS |
| reported by the 'tst_resm()' interfaces were stored and are used for the exit |
| value. |
| |
| |
| 2.2.3 Test temporary directory |
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| |
| If the test needs to create files (which is common case) it must create a test |
| temporary directory in LTP temp directory and work with files only inside of |
| this directory. Happily there are easy to use library functions exactly for |
| this purpose. |
| |
| [source,c] |
| ------------------------------------------------------------------------------- |
| void tst_tmpdir(void); |
| ------------------------------------------------------------------------------- |
| |
| Creates a directory with a unique name (based on the test 'TCID') under the |
| LTP temporary directory (defaults to '/tmp/') and changes the test working |
| directory to it. It's usually called from the test 'setup()'. |
| |
| [source,c] |
| ------------------------------------------------------------------------------- |
| void tst_rmdir(void); |
| ------------------------------------------------------------------------------- |
| |
| Removes the directory recursively. It's usually called from test 'cleanup()'. |
| |
| It's important to close all file descriptors (that points to files in test |
| temporary directory, even the unlinked ones) before the test calls |
| 'tst_rmdir()' otherwise the test may break on NFS mounted temp dir (look for |
| "NFS silly rename"). |
| |
| 2.2.4 Safe macros |
| ^^^^^^^^^^^^^^^^^ |
| |
| Safe macros aids to simplify error checking in test preparation. Instead of |
| calling system API functions, checking for their return value and aborting the |
| test if operation has failed you just use corresponding safe macro. |
| |
| Use them whenever it's possible. |
| |
| NOTE: You cannot use safe macros from a child processes because they call test |
| 'tst_' API. |
| |
| Instead of writing: |
| |
| [source,c] |
| ------------------------------------------------------------------------------- |
| fd = open("/dev/null", O_RDONLY); |
| if (fd < 0) |
| tst_brkm(TBROK | TERRNO, cleanup, "opening /dev/null failed"); |
| ------------------------------------------------------------------------------- |
| |
| You write just: |
| |
| [source,c] |
| ------------------------------------------------------------------------------- |
| fd = SAFE_OPEN(cleanup, "/dev/null", O_RDONLY); |
| ------------------------------------------------------------------------------- |
| |
| They can also simplify reading and writing of sysfs files, you can, for |
| example do: |
| |
| [source,c] |
| ------------------------------------------------------------------------------- |
| SAFE_FILE_SCANF(cleanup, "/proc/sys/kernel/pid_max", "%lu", &pid_max); |
| ------------------------------------------------------------------------------- |
| |
| See 'include/safe_macros.h', 'include/safe_stdio.h' and |
| 'include/safe_file_ops.h' for a complete list. |
| |
| NOTE: It's wise NOT to use safe macros in test cleanup(). This is because |
| all safe macros calls tst_brkm() which exits the test immediately which |
| may exit the cleanup() prematurely. (Actually this is hacked around in |
| the test library at the moment so that the cleanup() will finish, but |
| the hack will be removed in the future). |
| |
| 2.2.5 Runtime kernel version detection |
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| |
| [source,c] |
| ------------------------------------------------------------------------------- |
| int tst_kvercmp(int r1, int r2, int r3); |
| |
| struct tst_kern_exv { |
| char *dist_name; |
| char *extra_ver; |
| }; |
| |
| int tst_kvercmp2(int r1, int r2, int r3, struct tst_kern_exv *vers); |
| ------------------------------------------------------------------------------- |
| |
| These two functions are intended for runtime kernel version detection. They |
| parse the output from 'uname()' and compare them with the passed values. |
| |
| The return value is similar to the 'strcmp()' function, i.e. zero means equal, |
| negative value means that the kernel is older than than the expected value and |
| positive means that it's newer. |
| |
| The second function 'tst_kvercmp2()' allows for specifying per-vendor table of |
| kernel versions as vendors typically backport fixes to their kernels and the |
| test may be relevant even if the kernel version does not suggests so. See |
| 'testcases/kernel/syscalls/inotify/inotify04.c' for example usage. |
| |
| 2.2.6 Fork()-ing |
| ^^^^^^^^^^^^^^^^ |
| |
| Be wary that if the test forks and there are were messages printed by tst_* |
| interfaces the data may still be in kernel buffers and these are NOT flushed |
| automatically. |
| |
| This happens when 'stdout' gets redirected to a file. In this case the |
| 'stdout' is not line buffered but block buffered and buffered messages will be |
| printed by the parent and by each of the children. |
| |
| To avoid that you should either call 'tst_flush()' right before the 'fork()' |
| or even better use 'tst_fork()' instead. |
| |
| 2.2.7 Fork() and Parent-child synchronization |
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| |
| As LTP tests are written for Linux most of the test involves fork()-ing and |
| parent-child process synchronization. But first of all: |
| |
| WARNING: Usage of test interface ('tst_resm()', 'tst_brkm()', 'tst_exit()', |
| ...) from child process is forbidden. |
| |
| If you happen to use the interface from a child process the outcome would |
| likely be disastrous. Just to name a few examples: |
| |
| * Imagine the 'cleanup()' is called both from the parent and child, the test |
| will likely fail in 'tst_rmdir()' because the directory was removed |
| already by the child. |
| |
| * The test failures reported via 'tst_resm()' are not propagated from the |
| child to the parent (how could they, it's just a static variable in the |
| test library). And the test will report success even when the test in child |
| process has reported failure. |
| |
| * And many more... |
| |
| Now to the child-parent synchronization. We have a checkpoint library code |
| that works even for two different processes, all they need to is to run with |
| the same working directory (they use FIFO for synchronization). The checkpoint |
| interface provides two pairs or signal and wait functions. One pair to be used |
| to signal child from parent and second to signal parent from child. |
| |
| For the details of the interface look into the 'include/tst_checkpoint.h' and |
| 'lib/tests/tst_checkpoint_*'. |
| |
| There is also an interface that allows parent to wait until child is blocked |
| in kernel (for example waits in 'pause()') see 'include/tst_process_state.h' |
| for more. |
| |
| 2.2.8 Signal handlers |
| ^^^^^^^^^^^^^^^^^^^^^ |
| |
| If you need to use signal handlers keep the code short and simple. Don't |
| forget that the signal handler code runs concurrently to the rest of your code. |
| |
| If you call a non-async-safe function from a signal handler (functions that |
| have global state or locks, such as 'malloc()', 'free()', any operations on |
| 'FILE' and many more...) the test will run fine in most of the cases but will |
| deadlock occasionally. |
| |
| Quite common mistake is to call 'exit(3)' from a signal handler. Note that this |
| function is not signal-async-safe as it flushes buffers, etc. If you need to |
| exit a test immediately from a signal handler use '_exit(2)' instead. |
| |
| TIP: See 'man 7 singnal' for the list of signal-async-safe functions. |
| |
| If signal handler sets a variable it's declaration must be 'volatile' |
| otherwise compiler may misoptimize the code because the variable may not be |
| changed in the code flow analysis. There is 'sig_atomic_t' type defined in C99 |
| but this one DOES NOT imply volatile (it's just a typedef to int). So correct |
| type for a flag that is changed from a signal handler is either of 'volatile |
| int' or 'volatile sig_atomic_t'. |
| |
| 2.2.9 Kernel Modules |
| ^^^^^^^^^^^^^^^^^^^^ |
| |
| There are certain cases where the test needs a kernel part and user space part, |
| happily LTP can build a kernel module and then insert it to the kernel on test |
| start for you. See 'testcases/kernel/device-drivers/block' for details. |
| |
| 2.2.10 Usefull macros |
| ^^^^^^^^^^^^^^^^^^^^^ |
| |
| [source,c] |
| ------------------------------------------------------------------------------- |
| ARRAY_SIZE(arr) |
| ------------------------------------------------------------------------------- |
| |
| Returns the size of statically defined array, i.e. |
| '(sizeof(arr) / sizeof(*arr))' |
| |
| [source,c] |
| ------------------------------------------------------------------------------- |
| LTP_ALIGN(x, a) |
| ------------------------------------------------------------------------------- |
| |
| Aligns the x to be next multiple of a. The a must be power of 2. |
| |
| 2.2.11 Filesystem type detection |
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| |
| Some tests are known to fail on certain filesytems (you cannot swap on TMPFS, |
| there are unimplemented fcntl() etc.). |
| |
| If your test needs to be skipped on certain filesystems use the interface |
| bellow: |
| |
| [source,c] |
| ------------------------------------------------------------------------------- |
| #include "tst_fs_type.h" |
| |
| /* |
| * Unsupported only on NFS. |
| */ |
| if (tst_fs_type(cleanup, ".") == TST_NFS_MAGIC) { |
| tst_brkm(TCONF, cleanup, |
| "Test not supported on NFS filesystem"); |
| } |
| |
| /* |
| * Unsupported on NFS, TMPFS and RAMFS |
| */ |
| long type; |
| |
| switch ((type = tst_fs_type(cleanup, "."))) { |
| case TST_NFS_MAGIC: |
| case TST_TMPFS_MAGIC: |
| case TST_RAMFS_MAGIC: |
| tst_brkm(TCONF, cleanup, "Test not supported on %s filesytem", |
| tst_fs_type_name(type)); |
| break; |
| } |
| ------------------------------------------------------------------------------- |
| |
| 2.2.12 Thread-safety in the LTP library |
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| |
| It is safe to use library 'tst_' functions in multi-threaded tests. |
| Synchronization mechanism is enabled only if the test is linked with pthread |
| library, otherwise no-op stubs (defined in libc) are used. |
| |
| Also, LTP has a helper "TST_DECLARE_ONCE_FN" macro to declare a function which |
| must be run only once (e.g. init or cleanup function), but might be called by |
| multiple threads at the same time. See example below: |
| |
| [source,c] |
| ------------------------------------------------------------------------------- |
| #include "test.h" |
| |
| ... |
| |
| void do_cleanup(void) |
| { |
| /* cleanup code */ |
| ... |
| } |
| TST_DECLARE_ONCE_FN(cleanup, do_cleanup); |
| |
| ... |
| |
| void test01(void) |
| { |
| sfd = socket(AF_INET, SOCK_STREAM, 0); |
| if (sfd == -1) |
| tst_brkm(TBROK, cleanup, "Failed to create a socket"); |
| } |
| ------------------------------------------------------------------------------- |
| |
| 2.2.13 Formatting a device with a filesystem |
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| |
| [source,c] |
| ------------------------------------------------------------------------------- |
| #include "test.h" |
| |
| void tst_mkfs(void (cleanup_fn)(void), const char *dev, |
| const char *fs_type, const char *const fs_opts[]); |
| ------------------------------------------------------------------------------- |
| |
| This function takes a path to a device, filesystem type and an array of extra |
| options passed to mkfs. |
| |
| The extra options 'fs_opts' should either be 'NULL' if there are none or a |
| 'NULL' terminated array of strings such as '{"-b", "1024", NULL}'. |
| |
| 2.3 Writing a testcase in shell |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| LTP supports testcases to be written in a portable shell too. |
| |
| There is a shell library modeled closely to the C interface (the source is |
| located at 'testcases/lib/test.sh') and is installed to the same directory as |
| the rest of the LTP test binaries. |
| |
| WARNING: All identifiers starting with TST_ or tst_ are reserved for the |
| 'test.sh' library. |
| |
| 2.3.1 Basic shell test structure |
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| |
| [source,sh] |
| ------------------------------------------------------------------------------- |
| #!/bin/sh |
| # |
| # This is a basic test for true shell buildin |
| # |
| |
| TCID=true01 |
| TST_TOTAL=1 |
| . test.sh |
| |
| true |
| ret=$? |
| |
| if [ $ret -eq 0 ]; then |
| tst_resm TPASS "true returned 0" |
| else |
| tst_resm TFAIL "true rturned $ret" |
| fi |
| |
| tst_exit |
| ------------------------------------------------------------------------------- |
| |
| TIP: To execute this test the 'test.sh' library must be in '$PATH'. If you are |
| executing the test from a git checkout you can run it as |
| 'PATH="$PATH:../../lib" ./foo01.sh' |
| |
| WARNING: Do not forget to add the 'tst_exit' at the end of the test, |
| otherwise the test return value would be the return value of last |
| executed command. |
| |
| 2.3.2 Basic test interface |
| ^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| |
| Following functions similar to the LTP C intearface are available. |
| |
| * tst_resm() |
| * tst_brkm() |
| * tst_exit() |
| * tst_require_root() |
| * tst_tmpdir() |
| * tst_rmdir() |
| |
| There is one more function called 'tst_check_cmds()' that gets unspecified |
| number of parameters and asserts that each parameter is a name of an |
| executable in '$PATH' and exits the test with 'TCONF' on first missing. |
| |
| 2.3.3 Cleanup |
| ^^^^^^^^^^^^^ |
| |
| Due to differencies in C and shell, the cleanup callback is done using a |
| 'TST_CLEANUP' shell variable that, if not empty, is evaluated before the test |
| exits (either after calling 'tst_exit()' or 'tst_brkm()'). See example below. |
| |
| [source,sh] |
| ------------------------------------------------------------------------------- |
| #!/bin/sh |
| # |
| # Test cleanup example |
| # |
| |
| TCID=true01 |
| TST_TOTAL=1 |
| . test.sh |
| |
| cleanup() |
| { |
| tst_rmdir |
| } |
| |
| tst_tmpdir |
| TST_CLEANUP=cleanup |
| |
| # Do the test here |
| |
| tst_exit |
| ------------------------------------------------------------------------------- |
| |
| 2.3.4 Restarting daemons |
| ^^^^^^^^^^^^^^^^^^^^^^^^ |
| |
| Restarting system daemons is a complicated task for two reasons. |
| |
| * There are different init systems |
| (SysV init, systemd, etc...) |
| |
| * Daemon names are not unified between distributions |
| (apache vs httpd, cron vs crond, various syslog variations) |
| |
| To solve these problems LTP has 'testcases/lib/daemonlib.sh' library that |
| provides functions to start/stop/query daemons as well as variables that store |
| correct daemon name. |
| |
| .Supported operations |
| |============================================================================== |
| | start_daemon() | Starts daemon, name is passed as first parameter. |
| | stop_daemon() | Stops daemon, name is passed as first parameter. |
| | restart_daemon() | Restarts daemon, name is passed as first parameter. |
| | status_daemon() | Returns daemon status, TODO: what is return value? |
| |============================================================================== |
| |
| .Variables with detected names |
| |============================================================================== |
| | CROND_DAEMON | Cron daemon name (cron, crond). |
| | SYSLOG_DAEMON | Syslog daemon name (syslog, syslog-ng, rsyslog). |
| |============================================================================== |
| |
| .Cron daemon restart example |
| [source,sh] |
| ------------------------------------------------------------------------------- |
| #!/bin/sh |
| # |
| # Cron daemon restart example |
| # |
| TCID=cron01 |
| TST_COUNT=1 |
| . test.sh |
| . daemonlib.sh |
| |
| ... |
| |
| restart_daemon $CROND_DAEMON |
| |
| ... |
| |
| tst_exit |
| ------------------------------------------------------------------------------- |
| |
| |
| 3. Common problems |
| ------------------ |
| |
| This chapter describes common problems/misuses and less obvious desing patters |
| (quirks) in UNIX interfaces. Read it carefully :) |
| |
| 3.1 umask() |
| ~~~~~~~~~~~ |
| |
| I've been hit by this one several times already... When you are creating files |
| with 'open()' or 'creat()' etc, the mode specified as the last parameter *is |
| not* the mode the file is created with. The mode depends on current 'umask()' |
| settings which may clear some of the bits. If your test depends on specific |
| file permissions you need to 'chmod()' the file afterwards or use SAFE_TOUCH() |
| that does the 'chmod()' for you. |
| |
| |
| 4. Test Contribution Checklist |
| ------------------------------ |
| |
| 1. Test compiles and runs fine (check with -i 10 too) |
| 2. Checkpatch does not report any errors |
| 3. The runtest entires are in place |
| 4. Test files are added into corresponding .gitignore files |
| 5. Patches apply over the latest git |
| |
| |
| 4.1 About .gitignore files |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| There are numerous '.gitignore' files in the LTP tree. Usually there is a |
| '.gitignore' file per a group of tests. The reason for this setup is simple. |
| It's easier to maintain a '.gitignore' file per directory with tests rather |
| than having single file in the project root directory. This way we don't have |
| to update them when moving directories and they gets deleted automatically |
| when a directory with tests is removed. |