Linux Test Project
(Maintained by: Subrata Modak<subrata@linux.vnet.ibm.com>)

The latest image is always available at http://ltp.sf.net


About the Linux Test Project

The Linux Test Project is a joint project with SGI, IBM, OSDL, and Bull with a
goal to deliver test suites to the open source community that validate the
reliability, robustness, and stability of Linux. The Linux Test Project is a
collection of tools for testing the Linux kernel and related features. Our goal
is to improve the Linux kernel by bringing test automation to the kernel testing
effort. Interested open source contributors are encouraged to join the project.


In the Package
--------------
Here is a short description of what is included in this package

INSTALL
	Installation documentation and quick start guide

README
	This document

CREDITS
	Credits to those who have contributed to the project.

COPYING
		 GNU General Public License

ChangeLog
	An incomplete list of changes to the project.  We will try harder to
	keep this up to date.

Makefile
		 Top level make file for LTP.

runalltests.sh
	A simple script to run all of the packaged tests in sequential order and
		 report the overall result.

diskio.sh
	A simple script to run all the diskio tests that pertain to the CD-ROM
	and floppy disk file systems.

networktests.sh
	A simple script to run all the network related tests that require
	another remote machine.

doc/*
	Documentation for the project including man pages for some of the
	tools & library functions, as well as descriptions of every test.


include/*
lib/*
	The include and lib directories contain headers and codes for common
	routines used by many of the tests.  We have tried to keep this to a
		 minimum.

testcases/*
	Contains all tests that run under the LTP as well as the "bin"
		 directory, which has hardlinks to all the test executables.

runtest/*
	It houses command lists that are used by ltp-pan for automated testing.

pan/*
	The pan directory contains a simple, lightweight test harness.  lpt-pan
	has the ability to run tests randomly and in parallel.  See ltp-pan's man
	page for more information.

scratch/*
	The scratch directory is a dumping ground for snippets and test cases
	that we found but haven't integrated into LTP yet.  Feel free to suggest
	new snippets to be added, or take one of these and finish it off.

testcases/kernel/containers/*
	Contains all the testcases related to container functionality ( e.g: sysvipc, utsname etc..)
	For further information on the each of the container functionality refer
	the following file: testcases/kernel/containers/README


Warning!
--------
Be careful with these tests!

Don't run them on production systems.  Growfiles, doio, and iogen in particular
stress the I/O capabilities of systems and while they should not cause problems
on properly functioning systems, they are intended to find (or cause) problems.


Contact Information and Updates
-------------------------------

URL: http://ltp.sf.net

Questions and comments should be sent to the LTP mailing list located on the
project page.

Using KDUMP test automation suite
---------------------------------
For more information on how to use the kdump test automation suite please refer
to file testcases/kdump/README.

Using NUMA test automation suite
---------------------------------
For more information on how to use the 'numa' test automation suite please refer
to file testcases/kernel/numa/README.

---------------------------------
Enable CODE COVERAGE for your Kernel:
---------------------------------
1) Apply kernel-gcov patch(s) against the corresponding Kernel. They are available here:
http://ltp.cvs.sourceforge.net/viewvc/ltp/utils/analysis/gcov-kernel/linux-2.*.*-gcov.patch?view=log,
2) Also enable the following options in your .config file before building the kernel
CONFIG_GCOV_PROFILE=y
CONFIG_GCOV_ALL=y
CONFIG_GCOV_PROC=m
CONFIG_GCOV_HAMMER=y

---------------------------------
Enabling Kernel Configuration to test Containers/Namespaces
---------------------------------
CONFIG_NAMESPACES=y
CONFIG_UTS_NS=y
CONFIG_IPC_NS=y
CONFIG_USER_NS=y
CONFIG_PID_NS=y
CONFIG_NET_NS=y
CONFIG_VETH=y
CONFIG_MACVLAN=y

The IPC namespaces do not automatically enable IPC, so you may
also want to have:

CONFIG_SYSVIPC=y
CONFIG_SYSVIPC_SYSCTL=y
CONFIG_POSIX_MQUEUE=y

---------------------------------
Enabling Kernel Configuration to test Controllers
---------------------------------
CONFIG_CGROUPS=y
CONFIG_CGROUP_DEBUG=y
CONFIG_CGROUP_NS=y
CONFIG_GROUP_SCHED=y
CONFIG_FAIR_GROUP_SCHED=y
CONFIG_RT_GROUP_SCHED=y
CONFIG_CGROUP_SCHED=y
CONFIG_CGROUP_MEM_RES_CTLR=y
CONFIG_LOCKDEP=y
---------------------------------
Enabling Kernel Configuration to test Power Management features
---------------------------------
CONFIG_CPU_FREQ=y
CONFIG_CPU_FREQ_TABLE=y
CONFIG_CPU_FREQ_DEBUG=y
CONFIG_CPU_FREQ_STAT=y
CONFIG_CPU_FREQ_STAT_DETAILS=y
CONFIG_CPU_IDLE=y
CONFIG_CPU_IDLE_GOV_LADDER=y
CONFIG_CPU_IDLE_GOV_MENU=y
CONFIG_SCHED_MC=y
CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE=y
CONFIG_CPU_FREQ_DEFAULT_GOV_POWERSAVE=y
CONFIG_CPU_FREQ_DEFAULT_GOV_USERSPACE=y
CONFIG_CPU_FREQ_DEFAULT_GOV_ONDEMAND=y
CONFIG_CPU_FREQ_DEFAULT_GOV_CONSERVATIVE=y
CONFIG_CPU_FREQ_GOV_PERFORMANCE=y
CONFIG_CPU_FREQ_GOV_POWERSAVE=y
CONFIG_CPU_FREQ_GOV_USERSPACE=y
CONFIG_CPU_FREQ_GOV_ONDEMAND=y
CONFIG_CPU_FREQ_GOV_CONSERVATIVE=y
---------------------------------
Enabling Kernel Configuration to test filecaps security feature
---------------------------------
CONFIG_SECURITY_FILE_CAPABILITIES=y
---------------------------------
Enabling Kernel Configuration to test p9auth security feature
---------------------------------
CONFIG_CRYPTO=y
CONFIG_STAGING=y
# CONFIG_STAGING_EXCLUDE_BUILD is not set
CONFIG_PLAN9AUTH=m
(Or CONFIG_PLAN9AUTH=y)
You also will need openssl installed.
You will also need to have an user named 'ltp'
created to execute this test.
---------------------------------
Enabling Kernel Configuration to test SELinux security feature
---------------------------------
Your Kernel should have been built with the following options to
test SELinux:

CONFIG_SECURITY=y
CONFIG_SECURITY_NETWORK=y
CONFIG_SECURITY_NETWORK_XFRM=y

CONFIG_SECURITY_DEFAULT_MMAP_MIN_ADDR=0
This has to be set to a positive value if you want to test this check.
Fedora kernels set it to 65536.

CONFIG_SECURITY_SELINUX=y
CONFIG_SECURITY_SELINUX_BOOTPARAM=y
CONFIG_SECURITY_SELINUX_BOOTPARAM_VALUE=1
CONFIG_SECURITY_SELINUX_DEVELOP=y
CONFIG_SECURITY_SELINUX_AVC_STATS=y
CONFIG_SECURITY_SELINUX_CHECKREQPROT_VALUE=1
CONFIG_SECURITY_SELINUX_ENABLE_SECMARK_DEFAULT=y

CONFIG_SECURITY_SELINUX_POLICYDB_VERSION_MAX=y
You don't want this one unless you are running Fedora 3 or 4.  
On anything newer, it will cause unnecessary policy expansion.

CONFIG_SECURITY_SMACK=y
CONFIG_SECURITY_SELINUX=y

By default, if you boot with multiple LSMs compiled into the kernel, the
kernel won't boot succesfully - there can be only one (aside from
explicit internal "stacking" e.g. as is done for combining SELinux or
Smack with capabilities).  Unless you use the security= option to select
one at boot.  SELinux and Smack will honor the security= option.
---------------------------------
---------------------------------
Enabling Kernel Configuration to test SMACK security feature
---------------------------------
CONFIG_NETLABEL=y
CONFIG_SECURITY=y
CONFIG_SECURITY_NETWORK=y
CONFIG_SECURITY_SMACK=y
CONFIG_SECURITY_SELINUX should not be set

For more information to build/install/run these tests, look through:
ltp/testcases/kernel/security/smack/README
---------------------------------
---------------------------------
Enablement for Enhancement to kexec/kdump for implementing the following features:
- Backup/restore memory used by the original kernel before/after kexec.
- Save/restore CPU state before/after kexec.
Now, only the i386 architecture is supported. More from the following git logs:
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=3ab83521378268044a448113c6aa9a9e245f4d2f,
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=89081d17f7bb81d89fa1aa9b70f821c5cf4d39e9,
---------------------------------
CONFIG_X86_32=y
CONFIG_RELOCATABLE=y
CONFIG_KEXEC=y
CONFIG_CRASH_DUMP=y
CONFIG_PM=y
CONFIG_HIBERNATION=y
CONFIG_KEXEC_JUMP=y
---------------------------------
Enabling HOTPLUG for your Kernels
---------------------------------
CONFIG_HOTPLUG=y
CONFIG_HOTPLUG_CPU=y
CONFIG_ARCH_ENABLE_MEMORY_HOTPLUG=y
CONFIG_ACPI_HOTPLUG_CPU=y
CONFIG_HOTPLUG_PCI_PCIE=y
CONFIG_HOTPLUG_PCI=y
CONFIG_HOTPLUG_PCI_FAKE=y
CONFIG_HOTPLUG_PCI_COMPAQ=y
CONFIG_HOTPLUG_PCI_COMPAQ_NVRAM=y
CONFIG_HOTPLUG_PCI_IBM=y
CONFIG_HOTPLUG_PCI_ACPI=y
CONFIG_HOTPLUG_PCI_ACPI_IBM=y
CONFIG_HOTPLUG_PCI_CPCI=y
CONFIG_HOTPLUG_PCI_CPCI_ZT5550=y
CONFIG_HOTPLUG_PCI_CPCI_GENERIC=y
CONFIG_HOTPLUG_PCI_SHPC=y
---------------------------------
Video For Linux Two API testing Requirements:
You need to have a video device (i.e. webcam, tuner card, etc.) connected to your system and available under /dev/video0. If you don't have any hardware device available, you can still test the "Virtual Video Driver". To compile this you need to compile your kernel with CONFIG_VIDEO_VIVI=m under:
  -> Device Drivers
    -> Multimedia devices
      -> Video For Linux
        -> Video capture adapters
         -> Virtual Video Driver

The tests also require CUnit Framework to be installed before compiling the tests. Download & Install the same from:
http://sourceforge.net/projects/cunit/
---------------------------------
---------------------------------
Native language support (nls) testsuite requirements
----------------------------------------------------
CONFIG_NLS=m
CONFIG_BLOCK=y
---------------------------------
Enabling Controller area network (CAN) protocol support for your Kernels
---------------------------------
CONFIG_NET=y
CONFIG_CAN=m
CONFIG_CAN_RAW=m
CONFIG_CAN_BCM=m
# CAN Device Drivers
CONFIG_CAN_VCAN=m
---------------------------------
Enabling Fault Injection Support for your kernel (version 2.6.29).
Please check with the original kernel for the fault injection
types it supports. Following supports will be available:

/sys/kernel/debug/fail_io_timeout/interval
/sys/kernel/debug/fail_io_timeout/probability
/sys/kernel/debug/fail_io_timeout/reject-end
/sys/kernel/debug/fail_io_timeout/reject-start
/sys/kernel/debug/fail_io_timeout/require-end
/sys/kernel/debug/fail_io_timeout/require-start
/sys/kernel/debug/fail_io_timeout/space
/sys/kernel/debug/fail_io_timeout/stacktrace-depth
/sys/kernel/debug/fail_io_timeout/task-filter
/sys/kernel/debug/fail_io_timeout/times
/sys/kernel/debug/fail_io_timeout/verbose

/sys/kernel/debug/fail_make_request/interval
/sys/kernel/debug/fail_make_request/probability
/sys/kernel/debug/fail_make_request/reject-end
/sys/kernel/debug/fail_make_request/reject-start
/sys/kernel/debug/fail_make_request/require-end
/sys/kernel/debug/fail_make_request/require-start
/sys/kernel/debug/fail_make_request/space
/sys/kernel/debug/fail_make_request/stacktrace-depth
/sys/kernel/debug/fail_make_request/task-filter
/sys/kernel/debug/fail_make_request/times
/sys/kernel/debug/fail_make_request/verbose

/sys/kernel/debug/fail_page_alloc/ignore-gfp-highmem
/sys/kernel/debug/fail_page_alloc/ignore-gfp-wait
/sys/kernel/debug/fail_page_alloc/interval
/sys/kernel/debug/fail_page_alloc/min-order
/sys/kernel/debug/fail_page_alloc/probability
/sys/kernel/debug/fail_page_alloc/reject-end
/sys/kernel/debug/fail_page_alloc/reject-start
/sys/kernel/debug/fail_page_alloc/require-end
/sys/kernel/debug/fail_page_alloc/require-start
/sys/kernel/debug/fail_page_alloc/space
/sys/kernel/debug/fail_page_alloc/stacktrace-depth
/sys/kernel/debug/fail_page_alloc/task-filter
/sys/kernel/debug/fail_page_alloc/times
/sys/kernel/debug/fail_page_alloc/verbose

/sys/kernel/debug/failslab/ignore-gfp-wait
/sys/kernel/debug/failslab/interval
/sys/kernel/debug/failslab/probability
/sys/kernel/debug/failslab/reject-end
/sys/kernel/debug/failslab/reject-start
/sys/kernel/debug/failslab/require-end
/sys/kernel/debug/failslab/require-start
/sys/kernel/debug/failslab/space
/sys/kernel/debug/failslab/stacktrace-depth
/sys/kernel/debug/failslab/task-filter
/sys/kernel/debug/failslab/times
/sys/kernel/debug/failslab/verbose

when the below kernel config options are set:

CONFIG_FAULT_INJECTION=y
CONFIG_DEBUG_KERNEL=y
CONFIG_FAILSLAB=y (Fault-injection capability for kmalloc)
(CONFIG_SLAB=y || CONFIG_SLUB=y) if CONFIG_FAILSLAB=y
CONFIG_FAIL_PAGE_ALLOC=y (Fault-injection capabilitiy for alloc_pages())
CONFIG_FAIL_MAKE_REQUEST=y (Fault-injection capability for disk IO)
CONFIG_BLOCK=y if CONFIG_FAIL_MAKE_REQUEST=y
CONFIG_FAIL_IO_TIMEOUT=y (Faul-injection capability for faking disk interrupts)
CONFIG_BLOCK=y if CONFIG_FAIL_IO_TIMEOUT=y
CONFIG_FAULT_INJECTION_DEBUG_FS=y (Debugfs entries for fault-injection capabilities)
(CONFIG_SYSFS=y && CONFIG_DEBUG_FS=y) if CONFIG_FAULT_INJECTION_DEBUG_FS=y
CONFIG_FAULT_INJECTION_STACKTRACE_FILTER=y (stacktrace filter for fault-injection capabilities)
(CONFIG_FAULT_INJECTION_DEBUG_FS=y && CONFIG_STACKTRACE_SUPPORT=y && !CONFIG_X86_64) if
	CONFIG_FAULT_INJECTION_STACKTRACE_FILTER=y

For more information on Fault injection, please refer to:
linux-2.6/Documentation/fault-injection/fault-injection.txt,

You should also have made the following entries in your /etc/fstab file
once the kernel is booted with the above CONFIG options set:

debugfs		/sys/kernel/debug/		debugfs

# How the Kernel Fault Injection works for LTP ?

1) Build Kernel with all the above possible kernel CONFIG Options,
2) Create the above entry in /etc/fstab file,
3) Reboot in the new kernel,
4) Goto LTPROOT. Build and Install LTP as per ltp/INSTALL file,
5) Choose your own test(or default) to run with fault injection as follows:
	./runltp -f <command_file> -F <LOOPS>,<FAULT_PROBABILITY>

The agorithm will work as:
loop (for each testcase)
begin
	execute_testcase(inside_stable_kernel)
	begin
		insert_fault_into_kernel()
		loop X Times
		begin
			execute_testcase(inside_fault_kernel)
		end
		restore_kernel_to_normal()
	end
end

# Eternal TODOs:
1) Add as many framework as they get added/modifed in the kernel
---------------------------------

---------------------------------
Enabling Kernel Configuration to test ext4 new features
---------------------------------
CONFIG_EXT4_FS=y
CONFIG_EXT4DEV_COMPAT=y
CONFIG_EXT4_FS_XATTR=y
CONFIG_EXT4_FS_POSIX_ACL=y
CONFIG_EXT4_FS_SECURITY=y

Beside that, the following packages are necessary.
  e2fsprogs
  e2fsprogs-devel
  e2fsprogs-libs
And the version of packages must be 1.41.4 or above.

For more information to build/install/run these tests, look through:
ltp/testcases/kernel/fs/ext4-new-features/README
---------------------------------