blob: 084843c5fd4026f3617f50d100d7759372b14712 [file] [log] [blame]
Copyright (c) 2012 The Chromium OS Authors. All rights reserved.
Use of this source code is governed by a BSD-style license that can be
found in the LICENSE file.
Intro
-----
We test shill using two sets of tests: unit tests, and integration
tests. The unit tests are built using Google Test [1] and Google Mock
[2]; the integration tests use autotest [3].
Running unit tests
------------------
- build the shill_unittest target
- run the resulting shill_unittest binary
- if you're using our ebuild scripts, you can do both by running
(chroot)$ cros_run_unit_tests -p shill
- run the unit tests from your host machine under gdb
(chroot)$ FEATURES="test noclean" emerge-x86-generic shill
(chroot)$ gdb_x86_local --board x86-generic \
/build/x86-generic/tmp/portage/chromeos-base/shill-9999/work/shill-9999/shill_unittest
(Of course if the unit tests segfaulted, you wouldn't need the emerge
step since the build directory would have been retained in the course
of the test failing.)
- for an incremental workflow, you can setup like this:
(chroot)$ cros_workon --board x86-generic shill
- incrementally build like this (note: this may not work if your
host architecture does not match that of the target):
(chroot)$ CXXFLAGS=-g cros_workon_make --board x86-generic --reconf --test shill
- and then debug tests like this:
(chroot)$ gdb_x86_local --board x86-generic /path/to/checkout/src/platform/shill/shill_unittest
- you can specify arguments to the unit tests like this:
- full workflow:
(chroot)$ GTEST_ARGS="--v=1000 --gtest_filter=WiFiPropertyTest.*" \
FEATURES="test" emerge-x86-generic shill
- incremenetal workflow:
(chroot)$ GTEST_ARGS="--v=1000 --gtest_filter=WiFiPropertyTest.*" \
CXXFLAGS=-g cros_workon_make --board x86-generic --reconf --test shill
Running integration tests
-------------------------
- build a test image, suitable for a VM:
(chroot) src/scripts$ ./build_packages --board=x86-generic
(chroot) src/scripts$ ./build_image --board=x86-generic \
--noenable_rootfs_verification test
(chroot) src/scripts$ ./image_to_vm.sh --board=x86-generic --test_image
- start the VM
(host)$ sudo kvm -m 1024 -vga std -pidfile /tmp/kvm.pid \
-net nic,model=virtio -net user,hostfwd=tcp::9222-:22 \
-hda <path to chroot>/src/build/images/x86-generic/latest/chromiumos_qemu_image.bin
- open a root shell on the VM console
- in VM console, log in with any legal account (but not Guest)
- hit C-A-t to open a shell (use right control and alt, to avoid escaping
out of VM's mouse capture)
crosh> shell
chronos@localhost / $ sudo bash
- if you've modified the source after building the image, load shill onto VM image:
(chroot) src/scripts$ ./start_devserver
localhost / # FEATURES="nostrip" gmerge shill
- stop flimflam, and start shill
localhost / # stop flimflam
localhost / # shill --foreground --v=1000 \
--device-black-list=managed0,managed1
(the black list keep shill from configuring the devices used by the hostapd
processes.)
- run the tests
(chroot) src/scripts$ ./run_remote_tests.sh --board=x86-generic \
--remote=127.0.0.1 --ssh_port 9222 \
--args="config_file=wifi_vm_config" WiFiManager
To run a specific test out of the test suite, use test_pat option to --args.
# Example: To just run the 035CheckWEPKeySyntax test:
(chroot) src/scripts$ ./run_remote_tests.sh --board=x86-generic \
--remote=127.0.0.1 --ssh_port 9222 \
--args="config_file=wifi_vm_config test_pat=035CheckWEPKeySyntax" WiFiManager
- configuration note: if you use a different port
(e.g. hostfwd=tcp::9223-:22), you'll need to change:
- the ssh_port argument to run_remote_tests
- the port numbers in
<chroot>/third_party/autotest/files/client/config/wifi_vm_config
- debugging test failures
- "grep shill /var/log/messages" for log messages
- "grep wpa_supplicant /var/log/messages" for supplicant log messages
- "wpa_debug debug" to increase supplicant log level
- try resetting the test infrastructure
- rmmod mac80211_hwsim mac80211 cfg80211
- restart wpasupplicant
- rm /tmp/hostapd-test.control/*
- examine autotest log files
- check how far the test got before it failed
$ grep -a ': step ' <test output>/<suite name>/<suite name>.<test name>/debug/<suite name>.<test name>.INFO
e.g.
(chroot) $ grep -a ': step ' /tmp/run_remote_tests.abcd/network_WiFiRoaming/network_WiFiRoaming.002Suspend/debug/network_WiFiRoaming.002Suspend.INFO
- read the log file
(chroot) $ LESSOPEN= less /tmp/run_remote_tests.abcd/network_WiFiRoaming/network_WiFiRoaming.002Suspend/debug/network_WiFiRoaming.002Suspend.INFO
(LESSOPEN= prevents less from misinterpreting the logs as binary files,
and piping them through hexdump.)
- additional test suites: we have a number of other WiFi test suites
(in addition to WiFiManager). these are: WiFiMatFunc, WiFiPerf,
WiFiRoaming, WiFiSecMat. the WiFiPerf tests are probably not too
relevant to shill (just yet), but the rest probably are.
[1] http://code.google.com/p/googletest/
[2] http://code.google.com/p/googlemock/
[3] http://autotest.kernel.org/,
http://www.chromium.org/chromium-os/testing/testing-faq