mukesh agrawal | 8a3188d | 2011-12-01 20:56:44 +0000 | [diff] [blame] | 1 | Copyright (c) 2012 The Chromium OS Authors. All rights reserved. |
mukesh agrawal | 70d3ebe | 2011-10-13 14:45:54 -0700 | [diff] [blame] | 2 | Use of this source code is governed by a BSD-style license that can be |
| 3 | found in the LICENSE file. |
| 4 | |
| 5 | Intro |
| 6 | ----- |
| 7 | We test shill using two sets of tests: unit tests, and integration |
| 8 | tests. The unit tests are built using Google Test [1] and Google Mock |
| 9 | [2]; the integration tests use autotest [3]. |
| 10 | |
| 11 | Running unit tests |
| 12 | ------------------ |
| 13 | - build the shill_unittest target |
| 14 | - run the resulting shill_unittest binary |
| 15 | - if you're using our ebuild scripts, you can do both by running |
| 16 | (chroot)$ FEATURES="test" emerge-x86-generic shill |
Paul Stewart | 85a1bad | 2011-10-21 22:36:38 -0700 | [diff] [blame] | 17 | - run the unit tests from your host machine under gdb |
| 18 | (chroot)$ FEATURES="test noclean" emerge-x86-generic shill |
| 19 | (chroot)$ gdb_x86_local --board x86-generic \ |
| 20 | /build/x86-generic/tmp/portage/chromeos-base/shill-9999/work/shill-9999/shill_unittest |
| 21 | (Of course if the unit tests segfaulted, you wouldn't need the emerge |
| 22 | step since the build directory would have been retained in the course |
| 23 | of the test failing.) |
mukesh agrawal | 70d3ebe | 2011-10-13 14:45:54 -0700 | [diff] [blame] | 24 | |
Chris Masone | 5756a36 | 2011-10-24 10:34:10 -0700 | [diff] [blame] | 25 | - for an incremental workflow, you can setup like this: |
| 26 | (chroot)$ cros_workon --board x86-generic shill |
| 27 | - incrementally build like this: |
| 28 | (chroot)$ CXXFLAGS=-g cros_workon_make --board x86-generic --reconf --test shill |
| 29 | - and then debug tests like this: |
| 30 | (chroot)$ gdb_x86_local --board x86-generic /path/to/checkout/src/platform/shill/shill_unittest |
| 31 | |
mukesh agrawal | 1590839 | 2011-11-16 18:29:25 +0000 | [diff] [blame] | 32 | - you can specify arguments to the unit tests like this: |
| 33 | - full workflow: |
| 34 | (chroot)$ GTEST_ARGS="--v=1000 --gtest_filter=WiFiPropertyTest.*" \ |
| 35 | FEATURES="test" emerge-x86-generic shill |
| 36 | - incremenetal workflow: |
| 37 | (chroot)$ GTEST_ARGS="--v=1000 --gtest_filter=WiFiPropertyTest.*" \ |
| 38 | CXXFLAGS=-g cros_workon_make --board x86-generic --reconf --test shill |
| 39 | |
mukesh agrawal | 70d3ebe | 2011-10-13 14:45:54 -0700 | [diff] [blame] | 40 | Running integration tests |
| 41 | ------------------------- |
| 42 | - build a test image, suitable for a VM: |
| 43 | (chroot) src/scripts$ ./build_packages --board=x86-generic |
| 44 | (chroot) src/scripts$ ./build_image --board=x86-generic --withdev \ |
| 45 | --test --noenable_rootfs_verification |
| 46 | (chroot) src/scripts$ ./image_to_vm.sh --board=x86-generic --test_image |
| 47 | |
| 48 | - start the VM |
| 49 | (host)$ sudo kvm -m 1024 -vga std -pidfile /tmp/kvm.pid \ |
| 50 | -net nic,model=virtio -net user,hostfwd=tcp::9222-:22 \ |
| 51 | -hda <path to chroot>/src/build/images/x86-generic/latest/chromiumos_qemu_image.bin |
| 52 | |
| 53 | - open a root shell on the VM console |
| 54 | - in VM console, log in as guest |
| 55 | - hit C-A-t to open a shell (use right control and alt, to avoid escaping |
| 56 | out of VM's mouse capture) |
| 57 | |
| 58 | crosh> shell |
| 59 | chronos@localhost / $ sudo bash |
| 60 | |
| 61 | - load shill onto VM image |
| 62 | localhost / # FEATURES="nostrip" gmerge shill |
| 63 | |
| 64 | - stop flimflam, and start shill |
| 65 | localhost / # stop flimflam |
Gaurav Shah | 8558710 | 2011-11-29 10:34:22 -0800 | [diff] [blame] | 66 | localhost / # shill --foreground --v=1000 \ |
mukesh agrawal | 8a3188d | 2011-12-01 20:56:44 +0000 | [diff] [blame] | 67 | --device-black-list=managed0,managed1 |
mukesh agrawal | 70d3ebe | 2011-10-13 14:45:54 -0700 | [diff] [blame] | 68 | (the black list keep shill from configuring the devices used by the hostapd |
| 69 | processes.) |
| 70 | |
| 71 | - run the tests |
| 72 | (chroot) src/scripts$ ./run_remote_tests.sh --board=x86-generic \ |
| 73 | --remote=127.0.0.1 --ssh_port 9222 \ |
| 74 | --args="config_file=wifi_vm_config" WiFiManager |
| 75 | |
Gaurav Shah | d52caa5 | 2011-10-25 10:46:03 -0700 | [diff] [blame] | 76 | To run a specific test out of the test suite, use test_pat option to --args. |
| 77 | |
| 78 | # Example: To just run the 035CheckWEPKeySyntax test: |
| 79 | (chroot) src/scripts$ ./run_remote_tests.sh --board=x86-generic \ |
| 80 | --remote=127.0.0.1 --ssh_port 9222 \ |
| 81 | --args="config_file=wifi_vm_config test_pat=035CheckWEPKeySyntax" WiFiManager |
| 82 | |
mukesh agrawal | 70d3ebe | 2011-10-13 14:45:54 -0700 | [diff] [blame] | 83 | - configuration note: if you use a different port |
| 84 | (e.g. hostfwd=tcp::9223-:22), you'll need to change: |
| 85 | - the ssh_port argument to run_remote_tests |
| 86 | - the port numbers in |
| 87 | <chroot>/third_party/autotest/files/client/config/wifi_vm_config |
| 88 | |
| 89 | - debugging test failures |
| 90 | - "grep shill /var/log/messages" for log messages |
| 91 | - "grep wpa_supplicant /var/log/messages" for supplicant log messages |
| 92 | - "wpa_debug debug" to increase supplicant log level |
| 93 | - try resetting the test infrastructure |
| 94 | - rmmod mac80211_hwsim mac80211 cfg80211 |
| 95 | - restart wpasupplicant |
| 96 | - rm /tmp/hostapd-test.control/* |
mukesh agrawal | c2f39fc | 2011-11-07 22:07:23 +0000 | [diff] [blame] | 97 | - examine autotest log files |
| 98 | - check how far the test got before it failed |
| 99 | $ grep -a ': step ' <test output>/<suite name>/<suite name>.<test name>/debug/<suite name>.<test name>.INFO |
| 100 | e.g. |
| 101 | (chroot) $ grep -a ': step ' /tmp/run_remote_tests.abcd/network_WiFiRoaming/network_WiFiRoaming.002Suspend/debug/network_WiFiRoaming.002Suspend.INFO |
| 102 | - read the log file |
| 103 | (chroot) $ LESSOPEN= less /tmp/run_remote_tests.abcd/network_WiFiRoaming/network_WiFiRoaming.002Suspend/debug/network_WiFiRoaming.002Suspend.INFO |
| 104 | |
| 105 | (LESSOPEN= prevents less from misinterpreting the logs as binary files, |
| 106 | and piping them through hexdump.) |
mukesh agrawal | 70d3ebe | 2011-10-13 14:45:54 -0700 | [diff] [blame] | 107 | |
| 108 | - additional test suites: we have a number of other WiFi test suites |
| 109 | (in addition to WiFiManager). these are: WiFiMatFunc, WiFiPerf, |
| 110 | WiFiRoaming, WiFiSecMat. the WiFiPerf tests are probably not too |
| 111 | relevant to shill (just yet), but the rest probably are. |
| 112 | |
| 113 | [1] http://code.google.com/p/googletest/ |
| 114 | [2] http://code.google.com/p/googlemock/ |
| 115 | [3] http://autotest.kernel.org/, |
Gaurav Shah | d52caa5 | 2011-10-25 10:46:03 -0700 | [diff] [blame] | 116 | http://www.chromium.org/chromium-os/testing/testing-faq |