mukesh agrawal | 8a3188d | 2011-12-01 20:56:44 +0000 | [diff] [blame] | 1 | Copyright (c) 2012 The Chromium OS Authors. All rights reserved. |
mukesh agrawal | 70d3ebe | 2011-10-13 14:45:54 -0700 | [diff] [blame] | 2 | Use of this source code is governed by a BSD-style license that can be |
| 3 | found in the LICENSE file. |
| 4 | |
| 5 | Intro |
| 6 | ----- |
| 7 | We test shill using two sets of tests: unit tests, and integration |
| 8 | tests. The unit tests are built using Google Test [1] and Google Mock |
| 9 | [2]; the integration tests use autotest [3]. |
| 10 | |
| 11 | Running unit tests |
| 12 | ------------------ |
| 13 | - build the shill_unittest target |
| 14 | - run the resulting shill_unittest binary |
| 15 | - if you're using our ebuild scripts, you can do both by running |
Paul Stewart | 739742a | 2012-07-13 19:25:19 -0700 | [diff] [blame] | 16 | (chroot)$ cros_run_unit_tests -p shill |
Paul Stewart | 85a1bad | 2011-10-21 22:36:38 -0700 | [diff] [blame] | 17 | - run the unit tests from your host machine under gdb |
| 18 | (chroot)$ FEATURES="test noclean" emerge-x86-generic shill |
| 19 | (chroot)$ gdb_x86_local --board x86-generic \ |
| 20 | /build/x86-generic/tmp/portage/chromeos-base/shill-9999/work/shill-9999/shill_unittest |
| 21 | (Of course if the unit tests segfaulted, you wouldn't need the emerge |
| 22 | step since the build directory would have been retained in the course |
| 23 | of the test failing.) |
mukesh agrawal | 70d3ebe | 2011-10-13 14:45:54 -0700 | [diff] [blame] | 24 | |
Chris Masone | 5756a36 | 2011-10-24 10:34:10 -0700 | [diff] [blame] | 25 | - for an incremental workflow, you can setup like this: |
| 26 | (chroot)$ cros_workon --board x86-generic shill |
Paul Stewart | 739742a | 2012-07-13 19:25:19 -0700 | [diff] [blame] | 27 | - incrementally build like this (note: this may not work if your |
| 28 | host architecture does not match that of the target): |
Chris Masone | 5756a36 | 2011-10-24 10:34:10 -0700 | [diff] [blame] | 29 | (chroot)$ CXXFLAGS=-g cros_workon_make --board x86-generic --reconf --test shill |
| 30 | - and then debug tests like this: |
| 31 | (chroot)$ gdb_x86_local --board x86-generic /path/to/checkout/src/platform/shill/shill_unittest |
| 32 | |
mukesh agrawal | 1590839 | 2011-11-16 18:29:25 +0000 | [diff] [blame] | 33 | - you can specify arguments to the unit tests like this: |
| 34 | - full workflow: |
| 35 | (chroot)$ GTEST_ARGS="--v=1000 --gtest_filter=WiFiPropertyTest.*" \ |
| 36 | FEATURES="test" emerge-x86-generic shill |
| 37 | - incremenetal workflow: |
| 38 | (chroot)$ GTEST_ARGS="--v=1000 --gtest_filter=WiFiPropertyTest.*" \ |
| 39 | CXXFLAGS=-g cros_workon_make --board x86-generic --reconf --test shill |
| 40 | |
mukesh agrawal | 70d3ebe | 2011-10-13 14:45:54 -0700 | [diff] [blame] | 41 | Running integration tests |
| 42 | ------------------------- |
| 43 | - build a test image, suitable for a VM: |
| 44 | (chroot) src/scripts$ ./build_packages --board=x86-generic |
Wade Guthrie | 8624e43 | 2012-04-11 11:12:00 -0700 | [diff] [blame] | 45 | (chroot) src/scripts$ ./build_image --board=x86-generic \ |
| 46 | --noenable_rootfs_verification test |
mukesh agrawal | 70d3ebe | 2011-10-13 14:45:54 -0700 | [diff] [blame] | 47 | (chroot) src/scripts$ ./image_to_vm.sh --board=x86-generic --test_image |
| 48 | |
| 49 | - start the VM |
| 50 | (host)$ sudo kvm -m 1024 -vga std -pidfile /tmp/kvm.pid \ |
| 51 | -net nic,model=virtio -net user,hostfwd=tcp::9222-:22 \ |
| 52 | -hda <path to chroot>/src/build/images/x86-generic/latest/chromiumos_qemu_image.bin |
| 53 | |
| 54 | - open a root shell on the VM console |
Wade Guthrie | 8624e43 | 2012-04-11 11:12:00 -0700 | [diff] [blame] | 55 | - in VM console, log in with any legal account (but not Guest) |
mukesh agrawal | 70d3ebe | 2011-10-13 14:45:54 -0700 | [diff] [blame] | 56 | - hit C-A-t to open a shell (use right control and alt, to avoid escaping |
| 57 | out of VM's mouse capture) |
| 58 | |
| 59 | crosh> shell |
| 60 | chronos@localhost / $ sudo bash |
| 61 | |
Wade Guthrie | 8624e43 | 2012-04-11 11:12:00 -0700 | [diff] [blame] | 62 | - if you've modified the source after building the image, load shill onto VM image: |
| 63 | (chroot) src/scripts$ ./start_devserver |
mukesh agrawal | 70d3ebe | 2011-10-13 14:45:54 -0700 | [diff] [blame] | 64 | localhost / # FEATURES="nostrip" gmerge shill |
| 65 | |
| 66 | - stop flimflam, and start shill |
| 67 | localhost / # stop flimflam |
Gaurav Shah | 8558710 | 2011-11-29 10:34:22 -0800 | [diff] [blame] | 68 | localhost / # shill --foreground --v=1000 \ |
mukesh agrawal | 8a3188d | 2011-12-01 20:56:44 +0000 | [diff] [blame] | 69 | --device-black-list=managed0,managed1 |
mukesh agrawal | 70d3ebe | 2011-10-13 14:45:54 -0700 | [diff] [blame] | 70 | (the black list keep shill from configuring the devices used by the hostapd |
| 71 | processes.) |
| 72 | |
| 73 | - run the tests |
| 74 | (chroot) src/scripts$ ./run_remote_tests.sh --board=x86-generic \ |
| 75 | --remote=127.0.0.1 --ssh_port 9222 \ |
| 76 | --args="config_file=wifi_vm_config" WiFiManager |
| 77 | |
Gaurav Shah | d52caa5 | 2011-10-25 10:46:03 -0700 | [diff] [blame] | 78 | To run a specific test out of the test suite, use test_pat option to --args. |
| 79 | |
| 80 | # Example: To just run the 035CheckWEPKeySyntax test: |
| 81 | (chroot) src/scripts$ ./run_remote_tests.sh --board=x86-generic \ |
| 82 | --remote=127.0.0.1 --ssh_port 9222 \ |
| 83 | --args="config_file=wifi_vm_config test_pat=035CheckWEPKeySyntax" WiFiManager |
| 84 | |
mukesh agrawal | 70d3ebe | 2011-10-13 14:45:54 -0700 | [diff] [blame] | 85 | - configuration note: if you use a different port |
| 86 | (e.g. hostfwd=tcp::9223-:22), you'll need to change: |
| 87 | - the ssh_port argument to run_remote_tests |
| 88 | - the port numbers in |
| 89 | <chroot>/third_party/autotest/files/client/config/wifi_vm_config |
| 90 | |
| 91 | - debugging test failures |
| 92 | - "grep shill /var/log/messages" for log messages |
| 93 | - "grep wpa_supplicant /var/log/messages" for supplicant log messages |
| 94 | - "wpa_debug debug" to increase supplicant log level |
| 95 | - try resetting the test infrastructure |
| 96 | - rmmod mac80211_hwsim mac80211 cfg80211 |
| 97 | - restart wpasupplicant |
| 98 | - rm /tmp/hostapd-test.control/* |
mukesh agrawal | c2f39fc | 2011-11-07 22:07:23 +0000 | [diff] [blame] | 99 | - examine autotest log files |
| 100 | - check how far the test got before it failed |
| 101 | $ grep -a ': step ' <test output>/<suite name>/<suite name>.<test name>/debug/<suite name>.<test name>.INFO |
| 102 | e.g. |
| 103 | (chroot) $ grep -a ': step ' /tmp/run_remote_tests.abcd/network_WiFiRoaming/network_WiFiRoaming.002Suspend/debug/network_WiFiRoaming.002Suspend.INFO |
| 104 | - read the log file |
| 105 | (chroot) $ LESSOPEN= less /tmp/run_remote_tests.abcd/network_WiFiRoaming/network_WiFiRoaming.002Suspend/debug/network_WiFiRoaming.002Suspend.INFO |
| 106 | |
| 107 | (LESSOPEN= prevents less from misinterpreting the logs as binary files, |
| 108 | and piping them through hexdump.) |
mukesh agrawal | 70d3ebe | 2011-10-13 14:45:54 -0700 | [diff] [blame] | 109 | |
| 110 | - additional test suites: we have a number of other WiFi test suites |
| 111 | (in addition to WiFiManager). these are: WiFiMatFunc, WiFiPerf, |
| 112 | WiFiRoaming, WiFiSecMat. the WiFiPerf tests are probably not too |
| 113 | relevant to shill (just yet), but the rest probably are. |
| 114 | |
| 115 | [1] http://code.google.com/p/googletest/ |
| 116 | [2] http://code.google.com/p/googlemock/ |
| 117 | [3] http://autotest.kernel.org/, |
Gaurav Shah | d52caa5 | 2011-10-25 10:46:03 -0700 | [diff] [blame] | 118 | http://www.chromium.org/chromium-os/testing/testing-faq |