Johnny Chen | 6be30df | 2010-08-03 22:12:11 +0000 | [diff] [blame] | 1 | This README file describes the files and directories related to the Python test |
| 2 | suite under the current 'test' directory. |
| 3 | |
| 4 | o dotest.py |
| 5 | |
| 6 | Provides the test driver for the test suite. To invoke it, cd to the 'test' |
| 7 | directory and issue the './dotest.py' command or './dotest.py -v' for more |
| 8 | verbose output. '.dotest.py -h' prints out the help messge. |
| 9 | |
| 10 | A specific naming pattern is followed by the .py script under the 'test' |
| 11 | directory in order to be recognized by 'dotest.py' test driver as a module |
| 12 | which implements a test case, namely, Test*.py. |
| 13 | |
Johnny Chen | 840d8e3 | 2010-08-17 23:00:13 +0000 | [diff] [blame] | 14 | Some example usages: |
| 15 | |
| 16 | 1. ./dotest.py -v . 2> ~/Developer/Log/lldbtest.log0 |
| 17 | This runs the test suite and directs the run log to a file. |
| 18 | |
| 19 | 2. LLDB_LOG=/tmp/lldb.log GDB_REMOTE_LOG=/tmp/gdb-remote.log ./dotest.py -v . 2> ~/Developer/Log/lldbtest.log |
| 20 | This runs the test suite, with logging turned on for the lldb as well as |
| 21 | the process.gdb-remote channels and directs the run log to a file. |
| 22 | |
Johnny Chen | 6be30df | 2010-08-03 22:12:11 +0000 | [diff] [blame] | 23 | o lldbtest.py |
| 24 | |
| 25 | Provides an abstract base class of lldb test case named 'TestVase', which in |
| 26 | turn inherits from Python's unittest.TestCase. The concrete subclass can |
| 27 | override lldbtest.TestBase in order to inherit the common behavior for |
| 28 | unittest.TestCase.setUp/tearDown implemented in this file. |
| 29 | |
| 30 | To provide a test case, the concrete subclass provides methods whose names |
| 31 | start with the letters test. For more details about the Python's unittest |
| 32 | framework, go to http://docs.python.org/library/unittest.html. |
| 33 | |
| 34 | ./command_source/TestCommandSource.py provides a simple example of test case |
| 35 | which overrides lldbtest.TestBase to exercise the lldb's 'command source' |
| 36 | command. The subclass should override the attribute 'mydir' in order for the |
| 37 | runtime to locate the individual test cases when running as part of a large |
| 38 | test suite or when running each test case as a separate Python invocation. |
| 39 | |
| 40 | The doc string provides more details about the setup required for running a |
| 41 | test case on its own. To run the whole test suite, 'dotest.py' is all you |
| 42 | need to do. |
| 43 | |
| 44 | o subdirectories of 'test' |
| 45 | |
| 46 | Most of them predate the introduction of the python test suite and contain |
| 47 | example C/C++/ObjC source files which get compiled into executables which are |
| 48 | to be exercised by the debugger. |
| 49 | |
| 50 | For such subdirectory which has an associated Test*.py file, it was added as |
| 51 | part of the Python-based test suite to test lldb functionality. |
| 52 | |
| 53 | Some of the subdirectories, for example, the 'help' subdirectory, do not have |
| 54 | C/C++/ObjC source files; they were created to house the Python test case which |
| 55 | does not involve lldb reading in an executable file at all. |
| 56 | |
Johnny Chen | 8f24cd1 | 2010-08-23 23:56:08 +0000 | [diff] [blame] | 57 | o make directory |
| 58 | |
| 59 | Contains Makefile.rules, which can be utilized by test cases to write Makefile |
Johnny Chen | fe34c89 | 2011-07-28 21:15:39 +0000 | [diff] [blame] | 60 | based rules to build binaries for the inferiors. |
| 61 | |
| 62 | By default, the built executable name is a.out, which can be overwritten by |
| 63 | specifying your EXE make variable, via the Makefile under the specfic test |
| 64 | directory or via supplying a Python dictionary to the build method in your |
| 65 | Python test script. An example of the latter can be found in |
| 66 | test/lang/objc/radar-9691614/TestObjCMethodReturningBOOL.py, where: |
| 67 | |
| 68 | def test_method_ret_BOOL_with_dsym(self): |
| 69 | """Test that objective-c method returning BOOL works correctly.""" |
| 70 | d = {'EXE': self.exe_name} |
| 71 | self.buildDsym(dictionary=d) |
| 72 | self.setTearDownCleanup(dictionary=d) |
| 73 | self.objc_method_ret_BOOL(self.exe_name) |
| 74 | |
| 75 | def test_method_ret_BOOL_with_dwarf(self): |
| 76 | """Test that objective-c method returning BOOL works correctly.""" |
| 77 | d = {'EXE': self.exe_name} |
| 78 | self.buildDwarf(dictionary=d) |
| 79 | self.setTearDownCleanup(dictionary=d) |
| 80 | self.objc_method_ret_BOOL(self.exe_name) |
| 81 | |
| 82 | def setUp(self): |
| 83 | # Call super's setUp(). |
| 84 | TestBase.setUp(self) |
| 85 | # We'll use the test method name as the exe_name. |
| 86 | self.exe_name = self.testMethodName |
| 87 | # Find the line number to break inside main(). |
| 88 | self.main_source = "main.m" |
| 89 | self.line = line_number(self.main_source, '// Set breakpoint here.') |
| 90 | |
| 91 | The exe names for the two test methods are equal to the test method names and |
| 92 | are therefore guaranteed different. |
Johnny Chen | 8f24cd1 | 2010-08-23 23:56:08 +0000 | [diff] [blame] | 93 | |
Johnny Chen | 433cc8d | 2010-08-31 17:51:08 +0000 | [diff] [blame] | 94 | o plugins directory |
| 95 | |
| 96 | Contains platform specific plugin to build binaries with dsym/dwarf debugging |
| 97 | info. Other platform specific functionalities may be added in the future. |
| 98 | |
Johnny Chen | 8f24cd1 | 2010-08-23 23:56:08 +0000 | [diff] [blame] | 99 | o unittest2 directory |
Johnny Chen | 16fc694 | 2010-08-05 23:47:51 +0000 | [diff] [blame] | 100 | |
| 101 | Many new features were added to unittest in Python 2.7, including test |
| 102 | discovery. unittest2 allows you to use these features with earlier versions of |
| 103 | Python. |
| 104 | |
| 105 | It currently has unittest2 0.5.1 from http://pypi.python.org/pypi/unittest2. |
Johnny Chen | 3acc73a | 2010-08-06 21:07:38 +0000 | [diff] [blame] | 106 | Version 0.5.1 of unittest2 has feature parity with unittest in Python 2.7 |
| 107 | final. If you want to ensure that your tests run identically under unittest2 |
| 108 | and unittest in Python 2.7 you should use unittest2 0.5.1. |
| 109 | |
| 110 | Later versions of unittest2 include changes in unittest made in Python 3.2 and |
| 111 | onwards after the release of Python 2.7. |
Johnny Chen | 16fc694 | 2010-08-05 23:47:51 +0000 | [diff] [blame] | 112 | |
Johnny Chen | 6be30df | 2010-08-03 22:12:11 +0000 | [diff] [blame] | 113 | o dotest.pl |
| 114 | |
| 115 | In case you wonder, there is also a 'dotest.pl' perl script file. It was |
| 116 | created to visit each Python test case under the specified directory and |
| 117 | invoke Python's builtin unittest.main() on each test case. |
| 118 | |
| 119 | It does not take advantage of the test runner and test suite functionality |
| 120 | provided by Python's unitest framework. Its existence is because we want a |
| 121 | different way of running the whole test suite. As lldb and the Python test |
| 122 | suite become more reliable, we don't expect to be using 'dotest.pl' anymore. |
| 123 | |
Johnny Chen | 0cae7e2 | 2010-10-05 00:08:08 +0000 | [diff] [blame] | 124 | Note: dotest.pl has been moved to the attic directory. |
Johnny Chen | 0de6ab5 | 2011-01-19 02:10:40 +0000 | [diff] [blame] | 125 | |
| 126 | o Profiling dotest.py runs |
| 127 | |
| 128 | I used the following command line thingy to do the profiling on a SnowLeopard |
| 129 | machine: |
| 130 | |
| 131 | $ DOTEST_PROFILE=YES DOTEST_SCRIPT_DIR=/Volumes/data/lldb/svn/trunk/test /System/Library/Frameworks/Python.framework/Versions/Current/lib/python2.6/cProfile.py -o my.profile ./dotest.py -v -w 2> ~/Developer/Log/lldbtest.log |
| 132 | |
Johnny Chen | 27d098f | 2011-01-19 19:48:29 +0000 | [diff] [blame] | 133 | After that, I used the pstats.py module to browse the statistics: |
| 134 | |
| 135 | $ python /System/Library/Frameworks/Python.framework/Versions/Current/lib/python2.6/pstats.py my.profile |