blob: c1894eb4c64fd18a8840ae8ec909c5c30a557677 [file] [log] [blame]
the.mindstorma10e6b82006-08-06 21:51:57 +00001===========================================================================
25.1
cbeust336f0ef2006-08-01 22:50:26 +00003
cbeustca895ef2006-08-09 21:34:51 +00004Fixed: TESTNG-102 (Incorrect ordering of @BeforeMethod calls when a dependency is specified)
the.mindstorm997a5932006-08-11 19:34:04 +00005Fixed: TESTNG-101 (HTML output contains nested <P> tags and a missing <tr> tag)
the.mindstorma10e6b82006-08-06 21:51:57 +00006Added: support for specifying test-only classpath (http://forums.opensymphony.com/thread.jspa?messageID=78048&tstart=0)
7Fixed: TESTNG-93 (method selectors filtering @BeforeMethod)
8Fixed: TESTNG-81 (Assert.assertFalse() displays wrong expected, actual value)
9Fixed: TESTNG-59 (multiple method selectors usage results in no tests run)
10Fixed: TESTNG-56 (invocation of @Before/AfterClass methods in parallel/sequential scenarios)
11Fixed: TESTNG-40 (failures suite does not contain @Before/After Suite/Test methods)
12Fixed: TESTNG-37 (allow passing null parameter value from testng.xml)
13Fixed: TESTNG-7 (display classname when hovering method)
14
15
16Eclipse plug-in
17
18Added: run contextual test classes with parameters from suite definition files
19Added: TESTNG-100 (Show HTML reports after running tests)
20Added: TESTNG-84 (plug-in UI for suite option should support absolute path)
21Added: TESTNG-20 (copy stack trace)
22
23Fixed: TESTNG-72 (display groups with non-array values)
24Fixed: TESTNG-64 (Eclipse plug-in applies added groups to all launch configurations)
25Fixed: TESTNG-28 (Cannot select groups from dependent eclipse projects)
26Fixed: TESTNG-25 (do not display fully qualified method name when running contextual test class)
27
28Improved behavior:
29 TESTNG-98 (temporary files have guaranteed fixed names)
30 TESTNG-95 (Assertion failed comparison trims trailing ">")
31 TESTNG-70 (TestNG prevents eclpse from opening an older CVS version of a java class)
32 display of test hierarchy information (TESTNG-29)
33
34
35TODO
36 :Introduce ITestFactory, deprecate @Factory
37 :Introduce Annotation Transformers
cbeust336f0ef2006-08-01 22:50:26 +000038
39===========================================================================
40
415.0.1
42
43Eclipse plug-in
44
45Added: Output directory for the tests
46Added: Can now specify listener classes
47
48===========================================================================
495.0.1
50
51Fixed: reports generated by SuiteHTMLReporter do not work with JDK1.4
52
53===========================================================================
54
555.0
56
57Added: Ant task: support for JVM, workingDir, timeout
58Added: Stack traces can be interactively shown in the HTML reports
59Added: Link to testng.xml in the reports
60Added: New structure for reports, suites go in their individual directory
61Added: @Test(suiteName) and @Test(testName)
62Added: The stack traces in reports do not include TestNG frames (system property testng.exception)
63 (see: http://groups.google.com/group/testng-dev/browse_thread/thread/9f4d46ade10b0fda)
64Fixed: Exit with error when no methods are run
65 (see: http://groups.google.com/group/testng-dev/browse_thread/thread/3c26e8a5658f22ac)
66Added: List of methods in alphabetical order
67Fixed: Class-scoped annotations were not recognized when inherited
68Added: Deprecated @Configuration and introduced @BeforeSuite/Test/Class/TestMethod
69Added: Deprecated @ExpectedExceptions and moved it into @Test
70Added: expectedExceptions to @Test, deprecated @ExpectedExceptions
71Added: New annotations: @BeforeSuite, @BeforeTest, etc...
72Fixed: Was returning an exit code of 0 if a cyclic graph was detected
73Added: Interface org.testng.ITest so that tests can declare a name
74Fixed: The Text reporter was reporting the square of the actual number of methods
75Fixed: Bug reported by Eran about dependencies with an afterClass method
76Added: IHookCallBack now receives the ITestResult in its run() method
77Added: Name of suite for command line can be set with -Dtestng.suite.name=xxx
78Fixed: TestNGAntTask was hardcoding m_haltOnFSP to true
79Fixed: Passing a null parameter caused an NPE in the reports
80Added: "listener" to the ant task (and documentation)
81Added: if patch-testng-sourcedir.properties is found in the classpath
82 with a property "sourcedir" containing a ; separated list of
83 directories, this list will override -sourcedir.
84
85===========================================================================
86
87
884.7
89
90Added: Maven 2 plug-in
91Fixed: Message formattings in TestNG assertion utility class
92Fixed: @Factory methods were counted as @Test as well
93 http://jira.opensymphony.com/browse/TESTNG-51
94Fixed: All DataProvider parameters were shown in the HTML report
95Fixed: Bug in testng-failed.xml generation
96Fixed: <packages> bug when using a jar file to load the test classes
97Added: alwaysRun for before @Configuration methods
98 http://jira.opensymphony.com/browse/TESTNG-35
99Fixed: groupless @Configurations were not invoked if a method depends on a group
100 http://jira.opensymphony.com/browse/TESTNG-45
101Added: beforeGroups/afterGroups to @Configuration
102
103Eclipse plugin:
104
105Added: last contextual launch is available in Eclipse launcher lists
106Fixed: 3.2M5 integration (removed dependency on non-existing class)
107Fixed: testng-failures.xml generation
108
109===========================================================================
110
1114.6
112
113Added: Documentation contains the new reports
114Added: TestNG.setUseDefaultListeners(boolean)
115Added: Descriptions now appear in TextReporter (verbose>=2) and the HTML reports
116Added: description attribute to @Test and @Configuration
117Added: combined Reporter output in the reports
118Added: methods not run in the reports
119Added: org.testng.IReporter
120Added: threadPoolSize to @Test
121Added: Reports now show relative timings (start at 0)
122Added: Reports now show different colors depending on the methods' classes
123Added: Reports now show all parameters used to invoke the test method
124Added: org.testng.Reporter
125Added: DataProviders can accept a Method as first parameter
126Fixed: Extraneous implicit inclusion of a method
127
128Eclipse plugin:
129
130Added: Run/Debug as TestNG test from the editor contextual menu
131Fixed: TESTNG-24: 'Run as testng test' does not appear of the Test annotation does not have a group
132Fixed: TESTNG-18: Eclipse plugin ignores Factory annotation
133Fixed: TESTNG-21: Show differences when double clicking assertion exceptions
134Added: UI allows setting orientation (even more space)
135 http://forums.opensymphony.com/thread.jspa?threadID=17225&messageID=33805#33805
136
137===========================================================================
138
1394.5
140
141Core:
142
143Fixed: Methods were not implicitly included, only groups
144Fixed: Bug with failed parent @Configuration don't skip child @Configuration/@Test invocations
145Fixed: Bug with overridding @Configuration methods (both parent and child were run)
146Fixed: Bug when overriding beforeClass methods in base class (cyclic graph)
147Added: Support for JAAS (see org.testng.IHookable)
148Fixed: Problem with nested classes inside <package name="foo.*"
149Fixed: If a group is not found, mark the method as a skip instead of aborting
150Fixed: testng-failed.xml was not respecting dependencies
151Fixed: class/include method in testng.xml didn't work on default package
152Fixed: DTD only allowed one <define>
153Fixed: ArrayIndexOutOfBoundsException for jMock
154Added: dependsOnMethods can contain methods from another class
155Fixed: JUnitConverter required -restore, not any more (option is now a no-op)
156Fixed: JUnit mode wasn't invoking setName() on test classes
157Added: Regular expressions for classes in <package>
158Added: Distributed TestNG
159Fixed: Command line parameters and testng.xml are now cumulative
160Fixed: Reports now work for multiple suites
161Fixed: Was ignoring abstract classes even if they have non-abstract instances
162Fixed: If setUp() failed, methods were not skipped
163Fixed: Was not clearly indicating when beforeSuite fails
164Added: @Configuration.inheritGroups
165Fixed: inconsistency between testng.xml and objects regarding method selectors
166
167Eclipse plug-in:
168
169New look for the progress view.
170
171===========================================================================
172
1734.4
174
175Core:
176
177Fixed: testng-failures.xml was not excluding methods from base classes
178Fixed: Bug in suites of suites for JUnit mode
179
180===========================================================================
181
1824.3
183
184Core:
185
186Fixed: testng-failures.xml was not excluding methods from base classes
187Fixed: Bug in suites of suites for JUnit mode
188Added: Excluded groups on command line and ant task
189Fixed: When including a group, implicitly include groups depended upon
190Fixed: When depending on several groups, wasn't skipped if one of them failed
191Fixed: Failures weren't reported accurately in the JUnitReports report
192Fixed: Wasn't throwing an exception if depending on a non-existing group
193
194===========================================================================
195
1964.2
197
198Core:
199
200Fixed: wasn't excluding methods in base classes
201Added: alwaysRun for tests (soft dependencies)
202Fixed: Class-level enabled=false were not honored
203Fixed: Bug with multiple dataproviders on same class
204Fixed: Bug with dataprovider defined in the parent class
205Fixed: Bug with dataprovider defined in a subclass
206Fixed: Bug with dataprovider defined in an abstract class
207Fixed: testng-failures generation was excluding the methods even if a failed test depended on it
208
209===========================================================================
210
2114.1
212
213Core:
214
215Added: @DataProviders can return Iterable<Object[]>
216Fixed: Superclass test methods were not called in the presence of a class @Test
217Added: Reporter class to log messages in the HTML reports
218
219===========================================================================
220
2214.0
222
223Core:
224
225Fixed: suite methods now invoked only once in a hierarchy
226Added: @DataProvider and @testng.data-provider
227Fixed: Interleave order now respected for before/afterClass methods
228Added: Can now invoke java -jar testng-2.6.jar <...>
229Added: Support for BeanShell
230Added: Method Selectors (IMethodSelector)
231Fixed: In the absence of dependencies, @Configuration methods respect inheritance
232Fixed: Bug in multithreaded dependencies on methods
233Fixed: dependsOnGroups wasn't working on regular expressions
234Fixed: Bug in <package> when directories contain spaces in their names
235Fixed: Introduced a JDK5 dependency in the JDK1.4 build (getEnclosingClass())
236Fixed: Output directory in ant task was not honored if it didn't exist
237Fixed: Problem with timeout according to
238 http://forums.opensymphony.com/thread.jspa?threadID=6707
239
240Eclipse plug-in:
241
242Fixed: Wasn't handling linked directories correctly
243Fixed: Bug in QuickFix implementation
244Added: Quick Fix for JUnit conversion (Annotations and JavaDoc)
245Fixed: Methods Run as TestNG test
246Added: Package level Run as TestNG test
247Fixed: Resources from the linked directories are using a wrong path when
248 passed to command line TestNG
249
250IDEA plug-in:
251
252Added: Support for JDK 1.4 (both projects and IDEA itself)
253Fixed: Classes that contained only configuration were ignored
254
255===========================================================================
256
2572.5
258
259Added: ITestListener.onTestStart(ITestResult)
260Added: Support for <packages>
261Added: Resource files for easier ant taskdefs
262Fixed: @Configuration methods were not invoked with individual test methods
263Fixed: Bug with ExpectedExceptions
264Fixed: Didn't support nested factory classes
265Fixed: NPE if -target is omitted with JDK 1.4
266Fixed: @Configuration failures in a class would cause other classes to fail
267Added: alwaysRun
268Fixed: beforeTestClass/afterTestClass were broken for a pathological case
269Added: @Configuration(alwaysRun)
270Added: JUnitConverter task
271Fixed: < and > characters in reports were not escaped
272
273Eclipse plug-in:
274
275Fixed: Class dialog wasn't showing @Factory classes
276
277IDEA plug-in:
278
279First release!
280
281Documentation:
282
283Added: Brand new look!!!
284Added: Section on testng.xml
285Fixed: Numbering of sections
286
287===========================================================================
288
2892.4
290Changed: New package: testng.org
291Fixed: Bug with @ExpectedException occuring the parallel mode
292Fixed: Bug with parameters and beforeTest
293Added: IInstanceInfo support
294Fixed: methods were not excluded when included by groups
295Fixed: testng-failures.xml is now including also the beforeSuite/afterSuite methods
296Fixed: generating the testng-failures.xml is now working as expected
297Fixed: Factories call all the tests even if some of them fail along the way
298Fixed: Better JUnit support (wasn't creating individual instances)
299Fixed: dependsOnGroups didn't work across different classes
300Added: command line (and Ant) -groups option
301Added: @Parameters (and made parameters attribute deprecated)
302Added: Parameters for constructors
303Fixed: Better interleaving of before/afterTestMethods
304Fixed: Ant task
305Fixed: TestNGException thrown when TestNG conditions are not fulfilled
306
307Documentation:
308- New assert classes
309- New ways to launch
310- JUnitConverter documentation
311- new beforeSuite/afterSuite
312
313===========================================================================
314
3152.3
316
317Fixed: Spaces are now legal in JavaDoc comments
318Added: documentation for @Factory
319Fixed: factories were called multiple times
320Added: beforeSuite and afterSuite
321Fixed: inheritance and scope now working properly for annotations
322Fixed: dependsOnMethods wasn't working for 1.4
323Added: Better stack traces
324Added: Better syntax for included/excluded methods
325Fixed: Better verbose support
326Fixed: Various fixes for the Eclipse plug-in
327Added: Can specify a class name on the command line
328Fixed: Default package bug in JUnitConverter
329Added: Regression tests for JUnitConverter
330Added: -quiet option to JUnitConverter
331
332===========================================================================
333
3342.2
335
336Fixed: Wasn't handling several testng.xml files correctly
337Fixed: Renamed -src to -sourcedir
338Fixed: Complains if no sourcedir is specified in 1.4
339Added: In 1.4, don't require annotations="javadoc"
340Fixed: If setUp fails, complain and mark test methods as skips
341Fixed: Dependent methods weren't working for 1.4
342
343===========================================================================
344
3452.1
346
347Added: Parser can accept an InputStream for testng.xml
348Fixed: expected-exceptions now fails if test passes
349Fixed: reports now use the suite name in HTML
350Added: invocationCount and successPercentage
351Added: dependsOnMethods
352Added: timeOut works in non-parallel mode
353
354===========================================================================
355
3562.0
357
358Added: port on JDK 1.4
359
360===========================================================================
361
3621.3
363
364Added: new view: classes (still experimental)
365Added: timeout on methods
366Added: thread-count
367Added: TestNG is now multithread, see "parallel" in <suite>
368
369===========================================================================
370
3711.2
372
373Added: JUnitConverter
374Fixed: Bug with afterClasses (test: AfterClassCalledAtTheEnd)
375
376===========================================================================
377
3781.1
379
380Added: new links for methods and groups in the HTML report
381Added: <methods>
382Added: <fileset> to <testng>
383
384===========================================================================
385
3861.0
387
388Fixed: Updated to the new DTD
389Fixed: Suite table of contents displays failures first
390Fixed: Bug in afterTestClass
391Added: Validating testng.xml
392Added: Scoped parameters
393Added: testng.xml
394Removed: Property quiet
395Changed: Verbose is now an integer
396Added: Dependent methods
397
398===========================================================================
399
4000.9
401
402Added: Groups of groups
403Added: Groups for Configuration methods
404Added: Parameters
405
406===========================================================================
407
4080.2
409
410Fixed: Merged TestMethod and TestClass into Test
411Added: HTML report
412Added: Regexps for groups
413Fixed: Inheritance of methods
414Fixed: ExpectedException is now called ExpectedExceptions