blob: 860231c407d90a690fc7282bcffd11e0e92d505d [file] [log] [blame]
the.mindstorma10e6b82006-08-06 21:51:57 +00001===========================================================================
25.1
cbeust336f0ef2006-08-01 22:50:26 +00003
cbeustca895ef2006-08-09 21:34:51 +00004Fixed: TESTNG-102 (Incorrect ordering of @BeforeMethod calls when a dependency is specified)
the.mindstorm997a5932006-08-11 19:34:04 +00005Fixed: TESTNG-101 (HTML output contains nested <P> tags and a missing <tr> tag)
the.mindstorma10e6b82006-08-06 21:51:57 +00006Added: support for specifying test-only classpath (http://forums.opensymphony.com/thread.jspa?messageID=78048&tstart=0)
7Fixed: TESTNG-93 (method selectors filtering @BeforeMethod)
8Fixed: TESTNG-81 (Assert.assertFalse() displays wrong expected, actual value)
9Fixed: TESTNG-59 (multiple method selectors usage results in no tests run)
10Fixed: TESTNG-56 (invocation of @Before/AfterClass methods in parallel/sequential scenarios)
11Fixed: TESTNG-40 (failures suite does not contain @Before/After Suite/Test methods)
12Fixed: TESTNG-37 (allow passing null parameter value from testng.xml)
13Fixed: TESTNG-7 (display classname when hovering method)
14
15
16Eclipse plug-in
17
18Added: run contextual test classes with parameters from suite definition files
19Added: TESTNG-100 (Show HTML reports after running tests)
the.mindstorm13e8f502006-08-11 20:27:14 +000020Added: TESTNG-97 (Double click top stack to raise comparison)
the.mindstorma10e6b82006-08-06 21:51:57 +000021Added: TESTNG-84 (plug-in UI for suite option should support absolute path)
22Added: TESTNG-20 (copy stack trace)
23
24Fixed: TESTNG-72 (display groups with non-array values)
25Fixed: TESTNG-64 (Eclipse plug-in applies added groups to all launch configurations)
26Fixed: TESTNG-28 (Cannot select groups from dependent eclipse projects)
27Fixed: TESTNG-25 (do not display fully qualified method name when running contextual test class)
28
29Improved behavior:
30 TESTNG-98 (temporary files have guaranteed fixed names)
31 TESTNG-95 (Assertion failed comparison trims trailing ">")
32 TESTNG-70 (TestNG prevents eclpse from opening an older CVS version of a java class)
33 display of test hierarchy information (TESTNG-29)
34
35
36TODO
37 :Introduce ITestFactory, deprecate @Factory
38 :Introduce Annotation Transformers
cbeust336f0ef2006-08-01 22:50:26 +000039
40===========================================================================
41
425.0.1
43
44Eclipse plug-in
45
46Added: Output directory for the tests
47Added: Can now specify listener classes
48
49===========================================================================
505.0.1
51
52Fixed: reports generated by SuiteHTMLReporter do not work with JDK1.4
53
54===========================================================================
55
565.0
57
58Added: Ant task: support for JVM, workingDir, timeout
59Added: Stack traces can be interactively shown in the HTML reports
60Added: Link to testng.xml in the reports
61Added: New structure for reports, suites go in their individual directory
62Added: @Test(suiteName) and @Test(testName)
63Added: The stack traces in reports do not include TestNG frames (system property testng.exception)
64 (see: http://groups.google.com/group/testng-dev/browse_thread/thread/9f4d46ade10b0fda)
65Fixed: Exit with error when no methods are run
66 (see: http://groups.google.com/group/testng-dev/browse_thread/thread/3c26e8a5658f22ac)
67Added: List of methods in alphabetical order
68Fixed: Class-scoped annotations were not recognized when inherited
69Added: Deprecated @Configuration and introduced @BeforeSuite/Test/Class/TestMethod
70Added: Deprecated @ExpectedExceptions and moved it into @Test
71Added: expectedExceptions to @Test, deprecated @ExpectedExceptions
72Added: New annotations: @BeforeSuite, @BeforeTest, etc...
73Fixed: Was returning an exit code of 0 if a cyclic graph was detected
74Added: Interface org.testng.ITest so that tests can declare a name
75Fixed: The Text reporter was reporting the square of the actual number of methods
76Fixed: Bug reported by Eran about dependencies with an afterClass method
77Added: IHookCallBack now receives the ITestResult in its run() method
78Added: Name of suite for command line can be set with -Dtestng.suite.name=xxx
79Fixed: TestNGAntTask was hardcoding m_haltOnFSP to true
80Fixed: Passing a null parameter caused an NPE in the reports
81Added: "listener" to the ant task (and documentation)
82Added: if patch-testng-sourcedir.properties is found in the classpath
83 with a property "sourcedir" containing a ; separated list of
84 directories, this list will override -sourcedir.
85
86===========================================================================
87
88
894.7
90
91Added: Maven 2 plug-in
92Fixed: Message formattings in TestNG assertion utility class
93Fixed: @Factory methods were counted as @Test as well
94 http://jira.opensymphony.com/browse/TESTNG-51
95Fixed: All DataProvider parameters were shown in the HTML report
96Fixed: Bug in testng-failed.xml generation
97Fixed: <packages> bug when using a jar file to load the test classes
98Added: alwaysRun for before @Configuration methods
99 http://jira.opensymphony.com/browse/TESTNG-35
100Fixed: groupless @Configurations were not invoked if a method depends on a group
101 http://jira.opensymphony.com/browse/TESTNG-45
102Added: beforeGroups/afterGroups to @Configuration
103
104Eclipse plugin:
105
106Added: last contextual launch is available in Eclipse launcher lists
107Fixed: 3.2M5 integration (removed dependency on non-existing class)
108Fixed: testng-failures.xml generation
109
110===========================================================================
111
1124.6
113
114Added: Documentation contains the new reports
115Added: TestNG.setUseDefaultListeners(boolean)
116Added: Descriptions now appear in TextReporter (verbose>=2) and the HTML reports
117Added: description attribute to @Test and @Configuration
118Added: combined Reporter output in the reports
119Added: methods not run in the reports
120Added: org.testng.IReporter
121Added: threadPoolSize to @Test
122Added: Reports now show relative timings (start at 0)
123Added: Reports now show different colors depending on the methods' classes
124Added: Reports now show all parameters used to invoke the test method
125Added: org.testng.Reporter
126Added: DataProviders can accept a Method as first parameter
127Fixed: Extraneous implicit inclusion of a method
128
129Eclipse plugin:
130
131Added: Run/Debug as TestNG test from the editor contextual menu
132Fixed: TESTNG-24: 'Run as testng test' does not appear of the Test annotation does not have a group
133Fixed: TESTNG-18: Eclipse plugin ignores Factory annotation
134Fixed: TESTNG-21: Show differences when double clicking assertion exceptions
135Added: UI allows setting orientation (even more space)
136 http://forums.opensymphony.com/thread.jspa?threadID=17225&messageID=33805#33805
137
138===========================================================================
139
1404.5
141
142Core:
143
144Fixed: Methods were not implicitly included, only groups
145Fixed: Bug with failed parent @Configuration don't skip child @Configuration/@Test invocations
146Fixed: Bug with overridding @Configuration methods (both parent and child were run)
147Fixed: Bug when overriding beforeClass methods in base class (cyclic graph)
148Added: Support for JAAS (see org.testng.IHookable)
149Fixed: Problem with nested classes inside <package name="foo.*"
150Fixed: If a group is not found, mark the method as a skip instead of aborting
151Fixed: testng-failed.xml was not respecting dependencies
152Fixed: class/include method in testng.xml didn't work on default package
153Fixed: DTD only allowed one <define>
154Fixed: ArrayIndexOutOfBoundsException for jMock
155Added: dependsOnMethods can contain methods from another class
156Fixed: JUnitConverter required -restore, not any more (option is now a no-op)
157Fixed: JUnit mode wasn't invoking setName() on test classes
158Added: Regular expressions for classes in <package>
159Added: Distributed TestNG
160Fixed: Command line parameters and testng.xml are now cumulative
161Fixed: Reports now work for multiple suites
162Fixed: Was ignoring abstract classes even if they have non-abstract instances
163Fixed: If setUp() failed, methods were not skipped
164Fixed: Was not clearly indicating when beforeSuite fails
165Added: @Configuration.inheritGroups
166Fixed: inconsistency between testng.xml and objects regarding method selectors
167
168Eclipse plug-in:
169
170New look for the progress view.
171
172===========================================================================
173
1744.4
175
176Core:
177
178Fixed: testng-failures.xml was not excluding methods from base classes
179Fixed: Bug in suites of suites for JUnit mode
180
181===========================================================================
182
1834.3
184
185Core:
186
187Fixed: testng-failures.xml was not excluding methods from base classes
188Fixed: Bug in suites of suites for JUnit mode
189Added: Excluded groups on command line and ant task
190Fixed: When including a group, implicitly include groups depended upon
191Fixed: When depending on several groups, wasn't skipped if one of them failed
192Fixed: Failures weren't reported accurately in the JUnitReports report
193Fixed: Wasn't throwing an exception if depending on a non-existing group
194
195===========================================================================
196
1974.2
198
199Core:
200
201Fixed: wasn't excluding methods in base classes
202Added: alwaysRun for tests (soft dependencies)
203Fixed: Class-level enabled=false were not honored
204Fixed: Bug with multiple dataproviders on same class
205Fixed: Bug with dataprovider defined in the parent class
206Fixed: Bug with dataprovider defined in a subclass
207Fixed: Bug with dataprovider defined in an abstract class
208Fixed: testng-failures generation was excluding the methods even if a failed test depended on it
209
210===========================================================================
211
2124.1
213
214Core:
215
216Added: @DataProviders can return Iterable<Object[]>
217Fixed: Superclass test methods were not called in the presence of a class @Test
218Added: Reporter class to log messages in the HTML reports
219
220===========================================================================
221
2224.0
223
224Core:
225
226Fixed: suite methods now invoked only once in a hierarchy
227Added: @DataProvider and @testng.data-provider
228Fixed: Interleave order now respected for before/afterClass methods
229Added: Can now invoke java -jar testng-2.6.jar <...>
230Added: Support for BeanShell
231Added: Method Selectors (IMethodSelector)
232Fixed: In the absence of dependencies, @Configuration methods respect inheritance
233Fixed: Bug in multithreaded dependencies on methods
234Fixed: dependsOnGroups wasn't working on regular expressions
235Fixed: Bug in <package> when directories contain spaces in their names
236Fixed: Introduced a JDK5 dependency in the JDK1.4 build (getEnclosingClass())
237Fixed: Output directory in ant task was not honored if it didn't exist
238Fixed: Problem with timeout according to
239 http://forums.opensymphony.com/thread.jspa?threadID=6707
240
241Eclipse plug-in:
242
243Fixed: Wasn't handling linked directories correctly
244Fixed: Bug in QuickFix implementation
245Added: Quick Fix for JUnit conversion (Annotations and JavaDoc)
246Fixed: Methods Run as TestNG test
247Added: Package level Run as TestNG test
248Fixed: Resources from the linked directories are using a wrong path when
249 passed to command line TestNG
250
251IDEA plug-in:
252
253Added: Support for JDK 1.4 (both projects and IDEA itself)
254Fixed: Classes that contained only configuration were ignored
255
256===========================================================================
257
2582.5
259
260Added: ITestListener.onTestStart(ITestResult)
261Added: Support for <packages>
262Added: Resource files for easier ant taskdefs
263Fixed: @Configuration methods were not invoked with individual test methods
264Fixed: Bug with ExpectedExceptions
265Fixed: Didn't support nested factory classes
266Fixed: NPE if -target is omitted with JDK 1.4
267Fixed: @Configuration failures in a class would cause other classes to fail
268Added: alwaysRun
269Fixed: beforeTestClass/afterTestClass were broken for a pathological case
270Added: @Configuration(alwaysRun)
271Added: JUnitConverter task
272Fixed: < and > characters in reports were not escaped
273
274Eclipse plug-in:
275
276Fixed: Class dialog wasn't showing @Factory classes
277
278IDEA plug-in:
279
280First release!
281
282Documentation:
283
284Added: Brand new look!!!
285Added: Section on testng.xml
286Fixed: Numbering of sections
287
288===========================================================================
289
2902.4
291Changed: New package: testng.org
292Fixed: Bug with @ExpectedException occuring the parallel mode
293Fixed: Bug with parameters and beforeTest
294Added: IInstanceInfo support
295Fixed: methods were not excluded when included by groups
296Fixed: testng-failures.xml is now including also the beforeSuite/afterSuite methods
297Fixed: generating the testng-failures.xml is now working as expected
298Fixed: Factories call all the tests even if some of them fail along the way
299Fixed: Better JUnit support (wasn't creating individual instances)
300Fixed: dependsOnGroups didn't work across different classes
301Added: command line (and Ant) -groups option
302Added: @Parameters (and made parameters attribute deprecated)
303Added: Parameters for constructors
304Fixed: Better interleaving of before/afterTestMethods
305Fixed: Ant task
306Fixed: TestNGException thrown when TestNG conditions are not fulfilled
307
308Documentation:
309- New assert classes
310- New ways to launch
311- JUnitConverter documentation
312- new beforeSuite/afterSuite
313
314===========================================================================
315
3162.3
317
318Fixed: Spaces are now legal in JavaDoc comments
319Added: documentation for @Factory
320Fixed: factories were called multiple times
321Added: beforeSuite and afterSuite
322Fixed: inheritance and scope now working properly for annotations
323Fixed: dependsOnMethods wasn't working for 1.4
324Added: Better stack traces
325Added: Better syntax for included/excluded methods
326Fixed: Better verbose support
327Fixed: Various fixes for the Eclipse plug-in
328Added: Can specify a class name on the command line
329Fixed: Default package bug in JUnitConverter
330Added: Regression tests for JUnitConverter
331Added: -quiet option to JUnitConverter
332
333===========================================================================
334
3352.2
336
337Fixed: Wasn't handling several testng.xml files correctly
338Fixed: Renamed -src to -sourcedir
339Fixed: Complains if no sourcedir is specified in 1.4
340Added: In 1.4, don't require annotations="javadoc"
341Fixed: If setUp fails, complain and mark test methods as skips
342Fixed: Dependent methods weren't working for 1.4
343
344===========================================================================
345
3462.1
347
348Added: Parser can accept an InputStream for testng.xml
349Fixed: expected-exceptions now fails if test passes
350Fixed: reports now use the suite name in HTML
351Added: invocationCount and successPercentage
352Added: dependsOnMethods
353Added: timeOut works in non-parallel mode
354
355===========================================================================
356
3572.0
358
359Added: port on JDK 1.4
360
361===========================================================================
362
3631.3
364
365Added: new view: classes (still experimental)
366Added: timeout on methods
367Added: thread-count
368Added: TestNG is now multithread, see "parallel" in <suite>
369
370===========================================================================
371
3721.2
373
374Added: JUnitConverter
375Fixed: Bug with afterClasses (test: AfterClassCalledAtTheEnd)
376
377===========================================================================
378
3791.1
380
381Added: new links for methods and groups in the HTML report
382Added: <methods>
383Added: <fileset> to <testng>
384
385===========================================================================
386
3871.0
388
389Fixed: Updated to the new DTD
390Fixed: Suite table of contents displays failures first
391Fixed: Bug in afterTestClass
392Added: Validating testng.xml
393Added: Scoped parameters
394Added: testng.xml
395Removed: Property quiet
396Changed: Verbose is now an integer
397Added: Dependent methods
398
399===========================================================================
400
4010.9
402
403Added: Groups of groups
404Added: Groups for Configuration methods
405Added: Parameters
406
407===========================================================================
408
4090.2
410
411Fixed: Merged TestMethod and TestClass into Test
412Added: HTML report
413Added: Regexps for groups
414Fixed: Inheritance of methods
415Fixed: ExpectedException is now called ExpectedExceptions