blob: 7ed82dba895cd60b84db4a88c038f86ef58a20af [file] [log] [blame]
cbeust125d2472006-09-16 19:49:12 +00001
the.mindstorm7b73a742006-09-14 17:39:29 +00002===========================================================================
cbeust125d2472006-09-16 19:49:12 +000035.3
4
5Added: ant task can receive several listeners
the.mindstorm7b73a742006-09-14 17:39:29 +00006
7Eclipse plug-in
8Added: Support for configuring per project usedefaultlisteners
cbeust8855dc42006-08-14 20:20:52 +00009
the.mindstorma10e6b82006-08-06 21:51:57 +000010===========================================================================
cbeust8bee4072006-08-23 23:32:32 +0000115.2
12
cbeustae569972006-09-11 19:47:01 +000013Added: "-usedefaultlisteners true/false" to command line and ant
14Added: EmailableReporter (from Paul Mendelson)
cbeust4b9824b2006-09-01 17:34:08 +000015Added: parallel can now be "methods" or "tests". Boolean version deprecated
cbeustc2408c32006-08-29 17:23:22 +000016Added: TestNGAntTask now uses the @ syntax to invoke TestNG
cbeust26b0c782006-08-29 16:54:38 +000017Added: Command line understands @ syntax
cbeuste24b3802006-08-24 17:47:41 +000018Added: JUnitConverter uses the new syntax
19Added: -groups to JUnitConverter
cbeust8bee4072006-08-23 23:32:32 +000020Fixed: Throw proper exception when a DataProvider declares parameters
the.mindstormf91c2a62006-09-03 01:32:44 +000021Added: completely revamped JUnit support (should run all kind of JUnit tests)
22Fixed: TESTNG-40 (Bug in testng-failed.xml generation)
the.mindstorm2ea93332006-09-04 23:59:44 +000023Fixed: TESTNG-106 (Failed "@BeforeSuite" method just skipps the last test in xml-file)
24Fixed: Success on 0 tests (http://forums.opensymphony.com/thread.jspa?threadID=41213)
the.mindstormf91c2a62006-09-03 01:32:44 +000025
26Eclipse plug-in
27Added: TESTNG-105 Automaticaly define TESTNG_HOME classpath variable
cbeust8bee4072006-08-23 23:32:32 +000028
29===========================================================================
the.mindstorma10e6b82006-08-06 21:51:57 +0000305.1
cbeust336f0ef2006-08-01 22:50:26 +000031
cbeust8855dc42006-08-14 20:20:52 +000032Added: @Test(sequential = true)
cbeustca895ef2006-08-09 21:34:51 +000033Fixed: TESTNG-102 (Incorrect ordering of @BeforeMethod calls when a dependency is specified)
the.mindstorm997a5932006-08-11 19:34:04 +000034Fixed: TESTNG-101 (HTML output contains nested <P> tags and a missing <tr> tag)
the.mindstorma10e6b82006-08-06 21:51:57 +000035Added: support for specifying test-only classpath (http://forums.opensymphony.com/thread.jspa?messageID=78048&tstart=0)
36Fixed: TESTNG-93 (method selectors filtering @BeforeMethod)
37Fixed: TESTNG-81 (Assert.assertFalse() displays wrong expected, actual value)
38Fixed: TESTNG-59 (multiple method selectors usage results in no tests run)
39Fixed: TESTNG-56 (invocation of @Before/AfterClass methods in parallel/sequential scenarios)
40Fixed: TESTNG-40 (failures suite does not contain @Before/After Suite/Test methods)
41Fixed: TESTNG-37 (allow passing null parameter value from testng.xml)
42Fixed: TESTNG-7 (display classname when hovering method)
43
44
45Eclipse plug-in
46
47Added: run contextual test classes with parameters from suite definition files
48Added: TESTNG-100 (Show HTML reports after running tests)
the.mindstorm13e8f502006-08-11 20:27:14 +000049Added: TESTNG-97 (Double click top stack to raise comparison)
the.mindstorma10e6b82006-08-06 21:51:57 +000050Added: TESTNG-84 (plug-in UI for suite option should support absolute path)
51Added: TESTNG-20 (copy stack trace)
52
53Fixed: TESTNG-72 (display groups with non-array values)
54Fixed: TESTNG-64 (Eclipse plug-in applies added groups to all launch configurations)
55Fixed: TESTNG-28 (Cannot select groups from dependent eclipse projects)
56Fixed: TESTNG-25 (do not display fully qualified method name when running contextual test class)
57
58Improved behavior:
59 TESTNG-98 (temporary files have guaranteed fixed names)
60 TESTNG-95 (Assertion failed comparison trims trailing ">")
cbeust96761aa2006-08-15 17:31:35 +000061 TESTNG-70 (TestNG prevents eclipse from opening an older CVS version of a java class)
the.mindstorma10e6b82006-08-06 21:51:57 +000062 display of test hierarchy information (TESTNG-29)
63
cbeust336f0ef2006-08-01 22:50:26 +000064===========================================================================
65
665.0.1
67
68Eclipse plug-in
69
70Added: Output directory for the tests
71Added: Can now specify listener classes
72
73===========================================================================
745.0.1
75
76Fixed: reports generated by SuiteHTMLReporter do not work with JDK1.4
77
78===========================================================================
79
805.0
81
82Added: Ant task: support for JVM, workingDir, timeout
83Added: Stack traces can be interactively shown in the HTML reports
84Added: Link to testng.xml in the reports
85Added: New structure for reports, suites go in their individual directory
86Added: @Test(suiteName) and @Test(testName)
87Added: The stack traces in reports do not include TestNG frames (system property testng.exception)
88 (see: http://groups.google.com/group/testng-dev/browse_thread/thread/9f4d46ade10b0fda)
89Fixed: Exit with error when no methods are run
90 (see: http://groups.google.com/group/testng-dev/browse_thread/thread/3c26e8a5658f22ac)
91Added: List of methods in alphabetical order
92Fixed: Class-scoped annotations were not recognized when inherited
93Added: Deprecated @Configuration and introduced @BeforeSuite/Test/Class/TestMethod
94Added: Deprecated @ExpectedExceptions and moved it into @Test
95Added: expectedExceptions to @Test, deprecated @ExpectedExceptions
96Added: New annotations: @BeforeSuite, @BeforeTest, etc...
97Fixed: Was returning an exit code of 0 if a cyclic graph was detected
98Added: Interface org.testng.ITest so that tests can declare a name
99Fixed: The Text reporter was reporting the square of the actual number of methods
100Fixed: Bug reported by Eran about dependencies with an afterClass method
101Added: IHookCallBack now receives the ITestResult in its run() method
102Added: Name of suite for command line can be set with -Dtestng.suite.name=xxx
103Fixed: TestNGAntTask was hardcoding m_haltOnFSP to true
104Fixed: Passing a null parameter caused an NPE in the reports
105Added: "listener" to the ant task (and documentation)
106Added: if patch-testng-sourcedir.properties is found in the classpath
107 with a property "sourcedir" containing a ; separated list of
108 directories, this list will override -sourcedir.
109
110===========================================================================
111
112
1134.7
114
115Added: Maven 2 plug-in
116Fixed: Message formattings in TestNG assertion utility class
117Fixed: @Factory methods were counted as @Test as well
118 http://jira.opensymphony.com/browse/TESTNG-51
119Fixed: All DataProvider parameters were shown in the HTML report
120Fixed: Bug in testng-failed.xml generation
121Fixed: <packages> bug when using a jar file to load the test classes
122Added: alwaysRun for before @Configuration methods
123 http://jira.opensymphony.com/browse/TESTNG-35
124Fixed: groupless @Configurations were not invoked if a method depends on a group
125 http://jira.opensymphony.com/browse/TESTNG-45
126Added: beforeGroups/afterGroups to @Configuration
127
128Eclipse plugin:
129
130Added: last contextual launch is available in Eclipse launcher lists
131Fixed: 3.2M5 integration (removed dependency on non-existing class)
132Fixed: testng-failures.xml generation
133
134===========================================================================
135
1364.6
137
138Added: Documentation contains the new reports
139Added: TestNG.setUseDefaultListeners(boolean)
140Added: Descriptions now appear in TextReporter (verbose>=2) and the HTML reports
141Added: description attribute to @Test and @Configuration
142Added: combined Reporter output in the reports
143Added: methods not run in the reports
144Added: org.testng.IReporter
145Added: threadPoolSize to @Test
146Added: Reports now show relative timings (start at 0)
147Added: Reports now show different colors depending on the methods' classes
148Added: Reports now show all parameters used to invoke the test method
149Added: org.testng.Reporter
150Added: DataProviders can accept a Method as first parameter
151Fixed: Extraneous implicit inclusion of a method
152
153Eclipse plugin:
154
155Added: Run/Debug as TestNG test from the editor contextual menu
156Fixed: TESTNG-24: 'Run as testng test' does not appear of the Test annotation does not have a group
157Fixed: TESTNG-18: Eclipse plugin ignores Factory annotation
158Fixed: TESTNG-21: Show differences when double clicking assertion exceptions
159Added: UI allows setting orientation (even more space)
160 http://forums.opensymphony.com/thread.jspa?threadID=17225&messageID=33805#33805
161
162===========================================================================
163
1644.5
165
166Core:
167
168Fixed: Methods were not implicitly included, only groups
169Fixed: Bug with failed parent @Configuration don't skip child @Configuration/@Test invocations
170Fixed: Bug with overridding @Configuration methods (both parent and child were run)
171Fixed: Bug when overriding beforeClass methods in base class (cyclic graph)
172Added: Support for JAAS (see org.testng.IHookable)
173Fixed: Problem with nested classes inside <package name="foo.*"
174Fixed: If a group is not found, mark the method as a skip instead of aborting
175Fixed: testng-failed.xml was not respecting dependencies
176Fixed: class/include method in testng.xml didn't work on default package
177Fixed: DTD only allowed one <define>
178Fixed: ArrayIndexOutOfBoundsException for jMock
179Added: dependsOnMethods can contain methods from another class
180Fixed: JUnitConverter required -restore, not any more (option is now a no-op)
181Fixed: JUnit mode wasn't invoking setName() on test classes
182Added: Regular expressions for classes in <package>
183Added: Distributed TestNG
184Fixed: Command line parameters and testng.xml are now cumulative
185Fixed: Reports now work for multiple suites
186Fixed: Was ignoring abstract classes even if they have non-abstract instances
187Fixed: If setUp() failed, methods were not skipped
188Fixed: Was not clearly indicating when beforeSuite fails
189Added: @Configuration.inheritGroups
190Fixed: inconsistency between testng.xml and objects regarding method selectors
191
192Eclipse plug-in:
193
194New look for the progress view.
195
196===========================================================================
197
1984.4
199
200Core:
201
202Fixed: testng-failures.xml was not excluding methods from base classes
203Fixed: Bug in suites of suites for JUnit mode
204
205===========================================================================
206
2074.3
208
209Core:
210
211Fixed: testng-failures.xml was not excluding methods from base classes
212Fixed: Bug in suites of suites for JUnit mode
213Added: Excluded groups on command line and ant task
214Fixed: When including a group, implicitly include groups depended upon
215Fixed: When depending on several groups, wasn't skipped if one of them failed
216Fixed: Failures weren't reported accurately in the JUnitReports report
217Fixed: Wasn't throwing an exception if depending on a non-existing group
218
219===========================================================================
220
2214.2
222
223Core:
224
225Fixed: wasn't excluding methods in base classes
226Added: alwaysRun for tests (soft dependencies)
227Fixed: Class-level enabled=false were not honored
228Fixed: Bug with multiple dataproviders on same class
229Fixed: Bug with dataprovider defined in the parent class
230Fixed: Bug with dataprovider defined in a subclass
231Fixed: Bug with dataprovider defined in an abstract class
232Fixed: testng-failures generation was excluding the methods even if a failed test depended on it
233
234===========================================================================
235
2364.1
237
238Core:
239
240Added: @DataProviders can return Iterable<Object[]>
241Fixed: Superclass test methods were not called in the presence of a class @Test
242Added: Reporter class to log messages in the HTML reports
243
244===========================================================================
245
2464.0
247
248Core:
249
250Fixed: suite methods now invoked only once in a hierarchy
251Added: @DataProvider and @testng.data-provider
252Fixed: Interleave order now respected for before/afterClass methods
253Added: Can now invoke java -jar testng-2.6.jar <...>
254Added: Support for BeanShell
255Added: Method Selectors (IMethodSelector)
256Fixed: In the absence of dependencies, @Configuration methods respect inheritance
257Fixed: Bug in multithreaded dependencies on methods
258Fixed: dependsOnGroups wasn't working on regular expressions
259Fixed: Bug in <package> when directories contain spaces in their names
260Fixed: Introduced a JDK5 dependency in the JDK1.4 build (getEnclosingClass())
261Fixed: Output directory in ant task was not honored if it didn't exist
262Fixed: Problem with timeout according to
263 http://forums.opensymphony.com/thread.jspa?threadID=6707
264
265Eclipse plug-in:
266
267Fixed: Wasn't handling linked directories correctly
268Fixed: Bug in QuickFix implementation
269Added: Quick Fix for JUnit conversion (Annotations and JavaDoc)
270Fixed: Methods Run as TestNG test
271Added: Package level Run as TestNG test
272Fixed: Resources from the linked directories are using a wrong path when
273 passed to command line TestNG
274
275IDEA plug-in:
276
277Added: Support for JDK 1.4 (both projects and IDEA itself)
278Fixed: Classes that contained only configuration were ignored
279
280===========================================================================
281
2822.5
283
284Added: ITestListener.onTestStart(ITestResult)
285Added: Support for <packages>
286Added: Resource files for easier ant taskdefs
287Fixed: @Configuration methods were not invoked with individual test methods
288Fixed: Bug with ExpectedExceptions
289Fixed: Didn't support nested factory classes
290Fixed: NPE if -target is omitted with JDK 1.4
291Fixed: @Configuration failures in a class would cause other classes to fail
292Added: alwaysRun
293Fixed: beforeTestClass/afterTestClass were broken for a pathological case
294Added: @Configuration(alwaysRun)
295Added: JUnitConverter task
296Fixed: < and > characters in reports were not escaped
297
298Eclipse plug-in:
299
300Fixed: Class dialog wasn't showing @Factory classes
301
302IDEA plug-in:
303
304First release!
305
306Documentation:
307
308Added: Brand new look!!!
309Added: Section on testng.xml
310Fixed: Numbering of sections
311
312===========================================================================
313
3142.4
315Changed: New package: testng.org
316Fixed: Bug with @ExpectedException occuring the parallel mode
317Fixed: Bug with parameters and beforeTest
318Added: IInstanceInfo support
319Fixed: methods were not excluded when included by groups
320Fixed: testng-failures.xml is now including also the beforeSuite/afterSuite methods
321Fixed: generating the testng-failures.xml is now working as expected
322Fixed: Factories call all the tests even if some of them fail along the way
323Fixed: Better JUnit support (wasn't creating individual instances)
324Fixed: dependsOnGroups didn't work across different classes
325Added: command line (and Ant) -groups option
326Added: @Parameters (and made parameters attribute deprecated)
327Added: Parameters for constructors
328Fixed: Better interleaving of before/afterTestMethods
329Fixed: Ant task
330Fixed: TestNGException thrown when TestNG conditions are not fulfilled
331
332Documentation:
333- New assert classes
334- New ways to launch
335- JUnitConverter documentation
336- new beforeSuite/afterSuite
337
338===========================================================================
339
3402.3
341
342Fixed: Spaces are now legal in JavaDoc comments
343Added: documentation for @Factory
344Fixed: factories were called multiple times
345Added: beforeSuite and afterSuite
346Fixed: inheritance and scope now working properly for annotations
347Fixed: dependsOnMethods wasn't working for 1.4
348Added: Better stack traces
349Added: Better syntax for included/excluded methods
350Fixed: Better verbose support
351Fixed: Various fixes for the Eclipse plug-in
352Added: Can specify a class name on the command line
353Fixed: Default package bug in JUnitConverter
354Added: Regression tests for JUnitConverter
355Added: -quiet option to JUnitConverter
356
357===========================================================================
358
3592.2
360
361Fixed: Wasn't handling several testng.xml files correctly
362Fixed: Renamed -src to -sourcedir
363Fixed: Complains if no sourcedir is specified in 1.4
364Added: In 1.4, don't require annotations="javadoc"
365Fixed: If setUp fails, complain and mark test methods as skips
366Fixed: Dependent methods weren't working for 1.4
367
368===========================================================================
369
3702.1
371
372Added: Parser can accept an InputStream for testng.xml
373Fixed: expected-exceptions now fails if test passes
374Fixed: reports now use the suite name in HTML
375Added: invocationCount and successPercentage
376Added: dependsOnMethods
377Added: timeOut works in non-parallel mode
378
379===========================================================================
380
3812.0
382
383Added: port on JDK 1.4
384
385===========================================================================
386
3871.3
388
389Added: new view: classes (still experimental)
390Added: timeout on methods
391Added: thread-count
392Added: TestNG is now multithread, see "parallel" in <suite>
393
394===========================================================================
395
3961.2
397
398Added: JUnitConverter
399Fixed: Bug with afterClasses (test: AfterClassCalledAtTheEnd)
400
401===========================================================================
402
4031.1
404
405Added: new links for methods and groups in the HTML report
406Added: <methods>
407Added: <fileset> to <testng>
408
409===========================================================================
410
4111.0
412
413Fixed: Updated to the new DTD
414Fixed: Suite table of contents displays failures first
415Fixed: Bug in afterTestClass
416Added: Validating testng.xml
417Added: Scoped parameters
418Added: testng.xml
419Removed: Property quiet
420Changed: Verbose is now an integer
421Added: Dependent methods
422
423===========================================================================
424
4250.9
426
427Added: Groups of groups
428Added: Groups for Configuration methods
429Added: Parameters
430
431===========================================================================
432
4330.2
434
435Fixed: Merged TestMethod and TestClass into Test
436Added: HTML report
437Added: Regexps for groups
438Fixed: Inheritance of methods
439Fixed: ExpectedException is now called ExpectedExceptions