blob: 55712baf69cee7ce6a5d4d0f6829c82a296b05f2 [file] [log] [blame]
Steve Blocka7e24c12009-10-30 11:49:00 +00001# Copyright 2009 the V8 project authors. All rights reserved.
2# Redistribution and use in source and binary forms, with or without
3# modification, are permitted provided that the following conditions are
4# met:
5#
6# * Redistributions of source code must retain the above copyright
7# notice, this list of conditions and the following disclaimer.
8# * Redistributions in binary form must reproduce the above
9# copyright notice, this list of conditions and the following
10# disclaimer in the documentation and/or other materials provided
11# with the distribution.
12# * Neither the name of Google Inc. nor the names of its
13# contributors may be used to endorse or promote products derived
14# from this software without specific prior written permission.
15#
16# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
17# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
18# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
19# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
20# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
21# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
22# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
23# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
24# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
25# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
26# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
27
28prefix es5conform
29def UNIMPLEMENTED = PASS || FAIL
30def FAIL_OK = FAIL, OKAY
31
Ben Murdochb0fe1622011-05-05 13:52:32 +010032
33##############################################################################
Steve Block8defd9f2010-07-08 12:39:36 +010034# Non UTF8 characters in test files.
35chapter10/10.4/10.4.2/10.4.2-3-c-2-s: FAIL_OK
36chapter10/10.4/10.4.2/10.4.2-3-c-1-s: FAIL_OK
37chapter10/10.4/10.4.2/10.4.2-2-c-1: FAIL_OK
38
39# We do not implement the error chekcs specified in the production rules
40# of 11.1.5 (Object initializer).
41# We are compatible with Safari and Firefox.
42chapter11/11.1/11.1.5: UNIMPLEMENTED
43
Steve Block8defd9f2010-07-08 12:39:36 +010044# We do not have a global object called 'global' as required by tests.
45chapter15/15.1: FAIL_OK
Steve Blocka7e24c12009-10-30 11:49:00 +000046
Steve Block8defd9f2010-07-08 12:39:36 +010047# NaN is writable. We are compatible with JSC.
Leon Clarkee46be812010-01-19 14:06:41 +000048chapter15/15.2/15.2.3/15.2.3.3/15.2.3.3-4-178: FAIL_OK
Steve Block8defd9f2010-07-08 12:39:36 +010049# Infinity is writable. We are compatible with JSC.
Leon Clarkee46be812010-01-19 14:06:41 +000050chapter15/15.2/15.2.3/15.2.3.3/15.2.3.3-4-179: FAIL_OK
Steve Block8defd9f2010-07-08 12:39:36 +010051# undefined is writable. We are compatible with JSC.
Leon Clarkee46be812010-01-19 14:06:41 +000052chapter15/15.2/15.2.3/15.2.3.3/15.2.3.3-4-180: FAIL_OK
Leon Clarkee46be812010-01-19 14:06:41 +000053
Steve Block8defd9f2010-07-08 12:39:36 +010054# Our Function object has an "arguments" property which is used as a
55# non-property in the test.
Leon Clarkee46be812010-01-19 14:06:41 +000056chapter15/15.2/15.2.3/15.2.3.3/15.2.3.3-4-183: FAIL_OK
57
Steve Block8defd9f2010-07-08 12:39:36 +010058# Our Function object has a "caller" property which is used as a
59# non-property in in the test.
Leon Clarkee46be812010-01-19 14:06:41 +000060chapter15/15.2/15.2.3/15.2.3.3/15.2.3.3-4-184: FAIL_OK
61
Steve Block8defd9f2010-07-08 12:39:36 +010062# Our function object has a name property which is used as a
63# non-property in the test.
Leon Clarkee46be812010-01-19 14:06:41 +000064chapter15/15.2/15.2.3/15.2.3.3/15.2.3.3-4-188: FAIL_OK
Leon Clarkee46be812010-01-19 14:06:41 +000065
66# NOT IMPLEMENTED: RegExp.prototype.source
Steve Block8defd9f2010-07-08 12:39:36 +010067chapter15/15.2/15.2.3/15.2.3.3/15.2.3.3-4-212: UNIMPLEMENTED
Leon Clarkee46be812010-01-19 14:06:41 +000068
69# NOT IMPLEMENTED: RegExp.prototype.global
Steve Block8defd9f2010-07-08 12:39:36 +010070chapter15/15.2/15.2.3/15.2.3.3/15.2.3.3-4-213: UNIMPLEMENTED
Leon Clarkee46be812010-01-19 14:06:41 +000071
72# NOT IMPLEMENTED: RegExp.prototype.ignoreCase
Steve Block8defd9f2010-07-08 12:39:36 +010073chapter15/15.2/15.2.3/15.2.3.3/15.2.3.3-4-214: UNIMPLEMENTED
Leon Clarkee46be812010-01-19 14:06:41 +000074
75# NOT IMPLEMENTED: RegExp.prototype.multiline
Steve Block8defd9f2010-07-08 12:39:36 +010076chapter15/15.2/15.2.3/15.2.3.3/15.2.3.3-4-215: UNIMPLEMENTED
Leon Clarkee46be812010-01-19 14:06:41 +000077
Leon Clarkee46be812010-01-19 14:06:41 +000078# All of the tests below marked SUBSETFAIL (in 15.2.3.4) fail because
79# the tests assumes that objects can not have more properties
80# than those described in the spec - but according to spec they can
81# have additional properties.
82# All compareArray calls in these tests could be exchanged with a
Ben Murdochb0fe1622011-05-05 13:52:32 +010083# isSubsetOfArray call (I will upload a patch to the es5conform site).
Leon Clarkee46be812010-01-19 14:06:41 +000084
85# SUBSETFAIL
86chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-1: FAIL_OK
87
Steve Block8defd9f2010-07-08 12:39:36 +010088# SUBSETFAIL + we do not implement all methods on Object.
Leon Clarkee46be812010-01-19 14:06:41 +000089chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-2: FAIL_OK
90
91# SUBSETFAIL
92chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-3: FAIL_OK
93
Steve Block1e0659c2011-05-24 12:43:12 +010094# SUBSETFAIL
Leon Clarkee46be812010-01-19 14:06:41 +000095chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-4: FAIL_OK
96
97# SUBSETFAIL
98chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-5: FAIL_OK
99
100# SUBSETFAIL
101chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-6: FAIL_OK
102
103# SUBSETFAIL
104chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-7: FAIL_OK
105
106# SUBSETFAIL
Leon Clarkee46be812010-01-19 14:06:41 +0000107chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-11: FAIL_OK
108
Steve Block8defd9f2010-07-08 12:39:36 +0100109# We do not implement all methods on RegExp.
110chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-13: FAIL
Leon Clarkee46be812010-01-19 14:06:41 +0000111
112# SUBSETFAIL
113chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-14: FAIL_OK
114
Steve Block8defd9f2010-07-08 12:39:36 +0100115# EvalError.prototype does not have message property.
116chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-15: FAIL
Leon Clarkee46be812010-01-19 14:06:41 +0000117
Steve Block8defd9f2010-07-08 12:39:36 +0100118# Rangeerror.prototype does not have message property.
119chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-16: FAIL
Leon Clarkee46be812010-01-19 14:06:41 +0000120
Steve Block8defd9f2010-07-08 12:39:36 +0100121# ReferenceError.prototype does not have message property.
122chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-17: FAIL
Leon Clarkee46be812010-01-19 14:06:41 +0000123
Steve Block8defd9f2010-07-08 12:39:36 +0100124# SyntaxError.prototype does not have message property.
125chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-18: FAIL
Leon Clarkee46be812010-01-19 14:06:41 +0000126
Steve Block8defd9f2010-07-08 12:39:36 +0100127# TypeError.prototype does not have message property.
128chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-19: FAIL
Leon Clarkee46be812010-01-19 14:06:41 +0000129
Steve Block8defd9f2010-07-08 12:39:36 +0100130# URIError.prototype does not have message property.
131chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-20: FAIL
Leon Clarkee46be812010-01-19 14:06:41 +0000132
133# SUBSETFAIL
134chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-22: FAIL_OK
135
136# SUBSETFAIL
137chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-23: FAIL_OK
138
139# SUBSETFAIL
140chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-24: FAIL_OK
141
142# SUBSETFAIL
143chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-25: FAIL_OK
144
145# SUBSETFAIL
146chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-26: FAIL_OK
147
148# SUBSETFAIL
149chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-27: FAIL_OK
150
151# SUBSETFAIL
152chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-28: FAIL_OK
153
154# SUBSETFAIL
155chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-29: FAIL_OK
156
157# SUBSETFAIL
158chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-30: FAIL_OK
159
160# SUBSETFAIL
161chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-31: FAIL_OK
162
163# SUBSETFAIL
164chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-32: FAIL_OK
165
166# SUBSETFAIL
167chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-33: FAIL_OK
168
169# SUBSETFAIL
170chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-34: FAIL_OK
171
172# SUBSETFAIL
173chapter15/15.2/15.2.3/15.2.3.4/15.2.3.4-4-35: FAIL_OK
174
Steve Block8defd9f2010-07-08 12:39:36 +0100175# Bad test - the test at the end should be "i === true".
Leon Clarkee46be812010-01-19 14:06:41 +0000176chapter15/15.4/15.4.4/15.4.4.17/15.4.4.17-8-10: FAIL_OK
177
Steve Block8defd9f2010-07-08 12:39:36 +0100178# Bad test - according to spec some returns a Boolean, not a number.
179chapter15/15.4/15.4.4/15.4.4.17/15.4.4.17-4-9: FAIL_OK
Leon Clarkee46be812010-01-19 14:06:41 +0000180
Steve Block8defd9f2010-07-08 12:39:36 +0100181# Bad test - uses unitialized variable a in precondition check.
182chapter15/15.4/15.4.4/15.4.4.19/15.4.4.19-9-3: FAIL_OK
Leon Clarkee46be812010-01-19 14:06:41 +0000183
Steve Block8defd9f2010-07-08 12:39:36 +0100184# We do not implement Array mapping functions correctly if array
185# entries are added for nonexistent entries smaller than length by
186# the callback function. We are compatible with JSC.
187# See http://code.google.com/p/v8/issues/detail?id=755
188chapter15/15.4/15.4.4/15.4.4.22/15.4.4.22-9-1: FAIL_OK
Leon Clarkee46be812010-01-19 14:06:41 +0000189
Steve Block8defd9f2010-07-08 12:39:36 +0100190# Bad tests, path in test file is wrong. This will crash the test
191# script so we mark it SKIP.
192chapter15/15.4/15.4.4/15.4.4.22/15.4.4.22-9-c-ii-4: SKIP
193chapter15/15.4/15.4.4/15.4.4.22/15.4.4.22-9-c-ii-4-s: SKIP
Leon Clarkee46be812010-01-19 14:06:41 +0000194
Steve Block8defd9f2010-07-08 12:39:36 +0100195# Bad test - deleting the property on o in callbackfn will
196# have no effect on the actual array on which reduceRight is called.
197chapter15/15.4/15.4.4/15.4.4.22/15.4.4.22-9-7: FAIL_OK
Leon Clarkee46be812010-01-19 14:06:41 +0000198
Steve Block8defd9f2010-07-08 12:39:36 +0100199# We do not correctly recognize \uFEFF as whitespace
200chapter15/15.5/15.5.4/15.5.4.20/15.5.4.20-4-10: FAIL
201chapter15/15.5/15.5.4/15.5.4.20/15.5.4.20-4-18: FAIL
202chapter15/15.5/15.5.4/15.5.4.20/15.5.4.20-4-34: FAIL
203
204# RegExp.prototype is not of type RegExp - we are bug compatible with JSC.
205chapter15/15.10/15.10.6/15.10.6: FAIL_OK
206
207# We do not have the properties of a RegExp instance on RegExp.prototype.
208# The spec says we should - but we are currently bug compatible with JSC.
209chapter15/15.10/15.10.7/15.10.7.1/15.10.7.1-1: FAIL_OK
210chapter15/15.10/15.10.7/15.10.7.1/15.10.7.1-2: FAIL_OK
211chapter15/15.10/15.10.7/15.10.7.2/15.10.7.2-1: FAIL_OK
212chapter15/15.10/15.10.7/15.10.7.2/15.10.7.2-2: FAIL_OK
213chapter15/15.10/15.10.7/15.10.7.3/15.10.7.3-1: FAIL_OK
214chapter15/15.10/15.10.7/15.10.7.3/15.10.7.3-2: FAIL_OK
215chapter15/15.10/15.10.7/15.10.7.4/15.10.7.4-1: FAIL_OK
216chapter15/15.10/15.10.7/15.10.7.4/15.10.7.4-2: FAIL_OK
217chapter15/15.10/15.10.7/15.10.7.5/15.10.7.5-1: FAIL_OK
218chapter15/15.10/15.10.7/15.10.7.5/15.10.7.5-2: FAIL_OK
Andrei Popescu31002712010-02-23 13:46:05 +0000219
Steve Block1e0659c2011-05-24 12:43:12 +0100220##############################################################################
221# Unimplemented parts of strict mode
222# Setting expectations to fail only so that the tests trigger as soon as
223# the strict mode feature gets implemented
224
225# A directive preceeding an 'use strict' directive may not contain
226# an OctalEscapeSequence
227# Incorrect test - need double escape in eval.
228chapter07/7.8/7.8.4/7.8.4-1-s: FAIL
229
Steve Block1e0659c2011-05-24 12:43:12 +0100230# arguments.caller is non-configurable in strict mode
Steve Block44f0eee2011-05-26 01:26:41 +0100231# Invalid test case. Checks for "writable == true" and presence of "put"..
Steve Block1e0659c2011-05-24 12:43:12 +0100232chapter10/10.6/10.6-13-b-3-s: FAIL
Steve Block1e0659c2011-05-24 12:43:12 +0100233# arguments.callee is non-configurable in strict mode
Steve Block44f0eee2011-05-26 01:26:41 +0100234# Invalid test case. Checks for "put" property accessor.
Steve Block1e0659c2011-05-24 12:43:12 +0100235chapter10/10.6/10.6-13-c-3-s: FAIL
236
237# simple assignment throws TypeError if LeftHandSide is a property reference
238# with a primitive base value (this is undefined)
239chapter11/11.13/11.13.1/11.13.1-1-7-s: FAIL
240
241# simple assignment throws TypeError if LeftHandSide is a readonly property
242# in strict mode (Global.NaN)
243chapter11/11.13/11.13.1/11.13.1-4-2-s: FAIL
244# simple assignment throws TypeError if LeftHandSide is a readonly property
245# in strict mode (Global.Infinity)
246chapter11/11.13/11.13.1/11.13.1-4-3-s: FAIL
247# simple assignment throws TypeError if LeftHandSide is a readonly property
248# in strict mode (Global.length)
249chapter11/11.13/11.13.1/11.13.1-4-4-s: FAIL
250# simple assignment throws TypeError if LeftHandSide is a readonly property
Steve Block1e0659c2011-05-24 12:43:12 +0100251# in strict mode (Global.undefined)
252chapter11/11.13/11.13.1/11.13.1-4-27-s: FAIL
253
Steve Block1e0659c2011-05-24 12:43:12 +0100254# delete operator throws TypeError when when deleting a non-configurable
255# data property in strict mode (Global.NaN)
Ben Murdoche0cee9b2011-05-25 10:26:03 +0100256# Invalid test case - "this" is not a global object within the test case.
257# (http://es5conform.codeplex.com/workitem/29151)
258chapter11/11.4/11.4.1/11.4.1-4.a-4-s: FAIL_OK
Steve Block1e0659c2011-05-24 12:43:12 +0100259
260# delete operator throws ReferenceError when deleting a direct reference
261# to a var in strict mode
Ben Murdoche0cee9b2011-05-25 10:26:03 +0100262# Invalid test case. Test expects ReferenceError instead of SyntaxError.
263# http://es5conform.codeplex.com/workitem/29084
Steve Block1e0659c2011-05-24 12:43:12 +0100264chapter11/11.4/11.4.1/11.4.1-5-1-s: FAIL
265# delete operator throws ReferenceError when deleting a direct reference
266# to a function argument in strict mode
Ben Murdoche0cee9b2011-05-25 10:26:03 +0100267# Invalid test case. Test expects ReferenceError instead of SyntaxError.
268# http://es5conform.codeplex.com/workitem/29084
Steve Block1e0659c2011-05-24 12:43:12 +0100269chapter11/11.4/11.4.1/11.4.1-5-2-s: FAIL
270# delete operator throws ReferenceError when deleting a direct reference
271# to a function name in strict mode
Ben Murdoche0cee9b2011-05-25 10:26:03 +0100272# Invalid test case. Test expects ReferenceError instead of SyntaxError.
273# http://es5conform.codeplex.com/workitem/29084
Steve Block1e0659c2011-05-24 12:43:12 +0100274chapter11/11.4/11.4.1/11.4.1-5-3-s: FAIL
Steve Block1e0659c2011-05-24 12:43:12 +0100275
276# eval - a function declaring a var named 'eval' throws EvalError in strict mode
277# Invalid test case. SyntaxError should be expected instead of EvalError.
278chapter12/12.2/12.2.1/12.2.1-1-s: FAIL
279# eval - a function assigning into 'eval' throws EvalError in strict mode
280# Invalid test case. SyntaxError should be expected instead of EvalError.
281chapter12/12.2/12.2.1/12.2.1-2-s: FAIL
282# eval - a function expr declaring a var named 'eval' throws EvalError
283# in strict mode
284# Invalid test case. SyntaxError should be expected instead of EvalError.
285chapter12/12.2/12.2.1/12.2.1-3-s: FAIL
286# eval - a function expr assigning into 'eval' throws a EvalError in strict mode
287# Invalid test case. SyntaxError should be expected instead of EvalError.
288chapter12/12.2/12.2.1/12.2.1-4-s: FAIL
289# eval - a Function declaring var named 'eval' throws EvalError in strict mode
290# Invalid test case. SyntaxError should be expected instead of EvalError.
291chapter12/12.2/12.2.1/12.2.1-5-s: FAIL
292# eval - a Function assigning into 'eval' throws EvalError in strict mode
293# Invalid test case. SyntaxError should be expected instead of EvalError.
294chapter12/12.2/12.2.1/12.2.1-6-s: FAIL
295# eval - a direct eval declaring a var named 'eval' throws EvalError
296# in strict mode
297# Invalid test case. SyntaxError should be expected instead of EvalError.
298chapter12/12.2/12.2.1/12.2.1-7-s: FAIL
299# eval - a direct eval assigning into 'eval' throws EvalError in strict mode
300# Invalid test case. SyntaxError should be expected instead of EvalError.
301chapter12/12.2/12.2.1/12.2.1-8-s: FAIL
302# eval - an indirect eval declaring a var named 'eval' throws EvalError
303# in strict mode
304# Invalid test case. SyntaxError should be expected instead of EvalError.
305chapter12/12.2/12.2.1/12.2.1-9-s: FAIL
306# eval - an indirect eval assigning into 'eval' throws EvalError in strict mode
307# Invalid test case. SyntaxError should be expected instead of EvalError.
308chapter12/12.2/12.2.1/12.2.1-10-s: FAIL
309
310# SyntaxError if eval used as function identifier in function declaration
311# with strict body
312# Test fails to return true on success (invalid test case).
313chapter13/13.1/13.1-3-3-s: FAIL
314# SyntaxError if eval used as function identifier in function expression
315# with strict body
316# Test fails to return true on success (invalid test case).
317chapter13/13.1/13.1-3-4-s: FAIL
318# SyntaxError if eval used as function identifier in function declaration
319# in strict code
320# Test fails to return true on success (invalid test case).
321chapter13/13.1/13.1-3-5-s: FAIL
322# SyntaxError if eval used as function identifier in function expression
323# in strict code
324# Test fails to return true on success (invalid test case).
325chapter13/13.1/13.1-3-6-s: FAIL
326# SyntaxError if arguments used as function identifier in function declaration
327# with strict body
328# Test fails to return true on success (invalid test case).
329chapter13/13.1/13.1-3-9-s: FAIL
330# SyntaxError if arguments used as function identifier in function expression
331# with strict body
332# Test fails to return true on success (invalid test case).
333chapter13/13.1/13.1-3-10-s: FAIL
334# SyntaxError if arguments used as function identifier in function declaration
335# in strict code
336# Test fails to return true on success (invalid test case).
337chapter13/13.1/13.1-3-11-s: FAIL
338# SyntaxError if arguments used as function identifier in function expression
339# in strict code
340# Test fails to return true on success (invalid test case).
341chapter13/13.1/13.1-3-12-s: FAIL
342
Steve Block1e0659c2011-05-24 12:43:12 +0100343# Duplicate combined parameter name allowed in Function constructor called
344# in strict mode if body not strict
345# Test fails to return true on success (invalid test case).
346chapter15/15.3/15.3.2/15.3.2.1/15.3.2.1-11-6-s: FAIL
347
Steve Block1e0659c2011-05-24 12:43:12 +0100348# Array.prototype.reduce - null passed as thisValue to strict callbackfn
Ben Murdoche0cee9b2011-05-25 10:26:03 +0100349# Invalid test case: http://es5conform.codeplex.com/workitem/29085
Steve Block1e0659c2011-05-24 12:43:12 +0100350chapter15/15.4/15.4.4/15.4.4.21/15.4.4.21-9-c-ii-4-s: FAIL
351
Andrei Popescu31002712010-02-23 13:46:05 +0000352[ $arch == mips ]
353
354# Skip all tests on MIPS.
355*: SKIP