| Raymond Hettinger | 7c85fa4 | 2004-07-01 11:01:35 +0000 | [diff] [blame] | 1 | ------------------------------------------------------------------------ |
| 2 | -- quantize.decTest -- decimal quantize operation -- |
| Facundo Batista | 353750c | 2007-09-13 18:13:15 +0000 | [diff] [blame] | 3 | -- Copyright (c) IBM Corporation, 1981, 2007. All rights reserved. -- |
| Raymond Hettinger | 7c85fa4 | 2004-07-01 11:01:35 +0000 | [diff] [blame] | 4 | ------------------------------------------------------------------------ |
| 5 | -- Please see the document "General Decimal Arithmetic Testcases" -- |
| 6 | -- at http://www2.hursley.ibm.com/decimal for the description of -- |
| 7 | -- these testcases. -- |
| 8 | -- -- |
| 9 | -- These testcases are experimental ('beta' versions), and they -- |
| 10 | -- may contain errors. They are offered on an as-is basis. In -- |
| 11 | -- particular, achieving the same results as the tests here is not -- |
| 12 | -- a guarantee that an implementation complies with any Standard -- |
| 13 | -- or specification. The tests are not exhaustive. -- |
| 14 | -- -- |
| 15 | -- Please send comments, suggestions, and corrections to the author: -- |
| 16 | -- Mike Cowlishaw, IBM Fellow -- |
| 17 | -- IBM UK, PO Box 31, Birmingham Road, Warwick CV34 5JL, UK -- |
| 18 | -- mfc@uk.ibm.com -- |
| 19 | ------------------------------------------------------------------------ |
| Facundo Batista | 9401cbe | 2007-11-23 18:14:50 +0000 | [diff] [blame^] | 20 | version: 2.57 |
| Raymond Hettinger | 7c85fa4 | 2004-07-01 11:01:35 +0000 | [diff] [blame] | 21 | |
| 22 | -- Most of the tests here assume a "regular pattern", where the |
| 23 | -- sign and coefficient are +1. |
| 24 | -- 2004.03.15 Underflow for quantize is suppressed |
| Facundo Batista | 353750c | 2007-09-13 18:13:15 +0000 | [diff] [blame] | 25 | -- 2005.06.08 More extensive tests for 'does not fit' |
| Raymond Hettinger | 7c85fa4 | 2004-07-01 11:01:35 +0000 | [diff] [blame] | 26 | |
| 27 | extended: 1 |
| 28 | precision: 9 |
| 29 | rounding: half_up |
| 30 | maxExponent: 999 |
| 31 | minexponent: -999 |
| 32 | |
| 33 | -- sanity checks |
| 34 | quax001 quantize 0 1e0 -> 0 |
| 35 | quax002 quantize 1 1e0 -> 1 |
| 36 | quax003 quantize 0.1 1e+2 -> 0E+2 Inexact Rounded |
| 37 | quax005 quantize 0.1 1e+1 -> 0E+1 Inexact Rounded |
| 38 | quax006 quantize 0.1 1e0 -> 0 Inexact Rounded |
| 39 | quax007 quantize 0.1 1e-1 -> 0.1 |
| 40 | quax008 quantize 0.1 1e-2 -> 0.10 |
| 41 | quax009 quantize 0.1 1e-3 -> 0.100 |
| 42 | quax010 quantize 0.9 1e+2 -> 0E+2 Inexact Rounded |
| 43 | quax011 quantize 0.9 1e+1 -> 0E+1 Inexact Rounded |
| 44 | quax012 quantize 0.9 1e+0 -> 1 Inexact Rounded |
| 45 | quax013 quantize 0.9 1e-1 -> 0.9 |
| 46 | quax014 quantize 0.9 1e-2 -> 0.90 |
| 47 | quax015 quantize 0.9 1e-3 -> 0.900 |
| 48 | -- negatives |
| 49 | quax021 quantize -0 1e0 -> -0 |
| 50 | quax022 quantize -1 1e0 -> -1 |
| 51 | quax023 quantize -0.1 1e+2 -> -0E+2 Inexact Rounded |
| 52 | quax025 quantize -0.1 1e+1 -> -0E+1 Inexact Rounded |
| 53 | quax026 quantize -0.1 1e0 -> -0 Inexact Rounded |
| 54 | quax027 quantize -0.1 1e-1 -> -0.1 |
| 55 | quax028 quantize -0.1 1e-2 -> -0.10 |
| 56 | quax029 quantize -0.1 1e-3 -> -0.100 |
| 57 | quax030 quantize -0.9 1e+2 -> -0E+2 Inexact Rounded |
| 58 | quax031 quantize -0.9 1e+1 -> -0E+1 Inexact Rounded |
| 59 | quax032 quantize -0.9 1e+0 -> -1 Inexact Rounded |
| 60 | quax033 quantize -0.9 1e-1 -> -0.9 |
| 61 | quax034 quantize -0.9 1e-2 -> -0.90 |
| 62 | quax035 quantize -0.9 1e-3 -> -0.900 |
| 63 | quax036 quantize -0.5 1e+2 -> -0E+2 Inexact Rounded |
| 64 | quax037 quantize -0.5 1e+1 -> -0E+1 Inexact Rounded |
| 65 | quax038 quantize -0.5 1e+0 -> -1 Inexact Rounded |
| 66 | quax039 quantize -0.5 1e-1 -> -0.5 |
| 67 | quax040 quantize -0.5 1e-2 -> -0.50 |
| 68 | quax041 quantize -0.5 1e-3 -> -0.500 |
| 69 | quax042 quantize -0.9 1e+2 -> -0E+2 Inexact Rounded |
| 70 | quax043 quantize -0.9 1e+1 -> -0E+1 Inexact Rounded |
| 71 | quax044 quantize -0.9 1e+0 -> -1 Inexact Rounded |
| 72 | quax045 quantize -0.9 1e-1 -> -0.9 |
| 73 | quax046 quantize -0.9 1e-2 -> -0.90 |
| 74 | quax047 quantize -0.9 1e-3 -> -0.900 |
| 75 | |
| 76 | -- examples from Specification |
| 77 | quax060 quantize 2.17 0.001 -> 2.170 |
| 78 | quax061 quantize 2.17 0.01 -> 2.17 |
| 79 | quax062 quantize 2.17 0.1 -> 2.2 Inexact Rounded |
| 80 | quax063 quantize 2.17 1e+0 -> 2 Inexact Rounded |
| 81 | quax064 quantize 2.17 1e+1 -> 0E+1 Inexact Rounded |
| 82 | quax065 quantize -Inf Inf -> -Infinity |
| 83 | quax066 quantize 2 Inf -> NaN Invalid_operation |
| 84 | quax067 quantize -0.1 1 -> -0 Inexact Rounded |
| 85 | quax068 quantize -0 1e+5 -> -0E+5 |
| 86 | quax069 quantize +35236450.6 1e-2 -> NaN Invalid_operation |
| 87 | quax070 quantize -35236450.6 1e-2 -> NaN Invalid_operation |
| 88 | quax071 quantize 217 1e-1 -> 217.0 |
| 89 | quax072 quantize 217 1e+0 -> 217 |
| 90 | quax073 quantize 217 1e+1 -> 2.2E+2 Inexact Rounded |
| 91 | quax074 quantize 217 1e+2 -> 2E+2 Inexact Rounded |
| 92 | |
| 93 | -- general tests .. |
| 94 | quax089 quantize 12 1e+4 -> 0E+4 Inexact Rounded |
| 95 | quax090 quantize 12 1e+3 -> 0E+3 Inexact Rounded |
| 96 | quax091 quantize 12 1e+2 -> 0E+2 Inexact Rounded |
| 97 | quax092 quantize 12 1e+1 -> 1E+1 Inexact Rounded |
| 98 | quax093 quantize 1.2345 1e-2 -> 1.23 Inexact Rounded |
| 99 | quax094 quantize 1.2355 1e-2 -> 1.24 Inexact Rounded |
| 100 | quax095 quantize 1.2345 1e-6 -> 1.234500 |
| 101 | quax096 quantize 9.9999 1e-2 -> 10.00 Inexact Rounded |
| 102 | quax097 quantize 0.0001 1e-2 -> 0.00 Inexact Rounded |
| 103 | quax098 quantize 0.001 1e-2 -> 0.00 Inexact Rounded |
| 104 | quax099 quantize 0.009 1e-2 -> 0.01 Inexact Rounded |
| 105 | quax100 quantize 92 1e+2 -> 1E+2 Inexact Rounded |
| 106 | |
| 107 | quax101 quantize -1 1e0 -> -1 |
| 108 | quax102 quantize -1 1e-1 -> -1.0 |
| 109 | quax103 quantize -1 1e-2 -> -1.00 |
| 110 | quax104 quantize 0 1e0 -> 0 |
| 111 | quax105 quantize 0 1e-1 -> 0.0 |
| 112 | quax106 quantize 0 1e-2 -> 0.00 |
| 113 | quax107 quantize 0.00 1e0 -> 0 |
| 114 | quax108 quantize 0 1e+1 -> 0E+1 |
| 115 | quax109 quantize 0 1e+2 -> 0E+2 |
| 116 | quax110 quantize +1 1e0 -> 1 |
| 117 | quax111 quantize +1 1e-1 -> 1.0 |
| 118 | quax112 quantize +1 1e-2 -> 1.00 |
| 119 | |
| 120 | quax120 quantize 1.04 1e-3 -> 1.040 |
| 121 | quax121 quantize 1.04 1e-2 -> 1.04 |
| 122 | quax122 quantize 1.04 1e-1 -> 1.0 Inexact Rounded |
| 123 | quax123 quantize 1.04 1e0 -> 1 Inexact Rounded |
| 124 | quax124 quantize 1.05 1e-3 -> 1.050 |
| 125 | quax125 quantize 1.05 1e-2 -> 1.05 |
| 126 | quax126 quantize 1.05 1e-1 -> 1.1 Inexact Rounded |
| Raymond Hettinger | 7c85fa4 | 2004-07-01 11:01:35 +0000 | [diff] [blame] | 127 | quax131 quantize 1.05 1e0 -> 1 Inexact Rounded |
| 128 | quax132 quantize 1.06 1e-3 -> 1.060 |
| 129 | quax133 quantize 1.06 1e-2 -> 1.06 |
| 130 | quax134 quantize 1.06 1e-1 -> 1.1 Inexact Rounded |
| 131 | quax135 quantize 1.06 1e0 -> 1 Inexact Rounded |
| 132 | |
| 133 | quax140 quantize -10 1e-2 -> -10.00 |
| 134 | quax141 quantize +1 1e-2 -> 1.00 |
| 135 | quax142 quantize +10 1e-2 -> 10.00 |
| 136 | quax143 quantize 1E+10 1e-2 -> NaN Invalid_operation |
| 137 | quax144 quantize 1E-10 1e-2 -> 0.00 Inexact Rounded |
| 138 | quax145 quantize 1E-3 1e-2 -> 0.00 Inexact Rounded |
| 139 | quax146 quantize 1E-2 1e-2 -> 0.01 |
| 140 | quax147 quantize 1E-1 1e-2 -> 0.10 |
| 141 | quax148 quantize 0E-10 1e-2 -> 0.00 |
| 142 | |
| 143 | quax150 quantize 1.0600 1e-5 -> 1.06000 |
| 144 | quax151 quantize 1.0600 1e-4 -> 1.0600 |
| 145 | quax152 quantize 1.0600 1e-3 -> 1.060 Rounded |
| 146 | quax153 quantize 1.0600 1e-2 -> 1.06 Rounded |
| 147 | quax154 quantize 1.0600 1e-1 -> 1.1 Inexact Rounded |
| 148 | quax155 quantize 1.0600 1e0 -> 1 Inexact Rounded |
| 149 | |
| 150 | -- base tests with non-1 coefficients |
| 151 | quax161 quantize 0 -9e0 -> 0 |
| 152 | quax162 quantize 1 -7e0 -> 1 |
| 153 | quax163 quantize 0.1 -1e+2 -> 0E+2 Inexact Rounded |
| 154 | quax165 quantize 0.1 0e+1 -> 0E+1 Inexact Rounded |
| 155 | quax166 quantize 0.1 2e0 -> 0 Inexact Rounded |
| 156 | quax167 quantize 0.1 3e-1 -> 0.1 |
| 157 | quax168 quantize 0.1 44e-2 -> 0.10 |
| 158 | quax169 quantize 0.1 555e-3 -> 0.100 |
| 159 | quax170 quantize 0.9 6666e+2 -> 0E+2 Inexact Rounded |
| 160 | quax171 quantize 0.9 -777e+1 -> 0E+1 Inexact Rounded |
| 161 | quax172 quantize 0.9 -88e+0 -> 1 Inexact Rounded |
| 162 | quax173 quantize 0.9 -9e-1 -> 0.9 |
| 163 | quax174 quantize 0.9 0e-2 -> 0.90 |
| 164 | quax175 quantize 0.9 1.1e-3 -> 0.9000 |
| 165 | -- negatives |
| 166 | quax181 quantize -0 1.1e0 -> -0.0 |
| 167 | quax182 quantize -1 -1e0 -> -1 |
| 168 | quax183 quantize -0.1 11e+2 -> -0E+2 Inexact Rounded |
| 169 | quax185 quantize -0.1 111e+1 -> -0E+1 Inexact Rounded |
| 170 | quax186 quantize -0.1 71e0 -> -0 Inexact Rounded |
| 171 | quax187 quantize -0.1 -91e-1 -> -0.1 |
| 172 | quax188 quantize -0.1 -.1e-2 -> -0.100 |
| 173 | quax189 quantize -0.1 -1e-3 -> -0.100 |
| 174 | quax190 quantize -0.9 0e+2 -> -0E+2 Inexact Rounded |
| 175 | quax191 quantize -0.9 -0e+1 -> -0E+1 Inexact Rounded |
| 176 | quax192 quantize -0.9 -10e+0 -> -1 Inexact Rounded |
| 177 | quax193 quantize -0.9 100e-1 -> -0.9 |
| 178 | quax194 quantize -0.9 999e-2 -> -0.90 |
| 179 | |
| 180 | -- +ve exponents .. |
| 181 | quax201 quantize -1 1e+0 -> -1 |
| 182 | quax202 quantize -1 1e+1 -> -0E+1 Inexact Rounded |
| 183 | quax203 quantize -1 1e+2 -> -0E+2 Inexact Rounded |
| 184 | quax204 quantize 0 1e+0 -> 0 |
| 185 | quax205 quantize 0 1e+1 -> 0E+1 |
| 186 | quax206 quantize 0 1e+2 -> 0E+2 |
| 187 | quax207 quantize +1 1e+0 -> 1 |
| 188 | quax208 quantize +1 1e+1 -> 0E+1 Inexact Rounded |
| 189 | quax209 quantize +1 1e+2 -> 0E+2 Inexact Rounded |
| 190 | |
| 191 | quax220 quantize 1.04 1e+3 -> 0E+3 Inexact Rounded |
| 192 | quax221 quantize 1.04 1e+2 -> 0E+2 Inexact Rounded |
| 193 | quax222 quantize 1.04 1e+1 -> 0E+1 Inexact Rounded |
| 194 | quax223 quantize 1.04 1e+0 -> 1 Inexact Rounded |
| 195 | quax224 quantize 1.05 1e+3 -> 0E+3 Inexact Rounded |
| 196 | quax225 quantize 1.05 1e+2 -> 0E+2 Inexact Rounded |
| 197 | quax226 quantize 1.05 1e+1 -> 0E+1 Inexact Rounded |
| 198 | quax227 quantize 1.05 1e+0 -> 1 Inexact Rounded |
| 199 | quax228 quantize 1.05 1e+3 -> 0E+3 Inexact Rounded |
| 200 | quax229 quantize 1.05 1e+2 -> 0E+2 Inexact Rounded |
| 201 | quax230 quantize 1.05 1e+1 -> 0E+1 Inexact Rounded |
| 202 | quax231 quantize 1.05 1e+0 -> 1 Inexact Rounded |
| 203 | quax232 quantize 1.06 1e+3 -> 0E+3 Inexact Rounded |
| 204 | quax233 quantize 1.06 1e+2 -> 0E+2 Inexact Rounded |
| 205 | quax234 quantize 1.06 1e+1 -> 0E+1 Inexact Rounded |
| 206 | quax235 quantize 1.06 1e+0 -> 1 Inexact Rounded |
| 207 | |
| 208 | quax240 quantize -10 1e+1 -> -1E+1 Rounded |
| 209 | quax241 quantize +1 1e+1 -> 0E+1 Inexact Rounded |
| 210 | quax242 quantize +10 1e+1 -> 1E+1 Rounded |
| 211 | quax243 quantize 1E+1 1e+1 -> 1E+1 -- underneath this is E+1 |
| 212 | quax244 quantize 1E+2 1e+1 -> 1.0E+2 -- underneath this is E+1 |
| 213 | quax245 quantize 1E+3 1e+1 -> 1.00E+3 -- underneath this is E+1 |
| 214 | quax246 quantize 1E+4 1e+1 -> 1.000E+4 -- underneath this is E+1 |
| 215 | quax247 quantize 1E+5 1e+1 -> 1.0000E+5 -- underneath this is E+1 |
| 216 | quax248 quantize 1E+6 1e+1 -> 1.00000E+6 -- underneath this is E+1 |
| 217 | quax249 quantize 1E+7 1e+1 -> 1.000000E+7 -- underneath this is E+1 |
| 218 | quax250 quantize 1E+8 1e+1 -> 1.0000000E+8 -- underneath this is E+1 |
| 219 | quax251 quantize 1E+9 1e+1 -> 1.00000000E+9 -- underneath this is E+1 |
| 220 | -- next one tries to add 9 zeros |
| 221 | quax252 quantize 1E+10 1e+1 -> NaN Invalid_operation |
| 222 | quax253 quantize 1E-10 1e+1 -> 0E+1 Inexact Rounded |
| 223 | quax254 quantize 1E-2 1e+1 -> 0E+1 Inexact Rounded |
| 224 | quax255 quantize 0E-10 1e+1 -> 0E+1 |
| 225 | quax256 quantize -0E-10 1e+1 -> -0E+1 |
| 226 | quax257 quantize -0E-1 1e+1 -> -0E+1 |
| 227 | quax258 quantize -0 1e+1 -> -0E+1 |
| 228 | quax259 quantize -0E+1 1e+1 -> -0E+1 |
| 229 | |
| 230 | quax260 quantize -10 1e+2 -> -0E+2 Inexact Rounded |
| 231 | quax261 quantize +1 1e+2 -> 0E+2 Inexact Rounded |
| 232 | quax262 quantize +10 1e+2 -> 0E+2 Inexact Rounded |
| 233 | quax263 quantize 1E+1 1e+2 -> 0E+2 Inexact Rounded |
| 234 | quax264 quantize 1E+2 1e+2 -> 1E+2 |
| 235 | quax265 quantize 1E+3 1e+2 -> 1.0E+3 |
| 236 | quax266 quantize 1E+4 1e+2 -> 1.00E+4 |
| 237 | quax267 quantize 1E+5 1e+2 -> 1.000E+5 |
| 238 | quax268 quantize 1E+6 1e+2 -> 1.0000E+6 |
| 239 | quax269 quantize 1E+7 1e+2 -> 1.00000E+7 |
| 240 | quax270 quantize 1E+8 1e+2 -> 1.000000E+8 |
| 241 | quax271 quantize 1E+9 1e+2 -> 1.0000000E+9 |
| 242 | quax272 quantize 1E+10 1e+2 -> 1.00000000E+10 |
| 243 | quax273 quantize 1E-10 1e+2 -> 0E+2 Inexact Rounded |
| 244 | quax274 quantize 1E-2 1e+2 -> 0E+2 Inexact Rounded |
| 245 | quax275 quantize 0E-10 1e+2 -> 0E+2 |
| 246 | |
| 247 | quax280 quantize -10 1e+3 -> -0E+3 Inexact Rounded |
| 248 | quax281 quantize +1 1e+3 -> 0E+3 Inexact Rounded |
| 249 | quax282 quantize +10 1e+3 -> 0E+3 Inexact Rounded |
| 250 | quax283 quantize 1E+1 1e+3 -> 0E+3 Inexact Rounded |
| 251 | quax284 quantize 1E+2 1e+3 -> 0E+3 Inexact Rounded |
| 252 | quax285 quantize 1E+3 1e+3 -> 1E+3 |
| 253 | quax286 quantize 1E+4 1e+3 -> 1.0E+4 |
| 254 | quax287 quantize 1E+5 1e+3 -> 1.00E+5 |
| 255 | quax288 quantize 1E+6 1e+3 -> 1.000E+6 |
| 256 | quax289 quantize 1E+7 1e+3 -> 1.0000E+7 |
| 257 | quax290 quantize 1E+8 1e+3 -> 1.00000E+8 |
| 258 | quax291 quantize 1E+9 1e+3 -> 1.000000E+9 |
| 259 | quax292 quantize 1E+10 1e+3 -> 1.0000000E+10 |
| 260 | quax293 quantize 1E-10 1e+3 -> 0E+3 Inexact Rounded |
| 261 | quax294 quantize 1E-2 1e+3 -> 0E+3 Inexact Rounded |
| 262 | quax295 quantize 0E-10 1e+3 -> 0E+3 |
| 263 | |
| 264 | -- round up from below [sign wrong in JIT compiler once] |
| 265 | quax300 quantize 0.0078 1e-5 -> 0.00780 |
| 266 | quax301 quantize 0.0078 1e-4 -> 0.0078 |
| 267 | quax302 quantize 0.0078 1e-3 -> 0.008 Inexact Rounded |
| 268 | quax303 quantize 0.0078 1e-2 -> 0.01 Inexact Rounded |
| 269 | quax304 quantize 0.0078 1e-1 -> 0.0 Inexact Rounded |
| 270 | quax305 quantize 0.0078 1e0 -> 0 Inexact Rounded |
| 271 | quax306 quantize 0.0078 1e+1 -> 0E+1 Inexact Rounded |
| 272 | quax307 quantize 0.0078 1e+2 -> 0E+2 Inexact Rounded |
| 273 | |
| 274 | quax310 quantize -0.0078 1e-5 -> -0.00780 |
| 275 | quax311 quantize -0.0078 1e-4 -> -0.0078 |
| 276 | quax312 quantize -0.0078 1e-3 -> -0.008 Inexact Rounded |
| 277 | quax313 quantize -0.0078 1e-2 -> -0.01 Inexact Rounded |
| 278 | quax314 quantize -0.0078 1e-1 -> -0.0 Inexact Rounded |
| 279 | quax315 quantize -0.0078 1e0 -> -0 Inexact Rounded |
| 280 | quax316 quantize -0.0078 1e+1 -> -0E+1 Inexact Rounded |
| 281 | quax317 quantize -0.0078 1e+2 -> -0E+2 Inexact Rounded |
| 282 | |
| 283 | quax320 quantize 0.078 1e-5 -> 0.07800 |
| 284 | quax321 quantize 0.078 1e-4 -> 0.0780 |
| 285 | quax322 quantize 0.078 1e-3 -> 0.078 |
| 286 | quax323 quantize 0.078 1e-2 -> 0.08 Inexact Rounded |
| 287 | quax324 quantize 0.078 1e-1 -> 0.1 Inexact Rounded |
| 288 | quax325 quantize 0.078 1e0 -> 0 Inexact Rounded |
| 289 | quax326 quantize 0.078 1e+1 -> 0E+1 Inexact Rounded |
| 290 | quax327 quantize 0.078 1e+2 -> 0E+2 Inexact Rounded |
| 291 | |
| 292 | quax330 quantize -0.078 1e-5 -> -0.07800 |
| 293 | quax331 quantize -0.078 1e-4 -> -0.0780 |
| 294 | quax332 quantize -0.078 1e-3 -> -0.078 |
| 295 | quax333 quantize -0.078 1e-2 -> -0.08 Inexact Rounded |
| 296 | quax334 quantize -0.078 1e-1 -> -0.1 Inexact Rounded |
| 297 | quax335 quantize -0.078 1e0 -> -0 Inexact Rounded |
| 298 | quax336 quantize -0.078 1e+1 -> -0E+1 Inexact Rounded |
| 299 | quax337 quantize -0.078 1e+2 -> -0E+2 Inexact Rounded |
| 300 | |
| 301 | quax340 quantize 0.78 1e-5 -> 0.78000 |
| 302 | quax341 quantize 0.78 1e-4 -> 0.7800 |
| 303 | quax342 quantize 0.78 1e-3 -> 0.780 |
| 304 | quax343 quantize 0.78 1e-2 -> 0.78 |
| 305 | quax344 quantize 0.78 1e-1 -> 0.8 Inexact Rounded |
| 306 | quax345 quantize 0.78 1e0 -> 1 Inexact Rounded |
| 307 | quax346 quantize 0.78 1e+1 -> 0E+1 Inexact Rounded |
| 308 | quax347 quantize 0.78 1e+2 -> 0E+2 Inexact Rounded |
| 309 | |
| 310 | quax350 quantize -0.78 1e-5 -> -0.78000 |
| 311 | quax351 quantize -0.78 1e-4 -> -0.7800 |
| 312 | quax352 quantize -0.78 1e-3 -> -0.780 |
| 313 | quax353 quantize -0.78 1e-2 -> -0.78 |
| 314 | quax354 quantize -0.78 1e-1 -> -0.8 Inexact Rounded |
| 315 | quax355 quantize -0.78 1e0 -> -1 Inexact Rounded |
| 316 | quax356 quantize -0.78 1e+1 -> -0E+1 Inexact Rounded |
| 317 | quax357 quantize -0.78 1e+2 -> -0E+2 Inexact Rounded |
| 318 | |
| 319 | quax360 quantize 7.8 1e-5 -> 7.80000 |
| 320 | quax361 quantize 7.8 1e-4 -> 7.8000 |
| 321 | quax362 quantize 7.8 1e-3 -> 7.800 |
| 322 | quax363 quantize 7.8 1e-2 -> 7.80 |
| 323 | quax364 quantize 7.8 1e-1 -> 7.8 |
| 324 | quax365 quantize 7.8 1e0 -> 8 Inexact Rounded |
| 325 | quax366 quantize 7.8 1e+1 -> 1E+1 Inexact Rounded |
| 326 | quax367 quantize 7.8 1e+2 -> 0E+2 Inexact Rounded |
| 327 | quax368 quantize 7.8 1e+3 -> 0E+3 Inexact Rounded |
| 328 | |
| 329 | quax370 quantize -7.8 1e-5 -> -7.80000 |
| 330 | quax371 quantize -7.8 1e-4 -> -7.8000 |
| 331 | quax372 quantize -7.8 1e-3 -> -7.800 |
| 332 | quax373 quantize -7.8 1e-2 -> -7.80 |
| 333 | quax374 quantize -7.8 1e-1 -> -7.8 |
| 334 | quax375 quantize -7.8 1e0 -> -8 Inexact Rounded |
| 335 | quax376 quantize -7.8 1e+1 -> -1E+1 Inexact Rounded |
| 336 | quax377 quantize -7.8 1e+2 -> -0E+2 Inexact Rounded |
| 337 | quax378 quantize -7.8 1e+3 -> -0E+3 Inexact Rounded |
| 338 | |
| 339 | -- some individuals |
| 340 | precision: 9 |
| 341 | quax380 quantize 352364.506 1e-2 -> 352364.51 Inexact Rounded |
| 342 | quax381 quantize 3523645.06 1e-2 -> 3523645.06 |
| 343 | quax382 quantize 35236450.6 1e-2 -> NaN Invalid_operation |
| 344 | quax383 quantize 352364506 1e-2 -> NaN Invalid_operation |
| 345 | quax384 quantize -352364.506 1e-2 -> -352364.51 Inexact Rounded |
| 346 | quax385 quantize -3523645.06 1e-2 -> -3523645.06 |
| 347 | quax386 quantize -35236450.6 1e-2 -> NaN Invalid_operation |
| 348 | quax387 quantize -352364506 1e-2 -> NaN Invalid_operation |
| 349 | |
| 350 | rounding: down |
| 351 | quax389 quantize 35236450.6 1e-2 -> NaN Invalid_operation |
| 352 | -- ? should that one instead have been: |
| 353 | -- quax389 quantize 35236450.6 1e-2 -> NaN Invalid_operation |
| 354 | rounding: half_up |
| 355 | |
| 356 | -- and a few more from e-mail discussions |
| 357 | precision: 7 |
| 358 | quax391 quantize 12.34567 1e-3 -> 12.346 Inexact Rounded |
| 359 | quax392 quantize 123.4567 1e-3 -> 123.457 Inexact Rounded |
| 360 | quax393 quantize 1234.567 1e-3 -> 1234.567 |
| 361 | quax394 quantize 12345.67 1e-3 -> NaN Invalid_operation |
| 362 | quax395 quantize 123456.7 1e-3 -> NaN Invalid_operation |
| 363 | quax396 quantize 1234567. 1e-3 -> NaN Invalid_operation |
| 364 | |
| 365 | -- some 9999 round-up cases |
| 366 | precision: 9 |
| 367 | quax400 quantize 9.999 1e-5 -> 9.99900 |
| 368 | quax401 quantize 9.999 1e-4 -> 9.9990 |
| 369 | quax402 quantize 9.999 1e-3 -> 9.999 |
| 370 | quax403 quantize 9.999 1e-2 -> 10.00 Inexact Rounded |
| 371 | quax404 quantize 9.999 1e-1 -> 10.0 Inexact Rounded |
| 372 | quax405 quantize 9.999 1e0 -> 10 Inexact Rounded |
| 373 | quax406 quantize 9.999 1e1 -> 1E+1 Inexact Rounded |
| 374 | quax407 quantize 9.999 1e2 -> 0E+2 Inexact Rounded |
| 375 | |
| 376 | quax410 quantize 0.999 1e-5 -> 0.99900 |
| 377 | quax411 quantize 0.999 1e-4 -> 0.9990 |
| 378 | quax412 quantize 0.999 1e-3 -> 0.999 |
| 379 | quax413 quantize 0.999 1e-2 -> 1.00 Inexact Rounded |
| 380 | quax414 quantize 0.999 1e-1 -> 1.0 Inexact Rounded |
| 381 | quax415 quantize 0.999 1e0 -> 1 Inexact Rounded |
| 382 | quax416 quantize 0.999 1e1 -> 0E+1 Inexact Rounded |
| 383 | |
| 384 | quax420 quantize 0.0999 1e-5 -> 0.09990 |
| 385 | quax421 quantize 0.0999 1e-4 -> 0.0999 |
| 386 | quax422 quantize 0.0999 1e-3 -> 0.100 Inexact Rounded |
| 387 | quax423 quantize 0.0999 1e-2 -> 0.10 Inexact Rounded |
| 388 | quax424 quantize 0.0999 1e-1 -> 0.1 Inexact Rounded |
| 389 | quax425 quantize 0.0999 1e0 -> 0 Inexact Rounded |
| 390 | quax426 quantize 0.0999 1e1 -> 0E+1 Inexact Rounded |
| 391 | |
| 392 | quax430 quantize 0.00999 1e-5 -> 0.00999 |
| 393 | quax431 quantize 0.00999 1e-4 -> 0.0100 Inexact Rounded |
| 394 | quax432 quantize 0.00999 1e-3 -> 0.010 Inexact Rounded |
| 395 | quax433 quantize 0.00999 1e-2 -> 0.01 Inexact Rounded |
| 396 | quax434 quantize 0.00999 1e-1 -> 0.0 Inexact Rounded |
| 397 | quax435 quantize 0.00999 1e0 -> 0 Inexact Rounded |
| 398 | quax436 quantize 0.00999 1e1 -> 0E+1 Inexact Rounded |
| 399 | |
| 400 | quax440 quantize 0.000999 1e-5 -> 0.00100 Inexact Rounded |
| 401 | quax441 quantize 0.000999 1e-4 -> 0.0010 Inexact Rounded |
| 402 | quax442 quantize 0.000999 1e-3 -> 0.001 Inexact Rounded |
| 403 | quax443 quantize 0.000999 1e-2 -> 0.00 Inexact Rounded |
| 404 | quax444 quantize 0.000999 1e-1 -> 0.0 Inexact Rounded |
| 405 | quax445 quantize 0.000999 1e0 -> 0 Inexact Rounded |
| 406 | quax446 quantize 0.000999 1e1 -> 0E+1 Inexact Rounded |
| 407 | |
| 408 | precision: 8 |
| 409 | quax449 quantize 9.999E-15 1e-23 -> NaN Invalid_operation |
| 410 | quax450 quantize 9.999E-15 1e-22 -> 9.9990000E-15 |
| 411 | quax451 quantize 9.999E-15 1e-21 -> 9.999000E-15 |
| 412 | quax452 quantize 9.999E-15 1e-20 -> 9.99900E-15 |
| 413 | quax453 quantize 9.999E-15 1e-19 -> 9.9990E-15 |
| 414 | quax454 quantize 9.999E-15 1e-18 -> 9.999E-15 |
| 415 | quax455 quantize 9.999E-15 1e-17 -> 1.000E-14 Inexact Rounded |
| 416 | quax456 quantize 9.999E-15 1e-16 -> 1.00E-14 Inexact Rounded |
| 417 | quax457 quantize 9.999E-15 1e-15 -> 1.0E-14 Inexact Rounded |
| 418 | quax458 quantize 9.999E-15 1e-14 -> 1E-14 Inexact Rounded |
| 419 | quax459 quantize 9.999E-15 1e-13 -> 0E-13 Inexact Rounded |
| 420 | quax460 quantize 9.999E-15 1e-12 -> 0E-12 Inexact Rounded |
| 421 | quax461 quantize 9.999E-15 1e-11 -> 0E-11 Inexact Rounded |
| 422 | quax462 quantize 9.999E-15 1e-10 -> 0E-10 Inexact Rounded |
| 423 | quax463 quantize 9.999E-15 1e-9 -> 0E-9 Inexact Rounded |
| 424 | quax464 quantize 9.999E-15 1e-8 -> 0E-8 Inexact Rounded |
| 425 | quax465 quantize 9.999E-15 1e-7 -> 0E-7 Inexact Rounded |
| 426 | quax466 quantize 9.999E-15 1e-6 -> 0.000000 Inexact Rounded |
| 427 | quax467 quantize 9.999E-15 1e-5 -> 0.00000 Inexact Rounded |
| 428 | quax468 quantize 9.999E-15 1e-4 -> 0.0000 Inexact Rounded |
| 429 | quax469 quantize 9.999E-15 1e-3 -> 0.000 Inexact Rounded |
| 430 | quax470 quantize 9.999E-15 1e-2 -> 0.00 Inexact Rounded |
| 431 | quax471 quantize 9.999E-15 1e-1 -> 0.0 Inexact Rounded |
| 432 | quax472 quantize 9.999E-15 1e0 -> 0 Inexact Rounded |
| 433 | quax473 quantize 9.999E-15 1e1 -> 0E+1 Inexact Rounded |
| 434 | |
| Facundo Batista | 353750c | 2007-09-13 18:13:15 +0000 | [diff] [blame] | 435 | precision: 7 |
| 436 | quax900 quantize 9.999E-15 1e-22 -> NaN Invalid_operation |
| 437 | quax901 quantize 9.999E-15 1e-21 -> 9.999000E-15 |
| 438 | quax902 quantize 9.999E-15 1e-20 -> 9.99900E-15 |
| 439 | quax903 quantize 9.999E-15 1e-19 -> 9.9990E-15 |
| 440 | quax904 quantize 9.999E-15 1e-18 -> 9.999E-15 |
| 441 | quax905 quantize 9.999E-15 1e-17 -> 1.000E-14 Inexact Rounded |
| 442 | quax906 quantize 9.999E-15 1e-16 -> 1.00E-14 Inexact Rounded |
| 443 | quax907 quantize 9.999E-15 1e-15 -> 1.0E-14 Inexact Rounded |
| 444 | quax908 quantize 9.999E-15 1e-14 -> 1E-14 Inexact Rounded |
| 445 | quax909 quantize 9.999E-15 1e-13 -> 0E-13 Inexact Rounded |
| 446 | quax910 quantize 9.999E-15 1e-12 -> 0E-12 Inexact Rounded |
| 447 | quax911 quantize 9.999E-15 1e-11 -> 0E-11 Inexact Rounded |
| 448 | quax912 quantize 9.999E-15 1e-10 -> 0E-10 Inexact Rounded |
| 449 | quax913 quantize 9.999E-15 1e-9 -> 0E-9 Inexact Rounded |
| 450 | quax914 quantize 9.999E-15 1e-8 -> 0E-8 Inexact Rounded |
| 451 | quax915 quantize 9.999E-15 1e-7 -> 0E-7 Inexact Rounded |
| 452 | quax916 quantize 9.999E-15 1e-6 -> 0.000000 Inexact Rounded |
| 453 | quax917 quantize 9.999E-15 1e-5 -> 0.00000 Inexact Rounded |
| 454 | quax918 quantize 9.999E-15 1e-4 -> 0.0000 Inexact Rounded |
| 455 | quax919 quantize 9.999E-15 1e-3 -> 0.000 Inexact Rounded |
| 456 | quax920 quantize 9.999E-15 1e-2 -> 0.00 Inexact Rounded |
| 457 | quax921 quantize 9.999E-15 1e-1 -> 0.0 Inexact Rounded |
| 458 | quax922 quantize 9.999E-15 1e0 -> 0 Inexact Rounded |
| 459 | quax923 quantize 9.999E-15 1e1 -> 0E+1 Inexact Rounded |
| 460 | |
| 461 | precision: 6 |
| 462 | quax930 quantize 9.999E-15 1e-22 -> NaN Invalid_operation |
| 463 | quax931 quantize 9.999E-15 1e-21 -> NaN Invalid_operation |
| 464 | quax932 quantize 9.999E-15 1e-20 -> 9.99900E-15 |
| 465 | quax933 quantize 9.999E-15 1e-19 -> 9.9990E-15 |
| 466 | quax934 quantize 9.999E-15 1e-18 -> 9.999E-15 |
| 467 | quax935 quantize 9.999E-15 1e-17 -> 1.000E-14 Inexact Rounded |
| 468 | quax936 quantize 9.999E-15 1e-16 -> 1.00E-14 Inexact Rounded |
| 469 | quax937 quantize 9.999E-15 1e-15 -> 1.0E-14 Inexact Rounded |
| 470 | quax938 quantize 9.999E-15 1e-14 -> 1E-14 Inexact Rounded |
| 471 | quax939 quantize 9.999E-15 1e-13 -> 0E-13 Inexact Rounded |
| 472 | quax940 quantize 9.999E-15 1e-12 -> 0E-12 Inexact Rounded |
| 473 | quax941 quantize 9.999E-15 1e-11 -> 0E-11 Inexact Rounded |
| 474 | quax942 quantize 9.999E-15 1e-10 -> 0E-10 Inexact Rounded |
| 475 | quax943 quantize 9.999E-15 1e-9 -> 0E-9 Inexact Rounded |
| 476 | quax944 quantize 9.999E-15 1e-8 -> 0E-8 Inexact Rounded |
| 477 | quax945 quantize 9.999E-15 1e-7 -> 0E-7 Inexact Rounded |
| 478 | quax946 quantize 9.999E-15 1e-6 -> 0.000000 Inexact Rounded |
| 479 | quax947 quantize 9.999E-15 1e-5 -> 0.00000 Inexact Rounded |
| 480 | quax948 quantize 9.999E-15 1e-4 -> 0.0000 Inexact Rounded |
| 481 | quax949 quantize 9.999E-15 1e-3 -> 0.000 Inexact Rounded |
| 482 | quax950 quantize 9.999E-15 1e-2 -> 0.00 Inexact Rounded |
| 483 | quax951 quantize 9.999E-15 1e-1 -> 0.0 Inexact Rounded |
| 484 | quax952 quantize 9.999E-15 1e0 -> 0 Inexact Rounded |
| 485 | quax953 quantize 9.999E-15 1e1 -> 0E+1 Inexact Rounded |
| 486 | |
| 487 | precision: 3 |
| 488 | quax960 quantize 9.999E-15 1e-22 -> NaN Invalid_operation |
| 489 | quax961 quantize 9.999E-15 1e-21 -> NaN Invalid_operation |
| 490 | quax962 quantize 9.999E-15 1e-20 -> NaN Invalid_operation |
| 491 | quax963 quantize 9.999E-15 1e-19 -> NaN Invalid_operation |
| 492 | quax964 quantize 9.999E-15 1e-18 -> NaN Invalid_operation |
| 493 | quax965 quantize 9.999E-15 1e-17 -> NaN Invalid_operation |
| 494 | quax966 quantize 9.999E-15 1e-16 -> 1.00E-14 Inexact Rounded |
| 495 | quax967 quantize 9.999E-15 1e-15 -> 1.0E-14 Inexact Rounded |
| 496 | quax968 quantize 9.999E-15 1e-14 -> 1E-14 Inexact Rounded |
| 497 | quax969 quantize 9.999E-15 1e-13 -> 0E-13 Inexact Rounded |
| 498 | quax970 quantize 9.999E-15 1e-12 -> 0E-12 Inexact Rounded |
| 499 | quax971 quantize 9.999E-15 1e-11 -> 0E-11 Inexact Rounded |
| 500 | quax972 quantize 9.999E-15 1e-10 -> 0E-10 Inexact Rounded |
| 501 | quax973 quantize 9.999E-15 1e-9 -> 0E-9 Inexact Rounded |
| 502 | quax974 quantize 9.999E-15 1e-8 -> 0E-8 Inexact Rounded |
| 503 | quax975 quantize 9.999E-15 1e-7 -> 0E-7 Inexact Rounded |
| 504 | quax976 quantize 9.999E-15 1e-6 -> 0.000000 Inexact Rounded |
| 505 | quax977 quantize 9.999E-15 1e-5 -> 0.00000 Inexact Rounded |
| 506 | quax978 quantize 9.999E-15 1e-4 -> 0.0000 Inexact Rounded |
| 507 | quax979 quantize 9.999E-15 1e-3 -> 0.000 Inexact Rounded |
| 508 | quax980 quantize 9.999E-15 1e-2 -> 0.00 Inexact Rounded |
| 509 | quax981 quantize 9.999E-15 1e-1 -> 0.0 Inexact Rounded |
| 510 | quax982 quantize 9.999E-15 1e0 -> 0 Inexact Rounded |
| 511 | quax983 quantize 9.999E-15 1e1 -> 0E+1 Inexact Rounded |
| 512 | |
| 513 | -- Fung Lee's case & similar |
| 514 | precision: 3 |
| 515 | quax1001 quantize 0.000 0.001 -> 0.000 |
| 516 | quax1002 quantize 0.001 0.001 -> 0.001 |
| 517 | quax1003 quantize 0.0012 0.001 -> 0.001 Inexact Rounded |
| 518 | quax1004 quantize 0.0018 0.001 -> 0.002 Inexact Rounded |
| 519 | quax1005 quantize 0.501 0.001 -> 0.501 |
| 520 | quax1006 quantize 0.5012 0.001 -> 0.501 Inexact Rounded |
| 521 | quax1007 quantize 0.5018 0.001 -> 0.502 Inexact Rounded |
| 522 | quax1008 quantize 0.999 0.001 -> 0.999 |
| 523 | quax1009 quantize 0.9992 0.001 -> 0.999 Inexact Rounded |
| 524 | quax1010 quantize 0.9998 0.001 -> NaN Invalid_operation |
| 525 | quax1011 quantize 1.0001 0.001 -> NaN Invalid_operation |
| 526 | quax1012 quantize 1.0051 0.001 -> NaN Invalid_operation |
| 527 | quax1013 quantize 1.0551 0.001 -> NaN Invalid_operation |
| 528 | quax1014 quantize 1.5551 0.001 -> NaN Invalid_operation |
| 529 | quax1015 quantize 1.9999 0.001 -> NaN Invalid_operation |
| 530 | |
| Raymond Hettinger | 7c85fa4 | 2004-07-01 11:01:35 +0000 | [diff] [blame] | 531 | -- long operand checks [rhs checks removed] |
| 532 | maxexponent: 999 |
| 533 | minexponent: -999 |
| 534 | precision: 9 |
| 535 | quax481 quantize 12345678000 1e+3 -> 1.2345678E+10 Rounded |
| 536 | quax482 quantize 1234567800 1e+1 -> 1.23456780E+9 Rounded |
| 537 | quax483 quantize 1234567890 1e+1 -> 1.23456789E+9 Rounded |
| 538 | quax484 quantize 1234567891 1e+1 -> 1.23456789E+9 Inexact Rounded |
| 539 | quax485 quantize 12345678901 1e+2 -> 1.23456789E+10 Inexact Rounded |
| 540 | quax486 quantize 1234567896 1e+1 -> 1.23456790E+9 Inexact Rounded |
| 541 | -- a potential double-round |
| 542 | quax487 quantize 1234.987643 1e-4 -> 1234.9876 Inexact Rounded |
| 543 | quax488 quantize 1234.987647 1e-4 -> 1234.9876 Inexact Rounded |
| 544 | |
| 545 | precision: 15 |
| 546 | quax491 quantize 12345678000 1e+3 -> 1.2345678E+10 Rounded |
| 547 | quax492 quantize 1234567800 1e+1 -> 1.23456780E+9 Rounded |
| 548 | quax493 quantize 1234567890 1e+1 -> 1.23456789E+9 Rounded |
| 549 | quax494 quantize 1234567891 1e+1 -> 1.23456789E+9 Inexact Rounded |
| 550 | quax495 quantize 12345678901 1e+2 -> 1.23456789E+10 Inexact Rounded |
| 551 | quax496 quantize 1234567896 1e+1 -> 1.23456790E+9 Inexact Rounded |
| 552 | quax497 quantize 1234.987643 1e-4 -> 1234.9876 Inexact Rounded |
| 553 | quax498 quantize 1234.987647 1e-4 -> 1234.9876 Inexact Rounded |
| 554 | |
| 555 | -- Zeros |
| 556 | quax500 quantize 0 1e1 -> 0E+1 |
| 557 | quax501 quantize 0 1e0 -> 0 |
| 558 | quax502 quantize 0 1e-1 -> 0.0 |
| 559 | quax503 quantize 0.0 1e-1 -> 0.0 |
| 560 | quax504 quantize 0.0 1e0 -> 0 |
| 561 | quax505 quantize 0.0 1e+1 -> 0E+1 |
| 562 | quax506 quantize 0E+1 1e-1 -> 0.0 |
| 563 | quax507 quantize 0E+1 1e0 -> 0 |
| 564 | quax508 quantize 0E+1 1e+1 -> 0E+1 |
| 565 | quax509 quantize -0 1e1 -> -0E+1 |
| 566 | quax510 quantize -0 1e0 -> -0 |
| 567 | quax511 quantize -0 1e-1 -> -0.0 |
| 568 | quax512 quantize -0.0 1e-1 -> -0.0 |
| 569 | quax513 quantize -0.0 1e0 -> -0 |
| 570 | quax514 quantize -0.0 1e+1 -> -0E+1 |
| 571 | quax515 quantize -0E+1 1e-1 -> -0.0 |
| 572 | quax516 quantize -0E+1 1e0 -> -0 |
| 573 | quax517 quantize -0E+1 1e+1 -> -0E+1 |
| 574 | |
| 575 | -- Suspicious RHS values |
| 576 | maxexponent: 999999999 |
| 577 | minexponent: -999999999 |
| 578 | precision: 15 |
| 579 | quax520 quantize 1.234 1e999999000 -> 0E+999999000 Inexact Rounded |
| 580 | quax521 quantize 123.456 1e999999000 -> 0E+999999000 Inexact Rounded |
| 581 | quax522 quantize 1.234 1e999999999 -> 0E+999999999 Inexact Rounded |
| 582 | quax523 quantize 123.456 1e999999999 -> 0E+999999999 Inexact Rounded |
| 583 | quax524 quantize 123.456 1e1000000000 -> NaN Invalid_operation |
| 584 | quax525 quantize 123.456 1e12345678903 -> NaN Invalid_operation |
| 585 | -- next four are "won't fit" overflows |
| 586 | quax526 quantize 1.234 1e-999999000 -> NaN Invalid_operation |
| 587 | quax527 quantize 123.456 1e-999999000 -> NaN Invalid_operation |
| 588 | quax528 quantize 1.234 1e-999999999 -> NaN Invalid_operation |
| 589 | quax529 quantize 123.456 1e-999999999 -> NaN Invalid_operation |
| 590 | quax530 quantize 123.456 1e-1000000014 -> NaN Invalid_operation |
| 591 | quax531 quantize 123.456 1e-12345678903 -> NaN Invalid_operation |
| 592 | |
| 593 | maxexponent: 999 |
| 594 | minexponent: -999 |
| 595 | precision: 15 |
| 596 | quax532 quantize 1.234E+999 1e999 -> 1E+999 Inexact Rounded |
| 597 | quax533 quantize 1.234E+998 1e999 -> 0E+999 Inexact Rounded |
| 598 | quax534 quantize 1.234 1e999 -> 0E+999 Inexact Rounded |
| 599 | quax535 quantize 1.234 1e1000 -> NaN Invalid_operation |
| 600 | quax536 quantize 1.234 1e5000 -> NaN Invalid_operation |
| 601 | quax537 quantize 0 1e-999 -> 0E-999 |
| 602 | -- next two are "won't fit" overflows |
| 603 | quax538 quantize 1.234 1e-999 -> NaN Invalid_operation |
| 604 | quax539 quantize 1.234 1e-1000 -> NaN Invalid_operation |
| 605 | quax540 quantize 1.234 1e-5000 -> NaN Invalid_operation |
| 606 | -- [more below] |
| 607 | |
| 608 | -- check bounds (lhs maybe out of range for destination, etc.) |
| 609 | precision: 7 |
| 610 | quax541 quantize 1E+999 1e+999 -> 1E+999 |
| 611 | quax542 quantize 1E+1000 1e+999 -> NaN Invalid_operation |
| 612 | quax543 quantize 1E+999 1e+1000 -> NaN Invalid_operation |
| 613 | quax544 quantize 1E-999 1e-999 -> 1E-999 |
| 614 | quax545 quantize 1E-1000 1e-999 -> 0E-999 Inexact Rounded |
| 615 | quax546 quantize 1E-999 1e-1000 -> 1.0E-999 |
| 616 | quax547 quantize 1E-1005 1e-999 -> 0E-999 Inexact Rounded |
| 617 | quax548 quantize 1E-1006 1e-999 -> 0E-999 Inexact Rounded |
| 618 | quax549 quantize 1E-1007 1e-999 -> 0E-999 Inexact Rounded |
| 619 | quax550 quantize 1E-998 1e-1005 -> NaN Invalid_operation -- won't fit |
| 620 | quax551 quantize 1E-999 1e-1005 -> 1.000000E-999 |
| 621 | quax552 quantize 1E-1000 1e-1005 -> 1.00000E-1000 Subnormal |
| 622 | quax553 quantize 1E-999 1e-1006 -> NaN Invalid_operation |
| 623 | quax554 quantize 1E-999 1e-1007 -> NaN Invalid_operation |
| 624 | -- related subnormal rounding |
| 625 | quax555 quantize 1.666666E-999 1e-1005 -> 1.666666E-999 |
| 626 | quax556 quantize 1.666666E-1000 1e-1005 -> 1.66667E-1000 Subnormal Inexact Rounded |
| 627 | quax557 quantize 1.666666E-1001 1e-1005 -> 1.6667E-1001 Subnormal Inexact Rounded |
| 628 | quax558 quantize 1.666666E-1002 1e-1005 -> 1.667E-1002 Subnormal Inexact Rounded |
| 629 | quax559 quantize 1.666666E-1003 1e-1005 -> 1.67E-1003 Subnormal Inexact Rounded |
| 630 | quax560 quantize 1.666666E-1004 1e-1005 -> 1.7E-1004 Subnormal Inexact Rounded |
| 631 | quax561 quantize 1.666666E-1005 1e-1005 -> 2E-1005 Subnormal Inexact Rounded |
| 632 | quax562 quantize 1.666666E-1006 1e-1005 -> 0E-1005 Inexact Rounded |
| 633 | quax563 quantize 1.666666E-1007 1e-1005 -> 0E-1005 Inexact Rounded |
| 634 | |
| 635 | -- Specials |
| 636 | quax580 quantize Inf -Inf -> Infinity |
| 637 | quax581 quantize Inf 1e-1000 -> NaN Invalid_operation |
| 638 | quax582 quantize Inf 1e-1 -> NaN Invalid_operation |
| 639 | quax583 quantize Inf 1e0 -> NaN Invalid_operation |
| 640 | quax584 quantize Inf 1e1 -> NaN Invalid_operation |
| 641 | quax585 quantize Inf 1e1000 -> NaN Invalid_operation |
| 642 | quax586 quantize Inf Inf -> Infinity |
| 643 | quax587 quantize -1000 Inf -> NaN Invalid_operation |
| 644 | quax588 quantize -Inf Inf -> -Infinity |
| 645 | quax589 quantize -1 Inf -> NaN Invalid_operation |
| 646 | quax590 quantize 0 Inf -> NaN Invalid_operation |
| 647 | quax591 quantize 1 Inf -> NaN Invalid_operation |
| 648 | quax592 quantize 1000 Inf -> NaN Invalid_operation |
| 649 | quax593 quantize Inf Inf -> Infinity |
| 650 | quax594 quantize Inf 1e-0 -> NaN Invalid_operation |
| 651 | quax595 quantize -0 Inf -> NaN Invalid_operation |
| 652 | |
| 653 | quax600 quantize -Inf -Inf -> -Infinity |
| 654 | quax601 quantize -Inf 1e-1000 -> NaN Invalid_operation |
| 655 | quax602 quantize -Inf 1e-1 -> NaN Invalid_operation |
| 656 | quax603 quantize -Inf 1e0 -> NaN Invalid_operation |
| 657 | quax604 quantize -Inf 1e1 -> NaN Invalid_operation |
| 658 | quax605 quantize -Inf 1e1000 -> NaN Invalid_operation |
| 659 | quax606 quantize -Inf Inf -> -Infinity |
| 660 | quax607 quantize -1000 Inf -> NaN Invalid_operation |
| 661 | quax608 quantize -Inf -Inf -> -Infinity |
| 662 | quax609 quantize -1 -Inf -> NaN Invalid_operation |
| 663 | quax610 quantize 0 -Inf -> NaN Invalid_operation |
| 664 | quax611 quantize 1 -Inf -> NaN Invalid_operation |
| 665 | quax612 quantize 1000 -Inf -> NaN Invalid_operation |
| 666 | quax613 quantize Inf -Inf -> Infinity |
| 667 | quax614 quantize -Inf 1e-0 -> NaN Invalid_operation |
| 668 | quax615 quantize -0 -Inf -> NaN Invalid_operation |
| 669 | |
| 670 | quax621 quantize NaN -Inf -> NaN |
| 671 | quax622 quantize NaN 1e-1000 -> NaN |
| 672 | quax623 quantize NaN 1e-1 -> NaN |
| 673 | quax624 quantize NaN 1e0 -> NaN |
| 674 | quax625 quantize NaN 1e1 -> NaN |
| 675 | quax626 quantize NaN 1e1000 -> NaN |
| 676 | quax627 quantize NaN Inf -> NaN |
| 677 | quax628 quantize NaN NaN -> NaN |
| 678 | quax629 quantize -Inf NaN -> NaN |
| 679 | quax630 quantize -1000 NaN -> NaN |
| 680 | quax631 quantize -1 NaN -> NaN |
| 681 | quax632 quantize 0 NaN -> NaN |
| 682 | quax633 quantize 1 NaN -> NaN |
| 683 | quax634 quantize 1000 NaN -> NaN |
| 684 | quax635 quantize Inf NaN -> NaN |
| 685 | quax636 quantize NaN 1e-0 -> NaN |
| 686 | quax637 quantize -0 NaN -> NaN |
| 687 | |
| 688 | quax641 quantize sNaN -Inf -> NaN Invalid_operation |
| 689 | quax642 quantize sNaN 1e-1000 -> NaN Invalid_operation |
| 690 | quax643 quantize sNaN 1e-1 -> NaN Invalid_operation |
| 691 | quax644 quantize sNaN 1e0 -> NaN Invalid_operation |
| 692 | quax645 quantize sNaN 1e1 -> NaN Invalid_operation |
| 693 | quax646 quantize sNaN 1e1000 -> NaN Invalid_operation |
| 694 | quax647 quantize sNaN NaN -> NaN Invalid_operation |
| 695 | quax648 quantize sNaN sNaN -> NaN Invalid_operation |
| 696 | quax649 quantize NaN sNaN -> NaN Invalid_operation |
| 697 | quax650 quantize -Inf sNaN -> NaN Invalid_operation |
| 698 | quax651 quantize -1000 sNaN -> NaN Invalid_operation |
| 699 | quax652 quantize -1 sNaN -> NaN Invalid_operation |
| 700 | quax653 quantize 0 sNaN -> NaN Invalid_operation |
| 701 | quax654 quantize 1 sNaN -> NaN Invalid_operation |
| 702 | quax655 quantize 1000 sNaN -> NaN Invalid_operation |
| 703 | quax656 quantize Inf sNaN -> NaN Invalid_operation |
| 704 | quax657 quantize NaN sNaN -> NaN Invalid_operation |
| 705 | quax658 quantize sNaN 1e-0 -> NaN Invalid_operation |
| 706 | quax659 quantize -0 sNaN -> NaN Invalid_operation |
| 707 | |
| 708 | -- propagating NaNs |
| 709 | quax661 quantize NaN9 -Inf -> NaN9 |
| 710 | quax662 quantize NaN8 919 -> NaN8 |
| 711 | quax663 quantize NaN71 Inf -> NaN71 |
| 712 | quax664 quantize NaN6 NaN5 -> NaN6 |
| 713 | quax665 quantize -Inf NaN4 -> NaN4 |
| 714 | quax666 quantize -919 NaN31 -> NaN31 |
| 715 | quax667 quantize Inf NaN2 -> NaN2 |
| 716 | |
| 717 | quax671 quantize sNaN99 -Inf -> NaN99 Invalid_operation |
| 718 | quax672 quantize sNaN98 -11 -> NaN98 Invalid_operation |
| 719 | quax673 quantize sNaN97 NaN -> NaN97 Invalid_operation |
| 720 | quax674 quantize sNaN16 sNaN94 -> NaN16 Invalid_operation |
| 721 | quax675 quantize NaN95 sNaN93 -> NaN93 Invalid_operation |
| 722 | quax676 quantize -Inf sNaN92 -> NaN92 Invalid_operation |
| 723 | quax677 quantize 088 sNaN91 -> NaN91 Invalid_operation |
| 724 | quax678 quantize Inf sNaN90 -> NaN90 Invalid_operation |
| 725 | quax679 quantize NaN sNaN88 -> NaN88 Invalid_operation |
| 726 | |
| 727 | quax681 quantize -NaN9 -Inf -> -NaN9 |
| 728 | quax682 quantize -NaN8 919 -> -NaN8 |
| 729 | quax683 quantize -NaN71 Inf -> -NaN71 |
| 730 | quax684 quantize -NaN6 -NaN5 -> -NaN6 |
| 731 | quax685 quantize -Inf -NaN4 -> -NaN4 |
| 732 | quax686 quantize -919 -NaN31 -> -NaN31 |
| 733 | quax687 quantize Inf -NaN2 -> -NaN2 |
| 734 | |
| 735 | quax691 quantize -sNaN99 -Inf -> -NaN99 Invalid_operation |
| 736 | quax692 quantize -sNaN98 -11 -> -NaN98 Invalid_operation |
| 737 | quax693 quantize -sNaN97 NaN -> -NaN97 Invalid_operation |
| 738 | quax694 quantize -sNaN16 sNaN94 -> -NaN16 Invalid_operation |
| 739 | quax695 quantize -NaN95 -sNaN93 -> -NaN93 Invalid_operation |
| 740 | quax696 quantize -Inf -sNaN92 -> -NaN92 Invalid_operation |
| 741 | quax697 quantize 088 -sNaN91 -> -NaN91 Invalid_operation |
| 742 | quax698 quantize Inf -sNaN90 -> -NaN90 Invalid_operation |
| 743 | quax699 quantize NaN -sNaN88 -> -NaN88 Invalid_operation |
| 744 | |
| 745 | -- subnormals and underflow |
| 746 | precision: 4 |
| 747 | maxexponent: 999 |
| 748 | minexponent: -999 |
| 749 | quax710 quantize 1.00E-999 1e-999 -> 1E-999 Rounded |
| 750 | quax711 quantize 0.1E-999 2e-1000 -> 1E-1000 Subnormal |
| 751 | quax712 quantize 0.10E-999 3e-1000 -> 1E-1000 Subnormal Rounded |
| 752 | quax713 quantize 0.100E-999 4e-1000 -> 1E-1000 Subnormal Rounded |
| 753 | quax714 quantize 0.01E-999 5e-1001 -> 1E-1001 Subnormal |
| 754 | -- next is rounded to Emin |
| 755 | quax715 quantize 0.999E-999 1e-999 -> 1E-999 Inexact Rounded |
| 756 | quax716 quantize 0.099E-999 10e-1000 -> 1E-1000 Inexact Rounded Subnormal |
| 757 | |
| 758 | quax717 quantize 0.009E-999 1e-1001 -> 1E-1001 Inexact Rounded Subnormal |
| 759 | quax718 quantize 0.001E-999 1e-1001 -> 0E-1001 Inexact Rounded |
| 760 | quax719 quantize 0.0009E-999 1e-1001 -> 0E-1001 Inexact Rounded |
| 761 | quax720 quantize 0.0001E-999 1e-1001 -> 0E-1001 Inexact Rounded |
| 762 | |
| 763 | quax730 quantize -1.00E-999 1e-999 -> -1E-999 Rounded |
| 764 | quax731 quantize -0.1E-999 1e-999 -> -0E-999 Rounded Inexact |
| 765 | quax732 quantize -0.10E-999 1e-999 -> -0E-999 Rounded Inexact |
| 766 | quax733 quantize -0.100E-999 1e-999 -> -0E-999 Rounded Inexact |
| 767 | quax734 quantize -0.01E-999 1e-999 -> -0E-999 Inexact Rounded |
| 768 | -- next is rounded to Emin |
| 769 | quax735 quantize -0.999E-999 90e-999 -> -1E-999 Inexact Rounded |
| 770 | quax736 quantize -0.099E-999 -1e-999 -> -0E-999 Inexact Rounded |
| 771 | quax737 quantize -0.009E-999 -1e-999 -> -0E-999 Inexact Rounded |
| 772 | quax738 quantize -0.001E-999 -0e-999 -> -0E-999 Inexact Rounded |
| 773 | quax739 quantize -0.0001E-999 0e-999 -> -0E-999 Inexact Rounded |
| 774 | |
| 775 | quax740 quantize -1.00E-999 1e-1000 -> -1.0E-999 Rounded |
| 776 | quax741 quantize -0.1E-999 1e-1000 -> -1E-1000 Subnormal |
| 777 | quax742 quantize -0.10E-999 1e-1000 -> -1E-1000 Subnormal Rounded |
| 778 | quax743 quantize -0.100E-999 1e-1000 -> -1E-1000 Subnormal Rounded |
| 779 | quax744 quantize -0.01E-999 1e-1000 -> -0E-1000 Inexact Rounded |
| 780 | -- next is rounded to Emin |
| 781 | quax745 quantize -0.999E-999 1e-1000 -> -1.0E-999 Inexact Rounded |
| 782 | quax746 quantize -0.099E-999 1e-1000 -> -1E-1000 Inexact Rounded Subnormal |
| 783 | quax747 quantize -0.009E-999 1e-1000 -> -0E-1000 Inexact Rounded |
| 784 | quax748 quantize -0.001E-999 1e-1000 -> -0E-1000 Inexact Rounded |
| 785 | quax749 quantize -0.0001E-999 1e-1000 -> -0E-1000 Inexact Rounded |
| 786 | |
| 787 | quax750 quantize -1.00E-999 1e-1001 -> -1.00E-999 |
| 788 | quax751 quantize -0.1E-999 1e-1001 -> -1.0E-1000 Subnormal |
| 789 | quax752 quantize -0.10E-999 1e-1001 -> -1.0E-1000 Subnormal |
| 790 | quax753 quantize -0.100E-999 1e-1001 -> -1.0E-1000 Subnormal Rounded |
| 791 | quax754 quantize -0.01E-999 1e-1001 -> -1E-1001 Subnormal |
| 792 | -- next is rounded to Emin |
| 793 | quax755 quantize -0.999E-999 1e-1001 -> -1.00E-999 Inexact Rounded |
| 794 | quax756 quantize -0.099E-999 1e-1001 -> -1.0E-1000 Inexact Rounded Subnormal |
| 795 | quax757 quantize -0.009E-999 1e-1001 -> -1E-1001 Inexact Rounded Subnormal |
| 796 | quax758 quantize -0.001E-999 1e-1001 -> -0E-1001 Inexact Rounded |
| 797 | quax759 quantize -0.0001E-999 1e-1001 -> -0E-1001 Inexact Rounded |
| 798 | |
| 799 | quax760 quantize -1.00E-999 1e-1002 -> -1.000E-999 |
| 800 | quax761 quantize -0.1E-999 1e-1002 -> -1.00E-1000 Subnormal |
| 801 | quax762 quantize -0.10E-999 1e-1002 -> -1.00E-1000 Subnormal |
| 802 | quax763 quantize -0.100E-999 1e-1002 -> -1.00E-1000 Subnormal |
| 803 | quax764 quantize -0.01E-999 1e-1002 -> -1.0E-1001 Subnormal |
| 804 | quax765 quantize -0.999E-999 1e-1002 -> -9.99E-1000 Subnormal |
| 805 | quax766 quantize -0.099E-999 1e-1002 -> -9.9E-1001 Subnormal |
| 806 | quax767 quantize -0.009E-999 1e-1002 -> -9E-1002 Subnormal |
| 807 | quax768 quantize -0.001E-999 1e-1002 -> -1E-1002 Subnormal |
| 808 | quax769 quantize -0.0001E-999 1e-1002 -> -0E-1002 Inexact Rounded |
| 809 | |
| 810 | -- rhs must be no less than Etiny |
| 811 | quax770 quantize -1.00E-999 1e-1003 -> NaN Invalid_operation |
| 812 | quax771 quantize -0.1E-999 1e-1003 -> NaN Invalid_operation |
| 813 | quax772 quantize -0.10E-999 1e-1003 -> NaN Invalid_operation |
| 814 | quax773 quantize -0.100E-999 1e-1003 -> NaN Invalid_operation |
| 815 | quax774 quantize -0.01E-999 1e-1003 -> NaN Invalid_operation |
| 816 | quax775 quantize -0.999E-999 1e-1003 -> NaN Invalid_operation |
| 817 | quax776 quantize -0.099E-999 1e-1003 -> NaN Invalid_operation |
| 818 | quax777 quantize -0.009E-999 1e-1003 -> NaN Invalid_operation |
| 819 | quax778 quantize -0.001E-999 1e-1003 -> NaN Invalid_operation |
| 820 | quax779 quantize -0.0001E-999 1e-1003 -> NaN Invalid_operation |
| 821 | quax780 quantize -0.0001E-999 1e-1004 -> NaN Invalid_operation |
| 822 | |
| 823 | precision: 9 |
| 824 | maxExponent: 999999999 |
| 825 | minexponent: -999999999 |
| 826 | |
| 827 | -- some extremes derived from Rescale testcases |
| 828 | quax801 quantize 0 1e1000000000 -> NaN Invalid_operation |
| 829 | quax802 quantize 0 1e-1000000000 -> 0E-1000000000 |
| 830 | quax803 quantize 0 1e2000000000 -> NaN Invalid_operation |
| 831 | quax804 quantize 0 1e-2000000000 -> NaN Invalid_operation |
| 832 | quax805 quantize 0 1e3000000000 -> NaN Invalid_operation |
| 833 | quax806 quantize 0 1e-3000000000 -> NaN Invalid_operation |
| 834 | quax807 quantize 0 1e4000000000 -> NaN Invalid_operation |
| 835 | quax808 quantize 0 1e-4000000000 -> NaN Invalid_operation |
| 836 | quax809 quantize 0 1e5000000000 -> NaN Invalid_operation |
| 837 | quax810 quantize 0 1e-5000000000 -> NaN Invalid_operation |
| 838 | quax811 quantize 0 1e6000000000 -> NaN Invalid_operation |
| 839 | quax812 quantize 0 1e-6000000000 -> NaN Invalid_operation |
| 840 | quax813 quantize 0 1e7000000000 -> NaN Invalid_operation |
| 841 | quax814 quantize 0 1e-7000000000 -> NaN Invalid_operation |
| 842 | quax815 quantize 0 1e8000000000 -> NaN Invalid_operation |
| 843 | quax816 quantize 0 1e-8000000000 -> NaN Invalid_operation |
| 844 | quax817 quantize 0 1e9000000000 -> NaN Invalid_operation |
| 845 | quax818 quantize 0 1e-9000000000 -> NaN Invalid_operation |
| 846 | quax819 quantize 0 1e9999999999 -> NaN Invalid_operation |
| 847 | quax820 quantize 0 1e-9999999999 -> NaN Invalid_operation |
| 848 | quax821 quantize 0 1e10000000000 -> NaN Invalid_operation |
| 849 | quax822 quantize 0 1e-10000000000 -> NaN Invalid_operation |
| 850 | |
| 851 | quax843 quantize 0 1e999999999 -> 0E+999999999 |
| 852 | quax844 quantize 0 1e1000000000 -> NaN Invalid_operation |
| 853 | quax845 quantize 0 1e-999999999 -> 0E-999999999 |
| 854 | quax846 quantize 0 1e-1000000000 -> 0E-1000000000 |
| 855 | quax847 quantize 0 1e-1000000001 -> 0E-1000000001 |
| 856 | quax848 quantize 0 1e-1000000002 -> 0E-1000000002 |
| 857 | quax849 quantize 0 1e-1000000003 -> 0E-1000000003 |
| 858 | quax850 quantize 0 1e-1000000004 -> 0E-1000000004 |
| 859 | quax851 quantize 0 1e-1000000005 -> 0E-1000000005 |
| 860 | quax852 quantize 0 1e-1000000006 -> 0E-1000000006 |
| 861 | quax853 quantize 0 1e-1000000007 -> 0E-1000000007 |
| 862 | quax854 quantize 0 1e-1000000008 -> NaN Invalid_operation |
| 863 | |
| 864 | quax861 quantize 1 1e+2147483649 -> NaN Invalid_operation |
| 865 | quax862 quantize 1 1e+2147483648 -> NaN Invalid_operation |
| 866 | quax863 quantize 1 1e+2147483647 -> NaN Invalid_operation |
| 867 | quax864 quantize 1 1e-2147483647 -> NaN Invalid_operation |
| 868 | quax865 quantize 1 1e-2147483648 -> NaN Invalid_operation |
| 869 | quax866 quantize 1 1e-2147483649 -> NaN Invalid_operation |
| 870 | |
| Facundo Batista | 353750c | 2007-09-13 18:13:15 +0000 | [diff] [blame] | 871 | -- More from Fung Lee |
| 872 | precision: 16 |
| 873 | rounding: half_up |
| 874 | maxExponent: 384 |
| 875 | minExponent: -383 |
| 876 | quax1021 quantize 8.666666666666000E+384 1.000000000000000E+384 -> 8.666666666666000E+384 |
| 877 | quax1022 quantize 64#8.666666666666000E+384 64#1.000000000000000E+384 -> 8.666666666666000E+384 |
| 878 | quax1023 quantize 64#8.666666666666000E+384 128#1.000000000000000E+384 -> 8.666666666666000E+384 |
| 879 | quax1024 quantize 64#8.666666666666000E+384 64#1E+384 -> 8.666666666666000E+384 |
| 880 | quax1025 quantize 64#8.666666666666000E+384 64#1E+384 -> 64#8.666666666666000E+384 |
| 881 | quax1026 quantize 64#8.666666666666000E+384 128#1E+384 -> 64#9E+384 Inexact Rounded Clamped |
| 882 | quax1027 quantize 64#8.666666666666000E+323 64#1E+31 -> NaN Invalid_operation |
| 883 | quax1028 quantize 64#8.666666666666000E+323 128#1E+31 -> NaN Invalid_operation |
| 884 | quax1029 quantize 64#8.66666666E+3 128#1E+10 -> 64#0E10 Inexact Rounded |
| 885 | quax1030 quantize 8.66666666E+3 1E+3 -> 9E+3 Inexact Rounded |
| 886 | |
| 887 | -- Int and uInt32 edge values for testing conversions |
| 888 | quax1040 quantize -2147483646 0 -> -2147483646 |
| 889 | quax1041 quantize -2147483647 0 -> -2147483647 |
| 890 | quax1042 quantize -2147483648 0 -> -2147483648 |
| 891 | quax1043 quantize -2147483649 0 -> -2147483649 |
| 892 | quax1044 quantize 2147483646 0 -> 2147483646 |
| 893 | quax1045 quantize 2147483647 0 -> 2147483647 |
| 894 | quax1046 quantize 2147483648 0 -> 2147483648 |
| 895 | quax1047 quantize 2147483649 0 -> 2147483649 |
| 896 | quax1048 quantize 4294967294 0 -> 4294967294 |
| 897 | quax1049 quantize 4294967295 0 -> 4294967295 |
| 898 | quax1050 quantize 4294967296 0 -> 4294967296 |
| 899 | quax1051 quantize 4294967297 0 -> 4294967297 |
| 900 | -- and powers of ten for same |
| 901 | quax1101 quantize 5000000000 0 -> 5000000000 |
| 902 | quax1102 quantize 4000000000 0 -> 4000000000 |
| 903 | quax1103 quantize 2000000000 0 -> 2000000000 |
| 904 | quax1104 quantize 1000000000 0 -> 1000000000 |
| 905 | quax1105 quantize 0100000000 0 -> 100000000 |
| 906 | quax1106 quantize 0010000000 0 -> 10000000 |
| 907 | quax1107 quantize 0001000000 0 -> 1000000 |
| 908 | quax1108 quantize 0000100000 0 -> 100000 |
| 909 | quax1109 quantize 0000010000 0 -> 10000 |
| 910 | quax1110 quantize 0000001000 0 -> 1000 |
| 911 | quax1111 quantize 0000000100 0 -> 100 |
| 912 | quax1112 quantize 0000000010 0 -> 10 |
| 913 | quax1113 quantize 0000000001 0 -> 1 |
| 914 | quax1114 quantize 0000000000 0 -> 0 |
| 915 | -- and powers of ten for same |
| 916 | quax1121 quantize -5000000000 0 -> -5000000000 |
| 917 | quax1122 quantize -4000000000 0 -> -4000000000 |
| 918 | quax1123 quantize -2000000000 0 -> -2000000000 |
| 919 | quax1124 quantize -1000000000 0 -> -1000000000 |
| 920 | quax1125 quantize -0100000000 0 -> -100000000 |
| 921 | quax1126 quantize -0010000000 0 -> -10000000 |
| 922 | quax1127 quantize -0001000000 0 -> -1000000 |
| 923 | quax1128 quantize -0000100000 0 -> -100000 |
| 924 | quax1129 quantize -0000010000 0 -> -10000 |
| 925 | quax1130 quantize -0000001000 0 -> -1000 |
| 926 | quax1131 quantize -0000000100 0 -> -100 |
| 927 | quax1132 quantize -0000000010 0 -> -10 |
| 928 | quax1133 quantize -0000000001 0 -> -1 |
| 929 | quax1134 quantize -0000000000 0 -> -0 |
| 930 | |
| 931 | -- Some miscellany |
| 932 | precision: 34 |
| 933 | rounding: half_up |
| 934 | maxExponent: 6144 |
| 935 | minExponent: -6143 |
| 936 | -- 1 2 3 |
| 937 | -- 1 234567890123456789012345678901234 |
| 938 | quax0a1 quantize 8.555555555555555555555555555555555E+6143 1E+6143 -> 9E+6143 Inexact Rounded |
| 939 | quax0a2 quantize 128#8.555555555555555555555555555555555E+6143 128#1E+6143 -> 8.55555555555555555555555555555556E+6143 Rounded Inexact |
| 940 | quax0a3 quantize 128#8.555555555555555555555555555555555E+6144 128#1E+6144 -> 8.555555555555555555555555555555555E+6144 |
| 941 | |
| 942 | -- payload decapitate |
| 943 | precision: 5 |
| 944 | quax62100 quantize 11 -sNaN1234567890 -> -NaN67890 Invalid_operation |
| 945 | |
| Raymond Hettinger | 7c85fa4 | 2004-07-01 11:01:35 +0000 | [diff] [blame] | 946 | -- Null tests |
| Facundo Batista | 353750c | 2007-09-13 18:13:15 +0000 | [diff] [blame] | 947 | quax998 quantize 10 # -> NaN Invalid_operation |
| 948 | quax999 quantize # 1e10 -> NaN Invalid_operation |