Marc-Andre's third try at this bulk patch seems to work (except that
his copy of test_contains.py seems to be broken -- the lines he
deleted were already absent). Checkin messages:
New Unicode support for int(), float(), complex() and long().
- new APIs PyInt_FromUnicode() and PyLong_FromUnicode()
- added support for Unicode to PyFloat_FromString()
- new encoding API PyUnicode_EncodeDecimal() which converts
Unicode to a decimal char* string (used in the above new
APIs)
- shortcuts for calls like int(<int object>) and float(<float obj>)
- tests for all of the above
Unicode compares and contains checks:
- comparing Unicode and non-string types now works; TypeErrors
are masked, all other errors such as ValueError during
Unicode coercion are passed through (note that PyUnicode_Compare
does not implement the masking -- PyObject_Compare does this)
- contains now works for non-string types too; TypeErrors are
masked and 0 returned; all other errors are passed through
Better testing support for the standard codecs.
Misc minor enhancements, such as an alias dbcs for the mbcs codec.
Changes:
- PyLong_FromString() now applies the same error checks as
does PyInt_FromString(): trailing garbage is reported
as error and not longer silently ignored. The only characters
which may be trailing the digits are 'L' and 'l' -- these
are still silently ignored.
- string.ato?() now directly interface to int(), long() and
float(). The error strings are now a little different, but
the type still remains the same. These functions are now
ready to get declared obsolete ;-)
- PyNumber_Int() now also does a check for embedded NULL chars
in the input string; PyNumber_Long() already did this (and
still does)
Followed by:
Looks like I've gone a step too far there... (and test_contains.py
seem to have a bug too).
I've changed back to reporting all errors in PyUnicode_Contains()
and added a few more test cases to test_contains.py (plus corrected
the join() NameError).
diff --git a/Include/Python.h b/Include/Python.h
index 4179b2e..044ad5f 100644
--- a/Include/Python.h
+++ b/Include/Python.h
@@ -72,6 +72,7 @@
#include "pydebug.h"
+#include "unicodeobject.h"
#include "intobject.h"
#include "longobject.h"
#include "floatobject.h"
@@ -92,7 +93,6 @@
#include "cobject.h"
#include "traceback.h"
#include "sliceobject.h"
-#include "unicodeobject.h"
#include "codecs.h"
#include "pyerrors.h"
diff --git a/Include/intobject.h b/Include/intobject.h
index 781a4b5..507a9e2 100644
--- a/Include/intobject.h
+++ b/Include/intobject.h
@@ -60,6 +60,7 @@
#define PyInt_Check(op) ((op)->ob_type == &PyInt_Type)
extern DL_IMPORT(PyObject *) PyInt_FromString Py_PROTO((char*, char**, int));
+extern DL_IMPORT(PyObject *) PyInt_FromUnicode Py_PROTO((Py_UNICODE*, int, int));
extern DL_IMPORT(PyObject *) PyInt_FromLong Py_PROTO((long));
extern DL_IMPORT(long) PyInt_AsLong Py_PROTO((PyObject *));
extern DL_IMPORT(long) PyInt_GetMax Py_PROTO((void));
diff --git a/Include/longobject.h b/Include/longobject.h
index 89f2aa3..ca1adb7 100644
--- a/Include/longobject.h
+++ b/Include/longobject.h
@@ -82,6 +82,7 @@
#endif /* HAVE_LONG_LONG */
DL_IMPORT(PyObject *) PyLong_FromString Py_PROTO((char *, char **, int));
+DL_IMPORT(PyObject *) PyLong_FromUnicode Py_PROTO((Py_UNICODE*, int, int));
#ifdef __cplusplus
}
diff --git a/Include/unicodeobject.h b/Include/unicodeobject.h
index cfc8126..e1f5914 100644
--- a/Include/unicodeobject.h
+++ b/Include/unicodeobject.h
@@ -358,7 +358,7 @@
/* --- UTF-16 Codecs ------------------------------------------------------ */
-/* Decodes length bytes from a UTF-16 encoded buffer string and return
+/* Decodes length bytes from a UTF-16 encoded buffer string and returns
the corresponding Unicode object.
errors (if non-NULL) defines the error handling. It defaults
@@ -397,7 +397,7 @@
);
/* Returns a Python string object holding the UTF-16 encoded value of
- the Unicode data in s.
+ the Unicode data.
If byteorder is not 0, output is written according to the following
byte order:
@@ -587,6 +587,37 @@
#endif /* MS_WIN32 */
+/* --- Decimal Encoder ---------------------------------------------------- */
+
+/* Takes a Unicode string holding a decimal value and writes it into
+ an output buffer using standard ASCII digit codes.
+
+ The output buffer has to provide at least length+1 bytes of storage
+ area. The output string is 0-terminated.
+
+ The encoder converts whitespace to ' ', decimal characters to their
+ corresponding ASCII digit and all other Latin-1 characters except
+ \0 as-is. Characters outside this range (Unicode ordinals 1-256)
+ are treated as errors. This includes embedded NULL bytes.
+
+ Error handling is defined by the errors argument:
+
+ NULL or "strict": raise a ValueError
+ "ignore": ignore the wrong characters (these are not copied to the
+ output buffer)
+ "replace": replaces illegal characters with '?'
+
+ Returns 0 on success, -1 on failure.
+
+*/
+
+extern DL_IMPORT(int) PyUnicode_EncodeDecimal(
+ Py_UNICODE *s, /* Unicode buffer */
+ int length, /* Number of Py_UNICODE chars to encode */
+ char *output, /* Output buffer; must have size >= length */
+ const char *errors /* error handling */
+ );
+
/* --- Methods & Slots ----------------------------------------------------
These are capable of handling Unicode objects and strings on input