Added tests for tokenize, blocked corresponding checkin from trunk.
1 file changed