Implement BreakIterator in terms of icu4j.
Note that we need manual range checks in our wrapper because
the ICU4J implementation will clamp offsets to either end of
the input text.
benchmark results :
text benchmark us linear runtime
LIPSUM BreakIterator 326.02 =====
LIPSUM IcuBreakIterator 333.47 ======
LONGPARA BreakIterator 227.67 ====
LONGPARA IcuBreakIterator 228.68 ====
GERMAN BreakIterator 79.09 =
GERMAN IcuBreakIterator 81.30 =
THAI BreakIterator 1654.11 ==============================
THAI IcuBreakIterator 1646.75 =============================
THAI2 BreakIterator 44.28 =
THAI2 IcuBreakIterator 42.67 =
TABS BreakIterator 12.79 =
TABS IcuBreakIterator 12.53 =
ACCENT BreakIterator 16.26 =
ACCENT IcuBreakIterator 15.37 =
EMOJI BreakIterator 17.71 =
EMOJI IcuBreakIterator 18.34 =
SPACES BreakIterator 16.86 =
SPACES IcuBreakIterator 15.78 =
EMPTY BreakIterator 9.22 =
EMPTY IcuBreakIterator 9.96 =
NEWLINE BreakIterator 11.08 =
NEWLINE IcuBreakIterator 9.96 =
BIDI BreakIterator 19.54 =
BIDI IcuBreakIterator 18.96 =
Change-Id: I72023dd605da8e33974aa3181506f0d9c7bc180e
6 files changed