Replace ICU.initLocaleDataNative with ICU4J implementation
The goal is to remove the ICU C++ API usage in the libcore.
This also fixes some data bugs, and specifically the previous implementation:
1. Use uloc_getParent as fallback locale in ICU resource bundle format.
uloc_getParent only truncates the locale ID string to get the parent, but doesn't use the
parent-child relationships in the locale data.
2. Truncate single Unicode character, e.g. U+11136 CHAKMA DIGIT ZERO, into single high
surrogate, e.g. 0xD804.
3. Does not consider alias in Root Locale
For AM/PM markers, AmPmMarkersNarrow is specified in the Root Locale as the alias of
"/LOCALE/calendar/gregorian/AmPmMarkersAbbr".
It affects the following data and locales:
1. short/medium/long/full time/date format
Affected locales: es_EC, uz_AF_#Arab, es_NI, es_AR, es_CR, es_UY, uz__#Arab, es_SV, es_PY
Obvervation: Many es_* locales should fallback to es_419 (Latin
American Spainsh) data instead of es data
2. yesterday/today/tomorrow
Affected locales: pa_PK_#Arab, pa__#Arab
Obversation: Locale pa uses Gurmukhi script, not Arabic script. Thus, loading data from pa_PK_#Arab
should fallback to the Root locale data. Generally, English words are used, i.e. Yesterday, Today, Tomorrow.
3. Zero digit
All locales using numbering system with non-BMP codepoint as zero
digits. It does not affect the common numbering system used in
Android, e.g. Latin, Arabic.
Previously, it's truncated into the high surrogate.
Example: Zero digit in Chakma (ccp language code) - '𑄶' U+11136 CHAKMA DIGIT ZERO.
New mechanism, fallback to the default Arabic numerals, e.g. ‘0’
4. Narrow AM and PM
Impact: narrowAm and narrowPm are used by time picker UI, and only a subset of locales by this UI in AOSP.
lo-LA is the only locale being affected and being used by the AOSP time picker UI. With this fix,
Lao user will see the localized string “ກ່ອນທ່ຽງ” instead of “AM”.
Affected locales with user impact: lo-LA (Lao)
Affected locales: agq, agq_CM, ak, ak_GH, asa, asa_TZ, bas, bas_CM, bem, bem_ZM, bez, bez_TZ, bo, bo_CN, bo_IN, brx, brx_IN, bs__#Cyrl, bs_BA_#Cyrl, dav, dav_KE, dje, dje_NE, dua, dua_CM, ebu, ebu_KE, eo, eo_001, ewo, ewo_CM, ff, ff__#Latn, ff_BF_#Latn, ff_CM_#Latn, ff_GH_#Latn, ff_GM_#Latn, ff_GN_#Latn, ff_GW_#Latn, ff_LR_#Latn, ff_MR_#Latn, ff_NE_#Latn, ff_NG_#Latn, ff_SL_#Latn, ff_SN_#Latn, fur, fur_IT, ga, ga_GB, ga_IE, gsw, gsw_CH, gsw_FR, gsw_LI, guz, guz_KE, gv, gv_IM, ii, ii_CN, jgo, jgo_CM, jmc, jmc_TZ, kab, kab_DZ, kam, kam_KE, kde, kde_TZ, kea, kea_CV, khq, khq_ML, ki, ki_KE, kln, kln_KE, ksb, ksb_TZ, ksf, ksf_CM, ksh, ksh_DE, ku, ku_TR, kw, kw_GB, lag, lag_TZ, ln, ln_AO, ln_CD, ln_CF, ln_CG, lo, lo_LA, lu, lu_CD, luo, luo_KE, luy, luy_KE, mas, mas_KE, mas_TZ, mer, mer_KE, mgh, mgh_MZ, mua, mua_CM, naq, naq_NA, nmg, nmg_CM, nnh, nnh_CM, nus, nus_SS, om, om_ET, om_KE, pa__#Arab, pa_PK_#Arab, pt_MO, rn, rn_BI, rof, rof_TZ, rwk, rwk_TZ, saq, saq_KE, sbp, sbp_TZ, ses, ses_ML, sg, sg_CF, shi, shi__#Latn, shi_MA_#Latn, shi__#Tfng, shi_MA_#Tfng, teo, teo_KE, teo_UG, twq, twq_NE, tzm, tzm_MA, uz__#Arab, uz_AF_#Arab, vun, vun_TZ, xog, xog_UG, yav, yav_CM
This change also removes unused field in LocaleData:
1. currencySymbol
2. internationalCurrencySymbol
Performance:
Before:
Results:
runtime(ns): min=16317890.50, 1st qu.=16317890.50, median=16317890.50, mean=16317890.50, 3rd qu.=16317890.50, max=16317890.50
Moving DateFormatSymbols only:
Results:
runtime(ns): min=16366308.74, 1st qu.=16366308.74, median=16366308.74, mean=16366308.74, 3rd qu.=16366308.74, max=16366308.74
Moving DateFormatSymbols, DecimalFormat/Symbol:
Results:
runtime(ns): min=15602260.27, 1st qu.=15602260.27, median=15602260.27, mean=15602260.27, 3rd qu.=15602260.27, max=15602260.27
After (removed ICU.initLocaleDataNative):
Results:
runtime(ns): min=15593138.32, 1st qu.=15593138.32, median=15593138.32, mean=15593138.32, 3rd qu.=15593138.32, max=15593138.32
Various caches in ICU4J levels may contribute the speedup of subsequent
queries and could take more memory.
1. BreakIterator instances per locale. Soft cache
2. Calendar data. Soft cache
3. Locale -> ULocale mapping. Soft cache
As they are soft cache, they can be collected by the ART runtime during GC.
If there are boot/zygote memory regression, we can try to clean up the
cache before zygote forks.
Bug: 138994281
Test: CtsLibcoreTestCases
Change-Id: I804f35118cf58a2b83ea6e7a4dc05498b6ff4683
8 files changed