Search and sorting with the new letters: a note for developers

The reform changed the text, which means it changed your database, your search and your sorting too. Here are four mistakes almost everybody makes.

Search finds nothing

The database has Toshkent, the user types Toşkent — zero matches. There is one fix: normalise both the query and the index to the same form before comparing. Which form hardly matters; that it is the same one does.

Sorting comes out wrong

By default ö and ğ land after z, because their code points are higher. In JavaScript you want Intl.Collator("uz-Latn"); in PostgreSQL, COLLATE "uz-Latn-x-icu".

The Turkish i

The new letters exist in Turkish too, which makes the tr locale tempting as a "close enough" match. Do not take it: in the Turkish locale "i".toUpperCase() yields İ, while Uzbek needs I. Use uz, or no locale at all.

Unicode normalisation

ö arrives both as a single code point and as o plus a combining diaeresis — the latter is typical of files from macOS. Run all incoming text through normalize("NFC"), or two identical-looking strings will compare as different.

Apostrophes

People will type ', ’, `, ʻ and ʼ — all of them. Fold them to one character on input, and always render ʼ (U+02BC) on output.