Her apostrophe was cutting French words in half

Typography.ts rewrites every ' typed in the editor into a curly ’, but both
word regexes only counted the straight one. So "aujourd’hui" reached the
dictionary as "aujourd" + "hui", neither of them a French word, and one of the
commonest words in the language came back wearing two red underlines. Same for
quelqu’un, presqu’île, prud’homme. l’arbre only survived by accident, because
"l" happens to be a bare entry. withElision, written for exactly this, could
only ever fire on pasted text.

Both marks are word characters now, and combine() straightens on lookup — the
one place every lookup passes through — since the shipped word lists spell
theirs straight. Suggestions come back wearing whichever mark she actually
used, so accepting a pill never swaps her apostrophe.

œ was untokenizable too: U+0152/U+0153 sit outside the Latin-1 ranges, so
"cœur" split into "c" + "ur" and the orphan was long enough to underline. 586 œ
forms ship in fr.dic.gz and not one of them was reachable.

In the dictionary builder, the two cross-product paths added their forms
without the NEEDAFFIX check the single-affix paths apply, so a doubly-affixed
form that is still "not a word on its own" was accepted anyway — the exact
class of error the FLAG-aware rewrite exists to close. PFX and SFX are also
separate flag namespaces, and one shared `cross` dict let the second block
overwrite the first. Odd-length long-flag strings now stop the build instead of
dropping a character and expanding through the wrong paradigm.

The pt-PT and Québécois greps were case-sensitive against sentence-cased copy,
which let a leading "Actualmente…" through the guard added to catch it.

Note: this changes what the expander produces, but fr.dic.gz and pt-PT.dic.gz
are vendored and were built with the old behaviour. Both want regenerating on a
box that can fetch the upstream .deb, and BUILD_PLAN Phase 24's "pt-PT rebuild
is byte-identical" claim re-checked — if those bytes move, the NEEDAFFIX gap
was live in the Portuguese list too.

Claude-Session: https://claude.ai/code/session_016y6gyuHkQXPiEuW8RGQyua
This commit is contained in:
prosolis
2026-07-27 17:07:30 -07:00
parent 071ea7b835
commit be1ab5cef7
6 changed files with 92 additions and 19 deletions
+5 -2
View File
@@ -194,7 +194,10 @@ describe('the pt-PT pack', () => {
// is invisible to anyone who doesn't read Portuguese — including whoever
// reviews this diff.
it('is European Portuguese, not Brazilian', () => {
const text = JSON.stringify(ptPT, (_k, v) => (typeof v === 'function' ? v(1, 'x') : v))
// Lowercased: half this copy is sentences, and a form that only ever appears
// at the start of one ("Actualmente…") would otherwise walk straight past
// both greps below.
const text = JSON.stringify(ptPT, (_k, v) => (typeof v === 'function' ? v(1, 'x') : v)).toLowerCase()
// Brazilian spellings and vocabulary that would give the pack away.
for (const bad of ['sinônimo', 'acadêmico', 'arquivo', 'tela', 'salvar', 'deletar', 'usuário', 'você']) {
@@ -259,7 +262,7 @@ describe('the fr pack', () => {
// this file, which makes it exactly as invisible to a reviewer as the pt-BR
// forms were, and worth pinning the same way.
it('is metropolitan French, not Québécois', () => {
const text = JSON.stringify(fr, (_k, v) => (typeof v === 'function' ? v(1, 'x') : v))
const text = JSON.stringify(fr, (_k, v) => (typeof v === 'function' ? v(1, 'x') : v)).toLowerCase()
for (const bad of ['courriel', 'clavarder', 'magasiner', 'fin de semaine', 'baladodiffusion']) {
expect(text, `Québécois form "${bad}" in the fr pack`).not.toContain(bad)
}