The keystroke that isn't one: IME composition guards

Phase 26 scoped these and left them unbuilt, naming them as the likeliest
thing to be wrong the first time anyone types Chinese into Petal for real.

A composition is not a keystroke: the pinyin goes into the document as it
is typed, a candidate window sits over it, and all three decoration layers
recompute from the live document on every change — rewriting the DOM around
the node the browser is composing in, which is what eats half-typed input.

The layers now hold their redraws rather than skip them: a rebuild that falls
due mid-composition marks itself stale and its decorations are mapped through
the transaction, so they travel with the text and land correct the moment the
composition ends. The flag is read from the state before the transaction, so
the answer doesn't depend on plugin ordering; the end transaction is the one
deliberate exception, or nothing would ever release. The release is a
macrotask late because a custom handleDOMEvents handler runs before
ProseMirror's own and ProseMirror flushes the composition's last changes in a
microtask — so the held rebuild sees the committed hanzi, not the pinyin it
replaced.

Input rules needed no guard (Tiptap already returns early while composing),
which was checked rather than assumed: pinyin uses an apostrophe as a
syllable separator and Typography rewrites every ' into a curly one.

The save is deliberately not gated and the analysis is. A tablet keyboard can
hold one composition open for a whole sentence, and Petal never makes writing
wait for anything — so EditorChange carries the flag, auto-save ignores it,
and the checkpoint, rule pack and companion wait for the word to commit. One
more change is emitted the instant it does, so nothing is skipped.

Four places were taking keys that belong to the IME: the Find bar, the tag
picker, Ask Petal's chat box, and distraction-free mode's global Escape.

vitest 296/296, tsc, vite, go build/vet/test clean. Not verified with a real
IME — no browser or IME here, and that is the half the tests cannot reach.

BUILD_PLAN.md

@@ -393,14 +393,31 @@ Scope agreed with the user 2026-07-28: **a `direction` column, segmentation + ho
 6. [x] **The direction picker names each option in the language of the person who would choose it** — 英文 for the writer who is native in Chinese, "Chinese 中文" for the one who is native in English. The same self-naming principle the pair buttons follow, for the same reason: someone on the wrong side of this switch cannot read the side they are trying to reach. It renders only when the pack carries a `learner` block, which is the frontend's half of `auth.learnerPairs`.
 - **`@types/node` added as a devDependency**, which is a small thing with a real consequence: vitest can now read the *shipped* assets. Phases 21–25 all verified their dictionaries with throwaway scripts because the suite could not; `segment.test.ts` and `hanzi.test.ts` assert against the real files.
 - Verified: go build/vet, `go test ./...`, tsc, vite build, **vitest 284/284**; live smoke on a throwaway DB (:8071, LLM pointed at a dead port) — 公园 → gōngyuán, 得 → both readings, 猫书 → the character fallback, the word list served at 965,266 B, `PATCH {"direction":"learning_pair"}` accepted, `{"pair_lang":"fr"}` refused 400 while the account stayed put. The smoke also caught a cosmetic build bug: stripping CC-CEDICT's `CL:` field left "cat (" with an unbalanced paren, now fixed and the asset rebuilt.
-- ⚠️ **The IME guards were scoped into this phase and are NOT done.** No composition handling exists anywhere in the app — verified, not assumed. The risk is concrete and known: the decoration plugins (`SpellCheck`, `SuggestionHighlight`, `SearchHighlight`) recompute on every doc change, and a rebuild mid-composition is the classic bug that eats half-typed pinyin. This is the single most likely thing to be wrong the first time someone types Chinese into Petal with a real IME, and it is untouched rather than half-built.
+- ⚠️ **The IME guards were scoped into this phase and were NOT done** — no composition handling existed anywhere in the app. **Built 2026-07-28 as Phase 27 below.**
 - ⚠️ **Not deployed** (this one carries a migration, so it is a deploy and not a rebuild), **not seen in a browser**, and **no account has ever been in the learner direction** — every claim above about how this feels to use is inference from unit tests. The 错别字 pack has not been read by a native speaker either; unlike the Latin packs it is 24 mechanically-screened pairs rather than prose, which lowers the stakes without removing them.
 
+### Phase 27 — IME composition guards (2026-07-28, code half) — the keystroke that isn't one
+Phase 26's own outstanding item, and the thing it named as most likely to be wrong the first time anyone types Chinese into Petal for real. Code only; no migration, no server change at all, so it is a rebuild whenever the user wants it.
+
+1. [x] **One tracker, asked by everything** (`web/src/components/Editor/Composition.ts`). A composition is not a keystroke: the pinyin she types goes *into the document* as she types it, a candidate window sits over it, and only on choosing a candidate is the run replaced with hanzi. All three decoration layers recompute from the live document on every change, and recomputing rewrites the DOM around the node the browser is composing in — which is the classic bug that eats half-typed input.
+   - **The layers hold their redraws, they do not skip them.** A rebuild that falls due mid-composition marks itself `stale` and the existing decorations are **mapped through the transaction**, so they travel with the text growing under them; the moment the composition ends, the held rebuild happens. Skipping instead of holding would leave every highlight a character behind for as long as she kept typing — the same wrongness, arriving quietly instead of loudly.
+   - **The question is asked of the state *before* the transaction**, which is what makes the answer independent of plugin ordering: the flag was set by an earlier transaction (compositionstart), not by the one being applied. The end-of-composition transaction is the single exception — it releases rather than holds, or nothing ever would.
+   - **The release is a macrotask late, and that is the one piece of timing that matters.** A custom `handleDOMEvents` handler runs *before* ProseMirror's own, and ProseMirror's `compositionend` queues the composition's final DOM changes as a **microtask**. Ending on `setTimeout(…, 0)` puts the release after both, so the rebuild sees the committed 公园 rather than the *gongyuan* it replaced. If a transaction from that flush lands first, it rebuilds anyway — by then the flag is already false. Both orders land, which is why it is not a race.
+   - `blur` ends the composition too: clicking away mid-candidate doesn't always produce a `compositionend`, and without it the layers would stay held silently until she typed again.
+   - **Clearing is never held.** Closing the Find bar removes decorations rather than adding them, so it goes through immediately.
+2. [x] **Input rules needed no guard, and this was checked rather than assumed** — Tiptap's own input-rule plugin returns early while `view.composing`. It matters more here than in an English app: pinyin uses an apostrophe as a syllable separator (`xi'an` → 西安) and `Typography.ts` rewrites every `'` into a curly `’`, so an unguarded rule would corrupt the IME's buffer mid-word.
+3. [x] **The save is deliberately *not* gated; the analysis is.** A tablet keyboard can hold one composition open for a whole sentence, and Petal never makes writing wait for anything — so `EditorChange` carries a `composing` flag and the auto-save ignores it. What it gates is the checkpoint, the rule pack and the companion: asking them to read half-typed pinyin can only produce advice about text that is about to stop existing. The editor emits **one more change the moment the composition commits**, so nothing is skipped, only deferred by the length of a word. The flag stays out of the document patch itself — it describes the keyboard, not the document, and the draft a signed-out save stashes should be the document alone.
+4. [x] **Enter and Escape belong to the IME while a candidate window is open** (`web/src/lib/ime.ts`), and four places were taking them: the Find bar (Enter steps to the next match, Escape closes), the tag picker (Enter creates the tag — a field whose whole purpose is a name typed in Chinese), **Ask Petal's chat box** (the field she types Mandarin into, where Enter sends), and distraction-free mode's global Escape (pressed to fix a wrong candidate, it popped the sidebar back). `isComposing` plus the older `keyCode === 229`, which some Safari/IME combinations are still the only signal from.
+- Tests: `Composition.test.ts` (12) drives real plugin state through a real composition — pinyin typed in, candidate committed, composition ended — and asserts each layer holds, maps and then releases: no underline under half-typed pinyin, a suggestion list arriving mid-composition applied on end, matches held at two rather than briefly three, a checker that arrives mid-composition applied when it can be, and the release working on a transaction carrying **no document change at all**, which is what the timer-dispatched end signal usually is.
+- Verified: go build/vet/test (untouched, but the standing rule), tsc, vite build, **vitest 296/296**.
+- ⚠️ **Not verified with a real IME.** The tests exercise the layer where the decision is made — plugin state — and cannot exercise the layer where the bug bites, which is a browser deciding whether to abandon a composition after the DOM under it moved. This laptop has no Chrome extension and no IME. What is proven is that the redraws are held and released correctly; what is unproven is the browser's half of it.
+- **Not deployed.** No migration; a rebuild whenever the user wants it, along with Phases 24–26.
+
 ### Later / explicitly not now
 - Learner-facing Chinese writing (the zh pair's second direction) — own phase with its own spec (SUGGESTIONS §4); only after Phases 19–21 prove the pair model
 - ~~Spanish pair — gated on DreamDict growing an es dataset~~ **ungated 2026-07-26**, **shipped (code) 2026-07-28** — see Phase 25. What it still owes: a deploy, a native reader, and a writer who actually uses it.
 - ~~Voseo for the es pair~~ **resolved 2026-07-28 before shipping** — the fix was not to generate the paradigm but to stop using Debian's package, which is peninsular. RLA's generic build has it. See Phase 25 item 2.
-- **IME composition guards** — scoped into Phase 26 and not built; see that entry. The decoration plugins recompute on every doc change, and doing so mid-composition is what eats half-typed pinyin. First thing to do before anyone types Chinese into Petal in earnest.
+- ~~**IME composition guards** — scoped into Phase 26 and not built~~ **built 2026-07-28, see Phase 27.** What it still owes is a real IME in a real browser.
 - **`restoring` is untranslated in the fr and pt-PT packs** — surfaced by the es review, fixed only in es. One line each, whenever those packs are next touched.
 - Reactive-animation puppy companion — wishlist, low priority; `companions.ts` roster + mood engine is the drop-in point
 - Copyleaks Tier-2 — revisit once Phase 15 provides a public webhook endpoint
@@ -413,6 +430,7 @@ Scope agreed with the user 2026-07-28: **a `direction` column, segmentation + ho
 - [x] **Phase 14 — companion warmth + bedtime nag + night mode**: more encouraging phrases, a gentle "go to bed" nudge after 11pm, and a calm dark theme + falling stars at night. ✅ (see Phase 14 above)
 
 ## Session log
+- 2026-07-28: **Phase 27 — the keystroke that isn't one** (user asked to continue the build plan; the plan's own next item was Phase 26's unbuilt IME guards, named there as the likeliest thing to be wrong the first time anyone types Chinese into Petal for real). **The bug is that a composition is not a keystroke**: the pinyin goes into the document as it is typed, a candidate window sits over it, and all three decoration layers recompute from the live document on every change — rewriting the DOM around the node the browser is composing in, which is what eats half-typed input. **The fix is to hold the redraws, not skip them**: a rebuild that falls due mid-composition marks itself stale and its decorations are *mapped through the transaction*, so they travel with the text growing under them and land correct the moment the composition ends. Skipping would have left every highlight a character behind for as long as she kept typing — the same wrongness, arriving quietly. **The composition flag is read from the state *before* the transaction**, which makes it independent of plugin ordering: the flag was set by compositionstart, not by the transaction being applied, and the end transaction is the one deliberate exception or nothing would ever release. **One piece of timing genuinely matters and is written down where it happens**: a custom `handleDOMEvents` handler runs *before* ProseMirror's own, and ProseMirror's compositionend queues the composition's last DOM changes as a microtask — so the release is a macrotask late, and the held rebuild sees the committed 公园 rather than the *gongyuan* it replaced. If the flush's transaction arrives first it rebuilds anyway; both orders land, which is what makes it not a race. **One thing was checked rather than assumed and turned out already handled**: Tiptap's input-rule plugin returns early while composing — which matters more here than in an English app, because pinyin uses an apostrophe as a syllable separator (xi'an → 西安) and Typography rewrites every `'` into a curly `’`. **The save is deliberately not gated and the analysis is**: a tablet keyboard can hold one composition open for a whole sentence, and Petal never makes writing wait for anything, so `EditorChange` carries the flag, auto-save ignores it, and the checkpoint/rule pack/companion wait — with one more change emitted the instant the composition commits, so nothing is skipped, only deferred by the length of a word. **And four places were quietly stealing keys from the IME**: the Find bar (Enter = next match, Escape = close), the tag picker (Enter = create, in a field whose purpose is a name typed in Chinese), Ask Petal's chat box (the field she types Mandarin into, where Enter sends), and distraction-free mode's global Escape — pressed to fix a wrong candidate, it popped the sidebar back. tsc, vite, go build/vet/test, **vitest 296/296** (12 new, driving real plugin state through a real composition: pinyin in, candidate committed, composition ended). ⚠️ **Not verified with a real IME** — the tests exercise the layer where the decision is made and cannot exercise the layer where the bug bites, which is a browser deciding whether to abandon a composition after the DOM under it moved. Held-and-released is proven; the browser's half is not. **Not deployed** (no migration — a rebuild, along with Phases 24–26).
 - 2026-07-28: **Phase 26 — the zh pair's other direction, and a rule pack whose best feature is what it refuses** (user asked to continue the build plan, then chose a new phase over deploying fr/es; scope chosen with the user: **direction column, segmentation + hover pinyin/gloss, 错别字, IME guards**, code only). SUGGESTIONS §4 had called this its own epic, and the reason turned out to be one sentence: **`pair_lang` had always been answering a second question nobody asked.** It says which two languages; every surface built on it assumed English was the one being *learned*, which is why CJK is deliberately never tokenized, never spell-checked and never glossed. All correct for the writer this app was built for, all backwards for the other one. A `direction` column rather than a `zh-learner` pair code, because the two are genuinely separate questions and the column is what lets fr/es/pt inherit the capacity later. **The phase has three decisions in it and they are all about coverage.** The browser gets a word list and the server keeps the dictionary, and their gates come out *opposite*: the client list is frequency-gated at 5 because segmentation by the full union and by the gated list is **identical** on ordinary prose (measured, including 研究生命的起源 and 乒乓球拍卖完了 — the long tail is rare proper nouns and the max-probability walk never picks one), while the dictionary is gated by **nothing**, because its only power is to *explain* and the word a learner stops on is precisely the rare one. That is the es dictionary decision arrived at from both sides in one phase. **Writing the segmenter tests found the boundary bug**: a position names a gap and a word covers characters, so a hover on a boundary was resolving to the word that ended there rather than the one that starts. **The 错别字 pack is the part worth reading.** Chinese has no misspellings — every character an IME offers is real — so the unit of error is a substituted character inside a correct-looking word, and the pack is 24 confusable pairs held by two mechanical gates. Gate one admits a pair only if the wrong form is *not* a dictionary word and the right form is, which is what makes it refuse **自已 for 自己** — one of the commonest slips in written Chinese, whose wrong form is itself a headword — exactly as Phase 22 refused `married with`. Gate two is the one that matters: **自己经常 contains 己经, 睡觉的时候 contains 觉的, 不知到底 contains 知到**, so a substring match would corrupt correct sentences silently, into text still made of real characters. The segmenter settles it — two adjacent single-character tokens is what the walk produces when it has nothing better, which is what a mistyped compound looks like — and where the gate costs a real catch (我不知到他在哪里 *is* 知到 for 知道, but 不知 is a word) it declines rather than risk the identical-looking correct sentence beside it. **`@types/node` went in as a devDependency and quietly fixes something older**: phases 21–25 each verified their shipped dictionary with a throwaway script because vitest could not read files; the suite now asserts against the real assets. go build/vet/test, tsc, vite, **vitest 284/284**, live smoke on a throwaway DB which itself caught a cosmetic build bug ("cat (" left by stripping CC-CEDICT's CL: field). ⚠️ **The IME guards were in scope and are not done** — no composition handling exists anywhere in the app, and a decoration rebuild mid-composition eating half-typed pinyin is the likeliest thing to be wrong the first time anyone types Chinese into Petal for real. Left untouched rather than half-built, and named here rather than buried. ⚠️ **Not deployed** (it carries a migration), **not seen in a browser**, and **no account has ever been in the learner direction**, so everything above about how it feels to use is inference from tests.
 - 2026-07-28: **Phase 25 — the es pair, and a plan that had quietly chosen the wrong Spanish** (user asked where Spanish support had gone, then "yes" to starting the phase; scope chosen with the user: **Latin American neutral**, quorum review, code only). The starting point was a misreading worth recording: the plan *reads* as though Spanish shipped, because the DreamDict rebuild, the LLM language entry, the L1 rule gating and the TTS env-discovery are all `[x]` — every piece of groundwork was done and the pair itself had never been built. `shippedPairs` was the honest answer all along: the server had been refusing `es` on purpose. **The regional question was the phase.** pt-PT's was forced by packaging and fr's turned out not to exist; es had a real choice with no default, and once the user chose Latin American, the plan's own two concrete decisions were both wrong. It warned that `hunspell-es` is "packaged per country — check what `es_ES` actually is": it ships twenty country codes and **every one is a symlink to one pan-Hispanic file**, so the trap was not there. And it named **`es_ES-davefx-medium`** for the voice, which *is* the trap — six of Piper's nine Spanish voices are peninsular, so the obvious pick would have read Latin American copy in a Castilian accent, the pt-PT mistake arriving through a different door. `es_MX-ald-medium` instead. **Then the user asked "should we pick a different Spanish dictionary?" and the answer was yes** — the phase had shipped the wrong one and written a confident justification for it. Debian's `hunspell-es` symlinks twenty country codes to one file, which reads as pan-Hispanic; RLA actually publishes twenty-four builds per release, one per country **plus a generic `es` that is the union**, and Debian ships **peninsular `es_ES`**. The 58,622-form difference is essentially **voseo**: under the first build, *vení* and *tenés* — the ordinary present tense of Argentina, Uruguay, Paraguay and much of Central America — were underlined as misspellings, and this document called that a known gap handled on principle. **What makes it worth writing down is that the MUST_ACCEPT list was designed to catch exactly this and could not**: it asserted the pan-Hispanic *vocabulary*, and every RLA variant carries the full pan-Hispanic vocabulary — only the paradigms are localised — so it was satisfiable by all twenty-four. The `REP` table cited as the corroborating witness (yeísmo, seseo) is likewise shared by every build. Two independent-looking proofs, neither of which could distinguish anything, agreeing with each other. The profile now demands **voseo** (rejects es_ES and Debian), **vosotros** (rejects es_MX) and **another region's everyday words** — *arepa*, *chévere*, *bacán* (rejects es_AR, which has both paradigms and would otherwise pass); all four neighbours were run through it and confirmed refused. Shipping the union is the same call fr made between *coût* and *cout*: the dictionary's only power is to underline, so it holds every variety, while the *copy* picks a register because speaking requires one. 717,640 forms, 1.74 MB gzipped, **762 ms / 97 MB** in a real nspell, and **fr and pt-PT rebuild byte-identical** from their own upstream debs. **The quorum review earned its place twice**: four models, ≥2-of-4, 5 of 27 findings applied — one of which caught the pack's bedtime proverb being *Qui dort dîne* calqued into Spanish, English gloss and all, which is exactly the "a pack is not a translation of another pack" rule the fr header states and I had broken while writing it. And one below-threshold finding (a missing `¡` on an exclamative, seen by 1 of 4 because an absent *opening* mark has no closing `!` to look wrong against) was applied anyway and **turned into an assertion**: the suite now rejects any native line that closes `?`/`!` without opening one. That is Phase 24's lesson one level up — what a review finds once, a test should find every time. go build/vet/test, tsc, vite, **vitest 251/251**. ⚠️ **Not deployed, not seen in a browser, not read by a native speaker, and no es account exists** — all four accounts' worth of Spanish experience is still hypothetical, and the pack says so in its own header.
 - 2026-07-27: **Phase 24 — the fr pair, and a "generalizes" that did not** (user: "resume the build plan"; scope chosen with the user: French end to end, code only, deploy its own step). The plan's five items were meant to be mechanical, and four of them were — the Piper voice is a compose service and two env lines because Phase 21 made a language configuration; the lexicon needed nothing at all, fr having been measured at 63.1% during Phase 20's rebuild, better than the pair that already shipped; the sidebar picker grew a third entry without a line of UI because it derives itself from the shipped packs. **Item 3 was the one that had been recorded as done and wasn't.** `build_ptpt_dictionary.py` was said to generalize; it handled single-character flags and plain PFX/SFX and stopped on everything else, and `fr.aff` uses four of the things it stopped on. `FLAG long` is the dangerous one: French flags are two characters, so the pt-PT reader's `set(flagstr)` yields a bag of unrelated letters and expands every entry through the wrong paradigm without erroring. Plus continuation flags (French really does affix an affixed form), NEEDAFFIX on 68,075 of 84,140 stems, and FULLSTRIP. The rewritten `build_hunspell_dictionary.py` carries a per-language profile and asserts that CIRCUMFIX and FORBIDDENWORD are still unused — and **rebuilds pt-PT byte-identical to the shipped asset**, which is the only thing that makes "generalized" a claim rather than a hope. **The second decision was elision, and it was made by measuring both halves**: keeping `l'arbre` and its thirty-three siblings costs 3,159,832 forms and 8.25 MB gzipped; dropping them costs 473,326 and 1.19 MB. They are not new words, but the tokenizer keeps internal apostrophes, so they really would have been underlined — so they moved out of the dictionary and into `withElision`, which splits at a known clitic and still requires the remainder to be a word (`l'zzzz` stays flagged). Real nspell: 369 ms and 74 MB for the larger language, against pt-PT's 842 ms and 139 MB. **Where the regional trap lives is the mirror image of Portuguese's**: every `fr_*` Piper voice is fr_FR and every Debian fr dictionary is one shared word list, so nothing can be quietly wrong about the country — the whole decision is in the copy, which is why the pack is greped for *courriel* and *magasiner* the way pt-PT is greped for *arquivo*. What French does have instead is the 1990 reform, packaged three ways; Petal ships comprehensive, because Petal never corrects her French and *coût* and *cout* are both correct. go build/vet/test, tsc, vite, vitest 190/190. **Two things owed and both said plainly**: no native speaker has read the pack (SUGGESTIONS §3's bar, unmet for pt-PT too), and nothing here has been seen in a browser. **Then, same session, an interim answer to the first of those** (user: "perhaps for now, we could leverage multiple LLMs to act as reviewers?"): four models reviewed each Latin pack independently, and only findings ≥2 of them reached on their own were applied — five per pack. It earned its keep on the pack that was *already shipped*: pt-PT had **pre-Acordo spellings in a file whose own header commits to post-Acordo**, because the Phase 21 greps checked for Brazilian vocabulary and never checked the pack against its own spelling policy. That grep now exists and was confirmed to fail on the old text. Where reviewers agreed a line was wrong but split on the fix, the wording is mine and the reasoning is in the phase entry rather than averaged away. Still not a native speaker, and both packs now say so precisely.