Phase 21: Petal learns to be an English+Portuguese pair

The plan said "Hunspell pt-PT vendored like en-US". Measuring that first is
what saved it: nspell expands affixes eagerly on construction, and European
Portuguese's 1,340 rules over 44,257 stems want over a gigabyte of browser
heap — ~340 MB for the first 12,000 entries, and no return at all after three
minutes on the whole file. So the expansion runs once at build time instead:
1,039,058 forms, 2.66 MB gzipped, read by the same nspell in 842 ms.

The obvious npm package would also have shipped the wrong language. Both
dictionary-pt and dictionary-pt-br carry VERO, the Brazilian word list, so
vendoring by name puts pt-BR spellings behind a pt-PT label — the drift
SUGGESTIONS §3 warns about, arriving through the packaging where no reviewer
can see it. The source is Projecto Natura's, and the build script now asserts
the fault lines (receção in, recepção out) before writing anything.

Spellcheck consults both dictionaries and flags only what both reject, which
is the no-detector answer to a pair with no script boundary. The word card
does the same in the other direction: "data" is a word in both languages, so
Petal shows both readings rather than guessing which she meant.

Writing the tests caught the one real bug — extendedAlphabet was a snapshot
while correct/suggest read live, and her dictionary arrives after English, so
every lookup would have resolved "cora" while the underlines were already
right.

Not done, and not claimed: the pack has not been read by a pt-PT speaker, and
the Piper voice is deferred with the deploy.

Claude-Session: https://claude.ai/code/session_016y6gyuHkQXPiEuW8RGQyua

BUILD_PLAN.md

@@ -235,12 +235,21 @@ Option 3 ratified (import package, read-only `dict.db`).
 ### Phase 21 — The pt-PT pair (first Latin pair, proves the model)
 SUGGESTIONS §1/§3/§3a. French and **Spanish** follow the same groove afterwards — es is no longer gated now that DreamDict has Spanish data (2026-07-26). pt-PT still goes first: it's the pair with a real user behind it, and it's the one that proves the langpack + both-dictionaries model.
 Phase 20 left this ready: `dict.db` on the VPS now holds all five languages, and pt-PT gloss coverage of common English words is 62%.
-- [ ] Hunspell pt-PT vendored like en-US; **both-dictionaries spellcheck** (flag only if wrong in both; pills from both) — the no-detector stance, Q1 settled
-- [ ] Gloss/WordCard both directions; on en/pt collisions show both compactly, never hide either
-- [ ] Prompts pinned to **European Portuguese, never pt-BR** (explicit in every prompt); pt-PT langpack copy written and **reviewed by a pt-PT speaker before trusted**
-- [ ] Piper pt-PT voice instance on parodia; read-aloud + L1 voice wired; slow toggle (`length_scale`) while in there (SUGGESTIONS §5e)
-- [ ] Companion tips/cheers/bedtime lines in the pt-PT pack (the kitten speaks pt+en to this user)
-- [ ] Acceptance: a pt-PT-pair user gets the full experience end-to-end with the VPN down except LLM passes; zh-pair user sees zero change
+**Code half built 2026-07-27** (user: "let's continue the build plan"; scope confirmed: code only, no VPS work; the pack written but flagged unreviewed). Not deployed — no migration, so it is a rebuild whenever the user wants it.
+- [x] **pt-PT spelling dictionary — and it could not be "vendored like en-US".** nspell expands affixes *eagerly on construction*: English's ~50k stems and small rule set are fine, European Portuguese's **1,340 affix rules over 44,257 stems** are not. Measured here: ~340 MB of heap for the first 12,000 entries and no return at all after three minutes on the whole file, i.e. comfortably over a gigabyte for a browser to load a spellchecker. So `scripts/build_ptpt_dictionary.py` runs Hunspell's expansion **once at build time** — 1,039,058 surface forms, 15 MB of text, **2.66 MB gzipped**, which nspell then reads with no affix machinery at all in **842 ms / ~120 MB**. The runtime path is byte-for-byte the English one, which is the real prize. The shipped `.aff` keeps only upstream's TRY/KEY/REP/MAP, which shape *corrections* rather than membership — so "telemovel" still corrects to "telemóvel" and "cao" still knows about "ção".
+- [x] **npm's `dictionary-pt` is not European Portuguese.** Both it and `dictionary-pt-br` package VERO (*Verificador Ortográfico Livre*, Brasil) — the obvious vendoring step would have shipped Brazilian spellings under a pt-PT label, which is the §3 drift risk arriving through the packaging rather than through the model. The real source is Projecto Natura's (Universidade do Minho), which LibreOffice ships and Debian packages as `hunspell-pt-pt`; its aff declares `LANG pt_PT`. The build script **asserts the fault lines before it writes anything**: accepts `receção`, `húmido`, `telemóvel`, `autocarro`, `comboio`, `ótimo`, `pensámos`, `escrevêssemos`; rejects `recepção`, `úmido`, `ônibus`, `óptimo`. A source that fails those is not the dictionary it is for.
+- [x] **Both-dictionaries spellcheck** (`useSpellChecker`): English always loads; her language loads when a pair ships one; a token is flagged only when **every** loaded dictionary rejects it. Correction pills **interleave** the two rather than concatenating — otherwise English fills all five and a misspelt Portuguese word gets no Portuguese suggestion, which is the one case the second dictionary was loaded for. No dictionary at all accepts everything: a failed fetch must not underline the whole document.
+- [x] **The tokenizer had to become a property of the checker, not a constant.** `[A-Za-z]` cuts "coração" into "cora" and "o", both short enough that `isCheckable` discards them — so the word was silently never checked *and* a right-click would have offered a definition of "cora". The wide alphabet is `[A-Za-zÀ-ÖØ-öø-ÿ]`, deliberately skipping × and ÷, which hide inside that Latin-1 range. It stays **off** for a writer with no Latin second language: widening it there can only find new words to underline (the *café* and *naïve* she borrows) and no mistake she actually made. `wordAt` takes the same flag, so the underline and every lookup agree.
+- [x] **Gloss/WordCard both directions** — new `lexicon.Reverse` on the lookup and a `reverse` line on the hover tip. A Latin pair has no script boundary: *data*, *sale*, *comum*, *tarde* and *ali* are real words on both sides, and there is no honest way to look at one in a mixed document and know which was meant. Petal asks both directions and shows whatever answers — no detector, so it cannot be wrong about her writing, and for a learner the collision is the interesting part. The English de-inflection walk is deliberately **not** applied in reverse: `candidates` knows -s/-ed/-ing, and running it over Portuguese would be right by accident and wrong by rule.
+- [x] Prompts pinned to **European Portuguese, never pt-BR** — already done in Phase 19 (`internal/llm/lang.go` spells it out inside the prompt, with *porquê* carried alongside so the tutor recognises her question).
+- [x] pt-PT langpack written (`web/src/i18n/packs/pt-PT.ts`), and pt-PT is now a real switch rather than a fallback. Post-Acordo spellings with the European lexicon (*ficheiro*, *ecrã*, *guardar*, *sinónimo*, *académico*, *Iniciar sessão*), *estás a escrever* rather than the gerund, and second-person *tu* — a companion in a private notebook, not a form. A test greps the built pack for Brazilian forms, because that is exactly the error nobody reviewing the diff can see.
+- [ ] ⚠️ **The pack is NOT reviewed by a pt-PT speaker** — SUGGESTIONS §3's own bar, and the one item here I cannot meet. Flagged at the top of the file and left unchecked deliberately; expect a speaker to change the register before the vocabulary.
+- [x] Companion tips/cheers/bedtime lines in the pt-PT pack. Not a translation of the zh pack: the bedtime proverbs are Portuguese ones and there is a false-friends tip the Mandarin pair had no use for. The English wit in the bedtime lines is the user's own and is kept word for word across packs.
+- [ ] Piper pt-PT voice instance on parodia; read-aloud + L1 voice wired; slow toggle (`length_scale`) while in there (SUGGESTIONS §5e) — the infra half, deferred with the deploy
+- [ ] Acceptance: a pt-PT-pair user gets the full experience end-to-end with the VPN down except LLM passes; zh-pair user sees zero change — waits on the voice above and on a real pt-PT account
+- Tests: `internal/lexicon/dreamdict_test.go` gains a real collision in the fixture (*data*: English facts, Portuguese date) — both readings on a collision, **no** reverse block for an English-only word, the tooltip carrying only the reverse gloss, and the embedded/glossless providers staying silent (a Chinese reading of an English word is worse than none). Frontend: `spellchecker.test.ts` (either-accepts, flag-only-if-both-reject, a Portuguese word never flagged for being unknown to English, no-dictionary-accepts-everything, a dictionary arriving *after* the checker was built, interleaved pills) and `SpellCheck.test.ts` (the narrow alphabet still cutting "coração", the wide one not, CJK never tokenized under either, × and ÷ excluded). The i18n suite now runs its shape assertions over *every* pack — a shape only the first author's pack satisfies is a coincidence, not a shape.
+- **A bug the test found, not the code review**: `extendedAlphabet` was a value computed when the checker was built while `correct`/`suggest` read live. Her dictionary arrives *after* English, so the underlines would have been right while every lookup was still resolving "cora". It is a getter now.
+- Verified: go build/vet/test, tsc, vite build, vitest 116/116 clean. The shipped asset loaded in a real nspell (842 ms, 139 MB, pt-PT variants correct both ways). Live smoke on a throwaway DB (:8091): both dictionary files served (577 B aff, 2,661,813 B gz), the gz inflating to 1,039,058 forms with `receção` present, and the zh word lookup unchanged. **Not verified against real data**: this laptop has no `dict.db`, so the reverse-lookup path is exercised by the fixture only — the first real pt-PT collision lookup happens on the VPS.
 
 ### Phase 22 — Learning loop + code-first layers
 Each item independent and small; order within is free (SUGGESTIONS §5–§6).
@@ -265,6 +274,7 @@ Each item independent and small; order within is free (SUGGESTIONS §5–§6).
 - [x] **Phase 14 — companion warmth + bedtime nag + night mode**: more encouraging phrases, a gentle "go to bed" nudge after 11pm, and a calm dark theme + falling stars at night. ✅ (see Phase 14 above)
 
 ## Session log
+- 2026-07-27: **Phase 21 (code half) — the pt-PT pair, and the plan's one-line assumption about the dictionary** (user: "let's continue the build plan"; scope confirmed: code only, the Piper voice and the deploy deferred, the pack written but flagged unreviewed). The plan said "Hunspell pt-PT vendored like en-US", and that turned out to be the load-bearing sentence. **nspell expands affixes eagerly on construction** — it materialises every surface form the moment you build it. English survives that; European Portuguese's 1,340 affix rules over 44,257 stems do not. Measured before deciding anything: ~340 MB of heap for the first 12,000 entries, and no return at all after three minutes on the whole file — over a gigabyte, in a browser, on a tablet. So the expansion moved to build time: `scripts/build_ptpt_dictionary.py` writes 1,039,058 forms, 2.66 MB gzipped, which the *same* nspell then reads in 842 ms using ~120 MB, and the runtime path stays byte-for-byte the English one. The `.aff` keeps only TRY/KEY/REP/MAP, which shape corrections rather than membership, so "telemovel" still corrects to "telemóvel". **A second thing the obvious route would have got wrong quietly**: npm's `dictionary-pt` is not European Portuguese — both it and `dictionary-pt-br` package VERO (Brasil), so vendoring the obvious package name ships Brazilian spellings under a pt-PT label. That is §3's pt-BR drift arriving through the *packaging* rather than through the model, and nobody reviewing the diff would see it. The real source is Projecto Natura's, packaged as `hunspell-pt-pt`; the build script now asserts the fault lines (`receção`/`húmido`/`pensámos` in, `recepção`/`úmido`/`ônibus`/`óptimo` out) before it writes a byte, and a vitest greps the built langpack for *sinônimo*, *arquivo*, *tela*, *você*. **Both-dictionaries spellcheck** landed as §3a specifies — flag only what every loaded dictionary rejects, interleave the correction pills so English can't fill all five — and dragged a smaller thing with it: the tokenizer had to become a property of the checker rather than a constant, because `[A-Za-z]` cuts "coração" into "cora", which is both silently unchecked *and* what a right-click would have looked up. The wide alphabet stays off for a writer with no Latin second language, where it could only earn her new squiggles. **Gloss both directions**: a Latin pair has no script boundary, so *data*, *sale* and *comum* are words on both sides and there is no honest way to know which she meant — Petal asks both and shows what answers, which needs no detector and therefore cannot be wrong about her writing. The reverse direction deliberately skips the English de-inflection walk, which over Portuguese would be right by accident and wrong by rule. **Writing the tests found the bug**: `extendedAlphabet` was a snapshot taken when the checker was built while `correct`/`suggest` read live — and her dictionary arrives *after* English, so the underlines would have been right while every lookup still resolved "cora". go build/vet/test, tsc, vite, vitest 116/116 clean; the shipped asset loaded in a real nspell; live smoke on a throwaway DB served both files and left the zh lookup untouched. **Two things outstanding and both said plainly**: the pack has *not* been read by a pt-PT speaker (SUGGESTIONS §3's own bar, and not one I can meet), and this laptop has no `dict.db`, so the reverse-lookup path is covered by a fixture rather than by a real collision — the first of those happens on the VPS.
 - 2026-07-27: **dict.db rebuilt with Spanish, and a log line caught lying** (user: "if we need to redeploy DreamDict to add Spanish support, then do so"). Millenia's dreamdict checkout held ~490 lines of uncommitted work; rather than pull over it, comparing file contents showed an earlier draft of the regional-variant work already committed upstream — nothing unique, but not mine to discard, so it was left alone and the rebuild ran from a clean clone pushed over from the laptop (millenia has no GitHub SSH). Import took 6m15s and added **es: 102,971 words**, leaving en/fr/pt-PT/zh byte-identical — the check that distinguishes "added a language" from "quietly changed everything". Coverage measured before shipping: **es 68.6%**, the best of the four; **zh re-measured at 53.2%**, so the ECDICT decision stands on fresh evidence rather than on the earlier number. Shipped direct millenia→parodia over headscale, hashed both ends, kept the April file for rollback. **The rebuild's real find was in Petal, not DreamDict**: the startup line reported `dictionary.Langs()`, a compile-time constant of *supported* languages, so it had been printing a cheerful `[en fr pt-PT es zh]` over a database with no Spanish in it — the exact failure it existed to catch, reported as success, and something I had already claimed as proof the deploy was good. It now counts rows. Chasing a failed SUBTLEX-US download (benign — the loader falls back to `.txt`) also confirmed English "frequency" is mostly SCOWL's commonness bucket, which independently vindicates the band chip reading `difficulty` instead.
 - 2026-07-27: **Phase 20 — DreamDict becomes the dictionary for every pair but Chinese** (user: "let's continue the build plan"; scope confirmed: build the seam against the existing April `dict.db`, rebuild it later, code + local verification only). The prerequisite was bigger than the plan recorded: renaming DreamDict's module path was necessary but useless on its own, because the query layer lived in `internal/dictionary` and no module may import another's `internal`. Both fixed upstream — the package is now `dictionary`, with a comment saying why *reading* a built database is public API while the loaders that build one stay internal. In Petal, `Provider` is the two questions the popover already asked, so the embedded `*Lexicon` satisfied it with no changes at all, and `Set.For(lang)` is the one place the choice is made. **The measurement is the story of the phase.** `MULTIUSER_PLAN.md` mapped `Gloss ← Translate(word, "en", L1)` 1:1; against the real 452 MB database that table answers for **17%** of the 2,000 commonest English words into pt-PT. Wiktionary's translation sections are thin in that direction — "ephemeral", "think" and "quickly" have no en→pt-PT row at all. The shared-synset path answers for **61%**, so a new upstream `Equivalents` queries that and falls back to translations for 62% combined. Then the *ordering* was wrong in an instructive way: sorting by target frequency glosses "think" as *lembrar* — "remember" — because lembrar is commoner in Portuguese, even though pensar shares six of think's synsets to lembrar's one. Counting sense agreement first fixes it (think → pensar; write → escrever; garden → jardim). The same measurement is what kept **zh on ECDICT**: DreamDict reaches a Chinese gloss for 53% of those words where ECDICT reaches nearly all — the plan said converge only if quality holds, and it didn't. Two other decisions worth keeping: a missing `dict.db` is **not an error** (a laptop has never had one) but a present-and-unimported one is; and a pt-PT writer without a dictionary falls back to the embedded datasets **with the gloss suppressed**, keeping the English half rather than blanking the popover — an empty field reads as "not found", the wrong language reads as broken. The new fields surface as **three** bands, not five, because the difficulty score can separate "everyday" from "you'll have to explain this" but cannot rank *obfuscate* against *serendipity*, and a finer scale would be a confident-looking lie. Writing the tests found two bugs first: `trimEtymology` sliced by byte, which would have emitted invalid UTF-8 for precisely the Greek and Latin etymologies the feature exists for, and its ellipsis path overran its own cap. go build/vet/test, tsc, vite, vitest 96/96 clean in both repos; live smoke on a throwaway DB against the real dictionary, one instance flipped from zh to pt-PT mid-run. **Then deployed, with Phase 19** (user: "do it"): dreamdict pushed to GitHub, the `replace` swapped for a real pseudo-version, encrypted off-box backup first, `dict.db` copied into the LUKS volume and SHA-256-verified, then a rebuild — no migration in either phase, so `schema_migrations` stayed at 11 and her writing came through untouched (8 documents, 33 versions, 103 suggestions, FTS matching, integrity ok). Both accounts are on the zh pair, so **nothing she sees changed today**; what shipped is the capacity for the next pair. Outstanding: the deployed `dict.db` predates DreamDict's Spanish data and needs rebuilding before the es pair ships.
 - 2026-07-27: **Phase 18 deployed, and Phase 19 — the copy stops being hardcoded Mandarin** (user: "let's continue the build plan"; sequencing confirmed: rehearse + deploy 18, then start 19). The rehearsal the previous session was blocked from running went first: a `VACUUM INTO` snapshot of the live VPS database, migrated locally by the Phase-18 binary, every count unchanged and FTS/integrity/foreign keys clean, `personal_words` created empty — then the deploy itself (off-box encrypted backup, rebuild, all three containers healthy, `0011` applied to the live DB with her writing untouched, `/api/spell/words` 401 without a session over public HTTPS). **One check was refused and not worked around**: minting a probe session row to see the endpoint answer 200 for a real cookie reads as credential fabrication to this session's classifier; the endpoint's lifecycle is covered by tests and the shared middleware governs that last step for every other route. **Phase 19** then lifted every `中文 · English` literal out of ~29 files into `web/src/i18n` — one `Pack` type, a verbatim `zh` pack, and two access paths chosen by *when* copy is built: `usePack()` for components, `pack()` for the companion and prose checker, which compose a line when something happens rather than when something renders. The interesting decisions were about what a pack must be allowed to control: **every string with a value in it is a function** (`reviewDue(n)`, `daysAgo(n)`, even English pluralisation) because word order isn't universal; the roster constants keep only value + emoji so a label can never drift from its key; and `gradeBand` returns a band *name* rather than a label. On the server, `internal/llm/lang.go` replaces "Simplified Chinese" in the three prompts that name her language — with pt-PT spelled **"European Portuguese (pt-PT, never Brazilian Portuguese)"** in the prompt itself, and her word for "why" carried alongside so the tutor still recognises the question. `pair_lang` is read **in the row-scoped query each handler already ran**, not a second lookup that could disagree with it — and the test for that was checked by breaking the join and watching it fail. go build/vet/test, tsc, vite, vitest 90/90 clean; live smoke on a throwaway DB. **Phase 19 is not deployed** — no migration, so it's a rebuild whenever the user wants it.