Correct the language she wrote in, not the one she was practising
Every pass was English-shaped: CheckpointMessages took the text and the tone and
nothing else, so there was never a language decision to get wrong. On the live
build two pt-PT sentences drew no cards at all — Petal read the Portuguese, said
nothing about it, and filed a mechanics note about the one English line.
The rule is two decisions reading different state. What gets corrected follows
the document. What language the explanation is written in follows the writer —
the half of her pair she is not learning, from users.direction — because an
explanation is teaching, and teaching lands in the language she reads most
easily. Those coincide for every account that exists today (learnerPairs is
{"zh"}), which is a fact about the roster and not about the design, so Target
keeps them apart. It carries a third language too: the collocation gloss is
addressed to her rather than to the document, and folding it into Explain would
have quietly moved it into English on every English document.
The document verdict is a proportion, not a presence — one Portuguese quotation
must not flip an English essay. Per sentence, three-way: pair, English, or no
answer. The third value is the load-bearing one; counting the undecided as
English is exactly what would hold a journal of short Portuguese sentences in
English forever, so the Latin pairs needed an englishMarkers list curated against
pt/fr/es as carefully as latinMarkers was curated against English. Hysteresis at
70/40 because a bilingual paragraph would otherwise alternate its cards' language
every few keystrokes, and hysteresis needs a yesterday — hence the column. Plus a
corroboration floor: a ratio computed over "Não. Eu." is 100% of nothing, and a
flip rewrites every card in the document.
The verdict folds into the chunk salt beside the tone, so a document that changes
language re-opens every sentence rather than serving back cards in a language it
no longer speaks.
checkpointSystemPrompt could not simply take a language — it opens by naming the
reader an ESL learner, and appending "explain in Portuguese" hands the model two
contradictory framings. Separate constants, sharing the JSON contract below the
framing. Both carry a "never translate it into English" line, which is the
instruction the model will most want to disobey. The English prompts are
untouched byte for byte, and a golden says so out loud.
Collocation deliberately did not move: its prompt is per-language knowledge, not
framing, and "natives usually say" for Portuguese is a claim Petal cannot back.
Not deployed and not smoked against a real model. The tests drive the real router
and a real DB; what none of them prove is how Qwen behaves on a Portuguese
document, in particular whether the never-translate line holds.
Claude-Session: https://claude.ai/code/session_01GJHNvirh7Hzhc9RL3HAvz7
This commit is contained in:
@@ -423,6 +423,36 @@ Phase 26's own outstanding item, and the thing it named as most likely to be wro
|
||||
- ⚠️ **What this still does not prove is the counterfactual** — that the bug would have bitten *here* without the guard. Chrome's own composition handling is robust, and a clean run with the guard in place cannot distinguish "held correctly" from "would have been fine anyway". Settling it means building with the guard reverted and asking her to type once more; offered, not assumed.
|
||||
- **Not deployed.** No migration; a rebuild whenever the user wants it, along with Phases 24–26.
|
||||
|
||||
### Phase 28 — following the writing into her own language (planned 2026-07-28; step (a) built 2026-07-28)
|
||||
Today every correction and every explanation comes back in English, whatever she wrote. `CheckpointMessages` (`internal/llm/prompts.go:49`) takes the text and the tone and **nothing else** — there is no language parameter to pass, so there has never been a language decision to get wrong. The pair language reaches her only on demand: tapping Ask Petal fetches a translation of the English explanation (`AskPetal.tsx:93`), and that panel answers in her language because `AskPetalSystemPrompt` is given one. Right default for a writer practising English; wrong default for a document written in Portuguese, where Petal reads the Portuguese prose, says nothing about it, and files a mechanics note about the one English sentence at the end. **Observed on the live build 2026-07-28**: two pt-PT sentences drew no cards at all.
|
||||
|
||||
1. [x] **The rule is two decisions, not one, and they read different state.** What gets *corrected* follows the **document** — Portuguese prose gets Portuguese corrections, which is the whole point. What language the explanation is *written in* follows the **writer**: the half of her pair she is not learning, from the `direction` column (`internal/auth/users.go:96`), because an explanation is teaching and teaching lands in the language she reads most easily. So a native Portuguese speaker practising English, writing Portuguese, gets Portuguese corrections explained in Portuguese; a native English speaker learning French, writing French, gets French corrections explained in English. Neither is trapped — the other language stays one tap away, in both directions.
|
||||
- **What the direction lookup costs today is nothing.** `learnerPairs` is `{"zh"}` (`users.go:119`), so fr, es and pt-PT accounts are all `learning_en` and their non-learned half *is* the pair language: the rule produces exactly "explain in the document's language" for every writer who exists right now. It is there to stop that from being baked into the prompts, the way "English is the language being learned" was baked into `pair_lang` before migration 0016 — the same mistake this plan's own Phase 26 had to unpick.
|
||||
2. [x] **A document-level verdict, beside the span-level one already in `internal/suggestions/language.go`.** `readsAsPairLang` answers "is this quoted span her language" for labelling one card; it under-claims on purpose and trips at two marker words. A whole document needs a **proportion, not a presence**, or one Portuguese quotation inside an English essay flips the entire pass. Three properties, in the order they bite:
|
||||
- **Decided over the whole document, never a chunk.** A chunked pass sends only the sentences that changed (`splitChunks`/`changedChunks`); computing the verdict from `askText` would put an English card in a Portuguese journal the moment she edits its one English line. Compute from `contentText`, always.
|
||||
- **Hysteresis, for the same reason the mascot needed it** (see the session log below, 2026-07-28). A bilingual paragraph sits near whatever threshold we pick, and a document crossing it every few keystrokes would alternate card languages between passes. Flip to the pair at ≥70% pair-language sentences, back only below 40%. The band is the feature.
|
||||
- **Plain code, no model call** — the house rule that the LLM is garnish, never a gatekeeper. zh is a rune-script count (already written); pt-PT, fr and es reuse `latinMarkers`, aggregated per sentence rather than per span.
|
||||
3. [x] **`checkpointSystemPrompt` cannot simply take a language.** It opens with *"helping someone who speaks English as a second language"* and asks for ESL patterns — appending "explain in Portuguese" hands the model two contradictory instructions. Split it: shared JSON contract and tone clause, framing sentence filled per direction. `CheckpointMessages` and `VoiceMessages` (which has no language at all today) take **two** language arguments — corrected and explained-in — and **resist collapsing them into one `Lang` while every current account has them equal**: that equality is a fact about today's `learnerPairs`, not about the design.
|
||||
4. [x] **The verdict rides with `pairLang` in the row-scoped lookup and folds into the chunk salt** next to `tone` (`handlers.go`). That is the cheap correct answer to stale cards: when a document's language flips, every sentence's identity changes, so old-language cards are re-checked rather than left sitting there in the wrong language.
|
||||
5. [ ] **`isTranslation` learns which way it points.** It currently means "her language rendered into English"; in a flipped document the useful translate card is the mirror image, so the test takes the document verdict and checks the direction that matches.
|
||||
6. [ ] **The tap-through has to stop assuming its direction, and this is not optional dressing** — it is what makes rule 1 safe for a learner reading explanations in English. `/suggestions/{id}/translate` always renders into the pair language today; it should render into whichever half the explanation is *not* already in, and skip the seed entirely when those coincide rather than round-tripping Portuguese into Portuguese.
|
||||
- **`internal/llm/lang.go` already carries the precision the model needs** (`"European Portuguese (pt-PT, never Brazilian Portuguese)"`). Spanish wants the same care pointed the other way: the shipped dictionary deliberately accepts the whole Spanish-speaking world (Phase 25), so the prompt must not quietly impose peninsular usage.
|
||||
- **Spellcheck must not change.** "A word is a misspelling only when *both* dictionaries reject it" (`useSpellChecker.ts:13`) is a deliberate refusal to detect document language, and it is right: it makes English quotations inside Portuguese prose free, in both directions. Teaching it a document verdict buys nothing and costs a false positive on every borrowed word.
|
||||
- Tests worth writing first: doc-level detection per pair (monolingual, 80/20, 50/50, quotation-heavy English), asserting the hysteresis band **from both directions**; the flipped prompt names the target language and drops the ESL framing while the English-document prompt stays **byte-identical** to today's (the path every existing user is on); the two language arguments proven independent by the only pair that can exercise it — a `learning_pair` zh account writing Chinese wants Chinese corrections explained in English, a `learning_en` zh account writing Chinese wants both in Chinese; a handler test in the shape of `pairlang_test.go`; and a language flip invalidating checked chunks.
|
||||
- **Open, and needs a decision before the last step.** The **vocabulary garden** harvests phrases from documents, so a Portuguese document would seed it with Portuguese — tag entries by language and filter by the half being learned, or gate harvesting to English documents? Tagging looks right and touches stored rows. **Read-aloud** should follow the document too (all five voices now run on the VPS), which is probably small and lands where the verdict lands, but has not been traced.
|
||||
- **Order:** (a) detection + salt + checkpoint/voice prompts — the whole visible win, independently shippable; (b) translate-card direction, Ask Petal seed, `/translate` direction; (c) garden and read-aloud, once the two questions above are answered.
|
||||
|
||||
**Step (a) as built, 2026-07-28** — items 1–4. Step (b) and (c) are untouched, and the two open questions under (c) are still open.
|
||||
- `internal/llm/target.go` — `Target{Correct, Explain, Pair}` + `English` (a `Lang` the `langs` map has no business holding: that map answers "which half is hers"). `EnglishTarget(pair)` is the pre-phase behaviour named, and `Flipped()` is the one question the prompts ask. **Three fields, not two**: the collocation coach's parenthetical gloss is addressed to *her* and not to the document, so it reads `Pair` — collapsing it into `Explain` would have silently moved that gloss into English on every English document, which is every document today.
|
||||
- `internal/suggestions/doclang.go` — `documentLang(contentText, pairLang, prev)` → `"en"`/`"pair"`. Per **sentence** (reusing `splitChunks`, salt `""`), three-way: pair / English / **no answer**. The third value is the load-bearing one — a sentence with no evidence ("Bom dia.", a heading) is left out of the ratio rather than counted for the language it isn't, and counting the undecided as English is exactly what would hold a Portuguese journal of short sentences in English forever. So the Latin pairs needed an `englishMarkers` list curated against pt/fr/es with the same care `latinMarkers` was curated against English (no "on"/"son"/"as"/"no"/"para" — each a false English vote inside someone's own language). Band as specified, ≥70% / <40%, plus a **corroboration floor** the plan didn't call for: 3 distinct pair markers (8 Han runes) document-wide, because a ratio computed over "Não. Eu." is 100% of nothing and a flip rewrites every card in the document.
|
||||
- **Migration `0017_document_lang`** — `documents.doc_lang` (`'' | 'en' | 'pair'`). Stored, not recomputed, because hysteresis needs a yesterday; `'pair'` rather than a language code, so changing her pair re-reads her documents instead of stranding a stale language name on all of them. Backfill is the default `''`, which reads as English — true of every document that exists.
|
||||
- `runPass` reads `doc_lang` + `direction` in the row-scoped lookup that already proves ownership, writes the verdict back only when it changed, folds it into the chunk salt beside `tone`, and hands `targetFor(pairLang, direction, docLang)` to the pass. `pass` now takes an `llm.Target`.
|
||||
- Prompts: `pairCheckpointSystemPrompt` / `pairVoiceSystemPrompt` beside the originals — separate constants, not a clause, since the English ones open by naming the reader an ESL learner. Both carry a **"never translate it into English"** line, which is the instruction the model most wants to disobey: asked to improve Portuguese by an assistant that is English-shaped by training, it hands back an English rendering, and that is a translation card and not a correction. The English prompts are untouched, byte for byte, and a golden in `internal/llm/target_test.go` says so out loud (duplicated on purpose — a golden copied from the constant it guards guards nothing).
|
||||
- **Collocation deliberately did not move.** Its prompt is per-language *knowledge*, not framing: "natives usually say" for Portuguese is a claim Petal has no grounds to make yet. A flipped document gets the pass it always got.
|
||||
- Tests: `doclang_test.go` (monolingual pt/zh/en, one-English-line-in-a-Portuguese-journal, English-quoting-Portuguese, untested pair, emptied document holds its verdict, the band **from both directions**, corroboration floor, the two Target decisions proven independent by the only pair that can — a `learning_pair` zh account writing Chinese gets Chinese corrections explained in English — plus handler tests in the shape of `pairlang_test.go`: a pt-PT document reaches the model as a Portuguese checkpoint *and* a Portuguese voice pass, the verdict persists, and a language flip re-opens the already-checked English sentence). `target_test.go` for the prompts. go build/vet/test clean.
|
||||
- **Not deployed, and no live smoke.** The handler tests drive the real router and a real DB, which is what a smoke would have shown; what neither proves is how Qwen behaves on a Portuguese document — in particular whether the "never translate" line holds. That wants the deploy Phases 24–27 are also waiting on, and a pt-PT reader.
|
||||
- **Left standing, deliberately, and visible on a flipped document:** the offline mechanics rule pack is English (it is the one that filed a note about the stray English sentence), `isTranslation` still points one way (item 5), and the Ask Petal seed still round-trips (item 6).
|
||||
|
||||
### Later / explicitly not now
|
||||
- Learner-facing Chinese writing (the zh pair's second direction) — own phase with its own spec (SUGGESTIONS §4); only after Phases 19–21 prove the pair model
|
||||
- ~~Spanish pair — gated on DreamDict growing an es dataset~~ **ungated 2026-07-26**, **shipped (code) 2026-07-28** — see Phase 25. What it still owes: a deploy, a native reader, and a writer who actually uses it.
|
||||
|
||||
Reference in New Issue
Block a user