Correct the language she wrote in, not the one she was practising

Every pass was English-shaped: CheckpointMessages took the text and the tone and
nothing else, so there was never a language decision to get wrong. On the live
build two pt-PT sentences drew no cards at all — Petal read the Portuguese, said
nothing about it, and filed a mechanics note about the one English line.

The rule is two decisions reading different state. What gets corrected follows
the document. What language the explanation is written in follows the writer —
the half of her pair she is not learning, from users.direction — because an
explanation is teaching, and teaching lands in the language she reads most
easily. Those coincide for every account that exists today (learnerPairs is
{"zh"}), which is a fact about the roster and not about the design, so Target
keeps them apart. It carries a third language too: the collocation gloss is
addressed to her rather than to the document, and folding it into Explain would
have quietly moved it into English on every English document.

The document verdict is a proportion, not a presence — one Portuguese quotation
must not flip an English essay. Per sentence, three-way: pair, English, or no
answer. The third value is the load-bearing one; counting the undecided as
English is exactly what would hold a journal of short Portuguese sentences in
English forever, so the Latin pairs needed an englishMarkers list curated against
pt/fr/es as carefully as latinMarkers was curated against English. Hysteresis at
70/40 because a bilingual paragraph would otherwise alternate its cards' language
every few keystrokes, and hysteresis needs a yesterday — hence the column. Plus a
corroboration floor: a ratio computed over "Não. Eu." is 100% of nothing, and a
flip rewrites every card in the document.

The verdict folds into the chunk salt beside the tone, so a document that changes
language re-opens every sentence rather than serving back cards in a language it
no longer speaks.

checkpointSystemPrompt could not simply take a language — it opens by naming the
reader an ESL learner, and appending "explain in Portuguese" hands the model two
contradictory framings. Separate constants, sharing the JSON contract below the
framing. Both carry a "never translate it into English" line, which is the
instruction the model will most want to disobey. The English prompts are
untouched byte for byte, and a golden says so out loud.

Collocation deliberately did not move: its prompt is per-language knowledge, not
framing, and "natives usually say" for Portuguese is a claim Petal cannot back.

Not deployed and not smoked against a real model. The tests drive the real router
and a real DB; what none of them prove is how Qwen behaves on a Portuguese
document, in particular whether the never-translate line holds.

Claude-Session: https://claude.ai/code/session_01GJHNvirh7Hzhc9RL3HAvz7

internal/suggestions/doclang.go

@@ -0,0 +1,179 @@
+package suggestions
+
+import "strings"
+
+// What language is this DOCUMENT in — as opposed to this span.
+//
+// `readsAsPairLang` next door answers a span-level question for the purpose of
+// labelling one card, and it is written to under-claim: two marker words, or
+// nothing. A whole document needs the opposite temperament. A proportion, not a
+// presence — one Portuguese quotation inside an English essay must not flip the
+// entire pass into Portuguese, and one English sentence at the end of a
+// Portuguese journal must not keep it in English.
+//
+// Three properties, in the order they bite:
+//
+//   - Decided over the WHOLE document, never a chunk. The grammar checkpoint
+//     sends only the sentences that changed, so a verdict computed from what it
+//     asked about would put an English card in a Portuguese journal the moment
+//     she edits its one English line. The caller passes content_text, always.
+//
+//   - Hysteresis. A bilingual paragraph sits near whatever single threshold we
+//     pick, and a document crossing it every few keystrokes would alternate card
+//     languages between passes — the same instability the mascot needed a band
+//     for. Flip to the pair at 70% and back only below 40%; in between, whatever
+//     it was last time stands. The band is the feature, not a rounding
+//     tolerance.
+//
+//   - Plain code, no model call. The house rule is that the LLM is garnish,
+//     never a gatekeeper: a document must not become uncheckable because the
+//     inference box is down.
+const (
+	docLangEnglish = "en"
+	docLangPair    = "pair"
+)
+
+// The band. Deliberately wide: the cost of an unnecessary flip is every card in
+// the document changing language, which is far more startling than a paragraph
+// of mixed writing being read as whichever language it was a minute ago.
+const (
+	flipToPairAt       = 0.70
+	flipToEnglishBelow = 0.40
+)
+
+// documentLang returns the language verdict for a document, given the verdict it
+// carried before. `prev` is "" for a document that has never been read.
+//
+// The result is one of docLangEnglish / docLangPair — not a language code. Which
+// language "pair" means is the writer's `pair_lang`, and keeping the stored
+// verdict relative to her pair means changing her pair doesn't strand a stale
+// language name on every document she owns.
+func documentLang(contentText, pairLang, prev string) string {
+	if prev != docLangPair {
+		prev = docLangEnglish
+	}
+	p := normalizePairLang(pairLang)
+	if !hasLangTest(p) {
+		// No test for this pair: say English, which is what every surface did
+		// before this phase. A wrong flip is louder than a missing one.
+		return docLangEnglish
+	}
+
+	var pair, english int
+	for _, c := range splitChunks(contentText, "") {
+		switch sentenceLang(c.text, p) {
+		case docLangPair:
+			pair++
+		case docLangEnglish:
+			english++
+		}
+	}
+	decided := pair + english
+	if decided == 0 {
+		// Nothing to go on — an empty document, a list of numbers, a title. Hold
+		// the previous verdict rather than resetting a Portuguese journal to
+		// English because she cleared it to start again.
+		return prev
+	}
+
+	ratio := float64(pair) / float64(decided)
+	switch {
+	case ratio >= flipToPairAt && corroborated(contentText, p):
+		return docLangPair
+	case ratio < flipToEnglishBelow:
+		return docLangEnglish
+	default:
+		return prev
+	}
+}
+
+// sentenceLang classifies one sentence as pair-language, English, or neither.
+//
+// Neither is a real answer and carries weight: a sentence with no evidence
+// either way ("Bom dia.", "OK.", a heading) is left out of the ratio entirely
+// rather than counted for the language it isn't. Counting the undecided as
+// English is what would keep a Portuguese document in English forever, since
+// short sentences carry no markers.
+func sentenceLang(s, p string) string {
+	if p == "zh" {
+		han, latin := scriptCounts(s)
+		switch {
+		case han >= 2 && han > latin:
+			return docLangPair
+		case latin >= 3 && latin > han:
+			return docLangEnglish
+		}
+		return ""
+	}
+	// A Latin pair shares its alphabet with English, so both sides are counted
+	// the same way and the larger pile of evidence wins. A tie — including no
+	// evidence at all — is no answer, which is why the English list below is
+	// curated as carefully against the pair languages as theirs is against
+	// English.
+	pair := distinctMarkers(s, latinMarkers[p])
+	eng := distinctMarkers(s, englishMarkers)
+	switch {
+	case pair > eng:
+		return docLangPair
+	case eng > pair:
+		return docLangEnglish
+	}
+	return ""
+}
+
+// corroborated requires the document as a whole to carry real evidence of the
+// pair language before the pass flips into it. A two-sentence document of
+// "Sim." / "Não." would otherwise reach 100% on almost nothing; a flip changes
+// every card in the document, so it has to be earned document-wide and not only
+// in proportion.
+func corroborated(contentText, p string) bool {
+	if p == "zh" {
+		han, _ := scriptCounts(contentText)
+		return han >= 8
+	}
+	return distinctMarkers(contentText, latinMarkers[p]) >= 3
+}
+
+// hasLangTest reports whether readsAsPairLang / sentenceLang know how to test
+// this pair at all. Kept beside the tests it describes so a new pair that adds
+// markers without adding itself here fails loudly in review rather than quietly
+// at runtime.
+func hasLangTest(p string) bool {
+	if p == "zh" {
+		return true
+	}
+	return len(latinMarkers[p]) > 0
+}
+
+// English function words, curated against the pair languages exactly as
+// `latinMarkers` is curated against English.
+//
+// Every word here is one a Portuguese, French or Spanish sentence has no reason
+// to contain. Deliberately absent, each a false English vote waiting to happen
+// in someone's own language: "on" and "son" (French), "as", "a", "o", "e", "no",
+// "os" (Portuguese), "no", "para", "sin" (Spanish), and "is"-alikes that are
+// really other languages' words. The list is short on purpose — it does not need
+// coverage, only a reliable vote in the sentences where the pair list is silent.
+var englishMarkers = words(
+	"the", "and", "is", "are", "was", "were", "be", "been", "being",
+	"of", "to", "that", "this", "these", "those", "with", "from", "for",
+	"have", "has", "had", "they", "them", "their", "there", "then", "than",
+	"what", "which", "when", "where", "why", "how", "who",
+	"will", "would", "should", "could", "can", "about", "because",
+	"into", "some", "such", "only", "very", "much", "many", "other",
+	"our", "your", "its", "it", "he", "she", "we", "you", "but", "not",
+	"just", "like", "also", "most", "over", "after", "before", "between",
+	"through", "said", "says", "get", "got", "make", "made", "know", "think",
+	"thing", "things", "time", "people", "here", "always", "never", "something",
+	"want", "need", "feel", "day", "today", "good", "really", "still", "even",
+)
+
+// normalizeDocLang folds a stored verdict into the two values the rest of the
+// code reasons about: the column holds "" for a document nothing has read yet,
+// and that means English, which is what every surface did before this phase.
+func normalizeDocLang(v string) string {
+	if strings.TrimSpace(v) == docLangPair {
+		return docLangPair
+	}
+	return docLangEnglish
+}