The zh pair's other direction, and a rule pack that mostly says no

`pair_lang` had always been answering a second question nobody asked: it
says which two languages, and every surface built on it assumed English
was the one being learned. That is why hanzi is never tokenized, never
spell-checked, never glossed — correct for a Mandarin native practising
English, backwards for an English native practising Mandarin.
`users.direction` (migration 0016) separates the two questions; a
`zh-learner` pair code would have been cheaper and would have made two
directions of one pair look like two unrelated languages to every query.

Segmentation is what replaces `wordAt` where there are no spaces: a
shortest-path walk over log-probabilities, 232 ms and 14 MB for 188,522
words. The browser gets the word list because segmentation runs on hover;
the server keeps the whole dictionary. Their coverage gates come out
opposite on purpose — the client list is frequency-gated because the
segmentation is measurably identical without the tail, and the dictionary
is gated by nothing, because its only power is to explain and the word a
learner stops on is the rare one.

The 错别字 pack is 24 confusable pairs behind two mechanical gates. One
admits a pair only if the wrong form is not a dictionary word and the
right form is, which is why it refuses 自已 for 自己 — a real error whose
wrong form is a headword. The other asks the segmenter whether the two
characters already belong to two different words, without which 自己经常,
睡觉的时候 and 不知到底 would all be corrupted silently into text still
made of real characters.

Not deployed (this carries a migration), not seen in a browser, and no
account has ever been in the learner direction. The IME composition
guards were in scope and are not done — see BUILD_PLAN Phase 26.

web/src/components/Companion/hanzi.test.ts

@@ -0,0 +1,141 @@
+import { readFileSync } from 'node:fs'
+import { gunzipSync } from 'node:zlib'
+import { describe, expect, it } from 'vitest'
+
+import { CONFUSION_PAIRS, hanziFindings } from './hanzi'
+import { buildSegmenter } from '../../lib/segment'
+
+// The 错别字 pack, held to the bar Phase 22 set for the English rule pack: every
+// rule pinned in *two* directions — the mistake it must catch, and the correct
+// writing next to it that it must leave alone.
+//
+// Here the second direction is the one that matters, and it is unusually easy to
+// get wrong. Chinese has no spaces, so every one of these rules is a substring
+// match on running text, and for most of them there exists an ordinary correct
+// sentence that contains the substring across a word boundary. Those sentences
+// are the real test.
+
+const raw = gunzipSync(readFileSync(new URL('../../../public/dictionaries/zh/words.txt.gz', import.meta.url)))
+const seg = buildSegmenter(raw.toString('utf8'))
+
+const flagged = (text: string) => hanziFindings(text, seg).map((f) => `${f.original}→${f.replacement}`)
+
+describe('the gate that admits a rule', () => {
+  // The pack's own claim about itself, checked against the shipped dictionary
+  // rather than asserted in a comment. A pair whose wrong form is a real word
+  // cannot be decided mechanically and does not belong here.
+  it('every wrong form is not a word, and every right form is', () => {
+    for (const { wrong, right } of CONFUSION_PAIRS) {
+      expect(seg.has(wrong), `${wrong} is a dictionary word and must not be flagged`).toBe(false)
+      expect(seg.has(right), `${right} is not a dictionary word`).toBe(true)
+    }
+  })
+
+  // The errors this pack deliberately refuses, and why — each is a genuine
+  // mistake by a modern standard whose wrong form is itself a headword. If a
+  // dictionary rebuild ever drops one of these, this test fails and the pair
+  // becomes admissible; that is the intended way to find out.
+  it('refuses the well-known errors it cannot decide', () => {
+    for (const undecidable of ['自已', '好象', '倒底', '帐号', '部份']) {
+      expect(seg.has(undecidable), `${undecidable} is no longer a word — reconsider the rule`).toBe(true)
+      expect(flagged(`这是${undecidable}的例子`)).toEqual([])
+    }
+  })
+})
+
+describe('the mistakes it catches', () => {
+  it('已 / 己 / 以', () => {
+    expect(flagged('我己经写完了作业')).toEqual(['己经→已经'])
+    expect(flagged('我以经吃过饭了')).toEqual(['以经→已经'])
+    expect(flagged('下课已后我们去公园')).toEqual(['已后→以后'])
+  })
+
+  it('在 / 再', () => {
+    expect(flagged('明天在见')).toEqual(['在见→再见'])
+    expect(flagged('他正再看书')).toEqual(['正再→正在'])
+    expect(flagged('现再几点了')).toEqual(['现再→现在'])
+  })
+
+  it('做 / 作', () => {
+    expect(flagged('我的工做很忙')).toEqual(['工做→工作'])
+    expect(flagged('老师给我们很多做业')).toEqual(['做业→作业'])
+    expect(flagged('这本书的做者是谁')).toEqual(['做者→作者'])
+  })
+
+  it('the rest', () => {
+    expect(flagged('我觉的这个很好')).toEqual(['觉的→觉得'])
+    expect(flagged('你因该早点睡')).toEqual(['因该→应该'])
+    expect(flagged('即然你来了就坐下吧')).toEqual(['即然→既然'])
+    expect(flagged('你知到吗')).toEqual(['知到→知道'])
+    expect(flagged('请输入你的蜜码')).toEqual(['蜜码→密码'])
+  })
+
+  it('reports an exact span, so the card replaces the right characters', () => {
+    const text = '我己经到了'
+    const [f] = hanziFindings(text, seg)
+    expect(text.slice(f.from, f.to)).toBe('己经')
+    expect(text.slice(0, f.from) + f.replacement + text.slice(f.to)).toBe('我已经到了')
+  })
+
+  it('finds every occurrence, in document order', () => {
+    expect(flagged('我己经吃了，他也己经吃了')).toEqual(['己经→已经', '己经→已经'])
+    expect(flagged('我的工做很忙，所以我觉的很累')).toEqual(['工做→工作', '觉的→觉得'])
+  })
+})
+
+// ── the direction that matters ──────────────────────────────────────────────
+
+describe('the correct writing it must not touch', () => {
+  // Each of these is an ordinary sentence containing a flagged substring across
+  // a word boundary. Without the boundary gate, every one would be corrupted —
+  // and corrupted silently, into text that is still made of real characters.
+  it('leaves two real words alone where they happen to abut', () => {
+    // 自己 + 经常. The substring is 己经.
+    expect(flagged('他自己经常做饭')).toEqual([])
+    // 睡觉 + 的. The substring is 觉的.
+    expect(flagged('睡觉的时候不要看手机')).toEqual([])
+    // 感觉 + 的.
+    expect(flagged('这是我感觉的方向')).toEqual([])
+    // 不知 + 到底.
+    expect(flagged('我不知到底该怎么办')).toEqual([])
+    // 因 + 位置.
+    expect(flagged('因位置不好我们换了座位')).toEqual([])
+    // 已 + 后悔.
+    expect(flagged('他已后悔了')).toEqual([])
+  })
+
+  it('leaves ordinary correct prose entirely alone', () => {
+    for (const good of [
+      '我今天早上去公园跑步了',
+      '他的中文说得很好',
+      '我已经完成了我的作业',
+      '现在几点了，我们再见面吧',
+      '我觉得这个工作很有意思',
+      '既然你已经知道了，就按照计划做',
+    ]) {
+      expect(flagged(good), good).toEqual([])
+    }
+  })
+
+  // Where the gate costs the pack a real catch, and the trade it is making.
+  // 不知 is itself a word, so 我不知到他在哪里 — which really is 知到 for 知道 —
+  // reads to the segmenter as 不知 + 到 and is left alone. That is the gate
+  // preferring a missed error to a corrupted sentence, which is the whole
+  // premise: 我不知到底该怎么办 is the same three characters and is correct.
+  it('declines a real error rather than risk the sentence beside it', () => {
+    expect(flagged('我不知到他在哪里')).toEqual([])
+    expect(flagged('你知到吗')).toEqual(['知到→知道'])
+  })
+
+  it('says nothing about English, or about nothing', () => {
+    expect(flagged('I already finished my homework')).toEqual([])
+    expect(flagged('')).toEqual([])
+  })
+
+  // The direction gate. The word list is loaded only for an account learning
+  // Chinese, so without one this pack is silent — a writer practising English
+  // must never be told her own quoted Chinese is wrong.
+  it('is silent without a segmenter, which is how the direction gate works', () => {
+    expect(hanziFindings('我己经写完了', null)).toEqual([])
+  })
+})

web/src/components/Companion/hanzi.ts

@@ -0,0 +1,149 @@
+import type { MechanicsFinding } from '../../api/client'
+import type { Segmenter } from '../../lib/segment'
+
+// 错别字 — wrong-character detection, the Chinese counterpart of the spell
+// checker, and a different problem from the one Hunspell solves.
+//
+// Chinese has no misspellings in the English sense: every character a writer can
+// type is a real character, correctly formed, and an IME will not offer one that
+// is not. What it *will* offer is the wrong one. Typing pinyin `yijing` and
+// taking the first candidate gives 已经 or 己经 depending on the moment, and both
+// are made of real characters. So the unit of error is not a malformed word but
+// a **substituted character inside a correct-looking one** — which is why this
+// is a rule pack over confusable pairs rather than a dictionary membership test.
+//
+// The discipline is Phase 22's, and the bar is the same: **precision over
+// recall**. A wrong nudge costs more trust than a missed one earns, and it costs
+// double here, because a learner has no way to know the tool is wrong. Two
+// mechanical gates enforce it, and both are checked in the tests rather than
+// asserted in prose.
+
+// A confusable pair: `wrong` is never a word, `right` is what was meant.
+//
+// **Gate one — the pair must be decidable by the dictionary.** Each entry is
+// admitted only if `wrong` is absent from the 188k-word list *and* `right` is
+// present. That is what makes the correction a fact rather than a preference,
+// and it is checked against the shipped asset in hanzi.test.ts.
+//
+// It is also the gate that keeps out errors everyone knows are errors. 自已 for
+// 自己 is among the commonest slips in written Chinese, and 自已 is itself a
+// dictionary headword — so this pack does not flag it, exactly as Phase 22's
+// English pack left out `married with`. The same fate for 好象 (an older form of
+// 好像, still in the dictionary), 倒底, 帐号 and 部份: all real errors by a modern
+// standard, none of them decidable here.
+interface Confusion {
+  wrong: string
+  right: string
+  // The note on the card. English, because this pack only ever runs for a writer
+  // whose English is the language they think in — see the direction gate below.
+  why: string
+}
+
+const CONFUSIONS: Confusion[] = [
+  // 已 / 己 / 以 — three characters that differ by one stroke and share a
+  // syllable. The most productive source of 错别字 there is.
+  { wrong: '己经', right: '已经', why: '已经 (already) — 己 is the "self" character; the one you want is 已.' },
+  { wrong: '以经', right: '已经', why: '已经 (already) — 以 is a different word; 已 is the one that means "already".' },
+  { wrong: '已后', right: '以后', why: '以后 (afterwards) takes 以, not 已.' },
+
+  // 在 / 再 — same pinyin (zài), completely different jobs: one is location and
+  // ongoing action, the other is repetition.
+  { wrong: '在见', right: '再见', why: '再见 (goodbye) — 再 is "again", which is what "see you again" needs.' },
+  { wrong: '正再', right: '正在', why: '正在 (in the middle of doing) takes 在, the one about being somewhere.' },
+  { wrong: '现再', right: '现在', why: '现在 (now) takes 在.' },
+
+  // 做 / 作 — both zuò, both "to do", and which one a compound takes is simply
+  // fixed by convention. A learner cannot reason it out, which is what makes a
+  // reminder worth having.
+  { wrong: '工做', right: '工作', why: '工作 (work) is written with 作.' },
+  { wrong: '做业', right: '作业', why: '作业 (homework) is written with 作.' },
+  { wrong: '做者', right: '作者', why: '作者 (author) is written with 作.' },
+  { wrong: '做文', right: '作文', why: '作文 (an essay) is written with 作.' },
+  { wrong: '做用', right: '作用', why: '作用 (effect, function) is written with 作.' },
+
+  // 得 / 的 — the pair everyone knows about. Only the fixed compound is flagged:
+  // deciding 的 against 地 against 得 in the general case needs to know whether
+  // the next word is a verb or a noun, which nothing here can tell.
+  { wrong: '觉的', right: '觉得', why: '觉得 (to feel, to think) ends in 得.' },
+
+  // 即 / 既 — one stroke apart, opposite meanings ("namely" against "since").
+  { wrong: '即然', right: '既然', why: '既然 (since, given that) takes 既.' },
+  { wrong: '既使', right: '即使', why: '即使 (even if) takes 即.' },
+
+  // The rest: ordinary IME slips where the wrong character is a homophone.
+  { wrong: '因该', right: '应该', why: '应该 (should) — 因 means "because"; the word you want starts with 应.' },
+  { wrong: '因位', right: '因为', why: '因为 (because) ends in 为.' },
+  { wrong: '知到', right: '知道', why: '知道 (to know) ends in 道.' },
+  { wrong: '安照', right: '按照', why: '按照 (according to) takes 按.' },
+  { wrong: '蜜码', right: '密码', why: '密码 (password) takes 密 — 蜜 is honey.' },
+  { wrong: '犹其', right: '尤其', why: '尤其 (especially) takes 尤.' },
+  { wrong: '甘净', right: '干净', why: '干净 (clean) takes 干.' },
+  { wrong: '什末', right: '什么', why: '什么 (what) ends in 么.' },
+  { wrong: '一像', right: '一样', why: '一样 (the same) ends in 样 — 像 is "to resemble".' },
+  { wrong: '必须品', right: '必需品', why: '必需品 (a necessity) takes 需. 必须 is "must", which is a different word.' },
+]
+
+// **Gate two — the characters must not already belong to two different words.**
+//
+// This is the gate that stops the pack from destroying correct writing, and
+// without it every rule above is dangerous. 自己经常 ("oneself, often") contains
+// the string 己经. 睡觉的时候 ("when sleeping") contains 觉的. 不知到底 contains 知到.
+// A substring match would corrupt all three.
+//
+// The segmenter already knows the difference, so the test is: split the text,
+// and if the two characters land in different tokens *and* either token is a
+// real multi-character word, this is a word boundary and not an error. Two
+// adjacent single-character tokens is what the walk produces when it has nothing
+// better to offer — which is exactly what a mistyped compound looks like.
+function isWordBoundary(tokens: { word: string; from: number; to: number }[], at: number): boolean {
+  const left = tokens.find((t) => at >= t.from && at < t.to)
+  const right = tokens.find((t) => at + 1 >= t.from && at + 1 < t.to)
+  if (!left || !right || left === right) return false
+  return left.word.length > 1 || right.word.length > 1
+}
+
+// hanziFindings returns the 错别字 in a piece of text, as ordinary mechanics
+// findings — the same shape, the same rail, the same cards, the same accept.
+//
+// It needs the segmenter and does nothing without one, which is also the
+// direction gate: the word list is loaded only for an account learning Chinese
+// (useSegmenter), so a writer practising English can never be told her quoted
+// Chinese is wrong. That is not a nicety. Petal deliberately never corrects the
+// pair language — the fr and es dictionaries are chosen to hold every variety
+// precisely so they cannot underline correct writing — and a Mandarin native
+// does not need her own language checked by a rule pack of two dozen entries.
+export function hanziFindings(text: string, segmenter: Segmenter | null): MechanicsFinding[] {
+  if (!segmenter || !text) return []
+  // One segmentation for the whole text, shared by every rule. The walk is
+  // linear, but running it two dozen times over a long document would not be.
+  const tokens = segmenter.segment(text)
+  const found: MechanicsFinding[] = []
+
+  for (const c of CONFUSIONS) {
+    let from = text.indexOf(c.wrong)
+    while (from !== -1) {
+      // The boundary test is asked at the seam the substitution sits on: the
+      // gap between the first two characters, which is where a mistyped
+      // compound and two adjacent words look different from each other.
+      if (!isWordBoundary(tokens, from)) {
+        found.push({
+          from,
+          to: from + c.wrong.length,
+          original: c.wrong,
+          replacement: c.right,
+          explanation: c.why,
+          type: 'mechanics',
+        })
+      }
+      from = text.indexOf(c.wrong, from + 1)
+    }
+  }
+
+  // Document order, so the rail reads down the page rather than down this file.
+  return found.sort((a, b) => a.from - b.from)
+}
+
+// Exported for the tests, which check every pair against the shipped word list.
+// A pack whose own gate is only described in a comment is a pack whose gate can
+// rot; this is how the description is made to stay true.
+export const CONFUSION_PAIRS = CONFUSIONS.map((c) => ({ wrong: c.wrong, right: c.right }))