The zh pair's other direction, and a rule pack that mostly says no
`pair_lang` had always been answering a second question nobody asked: it says which two languages, and every surface built on it assumed English was the one being learned. That is why hanzi is never tokenized, never spell-checked, never glossed — correct for a Mandarin native practising English, backwards for an English native practising Mandarin. `users.direction` (migration 0016) separates the two questions; a `zh-learner` pair code would have been cheaper and would have made two directions of one pair look like two unrelated languages to every query. Segmentation is what replaces `wordAt` where there are no spaces: a shortest-path walk over log-probabilities, 232 ms and 14 MB for 188,522 words. The browser gets the word list because segmentation runs on hover; the server keeps the whole dictionary. Their coverage gates come out opposite on purpose — the client list is frequency-gated because the segmentation is measurably identical without the tail, and the dictionary is gated by nothing, because its only power is to explain and the word a learner stops on is the rare one. The 错别字 pack is 24 confusable pairs behind two mechanical gates. One admits a pair only if the wrong form is not a dictionary word and the right form is, which is why it refuses 自已 for 自己 — a real error whose wrong form is a headword. The other asks the segmenter whether the two characters already belong to two different words, without which 自己经常, 睡觉的时候 and 不知到底 would all be corrupted silently into text still made of real characters. Not deployed (this carries a migration), not seen in a browser, and no account has ever been in the learner direction. The IME composition guards were in scope and are not done — see BUILD_PLAN Phase 26.
This commit is contained in:
Binary file not shown.
@@ -0,0 +1,39 @@
|
||||
import { useEffect, useState } from 'react'
|
||||
|
||||
import { loadSegmenter, type Segmenter } from '../lib/segment'
|
||||
|
||||
// Loads the Chinese word list, once per session, and only for a writer who is
|
||||
// going to use it.
|
||||
//
|
||||
// Modelled on useSpellChecker, and gated harder. That hook loads for everyone,
|
||||
// because everyone's English gets spell-checked; this one loads a megabyte for
|
||||
// the one direction that needs it, and an account practising English would
|
||||
// never ask a single question of it. The gate is the writer's own setting rather
|
||||
// than a guess from their text: a Mandarin native drafting English quotes
|
||||
// Chinese in it constantly, and none of that is what this is for.
|
||||
//
|
||||
// A failure resolves to null, which every consumer already handles as "no
|
||||
// segmentation" — the Chinese hover quietly does nothing rather than the editor
|
||||
// refusing to open.
|
||||
export function useSegmenter(enabled: boolean): Segmenter | null {
|
||||
const [segmenter, setSegmenter] = useState<Segmenter | null>(null)
|
||||
|
||||
useEffect(() => {
|
||||
if (!enabled) {
|
||||
// Turning the direction back drops it. It is a megabyte of resident map
|
||||
// whose only consumer just switched off, and re-loading costs one fetch
|
||||
// that the browser cache answers.
|
||||
setSegmenter(null)
|
||||
return
|
||||
}
|
||||
let cancelled = false
|
||||
loadSegmenter().then((seg) => {
|
||||
if (!cancelled) setSegmenter(seg)
|
||||
})
|
||||
return () => {
|
||||
cancelled = true
|
||||
}
|
||||
}, [enabled])
|
||||
|
||||
return segmenter
|
||||
}
|
||||
@@ -42,5 +42,14 @@ export function useSession() {
|
||||
}
|
||||
}, [])
|
||||
|
||||
return { me, signedOut }
|
||||
// Turn the pair around. The account is the source of truth for which
|
||||
// direction the editor is in — it decides whether the word list loads at all —
|
||||
// so the state moves only once the server has agreed, and it moves to what the
|
||||
// server *stored* rather than to what was asked for.
|
||||
const setDirection = async (direction: string) => {
|
||||
const updated = await api.setDirection(direction)
|
||||
setMe(updated)
|
||||
}
|
||||
|
||||
return { me, signedOut, setDirection }
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user