a216614c81c6f3f7465f7c5fcdbcb4e10b989231
5
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 prosolis
|
77f284f65c |
The zh pair's other direction, and a rule pack that mostly says no
`pair_lang` had always been answering a second question nobody asked: it says which two languages, and every surface built on it assumed English was the one being learned. That is why hanzi is never tokenized, never spell-checked, never glossed — correct for a Mandarin native practising English, backwards for an English native practising Mandarin. `users.direction` (migration 0016) separates the two questions; a `zh-learner` pair code would have been cheaper and would have made two directions of one pair look like two unrelated languages to every query. Segmentation is what replaces `wordAt` where there are no spaces: a shortest-path walk over log-probabilities, 232 ms and 14 MB for 188,522 words. The browser gets the word list because segmentation runs on hover; the server keeps the whole dictionary. Their coverage gates come out opposite on purpose — the client list is frequency-gated because the segmentation is measurably identical without the tail, and the dictionary is gated by nothing, because its only power is to explain and the word a learner stops on is the rare one. The 错别字 pack is 24 confusable pairs behind two mechanical gates. One admits a pair only if the wrong form is not a dictionary word and the right form is, which is why it refuses 自已 for 自己 — a real error whose wrong form is a headword. The other asks the segmenter whether the two characters already belong to two different words, without which 自己经常, 睡觉的时候 and 不知到底 would all be corrupted silently into text still made of real characters. Not deployed (this carries a migration), not seen in a browser, and no account has ever been in the learner direction. The IME composition guards were in scope and are not done — see BUILD_PLAN Phase 26. |
||
|
prosolis
|
9224c44fff |
The es pair, and a dictionary that was quietly Spain's
Phase 25. Spanish was never built — the groundwork was all [x] (DreamDict data, the prompt language, the L1 rule gating, TTS env-discovery), which is why the plan read as though it had shipped. shippedPairs was the honest answer: the server had been refusing es on purpose. The langpack is neutral Latin American, chosen with the user: tú, ustedes, no vosotros, and the pan-American half of every vocabulary split. A vitest greps for the peninsular twins the way fr is greped for québécismes — including coger, which is not merely regional but obscene through most of Latin America. The dictionary is the story. Debian's hunspell-es symlinks twenty country codes to one file, which reads as pan-Hispanic; RLA publishes twenty-four builds per release, one per country plus a generic es that is the union, and Debian ships peninsular es_ES. The 58,622-form gap is essentially voseo, so the first version of this commit underlined vení and tenés as misspellings and called it a considered gap. The MUST_ACCEPT list was written to catch exactly that and structurally could not: it asserted the pan-Hispanic vocabulary, and every RLA variant carries the full pan-Hispanic vocabulary — only the paradigms are localised. The REP table cited as the second witness is shared by all builds too. Two independent-looking proofs, neither able to distinguish anything, agreeing with each other. The profile now demands what discriminates, each verified against the build it targets: voseo rejects es_ES and Debian, vosotros rejects es_MX, and arepa/chévere/bacán reject es_AR, which has both paradigms and would otherwise pass. 717,640 forms, 1.74 MB gzipped, 762 ms / 97 MB in a real nspell. fr and pt-PT rebuild byte-identical from their own upstream debs, so the shared script still means what it meant. Shipping the union is fr's call arrived at from the other side: coût and cout are both correct French, tienes and tenés are both correct Spanish. The dictionary holds every variety because underlining is all it can do; the copy picks a register because speaking requires one. Reviewed by four models at the usual >=2-of-4 threshold, 5 of 27 findings applied — one catching the bedtime proverb as fr's Qui dort dîne calqued into Spanish, gloss and all, which is the rule the fr header states. One below-threshold finding (a missing ¡, seen by 1 of 4 because an absent opening mark has no closing ! to look wrong against) was applied and turned into an assertion instead: the suite now rejects any native line that closes ? or ! without opening one. piper-es on es_MX-ald-medium, not the es_ES-davefx-medium the plan named — six of Piper's nine Spanish voices are peninsular, so the obvious pick was the pt-PT trap through a different door. go build/vet/test, tsc, vite, vitest 251/251. Not deployed, not seen in a browser, not read by a native speaker, and no es account exists. |
||
|
prosolis
|
071ea7b835 |
Petal learns French, and the pack that shipped was misspelling itself
Phase 24, the fr half: langpack, Hunspell dictionary, Piper voice, and the lexicon coverage that turned out to have been measured already (63.1%, better than pt-PT's 62.1%). No migration; not deployed. The plan recorded that build_ptpt_dictionary.py "generalizes" to French. It did not. It handled single-character flags and plain PFX/SFX and stopped on everything else, and fr.aff uses four of the things it stopped on. FLAG long is the dangerous one: French flags are two characters, so the old reader's set(flagstr) yields a bag of unrelated letters and expands every entry through the wrong paradigm without ever erroring. Plus continuation flags (French really does affix an affixed form), NEEDAFFIX on 68,075 of 84,140 stems, and FULLSTRIP. Renamed build_hunspell_dictionary.py with a per-language profile, asserting that CIRCUMFIX and FORBIDDENWORD are still unused rather than assuming it — and it rebuilds pt-PT byte-identical to the shipped asset, which is the only thing that makes "generalized" a claim rather than a hope. Elision was decided by building both halves and measuring. Keeping l'arbre and its thirty-three siblings: 3,159,832 forms, 8.25 MB gzipped. Dropping them: 473,326 and 1.19 MB. They are not new words, but the tokenizer keeps internal apostrophes, so they genuinely would have been underlined — so they moved out of the dictionary into withElision, which splits at a known clitic and still requires the remainder to be a word (l'zzzz stays flagged). Real nspell: 369 ms and 74 MB, against pt-PT's 842 ms and 139 MB, on the larger language. Where the regional trap lives is the mirror image of Portuguese's: every fr_* Piper voice is fr_FR and Debian's fr_FR/fr_CA/fr_BE dictionaries are one shared word list, so nothing can be quietly wrong about the country and the whole decision sits in the copy. What French has instead is the 1990 reform, packaged three ways; comprehensive ships, because Petal never corrects her French and coût and cout are both correct. Then the interim review pass, at the user's suggestion and explicitly "for now": four models read each Latin pack independently, and only findings at least two of them reached on their own were applied — five per pack. It earned its keep on the pack that was already live. pt-PT was carrying pre-Acordo spellings (adjectivos, actualmente) in a file whose own header commits to post-Acordo, plus Brazilian decepção, because the Phase 21 greps checked for Brazilian vocabulary and never checked the pack against its own spelling policy. That grep now exists and was confirmed to fail on the old text before being kept. Where reviewers agreed a line was wrong but split on the fix, the wording is mine and the reasoning is in BUILD_PLAN rather than averaged away. Still owed, and both packs now say so precisely: a quorum of models agreeing is agreement, not authority. No native speaker has read either pack, and none of this has been seen in a browser. go build/vet/test clean, tsc, vite build, vitest 190/190. Claude-Session: https://claude.ai/code/session_016y6gyuHkQXPiEuW8RGQyua |
||
|
prosolis
|
1f4ca4775a |
Let her choose her own pair
Raised by the user, not by the plan: there was no way to change language in the mobile UI. There was no way anywhere. `users.pair_lang` has been readable since Phase 19 and writable by nobody — /api/me was GET-only and Upsert deliberately skips the column — which is also why "no pt-PT account exists yet" has stood through two phases. Nothing could create one. PATCH /api/me answers with the whole user rather than 204, so the client re-reads the pair from the server instead of trusting its own request. One write reaches everything: langpack, Hunspell dictionary, Piper voice, lexicon provider and prompt language all read the column at use time. The server refuses a pair it has no copy for, and auth.shippedPairs is deliberately not internal/llm's list. That one names pairs the prompts can talk about (fr and es, since Phase 19); this one names pairs Petal can render itself in, which needs a langpack. Storing fr today would strand her on Chinese with no way back except a lucky guess at a button she cannot read. The picker sits in the sidebar footer because the sidebar is the mobile drawer — always one tap away. The status bar exists only while a document is open, which is the wrong moment to find the app speaking a language you can't read. Each language names itself, 中文 and Português: the one place bilingual copy would get in the way. Claude-Session: https://claude.ai/code/session_016y6gyuHkQXPiEuW8RGQyua |
||
|
prosolis
|
1cf207d73f |
Phase 16: Petal authenticates for itself
Petal is now an OIDC client in its own right rather than trusting a header from the proxy. The Phase-0 Resolver seam was the only integration point: main.go picks the session store when Authentik is configured and the static local user otherwise, and no handler or query moved for either. internal/auth gains three pieces. session.go issues an opaque cookie token and stores only its SHA-256, so a database copy yields nothing usable; the 30-day expiry slides on every request, throttled to one write an hour, and logout deletes the row rather than just the cookie. oidc.go runs the authorization-code flow with state, nonce and PKCE, and discovers the provider lazily and on retry — an Authentik outage should block new logins without stopping Petal booting or invalidating live sessions. users.go provisions accounts from the token's claims and gates them on an allowlist that matches emails as well as subject ids, since a subject is an opaque uuid that doesn't exist until someone has already logged in once. Migration 0010 lands sessions, images and users.pair_lang together. The images table closes the capability-URL hole the Phase-0 audit flagged: a hash was previously enough to fetch anyone's picture. Rows are keyed (name, user_id) so one file can have several owners and deduplication survives; a stranger gets 404 rather than 403, the cache header drops to private, and files already on disk are claimed at startup or every image already pasted into a document would 404. On the frontend a single 401 interceptor feeds a warm bilingual sign-in overlay, drawn over a still-visible editor because nothing has been taken away. Behind it is the part that matters: a save that comes back 401 stashes its body to localStorage before anything else and stops the auto-save loop, and reopening that document after signing in merges the draft back and saves it. An expired session must not cost writing. Writing the round-trip test against a stub identity provider turned up a real bug: the one-shot state/nonce/PKCE cookies were cleared in a defer, which runs after the redirect has written the response header, so the clearing Set-Cookie was silently dropped and they lingered for their full ten minutes. Also swaps the emoji favicon for a drawn sakura, which renders as Petal's own rose palette everywhere instead of whatever each platform's font decides, and doubles as the app tile in Authentik. Migration 0010 verified against a VACUUM INTO copy of the live millenia database: counts intact, FTS still matching, the one existing image claimed. Claude-Session: https://claude.ai/code/session_016y6gyuHkQXPiEuW8RGQyua |