Ratify the language-learning direction and expand it into Phases 15-22

SUGGESTIONS.md (new): the product rationale. Every user gets one
(English + X) pair, X in {zh, pt-PT, fr, maybe es} - bilingual UI in the
pair, type in either language, direction inferred without a detector
(both-dictionaries spellcheck, show-both gloss on collision). Langpacks
keyed by X. LLM-minimalism as a standing principle: the LLM never gates
essential functionality; grammar-lite rules and an embedded
miscollocation list are planned as code-first layers beneath the LLM
families.

MULTIUSER_PLAN.md: all OPEN decisions settled - in-app OIDC (the
parodia.dev VPS hosting plan decides it), 30-day sliding sessions,
allowlist, migration-by-script, image-store ownership fixed alongside
auth, DreamDict imported as a package reading dict.db (module rename
prereq lives in the dreamdict repo), zh stays on ECDICT until compared.

BUILD_PLAN.md: the deferred bucket becomes checkboxed Phases 15-22
(deploy plumbing / OIDC / local-user migration / client-state
namespacing / langpack extraction / DreamDict provider / pt-PT pair /
learning loop + code-first layers) with standing rules: isolation tests
in the same commit as any user-scoped endpoint, LLM-minimalism, and
bilingual aesthetic as acceptance criteria.
This commit is contained in:
prosolis
2026-07-26 22:24:04 -07:00
parent 316b6b305d
commit dae1213c68
3 changed files with 510 additions and 52 deletions
+364
View File
@@ -0,0 +1,364 @@
# Petal — product suggestions: becoming essential for language learners
**Status:** written 2026-07-26 against `feat/writing-passport`; **ratified by
the user 2026-07-26** (recommendations accepted — reviewer Q1Q3 settled
below; Q4Q6 remain genuinely open and don't block execution). This document
is the *why*; the checkboxed execution phases live in `BUILD_PLAN.md`
(Phase 15 onward).
**The brief:** make Petal essential for two audiences — ESL writers (native
Mandarin / pt-PT / French → English), and English natives learning Mandarin,
European Portuguese, French, maybe Spanish. Preserve privacy and warmth.
**The language model (settled by the user, 2026-07-26):** every user has
exactly one language **pair, with English always one half** — (en + X),
X ∈ {zh, pt-PT, fr, maybe es}. This is a deliberate scope decision: never an
X↔Y pair without English, never more than one pair per user. The UI is
bilingual in the pair everywhere (tips, pet responses, cards), and the user
may **type in either language of the pair**; Petal infers direction from the
text rather than asking.
---
## 1. What the pair model implies
The wife's zh setup is already exactly this — she writes Mandarin and English
mixed, the UI is zh+en bilingual, and Petal adapts per span (CJK is never
spellchecked, English words gloss to Chinese). So the pair model isn't a new
design; it's a *promotion of today's behavior to the spec*. Three consequences:
- **Schema:** one column, `users.pair_lang` (the X half; default `'zh'`).
No per-document language, no target/native split. Add it in whatever
migration Phase B's provisioning touches — one column now vs. a real
migration later, the same logic that put `user_id` in the schema on day one.
- **Direction is inferred, not declared.** The zh pair gets inference for free
(script boundaries separate the languages). Latin pairs don't — see §3a,
which is the one genuinely new problem the pair model creates.
- **Every bilingual surface stays two-language**, just parameterized: the
`中文 · English` pattern becomes `X · English`. Nothing about the UI's shape
changes, which is why the copy extraction in §2 is safe to do early.
## 2. Languages as data, not code ("langpacks")
Adding pt-PT today means editing code in many places. A quick census: **29
frontend files** carry hardcoded zh-first bilingual strings (`tips.ts`,
`GardenPanel`, `StatusBar`, `WordCard`, every popover…), plus the
Mandarin-first prompt copy in `internal/llm/prompts.go`. Adding each new
language by hunting through those files doesn't scale to four pairs and would
slowly erode the bilingual-copy quality that makes Petal feel cared-for.
Because English is always one half, a **langpack is keyed by X alone** — one
pack per pair, holding everything that varies:
- UI copy pairs — extract the existing `中文 · English` strings into a copy
module; the current strings become the `zh` pack verbatim, so nothing
visible changes. This is the biggest single chore in the whole effort;
better done once than per-language.
- Companion tip/cheer/bedtime lines (`tips.ts` is already data-shaped —
closest to done).
- LLM prompt copy: bilingual explanation phrasing, "natives usually say…"
example pairs, and the both-directions framing (the text may be English, X,
or mixed — respond appropriately).
- Hunspell dictionary for X where one exists (`pt-PT`, `fr`, `es` upstream;
zh has none — see §4). The en-US dictionary is shared by every pair.
- DreamDict wiring: gloss both directions (`en→X` and `X→en`), phonetics.
- Piper voices for X (both for reading X text aloud and as the L1 voice);
font stack (CJK stacks only for zh).
Shared across all pairs, untouched: nspell en-US, the English IPA dataset, the
EN Piper voice, and all of the editor machinery.
This is refactoring, not product, so it's tempting to skip. Don't: it's the
difference between "Spanish is a data drop" and "Spanish is a month."
## 3. Sequencing: Latin-script targets first, and in this order
**pt-PT → fr → es.** Everything needed for these exists already: Hunspell
dictionaries, Piper voices, DreamDict data (en/fr/pt-PT/zh), and — critically —
the entire decoration/anchoring machinery (`wordAt`, spell tokenizer, suggestion
re-anchoring) already works, because these languages are space-delimited and
Latin-script like English.
Caveats worth writing down now:
- **DreamDict has no Spanish.** "Maybe Spanish" is gated on adding es to
DreamDict first, or a separate dataset. Cheap to note, expensive to discover
later.
- **pt-BR drift is the main quality risk.** Qwen will default to Brazilian
Portuguese in both explanations and "natives say…" examples. Prompts must pin
European Portuguese explicitly, and the pt-PT pack should be reviewed by a
pt-PT speaker before it's trusted — same standard the zh copy got by being
written for a real reader. The multi-user plan's ECDICT-vs-DreamDict
compare-on-real-lookups discipline applies here too.
### 3a. The Latin+Latin wrinkle: inferring direction without a script boundary
The zh pair gets "which language is this word?" for free — the script answers
it, and all of today's behavior (CJK never spellchecked, English words gloss
to Chinese) hangs off that. In an en+fr or en+pt pair, both halves are Latin
script, so the two per-word decisions need a real answer:
- **Spellcheck:** load both Hunspell dictionaries and pass a token if *either*
accepts it; flag only words wrong in both. This never falsely squiggles
correct writing in either language — the failure mode is missing a French
word that happens to be a valid English word, which is the gentle direction
to fail in. Correction pills can offer both dictionaries' suggestions.
- **Gloss/WordCard:** look the word up in both directions via DreamDict; if it
exists in only one language, done. For collisions (*chat*, *pain*, *sale*
are all real words in both English and French), show both compactly — a
two-line card ("🇫🇷 chat → cat · 🇬🇧 chat → bavarder") is honest, needs no
detector, and is arguably *delightful* for a learner. A sentence-level
language guess can order the lines, but shouldn't hide either.
No trained language detector, no heuristics that can be wrong about someone's
writing — both-dictionaries membership plus show-both-on-collision covers it.
The LLM passes need nothing: the prompt already sees the mixed text whole.
- The Hunspell tokenizer's current rule "CJK is never tokenized" stays correct
for the zh pair unchanged.
## 4. The zh pair's *other* direction is a separate epic — say so explicitly
The en+zh pair already exists, but only one direction of it is built: today
Petal deliberately ignores typed hanzi (never tokenized, never flagged, never
glossed) — exactly right for a zh-native writer practicing English, and
exactly insufficient for an English native *learning* Chinese, for whom the
hanzi side is the whole point. Supporting that direction breaks assumptions
that are load-bearing everywhere:
- No spaces → `wordAt`, the spell tokenizer, and word-boundary lookups need
real word segmentation (a jieba-style segmenter, client- or server-side).
- Hunspell has no concept of Chinese; "spellcheck" becomes wrong-character
(错别字) detection — a different problem, probably LLM-assisted.
- Smart-typography input rules and the IME interact; input rules are currently
ASCII-gated, which is correct, but selection/caret behavior mid-IME
composition needs testing.
- The learning aids that matter are different: pinyin annotation (useful here,
unlike for the current user who reads hanzi), tone-mark help, HSK-level word
difficulty, hanzi stroke/handwriting practice.
None of this is unbuildable, but it is its **own phase with its own spec**, not
part of the langpack drop. Recommendation: ship the pt-PT/fr pairs first to
prove the pair model, and treat learner-facing Chinese writing as Petal's next
big product bet after that — it's also the most differentiated one (very few
warm, private tools exist for writing practice in Chinese).
## 5. Deepening the learning loop (all local, all gentle)
Petal's suggestion pipeline currently *corrects and forgets*. The vocabulary
garden proved that capturing what the user already does (lookups) creates a
learning surface for free. The same move is available twice more:
### 5a. Growth journal (patterns from accepted suggestions)
Accepted grammar/collocation suggestions are a record of what the writer is
learning. Aggregate them **locally** into gentle patterns: "this month you've
mostly stopped mixing 在/at" / "make a decision has stuck — you've used it
right 4 times since." Two framing rules that keep it warm: it reports *growth*,
never an error tally, and it only ever compares the writer to her own past
self. Feeds the companion's cheer pool with genuinely personal material
("上次你还问过这个词,这次自己用对了! 🌱"). Data is already in the
`suggestions` table (status + type + original/replacement); this is a read-side
feature, no new capture needed.
### 5b. Plant accepted collocations in the garden
An accepted collocation ("do a decision" → "make a decision") is a learnable
chunk, exactly like a looked-up word. Auto-capture it into the vocabulary
garden as a phrase card (the SM-2-lite scheduler doesn't care that it's two
words). The garden then reflects *both* halves of learning: words she sought
out, and phrasing she was gently given.
### 5c. Companion as tutor-lite: a daily invitation to write
The companion nudges about breaks and bedtime but never *invites writing*. A
once-a-day bilingual prompt ("写 50 个字:今天让你微笑的一件小事 · Write 50
words: one small thing that made you smile today"), offered when a session
starts with no doc open. Explicitly **no streaks, no guilt** — the existing
no-streak-shaming ethos in the SR scheduler is the right precedent; a declined
prompt just gets a sleepy "好吧,我继续睡 😴". Prompt lists live in the
native-language pack.
### 5d. Use DreamDict's richer fields
The multi-user plan notes DreamDict carries `Frequency`, `Difficulty`,
`Antonyms`, `Etymology` with "no equivalent" in Petal. Three cheap, high-value
surfaces:
- A **frequency/difficulty chip** in the WordCard ("common word" / "advanced")
— helps a learner decide whether a word is worth gardening.
- **Etymology for the en-native audience**: Romance-language learners live on
cognates; a one-line "from Latin *decidere*, like English *decide*" is the
single best memory hook for pt/fr/es vocabulary.
- **False friends**: a small curated list per pair (en↔pt: *embarrassed*
*embaraçada*-adjacent traps, *actually*/*atualmente*; en↔fr likewise),
surfaced as a warm heads-up in the WordCard and as a collocation-style
gentle flag when one is used suspiciously. Tiny data, disproportionate
trust-building — this is the mistake that makes learners feel foolish, and
catching it kindly is very Petal.
### 5e. Read-aloud, slower
Piper voices exist per target language; wire the target-language voice into the
existing 🔊 surfaces, and add a **slow toggle** (Piper's `length_scale`) —
learners replaying a sentence at 0.75× is one of the oldest, most-loved
listening aids, and it's a query parameter away.
## 6. LLM-minimalism: essential help in plain code, the model as garnish
**Stated by the user (2026-07-26):** with the LLM on the far side of a VPN,
preserve as much essential functionality as possible in ordinary code inside
Petal, and rely on the LLM as little as possible. This deserves to be a
standing design principle, not just a deployment reaction — it's also what
keeps Petal instant (no 38-second checkpoints for things a lookup can answer)
and private by construction.
Where Petal stands today, by dependency:
| Already pure code (survives VPN-down) | LLM-only today |
|---|---|
| Spellcheck (Hunspell), gloss/definitions/synonyms/phonetics (embedded lexicon → DreamDict), thesaurus, vocabulary garden + SR review, search, tags, versions + writing passport, export, find/replace, typography, TTS (Piper, VPS-local) | Grammar checkpoint, collocation coach, voice pass, Ask Petal, tone rewrite |
Everything in §5 lands in the left column by design (growth journal, garden
planting, daily prompts, DreamDict fields, false friends — all lookups and
local aggregation). The right column splits into two groups:
**Worth a code-first layer (the essential two):**
- **Grammar lite** — a rule-pack of high-precision, data-driven checks for
the classic ESL patterns: a/an before vowel sounds, uncountables
("informations", "advices", "furnitures"), subjectverb agreement in simple
clauses, doubled comparatives, common preposition pairs ("depend of" →
"depend on"), per-pair L1-interference rules (zh: dropped articles, he/she
slips; pt/fr: "have X years" for age). These run instantly on every edit —
no debounce, no 30s rate limit — as a fourth suggestion family through the
existing rail. The bar is **precision over recall**: an offline rule must be
near-certain before it flags, because a wrong correction is colder than a
missed one. LanguageTool's open rule corpus is a mineable source for
vetted patterns (extract data, not the Java).
- **Collocation data** — the same curated-list move as false friends: the
do/make, say/tell, strong-tea/heavy-rain families that fill every ESL
collocation workbook are a few hundred entries of data, not a model. A
small embedded miscollocation list catches the top offenders offline; the
LLM pass, when reachable, adds the long tail. Same family, same rail, same
warm phrasing — the writer never needs to know which engine spoke.
**Inherently LLM (degrade warmly, don't imitate):** Ask Petal, tone rewrite,
and the voice pass are open-ended language generation — a code fake would be
worse than the existing honest "小助手在休息" state. Leave them as the
garnish they are.
The framing that falls out: **the LLM never holds essential functionality
hostage.** Every daily-writing need — spell, define, gloss, pronounce, catch
the common mistakes, review vocabulary, prove authorship — works on a
disconnected VPS. The model adds depth and conversation when the tunnel is up.
## 7. The writing passport is an ESL flagship — treat it as one
The passport exists because AI detectors misfire on non-native English (the
commit message cites the Stanford TOEFL finding). That's not a side feature —
for the ESL audience it may be *the* reason to adopt Petal over any other
editor: **the tool that protects you from being wrongly accused, instead of
scoring you.** No product change needed beyond making sure it works identically
for any target language (it should — it's language-agnostic snapshot history).
Worth a prominent place in the README/landing copy when Petal gets one.
## 8. Ties into MULTIUSER_PLAN.md
For the open questions there, this document's brief implies:
- **OPEN #1 (auth):** Option B (in-app OIDC), and the planned deployment
settles it. The user's stated topology (2026-07-26) is: **Petal hosted on
the parodia.dev VPS, reaching vLLM on millenia over headscale VPN.** A
public-internet app is exactly the case where "must never be reachable
except through Traefik" is a footgun — one proxy misconfiguration on a VPS
and forged identity headers reach the app. In-app OIDC is safe to expose
directly.
- **Deployment topology consequences** worth writing into the plan's Phase 2
(deploy plumbing):
- The LLM becomes the only cross-VPN runtime dependency (Piper is already
installed on parodia.dev, so TTS stays VPS-local). The warm
"小助手在休息" degradation path was built for a flaky co-tenant Ollama; a
VPN link-down hits the same path, so the architecture already fails
gently — but checkpoint latency now includes a WAN+VPN round trip, worth
a look at the 60s LLM timeout.
- Everything offline-by-design (DreamDict lookups, spellcheck, gloss,
garden, search, the whole editor) keeps working when the VPN is down —
another argument for MULTIUSER_PLAN Option 3 over an HTTP dictionary
service, which would otherwise add a second cross-machine dependency.
- vLLM and Piper on millenia should bind to the headscale interface only,
never 0.0.0.0 on the LAN-facing side.
- The writing moves onto rented VPS disk. "The writing never leaves the
box" (§8) becomes "the box is a VPS" — at-rest encryption and an
off-VPS backup of `petal.db` (e.g. nightly to millenia over the same
VPN) deserve a line in the deploy phase.
- **OPEN #6a (DreamDict):** Option 3 (import, read-only `dict.db`), agreed —
it's the only option where four languages stay offline and instant, which
§6 and §9 treat as non-negotiable.
- **Phase B provisioning** should set `users.pair_lang` from the operator's
provisioning step or a first-run picker — add the column in the same
migration. (The plan's Phase D "native language becomes a `users` column"
becomes this: one column, the X half of the pair.)
- **localStorage namespacing** matters slightly more than the plan says once a
household mixes pairs: the personal spell dictionary is per-*language* as
well as per-user (a user's en words and pt-PT words must not merge into one
Hunspell overlay). Key by `user + lang`.
## 9. Privacy & warmth guardrails (the checklist for every item above)
Everything suggested here passes these; future ideas should too.
1. **Offline-first, always — and code-first (§6).** Every essential surface
must work with the LLM unreachable and the network unplugged
(DreamDict-as-local-file preserves this; an HTTP dictionary service would
not). The LLM only ever adds depth to something that already works.
Cloud APIs are off the table even when they'd be easier.
2. **The writing never leaves the box.** No telemetry, no "anonymous usage
stats," ever. The growth journal (§5a) is computed locally from local rows.
3. **No scores, no percentages, no red.** Petal already refuses AI-detection
scores and classic red squiggles; the growth journal and false-friend flags
must hold the same line — evidence and gentle phrasing, never grades.
4. **No streaks, no guilt.** The SR scheduler set the precedent (gentle
"again", no wipe). Daily prompts (§5c) are invitations, not obligations.
5. **Both languages of the pair, always visible** — every explanation, tip,
and pet response renders bilingual in (en + X). That's the warmth: being
helped in the language you think in, next to the one you're learning.
6. **The kitten stays asleep.** Every new companion behavior routes through
the existing mood/cooldown engine; 瞌睡猫 keeps mumbling helpful things
without waking up. (A reactive-animation puppy is on the wishlist — low
priority per the user; the `companions.ts` roster + mood engine is already
the drop-in point, richer per-mood Lottie segments are the only new work.)
## 10. Suggested sequence (interleaved with the multi-user plan's)
1. Add the `users.pair_lang` column (with the Phase B migration or sooner).
2. Extract the bilingual UI copy into the langpack (zh pack = today's strings
verbatim; pure refactor, no visible change).
3. DreamDict integration per MULTIUSER_PLAN Option 3 (module rename → lexicon
provider → pt-PT/fr wired first, zh compared before converging).
4. pt-PT as the first full second pair: Hunspell pt-PT, Piper pt-PT voices,
pinned-pt-PT prompts, native-speaker copy review, and the both-dictionaries
spellcheck + show-both-gloss behavior from §3a. French follows the same
groove; Spanish gated on DreamDict es data.
5. Learning-loop features (§5) — each is small and independent; growth journal
and garden-planting of collocations first, since they're read-side over
existing data.
6. Code-first layers (§6): the embedded miscollocation list first (same shape
as false friends, drops into the existing collocation family), then
grammar lite as its own suggestion family. Both are per-pair data, so
they slot naturally into the langpacks from step 2.
7. Learner-facing Chinese writing (the zh pair's second direction): spec it as
its own phase (§4) only after the pair model is proven on pt-PT/fr.
## 11. Questions for the reviewer (13 settled 2026-07-26)
1. ~~§3a's no-detector stance~~ **Settled: yes** — pass if either dictionary
accepts it, show both glosses on collision, no language detector.
2. ~~UI-copy extraction first?~~ **Settled: yes** — the extraction (§2) is a
prerequisite chore, done before pt-PT is wired.
3. ~~Growth journal framing~~ **Settled: build it** with the two framing rules
as hard constraints (growth only, self-comparison only).
4. When (not whether) to build the learner-facing hanzi direction of the zh
pair — after pt-PT/fr, or is it wanted sooner?
5. Spanish: worth asking DreamDict to grow an es dataset now, or park it?
6. Grammar lite (§6): hand-curate the rule pack from ESL teaching materials
(small, fully understood), or mine LanguageTool's open rule corpus for
vetted patterns (bigger head start, needs licensing + quality triage)?