Files
petal/SUGGESTIONS.md
prosolis dae1213c68 Ratify the language-learning direction and expand it into Phases 15-22
SUGGESTIONS.md (new): the product rationale. Every user gets one
(English + X) pair, X in {zh, pt-PT, fr, maybe es} - bilingual UI in the
pair, type in either language, direction inferred without a detector
(both-dictionaries spellcheck, show-both gloss on collision). Langpacks
keyed by X. LLM-minimalism as a standing principle: the LLM never gates
essential functionality; grammar-lite rules and an embedded
miscollocation list are planned as code-first layers beneath the LLM
families.

MULTIUSER_PLAN.md: all OPEN decisions settled - in-app OIDC (the
parodia.dev VPS hosting plan decides it), 30-day sliding sessions,
allowlist, migration-by-script, image-store ownership fixed alongside
auth, DreamDict imported as a package reading dict.db (module rename
prereq lives in the dreamdict repo), zh stays on ECDICT until compared.

BUILD_PLAN.md: the deferred bucket becomes checkboxed Phases 15-22
(deploy plumbing / OIDC / local-user migration / client-state
namespacing / langpack extraction / DreamDict provider / pt-PT pair /
learning loop + code-first layers) with standing rules: isolation tests
in the same commit as any user-scoped endpoint, LLM-minimalism, and
bilingual aesthetic as acceptance criteria.
2026-07-26 22:24:04 -07:00

21 KiB
Raw Permalink Blame History

Petal — product suggestions: becoming essential for language learners

Status: written 2026-07-26 against feat/writing-passport; ratified by the user 2026-07-26 (recommendations accepted — reviewer Q1Q3 settled below; Q4Q6 remain genuinely open and don't block execution). This document is the why; the checkboxed execution phases live in BUILD_PLAN.md (Phase 15 onward).

The brief: make Petal essential for two audiences — ESL writers (native Mandarin / pt-PT / French → English), and English natives learning Mandarin, European Portuguese, French, maybe Spanish. Preserve privacy and warmth.

The language model (settled by the user, 2026-07-26): every user has exactly one language pair, with English always one half — (en + X), X ∈ {zh, pt-PT, fr, maybe es}. This is a deliberate scope decision: never an X↔Y pair without English, never more than one pair per user. The UI is bilingual in the pair everywhere (tips, pet responses, cards), and the user may type in either language of the pair; Petal infers direction from the text rather than asking.


1. What the pair model implies

The wife's zh setup is already exactly this — she writes Mandarin and English mixed, the UI is zh+en bilingual, and Petal adapts per span (CJK is never spellchecked, English words gloss to Chinese). So the pair model isn't a new design; it's a promotion of today's behavior to the spec. Three consequences:

  • Schema: one column, users.pair_lang (the X half; default 'zh'). No per-document language, no target/native split. Add it in whatever migration Phase B's provisioning touches — one column now vs. a real migration later, the same logic that put user_id in the schema on day one.
  • Direction is inferred, not declared. The zh pair gets inference for free (script boundaries separate the languages). Latin pairs don't — see §3a, which is the one genuinely new problem the pair model creates.
  • Every bilingual surface stays two-language, just parameterized: the 中文 · English pattern becomes X · English. Nothing about the UI's shape changes, which is why the copy extraction in §2 is safe to do early.

2. Languages as data, not code ("langpacks")

Adding pt-PT today means editing code in many places. A quick census: 29 frontend files carry hardcoded zh-first bilingual strings (tips.ts, GardenPanel, StatusBar, WordCard, every popover…), plus the Mandarin-first prompt copy in internal/llm/prompts.go. Adding each new language by hunting through those files doesn't scale to four pairs and would slowly erode the bilingual-copy quality that makes Petal feel cared-for.

Because English is always one half, a langpack is keyed by X alone — one pack per pair, holding everything that varies:

  • UI copy pairs — extract the existing 中文 · English strings into a copy module; the current strings become the zh pack verbatim, so nothing visible changes. This is the biggest single chore in the whole effort; better done once than per-language.
  • Companion tip/cheer/bedtime lines (tips.ts is already data-shaped — closest to done).
  • LLM prompt copy: bilingual explanation phrasing, "natives usually say…" example pairs, and the both-directions framing (the text may be English, X, or mixed — respond appropriately).
  • Hunspell dictionary for X where one exists (pt-PT, fr, es upstream; zh has none — see §4). The en-US dictionary is shared by every pair.
  • DreamDict wiring: gloss both directions (en→X and X→en), phonetics.
  • Piper voices for X (both for reading X text aloud and as the L1 voice); font stack (CJK stacks only for zh).

Shared across all pairs, untouched: nspell en-US, the English IPA dataset, the EN Piper voice, and all of the editor machinery.

This is refactoring, not product, so it's tempting to skip. Don't: it's the difference between "Spanish is a data drop" and "Spanish is a month."

3. Sequencing: Latin-script targets first, and in this order

pt-PT → fr → es. Everything needed for these exists already: Hunspell dictionaries, Piper voices, DreamDict data (en/fr/pt-PT/zh), and — critically — the entire decoration/anchoring machinery (wordAt, spell tokenizer, suggestion re-anchoring) already works, because these languages are space-delimited and Latin-script like English.

Caveats worth writing down now:

  • DreamDict has no Spanish. "Maybe Spanish" is gated on adding es to DreamDict first, or a separate dataset. Cheap to note, expensive to discover later.
  • pt-BR drift is the main quality risk. Qwen will default to Brazilian Portuguese in both explanations and "natives say…" examples. Prompts must pin European Portuguese explicitly, and the pt-PT pack should be reviewed by a pt-PT speaker before it's trusted — same standard the zh copy got by being written for a real reader. The multi-user plan's ECDICT-vs-DreamDict compare-on-real-lookups discipline applies here too.

3a. The Latin+Latin wrinkle: inferring direction without a script boundary

The zh pair gets "which language is this word?" for free — the script answers it, and all of today's behavior (CJK never spellchecked, English words gloss to Chinese) hangs off that. In an en+fr or en+pt pair, both halves are Latin script, so the two per-word decisions need a real answer:

  • Spellcheck: load both Hunspell dictionaries and pass a token if either accepts it; flag only words wrong in both. This never falsely squiggles correct writing in either language — the failure mode is missing a French word that happens to be a valid English word, which is the gentle direction to fail in. Correction pills can offer both dictionaries' suggestions.
  • Gloss/WordCard: look the word up in both directions via DreamDict; if it exists in only one language, done. For collisions (chat, pain, sale are all real words in both English and French), show both compactly — a two-line card ("🇫🇷 chat → cat · 🇬🇧 chat → bavarder") is honest, needs no detector, and is arguably delightful for a learner. A sentence-level language guess can order the lines, but shouldn't hide either.

No trained language detector, no heuristics that can be wrong about someone's writing — both-dictionaries membership plus show-both-on-collision covers it. The LLM passes need nothing: the prompt already sees the mixed text whole.

  • The Hunspell tokenizer's current rule "CJK is never tokenized" stays correct for the zh pair unchanged.

4. The zh pair's other direction is a separate epic — say so explicitly

The en+zh pair already exists, but only one direction of it is built: today Petal deliberately ignores typed hanzi (never tokenized, never flagged, never glossed) — exactly right for a zh-native writer practicing English, and exactly insufficient for an English native learning Chinese, for whom the hanzi side is the whole point. Supporting that direction breaks assumptions that are load-bearing everywhere:

  • No spaces → wordAt, the spell tokenizer, and word-boundary lookups need real word segmentation (a jieba-style segmenter, client- or server-side).
  • Hunspell has no concept of Chinese; "spellcheck" becomes wrong-character (错别字) detection — a different problem, probably LLM-assisted.
  • Smart-typography input rules and the IME interact; input rules are currently ASCII-gated, which is correct, but selection/caret behavior mid-IME composition needs testing.
  • The learning aids that matter are different: pinyin annotation (useful here, unlike for the current user who reads hanzi), tone-mark help, HSK-level word difficulty, hanzi stroke/handwriting practice.

None of this is unbuildable, but it is its own phase with its own spec, not part of the langpack drop. Recommendation: ship the pt-PT/fr pairs first to prove the pair model, and treat learner-facing Chinese writing as Petal's next big product bet after that — it's also the most differentiated one (very few warm, private tools exist for writing practice in Chinese).

5. Deepening the learning loop (all local, all gentle)

Petal's suggestion pipeline currently corrects and forgets. The vocabulary garden proved that capturing what the user already does (lookups) creates a learning surface for free. The same move is available twice more:

5a. Growth journal (patterns from accepted suggestions)

Accepted grammar/collocation suggestions are a record of what the writer is learning. Aggregate them locally into gentle patterns: "this month you've mostly stopped mixing 在/at" / "make a decision has stuck — you've used it right 4 times since." Two framing rules that keep it warm: it reports growth, never an error tally, and it only ever compares the writer to her own past self. Feeds the companion's cheer pool with genuinely personal material ("上次你还问过这个词,这次自己用对了! 🌱"). Data is already in the suggestions table (status + type + original/replacement); this is a read-side feature, no new capture needed.

5b. Plant accepted collocations in the garden

An accepted collocation ("do a decision" → "make a decision") is a learnable chunk, exactly like a looked-up word. Auto-capture it into the vocabulary garden as a phrase card (the SM-2-lite scheduler doesn't care that it's two words). The garden then reflects both halves of learning: words she sought out, and phrasing she was gently given.

5c. Companion as tutor-lite: a daily invitation to write

The companion nudges about breaks and bedtime but never invites writing. A once-a-day bilingual prompt ("写 50 个字:今天让你微笑的一件小事 · Write 50 words: one small thing that made you smile today"), offered when a session starts with no doc open. Explicitly no streaks, no guilt — the existing no-streak-shaming ethos in the SR scheduler is the right precedent; a declined prompt just gets a sleepy "好吧,我继续睡 😴". Prompt lists live in the native-language pack.

5d. Use DreamDict's richer fields

The multi-user plan notes DreamDict carries Frequency, Difficulty, Antonyms, Etymology with "no equivalent" in Petal. Three cheap, high-value surfaces:

  • A frequency/difficulty chip in the WordCard ("common word" / "advanced") — helps a learner decide whether a word is worth gardening.
  • Etymology for the en-native audience: Romance-language learners live on cognates; a one-line "from Latin decidere, like English decide" is the single best memory hook for pt/fr/es vocabulary.
  • False friends: a small curated list per pair (en↔pt: embarrassedembaraçada-adjacent traps, actually/atualmente; en↔fr likewise), surfaced as a warm heads-up in the WordCard and as a collocation-style gentle flag when one is used suspiciously. Tiny data, disproportionate trust-building — this is the mistake that makes learners feel foolish, and catching it kindly is very Petal.

5e. Read-aloud, slower

Piper voices exist per target language; wire the target-language voice into the existing 🔊 surfaces, and add a slow toggle (Piper's length_scale) — learners replaying a sentence at 0.75× is one of the oldest, most-loved listening aids, and it's a query parameter away.

6. LLM-minimalism: essential help in plain code, the model as garnish

Stated by the user (2026-07-26): with the LLM on the far side of a VPN, preserve as much essential functionality as possible in ordinary code inside Petal, and rely on the LLM as little as possible. This deserves to be a standing design principle, not just a deployment reaction — it's also what keeps Petal instant (no 38-second checkpoints for things a lookup can answer) and private by construction.

Where Petal stands today, by dependency:

Already pure code (survives VPN-down) LLM-only today
Spellcheck (Hunspell), gloss/definitions/synonyms/phonetics (embedded lexicon → DreamDict), thesaurus, vocabulary garden + SR review, search, tags, versions + writing passport, export, find/replace, typography, TTS (Piper, VPS-local) Grammar checkpoint, collocation coach, voice pass, Ask Petal, tone rewrite

Everything in §5 lands in the left column by design (growth journal, garden planting, daily prompts, DreamDict fields, false friends — all lookups and local aggregation). The right column splits into two groups:

Worth a code-first layer (the essential two):

  • Grammar lite — a rule-pack of high-precision, data-driven checks for the classic ESL patterns: a/an before vowel sounds, uncountables ("informations", "advices", "furnitures"), subjectverb agreement in simple clauses, doubled comparatives, common preposition pairs ("depend of" → "depend on"), per-pair L1-interference rules (zh: dropped articles, he/she slips; pt/fr: "have X years" for age). These run instantly on every edit — no debounce, no 30s rate limit — as a fourth suggestion family through the existing rail. The bar is precision over recall: an offline rule must be near-certain before it flags, because a wrong correction is colder than a missed one. LanguageTool's open rule corpus is a mineable source for vetted patterns (extract data, not the Java).
  • Collocation data — the same curated-list move as false friends: the do/make, say/tell, strong-tea/heavy-rain families that fill every ESL collocation workbook are a few hundred entries of data, not a model. A small embedded miscollocation list catches the top offenders offline; the LLM pass, when reachable, adds the long tail. Same family, same rail, same warm phrasing — the writer never needs to know which engine spoke.

Inherently LLM (degrade warmly, don't imitate): Ask Petal, tone rewrite, and the voice pass are open-ended language generation — a code fake would be worse than the existing honest "小助手在休息" state. Leave them as the garnish they are.

The framing that falls out: the LLM never holds essential functionality hostage. Every daily-writing need — spell, define, gloss, pronounce, catch the common mistakes, review vocabulary, prove authorship — works on a disconnected VPS. The model adds depth and conversation when the tunnel is up.

7. The writing passport is an ESL flagship — treat it as one

The passport exists because AI detectors misfire on non-native English (the commit message cites the Stanford TOEFL finding). That's not a side feature — for the ESL audience it may be the reason to adopt Petal over any other editor: the tool that protects you from being wrongly accused, instead of scoring you. No product change needed beyond making sure it works identically for any target language (it should — it's language-agnostic snapshot history). Worth a prominent place in the README/landing copy when Petal gets one.

8. Ties into MULTIUSER_PLAN.md

For the open questions there, this document's brief implies:

  • OPEN #1 (auth): Option B (in-app OIDC), and the planned deployment settles it. The user's stated topology (2026-07-26) is: Petal hosted on the parodia.dev VPS, reaching vLLM on millenia over headscale VPN. A public-internet app is exactly the case where "must never be reachable except through Traefik" is a footgun — one proxy misconfiguration on a VPS and forged identity headers reach the app. In-app OIDC is safe to expose directly.
  • Deployment topology consequences worth writing into the plan's Phase 2 (deploy plumbing):
    • The LLM becomes the only cross-VPN runtime dependency (Piper is already installed on parodia.dev, so TTS stays VPS-local). The warm "小助手在休息" degradation path was built for a flaky co-tenant Ollama; a VPN link-down hits the same path, so the architecture already fails gently — but checkpoint latency now includes a WAN+VPN round trip, worth a look at the 60s LLM timeout.
    • Everything offline-by-design (DreamDict lookups, spellcheck, gloss, garden, search, the whole editor) keeps working when the VPN is down — another argument for MULTIUSER_PLAN Option 3 over an HTTP dictionary service, which would otherwise add a second cross-machine dependency.
    • vLLM and Piper on millenia should bind to the headscale interface only, never 0.0.0.0 on the LAN-facing side.
    • The writing moves onto rented VPS disk. "The writing never leaves the box" (§8) becomes "the box is a VPS" — at-rest encryption and an off-VPS backup of petal.db (e.g. nightly to millenia over the same VPN) deserve a line in the deploy phase.
  • OPEN #6a (DreamDict): Option 3 (import, read-only dict.db), agreed — it's the only option where four languages stay offline and instant, which §6 and §9 treat as non-negotiable.
  • Phase B provisioning should set users.pair_lang from the operator's provisioning step or a first-run picker — add the column in the same migration. (The plan's Phase D "native language becomes a users column" becomes this: one column, the X half of the pair.)
  • localStorage namespacing matters slightly more than the plan says once a household mixes pairs: the personal spell dictionary is per-language as well as per-user (a user's en words and pt-PT words must not merge into one Hunspell overlay). Key by user + lang.

9. Privacy & warmth guardrails (the checklist for every item above)

Everything suggested here passes these; future ideas should too.

  1. Offline-first, always — and code-first (§6). Every essential surface must work with the LLM unreachable and the network unplugged (DreamDict-as-local-file preserves this; an HTTP dictionary service would not). The LLM only ever adds depth to something that already works. Cloud APIs are off the table even when they'd be easier.
  2. The writing never leaves the box. No telemetry, no "anonymous usage stats," ever. The growth journal (§5a) is computed locally from local rows.
  3. No scores, no percentages, no red. Petal already refuses AI-detection scores and classic red squiggles; the growth journal and false-friend flags must hold the same line — evidence and gentle phrasing, never grades.
  4. No streaks, no guilt. The SR scheduler set the precedent (gentle "again", no wipe). Daily prompts (§5c) are invitations, not obligations.
  5. Both languages of the pair, always visible — every explanation, tip, and pet response renders bilingual in (en + X). That's the warmth: being helped in the language you think in, next to the one you're learning.
  6. The kitten stays asleep. Every new companion behavior routes through the existing mood/cooldown engine; 瞌睡猫 keeps mumbling helpful things without waking up. (A reactive-animation puppy is on the wishlist — low priority per the user; the companions.ts roster + mood engine is already the drop-in point, richer per-mood Lottie segments are the only new work.)

10. Suggested sequence (interleaved with the multi-user plan's)

  1. Add the users.pair_lang column (with the Phase B migration or sooner).
  2. Extract the bilingual UI copy into the langpack (zh pack = today's strings verbatim; pure refactor, no visible change).
  3. DreamDict integration per MULTIUSER_PLAN Option 3 (module rename → lexicon provider → pt-PT/fr wired first, zh compared before converging).
  4. pt-PT as the first full second pair: Hunspell pt-PT, Piper pt-PT voices, pinned-pt-PT prompts, native-speaker copy review, and the both-dictionaries spellcheck + show-both-gloss behavior from §3a. French follows the same groove; Spanish gated on DreamDict es data.
  5. Learning-loop features (§5) — each is small and independent; growth journal and garden-planting of collocations first, since they're read-side over existing data.
  6. Code-first layers (§6): the embedded miscollocation list first (same shape as false friends, drops into the existing collocation family), then grammar lite as its own suggestion family. Both are per-pair data, so they slot naturally into the langpacks from step 2.
  7. Learner-facing Chinese writing (the zh pair's second direction): spec it as its own phase (§4) only after the pair model is proven on pt-PT/fr.

11. Questions for the reviewer (13 settled 2026-07-26)

  1. §3a's no-detector stance Settled: yes — pass if either dictionary accepts it, show both glosses on collision, no language detector.
  2. UI-copy extraction first? Settled: yes — the extraction (§2) is a prerequisite chore, done before pt-PT is wired.
  3. Growth journal framing Settled: build it with the two framing rules as hard constraints (growth only, self-comparison only).
  4. When (not whether) to build the learner-facing hanzi direction of the zh pair — after pt-PT/fr, or is it wanted sooner?
  5. Spanish: worth asking DreamDict to grow an es dataset now, or park it?
  6. Grammar lite (§6): hand-curate the rule pack from ESL teaching materials (small, fully understood), or mine LanguageTool's open rule corpus for vetted patterns (bigger head start, needs licensing + quality triage)?