drwily/petal
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Ratify the language-learning direction and expand it into Phases 15-22

Browse Source 
SUGGESTIONS.md (new): the product rationale. Every user gets one
(English + X) pair, X in {zh, pt-PT, fr, maybe es} - bilingual UI in the
pair, type in either language, direction inferred without a detector
(both-dictionaries spellcheck, show-both gloss on collision). Langpacks
keyed by X. LLM-minimalism as a standing principle: the LLM never gates
essential functionality; grammar-lite rules and an embedded
miscollocation list are planned as code-first layers beneath the LLM
families.

MULTIUSER_PLAN.md: all OPEN decisions settled - in-app OIDC (the
parodia.dev VPS hosting plan decides it), 30-day sliding sessions,
allowlist, migration-by-script, image-store ownership fixed alongside
auth, DreamDict imported as a package reading dict.db (module rename
prereq lives in the dreamdict repo), zh stays on ECDICT until compared.

BUILD_PLAN.md: the deferred bucket becomes checkboxed Phases 15-22
(deploy plumbing / OIDC / local-user migration / client-state
namespacing / langpack extraction / DreamDict provider / pt-PT pair /
learning loop + code-first layers) with standing rules: isolation tests
in the same commit as any user-scoped endpoint, LLM-minimalism, and
bilingual aesthetic as acceptance criteria.
This commit is contained in:
prosolis
2026-07-26 22:24:04 -07:00
parent 316b6b305d
commit dae1213c68
3 changed files with 510 additions and 52 deletions
Download Patch File Download Diff File Expand all files Collapse all files
BUILD_PLAN.md
+78 -3
View File
@@ -136,9 +136,83 @@ Multi-session build. **Source of truth for what's done and what's next.** Update
### Deferred (post-v1-local)
- [x] **Multi-user groundwork** (2026-07-26) — request-scoped identity. New `internal/auth`: `Middleware(Resolver)` resolves the caller once per API request and stores the id in the context; handlers read it via `auth.UserID(r.Context())` instead of naming `db.LocalUserID`. `StaticResolver(db.LocalUserID)` keeps Petal single-user today. **Auth itself is still deferred** — but every query is now scoped to whoever the resolver says is calling, so landing Authentik is a one-line change in `main.go` plus a `Resolver` implementation.
- [ ] Authentik OIDC auth + session middleware ← **next step: write a `Resolver` that validates a session cookie; the plumbing is in place**
- [ ] Copyleaks Tier-2 + webhook HMAC
- [ ] Dockerfile, docker-compose, Traefik, deploy to write.parodia.dev
- [ ] Copyleaks Tier-2 + webhook HMAC (still parked — needs a public webhook; revisit after Phase 15)
- Authentik OIDC + deploy: **no longer deferred — expanded into Phases 15–17 below** (decisions ratified 2026-07-26; see `MULTIUSER_PLAN.md`)
## Execution phases 15–22 (added 2026-07-26)
Decisions behind these are ratified in `MULTIUSER_PLAN.md` (all OPENs settled) and `SUGGESTIONS.md` (the *why*; Q1–Q3 settled). **Standing rules for every phase below:**
- **Isolation tests in the same commit** as any new user-scoped endpoint (the `docs/isolation_test.go` suites are the template — this discipline caught a real unscoped-query bug once already).
- **LLM-minimalism** (SUGGESTIONS §6): the LLM never gates essential functionality; new essential features are code+data first.
- **Aesthetic + bilingual-in-the-pair copy remain acceptance criteria** on every user-visible change.
- Verify per project convention: go build/vet/test, tsc, vite build, vitest, live smoke on a throwaway DB/port.
### Phase 15 — Deploy plumbing (parodia.dev + headscale)
Petal hosted on the parodia.dev VPS; vLLM stays on millenia over headscale. Auth (Phase 16) needs the stable `BASE_URL`/redirect URI this phase creates.
- [ ] Dockerfile (multi-stage: `npm run build` → `go build`, single binary + data dir volume) + docker-compose
- [ ] Traefik route + HTTPS on the chosen hostname (e.g. `write.parodia.dev`); stable `BASE_URL` env
- [ ] `LLM_ENDPOINT` → millenia's **headscale** address; verify vLLM binds only to the headscale interface (not LAN/0.0.0.0); revisit `LLM_TIMEOUT` for the WAN+VPN round trip
- [ ] TTS → the Piper already installed on parodia (VPS-local; per-language instances as on millenia — EN now, zh; see `deploy/` artifacts for the service pattern)
- [ ] Off-VPS nightly backup of `petal.db` (+`-wal`/`-shm` coherently, e.g. `sqlite3 .backup`) to millenia over the VPN; document restore
- [ ] Migrate the live millenia data to the VPS (app stopped, backup first, counts verified) — or decide the millenia instance stays canonical until Phase 17 completes
- [ ] Acceptance: full editor works with the VPN link **down** (spell/gloss/garden/search/export/TTS all fine; checkpoint shows the warm 小助手在休息 state); `/api/health` public; HTTPS end-to-end
### Phase 16 — Auth (in-app OIDC) + image-store ownership
Option B ratified. `go-oidc` + `x/oauth2`; config fields already exist. The `Resolver` seam from Phase 0 is the only integration point.
- [ ] OIDC login flow: `/auth/login` → Authentik → `/auth/callback` (state/CSRF checked) → provision user (upsert from `sub`/email/name) → session
- [ ] `sessions` table migration (id, user_id, expires_at, created_at, user_agent); opaque cookie (`HttpOnly`, `SameSite=Lax`, `Secure` when https); **30-day sliding expiry**; `/auth/logout` revokes server-side
- [ ] Allowlist: `PETAL_ALLOWED_SUBS` env (comma-separated); rejected valid logins get a warm bilingual page, not an error dump
- [ ] `SessionResolver` replaces `StaticResolver` in `main.go` (keep `StaticResolver` for dev via env flag)
- [ ] Frontend 401 interceptor in `api/client.ts`: halt auto-save, preserve the draft (localStorage keyed by doc id), warm bilingual "请重新登录 · Please sign in again" overlay, resume cleanly after re-login — **a 401 mid-draft must never lose writing**
- [ ] **Image store ownership** (same phase, per OPEN #5): `images` table migration (hash, user_id, content_type, size, created_at); fetch joins on caller; dedup preserved (one file, N rows); last-row delete removes the file; backfill existing images to the current user
- [ ] `users.pair_lang` column (default `'zh'`) in this phase's provisioning migration — read by Phase 19+, cheap to add now
- [ ] Isolation tests: sessions (expiry, revocation, cross-user), images, allowlist paths
### Phase 17 — Migrate the `local` user
Script, app stopped, backup first (OPEN #4). Depends on: she logs in once so her OIDC `sub` exists.
- [ ] `scripts/migrate_local_user.*`: single transaction, `PRAGMA foreign_keys=OFF`, re-point `documents`/`tags`/`vocab_words` (versions/suggestions follow parents), delete the empty provisioned row, verify row counts before commit; refuses to run if the app is up or the target has data
- [ ] Runbook documented in the script header; dry-run mode
### Phase 18 — Per-user, per-language client state
- [ ] Namespace `localStorage` keys by user id once known: `petal.spell.personal`, `petal.companion`, sound/petals prefs
- [ ] Personal spell dictionary keyed by **user + language** (en and pt-PT word lists must not merge); consider promoting it to a server table so it follows her across devices (nice-to-have, decide during)
### Phase 19 — Langpack extraction (the copy chore)
Pure refactor, zero visible change; prerequisite for every new pair (SUGGESTIONS §2, Q2 settled).
- [ ] Extract the `中文 · English` strings from the ~29 frontend files + companion `tips.ts` into a langpack copy module keyed by the pair's X; today's strings become the `zh` pack **verbatim**
- [ ] Parameterize `internal/llm/prompts.go` bilingual copy the same way (explanation language, "natives usually say…" framing, both-directions preamble)
- [ ] Wire pack selection to `users.pair_lang`; acceptance: pixel-identical UI for the zh pair, vitest snapshots unchanged
### Phase 20 — DreamDict as a lexicon provider
Option 3 ratified (import package, read-only `dict.db`). **Prerequisite in the dreamdict repo:** rename its module path (or add a `replace` for dev).
- [ ] Provider seam behind the existing `lexicon` interface; DreamDict provider opens `dict.db` read-only (modernc driver, second handle beside `petal.db`); graceful "no data" when the file is absent
- [ ] pt-PT + fr wired to DreamDict (nothing to regress); **zh stays on ECDICT** until compared on real lookups from her documents — converge only if quality holds
- [ ] Surface the new fields where cheap: frequency/difficulty chip in WordCard; etymology line (cognate hook for en-natives)
- [ ] Deploy: `dict.db` ships in the data dir alongside `petal.db`
### Phase 21 — The pt-PT pair (first Latin pair, proves the model)
SUGGESTIONS §1/§3/§3a. French follows the same groove afterwards; Spanish stays gated on DreamDict es data.
- [ ] Hunspell pt-PT vendored like en-US; **both-dictionaries spellcheck** (flag only if wrong in both; pills from both) — the no-detector stance, Q1 settled
- [ ] Gloss/WordCard both directions; on en/pt collisions show both compactly, never hide either
- [ ] Prompts pinned to **European Portuguese, never pt-BR** (explicit in every prompt); pt-PT langpack copy written and **reviewed by a pt-PT speaker before trusted**
- [ ] Piper pt-PT voice instance on parodia; read-aloud + L1 voice wired; slow toggle (`length_scale`) while in there (SUGGESTIONS §5e)
- [ ] Companion tips/cheers/bedtime lines in the pt-PT pack (the kitten speaks pt+en to this user)
- [ ] Acceptance: a pt-PT-pair user gets the full experience end-to-end with the VPN down except LLM passes; zh-pair user sees zero change
### Phase 22 — Learning loop + code-first layers
Each item independent and small; order within is free (SUGGESTIONS §5–§6).
- [ ] **Growth journal** (Q3 settled): local aggregation over accepted suggestions; growth-only, self-comparison-only framing; feeds companion cheers
- [ ] **Plant accepted collocations** in the vocabulary garden as phrase cards (scheduler unchanged)
- [ ] **Daily writing invitation** from the companion (no streaks, declining is fine)
- [ ] **False-friend list** per pair (curated data, WordCard heads-up + gentle flag)
- [ ] **Embedded miscollocation list** (code-first under the collocation family; LLM adds the long tail when reachable)
- [ ] **Grammar lite** rule-pack as a fourth suggestion family: instant, offline, precision-over-recall (near-certain or silent); per-pair L1-interference rules; sourcing per SUGGESTIONS Q6 (hand-curate vs mine LanguageTool's corpus — decide at build time)
### Later / explicitly not now
- Learner-facing Chinese writing (the zh pair's second direction) — own phase with its own spec (SUGGESTIONS §4); only after Phases 19–21 prove the pair model
- Spanish pair — gated on DreamDict growing an es dataset
- Reactive-animation puppy companion — wishlist, low priority; `companions.ts` roster + mood engine is the drop-in point
- Copyleaks Tier-2 — revisit once Phase 15 provides a public webhook endpoint
### Next-up (post-v1 product, agreed with user 2026-06-26)
- [x] **Phase 9 — ESL superpowers**: inline Chinese gloss on hover/select; "say it more naturally" / tone-rewrite. ✅ (see Phase 9 above)
@@ -148,6 +222,7 @@ Multi-session build. **Source of truth for what's done and what's next.** Update
- [x] **Phase 14 — companion warmth + bedtime nag + night mode**: more encouraging phrases, a gentle "go to bed" nudge after 11pm, and a calm dark theme + falling stars at night. ✅ (see Phase 14 above)
## Session log
- 2026-07-26: **Product direction + execution plan ratified** (user: "make it so, number one"). New `SUGGESTIONS.md` (product rationale for the language-learning direction): the **pair model** — every user gets one (English + X) pair, X ∈ {zh, pt-PT, fr, maybe es}, bilingual UI in the pair, type in either language, direction inferred (no detector: both-dictionaries spellcheck, show-both gloss on collision); **langpacks** keyed by X; **LLM-minimalism** as a standing principle (LLM is garnish, never a gatekeeper — grammar-lite rule pack + embedded miscollocation list planned as code-first layers). Deployment settled: Petal on the **parodia.dev VPS**, vLLM on millenia over **headscale** (the only cross-VPN dependency; Piper is VPS-local). All `MULTIUSER_PLAN.md` OPENs ratified: in-app OIDC (B), 30-day sliding sessions, allowlist, migration script, image-store fix with auth, DreamDict via package import (Option 3, module rename prereq in the dreamdict repo), zh stays on ECDICT until compared. Everything expanded into **Phases 15–22** above with standing rules (isolation tests same-commit, LLM-minimalism, bilingual aesthetic). Ready for implementation handoff starting at Phase 15.
- 2026-07-26: **Multi-user groundwork** (user: "let's start preparing Petal for multi-user support"; scope agreed as plumbing-only, aimed at Authentik). New **`internal/auth`** package — context-carried identity (`WithUser`/`UserID`), a `Resolver` seam (`Resolve(*http.Request) (string, error)`), `StaticResolver` for today's single user, and `Middleware` that 401s anything unresolved. `main.go` splits `/api` into a **public group** (`/health`, `/version` — a monitoring probe must not need a session) and an **authenticated group** carrying everything else. All ~35 `db.LocalUserID` call sites across `docs`/`suggestions`/`vocab` now read the caller from the request; helpers that had no request in scope (`fetch`, `ownsDoc`, `ownsTag`, `tagsByDoc`, `fetchVersion`, `passportVersions`, `fetchPending`, vocab `fetch`) take an explicit `userID` param. `UserID` returns `""` rather than panicking when middleware is absent, so a mis-wired route **fails closed** (every query is `WHERE user_id = ?` → matches nothing). **Two real access-control gaps found and fixed while threading**: `setStatus` (accept/dismiss) updated a suggestion by bare id with **no ownership check at all**, and `fetchPending`/`listForDoc` read a document's suggestions by `doc_id` alone — a leak of the quoted source sentences. Both now scope through `documents.user_id`. **A third bug was caught by the new tests, not by the compiler**: `docs.fetch` gained a `userID` parameter but kept binding `db.LocalUserID` in the query — legal Go (unused params compile), silently unscoped, and it would have shipped. New tests: `internal/auth/auth_test.go` (round-trip, absent-context, both 401 paths) and **two-user isolation suites** (`docs/isolation_test.go`, `suggestions/isolation_test.go`) that mount the same routers twice behind two resolvers over one DB and assert a stranger gets 404 on get/update/delete/export/passport/snapshot/version-preview/restore/tag-assign/tag-rename/tag-delete/suggestion-accept/dismiss, sees nothing in list/search/version-list, and leaves the owner's data untouched. go build/vet/test all clean. **Still global, deliberately out of scope** (flagged for the auth phase): the image store is content-addressed with no per-user association or DB row — any authenticated user holding a hash can fetch any image (capability-URL security, needs a table + migration to fix); `export-all` is correctly scoped; frontend `localStorage` keys (`petal.spell.personal`, `petal.companion`, sound/petals prefs) are per-browser, not per-account, so they'd bleed across users sharing a device.
- 2026-06-26: **Phases 12 + 13 complete** (collocation coach + vocabulary garden — "finish the rest of the build plan except Authentik/Traefik"). **Phase 12**: collocation drops in as a third suggestion family reusing the whole `runPass`/`pendingScope` machinery — `llm/collocation.go` (`RunCollocation`, 25s floor, reuses `ParseCheckpoint`), `collocationSystemPrompt`/`CollocationMessages` (warm "Natives usually say…" + Mandarin gloss, defers grammar elsewhere), migration `0005` **rebuilds** the suggestions table to extend the `type` CHECK (SQLite can't ALTER a CHECK), `collocationScope` + `CollocationLimit` + `POST /{id}/collocation`. **Caught a latent bug**: `grammarScope` was `type != 'voice'` → would wipe collocation flags; fixed to `type NOT IN ('voice','collocation')`. Frontend: `--color-blossom` pink, "Make it sound natural 🌸" toolbar pill, `collocating`/`runCollocation` in `useCheckpoint`, StatusBar dot — all into the existing rail/card. **Phase 13**: new `internal/vocab` package — migration `0006_vocab_garden` (`vocab_words`, SM-2-lite columns, doc_id `ON DELETE SET NULL`, `UNIQUE(user_id,word)`), `scheduler.go` (Leitner ladder 1/3/7/16/35 → geometric; gentle "again", no streak-shaming), `handlers.go` (capture-upsert/list/due/review/delete, all owner-scoped, time math via SQLite `datetime()` so stored values stay canonical-UTC). Auto-capture wired into `EditorCore.openWordLookup` (only dictionary-known words, captures the surrounding sentence + doc_id) + a 🤍/💚 toggle on `WordCard`. `GardenPanel` slide-over: blossom grid (bloom stage by reps), flashcard review (sentence blanked, flip, again/good/easy, direction alternates recognition↔production), sleepy-kitten footer; opened from a global 🌷 header button. Tests: `TestCollocationPassCoexists`, vocab `scheduler_test.go` + `handlers_test.go`, db CHECK test extended. go build/vet/test + tsc + vite + vitest (51/51) all clean; migration verified against a copy of the live DB; live backend smoke (throwaway DB) walked the full vocab lifecycle + the warm-502 collocation path. **Remaining: only the deferred infra bucket** — Authentik auth, Copyleaks Tier-2 (needs a public webhook), Docker/Traefik/deploy — all on hold per the user's "except Authentik/Traefik".
- 2026-06-26: **Phase 14 complete** (companion warmth + bedtime nag + night mode). `tips.ts`: `ENCOURAGEMENTS` 5→10 lines; new `BEDTIME` array (4 lines, user-supplied English wit + gentle Mandarin leads). `useCompanion.ts`: bedtime branch in the 10s heartbeat (after idle-return + break, before the generic tip); only nudges while actively writing; own `lastBedtime` ref + 30min `BEDTIME_GAP`, respects `PROACTIVE_GAP`; new `'bedtime'` `BubbleTone` lingers ~4s longer. **Night mode** (added same session, user request): `lib/night.ts` centralizes `isBedtime()` + window (now shared by the nag too); `hooks/useNightMode.ts` toggles `petal-night` on `<html>` (60s re-check); `index.css` `html.petal-night` re-points only the palette tokens → whole UI flips via `var()` (no component edits), 600ms dusk fade, print stays white; `PetalFall` gains a `night` prop → chunky cartoon power stars (`makeCartoonStar`, Mario/Kirby-style, 5 candy colors) mixed ~70/30 with small twinkle sparkles, gentle spin + shallow shimmer, effect re-inits on flip; App: `useNightMode()` → `<PetalFall night={night}/>`. tsc + vite clean, companion vitest 45/45; verified with real-browser Playwright screenshots (clock mocked to 23:30) — day petals/cream vs night stars/dark-plum, both pretty. Bedtime window is `BEDTIME_FROM`/`BEDTIME_TO` (local clock) for easy retune.
MULTIUSER_PLAN.md
+68 -49
View File
@@ -1,11 +1,16 @@
# Petal multi-user plan
**Status:** Phase 0 (identity plumbing) landed 2026-07-26 in `6901cdb`. Everything
below it is proposed, not agreed.
**Status:** Phase 0 (identity plumbing) landed 2026-07-26 in `6901cdb`.
**All OPEN decisions ratified by the user 2026-07-26** (recommendations
accepted as written) — see each OPEN for its settled answer. Execution phases
live in `BUILD_PLAN.md` (Phase 15 onward); product rationale for the language
work is in `SUGGESTIONS.md`.
This document exists to be argued with. Sections marked **OPEN** are real
decisions with real trade-offs, not rhetorical questions — a reviewer should
push back on them. Where I have a recommendation I say so and say why.
Settled context that postdates the original draft: Petal will be hosted on the
**parodia.dev VPS**, reaching vLLM on millenia over **headscale VPN**; Piper
TTS is already installed on parodia (VPS-local, no VPN hop). The LLM is the
only cross-VPN dependency, and per the LLM-minimalism principle
(SUGGESTIONS.md §6) it must never gate essential functionality.
---
@@ -61,10 +66,12 @@ global."
## 3. Phase A — authentication
### OPEN #1: forward-auth vs. in-app OIDC
### OPEN #1: forward-auth vs. in-app OIDC — **SETTLED: Option B (in-app OIDC)**
This is the biggest fork and it should be settled first, because everything in
Phase B depends on it.
Ratified 2026-07-26. The deciding fact arrived with the deployment plan: Petal
will live on the public parodia.dev VPS, which is exactly the environment where
Option A's "must never be reachable except through Traefik" invariant is a
footgun. Original analysis kept below for the record.
**Option A — Traefik forward-auth to an Authentik outpost.**
Traefik is already in the deferred deploy bucket. Authentik's outpost terminates
@@ -109,10 +116,10 @@ Does the operator want Petal to be independently deployable?
Preferred over signed stateless cookies because it makes logout and
revocation actually work — worth the one table.
- Cookie: `HttpOnly`, `SameSite=Lax`, `Secure` when `BASE_URL` is https.
- **OPEN #2:** session lifetime. This is a personal writing tool used daily on
trusted devices; a 30-day sliding expiry is kind, an 8-hour one is
conventional. I lean long — an editor that logs you out mid-draft is hostile,
and the auto-save makes a surprise 401 genuinely costly. Needs a decision.
- **OPEN #2 — SETTLED: 30-day sliding expiry** (ratified 2026-07-26). An
editor that logs you out mid-draft is hostile, and auto-save makes a
surprise 401 genuinely costly. Sliding: each authenticated request extends
the session.
### The 401 problem
@@ -136,9 +143,11 @@ On first successful login, upsert a `users` row from the OIDC claims
(`sub` → `users.id`, plus email and name). No signup flow; whoever Authentik lets
in gets an account.
**OPEN #3:** should there be an allowlist? Authentik may host other applications
with a broader user set than Petal should have. A `PETAL_ALLOWED_SUBS` env var,
or an Authentik group check on a claim, would gate it. Probably yes, cheaply.
**OPEN #3 — SETTLED: yes, allowlist** (ratified 2026-07-26). Authentik may
host other applications with a broader user set than Petal should have. Gate
via a `PETAL_ALLOWED_SUBS` env var (or an Authentik group claim check —
implementer's choice, env var is simpler); a valid login not on the list gets
a warm bilingual "this Petal isn't yours to write in" page, not a 500.
### Migrating the existing `local` user
@@ -159,9 +168,9 @@ person logs in once. Sequence: deploy auth → she logs in → new empty account
created → stop app, back up, run script to move `local`'s rows onto her real id,
delete the empty row → restart.
**OPEN #4:** is that acceptable, or is a small admin endpoint preferable to a
script? The existing project convention (`scripts/`, hand-run Python) suggests a
script is in keeping.
**OPEN #4 — SETTLED: documented one-off script** (ratified 2026-07-26), run
with the app stopped and a DB backup taken first, per the existing `scripts/`
convention. No admin endpoint.
---
@@ -186,10 +195,9 @@ image uploaded by two users is stored once on disk and simply has two rows, so
deduplication survives. Deleting the last row referencing a hash removes the
file.
**OPEN #5:** is this worth doing before real multi-user, or is it acceptable to
ship auth first and treat this as a known limitation? I lean toward doing it *in
the same phase as auth*, since the moment a second account exists the exposure is
real and the fix requires a migration either way.
**OPEN #5 — SETTLED: fix it in the same phase as auth** (ratified
2026-07-26). The moment a second account exists the exposure is real, and the
fix requires a migration either way.
### Frontend `localStorage`
@@ -241,7 +249,13 @@ It also carries data Petal has no equivalent for and could use: `Antonyms`,
So Phase D stops being gated on data and becomes an integration decision.
### OPEN #6a (new): how to integrate
### OPEN #6a (new): how to integrate — **SETTLED: Option 3** (ratified 2026-07-26)
Import the package, open `dict.db` read-only. Prerequisite stands: DreamDict's
module path must be renamed (or `replace`-directed) first — **that change
lives in the dreamdict repo, not this one.** The migration caution below also
stands: pt-PT/fr wire to DreamDict first; zh stays on ECDICT until compared on
real lookups. Options kept below for the record.
**Option 1 — HTTP client.** Petal calls DreamDict on localhost:7777, exactly the
pattern already used for Piper TTS (including graceful degradation when it's
@@ -305,18 +319,26 @@ the phonetic dataset, the EN voice — is unaffected and stays shared.
## 7. Suggested sequence
1. **Settle OPEN #1** (forward-auth vs. in-app OIDC). Everything else follows.
2. Deploy plumbing: Dockerfile, Traefik, real hostname, HTTPS. Auth needs a
stable `BASE_URL` and a redirect URI regardless of which option wins.
3. Auth itself: `Resolver` implementation, sessions if applicable, frontend 401
handling.
1. ~~Settle OPEN #1~~ **Settled: in-app OIDC.**
2. Deploy plumbing: Dockerfile, Traefik, real hostname on parodia.dev, HTTPS,
headscale route to vLLM on millenia (bound to the headscale interface
only), VPS-local Piper, off-VPS DB backup. Auth needs a stable `BASE_URL`
and a redirect URI.
3. Auth itself: OIDC `Resolver`, sessions table (30-day sliding), allowlist,
frontend 401 handling with draft preservation.
4. Image store table + migration (same phase, per OPEN #5).
5. Provision the second real account; migrate `local`'s data.
6. `localStorage` namespacing.
7. Per-user language. **No longer gated on data** — DreamDict covers all four
languages. Sequence within it: rename DreamDict's module path → wire it in as
a lexicon provider → pt-PT/fr first → compare zh quality → converge if it
holds.
5. Provision the second real account; migrate `local`'s data (script, app
stopped, backup first).
6. `localStorage` namespacing (key by user **and** language — see
SUGGESTIONS.md §8).
7. Per-user language pair. **No longer gated on data** — DreamDict covers all
four languages. Sequence within it: rename DreamDict's module path → wire
it in as a lexicon provider → pt-PT/fr first → compare zh quality →
converge if it holds. The pair model and langpack shape are specified in
SUGGESTIONS.md §1–§3.
These are expanded into checkboxed execution phases in `BUILD_PLAN.md`
(Phase 15 onward) — that file remains the source of truth for progress.
---
@@ -337,19 +359,16 @@ the phonetic dataset, the EN voice — is unaffected and stays shared.
---
## 9. Questions for the reviewer
## 9. Questions for the reviewer — all answered 2026-07-26
1. Forward-auth or in-app OIDC? (OPEN #1 — the one that matters most)
2. Session lifetime, given a daily-use editor with auto-save? (OPEN #2)
3. Allowlist Petal accounts separately from Authentik's user set? (OPEN #3)
4. Migration script vs. admin endpoint for moving `local`'s data? (OPEN #4)
5. Fix the image store alongside auth, or ship auth with it as a known
limitation? (OPEN #5)
6. ~~Is there a usable open English↔European-Portuguese dictionary dataset?~~
**Answered: DreamDict**, which covers en/fr/pt-PT/zh. The live question is now
*how* to integrate it — HTTP service, build-time extraction, or importing the
package and opening `dict.db` read-only. (OPEN #6a; I recommend the third)
7. Does replacing the zh gloss (ECDICT → CC-CEDICT) risk regressing a feature in
daily use, and should zh stay on ECDICT until the two are compared?
Anything above that reads as settled but shouldn't be is also fair game.
1. ~~Forward-auth or in-app OIDC?~~ **In-app OIDC** (OPEN #1).
2. ~~Session lifetime?~~ **30-day sliding** (OPEN #2).
3. ~~Allowlist?~~ **Yes**, `PETAL_ALLOWED_SUBS` or group claim (OPEN #3).
4. ~~Script vs. admin endpoint?~~ **Script**, app stopped, backup first (OPEN #4).
5. ~~Image store timing?~~ **Same phase as auth** (OPEN #5).
6. ~~pt-PT dictionary data?~~ **DreamDict**, integrated per **Option 3**
(import package, read-only `dict.db`; module rename is the prerequisite)
(OPEN #6a).
7. ~~zh gloss regression risk?~~ **zh stays on ECDICT** until compared against
DreamDict on real lookups from her actual documents; converge only if
quality holds.
SUGGESTIONS.md
+364
View File
@@ -0,0 +1,364 @@
# Petal — product suggestions: becoming essential for language learners
**Status:** written 2026-07-26 against `feat/writing-passport`; **ratified by
the user 2026-07-26** (recommendations accepted — reviewer Q1–Q3 settled
below; Q4–Q6 remain genuinely open and don't block execution). This document
is the *why*; the checkboxed execution phases live in `BUILD_PLAN.md`
(Phase 15 onward).
**The brief:** make Petal essential for two audiences — ESL writers (native
Mandarin / pt-PT / French → English), and English natives learning Mandarin,
European Portuguese, French, maybe Spanish. Preserve privacy and warmth.
**The language model (settled by the user, 2026-07-26):** every user has
exactly one language **pair, with English always one half** — (en + X),
X ∈ {zh, pt-PT, fr, maybe es}. This is a deliberate scope decision: never an
X↔Y pair without English, never more than one pair per user. The UI is
bilingual in the pair everywhere (tips, pet responses, cards), and the user
may **type in either language of the pair**; Petal infers direction from the
text rather than asking.
---
## 1. What the pair model implies
The wife's zh setup is already exactly this — she writes Mandarin and English
mixed, the UI is zh+en bilingual, and Petal adapts per span (CJK is never
spellchecked, English words gloss to Chinese). So the pair model isn't a new
design; it's a *promotion of today's behavior to the spec*. Three consequences:
- **Schema:** one column, `users.pair_lang` (the X half; default `'zh'`).
No per-document language, no target/native split. Add it in whatever
migration Phase B's provisioning touches — one column now vs. a real
migration later, the same logic that put `user_id` in the schema on day one.
- **Direction is inferred, not declared.** The zh pair gets inference for free
(script boundaries separate the languages). Latin pairs don't — see §3a,
which is the one genuinely new problem the pair model creates.
- **Every bilingual surface stays two-language**, just parameterized: the
`中文 · English` pattern becomes `X · English`. Nothing about the UI's shape
changes, which is why the copy extraction in §2 is safe to do early.
## 2. Languages as data, not code ("langpacks")
Adding pt-PT today means editing code in many places. A quick census: **29
frontend files** carry hardcoded zh-first bilingual strings (`tips.ts`,
`GardenPanel`, `StatusBar`, `WordCard`, every popover…), plus the
Mandarin-first prompt copy in `internal/llm/prompts.go`. Adding each new
language by hunting through those files doesn't scale to four pairs and would
slowly erode the bilingual-copy quality that makes Petal feel cared-for.
Because English is always one half, a **langpack is keyed by X alone** — one
pack per pair, holding everything that varies:
- UI copy pairs — extract the existing `中文 · English` strings into a copy
module; the current strings become the `zh` pack verbatim, so nothing
visible changes. This is the biggest single chore in the whole effort;
better done once than per-language.
- Companion tip/cheer/bedtime lines (`tips.ts` is already data-shaped —
closest to done).
- LLM prompt copy: bilingual explanation phrasing, "natives usually say…"
example pairs, and the both-directions framing (the text may be English, X,
or mixed — respond appropriately).
- Hunspell dictionary for X where one exists (`pt-PT`, `fr`, `es` upstream;
zh has none — see §4). The en-US dictionary is shared by every pair.
- DreamDict wiring: gloss both directions (`en→X` and `X→en`), phonetics.
- Piper voices for X (both for reading X text aloud and as the L1 voice);
font stack (CJK stacks only for zh).
Shared across all pairs, untouched: nspell en-US, the English IPA dataset, the
EN Piper voice, and all of the editor machinery.
This is refactoring, not product, so it's tempting to skip. Don't: it's the
difference between "Spanish is a data drop" and "Spanish is a month."
## 3. Sequencing: Latin-script targets first, and in this order
**pt-PT → fr → es.** Everything needed for these exists already: Hunspell
dictionaries, Piper voices, DreamDict data (en/fr/pt-PT/zh), and — critically —
the entire decoration/anchoring machinery (`wordAt`, spell tokenizer, suggestion
re-anchoring) already works, because these languages are space-delimited and
Latin-script like English.
Caveats worth writing down now:
- **DreamDict has no Spanish.** "Maybe Spanish" is gated on adding es to
DreamDict first, or a separate dataset. Cheap to note, expensive to discover
later.
- **pt-BR drift is the main quality risk.** Qwen will default to Brazilian
Portuguese in both explanations and "natives say…" examples. Prompts must pin
European Portuguese explicitly, and the pt-PT pack should be reviewed by a
pt-PT speaker before it's trusted — same standard the zh copy got by being
written for a real reader. The multi-user plan's ECDICT-vs-DreamDict
compare-on-real-lookups discipline applies here too.
### 3a. The Latin+Latin wrinkle: inferring direction without a script boundary
The zh pair gets "which language is this word?" for free — the script answers
it, and all of today's behavior (CJK never spellchecked, English words gloss
to Chinese) hangs off that. In an en+fr or en+pt pair, both halves are Latin
script, so the two per-word decisions need a real answer:
- **Spellcheck:** load both Hunspell dictionaries and pass a token if *either*
accepts it; flag only words wrong in both. This never falsely squiggles
correct writing in either language — the failure mode is missing a French
word that happens to be a valid English word, which is the gentle direction
to fail in. Correction pills can offer both dictionaries' suggestions.
- **Gloss/WordCard:** look the word up in both directions via DreamDict; if it
exists in only one language, done. For collisions (*chat*, *pain*, *sale*
are all real words in both English and French), show both compactly — a
two-line card ("🇫🇷 chat → cat · 🇬🇧 chat → bavarder") is honest, needs no
detector, and is arguably *delightful* for a learner. A sentence-level
language guess can order the lines, but shouldn't hide either.
No trained language detector, no heuristics that can be wrong about someone's
writing — both-dictionaries membership plus show-both-on-collision covers it.
The LLM passes need nothing: the prompt already sees the mixed text whole.
- The Hunspell tokenizer's current rule "CJK is never tokenized" stays correct
for the zh pair unchanged.
## 4. The zh pair's *other* direction is a separate epic — say so explicitly
The en+zh pair already exists, but only one direction of it is built: today
Petal deliberately ignores typed hanzi (never tokenized, never flagged, never
glossed) — exactly right for a zh-native writer practicing English, and
exactly insufficient for an English native *learning* Chinese, for whom the
hanzi side is the whole point. Supporting that direction breaks assumptions
that are load-bearing everywhere:
- No spaces → `wordAt`, the spell tokenizer, and word-boundary lookups need
real word segmentation (a jieba-style segmenter, client- or server-side).
- Hunspell has no concept of Chinese; "spellcheck" becomes wrong-character
(错别字) detection — a different problem, probably LLM-assisted.
- Smart-typography input rules and the IME interact; input rules are currently
ASCII-gated, which is correct, but selection/caret behavior mid-IME
composition needs testing.
- The learning aids that matter are different: pinyin annotation (useful here,
unlike for the current user who reads hanzi), tone-mark help, HSK-level word
difficulty, hanzi stroke/handwriting practice.
None of this is unbuildable, but it is its **own phase with its own spec**, not
part of the langpack drop. Recommendation: ship the pt-PT/fr pairs first to
prove the pair model, and treat learner-facing Chinese writing as Petal's next
big product bet after that — it's also the most differentiated one (very few
warm, private tools exist for writing practice in Chinese).
## 5. Deepening the learning loop (all local, all gentle)
Petal's suggestion pipeline currently *corrects and forgets*. The vocabulary
garden proved that capturing what the user already does (lookups) creates a
learning surface for free. The same move is available twice more:
### 5a. Growth journal (patterns from accepted suggestions)
Accepted grammar/collocation suggestions are a record of what the writer is
learning. Aggregate them **locally** into gentle patterns: "this month you've
mostly stopped mixing 在/at" / "make a decision has stuck — you've used it
right 4 times since." Two framing rules that keep it warm: it reports *growth*,
never an error tally, and it only ever compares the writer to her own past
self. Feeds the companion's cheer pool with genuinely personal material
("上次你还问过这个词，这次自己用对了! 🌱"). Data is already in the
`suggestions` table (status + type + original/replacement); this is a read-side
feature, no new capture needed.
### 5b. Plant accepted collocations in the garden
An accepted collocation ("do a decision" → "make a decision") is a learnable
chunk, exactly like a looked-up word. Auto-capture it into the vocabulary
garden as a phrase card (the SM-2-lite scheduler doesn't care that it's two
words). The garden then reflects *both* halves of learning: words she sought
out, and phrasing she was gently given.
### 5c. Companion as tutor-lite: a daily invitation to write
The companion nudges about breaks and bedtime but never *invites writing*. A
once-a-day bilingual prompt ("写 50 个字：今天让你微笑的一件小事 · Write 50
words: one small thing that made you smile today"), offered when a session
starts with no doc open. Explicitly **no streaks, no guilt** — the existing
no-streak-shaming ethos in the SR scheduler is the right precedent; a declined
prompt just gets a sleepy "好吧，我继续睡 😴". Prompt lists live in the
native-language pack.
### 5d. Use DreamDict's richer fields
The multi-user plan notes DreamDict carries `Frequency`, `Difficulty`,
`Antonyms`, `Etymology` with "no equivalent" in Petal. Three cheap, high-value
surfaces:
- A **frequency/difficulty chip** in the WordCard ("common word" / "advanced")
— helps a learner decide whether a word is worth gardening.
- **Etymology for the en-native audience**: Romance-language learners live on
cognates; a one-line "from Latin *decidere*, like English *decide*" is the
single best memory hook for pt/fr/es vocabulary.
- **False friends**: a small curated list per pair (en↔pt: *embarrassed* ≠
*embaraçada*-adjacent traps, *actually*/*atualmente*; en↔fr likewise),
surfaced as a warm heads-up in the WordCard and as a collocation-style
gentle flag when one is used suspiciously. Tiny data, disproportionate
trust-building — this is the mistake that makes learners feel foolish, and
catching it kindly is very Petal.
### 5e. Read-aloud, slower
Piper voices exist per target language; wire the target-language voice into the
existing 🔊 surfaces, and add a **slow toggle** (Piper's `length_scale`) —
learners replaying a sentence at 0.75× is one of the oldest, most-loved
listening aids, and it's a query parameter away.
## 6. LLM-minimalism: essential help in plain code, the model as garnish
**Stated by the user (2026-07-26):** with the LLM on the far side of a VPN,
preserve as much essential functionality as possible in ordinary code inside
Petal, and rely on the LLM as little as possible. This deserves to be a
standing design principle, not just a deployment reaction — it's also what
keeps Petal instant (no 38-second checkpoints for things a lookup can answer)
and private by construction.
Where Petal stands today, by dependency:
| Already pure code (survives VPN-down) | LLM-only today |
|---|---|
| Spellcheck (Hunspell), gloss/definitions/synonyms/phonetics (embedded lexicon → DreamDict), thesaurus, vocabulary garden + SR review, search, tags, versions + writing passport, export, find/replace, typography, TTS (Piper, VPS-local) | Grammar checkpoint, collocation coach, voice pass, Ask Petal, tone rewrite |
Everything in §5 lands in the left column by design (growth journal, garden
planting, daily prompts, DreamDict fields, false friends — all lookups and
local aggregation). The right column splits into two groups:
**Worth a code-first layer (the essential two):**
- **Grammar lite** — a rule-pack of high-precision, data-driven checks for
the classic ESL patterns: a/an before vowel sounds, uncountables
("informations", "advices", "furnitures"), subject–verb agreement in simple
clauses, doubled comparatives, common preposition pairs ("depend of" →
"depend on"), per-pair L1-interference rules (zh: dropped articles, he/she
slips; pt/fr: "have X years" for age). These run instantly on every edit —
no debounce, no 30s rate limit — as a fourth suggestion family through the
existing rail. The bar is **precision over recall**: an offline rule must be
near-certain before it flags, because a wrong correction is colder than a
missed one. LanguageTool's open rule corpus is a mineable source for
vetted patterns (extract data, not the Java).
- **Collocation data** — the same curated-list move as false friends: the
do/make, say/tell, strong-tea/heavy-rain families that fill every ESL
collocation workbook are a few hundred entries of data, not a model. A
small embedded miscollocation list catches the top offenders offline; the
LLM pass, when reachable, adds the long tail. Same family, same rail, same
warm phrasing — the writer never needs to know which engine spoke.
**Inherently LLM (degrade warmly, don't imitate):** Ask Petal, tone rewrite,
and the voice pass are open-ended language generation — a code fake would be
worse than the existing honest "小助手在休息" state. Leave them as the
garnish they are.
The framing that falls out: **the LLM never holds essential functionality
hostage.** Every daily-writing need — spell, define, gloss, pronounce, catch
the common mistakes, review vocabulary, prove authorship — works on a
disconnected VPS. The model adds depth and conversation when the tunnel is up.
## 7. The writing passport is an ESL flagship — treat it as one
The passport exists because AI detectors misfire on non-native English (the
commit message cites the Stanford TOEFL finding). That's not a side feature —
for the ESL audience it may be *the* reason to adopt Petal over any other
editor: **the tool that protects you from being wrongly accused, instead of
scoring you.** No product change needed beyond making sure it works identically
for any target language (it should — it's language-agnostic snapshot history).
Worth a prominent place in the README/landing copy when Petal gets one.
## 8. Ties into MULTIUSER_PLAN.md
For the open questions there, this document's brief implies:
- **OPEN #1 (auth):** Option B (in-app OIDC), and the planned deployment
settles it. The user's stated topology (2026-07-26) is: **Petal hosted on
the parodia.dev VPS, reaching vLLM on millenia over headscale VPN.** A
public-internet app is exactly the case where "must never be reachable
except through Traefik" is a footgun — one proxy misconfiguration on a VPS
and forged identity headers reach the app. In-app OIDC is safe to expose
directly.
- **Deployment topology consequences** worth writing into the plan's Phase 2
(deploy plumbing):
- The LLM becomes the only cross-VPN runtime dependency (Piper is already
installed on parodia.dev, so TTS stays VPS-local). The warm
"小助手在休息" degradation path was built for a flaky co-tenant Ollama; a
VPN link-down hits the same path, so the architecture already fails
gently — but checkpoint latency now includes a WAN+VPN round trip, worth
a look at the 60s LLM timeout.
- Everything offline-by-design (DreamDict lookups, spellcheck, gloss,
garden, search, the whole editor) keeps working when the VPN is down —
another argument for MULTIUSER_PLAN Option 3 over an HTTP dictionary
service, which would otherwise add a second cross-machine dependency.
- vLLM and Piper on millenia should bind to the headscale interface only,
never 0.0.0.0 on the LAN-facing side.
- The writing moves onto rented VPS disk. "The writing never leaves the
box" (§8) becomes "the box is a VPS" — at-rest encryption and an
off-VPS backup of `petal.db` (e.g. nightly to millenia over the same
VPN) deserve a line in the deploy phase.
- **OPEN #6a (DreamDict):** Option 3 (import, read-only `dict.db`), agreed —
it's the only option where four languages stay offline and instant, which
§6 and §9 treat as non-negotiable.
- **Phase B provisioning** should set `users.pair_lang` from the operator's
provisioning step or a first-run picker — add the column in the same
migration. (The plan's Phase D "native language becomes a `users` column"
becomes this: one column, the X half of the pair.)
- **localStorage namespacing** matters slightly more than the plan says once a
household mixes pairs: the personal spell dictionary is per-*language* as
well as per-user (a user's en words and pt-PT words must not merge into one
Hunspell overlay). Key by `user + lang`.
## 9. Privacy & warmth guardrails (the checklist for every item above)
Everything suggested here passes these; future ideas should too.
1. **Offline-first, always — and code-first (§6).** Every essential surface
must work with the LLM unreachable and the network unplugged
(DreamDict-as-local-file preserves this; an HTTP dictionary service would
not). The LLM only ever adds depth to something that already works.
Cloud APIs are off the table even when they'd be easier.
2. **The writing never leaves the box.** No telemetry, no "anonymous usage
stats," ever. The growth journal (§5a) is computed locally from local rows.
3. **No scores, no percentages, no red.** Petal already refuses AI-detection
scores and classic red squiggles; the growth journal and false-friend flags
must hold the same line — evidence and gentle phrasing, never grades.
4. **No streaks, no guilt.** The SR scheduler set the precedent (gentle
"again", no wipe). Daily prompts (§5c) are invitations, not obligations.
5. **Both languages of the pair, always visible** — every explanation, tip,
and pet response renders bilingual in (en + X). That's the warmth: being
helped in the language you think in, next to the one you're learning.
6. **The kitten stays asleep.** Every new companion behavior routes through
the existing mood/cooldown engine; 瞌睡猫 keeps mumbling helpful things
without waking up. (A reactive-animation puppy is on the wishlist — low
priority per the user; the `companions.ts` roster + mood engine is already
the drop-in point, richer per-mood Lottie segments are the only new work.)
## 10. Suggested sequence (interleaved with the multi-user plan's)
1. Add the `users.pair_lang` column (with the Phase B migration or sooner).
2. Extract the bilingual UI copy into the langpack (zh pack = today's strings
verbatim; pure refactor, no visible change).
3. DreamDict integration per MULTIUSER_PLAN Option 3 (module rename → lexicon
provider → pt-PT/fr wired first, zh compared before converging).
4. pt-PT as the first full second pair: Hunspell pt-PT, Piper pt-PT voices,
pinned-pt-PT prompts, native-speaker copy review, and the both-dictionaries
spellcheck + show-both-gloss behavior from §3a. French follows the same
groove; Spanish gated on DreamDict es data.
5. Learning-loop features (§5) — each is small and independent; growth journal
and garden-planting of collocations first, since they're read-side over
existing data.
6. Code-first layers (§6): the embedded miscollocation list first (same shape
as false friends, drops into the existing collocation family), then
grammar lite as its own suggestion family. Both are per-pair data, so
they slot naturally into the langpacks from step 2.
7. Learner-facing Chinese writing (the zh pair's second direction): spec it as
its own phase (§4) only after the pair model is proven on pt-PT/fr.
## 11. Questions for the reviewer (1–3 settled 2026-07-26)
1. ~~§3a's no-detector stance~~ **Settled: yes** — pass if either dictionary
accepts it, show both glosses on collision, no language detector.
2. ~~UI-copy extraction first?~~ **Settled: yes** — the extraction (§2) is a
prerequisite chore, done before pt-PT is wired.
3. ~~Growth journal framing~~ **Settled: build it** with the two framing rules
as hard constraints (growth only, self-comparison only).
4. When (not whether) to build the learner-facing hanzi direction of the zh
pair — after pt-PT/fr, or is it wanted sooner?
5. Spanish: worth asking DreamDict to grow an es dataset now, or park it?
6. Grammar lite (§6): hand-curate the rule pack from ESL teaching materials
(small, fully understood), or mine LanguageTool's open rule corpus for
vetted patterns (bigger head start, needs licensing + quality triage)?