Ratify the language-learning direction and expand it into Phases 15-22
SUGGESTIONS.md (new): the product rationale. Every user gets one
(English + X) pair, X in {zh, pt-PT, fr, maybe es} - bilingual UI in the
pair, type in either language, direction inferred without a detector
(both-dictionaries spellcheck, show-both gloss on collision). Langpacks
keyed by X. LLM-minimalism as a standing principle: the LLM never gates
essential functionality; grammar-lite rules and an embedded
miscollocation list are planned as code-first layers beneath the LLM
families.
MULTIUSER_PLAN.md: all OPEN decisions settled - in-app OIDC (the
parodia.dev VPS hosting plan decides it), 30-day sliding sessions,
allowlist, migration-by-script, image-store ownership fixed alongside
auth, DreamDict imported as a package reading dict.db (module rename
prereq lives in the dreamdict repo), zh stays on ECDICT until compared.
BUILD_PLAN.md: the deferred bucket becomes checkboxed Phases 15-22
(deploy plumbing / OIDC / local-user migration / client-state
namespacing / langpack extraction / DreamDict provider / pt-PT pair /
learning loop + code-first layers) with standing rules: isolation tests
in the same commit as any user-scoped endpoint, LLM-minimalism, and
bilingual aesthetic as acceptance criteria.
This commit is contained in:
+78
-3
@@ -136,9 +136,83 @@ Multi-session build. **Source of truth for what's done and what's next.** Update
|
||||
|
||||
### Deferred (post-v1-local)
|
||||
- [x] **Multi-user groundwork** (2026-07-26) — request-scoped identity. New `internal/auth`: `Middleware(Resolver)` resolves the caller once per API request and stores the id in the context; handlers read it via `auth.UserID(r.Context())` instead of naming `db.LocalUserID`. `StaticResolver(db.LocalUserID)` keeps Petal single-user today. **Auth itself is still deferred** — but every query is now scoped to whoever the resolver says is calling, so landing Authentik is a one-line change in `main.go` plus a `Resolver` implementation.
|
||||
- [ ] Authentik OIDC auth + session middleware ← **next step: write a `Resolver` that validates a session cookie; the plumbing is in place**
|
||||
- [ ] Copyleaks Tier-2 + webhook HMAC
|
||||
- [ ] Dockerfile, docker-compose, Traefik, deploy to write.parodia.dev
|
||||
- [ ] Copyleaks Tier-2 + webhook HMAC (still parked — needs a public webhook; revisit after Phase 15)
|
||||
- Authentik OIDC + deploy: **no longer deferred — expanded into Phases 15–17 below** (decisions ratified 2026-07-26; see `MULTIUSER_PLAN.md`)
|
||||
|
||||
## Execution phases 15–22 (added 2026-07-26)
|
||||
|
||||
Decisions behind these are ratified in `MULTIUSER_PLAN.md` (all OPENs settled) and `SUGGESTIONS.md` (the *why*; Q1–Q3 settled). **Standing rules for every phase below:**
|
||||
- **Isolation tests in the same commit** as any new user-scoped endpoint (the `docs/isolation_test.go` suites are the template — this discipline caught a real unscoped-query bug once already).
|
||||
- **LLM-minimalism** (SUGGESTIONS §6): the LLM never gates essential functionality; new essential features are code+data first.
|
||||
- **Aesthetic + bilingual-in-the-pair copy remain acceptance criteria** on every user-visible change.
|
||||
- Verify per project convention: go build/vet/test, tsc, vite build, vitest, live smoke on a throwaway DB/port.
|
||||
|
||||
### Phase 15 — Deploy plumbing (parodia.dev + headscale)
|
||||
Petal hosted on the parodia.dev VPS; vLLM stays on millenia over headscale. Auth (Phase 16) needs the stable `BASE_URL`/redirect URI this phase creates.
|
||||
- [ ] Dockerfile (multi-stage: `npm run build` → `go build`, single binary + data dir volume) + docker-compose
|
||||
- [ ] Traefik route + HTTPS on the chosen hostname (e.g. `write.parodia.dev`); stable `BASE_URL` env
|
||||
- [ ] `LLM_ENDPOINT` → millenia's **headscale** address; verify vLLM binds only to the headscale interface (not LAN/0.0.0.0); revisit `LLM_TIMEOUT` for the WAN+VPN round trip
|
||||
- [ ] TTS → the Piper already installed on parodia (VPS-local; per-language instances as on millenia — EN now, zh; see `deploy/` artifacts for the service pattern)
|
||||
- [ ] Off-VPS nightly backup of `petal.db` (+`-wal`/`-shm` coherently, e.g. `sqlite3 .backup`) to millenia over the VPN; document restore
|
||||
- [ ] Migrate the live millenia data to the VPS (app stopped, backup first, counts verified) — or decide the millenia instance stays canonical until Phase 17 completes
|
||||
- [ ] Acceptance: full editor works with the VPN link **down** (spell/gloss/garden/search/export/TTS all fine; checkpoint shows the warm 小助手在休息 state); `/api/health` public; HTTPS end-to-end
|
||||
|
||||
### Phase 16 — Auth (in-app OIDC) + image-store ownership
|
||||
Option B ratified. `go-oidc` + `x/oauth2`; config fields already exist. The `Resolver` seam from Phase 0 is the only integration point.
|
||||
- [ ] OIDC login flow: `/auth/login` → Authentik → `/auth/callback` (state/CSRF checked) → provision user (upsert from `sub`/email/name) → session
|
||||
- [ ] `sessions` table migration (id, user_id, expires_at, created_at, user_agent); opaque cookie (`HttpOnly`, `SameSite=Lax`, `Secure` when https); **30-day sliding expiry**; `/auth/logout` revokes server-side
|
||||
- [ ] Allowlist: `PETAL_ALLOWED_SUBS` env (comma-separated); rejected valid logins get a warm bilingual page, not an error dump
|
||||
- [ ] `SessionResolver` replaces `StaticResolver` in `main.go` (keep `StaticResolver` for dev via env flag)
|
||||
- [ ] Frontend 401 interceptor in `api/client.ts`: halt auto-save, preserve the draft (localStorage keyed by doc id), warm bilingual "请重新登录 · Please sign in again" overlay, resume cleanly after re-login — **a 401 mid-draft must never lose writing**
|
||||
- [ ] **Image store ownership** (same phase, per OPEN #5): `images` table migration (hash, user_id, content_type, size, created_at); fetch joins on caller; dedup preserved (one file, N rows); last-row delete removes the file; backfill existing images to the current user
|
||||
- [ ] `users.pair_lang` column (default `'zh'`) in this phase's provisioning migration — read by Phase 19+, cheap to add now
|
||||
- [ ] Isolation tests: sessions (expiry, revocation, cross-user), images, allowlist paths
|
||||
|
||||
### Phase 17 — Migrate the `local` user
|
||||
Script, app stopped, backup first (OPEN #4). Depends on: she logs in once so her OIDC `sub` exists.
|
||||
- [ ] `scripts/migrate_local_user.*`: single transaction, `PRAGMA foreign_keys=OFF`, re-point `documents`/`tags`/`vocab_words` (versions/suggestions follow parents), delete the empty provisioned row, verify row counts before commit; refuses to run if the app is up or the target has data
|
||||
- [ ] Runbook documented in the script header; dry-run mode
|
||||
|
||||
### Phase 18 — Per-user, per-language client state
|
||||
- [ ] Namespace `localStorage` keys by user id once known: `petal.spell.personal`, `petal.companion`, sound/petals prefs
|
||||
- [ ] Personal spell dictionary keyed by **user + language** (en and pt-PT word lists must not merge); consider promoting it to a server table so it follows her across devices (nice-to-have, decide during)
|
||||
|
||||
### Phase 19 — Langpack extraction (the copy chore)
|
||||
Pure refactor, zero visible change; prerequisite for every new pair (SUGGESTIONS §2, Q2 settled).
|
||||
- [ ] Extract the `中文 · English` strings from the ~29 frontend files + companion `tips.ts` into a langpack copy module keyed by the pair's X; today's strings become the `zh` pack **verbatim**
|
||||
- [ ] Parameterize `internal/llm/prompts.go` bilingual copy the same way (explanation language, "natives usually say…" framing, both-directions preamble)
|
||||
- [ ] Wire pack selection to `users.pair_lang`; acceptance: pixel-identical UI for the zh pair, vitest snapshots unchanged
|
||||
|
||||
### Phase 20 — DreamDict as a lexicon provider
|
||||
Option 3 ratified (import package, read-only `dict.db`). **Prerequisite in the dreamdict repo:** rename its module path (or add a `replace` for dev).
|
||||
- [ ] Provider seam behind the existing `lexicon` interface; DreamDict provider opens `dict.db` read-only (modernc driver, second handle beside `petal.db`); graceful "no data" when the file is absent
|
||||
- [ ] pt-PT + fr wired to DreamDict (nothing to regress); **zh stays on ECDICT** until compared on real lookups from her documents — converge only if quality holds
|
||||
- [ ] Surface the new fields where cheap: frequency/difficulty chip in WordCard; etymology line (cognate hook for en-natives)
|
||||
- [ ] Deploy: `dict.db` ships in the data dir alongside `petal.db`
|
||||
|
||||
### Phase 21 — The pt-PT pair (first Latin pair, proves the model)
|
||||
SUGGESTIONS §1/§3/§3a. French follows the same groove afterwards; Spanish stays gated on DreamDict es data.
|
||||
- [ ] Hunspell pt-PT vendored like en-US; **both-dictionaries spellcheck** (flag only if wrong in both; pills from both) — the no-detector stance, Q1 settled
|
||||
- [ ] Gloss/WordCard both directions; on en/pt collisions show both compactly, never hide either
|
||||
- [ ] Prompts pinned to **European Portuguese, never pt-BR** (explicit in every prompt); pt-PT langpack copy written and **reviewed by a pt-PT speaker before trusted**
|
||||
- [ ] Piper pt-PT voice instance on parodia; read-aloud + L1 voice wired; slow toggle (`length_scale`) while in there (SUGGESTIONS §5e)
|
||||
- [ ] Companion tips/cheers/bedtime lines in the pt-PT pack (the kitten speaks pt+en to this user)
|
||||
- [ ] Acceptance: a pt-PT-pair user gets the full experience end-to-end with the VPN down except LLM passes; zh-pair user sees zero change
|
||||
|
||||
### Phase 22 — Learning loop + code-first layers
|
||||
Each item independent and small; order within is free (SUGGESTIONS §5–§6).
|
||||
- [ ] **Growth journal** (Q3 settled): local aggregation over accepted suggestions; growth-only, self-comparison-only framing; feeds companion cheers
|
||||
- [ ] **Plant accepted collocations** in the vocabulary garden as phrase cards (scheduler unchanged)
|
||||
- [ ] **Daily writing invitation** from the companion (no streaks, declining is fine)
|
||||
- [ ] **False-friend list** per pair (curated data, WordCard heads-up + gentle flag)
|
||||
- [ ] **Embedded miscollocation list** (code-first under the collocation family; LLM adds the long tail when reachable)
|
||||
- [ ] **Grammar lite** rule-pack as a fourth suggestion family: instant, offline, precision-over-recall (near-certain or silent); per-pair L1-interference rules; sourcing per SUGGESTIONS Q6 (hand-curate vs mine LanguageTool's corpus — decide at build time)
|
||||
|
||||
### Later / explicitly not now
|
||||
- Learner-facing Chinese writing (the zh pair's second direction) — own phase with its own spec (SUGGESTIONS §4); only after Phases 19–21 prove the pair model
|
||||
- Spanish pair — gated on DreamDict growing an es dataset
|
||||
- Reactive-animation puppy companion — wishlist, low priority; `companions.ts` roster + mood engine is the drop-in point
|
||||
- Copyleaks Tier-2 — revisit once Phase 15 provides a public webhook endpoint
|
||||
|
||||
### Next-up (post-v1 product, agreed with user 2026-06-26)
|
||||
- [x] **Phase 9 — ESL superpowers**: inline Chinese gloss on hover/select; "say it more naturally" / tone-rewrite. ✅ (see Phase 9 above)
|
||||
@@ -148,6 +222,7 @@ Multi-session build. **Source of truth for what's done and what's next.** Update
|
||||
- [x] **Phase 14 — companion warmth + bedtime nag + night mode**: more encouraging phrases, a gentle "go to bed" nudge after 11pm, and a calm dark theme + falling stars at night. ✅ (see Phase 14 above)
|
||||
|
||||
## Session log
|
||||
- 2026-07-26: **Product direction + execution plan ratified** (user: "make it so, number one"). New `SUGGESTIONS.md` (product rationale for the language-learning direction): the **pair model** — every user gets one (English + X) pair, X ∈ {zh, pt-PT, fr, maybe es}, bilingual UI in the pair, type in either language, direction inferred (no detector: both-dictionaries spellcheck, show-both gloss on collision); **langpacks** keyed by X; **LLM-minimalism** as a standing principle (LLM is garnish, never a gatekeeper — grammar-lite rule pack + embedded miscollocation list planned as code-first layers). Deployment settled: Petal on the **parodia.dev VPS**, vLLM on millenia over **headscale** (the only cross-VPN dependency; Piper is VPS-local). All `MULTIUSER_PLAN.md` OPENs ratified: in-app OIDC (B), 30-day sliding sessions, allowlist, migration script, image-store fix with auth, DreamDict via package import (Option 3, module rename prereq in the dreamdict repo), zh stays on ECDICT until compared. Everything expanded into **Phases 15–22** above with standing rules (isolation tests same-commit, LLM-minimalism, bilingual aesthetic). Ready for implementation handoff starting at Phase 15.
|
||||
- 2026-07-26: **Multi-user groundwork** (user: "let's start preparing Petal for multi-user support"; scope agreed as plumbing-only, aimed at Authentik). New **`internal/auth`** package — context-carried identity (`WithUser`/`UserID`), a `Resolver` seam (`Resolve(*http.Request) (string, error)`), `StaticResolver` for today's single user, and `Middleware` that 401s anything unresolved. `main.go` splits `/api` into a **public group** (`/health`, `/version` — a monitoring probe must not need a session) and an **authenticated group** carrying everything else. All ~35 `db.LocalUserID` call sites across `docs`/`suggestions`/`vocab` now read the caller from the request; helpers that had no request in scope (`fetch`, `ownsDoc`, `ownsTag`, `tagsByDoc`, `fetchVersion`, `passportVersions`, `fetchPending`, vocab `fetch`) take an explicit `userID` param. `UserID` returns `""` rather than panicking when middleware is absent, so a mis-wired route **fails closed** (every query is `WHERE user_id = ?` → matches nothing). **Two real access-control gaps found and fixed while threading**: `setStatus` (accept/dismiss) updated a suggestion by bare id with **no ownership check at all**, and `fetchPending`/`listForDoc` read a document's suggestions by `doc_id` alone — a leak of the quoted source sentences. Both now scope through `documents.user_id`. **A third bug was caught by the new tests, not by the compiler**: `docs.fetch` gained a `userID` parameter but kept binding `db.LocalUserID` in the query — legal Go (unused params compile), silently unscoped, and it would have shipped. New tests: `internal/auth/auth_test.go` (round-trip, absent-context, both 401 paths) and **two-user isolation suites** (`docs/isolation_test.go`, `suggestions/isolation_test.go`) that mount the same routers twice behind two resolvers over one DB and assert a stranger gets 404 on get/update/delete/export/passport/snapshot/version-preview/restore/tag-assign/tag-rename/tag-delete/suggestion-accept/dismiss, sees nothing in list/search/version-list, and leaves the owner's data untouched. go build/vet/test all clean. **Still global, deliberately out of scope** (flagged for the auth phase): the image store is content-addressed with no per-user association or DB row — any authenticated user holding a hash can fetch any image (capability-URL security, needs a table + migration to fix); `export-all` is correctly scoped; frontend `localStorage` keys (`petal.spell.personal`, `petal.companion`, sound/petals prefs) are per-browser, not per-account, so they'd bleed across users sharing a device.
|
||||
- 2026-06-26: **Phases 12 + 13 complete** (collocation coach + vocabulary garden — "finish the rest of the build plan except Authentik/Traefik"). **Phase 12**: collocation drops in as a third suggestion family reusing the whole `runPass`/`pendingScope` machinery — `llm/collocation.go` (`RunCollocation`, 25s floor, reuses `ParseCheckpoint`), `collocationSystemPrompt`/`CollocationMessages` (warm "Natives usually say…" + Mandarin gloss, defers grammar elsewhere), migration `0005` **rebuilds** the suggestions table to extend the `type` CHECK (SQLite can't ALTER a CHECK), `collocationScope` + `CollocationLimit` + `POST /{id}/collocation`. **Caught a latent bug**: `grammarScope` was `type != 'voice'` → would wipe collocation flags; fixed to `type NOT IN ('voice','collocation')`. Frontend: `--color-blossom` pink, "Make it sound natural 🌸" toolbar pill, `collocating`/`runCollocation` in `useCheckpoint`, StatusBar dot — all into the existing rail/card. **Phase 13**: new `internal/vocab` package — migration `0006_vocab_garden` (`vocab_words`, SM-2-lite columns, doc_id `ON DELETE SET NULL`, `UNIQUE(user_id,word)`), `scheduler.go` (Leitner ladder 1/3/7/16/35 → geometric; gentle "again", no streak-shaming), `handlers.go` (capture-upsert/list/due/review/delete, all owner-scoped, time math via SQLite `datetime()` so stored values stay canonical-UTC). Auto-capture wired into `EditorCore.openWordLookup` (only dictionary-known words, captures the surrounding sentence + doc_id) + a 🤍/💚 toggle on `WordCard`. `GardenPanel` slide-over: blossom grid (bloom stage by reps), flashcard review (sentence blanked, flip, again/good/easy, direction alternates recognition↔production), sleepy-kitten footer; opened from a global 🌷 header button. Tests: `TestCollocationPassCoexists`, vocab `scheduler_test.go` + `handlers_test.go`, db CHECK test extended. go build/vet/test + tsc + vite + vitest (51/51) all clean; migration verified against a copy of the live DB; live backend smoke (throwaway DB) walked the full vocab lifecycle + the warm-502 collocation path. **Remaining: only the deferred infra bucket** — Authentik auth, Copyleaks Tier-2 (needs a public webhook), Docker/Traefik/deploy — all on hold per the user's "except Authentik/Traefik".
|
||||
- 2026-06-26: **Phase 14 complete** (companion warmth + bedtime nag + night mode). `tips.ts`: `ENCOURAGEMENTS` 5→10 lines; new `BEDTIME` array (4 lines, user-supplied English wit + gentle Mandarin leads). `useCompanion.ts`: bedtime branch in the 10s heartbeat (after idle-return + break, before the generic tip); only nudges while actively writing; own `lastBedtime` ref + 30min `BEDTIME_GAP`, respects `PROACTIVE_GAP`; new `'bedtime'` `BubbleTone` lingers ~4s longer. **Night mode** (added same session, user request): `lib/night.ts` centralizes `isBedtime()` + window (now shared by the nag too); `hooks/useNightMode.ts` toggles `petal-night` on `<html>` (60s re-check); `index.css` `html.petal-night` re-points only the palette tokens → whole UI flips via `var()` (no component edits), 600ms dusk fade, print stays white; `PetalFall` gains a `night` prop → chunky cartoon power stars (`makeCartoonStar`, Mario/Kirby-style, 5 candy colors) mixed ~70/30 with small twinkle sparkles, gentle spin + shallow shimmer, effect re-inits on flip; App: `useNightMode()` → `<PetalFall night={night}/>`. tsc + vite clean, companion vitest 45/45; verified with real-browser Playwright screenshots (clock mocked to 23:30) — day petals/cream vs night stars/dark-plum, both pretty. Bedtime window is `BEDTIME_FROM`/`BEDTIME_TO` (local clock) for easy retune.
|
||||
|
||||
Reference in New Issue
Block a user