Phase 12 + 13: collocation coach + vocabulary garden

Phase 12 — collocation coach: a third suggestion family for gentle
"natives usually say…" hints on non-native word pairings, reusing the
existing runPass/pendingScope/rail machinery.
- llm/collocation.go (RunCollocation, 25s floor, reuses ParseCheckpoint)
  + collocationSystemPrompt/CollocationMessages (warm, Mandarin gloss,
  defers grammar/spelling to the grammar family)
- migration 0005 rebuilds the suggestions table to extend the type CHECK
  (SQLite can't ALTER a CHECK)
- collocationScope + CollocationLimit + POST /{id}/collocation
- fix: grammarScope was `type != 'voice'` and would wipe the new
  collocation flags; now `type NOT IN ('voice','collocation')`
- frontend: --color-blossom, "Make it sound natural 🌸" pill,
  collocating/runCollocation in useCheckpoint, StatusBar dot

Phase 13 — vocabulary garden: capture looked-up words and surface them
for gentle spaced repetition.
- new internal/vocab package: migration 0006 (vocab_words, SM-2-lite
  columns, doc_id ON DELETE SET NULL, UNIQUE(user_id,word)),
  scheduler.go (Leitner ladder 1/3/7/16/35 then geometric; gentle
  "again", no streak-shaming), handlers (capture-upsert/list/due/
  review/delete, owner-scoped, SQLite-side datetime math)
- auto-capture on word lookup (dictionary-known words only, captures
  the surrounding sentence + doc_id) + 🤍/💚 toggle on WordCard
- GardenPanel: blossom grid (bloom by reps), flashcard review (sentence
  blanked, flip, again/good/easy, recognition↔production), sleepy-kitten
  footer; opened from a global 🌷 header button

Tests: TestCollocationPassCoexists, vocab scheduler + handlers, db CHECK
extended. go build/vet/test + tsc + vite + vitest (51/51) clean;
migration verified against a copy of the live DB; live backend smoke
walked the full vocab lifecycle + the warm-502 collocation path.

Claude-Session: https://claude.ai/code/session_016Yr6jELuRc7hyzYLccQKZd
This commit is contained in:
prosolis
2026-06-26 15:50:25 -07:00
parent 20442d1356
commit 8aa437ec82
23 changed files with 1614 additions and 58 deletions

View File

@@ -105,25 +105,25 @@ Multi-session build. **Source of truth for what's done and what's next.** Update
- [x] **English phonetic (pivot from pinyin)** — for a native-Mandarin English learner the useful pronunciation aid is the English IPA, not pinyin (she reads Chinese fluently). `scripts/build_phonetic.py` extracts ECDICT's `phonetic` column (same source/freq-gate as the gloss); `phonetic.json.gz` embedded + lazily loaded; `Result.Phonetic` resolved via the same de-inflecting candidate walk; shown as `/ˈrɪvər/` in the WordCard. **Full dataset built from ECDICT: 46,579 words (361KB gz)**, in line with the other lexicon assets. The script also has a `--seed` mode (71 curated common words) that ships as a fallback / works without the csv. Curated seed entries (clean IPA) override the ECDICT form where both exist.
- Verified: go build/vet/test (incl. new `TestExportAll`), tsc, vite build all clean; live smoke vs throwaway binaries — `/word/river|running|serendipity|rivers` all return phonetic (`rivers``river` de-inflected), export-all returns a valid 2-entry zip with de-duped CJK filenames + dated name, route doesn't collide with `/{id}`.
### Phase 12 — Collocation coach 🔜 (planned 2026-06-26)
### Phase 12 — Collocation coach (2026-06-26)
**Why:** ESL writers nail grammar but miss *which words go together* — "do a decision" → "make a decision", "strong rain" → "heavy rain". These aren't *wrong*, so the grammar pass won't flag them; they're just non-native. Gentle "natives usually say…" hints are the single highest-leverage upgrade for making her writing sound native. **Build this first** — it's small and de-risks the migration-rebuild pattern Phase 13 also needs.
**Key insight:** the suggestion pipeline is already generic over a `pass` + a `pendingScope` "family" (`runPass` in `internal/suggestions/handlers.go`; grammar + voice already prove it). Collocation drops in as a **third family** and reuses the entire accept/dismiss/re-anchoring/rail/Mandarin-explanation machinery — no new frontend rendering layer.
- [ ] `internal/llm/collocation.go``CollocationInterval` (~25s) + `RunCollocation(...)`, reusing the existing `ParseCheckpoint` parser (same as `RunVoice`). No new parsing.
- [ ] `internal/llm/prompts.go``CollocationMessages(contentText, tone)`. **This prompt is the whole feature**: flag only non-wrong-but-non-native word pairings; explicitly defer real grammar errors to the grammar family; phrase every explanation as warm "natives usually say…" with a Mandarin gloss — never "error/wrong".
- [ ] `internal/db/models.go` + migration `0005` — new suggestion type `collocation`. ⚠️ The `type` column has a `CHECK(...)` and **SQLite can't `ALTER` a CHECK** → migration must **rebuild** the `suggestions` table (create new w/ extended CHECK, copy rows, drop, rename, recreate `idx_suggestions_doc_id`). Not a one-line ALTER.
- [ ] `internal/suggestions/handlers.go` add `collocationScope` (`deleteWhere: "type = 'collocation'"`, `forceType: collocation`), a `CollocationLimit` on `Handler` (+ wire in `New`), register `POST /{id}/collocation`. Mirrors `voice`, ~15 lines. Also extend `normalizeType` to accept the new type.
- [ ] Frontend: `suggestionMeta.ts` collocation entry (💡 icon + its own warm color so cards read as friendly tips); `client.ts` `collocation(docId)`; a gentle trigger like the existing "Check my voice" pill — **"Make it sound natural 🌸"** — rendering straight into `SuggestionRail`/`SuggestionCard`.
- [ ] Verify via the millenia UI-test harness (real-browser editor check) + go/tsc/vite clean. **Est. ~½ day** (mostly the new prompt + the table-rebuild migration).
- [x] `internal/llm/collocation.go``CollocationInterval` (25s) + `RunCollocation(...)`, reusing the existing `ParseCheckpoint` parser (same as `RunVoice`). Whole-document (no `TruncateDoc`), tone passed through.
- [x] `internal/llm/prompts.go``CollocationMessages(contentText, tone)` + `collocationSystemPrompt`. The prompt flags only non-wrong-but-non-native word pairings ("do a decision" → "make a decision"), explicitly defers grammar/spelling to the grammar family, and frames every explanation as warm "Natives usually say…" with a Mandarin gloss — never "error/wrong/mistake".
- [x] `internal/db/models.go` (`SuggestionTypeCollocation`) + migration `0005_collocation_suggestion_type` **rebuilds** the `suggestions` table (new table w/ extended CHECK, copy rows, drop, rename, recreate `idx_suggestions_doc_id`), since SQLite can't `ALTER` a CHECK. Verified against a copy of the live DB (5 migrations apply cleanly, collocation insert accepted, rows preserved).
- [x] `internal/suggestions/handlers.go``collocationScope` (`deleteWhere: "type = 'collocation'"`, `forceType: collocation`), `CollocationLimit` on `Handler` (+ wired in `New`), `POST /{id}/collocation`, `normalizeType` extended. **Also fixed `grammarScope`** from `type != 'voice'``type NOT IN ('voice','collocation')` so a grammar checkpoint no longer wipes the collocation pending flags (the third family must survive like voice does).
- [x] Frontend: `suggestionMeta.ts` collocation entry (`--color-blossom` warm pink, "Word pairing" label); `client.ts` `collocationDoc(docId)` + `SuggestionType` extended; `index.css` token + `.petal-suggestion-collocation` decoration; `useCheckpoint` `collocating`/`runCollocation` (mirrors `runVoice`, shares the run-token guard); `Toolbar` blossom **"Make it sound natural 🌸"** pill (→ "Reading…"); `StatusBar` breathing blossom dot "Finding natural phrasing…"; threaded through `EditorCore`/`App`. Renders straight into the existing `SuggestionRail`/`SuggestionCard` (Accept applies the native pairing).
- [x] Verified: go build/vet/test (`TestCollocationPassCoexists` — three families coexist, grammar checkpoint doesn't wipe voice/collocation), tsc, vite build, vitest 51/51 all clean; live smoke vs the binary (dead LLM) → collocation route returns the warm 502 like check/voice.
### Phase 13 — Vocabulary garden (spaced repetition) 🔜 (planned 2026-06-26)
### Phase 13 — Vocabulary garden (spaced repetition) (2026-06-26)
**Why:** the lexicon (`internal/lexicon`) is a stateless static-dataset lookup — **nothing records which words she's looked up.** Capturing them turns passive lookups into real vocabulary, and the review surface ties straight into the "petal garden" delight idea (words become blossoms; the sleeping kitten naps among them). Build **after** Phase 12.
- [ ] New `internal/vocab` package + migration `0005`/`0006``vocab_words` table: `word, gloss, phonetic, example` (sentence captured at lookup for context), `doc_id`, + SM-2-lite scheduling: `due_at, interval_days, ease, reps, lapses, last_reviewed`; `UNIQUE(user_id, word)`.
- [ ] Auto-capture: when `WordCard` calls `lookupWord`, also fire `POST /api/vocab` (upsert; new word → `due_at = now + 1 day`). Looking words up *is* the data source — zero extra effort from her.
- [ ] Endpoints: `POST /api/vocab` (record/upsert), `GET /api/vocab/due` (cards due now), `POST /api/vocab/{id}/review` (grade again/good/easy → reschedule), `GET /api/vocab` (full garden), `DELETE /api/vocab/{id}`.
- [ ] SR scheduler: gentle SM-2-lite / Leitner intervals (1d → 3d → 7d → 16d → 35d). No streak-shaming. This is the only genuinely new logic.
- [ ] Frontend Garden panel (sibling to `HistoryPanel`/`DocList`): each learned word a blossom, more reps → more bloomed; the `PetalCompanion` kitten naps among them. Flashcard review: captured sentence with the word blanked → recall → EN↔中文 flip → again/good/easy (direction flips for recognition *and* production). A 🤍 "save to garden" on `WordCard` for explicit saves alongside auto-capture.
- [ ] Verify via the millenia UI-test harness + go/tsc/vite clean. **Est. ~1½2 days** (new package + table + panel/review UI; reuses existing panel/companion patterns).
- [x] New `internal/vocab` package + migration `0006_vocab_garden` (0005 was taken by collocation)`vocab_words` table: `word, gloss, phonetic, example` (sentence captured at lookup), `doc_id` (`ON DELETE SET NULL` so a word outlives its source doc), + SM-2-lite scheduling: `due_at, interval_days, ease, reps, lapses, last_reviewed`; `UNIQUE(user_id, word)` + `idx_vocab_due`.
- [x] Auto-capture: `EditorCore.openWordLookup` fires `POST /api/vocab` after a successful lookup — **only for words the dictionary actually knows** (a real gloss or definition), so typos/proper-noun lookups don't clutter the garden. Captures the surrounding sentence (`sentenceAround`) + `doc_id`. Idempotent upsert: re-looking-up a word refreshes its gloss/phonetic/example but never resets its schedule.
- [x] Endpoints (`internal/vocab/handlers.go`): `POST /api/vocab` (upsert; new word → `due_at = datetime('now','+1 day')`), `GET /api/vocab/due` (due now, server-side `datetime('now')` comparison — avoids JS local-vs-UTC parsing bugs), `POST /api/vocab/{id}/review` (grade → reschedule via `datetime('now','+N days')`), `GET /api/vocab` (full garden), `DELETE /api/vocab/{id}`. All owner-scoped.
- [x] SR scheduler (`internal/vocab/scheduler.go`): gentle SM-2-lite / Leitner ladder (1d → 3d → 7d → 16d → 35d, then geometric by ease). "again" steps back to 1d + counts a lapse + nudges ease down (floored at 1.3) — no harsh wipe; "good" climbs one rung; "easy" climbs a rung and a bit more + raises ease. No streaks to break.
- [x] Frontend `GardenPanel` (slide-over drawer, sibling to `HistoryPanel`): each word a blossom that opens further with reps (🌱→🌿→🌷→🌸→🌺); a "复习 N 个词 · Review N due" button; per-word detail (phonetic/example/source-doc/remove); footer "🐱💤 N 朵花在花园里" — the sleepy kitten napping among the blossoms. **Flashcard review**: due queue, the example sentence with the word blanked (`blankOut`), flip to reveal word+phonetic+gloss+sentence, again/good/easy grades; **direction alternates** by cursor parity for recognition (EN→中文) *and* production (中文→EN). A 🤍/💚 "save to garden" toggle on `WordCard` alongside the silent auto-capture. Opened from a global 🌷 词汇花园 button in the app header. All copy bilingual zh-first.
- [x] Verified: go build/vet/test (`scheduler_test.go` — ladder/again-gentle/easy-further; `handlers_test.go` — capture/upsert/due/review/delete lifecycle + empty-word 400 + doc-delete SET NULL), tsc, vite build, vitest 51/51 all clean; live smoke vs the binary (throwaway DB) — full capture→list→due→review→delete flow + 400 on bad grade verified end-to-end.
### Phase 14 — Companion warmth + bedtime nag + night mode ✅
**Why:** the companion kitten is the heart of Petal's "built for her" feel. Three additions: (1) a wider, fresher pool of **encouraging phrases** so cheers don't repeat as quickly; (2) when she's still writing **late at night (≥11pm)**, the kitten gently nags her to go to bed; (3) at the same hour the whole app drifts into a calm **night mode** — dark moonlit theme + the falling petals become **falling stars**. Caring, never scolding — the sleepy-cat gag makes "you should be sleeping too 🐱💤" land perfectly. Self-contained, frontend-only.
@@ -142,11 +142,12 @@ Multi-session build. **Source of truth for what's done and what's next.** Update
### Next-up (post-v1 product, agreed with user 2026-06-26)
- [x] **Phase 9 — ESL superpowers**: inline Chinese gloss on hover/select; "say it more naturally" / tone-rewrite. ✅ (see Phase 9 above)
- [x] **Phase 10 — organization & polish**: cross-doc search, tags, tablet/touch polish, warm LLM-down failure states. ✅ (see Phase 10 above; "tags only" chosen over folders, FTS5 over LIKE)
- [ ] **Phase 12 — collocation coach**: gentle "natives usually say…" hints for non-native word pairings, as a third suggestion family. 🔜 (see Phase 12 above; build first)
- [ ] **Phase 13 — vocabulary garden**: spaced-repetition review built from looked-up words, surfaced as a blooming garden. 🔜 (see Phase 13 above)
- [x] **Phase 12 — collocation coach**: gentle "natives usually say…" hints for non-native word pairings, as a third suggestion family. (see Phase 12 above)
- [x] **Phase 13 — vocabulary garden**: spaced-repetition review built from looked-up words, surfaced as a blooming garden. (see Phase 13 above)
- [x] **Phase 14 — companion warmth + bedtime nag + night mode**: more encouraging phrases, a gentle "go to bed" nudge after 11pm, and a calm dark theme + falling stars at night. ✅ (see Phase 14 above)
## Session log
- 2026-06-26: **Phases 12 + 13 complete** (collocation coach + vocabulary garden — "finish the rest of the build plan except Authentik/Traefik"). **Phase 12**: collocation drops in as a third suggestion family reusing the whole `runPass`/`pendingScope` machinery — `llm/collocation.go` (`RunCollocation`, 25s floor, reuses `ParseCheckpoint`), `collocationSystemPrompt`/`CollocationMessages` (warm "Natives usually say…" + Mandarin gloss, defers grammar elsewhere), migration `0005` **rebuilds** the suggestions table to extend the `type` CHECK (SQLite can't ALTER a CHECK), `collocationScope` + `CollocationLimit` + `POST /{id}/collocation`. **Caught a latent bug**: `grammarScope` was `type != 'voice'` → would wipe collocation flags; fixed to `type NOT IN ('voice','collocation')`. Frontend: `--color-blossom` pink, "Make it sound natural 🌸" toolbar pill, `collocating`/`runCollocation` in `useCheckpoint`, StatusBar dot — all into the existing rail/card. **Phase 13**: new `internal/vocab` package — migration `0006_vocab_garden` (`vocab_words`, SM-2-lite columns, doc_id `ON DELETE SET NULL`, `UNIQUE(user_id,word)`), `scheduler.go` (Leitner ladder 1/3/7/16/35 → geometric; gentle "again", no streak-shaming), `handlers.go` (capture-upsert/list/due/review/delete, all owner-scoped, time math via SQLite `datetime()` so stored values stay canonical-UTC). Auto-capture wired into `EditorCore.openWordLookup` (only dictionary-known words, captures the surrounding sentence + doc_id) + a 🤍/💚 toggle on `WordCard`. `GardenPanel` slide-over: blossom grid (bloom stage by reps), flashcard review (sentence blanked, flip, again/good/easy, direction alternates recognition↔production), sleepy-kitten footer; opened from a global 🌷 header button. Tests: `TestCollocationPassCoexists`, vocab `scheduler_test.go` + `handlers_test.go`, db CHECK test extended. go build/vet/test + tsc + vite + vitest (51/51) all clean; migration verified against a copy of the live DB; live backend smoke (throwaway DB) walked the full vocab lifecycle + the warm-502 collocation path. **Remaining: only the deferred infra bucket** — Authentik auth, Copyleaks Tier-2 (needs a public webhook), Docker/Traefik/deploy — all on hold per the user's "except Authentik/Traefik".
- 2026-06-26: **Phase 14 complete** (companion warmth + bedtime nag + night mode). `tips.ts`: `ENCOURAGEMENTS` 5→10 lines; new `BEDTIME` array (4 lines, user-supplied English wit + gentle Mandarin leads). `useCompanion.ts`: bedtime branch in the 10s heartbeat (after idle-return + break, before the generic tip); only nudges while actively writing; own `lastBedtime` ref + 30min `BEDTIME_GAP`, respects `PROACTIVE_GAP`; new `'bedtime'` `BubbleTone` lingers ~4s longer. **Night mode** (added same session, user request): `lib/night.ts` centralizes `isBedtime()` + window (now shared by the nag too); `hooks/useNightMode.ts` toggles `petal-night` on `<html>` (60s re-check); `index.css` `html.petal-night` re-points only the palette tokens → whole UI flips via `var()` (no component edits), 600ms dusk fade, print stays white; `PetalFall` gains a `night` prop → chunky cartoon power stars (`makeCartoonStar`, Mario/Kirby-style, 5 candy colors) mixed ~70/30 with small twinkle sparkles, gentle spin + shallow shimmer, effect re-inits on flip; App: `useNightMode()``<PetalFall night={night}/>`. tsc + vite clean, companion vitest 45/45; verified with real-browser Playwright screenshots (clock mocked to 23:30) — day petals/cream vs night stars/dark-plum, both pretty. Bedtime window is `BEDTIME_FROM`/`BEDTIME_TO` (local clock) for easy retune.
- 2026-06-26: **Phase 11 complete** (writer power-ups, batch requested as "do it all"). Seven features: (1) in-doc **Find & Replace**`SearchHighlight` decoration extension + `FindReplace` bar (Ctrl/Cmd+F, match-case, replace-all back-to-front, DOM scroll that doesn't trigger the selection bubble); (2) **read-aloud** Web Speech util + 🔊 in WordCard & selection bubble; (3) **keyboard/touch access** — Ctrl/Cmd+D caret lookup, Ctrl/Cmd+J rewrite, touch long-press (refactored `handleContextMenu` → shared `openWordLookup(pos)`); (4) **export-all** backup zip (`GET /api/docs/export-all`, `TestExportAll`, sidebar download links); (5) **smart typography** input-rules extension (curly quotes/em-dash/ellipsis, ASCII-only so CJK untouched); (6) **duplicate doc + sidebar sort + outline popover**; (7) **English phonetic** (pivoted from pinyin — IPA is what an English learner needs; pinyin annotates Chinese she already reads) via `scripts/build_phonetic.py` + embedded `phonetic.json.gz` + `Result.Phonetic` + WordCard `/ˈrɪvər/` line — **full 46,579-word dataset built from ECDICT** (the csv re-download worked; `--seed` mode kept as a csv-free fallback). Also folded in this session: the **selection-bubble vs copy/paste fix** (bubble deferred to pointer-up + container `pointer-events:none` so it never sits where you click). go build/vet/test + tsc + vite all clean; live smoke verified word-phonetic (incl. de-inflection) + export-all zip (de-duped CJK names, route priority). Next: deferred bucket (auth/Copyleaks/deploy), still on hold per user.
- 2026-06-26: **Phase 10 complete** (organization & polish). Scope confirmed with user: all four areas, **tags** (not folders), **FTS5** search. Backend: migration `0004_tags_and_search` (tags + document_tags + `documents_fts` trigram virtual table with sync triggers + back-fill); `db.Tag` model + color constants; `internal/docs/tags.go` (tag CRUD + idempotent assignment + `tagsByDoc` helper, doc list now carries tags); `internal/docs/search.go` (`GET /api/search`, FTS for ≥3 runes + LIKE fallback for 1-2, Go-built sentinel-highlighted rune-aware snippets, owner-scoped). Mounted `/api/tags` + `/api/search` in main.go. Frontend: `useTags`, `TagChip`/`TagPicker`/`SearchBox`, rewritten `DocList`/`DocListItem` (chips + filter bar + search), `api.search`/tag methods + `splitSnippet`/`tagColorVar`; responsive sidebar drawer (hamburger + scrim, <768px) + `pointer:coarse` tap-target/affordance CSS; tap-to-open + outside-pointerdown-close for suggestion cards (touch); `useCheckpoint` `llmDown` flag → warm bilingual "小助手在休息" StatusBar note. Tests: `tags_test.go`, `search_test.go` (incl. update-trigger re-index). All builds/tests/vet/tsc/vite clean; live smoke vs binary on :8061 (dead LLM host) verified search EN/CJK/2-char, full tag lifecycle, check→502 warm path, bundle contents; FTS backfill of pre-existing docs verified. **All v1 phases (07) + post-v1 product (810) done.** Remaining: deferred bucket (auth/Copyleaks/deploy), on hold per user.