Files
petal/MULTIUSER_PLAN.md
T
prosolis 97e9c269ec Phase 20: the dictionary stops being English and Chinese only
Word lookups now come from DreamDict's dict.db for every pair but Chinese —
opened read-only beside petal.db, no service, nothing over the VPN, because a
hover gloss has to answer in milliseconds.

`Provider` is the two questions the popover and the tooltip already asked, so
the embedded *Lexicon satisfies it with no changes at all; Set.For(lang) is the
single place the choice between them is made. The prerequisite in the dreamdict
repo turned out to be two things, not one: the module path was unfetchable
*and* the query layer sat in internal/, which no other module may import
whatever the module is called. Both fixed upstream.

The plan's central assumption did not survive the data. It mapped
Gloss ← Translate(word, "en", L1) one-to-one; against the real 452 MB database
that table answers for 17% of the 2,000 commonest English words into pt-PT.
Wiktionary's translation sections are thin in that direction — "ephemeral",
"think" and "quickly" have no en→pt-PT row at all. Shared WordNet synsets
answer for 61%, so DreamDict gained Equivalents() and Petal glosses through it.
Ordering those was wrong in an instructive way too: sorting by frequency
glosses "think" as lembrar, "remember", because lembrar is the commoner
Portuguese word even though pensar shares six of think's synsets to lembrar's
one. Counting sense agreement first asks the right question.

The same measurement is why zh stays on ECDICT: DreamDict reaches a Chinese
gloss for 53% of those words, ECDICT for nearly all of them. The plan said
converge only if quality holds. It didn't, so nothing converged.

Two decisions about failure worth keeping. A missing dict.db is not an error —
a laptop checkout has never had one — but a present-and-never-imported one is,
because that is a half-finished deploy. And a pt-PT writer with no dictionary
falls back to the embedded datasets with the gloss suppressed, keeping
definitions, synonyms and phonetics rather than blanking the popover: an empty
field reads as "not found", the wrong language reads as broken.

The new fields surface as an etymology line and a three-band chip. Three, not
five: the difficulty score separates "everyday" from "you'll have to explain
this" but cannot rank obfuscate against serendipity, and a finer scale would be
a confident-looking lie. An unscored word gets no chip.

Writing the tests found two bugs first — trimEtymology sliced by byte, which
would have emitted invalid UTF-8 for exactly the Greek and Latin etymologies
the feature exists for, and its ellipsis path overran its own cap.

go build/vet/test, tsc, vite, vitest 96/96 clean; live smoke against the real
dict.db with one instance flipped from zh to pt-PT mid-run.

Not deployed: go.mod still replaces github.com/prosolis/dreamdict with
../dreamdict, so the Docker build needs the two upstream commits pushed and the
replace dropped. The deployed dict.db also predates DreamDict's Spanish data.

Claude-Session: https://claude.ai/code/session_016y6gyuHkQXPiEuW8RGQyua
2026-07-27 09:38:50 -07:00

391 lines
19 KiB
Markdown

# Petal multi-user plan
**Status:** Phase 0 (identity plumbing) landed 2026-07-26 in `6901cdb`.
**Phase A (authentication) + Phase C's image store built 2026-07-27** — in-app
OIDC, server-side sessions, allowlist, provisioning, the frontend 401 path and
per-owner images. Not yet configured against the live Authentik; see
`BUILD_PLAN.md` Phase 16 and `deploy/README.md` §4. Phase B (migrating the
`local` user) still waits on her first real login.
**All OPEN decisions ratified by the user 2026-07-26** (recommendations
accepted as written) — see each OPEN for its settled answer. Execution phases
live in `BUILD_PLAN.md` (Phase 15 onward); product rationale for the language
work is in `SUGGESTIONS.md`.
Settled context that postdates the original draft: Petal will be hosted on the
**parodia.dev VPS**, reaching vLLM on millenia over **headscale VPN**; Piper
TTS is already installed on parodia (VPS-local, no VPN hop). The LLM is the
only cross-VPN dependency, and per the LLM-minimalism principle
(SUGGESTIONS.md §6) it must never gate essential functionality.
---
## 1. Where Petal is today
Single user, by construction. `db.Open` seeds one row (`users.id = 'local'`) and,
until this week, every query named that constant directly.
What was already right: **the schema has been multi-user-shaped from day one.**
`documents`, `tags`, and `vocab_words` all carry `user_id`; `document_versions`
and `suggestions` scope through their parent document. No migration is needed to
support a second user — only a way to know which user is asking.
### What Phase 0 changed
- New `internal/auth`: `Middleware(Resolver)` resolves the caller once per API
request and stores the id in the request context. Handlers read
`auth.UserID(r.Context())`.
- `Resolver` is a one-method interface — `Resolve(*http.Request) (string, error)`
— and is the only thing a real identity provider has to implement.
- `StaticResolver(db.LocalUserID)` supplies today's single user, so behavior is
unchanged.
- `/api` is split into a public group (`/health`, `/version`) and an
authenticated group (everything else).
- Two pre-existing access-control gaps fixed: `setStatus` (accept/dismiss) had
**no ownership check at all**, and `fetchPending` read suggestions by `doc_id`
alone — which leaked the quoted source sentences.
- Two-user isolation test suites (`docs`, `suggestions`) mount the same routers
twice behind two resolvers over one database.
**The remaining work is not "make Petal multi-user."** It is "authenticate
someone, provision them, and clean up the three places where data is still
global."
---
## 2. Goals and non-goals
**Goals**
- Two or more people use one Petal instance without seeing each other's writing.
- The existing local user's data survives, attached to a real account.
- Adding a user is an operator action, not a code change.
**Non-goals (explicitly out, unless a reviewer argues otherwise)**
- Sharing, collaboration, or multi-author documents. Petal is a private writing
space; every feature to date assumes one reader. Sharing would change the
passport's meaning (authorship evidence) and is a product decision, not an
auth one.
- Roles, permissions, or an admin UI.
- Public signup. Accounts are provisioned deliberately.
---
## 3. Phase A — authentication
### OPEN #1: forward-auth vs. in-app OIDC — **SETTLED: Option B (in-app OIDC)**
Ratified 2026-07-26. The deciding fact arrived with the deployment plan: Petal
will live on the public parodia.dev VPS, which is exactly the environment where
Option A's "must never be reachable except through Traefik" invariant is a
footgun. Original analysis kept below for the record.
**Option A — Traefik forward-auth to an Authentik outpost.**
Traefik is already in the deferred deploy bucket. Authentik's outpost terminates
the login, and Petal receives a trusted header (`X-authentik-uid`, plus email and
name). The `Resolver` becomes ~20 lines: read the header, map to a user id.
- *For:* no OIDC library, no session store, no cookie handling, no redirect
plumbing, no token refresh, no logout endpoint. Petal keeps zero auth code.
Login/MFA/password reset are entirely Authentik's problem.
- *Against:* Petal is only secure if it is **never reachable except through
Traefik**. Anyone who can hit the container directly can forge the header and
become any user. That's a deployment invariant enforced by network config, not
by code — and it is exactly the kind of invariant that quietly breaks. It also
makes local development awkward (no proxy → no identity), though
`StaticResolver` covers that.
**Option B — Petal is an OIDC client itself.**
`github.com/coreos/go-oidc` + `golang.org/x/oauth2`, a `/auth/callback` route,
and a signed session cookie. The config fields already exist
(`AUTHENTIK_URL`, `AUTHENTIK_CLIENT_ID`, `AUTHENTIK_CLIENT_SECRET`,
`SESSION_SECRET`).
- *For:* self-contained and safe to expose directly. No trust-the-proxy
invariant. Works the same in dev and prod.
- *Against:* meaningfully more code — a session table or signed-cookie scheme,
CSRF on the callback, token expiry, logout. Two new dependencies in a project
that currently has exactly two.
**My recommendation: Option B**, but not confidently. The deciding factor for me
is that "must never be reachable directly" is a footgun that survives long after
whoever set it up has forgotten, and Petal already holds someone's private
journals. But if the deployment is definitively a single Traefik-fronted box on a
LAN and will stay that way, Option A is *much* less code and I'd not object.
A reviewer should weigh: how likely is this instance ever exposed beyond the LAN?
Does the operator want Petal to be independently deployable?
### Session handling (assumes Option B)
- Server-side sessions in a `sessions` table (id, user_id, expires_at,
created_at, user_agent), cookie holds an opaque random id.
Preferred over signed stateless cookies because it makes logout and
revocation actually work — worth the one table.
- Cookie: `HttpOnly`, `SameSite=Lax`, `Secure` when `BASE_URL` is https.
- **OPEN #2 — SETTLED: 30-day sliding expiry** (ratified 2026-07-26). An
editor that logs you out mid-draft is hostile, and auto-save makes a
surprise 401 genuinely costly. Sliding: each authenticated request extends
the session.
### The 401 problem
Every frontend fetch currently assumes success. Once a session can expire,
**any** call can return 401 mid-session — including the 1.5s auto-save, which is
the one that must not fail silently.
Proposal: a single interceptor in `web/src/api/client.ts` that, on 401, halts
auto-save, surfaces a warm bilingual "请重新登录 / Please sign in again" state
rather than a raw error, and preserves unsaved editor content across the
re-login (localStorage draft keyed by doc id). This is small but easy to forget,
and getting it wrong means lost writing.
---
## 4. Phase B — user provisioning and migration
### Provisioning
On first successful login, upsert a `users` row from the OIDC claims
(`sub``users.id`, plus email and name). No signup flow; whoever Authentik lets
in gets an account.
**OPEN #3 — SETTLED: yes, allowlist** (ratified 2026-07-26). Authentik may
host other applications with a broader user set than Petal should have. Gate
via a `PETAL_ALLOWED_SUBS` env var (or an Authentik group claim check —
implementer's choice, env var is simpler); a valid login not on the list gets
a warm bilingual "this Petal isn't yours to write in" page, not a 500.
### Migrating the existing `local` user
The live database on millenia holds real writing under `user_id = 'local'`. That
data must end up owned by the wife's real account.
Recommended: a migration that **renames** rather than copies — update the
`users.id` and let `ON UPDATE CASCADE`… except SQLite FKs here are declared
without `ON UPDATE`, so this needs either a deliberate multi-table update inside
one transaction (`documents`, `tags`, `vocab_words` — versions and suggestions
follow their parents) with `PRAGMA foreign_keys=OFF` around it, or an explicit
one-off admin command.
I'd rather do this as a **documented one-off script run with the app stopped and
a copy of the DB taken first** than as an automatic startup migration, because it
depends on knowing the new OIDC subject id, which isn't available until that
person logs in once. Sequence: deploy auth → she logs in → new empty account is
created → stop app, back up, run script to move `local`'s rows onto her real id,
delete the empty row → restart.
**OPEN #4 — SETTLED: documented one-off script** (ratified 2026-07-26), run
with the app stopped and a DB backup taken first, per the existing `scripts/`
convention. No admin endpoint.
---
## 5. Phase C — the data that is still global
Found during the Phase 0 audit. None of these break with two users; all of them
leak or bleed.
### Image store — the real one
`internal/images` is a flat content-addressed directory. There is no per-user
association and no database row at all. Any authenticated user who knows a
sha256 can fetch any other user's image.
That is capability-URL security. Hashes aren't guessable, so this is not an
emergency — but "unguessable filename" is not access control, and images pasted
into a private journal are exactly the content that shouldn't rely on it.
Proposal: an `images` table (hash, user_id, content_type, created_at, size) with
the fetch handler joining on the caller. Content addressing is kept — the same
image uploaded by two users is stored once on disk and simply has two rows, so
deduplication survives. Deleting the last row referencing a hash removes the
file.
**OPEN #5 — SETTLED: fix it in the same phase as auth** (ratified
2026-07-26). The moment a second account exists the exposure is real, and the
fix requires a migration either way.
### Frontend `localStorage`
`petal.spell.personal` (personal dictionary), `petal.companion` (chosen mascot),
plus sound and petal-effect preferences are all per-browser. Two users on one
device share them — and the personal dictionary is the one that matters, since
it's built from someone's own writing.
Cheapest fix: namespace every key by user id once the client knows who it is.
The honest fix for the dictionary is to move it server-side into a table, which
also means it follows a user between devices — arguably a feature.
### Not affected
`export-all` is correctly scoped. The TTS cache is content-addressed audio of
text the requester supplied, no cross-user inference. The lexicon is a static
dataset identical for everyone.
---
## 6. Phase D — per-user language (and DreamDict)
Tracked here because it lands on the same `users` row and shouldn't be designed
twice.
**English is always the target language.** What varies is the user's *native*
language — the one glosses and explanations are written in. Mandarin ships today;
European Portuguese (pt-PT, explicitly not pt-BR) is wanted; French is possible.
### OPEN #6 is answered: DreamDict
The original worry here was data sourcing — Petal's gloss comes from ECDICT
(English↔Chinese), and a pt-PT equivalent of comparable quality and license
looked like the blocker.
`~/git/dreamdict` already solves it, and more completely than expected. It
supports **en, fr, pt-PT, and zh** (~136k/56k/136k/121k words), and its shape maps
almost 1:1 onto `lexicon.Result`:
| Petal field | DreamDict |
|---|---|
| `Gloss` | `Translate(word, "en", L1)` |
| `Phonetic` | pronunciation (CMU + IPA for en, Wiktionary IPA elsewhere) |
| `Definitions` | `Define(word, lang)` — curated sources ranked above Wiktionary |
| `Synonyms` | `Synonyms(word, lang)` |
It also carries data Petal has no equivalent for and could use: `Antonyms`,
`Frequency`, `Difficulty`, and `Etymology`.
> **Correction (2026-07-27, Phase 20).** The `Gloss` row of that table is wrong.
> `Translate(word, "en", L1)` reads Wiktionary's translation sections, which are
> thin in the en→X direction: measured on the real `dict.db`, it answers for
> **17%** of the 2,000 commonest English words into pt-PT and 16% into fr.
> Meaning has to come through shared WordNet synset ids instead (**61%**), which
> is what DreamDict's new `Equivalents(word, from, to)` does — falling back to
> the translations table, for 62% combined. The 1:1 mapping was assumed from the
> API surface and never checked against the data; it did not survive contact
> with it. Same measurement on zh reads 53% against ECDICT's near-total coverage
> of those words, which is why the zh pair did **not** converge.
So Phase D stops being gated on data and becomes an integration decision.
### OPEN #6a (new): how to integrate — **SETTLED: Option 3** (ratified 2026-07-26)
Import the package, open `dict.db` read-only. Prerequisite stands: DreamDict's
module path must be renamed (or `replace`-directed) first — **that change
lives in the dreamdict repo, not this one.** The migration caution below also
stands: pt-PT/fr wire to DreamDict first; zh stays on ECDICT until compared on
real lookups. Options kept below for the record.
**Option 1 — HTTP client.** Petal calls DreamDict on localhost:7777, exactly the
pattern already used for Piper TTS (including graceful degradation when it's
down).
- *For:* zero coupling, DreamDict updates independently, all endpoints available.
- *Against:* a second service Petal now depends on at runtime, and the gloss is a
350ms hover tooltip where "the dictionary service is down" is a visible
regression from today's always-there embedded data.
**Option 2 — build-time extraction.** A script (sibling to the existing
`scripts/build_gloss.py`) generates Petal's embedded `.json.gz` datasets per
language from DreamDict's `dict.db`.
- *For:* preserves the embedded/offline property exactly; no runtime dependency;
no architectural change at all.
- *Against:* every language multiplies the binary (the four current gz files are
already ~11.6 MB); updating the dictionary means rebuilding and redeploying
Petal; the richer fields are lost unless separately extracted.
**Option 3 — import the package, open `dict.db` read-only.** DreamDict's
`internal/dictionary` is a plain library with `NewReadOnly(dbPath)`, and its only
dependency is `modernc.org/sqlite` — the same CGO-free driver Petal already uses.
Petal opens `dict.db` as a second read-only handle beside `petal.db`.
- *For:* no service, no HTTP, no new dependency, lookups stay local-file fast,
all four languages at once, and it deletes ~11.6 MB of embedded gz plus the
ECDICT build scripts. One dictionary, maintained once, shared with GogoBee.
- *Against:* Petal stops being a self-contained binary in the "just run it" sense
`dict.db` has to be deployed alongside. In practice Petal already ships a
data directory (`petal.db`, images, TTS cache), so this is a smaller loss than
it first sounds.
**My recommendation: Option 3.** It is the only one that gets all four languages,
keeps lookups offline and instant, and *removes* code rather than adding a
subsystem. Option 1's runtime dependency buys flexibility Petal doesn't need for
a dictionary that changes a few times a year.
**Prerequisite:** DreamDict's module path is currently `module dreamdict`, which
isn't fetchable. Importing it needs the module renamed to something like
`gitea.parodia.dev/drwily/dreamdict` (or a local `replace` directive for
development). Small, but it must happen first.
### Migration caution
Whichever option wins, the zh path is **currently working and in daily use**. The
gloss quality difference between ECDICT and CC-CEDICT is unknown and matters more
than the architecture.
Proposal: introduce DreamDict behind Petal's existing lexicon interface as a
*provider*, wire pt-PT and fr to it first (nothing to regress — they don't exist
yet), and keep zh on ECDICT until the two have been compared on real lookups from
her actual documents. Converge only if quality holds. This also de-risks the whole
change: if DreamDict turns out to be a poor fit, only the unshipped languages are
affected.
### Still per-user regardless
Native language becomes a `users` column, and these become per-user lookups:
LLM prompt copy (`internal/llm/prompts.go`, currently Mandarin-first), companion
tips (`tips.ts`), the L1 Piper voice (Piper has pt-PT voices), and the CJK font
stacks (not needed for Latin-script L1). English-side machinery — nspell en-US,
the phonetic dataset, the EN voice — is unaffected and stays shared.
## 7. Suggested sequence
1. ~~Settle OPEN #1~~ **Settled: in-app OIDC.**
2. Deploy plumbing: Dockerfile, Traefik, real hostname on parodia.dev, HTTPS,
headscale route to vLLM on millenia (bound to the headscale interface
only), VPS-local Piper, off-VPS DB backup. Auth needs a stable `BASE_URL`
and a redirect URI.
3. Auth itself: OIDC `Resolver`, sessions table (30-day sliding), allowlist,
frontend 401 handling with draft preservation.
4. Image store table + migration (same phase, per OPEN #5).
5. Provision the second real account; migrate `local`'s data (script, app
stopped, backup first).
6. `localStorage` namespacing (key by user **and** language — see
SUGGESTIONS.md §8).
7. Per-user language pair. **No longer gated on data** — DreamDict covers all
four languages. Sequence within it: rename DreamDict's module path → wire
it in as a lexicon provider → pt-PT/fr first → compare zh quality →
converge if it holds. The pair model and langpack shape are specified in
SUGGESTIONS.md §1–§3.
These are expanded into checkboxed execution phases in `BUILD_PLAN.md`
(Phase 15 onward) — that file remains the source of truth for progress.
---
## 8. Risks
- **Silent unscoping.** Phase 0 hit this exactly once: `docs.fetch` took a
`userID` parameter and kept binding `db.LocalUserID` in the query. Unused
parameters are legal Go — it compiled, `vet` was silent, and every existing
test passed while the lookup stayed unscoped. Only the two-user isolation test
caught it. **Every new user-scoped endpoint should get an isolation case in the
same commit**; the existing suites are the template.
- **Migrating live data.** The wife's real writing is the thing being moved.
Back up first, run with the app stopped, verify counts before deleting
anything.
- **A 401 mid-draft losing work.** See Phase A.
- **Scope creep into sharing.** Multi-user and collaboration are different
products. Adding accounts should not quietly become adding sharing.
---
## 9. Questions for the reviewer — all answered 2026-07-26
1. ~~Forward-auth or in-app OIDC?~~ **In-app OIDC** (OPEN #1).
2. ~~Session lifetime?~~ **30-day sliding** (OPEN #2).
3. ~~Allowlist?~~ **Yes**, `PETAL_ALLOWED_SUBS` or group claim (OPEN #3).
4. ~~Script vs. admin endpoint?~~ **Script**, app stopped, backup first (OPEN #4).
5. ~~Image store timing?~~ **Same phase as auth** (OPEN #5).
6. ~~pt-PT dictionary data?~~ **DreamDict**, integrated per **Option 3**
(import package, read-only `dict.db`; module rename is the prerequisite)
(OPEN #6a).
7. ~~zh gloss regression risk?~~ **zh stays on ECDICT** until compared against
DreamDict on real lookups from her actual documents; converge only if
quality holds.