From 1b4a5f26df8163d487f4a31463694cbeca9b5fe7 Mon Sep 17 00:00:00 2001 From: prosolis <5590409+prosolis@users.noreply.github.com> Date: Mon, 27 Jul 2026 07:34:36 -0700 Subject: [PATCH] Phase 16 is live: record what deploying found Claude-Session: https://claude.ai/code/session_016y6gyuHkQXPiEuW8RGQyua --- BUILD_PLAN.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/BUILD_PLAN.md b/BUILD_PLAN.md index 5f42459..359b081 100644 --- a/BUILD_PLAN.md +++ b/BUILD_PLAN.md @@ -162,7 +162,7 @@ Petal hosted on the parodia.dev VPS; vLLM stays on millenia over headscale. Auth - [x] **Interim edge gate** (not in the original plan; added once the instance was live). Petal authenticates nobody yet — `StaticResolver` hands every request the same `local` user — so on a public host the whole API was open to read/write and image upload. Traefik basic auth holds the door until Phase 16, with `/api/health` exempt on its own higher-priority router. Deleted when OIDC lands. - [x] Acceptance — verified over public HTTPS **with the LLM link down** (it genuinely is): Hunspell dictionaries 200, gloss + word lookup (incl. phonetic) 200, doc create/save, FTS search on 春天, md + docx export, vocab capture/list, read-aloud EN + zh (real mp3 via ffmpeg, cache hit on repeat, 404 for an unconfigured language so the client falls back). `POST /check` → the warm 502 that renders as 小助手在休息. `/api/health` public; HTTP 301 → HTTPS with a valid cert. -### Phase 16 — Auth (in-app OIDC) + image-store ownership — code complete (2026-07-27), not yet live +### Phase 16 — Auth (in-app OIDC) + image-store ownership ✅ (2026-07-27) — live on petal.parodia.dev Option B ratified. `go-oidc` + `x/oauth2`; config fields already existed. The `Resolver` seam from Phase 0 was the only integration point — no handler or query moved. - [x] OIDC login flow (`internal/auth/oidc.go`): `/auth/login` → Authentik → `/auth/callback` → provision → session. **state** (cookie vs param, constant-time) + **nonce** (ID-token claim vs cookie, so a token minted for another attempt is refused) + **PKCE S256**. Discovery is **lazy and retried**: an Authentik outage blocks new logins but leaves every existing session working, since those need only Petal's own DB — the app must not fail to boot because the IdP is briefly down. - [x] `sessions` table (migration `0010`); opaque token in `petal_session` (`HttpOnly`, `SameSite=Lax`, `Secure` only when `BASE_URL` is https — flagging it on a plain-http dev server makes the browser silently drop it). **The table stores only the token's SHA-256**, so a DB copy yields no usable session. **30-day sliding expiry**, all time math in SQLite `datetime()` (canonical UTC), the extension throttled to one write per hour per session. `/auth/logout` deletes the row, not just the cookie; `RevokeAll` signs one writer out everywhere; expired rows pruned at startup. @@ -174,7 +174,9 @@ Option B ratified. `go-oidc` + `x/oauth2`; config fields already existed. The `R - [x] Tests: `session_test.go` (lifecycle, expiry + prune, sliding renewal, hash-not-token storage, cross-user non-interchangeability, `RevokeAll`, FK cascade, middleware wiring, user upsert, allowlist matrix) and `oidc_test.go` — **the whole round trip against a stub IdP** (RSA-signed ID tokens, real discovery + JWKS): PKCE challenge present, verifier reaches the token endpoint, state mismatch → 400, **replayed nonce from another attempt → 400**, allowlist refusal → the bilingual 403 with no account created, provider error → 403, already-signed-in login short-circuits home. `images/handler_test.go` gained two-user isolation, cross-user dedup + last-owner file deletion, and backfill idempotency. `web/src/lib/drafts.test.ts` covers the rescue (round-trip, per-doc, take-consumes, expiry, corrupted entry, storage that throws). - **A real bug the round-trip test caught:** the one-shot state/nonce/PKCE cookies were cleared with `defer o.clearTemp(w)` — which runs *after* the redirect has written the response header, so the `Set-Cookie` was silently dropped and they lingered in the browser for their full 10 minutes. Now cleared up front. - Verified: go build/vet/test, tsc, vite build, vitest 76/76 all clean. Migration `0010` applied to **a copy of the live millenia DB** (`VACUUM INTO` snapshot): 10 migrations apply, documents/vocab/versions counts unchanged, FTS search still returns hits, the one existing image claimed, `pair_lang` defaulted. Live smoke against the binary on a throwaway DB: auth-off → `/api/me` is `local` and everything 200s; auth-on → `/api/docs` and `/api/me` 401, `/api/health` still public, `/auth/login` with an unreachable IdP renders the warm 503 page, `/auth/logout` redirects home; a hand-inserted session row → 200 with the cookie, 401 without it, with a bad one, and once expired. -- **Not done: this is not live yet.** Registering the provider in Authentik and switching `petal.parodia.dev` over is an outward-facing change on a shared public host, left for the user's go-ahead. `deploy/README.md` §4 is the runbook (provider fields, redirect URI, env, verification, how to force a sign-out). Until it's configured the Traefik basic-auth gate stays — Petal falls back to the single `local` user, so removing the gate before configuring OIDC would open the API to anyone. +- [x] **Deployed** (user: "do it! register it!"). Provider + application registered in Authentik (slug `petal`, confidential, strict redirect `https://petal.parodia.dev/auth/callback`, openid/profile/email, implicit-consent authorization flow), created through `ak shell` since the available API token is a limited invite-minter bot. `.env` filled in on the VPS, image rebuilt, container recreated. **The Traefik basic-auth gate is gone** along with the separate unauthenticated `/api/health` router that existed only to escape it — Petal 401s every `/api` route without a session, so an anonymous visitor gets the app shell and a redirect, and a second password in front of a real login is one more thing to lose. Verified over public HTTPS: `/api/health` 200, `/api/docs` **401 with no basic-auth challenge**, `/auth/login` → Authentik with state+nonce+PKCE in the URL, following it lands on the real sign-in page, `/petal.svg` served as the favicon. The final step — typing her password — is hers. +- **Two bugs deploying caught that the whole test suite could not**, both fatal before the login page ever renders: (1) the code trimmed the issuer's **trailing slash**, and Authentik's issuer has one — OIDC requires a byte-for-byte match, so discovery failed every time while the stub IdP (which advertised a slashless issuer) kept passing. Fixed, and the stub's issuer is now a knob with a regression test that ends in a slash. (2) A provider created through `ak shell` rather than the admin UI comes up with **`grant_types = []`**, which authentik reads as "no grant type is permitted here" and answers with `invalid_request` / *The request is otherwise malformed*. Both are written up in `deploy/README.md` §4. +- **Allowlist is currently `prosolis@proton.me` only.** That Authentik instance fronts ~40 accounts across several applications, so an empty list was not an option, and guessing which account is hers would either lock her out or let a stranger in. Adding her is one line in `.env` plus a restart. Note that an Authentik account with **no email set** (e.g. `akadmin`) can't match an email-based entry — use its subject id. ### Phase 17 — Migrate the `local` user Script, app stopped, backup first (OPEN #4). Depends on: she logs in once so her OIDC `sub` exists. @@ -231,7 +233,7 @@ Each item independent and small; order within is free (SUGGESTIONS §5–§6). - [x] **Phase 14 — companion warmth + bedtime nag + night mode**: more encouraging phrases, a gentle "go to bed" nudge after 11pm, and a calm dark theme + falling stars at night. ✅ (see Phase 14 above) ## Session log -- 2026-07-27: **Phase 16 built — Petal authenticates for itself** (user: "let's continue the build plan"; box access granted mid-session). New `internal/auth` surface on top of the Phase-0 `Resolver` seam: `session.go` (opaque cookie, **SHA-256-at-rest**, 30-day sliding expiry throttled to one write an hour, revoke/revoke-all/prune), `oidc.go` (login/callback/logout with state + nonce + PKCE, **lazy retried discovery** so an IdP outage can't stop Petal booting or invalidate live sessions), `users.go` (provisioning upsert keyed on `sub`, `/api/me`, allowlist). Migration `0010` lands `sessions`, `images` and `users.pair_lang` together. `main.go` picks the resolver from config, so a laptop build is unchanged. **Image ownership** closes the capability-URL hole flagged in the Phase-0 audit — one row per owner keeps dedup, a stranger gets 404 not 403, `Cache-Control` dropped to `private`, and pre-existing files are claimed at startup or they'd all 404. Frontend: a single 401 interceptor, a warm bilingual sign-in overlay over a still-visible editor, and a **draft rescue** to localStorage so an expired session can't cost writing — the auto-save stashes the body it couldn't send and reclaims it after re-login. **Three deliberate deviations from the plan**, all noted above: the allowlist matches emails as well as subject ids (a subject doesn't exist until first login, so a subject-only list is unusable in advance); `SESSION_SECRET` was dropped from config rather than left unused (nothing signs anything — sessions are opaque and server-side); and image rows are keyed `(name, user_id)` rather than owned singly, which is what preserves deduplication. **A real bug caught by writing the round-trip test rather than by reading the code**: the one-shot state/nonce/PKCE cookies were cleared in a `defer`, i.e. after the redirect had already written the header, so the clearing `Set-Cookie` was silently dropped. Verified: full go/tsc/vite/vitest suites, migration `0010` against a `VACUUM INTO` copy of the live millenia DB (counts intact, FTS still matching, image claimed), and a live smoke against the binary in both auth-off and auth-on modes including a hand-inserted session (valid → 200; absent/forged/expired → 401). **Deliberately not deployed**: registering the provider in Authentik and cutting `petal.parodia.dev` over is outward-facing on a shared public host and waits on the user's go-ahead; `deploy/README.md` §4 is the runbook, and the Traefik basic-auth gate stays until it's done. +- 2026-07-27: **Phase 16 built — Petal authenticates for itself** (user: "let's continue the build plan"; box access granted mid-session). New `internal/auth` surface on top of the Phase-0 `Resolver` seam: `session.go` (opaque cookie, **SHA-256-at-rest**, 30-day sliding expiry throttled to one write an hour, revoke/revoke-all/prune), `oidc.go` (login/callback/logout with state + nonce + PKCE, **lazy retried discovery** so an IdP outage can't stop Petal booting or invalidate live sessions), `users.go` (provisioning upsert keyed on `sub`, `/api/me`, allowlist). Migration `0010` lands `sessions`, `images` and `users.pair_lang` together. `main.go` picks the resolver from config, so a laptop build is unchanged. **Image ownership** closes the capability-URL hole flagged in the Phase-0 audit — one row per owner keeps dedup, a stranger gets 404 not 403, `Cache-Control` dropped to `private`, and pre-existing files are claimed at startup or they'd all 404. Frontend: a single 401 interceptor, a warm bilingual sign-in overlay over a still-visible editor, and a **draft rescue** to localStorage so an expired session can't cost writing — the auto-save stashes the body it couldn't send and reclaims it after re-login. **Three deliberate deviations from the plan**, all noted above: the allowlist matches emails as well as subject ids (a subject doesn't exist until first login, so a subject-only list is unusable in advance); `SESSION_SECRET` was dropped from config rather than left unused (nothing signs anything — sessions are opaque and server-side); and image rows are keyed `(name, user_id)` rather than owned singly, which is what preserves deduplication. **A real bug caught by writing the round-trip test rather than by reading the code**: the one-shot state/nonce/PKCE cookies were cleared in a `defer`, i.e. after the redirect had already written the header, so the clearing `Set-Cookie` was silently dropped. Verified: full go/tsc/vite/vitest suites, migration `0010` against a `VACUUM INTO` copy of the live millenia DB (counts intact, FTS still matching, image claimed), and a live smoke against the binary in both auth-off and auth-on modes including a hand-inserted session (valid → 200; absent/forged/expired → 401). **Then deployed** (user: "do it! register it!"): provider + application registered in Authentik via `ak shell`, `.env` filled in, image rebuilt, and the **Traefik basic-auth gate removed** — Petal holds its own door now. Deploying immediately found two things no test could: the issuer's **trailing slash is significant** (Authentik's has one, OIDC compares byte-for-byte, and my normalising it away broke discovery while the slashless stub kept passing — now a knob with a regression test), and a provider created through the shell rather than the admin UI comes up with **empty `grant_types`**, which authentik answers with `invalid_request` before the login page renders. Verified over public HTTPS: health 200, `/api/docs` 401 with no basic-auth challenge, `/auth/login` → Authentik with state+nonce+PKCE, following it lands on the real sign-in page. Also swapped the emoji favicon for a **drawn sakura** (`web/public/petal.svg`) that renders in Petal's own rose palette everywhere instead of at each platform's discretion, and doubles as the Authentik app tile (inlined as a data URI, since this authentik doesn't serve `/media`). **The allowlist is `prosolis@proton.me` only** — that IdP fronts ~40 accounts, so empty was not an option and guessing her account would either lock her out or let a stranger in; adding her is one `.env` line and a restart. - 2026-07-27: **Phase 15 finished off on the two boxes** (user granted millenia access mid-session: `ssh reala@192.168.1.212`, and parodia is `ssh reala@100.64.0.1` over headscale). **LLM link**: rather than rebinding vLLM as planned, `deploy/vllm-headscale-proxy.service` (socat) adds a listener on `100.64.0.2` only — `vllm-chat.service` is shared with **Gogobee** and **Open WebUI** (whose endpoint lives in its own DB, not env), so a rebind meant three consumer edits and a 35B reload; the forwarder cost nothing and no downtime. Grammar checkpoint from the VPS now returns real suggestions in ~3s. **Backups**: the user pointed out the VPS already has daily provider VM backups *and* an age-encrypted offsite `parodia-backup` job, so Petal was folded into the latter instead of running a parallel cron — and doing so **exposed a real bug in that job's `sqlite_dump` helper**: Python's `iterdump` does not reproduce an FTS5 virtual table, so any restore would have come back with cross-document search silently missing (fixed with a `VACUUM INTO`-based helper, round-trip verified). The **bigger** find: millenia, which holds her actual writing, had **no scheduled backup at all** — now `petal-backup.timer`, age-encrypted with the parodia public recipient and pushed off-box, neither machine able to decrypt it. **Encryption at rest** (user raised it; correctly): VPS data dir is now LUKS2 covering the DB, images *and* the TTS cache; key on-box as a deliberate availability tradeoff, documented for what it does and doesn't stop. Rehearsing a reboot caught two bugs a clean run would have hidden — the plaintext originals were still on the unencrypted root fs *under* the mount, and `systemd-cryptsetup` wasn't installed so crypttab was being ignored entirely and the volume would never have unlocked at boot. Added a `.volume-ok` guard so an unmounted volume fails loudly instead of serving a blank DB. **Millenia hygiene**: Piper had been dead since the Jul 26 reboot — **26,800+ failed restarts**, read-aloud silently degrading to browser Web Speech — because an OS upgrade moved `/usr/bin/python3` 3.13→3.14 and the venv's `site-packages` went invisible; venv recreated (lands Piper 1.6.0, which is what `TTS_PATH` exists for), both voices verified through Petal. Petal itself was running unsupervised at PPID 1 and is now `petal.service` (verified by `kill -9`); the Piper units got `StartLimitIntervalSec`/`Burst` so a broken service enters `failed` instead of looping forever unnoticed. Remaining: an external uptime-kuma probe (needs the UI), a true VPS reboot test (shared public host, user's call), and millenia is still unencrypted at rest. - 2026-07-26: **Phase 15 complete — Petal is deployed at https://petal.parodia.dev** (user: "let's start this build plan"; scope confirmed as artifacts **plus** the actual deploy, millenia stays canonical, hostname `petal.parodia.dev`). Stack: `Dockerfile` (node → go → alpine; CGO off, so the runtime layer carries only ffmpeg + tzdata), `docker-compose.yml` behind the host's existing Traefik, and **two Piper sidecars** instead of the planned host systemd units — Piper turned out never to have been installed on the VPS and the account has no lingering session, so containers on an internal network with no published ports are both simpler and tighter. **Three real problems found by deploying rather than by planning:** (1) the image's `petal` user (uid 10001) has no claim on a bind-mounted host directory → SQLite `unable to open database file (14)` and a restart loop; the container now runs as the stack directory's owner (still non-root, and the host account keeps write access the backup script needs); (2) piper-tts **1.6.0 moved synthesis from `POST /` to `POST /synthesize`** with an identical body → every read-aloud 405'd; rather than pin both deployments to one Piper release the path became config (`TTS_PATH`, default `/`, so millenia is untouched); (3) once the instance was live it was **a public, writable, unauthenticated API** — Petal authenticates nobody yet, so Traefik basic auth now holds the door until Phase 16, with `/api/health` exempt on its own higher-priority router. Backups: `db.Backup` via **`VACUUM INTO`** (WAL-coherent, no write lock, single file, refuses to overwrite) behind a `-backup` flag so the nightly job snapshots the running container; `deploy/backup-petal.sh` compresses, pushes to millenia with a size check, prunes both sides; cron at 03:15; restore documented and verified by round-tripping an archive through the binary. Tests: `internal/db/backup_test.go` (WAL capture, seeded user survives, no `-wal`/`-shm` companions, refuses an existing destination, missing source), `internal/tts` path-normalisation + configured-path. go build/vet/test clean. **Acceptance verified over public HTTPS with the LLM link genuinely down**: dictionaries, gloss, word lookup + phonetic, doc create/save, CJK FTS search, md/docx export, vocab capture, read-aloud EN + zh (real mp3, cache hit, 404-fallback for an unconfigured language) — all fine; `/check` → the warm 502 that renders as 小助手在休息; health public, HTTP→HTTPS with a valid cert. **Two items outstanding, both needing millenia access I don't have**: vLLM isn't bound to its headscale interface (so no AI pass works yet), and parodia's ssh key isn't authorized on millenia (so backups are VPS-local only — not yet a real off-box backup). Both have one-command fixes in `deploy/README.md` §3 and §5. Also this session: **DreamDict gained Spanish**, so the es pair is no longer gated — folded into Phases 20/21 and the "Later" bucket. Next: **Phase 16 (auth)** — Authentik already runs on the same VPS. - 2026-07-26: **Product direction + execution plan ratified** (user: "make it so, number one"). New `SUGGESTIONS.md` (product rationale for the language-learning direction): the **pair model** — every user gets one (English + X) pair, X ∈ {zh, pt-PT, fr, maybe es}, bilingual UI in the pair, type in either language, direction inferred (no detector: both-dictionaries spellcheck, show-both gloss on collision); **langpacks** keyed by X; **LLM-minimalism** as a standing principle (LLM is garnish, never a gatekeeper — grammar-lite rule pack + embedded miscollocation list planned as code-first layers). Deployment settled: Petal on the **parodia.dev VPS**, vLLM on millenia over **headscale** (the only cross-VPN dependency; Piper is VPS-local). All `MULTIUSER_PLAN.md` OPENs ratified: in-app OIDC (B), 30-day sliding sessions, allowlist, migration script, image-store fix with auth, DreamDict via package import (Option 3, module rename prereq in the dreamdict repo), zh stays on ECDICT until compared. Everything expanded into **Phases 15–22** above with standing rules (isolation tests same-commit, LLM-minimalism, bilingual aesthetic). Ready for implementation handoff starting at Phase 15.