prosolis
33e49ddb62
Petal's data directory (petal.db, uploaded images, and the TTS cache -- which is synthesized audio of her sentences) now lives on a LUKS volume. LUKS-on-a-file rather than gocryptfs because Petal is SQLite in WAL mode: WAL needs a shared-memory index mapped consistently across processes, and FUSE has a long history of subtle mmap/locking differences. A block device with ext4 behaves exactly like a disk to SQLite. The key sits on the same host, which is a deliberate availability tradeoff and is documented as such: it stops a decommissioned disk or a raw block-device read, not anyone holding the whole VM image. Two bugs found by rehearsing a reboot rather than trusting the setup: - Mounting over a directory HIDES its contents, it does not remove them. The first run left the original plaintext petal.db and WAL sitting on the unencrypted root filesystem, invisible under the mount -- exactly what the exercise was meant to eliminate. Now shredded before the mount, with a refusal if the mountpoint will not come up empty. - systemd-cryptsetup was not installed on the host, so /etc/crypttab was being ignored entirely and the volume would never have unlocked at boot. The script now refuses to run without the generator present.
Deploying Petal
Two deployments exist right now:
| host | shape | status | |
|---|---|---|---|
| millenia | 192.168.1.212 / 100.64.0.2 |
bare binary in a screen session on :8088, Piper as user systemd units |
canonical — her real writing lives here |
| parodia | petal.parodia.dev / 100.64.0.1 |
docker compose behind the host's Traefik | staging; empty database |
millenia stays canonical until Phase 16's auth lands, so she only moves accounts once. Everything below is the VPS side; the millenia Piper notes are kept in the appendix because that instance still runs them.
1. The VPS stack
docker-compose.yml at the repo root brings up three containers:
- petal — the single Go binary with the frontend embedded. Publishes no host port; Traefik is the only way in.
- piper-en / piper-zh — read-aloud. Each Piper HTTP server loads exactly one voice, so English and Chinese are separate containers off one image, with the models cached in a shared volume. They sit on an internal network with no published ports, so only Petal can reach them. Adding pt-PT in Phase 21 is a fourth service, not a new image.
They run as containers rather than the host systemd units millenia uses because
Piper was never actually installed on the VPS, and the reala account has no
lingering session to keep user units alive across logout.
Prerequisites on the host
- Docker with the compose plugin, and the existing external
traefiknetwork - A DNS A record for the hostname pointing at the VPS (
petal.parodia.devis already in place)
First deploy
ssh reala@100.64.0.1
git clone https://gitea.parodia.dev/drwily/petal.git ~/petal
cd ~/petal
cp deploy/petal.env.example .env
Then edit .env:
PETAL_UID/PETAL_GID—id -u/id -gfor this account../datais a bind mount, so the image's ownpetaluser has no claim on it; a mismatch shows up asunable to open database file (14)and a restart loop.PETAL_BASIC_AUTH— the interim edge gate, see §4.LLM_MODEL/LLM_CHAT_MODEL— see §3.
mkdir -p data/backups
docker compose up -d --build
docker compose ps # all three healthy
Updating
cd ~/petal && git pull && docker compose up -d --build
The frontend is embedded in the binary, so a rebuild is the whole deploy. The
client polls /api/version (a hash of the built index.html) and offers a
refresh when it changes.
2. What Traefik does
Labels follow the convention the other services on this box use: the external
traefik network, the web-secure entrypoint, the default cert resolver and
compression@file. Petal adds its own response-header middleware
(frame-ancestors 'self', HSTS, nosniff, Referrer-Policy: same-origin).
/api/health is deliberately on its own higher-priority router with no
middleware: a monitoring probe must not need a credential, and the endpoint
carries no user data.
3. The LLM link over headscale
The vLLM backend stays on millenia and is reached over headscale
(100.64.0.2). This is the only cross-VPN dependency, and by the
LLM-minimalism principle it never gates essential functionality — spell check,
gloss, vocabulary garden, search, export and read-aloud all keep working with
the link down, and the status bar shows the warm
🌙 小助手在休息 · Petal's helper is resting · 文字已保存.
LLM_TIMEOUT is raised from the local-network default of 30s to 90s: the
voice and collocation passes send a whole document, the timeout is a hard
deadline on the completion call, and a WAN+VPN round trip eats the margin.
How the link is exposed — a forwarder, not a rebind
vLLM stays bound to 127.0.0.1:8000. vllm-headscale-proxy.service (a socat
unit, in this directory) adds a second listener on 100.64.0.2:8000 that
forwards to it.
The plan originally said to rebind vLLM itself. That turned out to be the
expensive option: vllm-chat.service is shared — Petal, Gogobee and Open
WebUI all point at 127.0.0.1:8000, and Open WebUI stores its endpoint in its
own database rather than in env — so moving the bind address would mean editing
three consumers and reloading a 35B AWQ model, minutes of downtime for all of
them. The forwarder adds a door instead of moving one: local callers are
untouched, and the only new exposure is on the VPN interface.
It binds 100.64.0.2 specifically, never 0.0.0.0: the far end of this
link is a public host, and the LAN has no business seeing an unauthenticated
inference endpoint.
sudo install -m 0644 deploy/vllm-headscale-proxy.service /etc/systemd/system/
sudo systemctl daemon-reload && sudo systemctl enable --now vllm-headscale-proxy
ss -lntp | grep 8000 # expect BOTH 127.0.0.1:8000 and 100.64.0.2:8000
The model id (qwen3.6-35b) goes into LLM_MODEL / LLM_CHAT_MODEL on the
VPS. Verified end to end: a grammar checkpoint from petal.parodia.dev returns
real suggestions in ~3s over the VPN.
4. Interim edge gate (delete when Phase 16 lands)
Petal authenticates nobody yet — StaticResolver hands every request the same
local user. On a public host that means anyone who finds the hostname can read
and write documents and fill the disk with image uploads, so Traefik holds the
door with basic auth until the OIDC flow exists.
Generate a credential:
htpasswd -nbB petal 'your-password' # or any bcrypt htpasswd generator
and put the resulting user:hash pair in .env as PETAL_BASIC_AUTH.
When Phase 16 lands, delete the petal-auth middleware label, the
petal-health router labels, and this section.
5. Backups
On the VPS — folded into parodia-backup
Petal rides the host's existing offsite job (/usr/local/bin/parodia-backup,
parodia-backup.timer, nightly ~03:40): age-encrypted to S3, 14-day retention,
dead-man snitch. The host holds only the age public recipient, so it writes
backups it cannot itself decrypt.
push petal.db.age sqlite_file_dump /home/reala/petal/data/petal.db
/home/reala/petal/.env is in the same job's secrets tarball — it carries the
interim basic-auth hash and, from Phase 16, the OIDC client secret.
Why sqlite_file_dump and not the script's existing sqlite_dump: that
helper uses Python's iterdump, which does not reproduce an FTS5 virtual
table. It emits documents_fts as a raw sqlite_master row plus its shadow
tables, and replaying the result dies with no such table: documents_fts —
verified by round-tripping a real dump on 2026-07-27. Petal's cross-document
search would have been silently missing after any restore. sqlite_file_dump
runs VACUUM INTO instead: a genuine database file, virtual tables intact, WAL
folded in, no write lock. Restore is a copy rather than a replay.
If
apply.dbever gains a virtual table, it needs the same treatment.
Restore (VPS)
age -d -i <offline-identity> petal.db.age > /tmp/petal.db # from S3
cd ~/petal
docker compose stop petal # stop writers first
mv data/petal.db data/petal.db.before-restore # keep the current state
rm -f data/petal.db-wal data/petal.db-shm # a stale WAL against a new file
cp /tmp/petal.db data/petal.db
docker compose start petal
docker compose logs petal --tail 5 # expect "database ready"
To sanity-check an archive before committing to it, have Petal open it in a scratch directory — a clean exit means it reads end to end:
mkdir -p /tmp/restore-check && cp /tmp/petal.db /tmp/restore-check/petal.db
docker run --rm -v /tmp/restore-check:/data --user "$(id -u):$(id -g)" \
--entrypoint sh petal:local -c '/app/petal -backup /data/verify.db'
⚠️ millenia — the canonical instance — has no automated backup
This is the real gap. Her actual writing lives on millenia, and as of
2026-07-27 nothing backs it up on a schedule: ~/petal/backups holds only
ad-hoc pre-deploy snapshots, the newest a month old. The empty VPS staging
database is currently better protected than the live one.
deploy/backup-petal.sh is written for exactly this and is not yet installed
there — it snapshots via petal -backup (VACUUM INTO, safe against the live
app), compresses, pushes off-box with a post-transfer size check, and prunes
both ends. What it does not yet do is encrypt at rest, which is being
decided separately.
A manual snapshot any time, no tooling required:
cd ~/petal && ./petal -backup ~/petal/backups/manual-$(date -u +%Y%m%dT%H%M%SZ).db
⚠️ millenia's Petal is unsupervised
It runs as a bare ./petal with PPID 1 — no systemd unit, no screen session. A
crash or reboot leaves it down until someone notices. deploy/piper.service is
the pattern to copy if you want it supervised.
Appendix — Piper on millenia (user systemd units)
millenia still runs Piper as user services; these are the original notes.
scp deploy/piper.service deploy/setup-piper.sh 192.168.1.212:/tmp/
ssh 192.168.1.212 'cd /tmp && sudo ./setup-piper.sh'
setup-piper.sh creates ~/piper/venv, installs piper-tts[http], downloads
en_US-amy-medium into ~/piper/voices, installs and enables piper.service
(loopback :5005), and smoke-tests it. Idempotent. Check it with
systemctl status piper / journalctl -u piper -f.
Chinese runs as a second instance (piper-zh.service, :5006,
zh_CN-huayan-medium):
scp deploy/piper-zh.service 192.168.1.212:~/.config/systemd/user/
ssh 192.168.1.212 'export XDG_RUNTIME_DIR=/run/user/$(id -u)
~/piper/venv/bin/python -m piper.download_voices zh_CN-huayan-medium --data-dir ~/piper/voices
systemctl --user daemon-reload && systemctl --user enable --now piper-zh.service'
Petal's env then carries TTS_ENDPOINT=http://127.0.0.1:5005,
TTS_ENDPOINT_ZH=http://127.0.0.1:5006 and the matching voice ids. The handler
maps language → instance from config, so another language is another instance
plus an env pair, no code change.
Piper version note: piper-tts moved synthesis from POST / to
POST /synthesize in 1.6.0, with an identical request body. TTS_PATH selects
which — it defaults to /, and both the VPS compose and millenia's start.sh
now set /synthesize. If read-aloud starts returning 502 after a Piper upgrade,
that flag is the fix.
The venv is fragile across Python upgrades. On 2026-07-27 millenia's Piper
was found dead with 26,800+ failed restarts, silently since the Jul 26
reboot — read-aloud had been falling back to browser Web Speech the whole time.
Root cause: an OS upgrade moved /usr/bin/python3 from 3.13 to 3.14, and
venv/bin/python3 is a symlink to the system interpreter, so the venv's
lib/python3.13/site-packages became invisible — sys.path contained no
site-packages at all. The failure surfaced as the misleading
No module named piper.http_server even though http_server.py was sitting
right there on disk.
Fix (what was done — recreating the venv, not repairing it):
systemctl --user stop piper.service piper-zh.service
mv ~/piper/venv ~/piper/venv.broken-py313
python3 -m venv ~/piper/venv
~/piper/venv/bin/pip install "piper-tts[http]"
~/piper/venv/bin/python -c 'import piper.http_server' # must not raise
systemctl --user start piper.service piper-zh.service
That reinstall lands 1.6.0, so it must be paired with TTS_PATH=/synthesize in
start.sh and a binary new enough to read that variable. Voices in
~/piper/voices survive and do not need re-downloading.
Worth knowing: Restart=on-failure will retry forever without ever alerting.
Neither service reports its health anywhere, which is why this went unnoticed
for a day. A /api/tts probe in uptime-kuma would have caught it.
Verify end to end (through Petal, including the ffmpeg transcode):
curl -sf -X POST localhost:8088/api/tts \
-H 'Content-Type: application/json' \
-d '{"text":"hello there","lang":"en-US"}' -o /tmp/petal-tts.mp3 \
&& file /tmp/petal-tts.mp3 # expect: MPEG ADTS, layer III
A second identical call is served from the cache; a language with no configured instance returns 404 so the client falls back to Web Speech.