millenia: nightly encrypted backup, supervision, and hardened Piper units
The canonical instance -- the one with her actual writing -- turned out to be the least protected thing in the estate: - No scheduled backup at all; the newest snapshot was a month old. Now petal-backup.timer: VACUUM INTO, gzip, age-encrypt with the parodia public recipient, push to the VPS over headscale with a size check, prune both ends. Persistent=true because the box is not on 24/7. Neither machine can decrypt what it holds; the identity is offline. - Petal ran as a bare ./petal with PPID 1, so a crash or reboot left it down until somebody noticed. Now petal.service, verified by kill -9. - The Piper units retried forever without ever failing: RestartSec=3 against systemd's default 10s window means the burst limit is never reached, which is how a dead service logged 26,800+ restarts over a day while read-aloud silently fell back to browser speech. StartLimitIntervalSec=300 makes a broken Piper show up in --failed. backup-petal.sh now handles both deployment shapes (compose exec on the VPS, local binary on millenia) and encrypts before anything leaves the host. The VPS no longer uses it -- Petal rides parodia-backup there.
This commit is contained in:
+47
-15
@@ -1,15 +1,17 @@
|
||||
#!/usr/bin/env bash
|
||||
# Nightly off-VPS backup of Petal's database.
|
||||
# Nightly off-box backup of Petal's database.
|
||||
#
|
||||
# ./backup-petal.sh # snapshot, compress, push off-box, prune
|
||||
# ./backup-petal.sh # snapshot, compress, encrypt, push, prune
|
||||
# ./backup-petal.sh --local-only # snapshot + prune, skip the remote push
|
||||
#
|
||||
# Run it from cron on the VPS (see deploy/README.md). The snapshot itself goes
|
||||
# through `petal -backup`, which uses SQLite's VACUUM INTO: one coherent file
|
||||
# including anything still in the WAL, taken without a write lock, so it is
|
||||
# safe against the live running app. That is why this script never touches
|
||||
# petal.db / -wal / -shm directly — copying those three separately can capture
|
||||
# a torn mid-checkpoint state.
|
||||
# Used on millenia, driven by petal-backup.timer (see deploy/README.md). The
|
||||
# VPS does not use this script -- Petal rides parodia-backup there.
|
||||
#
|
||||
# The snapshot goes through `petal -backup`, which uses SQLite's VACUUM INTO:
|
||||
# one coherent file including anything still in the WAL, taken without a write
|
||||
# lock, so it is safe against the live running app. That is why this script
|
||||
# never touches petal.db / -wal / -shm directly — copying those three
|
||||
# separately can capture a torn mid-checkpoint state.
|
||||
#
|
||||
# Everything below is overridable from the environment.
|
||||
set -euo pipefail
|
||||
@@ -26,6 +28,9 @@ REMOTE_DIR="${REMOTE_DIR:-petal-backups}"
|
||||
# Retention, in days, on each side.
|
||||
KEEP_LOCAL_DAYS="${KEEP_LOCAL_DAYS:-7}"
|
||||
KEEP_REMOTE_DAYS="${KEEP_REMOTE_DAYS:-30}"
|
||||
# age public recipient. Set it and every archive is encrypted before it leaves
|
||||
# (and at rest locally too); leave it empty and the script says so loudly.
|
||||
AGE_RECIPIENT="${AGE_RECIPIENT:-}"
|
||||
|
||||
local_only=0
|
||||
[ "${1:-}" = "--local-only" ] && local_only=1
|
||||
@@ -35,17 +40,44 @@ name="petal-${stamp}.db"
|
||||
|
||||
cd "$STACK_DIR"
|
||||
|
||||
echo ">> snapshotting to data/backups/${name}"
|
||||
# The container writes to its own /data mount; ./data/backups is the same
|
||||
# directory seen from the host.
|
||||
docker compose exec -T petal /app/petal -backup "/data/backups/${name}"
|
||||
|
||||
mkdir -p "$LOCAL_DIR"
|
||||
snapshot="${LOCAL_DIR}/${name}"
|
||||
|
||||
# Two deployment shapes: the VPS runs the compose stack, millenia runs a bare
|
||||
# binary. Either way the snapshot goes through `petal -backup` (VACUUM INTO),
|
||||
# which is safe against the live process, so neither has to stop writing.
|
||||
if [ -f "$STACK_DIR/docker-compose.yml" ] && docker compose ps --status running 2>/dev/null | grep -q petal; then
|
||||
echo ">> snapshotting via the running container -> data/backups/${name}"
|
||||
# ./data/backups on the host is the container's /data/backups.
|
||||
docker compose exec -T petal /app/petal -backup "/data/backups/${name}"
|
||||
elif [ -x "$STACK_DIR/petal" ]; then
|
||||
echo ">> snapshotting via the local binary -> ${snapshot}"
|
||||
# DATABASE_PATH must match the running instance; start.sh is the source of
|
||||
# truth for it, so read it from there rather than guessing.
|
||||
DB_PATH="$(sed -n 's/^export DATABASE_PATH=//p' "$STACK_DIR/start.sh" 2>/dev/null | tail -1)"
|
||||
DATABASE_PATH="${DB_PATH:-$STACK_DIR/data/petal.db}" "$STACK_DIR/petal" -backup "$snapshot"
|
||||
else
|
||||
echo "no way to snapshot: neither a running petal container nor $STACK_DIR/petal" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
[ -s "$snapshot" ] || { echo "snapshot missing or empty: $snapshot" >&2; exit 1; }
|
||||
|
||||
echo ">> compressing"
|
||||
gzip -9 "$snapshot"
|
||||
archive="${snapshot}.gz"
|
||||
|
||||
# Encrypt with age when a recipient is configured. The recipient is a PUBLIC
|
||||
# key -- this host can write backups it cannot itself decrypt, and the private
|
||||
# identity stays offline. Same custody model as parodia-backup. Without this,
|
||||
# an off-box copy is just her writing sitting in plaintext on another machine.
|
||||
if [ -n "$AGE_RECIPIENT" ]; then
|
||||
age -r "$AGE_RECIPIENT" -o "${archive}.age" "$archive"
|
||||
shred -uz "$archive" 2>/dev/null || rm -f "$archive"
|
||||
archive="${archive}.age"
|
||||
else
|
||||
echo " (AGE_RECIPIENT unset: this backup is NOT encrypted)" >&2
|
||||
fi
|
||||
echo " $(du -h "$archive" | cut -f1) ${archive}"
|
||||
|
||||
if [ "$local_only" -eq 0 ] && [ -n "$REMOTE_HOST" ]; then
|
||||
@@ -68,7 +100,7 @@ if [ "$local_only" -eq 0 ] && [ -n "$REMOTE_HOST" ]; then
|
||||
|
||||
echo ">> pruning remote copies older than ${KEEP_REMOTE_DAYS} days"
|
||||
ssh -o BatchMode=yes "$target" \
|
||||
"find '${REMOTE_DIR}' -name 'petal-*.db.gz' -type f -mtime +${KEEP_REMOTE_DAYS} -delete"
|
||||
"find '${REMOTE_DIR}' \\( -name 'petal-*.db.gz' -o -name 'petal-*.db.gz.age' \\) -type f -mtime +${KEEP_REMOTE_DAYS} -delete"
|
||||
elif [ "$local_only" -eq 1 ]; then
|
||||
echo ">> --local-only: skipping the remote push"
|
||||
else
|
||||
@@ -76,6 +108,6 @@ else
|
||||
fi
|
||||
|
||||
echo ">> pruning local copies older than ${KEEP_LOCAL_DAYS} days"
|
||||
find "$LOCAL_DIR" -name 'petal-*.db.gz' -type f -mtime "+${KEEP_LOCAL_DAYS}" -delete
|
||||
find "$LOCAL_DIR" \( -name 'petal-*.db.gz' -o -name 'petal-*.db.gz.age' \) -type f -mtime "+${KEEP_LOCAL_DAYS}" -delete
|
||||
|
||||
echo ">> done"
|
||||
|
||||
Reference in New Issue
Block a user