A multi-session build turning Pete's read-only web UI into something people
return to. Five phases, signed-in features keyed off the OIDC subject; anonymous
visitors keep the reverse-chron feed and localStorage-only state.
Phase 1 — per-user read + bookmark state: user_story_state table +
storage/userstate.go; auth-gated /api/read, /api/bookmark, /api/state and a
/bookmarks page; reader.js syncs state server-side for signed-in users. Also
hides the Matrix-posting UI when posting.enabled=false (web-only mode).
Phase 2 — outbound feeds: storage.ListForFeed + web/feed.go hand-build RSS 2.0
(content:encoded) and JSON Feed 1.1 (no new dep); /feed.xml, /feed.json and
per-channel variants; <link rel=alternate> discovery tags.
Phase 3 — "For you" + related: storage/rank.go scores recent unread candidates
by channel/source affinity + recency decay; RelatedStories via FTS5. ForYou rail
+ /for-you page; public /api/related feeds the reader's "You might also like".
Phase 4 — source-health dashboard: source_health table + storage/sourcehealth.go
(RecordPollResult, ListSourceHealth, SourceContentStats), written by the poller;
admin-gated /status page behind web.admin_subs.
Phase 5 — PWA + offline reader + Web Push: root-scoped manifest.webmanifest and
sw.js (app-shell precache, /api/article runtime cache for offline reading,
offline fallback, push/notificationclick handlers); PNG icons from pete.avif;
pwa.js registers the SW and drives a notifications toggle. Web Push adds
webpush-go, a [web.push] config block (pete -genvapid mints VAPID keys), a
push_subscriptions table, auth-gated subscribe/unsubscribe endpoints, and a
digest sender that pings each subscriber "N new stories" past their watermark,
honoring disabled-sources and pruning gone endpoints.
Tests added beside each new storage/web file; go test ./... and go vet clean.
Stories are still ingested, classified, and served to the web UI; only
automatic Matrix posting is gated. Command replies (!post, !petestats)
still work. Pointer-bool so an absent key defaults to posting-on.
The Portugal News moved its RSS behind AWS WAF, which 405s Pete's honest
bot UA (confirmed: bot UA 0/6, browser UA 6/6 from the server IP). Add an
optional per-source user_agent that falls back to the default bot UA, and
set The Portugal News to a browser UA. Not load-related — 30-min polls and
the IP isn't banned.
- Route image-validation, Matrix image-download, and wayback/archive.today
fetches through safehttp so feed-controlled URLs can't reach loopback/
RFC1918/cloud-metadata IPs (incl. via redirects).
- Cap feed body size with safehttp.LimitedBody (16 MiB) to prevent OOM.
- Fail closed if matrix.pickle_key is unset/<16 chars; drop the hardcoded
"pete_pickle_key" default that silently weakened E2EE-at-rest.
- Gate !post behind a matrix.admins allowlist; empty = disabled (channel
rooms are public, so empty must not mean anyone).
- Reject /\host open-redirect bypass in post-login safeNext.
- Allowlist http(s) schemes for the Matrix link href; escape JSON embedded
in inline <script> (prefs/sources blobs).
Signed-in users get their preferences (hidden feeds, weather location,
weather toggle) stored server-side keyed by their OIDC subject and synced
across devices. Anonymous visitors keep using browser localStorage, so the
site stays public. First sign-in migrates existing localStorage prefs up.
- config: [web.auth] section (issuer, client_id/secret, redirect, session_secret)
- storage: user_preferences table + Get/PutUserPrefs
- web/auth: OIDC code flow, HMAC-signed session cookie, CSRF state + nonce
- web/prefs_api: GET/PUT /api/preferences (auth-gated, 64KB cap)
- frontend: prefs.js sync layer seeds localStorage from server, pushes on write
- header: sign-in / account control
OIDC discovery is non-fatal at boot: if Authentik is down, Pete serves
anonymously rather than refusing to start.
When a source sets language = "en", drop items whose per-item
language tag is present and doesn't prefix-match. Items without
a language tag pass through unchanged. Politico Europe is the
motivating case — same headlines appear in en, fr, and de.
Search: new FTS5-backed SearchStories query, /search JSON endpoint,
client-side overlay (search.js) wired into the layout header. EU
channel gets its own theme color (#003399) across bg/text/border/glow
classes. Sources can now route to non-Matrix channels without
validation error (web-only mode); a warning still flags typos.
Pete moves to a remote host without Ollama access. Every source must
declare a direct_route channel; the classifier, explainer, semantic
dedup, !explain summaries, feed_hint, and the recent_headlines /
classification_log tables are gone. Deterministic dedup (canonical URL,
headline_norm, per-channel cooldown) remains.
Serves Pete's classified-story archive over HTTP alongside the Matrix
bot. Three sections (gaming/tech/politics), Animal-Crossing-vibe Tailwind
templates, day/night palette driven by the visitor's browser clock.
Web port configurable via web.listen_addr and ${PETE_WEB_PORT} in
docker-compose. Tailwind built in a node stage in the Dockerfile so
deployments don't need node at runtime.
One story per interval_hours (default 4), cycling through enabled sources
in config order. Empty sources are skipped and the pointer advances to
whichever source actually posted. State persists across restarts.
Duplicate-flagged stories now get a _duplicate sentinel channel so they
stay out of the rotation pool alongside _discarded.
Four related fixes after Pete flooded a channel and ignored Ctrl-C:
1. Global daily cap (posting.daily_cap_total, default 5): hard ceiling on
posts across ALL channels in a rolling 24h window. Checked before the
per-channel min-interval and burst-cap.
2. Shutdown no longer flushes the queue. Previous drainAll posted every
remaining item with rate limits disabled — which was literally the
flood. Replaced with dropOnShutdown that clears queues and logs
the count.
3. Poller respects ctx mid-loop. pollOnceWithErr now takes ctx and bails
between items, so Ctrl-C doesn't have to wait for ~30s of network
per pending story before shutdown can complete.
4. Double-image fix. PostStory now reports imageSent; the queue clears
ImageURL before retry so a text-send failure after a successful
image upload doesn't re-post the image.
