Bypass-UA retry (Googlebot + Google referer) for soft paywalls, JSON-LD
gating scoped to Article-typed nodes, HTTP 402 treated as explicit
paywall, Wayback freshness filter (30d cap), archive.today as secondary
archive fallback, and transport failures no longer trigger snapshot
swaps. When gating is detected and no archive workaround succeeds, the
story is stored with paywalled=1 and the web card renders a diagonal
red rubber-stamp overlay so readers know the link is gated.
Decode AVIF via avifdec when Go's image.Decode can't, and pass-through
small AVIFs (<=800px wide) by caching the original bytes instead of
re-encoding.
Server picks variant from Lisbon-local date (rain/petals/jacaranda/motes/leaves)
and renders behind content via a canvas particle layer. Each variant has a
hand-drawn silhouette so shapes are recognizable. /weather demo route exposes
variant + intensity + phase pickers, locking the time-of-day phase override.
- Detect explicit paywall markers (article:content_tier meta, JSON-LD
isAccessibleForFree) so metered articles fall back to Wayback even
when body length is above threshold
- Blank-import go.mau.fi/util/dbutil/litestream so the sqlite3-fk-wal
driver mautrix's cryptohelper depends on is registered
- Fix /img route: Go ServeMux requires {wildcard} to be a whole segment,
so capture {name} and strip the .avif suffix in the handler
- Add music as a fourth channel (nav + theme color)
- Glowing themed border on cards that have been posted to Matrix
- Replace per-channel index sections with: "Pete just posted" strip,
channel dashboard (last post, 24h count, totals), unified latest feed
- /img proxy: SSRF-guarded thumbnail re-encoder that resizes to 800px
and runs avifenc -q 45, cached under data/img-cache
- Add -local flag: web/RSS-only mode that skips Matrix login and posting,
so the web UI can be exercised against live feeds without credentials
- Add Makefile (build/local/seed/test/clean) that handles Tailwind +
go build in one shot
- Fix Guardian thumbnails: NormalizeImageURL was rewriting width=1200
onto signed i.guim.co.uk URLs, invalidating the s= signature and
returning 401. Leave signed URLs alone and pick the widest
media:content variant up front instead
- Use pete.avif as the header logo, favicon, and footer mark; drop the
unused leaf.svg
Pete moves to a remote host without Ollama access. Every source must
declare a direct_route channel; the classifier, explainer, semantic
dedup, !explain summaries, feed_hint, and the recent_headlines /
classification_log tables are gone. Deterministic dedup (canonical URL,
headline_norm, per-channel cooldown) remains.
Serves Pete's classified-story archive over HTTP alongside the Matrix
bot. Three sections (gaming/tech/politics), Animal-Crossing-vibe Tailwind
templates, day/night palette driven by the visitor's browser clock.
Web port configurable via web.listen_addr and ${PETE_WEB_PORT} in
docker-compose. Tailwind built in a node stage in the Dockerfile so
deployments don't need node at runtime.
Source-keyed rotation skewed toward whichever channel had the most
feeds (4 of 7 sources routed to politics, so politics dominated the
rotation). Channel-keyed rotation guarantees variety regardless of
feed counts.
Schema: round_robin_state.last_source -> last_channel, added via
addColumnIfMissing so existing DBs migrate in place.
- !post falls back to newest unposted story for the channel when the
in-memory queue is empty (the steady state under round-robin).
- Accept ❓️/❔️ (U+FE0F variation selector) as question reactions —
the bare codepoints alone missed clients that render the colored emoji.
- Rewrite Guardian i.guim.co.uk thumbnails to width=1200 so we stop
rejecting real images as "tracking pixels"; relabel the size warning.
- Log decrypt failures and reactions on events not in post_log so future
silent drops surface instead of vanishing.
Typed in a configured channel room, !post pops the head of that channel's
queue and sends immediately, bypassing min-interval, burst cap, and daily
cap. Canonical-URL dedup still applies. Empty queue gets a threaded reply.
One story per interval_hours (default 4), cycling through enabled sources
in config order. Empty sources are skipped and the pointer advances to
whichever source actually posted. State persists across restarts.
Duplicate-flagged stories now get a _duplicate sentinel channel so they
stay out of the rotation pool alongside _discarded.
OllamaClient.Generate/GenerateText/call now take ctx and build the HTTP
request via NewRequestWithContext, so an in-flight LLM call is aborted
when the parent context is cancelled (Ctrl-C). Classifier.Classify and
its tier helpers take ctx too. ProcessFunc gets a ctx parameter so the
poller can forward its cancellable context down to classification.
Explainer.summarize manages its own 60s context since reaction-driven
flow has no parent ctx to inherit.
Four related fixes after Pete flooded a channel and ignored Ctrl-C:
1. Global daily cap (posting.daily_cap_total, default 5): hard ceiling on
posts across ALL channels in a rolling 24h window. Checked before the
per-channel min-interval and burst-cap.
2. Shutdown no longer flushes the queue. Previous drainAll posted every
remaining item with rate limits disabled — which was literally the
flood. Replaced with dropOnShutdown that clears queues and logs
the count.
3. Poller respects ctx mid-loop. pollOnceWithErr now takes ctx and bails
between items, so Ctrl-C doesn't have to wait for ~30s of network
per pending story before shutdown can complete.
4. Double-image fix. PostStory now reports imageSent; the queue clears
ImageURL before retry so a text-send failure after a successful
image upload doesn't re-post the image.
The previous parser treated any non-empty non-header line as a bullet,
so 'plain paragraph' answers were silently rewrapped as <ul><li>. Now
we only count a line if it had a recognized bullet marker (- * • ·);
if none did, fall back to raw + <pre> as originally intended.
Tests cover dash/asterisk/unicode bullets, blank-line tolerance,
Summary:/TL;DR header stripping, HTML escaping, single-bullet output,
the no-marker passthrough path, and the IsQuestionReaction set.
Now triggers on ❓❔ ⁉ ⁉️🤔 ? ？ — covers the obvious red/white/thinking
variants, the exclamation-question combo (with and without VS16), and
plain ascii / fullwidth question marks for keyboard users.
When a user reacts ❓ on one of Pete's posts, fetch the article body,
ask Ollama for a 3-bullet summary, and post it as a threaded reply
rooted at the original story event. Per-process cooldown of 5min per
story keeps repeated reactions from re-summarizing.
- ingestion.FetchArticleBody: visible <p> text capped at 8000 chars
- classifier.OllamaClient.GenerateText: non-JSON variant
- storage.GetStoryByGUID: full row lookup
- matrix.PostThreadedReply: m.thread + m.in_reply_to fallback
- poster.SetReactionCallback: optional hook fired after recording
- New package: internal/explainer
Fetch each new article once and measure visible <p> text. If body is below
500 chars (or fetch fails), resolve a Wayback snapshot via the
archive.org/wayback/available API and use that URL for both the og:image
fallback and the posted link. Dedup keys stay derived from the original
URL so paywalled/non-paywalled hits collide as before.
- New: internal/ingestion/article.go (FetchArticleMeta via goquery)
- New: internal/ingestion/wayback.go (ResolveWayback)
- Removed: internal/ingestion/og.go (folded into article.go)
- poller.go: dedup first, then one article fetch, then snapshot fallback
