Pete/README.md

Pete

A Matrix news bot that ingests RSS feeds from curated sources and routes each story to a configured channel.

Features

• RSS ingestion from configurable sources (Guardian, Ars Technica, Time Extension, …) with per-source polling intervals
• Direct routing — every source declares a direct_route channel; Pete posts there. No LLM, no classification step
• Deduplication — GUID exact match, canonical-URL match, normalized-headline match, and per-channel canonical-URL cooldown
• Metered release — per-channel post queues with minimum interval (5min) and burst cap (3/30min)
• Round-robin mode — opt-in pacing: one story per N hours (default 4), cycling through channels in sorted order, skip-and-advance over empty channels, state persists across restarts
• Reaction tracking — records emoji reactions on posts
• !post on demand — type !post in any configured channel room and Pete force-publishes the next queued story for that channel, bypassing min-interval, burst cap, and daily cap (canonical-URL dedup still applies); replies in-thread if the queue is empty
• Paywall detection — if an article's visible body text is below threshold, Pete swaps in a Wayback Machine snapshot URL for both the lead image and the posted link
• FTS5 search — full-text search across headlines and ledes
• Image validation — HEAD-based checks filter tracking pixels, uploads valid images via MXC
• Web UI — read-only browser interface (Tailwind, Animal-Crossing-ish vibe, day/night palette that follows the visitor's clock) at /, /gaming, /tech, /politics. Designed for a news.parodia.dev-style deployment

Channels

Channel	Purpose
tech	Technology news, product/industry stories
politics	Political, policy, current events
gaming	Gaming news, releases, platform announcements

Requirements

• Go 1.25+
• A Matrix account for Pete with access to target rooms
• SQLite (bundled via pure Go driver, no CGo)
• No CGo dependencies — E2EE uses pure Go crypto (goolm) via mautrix v0.28

Setup

# Clone and build
git clone <repo-url> && cd pete
go build -tags goolm .

# Create config from example
cp config.example.yaml config.yaml
# Edit config.yaml — fill in Matrix credentials, room IDs, and a direct_route for every source

# First run: seed current feed items as seen (prevents flood)
./pete -config config.yaml -seed

# Post one story to verify the pipeline
./pete -config config.yaml -test

# Run
./pete -config config.yaml

Configuration

See config.example.yaml for the full structure. Key sections:

• matrix — homeserver, credentials, channel room IDs, optional admin room
• posting — rate limiting (min interval, burst cap, daily cap), optional round_robin block
• storage — database path, retention windows
• sources — RSS feeds with tier, polling interval, and required direct_route (must match a key in matrix.channels)
• web — read-only HTTP UI (enabled toggle, listen address, site title, public base URL)

Environment variables can be referenced with ${VAR} syntax in the YAML.

Web UI

Set web.enabled: true (default port :8080) to expose Pete's classified-story archive over HTTP. Three sections (/gaming, /tech, /politics) plus a landing page at /, all pulling from the stories table. Adding a section is two lines in internal/web/server.go (the channels slice) plus a matching theme color in internal/web/static/css/input.css.

The frontend uses a small Tailwind build:

npm install
npm run build:css            # produces internal/web/static/css/output.css
# or for live editing:
npm run watch:css

The Docker image runs the build automatically as a first stage, so deployments don't need Node.

Front the server with a reverse proxy (Caddy, nginx, Traefik) terminating TLS for e.g. news.parodia.dev → 127.0.0.1:8080. The day/night palette is driven by the visitor's browser clock — no server-side timezone needed.

Round-robin mode

Set posting.round_robin.enabled: true to switch Pete from "post immediately" to a paced rotation. On each tick (interval_hours, default 4) Pete picks the newest unposted story routed to the next channel in sorted order, posting through the existing queue. Empty channels are skipped; the rotation pointer advances to whichever channel actually posted, and last_channel / last_tick_at are persisted so restarts don't reset the cycle. Rotating by channel (not by source) guarantees variety even when one channel has many more feeds than the others.

With a 4-hour cadence Pete will post at most 6 stories/day, so set posting.daily_cap_total to 0 (or ≥6) when enabling this — otherwise the daily cap can silently swallow a tick.

Flags

Flag	Description
-config <path>	Path to config file (default: config.yaml)
-seed	Ingest all current feed items as seen without posting, then exit
-test	Post one story to verify the full pipeline, then exit

Docker

docker compose up -d

Set PETE_PASSWORD in your environment or a .env file for the Matrix password.

Architecture

RSS Feed → Poller → GUID/Canonical/Headline Dedup → Article Fetch → Store → Route by direct_route → Image Validate → Queue → Matrix
                                                          ↓
                                              Paywall? → Wayback snapshot

Packages

Package	Purpose
internal/config	YAML config loading with ${ENV} expansion
internal/storage	SQLite with WAL, FTS5, all queries
internal/ingestion	Per-source RSS polling, feed parsing, image validation, paywall detection
internal/dedup	Canonical URL + headline normalization helpers
internal/matrix	Password auth with device persistence, posting, threaded replies, reaction + message listener
internal/poster	Per-channel metered release queue, reaction tracking
internal/scheduler	Round-robin posting scheduler: paced rotation across channels when enabled
internal/web	Read-only HTTP UI: Tailwind templates, day/night palette, per-channel feed pages

Post Format

[lead image if valid]
**Headline** → article_url
Lede text from feed
`source name`

Testing

go test ./...