Pete

A Matrix news bot that ingests RSS feeds from curated sources, classifies stories using a local LLM, deduplicates semantically, and routes posts to the appropriate channel.

Features

RSS ingestion from configurable sources (Guardian, Ars Technica) with per-source polling intervals
Two-tier classification — Tier 1 sources use direct routing or lightweight LLM routing; Tier 2 (wire services) use a keyword gating pipeline before LLM
Semantic deduplication — GUID exact match + LLM-based cross-source duplicate detection
Metered release — per-channel post queues with minimum interval (5min) and burst cap (3/30min)
Round-robin mode — opt-in pacing: one story per N hours (default 4), cycling through sources in config order, skip-and-advance over empty sources, state persists across restarts
Reaction tracking — records emoji reactions on posts for classifier tuning
!explain via ❓ reaction — react with ❓ (or ❔ ⁉ 🤔 ?) on any post; Pete fetches the article body and replies in-thread with a 3-bullet Ollama-generated TL;DR
Paywall detection — if an article's visible body text is below threshold, Pete swaps in a Wayback Machine snapshot URL for both the lead image and the posted link
FTS5 search — full-text search across headlines and ledes
Image validation — HEAD-based checks filter tracking pixels, uploads valid images via MXC

Channels

Channel	Purpose
`tech`	Technology news, product/industry stories
`politics`	Political, policy, current events (default/catch-all)
`gaming`	Gaming news, releases, platform announcements (no active sources yet)

Requirements

Go 1.25+
Ollama running locally or on a reachable host
A Matrix account for Pete with access to target rooms
SQLite (bundled via pure Go driver, no CGo)
No CGo dependencies — E2EE uses pure Go crypto (goolm) via mautrix v0.28

Setup

# Clone and build
git clone <repo-url> && cd pete
go build -tags goolm .

# Create config from example
cp config.example.yaml config.yaml
# Edit config.yaml — fill in Matrix credentials, room IDs, Ollama endpoint

# First run: seed current feed items as seen (prevents flood)
./pete -config config.yaml -seed

# Post one story to verify the pipeline
./pete -config config.yaml -test

# Run
./pete -config config.yaml

Configuration

See config.example.yaml for the full structure. Key sections:

matrix — homeserver, credentials, channel room IDs, optional admin room
ollama — base URL, model name, timeout
posting — rate limiting (min interval, burst cap, daily cap), optional round_robin block
storage — database path, retention windows
sources — RSS feeds with tier, polling interval, feed hint, optional direct routing

Environment variables can be referenced with ${VAR} syntax in the YAML.

Round-robin mode

Set posting.round_robin.enabled: true to switch Pete from "post on classify" to a paced rotation. On each tick (interval_hours, default 4) Pete picks the newest unposted classified story from the next source in config order, posting through the existing queue. Empty sources are skipped; the rotation pointer advances to whichever source actually posted, and last_source / last_tick_at are persisted so restarts don't reset the cycle.

With a 4-hour cadence Pete will post at most 6 stories/day, so set posting.daily_cap_total to 0 (or ≥6) when enabling this — otherwise the daily cap can silently swallow a tick.

Flags

Flag	Description
`-config <path>`	Path to config file (default: `config.yaml`)
`-seed`	Ingest all current feed items as seen without posting, then exit
`-test`	Post one story to verify the full pipeline, then exit

Docker

docker compose up -d

Set PETE_PASSWORD in your environment or a .env file for the Matrix password.

Architecture

RSS Feed → Poller → GUID Dedup → Canonical/Headline Dedup → Article Fetch → Store → Classifier → Semantic Dedup → Image Validate → Queue → Matrix
                                                                ↓                       ↓
                                                    Paywall? → Wayback           Direct Route (no LLM)
                                                                                       or
                                                                                  Ollama /api/chat (Tier 1 routing)
                                                                                       or
                                                                                  Keyword Gating → Ollama (Tier 2)

Posted message ← ❓ reaction → Explainer → Article Fetch → Ollama summary → Threaded reply

Packages

Package	Purpose
`internal/config`	YAML config loading with `${ENV}` expansion
`internal/storage`	SQLite with WAL, FTS5, all queries
`internal/ingestion`	Per-source RSS polling, feed parsing, image validation
`internal/classifier`	Ollama client, Tier 1/2 prompts, JSON repair, keyword gating
`internal/matrix`	Password auth with device persistence, posting, threaded replies, reaction listener
`internal/poster`	Per-channel metered release queue, reaction tracking, callback hook
`internal/explainer`	❓-reaction → article fetch → Ollama summary → threaded reply
`internal/scheduler`	Round-robin posting scheduler: paced rotation across sources when enabled

Post Format

[lead image if valid]
**Headline** → article_url
Lede text from feed
`source name`

Gaming stories append platform tags: `source` · `nintendo-switch` · `pc`

Testing

go test ./...

5.7 KiB Raw Blame History