Pete/README.md

# Pete

A Matrix news bot that ingests RSS feeds from curated sources and routes each story to a configured channel.

## Features

- **RSS ingestion** from configurable sources (Guardian, Ars Technica, Time Extension, …) with per-source polling intervals
- **Direct routing** — every source declares a `direct_route` channel; Pete posts there. No LLM, no classification step
- **Deduplication** — GUID exact match, canonical-URL match, normalized-headline match, and per-channel canonical-URL cooldown
- **Metered release** — per-channel post queues with minimum interval (5min) and burst cap (3/30min)
- **Round-robin mode** — opt-in pacing: one story per N hours (default 4), cycling through channels in sorted order, skip-and-advance over empty channels, state persists across restarts
- **Reaction tracking** — records emoji reactions on posts
- **`!post` on demand** — type `!post` in any configured channel room and Pete force-publishes the next queued story for that channel, bypassing min-interval, burst cap, and daily cap (canonical-URL dedup still applies); replies in-thread if the queue is empty
- **Paywall detection** — if an article's visible body text is below threshold, Pete swaps in a Wayback Machine snapshot URL for both the lead image and the posted link
- **FTS5 search** — full-text search across headlines and ledes
- **Image validation** — HEAD-based checks filter tracking pixels, uploads valid images via MXC
- **Web UI** — read-only browser interface (Tailwind, Animal-Crossing-ish vibe, day/night palette that follows the visitor's clock) at `/`, `/gaming`, `/tech`, `/politics`. Designed for a `news.parodia.dev`-style deployment

## Channels

| Channel | Purpose |
|---|---|
| `tech` | Technology news, product/industry stories |
| `politics` | Political, policy, current events |
| `gaming` | Gaming news, releases, platform announcements |

## Requirements

- Go 1.25+
- A Matrix account for Pete with access to target rooms
- SQLite (bundled via pure Go driver, no CGo)
- No CGo dependencies — E2EE uses pure Go crypto (goolm) via mautrix v0.28

## Setup

```bash
# Clone and build
git clone <repo-url> && cd pete
go build -tags goolm .

# Create config from example
cp config.example.yaml config.toml
# Edit config.toml — fill in Matrix credentials, room IDs, and a direct_route for every source

# First run: seed current feed items as seen (prevents flood)
./pete -config config.toml -seed

# Post one story to verify the pipeline
./pete -config config.toml -test

# Run
./pete -config config.toml
```

## Configuration

See `config.example.yaml` for the full structure. Key sections:

- `matrix` — homeserver, credentials, channel room IDs, optional admin room
- `posting` — rate limiting (min interval, burst cap, daily cap), optional `round_robin` block
- `storage` — database path, retention windows
- `sources` — RSS feeds with tier, polling interval, and **required** `direct_route` (must match a key in `matrix.channels`)
- `web` — read-only HTTP UI (enabled toggle, listen address, site title, public base URL)

Environment variables can be referenced with `${VAR}` syntax in the TOML.

### Web UI

Set `web.enabled: true` (default port `:8080`) to expose Pete's classified-story archive over HTTP. Three sections (`/gaming`, `/tech`, `/politics`) plus a landing page at `/`, all pulling from the `stories` table. Adding a section is two lines in `internal/web/server.go` (the `channels` slice) plus a matching theme color in `internal/web/static/css/input.css`.

The frontend uses a small Tailwind build:

```bash
npm install
npm run build:css            # produces internal/web/static/css/output.css
# or for live editing:
npm run watch:css
```

The Docker image runs the build automatically as a first stage, so deployments don't need Node.

Front the server with a reverse proxy (Caddy, nginx, Traefik) terminating TLS for e.g. `news.parodia.dev → 127.0.0.1:8080`. The day/night palette is driven by the visitor's browser clock — no server-side timezone needed.

### Round-robin mode

Set `posting.round_robin.enabled: true` to switch Pete from "post immediately" to a paced rotation. On each tick (`interval_hours`, default 4) Pete picks the newest unposted story routed to the next channel in sorted order, posting through the existing queue. Empty channels are skipped; the rotation pointer advances to whichever channel actually posted, and `last_channel` / `last_tick_at` are persisted so restarts don't reset the cycle. Rotating by channel (not by source) guarantees variety even when one channel has many more feeds than the others.

With a 4-hour cadence Pete will post at most 6 stories/day, so set `posting.daily_cap_total` to 0 (or ≥6) when enabling this — otherwise the daily cap can silently swallow a tick.

## Flags

| Flag | Description |
|---|---|
| `-config <path>` | Path to config file (default: `config.toml`) |
| `-seed` | Ingest all current feed items as seen without posting, then exit |
| `-test` | Post one story to verify the full pipeline, then exit |
| `-local` | Web/RSS-only mode: poll feeds and serve the web UI on `web.listen_addr`. Skips Matrix login and posting — useful for local testing |

## Docker

```bash
docker compose up -d
```

Set `PETE_PASSWORD` in your environment or a `.env` file for the Matrix password.

## Architecture

```
RSS Feed → Poller → GUID/Canonical/Headline Dedup → Article Fetch → Store → Route by direct_route → Image Validate → Queue → Matrix
                                                          ↓
                                              Paywall? → Wayback snapshot
```

### Packages

| Package | Purpose |
|---|---|
| `internal/config` | YAML config loading with `${ENV}` expansion |
| `internal/storage` | SQLite with WAL, FTS5, all queries |
| `internal/ingestion` | Per-source RSS polling, feed parsing, image validation, paywall detection |
| `internal/dedup` | Canonical URL + headline normalization helpers |
| `internal/matrix` | Password auth with device persistence, posting, threaded replies, reaction + message listener |
| `internal/poster` | Per-channel metered release queue, reaction tracking |
| `internal/scheduler` | Round-robin posting scheduler: paced rotation across channels when enabled |
| `internal/web` | Read-only HTTP UI: Tailwind templates, day/night palette, per-channel feed pages |

## Post Format

```
[lead image if valid]
**Headline** → article_url
Lede text from feed
`source name`
```

## Testing

```bash
go test ./...
```