Initial commit

README.md

@@ -0,0 +1,118 @@
+# Pete
+
+A Matrix news bot that ingests RSS feeds from curated sources, classifies stories using a local LLM, deduplicates semantically, and routes posts to the appropriate channel.
+
+## Features
+
+- **RSS ingestion** from configurable sources (Guardian, Ars Technica) with per-source polling intervals
+- **Two-tier classification** — Tier 1 sources use direct routing or lightweight LLM routing; Tier 2 (wire services) use a keyword gating pipeline before LLM
+- **Semantic deduplication** — GUID exact match + LLM-based cross-source duplicate detection
+- **Metered release** — per-channel post queues with minimum interval (5min) and burst cap (3/30min)
+- **Reaction tracking** — records emoji reactions on posts for classifier tuning
+- **FTS5 search** — full-text search across headlines and ledes
+- **Image validation** — HEAD-based checks filter tracking pixels, uploads valid images via MXC
+
+## Channels
+
+| Channel | Purpose |
+|---|---|
+| `tech` | Technology news, product/industry stories |
+| `politics` | Political, policy, current events (default/catch-all) |
+| `gaming` | Gaming news, releases, platform announcements (no active sources yet) |
+
+## Requirements
+
+- Go 1.25+
+- [Ollama](https://ollama.ai) running locally or on a reachable host
+- A Matrix account for Pete with access to target rooms
+- SQLite (bundled via pure Go driver, no CGo)
+- No CGo dependencies — E2EE uses pure Go crypto (goolm) via mautrix v0.26
+
+## Setup
+
+```bash
+# Clone and build
+git clone <repo-url> && cd pete
+go build -tags goolm .
+
+# Create config from example
+cp config.example.yaml config.yaml
+# Edit config.yaml — fill in Matrix credentials, room IDs, Ollama endpoint
+
+# First run: seed current feed items as seen (prevents flood)
+./pete -config config.yaml -seed
+
+# Post one story to verify the pipeline
+./pete -config config.yaml -test
+
+# Run
+./pete -config config.yaml
+```
+
+## Configuration
+
+See `config.example.yaml` for the full structure. Key sections:
+
+- `matrix` — homeserver, credentials, channel room IDs, optional admin room
+- `ollama` — base URL, model name, timeout
+- `posting` — rate limiting (min interval, burst cap)
+- `storage` — database path, retention windows
+- `sources` — RSS feeds with tier, polling interval, feed hint, optional direct routing
+
+Environment variables can be referenced with `${VAR}` syntax in the YAML.
+
+## Flags
+
+| Flag | Description |
+|---|---|
+| `-config <path>` | Path to config file (default: `config.yaml`) |
+| `-seed` | Ingest all current feed items as seen without posting, then exit |
+| `-test` | Post one story to verify the full pipeline, then exit |
+
+## Docker
+
+```bash
+docker compose up -d
+```
+
+Set `PETE_PASSWORD` in your environment or a `.env` file for the Matrix password.
+
+## Architecture
+
+```
+RSS Feed → Poller → GUID Dedup → Store → Classifier → Semantic Dedup → Image Validate → Queue → Matrix
+                                            ↓
+                                    Direct Route (no LLM)
+                                         or
+                                    Ollama /api/chat (Tier 1 routing)
+                                         or
+                                    Keyword Gating → Ollama (Tier 2)
+```
+
+### Packages
+
+| Package | Purpose |
+|---|---|
+| `internal/config` | YAML config loading with `${ENV}` expansion |
+| `internal/storage` | SQLite with WAL, FTS5, all queries |
+| `internal/ingestion` | Per-source RSS polling, feed parsing, image validation |
+| `internal/classifier` | Ollama client, Tier 1/2 prompts, JSON repair, keyword gating |
+| `internal/matrix` | Password auth with device persistence, posting, reaction listener |
+| `internal/poster` | Per-channel metered release queue, reaction tracking |
+
+## Post Format
+
+```
+[lead image if valid]
+**Headline** → article_url
+Lede text from feed
+`source name`
+```
+
+Gaming stories append platform tags: `` `source` · `nintendo-switch` · `pc` ``
+
+## Testing
+
+```bash
+go test ./...
+```