Initial commit

2026-05-22 17:25:27 -07:00
commit 652d6dfa38
40 changed files with 5855 additions and 0 deletions
--- a/newsbot-spec.md
+++ b/newsbot-spec.md
@@ -0,0 +1,826 @@
+# NewsBot Spec — Claude Code Handoff
+*Sibling bot to GogoBee, built on a shared Matrix/infrastructure library*
+
+---
+
+## Overview
+
+A Matrix news bot that ingests RSS feeds from curated Tier 1 sources, classifies stories using a local LLM, deduplicates semantically, and routes posts to the appropriate channel with a consistent post format. Designed as a sibling to GogoBee, sharing core plumbing via an extracted shared library.
+
+---
+
+## Architecture
+
+### Option Selected: Shared Library (Option B)
+
+Extract reusable GogoBee infrastructure into a shared Go library. The news bot is a separate binary that consumes the shared library. GogoBee remains unchanged initially but can migrate to use the shared library over time.
+
+**Rationale:** Two consumers validate the library's abstraction quality. Clean separation means the news bot can be deployed independently without the full GogoBee stack. Forces good abstractions at the seams.
+
+### Repository Structure
+
+```
+/
+├── lib/                        # Shared library (new)
+│   ├── matrix/                 # Matrix room/event handling primitives
+│   ├── storage/                # SQLite wrapper patterns
+│   ├── config/                 # Config loading / env scaffolding
+│   ├── http/                   # HTTP client + retry/backoff utilities
+│   └── ollama/                 # Ollama API client + polling loop pattern
+├── gogobee/                    # Existing bot (unchanged initially)
+└── newsbot/                    # New bot (this spec)
+    ├── main.go
+    ├── config/
+    ├── ingestion/              # RSS feed polling + parsing
+    ├── classifier/             # LLM classification + dedup
+    ├── storage/                # SQLite schemas for this bot
+    └── poster/                 # Matrix post formatting + delivery
+```
+
+---
+
+## Shared Library Surface Area
+
+Extract from GogoBee (do not duplicate):
+
+| Package | Contents |
+|---|---|
+| `lib/matrix` | Room/event handling primitives, connection lifecycle |
+| `lib/storage` | SQLite open/close, migration support, typed query helpers |
+| `lib/config` | Env var loading, config struct patterns, validation |
+| `lib/http` | HTTP client with retry, backoff, timeout defaults |
+| `lib/ollama` | Ollama API client, polling loop pattern, JSON response handling |
+
+**Rule:** Anything that goes into the shared library must work for two consumers without modification. If it needs a GogoBee-specific assumption, it stays in GogoBee.
+
+---
+
+## Sources
+
+### Initial Source List
+
+| Source | Feed URL | Pre-classification hint |
+|---|---|---|
+| The Guardian — World | `https://www.theguardian.com/world/rss` | politics |
+| The Guardian — Technology | `https://www.theguardian.com/technology/rss` | tech |
+| The Guardian — Politics | `https://www.theguardian.com/politics/rss` | politics |
+| The Guardian — US News | `https://www.theguardian.com/us-news/rss` | politics |
+| Ars Technica — Main | `https://feeds.arstechnica.com/arstechnica/index` | tech |
+| Ars Technica — Policy | `https://feeds.arstechnica.com/arstechnica/policy` | tech+politics |
+
+### Platform Taxonomy
+
+Used for gaming stories. Stored as a JSON array in SQLite, filtered via SQL alongside FTS5 search results.
+
+| Tag | Covers |
+|---|---|
+| `nintendo-switch` | Switch, Switch 2 |
+| `playstation` | PS5, PS4, PSN |
+| `xbox` | Xbox Series X/S, Game Pass |
+| `pc` | Steam, Epic, PC gaming broadly |
+| `retro` | Pre-7th gen, emulation, preservation |
+| `mobile` | iOS/Android gaming |
+| `multi` | Cross-platform releases |
+
+Platform tags are classifier output — derived from headline and lede, not hardcoded per source. A story about a multiplatform release gets `["multi"]`. A Switch exclusive gets `["nintendo-switch"]`. Retro content gets `["retro"]` regardless of original platform.
+
+### Source Metadata (per source in config)
+
+```yaml
+sources:
+  - name: "The Guardian — Technology"
+    feed_url: "https://www.theguardian.com/technology/rss"
+    tier: 1
+    poll_interval_minutes: 20
+    feed_hint: "tech"
+    direct_route: null          # null = LLM routing
+    enabled: true
+```
+
+The `direct_route` field bypasses LLM classification entirely and routes straight to the named channel. Set only when feed sampling confirms the source is clean enough to trust. This is a rare configuration change — sources don't get added often and each new source should be sampled before a routing decision is made.
+
+### Source Tier Taxonomy
+
+Sources are classified on two axes: **funding model** and **political lean**. Both matter independently.
+
+- **Funding model** determines editorial incentive alignment. Subscriber/trust-funded outlets optimize for reader retention and trust, not click-through rate. Ad-funded outlets face structural pressure toward volume, outrage, and shareability regardless of journalist quality.
+- **Political lean** filters for intent and framing. Far-right lean correlates strongly with content produced to cultivate grievance and dehumanize — high production values and deliberate disinformation are not mutually exclusive.
+
+| Tier | Funding | Lean | Criteria | Examples |
+|---|---|---|---|---|
+| 1 | Subscriber/trust funded | Center-left | Strong editorial standards, high signal-to-noise | Guardian, Ars Technica, ProPublica, The Markup, Mother Jones |
+| 2 | Institutional / non-commercial | Lean-agnostic by design | Wire services optimizing for factual accuracy over narrative | AP, Reuters, AFP |
+| 3 | Domain trade press | Lean largely irrelevant | Specialist audience expertise enforces accuracy | Eurogamer, Time Extension, Rock Paper Shotgun (future gaming channel) |
+
+The `tier` field drives classification pipeline behaviour. Tier 1 sources use direct routing or lightweight LLM routing only. Tier 2 sources go through the full gating pipeline before routing.
+
+---
+
+## Channels
+
+Three Matrix rooms:
+
+| Channel | Purpose |
+|---|---|
+| `tech` | Technology news, product/industry stories |
+| `politics` | Political, policy, and significant current events |
+| `gaming` | Gaming news, releases, platform announcements, industry |
+
+Each story is routed to **exactly one channel**. Politics is the catch-all — when a story doesn't clearly belong in tech or gaming, it goes to politics. This applies to tech×policy overlap stories too: an AI regulation story goes to politics, not tech.
+
+The gaming channel is defined in config from day one but will not have active sources until gaming feeds are added. The channel infrastructure, platform tagging, and FTS5 search are implemented now.
+
+---
+
+## Ingestion Pipeline
+
+### Poll Loop
+
+- Configurable interval per source (default 20 minutes)
+- Uses `lib/ollama` polling loop pattern from shared library
+- On each tick: fetch feed → parse items → filter seen GUIDs → pass new items to classifier
+
+### Feed Parsing
+
+- Standard RSS/Atom via `github.com/mmcdole/gofeed` or equivalent
+- Extract per item: GUID, headline, lede (first sentence or description field), image URL, article URL, source name, feed hint
+- Store seen GUIDs in SQLite immediately on receipt (before classification) to prevent reprocessing on next poll
+
+### Image Validation
+
+For each story with an image URL:
+1. HEAD request to image URL — must return 200
+2. Content-Type must be `image/*`
+3. Content-Length (if present) must be > 5KB (filters 1x1 tracking pixels)
+4. On validation failure: post story without image, log warning
+
+Tier 1 sources are trusted editorially — no LLM image relevance check needed.
+
+---
+
+## Post Format
+
+Every story Pete posts follows a consistent structure:
+
+```
+[lead image — rendered inline by Matrix]
+**{headline}** → {article_url}
+{lede}
+`{source_name}`
+```
+
+### Rules
+
+- Lead image is included if validation passes (see Image Validation). Omitted silently on failure — no placeholder, no error message to the room.
+- Headline is bold and hyperlinked to the article URL.
+- Lede is the first sentence of the story description from the feed. Not truncated, not summarized — verbatim from the feed.
+- Source tag is monospaced, lowercase, matches the `name` field from source config.
+- No additional commentary, no bot chatter. Pete posts and stays quiet.
+
+### Gaming Stories
+
+Gaming stories append platform tags after the source tag if platforms are non-empty:
+
+```
+[lead image]
+**{headline}** → {article_url}
+{lede}
+`{source_name}` · `nintendo-switch` · `retro`
+```
+
+
+---
+
+## Classification
+
+### Model
+
+- **Model:** Gemma4 27B (config value — never hardcode model name)
+- **Runtime:** Ollama, local hardware
+- **Config key:** `ollama.model`
+
+### Single Inference Pass
+
+**Tier 1 — direct routing (no LLM):**
+Guardian Politics, Guardian World, and Guardian US News route directly to politics. Feed sampling confirmed these are clean enough that LLM adds no value. GUID dedup check only.
+
+**Tier 1 — LLM routing:**
+Guardian Technology and Ars Technica feeds go through LLM routing. Feed sampling showed meaningful politics bleed in both — Ars Technica is cleaner but not clean enough to skip classification. One LLM call handles routing (exactly one channel) + platform tagging (gaming only) + semantic dedup.
+
+**Tier 2 — all stories:**
+Three-stage gating pipeline (see Tier 2 Gating Logic), then LLM routing for stories that pass.
+
+### Input Context
+
+```
+Incoming story:
+  Source: {source_name}
+  Feed hint: {feed_hint}         ← pre-classification signal from source config
+  Headline: {headline}
+  Lede: {lede}
+
+Recent stories (last 24hr, headline + GUID only):
+  [{guid}] {headline} — {source_name}
+  [{guid}] {headline} — {source_name}
+  ...
+```
+
+Cap recent stories at 50 items to manage token budget. Oldest items dropped first.
+
+### Prompt
+
+### Tier 1 Prompt (routing only)
+
+```
+You are a news router for a community discussion platform with three channels: tech, politics, and gaming.
+
+This story comes from a curated Tier 1 source. It will be posted. Your only job is to decide which channel it belongs in.
+
+Choose exactly one channel:
+- tech: technology products, software, hardware, industry — where technology itself is the primary subject
+- gaming: games, gaming platforms, gaming industry, esports
+- politics: everything else — political, policy, current events, public interest, significant world events
+
+Politics is the default. When in doubt, choose politics.
+
+Routing guidance:
+- GPU launch, software release, hardware review → tech
+- Game release, platform announcement, studio news → gaming
+- Election, legislation, court ruling → politics
+- AI regulation, antitrust action, surveillance policy → politics (policy is the story, not the technology)
+- Death of a public figure → politics
+- Violent crime, assassination attempt, arrest of public figure → politics
+- Natural disaster, humanitarian crisis → politics
+- Celebrity gossip, lifestyle with no broader significance → politics (still gets posted, still needs a home)
+
+For gaming stories only: identify which platforms are relevant from this list:
+nintendo-switch, playstation, xbox, pc, retro, mobile, multi
+Return an empty array if no specific platform applies.
+
+Also check: does this story cover the same underlying event as any story in the recent window? Different outlets covering the same event counts as a duplicate. If yes, set duplicate_of to that story's GUID. If no match, set duplicate_of to null.
+
+Respond ONLY with valid JSON. No preamble, no explanation outside the JSON:
+{
+  "channel": "tech|politics|gaming",
+  "platforms": [string],
+  "duplicate_of": string | null,
+  "rationale": "one sentence max, for logging only"
+}
+
+INCOMING STORY:
+Source: {source}
+Tier: {tier}
+Feed hint: {feed_hint}
+Headline: {headline}
+Lede: {lede}
+
+RECENT STORIES (last 24hr):
+{recent_stories}
+```
+
+### Tier 2 Prompt (gating + routing)
+
+```
+You are a news classifier for a community discussion platform with three channels: tech, politics, and gaming.
+
+This story comes from a wire service. Unlike curated sources, wire feeds publish everything — not all stories warrant posting. First decide if this story is worth posting, then decide where.
+
+Step 1 — Should this be posted?
+Post if the story has clear public significance: major world events, policy decisions, significant deaths, natural disasters, major technology developments. Do not post: local news with no broader relevance, routine financial filings, minor sports results, entertainment fluff.
+
+Step 2 — If posting, choose exactly one channel:
+- tech: technology products, software, hardware, industry
+- gaming: games, gaming platforms, gaming industry
+- politics: everything else — the default
+
+Politics is the default when in doubt.
+
+For gaming stories only: identify relevant platforms from: nintendo-switch, playstation, xbox, pc, retro, mobile, multi
+
+Also check for duplicates against the recent window. If this story covers the same underlying event as a recent story, set duplicate_of to that story's GUID.
+
+Respond ONLY with valid JSON:
+{
+  "post": true|false,
+  "channel": "tech|politics|gaming",
+  "platforms": [string],
+  "duplicate_of": string | null,
+  "rationale": "one sentence max, for logging only"
+}
+
+If post is false, channel may be omitted.
+
+INCOMING STORY:
+Source: {source}
+Tier: {tier}
+Feed hint: {feed_hint}
+Headline: {headline}
+Lede: {lede}
+
+RECENT STORIES (last 24hr):
+{recent_stories}
+```
+
+### Output Handling
+
+```go
+// Tier 1 — routing only
+type Tier1Result struct {
+    Channel     string   `json:"channel"`      // "tech" | "politics" | "gaming"
+    Platforms   []string `json:"platforms"`
+    DuplicateOf *string  `json:"duplicate_of"`
+    Rationale   string   `json:"rationale"`
+}
+
+// Tier 2 — gating + routing
+type Tier2Result struct {
+    Post        bool     `json:"post"`
+    Channel     string   `json:"channel"`      // "tech" | "politics" | "gaming"
+    Platforms   []string `json:"platforms"`
+    DuplicateOf *string  `json:"duplicate_of"`
+    Rationale   string   `json:"rationale"`
+}
+```
+
+**Parse-with-fallback pattern (mandatory):**
+- Wrap all JSON parsing in try/catch
+- On parse failure: log raw LLM output + story GUID → discard story → continue polling loop
+- On scores out of range [0.0, 1.0]: clamp, log warning
+- On missing fields: treat as discard
+- **Never let a malformed LLM response block the polling loop**
+
+Enable Ollama `format: json` parameter to constrain output to valid JSON.
+
+### Routing Logic
+
+Tier 1 stories are always posted — the classifier decides channel only. No threshold, no gating.
+
+Tier 2 stories require the classifier to explicitly set `post: true` before routing proceeds. The rationale field in the classification log is the primary tool for evaluating whether Tier 2 gating is calibrated correctly during the initial period.
+
+---
+
+## Deduplication
+
+### Two-Stage Approach
+
+**Stage 1 — GUID dedup (exact):**
+- Store all seen GUIDs in SQLite on ingestion
+- Fast O(1) lookup, catches same story from same source
+
+**Stage 2 — LLM semantic dedup (approximate):**
+- Handled inside the classification call (see above)
+- `duplicate_of` field — if non-null, discard story regardless of scores
+- Rolling 24-hour window of recent headlines passed as context
+- LLM identifies same underlying event across different outlets/URLs
+
+**No embedding similarity layer in v1** — the two-stage approach (GUID + LLM) is sufficient for two sources with low story velocity. Embeddings can be added later if source count grows significantly.
+
+### SQLite Dedup Schema
+
+```
+table: seen_guids
+  guid TEXT PRIMARY KEY
+  seen_at INTEGER
+
+table: recent_headlines
+  guid TEXT PRIMARY KEY
+  headline TEXT
+  source TEXT
+  seen_at INTEGER
+```
+
+Prune `recent_headlines` entries older than 24 hours on each poll cycle.
+
+---
+
+## Post Format
+
+```
+[image — rendered inline by Matrix]
+**{headline}** → {article_url}
+{lede sentence}
+`{source_name}`
+```
+
+### Cross-channel Posts
+
+When a story routes to both channels, post to each independently with a tag:
+
+```
+[image]
+**{headline}** → {article_url}
+{lede}
+`{source_name}` · `tech × policy`
+```
+
+### Rate Limiting
+
+- Minimum 5-minute gap between posts per channel
+- If multiple stories clear threshold in the same poll cycle, queue and release at interval
+- Prevents burst spam during breaking news events that generate many near-duplicate stories
+
+---
+
+## Storage Schema (SQLite)
+
+```sql
+-- Core story record
+CREATE TABLE stories (
+    id           INTEGER PRIMARY KEY AUTOINCREMENT,
+    guid         TEXT UNIQUE NOT NULL,
+    headline     TEXT NOT NULL,
+    lede         TEXT,
+    image_url    TEXT,
+    article_url  TEXT NOT NULL,
+    source       TEXT NOT NULL,
+    feed_hint    TEXT,
+    platforms    TEXT,              -- JSON array e.g. '["nintendo-switch","multi"]'
+    seen_at      INTEGER NOT NULL   -- unix timestamp
+);
+
+-- Dedup window (last 24hr headlines for LLM context)
+CREATE TABLE recent_headlines (
+    guid       TEXT PRIMARY KEY,
+    headline   TEXT NOT NULL,
+    source     TEXT NOT NULL,
+    seen_at    INTEGER NOT NULL
+);
+
+-- Classification log (7 day retention, for threshold tuning)
+CREATE TABLE classification_log (
+    guid       TEXT PRIMARY KEY,
+    result     TEXT NOT NULL,       -- JSON ClassificationResult
+    logged_at  INTEGER NOT NULL
+);
+
+-- Post log (rate limiting + audit)
+CREATE TABLE post_log (
+    id         INTEGER PRIMARY KEY AUTOINCREMENT,
+    guid       TEXT NOT NULL,
+    channel    TEXT NOT NULL,
+    posted_at  INTEGER NOT NULL
+);
+
+-- Reactions (see Reaction Tracking section)
+CREATE TABLE reactions (
+    id         INTEGER PRIMARY KEY AUTOINCREMENT,
+    post_guid  TEXT NOT NULL,
+    channel    TEXT NOT NULL,
+    event_id   TEXT NOT NULL,
+    emoji      TEXT NOT NULL,
+    user_id    TEXT NOT NULL,
+    reacted_at INTEGER NOT NULL
+);
+CREATE INDEX idx_reactions_post_guid ON reactions(post_guid);
+
+-- FTS5 full-text search index
+CREATE VIRTUAL TABLE stories_fts USING fts5(
+    guid UNINDEXED,
+    headline,
+    lede,
+    source UNINDEXED,
+    platforms UNINDEXED,
+    content='stories',
+    content_rowid='id'
+);
+```
+
+### FTS5 Search
+
+FTS5 is the right tool for freetext search against headlines and ledes — specifically for cases like "show me everything about Elden Ring" where the game title is embedded in headline text. Pure SQL LIKE queries would be slow and miss ranking entirely.
+
+Example queries:
+
+```sql
+-- Search by game title
+SELECT s.* FROM stories s
+JOIN stories_fts f ON s.id = f.rowid
+WHERE stories_fts MATCH 'Elden Ring'
+ORDER BY rank;
+
+-- Search by game title filtered to gaming channel posts
+SELECT s.* FROM stories s
+JOIN stories_fts f ON s.id = f.rowid
+JOIN post_log p ON p.guid = s.guid
+WHERE stories_fts MATCH 'Metroid Prime'
+AND p.channel = 'gaming'
+ORDER BY rank;
+
+-- Filter by platform tag (structured SQL, no FTS5 needed)
+SELECT s.* FROM stories s
+JOIN post_log p ON p.guid = s.guid
+WHERE p.channel = 'gaming'
+AND json_each(s.platforms) LIKE '%nintendo-switch%'
+ORDER BY s.seen_at DESC;
+```
+
+**Note on tokenization:** FTS5 default tokenizer handles game titles (proper nouns, consistent spelling) well. Platform abbreviations like "PS5" and "Xbox" may need a synonym map in v2 once real search patterns are known. Don't over-engineer tokenization before you have real queries to learn from.
+
+---
+
+## Metered Release
+
+When multiple stories clear threshold in the same poll cycle for a single channel, Pete queues and releases them at controlled intervals rather than burst posting.
+
+### Rules
+
+- **Minimum interval** — 5 minutes between posts per channel (configured via `posting.min_interval_seconds`)
+- **Burst cap** — maximum 3 stories per channel per 30 minutes regardless of queue depth
+- **Queue strategy** — FIFO within each channel queue; arrival order is respected
+- **Per-channel isolation** — tech and politics queues are independent; a burst in one channel does not affect the other
+
+### Queue Behaviour
+
+Stories that clear the threshold are added to an in-memory per-channel queue immediately. A ticker drains each queue independently at the minimum interval, subject to the burst cap. Stories that arrive while the burst cap is active wait in queue — they are not discarded.
+
+### Configuration
+
+```yaml
+posting:
+  min_interval_seconds: 300       # 5 minutes between posts per channel
+  burst_cap_count: 3              # max posts per channel per burst window
+  burst_cap_window_seconds: 1800  # 30 minute burst window
+```
+
+### Future Consideration
+
+RSS feed metadata (pubDate recency, outlet-supplied category tags, GUID update frequency) may provide a legitimate urgency signal worth incorporating into queue priority in a future version. Not implemented in v1 — evaluate once real burst patterns are observed.
+
+---
+
+## Configuration
+
+```yaml
+matrix:
+  homeserver: "https://matrix.example.org"
+  access_token: "${MATRIX_ACCESS_TOKEN}"
+  channels:
+    tech: "!roomid:matrix.example.org"
+    politics: "!roomid:matrix.example.org"
+    gaming: "!roomid:matrix.example.org"
+
+ollama:
+  base_url: "http://localhost:11434"
+  model: "gemma4:27b"           # always a config value, never hardcoded
+  timeout_seconds: 30
+
+posting:
+  min_interval_seconds: 300       # 5 minutes between posts per channel
+  burst_cap_count: 3              # max posts per channel per burst window
+  burst_cap_window_seconds: 1800  # 30 minute burst window
+
+storage:
+  db_path: "/var/lib/newsbot/newsbot.db"
+  recent_window_hours: 24
+  classification_log_days: 7
+
+sources:
+  - name: "The Guardian — World"
+    feed_url: "https://www.theguardian.com/world/rss"
+    tier: 1
+    poll_interval_minutes: 20
+    feed_hint: "politics"
+    direct_route: "politics"    # confirmed clean via feed sampling
+    enabled: true
+  - name: "The Guardian — Technology"
+    feed_url: "https://www.theguardian.com/technology/rss"
+    tier: 1
+    poll_interval_minutes: 20
+    feed_hint: "tech"
+    direct_route: null          # LLM routing — significant politics bleed confirmed
+    enabled: true
+  - name: "The Guardian — Politics"
+    feed_url: "https://www.theguardian.com/politics/rss"
+    tier: 1
+    poll_interval_minutes: 20
+    feed_hint: "politics"
+    direct_route: "politics"    # confirmed clean via feed sampling
+    enabled: true
+  - name: "The Guardian — US News"
+    feed_url: "https://www.theguardian.com/us-news/rss"
+    tier: 1
+    poll_interval_minutes: 20
+    feed_hint: "politics"
+    direct_route: "politics"    # confirmed clean via feed sampling
+    enabled: true
+  - name: "Ars Technica"
+    feed_url: "https://feeds.arstechnica.com/arstechnica/index"
+    tier: 1
+    poll_interval_minutes: 20
+    feed_hint: "tech"
+    direct_route: null          # LLM routing — politics bleed confirmed (~20%)
+    enabled: true
+  - name: "Ars Technica — Policy"
+    feed_url: "https://feeds.arstechnica.com/arstechnica/policy"
+    tier: 1
+    poll_interval_minutes: 20
+    feed_hint: "tech+politics"
+    direct_route: null          # LLM routing — feed empty at sampling time
+    enabled: true
+```
+
+---
+
+## Deployment
+
+Pete is deployment-agnostic. Recommended approach:
+- Docker Compose service alongside your existing stack
+- No HTTP ingress required (outbound only — no reverse proxy needed)
+- Config via environment variables + mounted config file
+- Ollama API on localhost or reachable host, shared with other consumers if applicable
+
+### Matrix Identity
+
+Pete requires a dedicated Matrix account:
+- Display name: `Pete`
+- Avatar: configured via Matrix client before deployment — not managed by the bot
+- Bot account should be invited to each channel room and granted appropriate permissions (post messages, read reactions)
+- Optionally configure an admin room in config for operational warnings (feed failures, Ollama unavailability)
+
+```yaml
+matrix:
+  homeserver: "https://matrix.example.org"
+  access_token: "${MATRIX_ACCESS_TOKEN}"
+  admin_room: "!roomid:matrix.example.org"   # optional — operational warnings only
+  channels:
+    tech: "!roomid:matrix.example.org"
+    politics: "!roomid:matrix.example.org"
+    gaming: "!roomid:matrix.example.org"
+```
+
+
+---
+
+## Reaction Tracking
+
+Pete listens for `m.reaction` events referencing his post event IDs and aggregates them per story in SQLite.
+
+### Reaction Categories
+
+Emojis are mapped to categories at query time, not write time. Raw emoji is always stored so categories can be remapped without data loss.
+
+| Category | Emojis | Signal |
+|---|---|---|
+| Relevant | 📰 | Classifier validation — story belonged in this channel |
+| Noise | 🗑️ | Classifier validation — story should not have been posted |
+| Positive | 👍 🔥 ❤️ 😂 🎉 | Story generated a positive feeling |
+| Negative | 👎 😢 😡 😤 😱 | Story generated a negative feeling |
+| Sad | 😢 💔 🕯️ | Grief, loss, mourning |
+| Angry | 😡 😤 🤬 | Outrage, injustice, frustration |
+| Shock | 😱 😨 🤯 | Disbelief, surprise, breaking news |
+
+Categories intentionally overlap — 😢 contributes to both Negative and Sad, 😡 to both Negative and Angry. They measure different dimensions simultaneously.
+
+### Engagement Signals
+
+- **Relevance** — 📰 vs 🗑️ ratio: is the classifier making good calls?
+- **Sentiment** — Positive vs Negative ratio: how did the story land emotionally?
+- **Tone** — Sad / Angry / Shock breakdown: what kind of negative response?
+- **Engagement volume** — total reaction count regardless of type: did the story resonate at all?
+- **Silence** — no reactions: possible threshold miscalibration or source fatigue
+
+A story about a beloved figure dying may generate zero 📰 or 🗑️ but a flood of 😢 💔 — high engagement, clearly worth posting, classifier did its job.
+
+### SQLite Schema
+
+```sql
+CREATE TABLE reactions (
+    id          INTEGER PRIMARY KEY AUTOINCREMENT,
+    post_guid   TEXT NOT NULL,
+    channel     TEXT NOT NULL,
+    event_id    TEXT NOT NULL,       -- Matrix event ID of the reaction
+    emoji       TEXT NOT NULL,
+    user_id     TEXT NOT NULL,       -- Matrix user ID
+    reacted_at  INTEGER NOT NULL     -- unix timestamp
+);
+
+CREATE INDEX idx_reactions_post_guid ON reactions(post_guid);
+```
+
+Categories are derived at query time by joining against an in-memory emoji→category mapping. No category column stored — remapping is free.
+
+### Classifier Tuning Use
+
+- Consistently high 🗑️ from a specific source → flag source for review
+- Consistently high 🗑️ on a specific feed_hint → routing prompt or direct routing logic may need calibration
+- High engagement (any emoji) with low 🗑️ → source and classifier performing well
+- High Angry reactions on stories that cleared politics threshold → expected, not a misfire
+
+---
+
+## Error Handling
+
+### Ollama Unavailable
+
+- On connection failure: log error, skip classification for current poll cycle
+- Do not discard stories — they will be reprocessed on next poll cycle if not yet seen
+- If Ollama is unavailable for 3 consecutive poll cycles: log a warning to Matrix bot admin room (if configured)
+- Never crash the poll loop on Ollama failure
+
+### Matrix Connection Drop
+
+- Use mautrix-go reconnection handling from shared library
+- On reconnect: resume normal operation — no replay of missed stories
+- Stories that arrived during disconnect are processed normally on next poll cycle
+
+### Feed Fetch Failure
+
+- On HTTP error or timeout: log warning, skip source for this cycle
+- If a source fails 5 consecutive polls: log persistent failure warning to admin room
+- Other sources continue unaffected — per-source failure isolation
+
+### Image Validation Failure
+
+- Post story without image, log warning
+- Never block story posting on image failure
+
+### Malformed LLM Response
+
+- Log raw response + story GUID
+- Discard story for this cycle — do not post unclassified content
+- Story is not marked as seen — will be reprocessed on next poll cycle
+- Never crash the poll loop on malformed LLM output
+
+### General Principle
+
+The poll loop must never crash. All errors are logged, isolated, and recovered from. A degraded Pete that posts fewer stories is always preferable to a Pete that stops entirely.
+
+
+---
+
+## Tier 2 Gating Logic
+
+Wire services publish hundreds of stories daily — not all warrant posting. Gating uses a three-stage pipeline to minimize unnecessary LLM calls and keep decisions deterministic where possible.
+
+### Stage 1 — Definite Post (no LLM)
+
+If any of the following match in the headline or lede, post without LLM gating. Route via Tier 2 prompt as normal.
+
+- Named head of state, head of government, or major international body (UN, WHO, NATO, EU, IMF, World Bank) as primary subject
+- Casualty language: "dead", "killed", "injured", "casualties" with a number
+- Disaster language: "earthquake", "hurricane", "tsunami", "flood", "wildfire" with named location
+- Conflict language: "war", "invasion", "coup", "airstrike", "assassination", "siege"
+- Crisis language: "emergency", "crisis", "collapse", "outbreak", "pandemic"
+- Major financial scale: "$Xbn", "$Xtn", "trillion" in context of policy or market event
+
+### Stage 2 — Definite Discard (no LLM)
+
+If any of the following match, discard without LLM call. Log reason.
+
+- Purely local scope: city council, municipal government, local crime, regional weather with no national angle
+- Sports scores, match results, league standings with no broader significance
+- Entertainment and celebrity lifestyle: awards, relationships, appearances, album releases
+- Routine corporate earnings with no systemic significance (not major banks, not Big Tech)
+- Obituaries of non-public figures
+
+### Stage 3 — Ambiguous (escalate to LLM)
+
+Everything that clears neither Stage 1 nor Stage 2 goes to Gemma4 via the Tier 2 prompt. The `post: true|false` decision in the LLM response applies only to this tier — Stages 1 and 2 never reach the model for gating.
+
+### Pipeline Flow
+
+```
+Wire story arrives
+  ↓
+Stage 1: matches definite post criteria?
+  → Yes: skip gating, send to Tier 2 routing prompt
+  ↓
+Stage 2: matches definite discard criteria?
+  → Yes: discard, log reason
+  ↓
+Stage 3: ambiguous — send to Tier 2 prompt (gating + routing)
+  → post: false: discard, log rationale
+  → post: true: route to channel
+```
+
+### Implementation Notes
+
+- Stage 1 and Stage 2 matching is keyword/pattern based — fast, no external calls
+- Keyword lists are config-driven, not hardcoded — adjustable without redeployment
+- All discards are logged with stage and reason for calibration review
+- Stage 3 LLM calls log the full rationale field — primary tool for tuning keyword lists over time
+
+### Future Consideration
+
+As Tier 2 source volume and patterns become known, Stage 1 and Stage 2 keyword lists should be reviewed against the classification log. Patterns that consistently produce correct LLM decisions can be promoted to deterministic rules, reducing LLM calls further.
+
+---
+
+## Out of Scope (v1)
+
+- Reaction data aggregation dashboards and automated threshold tuning based on reaction signals (future)
+- Embedding similarity dedup layer (future, if source count grows)
+- Gaming sources (channel infrastructure is implemented in v1, sources populated later)
+- LLM image relevance scoring (unnecessary for Tier 1 sources)
+- Web scraping (RSS-only in v1)
+- Admin Matrix commands (future — enable/disable sources, adjust thresholds)
+
+---
+
+## Open Questions for Engineering Review
+
+1. Shared library extraction scope — which GogoBee packages are cleanest to extract first?
+2. `gofeed` vs. alternative RSS library — any existing preference in the codebase?
+3. SQLite table naming conventions — follow GogoBee patterns?
+4. Ollama `format: json` parameter — confirm supported by current Ollama version on host
+5. Matrix image attachment method — `m.image` event or inline URL in `m.text`?