Commit Graph

3 Commits

Author SHA1 Message Date
prosolis
5ea64c1d32 Paywall signal detection, sqlite driver registration, /img route fix
- Detect explicit paywall markers (article:content_tier meta, JSON-LD
  isAccessibleForFree) so metered articles fall back to Wayback even
  when body length is above threshold
- Blank-import go.mau.fi/util/dbutil/litestream so the sqlite3-fk-wal
  driver mautrix's cryptohelper depends on is registered
- Fix /img route: Go ServeMux requires {wildcard} to be a whole segment,
  so capture {name} and strip the .avif suffix in the handler
2026-05-25 07:50:36 -07:00
prosolis
3537e073e9 !explain via reaction: thread-reply with a 3-bullet TL;DR
When a user reacts  on one of Pete's posts, fetch the article body,
ask Ollama for a 3-bullet summary, and post it as a threaded reply
rooted at the original story event. Per-process cooldown of 5min per
story keeps repeated reactions from re-summarizing.

- ingestion.FetchArticleBody: visible <p> text capped at 8000 chars
- classifier.OllamaClient.GenerateText: non-JSON variant
- storage.GetStoryByGUID: full row lookup
- matrix.PostThreadedReply: m.thread + m.in_reply_to fallback
- poster.SetReactionCallback: optional hook fired after recording
- New package: internal/explainer
2026-05-22 18:27:11 -07:00
prosolis
e26b69e43f Paywall detection: swap to Wayback snapshot when article body is gated
Fetch each new article once and measure visible <p> text. If body is below
500 chars (or fetch fails), resolve a Wayback snapshot via the
archive.org/wayback/available API and use that URL for both the og:image
fallback and the posted link. Dedup keys stay derived from the original
URL so paywalled/non-paywalled hits collide as before.

- New: internal/ingestion/article.go (FetchArticleMeta via goquery)
- New: internal/ingestion/wayback.go (ResolveWayback)
- Removed: internal/ingestion/og.go (folded into article.go)
- poller.go: dedup first, then one article fetch, then snapshot fallback
2026-05-22 17:53:54 -07:00