Commit Graph

3 Commits

Author SHA1 Message Date
prosolis
e26b69e43f Paywall detection: swap to Wayback snapshot when article body is gated
Fetch each new article once and measure visible <p> text. If body is below
500 chars (or fetch fails), resolve a Wayback snapshot via the
archive.org/wayback/available API and use that URL for both the og:image
fallback and the posted link. Dedup keys stay derived from the original
URL so paywalled/non-paywalled hits collide as before.

- New: internal/ingestion/article.go (FetchArticleMeta via goquery)
- New: internal/ingestion/wayback.go (ResolveWayback)
- Removed: internal/ingestion/og.go (folded into article.go)
- poller.go: dedup first, then one article fetch, then snapshot fallback
2026-05-22 17:53:54 -07:00
prosolis
310950e722 Bump indirect deps: x/crypto 0.52, x/net 0.55, x/sys 0.45 2026-05-22 17:27:00 -07:00
prosolis
652d6dfa38 Initial commit 2026-05-22 17:25:27 -07:00