Add feed/reader mode with full-article capture at ingest

Reader mode presents the stories on a page one at a time in a focused
overlay, marking each read as it's shown. Left/right arrows (or the header
book button / `f`) page through them; read stories dim on the grid. Read
state is device-local in localStorage.

Backing this required actually capturing article bodies, which Pete wasn't
doing — it kept only the RSS <description> lede and discarded content:encoded:

- stories.content column (idempotent migration; old rows fall back to lede)
- parser keeps content:encoded as paragraph-preserving text
- article fetch already done for paywall detection now also returns its body,
  so ingest stores the richer of feed-content vs scraped body with no extra
  request (prefers the archive snapshot body for paywalled stories)
- GET /api/article?id= serves the stored text; card queries now select id and
  expose it as data-id for the reader

Tests cover content extraction, the storage round-trip, and the article
endpoint + card rendering end to end.
This commit is contained in:
prosolis
2026-07-06 22:46:14 -07:00
parent 410f8dda0a
commit 55aa167151
17 changed files with 803 additions and 13 deletions

View File

@@ -65,6 +65,7 @@ const googlebotUA = "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.
// ArticleMeta is what we can learn from fetching an article page directly.
type ArticleMeta struct {
ImageURL string // og:image or twitter:image, absolute URL
BodyText string // extracted visible article body text (capped at MaxBodyChars)
BodyChars int // length of extracted visible body text
Fetched bool // true if we got an HTTP 200 with HTML
FetchError bool // true if the fetch failed at the network/HTTP layer
@@ -192,7 +193,10 @@ func fetchArticleMetaOnce(articleURL, ua, referer string) ArticleMeta {
meta.Fetched = true
meta.ImageURL = extractOGImage(doc, articleURL)
meta.BodyChars = extractBodyChars(doc)
// One body extraction serves both purposes: the text feeds reader mode and
// its length is the paywall/thin-body signal.
meta.BodyText = extractBodyText(doc)
meta.BodyChars = len(meta.BodyText)
meta.Paywalled = detectPaywall(doc)
meta.SubscriberOnly = detectSubscriberOnly(articleURL, doc)
return meta