drwily/Pete
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Fix SSRF, XSS, dedup, force-post, and DB hot-path issues

Browse Source 
- Add internal/safehttp: hardened HTTP client (DNS-resolved dial guard
  blocking loopback/RFC1918/CGNAT/link-local, redirect re-validation,
  body-size cap) and rewire article/feed/thumb clients through it
- Cap goquery body at 5 MiB so a hostile origin can't OOM the process
- search.js: reject non-http(s) hrefs to block stored XSS via javascript:
- dedup: tracking-param key "CMP" was unreachable (lookup lowercases);
  fixed to "cmp" so CMP= is actually stripped from canonical URLs
- ForcePost: postItem now returns bool; on dedup-skip ForcePost returns
  false so !post falls back to DB lookup instead of silently consuming
- Bound reaction callbacks behind an 8-slot semaphore; drop overflow
- Add stories indexes on (channel, classified, seen_at DESC),
  (classified, seen_at DESC), and partial image_url to kill full scans
  in IsKnownImageURL and ORDER BY seen_at hot paths
- Surface FTS5 probe Scan error instead of swallowing it
This commit is contained in:
prosolis
2026-05-25 12:23:37 -07:00
parent e428f37c9e
commit 9e5ba5aafc
12 changed files with 234 additions and 19 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
internal/storage/schema.go
View File

@@ -48,6 +48,9 @@ CREATE INDEX IF NOT EXISTS idx_post_log_event_id ON post_log(event_id);
CREATE INDEX IF NOT EXISTS idx_post_log_channel_posted ON post_log(channel, posted_at);
CREATE INDEX IF NOT EXISTS idx_post_log_canonical_channel ON post_log(url_canonical, channel, posted_at);
CREATE INDEX IF NOT EXISTS idx_stories_classified_source ON stories(classified, source);
CREATE INDEX IF NOT EXISTS idx_stories_channel_classified_seen ON stories(channel, classified, seen_at DESC);
CREATE INDEX IF NOT EXISTS idx_stories_classified_seen ON stories(classified, seen_at DESC);
CREATE INDEX IF NOT EXISTS idx_stories_image_url ON stories(image_url) WHERE image_url IS NOT NULL AND image_url <> '';
CREATE UNIQUE INDEX IF NOT EXISTS idx_stories_url_canonical ON stories(url_canonical) WHERE url_canonical IS NOT NULL AND url_canonical <> '';
CREATE UNIQUE INDEX IF NOT EXISTS idx_stories_source_headline_norm ON stories(source, headline_norm) WHERE headline_norm IS NOT NULL AND headline_norm <> '';
CREATE INDEX IF NOT EXISTS idx_reactions_post_guid ON reactions(post_guid);