adventure: tell Pete what the realm is, not just what happened in it

Adds the realm snapshot behind Pete's map, board and hall of firsts: every
zone with its clear history and who is standing in it, the ledger of things
that have happened exactly once, and one line per adventurer.

Snapshot semantics like the roster and the Siege — pushed whole, replaces
Pete's copy, dropped rather than retried. It rides the roster ticker but at a
ten-minute stride: these are aggregate scans over the whole run history, and a
first clear does not move.

Two things the real data corrected, neither of which any unit test would have
caught:

`abandoned` does not mean the player gave up. It means the run row was
retired, and abandonZoneRunByID exists precisely to retire a run whose boss is
already dead when the expedition travels on. 30 of prod's 32 boss kills carry
abandoned = 1, so filtering on `abandoned = 0` — as this did, copying the
shipped backfill — drew a realm in which almost nothing had ever been beaten.
boss_defeated = 1 is the clear.

And news_realm_firsts is not a usable history book on its own. It only covers
what happened after the news seam went live, its one-shot seeder carried that
same wrong filter, and its first_at is when the claim was written down, not
when the thing happened — every backfilled row in prod shares one timestamp.
So the zone half of the hall is derived from the run history instead, which is
complete and correctly dated, and the ledger supplies the kinds the run
history knows nothing about. That also makes the hall agree with the board by
construction rather than by coincidence.

Opt-out follows the rule that fits each surface: a first-clearer is anonymised
(deleting the claim would redraw a conquered zone as unbeaten), a player is
dropped from the board outright, and presence is dropped entirely — who is in
a dungeon right now is the live-location fact the liveblog already refuses.

Claude-Session: https://claude.ai/code/session_012bxpQQJDjC1mTtLN3VVtBQ

internal/peteclient/client.go

@@ -493,6 +493,105 @@ type RunBeat struct {
 	Prose string `json:"prose,omitempty"`
 }
 
+// RealmZone is one zone as the realm map draws it: what it is, who first got
+// through it, how many have since, and who is inside it right now.
+//
+// FirstClearBy is a character name and FirstClearToken the public board token,
+// exactly as the Siege muster pairs them — and the token is EMPTY for a player
+// who has opted out, keeping the name off the page too (see buildRealmSnapshot:
+// an opted-out first-clearer is anonymised, not deleted, because deleting the
+// claim would make the zone read as never-cleared, which is a different and
+// false statement about the realm).
+type RealmZone struct {
+	ID         string `json:"id"`
+	Display    string `json:"display"`
+	Tier       int    `json:"tier"`
+	LevelMin   int    `json:"level_min"`
+	LevelMax   int    `json:"level_max"`
+	Faction    string `json:"faction,omitempty"`
+	Atmosphere string `json:"atmosphere,omitempty"`
+	Postgame   bool   `json:"postgame,omitempty"` // T6 mythic: gated, drawn apart
+
+	FirstClearBy    string `json:"first_clear_by,omitempty"`
+	FirstClearToken string `json:"first_clear_token,omitempty"`
+	FirstClearAt    int64  `json:"first_clear_at,omitempty"`
+
+	Clears   int `json:"clears"`   // boss-defeated runs, all time
+	Clearers int `json:"clearers"` // distinct adventurers who have managed it
+
+	Occupants []RealmOccupant `json:"occupants,omitempty"` // in there right now
+}
+
+// RealmOccupant is somebody currently on an expedition in a zone. Same
+// name+token pair as everywhere else, and an opted-out player is omitted
+// outright rather than anonymised: unlike a first clear, presence is not part of
+// a shared tally that stops adding up without them, and "who is in there right
+// now" is exactly the live-location fact the liveblog is careful about.
+type RealmOccupant struct {
+	Token string `json:"token,omitempty"`
+	Name  string `json:"name"`
+	Level int    `json:"level,omitempty"`
+	Day   int    `json:"day,omitempty"`
+}
+
+// RealmFirst is one row of the hall of firsts: a thing that happened in the
+// realm exactly once ever, and who it happened to. The ledger
+// (news_realm_firsts) records only (kind, target, first_at) — the holder is
+// recovered by gogobee at push time from the run history, which is why this is
+// pushed rather than derived on Pete.
+type RealmFirst struct {
+	Kind    string `json:"kind"`              // "zone" | "treasure"
+	Target  string `json:"target"`            // the zone id or treasure key
+	Display string `json:"display"`           // the human name for it
+	Tier    int    `json:"tier,omitempty"`    // zone tier, when kind is "zone"
+	Holder  string `json:"holder,omitempty"`  // character name, empty when unrecoverable
+	Token   string `json:"token,omitempty"`   // board token; empty when opted out
+	AtUnix  int64  `json:"at_unix"`           // when the realm first saw it
+}
+
+// RealmStanding is one adventurer's line on the board. Every number here is a
+// lifetime total from the game's own run history — nothing is a rate, an
+// average, or anything that would move on its own while nobody played.
+type RealmStanding struct {
+	Token       string `json:"token,omitempty"`
+	Name        string `json:"name"`
+	Level       int    `json:"level"`
+	ClassRace   string `json:"class_race,omitempty"`
+	DeepestTier int    `json:"deepest_tier"` // deepest zone tier actually cleared
+	Clears      int    `json:"clears"`
+	Zones       int    `json:"zones"`  // distinct zones cleared
+	Firsts      int    `json:"firsts"` // realm-firsts held
+	SiegeDamage int    `json:"siege_damage"`
+	SiegeFights int    `json:"siege_fights"`
+}
+
+// RealmSnapshot is the whole realm as one photograph: every zone, the hall of
+// firsts, and the board. Snapshot semantics, like the roster and the Siege —
+// Pete replaces its copy and a failed push is dropped, not retried.
+//
+// It is pushed on the roster ticker but NOT every tick: none of it moves fast
+// enough to be worth the aggregate queries every two minutes, and the page's
+// staleness window is generous for exactly that reason. See realmPushInterval.
+type RealmSnapshot struct {
+	SnapshotAt int64           `json:"snapshot_at"`
+	Zones      []RealmZone     `json:"zones,omitempty"`
+	Firsts     []RealmFirst    `json:"firsts,omitempty"`
+	Standings  []RealmStanding `json:"standings,omitempty"`
+}
+
+// PushRealm sends the realm pages' backing data to Pete. Drop-on-failure, same
+// as the other two snapshots.
+func PushRealm(ctx context.Context, snap RealmSnapshot) error {
+	if !Enabled() {
+		return nil
+	}
+	payload, err := json.Marshal(snap)
+	if err != nil {
+		return err
+	}
+	return std.post(ctx, "/api/ingest/realm", payload)
+}
+
 // PushRunBeats delivers a batch of beats. Unlike the snapshots this is
 // append-only and IS retried — a dropped beat is a hole in a story, not a stale
 // number that the next tick corrects. The caller only marks rows sent on success.

internal/plugin/pete_realm.go

@@ -0,0 +1,633 @@
+package plugin
+
+import (
+	"context"
+	"database/sql"
+	"log/slog"
+	"sort"
+	"time"
+
+	"gogobee/internal/db"
+	"gogobee/internal/peteclient"
+
+	"maunium.net/go/mautrix/id"
+)
+
+// The realm snapshot: the world map, the hall of firsts, and the board.
+//
+// Everything Pete has shown so far is either the present moment (the roster, the
+// Siege bar) or one thing that happened (a dispatch, a run log). None of it says
+// what the realm *is* — that there are thirty-odd named places with a difficulty
+// order, that some of them have never been beaten by anybody, and that the
+// people playing have a history against them. That is what this carries.
+//
+// Snapshot semantics, like the roster and the Siege: pushed whole, replaces
+// Pete's copy, dropped rather than retried on failure. The difference is the
+// clock. The roster is a photograph of where people are standing and is worth
+// re-taking every two minutes; a realm-first is a thing that happened once, ever,
+// and re-deriving the whole ledger plus three aggregate scans at that rate would
+// be pure waste. So this rides the same ticker at a much longer stride.
+const (
+	// realmPushInterval — how often the realm is recomputed and pushed. The
+	// fastest-moving field in the whole snapshot is a zone's occupant list, and a
+	// ten-minute-old answer to "who is in the Sunken Vault" is still a true and
+	// useful one. Everything else moves on the scale of days.
+	realmPushInterval = 10 * time.Minute
+
+	// realmMaxOccupants bounds the per-zone occupant list. A realm has tens of
+	// players; this only stops a pathological case spooling a huge payload.
+	realmMaxOccupants = 50
+)
+
+// realmLastPush is when the realm snapshot last went out. Zero means never, so
+// the first tick after start-up always pushes — an operator restarting the bot
+// should not have to wait ten minutes to see whether the wire works.
+var realmLastPush time.Time
+
+// realmPushOK mirrors rosterPushOK: log the transitions and nothing else.
+var realmPushOK bool
+
+// pushRealm recomputes and sends the realm snapshot, at most once per
+// realmPushInterval however often the ticker calls it.
+func (p *AdventurePlugin) pushRealm() {
+	now := time.Now().UTC()
+	if !realmLastPush.IsZero() && now.Sub(realmLastPush) < realmPushInterval {
+		return
+	}
+
+	snap, err := buildRealmSnapshot(now)
+	if err != nil {
+		slog.Error("realm: build snapshot failed", "err", err)
+		return
+	}
+
+	ctx, cancel := context.WithTimeout(context.Background(), rosterPushTimeout)
+	defer cancel()
+
+	if err := peteclient.PushRealm(ctx, snap); err != nil {
+		if realmPushOK {
+			slog.Warn("realm: push failed, realm pages will go stale on Pete", "err", err)
+		} else {
+			slog.Debug("realm: push failed, dropping snapshot", "err", err)
+		}
+		realmPushOK = false
+		// Deliberately NOT stamping realmLastPush: a failed push should be retried
+		// on the next roster tick, not ten minutes from now. The stamp is a
+		// "we already told Pete this" marker, and we didn't.
+		return
+	}
+
+	realmLastPush = now
+	if !realmPushOK {
+		slog.Info("realm: snapshot accepted by Pete — realm pages are publishing",
+			"zones", len(snap.Zones), "firsts", len(snap.Firsts), "standings", len(snap.Standings))
+		realmPushOK = true
+	}
+}
+
+// realmClearStats is the per-zone clear history, read in one pass.
+type realmClearStats struct {
+	clears       int
+	clearers     int
+	firstUser    id.UserID
+	firstClearAt int64
+}
+
+// buildRealmSnapshot assembles the whole realm from the game's own tables.
+//
+// The three aggregate reads are done up front and once each, keyed into maps,
+// rather than per-zone or per-player: this runs against the live DB on a ticker
+// and a query-per-zone loop over a registry that only grows is the kind of thing
+// that is fine until it isn't.
+func buildRealmSnapshot(now time.Time) (peteclient.RealmSnapshot, error) {
+	snap := peteclient.RealmSnapshot{SnapshotAt: now.Unix()}
+
+	clearsByZone, err := loadRealmClearStats()
+	if err != nil {
+		return snap, err
+	}
+	occupantsByZone := loadRealmOccupants()
+
+	// ── Zones ───────────────────────────────────────────────────────────────
+	// zoneOrder is the design-doc ordering and is what the page draws in, so the
+	// realm reads the way it was designed rather than the way a map iterates.
+	for _, zid := range zoneOrder {
+		def, ok := dndZoneRegistry[zid]
+		if !ok {
+			continue
+		}
+		z := peteclient.RealmZone{
+			ID:         string(def.ID),
+			Display:    def.Display,
+			Tier:       int(def.Tier),
+			LevelMin:   def.LevelMin,
+			LevelMax:   def.LevelMax,
+			Faction:    def.Faction,
+			Atmosphere: def.Atmosphere,
+			Postgame:   def.Tier == ZoneTierMythic,
+			Occupants:  occupantsByZone[string(def.ID)],
+		}
+		if st, ok := clearsByZone[string(def.ID)]; ok {
+			z.Clears = st.clears
+			z.Clearers = st.clearers
+			z.FirstClearAt = st.firstClearAt
+			// An opted-out first-clearer keeps the claim and loses the identity —
+			// the Siege contributor rule, and for the same reason. Deleting the
+			// claim outright would leave the zone drawn as never-cleared, which is
+			// a false statement about the realm rather than a withheld one.
+			if !isNewsOptedOut(st.firstUser) {
+				z.FirstClearBy = charName(st.firstUser)
+				if z.FirstClearBy != "" {
+					z.FirstClearToken = eventToken(st.firstUser, "roster")
+				}
+			}
+		}
+		snap.Zones = append(snap.Zones, z)
+	}
+
+	snap.Firsts = loadRealmFirsts(clearsByZone)
+
+	// The three pages must not be able to contradict each other about the same
+	// zone. loadRealmFirsts derives the zone half of the hall from clearsByZone —
+	// the same map the tiles and the board use — but it also carries any ledger
+	// claim with no surviving run behind it, and that case would otherwise draw a
+	// place as never-beaten on the map while the hall named the year it fell.
+	//
+	// So an unbacked claim floors the clear count at one. Deliberately a floor and
+	// not an assignment: where the run history is intact it is the better answer
+	// and this only fills a hole. Nothing is attributed — the zone reads "cleared,
+	// by somebody", which is exactly what is known about it.
+	for i := range snap.Zones {
+		if snap.Zones[i].Clears > 0 {
+			continue
+		}
+		for _, f := range snap.Firsts {
+			if f.Kind == "zone" && f.Target == snap.Zones[i].ID {
+				snap.Zones[i].Clears = 1
+				snap.Zones[i].Clearers = 1
+				break
+			}
+		}
+	}
+
+	snap.Standings, err = loadRealmStandings(clearsByZone)
+	if err != nil {
+		return snap, err
+	}
+	return snap, nil
+}
+
+// loadRealmClearStats reads every zone's clear history in one pass: how many
+// successful runs, how many distinct people managed it, and who did it first.
+//
+// MIN(completed_at) with a bare user_id column is SQLite's bare-column min/max
+// rule — the user_id comes from the same row the minimum came from, so the
+// (zone, first clearer, when) triple is internally consistent. backfillZoneFirsts
+// relies on exactly this and has done since the news seam shipped.
+//
+// completed_at is selected raw as a string and parsed in Go rather than being
+// wrapped in anything: modernc.org/sqlite rebuilds a time.Time from the column's
+// declared type, and passing a DATETIME through MIN() alongside an aggregate is
+// close enough to the COALESCE() trap that it is not worth finding out. The
+// string always parses — it is written by SQLite's own CURRENT_TIMESTAMP.
+//
+// NOTE the absence of `AND abandoned = 0`, which looks like it belongs here and
+// does not. `abandoned` does not mean "the player gave up" — it means the run
+// ROW was retired, and abandonZoneRunByID exists specifically to retire a run
+// whose boss is already dead when the expedition travels onward (see its comment
+// in dnd_zone_run.go). In prod, 30 of the realm's 32 boss kills carry
+// abandoned = 1. Filtering them out drew a map on which almost nothing had ever
+// been beaten, while the hall of firsts — reading a different table — said six
+// zones had been. boss_defeated = 1 is the clear, full stop.
+func loadRealmClearStats() (map[string]realmClearStats, error) {
+	rows, err := db.Get().Query(`
+		SELECT zone_id,
+		       COUNT(*)                AS clears,
+		       COUNT(DISTINCT user_id) AS clearers,
+		       user_id,
+		       MIN(completed_at)
+		  FROM dnd_zone_run
+		 WHERE boss_defeated = 1 AND completed_at IS NOT NULL
+		 GROUP BY zone_id`)
+	if err != nil {
+		return nil, err
+	}
+	defer rows.Close()
+
+	out := map[string]realmClearStats{}
+	for rows.Next() {
+		var zoneID, userID, completedAt string
+		var st realmClearStats
+		if err := rows.Scan(&zoneID, &st.clears, &st.clearers, &userID, &completedAt); err != nil {
+			return nil, err
+		}
+		st.firstUser = id.UserID(userID)
+		if ts, ok := parseSQLiteTime(completedAt); ok {
+			st.firstClearAt = ts.Unix()
+		}
+		out[zoneID] = st
+	}
+	return out, rows.Err()
+}
+
+// loadRealmOccupants answers "who is in there right now", per zone.
+//
+// Presence is dropped for an opted-out player rather than anonymised. Unlike a
+// first clear it is not part of a tally that stops adding up without them, and
+// it is the same live-location fact the run liveblog refuses to publish — an
+// anonymous "somebody is in the Drowned Star" next to a roster showing exactly
+// one person out on expedition is not an anonymisation.
+//
+// Errors are swallowed to nil: an unreadable expedition table should cost the
+// realm map its occupant dots, not the whole page.
+func loadRealmOccupants() map[string][]peteclient.RealmOccupant {
+	rows, err := db.Get().Query(
+		`SELECT user_id, zone_id, current_day FROM dnd_expedition WHERE status = 'active'`)
+	if err != nil {
+		slog.Error("realm: occupants query", "err", err)
+		return nil
+	}
+	defer rows.Close()
+
+	type live struct {
+		uid    id.UserID
+		zoneID string
+		day    int
+	}
+	var found []live
+	for rows.Next() {
+		var uid, zoneID string
+		var day int
+		if err := rows.Scan(&uid, &zoneID, &day); err != nil {
+			slog.Error("realm: occupants scan", "err", err)
+			return nil
+		}
+		found = append(found, live{id.UserID(uid), zoneID, day})
+	}
+	if err := rows.Err(); err != nil {
+		slog.Error("realm: occupants rows", "err", err)
+		return nil
+	}
+
+	// Names and opt-out are resolved only after the cursor is drained. The pool
+	// is one connection wide and charName reads the DB; resolving inside the loop
+	// is the deadlock W2a shipped and then had to fix.
+	out := map[string][]peteclient.RealmOccupant{}
+	for _, l := range found {
+		if isNewsOptedOut(l.uid) {
+			continue
+		}
+		name := charName(l.uid)
+		if name == "" {
+			continue // never fall back to a Matrix handle on a public page
+		}
+		if len(out[l.zoneID]) >= realmMaxOccupants {
+			continue
+		}
+		out[l.zoneID] = append(out[l.zoneID], peteclient.RealmOccupant{
+			Token: eventToken(l.uid, "roster"),
+			Name:  name,
+			Level: charLevel(l.uid),
+			Day:   l.day,
+		})
+	}
+	for zid := range out {
+		sort.Slice(out[zid], func(i, j int) bool { return out[zid][i].Name < out[zid][j].Name })
+	}
+	return out
+}
+
+// loadRealmFirsts renders news_realm_firsts as a history book.
+//
+// The ledger stores only (kind, target, first_at) — it exists to tier a dispatch,
+// not to remember who. The holder is recovered here from the game's own history:
+// a zone first from the earliest boss-defeating run, a treasure first from the
+// earliest surviving row in adventure_treasures. Both can come back empty — a
+// treasure that was found and later discarded leaves no owner anywhere — and an
+// unattributed first is rendered as one rather than dropped. It still happened.
+// loadRealmFirsts renders the hall of firsts, and it takes the clear stats
+// rather than reading the ledger alone, for two reasons that only showed up
+// against real prod data:
+//
+//  1. news_realm_firsts is INCOMPLETE for zones. It has only been written since
+//     the news seam went live, and the one-shot that seeded it filtered on
+//     `abandoned = 0` — the same wrong filter loadRealmClearStats documents — so
+//     it missed every zone whose clears were all retired runs. In prod it holds 6
+//     zones where the run history knows 9.
+//  2. Its first_at is when the CLAIM was recorded, not when the thing happened.
+//     Every backfilled row in prod carries the same timestamp: the minute the
+//     backfill ran. A history book dated by when somebody wrote it down is not
+//     much of a history book.
+//
+// So the zone half is derived from the run history, which is complete and
+// correctly dated, and the ledger supplies the kinds the run history knows
+// nothing about (treasures, and whatever ships next) plus any zone claim with no
+// surviving run behind it. That also makes the hall agree with the board by
+// construction: both count a zone-first as "you were the first to clear it".
+func loadRealmFirsts(clearsByZone map[string]realmClearStats) []peteclient.RealmFirst {
+	rows, err := db.Get().Query(
+		`SELECT kind, target, first_at FROM news_realm_firsts ORDER BY first_at ASC`)
+	if err != nil {
+		slog.Error("realm: firsts query", "err", err)
+		return nil
+	}
+	defer rows.Close()
+
+	var out []peteclient.RealmFirst
+	for rows.Next() {
+		var f peteclient.RealmFirst
+		if err := rows.Scan(&f.Kind, &f.Target, &f.AtUnix); err != nil {
+			slog.Error("realm: firsts scan", "err", err)
+			return nil
+		}
+		out = append(out, f)
+	}
+	if err := rows.Err(); err != nil {
+		slog.Error("realm: firsts rows", "err", err)
+		return nil
+	}
+
+	// Same discipline as the occupants: the cursor is closed before anything
+	// else touches the database.
+	rows.Close()
+
+	// Drop the ledger's zone rows wherever the run history has the same zone —
+	// it is the better record of both who and when. A claim with no run behind it
+	// survives, unattributed, and is what floors that zone's clear count in
+	// buildRealmSnapshot.
+	kept := out[:0]
+	for _, f := range out {
+		if f.Kind == "zone" {
+			if _, ok := clearsByZone[f.Target]; ok {
+				continue
+			}
+		}
+		kept = append(kept, f)
+	}
+	out = kept
+
+	// The zone half, from the authority the map and the board also use.
+	for zoneID, st := range clearsByZone {
+		zone := zoneOrFallback(ZoneID(zoneID))
+		f := peteclient.RealmFirst{
+			Kind:    "zone",
+			Target:  zoneID,
+			Display: zone.Display,
+			Tier:    int(zone.Tier),
+			AtUnix:  st.firstClearAt,
+		}
+		if !isNewsOptedOut(st.firstUser) {
+			if name := charName(st.firstUser); name != "" {
+				f.Holder = name
+				f.Token = eventToken(st.firstUser, "roster")
+			}
+		}
+		out = append(out, f)
+	}
+
+	for i := range out {
+		switch out[i].Kind {
+		case "zone":
+			if out[i].Display == "" {
+				zone := zoneOrFallback(ZoneID(out[i].Target))
+				out[i].Display = zone.Display
+				out[i].Tier = int(zone.Tier)
+				out[i].Holder, out[i].Token = realmFirstZoneHolder(out[i].Target)
+			}
+		case "treasure":
+			if def := lookupAdvTreasureDef(out[i].Target); def != nil {
+				out[i].Display = def.Name
+				out[i].Tier = def.Tier
+			} else {
+				out[i].Display = out[i].Target
+			}
+			out[i].Holder, out[i].Token = realmFirstTreasureHolder(out[i].Target)
+		default:
+			// A kind nobody has taught this function about still belongs in the
+			// hall — it is a genuine realm-first and the ledger says so. It just
+			// arrives with the raw target as its name, which is the same
+			// degrade-don't-drop rule the unknown event_type inversion settled on.
+			out[i].Display = out[i].Target
+		}
+	}
+
+	// Oldest first: the order it happened in. Pete regroups it newest-year-first
+	// for the page, but the wire carries history in history's order.
+	sort.Slice(out, func(i, j int) bool {
+		if out[i].AtUnix != out[j].AtUnix {
+			return out[i].AtUnix < out[j].AtUnix
+		}
+		return out[i].Target < out[j].Target
+	})
+	return out
+}
+
+// realmFirstZoneHolder names the earliest clearer of a zone. Returns ("", "")
+// when the run history no longer has one, and ("Name", "") when it does but the
+// player has opted out — the claim survives the anonymisation, the link does not.
+func realmFirstZoneHolder(zoneID string) (name, token string) {
+	var userID string
+	err := db.Get().QueryRow(`
+		SELECT user_id
+		  FROM dnd_zone_run
+		 WHERE zone_id = ? AND boss_defeated = 1 AND completed_at IS NOT NULL
+		 ORDER BY completed_at ASC
+		 LIMIT 1`, zoneID).Scan(&userID)
+	if err != nil {
+		if err != sql.ErrNoRows {
+			slog.Error("realm: zone-first holder", "zone", zoneID, "err", err)
+		}
+		return "", ""
+	}
+	uid := id.UserID(userID)
+	if isNewsOptedOut(uid) {
+		return "", ""
+	}
+	name = charName(uid)
+	if name == "" {
+		return "", ""
+	}
+	return name, eventToken(uid, "roster")
+}
+
+// realmFirstTreasureHolder names the earliest holder of a treasure key. A
+// treasure writes one row per bonus, so the MIN is over what may be several rows
+// for the same acquisition; that is fine, they share a timestamp.
+func realmFirstTreasureHolder(key string) (name, token string) {
+	var userID string
+	err := db.Get().QueryRow(`
+		SELECT user_id
+		  FROM adventure_treasures
+		 WHERE treasure_key = ?
+		 ORDER BY acquired_at ASC
+		 LIMIT 1`, key).Scan(&userID)
+	if err != nil {
+		if err != sql.ErrNoRows {
+			slog.Error("realm: treasure-first holder", "key", key, "err", err)
+		}
+		return "", ""
+	}
+	uid := id.UserID(userID)
+	if isNewsOptedOut(uid) {
+		return "", ""
+	}
+	name = charName(uid)
+	if name == "" {
+		return "", ""
+	}
+	return name, eventToken(uid, "roster")
+}
+
+// loadRealmStandings builds the board: one line per living, named, opted-in
+// adventurer, every number a lifetime total.
+//
+// It walks player_meta the way buildRosterSnapshot does, and for the same
+// reason — that is the list of people who exist, and a standings table assembled
+// by grouping the run history instead would silently include characters that have
+// since been deleted or never finished setup.
+func loadRealmStandings(clearsByZone map[string]realmClearStats) ([]peteclient.RealmStanding, error) {
+	rows, err := db.Get().Query(`SELECT user_id FROM player_meta WHERE alive = 1`)
+	if err != nil {
+		return nil, err
+	}
+	defer rows.Close()
+
+	var uids []id.UserID
+	for rows.Next() {
+		var uid string
+		if err := rows.Scan(&uid); err != nil {
+			return nil, err
+		}
+		uids = append(uids, id.UserID(uid))
+	}
+	if err := rows.Err(); err != nil {
+		return nil, err
+	}
+	rows.Close()
+
+	// Who holds how many realm-firsts, from the same authority the zone column
+	// uses — so a zone's "first cleared by X" and X's firsts count can never
+	// disagree with each other.
+	firstsBy := map[id.UserID]int{}
+	for _, st := range clearsByZone {
+		firstsBy[st.firstUser]++
+	}
+
+	perZone, err := loadRealmPlayerClears()
+	if err != nil {
+		return nil, err
+	}
+	siege := loadRealmSiegeTotals()
+
+	var out []peteclient.RealmStanding
+	for _, uid := range uids {
+		if isNewsOptedOut(uid) {
+			continue // the board omits an opted-out player outright, as it always has
+		}
+		c, err := LoadDnDCharacter(uid)
+		if err != nil || c == nil || c.PendingSetup {
+			continue
+		}
+		name := charName(uid)
+		if name == "" {
+			continue
+		}
+		s := peteclient.RealmStanding{
+			Token:       eventToken(uid, "roster"),
+			Name:        name,
+			Level:       c.Level,
+			ClassRace:   classRaceLabel(c),
+			Firsts:      firstsBy[uid],
+			SiegeDamage: siege[uid].damage,
+			SiegeFights: siege[uid].fights,
+		}
+		for zoneID, n := range perZone[uid] {
+			s.Clears += n
+			s.Zones++
+			if t := int(zoneOrFallback(ZoneID(zoneID)).Tier); t > s.DeepestTier {
+				s.DeepestTier = t
+			}
+		}
+		out = append(out, s)
+	}
+
+	// Ranked here, not on Pete: the ordering is a statement about the game
+	// ("deepest tier beaten, then how much of the realm you have beaten"), and
+	// the game is the thing that gets to make it. Name breaks the tie so the
+	// board is stable between snapshots that are otherwise identical.
+	sort.Slice(out, func(i, j int) bool {
+		a, b := out[i], out[j]
+		switch {
+		case a.DeepestTier != b.DeepestTier:
+			return a.DeepestTier > b.DeepestTier
+		case a.Zones != b.Zones:
+			return a.Zones > b.Zones
+		case a.Clears != b.Clears:
+			return a.Clears > b.Clears
+		case a.Level != b.Level:
+			return a.Level > b.Level
+		default:
+			return a.Name < b.Name
+		}
+	})
+	return out, nil
+}
+
+// loadRealmPlayerClears returns clears[user][zone] = count, in one pass.
+func loadRealmPlayerClears() (map[id.UserID]map[string]int, error) {
+	rows, err := db.Get().Query(`
+		SELECT user_id, zone_id, COUNT(*)
+		  FROM dnd_zone_run
+		 WHERE boss_defeated = 1 AND completed_at IS NOT NULL
+		 GROUP BY user_id, zone_id`)
+	if err != nil {
+		return nil, err
+	}
+	defer rows.Close()
+
+	out := map[id.UserID]map[string]int{}
+	for rows.Next() {
+		var uid, zoneID string
+		var n int
+		if err := rows.Scan(&uid, &zoneID, &n); err != nil {
+			return nil, err
+		}
+		u := id.UserID(uid)
+		if out[u] == nil {
+			out[u] = map[string]int{}
+		}
+		out[u][zoneID] = n
+	}
+	return out, rows.Err()
+}
+
+type realmSiegeTotal struct{ damage, fights int }
+
+// loadRealmSiegeTotals sums every Siege a player has ever turned up to. Across
+// all bosses, not just the live one — the war room already shows the current
+// muster, and what the board is for is the person who has shown up to all six.
+func loadRealmSiegeTotals() map[id.UserID]realmSiegeTotal {
+	rows, err := db.Get().Query(
+		`SELECT user_id, SUM(damage), SUM(fights) FROM world_boss_contrib GROUP BY user_id`)
+	if err != nil {
+		slog.Error("realm: siege totals query", "err", err)
+		return nil
+	}
+	defer rows.Close()
+
+	out := map[id.UserID]realmSiegeTotal{}
+	for rows.Next() {
+		var uid string
+		var damage, fights int
+		if err := rows.Scan(&uid, &damage, &fights); err != nil {
+			slog.Error("realm: siege totals scan", "err", err)
+			return nil
+		}
+		out[id.UserID(uid)] = realmSiegeTotal{damage, fights}
+	}
+	if err := rows.Err(); err != nil {
+		slog.Error("realm: siege totals rows", "err", err)
+		return nil
+	}
+	return out
+}

internal/plugin/pete_realm_test.go

@@ -0,0 +1,351 @@
+package plugin
+
+import (
+	"testing"
+	"time"
+
+	"gogobee/internal/db"
+
+	"maunium.net/go/mautrix/id"
+)
+
+// seedRealmFixture builds a small but realistic realm: two players, three
+// cleared runs across two zones, one live expedition, and the realm-first ledger
+// that the zone clears would have seeded.
+func seedRealmFixture(t *testing.T) {
+	t.Helper()
+	dir := t.TempDir()
+	db.Close()
+	if err := db.Init(dir); err != nil {
+		t.Fatalf("db.Init: %v", err)
+	}
+	t.Cleanup(db.Close)
+
+	db.Exec("seed josie", `INSERT INTO player_meta (user_id, display_name, alive) VALUES (?, ?, 1)`,
+		"@josie:x", "Josie")
+	db.Exec("seed quack", `INSERT INTO player_meta (user_id, display_name, alive) VALUES (?, ?, 1)`,
+		"@quack:x", "Quack")
+
+	// The board walks player_meta and then loads a character, so both halves have
+	// to exist: a player_meta row with no character is somebody who never finished
+	// setup, and standings correctly leaves them off.
+	for _, c := range []*DnDCharacter{
+		{UserID: "@josie:x", Race: RaceHuman, Class: ClassFighter, Level: 12,
+			STR: 18, DEX: 14, CON: 16, INT: 10, WIS: 10, CHA: 10,
+			HPMax: 120, HPCurrent: 120, ArmorClass: 18},
+		{UserID: "@quack:x", Race: RaceElf, Class: ClassMage, Level: 8,
+			STR: 8, DEX: 16, CON: 12, INT: 18, WIS: 12, CHA: 10,
+			HPMax: 48, HPCurrent: 48, ArmorClass: 12},
+	} {
+		if err := SaveDnDCharacter(c); err != nil {
+			t.Fatalf("SaveDnDCharacter(%s): %v", c.UserID, err)
+		}
+	}
+
+	// Josie cleared the Warrens first, then again; Quack cleared them later. Only
+	// Josie has been through the Crypt — and it is a deeper tier, which is what
+	// makes the board's depth-before-breadth ordering testable.
+	for _, r := range []struct {
+		runID, user, zone, at string
+	}{
+		{"r1", "@josie:x", string(ZoneGoblinWarrens), "2026-01-10 12:00:00"},
+		{"r2", "@quack:x", string(ZoneGoblinWarrens), "2026-03-02 12:00:00"},
+		{"r3", "@josie:x", string(ZoneGoblinWarrens), "2026-04-05 12:00:00"},
+		{"r4", "@josie:x", string(ZoneCryptValdris), "2026-05-01 12:00:00"},
+	} {
+		db.Exec("seed run", `INSERT INTO dnd_zone_run
+			(run_id, user_id, zone_id, total_rooms, boss_defeated, abandoned, completed_at)
+			VALUES (?, ?, ?, 6, 1, 0, ?)`, r.runID, r.user, r.zone, r.at)
+	}
+	// A retired-but-won run: boss_defeated = 1 with abandoned = 1. This IS a
+	// clear — `abandoned` means the run row was retired (the expedition travelled
+	// on after the kill), not that anybody gave up, and in prod it is how 30 of
+	// the realm's 32 boss kills are stored. The fixture carries one so the
+	// aggregate can never quietly go back to filtering them out.
+	db.Exec("seed retired-win", `INSERT INTO dnd_zone_run
+		(run_id, user_id, zone_id, total_rooms, boss_defeated, abandoned, completed_at)
+		VALUES ('r5', '@quack:x', 'crypt_valdris', 6, 1, 1, '2026-05-02 12:00:00')`)
+	// A genuinely unfinished run: no boss, no completion. Not a clear.
+	db.Exec("seed inflight", `INSERT INTO dnd_zone_run
+		(run_id, user_id, zone_id, total_rooms, boss_defeated, abandoned, completed_at)
+		VALUES ('r6', '@quack:x', 'crypt_valdris', 6, 0, 0, NULL)`)
+
+	claimRealmFirst("zone", string(ZoneGoblinWarrens))
+	claimRealmFirst("zone", string(ZoneCryptValdris))
+}
+
+// TestRealmClearStatsPickTheEarliestClearer is the load-bearing query on the
+// whole map: "who first got through here" is the single most interesting fact a
+// zone has, and it has to be the person who was actually first.
+//
+// It leans on SQLite's bare-column min/max rule — the user_id comes from the same
+// row MIN(completed_at) came from — which is the same thing backfillZoneFirsts
+// has relied on since the news seam shipped. If that ever stopped holding, this
+// would attribute somebody else's conquest to whoever the grouping happened to
+// land on, silently.
+func TestRealmClearStatsPickTheEarliestClearer(t *testing.T) {
+	seedRealmFixture(t)
+
+	stats, err := loadRealmClearStats()
+	if err != nil {
+		t.Fatalf("loadRealmClearStats: %v", err)
+	}
+
+	warrens, ok := stats["goblin_warrens"]
+	if !ok {
+		t.Fatal("no stats for goblin_warrens")
+	}
+	if warrens.clears != 3 {
+		t.Errorf("warrens clears = %d, want 3", warrens.clears)
+	}
+	if warrens.clearers != 2 {
+		t.Errorf("warrens clearers = %d, want 2", warrens.clearers)
+	}
+	if warrens.firstUser != id.UserID("@josie:x") {
+		t.Errorf("warrens first clearer = %q, want @josie:x — the earliest run didn't win", warrens.firstUser)
+	}
+
+	// Two clears: Josie's, and Quack's retired-but-won run. The in-flight run is
+	// correctly excluded. A regression to `AND abandoned = 0` shows up here as 1.
+	crypt := stats["crypt_valdris"]
+	if crypt.clears != 2 {
+		t.Errorf("crypt clears = %d, want 2 — a won-then-retired run is a clear, "+
+			"and an in-flight one is not", crypt.clears)
+	}
+	if crypt.firstUser != id.UserID("@josie:x") {
+		t.Errorf("crypt first clearer = %q, want @josie:x", crypt.firstUser)
+	}
+}
+
+// TestOptedOutFirstClearerIsAnonymisedNotErased. The Siege contributor rule,
+// applied to the map: deleting an opted-out clearer's claim would leave the zone
+// drawn as never-cleared, and "nobody has ever come out of there" is the most
+// dramatic thing the page can say. Saying it falsely because somebody chose
+// privacy would be worse than saying nothing.
+func TestOptedOutFirstClearerIsAnonymisedNotErased(t *testing.T) {
+	seedRealmFixture(t)
+	setNewsOptout(id.UserID("@josie:x"), true)
+
+	snap, err := buildRealmSnapshot(time.Now().UTC())
+	if err != nil {
+		t.Fatalf("buildRealmSnapshot: %v", err)
+	}
+
+	var warrens *struct {
+		clears    int
+		by, token string
+	}
+	for _, z := range snap.Zones {
+		if z.ID == "goblin_warrens" {
+			warrens = &struct {
+				clears    int
+				by, token string
+			}{z.Clears, z.FirstClearBy, z.FirstClearToken}
+		}
+	}
+	if warrens == nil {
+		t.Fatal("goblin_warrens is not in the snapshot at all")
+	}
+	if warrens.clears != 3 {
+		t.Errorf("clears = %d, want 3 — an opt-out deleted the town's history", warrens.clears)
+	}
+	if warrens.by != "" || warrens.token != "" {
+		t.Errorf("opted-out clearer still named: by=%q token=%q", warrens.by, warrens.token)
+	}
+}
+
+// TestOptedOutPlayerLeavesTheBoardEntirely. Standings follow the board's rule,
+// not the Siege's: an opted-out player is omitted outright. Their level, class
+// and clear count would re-identify them, and unlike a siege contribution there
+// is no shared total that stops adding up without them.
+func TestOptedOutPlayerLeavesTheBoardEntirely(t *testing.T) {
+	seedRealmFixture(t)
+	setNewsOptout(id.UserID("@quack:x"), true)
+
+	snap, err := buildRealmSnapshot(time.Now().UTC())
+	if err != nil {
+		t.Fatalf("buildRealmSnapshot: %v", err)
+	}
+	for _, s := range snap.Standings {
+		if s.Name == "Quack" {
+			t.Fatal("an opted-out player is still on the standings board")
+		}
+	}
+}
+
+// TestOccupantsDropOptedOutPlayers. Presence is the strictest case on the page
+// and deliberately stricter than a first clear: "who is in the Crypt of Valdris
+// right now" is the live-location fact the run liveblog refuses to publish at
+// all, so an opted-out player is dropped rather than anonymised.
+func TestOccupantsDropOptedOutPlayers(t *testing.T) {
+	seedRealmFixture(t)
+	db.Exec("seed expedition", `INSERT INTO dnd_expedition
+		(expedition_id, user_id, zone_id, status, current_day)
+		VALUES ('e1', '@josie:x', 'crypt_valdris', 'active', 3)`)
+
+	if occ := loadRealmOccupants(); len(occ["crypt_valdris"]) != 1 {
+		t.Fatalf("opted-in occupant missing: %+v", occ)
+	} else if occ["crypt_valdris"][0].Name != "Josie" || occ["crypt_valdris"][0].Day != 3 {
+		t.Errorf("occupant = %+v, want Josie on day 3", occ["crypt_valdris"][0])
+	}
+
+	setNewsOptout(id.UserID("@josie:x"), true)
+	if occ := loadRealmOccupants(); len(occ["crypt_valdris"]) != 0 {
+		t.Errorf("opted-out player still shows as standing in a zone: %+v", occ["crypt_valdris"])
+	}
+}
+
+// TestStandingsCountFirstsFromTheSameAuthorityTheMapDoes. A zone's "first
+// through: Josie" and Josie's own firsts count come off one map in one pass, so
+// the two can never disagree — which they would if the board recounted the
+// ledger itself and the two queries drifted.
+func TestStandingsCountFirstsFromTheSameAuthorityTheMapDoes(t *testing.T) {
+	seedRealmFixture(t)
+
+	snap, err := buildRealmSnapshot(time.Now().UTC())
+	if err != nil {
+		t.Fatalf("buildRealmSnapshot: %v", err)
+	}
+
+	firstsByName := map[string]int{}
+	for _, s := range snap.Standings {
+		firstsByName[s.Name] = s.Firsts
+	}
+	// Josie was first through both zones; Quack was first through neither.
+	if firstsByName["Josie"] != 2 {
+		t.Errorf("Josie holds %d firsts, want 2", firstsByName["Josie"])
+	}
+	if firstsByName["Quack"] != 0 {
+		t.Errorf("Quack holds %d firsts, want 0", firstsByName["Quack"])
+	}
+
+	named := 0
+	for _, z := range snap.Zones {
+		if z.FirstClearBy == "Josie" {
+			named++
+		}
+	}
+	if named != firstsByName["Josie"] {
+		t.Errorf("the map names Josie on %d zones but the board credits her with %d — "+
+			"the two disagree about the same fact", named, firstsByName["Josie"])
+	}
+}
+
+// TestStandingsRankDeepestFirst. The ordering is the game's statement about what
+// it values, and Pete renders it without renumbering — so it has to be right
+// here. Depth beats breadth: somebody who has put down a Tier 5 boss is ahead of
+// somebody who has cleared the whole of Tier 1 forty times.
+func TestStandingsRankDeepestFirst(t *testing.T) {
+	seedRealmFixture(t)
+
+	rows, err := loadRealmStandings(map[string]realmClearStats{})
+	if err != nil {
+		t.Fatalf("loadRealmStandings: %v", err)
+	}
+	if len(rows) < 2 {
+		t.Fatalf("got %d standings rows, want 2", len(rows))
+	}
+	for i := 1; i < len(rows); i++ {
+		a, b := rows[i-1], rows[i]
+		if a.DeepestTier < b.DeepestTier {
+			t.Errorf("row %d (T%d) sorts above row %d (T%d) — the board is not deepest-first",
+				i-1, a.DeepestTier, i, b.DeepestTier)
+		}
+		if a.DeepestTier == b.DeepestTier && a.Zones < b.Zones {
+			t.Errorf("equal depth but row %d covers %d zones above row %d's %d",
+				i-1, a.Zones, i, b.Zones)
+		}
+	}
+}
+
+// TestFirstsLedgerIsRenderedNotJustCounted. news_realm_firsts has existed since
+// the news seam shipped and has only ever been used to decide a dispatch tier —
+// the ledger itself was never read back. This is the whole point of the hall: it
+// is a history book, and every row needs a name and a date on it.
+func TestFirstsLedgerIsRenderedNotJustCounted(t *testing.T) {
+	seedRealmFixture(t)
+
+	stats, err := loadRealmClearStats()
+	if err != nil {
+		t.Fatalf("loadRealmClearStats: %v", err)
+	}
+	firsts := loadRealmFirsts(stats)
+	if len(firsts) != 2 {
+		t.Fatalf("got %d firsts, want 2", len(firsts))
+	}
+	// Oldest first, which is the order it happened in.
+	if firsts[0].Target != "goblin_warrens" {
+		t.Errorf("ledger order starts with %q, want goblin_warrens", firsts[0].Target)
+	}
+	for _, f := range firsts {
+		if f.Display == "" {
+			t.Errorf("first %q has no display name — it would render as a blank row", f.Target)
+		}
+		if f.Holder != "Josie" {
+			t.Errorf("first %q holder = %q, want Josie (she cleared both zones first)", f.Target, f.Holder)
+		}
+		if f.Token == "" {
+			t.Errorf("first %q has a holder but no token, so the hall can't link to them", f.Target)
+		}
+	}
+}
+
+// TestUnrecoverableFirstStillGetsAnEntry. The ledger records (kind, target,
+// first_at) and nothing else; the holder is recovered at push time from the run
+// history. A treasure found and later discarded leaves no owner anywhere, and
+// that entry has to survive as an unattributed first rather than vanish — it
+// still happened, and the hall is a record of what happened.
+func TestUnrecoverableFirstStillGetsAnEntry(t *testing.T) {
+	seedRealmFixture(t)
+	claimRealmFirst("treasure", "a_hat_nobody_kept")
+
+	var found bool
+	stats, err := loadRealmClearStats()
+	if err != nil {
+		t.Fatalf("loadRealmClearStats: %v", err)
+	}
+	for _, f := range loadRealmFirsts(stats) {
+		if f.Target == "a_hat_nobody_kept" {
+			found = true
+			if f.Holder != "" {
+				t.Errorf("holder = %q, want empty — nothing in the game knows who had it", f.Holder)
+			}
+			if f.Display == "" {
+				t.Error("an unrecoverable first got no display name at all")
+			}
+		}
+	}
+	if !found {
+		t.Error("a first with no recoverable holder was dropped from the ledger entirely")
+	}
+}
+
+// TestRealmPushIsRateLimitedBelowTheRosterTick. The realm rides the 2-minute
+// roster ticker but is aggregate scans over the whole run history, and none of
+// it moves at roster speed. The self-limit is the thing that makes riding that
+// ticker acceptable, so it is worth pinning — and so is the other half: a FAILED
+// push must not stamp the clock, or an outage would be followed by ten minutes
+// of silence instead of a retry on the next tick.
+func TestRealmPushIsRateLimitedBelowTheRosterTick(t *testing.T) {
+	if realmPushInterval <= rosterTickInterval {
+		t.Fatalf("realmPushInterval (%v) is not longer than the roster tick (%v) — "+
+			"the realm would be recomputed every tick", realmPushInterval, rosterTickInterval)
+	}
+
+	// The gate itself: zero means never-pushed and must always go.
+	realmLastPush = time.Time{}
+	t.Cleanup(func() { realmLastPush = time.Time{} })
+	now := time.Now().UTC()
+	if !realmLastPush.IsZero() {
+		t.Fatal("fixture broken")
+	}
+	// A stamp inside the window suppresses; one outside it does not.
+	realmLastPush = now.Add(-realmPushInterval / 2)
+	if now.Sub(realmLastPush) >= realmPushInterval {
+		t.Error("a push half an interval old is not being suppressed")
+	}
+	realmLastPush = now.Add(-realmPushInterval - time.Minute)
+	if now.Sub(realmLastPush) < realmPushInterval {
+		t.Error("a push older than the interval is still being suppressed")
+	}
+}

internal/plugin/pete_roster.go

@@ -53,6 +53,9 @@ func (p *AdventurePlugin) peteRosterTicker() {
 		p.pushRoster()
 		p.pushDetails()
 		p.pushSiege()
+		// Self-rate-limited to realmPushInterval: the realm is aggregate scans
+		// over the whole run history and none of it moves at roster speed.
+		p.pushRealm()
 		p.pushRunBeats()
 		// After the beats, not before: the summary is the last beat of a run's
 		// story and has no business overtaking the log it is about. It is also the