Files
gogobee/internal/plugin/expedition_balance_test.go
prosolis 0f09a421bc Phase 2 (diag): cadence sweep, gear-tier fix, lethality probe
Three Phase 2 diagnostic artifacts. No tuning knob has moved on
production code yet — these tests calibrate the harness and surface
the real first lever for next session.

1) Cadence calibration sweep
   TestExpeditionBalance_Phase2_CadenceCalibration sweeps
   HarvestRollsPerDay ∈ {1,2,3,4} across the full Phase 1 matrix and
   logs per-cell + per-tier completion. Required a new
   HarvestRollsPerDay field on expeditionBalanceProfile so cells can
   override the package-default constant. Finding: cadence is NOT the
   dominant lever — at rolls=1 the T1 cell only reaches 2%, with
   bimodal hp_left (100% survivors / 0% deaths). Killed the cadence
   hypothesis from Phase 1's commit message.

2) Gear-tier centerline fix
   phase1TierCenterline bumped for T3/T4/T5 (8→9, 11→13, 15→17). The
   shared gearTier ladder (5/9/13/17 boundaries) was placing T3/T4/T5
   centerlines one gear bracket *below* the zone's tier, so those
   cells fought with under-spec'd weapons/armor. New centerlines are
   the lowest level in each tier's design-doc range where gearTier ==
   tier. All centerlines still inside their design-doc ranges. Effect
   in the sweep at rolls=1: underforge T3 1.0% → 10.5% comp, underdark
   T4 flipped from pure combat-death to 14% starve (i.e. fighter now
   survives combat, runs out of food). Real bug, but small — the
   structural lethality problem remains.

3) Lethality probe + traceFight hook
   TestExpeditionBalance_Phase2_LethalityProbe runs 5 trials at the
   cleanest cell (T1 goblin_warrens L3 fighter, rolls=1) with a new
   optional traceFight hook on expeditionHarness that logs
   monster/AC/atk/HP-pre/HP-post/outcome per fight. Hook is nil in
   production runs, zero cost when unused. Finding: at T1, the
   InterruptElite branch keeps drawing Hobgoblin Warchief (AC 18,
   atk +5) from goblin_warrens' elite roster, and an L3 fighter has
   ~coin-flip odds against a CR-6-ish elite. One bad draw = dead;
   that's the bimodal hp_left fingerprint from the sweep. Non-elite
   draws (Worg, AC 13) play out as normal multi-round combats and
   are winnable.

Next-session lever choices, in order of suspected impact:
  - Roster gate: Hobgoblin Warchief out of (or weighted down in)
    the T1 elite pool — it's tier-disproportionate for goblin_warrens.
  - InterruptElite threshold: rarer elite-bracket draws at low threat
    so a single d20 swing doesn't equal expedition end.
  - Tier-floor cap on already-over-tier bestiary entries.

Plan doc: gogobee_expedition_difficulty.md §Phase 2.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 09:39:31 -07:00

444 lines
16 KiB
Go
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
package plugin
import (
"fmt"
"math"
"sort"
"testing"
)
// TestExpeditionBalance_Phase0_Spike is the sanity check the
// expedition-difficulty plan doc's Phase 0 calls for: one cell, 100
// trials, sensible numbers, fully deterministic from the seed.
//
// The cell — T2 zone (Crypt Valdris) × L5 Fighter — was chosen
// because it's the smallest cell that actually exercises every part
// of the seam: tier scaling raises monster AC/atk, supply burn lands
// at 1.5×, the player is built off the class-balance loadout ladder
// at the T2/+1 step, and the night phase has roster entries to draw
// from. If the seam works here, Phase 1's full-matrix expansion is
// mostly typing.
//
// Pass criteria are intentionally loose — we are not asserting a
// target band yet (Phase 2 does that), only that nothing is wired up
// so badly the result is degenerate (0%/100%/NaN/zero days). Anything
// in (0..100%) completion at this cell is a green Phase 0.
func TestExpeditionBalance_Phase0_Spike(t *testing.T) {
const trials = 100
profile := expeditionBalanceProfile{
ZoneID: ZoneCryptValdris,
Class: ClassFighter,
Level: 5,
Supplies: makeSupplies(ZoneTierApprentice, SupplyPurchase{
StandardPacks: 3,
DeluxePacks: 0,
}),
CampType: CampTypeStandard,
}
res := runExpeditionBalanceCell(profile, trials, 0xC0FFEE)
t.Logf("cell: %s L%d %s — completions=%d/%d (%.1f%%), deaths=%d, starved=%d, "+
"median_days=%d, median_threat=%d, avg_encounters=%.1f, avg_hp_remaining=%.1f%%",
profile.ZoneID, profile.Level, profile.Class,
res.Completions, res.Trials, res.CompletionRate()*100,
res.Deaths, res.StarvedOuts,
res.MedianDays, res.MedianThreatEnd,
res.AvgEncounters, res.AvgHPRemainingPct*100,
)
// Degenerate-outcome gates. These are harness-broken sentinels,
// not difficulty assertions — Phase 2 layers the real band on
// top of this same test.
if res.Trials != trials {
t.Fatalf("trial count mismatch: got %d, want %d", res.Trials, trials)
}
if res.Completions == 0 {
t.Errorf("zero completions in %d trials at T2/L5 Fighter — the spike cell should not be unwinnable", trials)
}
if res.Completions == trials {
t.Errorf("100%% completions in %d trials at T2/L5 Fighter — interrupt rolls / night checks not pressuring the run", trials)
}
if res.MedianDays == 0 {
t.Fatalf("median days == 0; day loop never advanced")
}
if res.MedianDays > harnessMaxDays {
t.Fatalf("median days %d > cap %d; termination wiring broken", res.MedianDays, harnessMaxDays)
}
}
// TestExpeditionBalance_Phase0_SeedSpread confirms the RNG seam is
// actually wired — different seeds produce different trial outcomes
// across a small sample. Full byte-for-byte reproducibility under
// the same seed is *not* asserted at Phase 0: a couple of production
// helpers we lean on (surpriseRoundNick, pickWanderingMonster) draw
// from the package-global math/rand/v2, same as the class-balance
// harness. Phase 1 lifts those to seeded variants if matrix
// reproducibility becomes a real requirement; until then, "seeds
// differentiate" is the contract we can honestly hold.
func TestExpeditionBalance_Phase0_SeedSpread(t *testing.T) {
profile := expeditionBalanceProfile{
ZoneID: ZoneCryptValdris,
Class: ClassFighter,
Level: 5,
Supplies: makeSupplies(ZoneTierApprentice, SupplyPurchase{
StandardPacks: 3,
}),
CampType: CampTypeStandard,
}
const seedA uint64 = 0xDEADBEEF
const seedB uint64 = 0xFEEDFACE
a := runExpeditionBalanceTrial(profile, seedA)
b := runExpeditionBalanceTrial(profile, seedB)
if a == b {
t.Fatalf("two distinct seeds produced byte-identical trials — RNG seam may not be wired:\n a = %+v\n b = %+v", a, b)
}
}
// phase1TierCenterline maps each tier to its centerline player level.
// Originally the design doc's median per tier; bumped where the
// gear-ladder boundaries (gearTier in dnd_class_balance.go: 5/9/13/17)
// pushed the median into a gear bracket *below* the zone's tier. The
// harness's classLoadout is shared with the class-balance pass, and
// retuning its boundaries would shift class-balance numbers too, so the
// adjustment lands here instead: pick the lowest level inside each
// tier's design-doc level range that resolves to gearTier == tier.
//
// T1 (L1-4) → 3 gearTier 1 = mundane ✓
// T2 (L3-7) → 5 gearTier 2 = +1 weapon/armor ✓
// T3 (L5-10) → 9 gearTier 3 = +2 (was L8 → +1)
// T4 (L7-15) → 13 gearTier 4 = +3 (was L11 → +2)
// T5 (L10-20) → 17 gearTier 5 = +3 (was L15 → +3, but bumped
// for consistency with the rule, no stat impact)
//
// All centerlines stay inside their design-doc range. One level per
// tier keeps the matrix at zones × 1, not zones × range. Phase 4 may
// widen this if a tier band looks level-sensitive.
var phase1TierCenterline = map[ZoneTier]int{
ZoneTierBeginner: 3,
ZoneTierApprentice: 5,
ZoneTierJourneyman: 9,
ZoneTierVeteran: 13,
ZoneTierLegendary: 17,
}
// TestExpeditionBalance_Phase1_FullMatrix is the Phase 1 baseline-
// measurement test: every registered zone × its tier centerline level,
// 200 trials/cell, class fixed to Fighter so the cell-to-cell delta
// isolates zone difficulty (class parity is the class-balance pass's
// job, not this one).
//
// Per the plan doc, Phase 1's gate is *pathology-only*: no zone at T1
// reads 0% completion, no zone at T5 reads 100% completion, no NaN /
// zero-day cells. The real target-band assertion lands in Phase 2 after
// the global levers have been tuned to centerline. Until then the
// matrix's job is to *log numbers we can read*, so the per-cell line
// and per-tier mean+spread go through t.Log for the commit diff.
func TestExpeditionBalance_Phase1_FullMatrix(t *testing.T) {
if testing.Short() {
t.Skip("phase 1 matrix is heavy; -short skips it")
}
const trialsPerCell = 200
const baseSeed uint64 = 0xE0FFEE1
// Walk the zone registry in declared order (matches zoneOrder so
// the log table is stable across runs / commits).
type cellRow struct {
zone ZoneDefinition
result expeditionBalanceResult
}
rows := make([]cellRow, 0, len(zoneOrder))
for i, id := range zoneOrder {
zone, ok := getZone(id)
if !ok {
t.Fatalf("zoneOrder[%d]=%q not in registry", i, id)
}
level, ok := phase1TierCenterline[zone.Tier]
if !ok {
t.Fatalf("zone %q has tier %d with no phase1 centerline mapping", id, zone.Tier)
}
profile := expeditionBalanceProfile{
ZoneID: id,
Class: ClassFighter,
Level: level,
Supplies: makeSupplies(zone.Tier, SupplyPurchase{StandardPacks: 3}),
CampType: CampTypeStandard,
}
// Per-cell seed offset keeps cells independent — same seed
// across cells would correlate their RNG streams.
res := runExpeditionBalanceCell(profile, trialsPerCell, baseSeed+uint64(i)*1_000_003)
rows = append(rows, cellRow{zone: zone, result: res})
}
// Per-cell log. The format is grep-able: leading "CELL" tag, fixed
// columns. Future me will diff this between commits to see what
// Phase 2's lever moves actually did.
t.Logf("phase1 matrix — %d zones × %d trials, Fighter @ tier-centerline level", len(rows), trialsPerCell)
t.Logf("CELL %-18s %-3s %-3s %-12s %-12s %-13s %-11s %-13s %-9s %-12s",
"zone", "tier", "lvl", "comp%", "death%", "starve%", "med_days", "med_threat", "encs", "hp_left%")
for _, row := range rows {
r := row.result
t.Logf("CELL %-18s T%d L%-2d comp=%5.1f%% death=%5.1f%% starve=%5.1f%% med_days=%2d med_threat=%3d encs=%4.1f hp_left=%5.1f%%",
row.zone.ID, row.zone.Tier, r.Profile.Level,
r.CompletionRate()*100,
r.DeathRate()*100,
float64(r.StarvedOuts)/float64(r.Trials)*100,
r.MedianDays, r.MedianThreatEnd,
r.AvgEncounters, r.AvgHPRemainingPct*100,
)
}
// Per-tier diagnostic: mean completion% across the zones in the
// tier, and the spread (max min). Mirrors the class-balance
// per-(level,tier) spread diagnostic. A tight spread means the
// global lever pass alone can drag the tier onto target; a wide
// spread is a signal that Phase 3 per-zone outlier work is
// unavoidable.
tierRows := map[ZoneTier][]cellRow{}
for _, row := range rows {
tierRows[row.zone.Tier] = append(tierRows[row.zone.Tier], row)
}
tiers := []ZoneTier{
ZoneTierBeginner, ZoneTierApprentice, ZoneTierJourneyman,
ZoneTierVeteran, ZoneTierLegendary,
}
for _, tier := range tiers {
group := tierRows[tier]
if len(group) == 0 {
continue
}
var sum, lo, hi float64
lo = math.Inf(1)
hi = math.Inf(-1)
zoneNames := make([]string, 0, len(group))
for _, row := range group {
c := row.result.CompletionRate() * 100
sum += c
if c < lo {
lo = c
}
if c > hi {
hi = c
}
zoneNames = append(zoneNames, fmt.Sprintf("%s=%.1f%%", row.zone.ID, c))
}
sort.Strings(zoneNames)
mean := sum / float64(len(group))
t.Logf("TIER T%d n=%d mean_comp=%5.1f%% spread=%5.1f pp [%s]",
tier, len(group), mean, hi-lo, joinZones(zoneNames))
}
// Gates split into two buckets:
//
// *Wiring* pathologies (zero-day loop, days > cap) — fatal. These
// would mean the harness itself is broken and the matrix numbers
// above are noise.
//
// *Difficulty* sentinels (0% at T1, 100% at T5) — logged as WARN,
// not fatal. Phase 1's contract is "log a baseline"; the band
// assertion is Phase 2's job once the global levers have been
// tuned to land within ±10pp of target. Promoting these to
// t.Errorf here would turn Phase 1 into a tuning gate before
// Phase 2 has a chance to move the levers, which the plan doc
// explicitly defers.
for _, row := range rows {
r := row.result
if r.MedianDays == 0 {
t.Errorf("%s: median days == 0; day loop never advanced", row.zone.ID)
}
if r.MedianDays > harnessMaxDays {
t.Errorf("%s: median days %d > cap %d; termination wiring broken",
row.zone.ID, r.MedianDays, harnessMaxDays)
}
if row.zone.Tier == ZoneTierBeginner && r.Completions == 0 {
t.Logf("WARN %s (T1): 0%% completions in %d trials — Phase 2 should lift this",
row.zone.ID, r.Trials)
}
if row.zone.Tier == ZoneTierLegendary && r.Completions == r.Trials {
t.Logf("WARN %s (T5): 100%% completions in %d trials — Phase 2 should pressure this",
row.zone.ID, r.Trials)
}
}
}
// TestExpeditionBalance_Phase2_CadenceCalibration sweeps the harness's
// daytime combat-interrupt cadence across {1,2,3,4} rolls/day and logs
// the full matrix per cadence. Diagnostic-only — no gates beyond the
// Phase 1 wiring pathologies.
//
// Why this exists: Phase 1's baseline came back uniformly 0% / 100%
// death at every cell, and the commit message named cadence as the
// suspected lever. We can't compare against a live trace
// (prod is too low-traffic; the corpus is one in-flight expedition
// with zero interrupt rows), so the cadence constant is itself a
// tunable. This test surfaces the curve so Phase 2's global-lever
// pass starts from a cadence where the matrix has signal — i.e. where
// T1 is roughly in the 7090% band without flooring T5 to 100%.
//
// Cell shape mirrors Phase 1: every registered zone × its tier-
// centerline level, Fighter, 200 trials. The only thing that varies
// across runs is HarvestRollsPerDay.
func TestExpeditionBalance_Phase2_CadenceCalibration(t *testing.T) {
if testing.Short() {
t.Skip("phase 2 cadence sweep is heavy; -short skips it")
}
const trialsPerCell = 200
const baseSeed uint64 = 0xCAFEC1DE
cadences := []int{1, 2, 3, 4}
t.Logf("phase2 cadence calibration — %d zones × %d cadences × %d trials, Fighter @ tier-centerline level",
len(zoneOrder), len(cadences), trialsPerCell)
for _, rolls := range cadences {
// Per-tier aggregation for the headline view.
type tierStat struct {
cells int
sumC float64
lo float64
hi float64
}
tierStats := map[ZoneTier]*tierStat{}
t.Logf("─── HarvestRollsPerDay=%d ───", rolls)
for i, id := range zoneOrder {
zone, ok := getZone(id)
if !ok {
t.Fatalf("zoneOrder[%d]=%q not in registry", i, id)
}
level, ok := phase1TierCenterline[zone.Tier]
if !ok {
t.Fatalf("zone %q has tier %d with no phase1 centerline mapping", id, zone.Tier)
}
profile := expeditionBalanceProfile{
ZoneID: id,
Class: ClassFighter,
Level: level,
Supplies: makeSupplies(zone.Tier, SupplyPurchase{StandardPacks: 3}),
CampType: CampTypeStandard,
HarvestRollsPerDay: rolls,
}
// Seed schedule: same base + cell offset as Phase 1, plus a
// cadence-dependent salt so different cadences don't sample
// the same correlated streams.
seed := baseSeed + uint64(i)*1_000_003 + uint64(rolls)*7919
r := runExpeditionBalanceCell(profile, trialsPerCell, seed)
c := r.CompletionRate() * 100
t.Logf("CELL rolls=%d %-18s T%d L%-2d comp=%5.1f%% death=%5.1f%% starve=%5.1f%% med_days=%2d med_threat=%3d encs=%4.1f hp_left=%5.1f%%",
rolls, zone.ID, zone.Tier, r.Profile.Level,
c,
r.DeathRate()*100,
float64(r.StarvedOuts)/float64(r.Trials)*100,
r.MedianDays, r.MedianThreatEnd,
r.AvgEncounters, r.AvgHPRemainingPct*100,
)
ts, ok := tierStats[zone.Tier]
if !ok {
ts = &tierStat{lo: math.Inf(1), hi: math.Inf(-1)}
tierStats[zone.Tier] = ts
}
ts.cells++
ts.sumC += c
if c < ts.lo {
ts.lo = c
}
if c > ts.hi {
ts.hi = c
}
}
tiers := []ZoneTier{
ZoneTierBeginner, ZoneTierApprentice, ZoneTierJourneyman,
ZoneTierVeteran, ZoneTierLegendary,
}
for _, tier := range tiers {
ts := tierStats[tier]
if ts == nil || ts.cells == 0 {
continue
}
mean := ts.sumC / float64(ts.cells)
t.Logf("TIER rolls=%d T%d n=%d mean_comp=%5.1f%% spread=%5.1f pp",
rolls, tier, ts.cells, mean, ts.hi-ts.lo)
}
}
}
// TestExpeditionBalance_Phase2_LethalityProbe runs a handful of trials
// at the cleanest cell (T1 goblin_warrens L3 Fighter, gear-aligned so
// gear mismatch isn't a factor) and logs per-fight details: monster,
// max HP, surprise nick, HP pre/post fight, enemy AC/atk, outcome.
//
// The cadence sweep (TestExpeditionBalance_Phase2_CadenceCalibration)
// proved cadence isn't the dominant lever — even at rolls=1 the T1
// cell sits at 2% completion with bimodal hp_left (100% for survivors,
// 0% for the dead). This probe is the next diagnostic step: see
// whether the lethality comes from the pre-combat nick, the picked
// monster's tier-floored stats, or the combat fold itself.
//
// Diagnostic-only — no assertions beyond "produced trace lines."
// Next session reads the output and picks Phase 2's real first lever.
func TestExpeditionBalance_Phase2_LethalityProbe(t *testing.T) {
if testing.Short() {
t.Skip("phase 2 lethality probe writes verbose log; -short skips it")
}
const trials = 5
const baseSeed uint64 = 0x1E7A11
profile := expeditionBalanceProfile{
ZoneID: ZoneGoblinWarrens,
Class: ClassFighter,
Level: 3,
Supplies: makeSupplies(ZoneTierBeginner, SupplyPurchase{StandardPacks: 3}),
CampType: CampTypeStandard,
HarvestRollsPerDay: 1, // sparse cadence so each fight is legible
}
t.Logf("phase2 lethality probe — %d trials at %s L%d %s (rolls=1)",
trials, profile.ZoneID, profile.Level, profile.Class)
for trial := 0; trial < trials; trial++ {
exp := newHarnessExpedition(profile)
char := buildHarnessCharacter(classBalanceProfile{
Class: profile.Class,
Level: profile.Level,
})
t.Logf("─── trial %d: hp_max=%d ───", trial, char.HPMax)
h := &expeditionHarness{
exp: exp,
char: char,
rng: newHarnessRNG(baseSeed + uint64(trial)),
rollsPerDay: profile.HarvestRollsPerDay,
traceFight: func(line string) {
t.Logf(" %s", line)
},
}
for {
res := h.advanceExpeditionOneDay()
if res.EndedReason != "" {
t.Logf("END trial %d: reason=%s days=%d threat=%d encs=%d hp_left_pct=%.1f%%",
trial, res.EndedReason, res.DaysElapsed, res.ThreatAtEnd,
res.CombatEncounters, res.HPRemainingPct*100)
break
}
}
}
}
// joinZones is a tiny helper kept local to the test file so the
// per-tier log line reads in one logical chunk without pulling in
// strings.Join's import for production code.
func joinZones(parts []string) string {
out := ""
for i, p := range parts {
if i > 0 {
out += ", "
}
out += p
}
return out
}