Files
gogobee/internal/plugin/expedition_balance_test.go
prosolis 04aa887d18 Phase 2a (lever): expedition retreat semantics split
Phase 2 diagnostics named the InterruptElite bracket as the likely
first lever; tier-walking the lethality probe at the matrix cadence
told a different story.  Phase 1's uniform-0% baseline isn't an
elite-bracket calibration issue at all — every tier reads 0% because
the engine's TimedOut contract was being ignored by every expedition
caller.

combat_engine.go:451 says: "Timeout = retreat, not lethal blow.
Caller treats a timeout loss as 'fight ended, no character death'".
But runHarvestInterrupt / tryPatrolEncounter / resolveCombatRoom all
called abandonZoneRun + retireAllRegionRuns on any !PlayerWon —
ending the expedition outright on a retreat.  The retreat flavor line
("X outlasts you. You retreat from the expedition, wounded but alive")
was already in the code, just stapled to an actual run-abort.

Splits the policy by caller:

  • runHarvestInterrupt   — autopilot daytime interrupt.  TimedOut →
    retreat: threat +5, HP carries over, run continues, harvest slot
    forfeit (no kill / loot).  HP<=0 still ends the run + marks dead.
  • tryPatrolEncounter   — !advance pre-room patrol roll.  Same
    retreat policy: patrols don't gate progress, so retreating from
    one and walking into the next room is the right shape.
  • resolveCombatRoom    — !advance room/elite combat.  Unchanged —
    this path gates room progression; a retreat has nowhere to go, so
    any loss still ends the run.  (Manual zone runs were always
    intended to end here.)

Harness mirrored: daytime interrupt timeout → carry HP + threat bump
+ continue day; night-encounter loss → terminate (mirrors
resolveCombatRoom, since live night encounters defer to !advance).

retreatThreatBump = 5 is the per-retreat threat penalty.  Low enough
not to compound brutally with chained retreats, high enough that 3–4
retreats noticeably walks the threat clock toward Stirring.  Easy to
dial in Phase 3 if zones go off-band.

Phase 1 matrix after the change still reads 0% completion at every
cell — but the encounter counts and survival shape are dramatically
different (T4 underdark 3.6→7.5 encs; T3 underforge trial saw 18
encounters across 10 days where the pre-change run died on day 2/3).
Adds TestExpeditionBalance_Phase2_TierLethality, a tier-walking
companion to the T1/rolls=1 probe, that traces every fight at the
matrix cadence across one zone per tier — the actual Phase 2b lever
work picks from this data, not the old T1-only probe.

The remaining 0% is now legibly driven by tier-disproportionate elite
rosters (Hobgoblin Warchief at T1, Green Hag at T2, Roper/Helmed
Horror higher up) that one-shot or two-shot tier-appropriate
fighters.  Phase 2b's lever shortlist:

  1. Roster gate / SpawnWeight tuning to dilute over-tier elites.
  2. Surprise-nick floor reduction on chained interrupts (carryover
     HP + nick is the death-spiral fingerprint at T1 specifically).
  3. Per-day cadence reduction if 1+2 don't carry T1 to band.

Pre-existing test failures (TestAdv2Scenario_ZoneRunGoblinWarrens,
TestMageSpellbookLineInRender) verified to fail identically on HEAD;
no new test regressions.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 10:01:07 -07:00

511 lines
18 KiB
Go
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
package plugin
import (
"fmt"
"math"
"sort"
"testing"
)
// TestExpeditionBalance_Phase0_Spike is the sanity check the
// expedition-difficulty plan doc's Phase 0 calls for: one cell, 100
// trials, sensible numbers, fully deterministic from the seed.
//
// The cell — T2 zone (Crypt Valdris) × L5 Fighter — was chosen
// because it's the smallest cell that actually exercises every part
// of the seam: tier scaling raises monster AC/atk, supply burn lands
// at 1.5×, the player is built off the class-balance loadout ladder
// at the T2/+1 step, and the night phase has roster entries to draw
// from. If the seam works here, Phase 1's full-matrix expansion is
// mostly typing.
//
// Pass criteria are intentionally loose — we are not asserting a
// target band yet (Phase 2 does that), only that nothing is wired up
// so badly the result is degenerate (0%/100%/NaN/zero days). Anything
// in (0..100%) completion at this cell is a green Phase 0.
func TestExpeditionBalance_Phase0_Spike(t *testing.T) {
const trials = 100
profile := expeditionBalanceProfile{
ZoneID: ZoneCryptValdris,
Class: ClassFighter,
Level: 5,
Supplies: makeSupplies(ZoneTierApprentice, SupplyPurchase{
StandardPacks: 3,
DeluxePacks: 0,
}),
CampType: CampTypeStandard,
}
res := runExpeditionBalanceCell(profile, trials, 0xC0FFEE)
t.Logf("cell: %s L%d %s — completions=%d/%d (%.1f%%), deaths=%d, starved=%d, "+
"median_days=%d, median_threat=%d, avg_encounters=%.1f, avg_hp_remaining=%.1f%%",
profile.ZoneID, profile.Level, profile.Class,
res.Completions, res.Trials, res.CompletionRate()*100,
res.Deaths, res.StarvedOuts,
res.MedianDays, res.MedianThreatEnd,
res.AvgEncounters, res.AvgHPRemainingPct*100,
)
// Degenerate-outcome gates. These are harness-broken sentinels,
// not difficulty assertions — Phase 2 layers the real band on
// top of this same test.
if res.Trials != trials {
t.Fatalf("trial count mismatch: got %d, want %d", res.Trials, trials)
}
if res.Completions == 0 {
t.Errorf("zero completions in %d trials at T2/L5 Fighter — the spike cell should not be unwinnable", trials)
}
if res.Completions == trials {
t.Errorf("100%% completions in %d trials at T2/L5 Fighter — interrupt rolls / night checks not pressuring the run", trials)
}
if res.MedianDays == 0 {
t.Fatalf("median days == 0; day loop never advanced")
}
if res.MedianDays > harnessMaxDays {
t.Fatalf("median days %d > cap %d; termination wiring broken", res.MedianDays, harnessMaxDays)
}
}
// TestExpeditionBalance_Phase0_SeedSpread confirms the RNG seam is
// actually wired — different seeds produce different trial outcomes
// across a small sample. Full byte-for-byte reproducibility under
// the same seed is *not* asserted at Phase 0: a couple of production
// helpers we lean on (surpriseRoundNick, pickWanderingMonster) draw
// from the package-global math/rand/v2, same as the class-balance
// harness. Phase 1 lifts those to seeded variants if matrix
// reproducibility becomes a real requirement; until then, "seeds
// differentiate" is the contract we can honestly hold.
func TestExpeditionBalance_Phase0_SeedSpread(t *testing.T) {
profile := expeditionBalanceProfile{
ZoneID: ZoneCryptValdris,
Class: ClassFighter,
Level: 5,
Supplies: makeSupplies(ZoneTierApprentice, SupplyPurchase{
StandardPacks: 3,
}),
CampType: CampTypeStandard,
}
const seedA uint64 = 0xDEADBEEF
const seedB uint64 = 0xFEEDFACE
a := runExpeditionBalanceTrial(profile, seedA)
b := runExpeditionBalanceTrial(profile, seedB)
if a == b {
t.Fatalf("two distinct seeds produced byte-identical trials — RNG seam may not be wired:\n a = %+v\n b = %+v", a, b)
}
}
// phase1TierCenterline maps each tier to its centerline player level.
// Originally the design doc's median per tier; bumped where the
// gear-ladder boundaries (gearTier in dnd_class_balance.go: 5/9/13/17)
// pushed the median into a gear bracket *below* the zone's tier. The
// harness's classLoadout is shared with the class-balance pass, and
// retuning its boundaries would shift class-balance numbers too, so the
// adjustment lands here instead: pick the lowest level inside each
// tier's design-doc level range that resolves to gearTier == tier.
//
// T1 (L1-4) → 3 gearTier 1 = mundane ✓
// T2 (L3-7) → 5 gearTier 2 = +1 weapon/armor ✓
// T3 (L5-10) → 9 gearTier 3 = +2 (was L8 → +1)
// T4 (L7-15) → 13 gearTier 4 = +3 (was L11 → +2)
// T5 (L10-20) → 17 gearTier 5 = +3 (was L15 → +3, but bumped
// for consistency with the rule, no stat impact)
//
// All centerlines stay inside their design-doc range. One level per
// tier keeps the matrix at zones × 1, not zones × range. Phase 4 may
// widen this if a tier band looks level-sensitive.
var phase1TierCenterline = map[ZoneTier]int{
ZoneTierBeginner: 3,
ZoneTierApprentice: 5,
ZoneTierJourneyman: 9,
ZoneTierVeteran: 13,
ZoneTierLegendary: 17,
}
// TestExpeditionBalance_Phase1_FullMatrix is the Phase 1 baseline-
// measurement test: every registered zone × its tier centerline level,
// 200 trials/cell, class fixed to Fighter so the cell-to-cell delta
// isolates zone difficulty (class parity is the class-balance pass's
// job, not this one).
//
// Per the plan doc, Phase 1's gate is *pathology-only*: no zone at T1
// reads 0% completion, no zone at T5 reads 100% completion, no NaN /
// zero-day cells. The real target-band assertion lands in Phase 2 after
// the global levers have been tuned to centerline. Until then the
// matrix's job is to *log numbers we can read*, so the per-cell line
// and per-tier mean+spread go through t.Log for the commit diff.
func TestExpeditionBalance_Phase1_FullMatrix(t *testing.T) {
if testing.Short() {
t.Skip("phase 1 matrix is heavy; -short skips it")
}
const trialsPerCell = 200
const baseSeed uint64 = 0xE0FFEE1
// Walk the zone registry in declared order (matches zoneOrder so
// the log table is stable across runs / commits).
type cellRow struct {
zone ZoneDefinition
result expeditionBalanceResult
}
rows := make([]cellRow, 0, len(zoneOrder))
for i, id := range zoneOrder {
zone, ok := getZone(id)
if !ok {
t.Fatalf("zoneOrder[%d]=%q not in registry", i, id)
}
level, ok := phase1TierCenterline[zone.Tier]
if !ok {
t.Fatalf("zone %q has tier %d with no phase1 centerline mapping", id, zone.Tier)
}
profile := expeditionBalanceProfile{
ZoneID: id,
Class: ClassFighter,
Level: level,
Supplies: makeSupplies(zone.Tier, SupplyPurchase{StandardPacks: 3}),
CampType: CampTypeStandard,
}
// Per-cell seed offset keeps cells independent — same seed
// across cells would correlate their RNG streams.
res := runExpeditionBalanceCell(profile, trialsPerCell, baseSeed+uint64(i)*1_000_003)
rows = append(rows, cellRow{zone: zone, result: res})
}
// Per-cell log. The format is grep-able: leading "CELL" tag, fixed
// columns. Future me will diff this between commits to see what
// Phase 2's lever moves actually did.
t.Logf("phase1 matrix — %d zones × %d trials, Fighter @ tier-centerline level", len(rows), trialsPerCell)
t.Logf("CELL %-18s %-3s %-3s %-12s %-12s %-13s %-11s %-13s %-9s %-12s",
"zone", "tier", "lvl", "comp%", "death%", "starve%", "med_days", "med_threat", "encs", "hp_left%")
for _, row := range rows {
r := row.result
t.Logf("CELL %-18s T%d L%-2d comp=%5.1f%% death=%5.1f%% starve=%5.1f%% med_days=%2d med_threat=%3d encs=%4.1f hp_left=%5.1f%%",
row.zone.ID, row.zone.Tier, r.Profile.Level,
r.CompletionRate()*100,
r.DeathRate()*100,
float64(r.StarvedOuts)/float64(r.Trials)*100,
r.MedianDays, r.MedianThreatEnd,
r.AvgEncounters, r.AvgHPRemainingPct*100,
)
}
// Per-tier diagnostic: mean completion% across the zones in the
// tier, and the spread (max min). Mirrors the class-balance
// per-(level,tier) spread diagnostic. A tight spread means the
// global lever pass alone can drag the tier onto target; a wide
// spread is a signal that Phase 3 per-zone outlier work is
// unavoidable.
tierRows := map[ZoneTier][]cellRow{}
for _, row := range rows {
tierRows[row.zone.Tier] = append(tierRows[row.zone.Tier], row)
}
tiers := []ZoneTier{
ZoneTierBeginner, ZoneTierApprentice, ZoneTierJourneyman,
ZoneTierVeteran, ZoneTierLegendary,
}
for _, tier := range tiers {
group := tierRows[tier]
if len(group) == 0 {
continue
}
var sum, lo, hi float64
lo = math.Inf(1)
hi = math.Inf(-1)
zoneNames := make([]string, 0, len(group))
for _, row := range group {
c := row.result.CompletionRate() * 100
sum += c
if c < lo {
lo = c
}
if c > hi {
hi = c
}
zoneNames = append(zoneNames, fmt.Sprintf("%s=%.1f%%", row.zone.ID, c))
}
sort.Strings(zoneNames)
mean := sum / float64(len(group))
t.Logf("TIER T%d n=%d mean_comp=%5.1f%% spread=%5.1f pp [%s]",
tier, len(group), mean, hi-lo, joinZones(zoneNames))
}
// Gates split into two buckets:
//
// *Wiring* pathologies (zero-day loop, days > cap) — fatal. These
// would mean the harness itself is broken and the matrix numbers
// above are noise.
//
// *Difficulty* sentinels (0% at T1, 100% at T5) — logged as WARN,
// not fatal. Phase 1's contract is "log a baseline"; the band
// assertion is Phase 2's job once the global levers have been
// tuned to land within ±10pp of target. Promoting these to
// t.Errorf here would turn Phase 1 into a tuning gate before
// Phase 2 has a chance to move the levers, which the plan doc
// explicitly defers.
for _, row := range rows {
r := row.result
if r.MedianDays == 0 {
t.Errorf("%s: median days == 0; day loop never advanced", row.zone.ID)
}
if r.MedianDays > harnessMaxDays {
t.Errorf("%s: median days %d > cap %d; termination wiring broken",
row.zone.ID, r.MedianDays, harnessMaxDays)
}
if row.zone.Tier == ZoneTierBeginner && r.Completions == 0 {
t.Logf("WARN %s (T1): 0%% completions in %d trials — Phase 2 should lift this",
row.zone.ID, r.Trials)
}
if row.zone.Tier == ZoneTierLegendary && r.Completions == r.Trials {
t.Logf("WARN %s (T5): 100%% completions in %d trials — Phase 2 should pressure this",
row.zone.ID, r.Trials)
}
}
}
// TestExpeditionBalance_Phase2_CadenceCalibration sweeps the harness's
// daytime combat-interrupt cadence across {1,2,3,4} rolls/day and logs
// the full matrix per cadence. Diagnostic-only — no gates beyond the
// Phase 1 wiring pathologies.
//
// Why this exists: Phase 1's baseline came back uniformly 0% / 100%
// death at every cell, and the commit message named cadence as the
// suspected lever. We can't compare against a live trace
// (prod is too low-traffic; the corpus is one in-flight expedition
// with zero interrupt rows), so the cadence constant is itself a
// tunable. This test surfaces the curve so Phase 2's global-lever
// pass starts from a cadence where the matrix has signal — i.e. where
// T1 is roughly in the 7090% band without flooring T5 to 100%.
//
// Cell shape mirrors Phase 1: every registered zone × its tier-
// centerline level, Fighter, 200 trials. The only thing that varies
// across runs is HarvestRollsPerDay.
func TestExpeditionBalance_Phase2_CadenceCalibration(t *testing.T) {
if testing.Short() {
t.Skip("phase 2 cadence sweep is heavy; -short skips it")
}
const trialsPerCell = 200
const baseSeed uint64 = 0xCAFEC1DE
cadences := []int{1, 2, 3, 4}
t.Logf("phase2 cadence calibration — %d zones × %d cadences × %d trials, Fighter @ tier-centerline level",
len(zoneOrder), len(cadences), trialsPerCell)
for _, rolls := range cadences {
// Per-tier aggregation for the headline view.
type tierStat struct {
cells int
sumC float64
lo float64
hi float64
}
tierStats := map[ZoneTier]*tierStat{}
t.Logf("─── HarvestRollsPerDay=%d ───", rolls)
for i, id := range zoneOrder {
zone, ok := getZone(id)
if !ok {
t.Fatalf("zoneOrder[%d]=%q not in registry", i, id)
}
level, ok := phase1TierCenterline[zone.Tier]
if !ok {
t.Fatalf("zone %q has tier %d with no phase1 centerline mapping", id, zone.Tier)
}
profile := expeditionBalanceProfile{
ZoneID: id,
Class: ClassFighter,
Level: level,
Supplies: makeSupplies(zone.Tier, SupplyPurchase{StandardPacks: 3}),
CampType: CampTypeStandard,
HarvestRollsPerDay: rolls,
}
// Seed schedule: same base + cell offset as Phase 1, plus a
// cadence-dependent salt so different cadences don't sample
// the same correlated streams.
seed := baseSeed + uint64(i)*1_000_003 + uint64(rolls)*7919
r := runExpeditionBalanceCell(profile, trialsPerCell, seed)
c := r.CompletionRate() * 100
t.Logf("CELL rolls=%d %-18s T%d L%-2d comp=%5.1f%% death=%5.1f%% starve=%5.1f%% med_days=%2d med_threat=%3d encs=%4.1f hp_left=%5.1f%%",
rolls, zone.ID, zone.Tier, r.Profile.Level,
c,
r.DeathRate()*100,
float64(r.StarvedOuts)/float64(r.Trials)*100,
r.MedianDays, r.MedianThreatEnd,
r.AvgEncounters, r.AvgHPRemainingPct*100,
)
ts, ok := tierStats[zone.Tier]
if !ok {
ts = &tierStat{lo: math.Inf(1), hi: math.Inf(-1)}
tierStats[zone.Tier] = ts
}
ts.cells++
ts.sumC += c
if c < ts.lo {
ts.lo = c
}
if c > ts.hi {
ts.hi = c
}
}
tiers := []ZoneTier{
ZoneTierBeginner, ZoneTierApprentice, ZoneTierJourneyman,
ZoneTierVeteran, ZoneTierLegendary,
}
for _, tier := range tiers {
ts := tierStats[tier]
if ts == nil || ts.cells == 0 {
continue
}
mean := ts.sumC / float64(ts.cells)
t.Logf("TIER rolls=%d T%d n=%d mean_comp=%5.1f%% spread=%5.1f pp",
rolls, tier, ts.cells, mean, ts.hi-ts.lo)
}
}
}
// TestExpeditionBalance_Phase2_LethalityProbe runs a handful of trials
// at the cleanest cell (T1 goblin_warrens L3 Fighter, gear-aligned so
// gear mismatch isn't a factor) and logs per-fight details: monster,
// max HP, surprise nick, HP pre/post fight, enemy AC/atk, outcome.
//
// The cadence sweep (TestExpeditionBalance_Phase2_CadenceCalibration)
// proved cadence isn't the dominant lever — even at rolls=1 the T1
// cell sits at 2% completion with bimodal hp_left (100% for survivors,
// 0% for the dead). This probe is the next diagnostic step: see
// whether the lethality comes from the pre-combat nick, the picked
// monster's tier-floored stats, or the combat fold itself.
//
// Diagnostic-only — no assertions beyond "produced trace lines."
// Next session reads the output and picks Phase 2's real first lever.
func TestExpeditionBalance_Phase2_LethalityProbe(t *testing.T) {
if testing.Short() {
t.Skip("phase 2 lethality probe writes verbose log; -short skips it")
}
const trials = 5
const baseSeed uint64 = 0x1E7A11
profile := expeditionBalanceProfile{
ZoneID: ZoneGoblinWarrens,
Class: ClassFighter,
Level: 3,
Supplies: makeSupplies(ZoneTierBeginner, SupplyPurchase{StandardPacks: 3}),
CampType: CampTypeStandard,
HarvestRollsPerDay: 1, // sparse cadence so each fight is legible
}
t.Logf("phase2 lethality probe — %d trials at %s L%d %s (rolls=1)",
trials, profile.ZoneID, profile.Level, profile.Class)
for trial := 0; trial < trials; trial++ {
exp := newHarnessExpedition(profile)
char := buildHarnessCharacter(classBalanceProfile{
Class: profile.Class,
Level: profile.Level,
})
t.Logf("─── trial %d: hp_max=%d ───", trial, char.HPMax)
h := &expeditionHarness{
exp: exp,
char: char,
rng: newHarnessRNG(baseSeed + uint64(trial)),
rollsPerDay: profile.HarvestRollsPerDay,
traceFight: func(line string) {
t.Logf(" %s", line)
},
}
for {
res := h.advanceExpeditionOneDay()
if res.EndedReason != "" {
t.Logf("END trial %d: reason=%s days=%d threat=%d encs=%d hp_left_pct=%.1f%%",
trial, res.EndedReason, res.DaysElapsed, res.ThreatAtEnd,
res.CombatEncounters, res.HPRemainingPct*100)
break
}
}
}
}
// TestExpeditionBalance_Phase2_TierLethality is the tier-walking
// companion to the T1-only lethality probe. Phase 1's matrix shows
// uniform 0% across every tier at the default rolls=4 cadence, which
// the T1/rolls=1 probe alone can't explain (its bimodal-warchief
// finding wouldn't show at T5 vs an L17 fighter). This probe traces
// every fight at the matrix cadence across one zone per tier so we
// can read whether the deaths are elite-driven, standard-fight
// chained, or something the harness combat fold itself is doing.
//
// Diagnostic-only — no assertions.
func TestExpeditionBalance_Phase2_TierLethality(t *testing.T) {
if testing.Short() {
t.Skip("phase 2 tier-lethality probe writes verbose log; -short skips it")
}
const trialsPerTier = 3
const baseSeed uint64 = 0x7E11A1
cells := []struct {
zone ZoneID
tier ZoneTier
}{
{ZoneGoblinWarrens, ZoneTierBeginner},
{ZoneForestShadows, ZoneTierApprentice},
{ZoneUnderforge, ZoneTierJourneyman},
{ZoneUnderdark, ZoneTierVeteran},
{ZoneDragonsLair, ZoneTierLegendary},
}
for _, cell := range cells {
level := phase1TierCenterline[cell.tier]
profile := expeditionBalanceProfile{
ZoneID: cell.zone,
Class: ClassFighter,
Level: level,
Supplies: makeSupplies(cell.tier, SupplyPurchase{StandardPacks: 3}),
CampType: CampTypeStandard,
HarvestRollsPerDay: 4, // matrix default, not the sparse probe
}
t.Logf("═══ %s T%d L%d Fighter rolls=4 ═══", cell.zone, cell.tier, level)
for trial := 0; trial < trialsPerTier; trial++ {
exp := newHarnessExpedition(profile)
char := buildHarnessCharacter(classBalanceProfile{
Class: profile.Class,
Level: profile.Level,
})
t.Logf("─── %s trial %d: hp_max=%d ───", cell.zone, trial, char.HPMax)
h := &expeditionHarness{
exp: exp,
char: char,
rng: newHarnessRNG(baseSeed + uint64(trial) + uint64(cell.tier)*101),
rollsPerDay: profile.HarvestRollsPerDay,
traceFight: func(line string) {
t.Logf(" %s", line)
},
}
for {
res := h.advanceExpeditionOneDay()
if res.EndedReason != "" {
t.Logf("END %s trial %d: reason=%s days=%d threat=%d encs=%d hp_left_pct=%.1f%%",
cell.zone, trial, res.EndedReason, res.DaysElapsed, res.ThreatAtEnd,
res.CombatEncounters, res.HPRemainingPct*100)
break
}
}
}
}
}
// joinZones is a tiny helper kept local to the test file so the
// per-tier log line reads in one logical chunk without pulling in
// strings.Join's import for production code.
func joinZones(parts []string) string {
out := ""
for i, p := range parts {
if i > 0 {
out += ", "
}
out += p
}
return out
}