Build v7 into the player, and find the cost model 18% wrong on the block it made commonest

src/player/decode.s now paints v7 literal spans, pixel-exact under MAME and
px68k's C68K core over a container where every frame carries 128-216 spans
covering up to 38% of the picture. The span pass is blit.s v7 verbatim: the
66.0/9.143/9.978 fit was measured on that instruction sequence.

The container is DLX3 -- a span section between the mode header and the block
payload, since that is the only place the 68000 can reach without first parsing
something of variable length. 16_span_roundtrip.py gates it in check.sh, and
asserts it emitted enough spans to have tested anything.

Two synthetic all-SPAN anchors price v7 inside decode.s at 151.2 and 225.6
clocks per 4x4 block, against FINDINGS 40's table of 151 and 226 -- 0.2% on
both emulators. The measured mode costs what it was said to cost.

Two things that were not on the list:

TWO BYTE BUDGETS. FINDINGS 40's 18/120 was scored against the 488 KB/s PIPE,
not the 280 KB/s profile, and at the profile rate the lam search has already
spent the allowance -- spans fired on 5 frames of 120 and looked like a
regression. The profile is a chosen quality rate point; the pipe is hardware.
--kbps and --span-kbps are now separate and spans run before mu, because a span
pays in bytes and mu pays in picture. Delivered: 86/120 over budget without
spans, 77/120 at the profile budget, 34/120 on the pipe for +0.36 dB.

C_SKIP_MIXED WAS NEVER MEASURED, and it was 18% low -- 45.0, now 55.0. It is
the one constant in the table that came from a derivation, because the
synthetic frame that would measure it cannot exist: a byte needs a coded block
for its SKIP to be mixed. Four bracketing anchors measure it on both emulators
with the header byte rotated through all four positions, and the partner mode
solves back to its own anchored value to 0.2%. With it corrected the model
predicts a real spanned decode to -0.06% mean / 0.09% worst, against -2.99% /
4.30%. It matters because a span marks its run SKIP, so mixed SKIPs dominate
exactly the frames spans are judged on.

Also: the rig had been writing its synthetic timing frames 26 KB past the top
of a 2 MB machine, and got away with it because the modes it overran are
data-independent. A span's jump displacements come out of the stream, so it is
not. And frames-over-budget is no longer a safe headline -- the controller aims
at the deadline, so 55 of 120 frames sit within 5% of it and a 1% cost shift
moves 22 frames.

FINDINGS 41. check.sh ALL GREEN, now gating on a span-heavy DLX3 container.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
This commit is contained in:
prosolis
2026-08-23 20:02:03 -07:00
parent c520a89e14
commit b49bbdc939
16 changed files with 1342 additions and 203 deletions
+163
View File
@@ -2532,3 +2532,166 @@ fine**. `span.sh` now runs at `-seconds_to_run 200`, measured at 30 s wall for
all 36 configs, and asserts the snapshot count against the number of configs in
the generated metadata rather than a literal 23 -- so adding a config can no
longer silently weaken the pixel-exactness gate.
## 41. v7 is in the player, and the model it is scored by was 18% wrong (session 12)
FINDINGS 40 measured v7 in `tools/bench/blit.s` and left it there. This builds it
into `src/player/decode.s`, defines the container that carries it, and scores
what the encoder actually delivers rather than what a selection model predicts.
Three things came out of it that were not on the list.
### 41.1 The decoder, and the format
`src/player/decode.s` gains `paint_spans`, which is `blit.s` v7 verbatim -- the
same instruction sequence, deliberately, because the 66.0/9.143/9.978 fit was
measured on that sequence and a tidier rewrite would silently invalidate it.
The container is **DLX3**: the span section sits between the 768-byte mode
header and the block payload, because that is the only place the 68000 can
reach without first parsing something of variable length.
```
u32 payload length
768 B mode header spanned blocks read SKIP
u16 nspans
nspans * { u32 GVRAM address, u16 coarse disp, c*48 B,
u16 fine disp, f*4 B }
block payload V1 -> 1 B, V4 -> 4 B, RAW -> 16 B
```
Every span record is a multiple of 4 bytes (4+2+48c+2+4f), so the section needs
no internal padding and the block payload starts aligned. `a1`, the mode-header
cursor, is one of v7's twelve payload registers, so it goes on the stack across
the pass: two long accesses a frame, against the 24 pixels a register buys per
chain unit.
**Pixel-exact under both CPU cores on the first run**, over a container where
every frame carries 128-216 spans painting up to 38% of the picture, with full
temporal recursion. `tools/analysis/16_span_roundtrip.py` is the new gate and it
is in `check.sh`: encode, write the container, read it back with the reference
decoder, compare to what the encoder recorded. It asserts it emitted enough
spans to have tested anything -- a round-trip over a span-less container is
green by vacuity, which is FINDINGS 40.6's lesson about the snapshot count.
### 41.2 There are TWO byte budgets, and conflating them hides the whole win
The first measured span encode looked like a regression: at the `scsi` profile
spans fired on 5 of 120 frames and bought almost nothing. The cause is not the
codec. **The lam search had already spent the byte allowance**, so the span pass
inherited a few hundred bytes of room.
FINDINGS 40's 18/120 was never scored at 280 KB/s. `14_dmac_chain.py` defaults
to `--bus 488` -- the PIPE -- and gives each frame 40,977 bytes. The profile's
is 23,228. **Those are two different budgets and only one of them is hardware.**
The profile is a chosen quality rate point; the pipe is a ceiling. Bytes between
the two buy a better picture if spent on `lam`, the 68000's deadline if spent on
spans, and nothing at all if left unspent.
So the encoder now takes both: `--kbps` sets the quality target and
`--span-kbps` the ceiling the span pass may draw on, flat per frame and not
banked, because a pipe cannot be saved up. The quality bucket is credited with
the BLOCK payload only -- charging it the span bytes drives it to its floor on
the first spanned frame and starves every later frame of quality for a budget
the spans were never drawing on.
Spans also run BEFORE `mu`, and that ordering is the point. Both controllers
make a frame decode in time; `mu` pays in quality and a span pays in bytes, and
a span carries literal source pixels so it *removes* that run's quantisation
error. Spending bytes we already have beats spending picture.
| 120-frame `scsi` window | KB/s | over budget | PSNR |
|---|---:|---:|---:|
| no spans | 278.3 | 86/120 | 29.27 dB |
| spans, profile budget only | 280.0 | 77/120 | 29.23 dB |
| **spans on the 488 KB/s pipe** | 487.7 | **34/120** | **29.63 dB** |
Scored with `17_span_delivered.py`, which reads the emitted span section and
prices exactly those spans -- no selection model at all -- in 14's additive
model: block decode + span painting + disk DMA.
### 41.3 The blit.s fit transfers into the player, to 0.2%
`prep_dlx.py` gained two synthetic all-SPAN frames (full-row runs, and 4-block
runs at the break-even). They price v7 inside `decode.s` against the constants
`span.sh` fitted in `blit.s`:
| | predicted | MAME | C68K | error |
|---|---:|---:|---:|---:|
| all-SPAN-64 (192 spans x 256 px) | 505,636 | 506,533 | 506,824 | +0.18% |
| all-SPAN-4 (3072 spans x 16 px) | 734,193 | 735,133 | 735,304 | +0.13% |
Per 4x4 block that is **151.2 and 225.6 clocks, against FINDINGS 40's table of
151 and 226**. The mode costs what it was said to cost, in the real decoder, on
two emulators.
### 41.4 The rig had been writing past the top of RAM
`prep_dlx.py` truncated the real frames to a RAM budget and then appended its
synthetic timing frames ON TOP, 26 KB past the 0x200000 top of a 2 MB machine.
Survivable while it lasted, because the modes it overran are data-independent:
reading junk payload costs a V1 or a RAW block exactly what reading pixels
costs, so the anchors timed correctly by luck.
**A span is not data-independent.** Its two jump displacements come out of the
stream, so an out-of-RAM span record jumps into open bus. The synthetic frames
are now built first and their size comes out of the budget, with an assertion
that the stream ends below the top of RAM.
### 41.5 C_SKIP_MIXED was never measured, and it was 18% low
Chasing a 3% gap between the model and the measured decode turned up the one
constant in `vq_hybrid`'s cost table that came from a derivation rather than a
measurement: **the cost of a SKIP block sharing its header byte with a coded
block.** It was 45.0 from session 7 to session 12. It is **55.0**.
Every other constant comes from a synthetic frame of a single mode, and there
was no such frame for a mixed SKIP, *because one cannot exist* -- the byte has
to hold a coded block for the SKIP to be mixed at all. So `prep_dlx.py` now
emits four frames that bracket it, (3 SKIP + 1 V1), (1 SKIP + 3 V1), and the
same pair with RAW, each pair solving for the SKIP cost and its partner's
together:
| | MAME | C68K |
|---|---:|---:|
| mixed SKIP, from the V1 pair | 55.03 | 56.50 |
| mixed SKIP, from the RAW pair | 55.83 | 56.50 |
| V1, solved back out | 300.66 | 300.50 |
The partner solves back to its own anchored value to 0.2%, which is what says
the pair is measuring the SKIP rather than absorbing it. 55.0 is taken because
every other constant in the table is MAME's.
**The header bytes ROTATE through all four positions, and that is load-bearing.**
`decode.s` reaches a block's mode bits with `lsr.b #6/#4/#2` and no shift at all
for the last one, so a block costs 52/48/44/34 clocks of dispatch depending on
where in its byte it sits. A fixed pattern like 0x01 pins every SKIP to the
three expensive slots and every V1 to the free one, and solving two such
equations returns a number that describes no real frame. The first attempt did
exactly that, and a single-parameter fit against real frames then "confirmed"
87.7 -- collinear with the span term, and wrong.
With the constant corrected the model predicts the measured decode of a real
spanned container to **-0.06% on the mean and 0.09% worst frame**, against
-2.99% and 4.30% as it stood.
It matters more than 10 clocks a block sounds, because **a span marks its run
SKIP**: a spanned container is made largely of mixed SKIPs, so this is the
dominant population in exactly the frames spans are judged on. It is imported
now, not copied, in `spans.py` and `14_dmac_chain.py`.
*Incidental, and it resolved a false lead:* RAW and V4 read 3.2-3.5% higher on
C68K than on MAME, on mixed and pure frames alike. That is FINDINGS 37's known
table spread, not a property of mixed bytes -- but comparing a C68K-derived
solve against a MAME-derived anchor made it look like one for an hour.
### 41.6 Frames-over-budget is not a safe headline any more
The delivered 34/120 against 14's simulated 18/120 is a **1.4% difference in
mean frame cost** (786,381 clocks against 774,356). The metric is that
sensitive because the rate controller *aims* at the deadline: 55 of 120 frames
land within 5% of it, and shifting every frame by 1% moves the count from 25 to
47.
| every frame shifted by | -3% | -2% | -1% | 0 | +1% | +2% | +3% |
|---|---:|---:|---:|---:|---:|---:|---:|
| frames over budget | 18 | 21 | 25 | **31** | 47 | 61 | 63 |
This was a fair metric when nothing controlled to the budget. It is now a
measurement of where the controller aims, and any cost-model error is amplified
into a large count change -- which is how 41.5's 18% error stayed invisible.
**Report the cost distribution; quote the count only with its sensitivity.**
Add this to the §4 measurement traps.