Build v7 into the player, and find the cost model 18% wrong on the block it made commonest

src/player/decode.s now paints v7 literal spans, pixel-exact under MAME and px68k's C68K core over a container where every frame carries 128-216 spans covering up to 38% of the picture. The span pass is blit.s v7 verbatim: the 66.0/9.143/9.978 fit was measured on that instruction sequence. The container is DLX3 -- a span section between the mode header and the block payload, since that is the only place the 68000 can reach without first parsing something of variable length. 16_span_roundtrip.py gates it in check.sh, and asserts it emitted enough spans to have tested anything. Two synthetic all-SPAN anchors price v7 inside decode.s at 151.2 and 225.6 clocks per 4x4 block, against FINDINGS 40's table of 151 and 226 -- 0.2% on both emulators. The measured mode costs what it was said to cost. Two things that were not on the list: TWO BYTE BUDGETS. FINDINGS 40's 18/120 was scored against the 488 KB/s PIPE, not the 280 KB/s profile, and at the profile rate the lam search has already spent the allowance -- spans fired on 5 frames of 120 and looked like a regression. The profile is a chosen quality rate point; the pipe is hardware. --kbps and --span-kbps are now separate and spans run before mu, because a span pays in bytes and mu pays in picture. Delivered: 86/120 over budget without spans, 77/120 at the profile budget, 34/120 on the pipe for +0.36 dB. C_SKIP_MIXED WAS NEVER MEASURED, and it was 18% low -- 45.0, now 55.0. It is the one constant in the table that came from a derivation, because the synthetic frame that would measure it cannot exist: a byte needs a coded block for its SKIP to be mixed. Four bracketing anchors measure it on both emulators with the header byte rotated through all four positions, and the partner mode solves back to its own anchored value to 0.2%. With it corrected the model predicts a real spanned decode to -0.06% mean / 0.09% worst, against -2.99% / 4.30%. It matters because a span marks its run SKIP, so mixed SKIPs dominate exactly the frames spans are judged on. Also: the rig had been writing its synthetic timing frames 26 KB past the top of a 2 MB machine, and got away with it because the modes it overran are data-independent. A span's jump displacements come out of the stream, so it is not. And frames-over-budget is no longer a safe headline -- the controller aims at the deadline, so 55 of 120 frames sit within 5% of it and a 1% cost shift moves 22 frames. FINDINGS 41. check.sh ALL GREEN, now gating on a span-heavy DLX3 container. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
2026-08-23 20:02:03 -07:00
parent c520a89e14
commit b49bbdc939
16 changed files with 1342 additions and 203 deletions
@@ -11,9 +11,15 @@ decoder occupies 86.7% of it once instruction prefetch is counted, and 52 of the
 53 frames that miss the 12fps budget miss it on the bus, not the CPU
 (FINDINGS 38). Read that before optimising anything for cycles.
-The largest measured win on the table is the **literal span with a fine tail**
+The **literal span with a fine tail** (v7) is now IN the player: `decode.s`
-(`blit.s` v7): it takes the worst `scsi` window from 84/120 frames over budget
+paints it, pixel-exact under both CPU cores, and it costs inside the decoder
-to 18/120, and `src/player/decode.s` does not implement it yet (FINDINGS 40).
+what `blit.s` said it would to 0.2% (FINDINGS 41).
 **It only pays if the stream is allowed to run near the pipe.** Spans buy the
 68000's deadline with bytes, and at the 280 KB/s profile the mode decision has
 already spent them: 77/120 frames over budget against 86 without spans. Given
 the full 488 KB/s pipe it is **34/120, and 0.36 dB better** — so the next
 decision is a rate point, not an optimisation (FINDINGS 41.2, docs/STATUS.md).
 **Green-light check:** `./tools/bench/check.sh` (~3 min, needs the Blu-ray
 mounted) re-runs both display regression tests, the rate-control drift test, the
@@ -58,7 +64,15 @@ tools/analysis/  measurement scripts, numbered in the order they were written
                 15 measures how much of the 68000's LOCAL bus the decoder
                 occupies (FINDINGS 38) and exits non-zero if its derived
                 model stops matching the harness's measurement.
-                 buscost.py is the shared bus-cycle table both import.
+                 16 is the DLX3 span container ROUND-TRIP gate (part of
                 check.sh): it encodes, writes the container, reads it back with
                 the reference decoder and fails if a pixel differs -- or if it
                 emitted too few spans to have tested anything. 17 prices the
                 spans the encoder ACTUALLY emitted, with no selection model,
                 which is what 12 and 14 could only simulate.
                 buscost.py is the shared bus-cycle table both import; the
                 per-BLOCK constants live in tools/encoder/vq_hybrid.py and are
                 imported, never copied (session 12 corrected one of them).
 tools/bench/     MAME Lua injection harness + 68000 benchmark sources.
                 `check.sh` re-runs both display regression tests (~40 s).
                 `blit.s`/`blit.lua` time the full-frame GVRAM blit on the
@@ -85,13 +99,19 @@ tools/bench/c68k/ headless px68k C68K harness -- a SECOND emulator for every
                 The Makefile's -no-pie and the harness's MAP_32BIT arena are
                 load-bearing: C68K truncates host pointers to 32 bits.
 tools/vasm/      vasm m68k assembler (built from source)
-tools/encoder/   hybrid VQ encoder + DLX2 container writer (working).
+tools/encoder/   hybrid VQ encoder + DLX3 container writer (working).
                 spans.py is the v7 span geometry, selection and serialiser, and
                 the single place the chain layout is stated on the encoder side
                 -- it must match blit.s/decode.s (11 coarse units of 24 px, 11
                 fine of 2).
                 DLX2 4-byte-aligns every frame record: an odd `move.l` is an
                 ADDRESS ERROR on a 68000, not a slow read (FINDINGS 28.3).
                 dlx.py is the reference DECODER -- ground truth for the 68000.
-src/player/      decode.s: the 68000 DLX decoder. Pixel-exact; 1 frame of 120
+src/player/      decode.s: the 68000 DLX3 decoder. Pixel-exact under MAME and
-                 over the 12fps CPU budget once the mode decision prices
+                 px68k's C68K core, blocks and v7 literal spans both. The span
-                 cycles. See FINDINGS 28 and 31.
+                 pass is blit.s v7 verbatim -- the same instruction sequence the
                 66.0/9.143/9.978 fit was measured on, so do not tidy it.
                 See FINDINGS 28, 31, 40 and 41.
 assets/          extracted frames/audio (gitignored)
 ```
@@ -102,6 +122,13 @@ python3 tools/encoder/extract.py 00020 /tmp/fr 12 crop
 python3 tools/encoder/encode.py  /tmp/fr out.dlx --profile scsi --preview p.png
 ```
 **Two budgets, not one.** `--kbps` is the quality rate point and `--span-kbps`
 is the ceiling the span pass may draw on. They are different things: the profile
 is chosen, the pipe is hardware, and bytes between them buy a better picture if
 spent on `lam`, the 68000's deadline if spent on spans, and nothing if left
 unspent. Spans run before `mu` because a span pays in bytes and `mu` pays in
 picture (FINDINGS 41.2).
 **One profile: `scsi`, 280 KB/s.** The 110 KB/s `sasi` profile was dropped in
 session 9 on capacity, not bandwidth — a SASI volume is limited to 40 MB, and
 the game's 22.8 minutes of footage is 146 MiB even at that rate (FINDINGS 32).
@@ -2532,3 +2532,166 @@ fine**. `span.sh` now runs at `-seconds_to_run 200`, measured at 30 s wall for
 all 36 configs, and asserts the snapshot count against the number of configs in
 the generated metadata rather than a literal 23 -- so adding a config can no
 longer silently weaken the pixel-exactness gate.
 ## 41. v7 is in the player, and the model it is scored by was 18% wrong (session 12)
 FINDINGS 40 measured v7 in `tools/bench/blit.s` and left it there. This builds it
 into `src/player/decode.s`, defines the container that carries it, and scores
 what the encoder actually delivers rather than what a selection model predicts.
 Three things came out of it that were not on the list.
 ### 41.1 The decoder, and the format
 `src/player/decode.s` gains `paint_spans`, which is `blit.s` v7 verbatim -- the
 same instruction sequence, deliberately, because the 66.0/9.143/9.978 fit was
 measured on that sequence and a tidier rewrite would silently invalidate it.
 The container is **DLX3**: the span section sits between the 768-byte mode
 header and the block payload, because that is the only place the 68000 can
 reach without first parsing something of variable length.
 ```
 u32 payload length
 768 B mode header          spanned blocks read SKIP
 u16 nspans
  nspans * { u32 GVRAM address, u16 coarse disp, c*48 B,
             u16 fine disp, f*4 B }
 block payload              V1 -> 1 B, V4 -> 4 B, RAW -> 16 B
 ```
 Every span record is a multiple of 4 bytes (4+2+48c+2+4f), so the section needs
 no internal padding and the block payload starts aligned. `a1`, the mode-header
 cursor, is one of v7's twelve payload registers, so it goes on the stack across
 the pass: two long accesses a frame, against the 24 pixels a register buys per
 chain unit.
 **Pixel-exact under both CPU cores on the first run**, over a container where
 every frame carries 128-216 spans painting up to 38% of the picture, with full
 temporal recursion. `tools/analysis/16_span_roundtrip.py` is the new gate and it
 is in `check.sh`: encode, write the container, read it back with the reference
 decoder, compare to what the encoder recorded. It asserts it emitted enough
 spans to have tested anything -- a round-trip over a span-less container is
 green by vacuity, which is FINDINGS 40.6's lesson about the snapshot count.
 ### 41.2 There are TWO byte budgets, and conflating them hides the whole win
 The first measured span encode looked like a regression: at the `scsi` profile
 spans fired on 5 of 120 frames and bought almost nothing. The cause is not the
 codec. **The lam search had already spent the byte allowance**, so the span pass
 inherited a few hundred bytes of room.
 FINDINGS 40's 18/120 was never scored at 280 KB/s. `14_dmac_chain.py` defaults
 to `--bus 488` -- the PIPE -- and gives each frame 40,977 bytes. The profile's
 is 23,228. **Those are two different budgets and only one of them is hardware.**
 The profile is a chosen quality rate point; the pipe is a ceiling. Bytes between
 the two buy a better picture if spent on `lam`, the 68000's deadline if spent on
 spans, and nothing at all if left unspent.
 So the encoder now takes both: `--kbps` sets the quality target and
 `--span-kbps` the ceiling the span pass may draw on, flat per frame and not
 banked, because a pipe cannot be saved up. The quality bucket is credited with
 the BLOCK payload only -- charging it the span bytes drives it to its floor on
 the first spanned frame and starves every later frame of quality for a budget
 the spans were never drawing on.
 Spans also run BEFORE `mu`, and that ordering is the point. Both controllers
 make a frame decode in time; `mu` pays in quality and a span pays in bytes, and
 a span carries literal source pixels so it *removes* that run's quantisation
 error. Spending bytes we already have beats spending picture.
 | 120-frame `scsi` window | KB/s | over budget | PSNR |
 |---|---:|---:|---:|
 | no spans | 278.3 | 86/120 | 29.27 dB |
 | spans, profile budget only | 280.0 | 77/120 | 29.23 dB |
 | **spans on the 488 KB/s pipe** | 487.7 | **34/120** | **29.63 dB** |
 Scored with `17_span_delivered.py`, which reads the emitted span section and
 prices exactly those spans -- no selection model at all -- in 14's additive
 model: block decode + span painting + disk DMA.
 ### 41.3 The blit.s fit transfers into the player, to 0.2%
 `prep_dlx.py` gained two synthetic all-SPAN frames (full-row runs, and 4-block
 runs at the break-even). They price v7 inside `decode.s` against the constants
 `span.sh` fitted in `blit.s`:
 | | predicted | MAME | C68K | error |
 |---|---:|---:|---:|---:|
 | all-SPAN-64 (192 spans x 256 px) | 505,636 | 506,533 | 506,824 | +0.18% |
 | all-SPAN-4 (3072 spans x 16 px) | 734,193 | 735,133 | 735,304 | +0.13% |
 Per 4x4 block that is **151.2 and 225.6 clocks, against FINDINGS 40's table of
 151 and 226**. The mode costs what it was said to cost, in the real decoder, on
 two emulators.
 ### 41.4 The rig had been writing past the top of RAM
 `prep_dlx.py` truncated the real frames to a RAM budget and then appended its
 synthetic timing frames ON TOP, 26 KB past the 0x200000 top of a 2 MB machine.
 Survivable while it lasted, because the modes it overran are data-independent:
 reading junk payload costs a V1 or a RAW block exactly what reading pixels
 costs, so the anchors timed correctly by luck.
 **A span is not data-independent.** Its two jump displacements come out of the
 stream, so an out-of-RAM span record jumps into open bus. The synthetic frames
 are now built first and their size comes out of the budget, with an assertion
 that the stream ends below the top of RAM.
 ### 41.5 C_SKIP_MIXED was never measured, and it was 18% low
 Chasing a 3% gap between the model and the measured decode turned up the one
 constant in `vq_hybrid`'s cost table that came from a derivation rather than a
 measurement: **the cost of a SKIP block sharing its header byte with a coded
 block.** It was 45.0 from session 7 to session 12. It is **55.0**.
 Every other constant comes from a synthetic frame of a single mode, and there
 was no such frame for a mixed SKIP, *because one cannot exist* -- the byte has
 to hold a coded block for the SKIP to be mixed at all. So `prep_dlx.py` now
 emits four frames that bracket it, (3 SKIP + 1 V1), (1 SKIP + 3 V1), and the
 same pair with RAW, each pair solving for the SKIP cost and its partner's
 together:
 | | MAME | C68K |
 |---|---:|---:|
 | mixed SKIP, from the V1 pair | 55.03 | 56.50 |
 | mixed SKIP, from the RAW pair | 55.83 | 56.50 |
 | V1, solved back out | 300.66 | 300.50 |
 The partner solves back to its own anchored value to 0.2%, which is what says
 the pair is measuring the SKIP rather than absorbing it. 55.0 is taken because
 every other constant in the table is MAME's.
 **The header bytes ROTATE through all four positions, and that is load-bearing.**
 `decode.s` reaches a block's mode bits with `lsr.b #6/#4/#2` and no shift at all
 for the last one, so a block costs 52/48/44/34 clocks of dispatch depending on
 where in its byte it sits. A fixed pattern like 0x01 pins every SKIP to the
 three expensive slots and every V1 to the free one, and solving two such
 equations returns a number that describes no real frame. The first attempt did
 exactly that, and a single-parameter fit against real frames then "confirmed"
 87.7 -- collinear with the span term, and wrong.
 With the constant corrected the model predicts the measured decode of a real
 spanned container to **-0.06% on the mean and 0.09% worst frame**, against
 -2.99% and 4.30% as it stood.
 It matters more than 10 clocks a block sounds, because **a span marks its run
 SKIP**: a spanned container is made largely of mixed SKIPs, so this is the
 dominant population in exactly the frames spans are judged on. It is imported
 now, not copied, in `spans.py` and `14_dmac_chain.py`.
 *Incidental, and it resolved a false lead:* RAW and V4 read 3.2-3.5% higher on
 C68K than on MAME, on mixed and pure frames alike. That is FINDINGS 37's known
 table spread, not a property of mixed bytes -- but comparing a C68K-derived
 solve against a MAME-derived anchor made it look like one for an hour.
 ### 41.6 Frames-over-budget is not a safe headline any more
 The delivered 34/120 against 14's simulated 18/120 is a **1.4% difference in
 mean frame cost** (786,381 clocks against 774,356). The metric is that
 sensitive because the rate controller *aims* at the deadline: 55 of 120 frames
 land within 5% of it, and shifting every frame by 1% moves the count from 25 to
 47.
 | every frame shifted by | -3% | -2% | -1% | 0 | +1% | +2% | +3% |
 |---|---:|---:|---:|---:|---:|---:|---:|
 | frames over budget | 18 | 21 | 25 | **31** | 47 | 61 | 63 |
 This was a fair metric when nothing controlled to the budget. It is now a
 measurement of where the controller aims, and any cost-model error is amplified
 into a large count change -- which is how 41.5's 18% error stayed invisible.
 **Report the cost distribution; quote the count only with its sensitivity.**
 Add this to the §4 measurement traps.
@@ -1,107 +1,108 @@
-# Status & next-session handoff — end of session 11 (2026-08-23)
+# Status & next-session handoff — end of session 12 (2026-08-23)
 ## Where this stands
-Session 11 measured the one item session 10 left at the top of the list, and it
+Session 12 built v7 into the player. **`src/player/decode.s` paints v7 literal
-paid: **`blit.s` v7, the literal span with a fine tail, is MEASURED and takes
+spans, and it is pixel-exact under both CPU cores** over a container where every
-the `scsi` window from 84/120 frames over budget to 18/120.** FINDINGS 40.
+frame carries 128-216 spans covering up to 38% of the picture. FINDINGS 41.
-```
+The container is **DLX3**: a span section between the mode header and the block
-v7:  cycles = 66.0 per span + 9.143 per COARSE pixel + 9.978 per FINE pixel
+payload, `{u32 GVRAM address, u16 coarse disp}` per span with the fine
-     13 span lengths, all fitted to within 0.2%, all pixel-exact
+displacement mid-stream. `tools/analysis/16_span_roundtrip.py` gates it and is
-```
+in `check.sh`.
-| | frames over budget, 120-frame `scsi` window |
+**The measured cost transfers.** Two synthetic all-SPAN anchors price v7 inside
-|---|---:|
+`decode.s` at **151.2 and 225.6 clocks per 4x4 block**, against FINDINGS 40's
-| today (no spans) | 84/120 |
+table of 151 and 226 — 0.2% on both emulators.
 | v6 span as built | 55/120 |
 | **v7, measured** | **18/120** |
 | DMAC chain (datasheet) | 12/120 |
-**The DMAC stays dropped, and now on a measurement rather than an argument.**
+### The two things that were not on the list
 v7 takes back 37 of the 43 frames the DMAC chain would, with no reserved
 channel, no two-region container, and no transfer timing neither emulator here
 can verify. FINDINGS 39.1 still holds if that ever changes: a chain array entry
 and a v6/v7 span record are the same six bytes.
-**Break-even against all-V1 moves from L=4 blocks to L=2.** 39.4 predicted L=3.
+**1. There are TWO byte budgets, and FINDINGS 40's 18/120 was scored at the
 wrong one.** The `scsi` profile is 280 KB/s; `14_dmac_chain.py` scores spans
 against the 488 KB/s PIPE, which is 40,977 B/frame against 23,228. At the
 profile rate the lam search has already spent the allowance and spans fire on 5
 frames of 120. The profile is a chosen quality rate point; the pipe is hardware.
 `--kbps` and `--span-kbps` are now separate, and spans run before `mu` because a
 span pays in bytes and `mu` pays in picture. FINDINGS 41.2.
-**18/120 is exactly what 39.4 derived, and that is a coincidence of two
+| 120-frame `scsi` window | KB/s | over budget | PSNR |
-cancelling errors** — worth knowing before the next derived figure gets trusted
+|---|---:|---:|---:|
-for landing on its measurement. 39.4 assumed a 2-register `movem` tail at 14.0
+| no spans | 278.3 | 86/120 | 29.27 dB |
-clocks/pixel (the real tail is 9.978, 29% cheaper) and assumed the second chain
+| spans, profile budget only | 280.0 | 77/120 | 29.23 dB |
-entry costs nothing per span (it costs 22.3 clocks). The two nearly cancel over
+| **spans on the 488 KB/s pipe** | 487.7 | **34/120** | **29.63 dB** |
 this window. FINDINGS 40.2.
-**The tail instruction the derivation should have picked is `move.l (a0)+,(a2)+`.**
+**2. `C_SKIP_MIXED` was never measured, and it was 18% low — 45.0, now 55.0.**
-A 2-register `movem` pays two instruction words and a `lea` to move what two
+It is the one constant in the cost table that came from a derivation, because
-post-incrementing `move.l`s move: 14 bus cycles against 10 for the same 4
+the synthetic frame that would measure it cannot exist (a byte needs a coded
-pixels. Taking the plain instruction also makes the padding quantum **2 pixels**
+block for its SKIP to be mixed). Four new bracketing anchors measure it on both
-instead of 4 — and a span is a run of 4x4 blocks, so **its padding is exactly
+emulators, and with it corrected the model predicts a real spanned decode to
-zero**. FINDINGS 40.3.
+**-0.06% mean / 0.09% worst**, against -2.99% / 4.30% before. It matters here
 because **a span marks its run SKIP**, so mixed SKIPs are the dominant
 population in exactly the frames spans are judged on. FINDINGS 41.5.
-**The fine displacement lives in the STREAM, not in the span record**, after the
+**And the metric everything has been quoted in is unstable.** 34/120 delivered
-coarse pixels and before the fine ones: the coarse chain falls out into
+against 14's simulated 18/120 is a 1.4% difference in mean frame cost. 55 of 120
-`move.w (a0)+,d0 / jmp`, where `d0` is dead payload and `a0` already points at
+frames sit within 5% of the deadline because the rate controller aims there, so
-it. That is what lets v7 keep all 12 payload registers, which is the entire
+a 1% cost shift moves 22 frames. Quote the distribution, not the count.
-reason v6's unit is 24 pixels. The container costs 2 more bytes a span.
+FINDINGS 41.6.
 FINDINGS 40.4.
-**One process note.** `span.sh` ran 13 minutes producing an empty log and zero
+Green light: `./tools/bench/check.sh` **ALL GREEN**, now gating on a span-heavy
-snapshots; the same command with a shorter `-seconds_to_run` did the identical
+DLX3 container.
 work in 30 s, and the wedge never reproduced. The cause is unidentified. What
 resolved it was not chasing the hang but **shrinking the stimulus**:
 `tmp/spans_meta.lua` holds byte offsets into a blob `prep_spans.py` writes once,
 so deleting lines from the metadata runs any subset in seconds against the same
 stream file. Keep that trick. `span.sh` is now `-seconds_to_run 200` (30 s wall
 for all 36 configs) and takes its expected snapshot count from the metadata
 instead of a literal 23. FINDINGS 40.6.
 Green light: `./tools/bench/check.sh` **ALL GREEN** at the end of this session.
 ## NEXT SESSION, in order
-0. **Green light first.** `./tools/bench/check.sh` (~5 min, Blu-ray mounted).
+0. **Green light first.** `./tools/bench/check.sh` (~6 min, Blu-ray mounted).
-   Verified green at end of session 11.
+   Verified green at end of session 12. The gate container is now
   `tmp/rc_fr_singe_scsi_span.dlx` (scsi modes, spans on the full pipe) and the
   rig fits 37 of 120 frames in a 2 MB machine.
-1. **Build v7 into `src/player/decode.s`.** This is now the largest thing
+1. **Decide the rate point, because the span result now depends on it.** This is
-   standing between the measured decoder and the budget: 84/120 to 18/120, the
+   the user's call and it is the first real fork since the profile was set:
-   format is fully specified (FINDINGS 30.2, costs 40.1), the executor is
+   spans only pay if the stream is allowed to run near the pipe (487.7 KB/s
-   written and pixel-exact in `blit.s`, and the encoder side is
+   delivered, 34/120 over budget) rather than at the 280 KB/s profile (77/120).
-   `prep_spans.py`'s v7 emitter. The container is `{u32 absolute GVRAM address,
+   That is a delivery-medium question — FINDINGS 32 dropped SASI on capacity and
-   u16 coarse displacement}` per span plus one `u16` fine displacement carried
+   parked the 110 KB/s point for CD-ROM, and 488 KB/s is 93% of a 4 Mbps figure
-   mid-stream — see FINDINGS 40.4 before changing that layout, the register
+   whose provenance is still unconfirmed (FINDINGS 29.5 item 3). **Do not spend
-   pressure is the reason for it.
+   another session optimising against a budget nobody has chosen.**
-2. **Then re-run `14_dmac_chain.py` and `13_cpu_ratectl.py` against a container
+2. **Re-derive the span selection jointly with lam, not after it.** The encoder
-   the encoder actually emits with spans in it.** Every span figure so far is
+   picks modes at one budget and then spans what is left, which FINDINGS 39.3
-   scored against mode maps chosen without spans available, which FINDINGS 39.3
+   already called a lower bound. A frame that misses its deadline would often do
-   flags as a lower bound on what a span-aware encoder would find.
+   better raising lam to free room for spans than lowering it — spans are
   pixel-exact, so the quality trade is not what it looks like. Bisecting a
   span reserve fraction inside the existing search is the tractable version.
-3. **Make sure the player actually gets DMA** — unchanged from session 10, and
+3. **Re-run `13_cpu_ratectl.py` against a DLX3 container.** 14 and 15 are done
-   still not an optimisation. Benchmark `x68000 -exp1 cz6bs1`, **never
+   (15 now counts span bus traffic and still reproduces the C68K measurement to
-   `x68ksupr`**; MAME's internal SCSI has no DMA glue (`// TODO: duplicate DMA
+   0.04%); 13 has not been re-run since the constant changed.
   glue from CZ-6BS1`) and would measure a PIO fallback the real machine does
   not have.
-4. **Re-decide the framerate.** 10 fps absorbs the DMA steal on current
+4. **Make sure the player actually gets DMA** — unchanged from sessions 10-11,
-   estimates. Still the user's call, and now cheaper to defer: v7 buys back
+   and still not an optimisation. Benchmark `x68000 -exp1 cz6bs1`, **never
-   enough of the budget that 12 fps is no longer obviously out of reach.
+   `x68ksupr`**; MAME's internal SCSI has no DMA glue and would measure a PIO
   fallback the real machine does not have. This is now more load-bearing, not
   less: the delivered stream is 487.7 KB/s and the disk debit is 163,798
   clocks a frame, 20% of the budget.
-5. **Re-run the ring-buffer simulation at the surviving rate** and confirm the
+5. **Re-run the ring-buffer simulation at the surviving rate** (FINDINGS
-   488 KB/s figure's provenance (FINDINGS 29.5/30.7, still open).
+   29.5/30.7, still open) and confirm the 488 KB/s figure's provenance.
-**Do not start by hand-optimising `decode.s`.** Unchanged and still true: the
+**Do not start by hand-optimising `decode.s`.** Unchanged and still true. The
-hand-derived timings agree with the measurements to 0.5% on V1 and 1% on RAW
+cycles to be won are in the budget, not the loop — and session 12 is the second
-(FINDINGS 28.4), 34 confirms the model on a second container, and the cycles to
+demonstration that the *model* of the budget is where the errors live.
 be won are in the budget, not the loop.
-**Always `stdbuf -oL` a MAME job that prints progress — and do not trust it.**
+**A new trap, worth reading before quoting any figure:** the rig had been
-Session 11 added the case where even that is not enough (40.6). If a run is not
+writing its synthetic timing frames 26 KB past the top of a 2 MB machine, and
-producing observable output, shrink the stimulus rather than waiting.
+got away with it because the modes it overran are data-independent. A span is
 not — its jump displacements come out of the stream. FINDINGS 41.4.
 ---
 ## What session 11 settled
 Session 11 measured v7 in `blit.s` and left it there; session 12 built it into
 the player. Items 0 and 1 of session 11's list are done (FINDINGS 40, 41) and
 the rest are carried forward in the list above.
 ## What session 10 settled
 Session 10 cross-checked the whole cycle model against a second emulator, then
@@ -38,6 +38,39 @@
 ; paid only by blocks that are NOT all-SKIP: a header byte of zero clears four
 ; blocks with one tst.b, and SKIP is the median block.
 ;
 ; LITERAL SPANS (v7, FINDINGS 40).  A run of horizontally adjacent dirty blocks
 ; is cheaper to paint as four ROW-LINEAR runs of word-expanded literal pixels
 ; than as blocks: 226 clocks per 4x4 block at a run of 4, against V1's 299.9,
 ; and the break-even is a run of 2.  The run's blocks read SKIP in the mode
 ; header and the span section paints them instead, so the block loop below is
 ; unchanged -- it sees a SKIP and advances, exactly as it does for a genuinely
 ; held block.
 ;
 ; The section sits BETWEEN the mode header and the block payload because that is
 ; the only place the 68000 can reach without first parsing something of variable
 ; length: the header is a fixed 768 bytes.  Per span the record is {u32 absolute
 ; GVRAM address, u16 coarse displacement}, then the coarse pixels, then a u16
 ; FINE displacement, then the fine pixels.
 ;
 ; The two displacements are jumps into two unrolled copy chains -- 24 pixels per
 ; coarse unit (a 12-register movem pair) and 2 per fine unit (one
 ; `move.l (a0)+,(a2)+`) -- so a span of any length is straight-line code with no
 ; loop, no remainder and no address arithmetic.  A run of 4x4 blocks is always a
 ; multiple of 4 pixels long, and 4 is a multiple of the 2-pixel fine quantum, so
 ; NOTHING is padded (FINDINGS 40.3).
 ;
 ; The fine displacement is in the STREAM rather than in the span record because
 ; that is what pays for the second dispatch: when the coarse chain falls out
 ; into `move.w (a0)+,d0 / jmp`, d0 is dead payload and a0 is already pointing at
 ; it, so the decoder holds nothing extra across the copy and keeps all twelve
 ; payload registers (FINDINGS 40.4).  Twelve is why the coarse unit is 24 pixels
 ; and not V5's 16, and it is the whole reason the per-pixel cost is 9.143 rather
 ; than 10.459 (FINDINGS 30.4).
 ;
 ; a1 (the mode header cursor) is one of those twelve, so it goes on the stack
 ; across the span pass.  Two long accesses per frame, against the 24 pixels a
 ; register buys per chain unit.
 ;
 ; ALIGNMENT.  Frame records are [u32 length][768-byte mode header][payload] laid
 ; end to end, and payload lengths are arbitrary -- so record boundaries land on
 ; odd addresses, and `move.l (a0)+,d0` on an odd address is an ADDRESS ERROR on
@@ -63,6 +96,10 @@ DSTE    = $C38000               ; GVRAM + 224*1024  (one past last)
 BROW    = 4096                  ; bytes per block row (4 picture rows)
 ROWLEN  = 512                   ; bytes per block row of blocks (64 * 8)
 MODEB   = 768                   ; packed mode header, 3072 blocks * 2 bits
 SPCU    = 12                    ; bytes of code per COARSE span unit (24 px)
 SPCN    = 11                    ; coarse units: 11*24 = 264 px >= one row
 SPFU    = 2                     ; bytes of code per FINE span unit (2 px)
 SPFN    = 11                    ; fine units: 11*2 = 22 px > one coarse unit
        org     $10000
 start:
@@ -75,7 +112,8 @@ frameloop:
        lea     0(a0,d0.l),a1
        move.l  a1,SCR_END.l            ; where the payload must end
        move.l  a0,a1                   ; a1 = packed mode header
-        lea     MODEB(a0),a0            ; a0 = payload
+        lea     MODEB(a0),a0            ; a0 = span section
        bsr     paint_spans             ; -> a0 = block payload, a1 preserved
        bsr     decode_frame
        cmpa.l  SCR_END.l,a0            ; bitstream desync is silent otherwise
        bne     desync
@@ -169,6 +207,81 @@ RAWPAIR macro
        move.l  d0,\1(a4)
        endm
 ; ------------------------------------------------------- the span section
 ; in:  a0 = span section, a1 = mode header (preserved across the call)
 ; out: a0 = one past the section, i.e. the block payload
 ;
 ; This is tools/bench/blit.s v7 verbatim, and deliberately so: the 66.0 clocks
 ; per span + 9.143 per coarse pixel + 9.978 per fine pixel of FINDINGS 40 were
 ; measured on exactly this instruction sequence, over thirteen span lengths, and
 ; a "tidier" rewrite here would silently invalidate every span figure in
 ; FINDINGS 39/40 and in tools/analysis/14_dmac_chain.py.
 ;
 ; The fine chain is entered by FALLING OUT of the coarse one, so a span with no
 ; coarse units enters at v7cx with d0 already reloaded -- which is why the
 ; coarse displacement for c=0 is SPCN*SPCU, one past the last coarse unit,
 ; rather than a special case.
 paint_spans:
        move.w  (a0)+,d7                ; spans in this frame
        subq.w  #1,d7
        bmi     spnone                  ; a frame may legitimately have none (the
                                        ; chain is far past a short branch)
        move.l  a1,-(sp)                ; a1 is a payload register below
 spspan: move.l  (a0)+,a2                ; absolute GVRAM destination
        move.w  (a0)+,d0                ; (SPCN - coarse) * SPCU
        jmp     spch(pc,d0.w)
 spch:
        movem.l (a0)+,d0-d6/a1/a3-a6
        movem.l d0-d6/a1/a3-a6,(a2)
        lea     48(a2),a2
        movem.l (a0)+,d0-d6/a1/a3-a6
        movem.l d0-d6/a1/a3-a6,(a2)
        lea     48(a2),a2
        movem.l (a0)+,d0-d6/a1/a3-a6
        movem.l d0-d6/a1/a3-a6,(a2)
        lea     48(a2),a2
        movem.l (a0)+,d0-d6/a1/a3-a6
        movem.l d0-d6/a1/a3-a6,(a2)
        lea     48(a2),a2
        movem.l (a0)+,d0-d6/a1/a3-a6
        movem.l d0-d6/a1/a3-a6,(a2)
        lea     48(a2),a2
        movem.l (a0)+,d0-d6/a1/a3-a6
        movem.l d0-d6/a1/a3-a6,(a2)
        lea     48(a2),a2
        movem.l (a0)+,d0-d6/a1/a3-a6
        movem.l d0-d6/a1/a3-a6,(a2)
        lea     48(a2),a2
        movem.l (a0)+,d0-d6/a1/a3-a6
        movem.l d0-d6/a1/a3-a6,(a2)
        lea     48(a2),a2
        movem.l (a0)+,d0-d6/a1/a3-a6
        movem.l d0-d6/a1/a3-a6,(a2)
        lea     48(a2),a2
        movem.l (a0)+,d0-d6/a1/a3-a6
        movem.l d0-d6/a1/a3-a6,(a2)
        lea     48(a2),a2
        movem.l (a0)+,d0-d6/a1/a3-a6
        movem.l d0-d6/a1/a3-a6,(a2)
        lea     48(a2),a2
 spcx:   move.w  (a0)+,d0                ; (SPFN - fine) * SPFU, from mid-stream
        jmp     spfh(pc,d0.w)
 spfh:
        move.l  (a0)+,(a2)+
        move.l  (a0)+,(a2)+
        move.l  (a0)+,(a2)+
        move.l  (a0)+,(a2)+
        move.l  (a0)+,(a2)+
        move.l  (a0)+,(a2)+
        move.l  (a0)+,(a2)+
        move.l  (a0)+,(a2)+
        move.l  (a0)+,(a2)+
        move.l  (a0)+,(a2)+
        move.l  (a0)+,(a2)+
        dbra    d7,spspan
        move.l  (sp)+,a1
 spnone: rts
 ; ------------------------------------------------------------- one frame
 ; in:  a0 = payload, a1 = packed mode header
 ; out: a0 = one past the last payload byte consumed
@@ -56,8 +56,11 @@ import buscost as B
 FRAME_CYC  = 833333.0
 AUDIO_KBPS = 7.8
-C_V1, C_V4, C_RAW = 299.9, 448.2, 400.4          # FINDINGS 28.2 (MEASURED)
+import vq_hybrid as _H
-C_SKIP_CLUSTERED, C_SKIP_MIXED = 13.25, 45.0
+C_V1, C_V4, C_RAW = _H.C_V1, _H.C_V4, _H.C_RAW   # FINDINGS 28.2 (MEASURED)
 # 45.0 until session 12 measured it at 55.0 (FINDINGS 41.5) -- imported now, so
 # the correction cannot be undone by a stale copy.
 C_SKIP_CLUSTERED, C_SKIP_MIXED = _H.C_SKIP_CLUSTERED, _H.C_SKIP_MIXED
 SPAN_BYTES_PX, SPAN_HDR = 2, 6
 ap = argparse.ArgumentParser()
@@ -25,8 +25,11 @@ A 68000 bus cycle is 4 clocks, so a frame of C clocks holds C/4 bus slots.
 """
 import sys, os, argparse, csv
 sys.path.insert(0, "tools/encoder")
 sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
 import numpy as np
 from dlx import DLX
 import buscost as B
 from buscost import V7_FRAME_PREF, V7_FRAME_DATA
 BUS_CLK = 4
@@ -90,6 +93,16 @@ for f in range(NF):
                pw, pd = BODY[b]
                pref += DISPATCH[b] + pw + SK_TAIL
                data += 1 + pd
    # The span section is bus traffic too, and it is most of the frame's data
    # accesses in a span-heavy container: 48 per 24-pixel chain unit. Leaving it
    # out would not merely understate the total -- it would break the CHECK
    # below, which is the whole licence for the prefetch figure.
    sp, _ = d.spans(f)
    if sp:
        pref += V7_FRAME_PREF; data += V7_FRAME_DATA
    for _, _, px in sp:
        sp_p, sp_d = B.v7_span_split(len(px))
        pref += sp_p; data += sp_d
    pref_t.append(pref); data_t.append(data)
    cyc_t.append(meas.get(f, (0, 0))[0])
@@ -0,0 +1,107 @@
 #!/usr/bin/env python3
 """GATE for the DLX3 span container: does the reference decoder reproduce the
 encoder's own reconstruction, from the emitted bytes?
    python3 tools/analysis/16_span_roundtrip.py [frames_dir] [--kbps 488]
 Exits non-zero if any frame differs by a single pixel.
 WHY THIS EXISTS SEPARATELY FROM 09.  `09_ratectl_drift.py` replays SKIP
 semantics in Python against the mode maps the encoder returned; it never reads
 a container.  A span breaks exactly that shortcut: a spanned block reads SKIP
 in the mode header and is painted by the span section instead, so a replay that
 knows only about mode maps reports drift where there is none, and -- far worse
 -- a container whose span section is malformed would still pass, because 09
 never parses one.  This gate closes that: encode, WRITE THE CONTAINER, read it
 back with tools/encoder/dlx.py (the byte-for-byte reference decoder the 68000
 is checked against), and compare to what ratectl recorded.
 It also has to prove it tested something.  A round-trip over a container with
 no spans in it is green by vacuity, which is the failure mode FINDINGS 40.6
 named for the snapshot count: a gate must take its expected work from the
 generated artefact, not from an assumption.  So the thresholds below are
 asserted, not printed.
 The `--kbps` default is the BUS rate, not the `scsi` profile's 280: spans are
 bought with bytes, and 14_dmac_chain.py scores them against the 488 KB/s pipe.
 At the profile rate the lam search has already spent the allowance and there is
 nothing left to buy a span with -- which is a real finding about the encoder
 (FINDINGS 41.2), not a reason for the gate to test nothing.
 """
 import argparse, os, pickle, sys, time
 sys.path.insert(0, "tools/encoder")
 import numpy as np
 import vq_hybrid as H, ratectl as RC, encode as E
 from dlx import DLX
 ap = argparse.ArgumentParser()
 ap.add_argument("frames_dir", nargs="?", default="tmp/fr_singe")
 ap.add_argument("--kbps", type=float, default=488.0)
 ap.add_argument("--out", default="tmp/s12_roundtrip")
 ap.add_argument("--cache", default=None)
 a = ap.parse_args()
 cache = a.cache or f"tmp/model_{os.path.basename(a.frames_dir.rstrip('/'))}.pkl"
 if os.path.exists(cache):
    m = pickle.load(open(cache, "rb"))
    print(f"model from {cache}")
 else:
    t = time.time()
    m = H.build(a.frames_dir, k1=256, k4=256, iters=16)
    pickle.dump(m, open(cache, "wb"))
    print(f"built model in {time.time()-t:.0f} s -> {cache}")
 bad = 0
 for span_mode in ("need", "all"):
    print(f"\n=== spans={span_mode}, {a.kbps:g} KB/s ===")
    m.pop("_sym", None)
    enc = RC.encode_rate_controlled(m, target_kbps=a.kbps, lam_lo=1.0,
                                    cycle_budget=RC.FRAME_CYCLES,
                                    span_mode=span_mode)
    recs = E.build_records(m, enc, span_mode)
    path = f"{a.out}_{span_mode}.dlx"
    total, vid, _ = E.write_container(path, m, recs, 12, m["k1"], m["k4"],
                                      span_mode)
    nsp = sum(len(x) for x in enc["spans"])
    nfr = sum(1 for x in enc["spans"] if x)
    px = sum(len(p) for x in enc["spans"] for _, _, p in x)
    print(f"{path}: {total:,} B, {len(recs)} frames, "
          f"{nsp:,} spans on {nfr} frames, {px:,} pixels painted by one "
          f"({100*px/(len(recs)*m['H']*m['W']):.1f}% of all pixels)")
    d = DLX(path)
    if d.version != 3:
        print(f"FAIL: container is DLX{d.version}, not DLX3"); bad += 1; continue
    # The decoder's own walk of the span section must land exactly where the
    # block payload starts, and blocks() already raises if the payload does not
    # consume the record -- so this reads the spans back through the same code
    # path the 68000 is modelled on rather than trusting the writer.
    got = d.decode_all()
    diff = np.array([(g != r).sum() for g, r in zip(got, enc["recon"])])
    print(f"pixels differing from the encoder's reconstruction: "
          f"{diff.sum()} total, worst frame {diff.max()}, "
          f"frames with any: {int((diff>0).sum())}/{len(diff)}")
    if diff.sum():
        f = int(np.argmax(diff))
        ys, xs = np.where(got[f] != enc["recon"][f])
        print(f"FAIL: frame {f} differs at {diff[f]} px, first (x={xs[0]}, "
              f"y={ys[0]}), block (bx={xs[0]//4}, by={ys[0]//4}), "
              f"mode there = {d.modes(f)[(ys[0]//4)*d.nbx + xs[0]//4]}")
        bad += 1
    # A green round-trip over a container with no spans in it proves nothing.
    if span_mode == "all":
        if nsp < 1000:
            print(f"FAIL: only {nsp} spans emitted -- this gate did not "
                  f"exercise the span path"); bad += 1
        if not (px and max(len(x) for x in enc["spans"]) > 50):
            print(f"FAIL: no frame carries a substantial span table"); bad += 1
 print()
 if bad:
    print(f"FAILED: {bad} check(s)")
    sys.exit(1)
 print("OK  the DLX3 span container round-trips: the reference decoder rebuilds "
      "the\n    encoder's reconstruction exactly, from the emitted bytes.")
@@ -0,0 +1,100 @@
 #!/usr/bin/env python3
 """What do the spans the ENCODER actually emitted cost, and what do they buy?
    python3 tools/analysis/17_span_delivered.py a.dlx [b.dlx ...] [--bus 488]
 Every span figure before this one -- FINDINGS 29 through 40, and
 tools/analysis/12 and 14 -- was scored by SIMULATING span selection over mode
 maps that were chosen without spans available.  FINDINGS 39.3 flagged that as a
 lower bound on what a span-aware encoder would find, and docs/STATUS.md's item 2
 asks for the figures to be re-run "against a container the encoder actually
 emits with spans in it".  This is that script: it reads the span section out of
 a DLX3 container and prices exactly those spans, with no selection model at all.
 THE MODEL IS 14_dmac_chain.py's, deliberately unchanged, so the columns are
 comparable:
    frame clocks = block decode + span painting + disk DMA
 additive, because a 68000 has no cache and a two-word prefetch queue and stalls
 the moment another master takes the bus (FINDINGS 38.3).  Block cost is
 vq_hybrid.cycles(), which reads a spanned block as SKIP -- correct, because the
 span section is what paints it, and its cost is the second term.
 The span term is the MEASURED v7 fit (FINDINGS 40), and as of session 12 that
 fit is confirmed inside src/player/decode.s itself rather than only in
 tools/bench/blit.s: the synthetic all-SPAN anchors of tools/bench/prep_dlx.py
 reproduce it to 0.23% on both emulators (FINDINGS 41.3).
 """
 import argparse, os, sys
 sys.path.insert(0, "tools/encoder")
 sys.path.insert(0, "tools/analysis")
 import numpy as np
 import vq_hybrid as H
 import spans as SP
 import buscost as B
 from dlx import DLX
 FRAME_CYC  = 833333.0
 AUDIO_KBPS = 7.8
 ap = argparse.ArgumentParser()
 ap.add_argument("containers", nargs="+")
 ap.add_argument("--bus", type=float, default=488.0, help="SCSI pipe, KB/s")
 ap.add_argument("--fps", type=float, default=12.0)
 ap.add_argument("--disk-clk-word", type=float, default=8.0,
                help="clocks the SCSI DMA steals per word (FINDINGS 39.7 "
                     "brackets it at 5..12; 8 is the midpoint)")
 a = ap.parse_args()
 def score(path):
    d = DLX(path)
    rows = []
    for f in range(d.nframes):
        mode = d.modes(f)
        sp, _ = d.spans(f)
        _, n = d.frames[f]
        blk = H.cycles(mode)
        spc = sum(SP.clocks(len(p)) for _, _, p in sp)
        disk = n / 2.0 * a.disk_clk_word
        rows.append((blk, spc, disk, n, len(sp),
                     sum(len(p) for _, _, p in sp)))
    return d, np.array(rows).T
 print(f"{'container':<34}{'KB/s':>8}{'spans':>9}{'span px':>9}"
      f"{'median':>9}{'worst':>9}{'over':>9}")
 print(f"{'':<34}{'':>8}{'/frame':>9}{'%':>9}"
      f"{'% frame':>9}{'% frame':>9}{'budget':>9}")
 for path in a.containers:
    if not os.path.exists(path):
        print(f"{path:<34}  missing"); continue
    d, r = score(path)
    blk, spc, disk, byt, nsp, spx = r
    tot = blk + spc + disk
    kbps = byt.mean() * a.fps / 1024 + AUDIO_KBPS
    print(f"{os.path.basename(path):<34}{kbps:>8.1f}{nsp.mean():>9.0f}"
          f"{100*spx.mean()/(d.W*d.H):>9.1f}"
          f"{100*np.median(tot)/FRAME_CYC:>9.1f}"
          f"{100*tot.max()/FRAME_CYC:>9.1f}"
          f"{int((tot > FRAME_CYC).sum()):>6}/{d.nframes:<3}")
 print(f"\n  ADDITIVE: frame = block decode + span painting + disk DMA, the model"
      f"\n  of 14_dmac_chain.py.  Disk debited at {a.disk_clk_word:g} clocks/word "
      f"over the\n  container's own byte count; CPU budget {FRAME_CYC:,.0f} "
      f"clocks at {a.fps:g} fps.")
 # The decomposition is the point: a span moves work out of the block loop and
 # into the span section, and it pays for it in bytes -- which the disk term
 # then charges back. A design that only counted the CPU would show a win that
 # the I/O it created takes away again (docs/FINDINGS.md 33).
 print(f"\nWHERE EACH FRAME'S CLOCKS GO, mean over the container")
 print(f"  {'container':<34}{'blocks':>12}{'spans':>12}{'disk':>12}{'total':>12}")
 for path in a.containers:
    if not os.path.exists(path):
        continue
    d, r = score(path)
    blk, spc, disk = r[0], r[1], r[2]
    print(f"  {os.path.basename(path):<34}{blk.mean():>12,.0f}{spc.mean():>12,.0f}"
          f"{disk.mean():>12,.0f}{(blk+spc+disk).mean():>12,.0f}")
@@ -154,8 +154,35 @@ def v7_span(npix):
 def v7_span_bus(npix):
    """Bus CYCLES a v7 span occupies -- instruction words plus data accesses."""
    p, d = v7_span_split(npix)
    return p + d
 def v7_span_split(npix):
    """(instruction words, data accesses) for one v7 span, separately.
    15_bus_occupancy.py needs the two apart, because the DATA half is what the
    C68K harness can check and the PREFETCH half is what rides on that check.
      per span    move.l (a0)+,a2   1 word + 2 reads
                  move.w (a0)+,d0   1 word + 1 read     (coarse displacement)
                  jmp    (pc,d0.w)  2 words
                  move.w (a0)+,d0   1 word + 1 read     (fine, from mid-stream)
                  jmp    (pc,d0.w)  2 words
                  dbra              2 words             -> 9 words, 4 accesses
      per coarse  2 movem.l of 12 + lea = 6 words, 24 reads + 24 writes
      per fine    move.l (a0)+,(a2)+ = 1 word, 2 reads + 2 writes
    """
    k, r = divmod(pad2(npix), V6_UNIT_PX)
-    return V7_SPAN_BUS + k * V6_UNIT_BUS + (r // V7_FINE_PX) * V7_FINE_BUS
+    f = r // V7_FINE_PX
    return (9 + k * 6 + f * 1,
            4 + k * 48 + f * 4)
 # Per FRAME, decode.s's paint_spans entry and exit: the span count read, the
 # guard branch, and the push/pop of a1 that buys back a twelfth payload
 # register.  Two long accesses a frame against 24 pixels a chain unit.
 V7_FRAME_PREF, V7_FRAME_DATA = 7, 7
 def v6_span_bus(npix):
@@ -40,6 +40,16 @@ python3 tools/analysis/09_ratectl_drift.py > tmp/drift_check.log 2>&1 \
  || { cat tmp/drift_check.log; exit 1; }
 tail -9 tmp/drift_check.log
 echo "--- session 12: the DLX3 span container round-trips (FINDINGS 41) ---"
 # 09 above replays SKIP semantics in Python and never reads a container. A v7
 # span breaks exactly that shortcut -- a spanned block reads SKIP in the mode
 # header and is painted by the span section instead -- so this encodes, WRITES
 # the container, reads it back with the reference decoder and compares. It also
 # asserts that it emitted enough spans to have tested anything.
 python3 tools/analysis/16_span_roundtrip.py > tmp/span_roundtrip.log 2>&1 \
  || { cat tmp/span_roundtrip.log; exit 1; }
 tail -4 tmp/span_roundtrip.log
 echo "--- session 7: display-path coherency (FINDINGS 28.1) ---"
 # 10_pathmix_drift.py is a COUNTEREXAMPLE, kept runnable: the dual-path plan of
 # FINDINGS 24.5/25.6 must still be shown to corrupt frames, and the strategy the
@@ -57,13 +67,15 @@ echo "--- session 7: 68000 decoder is pixel-exact (FINDINGS 28) ---"
 # 68000 code, every block mode, full temporal recursion. A SKIP block is a claim
 # about the previous frame still being on screen, so the last frame is only
 # right if all 120 were.
-# The gate container is the CURRENT default encode: scsi (the only profile left
+# The gate container is the HEAVIEST stream the encoder emits: the scsi mode
-# after session 9 dropped sasi on capacity, FINDINGS 32), cost-aware mode
+# decision (the only profile left after session 9 dropped sasi on capacity,
-# decision on, DLX2 4-byte-aligned records. It is also the heavier stream --
+# FINDINGS 32) with the span pass drawing on the full 488 KB/s pipe, so every
-# 43% RAW against sasi's 10% -- so it exercises the decoder harder than the
+# frame carries a span table and all four block modes are still exercised.
-# session-7 container this gate used to run on.
+# Spans are the newest and least-proven path in decode.s; gating on a container
-DLX=tmp/rc_fr_singe_scsi_cpufit.dlx
+# where they are rare would be gating on the old decoder. FINDINGS 41.
-[ -f "$DLX" ] || python3 tools/encoder/encode.py tmp/fr_singe "$DLX" --profile scsi
+DLX=tmp/rc_fr_singe_scsi_span.dlx
 [ -f "$DLX" ] || python3 tools/encoder/encode.py tmp/fr_singe "$DLX" --profile scsi \
     --kbps 280 --span-kbps 488 --spans all
 python3 tools/bench/prep_dlx.py "$DLX" > tmp/prep_dlx.log
 # The rig loads the whole stream into a 2 MB machine, so a scsi window does not
 # fit and prep_dlx truncates it. Verify against exactly the prefix it emitted.
@@ -83,7 +95,7 @@ rm -f tmp/snap_decode/x68000/*.png
 ( cd tmp && DLX_VERIFY_ONLY=1 SDL_VIDEODRIVER=dummy stdbuf -oL timeout -k 5 300 mame x68000 \
    -bios ipl10 -ramsize 2M -video soft -window -sound none -nothrottle -plugins \
    -autoboot_script ../tools/bench/decode.lua \
-    -snapshot_directory ./snap_decode -snapview native -seconds_to_run 45 \
+    -snapshot_directory ./snap_decode -snapview native -seconds_to_run 60 \
    > decode_check.log 2>&1 )
 # A truncated run must fail as a truncated run. Without this the only symptom is
 # a pixel diff against a half-drawn frame.
@@ -28,6 +28,7 @@ import sys, os, argparse
 sys.path.insert(0, "tools/encoder")
 import numpy as np
 from dlx import DLX
 import spans as SP
 # The harness loads the WHOLE container into emulated RAM at STREAM=0x30000 and
 # the target is a stock 2 MB machine, so there is a hard ceiling on how much of
@@ -54,6 +55,14 @@ a = ap.parse_args()
 d = DLX(a.container)
 if d.idx_bytes != 1:
    sys.exit("2-byte codebook indices: decode.s assumes 1 (k<=256)")
 # decode.s reads a u16 span count out of every frame record (FINDINGS 41), so a
 # DLX2 container is not merely span-less to it -- the first two bytes of the
 # block payload would be read as a count and the frame would decode as garbage.
 # Fail here rather than there.
 if not d.has_spans:
    sys.exit(f"{a.container} is DLX{d.version}: src/player/decode.s expects the "
             f"DLX3 span section. Re-encode (tools/encoder/encode.py emits DLX3 "
             f"by default) or pass --spans off and use an older decoder.")
 # --- codebooks, expanded to one WORD per pixel (high byte is discarded by
 #     gvram_w, so it is left zero and never has to be cleared)
@@ -72,6 +81,69 @@ palb = np.zeros((256, 2), np.uint8)
 palb[:, 0], palb[:, 1] = words >> 8, words & 0xFF
 dark = int(((render(I).astype(int)) ** 2).sum(1).argmin())
 def build_synth(d):
    """The synthetic timing frames, as record bodies.
    No real frame is all one mode, but the mix is exactly what the "76.6% x
    non-SKIP fraction" model of FINDINGS 24.5 assumes away: it prices every
    non-SKIP block as one V1-style burst.  These price the modes separately,
    which is the only way to see which one is expensive.
    Every record carries the DLX3 span section, empty or not -- decode.s reads a
    u16 count out of all of them, and a synthetic frame that omitted it would
    desync the bitstream exactly where the harness is least likely to look.
    The last two are the mode the block loop cannot express: a frame that is ALL
    SPAN, its mode header entirely SKIP.  Two run lengths, because a span costs
    per-span plus per-pixel and one length cannot separate them --
      all-SPAN-64  full-row runs, the floor of the mode (154 clocks/block,
                   FINDINGS 30.4)
      all-SPAN-4   4-block runs, the break-even against V1 (FINDINGS 40.1)
    They price v7 INSIDE decode.s against the constants tools/bench/span.sh
    fitted in blit.s.  Agreement cross-checks both; disagreement means the
    decoder's span pass is not the sequence that was measured.
    """
    out = {}
    empty = SP.serialise([])
    for name, mo, per in (("all-SKIP", 0, 0), ("all-V1", 1, 1),
                          ("all-V4", 2, 4), ("all-RAW", 3, 16)):
        out[name] = (bytes([mo * 0x55] * d.mode_bytes) + empty
                     + bytes(d.nb * per))
    # MIXED-SKIP frames.  Every other synthetic frame here is a pure population,
    # which is exactly why none of them prices the commonest block in a real
    # container: a SKIP that shares its header byte with a coded block, and so
    # cannot take the all-SKIP fast path.  vq_hybrid's C_SKIP_MIXED has never
    # been measured -- it was derived -- and a spanned container is made mostly
    # of them, because a spanned block reads SKIP.  FINDINGS 41.5.
    #
    # Two mixes per coded mode, because one equation cannot separate the SKIP
    # cost from the cost of the block it shares a group with.
    #
    # THE HEADER BYTES ROTATE, and that is not decoration.  decode.s reaches a
    # block's 2 mode bits with `lsr.b #6/#4/#2` and no shift at all for the last
    # one, so a block costs 52/48/44/34 clocks of dispatch depending on WHERE in
    # its header byte it sits.  A fixed byte like 0x01 puts every SKIP at the
    # three expensive positions and every V1 at the free one, and solving two
    # such equations returns a number that describes no real frame.  Cycling the
    # byte through the four rotations puts each mode at each position equally,
    # which is what a real mode map does.
    for nm, bys, per in (("mix-3SKIP-V1",  (0x01, 0x04, 0x10, 0x40), 1),
                         ("mix-1SKIP-3V1", (0x54, 0x51, 0x45, 0x15), 1),
                         ("mix-3SKIP-RAW", (0x03, 0x0C, 0x30, 0xC0), 16),
                         ("mix-1SKIP-3RAW", (0xFC, 0xF3, 0xCF, 0x3F), 16)):
        hdr = bytes(bys[i % 4] for i in range(d.mode_bytes))
        ncoded = sum(bin(b).count("1") and
                     sum(1 for k in range(4) if (b >> (2 * k)) & 3) for b in hdr[:1])
        ncoded = sum(sum(1 for k in range(4) if (b >> (2 * k)) & 3) for b in hdr)
        out[nm] = hdr + empty + bytes(ncoded * per)
    pat = np.tile(np.arange(d.W, dtype=np.uint8), (d.H, 1))
    for name, blocks in (("all-SPAN-64", d.W // 4), ("all-SPAN-4", 4)):
        sp = [(y, x, pat[y, x:x + blocks * 4])
              for y in range(d.H) for x in range(0, d.W, blocks * 4)]
        out[name] = bytes(d.mode_bytes) + SP.serialise(sp)
    return out
 # --- frame stream: [u32 len][modes][payload] per frame, each record start
 #     rounded up to a 4-byte boundary.
 #
@@ -85,6 +157,17 @@ dark = int(((render(I).astype(int)) ** 2).sum(1).argmin())
 budget = a.ram - STREAM_BASE - MARGIN
 stream, rec_off, pad = bytearray(), [], 0
 dropped = 0
 # The synthetic timing frames are built FIRST, so their size comes out of the
 # RAM budget rather than being appended past it.  It used to be appended: the
 # stream ran 26 KB beyond the top of a 2 MB machine, which was survivable only
 # because the modes it overran are data-independent -- their cost is in the
 # mode header, and reading junk payload costs the same as reading pixels.  A
 # span is not: its two jump DISPLACEMENTS come out of the stream, so an
 # out-of-RAM span record jumps into open bus.  FINDINGS 41.4.
 SYNTH = build_synth(d)
 budget -= sum(4 + len(b) + 3 for b in SYNTH.values())
 for (o, n) in d.frames:
    while len(stream) % 4:
        stream += b"\0"; pad += 1
@@ -101,21 +184,26 @@ if dropped:
          f"  This is the TEST RIG's limit, not the player's -- the player "
          f"streams into a ring buffer.")
-# Synthetic single-mode frames.  No real frame is all one mode, but the mix is
+# Append the synthetic frames the budget above already reserved.
 # exactly what the "76.6% x non-SKIP fraction" model of FINDINGS 24.5 assumes
 # away: it prices every non-SKIP block as one V1-style burst.  These four price
 # the modes separately, which is the only way to see which one is expensive.
 synth = {}
-for name, mo, per in (("all-SKIP", 0, 0), ("all-V1", 1, 1),
+for name, body in SYNTH.items():
                      ("all-V4", 2, 4), ("all-RAW", 3, 16)):
    while len(stream) % 4:
        stream += b"\0"; pad += 1
    synth[name] = len(stream)
-    hdr = bytes([mo * 0x55] * d.mode_bytes)
+    stream += len(body).to_bytes(4, "big") + body
-    stream += (d.mode_bytes + d.nb * per).to_bytes(4, "big") + hdr + bytes(d.nb * per)
+assert STREAM_BASE + len(stream) <= a.ram, (
    f"stream ends at 0x{STREAM_BASE+len(stream):X}, past the 0x{a.ram:X} top "
    f"of RAM -- the budget arithmetic above is wrong")
 # --- timing anchors: the distribution, not its mean (FINDINGS 25.6's lesson)
 #
 # A spanned block reads SKIP here, so this fraction is the BLOCK-LOOP workload
 # and no longer the frame's whole cost: the span section is the rest of it. The
 # anchors still pick out the extremes of the block loop, which is what they are
 # for, but a frame's total decode time now has two terms.
 ns = np.array([100 * (d.modes(i) != 0).mean() for i in range(NFRAMES)])
 nsp = np.array([len(d.spans(i)[0]) for i in range(NFRAMES)])
 spx = np.array([sum(len(p) for _, _, p in d.spans(i)[0]) for i in range(NFRAMES)])
 order = np.argsort(ns)
 pick = {
    "min non-SKIP  %.1f%%" % ns[order[0]]: int(order[0]),
@@ -124,9 +212,11 @@ pick = {
    "max non-SKIP  %.1f%%" % ns[order[-1]]: int(order[-1]),
 }
 anchors = [(n, rec_off[i], float(ns[i])) for n, i in pick.items()]
-for name in ("all-SKIP", "all-V1", "all-V4", "all-RAW"):
+for name in ("all-SKIP", "all-V1", "all-V4", "all-RAW",
             "all-SPAN-64", "all-SPAN-4", "mix-3SKIP-V1", "mix-1SKIP-3V1",
             "mix-3SKIP-RAW", "mix-1SKIP-3RAW"):
    anchors.append((f"synthetic {name}", synth[name],
-                    0.0 if name == "all-SKIP" else 100.0))
+                    0.0 if name.startswith(("all-SKIP", "all-SPAN")) else 100.0))
 blob = cb1.tobytes() + cb4.tobytes() + palb.tobytes() + bytes(stream)
 open(a.out + "_data.bin", "wb").write(blob)
@@ -147,6 +237,10 @@ print(f"  cb1 {cb1.nbytes} B + cb4 {cb4.nbytes} B expanded, palette {palb.nbytes
      f"stream {len(stream)} B  ->  {a.out}_data.bin ({len(blob)} B)")
 print(f"  non-SKIP blocks/frame: median {np.median(ns):.1f}%  "
      f"p90 {np.percentile(ns,90):.1f}%  max {ns.max():.1f}%")
 print(f"  v7 spans/frame: median {np.median(nsp):.0f}  max {nsp.max()}  "
      f"({int((nsp>0).sum())}/{NFRAMES} frames); pixels painted by one: "
      f"median {100*np.median(spx)/(d.W*d.H):.1f}%  "
      f"max {100*spx.max()/(d.W*d.H):.1f}% of the picture")
 print(f"  darkest palette entry: index {dark} -> {tuple(render(I)[dark])}")
 # A DLX2 container already carries this padding (FINDINGS 28.3 closed, session
 # 9), so the realignment above re-derives bytes that were already there and the
@@ -1,5 +1,5 @@
 #!/usr/bin/env python3
-"""Reference DLX1 reader/decoder -- the ground truth the 68000 player is checked against.
+"""Reference DLX reader/decoder -- the ground truth the 68000 player is checked against.
 This is deliberately a *decoder*, not a re-run of the encoder: it parses the
 container byte-for-byte the way `src/player/` must, so that any disagreement
@@ -12,9 +12,19 @@ Everything is big-endian (see the `encode.py` docstring).  Block raster order,
 V4 sub-block order is (sub_y, sub_x) row-major -- TL, TR, BL, BR -- matching
 `vq_hybrid.paint`'s reshape(-1,2,2,2,2).transpose(0,1,3,2,4).
 DLX3 adds the v7 LITERAL SPAN section between the mode header and the block
 payload (FINDINGS 40, tools/encoder/spans.py).  A spanned block reads SKIP in
 the mode header and is painted by a span instead, so a reader that ignores the
 section does not merely lose the spans -- it displays stale pixels wherever one
 was.  The section is at a KNOWN offset (header end) rather than behind the
 block payload precisely so that the 68000 can paint it before it has parsed
 anything of variable length.
 """
-import struct
+import os, sys, struct
 import numpy as np
 sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
 import spans as SP
 MODE_SKIP, MODE_V1, MODE_V4, MODE_RAW = 0, 1, 2, 3
@@ -28,10 +38,11 @@ class DLX:
        # (FINDINGS 28.3), so the padding is part of the format, not a loader
        # convenience -- but DLX1 containers stay readable, because every
        # measurement in FINDINGS 28-31 was taken on one.
-        if b[:4] not in (b"DLX1", b"DLX2"):
+        if b[:4] not in (b"DLX1", b"DLX2", b"DLX3"):
            raise ValueError(f"{path}: not a DLX container")
        self.version = int(b[3:4])
        self.aligned = self.version >= 2
        self.has_spans = self.version >= 3
        (self.W, self.H, self.fps, self.nframes,
         self.k1, self.k4) = struct.unpack(">HHHHHH", b[4:16])
        off_pal, off_cb1, off_cb4, off_frm = struct.unpack(">IIII", b[16:32])
@@ -72,6 +83,60 @@ class DLX:
        m = np.stack([(h >> 6) & 3, (h >> 4) & 3, (h >> 2) & 3, h & 3], axis=1)
        return m.reshape(-1)[:self.nb].copy()
    def spans(self, f):
        """The frame's literal spans: (list of (y, x, pixels), payload offset).
        Layout, big-endian, at the end of the mode header (DLX3 only):
            u16 nspans
            nspans * { u32 GVRAM address, u16 coarse disp, c*48 B pixels,
                       u16 fine disp,     f*4 B pixels }
        The displacements are JUMP offsets into the decoder's two unrolled copy
        chains, so the pixel counts are read back out of them -- which is the
        strongest available check that the encoder and `blit.s` agree about the
        chain geometry, because a wrong displacement lands mid-chain and paints
        the wrong number of pixels rather than failing loudly.
        """
        o, n = self.frames[f]
        p = o + self.mode_bytes
        if not self.has_spans:
            return [], p
        b = self.raw
        (ns,) = struct.unpack(">H", b[p:p + 2])
        p += 2
        out = []
        for _ in range(ns):
            addr, cd = struct.unpack(">IH", b[p:p + 6])
            p += 6
            c = SP.COARSE_N - cd // SP.COARSE_CODE
            if cd % SP.COARSE_CODE or not 0 <= c <= SP.COARSE_N:
                raise ValueError(f"frame {f}: coarse displacement {cd} is not "
                                 f"an entry point in an {SP.COARSE_N}-unit chain")
            px = list(np.frombuffer(b, ">u2", c * SP.COARSE_PX, p) & 0xFF)
            p += c * SP.COARSE_PX * 2
            (fd,) = struct.unpack(">H", b[p:p + 2])
            p += 2
            fu = SP.FINE_N - fd // SP.FINE_CODE
            if fd % SP.FINE_CODE or not 0 <= fu <= SP.FINE_N:
                raise ValueError(f"frame {f}: fine displacement {fd} is not "
                                 f"an entry point in a {SP.FINE_N}-unit chain")
            px += list(np.frombuffer(b, ">u2", fu * SP.FINE_PX, p) & 0xFF)
            p += fu * SP.FINE_PX * 2
            a = addr - SP.GVRAM
            y, x = divmod(a, SP.STRIDE)
            y -= SP.YOFF
            if x % 2 or not (0 <= y < self.H) or not (0 <= x // 2 < self.W):
                raise ValueError(f"frame {f}: span destination {addr:#x} is "
                                 f"not a pixel of the {self.W}x{self.H} picture")
            # blit.s tolerates a span running past the visible 256 pixels (the
            # line stride is 1024 bytes and only the first 512 are displayed),
            # but nothing an encoder emits should need to: a span is a run of
            # whole blocks.  numpy would truncate it here in silence.
            if x // 2 + len(px) > self.W:
                raise ValueError(f"frame {f}: span at ({x//2},{y}) of "
                                 f"{len(px)} px overruns the picture width")
            out.append((y, x // 2, np.array(px, np.uint8)))
        return out, p
    def blocks(self, f):
        """Decoded 4x4 palette-index blocks for the non-SKIP blocks of frame f.
@@ -82,7 +147,8 @@ class DLX:
        """
        mode = self.modes(f)
        o, n = self.frames[f]
-        p, end = o + self.mode_bytes, o + n
+        _, p = self.spans(f)
        end = o + n
        ib, out = self.idx_bytes, {}
        b = self.raw
        for i, mo in enumerate(mode):
@@ -115,6 +181,19 @@ class DLX:
        for i, blk in blks.items():
            by, bx = divmod(i, self.nbx)
            canvas[by * 4:by * 4 + 4, bx * 4:bx * 4 + 4] = blk
        sp, _ = self.spans(f)
        for y, x, pix in sp:
            # A span paints blocks the mode header calls SKIP.  If it ever
            # overlaps a coded block the two disagree about the same pixels and
            # the 68000's answer depends on which it does last -- so this is a
            # format invariant, not a courtesy check.
            b0, b1 = x // 4, -(-(x + len(pix)) // 4)
            bad = [b for b in range(b0, b1)
                   if mode[(y // 4) * self.nbx + b] != MODE_SKIP]
            if bad:
                raise ValueError(f"frame {f}: span at ({x},{y}) covers "
                                 f"non-SKIP block(s) {bad} of block row {y//4}")
            canvas[y, x:x + len(pix)] = pix
        return mode
    def decode_all(self):
@@ -30,8 +30,17 @@ multi-byte field is big-endian and the decoder can read it with a plain move.w):
  `move.l` -- FINDINGS 28.3):
    u32 payload length, then
    ceil(nblocks*2/8) bytes of 2-bit mode headers, MSB-first, block raster order
    DLX3 only: the v7 LITERAL SPAN section (tools/encoder/spans.py) --
      u16 nspans, then per span { u32 GVRAM address, u16 coarse displacement,
      c*48 B pixels, u16 fine displacement, f*4 B pixels }
    then payloads in block order: V1 -> 1 byte, V4 -> 4 bytes, RAW -> 16 bytes
  The span section is between the header and the block payload, not after it,
  because the 68000 has to reach it without first parsing something of variable
  length: the mode header is a fixed 768 bytes, so the section starts at a known
  offset and the block payload starts wherever the span walk finishes.  Every
  span record is a multiple of 4 bytes long, so nothing inside needs padding.
 Codebooks are emitted as palette INDICES, not pixels. The player expands them
 once at load time into word-per-pixel form so the blitter can movem them
 straight into GVRAM -- k1=1024 costs 1024*16*2 = 32 KB of the 2 MB.
@@ -39,7 +48,7 @@ straight into GVRAM -- k1=1024 costs 1024*16*2 = 32 KB of the 2 MB.
 import argparse, struct, sys, os
 sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
 import numpy as np
-import vq as VQ, vq_hybrid as H, ratectl as RC
+import vq as VQ, vq_hybrid as H, ratectl as RC, spans as SP
 # Measured on the emulated 68000, FINDINGS 24. Instruction cycles against
 # zero-wait-state memory, so these are floors, not hardware predictions.
@@ -81,81 +90,35 @@ def _idx(v):
 _IDX_BYTES = 1
-def main():
+def build_records(m, enc, span_mode):
-    global _IDX_BYTES
+    """The per-frame records of the container, in order.
    ap = argparse.ArgumentParser()
    ap.add_argument("frames_dir"); ap.add_argument("out")
    ap.add_argument("--profile", choices=list(RC.PROFILES), default="scsi")
    ap.add_argument("--lam", type=float, default=None)
    ap.add_argument("--fps", type=int, default=12)
    ap.add_argument("--iters", type=int, default=16)
    ap.add_argument("--fixed-lam", action="store_true",
                    help="disable rate control (session 5 behaviour)")
    ap.add_argument("--rc-floor", choices=("profile", "open"), default="profile",
                    help="quality floor for rate control")
    ap.add_argument("--bucket-frames", type=int, default=8,
                    help="leaky-bucket depth, in frame budgets")
    ap.add_argument("--no-cpu-fit", action="store_true",
                    help="drop the per-frame 68000 decode ceiling (session 7 "
                         "behaviour: 31%% of frames on hard content do not fit)")
    ap.add_argument("--prefill", type=float, default=0.0,
                    help="how full the player's buffer is assumed to be at "
                         "scene start, as a fraction of the bucket (0 = cold "
                         "buffer after a seek, the conservative assumption)")
    ap.add_argument("--preview")
    a = ap.parse_args()
-    prof = RC.PROFILES[a.profile]
+    The encoder hands back the symbols it actually chose. Re-deriving them here
-    lam = a.lam if a.lam is not None else prof["lam"]
+    (as session 5 did) is a second chance to disagree with the encoder, and with
-    k1, k4 = prof["k1"], prof["k4"]
+    per-frame rate control the mode map is no longer reproducible from a single
-    _IDX_BYTES = 1 if max(k1, k4) <= 256 else 2
+    lam anyway.
-    # An explicit --lam is a request for that lam, so it implies --fixed-lam.
+    Factored out of main() so tools/analysis/16_span_roundtrip.py can build the
-    rc = not (a.fixed_lam or a.lam is not None)
+    same bytes the shipping encoder does -- a round-trip gate that rebuilt the
-    lam_lo = lam if a.rc_floor == "profile" else 1.0
+    records itself would be testing its own copy of the format.
-    # The CPU ceiling is hardware, not taste: without it 31%% of frames on the
+    """
-    # worst sustained window do not decode in time on a stock 68000, and with
+    nbx = m["W"] // 4
-    # it that is one frame -- the intra frame -- for 0.26 dB. FINDINGS 31.
+    out = []
-    cyc_budget = None if a.no_cpu_fit else RC.FRAME_CYCLES
+    for f, im in enumerate(m["idx"]):
    print(f"profile {a.profile}: {prof['desc']}")
    if rc:
        print(f"  target {prof['kbps']} KB/s CEILING, rate-controlled: "
              f"lam bisected per frame in [{lam_lo:g}, {RC.LAM_CLIFF:g}], "
              f"{a.bucket_frames}-frame bucket")
        print(f"  CPU ceiling: " + (f"mu bisected per frame against "
              f"{RC.FRAME_CYCLES:,.0f} cycles (12fps, stock 68000)"
              if cyc_budget else "OFF (--no-cpu-fit)"))
    else:
        print(f"  target {prof['kbps']} KB/s, FIXED lam={lam} (no rate control)")
    print(f"  k1={k1} k4={k4}, {_IDX_BYTES}-byte indices")
    m = H.build(a.frames_dir, k1=k1, k4=k4, iters=a.iters)
    if rc:
        enc = RC.encode_rate_controlled(m, prof["kbps"], fps=a.fps,
                                        bucket_frames=a.bucket_frames,
                                        lam_lo=lam_lo, prefill=a.prefill,
                                        cycle_budget=cyc_budget)
    else:
        enc = H.encode(m, lam=lam)
    r = H.evaluate(m, enc, fps=a.fps)
    H_, W_ = m["H"], m["W"]; nbx = W_ // 4
    pal, idx = m["pal"], m["idx"]
    # The encoder hands back the symbols it actually chose. Re-deriving them
    # here (as session 5 did) is a second chance to disagree with the encoder,
    # and with per-frame rate control the mode map is no longer reproducible
    # from a single lam anyway.
    frames = []
    for f, im in enumerate(idx):
        mode = enc["modes"][f]
-        frames.append(pack_modes(mode)
+        sp = enc.get("spans", [[]] * len(m["idx"]))[f]
        rec = (pack_modes(mode)
               + (SP.serialise(sp) if span_mode else b"")
               + frame_payload(mode, enc["l1"][f], enc["l4g"][f], im, nbx))
        # the rate controller budgets exactly these bytes -- if that ever drifts
-        # from the container, every bitrate figure below is fiction
+        # from the container, every bitrate figure reported is fiction
-        assert len(frames[-1]) == enc["sizes"][f], (f, len(frames[-1]), enc["sizes"][f])
+        assert len(rec) == enc["sizes"][f], (f, len(rec), enc["sizes"][f])
        out.append(rec)
    return out
 def write_container(path, m, frames, fps, k1, k4, span_mode):
    """Write the whole container.  Returns (total bytes, video bytes, pad)."""
    palette = m["pal"][:256]
    if len(palette) < 256:
        palette = np.vstack([palette, np.zeros((256 - len(palette), 3), np.uint8)])
@@ -175,22 +138,119 @@ def main():
    # realigning at load time; the container now carries it.
    tbl_pad = -off_frm % 4
    off_frm += tbl_pad
-    hdr = (b"DLX2" + struct.pack(">HHHHHH", W_, H_, a.fps, len(idx), k1, k4)
+    hdr = ((b"DLX3" if span_mode else b"DLX2")
           + struct.pack(">HHHHHH", m["W"], m["H"], fps, len(frames), k1, k4)
           + struct.pack(">IIII", off_pal, off_cb1, off_cb4, off_frm))
    assert len(hdr) == 32, len(hdr)
    frm_pad = 0
-    with open(a.out, "wb") as fh:
+    with open(path, "wb") as fh:
        fh.write(hdr); fh.write(pal_b); fh.write(cb1_b); fh.write(cb4_b)
        fh.write(b"\0" * tbl_pad)
-        for i, p in enumerate(frames):
+        for i, rec in enumerate(frames):
-            fh.write(struct.pack(">I", len(p))); fh.write(p)
+            fh.write(struct.pack(">I", len(rec))); fh.write(rec)
            if i + 1 < len(frames):          # nothing follows the last record
-                n = -(4 + len(p)) % 4
+                n = -(4 + len(rec)) % 4
                fh.write(b"\0" * n); frm_pad += n
    total = os.path.getsize(path)
    return total, sum(len(r) + 4 for r in frames) + frm_pad, frm_pad
-    total = os.path.getsize(a.out)
+
-    vid = sum(len(p) + 4 for p in frames) + frm_pad
+def main():
    global _IDX_BYTES
    ap = argparse.ArgumentParser()
    ap.add_argument("frames_dir"); ap.add_argument("out")
    ap.add_argument("--profile", choices=list(RC.PROFILES), default="scsi")
    ap.add_argument("--lam", type=float, default=None)
    ap.add_argument("--fps", type=int, default=12)
    ap.add_argument("--iters", type=int, default=16)
    ap.add_argument("--fixed-lam", action="store_true",
                    help="disable rate control (session 5 behaviour)")
    ap.add_argument("--rc-floor", choices=("profile", "open"), default="profile",
                    help="quality floor for rate control")
    ap.add_argument("--bucket-frames", type=int, default=8,
                    help="leaky-bucket depth, in frame budgets")
    ap.add_argument("--kbps", type=float, default=None,
                    help="override the profile's bitrate CEILING. The profile "
                         "is a rate point on a delivery medium; this is for "
                         "asking what the codec does at another one -- e.g. "
                         "the 488 KB/s bus figure the span analyses of "
                         "FINDINGS 30/40 are scored against.")
    ap.add_argument("--span-kbps", type=float, default=None,
                    help="byte ceiling the SPAN pass may draw on, if it differs "
                         "from the profile's. The profile is a quality rate "
                         "point; the pipe is hardware. Bytes between the two "
                         "buy a better picture if spent on lam and the 68000's "
                         "deadline if spent on spans -- and nothing at all if "
                         "left unspent (FINDINGS 41.2). 488 is the bus figure "
                         "tools/analysis/14_dmac_chain.py scores against.")
    ap.add_argument("--spans", choices=("off", "need", "all"), default="need",
                    help="v7 literal spans (FINDINGS 40). `need` (default) "
                         "spends container bytes on spans only where a frame "
                         "misses the 68000's decode deadline; `all` spends "
                         "every profitable byte, which is the model "
                         "14_dmac_chain.py scores; `off` emits DLX2.")
    ap.add_argument("--no-cpu-fit", action="store_true",
                    help="drop the per-frame 68000 decode ceiling (session 7 "
                         "behaviour: 31%% of frames on hard content do not fit)")
    ap.add_argument("--prefill", type=float, default=0.0,
                    help="how full the player's buffer is assumed to be at "
                         "scene start, as a fraction of the bucket (0 = cold "
                         "buffer after a seek, the conservative assumption)")
    ap.add_argument("--preview")
    a = ap.parse_args()
    prof = dict(RC.PROFILES[a.profile])
    if a.kbps is not None:
        prof["kbps"] = a.kbps
        prof["desc"] = f"{prof['desc']} -- bitrate overridden to {a.kbps:g} KB/s"
    lam = a.lam if a.lam is not None else prof["lam"]
    k1, k4 = prof["k1"], prof["k4"]
    _IDX_BYTES = 1 if max(k1, k4) <= 256 else 2
    # An explicit --lam is a request for that lam, so it implies --fixed-lam.
    rc = not (a.fixed_lam or a.lam is not None)
    lam_lo = lam if a.rc_floor == "profile" else 1.0
    # The CPU ceiling is hardware, not taste: without it 31%% of frames on the
    # worst sustained window do not decode in time on a stock 68000, and with
    # it that is one frame -- the intra frame -- for 0.26 dB. FINDINGS 31.
    cyc_budget = None if a.no_cpu_fit else RC.FRAME_CYCLES
    span_mode = None if (a.spans == "off" or not rc) else a.spans
    print(f"profile {a.profile}: {prof['desc']}")
    if rc:
        print(f"  target {prof['kbps']} KB/s CEILING, rate-controlled: "
              f"lam bisected per frame in [{lam_lo:g}, {RC.LAM_CLIFF:g}], "
              f"{a.bucket_frames}-frame bucket")
        print(f"  CPU ceiling: " + (f"mu bisected per frame against "
              f"{RC.FRAME_CYCLES:,.0f} cycles (12fps, stock 68000)"
              if cyc_budget else "OFF (--no-cpu-fit)"))
    else:
        print(f"  target {prof['kbps']} KB/s, FIXED lam={lam} (no rate control)")
    print(f"  k1={k1} k4={k4}, {_IDX_BYTES}-byte indices")
    m = H.build(a.frames_dir, k1=k1, k4=k4, iters=a.iters)
    if rc:
        enc = RC.encode_rate_controlled(m, prof["kbps"], fps=a.fps,
                                        bucket_frames=a.bucket_frames,
                                        lam_lo=lam_lo, prefill=a.prefill,
                                        cycle_budget=cyc_budget,
                                        span_mode=span_mode,
                                        span_kbps=a.span_kbps)
    else:
        # Spans are a rate-control-era mode: `need` has no meaning without a
        # per-frame byte allowance to spend, so --fixed-lam emits DLX2.
        enc = H.encode(m, lam=lam)
    r = H.evaluate(m, enc, fps=a.fps)
    H_, W_ = m["H"], m["W"]; nbx = W_ // 4
    pal, idx = m["pal"], m["idx"]
    frames = build_records(m, enc, span_mode)
    nspans = sum(len(x) for x in enc.get("spans", []))
    total, vid, frm_pad = write_container(a.out, m, frames, a.fps, k1, k4,
                                          span_mode)
    print(f"  wrote {a.out}: {total} B  "
          f"(header+tables {total-vid} B, video {vid} B)")
    print(f"  DLX2 4-byte record alignment: {frm_pad} B over {len(frames)} frames "
@@ -201,6 +261,20 @@ def main():
          f"loss {r['loss']:.2f} dB")
    print(f"  modes: SKIP {r['skip']:.1f}%  V1 {r['v1']:.1f}%  "
          f"V4 {r['v4']:.1f}%  RAW {r['raw']:.1f}%")
    if span_mode:
        spf = np.array([len(x) for x in enc["spans"]])
        spb = np.array([SP.section_bytes(x) for x in enc["spans"]])
        # a run of L blocks is four spans of 4L pixels, so a block is 16 span
        # pixels -- not 4, which would count each block four times over
        blk = np.array([sum(len(p) for _, _, p in x) // 16 for x in enc["spans"]])
        print(f"  v7 spans ({span_mode}): {nspans:,} over {len(idx)} frames, "
              f"median {np.median(spf):.0f}/frame, max {spf.max()}/frame; "
              f"{100*np.mean(spb)/np.mean([len(p) for p in frames]):.1f}% of the "
              f"container")
        print(f"    frames with any span: {int((spf>0).sum())}/{len(idx)}; "
              f"blocks painted by one: median {np.median(blk):.0f}, "
              f"max {blk.max()} of {m['nb']} "
              f"({100*blk.max()/m['nb']:.1f}%)")
    if rc:
        rr = RC.summarise(m, enc, prof["kbps"], fps=a.fps)
        lm = enc["lam"]
@@ -225,7 +299,12 @@ def main():
    # begin with, because the compose path pays the blit ON TOP of decoding.
    # FINDINGS 28.1/28.4. The player has one path and no reference frame.
    ns = np.array([100 * (mm != 0).mean() for mm in enc["modes"]])
-    cyc = np.array([H.cycles(mm) for mm in enc["modes"]])
+    # enc["cycles"] already carries the span PAINTING clocks; H.cycles() sees
    # only the mode map, in which a spanned block reads SKIP, so re-deriving
    # here would report a frame as fitting on the strength of work the encoder
    # moved into the span section rather than removed.
    cyc = (np.asarray(enc["cycles"]) if "cycles" in enc
           else np.array([H.cycles(mm) for mm in enc["modes"]]))
    pct = 100 * cyc / RC.FRAME_CYCLES
    miss = int((pct > 100).sum())
    print(f"  non-SKIP blocks/frame: median {np.median(ns):.1f}%  "
@@ -21,6 +21,7 @@ fixed by tuning. Regression test: tools/analysis/09_ratectl_drift.py.
 """
 import numpy as np
 import vq_hybrid as H
 import spans as SP
 # Profiles. Bandwidths are the sustained-read figures the player can rely on;
 # see docs/FINDINGS.md 5 -- these are FOLKLORE-grade until the disk benchmark
@@ -192,9 +193,51 @@ def _search_mu(ctx, allow, lam_lo, lam_hi, cyc_budget, iters=10):
    return (*best, False)
 def _fit_spans(m, ctx, mode, sz, room, cyc_budget, span_mode, ib):
    """Buy 68000 cycles with container bytes, by painting runs as v7 spans.
    Returns (mode, size, cycles, sel) where `sel` is spans.select()'s result.
    ORDER MATTERS, and it is the reason this runs before the mu search rather
    than inside it.  Both controllers make a frame decode in time, but they pay
    for it differently: mu buys cycles with QUALITY (it pushes blocks down to
    cheaper modes and ultimately to SKIP), and a span buys them with BYTES --
    and it carries literal source pixels, so it *removes* that run's
    quantisation error.  Spending bytes we already have is strictly better than
    spending picture, so spans go first and mu is what is left when the byte
    allowance runs out.
    `span_mode` is "need" (stop as soon as the frame fits its cycle budget --
    the default, and the cheapest way to make the deadline) or "all" (spend
    every profitable byte, which is the model tools/analysis/14_dmac_chain.py
    scores and costs several times the bitrate for a little more headroom).
    `room` is a byte ceiling for the WHOLE frame, and it is not necessarily the
    same one the lam search ran under.  Those are two different budgets and
    conflating them is what made the first measured span encode look like a
    regression (FINDINGS 41.2): the profile's bitrate is a chosen quality rate
    point, while the pipe is a hardware ceiling, and bytes left between them
    buy nothing if they are not spent.  Spending them on lam gets a better
    picture; spending them on spans gets the deadline.  `--span-kbps` picks.
    """
    src = m["idx"][ctx["f"]]
    room = room - sz - 2             # the u16 span count is always emitted
    if room <= 0:
        return mode, sz, H.cycles(mode), None
    sel = SP.select(mode, src, m["nbx"], m["nby"], room,
                    need_clocks=(None if span_mode == "all" else cyc_budget),
                    idx_bytes=ib)
    if not sel["spans"]:
        return mode, sz, H.cycles(mode), None
    nmode = sel["mode"]
    nsz = (H.frame_bytes(nmode, ctx["nb"], ib) + SP.section_bytes(sel["spans"]))
    return nmode, nsz, H.cycles(nmode) + sel["clocks"], sel
 def encode_rate_controlled(m, target_kbps, fps=12, bucket_frames=8,
                           lam_lo=1.0, lam_hi=LAM_CLIFF, prefill=0.0,
-                           steps=None, verbose=False, cycle_budget=None):
+                           steps=None, verbose=False, cycle_budget=None,
                           span_mode=None, span_kbps=None):
    """Per-frame lam search under a leaky bucket, driving the encoder ONE FRAME
    AT A TIME and feeding back the frame actually emitted.
@@ -234,32 +277,71 @@ def encode_rate_controlled(m, target_kbps, fps=12, bucket_frames=8,
    if steps is not None and verbose:
        print("  note: `steps` is ignored; lam is now bisected per frame")
    budget = frame_budget(target_kbps, fps)
    span_budget = None if span_kbps is None else frame_budget(span_kbps, fps)
    cap = bucket_frames * budget
    bucket = prefill * cap    # banked bytes; bounded by the player's buffer both ways
    out = dict(recon=[], modes=[], sizes=[], lam=[], l1=[], l4g=[], overrun=[],
-               mu=[], cycles=[], late=[])
+               mu=[], cycles=[], late=[], spans=[])
    ib = H.default_idx_bytes(m)
    prev = None
    for f in range(len(m["idx"])):
        ctx = H.frame_ctx(m, f, prev)
        allow = budget + bucket
-        if cycle_budget is None:
+        # The span pass may draw on a DIFFERENT ceiling: flat per frame, not
        # banked, because it is the delivery pipe rather than a quality target
        # and a pipe cannot be saved up. None means "the same allowance the lam
        # search had", which is what leaves spans nothing to buy with at a rate
        # point the block coder has already spent (FINDINGS 41.2).
        span_allow = allow if span_budget is None else span_budget
        sel = None
        lam, mode, sz, ovr = _search_lam(ctx, allow, lam_lo, lam_hi)
        mu, cyc, late = 0.0, H.cycles(mode), False
-        else:
+        if span_mode and (span_mode == "all"
                          or (cycle_budget is not None and cyc > cycle_budget)):
            mode_pre = mode
            mode, sz, cyc, sel = _fit_spans(m, ctx, mode, sz, span_allow,
                                            cycle_budget, span_mode, ib)
        if cycle_budget is not None and cyc > cycle_budget:
            # The byte allowance could not buy the frame's deadline, so fall
            # back to the controller that pays in picture -- and then offer
            # spans the bytes the smaller mode map just freed.
            mu, lam, mode, sz, cyc, ovr, late = _search_mu(
                ctx, allow, lam_lo, lam_hi, cycle_budget)
-        rec = H.paint(m, ctx, mode)
+            if span_mode:
-        bucket = float(np.clip(bucket + budget - sz, -cap, cap))
+                mode_pre = mode
                mode, sz, cyc, sel = _fit_spans(m, ctx, mode, sz, span_allow,
                                                cycle_budget, span_mode, ib)
                late = cyc > cycle_budget
        # Paint from the mode map as it was BEFORE spanning.  A spanned run's
        # blocks read SKIP in the emitted header, but SKIP means "hold the
        # previous reconstruction" and on the first frame there is none -- and
        # more generally the held pixels would be wrong.  The span overwrites
        # exactly the run it covers (4 rows x 4L pixels = the blocks), so
        # painting the pre-span modes and then laying the spans over them is
        # what the 68000 produces, and it is defined on frame 0.
        if span_mode and sel is None:
            sz += 2          # the u16 span count is in every DLX3 frame record
        # What the quality bucket banks is the BLOCK payload. Charging it the
        # span bytes too would drive it to its floor on the first spanned frame
        # and starve every later frame of quality for a budget the spans were
        # never drawing on.
        sz_quality = sz if (sel is None or span_budget is None) else sz - sel["bytes"]
        rec = H.paint(m, ctx, mode if sel is None else mode_pre)
        if sel is not None:
            for y, x, pix in sel["spans"]:
                rec[y, x:x + len(pix)] = pix
        bucket = float(np.clip(bucket + budget - sz_quality, -cap, cap))
        out["recon"].append(rec); out["modes"].append(mode)
        out["sizes"].append(sz); out["lam"].append(lam); out["overrun"].append(ovr)
        out["mu"].append(mu); out["cycles"].append(cyc); out["late"].append(late)
        out["l1"].append(ctx["sym"]["l1"]); out["l4g"].append(ctx["sym"]["l4g"])
        out["spans"].append([] if sel is None else sel["spans"])
        prev = rec
        if verbose:
            print(f"  f{f:04d} lam={lam:8.2f} mu={mu:8.4f} {sz:7.0f} B "
                  f"(allow {allow:7.0f}) {100*cyc/FRAME_CYCLES:5.1f}% cpu"
                  f"{' OVER' if ovr else ''}{' LATE' if late else ''}")
-    return dict(recon=out["recon"], modes=out["modes"],
+    return dict(recon=out["recon"], modes=out["modes"], spans=out["spans"],
                sizes=np.array(out["sizes"]), lam=np.array(out["lam"]),
                l1=out["l1"], l4g=out["l4g"], overrun=np.array(out["overrun"]),
                mu=np.array(out["mu"]), cycles=np.array(out["cycles"]),
@@ -0,0 +1,216 @@
 #!/usr/bin/env python3
 """v7 literal spans: geometry, selection, and the bytes that go in the container.
 A span is a ROW-LINEAR run of word-expanded literal pixels that the 68000
 copies straight from the stream buffer into GVRAM through an unrolled chain of
 `movem.l` units, with no address arithmetic, no loop and no remainder logic.
 It is the mode FINDINGS 29 derived, FINDINGS 30 measured as v6, and FINDINGS 40
 re-measured as v7 -- v6's 24-pixel coarse chain with a 2-pixel fine chain
 appended, at
    66.0 clocks/span + 9.143/coarse pixel + 9.978/fine pixel   (MEASURED)
 The span constants live in tools/analysis/buscost.py and the per-block ones in
 tools/encoder/vq_hybrid.py; both are imported rather than copied, which is what
 kept session 12's correction to C_SKIP_MIXED from having to be made twice.
 WHAT A SPAN COVERS.  A run of L horizontally adjacent 4x4 blocks inside one
 block row, coded as FOUR spans of 4L pixels -- one per picture row.  The run's
 blocks are marked SKIP in the mode header and the span paints them instead, so
 a span costs the mode-map dispatch but not the block body.  That is exactly the
 accounting tools/analysis/14_dmac_chain.py scores.
 WHY THE PADDING IS ZERO.  v7's fine unit is one `move.l (a0)+,(a2)+` = 2
 pixels, and a span is a run of 4x4 blocks, so its length is always a multiple
 of 4 and splits into 24*c + 2*f with nothing left over (FINDINGS 40.3).  v6's
 24-pixel quantum wasted ~11 pixels a span and was 86% of the DMAC's advantage
 over it.
 SPANS ARE LITERAL, SO THEY ARE PIXEL-EXACT.  A span carries palette indices
 straight out of the palettised source, exactly as a RAW block does.  Spanning a
 run therefore does not just buy cycles, it removes that run's quantisation
 error -- which is why the selection below can only improve PSNR, and why the
 reconstruction the encoder feeds back to the next frame has to include spans
 (a temporally recursive codec drifts otherwise -- FINDINGS 26.1).
 """
 import os, sys
 import numpy as np
 sys.path.insert(0, os.path.join(os.path.dirname(os.path.abspath(__file__)),
                                "..", "analysis"))
 import buscost as B
 # Display geometry, and it must match tools/bench/crtc_mode.lua: 256-colour
 # page, one pixel per WORD of CPU address space, 1024-byte line stride, picture
 # in rows 32..223 of a 256-row page.  GVRAM is at a fixed $C00000 on every
 # X68000, which is what makes an absolute destination address a legitimate
 # thing for an encoder to bake into a stream (FINDINGS 30.2).
 GVRAM, YOFF, STRIDE = 0xC00000, 32, 1024
 # The two chains, and these must match tools/bench/blit.s v7 exactly.
 COARSE_PX, COARSE_CODE, COARSE_N = 24, 12, 11
 FINE_PX,   FINE_CODE,   FINE_N   = 2,  2,  11
 SPAN_HDR   = B.V7_SPAN_HDR        # {u32 address, u16 coarse disp} + u16 fine
 BYTES_PX   = 2                    # word-expanded, high byte discarded by gvram_w
 def split(npix):
    """(coarse units, fine units) for a span of npix pixels.  Exact: npix is a
    multiple of 4 for any real span, and 4 is a multiple of the 2-pixel fine
    quantum, so nothing is padded."""
    if npix % FINE_PX:
        raise ValueError(f"span of {npix} px is not a multiple of {FINE_PX}")
    c, r = divmod(npix, COARSE_PX)
    f = r // FINE_PX
    if c > COARSE_N or f > FINE_N:
        raise ValueError(f"span of {npix} px exceeds the chain "
                         f"({c} coarse > {COARSE_N} or {f} fine > {FINE_N})")
    return c, f
 def clocks(npix):
    """68000 clocks to paint one span of npix pixels (MEASURED, FINDINGS 40)."""
    c, f = split(npix)
    return (B.V7_SPAN_CYC + c * COARSE_PX * B.V7_CPX_CYC
            + f * FINE_PX * B.V7_FPX_CYC)
 def run_clocks(L):
    """Clocks for a run of L blocks: four spans of 4L pixels."""
    return 4.0 * clocks(4 * L)
 def run_bytes(L):
    """Container bytes for a run of L blocks."""
    return 4 * (SPAN_HDR + 4 * L * BYTES_PX)
 def dest(y, x):
    """Absolute GVRAM address of picture pixel (x, y)."""
    return GVRAM + (YOFF + y) * STRIDE + x * 2
 def dirty_runs(mode2d, nbx):
    """Maximal runs of horizontally adjacent non-SKIP blocks, per block row."""
    for by in range(mode2d.shape[0]):
        d = mode2d[by] != 0
        i = 0
        while i < nbx:
            if not d[i]:
                i += 1
                continue
            j = i
            while j < nbx and d[j]:
                j += 1
            yield by, i, j
            i = j
 # Per-block decode cost, the same measured table vq_hybrid.cycles() uses --
 # imported rather than copied, because session 12 corrected one of them and a
 # second copy is how a corrected constant stops being corrected everywhere.
 import vq_hybrid as _H
 C_SKIP_MIXED = _H.C_SKIP_MIXED    # a spanned block still pays its dispatch
 BLK_CLK = {1: _H.C_V1, 2: _H.C_V4, 3: _H.C_RAW}
 BLK_BYT = {1: 1, 2: 4, 3: 16}
 def select(mode, src_idx, nbx, nby, byte_room, need_clocks=None,
           idx_bytes=1):
    """Choose which runs to paint as spans.
    `mode`        1-D mode map, modified nowhere (a new one is returned)
    `src_idx`     (H, W) palettised source -- what the spans will carry
    `byte_room`   container bytes the frame may still spend
    `need_clocks` stop as soon as the frame's decode cost is at or below this;
                  None spends every profitable byte instead (the model
                  tools/analysis/14_dmac_chain.py scores).
    Ranked by clocks saved per byte spent, which is the same greedy 12 and 14
    use.  Selection is deliberately conservative in two ways and the reported
    figures are exact rather than greedy: a run is only offered if the span
    beats the blocks it replaces on cycles ALONE, and the saving credited here
    ignores the extra all-SKIP header bytes spanning tends to create.  The
    caller recomputes the frame's real cost from the returned mode map.
    Returns dict(mode, spanned, spans, bytes, clocks).
    """
    m2 = np.asarray(mode).reshape(nby, nbx)
    spanned = np.zeros((nby, nbx), bool)
    cand = []
    for by, i, j in dirty_runs(m2, nbx):
        L = j - i
        cur_c = sum(BLK_CLK[int(b)] for b in m2[by][i:j])
        cur_b = sum(BLK_BYT[int(b)] * (idx_bytes if int(b) != 3 else 1)
                    for b in m2[by][i:j])
        sc = run_clocks(L) + L * C_SKIP_MIXED     # the dispatch still happens
        if sc >= cur_c:
            continue
        db = run_bytes(L) - cur_b
        cand.append(((cur_c - sc) / max(db, 1), cur_c - sc, db, by, i, j))
    cand.sort(key=lambda s: -s[0])
    # `need_clocks` is measured against the frame as it stands, so the loop
    # tracks the real running total rather than a delta: a spanned run's blocks
    # become SKIP, and four SKIPs sharing a header byte cost 53 cycles instead
    # of 4x55, which the greedy's per-run delta does not see.
    import vq_hybrid as H
    cur = m2.copy()
    total_b, total_c = 0.0, 0.0
    chosen = []
    for _, dc, db, by, i, j in cand:
        if need_clocks is not None and H.cycles(cur) + total_c <= need_clocks:
            break
        if total_b + db > byte_room:
            continue
        total_b += db
        total_c += run_clocks(j - i)
        cur[by][i:j] = 0
        spanned[by][i:j] = True
        chosen.append((by, i, j))
    spans = []
    for by, i, j in sorted(chosen):
        x, npix = i * 4, (j - i) * 4
        for k in range(4):
            y = by * 4 + k
            spans.append((y, x, src_idx[y, x:x + npix].astype(np.uint8)))
    spans.sort()
    return dict(mode=cur.reshape(-1), spanned=spanned, spans=spans,
                bytes=int(total_b), clocks=float(total_c))
 def serialise(spans):
    """The span section of a frame record, exactly as blit.s v7 reads it.
        u16 nspans
        nspans * { u32 GVRAM address, u16 coarse disp, c*48 B pixels,
                   u16 fine disp,     f*4 B pixels }
    The fine displacement sits MID-STREAM rather than in the record because
    that is what lets the decoder keep all 12 payload registers: the coarse
    chain falls out into `move.w (a0)+,d0 / jmp` with d0 dead payload and a0
    already pointing at it (FINDINGS 40.4).
    Every field is big-endian and every span record is a multiple of 4 bytes
    long (4 + 2 + 48c + 2 + 4f), so the section needs no internal padding.
    """
    out = bytearray()
    out += len(spans).to_bytes(2, "big")
    for y, x, pix in spans:
        c, f = split(len(pix))
        w = np.zeros((len(pix), 2), np.uint8)
        w[:, 1] = pix                       # high byte discarded by gvram_w
        w = w.tobytes()
        out += dest(y, x).to_bytes(4, "big")
        out += ((COARSE_N - c) * COARSE_CODE).to_bytes(2, "big")
        out += w[:c * COARSE_PX * 2]
        out += ((FINE_N - f) * FINE_CODE).to_bytes(2, "big")
        out += w[c * COARSE_PX * 2:]
    return bytes(out)
 def section_bytes(spans):
    return 2 + sum(SPAN_HDR + len(p) * BYTES_PX for _, _, p in spans)
@@ -61,7 +61,30 @@ RAW_BYTES = 16.0                  # literal palette bytes, never indices
 #     mode decision uses the ranking constant.
 C_V1, C_V4, C_RAW = 299.9, 448.2, 400.4
 C_SKIP_CLUSTERED = 53.0 / 4       # all-SKIP header byte: one tst.b for four
-C_SKIP_MIXED = 45.0               # a SKIP block inside a mixed byte
+# C_SKIP_MIXED WAS THE ONE CONSTANT HERE THAT HAD NEVER BEEN MEASURED. It was
 # 45.0, hand-derived, from session 7 until session 12 measured it -- and it was
 # 18% low. Every other figure in this table comes from a synthetic frame of a
 # single mode, and there was no such frame for a SKIP in a MIXED byte, because
 # a frame of nothing but mixed SKIPs cannot exist: the byte has to hold a coded
 # block for the SKIP to be mixed at all.
 #
 # tools/bench/prep_dlx.py now emits four that bracket it -- (3 SKIP + 1 V1),
 # (1 SKIP + 3 V1), (3 SKIP + 1 RAW), (1 SKIP + 3 RAW), each with the header byte
 # ROTATED through all four positions so no mode is pinned to the free `lsr`
 # slot -- and each pair solves for the SKIP cost and its partner's together:
 #
 #            MAME     C68K        (the partner solves back to its own anchored
 #   V1 pair  55.03    56.50        value to 0.2%, which is what says the pair
 #   RAW pair 55.83    56.50        is measuring the SKIP and not absorbing it)
 #
 # 55.0 is taken because every other constant here is MAME's; C68K reads V4 and
 # RAW 3.2-3.5% higher on pure frames too, which is FINDINGS 37's known table
 # spread and not a property of mixed bytes.
 #
 # It matters more than 10 clocks a block sounds, because a v7 SPAN marks its
 # run SKIP: a spanned container is made largely of mixed SKIPs, so this is the
 # dominant population in exactly the frames spans are judged on. FINDINGS 41.5.
 C_SKIP_MIXED = 55.0               # a SKIP block inside a mixed byte, MEASURED
 C_SKIP_RANK = C_SKIP_CLUSTERED    # ranking only -- see above
 MODE_CYCLES = np.array([C_SKIP_RANK, C_V1, C_V4, C_RAW], dtype=np.float64)