Take the project's own story out of the README, starting with a claim about elapsed time

USER CORRECTION: the README is a description, not a chronicle, and the intro
had it narrating an arc -- "for most of this project's life that was read as a
video codec problem" -- which is also a claim about elapsed time that nothing
measured supports. The tree is days old.

The intro now states the design: it is a delivery problem, the live design has
no decoder in it, and the comparison that settles it is a measurement (37,585
decoded costs more than 49,152 not decoded; 110.4% against 54.9% at the 9 clk/B
floor). The codec is "in the tree and parked (USER DECISION)" -- no session
number attached, because which sitting it happened in is STATUS's business.

Four more of the same, found by grepping rather than by eye:
  - B2 "is not a headroom question any more" -> it decides which player exists
    rather than how much headroom one has.
  - "Other things the machine has said" -> "What else a player has to carry".
  - ring.i/xfer.i/stream.s "left the video path when a DMAC-direct player
    deleted the ring" -> a DMAC-direct player has no use for them, because it
    has no ring. Same edit in the parked list and in Layout.
  - "Why it is parked" kept the standing reason and dropped the retelling.

And one where the story was carrying a fact that had to stay: the sasi profile
was "dropped on capacity rather than bandwidth" with the numbers left in
FINDINGS. They are in the sentence now -- a 40 MB volume against 146 MiB of
game -- so the clause survives without the reader needing the episode.

One "session" reference stays, at the STATUS.md bullet: that file IS ordered
newest session first, and saying so is navigation for another document rather
than this one telling a story.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6

.gitignore

@@ -11,6 +11,7 @@ roms/
 __pycache__/
 *.pyc
 *.dlx
+*.dlxp
 a.out
 tmp/
 tools/bench/c68k/c68k_bench

README.md

@@ -1,350 +1,476 @@
-# Dragon's Lair — Sharp X68000 port
+# Dragon's Lair: Sharp X68000 port
 
 Porting Dragon's Lair to a stock X68000 (68000 @ 10MHz, 2MB, SCSI).
 
-This is fundamentally a **video codec problem**, not a game-logic problem: the
-game logic is a scene table with branching input windows; the difficulty is
-pushing ~22 minutes of Don Bluth animation through a 10MHz 68000.
+The game logic is a scene table with branching input windows. The difficulty is
+pushing ~22 minutes of Don Bluth animation through a 10MHz 68000, and it is a
+**delivery** problem rather than a compression one — bus cycles and bytes off a
+disc.
 
----
+**The live design has no decoder in it.** The disc holds packed 8-bit frames and
+a DMA channel walks them into GVRAM with the CPU halted. Measured on the machine,
+decoding 37,585 bytes costs more than not decoding 49,152: at the 9 clk/B
+dual-address floor the codec is 110.4% of a 12 fps frame and the decoder-free
+packed player is 54.9%.
+
+A working codec is in the tree and is **parked** (USER DECISION). Both branches
+are described below; the packed one is where the work goes.
 
 ## What it looks like
 
-![Blu-ray source next to the 68000's output](docs/img/source-vs-decoded.png)
+**The packed player, running off a real volume, with sound.** `src/player/packed.s`
+on an emulated stock X68000: the 68000 brings up its own display, builds its own
+193-entry DMA chain, keeps its own frame clock off the CRTC's V-DISP, and fetches
+every record itself with `READ(10)` from a real MB89352. **There is no decoder in
+the machine.** Source on the left; on the right, MAME's own snapshots of what the
+68000 put on screen. **120 of 120 frames are pixel-exact against the container**,
+every one compared, and the clip refuses to build otherwise.
 
-Left, the Blu-ray frame cropped to 256x192. Right, **the same frame as the
-emulated 68000 actually drew it** — 256 colours out of the X68000's 65536, one
-16-colour-per-4x4-block codebook, decoded by `src/player/decode.s` from the
-container. Not a re-render: these are the pixels MAME had on screen, extracted
-from its own snapshot. 2x nearest-neighbour, no filtering.
+<video src="docs/img/packed-player.webm" controls muted loop width="100%"></video>
 
-**The player, running.** 119 frames out of a **256 KB ring buffer on an emulated
-stock 2 MB X68000**, paced to a 12 fps frame clock, streamed from a host file at
-488 KB/s — `src/player/stream.s`, no Lua in the decode path. Source on the left,
-the machine's screen on the right.
+[`docs/img/packed-player.webm`](docs/img/packed-player.webm) (120 frames, 12 fps,
+VP9, with the chip's own audio)
+
+**The clip is a composite of two runs and that is the point.** The picture is the
+gate run — paced at half rate so each snapshot lands inside the write window,
+cycle stealing, no sound. The sound is the audio run — the same container at
+12 fps with the MSM6258 on channel 3, captured by MAME off the speaker, cut at
+the first sample the chip produced and gated sample-exact against lump 0 before
+anything is written. **What it is not is a real-time capture of the shipping
+configuration**, which at this container's own burst rate would show a blank
+layer for 99.5% of every slot (FINDINGS 64.2, below).
+
+![Blu-ray source next to the packed player's own screen](docs/img/packed-player.png)
+
+The still is the frame whose PSNR is closest to the mean over the gated window,
+so it is not the flattering one: **33.13 dB against the window's mean of 33.10**,
+which is the packed container's predicted GRB555 figure to the digit.
+
+**Name the layer.** Everything here is **emulated**: MAME 0.277 `x68000`,
+`-bios ipl10`, stock 10 MHz / 2 MB, with every 68000 cycle figure cross-checked
+on a second CPU core (px68k's C68K) and the display path cross-checked against
+px68k's own `gvram.c`. **Nothing in this project has run on real hardware yet.**
+
+## How the packed player works
+
+**256-colour GVRAM throws away the high byte of every word a CPU writes**, so a
+picture byte normally costs two disc bytes. **CRTC R20 bit 11 turns the masking
+off**, and with the two 256-colour pages scrolled apart one word carries two
+pixels (FINDINGS 46/47). That makes a frame 1.0 B/pixel — 49,152 bytes — and it
+makes the frame a *literal*: no codebook, no recursion, no decode.
+
+A record is that picture plus its own 256-entry palette: **49,664 bytes, which is
+97 sectors exactly**. A packed record's length is geometry, so **the container
+carries no index and no length word** — record *i* is at `off + i*rec` and a seek
+is arithmetic. A DMA channel fills GVRAM in buffer mode straight off the disc and
+**walks the 1,024-byte line stride itself** through array chaining, so a frame is
+**one channel start and not 192** — and the chain crosses from the palette
+registers at `$E82000` into GVRAM in the same start (FINDINGS 62). The array is
+scene-constant, because the packed layout spends both 256-colour pages and there
+is no page to flip.
+
+**Audio rides a fixed cadence, not the record.** The MSM6258 wants 7,812.5 B/s
+and 12 fps wants 651.0417 B a slot, a ratio with no arithmetic relationship, so
+audio in the record would make records variable-length and bring the index back.
+DLXP2 groups instead: **one 14-sector audio lump, then 11 records**, so
+`record i = off_frm + i*rec + (i//11)*7168` — six instructions once a frame. The
+lump is 7,168 bytes of space and eleven frames of audio is 7,161.4583…, so **the
+payload alternates 7,161 and 7,162** and a player that fed the chip whole lumps
+runs 0.09% fast: **1.25 s of lip-sync drift over the game's 22.8 minutes**,
+predicted, and played at 1.26 (FINDINGS 67.2/68.3). The player carries one
+accumulator instead, which is the frame clock's shape for the frame clock's
+reason.
+
+**The wire is fixed and there is no lever on it: 582.0 KB/s of picture +
+7.64 KB/s of audio cadence = 589.6 KB/s.** A codec's bitrate is adjustable; a
+literal frame's is geometry, and no scene costs less than another.
+
+| | |
+|---|---:|
+| record | 49,664 B = 97 sectors |
+| wire | **589.6 KB/s** |
+| picture, as the display renders it | **33.10 dB** mean over the gate window |
+| CPU per frame, video path | one channel start and one `READ(10)` — no paint |
+| cost at the 9 clk/B dual-address floor | **54.9% of a 12 fps frame** (FINDINGS 64.2) |
+| the codec, at the same floor | 110.4% (FINDINGS 61.4) |
+
+## Where it stands
+
+**What runs, end to end, on the emulated machine off a real volume:**
+
+- **120 of 120 frames pixel-exact**, in both palette orders, every frame compared
+  rather than the last — a packed frame is a literal, so frame 119 says nothing
+  about frame 60 (FINDINGS 64.1).
+- **All 78,125 B of a container's audio, read back off the speaker**, sample-exact
+  against the four ADPCM axes carried in the container's own header, one
+  delivered byte at a time — because a player's counters all stay right through
+  a bug that overwrites the buffer the channel is reading (FINDINGS 68.1).
+- **A mid-stream seek with sound on it**: `pg_aseek` rebuilds the lump index, the
+  stream position, the remainder accumulator and the byte offset into the group
+  and issues a second read — **132,162 B of spliced stream accounted for byte by
+  byte** across a branch deliberately not on a group boundary (FINDINGS 71.1).
+- **The scene graph**: 40 scenes, 516 sequences, 906 input windows, imported from
+  the arcade's own graph into this project's schema (FINDINGS 56).
+
+**The binding resource is the 68000's local BUS, not its clock**, and every item
+above is priced in one of four units:
+
+| resource | state |
+|---|---|
+| **68000 local bus** | the binding one. The codec's decoder occupies 86.7% of it, and 52 of the 53 frames that miss the 12 fps budget miss on the bus (FINDINGS 38). |
+| **68000 clocks** | measured, on two independent cores. |
+| **delivery rate** | **no working figure, deliberately** (FINDINGS 50, USER DECISION). `--bus`, `--kbps` and `DLX_STREAM_KBPS` are required arguments with no defaults, so no table can be scored against a rate its own output does not state. |
+| **seek time** | **no figure at all, and never had one.** |
+| **W, clocks stolen per delivered byte** | 5 single-address held, 9 dual held, 12 single arbitrated, 16..19 for the IPL ROM's own disk channel. **Still the largest open number.** The CPU doing the transfer itself is **87.28 clocks a byte, measured** — 4.6x the worst DMA configuration found here and 17.5x the best (FINDINGS 58). |
+
+### The open question that decides which player gets built
+
+**The write window is the frame.** A packed write *requires* R20 bit 11; buffer
+mode blanks the layer while it is set; and a DMAC-direct player holds the window
+open for the whole data phase, because there is no second page to hide behind.
+
+    dark fraction of a slot = record bytes / (DATA-PHASE rate x slot)
+
+**The rate in that expression is the BURST rate, not the sustained one**, and at
+the container's own 582.0 KB/s the dark fraction is **1.0**: every frame
+delivered, on time, pixel-exact, and none of them displayed. Free-running, the
+run reported exactly that — **the window was open on 99.5% of host frames**.
+
+It **reverses the ranking** in clocks, so there are two packed players and the
+difference between them is *when* the window is open:
+
+| | clocks (W=9) | on screen | RAM |
+|---|---:|---:|---:|
+| **A — DMAC-direct** (built, FINDINGS 64) | **54.9%** of a slot | 0% at the container's wire, 72.7% only at 2,131 KB/s | none |
+| **B — DMA to RAM + CPU paint** (K4, not built) | 82.2% | **72.7% at any rate** | 99,328 B |
+
+B's paint is **measured**: the packed `movem` blit is 227,553 clocks, 27.3% of a
+slot, independent of the medium. They are equally visible only at **3.7x the
+wire**. So B2 — *does buffer mode blank the display?* — decides which player
+exists rather than how much headroom one has, and **K4 is not built until it is
+answered** (FINDINGS 64.2, ROADMAP K4).
+
+### What else a player has to carry
+
+**A held channel costs the frame clock half its ticks, without the clock being
+able to tell.** `clock.i` counts V-DISP interrupts; a held channel halts the
+68000; the MFP's pending bit is one bit. Held at 12 fps, **487 of 1,038 edges are
+lost** and the player reports **zero late frames**, because the tick it grades
+itself against is advanced by the interrupt the channel stopped it from taking.
+It believed it was at 12 fps; the screen was at **6.37** (FINDINGS 64.3).
+
+**Held, the bus costs the audio 463 times the seam.** The MSM6258 has no
+starvation state — it goes on decoding nibbles out of whatever byte its data
+register still holds — so the interval between a channel counting out and the CPU
+arming the next lump is a held nibble pair driving the predictor. Stealing, that
+seam is **0.51 ms over ten seconds**, because the audio service runs from inside
+`dma.i`'s transfer wait. Held, it is **236 ms, 2.31% of the audio**, with every
+byte still correct (FINDINGS 68.2).
+
+**The chip is not the datasheet and not ffmpeg, and it was asked.** Sixteen
+candidate decoder models were fitted to what the emulated MSM6258 produced from a
+designed nibble stream; **exactly one reproduces it sample-exact**, with a
+negative control on every axis. The chip runs the datasheet's truncation, takes
+the **low** nibble first, clamps its accumulator at **10 bits** and starts it at
+**−2**. Getting the delta formula wrong costs −2.88 dB; **getting the nibble order
+wrong costs −25.74 dB** (FINDINGS 66). The four axes ride in the container header
+as fields rather than a version number.
+
+**The audio level is measured off the whole disc and does not change.** All 201
+streams that have audio, 21.5 minutes: the **disc peaks at 946 of 2048, 5.35 dB
+over the chip's clamp**, in 402 events totalling 44.0 ms. Forty windows encoded at
+six gains price the choice, and the disc's own level has the **best mean SNR
+(22.03 dB)** — the gain that guarantees zero clamping costs 0.85 dB across the
+game to buy back 1.90 dB on the 2.11 seconds that clamp, because the OKI step
+table's floor is a constant and does not scale (FINDINGS 69).
+
+**The predictor does not seek.** The MSM6258's accumulator is a pure integrator
+with no leakage term, so a branch that hands the chip bytes chosen for a state it
+is not in produces **a DC offset that does not decay**: playing through, −355 of
+511 with AC 0.00, still −108 four seconds later; stopping and re-PLAYing, a single
+permanent constant of −65. **A re-PLAY is 5.5x better and neither is zero.** The
+only fix that reaches zero is the encoder's — reset the predictor where a branch
+can land — and it costs **0.33 dB** (21.99 → 21.66) because the step table
+re-converges in a few samples (FINDINGS 71).
+
+**Branch points do not wait for the buffer.** 5.4% of the game's 612 branch
+transitions open an input window on the first frame of a clip the disc *seeked
+to*, so two seeks can fall back to back with no play between them. On the packed
+branch there is **no refill climb at all** — a record goes straight into GVRAM and
+the video lookahead is zero records — so the only consumer with any lookahead is
+the audio one: **1.833 s of sound held against 0.000 s of picture** (FINDINGS
+56/70.2). And a DLXP2 group puts its lump *in front* of its records, so a seek
+lands with its audio behind it: **mean 416.5 ms of silence entering a branch**
+over the arcade's 409 within-container targets, which is what `pg_aseek`'s second
+read removes for 11.7 ms (FINDINGS 70.3).
+
+### The parked branch: the codec
+
+The codec is a Cinepak-style hybrid — each 4x4 block coded as SKIP, one 4x4
+codeword, four 2x2 codewords, or RAW literal pixels, chosen per block by
+rate-distortion, with a v7 literal-span mode over the top. It works, it is
+measured, and it is **pixel-exact on the 68000 under two independent CPU cores**;
+`stream.s` decodes a 120-frame window out of a 256 KB ring on a stock 2 MB
+machine with the container in a file rather than in RAM.
 
 <video src="docs/img/player.webm" controls muted loop width="100%"></video>
 
-[`docs/img/player.webm`](docs/img/player.webm) — 119 frames, 12 fps, VP9
+[`docs/img/player.webm`](docs/img/player.webm) — the codec player, 119 frames,
+12 fps. 116 are pixel-exact against the reference decoder; **three are torn**,
+frame *n* on top of frame *n-1*, because MAME captured the screen while the block
+loop was partway down it. `decode.s` writes straight to the displayed page, so a
+real player tears the same way, and the media builder **asserts the tear** rather
+than trimming it. [`docs/img/modes.webm`](docs/img/modes.webm) is the same window
+with the block-mode map beside it — black SKIP, blue V1, amber V4, red RAW —
+which is what every cost table in FINDINGS is really about.
 
-116 of those 119 frames are **pixel-exact** against `tools/encoder/dlx.py`'s
-reference reconstruction. The other three are **torn** — the top of the picture
-is frame *n* and the bottom still holds frame *n-1*, because MAME captured the
-screen while the block loop was partway down it. That is not a rig artefact:
-`decode.s` writes straight to the displayed page (one display path, FINDINGS
-28.1 — the dual-path plan is kept runnable as a counterexample precisely because
-it corrupts frames), so a real player tears the same way.
-`tools/media/make_readme_media.py` asserts the tear rather than trimming it —
-every differing pixel has to come from the previous frame, or it refuses to
-build.
+**Why it is parked.** Its remaining path is a conjunction and the packed one is
+not: reaching a fitting rate needs a 35% byte reduction, two of its three levers
+measure inert, and the reward on success is a design at ~100% of the frame
+against the packed player's 54.9%. **It is kept on disk and not built on** — if
+B2 comes back "buffer mode blanks", it is the only thing left (FINDINGS 61.8,
+48.3).
 
-**What the decoder is actually doing.** The same window with the block-mode map
-beside it: **black = SKIP** (costs nothing, draws nothing — the previous frame
-stands), **blue = V1** (one codebook index for a whole 4x4 block), **amber = V4**
-(four indices), **red = RAW** (sixteen bytes verbatim). The mode mix is what
-every cost table in `docs/FINDINGS.md` is really about — V4 costs 1.5x V1, and
-since session 8 the mode decision is charged both bytes *and* cycles, which is
-why a byte-rich profile buys its way out to RAW instead of V4.
+## What is open
 
-<video src="docs/img/modes.webm" controls muted loop width="100%"></video>
+**Hardware — this list is the user's, and nothing here can be settled by an
+emulator.**
 
-[`docs/img/modes.webm`](docs/img/modes.webm) — the same 119 frames with the mode map
+- **B1. Measure the medium.** Three thresholds, not one: **sustained ≥ 589.6 KB/s**
+  or frames arrive late; **the data-phase BURST rate**, which decides how much of
+  the slot the picture is on screen; and **seek time**, which has no figure at
+  all. Plus what one extra SCSI command costs, which decides the audio cadence.
+  The 0.7–1.7 MB/s usually quoted for BlueSCSI on an X68000 is **folklore with no
+  published benchmark behind it**.
+- **B2. Does buffer mode blank the display?** `probe_bit11_blank.lua` is written
+  and settles it in minutes on a real board. It decides A vs B above.
+- **B3. Does a real CZ-6BS1 drive `#EXREQ`?** MAME's card has no request line to
+  the DMAC at all, so every configuration that can be run against it is
+  auto-requested and **charged by time rather than by byte**. A real request line
+  is what single-address 5 clk/B needs.
+- **B4. Does a real palette register take a byte write?** A negative answer costs
+  0.28% of a frame and nothing else.
+- **The MSM6258V, on silicon**: the four axes; whether it resets accumulator, step
+  index and nibble select on PLAY only when it was not already playing; and
+  whether it goes on asserting `#DRQ` while STOPped.
 
-**Name the layer:** everything above is **emulated** (MAME 0.277 `x68000`,
-`-bios ipl10`, stock 10 MHz / 2 MB), cross-checked frame-for-frame on a second
-CPU core (px68k's C68K). Nothing in this project has run on real hardware yet.
+**Software, in order.**
 
----
+1. **The predictor-reset container (DLXP3).** The cheapest thing that takes a
+   measured cost to zero rather than down. The player half already exists
+   (`PG_ARST`).
+2. **The audio buffering depth.** `PG_ANBUF` is 3; two slots is one constant and
+   one run. The audio buffer is the packed branch's *only* buffer.
+3. **The silent-clip and short-audio cases**: a scene whose audio is shorter than
+   its frames, and a scene with no audio track at all.
+4. **The cadence pick.** With a working seek path the silence F=11 costs is zero,
+   so the trade is padding against RAM — plus one SCSI command per branch, which
+   is B1's.
 
-**And the binding resource is the 68000's local BUS, not its clock.** The
-decoder occupies 86.7% of it once instruction prefetch is counted, and 52 of the
-53 frames that miss the 12fps budget miss it on the bus, not the CPU
-(FINDINGS 38). Read that before optimising anything for cycles.
-
-The **literal span with a fine tail** (v7) is now IN the player: `decode.s`
-paints it, pixel-exact under both CPU cores, and it costs inside the decoder
-what `blit.s` said it would to 0.2% (FINDINGS 41).
-
-**The delivery path is built and tested too** (FINDINGS 49). `src/player/stream.s`
-decodes the whole 120-frame window **out of a 256 KB ring on a stock 2 MB
-machine**, final frame pixel-exact, with the container in a host file rather than
-preloaded into RAM. The constraint turned out to be **contiguity, not byte
-count** — the block loop reads with a monotonically increasing `a0` and no bounds
-check, so the ring needs the whole next record resident *and contiguous*, which
-is a condition no byte-counting buffer simulation can see.
-
-**And building it caught a live defect**, then cost the project a constant.
-The shipping candidate is 496.7 KB/s; the pipe figure the design had been
-simulated against since session 2 was smaller, and nothing in the tree was
-comparing the two — the rate controller binds on clocks and has no pipe term at
-all, while the buffer sizing kept standing on a constant the design had stopped
-enforcing.
-
-**So the pipe constant is retired (session 18, USER DECISION).** It was never a
-bus measurement — user-supplied, no provenance, 10% of SCSI-1's asynchronous
-rating (FINDINGS 42.1). It is gone as a default from every analysis tool and
-from `stream.lua`; `--bus` / `--kbps` / `DLX_STREAM_KBPS` are now **required
-arguments**, so no table can be scored against a rate its own output does not
-state. **There is no working delivery figure, and that is the honest state.**
-
-What replaces it is a requirement rather than a constant:
-`tools/analysis/19_ring_stream.py` reports the **zero-prefill pipe**, the rate a
-medium must clear for a container to need no prefill. For the candidate that is
-**513.2 KB/s** — a hardware acceptance test to measure a BlueSCSI against.
-
-**Bytes are not free, and the number that said they were was in the wrong
-unit.** Session 13 found the pipe figure the design was built against was never
-a bus figure (SCSI-1 is 1.5 MB/s asynchronous) and concluded the span pass
-saturates at ~837 KB/s, 0/120 frames over budget. Session 14 found the disk
-debit behind that was charged **per word of stream to a byte-wide port** — the
-MB89352 is an 8-bit SPC, so the DMAC pays per BYTE, and the debit is 2x every
-table since FINDINGS 5. No 68000 bus cycle is shorter than four clocks, so the
-old figure was below a physical floor.
-
-**What survives, re-encoded honestly: 496.7 KB/s at 29.19 dB, 1 frame of 120
-over the 12fps budget** — and that one is frame 0, the intra frame, late on
-purpose. The remaining lever is not ours: **whether the CZ-6BS1 wires the SPC's
-DACK to the bus's `#EXACK`**, which decides 5 clocks/byte against 9, and with
-it 242 KB/s and 0.69 dB. Read FINDINGS 43 before quoting any rate figure.
-
-**And session 20 read the answer the machine already had.** The X68000's IPL ROM
-programs all four HD63450 channels itself, and MAME boots the rig with it, so
-`tools/analysis/21_iplrom_dmac.py` decodes the configuration straight out of the
-image and gates on the bytes still being there. The audio channel is
-dual-address, 8-bit port, cycle steal **without hold**, one external request per
-byte: **16..19 clocks a byte, not 5** — which prices the ADPCM stream at
-1.25%..1.48% of a frame and closes ROADMAP's "do this first" item. The disk
-channel is programmed **identically**. That is 16..19 clocks per delivered byte,
-above the whole 5..12 bracket the project costs the transport in, and at that
-price nothing fits. It is SASI and not the MB89352, so it does not settle the
-question — but **a cheap configuration is now the thing that has to be shown,
-not the thing assumed.** FINDINGS 52.
-
-**Green-light check:** `./tools/bench/check.sh` (~3 min, needs the Blu-ray
-mounted) re-runs both display regression tests, the rate-control drift test, the
-display-path coherency counterexample and a 120-frame 68000 decode, then prints
-`ALL GREEN`.
+**Parked, so it is not re-proposed:** the codec's remaining encoder work (E7, E4,
+C1); `ring.i`, `xfer.i` and most of `stream.s`, which a DMAC-direct player has no
+use for because it has no ring, and P4a's wiring with them. **K4 is blocked
+on B2**, not parked.
 
 ## Reproducing this
 
-**No media ships in this repo, and none of it is redistributable.** Bring your
-own Dragon's Lair Blu-ray. Everything else needed to rebuild every number and
-every picture above is here or is packaged.
-
-You need:
+**No media ships in this repo and none of it is redistributable.** Bring your own
+Dragon's Lair Blu-ray. Everything else needed to rebuild every number and every
+picture above is either here or is packaged.
 
 | | |
 |---|---|
-| the disc | loop-mounted read-only. `udisksctl loop-setup -r -f DRAGONS_LAIR.iso` — the tree was built against a decrypted UDF 2.x image (7-Zip cannot read UDF 2.x; use the loop mount) |
-| `python3` | plus **numpy** and **Pillow**. Nothing else — the k-means is hand-rolled rather than pulling in sklearn |
-| `ffmpeg` / `ffprobe` | frame extraction, and the README clips |
+| the disc | loop-mounted read-only: `udisksctl loop-setup -r -f DRAGONS_LAIR.iso`. Built against a decrypted UDF 2.x image; 7-Zip cannot read UDF 2.x, so use the loop mount |
+| `python3` | plus **numpy** and **Pillow**, and nothing else. The k-means is hand-rolled rather than pulling in sklearn |
+| `ffmpeg` / `ffprobe` | frame extraction, and the clips above |
 | **MAME** | tested on 0.277, with the `x68000` ROM set. The rigs drive it headless via `-autoboot_script` |
 | vasm (m68k, Motorola syntax) | **vendored**: `tools/vasm/vasmm68k_mot` is a Linux x86-64 binary, with the source tarball beside it to rebuild elsewhere |
 
-Then:
-
 ```sh
 export DLX_BDROM=/path/to/your/mounted/bluray    # if not /media/$USER/BDROM
-./tools/bench/check.sh                           # ~3 min, prints ALL GREEN
+./tools/bench/check.sh                           # the green light, prints ALL GREEN
 ```
 
-`DLX_BDROM` is honoured by every tool that reads the disc. Two stages are
-optional and **skip rather than fail** when their input is absent, because both
-live outside this repo:
+`check.sh` re-runs everything above that a host can re-run: both display
+regression tests, the rate-control drift gate, the display-path coherency
+counterexample, a 120-frame 68000 decode on two CPU cores, the ring and paced-ring
+passes, the DMAC configuration gate, the load-time transforms on both cores, the
+scene-graph import when a DirkSimple checkout is present, the packed container
+(with audio, and a silent control beside it) rendered through px68k's own GVRAM
+model, **the packed player for 120 frames off a real volume with every frame
+compared**, the audio encoder against ffmpeg's decoder, the sixteen-way decoder
+identification against the emulated chip, the DLXP2 container and each of its four
+axes with a negative control on each, the audio level off every stream of the
+game's own footage, and the refill climb against the arcade's own 612 branch
+points. `DLX_BDROM` is honoured by every tool that reads the disc.
 
-- `PX68K=/path/to/px68k` — a px68k checkout, for the second-CPU-core gate. This
-  is the cheapest strong test in the tree (seconds, no MAME, no ROMs), and it is
-  what licenses the bus and cycle figures.
-- `IPLROM=/path/to/iplrom.dat` — the X68000 IPL ROM, for the DMAC-configuration
-  gate (FINDINGS 52). Defaults to `~/mame/roms/iplrom.dat`.
+Two stages **skip rather than fail** when their input is absent, because both live
+outside this repo: `PX68K=/path/to/px68k` for the second-CPU-core gate — the
+cheapest strong test in the tree, and what licenses the bus and cycle figures —
+and `IPLROM=/path/to/iplrom.dat` for the DMAC configuration gate.
 
-To rebuild the stills and clips in `docs/img/` you also need a paced recording
-run — see the header of `tools/media/make_readme_media.py`.
+The player runs and the media are rebuilt from them:
 
-**Scene selection is a hard-coded stream number**, not a search: the gates use
-stream `00020` and `00223` of the disc's 224 `.m2ts` files, which are the ones
-FINDINGS §1 and §25 characterise. A different pressing may number them
-differently, and if so the green light will extract the wrong footage rather
-than fail — check that `tmp/fr_singe/` looks like the Singe encounter (which is
-what the directory is named for) before trusting any figure.
+```sh
+bash tools/bench/packed_run.sh                        # the packed player, 7 runs
+python3 tools/media/make_packed_media.py              # docs/img/packed-player.{png,webm}
+python3 tools/media/make_readme_media.py <c.dlx>      # the parked codec's stills and clips
+```
 
-## Read first
-- **`docs/FINDINGS.md`** — measured hardware facts, content statistics, codec
-  decision, and a section on measurement traps that produced three separate
-  false results. Read §4 before trusting any pipeline number.
-- **`docs/STATUS.md`** — current state, working setup, blockers, next steps.
-  **Start here.** It also lists what has been explicitly abandoned, so old ideas
-  do not get re-proposed.
-- **`docs/ROADMAP.md`** — the remaining work to a completion target, and which
-  milestone that target is. Read it with STATUS, not instead of it: STATUS holds
-  the measurements, ROADMAP holds the shape and goes stale first.
-- **`docs/BENCHMARK.md`** — how to measure the storage subsystem, and why a
+Both media builders **gate before they write**: a still or a clip of the player is
+a claim that the player drew it, so every frame is checked pixel-exact against the
+container first and the audio cut is checked sample-exact against lump 0. A README
+that illustrated a pixel-exact player with an approximate picture would be a small
+lie about the one property this project keeps testing.
+
+**Scene selection is a hard-coded stream number, not a search.** The gates use
+streams `00020` and `00223` of the disc's 224 `.m2ts` files. A different pressing
+may number them differently, and if so the green light will extract the wrong
+footage rather than fail, so check that `tmp/fr_singe/` looks like the Singe
+encounter before trusting any figure. **Not every large stream is game footage**:
+`00216` is the feature with a burned-in commentary picture-in-picture and `00215`
+is the commentary itself, the two largest files on the disc. The clean
+9.4-minute animation is **`00223`** (FINDINGS 25.1).
+
+**Nothing outside-derived is committed here.** The scene graph is regenerated from
+a reader's own clones into gitignored `tmp/`, and `tools/import/scenegraph.py` is
+the single file in the repo coupled to those projects — everything downstream reads
+`DLXSCENE1`, this project's own schema, with the sources' attribution carried in
+it. DirkSimple is zlib (Ryan C. Gordon); the SNES chapter set is MIT (Chad
+Doebelin) and, by its own README, *derived* from DirkSimple rather than an
+independent transcription.
+
+## Encoders
+
+**The live one is the packed container**, and it has no rate control in it because
+there is no rate lever:
+
+```sh
+# the gate window: stream 00223, 10.0 s from 539.4 s, the same seconds both times
+python3 tools/encoder/extract.py       00223 tmp/fr_singe   12    crop 539.4 10.0
+python3 tools/encoder/extract_audio.py 00223 tmp/au_singe.raw 15625     539.4 10.0
+python3 tools/encoder/pack.py tmp/fr_singe out.dlxp --audio tmp/au_singe.raw
+```
+
+`dlxp.py` is the format and the one place the interleave, the 97-sector record,
+the audio cadence and the lump payload are stated. It encodes in ~3 seconds
+because there is no k-means in it. `--scene-palette --no-palette` is the fallback
+container: **30.79 dB, zero palette churn, 576.0 KB/s**, still ahead of the codec
+as the display renders both, and it is what ships if B2 says the layer blanks —
+because a per-frame palette changes **231 of 256 entries every frame**, and a
+picture under the neighbouring frame's palette is **12.8 dB worse**
+([`docs/img/palette-mismatch.png`](docs/img/palette-mismatch.png)).
+
+`adpcm.py` is the MSM6258 codec and it carries **two decoders on purpose**: the
+module defaults are ffmpeg's, so `tools/bench/verify_adpcm.py` stays a check
+against an independent implementation, and `adpcm.CHIP` is the set measured out of
+the machine's own chip. Anything that encodes *for* the machine passes `CHIP`
+explicitly.
+
+**The parked codec's encoder is kept and still runs:**
+
+```sh
+python3 tools/encoder/encode.py tmp/fr_singe out.dlx --profile scsi --preview p.png
+```
+
+`--kbps` is the quality rate point and `--span-kbps` the ceiling the span pass may
+draw on; `mu` is bisected per frame against the 68000's own decode budget so a
+frame also *decodes* in time. One profile, `scsi`; the 110 KB/s `sasi` profile was
+dropped on capacity rather than bandwidth — a SASI volume is 40 MB and the
+game is 146 MiB even at that rate. The RAW escape means `lam=0` is
+pixel-exact against the palettised frame. See FINDINGS 31/41/44 before changing
+any of it.
+
+## Documentation
+
+- **`docs/STATUS.md`** is the current state, working setup, blockers and next
+  steps, newest session first. **Start here.** It also lists what has been
+  explicitly abandoned, so old ideas do not get re-proposed.
+- **`docs/ROADMAP.md`** is the remaining work to a completion target. Read it with
+  STATUS rather than instead of it: STATUS holds the measurements, ROADMAP holds
+  the shape and goes stale first.
+- **`docs/FINDINGS.md`** is measured hardware facts, content statistics, the codec
+  decision, and a section on measurement traps that produced three separate false
+  results. Read §4 before trusting any pipeline number. It is append-only and
+  later sections overturn earlier ones; superseded sections carry a blockquote
+  pointing at the correction.
+- **`docs/BENCHMARK.md`** is how to measure the storage subsystem, and why a
   bandwidth figure out of MAME would be meaningless.
-- **`docs/HARDWARE.md`** — X68000 GVRAM/CRTC reference.
+- **`docs/HARDWARE.md`** is the X68000 GVRAM/CRTC reference.
 
 ## Layout
-```
-docs/            findings, status, hardware reference
-tools/analysis/  measurement scripts, numbered in the order they were written
-                 (01/02 marked BROKEN deliberately, kept as regression refs).
-                 Run from the repo root — they import from tools/encoder/.
-                 07 finds the hottest sustained window in a stream; 08 renders
-                 source | decoded | block-mode map as .webm; 09 is the
-                 rate-control drift gate (FINDINGS 26/27) and is part of
-                 check.sh -- it exits non-zero if the encoder ever again
-                 reports a reconstruction no decoder would produce.
-                 10 is a COUNTEREXAMPLE, and exits non-zero by design: it
-                 demonstrates that the two-display-path plan of FINDINGS
-                 24.5/25.6 corrupts 70 of 120 frames (FINDINGS 28.1).
-                 11 scores a container against the MEASURED per-mode block
-                 costs without needing MAME; 12 prices the literal-span mode of
-                 FINDINGS 30 against those same mode maps, and prints whether a
-                 scene cut still fits at 12fps; 13 measures what fitting the
-                 CPU budget costs in dB (FINDINGS 31) and caches H.build so the
-                 search loop is seconds, not minutes.
-                 14 prices the HD63450 array-chain against the v6 and v7
-                 spans (FINDINGS 39/40) and prints the sensitivity that decides
-                 it -- v7 is measured, and takes 37 of the 43 frames the DMAC
-                 would, so the DMAC stays dropped;
-                 15 measures how much of the 68000's LOCAL bus the decoder
-                 occupies (FINDINGS 38) and exits non-zero if its derived
-                 model stops matching the harness's measurement.
-                 16 is the DLX3 span container ROUND-TRIP gate (part of
-                 check.sh): it encodes, writes the container, reads it back with
-                 the reference decoder and fails if a pixel differs -- or if it
-                 emitted too few spans to have tested anything. 17 prices the
-                 spans the encoder ACTUALLY emitted, with no selection model,
-                 which is what 12 and 14 could only simulate.
-                 18 measures what a 16-colour text-plane literal would cost in
-                 dB, and closes that direction (FINDINGS 46.3).
-                 19 is the RING-BUFFER simulation, and it supersedes 09_buffer_
-                 sim.py's question rather than repeating it: it models the ring's
-                 ADDRESSES, because src/player/ needs each record contiguous and
-                 not merely resident. It reports the ZERO-PREFILL PIPE -- the
-                 rate a medium must clear for a container to need no prefill --
-                 and warns explicitly when demand exceeds supply on the MEAN,
-                 where a "required prefill" figure would flatter a sustained
-                 overrun. Its wrap count and hole size match tools/bench/
-                 stream.lua's, measured on a real 68000, to the digit.
-                 buscost.py is the shared bus-cycle table both import; the
-                 per-BLOCK constants live in tools/encoder/vq_hybrid.py and are
-                 imported, never copied (session 12 corrected one of them).
-tools/bench/     MAME Lua injection harness + 68000 benchmark sources.
-                 `check.sh` re-runs both display regression tests (~40 s).
-                 `blit.s`/`blit.lua` time the full-frame GVRAM blit on the
-                 68000 itself (FINDINGS 24) — not part of check.sh, because
-                 wall timings would make the green-light check host-sensitive.
-                 `span.sh` (prep_spans.py + span.lua + blit.s v5/v6/v7)
-                 measures the literal-span mode the same way (FINDINGS 30 and
-                 40, ~30 s); it also asserts that every one of its 36 timing
-                 configs drew a pixel-exact frame, the count taken from the
-                 generated metadata so a new config cannot weaken the gate.
-                 v7 is v6 with a second, 2-pixel chain for the span tail:
-                 66.0 cycles/span + 9.143 per coarse pixel + 9.978 per fine
-                 pixel, MEASURED, which is the win FINDINGS 39.4 predicted.
-                 `crtc_mode.lua` is the single source of truth for CRTC R00-R08
-                 and R20 — do not write CRTC values anywhere else.
-                 `prep_dlx.py`/`decode.lua`/`verify_decode.py` load, time and
-                 verify `src/player/decode.s`; the verify pass is in check.sh.
-                 `prep_stream.py`/`stream.lua` do the same for `stream.s`, but
-                 lay the container out as a DISK in a host file and feed it
-                 through a bounded ring at a modelled pipe rate -- so the rig is
-                 no longer bounded by the emulated machine's RAM, and a stock
-                 2 MB machine runs the whole window. `dlxload.py` holds the
-                 codebook/palette load-time maths both preps share.
-tools/bench/c68k/ headless px68k C68K harness -- a SECOND emulator for every
-                 68000 cycle figure (FINDINGS 37). Links only px68k's CPU core:
-                 no SDL, no ROMs, no emulated machine. `make PX68K=~/src/px68k`
-                 then `run.sh`; `verify_c68k.py` checks the decode is
-                 pixel-exact, which is what licenses the cycle numbers. It also
-                 counts BUS cycles, which MAME cannot report.
-                 The Makefile's -no-pie and the harness's MAP_32BIT arena are
-                 load-bearing: C68K truncates host pointers to 32 bits.
-tools/vasm/      vasm m68k assembler (built from source)
-tools/encoder/   hybrid VQ encoder + DLX3 container writer (working).
-                 spans.py is the v7 span geometry, selection and serialiser, and
-                 the single place the chain layout is stated on the encoder side
-                 -- it must match blit.s/decode.s (11 coarse units of 24 px, 11
-                 fine of 2).
-                 DLX2 4-byte-aligns every frame record: an odd `move.l` is an
-                 ADDRESS ERROR on a 68000, not a slow read (FINDINGS 28.3).
-                 dlx.py is the reference DECODER -- ground truth for the 68000.
-src/player/      decode.s: the 68000 DLX3 decoder, PRELOADED-stream front-end.
-                 stream.s: the same decoder behind a bounded RING (FINDINGS 49).
-                 Both include frame.i (the block loop and span chain) and geom.i
-                 (the constants) so there is exactly ONE copy of the bytes every
-                 cycle constant in FINDINGS 24/30/40/41 is fitted to. check.sh
-                 asserts decode.s still assembles to the same 1,296 bytes.
-                 decode.s: the 68000 DLX3 decoder. Pixel-exact under MAME and
-                 px68k's C68K core, blocks and v7 literal spans both. The span
-                 pass is blit.s v7 verbatim -- the same instruction sequence the
-                 66.0/9.143/9.978 fit was measured on, so do not tidy it.
-                 See FINDINGS 28, 31, 40 and 41.
-assets/          extracted frames/audio (gitignored)
-```
-
-## Encoder
 
 ```
-python3 tools/encoder/extract.py 00020 /tmp/fr 12 crop
-python3 tools/encoder/encode.py  /tmp/fr out.dlx --profile scsi --preview p.png
+src/player/      packed.s    THE LIVE PLAYER: display bring-up, the 193-entry DMA
+                             chain, its own V-DISP clock, READ(10) off a real
+                             volume, and the MSM6258 on channel 3 -- a lump ring,
+                             the remainder accumulator that stops the padding
+                             becoming drift, a service routine that runs from
+                             INSIDE dma.i's transfer wait (0.51 ms of seam
+                             against 236), and pg_aseek.
+                 dma.i       the HD63450 channel: array chaining, held and
+                             stealing, and DM_HOOK.
+                 scsi.i      the MB89352: selection, READ(10), the data phase.
+                 adpcm.i     the audio transport, every register byte of it
+                             decoded out of the IPL ROM rather than recalled.
+                 clock.i     the frame clock, off the CRTC's V-DISP.
+                 geom.i      the constants, in one place.
+                 decode.s    the parked codec's 68000 decoder (1,296 bytes,
+                 stream.s    asserted), the same decoder behind a bounded ring,
+                 ring.i      the ring producer, the transport under it, and the
+                 xfer.i      load-time codebook/palette transforms. OUT OF THE
+                 load.i      VIDEO PATH: a DMAC-direct player has no ring.
+                 frame.i     the block loop and span chain, included by both, so
+                             there is exactly ONE copy of the bytes every cycle
+                             constant is fitted to. The span pass is blit.s v7
+                             verbatim -- do not tidy it.
+tools/encoder/   dlxp.py     THE LIVE CONTAINER (DLXP2) and pack.py writes it.
+                 pack.py     Nothing is shared with the codec's writer on purpose.
+                 adpcm.py    the MSM6258 codec, carrying ffmpeg's decoder and the
+                             chip's measured one side by side.
+                 dlx.py      the codec's container and its REFERENCE DECODER,
+                 encode.py   ground truth for the 68000; the encoder, rate
+                 spans.py    control, and the v7 span geometry. Parked, kept.
+                 extract.py  frames and audio off the disc.
+tools/bench/     check.sh    the green light.
+                 packed_run.sh   the packed player: gate, rate, audio, held, seek.
+                 verify_packed.py / verify_packed_audio.py
+                             the two gates that matter. The audio one reads the
+                             SPEAKER, one delivered byte at a time, because every
+                             counter in the player stayed right through a bug
+                             that overwrote the buffer the channel was reading.
+                 crtc_mode.lua   the SINGLE SOURCE OF TRUTH for CRTC R00-R08 and
+                             R20. Do not write CRTC values anywhere else.
+                 probe_bit11_blank.lua   B2, ready to run on a real board.
+                 c68k/       a SECOND emulator for every 68000 cycle figure:
+                             px68k's CPU core, no SDL, no ROMs. It also counts
+                             BUS cycles, which MAME cannot report.
+                 gvpack/     the same argument for the DISPLAY: px68k's own
+                             x68k/gvram.c, so the address decode, the R20 bit-11
+                             write path and the scroll wrap are its code and not
+                             a model of it.
+                 blit.s      the full-frame GVRAM blit, timed on the 68000
+                             itself. V8 is the packed one: 227,553 clocks.
+tools/analysis/  measurement scripts, numbered in the order they were written.
+                 Run from the repo root. 01 and 02 are marked BROKEN
+                 deliberately and kept as regression references; 10 is a
+                 COUNTEREXAMPLE and exits non-zero by design, demonstrating that
+                 the two-display-path plan corrupts 70 of 120 frames. 15 is the
+                 bus occupancy model, 19 the ring's ADDRESSES (not its
+                 occupancy), 21 the IPL ROM's DMAC configuration, 25 the scene
+                 graph, 30/34 the packed containers, 31 the display duty, 35 the
+                 audio level off the whole disc, 36 the climb against real
+                 branch points, 37 what a branch costs the chip. buscost.py is
+                 the shared bus-cycle table; the per-block constants live in
+                 vq_hybrid.py and are imported, never copied.
+tools/import/    scenegraph.py -- the ONLY code in this tree coupled to somebody
+                 else's source. Output is gitignored derived data.
+tools/media/     make_packed_media.py builds the live player's still and clip;
+                 make_readme_media.py builds the parked codec's. Both gate
+                 before they write.
+tools/vasm/      vasm m68k assembler, binary plus source tarball.
+docs/img/        the stills and clips above, built from real emulated runs.
+assets/, tmp/    extracted frames, audio and run output (gitignored).
 ```
 
-**Two budgets, not one.** `--kbps` is the quality rate point and `--span-kbps`
-is the ceiling the span pass may draw on. They are different things: the profile
-is chosen, the pipe is hardware, and bytes between them buy a better picture if
-spent on `lam`, the 68000's deadline if spent on spans, and nothing if left
-unspent. Spans run before `mu` because a span pays in bytes and `mu` pays in
-picture (FINDINGS 41.2).
-
-**One profile: `scsi`, 280 KB/s.** The 110 KB/s `sasi` profile was dropped in
-session 9 on capacity, not bandwidth — a SASI volume is limited to 40 MB, and
-the game's 22.8 minutes of footage is 146 MiB even at that rate (FINDINGS 32).
-The rate point may return under another name once the delivery medium is
-settled, because a 1x CD-ROM sustains ~150 KB/s and CD-ROM is the only period
-medium with the capacity.
-
-The profile bitrate is a **ceiling**: lam is bisected per frame under a leaky
-bucket, so the profile's `lam` is a quality floor rather than a setting
-(`--fixed-lam` opts out).
-
-**But at `--spans all` none of that binds.** A 32-frame bucket emits the same
-container byte for byte as an 8-frame one, and `lam` never leaves its floor on
-any frame of the reference window: the rate is set by the span pass and by `mu`,
-not by `--kbps` or the bucket (FINDINGS 44.3). Two known unit inconsistencies on
-that side are implemented and default OFF because they measure as a wash --
-`--joint-decide` (the per-block lagrangian prices a byte at `lam + mu*c` rather
-than `lam`) and `--joint-bucket` (the bucket may not lend clocks it cannot
-repay). FINDINGS 44.
-
-An encode is ~95% k-means; a 120-frame window is ~29 s, of which ~22 s is
-training the two codebooks (FINDINGS 44.5).
-
-There are **two** ceilings, on two different axes. The second is the 68000's
-decode budget: `mu` is bisected per frame against 833,333 cycles so the frame
-also *decodes* in time, which takes the worst sustained window from 37 frames
-over budget to 1 for 0.62 dB at `scsi` (FINDINGS 31). It is on by default; `--no-cpu-fit`
-restores session 7 behaviour. Unlike bytes, cycles have no bucket — there is no
-double buffer to decode ahead into, so it is a hard per-frame ceiling. The codec is
-a Cinepak-style hybrid: each 4x4 block is coded as SKIP, one 4x4 codeword, four
-2x2 codewords, or RAW literal pixels, chosen per block by rate-distortion.
-
-The RAW escape means `lam=0` is pixel-exact against the palettised frame, so the
-quality knob spans lossless to heavily-compressed without changing the bitstream.
-
-Profiles are derived from a bandwidth figure, not chosen by eye:
-
-```
-python3 tools/encoder/profile_gen.py --bw-mbps 4 --name scsi
-```
-
-> **On reading `docs/FINDINGS.md`:** it is append-only and several later sections
-> overturn earlier ones. Superseded sections carry a blockquote at the top
-> pointing to the correction — heed those, especially 18 (reversed by 21).
-
-Source media (`DRAGONS_LAIR.iso`) and ROMs are gitignored — supply your own.
-
-**Not every large stream is game footage.** `00216` is the feature with a
-burned-in commentary picture-in-picture and `00215` is the commentary itself —
-the two largest files on the disc. The clean 9.4-minute animation is **`00223`**.
-See FINDINGS 25.1 before running any size-ranked survey.
+Source media (`DRAGONS_LAIR.iso`) and ROMs are gitignored. Supply your own.

src/player/adpcm.i

@@ -0,0 +1,131 @@
+; ---------------------------------------------------------------------------
+; adpcm.i -- the MSM6258V, driven the way the machine's own ROM drives it.
+; ROADMAP P6a, and it is P6b's transport arriving early rather than scaffolding.
+;
+; NOTHING HERE IS INVENTED.  Every register value below is one that
+; tools/analysis/21_iplrom_dmac.py decodes OUT OF THE IPL ROM's own bytes, at
+; the addresses it prints: channel 3's DCR/SCR/MFC/CPR/DFC/DAR at $FF0C2E and
+; the per-transfer OCR = $32 plus command $02 at $FF9A82.  That is the one
+; ADPCM path on this board that is known-correct because Sharp wrote it.
+;
+; WHY THIS FILE EXISTS AT ALL.  Session 33 fed the chip from Lua and got
+; silence, swept control 0..3 against port C 0..15, and stopped rather than
+; guess (65.5).  Two of the reasons are visible from here and neither is a
+; register semantic anybody had to guess:
+;
+;   * THE PPI'S PORT C IS NOT AN OUTPUT UNTIL IT IS TOLD TO BE.  The ADPCM pan
+;     and the sample-rate divider are port C bits, and an i8255 in its reset
+;     state has every port an INPUT -- so a write to $E9A005 changes a latch
+;     nobody is reading and the pan never leaves wherever it was.  Control word
+;     $92 (mode 0, A and B input, both halves of C OUTPUT) is what makes the
+;     other write mean anything.
+;   * AND FEEDING IT SLOWLY IS NOT FEEDING IT.  The chip has no FIFO and no
+;     starvation state: it consumes a nibble every sample period out of whatever
+;     its data register last held, forever.  A byte per host frame is not a
+;     quiet chip, it is the same two nibbles 130 times, which saturates in six
+;     samples.  The feed has to be paced by the chip, which is what channel 3
+;     and its request line are FOR.
+;
+; THE CLOCK IS TWO WRITES AND THEY ARE IN DIFFERENT DEVICES.  15,625 Hz is
+; 8 MHz / 512: the 8 MHz comes from CT1 in the YM2151's port register $1B, and
+; the /512 from port C bits 3,2 = 10.  Neither is readable, so the rate is
+; verified from the OTHER end -- the capture's own sample count.
+
+AD_CTRLR = $E92001              ; W: command   R: status (bit7 = NOT playing)
+AD_DATAR = $E92003              ; W: the byte the chip takes two nibbles from
+AD_PLAY  = $02                  ; COMMAND_PLAY -- session 33's probes wrote $01,
+AD_STOP  = $01                  ; which is COMMAND_STOP
+PPI_PC   = $E9A005
+PPI_CTL  = $E9A007
+PPI_COUT = $92                  ; mode 0, A/B input, BOTH halves of C output
+PPI_RATE = $08                  ; pan 00 = both, rate 10 = /512 = 15,625 Hz
+YM_ADDR  = $E90001
+YM_DATA  = $E90003
+YM_CT    = $1B                  ; CT1 in bit 1: 0 = ADPCM master clock 8 MHz
+
+AD_DMAC  = $E840C0              ; HD63450 channel 3 -- the ADPCM channel, and
+A3_CSR   = AD_DMAC+$00          ; the one the ROM points at $E92003
+A3_CER   = AD_DMAC+$01
+A3_DCR   = AD_DMAC+$04
+A3_OCR   = AD_DMAC+$05
+A3_SCR   = AD_DMAC+$06
+A3_CCR   = AD_DMAC+$07
+A3_MTC   = AD_DMAC+$0A
+A3_MAR   = AD_DMAC+$0C
+A3_DAR   = AD_DMAC+$14
+A3_MFC   = AD_DMAC+$29
+A3_CPR   = AD_DMAC+$2D
+A3_DFC   = AD_DMAC+$31
+
+A3_DCRV  = $80                  ; XRM 10 cycle steal w/o hold, DTYP 00 dual
+                                ; address, DPS 0 8-bit port  (IPL $FF0C2E)
+A3_OCRV  = $32                  ; DIR memory->device, SIZE 11 byte unpacked,
+                                ; CHAIN 00, REQG 10 EXTERNAL REQUEST (IPL
+                                ; $FF9A82).  External request is what makes the
+                                ; chip the pacemaker: one byte per #DRQ3, and
+                                ; #DRQ3 ticks at half the sample rate.
+A3_SCRV  = $04                  ; MAC 01 memory increment, DAC 00 -- the device
+                                ; address is a REGISTER and must not walk
+A3_CCRST = $80
+
+; --------------------------------------------------------------- ad_setup
+; The clock and the pan.  No arguments, no result; trashes d0.
+ad_setup:
+        move.b  #YM_CT,YM_ADDR
+        moveq   #60,d0                  ; the YM2151 wants settling between the
+.ymw:   subq.l  #1,d0                   ; address write and the data write
+        bne.s   .ymw
+        move.b  #$00,YM_DATA            ; CT1 = 0 -> ADPCM master clock 8 MHz
+        move.b  #PPI_COUT,PPI_CTL       ; ...and NOW port C drives something
+        move.b  #PPI_RATE,PPI_PC        ; pan both, /512
+        rts
+
+; ----------------------------------------------------------------- ad_arm
+; Arm channel 3 to feed (a1) for d1 bytes and start it.  Trashes d0.
+; The channel is started BEFORE the chip is told to play (see ad_play), so that
+; byte 0 is already in the data register when the accumulator is reset.
+ad_arm:
+        move.b  #$FF,A3_CSR             ; CSR is write-one-to-clear: a stale COC
+                                        ; would pass the wait loop instantly
+        move.b  #A3_DCRV,A3_DCR
+        move.b  #A3_SCRV,A3_SCR
+        move.b  #$05,A3_MFC
+        move.b  #$05,A3_DFC
+        move.b  #$01,A3_CPR             ; the ROM's own priority: ADPCM outranks
+                                        ; the disk at the arbiter (52.5 item 5)
+        move.l  #AD_DATAR,A3_DAR
+        move.b  #A3_OCRV,A3_OCR
+        move.l  a1,A3_MAR
+        move.w  d1,A3_MTC
+        move.b  #A3_CCRST,A3_CCR
+        rts
+
+ad_play:
+        move.b  #AD_PLAY,AD_CTRLR
+        rts
+ad_halt:
+        move.b  #AD_STOP,AD_CTRLR
+        rts
+
+; --------------------------------------------------------------- ad_abort
+; STOP channel 3 where it stands, and clear what stopping it posts.
+;
+; A SEEK IS THE ONLY THING IN THIS PLAYER THAT NEEDS THIS.  Every other re-arm
+; happens at COC, where the channel has already counted itself out and there is
+; nothing to stop; a seek arrives MID-LUMP, because a branch is a frame index
+; and a frame does not know about the cadence (FINDINGS 70.3).
+;
+; SAB is CCR bit 4.  CSR is then written $FF -- write-one-to-clear -- to take
+; down COC and ERR together, because the abort posts a channel error (CER $11)
+; and pg_aserv's whole test is "did the channel count out": a stale COC would
+; make the very next service call arm a lump that is already playing.
+;
+; NAME THE LAYER.  MAME's hd63450 does not NEED the abort -- its
+; dma_transfer_start reloads MAR/MTC and restarts the timer whatever the channel
+; was doing.  The MC68450 does: STR written to an active channel is an operation
+; timing error, and the transfer that is running is not the one that was asked
+; for.  The abort is here for the silicon, and the run below cannot tell.
+ad_abort:
+        move.b  #$10,A3_CCR             ; SAB -- software abort
+        move.b  #$FF,A3_CSR             ; ...and the COC/ERR it posts
+        rts

src/player/adpcmgate.s

@@ -0,0 +1,112 @@
+; Front-end for the ADPCM transport (ROADMAP P6a), for the rig.
+;
+; It plays ONE buffer of nibbles the host pushed into RAM and reports what the
+; channel did.  What is being measured is not this code -- it is the CHIP: which
+; delta formula, which nibble of a byte first, where the accumulator clamps, and
+; what it starts at.  tools/bench/verify_adpcm_chip.py reads all four out of
+; MAME's own -wavwrite capture.
+;
+; THE ONE THING THIS FILE HAS TO GET RIGHT is the order of the two starts.  The
+; chip resets its accumulator, its step index AND its nibble select when it is
+; told to PLAY, and it begins consuming immediately out of whatever its data
+; register holds.  So the channel goes first and the CPU waits for MTC to move
+; -- proof that a byte has actually been taken -- before the PLAY.  The stream
+; still has a prologue of unknown length, because the gap between PLAY and the
+; NEXT #DRQ3 is not ours to set; the prologue is 16 zero nibbles for exactly
+; that reason and the verifier reads its length off the capture.
+
+        include "src/player/geom.i"
+
+AD_FLAG = $18600                ; u32 0 idle, 1 armed, 2 playing, $FF done, $EE error
+AD_BUF  = $18604                ; u32 where the nibble bytes are
+AD_LEN  = $18608                ; u32 how many bytes
+AD_MTC0 = $1860C                ; u32 MTC at the instant PLAY was written
+AD_CSRF = $18610                ; u32 CSR at completion
+AD_CERF = $18614                ; u32 CER with it
+AD_MTCF = $18618                ; u32 MTC with it
+AD_MARF = $1861C                ; u32 MAR with it -- where the channel stopped
+AD_SPIN = $18620                ; u32 trips round the wait loop
+AD_STAT = $18624                ; u32 the chip's own status byte while playing
+AD_PATIENCE = 60000000
+AD_SETTLE   = 60000          ; ~100 ms at 10 MHz, 18 clocks a trip          ; the wait is bounded like every other
+
+        org     $10000
+start:
+        bsr     ad_setup
+
+        ; ---- SETTLE, and it is not superstition.  The 8 MHz ADPCM clock is
+        ; CT1 in the YM2151's port register, and this machine delivers that
+        ; write to the ADPCM chip on the SOUND system's own schedule rather
+        ; than at the instant of the store -- so a transfer started in the same
+        ; breath as ad_setup plays its first ~17 ms at the PREVIOUS clock.  The
+        ; symptom is exact: the capture's first 130-odd samples come out in
+        ; identical PAIRS, because the chip is clocking half as fast as the
+        ; capture, and every model then fails to fit a stream that changed rate
+        ; part way through.  ~100 ms of nothing costs the gate nothing and a
+        ; player sets its clock once at boot.
+        move.l  #AD_SETTLE,d0
+.settle:subq.l  #1,d0
+        bne.s   .settle
+
+        movea.l AD_BUF.l,a1
+        move.l  AD_LEN.l,d1
+        bsr     ad_arm
+        move.l  #1,AD_FLAG.l
+
+        ; ---- wait for the channel to actually take byte 0.  Not a delay loop:
+        ; the condition is MTC having moved, which is the channel's own account.
+        move.l  #AD_PATIENCE,d3
+.first: move.w  A3_MTC,d0
+        andi.l  #$FFFF,d0
+        cmp.l   AD_LEN.l,d0
+        bne.s   .go
+        subq.l  #1,d3
+        bne.s   .first
+        bra     bad
+
+.go:    move.l  d0,AD_MTC0.l
+        bsr     ad_play
+        move.l  #2,AD_FLAG.l
+        moveq   #0,d0
+        move.b  AD_CTRLR,d0             ; bit 7 clear = the chip says it is playing
+        move.l  d0,AD_STAT.l
+
+        clr.l   AD_SPIN.l
+        move.l  #AD_PATIENCE,d3
+.wait:  addq.l  #1,AD_SPIN.l
+        move.b  A3_CSR,d4
+        btst    #4,d4                   ; ERR -- CER says which
+        bne.s   bad
+        btst    #7,d4                   ; COC
+        bne.s   .fin
+        subq.l  #1,d3
+        bne.s   .wait
+        bra.s   bad
+
+.fin:   bsr     report
+        ; The chip is left PLAYING deliberately: it goes on replaying the last
+        ; byte it was given, which the verifier ignores.  Stopping here would
+        ; put a silence in the capture at a point the host would then have to
+        ; find, and the capture already has a length it knows.
+        move.l  #$FF,AD_FLAG.l
+hold:   bra.s   hold
+
+bad:    bsr     report
+        move.l  #$EE,AD_FLAG.l
+        bra.s   hold
+
+report:
+        moveq   #0,d0
+        move.b  A3_CSR,d0
+        move.l  d0,AD_CSRF.l
+        moveq   #0,d0
+        move.b  A3_CER,d0
+        move.l  d0,AD_CERF.l
+        move.w  A3_MTC,d0
+        andi.l  #$FFFF,d0
+        move.l  d0,AD_MTCF.l
+        move.l  A3_MAR,d0
+        move.l  d0,AD_MARF.l
+        rts
+
+        include "src/player/adpcm.i"

src/player/clock.i

@@ -0,0 +1,202 @@
+; ---------------------------------------------------------------------------
+; clock.i -- the FRAME CLOCK, on the 68000 itself.  ROADMAP P3.
+;
+; src/player/stream.s has a pace gate: frame i may not START before tick i, and
+; PACE is the tick counter.  Until now PACE was written by tools/bench/
+; stream.lua, i.e. by the host, off the host's idea of what 12 fps means.  That
+; was honest for what FINDINGS 49/51 were measuring -- arrival times against a
+; deadline -- and it is not a player.  A player has no host.  These are the
+; bytes that replace it.
+;
+; WHAT THE MACHINE ACTUALLY OFFERS, because "use the MFP timer or vblank" hides
+; a real constraint.  The MC68901's timer clock on this board is 16 MHz / 4 =
+; 4 MHz (MAME 0.277 src/mame/sharp/x68k.cpp:1027-1028), its prescaler ladder is
+; {4, 10, 16, 50, 64, 100, 200} (src/devices/machine/mc68901.cpp:173) and its
+; data register is 8 bits.  The SLOWEST tick a single MFP timer can produce is
+; therefore 4e6 / (200*256) = 78.125 Hz, and 4e6/12 = 333,333.33 is not even an
+; integer -- so no prescaler/data pair ticks at 12 Hz, and no timer at any
+; setting ticks as slowly as a 12 fps frame.  A frame clock needs a divider in
+; software whichever source it is built on.  tools/analysis/23_frame_clock.py
+; enumerates the whole space rather than asserting this.
+;
+; So the source is the RASTER, and that is a better answer than a timer anyway.
+; GPIP4 on the MFP is V-DISP (x68k.cpp:1139, `m_crtc->vdisp_cb().set(i4_w)`),
+; high while the display is active; the same signal is the MFP's Timer A event
+; input (mc68901.cpp:167, GPIO_TIMER = {GPIP_4, GPIP_3}).  Its interrupt is
+; channel 6, IR_GPIP_4 = $40 in IERB/IPRB/IMRB (mc68901.cpp:76).  We take the
+; FALLING edge (AER bit 4 = 0), which is the start of vertical blanking -- the
+; instant a player would present a finished frame, so the clock and the flip
+; are the same event rather than two events with a phase between them.
+;
+; THE DIVIDER IS EXACT, AND IT IS EXACT BY CONSTRUCTION.  The raster is
+;
+;     31,500 lines/s / (R04 + 1) lines/frame
+;
+; and 31,500 / 568 = 55.4577 Hz is not a multiple of 12, so a whole-number
+; divide cannot do it: 4 refreshes is 13.87 fps and 5 is 11.09 fps.  Instead
+; each V-DISP adds `fps * (R04+1)` to an accumulator and a frame tick is emitted
+; whenever it reaches 31,500, keeping the remainder:
+;
+;     acc += fps*VTOTAL ;  if acc >= HFREQ:  acc -= HFREQ ;  PACE += 1
+;
+; Over VTOTAL/gcd raster frames that emits exactly fps*VTOTAL/gcd ticks, so the
+; long-run rate is fps*VTOTAL/VTOTAL = fps EXACTLY, with a bounded remainder and
+; ZERO accumulated drift -- not 12.0001, not 11.9998.  It holds for any fps and
+; any vertical geometry, which is why the two constants are READ OUT OF THE
+; CRTC at init rather than assembled in: the clock is derived from the same
+; registers that generate the raster it is counting, so the two cannot disagree.
+;
+; WHAT IT COSTS IN CADENCE, WHICH IS THE PART THAT IS NOT FREE.  12 fps on a
+; 55.4577 Hz raster is 4.6215 refreshes per frame, so a frame is shown for
+; either 4 or 5 refreshes -- 72.13 ms or 90.16 ms.  Nothing can change that;
+; it is the display's quantisation, not the clock's error, and a timer-derived
+; clock would have exactly the same cadence with an arbitrary phase against the
+; raster on top.  It does mean the slot a frame gets is NOT always the 83.33 ms
+; every budget in this project is priced against, and the short slot is 13.4%
+; under it.  tools/analysis/23_frame_clock.py prices that; do not read this file
+; as a claim that the clock made the budget bigger.
+;
+; INTERRUPTS, AND WHAT HAD TO BE TURNED OFF.  The rigs launch the 68000 at
+; SR=$2700 with everything masked, into a machine the IPL ROM has already booted
+; -- so the MFP arrives with whatever IOCS enabled on it (keyboard receive,
+; Timer C, its own V-DISP handler) and vectors pointing into IOCS.  Lowering the
+; mask without disarming the MFP would vector into code we did not put there.
+; clk_init therefore writes IERA = IERB = 0 first, which on the MC68901 also
+; clears the matching pending bits (mc68901.cpp REGISTER_IERA/B: `m_ipr &=
+; m_ier`), and only then arms GPIP4 alone.  Levels 1-5 stay masked at SR=$2500,
+; so the DMAC (IRQ3) and the SCC (IRQ5) cannot get in either; level 7 is the
+; front-panel NMI and is not ours to mask.
+;
+; The vector is the MFP's own: VR is written with the S bit CLEAR, so the
+; in-service register is not used and an acknowledge clears the pending bit by
+; itself (mc68901.cpp get_vector).  No end-of-interrupt write in the handler.
+; ---------------------------------------------------------------------------
+
+; --- MFP registers.  The device sits on D0-D7 of a 16-bit bus (x68k.cpp:793,
+;     `.umask16(0x00ff)`), so register n is one BYTE at $E88001 + 2n.
+MFP     = $E88001
+MFP_GPIP  = MFP+0*2
+MFP_AER   = MFP+1*2
+MFP_DDR   = MFP+2*2
+MFP_IERA  = MFP+3*2
+MFP_IERB  = MFP+4*2
+MFP_IPRA  = MFP+5*2
+MFP_IPRB  = MFP+6*2
+MFP_ISRA  = MFP+7*2
+MFP_ISRB  = MFP+8*2
+MFP_IMRA  = MFP+9*2
+MFP_IMRB  = MFP+10*2
+MFP_VR    = MFP+11*2
+
+MFP_GPIP4 = 4                   ; bit number of V-DISP in GPIP/AER/DDR
+MFP_IVDISP = $40                ; IR_GPIP_4, channel 6, in IERB/IPRB/IMRB
+MFP_VBASE = $40                 ; vector base; S clear -> no in-service register
+CLK_VEC   = (MFP_VBASE+6)*4     ; $118: MFP channel 6 vector, as an ADDRESS
+
+CRTC    = $E80000
+CRTC_R04 = CRTC+4*2             ; V total, in scanlines, minus one
+CRTC_R20 = CRTC+20*2            ; mode; bit 4 = 31.5 kHz
+HFREQ   = 31500                 ; lines/s in the 31.5 kHz modes.  Exact: the
+                                ; 768-wide IPL mode is 34.776 MHz / 1104 dots
+                                ; and the 256-wide mode 11.592 MHz / 368, both
+                                ; 31500.0 (tools/bench/crtc_mode.lua).
+
+; --- state.  PACE is stream.s's, deliberately: the whole point is that the
+;     68000 now writes the word the host used to write, and the pace gate that
+;     reads it does not change by a single byte.
+CLK_PACE  = $18034              ; == stream.s PACE
+CLK_ACC   = $18060              ; word: Bresenham remainder, < HFREQ
+CLK_INCR  = $18062              ; word: fps * (R04+1), computed by clk_init
+CLK_VDISP = $18064              ; long: V-DISP edges taken.  An INSTRUMENT --
+                                ; it is what lets a rig check that the tick
+                                ; count and the raster count are the same clock.
+CLK_FPS   = $18068              ; long: requested fps, an argument to clk_init
+CLK_ERR   = $1806C              ; long: 0 ok / 1 not a 31.5 kHz mode
+                                ;       / 2 fps*VTOTAL would overflow 16 bits
+
+; ---------------------------------------------------------------------------
+; clk_init -- arm the frame clock.  Reads CLK_FPS, leaves CLK_ERR.
+; Clobbers d0-d2.  Leaves the CPU at SR=$2500 on success.
+; ---------------------------------------------------------------------------
+clk_init:
+        clr.l   CLK_ERR.l
+        clr.l   CLK_VDISP.l
+        clr.w   CLK_ACC.l
+        clr.l   CLK_PACE.l
+
+; The mode has to be the one HFREQ describes.  A 15 kHz mode would halve the
+; line rate and the divider would run at double speed while looking correct,
+; which is the failure this test exists to prevent.
+        move.w  CRTC_R20.l,d0
+        btst    #4,d0
+        bne.s   .modeok
+        move.l  #1,CLK_ERR.l
+        rts
+.modeok:
+; VTOTAL and the increment.  Both out of the CRTC, so a change of mode changes
+; the clock with it.  acc is 16-bit and reaches at most HFREQ-1+incr, so incr
+; must leave room: 65536 - 31500 = 34036.  At VTOTAL=568 that is fps < 59.9,
+; which is every rate this machine can display anyway -- but it is checked
+; rather than argued.
+        move.w  CRTC_R04.l,d0
+        addq.w  #1,d0                   ; VTOTAL scanlines
+        move.w  d0,d1
+        move.w  CLK_FPS+2.l,d2          ; low word of the long
+        mulu    d2,d1                   ; fps * VTOTAL  (see FINDINGS 53.4 on
+                                        ; C68K's flat MULU charge; this is boot
+                                        ; code and is not cost-measured there)
+        cmp.l   #65536-HFREQ,d1
+        bcs.s   .fitok
+        move.l  #2,CLK_ERR.l
+        rts
+.fitok:
+        move.w  d1,CLK_INCR.l
+
+; The vector, before the source is armed.
+        move.l  #clk_isr,CLK_VEC.w
+
+; Disarm everything the IPL left running, then arm GPIP4 alone.  Order matters:
+; IER first (which clears IPR with it), then the edge, then the mask.
+        move.b  #0,MFP_IERA.l
+        move.b  #0,MFP_IERB.l
+        move.b  #0,MFP_IMRA.l
+        move.b  #MFP_VBASE,MFP_VR.l     ; S clear: acknowledge clears pending
+        bclr    #MFP_GPIP4,MFP_DDR.l    ; V-DISP is an input
+        bclr    #MFP_GPIP4,MFP_AER.l    ; interrupt on the FALLING edge, i.e.
+                                        ; at the start of vertical blanking
+        move.b  #MFP_IVDISP,MFP_IERB.l
+        move.b  #MFP_IVDISP,MFP_IMRB.l
+        move.w  #$2500,sr               ; let level 6 in; 1-5 stay masked
+        rts
+
+; ---------------------------------------------------------------------------
+; clk_stop -- disarm, and put the mask back where the rigs expect it.
+; ---------------------------------------------------------------------------
+clk_stop:
+        move.w  #$2700,sr
+        move.b  #0,MFP_IERB.l
+        move.b  #0,MFP_IMRB.l
+        rts
+
+; ---------------------------------------------------------------------------
+; clk_isr -- one V-DISP.  Every instruction here is charged to every frame the
+; decoder draws, so it is deliberately the shortest thing that is still exact:
+; four word operations and one long increment.
+;
+; Only the LOW WORD of d0 is touched, so only the low word is saved.  The
+; accumulator, the increment and the threshold all fit in 16 bits by the check
+; in clk_init, which is what makes that legal.
+; ---------------------------------------------------------------------------
+clk_isr:
+        move.w  d0,-(sp)
+        addq.l  #1,CLK_VDISP.l
+        move.w  CLK_ACC.l,d0
+        add.w   CLK_INCR.l,d0
+        cmp.w   #HFREQ,d0
+        bcs.s   .nf
+        sub.w   #HFREQ,d0
+        addq.l  #1,CLK_PACE.l
+.nf:
+        move.w  d0,CLK_ACC.l
+        move.w  (sp)+,d0
+        rte

src/player/clockgate.s

@@ -0,0 +1,59 @@
+; Front-end for the frame clock (ROADMAP P3), for the rig.
+;
+; It exists to answer two questions that the streaming rig cannot answer on its
+; own, because there the clock is buried under a decoder:
+;
+;   1. does the tick actually come from the raster, and at exactly the rate
+;      asked for -- measured over thousands of refreshes, not four;
+;   2. WHAT IT COSTS, in clocks, per interrupt.  This project's currency is
+;      68000 clocks and the decoder already occupies 86.7% of the bus, so a
+;      frame clock is not free until someone has priced it.
+;
+; THE INSTRUMENT, and why it is a busy loop.  MAME's Lua only sees the machine
+; at frame boundaries, so it can time to 1/55.46 s and no finer -- 18 ms, where
+; the whole per-frame cost of this clock is microseconds.  Differencing two
+; wall timings would measure nothing.  So the 68000 counts instead: a loop with
+; ONE instruction in its body runs for a fixed number of refreshes, and the
+; iteration count is read out at both ends.
+;
+;   clock off:   iters0 * L            = clocks in the window   ->  L
+;   clock on:    iters1 * L + ints * H = clocks in the window   ->  H
+;
+; The window is an exact number of raster frames, so its length in clocks is
+; exact and does not depend on the host at all.  L is calibrated out by the
+; first run rather than assumed from a cycle table, which matters: the point of
+; the exercise is to price this code on the machine that will run it, and a
+; table is the thing being checked.  With ~2.7e7 iterations behind it, L carries
+; enough digits that the interrupt cost -- 0.08% of the window -- survives the
+; subtraction.
+;
+; The body is `addq.l #1,CGCNT.l` and nothing else: no compare, no counter in a
+; register that an interrupt could be accused of disturbing, and a value that
+; the host can read at any moment without stopping the CPU.
+;
+; The gate does NOT decode anything.  What the clock does to a real frame is
+; tools/bench/stream.lua's question, with DLX_PACE=2.
+
+CGFLAG  = $18070                ; 0 idle / 1 running / $EE clk_init refused
+CGON    = $18074                ; 1 = arm the frame clock, 0 = leave it off
+CGCNT   = $18078                ; <- loop iterations, read by the host at both
+                                ;    ends of the window
+
+        org     $10000
+start:
+        clr.l   CGCNT.l
+        move.l  CGON.l,d0
+        beq.s   noclk
+        bsr     clk_init
+        tst.l   CLK_ERR.l
+        bne.s   bad
+noclk:
+        move.l  #1,CGFLAG.l             ; the host starts its window here
+loop:
+        addq.l  #1,CGCNT.l
+        bra.s   loop
+bad:
+        move.l  #$EE,CGFLAG.l
+hold:   bra.s   hold
+
+        include "src/player/clock.i"

src/player/dma.i

@@ -0,0 +1,273 @@
+; The HD63450 driving the SCSI data phase.  ROADMAP P4a.
+;
+; WHAT P4a HAS TO SHOW, and why it needed a new kind of evidence.  FINDINGS 58
+; measured the CPU moving every byte itself at 87.28 clocks per delivered byte
+; -- 391.8% of a 12 fps frame, against 22.4% for the cheapest DMA row of the
+; ladder and 85.3% for the dearest.  So the whole of what is left before M2 is a
+; DMAC configuration that HOLDS THE BUS.  57.3 is why it could not simply be
+; watched into existence: x68k_scsiext.cpp glues $EA0015 so that with the DMAC's
+; OWN asserted -- which it is at idle on this machine -- MAME CANNOT DISTINGUISH
+; a CPU-driven byte at that address from a DMAC-driven one.  Watching the data
+; register cannot answer the question it looks like it answers.
+;
+; THE DISCRIMINATOR USED HERE IS THE CPU'S OWN PROGRESS, and it never reads
+; $EA0015 at all.  A DMAC that holds the bus is one the CPU is not running
+; against; so the witness is a single instruction:
+;
+;       move.b  #CCR_START,DM_CCR       ; the channel is told to go
+;       move.w  DM_MTC,d0               ; <- sampled by the VERY NEXT instruction
+;
+; If the bus was held for the transfer, the whole transfer happened between
+; those two instructions and d0 reads ZERO.  If it was not, d0 reads very nearly
+; the full count and the CPU goes on to spin thousands of times while the
+; channel trickles.  Both configurations deliver the same bytes; what separates
+; them is whether the 68000 got to execute anything meanwhile, which is exactly
+; what "holds the bus" means and is not a fact about $EA0015.
+;
+; WHAT MAME CAN AND CANNOT BE ASKED, stated here because it bounds the claim and
+; it is not obvious from the outside:
+;
+;   * THE CARD HAS NO EXREQ PATH.  x68k_scsiext.cpp's drq_w only stores a flag;
+;     the expansion slot has no request line to the DMAC at all (x68k.cpp wires
+;     drq0 from the FDC and drq3 from ADPCM, and nothing else).  The card's flow
+;     control is DTACK: on a DMAC cycle with DRQ low the card NEGATES DTACK and
+;     the HD63450 discards that operand and retries.  So every configuration
+;     below is AUTO-REQUEST; REQG=10, external request -- the mode the ladder's
+;     W=5 and W=12 rows assume -- has no wiring in this model and cannot be run.
+;   * SINGLE ADDRESS CANNOT BE RUN EITHER.  hd63450.cpp only takes the implicit
+;     path when a channel has a dma_read/dma_write callback, and on this machine
+;     only channel 0 (the FDC) has one.  DTYP=10/11 on channels 1..3 falls
+;     through to the dual-address code.
+;   * ONLY BURST IS MODELLED AS HELD.  The device tests `(dcr & 0xc0) == 0`, so
+;     XRM=10 (cycle steal without hold) and XRM=11 (cycle steal WITH hold) are
+;     one code path.  The bus is held, and the CPU halted, only for XRM=00 burst
+;     with REQG=01 max rate.
+;
+; So of the four rows of the per-byte ladder, exactly ONE -- dual address, bus
+; held, 9 clk/B -- has a code path in this model, and it is the one demonstrated
+; below.  That is a bound on the apparatus and not a result about the board.
+;
+; AND IT IS STILL NOT A RATE.  MAME's DMAC is configured in wall-clock attotimes
+; (42.5), not per-operand cycles: set_burst_clocks gives channel 1 450 ns an
+; operand no matter what the 68000 is doing.  `W` is untouched by every line in
+; this file and still wants a board (ROADMAP B1/B3).
+
+; ---- the channel.  1, not 0: channel 0 is the FDC's and is the one channel
+; with device callbacks, which would silently take the implicit-address path.
+; Channel 1 is also the channel the IPL ROM points at the SASI data register
+; (52.5), so this is the machine's own disk channel programmed differently.
+DMA_CH  = 1
+DMACB   = DMAC+DMA_CH*DMAC_CH           ; $E84040
+DM_CSR  = DMACB+$00                     ; channel status  (write 1s to clear)
+DM_CER  = DMACB+$01                     ; channel error   (read only)
+DM_DCR  = DMACB+$04                     ; device control
+DM_OCR  = DMACB+$05                     ; operation control
+DM_SCR  = DMACB+$06                     ; sequence control
+DM_CCR  = DMACB+$07                     ; channel control
+DM_MTC  = DMACB+$0A                     ; memory transfer count, WORD
+DM_MAR  = DMACB+$0C                     ; memory address, LONG
+DM_DAR  = DMACB+$14                     ; device address, LONG
+DM_BTC  = DMACB+$1A                     ; base transfer count, WORD (array chain)
+DM_BAR  = DMACB+$1C                     ; base address, LONG    (array chain)
+DM_MFC  = DMACB+$29
+DM_CPR  = DMACB+$2D
+DM_DFC  = DMACB+$31
+
+; CSR bits
+CSR_COC = $80                           ; channel operation complete
+CSR_BTC = $40
+CSR_NDT = $20                           ; normal device termination
+CSR_ERR = $10                           ; channel error -- CER says which
+CSR_ACT = $08                           ; channel active
+CCR_START = $80
+
+; ---- the two configurations, as (DCR, OCR) pairs.  Both are decoded by
+; tools/analysis/27_dmac_config.py out of THESE bytes, using the same MC68450
+; field tables 21_iplrom_dmac.py reads the IPL ROM's channels with -- so what
+; the run claims it programmed and what it programmed cannot drift apart.
+;
+;   HELD : DCR $00 = XRM 00 burst, DTYP 00 dual address, DPS 0 8-bit port
+;          OCR $81 = DIR device->memory, SIZE byte, no chain, REQG 01 max rate
+;   STEAL: DCR $80 = XRM 10 cycle steal WITHOUT hold, otherwise identical
+;          OCR $80 = REQG 00 auto-request at limited rate
+DM_HELD_DCR  = $00
+DM_HELD_OCR  = $81
+DM_STEAL_DCR = $80
+DM_STEAL_OCR = $80
+
+; ---- what the run reports.  Every one of these is a DMAC register or a count
+; of the CPU's own instructions; none of them is a read of $EA0015.
+; $18500 AND NOT $18300, WHICH IS WHERE THIS FIRST WENT.  scsi.i's trace ends at
+; $182FF and the next 160 bytes are the RING's: $18300 is ring.i's XF_SLOT
+; mailbox, and tools/bench/stream.lua reads the same addresses from outside.
+; dma.i is included by stream.s as well as by the gate, so DM_USE landed on the
+; transfer request slot and the ring rig's first record request read as "use the
+; DMAC" -- P4b's stage went red on a run that never reached its snapshot.  The
+; symptom was in a stage this session did not touch, which is the whole argument
+; for check.sh being run before and after rather than only after.
+DM_USE   = $18500                       ; u32 0 = PIO data phase, 1 = this file
+DM_DCRV  = $18504                       ; u32 the DCR byte to program
+DM_OCRV  = $18508                       ; u32 the OCR byte to program
+DM_MTC0  = $1850C                       ; u32 MTC one instruction after START
+DM_SPIN  = $18510                       ; u32 times the CPU went round the wait
+DM_CSRF  = $18514                       ; u32 CSR when the channel finished
+DM_CERF  = $18518                       ; u32 CER with it
+DM_MTCF  = $1851C                       ; u32 MTC with it
+DM_MARF  = $18520                       ; u32 MAR with it -- where it stopped
+DM_LEN   = $18524                       ; u32 bytes the channel was asked for
+; ---- SEQUENTIAL ARRAY CHAINING, and it is OFF unless a caller asks for it.
+; A device->GVRAM transfer cannot be one contiguous run: a picture row is 256 B
+; of a 1024 B line stride, so 192 rows want 192 destinations.  The MC68450 walks
+; an array of 6-byte {u32 MAR, u16 MTC} entries for exactly this, and MAME's
+; hd63450 implements it (`(ocr & 0x0c) == 0x08`, dma_transfer_start and
+; dma_transfer_continue).  DM_BARV = 0 means no chaining and NOTHING below
+; changes, which is what stream.s gets: this file is included by the player as
+; well as by the gate, and a mailbox that defaults to a new behaviour is how
+; DM_USE landed on ring.i's slot and turned a stage red (above).
+DM_BARV  = $18528                       ; u32 array base address, 0 = no chain
+DM_BTCV  = $1852C                       ; u32 array entry count (BTC)
+; ---- THE SERVICE HOOK, and it is the whole of what a SECOND consumer needs
+; from this file.  ROADMAP P6c.  A player that feeds ADPCM has to look at the
+; audio channel more often than once a frame: the MSM6258 has no FIFO, so the
+; instant its channel counts out the chip goes on replaying whatever byte its
+; data register still holds -- and at 12 fps a once-a-frame re-arm makes that
+; replay 90 ms long, which is not a gap, it is a buzz.
+;
+; THE ONLY PLACE A 68000 HAS TO SPARE IS INSIDE THIS WAIT.  In the STEALING
+; configuration the CPU goes round the loop below thousands of times per record
+; (measured: 1,100,520 trips over 120 frames) and every one of them is time the
+; disc is delivering and the CPU is not.  So the hook is called from there, and
+; the second consumer costs the video path nothing it was using.
+;
+; AND IN THE HELD CONFIGURATION THE HOOK CANNOT RUN AT ALL, which is not a bug
+; in it: a burst channel HALTS the 68000 (dma_transfer_start asserts
+; INPUT_LINE_HALT) and the CPU does not execute the loop, or anything else,
+; until the record has landed.  That asymmetry is the measurement -- FINDINGS
+; 64.3 showed a held channel costs the frame CLOCK half its ticks, and this is
+; the same fact reaching the audio.
+;
+; ZERO BY DEFAULT and every other front-end in this tree leaves it zero, so the
+; cost to them is a `move.l` and a `beq` per trip.  A mailbox that defaulted to
+; a new behaviour is how DM_USE landed on ring.i's slot (above).
+DM_HOOK  = $18530                       ; u32 0 = none, else a routine to call
+                                        ; on every trip round the transfer wait.
+                                        ; d0 is dead here and a0 is saved round
+                                        ; the call, so the hook may trash both;
+                                        ; it must preserve EVERYTHING else,
+                                        ; because sc_in_dma's own d3/d4/d5 and
+                                        ; scsi_read's a1 are live across it.
+DM_PATIENCE = 4000000                   ; the wait is bounded like every other
+
+; ---------------------------------------------------------------- sc_in_dma
+; Receive d1 bytes into (a1) in phase d2, WITHOUT the CPU touching one of them.
+; Entered from sc_in_data when DM_USE is set; same registers, same contract.
+;
+; ORDER MATTERS AND IT IS NOT THE OBVIOUS ONE.  The SPC is put into DMA transfer
+; BEFORE the channel is started, because in the held configuration the 68000
+; stops executing at the CCR write and does not run again until the transfer is
+; over -- so anything the SPC needs to be told has to have been told already.
+sc_in_dma:
+        movem.l d3-d5,-(sp)
+        move.l  d1,d5                   ; keep the length for the report
+        move.l  d5,DM_LEN.l
+        move.b  d2,SC_PCTL
+        move.l  d1,d0
+        bsr     sc_settc                ; the SPC counts the same bytes down
+
+        ; ---- the channel, quiet first: CSR is write-one-to-clear and a stale
+        ; COC from a previous record would pass the wait loop instantly.
+        move.b  #$FF,DM_CSR
+        move.l  DM_DCRV.l,d0
+        move.b  d0,DM_DCR
+        move.l  DM_OCRV.l,d0
+        move.b  d0,DM_OCR
+        move.b  #$04,DM_SCR             ; MAC 01 memory increment, DAC 00 none:
+                                        ; the device address is a REGISTER and
+                                        ; must not walk off it.
+        move.b  #$05,DM_MFC             ; the function codes the IPL ROM uses
+        move.b  #$05,DM_DFC
+        move.b  #$01,DM_CPR
+        move.l  DM_BARV.l,d0
+        bne.s   .chain
+        move.w  d5,DM_MTC
+        move.l  a1,DM_MAR
+        bra.s   .darset
+.chain:
+        ; MAR and MTC are NOT written: the channel loads both from the array's
+        ; first entry when it starts, and reloads them from the next entry at
+        ; every count-out.  Writing them here would be writing registers the
+        ; hardware is about to overwrite, which reads like a contract and is not.
+        move.l  d0,DM_BAR
+        move.l  DM_BTCV.l,d0
+        move.w  d0,DM_BTC
+.darset:
+        move.l  #SC_DREG,DM_DAR         ; $EA0015 -- the DMAC's door, and now
+                                        ; the DMAC is the one going through it
+        move.b  #SCMD_XFER,SC_SCMD      ; no PROGRAM bit: the SPC raises DRQ
+        move.l  #11,SC_TAG.l            ; 11 = channel armed, SPC in DMA mode
+        bsr     sc_snap
+
+        ; ---- START, and the witness immediately after it
+        move.b  #CCR_START,DM_CCR
+        move.w  DM_MTC,d0               ; THE DISCRIMINATOR.  Held: zero.
+        andi.l  #$FFFF,d0
+        move.l  d0,DM_MTC0.l
+
+        ; ---- wait for the channel, counting the CPU's own trips round the loop.
+        ; In the held configuration this is one trip, because the CPU did not
+        ; get to run until the transfer was over.  In the stealing one it is
+        ; thousands, and every one of them is a 68000 instruction that executed
+        ; while the disc was delivering -- which is the whole point of P4a.
+        clr.l   DM_SPIN.l
+        move.l  #DM_PATIENCE,d3
+.wait:  addq.l  #1,DM_SPIN.l
+        move.l  DM_HOOK.l,d0            ; the second consumer's slot -- see above
+        beq.s   .nohook
+        move.l  a0,-(sp)
+        movea.l d0,a0
+        jsr     (a0)
+        movea.l (sp)+,a0
+.nohook:
+        move.b  DM_CSR,d4
+        btst    #4,d4                   ; ERR
+        bne.s   .err
+        btst    #7,d4                   ; COC
+        bne.s   .fin
+        subq.l  #1,d3
+        bne.s   .wait
+        bsr     .report
+        movem.l (sp)+,d3-d5
+        move.l  #SCE_TIMEOUT,SC_ERR.l
+        moveq   #-1,d0
+        rts
+.err:   bsr     .report
+        movem.l (sp)+,d3-d5
+        move.l  #SCE_TIMEOUT,SC_ERR.l   ; a channel error is a dead transport
+        moveq   #-1,d0
+        rts
+.fin:   bsr     .report
+        move.b  #$FF,DM_CSR             ; leave the channel as we found it
+        move.l  #12,SC_TAG.l            ; 12 = channel reported COC
+        bsr     sc_snap
+        movem.l (sp)+,d3-d5
+        bsr     sc_xferend              ; the SPC's own transfer, not the DMAC's
+        move.l  d0,-(sp)
+        move.l  #9,SC_TAG.l
+        bsr     sc_snap
+        move.l  (sp)+,d0
+        rts
+
+; ---- the channel's own account of what it did, read out of its registers
+.report:
+        moveq   #0,d0
+        move.b  DM_CSR,d0
+        move.l  d0,DM_CSRF.l
+        moveq   #0,d0
+        move.b  DM_CER,d0
+        move.l  d0,DM_CERF.l
+        move.w  DM_MTC,d0
+        andi.l  #$FFFF,d0
+        move.l  d0,DM_MTCF.l
+        move.l  DM_MAR,d0
+        move.l  d0,DM_MARF.l
+        rts

src/player/dmagate.s

@@ -0,0 +1,403 @@
+; Front-end for the HD63450 DATA PHASE (ROADMAP P4a), for the rig.
+;
+; THE QUESTION.  FINDINGS 58 put the transport on the 68000 and priced it: the
+; CPU moving every byte itself costs 87.28 clocks per delivered byte, 391.8% of
+; a 12 fps frame.  Against that, the cheapest DMA row of the ladder is 22.4% and
+; the dearest is 85.3%, so everything left before M2 turns on getting the DMAC
+; to drive the data phase with the bus HELD.  57.3 is why it cannot be shown by
+; watching the data register: with the DMAC's OWN asserted, which it is at idle
+; here, MAME cannot tell a CPU-driven byte at $EA0015 from a DMAC-driven one.
+;
+; THE EVIDENCE THIS GATE PRODUCES, and none of it is a read of $EA0015:
+;
+;   1. THE SAME BYTES.  The same sectors are read three times -- once by the PIO
+;      path FINDINGS 58 measured, once by the channel with the bus held, once by
+;      the channel stealing cycles -- and the HOST compares all three against its
+;      own copy of the image.  A transport that returns the wrong bytes without
+;      saying so is the failure a checksum-free ring cannot survive (49.2).
+;   2. THE CPU'S OWN PROGRESS.  MTC is sampled by the INSTRUCTION AFTER the one
+;      that starts the channel.  Held, it reads zero: the entire transfer
+;      happened between two instructions, because the 68000 did not execute in
+;      between.  Stealing, it reads nearly the full count and the CPU then goes
+;      round its wait loop thousands of times while the bytes arrive.  That
+;      difference IS "the DMAC held the bus", and it is a fact about the CPU.
+;   3. THE CHANNEL'S OWN ACCOUNT.  CSR, CER, the final MTC and the final MAR:
+;      the channel says it completed without error, moved every byte, and left
+;      its memory pointer exactly one transfer-length past where it started.
+;   4. THE WINDOW IS REFUSED.  A windowed read (58.3: 117 of 120 records start
+;      part way into a sector) is rejected by the transport rather than silently
+;      delivering the neighbouring records' bytes into the ring.  P4a's
+;      precondition is stated by the code that has it, not by a comment.
+;
+; WHAT IT DOES NOT SHOW.  Not `W`.  Not one clock of it.  MAME's DMAC runs on
+; wall-clock attotimes (42.5) and its burst mode halts the CPU outright rather
+; than costing it cycles per operand, so this gate settles WHICH CONFIGURATION
+; WORKS and not what one costs.  See src/player/dma.i for the three ways this
+; model bounds the question -- no EXREQ wiring, no single-address path, and only
+; burst modelled as held.
+
+DGFLAG  = $18600                ; 0 idle / 1 done
+DGREC   = $18800                ; 9 x 32 B: rc, err, mtc0, spin, csr, cer, mtc, mar
+                                ; $18800 AND NOT $18610, WHERE THIS LIVED: nine
+                                ; records of 32 B run to $188FF, and from $18610
+                                ; they would have run over DGWIN at $18700 --
+                                ; the window run's own result, which run 10 then
+                                ; writes back.  A silent overlap between two
+                                ; runs' evidence is the kind of thing that makes
+                                ; a gate report the wrong run's numbers.
+DGREC_SZ = 32
+DGWIN   = $18700                ; u32 return of the WINDOWED dma read (want -1)
+DGWERR  = $18704                ; u32 SC_ERR after it            (want SCE_WINDOW)
+DGR20   = $18708                ; u32 R20 as it stood during run 4
+DGR20N  = $1870C                ; u32 R20 as it stood during run 5 (the control)
+DGR20C  = $18710                ; u32 R20 as it stood during run 6 (chained)
+DGR20P  = $18714                ; u32 R20 as it stood during run 9 (palette+rows)
+R20_BUF = $0916                 ; 256-colour, 31.5 kHz, G-VRAM SET TO BUFFER
+R20_MSK = $0116                 ; the SAME, bit 11 CLEAR -- the negative control
+DGDST4  = $C0C000               ; where the control writes
+DGCHA   = $19000                ; the array-chain table: 8 x {u32 MAR, u16 MTC}
+DGCHN   = 8                     ; entries
+DGCROW  = 256                   ; bytes an entry carries -- one packed picture row
+DGDST5  = $C10000               ; and the 8 row bases, at the 1024 B line stride
+CRTC20  = $E80028               ; CRTC R20
+DGDST3  = $C08000               ; DMA straight into GVRAM, BUFFER MODE (47.6.2)
+DGLBA   = 1000                  ; a NON-ZERO LBA throughout: a driver that emits
+                                ; a malformed LBA field still passes LBA 0
+DGBLK   = 4                     ; 4 x 512 = 2,048 B
+DGDST0  = $20000                ; PIO
+DGDST1  = $24000                ; DMA, bus held
+DGDST2  = $28000                ; DMA, cycle stealing
+
+; ---- runs 7-9: THE PALETTE (ROADMAP K1, FINDINGS 61.9's first open item).
+DGPAL   = $E82000               ; the GRAPHIC palette: 256 words, GGGGGRRRRRBBBBBI
+DGPALN  = 256                   ; words in it -- and 512 B is exactly ONE sector
+DGPBLK  = 1                     ; so the whole palette is one block off the disc
+DGPOIS  = $A500                 ; the poison: word i = DGPOIS|i.  A palette that
+                                ; still reads this was not written by anything.
+DGDST6  = $2C000                ; run 8's destination: RAM, so the palette is
+                                ; left alone and must still read poison
+DGDST7  = $C14000               ; run 9's six GVRAM rows, at the 1024 B stride
+DGCHA2  = $19100                ; run 9's array: 7 x {u32 MAR, u16 MTC}
+DGCHN2  = 7                     ; palette 512 B + 6 rows x 256 B = 2,048 B
+DGCROW2 = 256
+DGPS7   = $1A000                ; the palette as it stood after run 7 ...
+DGPS8   = $1A200                ; ... after run 8 (the control: poison) ...
+DGPS9   = $1A400                ; ... and after run 9 (chained).  SNAPSHOTS, not
+                                ; a late read: each run overwrites the previous
+                                ; run's palette, so a host that looks once at
+                                ; the end sees only the last of the three.
+
+        org     $10000
+start:
+        clr.l   DGFLAG.l
+
+; ---- 1. the PIO path, unchanged, as the reference the other two are measured
+; against.  It is here so that a DMA failure cannot be confused with a SCSI
+; protocol failure: if this one is wrong, nothing below is about the DMAC.
+        bsr     scsi_init
+        move.l  #DGLBA,d3
+        moveq   #DGBLK,d4
+        lea     DGDST0,a1
+        bsr     scsi_read
+        lea     DGREC,a0
+        bsr     dg_save
+
+; ---- 2. the channel, WITH THE BUS HELD
+        bsr     scsi_init
+        move.l  #DM_HELD_DCR,DM_DCRV.l
+        move.l  #DM_HELD_OCR,DM_OCRV.l
+        move.l  #1,DM_USE.l             ; after scsi_init, which clears it
+        move.l  #DGLBA,d3
+        moveq   #DGBLK,d4
+        lea     DGDST1,a1
+        bsr     scsi_read
+        lea     DGREC+DGREC_SZ,a0
+        bsr     dg_save
+
+; ---- 3. the channel, STEALING CYCLES.  Same bytes, same code, two register
+; values different -- which is what makes the comparison a comparison.
+        bsr     scsi_init
+        move.l  #DM_STEAL_DCR,DM_DCRV.l
+        move.l  #DM_STEAL_OCR,DM_OCRV.l
+        move.l  #1,DM_USE.l
+        move.l  #DGLBA,d3
+        moveq   #DGBLK,d4
+        lea     DGDST2,a1
+        bsr     scsi_read
+        lea     DGREC+2*DGREC_SZ,a0
+        bsr     dg_save
+
+; ---- 4. THE CHANNEL WRITING GVRAM, IN BUFFER MODE.  47.6.2: "the DMAC has not
+; been near this" -- 44.7 costed a device->GVRAM transfer and 47 built the packed
+; layout, and no run in this tree has ever pointed a channel at $C00000.  Two
+; separate things are being asked at once and both are write-path questions:
+;
+;   a. can a channel write GVRAM AT ALL?  Nothing says it cannot -- the DMAC
+;      writes through the same program address space the CPU does -- but a
+;      transport that silently drops its writes into a device handler is exactly
+;      the failure this gate exists to catch, and the SPC's own bytes make a
+;      better witness than a fill pattern.
+;   b. does a BYTE-wide channel fill the PACKED layout?  47.1 measured the write
+;      path with word writes from the CPU.  A dual-address channel with an 8-bit
+;      device port writes BYTES, and MAME's gvram_w in buffer mode passes
+;      mem_mask straight through -- so an even byte should land in the HIGH half
+;      of its word (page 1) and an odd byte in the LOW half (page 0).  If it
+;      does, a linear DMA of a stream interleaved (right<<8)|left -- which is
+;      exactly show_frame256_packed.lua's layout -- fills the screen with no CPU
+;      in the loop at all.
+;
+; R20 bit 11 is left SET across the readback on purpose: gvram_r returns the raw
+; word in buffer mode and the masked byte outside it, so clearing it first would
+; hide half of what is being measured.
+;
+; R20 IS WRITTEN OUTRIGHT AND NOT OR-ED INTO.  The first cut of this run set bit
+; 11 on top of whatever the IPL left, and the IPL leaves $0B16 (22.1) -- bit 11
+; ALREADY SET, and COL = %11, the 65,536-colour setup, which writes whole words
+; with or without the bit.  The run passed and proved nothing: it was a test that
+; could not fail.  Run 5 below is the control that makes this one mean something,
+; and the two differ in EXACTLY BIT 11.
+        bsr     scsi_init
+        move.w  #R20_BUF,CRTC20.l
+        move.l  #R20_BUF,DGR20.l
+        move.l  #DM_HELD_DCR,DM_DCRV.l
+        move.l  #DM_HELD_OCR,DM_OCRV.l
+        move.l  #1,DM_USE.l
+        move.l  #DGLBA,d3
+        moveq   #DGBLK,d4
+        lea     DGDST3,a1
+        bsr     scsi_read
+        lea     DGREC+3*DGREC_SZ,a0
+        bsr     dg_save
+
+; ---- 5. THE NEGATIVE CONTROL: the same transfer with bit 11 CLEAR.  In masked
+; 256-colour mode gvram_w takes `data & 0x00ff` and IGNORES mem_mask, so a byte
+; written to an EVEN address -- where the 68000 puts the MS byte, and where every
+; other disc byte lands -- contributes nothing and cannot be read back.  Half the
+; transfer must be lost, and if it is not, run 4 was not measuring the bit.
+        bsr     scsi_init
+        move.w  #R20_MSK,CRTC20.l
+        move.l  #R20_MSK,DGR20N.l
+        move.l  #DM_HELD_DCR,DM_DCRV.l
+        move.l  #DM_HELD_OCR,DM_OCRV.l
+        move.l  #1,DM_USE.l
+        move.l  #DGLBA,d3
+        moveq   #DGBLK,d4
+        lea     DGDST4,a1
+        bsr     scsi_read
+        lea     DGREC+4*DGREC_SZ,a0
+        bsr     dg_save
+        move.w  #R20_BUF,CRTC20.l       ; back to buffer mode, so the host reads
+                                        ; RAW WORDS out of both destinations
+
+; ---- 6. THE ROW STRIDE, WHICH IS THE REAL SHAPE OF THE TRANSFER.  Runs 4 and 5
+; wrote 2,048 contiguous bytes, and no picture is contiguous: a packed row is
+; 256 B of a 1024 B line stride, so a frame is 192 destinations and not one.
+; 46.6 said "no stride for a DMAC to skip" about the bytes WITHIN a row and left
+; the rows themselves unexamined; a channel cannot skip 768 B any more than it
+; could skip the 300 B in front of a record (run 10 below).
+;
+; The MC68450 answers this with SEQUENTIAL ARRAY CHAINING -- an array of 6-byte
+; {u32 MAR, u16 MTC} entries it walks by itself -- and MAME implements it.  So
+; the question is not whether the CPU can restart the channel 192 times a frame;
+; it is whether it has to at all.  Eight rows here, out of the same LBA and the
+; same 2,048 B, so the ONLY difference from run 4 is where the bytes land.
+        bsr     scsi_init
+        move.w  #R20_BUF,CRTC20.l
+        move.l  #R20_BUF,DGR20C.l
+        lea     DGCHA,a0
+        lea     DGDST5,a1
+        moveq   #DGCHN-1,d5
+dg_mkch:
+        move.l  a1,(a0)+                ; MAR: this row's base
+        move.w  #DGCROW,(a0)+           ; MTC: 256 bytes of it
+        lea     1024(a1),a1             ; the next row is a line stride away
+        dbra    d5,dg_mkch
+        move.l  #DM_HELD_DCR,DM_DCRV.l
+        move.l  #DM_HELD_OCR|$08,DM_OCRV.l      ; OCR CHAIN = %10, array chain
+        move.l  #DGCHA,DM_BARV.l
+        move.l  #DGCHN,DM_BTCV.l
+        move.l  #1,DM_USE.l
+        move.l  #DGLBA,d3
+        moveq   #DGBLK,d4
+        lea     DGDST5,a1               ; ignored under chaining; passed so the
+                                        ; call site reads the same as the others
+        bsr     scsi_read
+        lea     DGREC+5*DGREC_SZ,a0
+        bsr     dg_save
+        clr.l   DM_BARV.l               ; and OFF again, so run 7 is unchained
+
+; ---- 7. THE PALETTE.  Can a channel write $E82000?  (FINDINGS 61.9, ROADMAP
+; K1.)  Runs 4-6 put the PICTURE on the channel; a packed frame is a picture AND
+; a palette, and if the palette registers take a byte-wide DMA the way GVRAM
+; does in buffer mode then the palette is a 193rd array-chain entry and ONE
+; channel start paints a whole frame -- no per-frame CPU work in the video path
+; at all.  If they do not, the CPU writes 256 words a frame (61.9 derives that
+; at ~2,370 clocks, 0.28% of a frame) and the architecture still stands; this is
+; the difference between cheap and free, and it is worth one run to know which.
+;
+; 512 B IS THE WHOLE PALETTE AND EXACTLY ONE SECTOR, which is why this run reads
+; one block where the others read four: a transfer that ran long would write
+; $E82200 (the TEXT palette) and then $E82400 (the video controller's own
+; registers, priority included), and a probe that reconfigures the video
+; controller as a side effect is not a probe.
+;
+; THE PALETTE IS POISONED FIRST, and that is what stops this being run 4's trap
+; a second time.  A destination that already holds the right bytes cannot tell a
+; channel that wrote them from a channel that did nothing; RAM at $20000 was
+; zero and the record is mostly pad, so "it matches" has been a weak claim all
+; session.  Word i is set to $A500|i, which no 512 B of container matches by
+; accident, and the host reports how many of the 512 positions the poison and
+; the disc actually differ in rather than assuming all of them.
+        bsr     scsi_init
+        bsr     dg_poison
+        move.l  #DM_HELD_DCR,DM_DCRV.l
+        move.l  #DM_HELD_OCR,DM_OCRV.l
+        move.l  #1,DM_USE.l
+        move.l  #DGLBA,d3
+        moveq   #DGPBLK,d4
+        lea     DGPAL,a1
+        bsr     scsi_read
+        lea     DGPS7,a1
+        bsr     dg_palsnap              ; before run 8 overwrites it
+        lea     DGREC+6*DGREC_SZ,a0
+        bsr     dg_save
+
+; ---- 8. THE NEGATIVE CONTROL, and it is an ATTRIBUTION control rather than a
+; mechanism one.  Run 5's control could point at a mode bit; there is no mode
+; bit here, so what has to be excluded is that run 7's palette held the disc's
+; bytes for some reason OTHER than the channel having written them there --
+; a readback that aliases somewhere else, the SPC's own path touching the
+; registers, the poison never having landed.
+;
+; Same transfer, same channel, same bytes; ONE thing different, the destination
+; address.  The disc's bytes must appear at $2C000, and the palette must still
+; read poison in all 256 words.  Two claims from one run, and the second is the
+; one that makes run 7 mean anything: it also proves the CPU's poison writes
+; reach the registers the host reads back, which is the positive half.
+        bsr     scsi_init
+        bsr     dg_poison
+        move.l  #DM_HELD_DCR,DM_DCRV.l
+        move.l  #DM_HELD_OCR,DM_OCRV.l
+        move.l  #1,DM_USE.l
+        move.l  #DGLBA,d3
+        moveq   #DGPBLK,d4
+        lea     DGDST6,a1
+        bsr     scsi_read
+        lea     DGPS8,a1
+        bsr     dg_palsnap              ; must be poison, word for word
+        lea     DGREC+7*DGREC_SZ,a0
+        bsr     dg_save
+
+; ---- 9. THE 193rd ENTRY: one start, the palette AND the picture rows.  This is
+; the run K1 exists for.  Runs 7 and 8 only show that a channel can write the
+; palette registers; what the architecture needs is that ONE array chain can
+; cross from a hardware register area into GVRAM without the CPU between them,
+; because a frame is a palette entry followed by 192 row entries and the whole
+; claim is that the CPU starts the channel once.
+;
+; Seven entries, 2,048 B, out of the same LBA as everything else: 512 B into the
+; palette and then six rows of 256 B at the 1024 B line stride.  The destination
+; regions are of two different KINDS -- device registers and video RAM in buffer
+; mode -- which is exactly the crossing that has never been run.
+        bsr     scsi_init
+        bsr     dg_poison
+        move.w  #R20_BUF,CRTC20.l
+        move.l  #R20_BUF,DGR20P.l
+        lea     DGCHA2,a0
+        move.l  #DGPAL,(a0)+            ; entry 0: the palette, a whole sector
+        move.w  #DGPALN*2,(a0)+
+        lea     DGDST7,a1
+        moveq   #DGCHN2-2,d5            ; the remaining six are picture rows
+dg_mkch2:
+        move.l  a1,(a0)+
+        move.w  #DGCROW2,(a0)+
+        lea     1024(a1),a1
+        dbra    d5,dg_mkch2
+        move.l  #DM_HELD_DCR,DM_DCRV.l
+        move.l  #DM_HELD_OCR|$08,DM_OCRV.l      ; OCR CHAIN = %10, array chain
+        move.l  #DGCHA2,DM_BARV.l
+        move.l  #DGCHN2,DM_BTCV.l
+        move.l  #1,DM_USE.l
+        move.l  #DGLBA,d3
+        moveq   #DGBLK,d4
+        lea     DGDST7,a1               ; ignored under chaining, as in run 6
+        bsr     scsi_read
+        lea     DGPS9,a1
+        bsr     dg_palsnap
+        lea     DGREC+8*DGREC_SZ,a0
+        bsr     dg_save
+        clr.l   DM_BARV.l               ; and OFF again, so run 10 is unchained
+
+; ---- 10. and a WINDOWED read through the channel, which must be REFUSED.  This
+; is the one test here that is expected to fail, and it has to fail LOUDLY: the
+; alternative is a channel writing a whole sector into a ring that has room for
+; a record, over the top of records the decoder has not finished with.
+        bsr     scsi_init
+        move.l  #DM_HELD_DCR,DM_DCRV.l
+        move.l  #DM_HELD_OCR,DM_OCRV.l
+        move.l  #1,DM_USE.l
+        move.l  #300,SC_WSKIP.l         ; a record that starts 300 B into a sector
+        move.l  #1024,SC_WKEEP.l
+        move.l  #DGLBA,d3
+        moveq   #DGBLK,d4
+        lea     DGDST2,a1
+        bsr     scsi_read_win
+        move.l  d0,DGWIN.l
+        move.l  SC_ERR.l,DGWERR.l
+
+        move.l  #1,DGFLAG.l
+hold:   bra.s   hold
+
+; ---- one config's result, copied out of the shared reporting words before the
+; next run overwrites them.  d0 = scsi_read's return; a0 = where it goes.
+dg_save:
+        move.l  d0,(a0)+
+        move.l  SC_ERR.l,(a0)+
+        move.l  DM_MTC0.l,(a0)+
+        move.l  DM_SPIN.l,(a0)+
+        move.l  DM_CSRF.l,(a0)+
+        move.l  DM_CERF.l,(a0)+
+        move.l  DM_MTCF.l,(a0)+
+        move.l  DM_MARF.l,(a0)+
+        ; and clear them, so a config that never reached the channel reports
+        ; zeros of its own rather than the previous config's numbers.
+        clr.l   DM_MTC0.l
+        clr.l   DM_SPIN.l
+        clr.l   DM_CSRF.l
+        clr.l   DM_CERF.l
+        clr.l   DM_MTCF.l
+        clr.l   DM_MARF.l
+        rts
+
+; ---- the poison.  Word i of the graphic palette <- $A500|i, written by the
+; 68000 itself.  Two jobs: a destination that cannot hold the right answer by
+; accident, and a known pattern the control run reads back out of the registers
+; to show the readback path shows what is actually in them.
+dg_poison:
+        movem.l d0-d2/a0,-(sp)
+        lea     DGPAL,a0
+        moveq   #0,d1                   ; i
+        move.w  #DGPALN-1,d0
+.p:     move.w  d1,d2
+        ori.w   #DGPOIS,d2              ; $A500|i, and i never exceeds 255
+        move.w  d2,(a0)+
+        addq.w  #1,d1
+        dbra    d0,.p
+        movem.l (sp)+,d0-d2/a0
+        rts
+
+; ---- the palette as it stands, copied to (a1) by the 68000 READING THE
+; REGISTERS.  A snapshot and not a late host read: each palette run overwrites
+; the last one's result, so all three have to be kept while they are true.  It
+; is also the same shape of evidence 53.3 used for pal_pack -- the words come
+; back out of $E82000 rather than out of the buffer they were built in.
+dg_palsnap:
+        movem.l d0/a0-a1,-(sp)
+        lea     DGPAL,a0
+        move.w  #DGPALN-1,d0
+.s:     move.w  (a0)+,(a1)+
+        dbra    d0,.s
+        movem.l (sp)+,d0/a0-a1
+        rts
+
+        include "src/player/scsi.i"

src/player/geom.i

@@ -25,3 +25,17 @@ SPCU    = 12                    ; bytes of code per COARSE span unit (24 px)
 SPCN    = 11                    ; coarse units: 11*24 = 264 px >= one row
 SPFU    = 2                     ; bytes of code per FINE span unit (2 px)
 SPFN    = 11                    ; fine units: 11*2 = 22 px > one coarse unit
+
+; RECORD ALIGNMENT, and it is a property of the CONTAINER rather than of the
+; 68000.  DLX2 padded each record up to 4, which is all `move.l (a0)+` needs
+; (FINDINGS 28.3).  DLX5 pads up to a 512 B SECTOR, so that a DMA channel can
+; read a record as whole sectors straight into the ring with no window and no
+; bounce copy -- `sc_in_data` REFUSES a windowed read when the data phase is
+; the channel's (59.4), and 117 of 120 records needed one under DLX4.
+;
+; The consumer has to know it too: the decoder releases the ring up to the end
+; of the record it was handed, and a decoder that released only the bytes it
+; READ would leave the pad unreclaimed and drift the producer's free-space
+; arithmetic by up to RECALN-1 per record.  The ring base must therefore be
+; RECALN-aligned, which is asserted where the ring is placed.
+RECALN  = 512

src/player/load.i

@@ -0,0 +1,256 @@
+; ---------------------------------------------------------------------------
+; load.i -- the two LOAD-TIME transforms, on the 68000 itself.  ROADMAP P1+P2.
+;
+; Until now both of these were done host-side, in tools/bench/dlxload.py, and
+; the rigs pushed the RESULT into emulated RAM.  That was the right call while
+; the inner loop was the thing being measured -- charging a once-per-scene cost
+; to the per-frame path would have flattered or damned it for no reason -- but
+; a player has no host.  These are the bytes that replace it.
+;
+; The reference is tools/bench/dlxload.py and it stays the reference: this code
+; is gated BYTE-FOR-BYTE against it (tools/bench/verify_load.py), palette words
+; and darkest-entry index included.  If the two ever disagree, the symptom in a
+; rig would be wrong colours rather than a crash, which is exactly the class of
+; bug the split was made to prevent.
+;
+; WHAT IT READS.  The RAW container as it comes off the disc.  The DLX header is
+; fixed-layout and big-endian (tools/encoder/dlx.py):
+;   +0  magic 'DLX3'          +12 k1 u16      +16 off_pal  u32
+;   +4  W u16                 +14 k4 u16      +20 off_cb1  u32
+;   +6  H u16                                 +24 off_cb4  u32
+;   +8  fps u16                               +28 off_frm  u32
+;   +10 nframes u16
+; The three offsets are container-relative, so every one of them is an add of
+; the base the loader was handed.  Nothing here parses a frame record.
+;
+; WHAT IT WRITES.  CB1 (8 KB) and CB4 (2 KB) expanded to one WORD per pixel at
+; the addresses geom.i names, and 256 packed palette words straight into the
+; graphics palette at $E82000.  It also reports the darkest entry, which is what
+; the letterbox is filled with until the encoder reserves a black one (23.4,
+; still open).
+;
+; WHY WORD-PER-PIXEL.  The block loop movems codebook entries straight into
+; GVRAM with no unpacking, and the high byte of a GVRAM word write is discarded
+; by the hardware, so the high byte is left zero and never has to be cleared.
+; It also makes index scaling a shift rather than a multiply (lsl.w #5 / #3).
+;
+; SCRATCH.  Three tables, built here and dead the moment the palette is packed:
+;   P6TAB  64 B    6-bit level -> the 8-bit value the hardware renders it as
+;   SQTAB  256 B   the square of that, so the darkest-entry search has no muls
+;   DTAB   512 B   err(v, I=0) - err(v, I=1) per 8-bit channel value, signed
+; DTAB is what turns P2's per-entry minimum-squared-error choice of the shared
+; LSB into three table reads and a sign test.  Choosing I per entry rather than
+; fixing it is worth 1.96 dB (FINDINGS 23.3), and it is a per-ENTRY decision
+; across three channels, so it cannot be folded into a per-channel table alone.
+; ---------------------------------------------------------------------------
+
+LFLAG   = $18040                ; 0 idle / 1 running / $FF done / $EE bad header
+LHDR    = $18044                ; -> raw container base
+LDARK   = $18048                ; <- index of the darkest palette entry
+LK1     = $1804C                ; <- k1, as the 68000 read it out of the header
+LK4     = $18050                ; <- k4
+LMODE   = $18054                ; bit0 codebooks, bit1 palette entries,
+                                ; bit2 the three scratch tables
+LITER   = $18058                ; repeat count, so a 55 Hz host clock can time it
+
+P6TAB   = $19000                ; 64 bytes
+SQTAB   = $19040                ; 64 longs
+DTAB    = $19140                ; 256 words
+GPAL    = $E82000               ; graphics palette, 256 words
+
+; ---------------------------------------------------------------- do_load
+; in:  a0 = container base, d1 = mode bits: 1 codebooks, 2 palette entries,
+;      4 the scratch tables.  A player builds the tables ONCE at boot (they
+;      describe the hardware's colour rendering and nothing about the scene) and
+;      then loads each scene with 3.
+; out: d0 = 0 ok, -1 not a DLX3/DLX4 container.  a0-a4 clobbered, a5 = base.
+;
+; The magic is accepted as 'DLX' plus a version byte of '3' OR ABOVE rather than
+; as one constant.  DLX4 (ROADMAP P5) adds the per-record index and a fifth
+; header offset at +32; every field this routine reads is at its DLX3 place, so
+; the transforms are version-independent and the check should be too.  A version
+; this loader has never seen is still refused -- '3' or above, not "anything
+; that begins DLX".
+do_load:
+        movea.l a0,a5
+        move.l  (a5),d0
+        andi.l  #$FFFFFF00,d0
+        cmpi.l  #$444C5800,d0           ; 'DLX'
+        bne     .bad
+        cmpi.b  #'3',3(a5)              ; ... version 3 or above
+        bcs     .bad
+        move.w  12(a5),d0
+        ext.l   d0
+        move.l  d0,LK1.l
+        move.w  14(a5),d0
+        ext.l   d0
+        move.l  d0,LK4.l
+
+        btst    #2,d1
+        beq.s   .notab
+        move.l  d1,-(sp)
+        bsr     pal_tables
+        move.l  (sp)+,d1
+.notab:
+        btst    #0,d1
+        beq.s   .nocb
+        moveq   #0,d2                   ; the count is built as a LONG and the
+        move.w  12(a5),d2               ; high word must not carry junk into it
+        lsl.l   #4,d2                   ; k1 entries x 16 source bytes
+        movea.l 20(a5),a0
+        adda.l  a5,a0
+        lea     CB1,a1
+        bsr     expand
+        moveq   #0,d2
+        move.w  14(a5),d2
+        lsl.l   #2,d2                   ; k4 entries x 4 source bytes
+        movea.l 24(a5),a0
+        adda.l  a5,a0
+        lea     CB4,a1
+        bsr     expand
+.nocb:
+        btst    #1,d1
+        beq.s   .nopal
+        bsr     pal_pack
+.nopal:
+        moveq   #0,d0
+        rts
+.bad:   moveq   #-1,d0
+        rts
+
+; ---------------------------------------------------------------- expand
+; One source byte -> one destination word, high byte zero.
+; in: a0 src, a1 dst, d2 = source byte count.  Always a multiple of 4: CB1 is
+; k1*16 and CB4 is k4*4, so no remainder case can exist and none is written.
+; A junk high word here is not a slow path, it is a WRONG one: `lsr.l #2` walks
+; two of its bits down into the low word and the dbra count comes out long.
+expand:
+        lsr.l   #2,d2
+        subq.l  #1,d2                   ; k<=256, so the count fits a dbra
+        moveq   #0,d0
+.e1:    move.b  (a0)+,d0
+        move.w  d0,(a1)+
+        move.b  (a0)+,d0
+        move.w  d0,(a1)+
+        move.b  (a0)+,d0
+        move.w  d0,(a1)+
+        move.b  (a0)+,d0
+        move.w  d0,(a1)+
+        dbra    d2,.e1
+        rts
+
+; ---------------------------------------------------------------- pal_tables
+; The three scratch tables.  SCENE-INDEPENDENT, every one of them: they describe
+; how the CRTC renders a 5-bit channel plus the shared LSB, which is a property
+; of the machine.  A player builds them once at boot and never again, which is
+; why they are a separate entry point rather than the head of pal_pack -- see
+; FINDINGS 53.3 for what that is worth.
+pal_tables:
+        ; -- P6TAB[x] = ((x<<2)|(x>>4)) & $FF, and SQTAB[x] = P6TAB[x]^2
+        lea     P6TAB,a0
+        lea     SQTAB,a1
+        moveq   #0,d1
+.p1:    move.w  d1,d0
+        lsl.w   #2,d0
+        move.w  d1,d2
+        lsr.w   #4,d2
+        or.w    d2,d0
+        andi.w  #$FF,d0
+        move.b  d0,(a0)+
+        move.w  d0,d2
+        mulu    d2,d2
+        move.l  d2,(a1)+
+        addq.w  #1,d1
+        cmpi.w  #64,d1
+        bne.s   .p1
+
+        ; -- DTAB[v] = (render(v,0)-v)^2 - (render(v,1)-v)^2, signed
+        lea     P6TAB,a0
+        lea     DTAB,a1
+        moveq   #0,d1
+.p2:    move.w  d1,d2
+        lsr.w   #2,d2
+        andi.w  #$3E,d2                 ; x0 = (v>>3)<<1
+        moveq   #0,d3
+        move.b  0(a0,d2.w),d3
+        sub.w   d1,d3
+        muls    d3,d3
+        moveq   #0,d4
+        move.b  1(a0,d2.w),d4
+        sub.w   d1,d4
+        muls    d4,d4
+        sub.l   d4,d3
+        move.w  d3,(a1)+
+        addq.w  #1,d1
+        cmpi.w  #256,d1
+        bne.s   .p2
+        rts
+
+; ---------------------------------------------------------------- pal_pack
+; 24-bit RGB -> GGGGGRRRRRBBBBBI, the shared LSB chosen per entry by minimum
+; squared error, written to the palette registers.  Identical arithmetic to
+; dlxload.pack_palette, including its tie-breaks: I stays 0 when the two errors
+; are equal, and the darkest entry is the FIRST index at the minimum.
+; in: a5 = container base, and pal_tables already run.
+pal_pack:
+        movea.l 16(a5),a0
+        adda.l  a5,a0                   ; -> 256 x RGB888
+        lea     GPAL,a1
+        lea     DTAB,a2
+        lea     SQTAB,a4                ; P6TAB is not needed here: the rendered
+                                        ; value is only ever wanted SQUARED
+        move.l  #$7FFFFFFF,d6
+        clr.l   LDARK.l
+        moveq   #0,d7
+.p3:    moveq   #0,d1
+        move.b  (a0)+,d1                ; R
+        moveq   #0,d2
+        move.b  (a0)+,d2                ; G
+        moveq   #0,d3
+        move.b  (a0)+,d3                ; B
+        move.w  d1,d0
+        add.w   d0,d0
+        move.w  0(a2,d0.w),d4
+        move.w  d2,d0
+        add.w   d0,d0
+        add.w   0(a2,d0.w),d4
+        move.w  d3,d0
+        add.w   d0,d0
+        add.w   0(a2,d0.w),d4           ; sum of err0-err1 over the three
+        moveq   #0,d5
+        tst.w   d4
+        ble.s   .p4
+        moveq   #1,d5                   ; I=1 only when it is STRICTLY better
+.p4:    lsr.w   #3,d1                   ; fR
+        lsr.w   #3,d2                   ; fG
+        lsr.w   #3,d3                   ; fB
+        move.w  d2,d4
+        lsl.w   #5,d4
+        or.w    d1,d4
+        lsl.w   #6,d4                   ; (fG<<11)|(fR<<6)
+        move.w  d3,d0
+        add.w   d0,d0
+        or.w    d0,d4
+        or.w    d5,d4
+        move.w  d4,(a1)+                ; -> the palette register
+
+        add.w   d1,d1                   ; x = (f<<1)|I, per channel
+        or.w    d5,d1
+        add.w   d2,d2
+        or.w    d5,d2
+        add.w   d3,d3
+        or.w    d5,d3
+        lsl.w   #2,d1                   ; SQTAB holds longs
+        move.l  0(a4,d1.w),d0
+        lsl.w   #2,d2
+        add.l   0(a4,d2.w),d0
+        lsl.w   #2,d3
+        add.l   0(a4,d3.w),d0           ; squared distance from black
+        cmp.l   d6,d0
+        bge.s   .p5
+        move.l  d0,d6
+        move.l  d7,LDARK.l              ; first index at the minimum wins
+.p5:    addq.w  #1,d7
+        cmpi.w  #256,d7
+        bne     .p3
+        rts

src/player/loadgate.s

@@ -0,0 +1,39 @@
+; Front-end for the load-time transforms (ROADMAP P1+P2), for the rig.
+;
+; It is to load.i what decode.s is to frame.i: a timing and control wrapper that
+; does nothing the shipping player would not do, so that the bytes being
+; measured are the bytes that will ship.  The player's own boot path will call
+; do_load once with the mode bits set to 3; this repeats it LITER times so a
+; host clock with 1/56.69 s granularity (tools/bench/crtc_mode.lua) can time a
+; job that takes milliseconds,
+; and splits it by LMODE so the codebook expansion and the palette pack can be
+; priced apart.  A player calls do_load with mode 7 once at boot -- the three
+; scratch tables describe the machine, not the scene -- and with mode 3 at every
+; scene change after that.
+;
+; Repeating is honest here in a way it would not be for a frame: nothing in
+; do_load is temporally recursive.  Pass n writes exactly what pass n-1 wrote,
+; over the top of it, out of the same source bytes.
+
+        include "src/player/geom.i"
+
+        org     $10000
+start:
+        move.l  LMODE.l,d1
+        move.l  LITER.l,d3
+        move.l  #1,LFLAG.l              ; timer starts here
+loop:
+        movem.l d1/d3,-(sp)
+        movea.l LHDR.l,a0
+        bsr     do_load
+        movem.l (sp)+,d1/d3
+        tst.l   d0
+        bne.s   bad
+        subq.l  #1,d3
+        bne.s   loop
+        move.l  #$FF,LFLAG.l            ; timer stops here
+hold:   bra.s   hold
+bad:    move.l  #$EE,LFLAG.l
+        bra.s   hold
+
+        include "src/player/load.i"

src/player/ring.i

@@ -0,0 +1,506 @@
+; ---------------------------------------------------------------- ring.i
+; The RING PRODUCER, on the 68000.  ROADMAP P5.
+;
+; WHAT MOVED.  FINDINGS 49 and 51 measured a ring that a HOST filled:
+; tools/bench/stream.lua held the record index, decided where each record went,
+; wrote the descriptor and advertised it.  The 68000 only ever consumed.  That
+; is the same shape session 21 found in the loader and session 22 in the frame
+; clock -- a policy living outside the machine that has to run inside it -- and
+; it is the last one in the delivery path.  A player has no host to place its
+; records.
+;
+; So the placement policy is here now, and the host keeps only the part that is
+; genuinely not the CPU's: moving bytes off a disc at a rate.  What the rig
+; supplies is a TRANSPORT, one request at a time, which is what a single SPC and
+; one DMAC channel are (FINDINGS 52.5); what this file supplies is every
+; decision about WHICH record, WHERE in the ring, and WHEN it is safe to start.
+;
+; THE POLICY IS `aligned`, and it is the same one 19_ring_stream.py scored and
+; 49.3 chose: never start a record that will not finish before the end of the
+; ring; leave the hole, restart at the base.  It costs a mean hole of 5.7% of
+; the ring on the gate container and ZERO clocks in the block loop, against
+; `split`'s 3.64% of every frame budget forever.
+;
+; WHY IT NEEDS AN INDEX, and why that is a container change (DLX4).  `aligned`
+; asks "does the NEXT record fit before the end of the ring", which is a
+; question about a record's length asked BEFORE it is fetched.  Every reader in
+; this tree learned record lengths by walking the frame stream -- reading each
+; record's length word to find the next -- and that is exactly what a player
+; streaming off a disc cannot do: the length word of record i+1 is one of the
+; bytes it has not fetched yet.  DLX4 puts nframes u16 longword-counts in the
+; scene header for this, and the same table gives a branch point the disc
+; address of an arbitrary record without reading what lies between (`ring_seek`).
+;
+; THE HANDSHAKE WITH THE DECODER IS UNCHANGED, deliberately.  FR_HEAD/FR_TAIL/
+; DESC[] are the same words src/player/stream.s already reads, written in the
+; same order, so the decoder cannot tell a host-filled ring from a self-filled
+; one -- which is what makes the self-filled run a test of THIS file and not of
+; a new rig.  Two monotonic counters, single reader, single writer, no atomics.
+;
+; THE TRANSPORT MAILBOX, XF_QD outstanding requests:
+;   XF_SLOT[seq & 1]       what to fetch, where to put it, how much, and which
+;                          record it is
+;   XF_GO                  requests issued -- bumped LAST, after the slot
+;   XF_ACK                 requests completed, in order, by the transport
+; In the player XF_* is an MB89352 command and a DMAC channel; here it is
+; tools/bench/stream.lua delivering at a modelled rate.  Either way the CPU
+; issues and polls, and the bytes arrive on somebody else's time.
+;
+; AND THAT IS WHERE THE COST IS.  The channel only moves bytes while a request
+; is outstanding, and only the CPU can issue the next one.  Between the
+; completion of record i and the issue of record i+1 the disc is IDLE, and the
+; length of that gap is a property of the PLAYER's loop, not of the medium.
+; `ring_poll` is therefore called from the pace wait -- the idle the frame clock
+; already creates -- rather than once a frame: once a frame would cap the fill
+; at one record per slot, which is the wire rate exactly, and a ring that can
+; only just keep up can never accumulate the slack a branch point spends
+; (51.3).  A frame that uses its whole slot does not merely present late
+; (54.4); it stops the disc for a frame time.  The rig counts that gap.
+
+; ---- transport mailbox.  TWO REQUEST SLOTS, and the depth is a knob.
+; A channel only moves bytes while it has a request, and only the CPU can give
+; it one.  With ONE slot the disc stands still from the moment a transfer
+; completes until the player next polls -- and a player polls in its idle, which
+; is the end of a frame slot, so the gap is up to a whole frame's decode.  With
+; TWO the next request is already queued when the current one lands and the
+; channel need never stop.  XF_QD selects which, so the cost of the first is
+; measurable against the second in one rig rather than argued about.
+XF_SLOT = $18300                ; 2 x 16 B: u32 disc offset, u32 destination,
+                                ;           u32 length, u32 record index
+XF_SLSZ = 16
+XF_SLM  = 1                     ; slot = sequence & XF_SLM
+XF_GO   = $18320                ; u32 requests ISSUED, written by the 68000
+XF_ACK  = $18324                ; u32 requests COMPLETED, written by transport
+XF_QD   = $18328                ; u32 queue depth, 1 or 2 (input)
+
+; ---- producer state
+RINGOWN = $1832C                ; 1 = the 68000 owns placement (this file)
+RNG_B   = $18330                ; ring base address
+RNG_SZ  = $18334                ; ring size in bytes
+IDX_B   = $18338                ; base of the DLX4 record index, nframes u16
+RQ_NEXT = $1833C                ; next record to REQUEST
+WCUR    = $18340                ; write cursor, a ring OFFSET
+RCUR    = $18344                ; read cursor: ring offset of the oldest record
+                                ;   the decoder has not finished with
+RTAILN  = $18348                ; records RCUR has stepped over; chases FR_TAIL
+NRETIRE = $1834C                ; requests this file has published; chases XF_ACK
+DOFF    = $18350                ; running disc offset of record RQ_NEXT
+; ---- instruments.  None of these is read by the policy.
+N_HOLE  = $18354                ; wraps that left a hole
+N_HOLEB = $18358                ; total bytes in those holes
+N_FULL  = $1835C                ; polls that refused for SPACE (ring-bound)
+N_POLL  = $18360                ; ring_poll calls
+N_ISSUE = $18364                ; requests issued
+SLK_MIN = $18368                ; least slack seen at a frame start, in records
+SLK_AT  = $1836C                ; and the frame it was seen at
+PF_REC  = $18370                ; prefill target, in whole records (input)
+PF_DONE = $18374                ; records resident when the prefill released
+N_SEEK  = $18378                ; ring_seek calls
+SK_WAIT = $1837C                ; polls spent waiting for the channel to go
+                                ;   quiet before the last seek could start
+ROFF    = $19400                ; u32 per record: disc offset, built at load.
+                                ;   The seek half of the index -- a running sum
+                                ;   is enough to PLAY, but a branch point needs
+                                ;   record j's address without summing to it.
+ROFFMAX = 1024                  ; entries; $19400..$1A400, below CB1 at $20000
+
+RPOLLMAX = 4000000              ; ring_poll calls with no progress before the
+                                ; producer is declared wedged
+
+; ---------------------------------------------------------------- ring_init
+; in:  IDX_B, RNG_B, RNG_SZ, NFR set by the caller.
+; out: d0 = 0 ok, -1 the index is longer than ROFF can hold.  Builds the disc
+;      offset table and leaves the ring empty at record 0.
+; Clobbers d0-d2/a0-a1.
+ring_init:
+        move.l  NFR.l,d0
+        cmpi.l  #ROFFMAX,d0
+        bhi     .toobig
+        movea.l IDX_B.l,a0
+        lea     ROFF.l,a1
+        moveq   #0,d1                   ; running disc offset
+        move.l  d0,d2
+        beq.s   .noidx
+.sum:   move.l  d1,(a1)+
+        moveq   #0,d0
+        move.w  (a0)+,d0                ; longwords in this padded record
+        lsl.l   #2,d0
+        add.l   d0,d1
+        subq.l  #1,d2
+        bne.s   .sum
+.noidx:
+        clr.l   NRETIRE.l
+        clr.l   N_HOLE.l
+        clr.l   N_HOLEB.l
+        clr.l   N_FULL.l
+        clr.l   N_POLL.l
+        clr.l   N_ISSUE.l
+        clr.l   N_SEEK.l
+        clr.l   SK_WAIT.l
+        clr.l   XF_GO.l
+        clr.l   XF_ACK.l
+        move.l  #$7FFFFFFF,SLK_MIN.l
+        move.l  #-1,SLK_AT.l
+        moveq   #0,d0
+        bsr     ring_seek               ; a scene starts with a seek to record 0
+        moveq   #0,d0
+        rts
+.toobig:
+        moveq   #-1,d0
+        rts
+
+; ---------------------------------------------------------------- ring_seek
+; in:  d0 = record index to play from.
+; out: the ring is empty, the cursors are at its base, and the next request
+;      will be for record d0.  FR_HEAD is reset; the CALLER must reset FR_TAIL
+;      (it is the decoder's word, and this file never writes the decoder's).
+;
+; A SEEK CANNOT START WHILE THE CHANNEL IS BUSY.  An outstanding request is
+; bytes already on their way to an address this routine is about to declare
+; free, so it is waited out and thrown away rather than cancelled -- a real
+; SPC would need the transfer aborted and the bus handed back before a new
+; command, and waiting is the version of that a rig can be honest about.  What
+; it costs is up to one record's delivery time, charged to the seek, and
+; SK_WAIT counts the polls it took.
+; Clobbers d0-d2/a0.
+ring_seek:
+        movem.l d0-d2/a0,-(sp)
+        addq.l  #1,N_SEEK.l
+        clr.l   SK_WAIT.l
+.wait:  move.l  XF_ACK.l,d1
+        cmp.l   XF_GO.l,d1
+        beq.s   .quiet
+        addq.l  #1,SK_WAIT.l
+        bsr     xf_service              ; the transport hook, and here it is not
+                                        ; optional: with the transport INSIDE
+                                        ; the machine (src/player/xfer.i) the
+                                        ; only thing that can retire the
+                                        ; outstanding request is this loop, so
+                                        ; without it a seek issued with a
+                                        ; request in flight spins forever.  A
+                                        ; host transport retired it on its own
+                                        ; time, which is exactly the kind of
+                                        ; difference the seam exists to hide and
+                                        ; this one it could not.
+        bra.s   .wait
+.quiet:
+        move.l  XF_GO.l,NRETIRE.l       ; whatever landed belongs to the scene
+                                        ; we came FROM, and is discarded
+        move.l  d0,RQ_NEXT.l
+        lsl.l   #2,d0
+        lea     ROFF.l,a0
+        move.l  (a0,d0.l),DOFF.l        ; the index's second job: record j's
+                                        ; disc address without reading to it
+        clr.l   WCUR.l
+        clr.l   RCUR.l
+        clr.l   RTAILN.l
+        clr.l   FR_HEAD.l
+        movem.l (sp)+,d0-d2/a0
+        rts
+
+; ---------------------------------------------------------------- ring_poll
+; Advance the producer by at most one step: retire a completed request, catch
+; the read cursor up with the decoder, and issue the next request if one fits.
+; Preserves every register -- it is called from inside the decoder's wait loops
+; and must be invisible to them.
+ring_poll:
+        movem.l d0-d3/a0-a1,-(sp)
+        addq.l  #1,N_POLL.l
+; ---- 0. the TRANSPORT, if it lives in this machine.  src/player/xfer.i answers
+;      at most one outstanding request per call and preserves every register;
+;      with XF_SCSI = 0 it is a tst and a branch, and the host is the transport
+;      exactly as it was in FINDINGS 55.  It goes BEFORE the retire step so that
+;      a transfer completed here is published in the same poll.
+        bsr     xf_service
+
+; ---- 1. retire.  The descriptor is written BEFORE the count that advertises
+;      it, which is the same order tools/bench/stream.lua used and the reason
+;      src/player/stream.s reads them the other way round.
+        move.l  NRETIRE.l,d3
+.retire:
+        cmp.l   XF_ACK.l,d3
+        bcc.s   .retired                ; d3 >= XF_ACK: nothing new has landed
+        move.l  d3,d0
+        and.l   #XF_SLM,d0
+        lsl.l   #4,d0                   ; * XF_SLSZ
+        lea     XF_SLOT.l,a1
+        adda.l  d0,a1                   ; a1 = the completed request's slot
+        move.l  12(a1),d0               ; its record index
+        lsl.l   #2,d0
+        and.w   #DESCM,d0
+        lea     DESC.l,a0
+        move.l  4(a1),(a0,d0.w)         ; its destination -> the descriptor
+        addq.l  #1,FR_HEAD.l            ; ...advertised only after the address
+        addq.l  #1,d3
+        bra.s   .retire
+.retired:
+        move.l  d3,NRETIRE.l
+
+; ---- 2. catch the read cursor up.  The decoder publishes FR_TAIL and nothing
+;      else the producer needs: RCUR walks the SAME placement rule the writer
+;      used, so it steps over the holes in exactly the places they were left.
+;      That is what makes the free space a single circular gap rather than a
+;      list of live records -- the host producer kept a list because it could
+;      afford to.
+;      THE RULE MUST BE THE WRITER'S, APPLIED TO THE SAME RECORD.  Stepping
+;      the reader past record i lands on the END of record i, which is where
+;      record i+1 went only if i+1 FITTED there -- and if it did not, the writer
+;      put it at the ring base and left a hole.  So the wrap is decided by the
+;      length of the record being stepped ONTO, exactly as the placement was.
+;
+;      Deciding it with the wrong record's length was a real bug and not a
+;      conservative one: it left RCUR pointing into the hole, and one more
+;      retirement then pushed it past the end of the ring and wrapped it to a
+;      low address unrelated to any record.  The live span computed from that is
+;      SHORTER than the truth, so the producer places on top of a record the
+;      decoder has not finished, and the symptom is a bitstream desync -- the
+;      decoder's a0 walking off the end of a record that changed underneath it.
+        move.l  RTAILN.l,d1
+.catch: cmp.l   FR_TAIL.l,d1
+        bcc.s   .caught
+        movea.l IDX_B.l,a0
+        move.l  d1,d0
+        add.l   d0,d0
+        moveq   #0,d2
+        move.w  (a0,d0.l),d2
+        lsl.l   #2,d2                   ; length of the record being retired
+        move.l  RCUR.l,d0
+        add.l   d2,d0                   ; d0 = one past its end
+        addq.l  #1,d1
+        cmp.l   NFR.l,d1
+        bcc.s   .last                   ; nothing follows it in this scene
+        move.l  d1,d2
+        add.l   d2,d2
+        moveq   #0,d3
+        move.w  (a0,d2.l),d3
+        lsl.l   #2,d3                   ; length of the record after it
+        add.l   d0,d3
+        cmp.l   RNG_SZ.l,d3
+        bls.s   .last
+        moveq   #0,d0                   ; it did not fit: the writer restarted
+                                        ; at the base, so the reader does too
+.last:
+        move.l  d0,RCUR.l
+        bra.s   .catch
+.caught:
+        move.l  d1,RTAILN.l
+
+; ---- 3. issue, if there is anything left and it fits.
+        move.l  RQ_NEXT.l,d1
+        cmp.l   NFR.l,d1
+        bcc     .out                    ; whole scene requested
+; ---- MEASURED AGAINST WHAT HAS BEEN RETIRED, NOT WHAT HAS BEEN ACKED, and
+;      the difference is a slot.  A request's slot stays in use until this file
+;      has read the record index and destination out of it -- which happens in
+;      step 1 above, one poll later than the ack at the earliest.  Gating on
+;      XF_ACK let the CPU write a slot whose descriptor had not been published
+;      yet: the transport had finished the transfer, the retire loop then read
+;      the OVERWRITTEN slot, and DESC for that frame stayed zero.  The decoder
+;      duly decoded address zero and reported a bitstream desync.
+        move.l  XF_GO.l,d0
+        sub.l   NRETIRE.l,d0            ; slots still spoken for
+        cmp.l   XF_QD.l,d0
+        bcc     .out                    ; the queue is as deep as it may go
+        move.l  d1,d0
+        add.l   d0,d0
+        movea.l IDX_B.l,a0
+        moveq   #0,d2
+        move.w  (a0,d0.l),d2
+        lsl.l   #2,d2                   ; d2 = length to place
+; ---- WHERE IT GOES, AND WHETHER IT MAY.  The live bytes are the circular
+;      interval [RCUR, WCUR) -- oldest record the decoder has not finished with,
+;      up to the write cursor -- so the FREE bytes are its complement, and a
+;      record has to fit in ONE piece of it because the block loop reads with a
+;      monotonically increasing a0 (49.2).
+;
+;      There are three shapes and they are not symmetric, which is the trap:
+;        empty            the whole ring is free
+;        RCUR <= WCUR     live is one run; free is [WCUR, SZ) THEN [0, RCUR),
+;                         so a record that will not fit before the end may
+;                         restart at the base -- this is `aligned`, and the
+;                         skipped bytes are the hole
+;        RCUR >  WCUR     LIVE is the one that wraps; free is only [WCUR, RCUR)
+;                         and the ring base is NOT ours -- a record that will
+;                         not fit must simply wait
+;      Deciding the wrap from `WCUR + len > SZ` alone, before knowing which
+;      shape it is, was the second bug in this file: in the third shape it
+;      restarted at a base that was live and overwrote records the decoder had
+;      not read, and the symptom was a bitstream desync rather than a fault.
+        move.l  WCUR.l,d3               ; d3 = candidate offset
+        moveq   #0,d0                   ; d0 = hole bytes, if any
+        move.l  RQ_NEXT.l,d1
+        cmp.l   RTAILN.l,d1
+        beq.s   .isempty
+        move.l  RCUR.l,d1
+        cmp.l   d3,d1
+        beq     .full                   ; RCUR == WCUR and not empty: the ring
+                                        ; is completely full
+        bhi.s   .freehi
+; RCUR < WCUR: free is [WCUR, SZ) then [0, RCUR).
+        move.l  d3,d0
+        add.l   d2,d0
+        cmp.l   RNG_SZ.l,d0
+        bls.s   .nohole2                ; fits before the end of the ring
+        move.l  RCUR.l,d0
+        cmp.l   d2,d0
+        bcs     .full                   ; it will not fit at the base either
+        move.l  RNG_SZ.l,d0
+        sub.l   d3,d0                   ; the hole `aligned` is about to leave.
+                                        ; Charged here, where the record is
+                                        ; actually PLACED, and not where the
+                                        ; wrap is decided: charging it at the
+                                        ; decision counts one hole per retry
+                                        ; while the decoder still owns the base,
+                                        ; which is every poll of a fast pipe,
+                                        ; and reported 105 wraps where there
+                                        ; are 18.
+        moveq   #0,d3
+        bra.s   .place
+.nohole2:
+        moveq   #0,d0
+        bra.s   .place
+.freehi:
+; RCUR > WCUR: the LIVE span wraps, so the only free run is [WCUR, RCUR).
+        move.l  d3,d0
+        add.l   d2,d0
+        cmp.l   RCUR.l,d0
+        bhi     .full
+        moveq   #0,d0
+        bra.s   .place
+.isempty:
+; Nothing live, so the whole ring is free and the live span is about to start
+; here.  Moving RCUR is what keeps the invariant true across a drained ring;
+; without it the reader's cursor would still point at the last consumed record.
+        move.l  d3,d0
+        add.l   d2,d0
+        cmp.l   RNG_SZ.l,d0
+        bls.s   .enohole
+        move.l  RNG_SZ.l,d0
+        sub.l   d3,d0
+        moveq   #0,d3
+        bra.s   .esetr
+.enohole:
+        moveq   #0,d0
+.esetr:
+        move.l  d3,RCUR.l
+.place:
+        tst.l   d0
+        beq.s   .nohole
+        addq.l  #1,N_HOLE.l
+        add.l   d0,N_HOLEB.l
+.nohole:
+        move.l  XF_GO.l,d1
+        and.l   #XF_SLM,d1
+        lsl.l   #4,d1                   ; * XF_SLSZ
+        lea     XF_SLOT.l,a1
+        adda.l  d1,a1
+        move.l  DOFF.l,(a1)
+        move.l  RNG_B.l,d0
+        add.l   d3,d0
+        move.l  d0,4(a1)
+        move.l  d2,8(a1)
+        move.l  RQ_NEXT.l,12(a1)
+        add.l   d2,d3
+        move.l  d3,WCUR.l
+        add.l   d2,DOFF.l
+        addq.l  #1,RQ_NEXT.l
+        addq.l  #1,N_ISSUE.l
+        addq.l  #1,XF_GO.l              ; LAST: the three words above must be
+                                        ; visible before the request is
+        bra.s   .out
+.full:
+        addq.l  #1,N_FULL.l
+.out:
+        movem.l (sp)+,d0-d3/a0-a1
+        rts
+
+; ---------------------------------------------------------------- ring_prefill
+; Fill until PF_REC whole records are resident, then return.  THE POLICY, not a
+; convenience: the decoder must not be released at slack 1, because 51.2
+; measured that n resident records buy n-1 frame times of stall -- the last one
+; is spent covering the pipe's restart.  Releasing at 1 therefore starts a scene
+; with a stall budget of zero, and the first hiccup is an underrun.
+;
+; It is also the ONLY place a player can buy lookahead cheaply.  51.3: slack is
+; accumulated out of `pipe - wire` over seconds of play, so a scene that starts
+; empty climbs for 4.83 s at 488 KB/s before it can afford a branch.  Bytes
+; bought here are bought before the frame clock starts and cost nothing but the
+; wait -- which is the one moment in a scene when the decoder has nothing else
+; to do anyway.
+;
+; out: d0 = 0 ok, -1 the transport never delivered.  PF_DONE = records resident.
+ring_prefill:
+        movem.l d1-d2,-(sp)
+        moveq   #0,d1
+.loop:  bsr     ring_poll
+        move.l  FR_HEAD.l,d0
+        sub.l   FR_TAIL.l,d0
+        cmp.l   PF_REC.l,d0
+        bcc.s   .done
+        addq.l  #1,d1
+        cmp.l   #RPOLLMAX,d1
+        bcs.s   .loop
+        movem.l (sp)+,d1-d2
+        moveq   #-1,d0
+        rts
+.done:  move.l  d0,PF_DONE.l
+        movem.l (sp)+,d1-d2
+        moveq   #0,d0
+        rts
+
+; ---------------------------------------------------------------- ring_slack
+; out: d0 = whole records resident and unconsumed.
+;
+; This is the number FINDINGS 51 spent a session establishing the meaning of,
+; and it is only worth what it is worth when the decoder is PACED: free-running,
+; the decoder outruns any pipe and the ring never backs up, so the difference is
+; a statement about earliness (49.7.2).
+ring_slack:
+        move.l  FR_HEAD.l,d0
+        sub.l   FR_TAIL.l,d0
+        rts
+
+; ---------------------------------------------------------------- ring_may_seek
+; in:  d0 = the stall a branch would cost, in whole frame times.
+; out: d0 = 0 the ring can cover it, -1 it cannot.  Z set on ok.
+;
+; THE RULE, from 51.2, measured and not assumed: n resident records buy n-1
+; frame times, because the record due immediately after the pipe restarts is
+; still arriving when its slot opens.  A design that reads the resident count
+; as its stall budget is over by one record every time.
+;
+; What a player does with a `no` is not this file's business -- delay the
+; branch, take the outcome that needs no seek, or accept a late present -- but
+; it must be able to ASK, and until now the answer only existed in the rig's
+; log.
+ring_may_seek:
+        move.l  d1,-(sp)
+        move.l  FR_HEAD.l,d1
+        sub.l   FR_TAIL.l,d1
+        beq.s   .no                     ; nothing resident: the subtraction
+                                        ; below would wrap to $FFFFFFFF and an
+                                        ; unsigned compare would then answer YES
+                                        ; to any request, from an empty ring
+        subq.l  #1,d1                   ; the restart record is not spendable
+        cmp.l   d0,d1
+        bcs.s   .no
+        move.l  (sp)+,d1
+        moveq   #0,d0
+        rts
+.no:    move.l  (sp)+,d1
+        moveq   #-1,d0
+        rts
+
+; ---------------------------------------------------------------- ring_mark
+; Sample the slack at a frame boundary, for the instruments only.
+; in: d0 = frame number.  Clobbers nothing.
+ring_mark:
+        movem.l d0-d1,-(sp)
+        move.l  FR_HEAD.l,d1
+        sub.l   FR_TAIL.l,d1
+        cmp.l   SLK_MIN.l,d1
+        bcc.s   .out
+        move.l  d1,SLK_MIN.l
+        move.l  d0,SLK_AT.l
+.out:   movem.l (sp)+,d0-d1
+        rts

src/player/scsi.i

@@ -0,0 +1,695 @@
+; ---------------------------------------------------------------- scsi.i
+; The MB89352 TRANSPORT, on the 68000.  ROADMAP P4.
+;
+; WHAT THIS REPLACES.  src/player/ring.i decides which record to fetch, where in
+; the ring to put it and when it is safe; it hands that to a TRANSPORT through
+; the XF_* mailbox and polls for completion.  Until now the transport was
+; tools/bench/stream.lua -- a host moving bytes at a modelled rate.  A player has
+; no host.  This file is the transport: a Fujitsu MB89352 SPC and, later, one
+; HD63450 channel.
+;
+; THE REGISTER MAP IS MEASURED, NOT ASSUMED.  src/player/scsigate.s probes
+; $EA0000..$EA003F one address at a time and survives the bus errors, and the
+; map below is what answered:
+;
+;   registers sit on the ODD bytes, $EA0001 + 2n, n = 0..14
+;   n=3 (TMOD) and n=15 (EXBF) BUS ERROR -- the MB89352 omits both, where the
+;     MB87030 has them, and MAME leaves HOLES rather than shifting the
+;     later indices down.  DREG is index 10 at $EA0015 either way, which is the
+;     one address FINDINGS 32.4 had quoted.
+;   TEMP ($EA0017) took $A5 and gave it back, so these are registers and not a
+;     mirror of something.
+;
+; That last point is why the probe was worth a run: MAME's own device summary
+; says the MB89352 "shifts subsequent indices", and the machine says it does
+; not.  The bytes win.
+;
+; THE DATA REGISTER IS DMA-ONLY, AND THAT IS NOT A CHOICE THIS CODE MADE.
+; x68k_scsiext.cpp puts its own glue on $EA0015 and on no other address:
+;
+;   write:  if (exown()) { if (!drq) dtack_w(1); else dma_w(data); }
+;           else         dreg_w(data);
+;
+; On this machine `exown()` -- the HD63450's OWN, fed back to the slot by
+; x68k.cpp -- is asserted where a PIO write needs it not to be, so the `else`
+; arm is unreachable and a byte written to $EA0015 with the SPC in PROGRAM
+; transfer mode is DISCARDED.  Silently: no error bit, no status change, no
+; interrupt.  It was measured rather than reasoned about -- scsigate.s writes
+; $5A to $EA0015 and reads it straight back, and gets $00 with the FIFO still
+; empty -- because ten command bytes vanishing without trace looks exactly like
+; a target refusing a command.
+;
+; So every transfer here issues SCMD WITHOUT the PROGRAM bit, which puts the SPC
+; in DMA mode and makes it raise DRQ; the CPU then moves the bytes through
+; $EA0015 itself and they go in via `dma_w`/`dma_r`.  The CPU is standing in for
+; the DMAC, through the DMAC's own door.
+;
+; WHAT THAT COSTS THE ARGUMENT, stated because it is easy to overclaim here:
+; with `exown` asserted at idle, MAME cannot distinguish a CPU-driven byte at
+; $EA0015 from a DMAC-driven one.  So this rig demonstrates THE DATA PATH and
+; cannot, on its own, demonstrate that the HD63450 is the thing driving it.
+; Whether a real CZ-6BS1 also refuses PIO here is NOT settled by this -- it is a
+; property of MAME's model, and it wants a board (ROADMAP B1/B3).
+;
+; PIO FIRST, DMA SECOND, DELIBERATELY.  The thing P4 has to demonstrate is a
+; DMAC configuration that HOLDS THE BUS (ROADMAP: "getting the DMAC to hold the
+; bus is the difference between 9 and 19 clocks per byte, and demonstrating a
+; configuration that does it is P4's first job").  But a DMA bring-up that fails
+; cannot tell "the SCSI protocol is wrong" from "the DMAC is misprogrammed".  So
+; the protocol is settled in PIO, where every byte is the CPU's and nothing else
+; can be blamed, and only then does the data phase move to the channel.
+;
+; NOTHING HERE IS A RATE MEASUREMENT, and it cannot become one.  MAME's device
+; models are functional, not transfer-timing accurate (docs/BENCHMARK.md), and
+; 42.5 reads its DMAC configured in wall-clock attotimes rather than per-operand
+; cycles.  `W` -- clocks stolen per delivered byte -- is untouched by every line
+; below.  What this settles is which handshake the player's own code provokes.
+
+; ---- the SPC, at the CZ-6BS1's decode
+SPCB    = $EA0001                       ; register 0; stride 2, odd lane
+SC_BDID = SPCB+0                        ; own ID (write the NUMBER; reads a MASK)
+SC_SCTL = SPCB+2
+SC_SCMD = SPCB+4
+;               SPCB+6  = TMOD, ABSENT on the MB89352 -- reading it BUS ERRORS
+SC_INTS = SPCB+8
+SC_PSNS = SPCB+10
+SC_SSTS = SPCB+12
+SC_SERR = SPCB+14
+SC_PCTL = SPCB+16
+SC_MBC  = SPCB+18
+SC_DREG = SPCB+20                       ; $EA0015, and the DMAC's single address
+SC_TEMP = SPCB+22
+SC_TCH  = SPCB+24
+SC_TCM  = SPCB+26
+SC_TCL  = SPCB+28
+;               SPCB+30 = EXBF, ABSENT -- reading it BUS ERRORS
+
+; SCTL
+SCTL_RESET   = $80                      ; reset & disable
+; SCMD, command in bits 7-5
+SCMD_RELEASE = $00                      ; command 000, let go of the bus
+SCMD_RSTACK  = $C0                      ; command 110, drop ACK/REQ
+SCMD_SELECT  = $20
+SCMD_RSTATN  = $40                      ; command 010, drop ATN
+SCMD_XFER    = $80
+SCMD_PROGRAM = $04                      ; set = PIO, clear = DMA.  NOT USED, and
+                                        ; the reason is the whole of 57.x -- see
+                                        ; "THE DATA REGISTER IS DMA-ONLY" above.
+; INTS
+INTS_RESET   = $01
+INTS_HARDERR = $02
+INTS_TIMEOUT = $04
+INTS_SERVICE = $08
+INTS_CMDCOMP = $10
+INTS_DISCON  = $20
+; SSTS
+SSTS_DREG_E  = $01                      ; DREG empty
+SSTS_DREG_F  = $02                      ; DREG full
+SSTS_TC0     = $04
+SSTS_BUSY    = $20
+SSTS_INITCON = $80
+; SCSI bus phases, as PSNS bits 2..0 and as PCTL's low three
+PH_DATAOUT   = 0
+PH_DATAIN    = 1
+PH_CMD       = 2
+PH_STATUS    = 3
+PH_MSGOUT    = 6
+PH_MSGIN     = 7
+
+SCSI_ID  = 7                            ; the player is the initiator
+SCSI_TGT = 0                            ; the disc
+
+; ---- error codes, reported through SC_ERR
+SCE_OK      = 0
+SCE_SELTMO  = 1                         ; the target never answered selection
+SCE_PHASE   = 2                         ; the bus went somewhere unexpected
+SCE_TIMEOUT = 3                         ; a poll loop ran out of patience
+SCE_STATUS  = 4                         ; the target returned non-zero status
+SCE_WINDOW  = 5                         ; a WINDOWED read was asked of the DMAC,
+                                        ; which cannot drop bytes (58.3/P4a)
+
+SC_ERR   = $18200                       ; u32 last error
+SC_STAT  = $18204                       ; u32 SCSI status byte from the last cmd
+SC_PH    = $18208                       ; u32 phase we were in when it went wrong
+SC_CDB   = $18210                       ; 12 B command block, built here
+SC_MSG   = $1821C                       ; 4 B message byte, either direction
+
+; ---- THE RECORD WINDOW, and why a transport needs one.  ROADMAP P4b.
+; src/player/ring.i asks for a RECORD: a byte offset into the scene's frame
+; stream and a length, both of them 4-byte aligned and neither of them a
+; multiple of 512.  A SCSI target deals in BLOCKS.  On the gate container 117
+; of 120 records start part way into a sector, so a transport that reads only
+; whole sectors delivers the record plus up to 511 bytes in front of it and up
+; to 511 behind, and those neighbouring bytes belong to records the decoder may
+; still be reading -- the block loop walks a0 with no bounds check (49.2), so
+; landing them in the ring is a corruption, not a waste.
+;
+; IN PIO THE FIX IS FREE, and that is the only reason this is affordable here:
+; the CPU is already touching every byte, so it simply does not STORE the ones
+; outside the window.  SC_WSKIP bytes are pulled from the FIFO and dropped, the
+; next SC_WKEEP are stored, the rest are pulled and dropped.  Three loops rather
+; than one steered loop, deliberately: the middle one is then byte-for-byte as
+; tight as the un-windowed sc_in_pio, so the per-byte cost this rig reports is
+; the transport's and not the window's.
+;
+; UNDER A DMAC IT IS NOT FREE, and that is P4a's problem arriving early.  A
+; channel writes a contiguous run to a contiguous address; it cannot be told to
+; drop the first 300 bytes.  So when the data phase moves to the HD63450 the
+; choice is a bounce buffer plus a copy of every byte (the cost `aligned` was
+; chosen to avoid, 49.3) or sector-aligned records in the container -- which is
+; a re-encode.  57 measured which is cheaper; see FINDINGS 58.3.
+SC_WSKIP = $18220                       ; u32 bytes to drop before the window
+SC_WKEEP = $18224                       ; u32 bytes of window to store
+
+; ---- a TRACE, because a SCSI bring-up cannot be debugged from one error code.
+; Four registers at each interesting instant: SSTS, PSNS, INTS, SERR.  MAME's
+; SCMD_CMD_TRANSFER is a NO-OP unless SSTS_INIT_CONNECTED is set -- it `break`s
+; out of the switch without complaint -- so "the transfer did nothing" and "the
+; transfer went wrong" look identical from the outside.  The trace separates
+; them.
+SC_TAG   = $1822C                       ; u32 where the next snapshot came from
+SC_TRN   = $18230                       ; u32 trace entries used
+SC_TR    = $18240                       ; 24 x 8 B: SSTS PSNS INTS SERR TCH TCM TCL TAG
+
+; A poll bound.  Every wait in this file is bounded, because a SCSI phase that
+; never arrives must be REPORTED -- an unbounded spin is indistinguishable from
+; a wedged emulator, and 34.1 already cost this project fifteen minutes to that
+; exact confusion.
+SC_PATIENCE = 200000
+
+; ---------------------------------------------------------------- sc_snap
+; Append SSTS/PSNS/INTS/SERR to the trace.  Clobbers nothing the callers use.
+sc_snap:
+        movem.l d0/a0,-(sp)
+        move.l  SC_TRN.l,d0
+        cmp.l   #24,d0
+        bge.s   sn_out
+        lea     SC_TR.l,a0
+        lsl.l   #3,d0
+        adda.l  d0,a0
+        move.b  SC_SSTS,(a0)+
+        move.b  SC_PSNS,(a0)+
+        move.b  SC_INTS,(a0)+
+        move.b  SC_SERR,(a0)+
+        move.b  SC_TCH,(a0)+
+        move.b  SC_TCM,(a0)+
+        move.b  SC_TCL,(a0)+
+        move.b  SC_TAG+3,(a0)+          ; WHERE this snapshot was taken
+        addq.l  #1,SC_TRN.l
+sn_out: movem.l (sp)+,d0/a0
+        rts
+
+; ---- the HD63450, so that PIO through the card's data register works at all.
+; x68k_scsiext.cpp puts DMA-AWARE GLUE on $EA0015 and nowhere else:
+;
+;   write:  if (exown())  { if (!drq) dtack_w(1); else dma_w(data); }
+;           else          dreg_w(data);
+;
+; With OWN asserted and DRQ low the byte is DROPPED, silently.  That is the
+; bring-up's fourth bug and the least guessable: ten command bytes went into
+; $EA0015, the FIFO stayed empty, the transfer counter stayed at 10, and every
+; register the SPC has said "waiting".  Nothing reports a discarded write.
+; OWN is the DMAC's, and the IPL ROM has been running for three seconds before
+; the player's first instruction (52.5 reads its channel setup out of the ROM),
+; so the player does not inherit a quiet DMAC -- it has to make one.
+DMAC    = $E84000
+DMAC_CH = $40                           ; channels are 64 B apart
+dmac_quiet:
+        lea     DMAC,a0
+        moveq   #3,d1
+dq1:    move.b  #0,7(a0)                ; CCR: no operation
+        move.b  #$FF,0(a0)              ; CSR: write-one-to-clear
+        adda.w  #DMAC_CH,a0
+        dbra    d1,dq1
+        rts
+
+; ---------------------------------------------------------------- scsi_init
+; Reset the SPC and claim an initiator ID.  Leaves interrupts DISABLED: the
+; player polls, because the ring producer is already a polling loop living in
+; the pace wait (ring.i) and an interrupt would buy it nothing it does not
+; already have.
+scsi_init:
+        bsr     dmac_quiet
+        move.b  #SCTL_RESET,SC_SCTL     ; reset & disable
+        moveq   #40,d0
+sci1:   nop
+        dbra    d0,sci1
+        move.b  #SCSI_ID,SC_BDID
+        move.b  #0,SC_SCTL              ; out of reset; no arbitration, no ints
+        move.b  #$FF,SC_INTS            ; INTS is cleared by writing its bits
+        move.b  #0,SC_PCTL
+        clr.l   SC_ERR.l
+        clr.l   DM_USE.l                ; PIO unless a caller asks otherwise,
+                                        ; AFTER this call (src/player/dma.i)
+        clr.l   DM_HOOK.l               ; ...and no second consumer is being
+                                        ; serviced until one installs itself,
+                                        ; for the same reason and in the same
+                                        ; direction: a mailbox that defaults to
+                                        ; a behaviour is how DM_USE landed on
+                                        ; ring.i's slot.
+        clr.l   SC_TRN.l
+        move.l  #0,SC_TAG.l
+        bsr     sc_snap
+        rts
+
+; ---------------------------------------------------------------- sc_settc
+; d0 = 24-bit transfer count -> TCH/TCM/TCL
+; Written LOW BYTE FIRST with lsr, not as a chain of rol.l #8.  The rol version
+; was the bring-up's second bug: three rotations put the ORIGINAL bits 31..24,
+; 23..16 and 15..8 into TCH/TCM/TCL, so a count of 10 loaded a transfer counter
+; of ZERO.  MAME then completed the TRANSFER instantly and silently -- SSTS came
+; back $85, TC0 set and XFER_IN_PROGRESS clear -- and the bus sat in command
+; phase, which surfaced as the same `UNEXPECTED PHASE` as a protocol error.
+sc_settc:
+        move.l  d0,-(sp)
+        move.b  d0,SC_TCL               ; bits 7..0
+        lsr.l   #8,d0
+        move.b  d0,SC_TCM               ; bits 15..8
+        lsr.l   #8,d0
+        move.b  d0,SC_TCH               ; bits 23..16
+        move.l  (sp)+,d0
+        rts
+
+; ---------------------------------------------------------------- sc_waitreq
+; Wait until the SPC reports a REQ with a phase, or patience runs out.
+; Returns the phase in d0; sets SC_ERR and returns -1 on timeout.
+sc_waitreq:
+        move.l  #SC_PATIENCE,d1
+swr1:   move.b  SC_PSNS,d0
+        btst    #7,d0                   ; REQ
+        bne.s   swr2
+        subq.l  #1,d1
+        bne.s   swr1
+        move.l  #SCE_TIMEOUT,SC_ERR.l
+        moveq   #-1,d0
+        rts
+swr2:   and.l   #7,d0
+        rts
+
+; ---------------------------------------------------------------- sc_waitfree
+; Wait for BUS FREE.  A command is not over when its last message byte has been
+; read: the target still has BSY asserted, and an initiator that starts
+; arbitrating into that gets a selection timeout.
+;
+; This is the bring-up's fifth bug, and it only appeared once there were TWO
+; reads.  One read passed, byte-exact, and every conclusion drawn from it was
+; sound; the SECOND could not select, because nothing had waited for the first
+; to let go of the bus.  A player issues one of these per record, so the failure
+; would have been universal in the ring and invisible in the demonstration.
+sc_waitfree:
+        move.l  #SC_PATIENCE,d1
+swf1:   move.b  SC_PSNS,d0
+        btst    #3,d0                   ; BSY
+        beq.s   swf2
+        subq.l  #1,d1
+        bne.s   swf1
+        move.l  #SCE_TIMEOUT,SC_ERR.l
+        moveq   #-1,d0
+        rts
+swf2:   moveq   #0,d0
+        rts
+
+; ---------------------------------------------------------------- sc_select
+; Select SCSI_TGT.  The selection bitmask goes in TEMP -- both IDs, ours and
+; theirs -- and the transfer counter doubles as the selection timeout (MAME:
+; SelectionWaitBSY is derived from TC's upper bits, which is the datasheet's
+; behaviour too).
+sc_select:
+        move.b  #$FF,SC_INTS
+        move.b  #(1<<SCSI_ID)|(1<<SCSI_TGT),SC_TEMP
+        move.l  #$002000,d0
+        bsr     sc_settc
+        move.b  #0,SC_PCTL
+        move.b  #SCMD_SELECT,SC_SCMD
+        move.l  #SC_PATIENCE,d1
+ssel1:  move.b  SC_INTS,d0
+        btst    #4,d0                   ; COMMAND COMPLETE = selection won
+        bne.s   ssel_ok
+        btst    #2,d0                   ; TIMEOUT = nobody there
+        bne.s   ssel_tmo
+        subq.l  #1,d1
+        bne.s   ssel1
+        move.l  #SCE_TIMEOUT,SC_ERR.l
+        moveq   #-1,d0
+        rts
+ssel_tmo:
+        move.b  #$FF,SC_INTS
+        move.l  #SCE_SELTMO,SC_ERR.l
+        moveq   #-1,d0
+        rts
+ssel_ok:
+        move.l  #1,SC_TAG.l
+        bsr     sc_snap
+        move.b  #$FF,SC_INTS
+        move.l  #2,SC_TAG.l
+        bsr     sc_snap
+        moveq   #0,d0
+        rts
+
+; ---------------------------------------------------------------- sc_xferend
+; Wait for the SPC to finish the TRANSFER it was given, rather than for the last
+; byte to have been HANDED to it.
+;
+; This is the bring-up's one real bug and it is worth recording.  Without it,
+; sc_out_pio wrote all ten command bytes and returned, the caller immediately
+; asked what phase the bus was in, and the answer was STILL COMMAND -- because
+; the SPC had the last byte in its FIFO and had not yet run the REQ/ACK for it.
+; The symptom was `UNEXPECTED PHASE, phase=2` at the DATA-IN check, which reads
+; like a target refusing the command and is nothing of the kind.  A byte handed
+; to a FIFO is not a byte on the bus.
+sc_xferend:
+        move.l  #SC_PATIENCE,d3
+sxe1:   move.b  SC_SSTS,d0
+        btst    #4,d0                   ; XFER IN PROGRESS
+        beq.s   sxe2
+        subq.l  #1,d3
+        bne.s   sxe1
+        move.l  #SCE_TIMEOUT,SC_ERR.l
+        moveq   #-1,d0
+        rts
+sxe2:   moveq   #0,d0
+        rts
+
+; ---------------------------------------------------------------- sc_out_pio
+; Send d1 bytes from (a0) in phase d2.  Command blocks and nothing else, so it
+; is the small, simple one.
+sc_out_pio:
+        move.b  d2,SC_PCTL
+        move.l  d1,d0
+        bsr     sc_settc
+        move.b  #SCMD_XFER,SC_SCMD
+        move.l  #4,SC_TAG.l             ; 4 = TRANSFER issued for an OUT phase
+        bsr     sc_snap
+sop1:   move.l  #SC_PATIENCE,d3
+sop2:   move.b  SC_SSTS,d0
+        btst    #1,d0                   ; DREG FULL -- wait for room
+        beq.s   sop3
+        subq.l  #1,d3
+        bne.s   sop2
+        move.l  #SCE_TIMEOUT,SC_ERR.l
+        moveq   #-1,d0
+        rts
+sop3:   move.b  (a0)+,SC_DREG
+        subq.l  #1,d1
+        bne.s   sop1
+        move.l  #5,SC_TAG.l             ; 5 = every byte handed to the FIFO
+        bsr     sc_snap
+        bsr     sc_xferend
+        move.l  d0,-(sp)
+        move.l  #6,SC_TAG.l             ; 6 = after waiting for the transfer
+        bsr     sc_snap
+        move.l  (sp)+,d0
+        rts
+
+; ---------------------------------------------------------------- sc_in_pio
+; Receive d1 bytes into (a1) in phase d2.  This is the path the DMA version
+; replaces; it stays because it is what makes a DMA failure diagnosable.
+sc_in_pio:
+        move.b  d2,SC_PCTL
+        move.l  d1,d0
+        bsr     sc_settc
+        move.b  #SCMD_XFER,SC_SCMD
+        move.l  #7,SC_TAG.l             ; 7 = TRANSFER issued for an IN phase
+        bsr     sc_snap
+sip1:   move.l  #SC_PATIENCE,d3
+sip2:   move.b  SC_SSTS,d0
+        btst    #0,d0                   ; DREG EMPTY -- wait for a byte
+        beq.s   sip3
+        subq.l  #1,d3
+        bne.s   sip2
+        move.l  #SCE_TIMEOUT,SC_ERR.l
+        moveq   #-1,d0
+        rts
+sip3:   move.b  SC_DREG,(a1)+
+        subq.l  #1,d1
+        bne.s   sip1
+        move.l  #8,SC_TAG.l             ; 8 = every byte taken from the FIFO
+        bsr     sc_snap
+        bsr     sc_xferend
+        move.l  d0,-(sp)
+        move.l  #9,SC_TAG.l             ; 9 = after waiting for the IN transfer
+        bsr     sc_snap
+        move.l  (sp)+,d0
+        rts
+
+; ---------------------------------------------------------------- sc_in_data
+; Receive d1 bytes in phase d2, storing only the WINDOW: drop SC_WSKIP, store
+; SC_WKEEP at (a1), drop whatever is left.  This is the DATA IN path; STATUS and
+; MESSAGE IN keep sc_in_pio, which is one byte and has no window.
+;
+; THREE LOOPS, NOT ONE STEERED LOOP.  A single loop with a `which third am I in`
+; test per byte would cost ~20 clocks on every byte of every record, and the
+; number this rig exists to produce is the transport's per-byte cost -- so the
+; middle loop is byte-for-byte sc_in_pio's and the window is paid for once at
+; each boundary instead of once per byte.
+;
+; A SPLIT DATA PHASE WOULD RE-SKIP.  The counters are re-read from memory on
+; every entry, so a target that broke one READ(10) across two DATA IN phases
+; would drop the head of the second phase as well.  This one does not split --
+; the same limitation sc_in_pio's caller already carries -- and the fix is the
+; same one: d5 has to become what each phase actually delivered.
+sc_in_data:
+        ; ---- P4a: the DATA IN phase can be handed to the HD63450 instead, and
+        ; when it is, the CPU touches none of these bytes.  src/player/dma.i.
+        ; The window is REFUSED rather than ignored: a channel writes a
+        ; contiguous run and cannot be told to drop the first 300 bytes, so a
+        ; windowed DMA read would deliver the neighbours' bytes into the ring
+        ; and the block loop has no bounds check to catch it (49.2, 58.3).
+        ; Refusing it here is what makes "sector-aligned records" a PRECONDITION
+        ; the transport states rather than an assumption it carries.
+        tst.l   DM_USE.l
+        beq.s   .pio
+        tst.l   SC_WSKIP.l
+        bne.s   .nowin
+        move.l  SC_WKEEP.l,d0
+        cmp.l   d1,d0
+        bne.s   .nowin
+        bra     sc_in_dma
+.nowin: move.l  #SCE_WINDOW,SC_ERR.l
+        moveq   #-1,d0
+        rts
+.pio:
+        movem.l d6-d7,-(sp)
+        move.b  d2,SC_PCTL
+        move.l  d1,d0
+        bsr     sc_settc
+        move.b  #SCMD_XFER,SC_SCMD
+        move.l  #7,SC_TAG.l             ; 7 = TRANSFER issued for an IN phase
+        bsr     sc_snap
+        move.l  SC_WSKIP.l,d6
+        move.l  SC_WKEEP.l,d7
+        sub.l   d6,d1
+        sub.l   d7,d1                   ; d1 = trailing bytes to drop
+        tst.l   d6
+        beq.s   .keep
+.drop1: move.l  #SC_PATIENCE,d3
+.dw1:   move.b  SC_SSTS,d0
+        btst    #0,d0                   ; DREG EMPTY -- wait for a byte
+        beq.s   .dg1
+        subq.l  #1,d3
+        bne.s   .dw1
+        bra     .tmo
+.dg1:   tst.b   SC_DREG                 ; popped and thrown away
+        subq.l  #1,d6
+        bne.s   .drop1
+.keep:  tst.l   d7
+        beq.s   .tail
+.keep1: move.l  #SC_PATIENCE,d3
+.kw1:   move.b  SC_SSTS,d0
+        btst    #0,d0
+        beq.s   .kg1
+        subq.l  #1,d3
+        bne.s   .kw1
+        bra     .tmo
+.kg1:   move.b  SC_DREG,(a1)+
+        subq.l  #1,d7
+        bne.s   .keep1
+.tail:  tst.l   d1
+        beq.s   .fin
+.tail1: move.l  #SC_PATIENCE,d3
+.tw1:   move.b  SC_SSTS,d0
+        btst    #0,d0
+        beq.s   .tg1
+        subq.l  #1,d3
+        bne.s   .tw1
+        bra.s   .tmo
+.tg1:   tst.b   SC_DREG
+        subq.l  #1,d1
+        bne.s   .tail1
+.fin:   movem.l (sp)+,d6-d7
+        move.l  #8,SC_TAG.l             ; 8 = every byte taken from the FIFO
+        bsr     sc_snap
+        bsr     sc_xferend
+        move.l  d0,-(sp)
+        move.l  #9,SC_TAG.l
+        bsr     sc_snap
+        move.l  (sp)+,d0
+        rts
+.tmo:   movem.l (sp)+,d6-d7
+        move.l  #SCE_TIMEOUT,SC_ERR.l
+        moveq   #-1,d0
+        rts
+
+; ---------------------------------------------------------------- scsi_read
+; READ(10) of d4 blocks from LBA d3 into (a1).  READ(10) rather than READ(6)
+; because a 21-bit LBA and a 256-block ceiling are limits this container will
+; reach -- 4,488,588 B of frame records is already 8,767 sectors, and a full
+; disc is 1.09 GiB (ROADMAP C3).
+;
+; DRIVEN BY THE PHASE, NOT BY A SCRIPT, and that is the third thing the bring-up
+; taught.  The first version ran a fixed sequence -- select, command, data,
+; status, message -- and broke the moment the target asked for something else:
+; it came up in MESSAGE OUT with ATN asserted and the driver, which "knew" the
+; next phase was COMMAND, called it an unexpected phase and gave up.  The bus
+; decides the order.  A driver that reads the phase and services whatever it
+; finds is both shorter and correct, and it is what the target is entitled to.
+scsi_read:
+        movem.l d3-d5/a1,-(sp)
+        move.l  d4,d5
+        lsl.l   #8,d5
+        lsl.l   #1,d5                   ; blocks * 512
+        clr.l   SC_WSKIP.l              ; no window: keep the whole transfer
+        move.l  d5,SC_WKEEP.l
+        bra.s   scr_body
+; ---- the same read, delivering only SC_WSKIP..SC_WSKIP+SC_WKEEP of it.  The
+; caller sets the two words; everything else is identical, which is the point --
+; a windowed read and a whole one must not be able to differ in the protocol.
+scsi_read_win:
+        movem.l d3-d5/a1,-(sp)
+        move.l  d4,d5
+        lsl.l   #8,d5
+        lsl.l   #1,d5
+scr_body:
+        ; ---- the command block, built before anything is on the bus
+        lea     SC_CDB.l,a0
+        move.b  #$28,(a0)+              ; READ(10)
+        clr.b   (a0)+
+        move.l  d3,d0                   ; LBA, big-endian u32
+        rol.l   #8,d0
+        move.b  d0,(a0)+                ; 31..24
+        rol.l   #8,d0
+        move.b  d0,(a0)+                ; 23..16
+        rol.l   #8,d0
+        move.b  d0,(a0)+                ; 15..8
+        rol.l   #8,d0
+        move.b  d0,(a0)+                ; 7..0
+        clr.b   (a0)+
+        move.l  d4,d0                   ; block count, big-endian u16.  Same
+        lsr.l   #8,d0                   ; trap as sc_settc had: a rol chain here
+        move.b  d0,(a0)+                ; would have emitted bits 31..24/23..16
+        move.b  d4,(a0)+                ; of a count that lives in 15..0.
+        clr.b   (a0)+
+        clr.l   SC_STAT.l
+        bsr     sc_select
+        tst.l   d0
+        bmi     scr_out
+        ; Drop ATN.  We have no message to send, so asking the target not to ask
+        ; for one is cheaper than answering.  The MSGOUT arm below still exists,
+        ; because "cheaper" is not "guaranteed".
+        move.b  #SCMD_RSTATN,SC_SCMD
+; ---- service whatever the bus asks for, until the target ends the command
+scr_ph:
+        bsr     sc_waitreq
+        tst.l   d0
+        bmi     scr_out
+        move.l  d0,-(sp)
+        move.l  #3,SC_TAG.l             ; 3 = the phase loop saw a REQ
+        bsr     sc_snap
+        move.l  (sp)+,d0
+        cmp.l   #PH_CMD,d0
+        beq.s   scr_cmd
+        cmp.l   #PH_DATAIN,d0
+        beq.s   scr_din
+        cmp.l   #PH_STATUS,d0
+        beq.s   scr_st
+        cmp.l   #PH_MSGIN,d0
+        beq.s   scr_min
+        cmp.l   #PH_MSGOUT,d0
+        beq.s   scr_mout
+        bra     scr_phase
+scr_cmd:
+        lea     SC_CDB.l,a0
+        moveq   #10,d1
+        moveq   #PH_CMD,d2
+        bsr     sc_out_pio
+        tst.l   d0
+        bmi     scr_out
+        bra     scr_ph
+scr_din:
+        ; KNOWN LIMITATION, harmless here and not harmless forever: this asks
+        ; for the WHOLE remaining count every time the bus enters DATA IN.  A
+        ; target that split one READ(10) across two data phases would be served
+        ; the full length twice and overrun the caller's buffer.  This one does
+        ; not split -- 4,096 B and 2,048 B both arrive in a single phase -- but
+        ; a real drive may, and P4b's mailbox integration is where d5 has to
+        ; start being decremented by what each phase actually delivered.
+        move.l  12(sp),a1               ; the caller's destination.  movem.l
+                                        ; d3-d5/a1,-(sp) lays them out ASCENDING
+                                        ; from sp as d3,d4,d5,a1 -- a1 is at 12.
+        move.l  d5,d1
+        moveq   #PH_DATAIN,d2
+        bsr     sc_in_data
+        tst.l   d0
+        bmi     scr_out
+        bra     scr_ph
+scr_st:
+        lea     SC_STAT.l,a1
+        addq.l  #3,a1                   ; the byte lands in the u32's low end
+        moveq   #1,d1
+        moveq   #PH_STATUS,d2
+        bsr     sc_in_pio
+        tst.l   d0
+        bmi     scr_out
+        bra     scr_ph
+scr_min:
+        lea     SC_MSG.l,a1
+        moveq   #1,d1
+        moveq   #PH_MSGIN,d2
+        bsr     sc_in_pio
+        tst.l   d0
+        bmi     scr_out
+        ; A message in ends the command.  Anything non-zero in the status byte
+        ; is the target refusing, and a transport that ignored it would hand the
+        ; ring a buffer of stale bytes and call it a record.
+        move.l  SC_STAT.l,d0
+        beq.s   scr_ok
+        move.l  #SCE_STATUS,SC_ERR.l
+        moveq   #-1,d0
+        bra.s   scr_out
+scr_mout:
+        ; Nothing to say: IDENTIFY, no disconnect, LUN 0.
+        lea     SC_MSG.l,a1
+        move.b  #$80,(a1)
+        move.l  a1,a0
+        moveq   #1,d1
+        moveq   #PH_MSGOUT,d2
+        bsr     sc_out_pio
+        tst.l   d0
+        bmi     scr_out
+        bra     scr_ph
+scr_ok: ; END OF COMMAND, and it takes two steps rather than one.  After the
+        ; final message byte the SPC is still holding ACK -- PSNS reads $4F,
+        ; REQ low and ACK high -- and a target cannot drop BSY into that.  So
+        ; ACK is dropped explicitly, and only then is the bus released.
+        move.b  #SCMD_RSTACK,SC_SCMD
+        move.b  #SCMD_RELEASE,SC_SCMD
+        move.l  #10,SC_TAG.l            ; 10 = after the bus release command
+        bsr     sc_snap
+        bsr     sc_waitfree             ; leave the bus as we found it
+        tst.l   d0
+        bmi     scr_out
+        moveq   #0,d0
+scr_out:
+        movem.l (sp)+,d3-d5/a1
+        rts
+scr_phase:
+        move.l  d0,SC_PH.l
+        move.l  #SCE_PHASE,SC_ERR.l
+        moveq   #-1,d0
+        bra.s   scr_out
+
+; ---- the DMAC side of the data phase, ROADMAP P4a.  Included unconditionally
+; so that there is ONE transport file: sc_in_data dispatches on DM_USE, which
+; scsi_init clears, so a front-end that never sets it assembles the same PIO
+; path FINDINGS 58 measured and executes not one instruction of the following.
+        include "src/player/dma.i"

src/player/scsigate.s

@@ -0,0 +1,150 @@
+; Front-end for the MB89352 PROBE (ROADMAP P4, first step), for the rig.
+;
+; WHY A PROBE AND NOT A DRIVER.  P4 replaces tools/bench/stream.lua's modelled
+; transport with a real SPC behind the XF_* mailbox src/player/ring.i already
+; talks to.  Before any of that can be written, the register map has to be a
+; FACT on the emulated machine rather than a reading of somebody's datasheet.
+; FINDINGS 32.4 quotes MAME mapping the data register at $EA0015 -- register
+; index 10 at a stride of 2 from $EA0001, i.e. registers on the ODD bytes.  That
+; is an inference from ONE address, and every access the driver makes rests on
+; it.
+;
+; WHY IT ENUMERATES INSTEAD OF DUMPING.  The first version of this walked
+; $EA0000 upwards with a plain `move.b (a0)+`, and took a bus error at $EA0006 --
+; at which point it knew one address was dead and nothing about the other 57.
+; A sequential dump stops at the first hole and reports the hole as the answer.
+; So this probes ONE ADDRESS AT A TIME with the index in MEMORY, and a bus error
+; handler that records the fault, steps the index and re-enters the loop.  A
+; dead address costs an entry in the map rather than the rest of the run.
+;
+; The 68000 cannot resume a faulted instruction -- RTE re-runs it and faults
+; again -- so the handler does not try.  It restores a stack pointer saved
+; before the loop and jumps back to the loop head, which reloads everything it
+; needs from memory.  Nothing lives in a register across a fault.
+;
+; This gate drives no SCSI bus and moves no data.  It is the smallest thing that
+; can turn "MAME instantiates an MB89352" into "the 68000 can reach it, HERE".
+
+SCFLAG  = $18080                ; 0 idle / 1 done
+SCN     = 64                    ; addresses probed, from SPCBASE up
+SCIDX   = $18084                ; u32 probe index, lives in memory across faults
+SCSAVSP = $18088                ; u32 stack pointer saved before the loop
+SCVAL   = $18100                ; SCN bytes: what each address read
+SCOK    = $18140                ; SCN bytes: 1 = answered, 0 = bus error
+SCTMP   = $180D0                ; u32: TEMP writeback -- $A5 written, read back
+SCTMPOK = $180D4                ; u32: 1 = the writeback completed without fault
+SCRD    = $180D8                ; u32: scsi_read's return, 0 = the read worked
+SCDRG   = $180DC                ; u32: $5A written to DREG then read straight back
+SCDRS   = $180E0                ; u32: SSTS between that write and that read
+SCDST   = $20000                ; where the read lands
+SCBLKS  = 8                     ; 8 x 512 B, enough to cross a sector boundary
+SCRD2   = $180E4                ; u32: the second read's return
+SCER1   = $180E8                ; u32: SC_ERR as it stood after the first read
+SCER2   = $180EC                ; u32: ...and after the second
+SCDST2  = $28000                ; where the second read lands
+SCLBA2  = 1000                  ; a NON-ZERO LBA: block 0 would pass even if the
+                                ; LBA bytes of the command block were ignored
+SCBLK2  = 4
+
+SPCBASE = $EA0000
+
+        org     $10000
+start:
+        move.l  #buserr,$8.w            ; vector 2
+        clr.l   SCFLAG.l
+        clr.l   SCTMP.l
+        clr.l   SCTMPOK.l
+        clr.l   SCIDX.l
+        move.l  sp,SCSAVSP.l
+
+; ---- probe SCN addresses, one at a time, surviving each fault
+ploop:
+        move.l  SCIDX.l,d0
+        cmp.l   #SCN,d0
+        bge.s   pdone
+        lea     SPCBASE,a0
+        adda.l  d0,a0
+        lea     SCVAL.l,a1
+        lea     SCOK.l,a2
+        move.b  #1,0(a2,d0.l)           ; assume it answers; the handler undoes
+        move.b  (a0),d1                 ; <- the access under test
+        move.b  d1,0(a1,d0.l)
+        addq.l  #1,SCIDX.l
+        bra.s   ploop
+pdone:
+
+; ---- TEMP (register 11 on the believed map, $EA0017) is a scratch latch on a
+; real MB89352.  Writing a pattern and reading it back separates "these odd
+; bytes are registers" from "these odd bytes are a mirror of something".
+; Guarded the same way: if it faults, the handler lands in ploop with SCIDX
+; already past the end, falls through here again, and SCTMPOK stays 0.
+        move.b  #$A5,SPCBASE+23
+        moveq   #0,d0
+        move.b  SPCBASE+23,d0
+        move.l  d0,SCTMP.l
+        move.l  #1,SCTMPOK.l
+
+; ---- DOES A WRITE TO THE DATA REGISTER REACH THE CHIP AT ALL?
+; $EA0015 is the one address x68k_scsiext.cpp puts its own glue on, and that
+; glue DROPS a write when the DMAC's OWN is asserted and DRQ is low.  A dropped
+; write is invisible: no error, no status bit, nothing.  So it is tested
+; directly, before any SCSI protocol can be blamed for it.  dreg_w enqueues into
+; the FIFO, so DREG_EMPTY must fall between the write and the read, and the read
+; must give the byte back.
+        bsr     scsi_init
+        move.b  #$5A,SPCBASE+21
+        moveq   #0,d0
+        move.b  SPCBASE+13,d0           ; SSTS: is the FIFO still empty?
+        move.l  d0,SCDRS.l
+        moveq   #0,d0
+        move.b  SPCBASE+21,d0
+        move.l  d0,SCDRG.l
+
+; ---- the SPC is reachable; now make it fetch something.  A read of the first
+; SCBLKS sectors, in PIO, verified BY THE HOST against the same bytes in
+; tmp/dlxdisk.img.  That is the whole of P4's correctness half in one line: the
+; player's own code selected a target, issued a READ(10) and got the disc's
+; bytes back, with no IOCS and no host in the path.
+        bsr     scsi_init
+        moveq   #0,d3                   ; LBA 0
+        moveq   #SCBLKS,d4
+        lea     SCDST,a1
+        bsr     scsi_read
+        move.l  d0,SCRD.l
+        move.l  SC_ERR.l,SCER1.l        ; SC_ERR is the LAST error, so it is
+                                        ; captured per read: reading it once at
+                                        ; the end reported the second read's
+                                        ; failure against the first read's name.
+
+; ---- and again, somewhere else on the disc.  A read of LBA 0 is passed by a
+; driver that emits a malformed LBA field, because zero is what a malformed
+; field usually is.  This one is not.
+        move.l  #SCLBA2,d3
+        moveq   #SCBLK2,d4
+        lea     SCDST2,a1
+        bsr     scsi_read
+        move.l  d0,SCRD2.l
+        move.l  SC_ERR.l,SCER2.l
+
+        move.l  #1,SCFLAG.l
+hold:   bra.s   hold
+
+; ---- bus error.  Mark the address dead, step past it, re-enter the loop with a
+; stack pointer that is known good.  The stacked frame is abandoned deliberately:
+; there is nothing in it worth more than the next 57 addresses.
+buserr:
+        move.l  SCSAVSP.l,sp
+        move.l  SCIDX.l,d0
+        cmp.l   #SCN,d0
+        bge.s   btmp
+        lea     SCOK.l,a2
+        clr.b   0(a2,d0.l)
+        lea     SCVAL.l,a1
+        move.b  #$FF,0(a1,d0.l)
+        addq.l  #1,SCIDX.l
+        jmp     ploop
+btmp:                                   ; the fault was the TEMP writeback
+        move.l  #1,SCFLAG.l
+be:     bra.s   be
+
+        include "src/player/scsi.i"

src/player/stream.s

@@ -81,7 +81,22 @@ PACE    = $18034                ; producer -> decoder: frame ticks elapsed since
                                 ; release.  Frame i may not START before tick i.
 PACEON  = $18038                ; 1 = obey PACE.  0 leaves the loop free-running,
                                 ; byte for byte the loop FINDINGS 49 measured.
+LATEFR  = $18080                ; frames that reached the pace gate with their
+                                ; tick ALREADY ARRIVED, i.e. did not idle for a
+                                ; single poll -- the previous frame used its
+                                ; whole slot.  See the gate below.
+LATEMAX = $18084                ; the worst of those, in WHOLE ticks overrun
+LATE1ST = $18088                ; index of the FIRST such frame, so that a
+                                ; count can be told apart from a start-up
+                                ; transient without re-running anything
+CLKON   = $1803C                ; 1 = the 68000 paces ITSELF: src/player/clock.i
+                                ; drives PACE off the CRTC's V-DISP instead of
+                                ; the host writing it.  Needs PACEON=1; the gate
+                                ; below cannot tell the two apart and must not.
 DESC    = $18100                ; DESCN x u32, record base addresses
+; The producer's own words -- RINGOWN, the transport mailbox and its
+; instruments -- are in src/player/ring.i, at $18300 and up, clear of DESC's
+; 256 bytes.
 
 DESCN   = 64                    ; power of two; the index is masked, not compared
 DESCM   = (DESCN-1)*4           ; mask for a BYTE offset into DESC
@@ -92,13 +107,76 @@ SPINMAX = 2000000               ; polls with no progress before giving up
 
         org     $10000
 start:
+; ---- the ring producer, if this run is asking the 68000 to fill its own ring
+; (ROADMAP P5, src/player/ring.i).  It goes FIRST because it only builds tables
+; and touches no hardware: a run that cannot build them should not have armed an
+; interrupt source first.
+        tst.l   RINGOWN.l
+        beq.s   noring
+; ---- the transport, before the producer that will ask it for something.
+; ring_init ends in a ring_seek and a seek WAITS for the channel to go quiet, so
+; the thing that makes the channel quiet has to exist first.  With XF_SCSI = 0
+; this brings up nothing and the host is the transport (FINDINGS 55).
+        bsr     xf_init
+        bsr     ring_init
+        tst.l   d0
+        bpl.s   noring
+        move.l  #$E3,FLAG.l             ; the record index is longer than ROFF
+        bra     hold
+noring:
+; ---- the frame clock, if this run is asking the 68000 to keep its own time.
+; It goes here rather than inside the frame loop because clk_init CLEARS PACE:
+; tick 0 has to be the instant the decoder was released, exactly as it is when
+; the host writes PACE, or the first frame's deadline moves.
+        tst.l   CLKON.l
+        beq.s   noclk
+        bsr     clk_init
+        tst.l   CLK_ERR.l
+        beq.s   noclk
+        move.l  #$E2,FLAG.l             ; the clock refused; CLK_ERR says why
+        bra     hold
+noclk:
         move.l  #1,FLAG.l               ; timer starts here
 outer:
         move.l  NFR.l,SCR_N.l
         clr.l   FR_TAIL.l
         clr.l   STALLS.l
         clr.l   SPINS.l
+        clr.l   LATEFR.l
+        clr.l   LATEMAX.l
+        move.l  #-1,LATE1ST.l
+; ---- SEEK AND PREFILL.  Every pass starts with a seek to record 0 -- which on
+; the first pass is just "start of scene" and on any later one is a REAL seek:
+; the channel has to go quiet, the ring is declared empty, and the whole
+; lookahead 51.3 says takes seconds of play to accumulate is thrown away and
+; rebuilt from the prefill up.  That is the branch point rehearsed with the one
+; thing a rig can check afterwards -- the decode has to still be pixel-exact.
+;
+; The clock is REBASED here rather than at clk_init, because tick 0 must be the
+; instant the decoder is released and the prefill happens before that.  Under a
+; host-written PACE this word belongs to the host, so it is only touched when
+; the 68000 is keeping its own time; ITER>1 therefore needs CLKON.
+        tst.l   RINGOWN.l
+        beq.s   noseek
+        moveq   #0,d0
+        bsr     ring_seek
+        bsr     ring_prefill
+        tst.l   d0
+        bpl.s   .pfok
+        move.l  #$E1,FLAG.l             ; the transport never delivered
+        bra     hold
+.pfok:
+        tst.l   CLKON.l
+        beq.s   noseek
+        clr.l   PACE.l                  ; at most one tick is lost to a V-DISP
+                                        ; landing between the ISR and here
+noseek:
 frameloop:
+        tst.l   RINGOWN.l
+        beq.s   nomark
+        move.l  FR_TAIL.l,d0
+        bsr     ring_mark
+nomark:
 ; ---- PACE GATE (FINDINGS 49.7.2, and it is the whole point of this session).
 ; Free-running, this loop asks for record i the instant it finishes record i-1,
 ; so it outruns any finite pipe, the ring NEVER backs up, and the producer's
@@ -117,8 +195,63 @@ frameloop:
 ;
 ; It also makes STALLS mean something.  Free-running, a stall is EARLINESS
 ; (49.6); paced, a frame that has to wait for its record is a real underrun.
+;
+; AND IT COUNTS THE FRAMES THAT WERE ALREADY LATE, which is a question only a
+; REAL frame clock raises.  12 fps on a 55.4577 Hz raster is 4.6215 refreshes
+; per frame, so the divider hands out slots of 4 refreshes (72.13 ms) and 5
+; (90.16 ms), 37.9% of them short -- and the SHORT one is 13.4% under the
+; 83.33 ms every budget in this project is priced against (FINDINGS 54).  A
+; frame that does not fit its slot does not fail here: PACE has already moved
+; on, so the next frame starts the instant this one finishes and the clock
+; catches up by itself.  What it costs is one late PRESENT, and nothing in this
+; tree counted those because until now the tick was a host model with no
+; cadence in it at all.
+;
+; The test is "did this frame have to WAIT", not "is it a whole tick behind".
+; Frame i waits while PACE < i; so PACE >= i on arrival means the decoder came
+; to the gate with slot i already open and idled for zero polls, which is the
+; same statement as "frame i-1 ran to the end of its slot".  A whole tick of
+; overrun -- PACE - FR_TAIL >= 1 -- is the much rarer case where it ran past
+; the end of the NEXT one, and is reported separately as the worst seen.
+;
+; Frame 0 is excluded: it starts at tick 0 by definition and has no predecessor
+; to have overrun.  The wait loop below is untouched -- all of this is ahead of
+; it, and the free-running path executes none of it.
         tst.l   PACEON.l
         beq.s   nopace
+        move.l  PACE.l,d0
+        cmp.l   FR_TAIL.l,d0            ; PACE < FR_TAIL: the slot has not come
+        bcs.s   pacesel                 ;   round yet, so this frame is EARLY
+        tst.l   FR_TAIL.l
+        beq.s   pacesel                 ; frame 0 starts AT tick 0 by definition
+        tst.l   LATEFR.l
+        bne.s   .nf1
+        move.l  FR_TAIL.l,LATE1ST.l
+.nf1:
+        addq.l  #1,LATEFR.l
+        sub.l   FR_TAIL.l,d0            ; whole ticks overrun; 0 = inside the
+        cmp.l   LATEMAX.l,d0            ;   slot but with nothing left of it
+        bls.s   pacewait
+        move.l  d0,LATEMAX.l
+pacesel:
+; Both branches above -- the early frame and frame 0 -- come here rather than
+; jumping straight into the legacy wait, because the early frame is the COMMON
+; case and it is the one with idle in it.  Routing it past this test was a real
+; bug and not a tidy-up: the producer then only ever ran from the record wait,
+; about once a frame, and the disc spent most of the scene stopped.
+        tst.l   RINGOWN.l
+        beq.s   pacewait
+; ---- THE IDLE IS WHERE THE DISC RUNS.  ring_poll retires the completed
+; request and issues the next one, and this loop is the only place in a paced
+; player with time to spare.  Polling once a FRAME instead would cap the fill at
+; one record per slot -- the wire rate exactly -- and a ring that can only keep
+; up never accumulates the slack a branch point spends (51.3).
+pacewaitR:
+        bsr     ring_poll
+        move.l  PACE.l,d0
+        cmp.l   FR_TAIL.l,d0
+        bcs.s   pacewaitR
+        bra.s   nopace
 pacewait:
         move.l  PACE.l,d0
         cmp.l   FR_TAIL.l,d0            ; d0 - FR_TAIL; carry = tick not reached
@@ -128,6 +261,22 @@ nopace:
 ; d1 counts polls for this frame; a nonzero d1 on exit means the frame stalled.
         moveq   #0,d1
         move.l  FR_TAIL.l,d2
+        tst.l   RINGOWN.l
+        beq.s   waitrec
+; ---- the same wait, with the producer inside it.  Here d1 counts POLLS rather
+; than spins, so STALLS still means "frames that had to wait" and SPINS is not
+; comparable with a host-filled run's.  A frame that waits here is a real
+; underrun either way: paced, its slot has already opened.
+waitrecR:
+        bsr     ring_poll
+        move.l  FR_HEAD.l,d0
+        cmp.l   d2,d0
+        bhi     gotrec
+        addq.l  #1,d1
+        cmp.l   #SPINMAX,d1
+        bcs.s   waitrecR
+        move.l  #$E1,FLAG.l
+        bra     hold
 waitrec:
         move.l  FR_HEAD.l,d0
         cmp.l   d2,d0
@@ -163,12 +312,16 @@ nostall:
         cmpa.l  SCR_END.l,a0            ; bitstream desync is silent otherwise
         bne     desync
 
-; ---- release.  Round up to 4 the same way decode.s does: the producer lays
-;      records on 4-byte boundaries, so the byte one past this record's padded
-;      end is the first byte the producer may reuse.
+; ---- release.  Round up to RECALN: the producer lays records on the
+;      container's own record boundaries (geom.i), so the byte one past this
+;      record's PADDED end is the first byte it may reuse.  Releasing only the
+;      bytes actually read would strand up to RECALN-1 of pad per record and
+;      the producer's free-space arithmetic would drift by that much a frame.
+;      This is an absolute address, so it is only the record's padded end
+;      because the ring base is RECALN-aligned as well.
         move.l  a0,d0
-        addq.l  #3,d0
-        and.b   #$FC,d0
+        addi.l  #RECALN-1,d0
+        andi.l  #~(RECALN-1),d0
         move.l  d0,RD_PTR.l
         addq.l  #1,FR_TAIL.l
 
@@ -176,9 +329,20 @@ nostall:
         bne     frameloop
         subq.l  #1,ITER.l
         bne     outer
+; ---- leave the MFP as it was found.  A rig that exits with a live interrupt
+;      source and a lowered mask hands the next thing that runs an interrupt it
+;      has no vector for, and the failure would land somewhere else entirely.
+        tst.l   CLKON.l
+        beq.s   noclk2
+        bsr     clk_stop
+noclk2:
         move.l  #$FF,FLAG.l             ; timer stops here
 hold:   bra.s   hold
 desync: move.l  #$EE,FLAG.l
         bra.s   hold
 
         include "src/player/frame.i"
+        include "src/player/clock.i"
+        include "src/player/ring.i"
+        include "src/player/scsi.i"
+        include "src/player/xfer.i"

src/player/xfer.i

@@ -0,0 +1,148 @@
+; ---------------------------------------------------------------- xfer.i
+; THE TRANSPORT BEHIND THE MAILBOX.  ROADMAP P4b.
+;
+; src/player/ring.i has always ended at a seam: it decides which record to
+; fetch, where in the ring to put it and when that is safe, writes those four
+; words into XF_SLOT and bumps XF_GO, and then polls XF_ACK.  On the other side
+; of that seam, until now, was tools/bench/stream.lua -- a host moving bytes at
+; a MODELLED rate, with XF_ACK synthesised out of emulated time.  A player has
+; no host.  This file is the other side: XF_GO is answered by src/player/scsi.i
+; issuing a real READ(10) to a real MB89352, and XF_ACK is a word the 68000
+; bumps when the bytes have landed.
+;
+; NOTHING ABOVE THE SEAM CHANGED, and that is deliberate for the same reason it
+; was in sessions 22 and 23: ring.i cannot tell which transport answered it, so
+; a green run here is a test of THIS file rather than of a new producer.  The
+; two hooks in ring.i (one in ring_poll, one in ring_seek's quiet-wait) are the
+; whole of the change on that side, and with XF_SCSI = 0 they are a tst and a
+; branch.
+;
+; IT IS SYNCHRONOUS, AND THAT IS NOT A SHORTCUT -- IT IS THE FINDING.  The
+; modelled transport overlapped: a request issued at time t landed at t + len/
+; rate while the 68000 got on with decoding, which is what a DMAC channel does.
+; Here the CPU moves every byte itself through $EA0015 (57.3: a PIO write to
+; that address is discarded, so even "PIO" runs the SPC in DMA mode with the CPU
+; standing in for the channel), so `bsr xf_service` does not start a transfer,
+; it PERFORMS one.  A two-deep request queue therefore buys nothing at all: the
+; second slot is filled and drained by the same instruction stream that would
+; have been decoding.  FINDINGS 55.3's whole result -- that a one-deep queue
+; gives away 6.8% of the pipe -- is about a transport that runs in parallel with
+; the CPU, and this one does not.
+;
+; So what this file is FOR is not to be the shipping transport.  It is to make
+; the shipping transport's cost measurable: the same 120 pixel-exact frames,
+; delivered by the machine off a real volume, with the CPU cost of doing it
+; charged where a rate model cannot hide it.  P4a -- the HD63450 holding the bus
+; -- is what makes the transfer overlap again, and until it exists this is the
+; honest floor.
+
+; ---- state.  Above src/player/ring.i's instruments (last: SK_WAIT at $1837C)
+; and below the disc-offset table at $19400.
+XF_SCSI  = $18380               ; 1 = the 68000 is the transport (input)
+XS_LBA0  = $18384               ; LBA of byte 0 of the scene's frame stream
+XS_NXFER = $18388               ; transfers completed
+XS_NBYTE = $1838C               ; record bytes delivered into the ring
+XS_NWIRE = $18390               ; bytes actually read off the disc, sectors and
+                                ;   all -- the two differ and 58.3 is why
+XS_ERR   = $18394               ; SC_ERR of the FIRST failure, 0 = none
+XS_ERRAT = $18398               ; ...and the request index it failed on
+
+; ---------------------------------------------------------------- xf_init
+; Clears the instruments and brings the SPC up, if this run has one.  Called
+; before ring_init, because ring_init ends in a ring_seek and a seek waits on
+; the transport.
+xf_init:
+        clr.l   XS_NXFER.l
+        clr.l   XS_NBYTE.l
+        clr.l   XS_NWIRE.l
+        clr.l   XS_ERR.l
+        move.l  #-1,XS_ERRAT.l
+        tst.l   XF_SCSI.l
+        beq.s   .out
+        bsr     scsi_init
+.out:   rts
+
+; ---------------------------------------------------------------- xf_service
+; Answer at most ONE outstanding request, then return.  Preserves every
+; register: it is called from inside ring_poll, which is itself called from
+; inside the decoder's wait loops and must be invisible to them.
+;
+; ONE PER CALL, not "drain the queue".  ring_poll retires exactly one completed
+; request per call as well, and a transport that answered both queued requests
+; in one visit would hand the retire loop two acks it can only take one poll at
+; a time -- which is legal, but it also means the decoder's wait loop would
+; disappear for two record times instead of one.  One per call keeps the two
+; sides stepping at the same rate.
+;
+; A RECORD IS NOT A SECTOR, and this is where that is dealt with.  ring.i asks
+; for a byte offset and a length; the target answers in 512 B blocks.  So the
+; command covers the sectors the record lies in, and SC_WSKIP/SC_WKEEP tell
+; src/player/scsi.i's DATA IN loop which of those bytes to store.  The ones
+; outside the window are pulled from the FIFO and dropped -- they are NOT
+; written past the ends of the destination, because the bytes on either side of
+; a record in the stream belong to records the decoder may still be reading and
+; the block loop has no bounds check (49.2).
+;
+; XS_NWIRE counts what the disc actually moved and XS_NBYTE what the ring got.
+; They are not the same number and the gap is a delivery cost, not an accounting
+; detail: it is bytes on the wire that no frame contains.
+xf_service:
+        tst.l   XF_SCSI.l
+        beq.s   .idle
+        move.l  XF_ACK.l,d0
+        cmp.l   XF_GO.l,d0
+        bcs.s   .work                   ; XF_ACK < XF_GO: something outstanding
+.idle:  rts
+.work:
+        movem.l d0-d7/a0-a2,-(sp)
+        move.l  XF_ACK.l,d0
+        move.l  d0,d1
+        and.l   #XF_SLM,d1
+        lsl.l   #4,d1                   ; * XF_SLSZ
+        lea     XF_SLOT.l,a0
+        adda.l  d1,a0
+        move.l  (a0),d1                 ; disc byte offset within the stream
+        movea.l 4(a0),a1                ; destination in the ring
+        move.l  8(a0),d2                ; length
+; ---- sector arithmetic
+        move.l  d1,d3
+        and.l   #511,d3                 ; bytes of the first sector to drop
+        move.l  d1,d4
+        lsr.l   #8,d4
+        lsr.l   #1,d4                   ; sector index within the stream
+        add.l   XS_LBA0.l,d4            ; ...and where the stream begins
+        move.l  d3,d5
+        add.l   d2,d5
+        addi.l  #511,d5
+        lsr.l   #8,d5
+        lsr.l   #1,d5                   ; sectors the record lies in
+        move.l  d3,SC_WSKIP.l
+        move.l  d2,SC_WKEEP.l
+        add.l   d2,XS_NBYTE.l
+        move.l  d5,d0
+        lsl.l   #8,d0
+        lsl.l   #1,d0
+        add.l   d0,XS_NWIRE.l
+        move.l  d4,d3                   ; d3 = LBA
+        move.l  d5,d4                   ; d4 = blocks
+        bsr     scsi_read_win
+        tst.l   d0
+        bmi.s   .err
+        addq.l  #1,XS_NXFER.l
+        addq.l  #1,XF_ACK.l             ; LAST: the bytes are all in the ring
+                                        ; before the request is called done
+        movem.l (sp)+,d0-d7/a0-a2
+        rts
+; ---- a failed read is NOT acked.  The record never becomes resident, the
+; decoder spins out in waitrec and reports a stalled producer, and XS_ERR says
+; which request failed and why.  Acking a failed transfer would publish a
+; descriptor for a buffer full of whatever was there before -- and the decoder
+; would find a plausible-looking length word in it and desync somewhere else
+; entirely.
+.err:
+        tst.l   XS_ERR.l
+        bne.s   .err2
+        move.l  SC_ERR.l,XS_ERR.l
+        move.l  XF_ACK.l,XS_ERRAT.l
+.err2:  movem.l (sp)+,d0-d7/a0-a2
+        rts

tools/analysis/13_cpu_ratectl.py

@@ -40,12 +40,24 @@ BUDGET = RC.FRAME_CYCLES
 # H.build is ~55 s, nearly all k-means, and it does not depend on the profile:
 # both ship k1=k4=256. One build, cached, serves every row of the table.
 cache = a.cache or f"tmp/model_{os.path.basename(a.frames_dir.rstrip('/'))}.pkl"
+# The build parameters are stored with the model and a mismatch rebuilds: the
+# cache is keyed on the frames directory alone, and once H.build acquired an
+# option (session 28's reserved black entry, 23.4) a stale pickle would quietly
+# serve a model the shipping encoder no longer builds.  Same guard as
+# tools/analysis/16_span_roundtrip.py.
+SIG = dict(k1=256, k4=256, iters=16, reserve_black=True)
+m = None
 if os.path.exists(cache):
     m = pickle.load(open(cache, "rb"))
-    print(f"model from {cache}")
-else:
+    if m.get("sig") != SIG:
+        print(f"{cache}: built with {m.get('sig')}, wanted {SIG} -- rebuilding")
+        m = None
+    else:
+        print(f"model from {cache}")
+if m is None:
     t = time.time()
-    m = H.build(a.frames_dir, k1=256, k4=256, iters=16)
+    m = H.build(a.frames_dir, **SIG)
+    m["sig"] = SIG
     pickle.dump(m, open(cache, "wb"))
     print(f"built model in {time.time()-t:.0f} s -> {cache}")
 print(f"{a.frames_dir}: {len(m['idx'])} frames, {m['nb']} blocks, "

tools/analysis/15_bus_occupancy.py

@@ -62,6 +62,11 @@ ap.add_argument("container", nargs="?", default="tmp/rc_fr_singe_scsi_cpufit.dlx
 ap.add_argument("--csv", default="tmp/c68k_frames.csv",
                 help="per-frame output of tools/bench/c68k/run.sh")
 ap.add_argument("--nframes", type=int, default=None)
+ap.add_argument("--kbps", type=float, default=None,
+                help="delivery rate in KB/s. OPTIONAL and there is no default "
+                     "(FINDINGS 50): supply it and the AUTO-REQUEST rows are "
+                     "added, which are the only rows whose cost depends on how "
+                     "long the record takes to arrive (59.3).")
 a = ap.parse_args()
 if not os.path.exists(a.container):
     sys.exit(f"missing {a.container}")
@@ -162,7 +167,13 @@ print("THE OTHER TWO MASTERS -- what the DMAC takes out of the same frame\n")
 FPS = d.fps
 CPUHZ = 10e6                       # stock X68000, MAME 0.277 x68k.cpp:1133
 FRAME_CLK = CPUHZ / FPS
-vid_bpf = sum(n + 4 for (_, n) in d.frames[:NF]) / NF     # DLX2 record padding
+# WHAT THE TRANSPORT MOVES, which is the PADDED record and not the payload.
+# Under DLX2/3/4 the pad was 0..3 B and the distinction was noise; under DLX5 it
+# is 0..511 B, and charging the payload would price the channel for bytes it
+# does not carry while the disc carries them anyway.  A budget that debits only
+# the bytes a frame CONTAINS is the same incomplete accounting this project has
+# been caught by before -- the pad is delivered, so the pad is charged.
+vid_bpf = sum(d.record_lengths()[:NF]) / NF
 aud_bpf = B.ADPCM_BYTES_PER_S / FPS
 a_lo = aud_bpf * B.ADPCM_CLK_BYTE_BEST
 a_hi = aud_bpf * B.ADPCM_CLK_BYTE_WORST
@@ -186,19 +197,102 @@ print(f"    {'W (clk/byte)':<16}{'clk/frame':>12}{'% of frame':>12}   "
       f"{'CPU+audio+video':>18}")
 for W, note in ((5.0,  "single address, bus held (11_cpu_budget.py default)"),
                 (8.0,  "FINDINGS 5's long-standing per-word ESTIMATE"),
+                (9.0,  "DUAL address, bus held -- and the FLOOR of every "
+                       "dual-address\n                                        "
+                       "         configuration, auto-request included (59.3)"),
                 (12.0, "single address, arbitrated per byte"),
                 (16.0, "what the ROM programs for SASI (best case)"),
-                (19.0, "what the ROM programs for SASI (worst case)")):
+                (19.0, "what the ROM programs for SASI (worst case)"),
+                (87.28, "PIO -- MEASURED, FINDINGS 58.2, the CPU doing it itself")):
     v = vid_bpf * W
     tot_clk = (cpu_clk if cyc_t.any() else 0) + a_lo + v
-    print(f"    {W:<16.0f}{v:>12,.0f}{100*v/FRAME_CLK:>11.1f}%   "
+    print(f"    {W:<16.6g}{v:>12,.0f}{100*v/FRAME_CLK:>11.1f}%   "
           f"{100*tot_clk/FRAME_CLK:>17.1f}%   {note}")
 print(f"\n  (the last column adds the MEASURED mean decode and the BEST-CASE "
       f"audio, so it is\n   the optimistic end of every row. 100% is the frame "
       f"deadline at {FPS:g} fps.)")
 print(f"""
-Audio is {100*a_lo/FRAME_CLK:.2f}%..{100*a_hi/FRAME_CLK:.2f}% of the frame and video is {vid_bpf*5/FRAME_CLK*100:.0f}%..{vid_bpf*19/FRAME_CLK*100:.0f}%. The unpriced audio
-stream was never the risk P6 called it -- ON THE BUS. What the same reading of
-the ROM found is that the DISK's per-byte cost has a worked example on this
-machine, it is 16..19 clocks, and at that price this design does not fit at any
-container size. W is the number to attack, and it is a PLAYER decision.""")
+Audio is {100*a_lo/FRAME_CLK:.2f}%..{100*a_hi/FRAME_CLK:.2f}% of the frame and video is {vid_bpf*5/FRAME_CLK*100:.0f}%..{vid_bpf*19/FRAME_CLK*100:.0f}% over the ladder, against
+{vid_bpf*87.28/FRAME_CLK*100:.0f}% for the PIO transport FINDINGS 58.2 measured. The unpriced audio stream
+was never the risk P6 called it -- ON THE BUS.""")
+
+# --- HEADROOM, AND THE FLOOR UNDER THE LADDER -----------------------------
+# Added session 27. The sweep above answers "what does each W cost"; it never
+# answered "what can this frame afford", and the two are not the same question.
+# FINDINGS 59.2 is why it matters now: with no external request line the only
+# configurations that can be run are dual-address, and a dual-address byte has
+# a FLOOR -- one 4-clock read of the device plus one 5-clock write to memory,
+# buscost.DMA_DUAL_BYTE_CLK. No GCR share and no delivery rate goes under it.
+print("\n" + "=" * 72)
+print("WHAT THE FRAME CAN AFFORD, AND THE FLOOR UNDER THE LADDER\n")
+head_clk = FRAME_CLK - (cpu_clk if cyc_t.any() else 0) - a_lo
+head_wb = head_clk / vid_bpf
+print(f"  headroom after the MEASURED decode and best-case audio: "
+      f"{head_clk:,.0f} clk = {100*head_clk/FRAME_CLK:.1f}%")
+print(f"  at {vid_bpf:,.0f} B a frame that is {head_wb:.2f} CLOCKS PER BYTE, and "
+      f"that is the number\n  a transport has to come in under.\n")
+floor = B.DMA_DUAL_BYTE_CLK
+print(f"  dual-address floor  {floor} clk/B  ({B.DMA_READ_CLK} read of the "
+      f"device + {B.DMA_WRITE_CLK} write to memory, Fig 4-25)")
+print(f"  single-address held {B.DMA_DISK_CLK_WORD_HELD} clk/B  (one memory "
+      f"write; needs the device to ACK, i.e. a REQUEST LINE)")
+if head_wb < floor:
+    print(f"""
+  SO DUAL ADDRESS DOES NOT FIT THIS CONTAINER AT {FPS:g} fps -- not at any
+  delivery rate and not at any GCR share, because {head_wb:.2f} < {floor}. A share
+  decides whether the channel sits AT the floor or above it; it cannot
+  go under it. That is FINDINGS 59.2's three bounds arriving in the
+  budget: the configurations this machine can run are exactly the ones
+  the frame cannot afford, and the one it can afford -- single address,
+  {B.DMA_DISK_CLK_WORD_HELD} clk/B, {100*vid_bpf*B.DMA_DISK_CLK_WORD_HELD/FRAME_CLK:.1f}% -- needs the request line ROADMAP B3 asks about.""")
+    for w, what in ((floor, "dual address"), (B.DMA_DISK_CLK_WORD_HELD, "single address")):
+        tgt = head_clk / w
+        print(f"\n  TO FIT AT {w} clk/B ({what}) THIS CONTAINER MUST COME DOWN TO")
+        print(f"    {tgt:,.0f} B a frame = {tgt*FPS/1024:,.0f} KB/s of payload "
+              f"(it is {vid_bpf:,.0f} B, {vid_bpf*FPS/1024:,.0f} KB/s)"
+              + ("  -- already met" if vid_bpf <= tgt else
+                 f"  -- {100*(vid_bpf/tgt-1):.0f}% too big"))
+    print(f"""
+  AND THAT IS THE PESSIMISTIC READING OF THE ENCODER LEVER: a lighter
+  container also DECODES cheaper, so the decode term above falls with
+  the byte term. The figure to re-derive it against is this tool run on
+  the lighter container -- with its OWN C68K measurement, because the
+  cross-check at the top is what licenses every number below it.""")
+else:
+    print(f"\n  The frame affords {head_wb:.2f} clk/B, which is at or above the "
+          f"{floor} clk/B dual-address floor.")
+
+# --- AUTO-REQUEST, and only when a rate is supplied ------------------------
+# These are the rows 59.3 added and they are the only ones here whose cost is
+# not a property of the transfer: an auto-requested channel spends its share of
+# the bus whether or not a byte is there, so what a record costs depends on how
+# long it takes to ARRIVE. No default rate, deliberately (FINDINGS 50).
+if a.kbps:
+    RATE = a.kbps * 1024.0
+    wire_clk = vid_bpf / RATE * CPUHZ
+    cap = 0.5 * CPUHZ / B.DMA_DUAL_BYTE_CLK          # the 50% share's ceiling
+    print("\n" + "=" * 72)
+    print(f"AUTO-REQUEST AT {a.kbps:g} KB/s -- charged by TIME, not by byte "
+          f"(59.3)\n")
+    print(f"  the record takes {wire_clk:,.0f} clk to arrive = "
+          f"{100*wire_clk/FRAME_CLK:.1f}% of a frame\n")
+    print(f"    {'configuration':<34}{'clk/B':>8}{'% of frame':>12}"
+          f"{'CPU+audio+video':>18}")
+    rows = [("REQG 01, max rate (100% of the bus)", 1.0, None)]
+    for br, share in ((0, .5), (1, .25), (2, .125), (3, .0625)):
+        rows.append((f"REQG 00, LRAR BR={br:02b}, {share*100:g}% share", share,
+                     share * CPUHZ / B.DMA_DUAL_BYTE_CLK))
+    for name, share, sustains in rows:
+        v = share * wire_clk
+        tot = (cpu_clk if cyc_t.any() else 0) + a_lo + v
+        flag = ""
+        if sustains is not None and sustains < RATE:
+            flag = f"  cannot carry the rate ({sustains/1024:.0f} KB/s max)"
+        print(f"    {name:<34}{v/vid_bpf:>8.2f}{100*v/FRAME_CLK:>11.1f}%"
+              f"{100*tot/FRAME_CLK:>17.1f}%{flag}")
+    print(f"""
+  A FASTER DISC MAKES AUTO-REQUEST CHEAPER, which no W does -- the share is
+  spent over a shorter wire time. But it cannot reach the floor: a 50% share
+  tops out at {cap/1024:,.0f} KB/s, above which the CHANNEL is the bottleneck and the
+  delivered rate falls back to it. At that ceiling the cost is exactly the
+  {B.DMA_DUAL_BYTE_CLK} clk/B floor, which is where the section above already put it.""")

tools/analysis/16_span_roundtrip.py

@@ -43,12 +43,26 @@ ap.add_argument("--cache", default=None)
 a = ap.parse_args()
 
 cache = a.cache or f"tmp/model_{os.path.basename(a.frames_dir.rstrip('/'))}.pkl"
+# The cache is keyed on the frames directory ALONE, which was fine while
+# H.build had no options and became a trap the moment it did: session 28's
+# reserved black entry (23.4) changes the palette, the codebooks and every
+# index in the model, and a pickle from before it would have let this gate
+# round-trip a container the shipping encoder no longer emits -- green, and
+# testing the wrong artefact.  So the build parameters are stored WITH the
+# model and a mismatch rebuilds.
+SIG = dict(k1=256, k4=256, iters=16, reserve_black=True)
+m = None
 if os.path.exists(cache):
     m = pickle.load(open(cache, "rb"))
-    print(f"model from {cache}")
-else:
+    if m.get("sig") != SIG:
+        print(f"{cache}: built with {m.get('sig')}, wanted {SIG} -- rebuilding")
+        m = None
+    else:
+        print(f"model from {cache}")
+if m is None:
     t = time.time()
-    m = H.build(a.frames_dir, k1=256, k4=256, iters=16)
+    m = H.build(a.frames_dir, **SIG)
+    m["sig"] = SIG
     pickle.dump(m, open(cache, "wb"))
     print(f"built model in {time.time()-t:.0f} s -> {cache}")
 
@@ -72,8 +86,15 @@ for span_mode in ("need", "all"):
           f"({100*px/(len(recs)*m['H']*m['W']):.1f}% of all pixels)")
 
     d = DLX(path)
-    if d.version != 3:
-        print(f"FAIL: container is DLX{d.version}, not DLX3"); bad += 1; continue
+    if not d.has_spans:
+        print(f"FAIL: container is DLX{d.version}, which has no span section")
+        bad += 1; continue
+    # DLX4 adds the record index and DLX() cross-checks it against its own walk
+    # of the frame stream, so simply constructing it above has already gated
+    # that. Said out loud here because it is easy to read this as version drift.
+    if d.has_index:
+        print(f"  DLX{d.version}: record index agrees with the frame stream on all "
+              f"{d.nframes} records ({2*d.nframes:,} B of scene header)")
 
     # The decoder's own walk of the span section must land exactly where the
     # block payload starts, and blocks() already raises if the payload does not
@@ -104,5 +125,5 @@ print()
 if bad:
     print(f"FAILED: {bad} check(s)")
     sys.exit(1)
-print("OK  the DLX3 span container round-trips: the reference decoder rebuilds "
+print("OK  the span container round-trips: the reference decoder rebuilds "
       "the\n    encoder's reconstruction exactly, from the emitted bytes.")

tools/analysis/19_ring_stream.py

@@ -71,7 +71,7 @@ def records(path):
     the ring must hold per frame are the padded ones, not the payload.
     """
     d = DLX(path)
-    rec = np.array([4 + n + (-(4 + n) % 4) for _, n in d.frames], np.int64)
+    rec = np.array(d.record_lengths(), np.int64)
     return d, rec
 
 

tools/analysis/20_seek_slack.py

@@ -32,7 +32,7 @@ SECTOR = 512
 
 def records(path):
     d = DLX(path)
-    rec = np.array([4 + n + (-(4 + n) % 4) for _, n in d.frames], np.int64)
+    rec = np.array(d.record_lengths(), np.int64)
     return d, rec
 
 

tools/analysis/21_iplrom_dmac.py

@@ -31,6 +31,7 @@ registers itself. It is evidence about what Sharp's engineers could get the
 board to do, from the vendor, for these exact devices.
 """
 import sys, os, argparse, hashlib
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
 
 BASE = 0xFE0000          # where the IPL ROM is mapped (and its 0xFF0000 alias)
 
@@ -41,50 +42,7 @@ KNOWN = {
         "IPL 1.0 (MAME x68000 -bios ipl10), 131,072 B",
 }
 
-# --- MC68450 register map, by offset inside a channel's 0x40 block ----------
-REG = {0x00: "CSR", 0x01: "CER", 0x04: "DCR", 0x05: "OCR", 0x06: "SCR",
-       0x07: "CCR", 0x0A: "MTC", 0x0C: "MAR", 0x14: "DAR", 0x1A: "BTC",
-       0x1C: "BAR", 0x25: "NIV", 0x27: "EIV", 0x29: "MFC", 0x2D: "CPR",
-       0x31: "DFC", 0x39: "BFC"}
-
-XRM  = {0: "burst",
-        1: "UNDEFINED",
-        2: "cycle steal WITHOUT hold  (bus released between operands)",
-        3: "cycle steal with hold"}
-DTYP = {0: "68000-compatible, EXPLICITLY addressed  -> DUAL ADDRESS",
-        1: "6800-compatible, EXPLICITLY addressed   -> DUAL ADDRESS",
-        2: "device with ACK, implicitly addressed   -> SINGLE ADDRESS",
-        3: "device with ACK and RDY, implicit       -> SINGLE ADDRESS"}
-DPS  = {0: "8-bit port", 1: "16-bit port"}
-PCL  = {0: "status input", 1: "status input with interrupt",
-        2: "start pulse", 3: "abort input"}
-SIZE = {0: "byte", 1: "word", 2: "long word", 3: "byte, unpacked"}
-CHAIN= {0: "none", 1: "UNDEFINED", 2: "array", 3: "linked array"}
-REQG = {0: "auto-request at limited rate", 1: "auto-request at max rate",
-        2: "EXTERNAL request  (one operand per device request)",
-        3: "auto-request first operand, external thereafter"}
-
-
-def dcr(v):
-    return [f"XRM  = {v>>6&3:02b}  {XRM[v>>6&3]}",
-            f"DTYP = {v>>4&3:02b}  {DTYP[v>>4&3]}",
-            f"DPS  = {v>>3&1:b}   {DPS[v>>3&1]}",
-            f"PCL  = {v&3:02b}  {PCL[v&3]}"]
-
-
-def ocr(v):
-    return [f"DIR  = {v>>7&1:b}   " +
-            ("device -> memory (read)" if v & 0x80 else "memory -> device (write)"),
-            f"SIZE = {v>>4&3:02b}  {SIZE[v>>4&3]}",
-            f"CHAIN= {v>>2&3:02b}  {CHAIN[v>>2&3]}",
-            f"REQG = {v&3:02b}  {REQG[v&3]}"]
-
-
-def scr(v):
-    m = {0: "no count", 1: "increment", 2: "decrement", 3: "UNDEFINED"}
-    return [f"MAC  = {v>>2&3:02b}  memory address {m[v>>2&3]}",
-            f"DAC  = {v&3:02b}  device address {m[v&3]}"]
-
+from mc68450 import REG, XRM, DTYP, DPS, PCL, SIZE, CHAIN, REQG, dcr, ocr, scr
 
 # --- the evidence ----------------------------------------------------------
 # (address, expected bytes, one-line description). Every register value quoted

tools/analysis/22_scene_load.py

@@ -0,0 +1,106 @@
+#!/usr/bin/env python3
+"""What a scene change costs, now that the loader runs on the 68000 (FINDINGS 53).
+
+    python3 tools/analysis/22_scene_load.py [container ...] --kbps R [R ...]
+
+ROADMAP P1 asked for the codebook expansion to be priced "against the refill
+climb, not treated as free setup", and that is the whole job of this file.  A
+scene change is the one moment where every cost in this project lands at once:
+the ring is empty because of the seek, the header has to arrive before a single
+frame can be drawn, and the 68000 cannot decode anything until it has expanded
+the codebooks out of that header.
+
+THREE COSTS, IN THREE DIFFERENT UNITS, and they are not interchangeable:
+
+  * BYTES.  The container's header region -- palette, CB1, CB4 -- must be
+    delivered before frame 0 can be decoded.  It is not part of any frame
+    record, so no rate table in this tree has ever counted it.
+  * CLOCKS.  What src/player/load.i costs to turn that header into what the
+    block loop reads, MEASURED on the emulated 68000 by tools/bench/load.lua
+    and parsed out of its log rather than copied in here as a constant.
+  * ACCUMULATED SLACK.  The bytes above are bytes the pipe did not spend
+    filling the ring, so they cost play-time at the surplus rate (pipe - wire),
+    which is the currency FINDINGS 51.3 established a branch point spends.
+    This is the one that compounds: it is charged on top of the seek itself.
+
+`--kbps` is REQUIRED and takes no default, for the reason FINDINGS 50 gives.
+Rates are decimal-KB per the rest of the tree's tooling; sizes are KiB.
+"""
+import sys, os, re, argparse
+sys.path.insert(0, "tools/encoder")
+import numpy as np
+from dlx import DLX
+import ratectl as RC
+
+FPS = 12
+CPUHZ = 10_000_000
+
+
+def rig_cycles(path):
+    """The measured per-stage cost, out of tools/bench/load.lua's own log.
+
+    Parsed rather than pasted: a constant copied in here would go stale the
+    first time load.i changed, and it would go stale SILENTLY -- the arithmetic
+    below would keep working and keep being wrong.
+    """
+    if not os.path.exists(path):
+        sys.exit(f"no rig log at {path} -- run tools/bench/load_run.sh first")
+    out = {}
+    for line in open(path, "rb").read().decode("utf-8", "replace").splitlines():
+        m = re.search(r"^\[LOD\]\s+(\S.*?)\s{2,}(\d+) cyc", line)
+        if m:
+            out[m.group(1).strip()] = int(m.group(2))
+    need = ("SCENE CHANGE: codebooks + palette", "scratch tables only (boot, once)")
+    for k in need:
+        if k not in out:
+            sys.exit(f"{path} has no '{k}' line -- is it a load.lua summary?")
+    return out
+
+
+ap = argparse.ArgumentParser()
+ap.add_argument("containers", nargs="*", default=["tmp/rc_fr_singe_scsi_span.dlx"])
+ap.add_argument("--kbps", type=float, nargs="+", required=True,
+                help="delivered pipe rates, KB/s. REQUIRED, no default (FINDINGS 50)")
+ap.add_argument("--log", default="tmp/load_check.log",
+                help="tools/bench/load.lua's log, for the measured cycle counts")
+a = ap.parse_args()
+
+cyc = rig_cycles(a.log)
+scene_cyc = cyc["SCENE CHANGE: codebooks + palette"]
+boot_cyc = cyc["scratch tables only (boot, once)"]
+frame_cyc = CPUHZ / FPS
+
+print(f"measured on the emulated 68000 ({a.log}):")
+print(f"  per scene change   {scene_cyc:>8,} clocks = {1000*scene_cyc/CPUHZ:6.2f} ms "
+      f"= {100*scene_cyc/frame_cyc:.1f}% of one {FPS}fps frame")
+print(f"  once at boot       {boot_cyc:>8,} clocks = {1000*boot_cyc/CPUHZ:6.2f} ms "
+      f"  (the three scratch tables: scene-independent)")
+
+for path in a.containers:
+    d = DLX(path)
+    hdr = int.from_bytes(d.raw[28:32], "big")
+    rec = np.array(d.record_lengths(), np.int64)
+    wire = rec.mean() * FPS / 1024 + RC.AUDIO_KBPS
+    print(f"\n=== {path}: header region {hdr:,} B "
+          f"(pal 768 + cb1 {d.k1*16:,} + cb4 {d.k4*4:,} + 32), wire {wire:.1f} KB/s")
+    print(f"{'pipe':>6} {'header ms':>10} {'+load ms':>9} {'total':>7} "
+          f"{'frames':>7} {'surplus':>9} {'slack s':>9}")
+    for kbps in a.kbps:
+        hdr_ms = 1000 * hdr / (kbps * 1024)
+        load_ms = 1000 * scene_cyc / CPUHZ
+        total = hdr_ms + load_ms
+        surplus = kbps - wire
+        # What the header costs in the currency of 51.3: play-time at the
+        # surplus rate.  A negative surplus means the container does not fit the
+        # pipe at all and no amount of play buys the bytes back.
+        slack = f"{hdr/(surplus*1024):8.3f}" if surplus > 0 else "  NEVER"
+        print(f"{kbps:>6.0f} {hdr_ms:>10.2f} {load_ms:>9.2f} {total:>7.2f} "
+              f"{total/(1000/FPS):>7.2f} {surplus:>9.1f} {slack:>9}")
+
+print("""
+Reading it.  The 'frames' column is the scene change's FIXED cost in 12fps
+frame slots, before the ring has been given a single frame of lookahead -- so it
+is a floor under the black gap at a branch point, not the gap itself.  The
+'slack s' column is the one that compounds with FINDINGS 51.3: the header's
+bytes are bytes that did not go into the ring, so they lengthen the climb back
+to the seek-slack ceiling by that much play-time, every time.""")

tools/analysis/23_frame_clock.py

@@ -0,0 +1,254 @@
+#!/usr/bin/env python3
+"""What a real frame clock can be built from, and what its cadence costs.
+
+    python3 tools/analysis/23_frame_clock.py [--fps 12] [--vtotal 568]
+                                             [--csv tmp/c68k_frames.csv]
+
+ROADMAP P3 says "needs MFP timer or VBL", which hides the fact that ONE OF
+THOSE CANNOT DO IT and the other cannot do it either without a divider.  This
+file enumerates the space rather than asserting a conclusion, the way FINDINGS
+47 had to be re-done once a hardware "no" turned out to be a claim about a whole
+configuration space nobody had walked.
+
+EVERY CONSTANT HERE IS SOURCED, and from a file on this machine:
+
+  MFP timer clock   16 MHz / 4        MAME 0.277 sharp/x68k.cpp:1027-1028
+  prescaler ladder  4,10,16,50,64,100,200   machine/mc68901.cpp:173
+  timer data reg    8 bits, 0 means 256     machine/mc68901.cpp (TCDR/TADR)
+  V-DISP -> GPIP4   x68k.cpp:1139, and it is also Timer A's event input,
+                    mc68901.cpp:167 GPIO_TIMER = {GPIP_4, GPIP_3}
+  line rate         31,500 Hz exactly in both 31.5 kHz modes, derived in
+                    tools/bench/crtc_mode.lua from the dot clocks
+  interrupt cost    MEASURED, not tabled: tools/bench/clock_run.sh
+
+THE PART THAT IS NOT A CLOCK PROBLEM AT ALL.  12 fps on a 55.4577 Hz raster is
+4.6215 refreshes per frame, so every frame is shown for 4 refreshes or 5 --
+72.13 ms or 90.16 ms -- and 37.9% of them get the short one.  That is the
+display's quantisation and no choice of clock changes it.  What it changes is
+the BUDGET: 833,333 clocks is the mean slot, not the slot, and the short slot is
+721,270.  With the per-frame decode costs in hand this file says exactly how
+many frames do not fit theirs, which is a thing this project has never had to
+ask because until now the tick came from a host that could not miss.
+"""
+import argparse
+import csv
+import os
+import sys
+
+MFP_HZ = 16_000_000 // 4                 # x68k.cpp:1027-1028
+PRESCALER = [4, 10, 16, 50, 64, 100, 200]   # mc68901.cpp:173
+HFREQ = 31500                            # lines/s, crtc_mode.lua
+CPUHZ = 10_000_000                       # 40 MHz / 4, x68k.cpp:1133
+
+
+def timer_space(fps):
+    """Every (prescale, data) the MFP can be set to, against a target fps."""
+    slowest = MFP_HZ / (PRESCALER[-1] * 256)
+    print(f"\n=== 1. THE MFP TIMER, WHICH CANNOT DO IT ALONE")
+    print(f"  timer clock {MFP_HZ:,} Hz, prescalers {PRESCALER}, data 1..256")
+    print(f"  slowest tick any single timer can produce: "
+          f"{MFP_HZ}/({PRESCALER[-1]}*256) = {slowest:.3f} Hz")
+    print(f"  a {fps} fps frame needs {fps} Hz, which is {slowest/fps:.1f}x "
+          f"slower than that -- so a software divider is REQUIRED whatever the "
+          f"source, and 'use an MFP timer' is not by itself an answer.")
+    exact = [(p, d) for p in PRESCALER for d in range(1, 257)
+             if (MFP_HZ * d * p) and (MFP_HZ % (p * d) == 0)
+             and (MFP_HZ // (p * d)) % fps == 0]
+    print(f"  settings whose tick rate is a whole multiple of {fps} Hz, so that "
+          f"a plain counter would be exact: {len(exact)}")
+    if exact:
+        best = min(exact, key=lambda pd: MFP_HZ / (pd[0] * pd[1]))
+        p, d = best
+        tick = MFP_HZ / (p * d)
+        print(f"    slowest of them: prescale /{p} data {d} = {tick:.4f} Hz, "
+              f"{tick/fps:.0f} ticks per frame")
+        print(f"    -> {tick/fps:.0f} interrupts per frame, against 4.6215 for "
+              f"the raster: {tick/fps/(HFREQ/(fps*568)):.1f}x the cost, and its "
+              f"phase against the raster is arbitrary, so a frame would be "
+              f"presented mid-scan.")
+    else:
+        print(f"    NONE.  {MFP_HZ}/{fps} = {MFP_HZ/fps:,.2f} is not an "
+              f"integer, so no prescale/data pair divides to {fps} Hz at all.")
+
+
+def raster_space(fps, vtotal):
+    hz = HFREQ / vtotal
+    print(f"\n=== 2. THE RASTER, WHICH IS THE RIGHT SOURCE AND IS ALSO NOT "
+          f"A WHOLE DIVIDE")
+    print(f"  V-DISP is {HFREQ}/{vtotal} = {hz:.4f} Hz, and it is BOTH the "
+          f"GPIP4 interrupt and Timer A's event-count input")
+    print(f"  whole divides -- all the MFP can do in hardware, no software:")
+    for n in (3, 4, 5, 6):
+        f = hz / n
+        print(f"    Timer A event count = {n}: {f:7.4f} fps  "
+              f"({100*(f/fps-1):+6.2f}% from {fps})")
+    print(f"  {fps} fps needs {hz/fps:.4f} refreshes per frame, which is not a "
+          f"whole number, so no event-count setting is exact either.")
+    print(f"\n  THE DIVIDER THAT IS EXACT: add fps*VTOTAL = {fps*vtotal} per "
+          f"V-DISP, emit a tick at {HFREQ}, keep the remainder.")
+    print(f"    long-run rate = {fps}*{vtotal}/{vtotal} = {fps} fps EXACTLY, "
+          f"with a remainder that never accumulates")
+    print(f"    the accumulator stays under {HFREQ + fps*vtotal:,}, so it is "
+          f"16-bit arithmetic on a 68000 (the ceiling is fps < "
+          f"{(65536-HFREQ)/vtotal:.1f} at this VTOTAL, and clk_init checks it)")
+
+
+def divider_gaps(fps, vtotal, n):
+    """The tick sequence src/player/clock.i emits, in refreshes per tick."""
+    acc, gaps, since = 0, [], 0
+    while len(gaps) < n:
+        acc += fps * vtotal
+        since += 1
+        if acc >= HFREQ:
+            acc -= HFREQ
+            gaps.append(since)
+            since = 0
+    return gaps
+
+
+def host_gaps(fps, refresh_hz, n):
+    """The tick sequence tools/bench/stream.lua's HOST clock emits.
+
+    It looks uniform in the source -- `floor((t - t_rel) * fps)` -- and is not.
+    Lua only sees the machine at frame boundaries, so tick k lands on the first
+    refresh at or after k/fps, and the gaps between ticks come out as the same
+    two whole numbers of refreshes the divider produces.  The host-paced runs of
+    FINDINGS 49 and 51 therefore already had this cadence in them; what ROADMAP
+    P3 changes is who produces it, not whether it exists.
+    """
+    at = [-(-int(k * refresh_hz * 1000000 // fps) // 1000000) for k in range(n + 1)]
+    return [at[k + 1] - at[k] for k in range(n)]
+
+
+def cadence(fps, vtotal, costs, label, refresh_hz, gaps, uniform=False):
+    """Charge each frame the slot it really gets, and run the pace gate."""
+    rpf = 1.0 if uniform else HFREQ / (fps * vtotal)
+    lo, hi = (1, 1) if uniform else (int(rpf), int(rpf) + 1)
+    slot_lo = lo / refresh_hz * CPUHZ
+    slot_hi = hi / refresh_hz * CPUHZ
+    nominal = CPUHZ / fps
+
+    n_lo = sum(1 for g in gaps[:len(costs)] if g == lo)
+    print(f"\n  --- {label}: refresh {refresh_hz:.4f} Hz")
+    print(f"      slots are {lo} refreshes = {slot_lo:,.0f} clk "
+          f"({1000*lo/refresh_hz:.2f} ms) or {hi} = {slot_hi:,.0f} clk "
+          f"({1000*hi/refresh_hz:.2f} ms)")
+    print(f"      the nominal {fps} fps budget every figure in this project is "
+          f"priced against is {nominal:,.0f} clk; the SHORT slot is "
+          f"{100*(slot_lo/nominal-1):+.1f}% of it")
+    over_lo = sum(1 for c in costs if c > slot_lo)
+    over_hi = sum(1 for c in costs if c > slot_hi)
+    over_nom = sum(1 for c in costs if c > nominal)
+    print(f"      frames that do not fit: {over_lo}/{len(costs)} the short "
+          f"slot, {over_hi}/{len(costs)} the long one, {over_nom}/{len(costs)} "
+          f"the nominal budget")
+
+    # The schedule, with the catch-up the pace gate actually performs: frame i
+    # starts at max(finish of i-1, tick i).  A frame that overruns does not fail
+    # -- it eats the next frame's idle, and the clock catches up by itself.
+    t, tick, late, worst, first = 0.0, 0.0, 0, 0.0, None
+    for i, c in enumerate(costs):
+        if t <= tick:
+            t = tick                      # idled: on time
+        else:
+            if i:                         # frame 0 has no predecessor
+                late += 1
+                if first is None:
+                    first = i
+            worst = max(worst, t - tick)
+        t += c
+        tick += gaps[i] / refresh_hz * CPUHZ
+    print(f"      through the pace gate: {late}/{len(costs)} frames found "
+          f"their slot already open (first at frame {first}), worst start "
+          f"{worst:,.0f} clk = {1000*worst/CPUHZ:.1f} ms behind its tick")
+    if not uniform:
+        print(f"      {n_lo}/{len(costs)} slots were the short one "
+              f"({100*n_lo/len(costs):.1f}%; the exact share is "
+              f"{100*(hi-rpf):.1f}%)")
+    return late
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--fps", type=int, default=12)
+    ap.add_argument("--vtotal", type=int, default=568,
+                    help="CRTC R04+1; 568 is the 31.5 kHz 256-line mode")
+    ap.add_argument("--csv", default="tmp/c68k_frames.csv",
+                    help="per-frame decode cost, from tools/bench/c68k/run.sh")
+    a = ap.parse_args()
+
+    timer_space(a.fps)
+    raster_space(a.fps, a.vtotal)
+
+    print(f"\n=== 3. WHAT THE CADENCE COSTS")
+    if not os.path.exists(a.csv):
+        print(f"  {a.csv} not found -- run tools/bench/c68k/run.sh first. The "
+              f"cadence question CANNOT be answered from percentiles: it needs "
+              f"the per-frame series, because what matters is whether an "
+              f"expensive frame lands in a short slot and how long the catch-up "
+              f"takes afterwards.")
+        return 1
+    costs = [float(r["cycles"]) for r in csv.DictReader(open(a.csv))]
+    print(f"  {len(costs)} frames from {a.csv}: mean {sum(costs)/len(costs):,.0f} "
+          f"clk, max {max(costs):,.0f} (frame {costs.index(max(costs))})")
+
+    hw = HFREQ / a.vtotal
+    n = len(costs)
+    # MAME's screen is fast by htotal/(htotal-8): refresh_mode() builds the
+    # frame period from scr.max_x*scr.max_y with scr.max_x = m_htotal - 8.  Both
+    # rasters are run, because the rig measures against the fast one and the
+    # player will run on the other -- reporting only one would leave the rig's
+    # count and this file's differing with nobody able to say which was wrong.
+    htotal = 368
+    mame = hw * htotal / (htotal - 8)
+
+    # The budget model every figure in FINDINGS assumes: a slot of exactly
+    # 1/fps.  No machine has this; it is the yardstick, run through the same
+    # schedule so that what the cadence ADDS can be read off.
+    cadence(a.fps, a.vtotal, costs, "the NOMINAL model (a slot of exactly "
+            "1/fps, which no raster produces)", float(a.fps),
+            [1] * (n + 1), uniform=True)
+    cadence(a.fps, a.vtotal, costs, "the HOST tick, as tools/bench/stream.lua "
+            "actually emits it", mame, host_gaps(a.fps, mame, n + 1))
+    cadence(a.fps, a.vtotal, costs, "the 68000's own clock on the HARDWARE "
+            "raster", hw, divider_gaps(a.fps, a.vtotal, n + 1))
+    cadence(a.fps, a.vtotal, costs, f"the 68000's own clock on MAME's raster "
+            f"(fast by {htotal}/{htotal-8})", mame,
+            divider_gaps(a.fps, a.vtotal, n + 1))
+
+    print(f"""
+Reading it.
+
+THE CADENCE WAS ALREADY THERE.  The nominal row is the model every budget in
+this project is priced against -- a slot of exactly 1/fps -- and no raster
+produces it.  The host row is what tools/bench/stream.lua has been emitting all
+along: `floor((t - t_rel) * fps)` looks uniform, but Lua only sees the machine at
+frame boundaries, so its ticks land on refreshes and its gaps are the same two
+whole numbers.  The host-paced results of FINDINGS 49 and 51 therefore already
+carried a 4/5 cadence that nothing named.  ROADMAP P3 did not introduce it; it
+moved who produces it onto the machine, where it belongs, and made it visible.
+
+THE SHORT SLOT IS REAL AND IT IS NOT A FAILURE.  {sum(1 for c in costs if c > 721270)}/{len(costs)} frames do not fit
+721,270 clocks.  The pace gate only says "not before tick i", so a frame that
+overruns spends the next frame's idle and the clock recovers by itself; the cost
+is one frame presented a refresh late, not a dropped frame.  What the counts
+above measure is frames with no idle left, and the difference between the
+nominal row and the raster rows -- 1 against 4 -- is the whole price of the
+cadence on this container.
+
+THE EXPENSIVE FRAME IS FRAME 0, at {max(costs)/(CPUHZ/a.fps)*100:.0f}% of the nominal budget: the first
+frame of a scene has nothing to SKIP against, so it is the whole picture in one
+slot.  Most of what follows it in these counts is that transient draining, which
+is why the first index is printed next to the total.  It also means the cost is
+paid AT A SCENE CHANGE, alongside the 18.96 ms of loader (FINDINGS 53.2) and the
+seek -- not spread over the window.
+
+SCOPE.  Decode costs are C68K's, on zero-wait-state memory, so they are a lower
+bound; real DRAM moves every row here in the same direction.  The MAME rows are
+the emulator's fast raster and exist to be compared with tools/bench/pace_run.sh,
+not to describe hardware.""")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

tools/analysis/24_ring_owner.py

@@ -0,0 +1,305 @@
+"""What the player's own fill loop costs the pipe (ROADMAP P5, FINDINGS 55).
+
+19_ring_stream.py and 20_seek_slack.py model a ring whose producer is free to
+act whenever it likes: bytes arrive at a rate and the only questions are where
+they go and whether the ring can hold them.  That is what a HOST-filled ring is,
+and it is what every delivery figure in FINDINGS 49 and 51 was measured on.
+
+A player has no host.  The 68000 owns the ring (src/player/ring.i), and it can
+only act when it is not decoding -- which turns the producer into a consumer of
+the same resource the decoder is short of, and puts an idle CHANNEL between
+every pair of records:
+
+    a transfer ends  ->  the disc has nothing to do  ->  the CPU next polls
+                     ->  it issues  ->  the disc starts again
+
+The gap in the middle is bytes the medium could have delivered and did not, and
+no rate table in this tree contains it.  Its size is set by the PLAYER: how many
+requests it may have outstanding (a DMAC channel takes one at a time; two slots
+mean the next is already queued when the current lands), and when it polls.
+
+This is that model, written from record sizes and per-frame decode costs, and
+sharing no code with the Lua rig it is compared against -- the same arrangement
+as 49.4 and 51.5.  The rig drives a real 68000 through a real ring and is the
+measurement; this says where to point it and what to expect.
+
+    python3 tools/analysis/24_ring_owner.py <container.dlx> --kbps R [R ...]
+            [--ring KB] [--qdepth N [N ...]] [--prefill RECORDS]
+            [--cadence raster|nominal]
+
+`--kbps` is REQUIRED and has no default, for the reason FINDINGS 50 gives.
+
+THE INDEX IS READ FROM THE CONTAINER, not derived by walking it.  A DLX4
+container carries nframes u16 record lengths in its scene header precisely
+because the producer needs the length of a record it has not fetched; this model
+reads the same table the 68000 does, so a container whose index disagreed with
+its stream would be caught here as well as in dlx.py's constructor.
+"""
+import sys, os, argparse
+sys.path.insert(0, "tools/encoder")
+import numpy as np
+from dlx import DLX
+import vq_hybrid as H
+import spans as SP
+
+CPUHZ = 10_000_000
+HFREQ = 31500          # lines/s in the 31.5 kHz modes (src/player/clock.i)
+VTOTAL = 568           # CRTC R04+1 in the 256x256 mode (tools/bench/crtc_mode.lua)
+CLK_ISR = 181.35       # clocks per V-DISP, MEASURED (FINDINGS 54.3)
+AUDIO_KBPS = 7.8       # ratectl.AUDIO_KBPS; the pipe carries it too
+
+
+def frame_costs(d):
+    """Per-frame decode cost in 68000 clocks: blocks plus v7 spans.
+
+    Both halves come from the encoder's own measured constants -- H.cycles is
+    the single source 11_cpu_budget.py uses, and SP.clocks is the v7 fit of
+    FINDINGS 40 -- so this is the same cost model the rate controller fits `mu`
+    against, applied to the emitted container rather than to a candidate.
+    """
+    out = []
+    for f in range(d.nframes):
+        c = H.cycles(d.modes(f))
+        for _, _, px in d.spans(f)[0]:
+            c += SP.clocks(len(px))
+        out.append(c)
+    return np.array(out)
+
+
+def ticks(n, fps):
+    """Frame tick times from src/player/clock.i's divider, or a nominal clock.
+
+    The player's clock is the raster with a remainder: a frame gets 4 refreshes
+    (72.13 ms) or 5 (90.16 ms) and there is no 83.33 ms frame (FINDINGS 54.4).
+    A model that hands out uniform slots gives every frame 13.4% more time than
+    the short one really has, so the cadence is reproduced here rather than
+    averaged away.
+    """
+    R = VTOTAL / HFREQ                     # one refresh, seconds
+    acc, out, t = 0, [0.0], 0.0
+    while len(out) < n:
+        t += R
+        acc += fps * VTOTAL
+        if acc >= HFREQ:
+            acc -= HFREQ
+            out.append(t)
+    return np.array(out)
+
+
+def simulate(rec, dec, tick, bps, ringsz, qdepth, prefill):
+    """One pass of the machine-owned ring.  Returns a dict of instruments.
+
+    The rules are src/player/ring.i's, stated as events:
+      * the CPU polls whenever it is NOT decoding -- the pace wait and the
+        record wait both call ring_poll, and nothing else in the frame does;
+      * a request occupies a slot until it is RETIRED, which happens at a poll,
+        so the queue is measured against retirement and not against completion;
+      * placement is `aligned`: a record that will not fit before the end of the
+        ring restarts at the base, and only if the base is free;
+      * the channel serves one transfer at a time, in order.
+    """
+    n = len(rec)
+    # ring state, in ring offsets
+    wcur = rcur = 0
+    rq = 0            # next record to request
+    retired = 0       # requests retired (== FR_HEAD)
+    consumed = 0      # records the decoder has finished (== FR_TAIL)
+    inflight = []     # [(record, done_time)] in issue order
+    chan_free = 0.0   # when the channel finishes what it has
+    gaps, gap_tot, gap_max = 0, 0.0, 0.0
+    busy = 0.0
+    full_refusals = 0
+    started = False   # the first transfer has no gap before it
+
+    def live_empty():
+        return rq == consumed
+
+    def place(length):
+        """Where the next record goes: (offset, hole) or None if it cannot."""
+        nonlocal full_refusals
+        if live_empty():
+            if wcur + length <= ringsz:
+                return wcur, 0
+            return 0, ringsz - wcur
+        if rcur == wcur:
+            return None                    # completely full
+        if rcur < wcur:                    # free is [wcur, SZ) then [0, rcur)
+            if wcur + length <= ringsz:
+                return wcur, 0
+            if length <= rcur:
+                return 0, ringsz - wcur
+            return None
+        if wcur + length <= rcur:          # live wraps; free is [wcur, rcur)
+            return wcur, 0
+        return None
+
+    def issue(now):
+        """Issue as many requests as the queue and the ring allow, at `now`."""
+        nonlocal wcur, rcur, rq, chan_free, gaps, gap_tot, gap_max, busy
+        nonlocal full_refusals, started
+        while rq < n and (rq - retired) < qdepth:
+            p = place(rec[rq])
+            if p is None:
+                full_refusals += 1
+                return
+            off, _hole = p
+            if live_empty():
+                rcur = off
+            start = max(now, chan_free)
+            if started:
+                g = start - chan_free
+                if g > 1e-12:
+                    gaps += 1
+                    gap_tot += g
+                    gap_max = max(gap_max, g)
+            started = True
+            dur = rec[rq] / bps
+            busy += dur
+            chan_free = start + dur
+            inflight.append((rq, chan_free))
+            wcur = off + rec[rq]
+            rq += 1
+
+    def retire(now):
+        """Publish every transfer that has landed by `now`.  In order."""
+        nonlocal retired
+        while inflight and inflight[0][1] <= now:
+            inflight.pop(0)
+            retired += 1
+
+    def advance_reader():
+        """Step rcur over the records the decoder has finished with."""
+        nonlocal rcur
+        i = consumed_seen[0]
+        while i < consumed:
+            end = rcur + rec[i]
+            if i + 1 < n and end + rec[i + 1] > ringsz:
+                end = 0
+            rcur = end
+            i += 1
+        consumed_seen[0] = i
+
+    consumed_seen = [0]
+
+    # ---- prefill.  The decoder is not running, so the CPU polls continuously
+    # and the channel never waits for it: this is the one part of a scene where
+    # the request loop costs nothing.
+    now = 0.0
+    while retired < prefill and rq < n:
+        issue(now)
+        if not inflight:
+            break
+        now = inflight[0][1]
+        retire(now)
+    prefill_done = now
+    t0 = now
+
+    underruns, worst_late, noidle = 0, 0.0, 0
+    slack_series = []
+    for i in range(n):
+        deadline = t0 + tick[i]
+        # TWO WAYS A FRAME CAN START LATE, AND THEY ARE NOT THE SAME FAILURE.
+        # The decoder reaches the record wait at max(its own finish, the tick):
+        # if it got there after the tick, the PREVIOUS frame used its whole slot
+        # and this is the CPU (54.4's cadence).  If it got there on time and the
+        # record was not resident, that is the PIPE.  The rig counts them
+        # separately -- NO IDLE and UNDERRUNS -- so conflating them here would
+        # have made the model disagree with it for a reason that is not about
+        # delivery at all.
+        if now > deadline + 1e-9:
+            noidle += 1
+        arrive = max(now, deadline)
+        now = arrive
+        # the record wait: the CPU polls, so it retires and issues while it waits
+        starved = retired <= i
+        while retired <= i:
+            issue(now)
+            if not inflight:
+                break
+            now = max(now, inflight[0][1])
+            retire(now)
+        if starved:
+            underruns += 1
+            worst_late = max(worst_late, now - arrive)
+        slack_series.append(retired - consumed)
+        issue(now)
+        # ---- decode.  No polls: whatever the channel finishes now waits.
+        now += dec[i] / CPUHZ
+        consumed += 1
+        advance_reader()
+        retire(now)
+        issue(now)
+        # ---- idle until the next tick.  The CPU polls throughout, so every
+        # completion is retired and every free slot is refilled at once.
+        nxt = t0 + tick[i + 1] if i + 1 < n else now
+        while inflight and inflight[0][1] < nxt:
+            now = max(now, inflight[0][1])
+            retire(now)
+            issue(now)
+        now = max(now, min(nxt, now))
+    span = max(now - t0, 1e-9)
+    return dict(underruns=underruns, worst_late=worst_late, gaps=gaps,
+                noidle=noidle,
+                gap_tot=gap_tot, gap_max=gap_max, busy=busy, span=span,
+                ceiling=max(slack_series), mean_slack=float(np.mean(slack_series)),
+                full=full_refusals, prefill_s=prefill_done)
+
+
+ap = argparse.ArgumentParser()
+ap.add_argument("container")
+ap.add_argument("--kbps", type=float, nargs="+", required=True,
+                help="delivery rates to model. REQUIRED: this tree has no "
+                     "default rate (FINDINGS 50)")
+ap.add_argument("--ring", type=int, default=256, help="ring size, KB")
+ap.add_argument("--qdepth", type=int, nargs="+", default=[1, 2],
+                help="requests the player may have outstanding")
+ap.add_argument("--prefill", type=int, default=2, help="records before release")
+ap.add_argument("--cadence", choices=["raster", "nominal"], default="raster")
+a = ap.parse_args()
+
+d = DLX(a.container)
+if not d.has_index:
+    sys.exit(f"{a.container} is DLX{d.version}: this model reads the record "
+             f"index the player reads, and only DLX4 carries one.")
+rec = np.array([q * 4 for q in d.index], np.int64)
+dec = frame_costs(d) + CLK_ISR * (HFREQ / VTOTAL) / d.fps   # + the clock's own
+if a.cadence == "raster":
+    tick = ticks(d.nframes + 1, d.fps)
+else:
+    tick = np.arange(d.nframes + 1) / d.fps
+
+wire = rec.mean() * d.fps / 1024
+print(f"{a.container}: {d.nframes} records, {rec.mean()/1024:.1f} KB mean, "
+      f"{rec.max()/1024:.1f} KB max, wire {wire:.1f} KB/s")
+print(f"  index: {2*d.nframes:,} B of scene header -- read, not walked")
+print(f"  decode: mean {dec.mean():,.0f} clk/frame ({100*dec.mean()/(CPUHZ/d.fps):.1f}% "
+      f"of a mean slot), p90 {np.percentile(dec,90):,.0f}")
+print(f"  cadence: {a.cadence}"
+      + ("  (4 or 5 refreshes a frame, 72.13/90.16 ms -- FINDINGS 54.4)"
+         if a.cadence == "raster" else "  (uniform 1/fps slots)"))
+print(f"  ring {a.ring} KB, prefill {a.prefill} records\n")
+
+hdr = (f"{'pipe':>8} {'Q':>2} {'idle':>9} {'gaps':>5} {'worst':>8} "
+       f"{'under':>7} {'late by':>8} {'noidl':>5} {'ceil':>5} {'mean':>5} "
+       f"{'refus':>6}")
+print(hdr)
+print("-" * len(hdr))
+for kb in a.kbps:
+    bps = (kb - AUDIO_KBPS) * 1024
+    for q in a.qdepth:
+        r = simulate(rec, dec, tick, bps, a.ring * 1024, q, a.prefill)
+        print(f"{kb:8.0f} {q:2d} {100*r['gap_tot']/r['span']:8.1f}% "
+              f"{r['gaps']:5d} {r['gap_max']*1000:7.1f}ms "
+              f"{r['underruns']:3d}/{d.nframes:<3d} {r['worst_late']*1000:7.1f}ms "
+              f"{r['noidle']:5d} {r['ceiling']:5d} {r['mean_slack']:5.1f} "
+              f"{r['full']:6d}")
+print()
+print("idle   = the channel with no request to work on, as a fraction of the")
+print("         window. Bytes the medium could have delivered and did not.")
+print("under  = frames whose record was not resident when the decoder asked")
+print("         for it. The PIPE.")
+print("noidl  = frames that reached the gate after their tick, because the one")
+print("         before used its whole slot. The CPU, and 54.4's cadence.")
+print("ceil   = most records resident and unconsumed at a frame start: what a")
+print("         branch point could spend, minus one for the restart (51.2).")
+print("refus  = placements refused for SPACE. Nonzero means the ring filled.")

tools/analysis/25_scene_graph.py

@@ -0,0 +1,363 @@
+"""The worst gap between two decision points, out of the scene graph (G1).
+
+FINDINGS 51.3 measured that a ring's lookahead is ACCUMULATED out of
+`pipe - wire` and that a seek spends all of it, so what a branch point costs is
+set by the rate and by the time since the last branch.  55.5 rehearsed a seek on
+the machine and could not ask the question that matters, because nothing in this
+tree knew where the branch points ARE:
+
+    what is the WORST gap, in seconds of play, between two consecutive
+    decision points, and does the refill climb survive it?
+
+Only the arcade scene graph knows.  This answers it in the currency 51.3
+established.
+
+  python3 tools/import/scenegraph.py                 # writes tmp/scenegraph.json
+  python3 tools/analysis/25_scene_graph.py --kbps R [R ...] [--ring KB [KB ...]]
+
+`--kbps` is REQUIRED and takes no default, for the reason FINDINGS 50 gives.
+
+THIS FILE KNOWS NOTHING ABOUT WHERE THE TABLE CAME FROM, deliberately.  It reads
+`DLXSCENE1`, which is this project's own schema; `tools/import/scenegraph.py` is
+the single file in the tree that knows anything about the outside projects the
+table is built from, and it carries their attribution.  Nothing is vendored.
+"""
+import sys, os, json, argparse, importlib.util
+
+sys.path.insert(0, "tools/encoder")
+
+TABLE = os.environ.get("DLX_SCENEGRAPH", "tmp/scenegraph.json")
+LD_FPS = 23.976     # the medium's frame rate; one frame is the comparison floor
+
+
+# ------------------------------------------------------------------- graph
+
+class Node:
+    """One (scene, sequence): a clip, its exits, and whether entering it seeks."""
+
+    def __init__(self, scene, name, seq):
+        self.scene, self.name, self.seq = scene, name, seq
+        self.start = seq["start_ms"]                 # ms, or -1 for no seek
+        self.seeks = self.start >= 0
+        self.timeout_ms = seq["timeout_ms"]
+        self.exits = [(seq["timeout_ms"], "timeout", seq["timeout_next"])]
+        for a in seq["actions"]:
+            # The player may press as early as `from_ms`, so that is the least
+            # play this clip can deliver before the branch it leads to.
+            self.exits.append((a["from_ms"], "action", a["next"]))
+
+    @property
+    def key(self):
+        return f"{self.scene}.{self.name}"
+
+
+def build_graph(scenes):
+    nodes = {}
+    for scene, seqs in scenes.items():
+        for name, seq in seqs.items():
+            n = Node(scene, name, seq)
+            nodes[n.key] = n
+    return nodes
+
+
+def worst_gap(nodes):
+    """Least play time, in ms, between one seek and the next.
+
+    A seek is entering a sequence whose start is >= 0; a negative start means
+    the disc keeps playing, so play ACCUMULATES across such sequences and the
+    gap is a shortest path over them.  Bellman-Ford rather than Dijkstra
+    because a zero-length timeout is common (`start_alive` chains) and the
+    graph has cycles; all weights are non-negative so it terminates.
+
+    A null exit ends the scene: the player moves to another scene entirely,
+    which is a seek AND a container change (FINDINGS 53), so it counts as a
+    seek and is flagged.
+    """
+    INF = float("inf")
+    dist = {k: (0.0 if n.seeks else INF) for k, n in nodes.items()}
+    for _ in range(len(nodes) + 1):
+        changed = False
+        for n in nodes.values():
+            if dist[n.key] == INF:
+                continue
+            for elapsed, kind, tgt in n.exits:
+                if tgt is None:
+                    continue
+                tk = f"{n.scene}.{tgt}"
+                if tk not in nodes:
+                    continue
+                d = dist[n.key] + max(0.0, float(elapsed))
+                if not nodes[tk].seeks and d < dist[tk] - 1e-9:
+                    dist[tk], changed = d, True
+        if not changed:
+            break
+
+    best = {}
+    for n in nodes.values():
+        if dist[n.key] == INF:
+            continue
+        for elapsed, kind, tgt in n.exits:
+            tk = f"{n.scene}.{tgt}" if tgt is not None else None
+            ends_scene = tgt is None
+            if ends_scene or (tk in nodes and nodes[tk].seeks):
+                g = dist[n.key] + max(0.0, float(elapsed))
+                # One entry per (source, destination): several input windows can
+                # lead to the same clip and only the earliest of them binds.
+                k = (n.key, tgt)
+                if k not in best or g < best[k][0]:
+                    best[k] = (g, n.key, tgt if tgt else "<end of scene>",
+                               kind, ends_scene, n.scene)
+    return sorted(best.values()), dist
+
+
+# ------------------------------------------------------ 51.3's currency
+
+def slack_model(gap_s, kbps, wire_kbps, mean_rec_b):
+    """Records of lookahead accrued in `gap_s` seconds of play at `kbps`, and
+    the seconds one record of lookahead costs.
+
+    This is 51.3's surplus model and nothing more: slack accrues at
+    (pipe - wire) bytes per second.  The paced rig is the measurement; this
+    says whether the gap is even in the right order of magnitude.
+    """
+    surplus = (kbps - wire_kbps) * 1024.0
+    if surplus <= 0:
+        return 0.0, float("inf")
+    return surplus * gap_s / mean_rec_b, mean_rec_b / surplus
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--kbps", type=float, nargs="+", required=True,
+                    help="delivered pipe rates, KB/s. REQUIRED, no default "
+                         "(FINDINGS 50).")
+    ap.add_argument("--table", default=TABLE, help="DLXSCENE1 scene table")
+    ap.add_argument("--container", default="tmp/rc_fr_singe_scsi_span.dlx",
+                    help="container the wire demand and mean record come from")
+    ap.add_argument("--ring", type=float, nargs="+", default=[256, 512])
+    ap.add_argument("--top", type=int, default=12)
+    a = ap.parse_args()
+
+    if not os.path.exists(a.table):
+        print(f"no scene table at {a.table} -- run:\n"
+              f"  python3 tools/import/scenegraph.py")
+        return 2
+    doc = json.load(open(a.table))
+    if doc.get("format") != "DLXSCENE1":
+        print(f"{a.table}: not a DLXSCENE1 table")
+        return 2
+
+    nodes = build_graph(doc["scenes"])
+    c = doc["counts"]
+    nseek = sum(1 for n in nodes.values() if n.seeks)
+
+    print(f"=== the scene graph ({a.table})")
+    for s in doc["sources"]:
+        print(f"  from {s['name']} ({s['licence']}, {s['holder']}): {s['role']}")
+    print(f"  scenes {c['scenes']}, sequences {c['sequences']}, "
+          f"input windows {c['windows']}")
+    print(f"  sequences entered by a SEEK: {nseek}/{c['sequences']} "
+          f"({100*nseek/c['sequences']:.1f}%); the rest play on from where the "
+          f"disc is")
+    print(f"  scene order: {len(doc['rows'])} rows x {len(doc['rows'][0])}")
+    # Gates.  A parser that quietly dropped a branch would produce a SMALLER
+    # graph and a LONGER worst gap -- it would fail in the flattering direction.
+    assert c["sequences"] == 516, f"expected 516 sequences, got {c['sequences']}"
+    assert c["windows"] == 906, f"expected 906 input windows, got {c['windows']}"
+
+    # ---- the measurement
+    gaps, dist = worst_gap(nodes)
+    play = [g for g in gaps if g[5] != "attract_mode"]
+    zero = [g for g in play if g[0] <= 1e-9]
+    print()
+    print("=== the worst gap between two consecutive decision points")
+    print("  A 'gap' is the LEAST play time the disc delivers between one seek")
+    print("  and the next: the earliest an input window opens, chained across")
+    print("  sequences the disc plays through without seeking.  One entry per")
+    print("  (source, destination); attract mode is excluded and reported")
+    print("  separately, because nothing branches there under a 12 fps budget.")
+    print(f"  {'gap s':>7}  from -> to")
+    for g, src, tgt, kind, ends, scene in play[:a.top]:
+        print(f"  {g/1000:>7.3f}  {src} -{kind}-> {tgt}"
+              + ("  [SCENE CHANGE]" if ends else ""))
+    sc_gaps = [g for g in play if g[4]]
+    print(f"  ... {len(play)} distinct transitions into a seek "
+          f"({len(gaps)-len(play)} more in attract mode)")
+    worst = play[0][0] / 1000.0
+    med = play[len(play) // 2][0] / 1000.0
+    print(f"  WORST {worst:.3f} s, median {med:.3f} s, "
+          f"best {play[-1][0]/1000:.3f} s")
+    print(f"  SCENE CHANGES specifically ({len(sc_gaps)} of them, and each also")
+    print(f"  needs a header before its frame 0, FINDINGS 53/55.1): worst "
+          f"{sc_gaps[0][0]/1000:.3f} s, median "
+          f"{sc_gaps[len(sc_gaps)//2][0]/1000:.3f} s")
+    print(f"  ZERO-PLAY BRANCHES: {len(zero)} of {len(play)} "
+          f"({100*len(zero)/len(play):.1f}%) open an input window at t=0 of a")
+    print("  clip the disc SEEKED to, so two seeks can fall back to back with no")
+    print("  play between them at all.  A rule of the form 'has there been")
+    print("  enough play since the last branch' (51.2's ring_may_seek) can be")
+    print("  answered NO by the content, not by the buffer.")
+
+    # ---- what the input layer has to survive, from the same table
+    inputs, windows = {}, []
+    for n in nodes.values():
+        for x in n.seq["actions"]:
+            inputs[x["input"]] = inputs.get(x["input"], 0) + 1
+            windows.append(x["to_ms"] - x["from_ms"])
+    windows.sort()
+    print()
+    print("=== what the input layer has to survive")
+    print("  " + ", ".join(f"{k} {v}" for k, v in
+                           sorted(inputs.items(), key=lambda x: -x[1])))
+    diag = sum(v for k, v in inputs.items()
+               if k in ("upleft", "upright", "downleft", "downright"))
+    print(f"  diagonals are {diag} windows of {len(windows)}: rare enough for a "
+          f"port to drop\n  and not droppable by one aiming at the arcade")
+    print(f"  window length: shortest {windows[0]:.0f} ms "
+          f"({windows[0]/(1000/12):.2f} frame slots at 12 fps), p10 "
+          f"{windows[len(windows)//10]:.0f} ms, median "
+          f"{windows[len(windows)//2]:.0f} ms")
+    print("  the floor is one to two frames wide (54.4: a slot is 72.13 or")
+    print("  90.16 ms, never 83.33), so input cannot be polled on the frame tick")
+
+    # ---- 51.3's currency
+    if os.path.exists(a.container):
+        spec = importlib.util.spec_from_file_location(
+            "seek_slack", "tools/analysis/20_seek_slack.py")
+        m = importlib.util.module_from_spec(spec)
+        spec.loader.exec_module(m)
+        import ratectl as RC
+        d, rec = m.records(a.container)
+        mean_rec = float(rec.mean())
+        wire = mean_rec * 12 / 1024 + RC.AUDIO_KBPS
+        gs = [g[0] / 1000.0 for g in play]
+        print()
+        print(f"=== what that gap buys, at explicit rates "
+              f"({os.path.basename(a.container)}: mean record "
+              f"{mean_rec/1024:.1f} KB, wire {wire:.1f} KB/s)")
+        print(f"  {'ring KB':>8} {'pipe':>7} {'ceiling':>8} {'climb s':>8} "
+              f"{'median gap':>11} {'accrued':>8} {'under climb':>12}")
+        for ring_kb in a.ring:
+            ring = int(ring_kb * 1024)
+            for kbps in a.kbps:
+                fill = (kbps - RC.AUDIO_KBPS) * 1024 / 12
+                lo, hi, ring_ref, rate_ref = m.paced_sim(rec, ring, fill)
+                ceiling = int(hi.max())
+                accrued, per_rec = slack_model(med, kbps, wire, mean_rec)
+                climb = ceiling * per_rec
+                under = sum(1 for g in gs if g < climb)
+                print(f"  {ring_kb:>8.0f} {kbps:>7.1f} {ceiling:>8} "
+                      f"{climb:>8.2f} {med:>11.3f} {accrued:>8.2f} "
+                      f"{f'{under}/{len(gs)}':>12}  {100*under/len(gs):.0f}%")
+
+        # What a branch costs when the gap before it bought nothing.  The ring
+        # is empty after a seek and the decoder is released at the prefill depth
+        # (55.4's shipped policy is 2 records), so this is the stall the player
+        # eats every time -- not the climb to the ceiling, which is what it
+        # needs in order to TOLERATE the next one.  A scene change additionally
+        # needs its header before frame 0; 22_scene_load.py prices that case
+        # properly, clocks included.
+        PREFILL_REC, HDR_B = 2, 6164
+        print()
+        print(f"  A branch taken on an empty ring, at the shipped prefill of "
+              f"{PREFILL_REC} records:")
+        for kbps in a.kbps:
+            b = PREFILL_REC * mean_rec
+            ms = b / (kbps * 1024) * 1000
+            hms = (b + HDR_B) / (kbps * 1024) * 1000
+            print(f"    {kbps:>7.1f} KB/s: {ms:>7.1f} ms "
+                  f"({ms/(1000/12):.2f} frame slots), and {hms:>7.1f} ms "
+                  f"({hms/(1000/12):.2f}) if it is a scene change carrying "
+                  f"{HDR_B:,} header bytes")
+        print()
+        print("  'climb s' is 51.3's: seconds of play to refill from empty to")
+        print("  the ceiling.  'under climb' is how many of this game's own")
+        print("  branch points arrive sooner than that, i.e. are reached with")
+        print("  LESS lookahead than the one before them.  The worst gap is")
+        print(f"  {worst:.3f} s and buys nothing at any rate in this table.")
+    else:
+        print(f"\n{a.container}: MISSING -- rate half skipped")
+
+    # ---- the cross-check, and what it is worth
+    print()
+    print("=== the second table, and why it is not a second transcription")
+    cross, meta = doc.get("crosscheck"), doc.get("crosscheck_meta")
+    if not cross:
+        print("  none in this table -- the import ran without it")
+        return 0
+    print(f"  {meta['chapters']} chapters, {meta['scenes_mapped']} scenes")
+    print(f"  PROVENANCE: {meta['provenance']}")
+    print("  FINDINGS 16 planned to diff two INDEPENDENT transcriptions to")
+    print("  catch transcription errors.  There is only one transcription.")
+    print("  This diff catches CONVERSION errors and nothing more.")
+
+    FRAME_MS = 1000.0 / LD_FPS
+    offs, durs, same, diff, missing = [], [], 0, 0, 0
+    renames, inputs_differ, examples = 0, [], []
+    for scene, seqs in sorted(cross.items()):
+        for seq, ch in sorted(seqs.items()):
+            n = nodes.get(f"{scene}.{seq}")
+            if n is None:
+                missing += 1
+                continue
+            if n.seeks:
+                offs.append(ch["start_ms"] - n.start)
+            durs.append(((ch["end_ms"] - ch["start_ms"]) - n.timeout_ms,
+                         ch["chapter"]))
+            ce = sorted((x["input"], x["next"]) for x in ch["actions"])
+            de = sorted((x["input"], x["next"]) for x in n.seq["actions"])
+            if ce == de:
+                same += 1
+                continue
+            diff += 1
+            ci = sorted(i for i, _ in ce)
+            di = sorted(i for i, _ in de)
+            if ci != di:
+                inputs_differ.append((ch["chapter"], ci, di))
+            else:
+                renames += 1
+            if len(examples) < 3:
+                examples.append((ch["chapter"], ce, de))
+
+    if offs:
+        offs.sort()
+        print(f"  START TIMES are on different timelines and do not compare: "
+              f"{len(offs)} seeking chapters,")
+        print(f"    offset spread {min(offs)/1000:,.1f} s .. "
+              f"{max(offs)/1000:,.1f} s, median {offs[len(offs)//2]/1000:,.1f} s"
+              f" -- not a constant, and not even one sign.")
+    if durs:
+        dd = sorted(x for x, _ in durs)
+        agree = sum(1 for x in dd if abs(x) <= FRAME_MS)
+        print(f"  DURATIONS compare (offset-invariant): {agree}/{len(dd)} "
+              f"within one frame of the medium ({100*agree/len(dd):.1f}%), "
+              f"median {dd[len(dd)//2]:,.0f} ms")
+        for x, ch in sorted(durs, key=lambda x: -abs(x[0]))[:3]:
+            print(f"    widest {ch}: {x/1000:+.3f} s")
+    if same + diff:
+        print(f"  BRANCH STRUCTURE compares: {same}/{same+diff} chapters have "
+              f"the identical set of (input -> target) edges "
+              f"({100*same/(same+diff):.1f}%)")
+    if diff:
+        print(f"    {renames} of the {diff} differ only in what a target "
+              f"sequence is NAMED")
+        DIAG = ("upleft", "upright", "downleft", "downright")
+        lost = [ch for ch, ci, di in inputs_differ
+                if set(di) - set(ci) and all(x in DIAG for x in set(di) - set(ci))]
+        print(f"    {len(inputs_differ)} differ in the INPUT SET, and "
+              f"{len(lost)} of those are the other table dropping the arcade's")
+        print("      DIAGONALS: a controller decision, not a transcription "
+              "difference.")
+        for ch, ci, di in inputs_differ:
+            if ch not in lost:
+                print(f"      the remaining one: {ch}, {ci} vs {di}")
+        for ch, ce, de in examples:
+            print(f"    e.g. {ch}\n      cross {ce}\n      graph {de}")
+    if missing:
+        print(f"  {missing} chapters have no sequence in the graph "
+              f"(the other project added them)")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

tools/analysis/26_sector_align.py

@@ -0,0 +1,150 @@
+"""A record is not a sector: what the mismatch costs, three ways (P4b, 58.3).
+
+src/player/ring.i asks the transport for a RECORD -- a byte offset into the
+scene's frame stream and a length, both 4-byte aligned because that is what
+`move.l (a0)+` needs (28.3) and neither of them a multiple of 512.  A SCSI
+target answers in 512 B BLOCKS.  On the gate container 117 of 120 records start
+part way into a sector, so something has to reconcile the two, and the three
+ways of doing it are not close.
+
+WHY IT IS NOT AN IMPLEMENTATION DETAIL.  The bytes on either side of a record in
+the stream belong to OTHER records -- ones the decoder may still be reading --
+and the block loop walks a0 with no bounds check at all (49.2).  So a transport
+that reads whole sectors straight into the ring does not waste 500 bytes, it
+CORRUPTS the neighbours, and the symptom is wrong pixels rather than a fault.
+
+  A. WINDOWED PIO.  Read the sectors the record lies in, store only the record.
+     src/player/scsi.i does this and it is what FINDINGS 58 measured.  It costs
+     nothing in clocks -- the CPU is touching every byte anyway -- and it costs
+     the extra sectors on the wire.  It CANNOT be done by a DMAC: a channel
+     writes a contiguous run to a contiguous address and cannot be told to drop
+     the first 300 bytes.
+  B. BOUNCE BUFFER.  Let the DMAC write whole sectors somewhere else, then copy
+     the record into the ring.  Works under DMA, and costs a copy of every
+     delivered byte -- which is precisely the cost `aligned` was chosen over
+     `split` to avoid (49.3, 19_ring_stream.py).
+  C. SECTOR-ALIGNED RECORDS.  Pad each record up to 512 in the container
+     instead of up to 4.  Costs bytes on the disc and in every delivery, and
+     nothing else at all; the transport becomes a whole-sector read into the
+     ring with no window and no copy.  It is a CONTAINER change -- a re-encode
+     and a re-measurement of every constant fitted to the gate container, which
+     is the class of change ROADMAP already has bundled with P2's other half.
+
+    python3 tools/analysis/26_sector_align.py <in.dlx> [--ring KB]
+
+No rate is taken and none is needed: every figure here is a fraction of the
+delivered bytes or a count of clocks, and both are rate-free.  What a given
+delivery rate does with them is 15_bus_occupancy.py's question.
+"""
+import sys, os, argparse
+sys.path.insert(0, "tools/encoder")
+from dlx import DLX
+
+SECTOR = 512
+CPUHZ = 10_000_000
+# 5.0 clocks/byte, and it is 19_ring_stream.py's constant rather than a new one:
+# a 68000 `move.l (a0)+,(a1)+` moves 4 bytes in 20 clocks on a 16-bit bus.  It
+# is the OPTIMISTIC figure there and it is the optimistic figure here.
+COPY_CLK_PER_BYTE = 5.0
+# The windowed PIO loop in src/player/scsi.i, from the 68000's cycle table:
+#   12  move.l #SC_PATIENCE,d3      patience reload
+#   16  move.b SC_SSTS,d0           (xxx).L -> Dn
+#   10  btst #0,d0
+#   10  beq.s taken
+#   20  move.b SC_DREG,(a1)+        (xxx).L -> (An)+
+#    8  subq.l #1,d7
+#   10  bne.s taken
+# FINDINGS 58.2 measured 87.28 clocks per delivered byte against this loop's 86
+# plus 1.15 for the dropped window bytes -- 0.2% apart, which is what says the
+# cost is the instruction stream and not MAME's device model.
+PIO_CLK_PER_BYTE = 86.0
+
+ap = argparse.ArgumentParser()
+ap.add_argument("container")
+ap.add_argument("--ring", type=int, default=256, help="ring size in KB")
+a = ap.parse_args()
+
+d = DLX(a.container)
+
+# The disc layout the 68000 walks: [u32 len][body], each record padded up to the
+# container's own alignment -- 4 on DLX2/3/4, 512 on DLX5.  Exactly
+# tools/bench/prep_stream.py's, and it comes from the reader rather than from a
+# second copy of the rule here, so pointing this tool at a DLX5 container asks
+# it the RIGHT question: what does the mismatch still cost once the container
+# has been changed to remove it?  (The answer had better be nothing.)
+off, recs = 0, []
+for ln in d.record_lengths():
+    recs.append((off, ln))
+    off += ln
+# The DENOMINATOR is the record bytes the decoder actually reads -- [u32 len]
+# plus payload -- and NOT the padded length, because on a DLX5 container the
+# padding IS the cost being measured.  Scoring against the padded length would
+# make an already-aligned container report +0.00% and look free.
+payload = sum(4 + n for _, n in d.frames)
+nfr = len(recs)
+budget = CPUHZ / d.fps
+
+print(f"{a.container}: {nfr} records, {payload:,} B, {d.fps} fps")
+print(f"  mean record {payload/nfr:,.0f} B; a {d.fps} fps frame is "
+      f"{budget:,.0f} clocks")
+aligned0 = sum(1 for o, _ in recs if o % SECTOR == 0)
+print(f"  records that already start on a sector boundary: {aligned0}/{nfr}")
+print()
+
+# ---- A. windowed PIO: the sectors the record lies in, and only the record kept
+wire_a = sum(((o % SECTOR) + ln + SECTOR - 1) // SECTOR for o, ln in recs) * SECTOR
+drop_a = wire_a - payload
+print("A. WINDOWED PIO (src/player/scsi.i, what FINDINGS 58 ran)")
+print(f"     wire {wire_a:,} B for {payload:,} B of record "
+      f"= +{100*drop_a/payload:.2f}%")
+print(f"     clocks {PIO_CLK_PER_BYTE:.0f}/B on EVERY byte off the FIFO, "
+      f"dropped ones included:")
+print(f"       {PIO_CLK_PER_BYTE*wire_a/nfr:,.0f} clk/frame "
+      f"= {100*PIO_CLK_PER_BYTE*wire_a/nfr/budget:.0f}% of the frame")
+print( "     and it does not survive the move to the DMAC at all: a channel "
+       "cannot drop bytes.")
+print()
+
+# ---- B. bounce buffer: DMA whole sectors elsewhere, copy the record in
+print("B. BOUNCE BUFFER (whole sectors by DMA, then a copy)")
+print(f"     wire {wire_a:,} B, the same +{100*drop_a/payload:.2f}% -- the "
+      f"command is identical")
+print(f"     clocks {COPY_CLK_PER_BYTE:g}/B of copy on every DELIVERED byte, "
+      f"on top of whatever W the")
+print(f"       channel steals: {COPY_CLK_PER_BYTE*payload/nfr:,.0f} clk/frame "
+      f"= {100*COPY_CLK_PER_BYTE*payload/nfr/budget:.1f}% of the frame")
+print( "     which is the cost `aligned` was chosen over `split` to avoid "
+       "(49.3), arriving")
+print( "     by a different door and on every byte instead of on a wrap.")
+print()
+
+# ---- C. sector-aligned records in the container
+cur, pad = 0, 0
+for _, ln in recs:
+    if cur % SECTOR:
+        pad += SECTOR - (cur % SECTOR)
+        cur += SECTOR - (cur % SECTOR)
+    cur += ln
+print("C. SECTOR-ALIGNED RECORDS (a container change; a re-encode)"
+      + ("  -- THIS CONTAINER ALREADY IS ONE" if d.sector_aligned else ""))
+print(f"     wire {cur:,} B for {payload:,} B of record = "
+      f"+{100*(cur-payload)/payload:.2f}%")
+print( "     clocks ZERO: the read is a whole-sector read straight into the "
+       "ring, no window,")
+print( "     no copy, and the DMAC can do it.")
+print()
+
+ringsz = a.ring * 1024
+print(f"  VERDICT, in the currency this project prices delivery in.  C is "
+      f"cheaper on the wire")
+print(f"  than A and B by {100*(wire_a-cur)/payload:.2f} points of the payload "
+      f"({wire_a-cur:,} B on this scene),")
+print(f"  and it is the only one of the three a DMA channel can run without a "
+      f"copy. What it")
+print(f"  costs is a container revision and the re-measurement that comes with "
+      f"one.")
+maxrec = max(ln for _, ln in recs)
+maxpad = maxrec + (-maxrec) % SECTOR
+print(f"  It also grows the largest record from {maxrec:,} to {maxpad:,} B, "
+      f"which a {a.ring} KB")
+print(f"  ring still holds {ringsz//maxpad} times over.")

tools/analysis/27_dmac_config.py

@@ -0,0 +1,111 @@
+#!/usr/bin/env python3
+"""What the PLAYER programs into the DMAC -- read out of the assembler source.
+
+    python3 tools/analysis/27_dmac_config.py [src/player/dma.i]
+
+ROADMAP P4a asks for a DMAC configuration that HOLDS THE BUS, and the whole
+weight of the claim is in four register bytes.  tools/analysis/21_iplrom_dmac.py
+already reads the IPL ROM's four channels the same way, out of the shipping
+image, and found Sharp's own disk channel at 16..19 clocks a byte (FINDINGS
+52.5) -- above the entire bracket this project costs P4 in.  This is the other
+half of that comparison: the same MC68450 field tables (tools/analysis/
+mc68450.py, one copy) applied to the bytes src/player/dma.i actually programs.
+
+IT PARSES THE SOURCE RATHER THAN RESTATING IT.  A constant typed into this file
+would be a claim about the player that the player could quietly stop honouring;
+the equates are read out of src/player/dma.i, so a change there changes what is
+printed here and a mismatch between the two is not expressible.
+
+IT IS A GATE.  Each configuration is checked against what it is FOR -- held
+must decode as a mode that keeps the bus, stealing must decode as one that does
+not -- and a disagreement exits non-zero rather than printing a paragraph.
+
+WHAT IT IS NOT: a rate.  Nothing here is a measurement of anything.  It says
+which mode the player asks the chip for; tools/bench/dma_run.sh shows the
+machine doing it, and `W` -- the clocks it costs on real silicon -- remains the
+project's largest open number (ROADMAP B1/B3).
+"""
+import sys, os, re
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+from mc68450 import dcr, ocr, scr, XRM, DTYP, REQG
+
+src = sys.argv[1] if len(sys.argv) > 1 else "src/player/dma.i"
+if not os.path.exists(src):
+    sys.exit(f"missing {src} -- run from the repo root.")
+text = open(src).read()
+
+def equ(name):
+    m = re.search(rf"^{name}\s*=\s*\$([0-9A-Fa-f]+)", text, re.M)
+    if not m:
+        sys.exit(f"{src} no longer defines {name}. This script reads the "
+                 f"player's own equates; it does not keep a copy of them.")
+    return int(m.group(1), 16)
+
+# The SCR the channel is given is written inline rather than equated, because it
+# is the same for both configurations and there is nothing to choose about it.
+m = re.search(r"move\.b\s+#\$([0-9A-Fa-f]+),DM_SCR", text)
+if not m:
+    sys.exit(f"{src} no longer writes DM_SCR with a literal.")
+SCR = int(m.group(1), 16)
+
+CFG = [("BUS HELD",       "DM_HELD_DCR",  "DM_HELD_OCR"),
+       ("CYCLE STEALING", "DM_STEAL_DCR", "DM_STEAL_OCR")]
+
+print(f"WHAT src/player/dma.i PROGRAMS -- decoded from {src}\n")
+bad = 0
+for label, dn, on in CFG:
+    D, O = equ(dn), equ(on)
+    print(f"  {label}   DCR = ${D:02X}   OCR = ${O:02X}   SCR = ${SCR:02X}")
+    for line in dcr(D):
+        print(f"      {line}")
+    for line in ocr(O):
+        print(f"      {line}")
+    for line in scr(SCR):
+        print(f"      {line}")
+    holds = (D >> 6 & 3) in (0, 3)          # burst, or cycle steal WITH hold
+    dual  = (D >> 4 & 3) in (0, 1)
+    tomem = bool(O & 0x80)
+    checks = [
+        (dual,  "DTYP must be explicitly addressed: only channel 0 has device "
+                "callbacks in this machine, so an implicit-address DTYP on "
+                "channel 1 falls through to the dual-address path anyway"),
+        (tomem, "OCR DIR must be device -> memory; this is a READ"),
+        ((O >> 4 & 3) == 0, "OCR SIZE must be byte: the SPC's port is 8 bits"),
+        ((O >> 2 & 3) == 0, "OCR CHAIN must be none until P5a picks a chaining "
+                            "scheme for the two-deep request queue (FINDINGS 55.3)"),
+        ((SCR & 3) == 0, "SCR DAC must not count: the device address is a "
+                         "REGISTER at $EA0015 and must not walk off it"),
+        ((SCR >> 2 & 3) == 1, "SCR MAC must increment: the record is contiguous"),
+        ((O & 3) in (0, 1), "OCR REQG must be an AUTO-request mode: the "
+                            "expansion slot has no request line to the DMAC in "
+                            "this machine, so external request cannot be run"),
+    ]
+    if label == "BUS HELD":
+        checks.append((holds, "the held configuration must decode as a mode "
+                              "that KEEPS the bus between operands"))
+        checks.append(((O & 3) == 1, "and as max-rate auto-request: MAME models "
+                                     "a held bus only for burst + REQG 01"))
+    else:
+        checks.append((not holds, "the stealing configuration must decode as a "
+                                  "mode that RELEASES the bus between operands "
+                                  "-- otherwise the two have no contrast"))
+    for ok, why in checks:
+        if not ok:
+            print(f"      FAIL: {why}")
+            bad += 1
+    print()
+
+print("""AGAINST THE MACHINE'S OWN DISK CHANNEL (21_iplrom_dmac.py, FINDINGS 52.5)
+
+  IPL ROM ch1, SASI    DCR $80 OCR $B2   dual address, 8-bit port, cycle steal
+                                         WITHOUT hold, EXTERNAL request
+                                         -> a full arbitration per byte, 16..19
+  player,  held        DCR $00 OCR $81   dual address, 8-bit port, BURST,
+                                         auto-request at max rate
+                                         -> the ladder's dual-address held row, 9
+
+Sharp's own configuration and the player's differ in exactly the field that
+decides the project. That is 52.5's finding read the other way round: a cheaper
+configuration IS reachable for an explicitly-addressed 8-bit port, and what it
+costs on real silicon is still ROADMAP B3's question and not this file's.""")
+sys.exit(1 if bad else 0)

tools/analysis/28_autorequest_cost.py

@@ -0,0 +1,172 @@
+#!/usr/bin/env python3
+"""What AUTO-REQUEST DMA costs the 68000, when there is no request line.
+
+    python3 tools/analysis/28_autorequest_cost.py --kbps 460 [--record 37405]
+
+WHY THIS EXISTS.  The project's per-byte ladder -- W = 5 single-address held, 9
+dual held, 12 single arbitrated, 16..19 dual arbitrated (FINDINGS 42.4, 52.5) --
+prices a transfer that the DEVICE asks for: one external request, one operand,
+a known number of stolen clocks per delivered byte.  Session 27 found that the
+CZ-6BS1 as MAME models it has NO REQUEST LINE to the DMAC at all (FINDINGS
+59.2): the card's flow control is DTACK, and every configuration that can be run
+against it is AUTO-REQUEST, where the channel transfers because its own counter
+says so and not because a byte has arrived.
+
+THAT CHANGES THE CURRENCY, and it is the reason this file is not a line in
+another one.  An externally requested transfer is charged PER DELIVERED BYTE.
+An auto-requested one is charged PER UNIT OF TIME THE CHANNEL IS ACTIVE, because
+the channel has no way to know the device is not ready: it takes its allotted
+share of the bus and spends it whether or not a byte comes back.  So the cost of
+delivering a record depends on HOW LONG THE RECORD TAKES TO ARRIVE -- i.e. on
+the delivery rate, the figure this tree deliberately has no default for (FINDINGS
+50) -- and the tool REQUIRES one rather than assuming it.
+
+SOURCED: MC68450 Direct Memory Access Controller, Motorola, Jul 1989
+(bitsavers), sections 3.8 and 5.2.3.3, the same document buscost.py's transfer
+timings come from.  Section 5.2.3.3.1: under maximum-rate auto-request "all
+operands in the data block will be transferred in one burst, so that the DMAC
+will use 100% of the available bus bandwidth" -- which is the datasheet saying,
+in its own words, what session 27 measured MAME's model doing when it HALTED the
+68000 for the whole data phase (FINDINGS 59.1).
+
+THE ONE LOAD-BEARING ASSUMPTION, stated because the whole table rests on it:
+that the channel SPENDS its allotted share whether or not the device has a byte.
+Under auto-request a request is pending until MTC is exhausted, so the DMAC
+takes the bus during every burst window it is entitled to; when the device is
+not ready the cycle is stretched by wait states (a real CZ-6BS1 negating DTACK)
+or retried later (MAME's model discards the operand), and either way the window
+is gone from the CPU's point of view.  If a real card instead lets the DMAC off
+the bus early when no byte is there, these figures are UPPER BOUNDS.  That is a
+board question and it is ROADMAP B3's.
+
+NOT A MEASUREMENT.  Every figure below is arithmetic over datasheet constants
+and an explicit rate.  `W` is still unmeasured and still wants a board.
+"""
+import sys, os, argparse
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+from buscost import DMA_DUAL_BYTE_CLK, DMA_READ_CLK, DMA_WRITE_CLK
+
+CPU_HZ = 10_000_000.0            # the X68000 the whole tree is costed against
+FPS = 12.0
+
+ap = argparse.ArgumentParser()
+ap.add_argument("--kbps", type=float, required=True,
+                help="delivery rate in KB/s. REQUIRED: this tree has no default "
+                     "rate and the whole answer scales with it (FINDINGS 50).")
+ap.add_argument("--record", type=int, default=37405,
+                help="mean record size in bytes (default: the gate container's)")
+a = ap.parse_args()
+RATE = a.kbps * 1024.0
+
+# --- 3.8 GENERAL CONTROL REGISTER, decoded from the formulas in 5.2.3.3.2 ---
+#   burst time    = 2^(BT + 4) clocks
+#   sample period = 2^(BT + BR + 5) clocks
+#   DMAC's share  = 2^-(BR + 1)
+# and Table 5-3 prints all sixteen combinations, so the formulas are GATED
+# against the table rather than trusted.
+TABLE = {           # (BR, BT): (burst, MPU period, share, sample period)
+ (0,0):(16,16,.5,32),   (0,1):(32,32,.5,64),   (0,2):(64,64,.5,128),  (0,3):(128,128,.5,256),
+ (1,0):(16,48,.25,64),  (1,1):(32,96,.25,128), (1,2):(64,192,.25,256),(1,3):(128,384,.25,512),
+ (2,0):(16,112,.125,128),(2,1):(32,224,.125,256),(2,2):(64,448,.125,512),(2,3):(128,896,.125,1024),
+ (3,0):(16,240,.0625,256),(3,1):(32,480,.0625,512),(3,2):(64,960,.0625,1024),
+ (3,3):(128,1920,.0625,2048),
+}
+bad = 0
+for (br, bt), (burst, mpu, share, sample) in sorted(TABLE.items()):
+    f_burst, f_sample, f_share = 2**(bt+4), 2**(bt+br+5), 2.0**-(br+1)
+    for got, want, what in ((f_burst, burst, "burst time"),
+                            (f_sample, sample, "sample period"),
+                            (f_share, share, "bandwidth share"),
+                            (f_sample - f_burst, mpu, "MPU period")):
+        if got != want:
+            print(f"  FAIL BR={br:02b} BT={bt:02b} {what}: formula {got}, "
+                  f"Table 5-3 {want}")
+            bad += 1
+if bad:
+    sys.exit(f"\n{bad} disagreements between 5.2.3.3.2's formulas and Table 5-3. "
+             "Everything below\nis those formulas, so it is not printed.")
+print(f"MC68450 5.2.3.3.2's formulas reproduce all 16 rows of Table 5-3.\n")
+
+BYTE_CLK = DMA_DUAL_BYTE_CLK     # dual address, 8-bit port: a 4-clock read of
+                                 # $EA0015 and a 5-clock write to the ring
+frame_clk = CPU_HZ / FPS
+wire_s = a.record / RATE                     # how long the record takes to land
+wire_clk = wire_s * CPU_HZ                   # ...in 68000 clocks
+per_byte_wire = wire_clk / a.record          # clocks of wall time per byte
+
+print(f"THE RECORD: {a.record:,} B at {a.kbps:g} KB/s = {wire_s*1000:.2f} ms "
+      f"= {wire_clk:,.0f} clocks = {100*wire_clk/frame_clk:.1f}% of a "
+      f"{FPS:g} fps frame")
+print(f"  one byte of WIRE TIME is {per_byte_wire:.2f} clocks; one byte of DMAC "
+      f"WORK is {BYTE_CLK} ({DMA_READ_CLK} read + {DMA_WRITE_CLK} write, "
+      f"buscost.py)\n")
+
+print("REQG 01, AUTO-REQUEST AT MAXIMUM RATE -- what session 27 demonstrated")
+print(f"  The channel holds the bus until MTC is exhausted (5.2.3.3.1: 100% of "
+      f"the\n  bandwidth), so the CPU gets NOTHING for the whole delivery:")
+print(f"    cost to the 68000 = the whole {100*wire_clk/frame_clk:.1f}% of a "
+      f"frame, or {per_byte_wire:.2f} clk/B")
+print(f"  It is the cheapest configuration per BYTE MOVED and the dearest per "
+      f"byte\n  DELIVERED, and the gap between those is the device's own "
+      f"slowness:\n    {BYTE_CLK} clocks of work in {per_byte_wire:.1f} clocks "
+      f"of waiting = {100*BYTE_CLK/per_byte_wire:.1f}% of the held bus does "
+      f"anything.\n")
+
+print("REQG 00, LIMITED-RATE AUTO-REQUEST -- the lever the GCR actually gives")
+print("  The DMAC takes its programmed share of the bus and spends it whether "
+      "or not\n  a byte is there, so the CPU pays the SHARE for the WHOLE "
+      "delivery -- and the\n  share must also be big enough to carry the rate. "
+      "Both, or it does not fit.\n")
+print("   BR  share   sustains   clk/B charged   % of a frame   fits "
+      f"{a.kbps:g} KB/s?")
+fits_any = []
+for br in range(4):
+    burst, mpu, share, sample = TABLE[(br, 3)]     # BT=11, the longest burst
+    # bytes the channel can move inside one burst window, and how often that
+    # window comes round
+    bytes_per_burst = burst // BYTE_CLK
+    sustains = bytes_per_burst * CPU_HZ / sample
+    charged = share * per_byte_wire                # clocks the CPU loses per
+                                                   # DELIVERED byte
+    pct = 100 * share * wire_clk / frame_clk
+    ok = sustains >= RATE
+    if ok:
+        fits_any.append((br, share, charged, pct))
+    print(f"   {br:02b}  {share*100:5.2f}%  {sustains/1024:7.1f} KB/s  "
+          f"{charged:9.2f}      {pct:8.1f}%      {'yes' if ok else 'NO'}")
+print(f"\n  (BT = 11 throughout: the longest burst, 128 clocks, which is the "
+      f"most\n  favourable row -- a shorter burst moves fewer bytes per window "
+      f"at the same\n  share and sustains proportionally less.)")
+
+if not fits_any:
+    print(f"\n  NOTHING FITS. At {a.kbps:g} KB/s no limited-rate share can "
+          f"carry the record,\n  so the only auto-request configuration that "
+          f"delivers is maximum rate --\n  and that one stops the CPU for the "
+          f"whole {100*wire_clk/frame_clk:.1f}% of a frame the record takes.")
+else:
+    br, share, charged, pct = fits_any[0]
+    print(f"\n  CHEAPEST THAT FITS: BR = {br:02b}, {share*100:g}% of the bus, "
+          f"{charged:.2f} clk/B charged to the\n  68000 -- {pct:.1f}% of a frame "
+          f"per record.")
+    print(f"  Against the ladder: W=5 held costs {5*a.record/frame_clk*100:.1f}%, "
+          f"W=9 dual held {9*a.record/frame_clk*100:.1f}%,\n  W=19 the IPL ROM's "
+          f"own {19*a.record/frame_clk*100:.1f}%, and PIO measured "
+          f"{87.28*a.record/frame_clk*100:.1f}% (FINDINGS 58.2).")
+
+print(f"""
+WHAT THIS SETTLES, AND WHAT IT DOES NOT
+
+1. AUTO-REQUEST IS CHARGED BY TIME, NOT BY BYTE. Every W in this project is
+   clocks per DELIVERED byte, which presumes the device asks. With no request
+   line the channel spends its share of the bus at a rate it was told, so the
+   record's cost scales with how long the disc takes -- halve the delivery rate
+   and the CPU cost of the same record DOUBLES. No W does that.
+
+2. THE GCR IS A DESIGN LEVER NOBODY HAD NAMED. BT and BR are two bits each and
+   they set what fraction of the bus the player gives away. That is the same
+   kind of choice as `aligned` vs `split` and it belongs in the same list.
+
+3. IT IS STILL NOT A MEASUREMENT. These are datasheet constants and an explicit
+   rate. Whether the real CZ-6BS1 drives #EXREQ (pin B36 exists on the slot, and
+   MAME's model simply does not connect it) is ROADMAP B3's question, and if it
+   does, the ladder applies and this file is the fallback rather than the plan.""")

tools/analysis/29_packed_player.py

@@ -0,0 +1,405 @@
+#!/usr/bin/env python3
+"""The DECODER-FREE PACKED player, priced against the measured cost model.
+
+    python3 tools/analysis/29_packed_player.py [container.dlx] [--kbps R]
+
+THE QUESTION, and why it is being asked again.  FINDINGS 44.7 removed the codec
+and asked what a player that just puts literal frames on screen would cost.  It
+answered "it fits the clocks and dies on the medium": 1,152 KB/s and 1.61 GB,
+because 256-colour GVRAM's default write path throws away the high byte of every
+word and a picture byte therefore costs two disc bytes.  46.5/47.1 then found
+the off switch -- CRTC R20 bit 11 -- and 47.2 built the layout and rendered it
+pixel-exactly on both emulators at 1.0 B/pixel.  47.5 re-derived the budget on
+that and withdrew 44.7's conclusion CONDITIONALLY.
+
+Everything in 47.5 is arithmetic over a cost model that has since been REPLACED.
+When it was written the transport was an unmeasured `c`; sessions 25b-28 put the
+transport on the 68000 and measured it (58.2: PIO is 87.28 clk/B), put it on the
+DMAC and bounded it (59.2: this machine can run dual-address only, and a
+dual-address byte has a 9 clk/B FLOOR), and re-derived what a frame can afford
+(59.7/60.7: 6.69 clk/B on the gate container).  47.6.1 also filed the CPU paint
+cost as an ASSUMPTION -- "the `movem` shape of the packed writes is an
+assumption", no clock in 47.5 measured.
+
+So this tool re-asks 44.7's question with:
+
+  * the paint MEASURED, not assumed -- tools/bench/blit.s V8 is V1 with 128
+    words a row instead of 256, and tools/bench/blit.lua times it next to V1,
+    V2 and V3 in the same run, so the packed number is quoted against a
+    variant whose value (53.6%) is a session-9 result that has not moved;
+  * the transport swept over the SAME `W` ladder 15_bus_occupancy.py uses,
+    every rung of it sourced or measured (buscost.py);
+  * the audio DMA charged, at the rate the IPL ROM's own channel-3 setup
+    implies (21_iplrom_dmac.py) -- 60.x's rule that a budget debits I/O;
+  * and the wire and the volume stated for each, because 44.7's answer was
+    never about clocks.
+
+WHAT IT DOES NOT DO.  It does not settle 47.4 -- whether buffer mode BLANKS the
+graphics layer, which MAME asserts and px68k is silent about (48.1), and which
+needs a real board.  It PRICES both branches instead, and the blanking section
+is where the measured paint earns its keep: the black interval is the paint, and
+until now the paint was a range read off an unpacked measurement ("~27% to ~54%",
+48.3) rather than a number.
+"""
+import sys, os, re, argparse, csv
+sys.path.insert(0, "tools/encoder")
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+import numpy as np
+from dlx import DLX
+import buscost as B
+
+CPUHZ = 10e6                      # stock X68000, MAME 0.277 x68k.cpp:1133
+GAME_S = 22.8 * 60                # the full-disc survey's runtime (ROADMAP C1)
+
+ap = argparse.ArgumentParser()
+ap.add_argument("container", nargs="?", default="tmp/rc_fr_singe_scsi_span.dlx",
+                help="the CODEC baseline this is compared against")
+ap.add_argument("--csv", default="tmp/c68k_frames.csv",
+                help="per-frame C68K measurement of that container")
+ap.add_argument("--blit-log", default="tmp/blit_v8.log",
+                help="tools/bench/blit.lua's log -- where the MEASURED paint "
+                     "costs are read from. No defaults are compiled in.")
+ap.add_argument("--kbps", type=float, default=None,
+                help="a delivery rate to score the wire against. OPTIONAL and "
+                     "there is no default (FINDINGS 50).")
+a = ap.parse_args()
+
+# --- the measured paint, read out of the run's own log ---------------------
+# NOT transcribed into this file.  A constant copied out of a log is a constant
+# that stops tracking the log, and this project has been caught by a stale
+# number twice (60.8).  If the log is not there the tool refuses rather than
+# substituting a plausible one.
+if not os.path.exists(a.blit_log):
+    sys.exit(f"missing {a.blit_log} -- run tools/bench/blit.lua first:\n"
+             f"  cd tmp && mame x68000 -bios ipl10 -ramsize 2M -video soft "
+             f"-window -sound none -nothrottle -plugins \\\n"
+             f"      -autoboot_script ../tools/bench/blit.lua -seconds_to_run 60")
+blit = {}
+for line in open(a.blit_log, errors="replace"):
+    m = re.search(r"V(\d+)\s+(\d+) cyc", line)
+    if m:
+        blit[int(m.group(1))] = int(m.group(2))
+for v in (1, 2, 3, 4, 8, 9, 10):
+    if v not in blit:
+        sys.exit(f"{a.blit_log} has no V{v} result -- the summary is incomplete, "
+                 f"so the run did not finish and nothing here can be quoted.")
+
+# --- the codec baseline: the container, and its MEASURED decode ------------
+d = DLX(a.container)
+FPS = d.fps
+FRAME_CLK = CPUHZ / FPS
+meas = {}
+if os.path.exists(a.csv):
+    for r in csv.DictReader(open(a.csv)):
+        meas[int(r["frame"])] = int(r["cycles"])
+if not meas:
+    sys.exit(f"missing {a.csv} -- the codec row's decode term is MEASURED and "
+             f"there is no derived stand-in for it.")
+NF = max(meas) + 1
+codec_decode = np.mean([meas[f] for f in range(NF)])
+codec_bpf = sum(d.record_lengths()[:NF]) / NF     # the PADDED record (60.7)
+
+# --- geometry, which is where the decoder-free rows come from -------------
+W_PX, H_PX = d.W, d.H
+NPX = W_PX * H_PX
+UNPACKED_BPF = NPX * 2            # one pixel per word, high byte discarded
+PACKED_BPF = NPX * 1              # R20 bit 11 + page scroll (47.2, measured)
+
+aud_bpf = B.ADPCM_BYTES_PER_S / FPS
+AUD_CLK = aud_bpf * B.ADPCM_CLK_BYTE_BEST         # best case, so every row is
+                                                  # the optimistic end
+# A device->GVRAM channel cannot walk a 1024-byte line stride inside one
+# transfer: it writes a contiguous run.  192 rows therefore need 192 array-chain
+# entries -- and SESSION 29 RAN THAT, off the disc, through src/player/dma.i's
+# DM_BARV/DM_BTCV: eight rows at the 1024 B stride landed from ONE channel start
+# with the CPU halted throughout (tools/bench/dma_run.sh, `[chain]`).  So the
+# MECHANISM is demonstrated and the CPU does not restart the channel per row.
+# The COST is still datasheet arithmetic -- 36 clocks an entry, Fig 4-25 sheet 1,
+# buscost.DMA_CHAIN_CLK -- because MAME's DMAC runs on wall-clock attotimes and
+# cannot be asked what anything costs (42.5).
+CHAIN_CLK = H_PX * B.DMA_CHAIN_CLK
+
+print(f"""{a.container}: {NF} frames of {W_PX}x{H_PX} at {FPS:g} fps
+frame slot on a 10 MHz 68000: {FRAME_CLK:,.0f} clocks
+paint costs MEASURED by tools/bench/blit.lua, read from {a.blit_log}:
+  V1 unpacked movem blit   {blit[1]:>9,} clk   {100*blit[1]/FRAME_CLK:5.1f}%   (96 KB read + 96 KB write)
+  V2 byte-source expansion {blit[2]:>9,} clk   {100*blit[2]/FRAME_CLK:5.1f}%   (48 KB read + 96 KB write)
+  V3 write-only floor      {blit[3]:>9,} clk   {100*blit[3]/FRAME_CLK:5.1f}%   (no source read at all)
+  V8 PACKED movem blit     {blit[8]:>9,} clk   {100*blit[8]/FRAME_CLK:5.1f}%   (48 KB read + 48 KB write)
+
+  V8 is {100*blit[8]/blit[1]:.1f}% of V1 and {100*blit[8]/blit[3]:.1f}% of V3 -- so PACKED PAINT COSTS WHAT THE
+  UNPACKED PATH PAYS TO WRITE ALONE, with its source read thrown in free.
+  It is not exactly half of V1 because the 192-row loop does not halve with
+  the words: per word V1 is {blit[1]/(NPX):.3f} clk and V8 is {blit[8]/(NPX//2):.3f}.""")
+
+# --- the architectures ----------------------------------------------------
+# Each is (label, bytes on the wire per frame, CPU clocks per frame that are
+# NOT the transport, and whether the transport lands in GVRAM or in RAM).
+ARCH = [
+    ("CODEC, CPU-decoded (the shipping design)", codec_bpf, codec_decode, "ring"),
+    ("free / DMAC device->GVRAM / unpacked",     UNPACKED_BPF, CHAIN_CLK, "gvram"),
+    ("free / DMAC device->GVRAM / PACKED",       PACKED_BPF,   CHAIN_CLK, "gvram"),
+    ("free / CPU-painted / unpacked, 2 B/px wire", UNPACKED_BPF, blit[1], "ring"),
+    ("free / CPU-painted / unpacked, 1 B/px wire", PACKED_BPF,   blit[2], "ring"),
+    ("free / CPU-painted / PACKED",              PACKED_BPF,   blit[8], "ring"),
+]
+
+LADDER = [
+    (5.0,   "single address, held      -- needs a request line (B3)"),
+    (9.0,   "dual address, held        -- the FLOOR (59.2/59.7)"),
+    (12.0,  "single address, arbitrated"),
+    (16.0,  "what the ROM programs for SASI, best"),
+    (19.0,  "what the ROM programs for SASI, worst"),
+    (87.28, "PIO -- MEASURED, 58.2"),
+]
+
+print("\n" + "=" * 78)
+print("WHAT EACH ARCHITECTURE COSTS A FRAME, over the transport ladder\n")
+print(f"  audio DMA is charged in every row at {AUD_CLK:,.0f} clk "
+      f"({100*AUD_CLK/FRAME_CLK:.2f}%), best case.\n")
+hdr = f"  {'architecture':<44}{'B/frame':>9}" + "".join(f"{f'W={w:g}':>9}" for w, _ in LADDER)
+print(hdr)
+print("  " + "-" * (len(hdr) - 2))
+for label, bpf, cpu, dest in ARCH:
+    cells = []
+    for w, _ in LADDER:
+        tot = bpf * w + cpu + AUD_CLK
+        pct = 100 * tot / FRAME_CLK
+        cells.append(f"{pct:>8.1f}%" if pct < 1000 else f"{pct:>8.0f}%")
+    print(f"  {label:<44}{bpf:>9,.0f}" + "".join(cells))
+print(f"""
+  100% is the frame deadline.  Every cell is CPU work plus transport plus
+  best-case audio; none of them overlap, because the 68000 has no cache and a
+  two-word prefetch queue that empties at once (buscost.DMA_OVERLAPS = False).
+
+  THE TWO ROWS THAT MATTER ARE THE FLOOR COLUMN, W=9, because 59.2 found that
+  the only configurations this machine can be shown to run are dual-address,
+  and a dual-address byte is a 4-clock read of the device plus a 5-clock write
+  to memory.  Every column left of it is a hardware fact nobody here has.""")
+
+# --- the wire, which is what 44.7 actually died on ------------------------
+print("\n" + "=" * 78)
+print("THE WIRE AND THE MEDIUM -- 44.7's real objection\n")
+print(f"  {'architecture':<44}{'B/frame':>9}{'KB/s':>9}{'GB for 22.8 min':>18}")
+print("  " + "-" * 78)
+seen = set()
+for label, bpf, cpu, dest in ARCH:
+    kbs = bpf * FPS / 1024
+    gb = bpf * FPS * GAME_S / 1e9
+    print(f"  {label:<44}{bpf:>9,.0f}{kbs:>9.1f}{gb:>18.2f}")
+print(f"""
+  The codec row is the gate container, which is deliberately the heaviest thing
+  the encoder emits (59.7).  The default `need` recipe is 267.9 KB/s and E7's
+  byte target at the 9 clk/B floor is 327 KB/s (60.7).
+
+  SO THE PACKED DECODER-FREE PLAYER ASKS FOR {PACKED_BPF*FPS/1024:.0f} KB/s -- {PACKED_BPF*FPS/1024/327:.2f}x E7's target and
+  {PACKED_BPF*FPS/1024/(codec_bpf*FPS/1024):.2f}x the gate container -- and it asks for it AT A FIXED RATE.  A codec's
+  bitrate is a lever; a literal frame's is geometry, and there is no scene in
+  the picture that costs less than another.""")
+if a.kbps:
+    R = a.kbps * 1024
+    print(f"\n  against a supplied {a.kbps:g} KB/s:")
+    for label, bpf, cpu, dest in ARCH:
+        need = bpf * FPS
+        print(f"    {label:<44}{'FITS' if need <= R else 'SHORT BY '}"
+              f"{'' if need <= R else f'{(need-R)/1024:.0f} KB/s'}"
+              f"   ({need/1024:.0f} KB/s wanted)")
+
+# --- 47.4's two branches, priced -----------------------------------------
+print("\n" + "=" * 78)
+print("IF BUFFER MODE BLANKS THE LAYER (47.4 / 48, MAME's reading)\n")
+print("""  R20 bit 11 only has to be SET across the GVRAM writes, so the black
+  interval is the paint and not the frame -- and which paint depends on where
+  the transport lands.  That asymmetry has not been stated before:\n""")
+print(f"  {'architecture':<44}{'black interval':>16}   {'set for':<14}")
+print("  " + "-" * 78)
+for label, bpf, cpu, dest in ARCH[1:]:
+    if dest == "gvram":
+        # the channel writes GVRAM, so the bit is set for the whole transfer
+        # DMA rungs only: a PIO transport is not a channel writing GVRAM, so
+        # 87.28 has no meaning in a device->GVRAM row.
+        rows = [bpf * w + CHAIN_CLK for w, _ in LADDER if w < 20]
+        span = f"{100*min(rows)/FRAME_CLK:.0f}%..{100*max(rows)/FRAME_CLK:.0f}%"
+        note = "the whole DMA"
+    else:
+        span = f"{100*cpu/FRAME_CLK:.1f}%"
+        note = "the blit only"
+    print(f"  {label:<44}{span:>16}   {note:<14}")
+print(f"""
+  THE CPU-PAINTED PACKED PATH HAS THE SMALLEST BLACK WINDOW OF ANY OF THEM --
+  {100*blit[8]/FRAME_CLK:.1f}% -- because its transport lands in RAM, where bit 11 is irrelevant,
+  and only the {blit[8]:,}-clock blit needs the bit set.  The DMAC-direct path,
+  which is cheaper in clocks at every rung of the ladder, is the one that must
+  hold the bit across its whole transfer.  Under MAME's reading the cheap
+  architecture is the dark one.
+
+  Both are a strobe at the frame rate over the whole picture, and the packed
+  layout has no page to flip to: both 256-colour pages carry picture, which is
+  the entire point of it (48.3).  {100*blit[8]/FRAME_CLK:.1f}% black at 12 Hz is not a tear.
+
+  IF PX68K IS RIGHT AND IT DOES NOT BLANK, every number above stands as
+  written.  Neither emulator is authority and 48.1 is why the prior leans
+  MAME's way: MAME asserts the semantic twice and deliberately, px68k's
+  display path never reads the bit at all.  That is an assertion against a
+  silence, not a tie, and it is settled by a board and the two-line probe in
+  tools/bench/probe_bit11_blank.lua.""")
+
+# --- 47.6.4: does the CODEC survive the packed layout? --------------------
+# Open since session 16 and never touched: "under the packed layout a word spans
+# two columns 128 apart. Whether the existing codec survives that is untouched."
+# There are exactly two ways it could, and blit.s V9 and V10 are them.
+sk_blocks = sk_tot = pair_sk = pair_tot = 0
+for f in range(NF):
+    m = d.modes(f).reshape(d.nby, d.nbx)
+    L, R = m[:, :d.nbx // 2], m[:, d.nbx // 2:]
+    sk_blocks += int((m == 0).sum()); sk_tot += m.size
+    pair_sk += int(((L == 0) & (R == 0)).sum()); pair_tot += L.size
+paint_now = 1 - sk_blocks / sk_tot
+paint_pair = 1 - pair_sk / pair_tot
+
+print("\n" + "=" * 78)
+print("CAN THE CODEC BE PACKED TOO? -- 47.6.4, open since session 16\n")
+print(f"""  A 4x4 block owns four bytes at STRIDE 2 under the packed layout, because
+  the high bytes of its four words belong to the block 128 columns away. So a
+  block decoder has two options and neither is free:
+
+  {'V4  block order, UNPACKED (the shipping shape)':<52}{blit[4]:>9,} clk  {100*blit[4]/FRAME_CLK:5.1f}%
+  {'V9  block order, PACKED, 16 move.b at stride 2':<52}{blit[9]:>9,} clk  {100*blit[9]/FRAME_CLK:5.1f}%
+  {'V10 block order, PACKED, blocks PAIRED (movem back)':<52}{blit[10]:>9,} clk  {100*blit[10]/FRAME_CLK:5.1f}%
+
+  V9 IS {100*blit[9]/blit[4]-100:.0f}% DEARER THAN V4, not cheaper. Packing buys a block decoder
+  nothing on the wire either -- a codeword is already one byte a pixel -- so
+  that route buys NOTHING and costs {blit[9]-blit[4]:,} clocks a frame to buy it.
+
+  V10 halves the paint, and pays for it in the mode map. A pair skips only if
+  BOTH its blocks skip, and on this container:
+
+    SKIP blocks now                 {100*sk_blocks/sk_tot:5.1f}%      painted now    {100*paint_now:5.1f}%
+    SKIP block PAIRS                {100*pair_sk/pair_tot:5.1f}%      painted paired {100*paint_pair:5.1f}%
+
+  So pairing paints {paint_pair/paint_now:.2f}x as many blocks for {blit[10]/blit[4]:.2f}x the paint per block --
+  {100*(paint_pair/paint_now)*(blit[10]/blit[4])-100:+.0f}% on the clock, and about {100*(paint_pair/paint_now-1):+.0f}% on the BYTES, because a coded
+  block is bytes in the container whether its half of the pair changed or not.
+  E7 needs the bytes DOWN {100*(codec_bpf*FPS/1024)/327-100:.0f}%.
+
+  SO PACKING BELONGS TO THE LITERAL PLAYER AND ONLY TO IT. 47.6.4 is closed:
+  the packed layout is not an upgrade the existing codec can take, it is the
+  thing you get INSTEAD of the codec.""")
+
+# --- the palette, which is where the literal player stops being a compromise --
+# 46.3 measured these while pricing the TEXT PLANE and the 256-colour rows were
+# only there for scale.  They answer a question nobody put to them: a literal
+# player has no codebooks, so it is not tied to a scene-wide palette the way the
+# codec is (vq.scene_palette exists BECAUSE codewords are indices into it), and
+# per-frame palettes become legal.  Re-run 18_text_plane_16col.py to reproduce.
+PSNR_SHIPPED   = 29.19        # docs/STATUS.md, --spans all, c=5, 496.7 KB/s
+PSNR_SCENE_256 = 31.33        # 18_text_plane_16col.py, tmp/fr_singe, 120 frames
+PSNR_FRAME_256 = 34.08        # the same window, per-frame palettes
+PAL_BYTES = 512               # 256 entries x 1 word
+
+pal_bpf = PACKED_BPF + PAL_BYTES
+# The palette write, DERIVED from a MEASURED per-word constant: V8 moves a word
+# into GVRAM for blit[8]/(NPX//2) clocks and the palette is 256 consecutive
+# words at $E82000 in the same movem shape.
+pal_clk = 256 * blit[8] / (NPX // 2)
+
+print("\n" + "=" * 78)
+print("AND THE PICTURE IS BETTER, WHICH NOBODY HAD ASKED\n")
+print(f"""  PSNR against the 24-bit source, 18_text_plane_16col.py over the same
+  120-frame window the whole tree is measured on:
+
+    {'shipping container (the codec, as it ships)':<48}{PSNR_SHIPPED:6.2f} dB
+    {'256 colours, SCENE palette -- the codec CEILING':<48}{PSNR_SCENE_256:6.2f} dB
+    {'256 colours, PER-FRAME palette':<48}{PSNR_FRAME_256:6.2f} dB
+
+  THE MIDDLE ROW IS A CEILING AND NOT A RIVAL. Every codeword the codec emits
+  is an index INTO the scene palette, so no amount of bitrate takes it past
+  {PSNR_SCENE_256:.2f} dB; it spends {codec_bpf*FPS/1024:.0f} KB/s to get within {PSNR_SCENE_256-PSNR_SHIPPED:.2f} dB of it.
+
+  A LITERAL FRAME HAS NO CODEBOOKS, so the scene palette is not forced on it,
+  and the bottom row is what it simply IS -- {PSNR_FRAME_256-PSNR_SHIPPED:+.2f} dB on the shipping
+  container and {PSNR_FRAME_256-PSNR_SCENE_256:+.2f} dB past the ceiling the codec cannot cross.
+
+  WHAT THE PER-FRAME PALETTE COSTS:
+    on the wire   {PAL_BYTES} B a frame -> {pal_bpf:,} B, {pal_bpf*FPS/1024:.1f} KB/s (+{100*PAL_BYTES/PACKED_BPF:.1f}%)
+    in clocks     ~{pal_clk:,.0f} ({100*pal_clk/FRAME_CLK:.2f}% of a frame) if the CPU writes it, DERIVED
+                  from V8's measured {blit[8]/(NPX//2):.3f} clk/word in the same movem shape
+    in colours    254, not 256: the packed layout spends index 0 on the
+                  transparency key and puts black at 255 (47.2,
+                  prep_frame.py --pack-transparent), against --reserve-black's
+                  one entry. The tree has already measured a reserved entry at
+                  0.04 dB (60.3), so this is noise against {PSNR_FRAME_256-PSNR_SHIPPED:+.2f}.
+
+  SETTLED IN SESSION 30, AND THE ANSWER IS YES (FINDINGS 62): a channel writes
+  the palette registers at $E82000 byte-exact, and ONE array-chained start
+  crosses from those registers into GVRAM -- so the palette IS a 193rd chain
+  entry and the clocks row above is what the CPU pays only if it does the write
+  itself. dmagate.s runs 7-9. What that does NOT settle is the board: MAME maps
+  the palette to palette_device over memory_array, whose write16 is a plain
+  COMBINE_DATA, so there is no handler that could refuse a byte write and the
+  model cannot discriminate. ROADMAP B4.
+
+  AND THE PSNR FIGURES ARE PIL's MEDIANCUT, not this project's own palette
+  builder (vq.scene_palette / H.build). The DIRECTION is measured and the
+  magnitude is about right; if the packed player gets built, re-derive the
+  per-frame number against the builder that will actually ship it.""")
+
+# --- the answer ----------------------------------------------------------
+w9 = 9.0
+free_packed_dma = PACKED_BPF * w9 + CHAIN_CLK + AUD_CLK
+free_packed_cpu = PACKED_BPF * w9 + blit[8] + AUD_CLK
+# ... and the same two rows with the PER-FRAME PALETTE actually charged, which
+# is what a player ships.  The picture rows above are the comparison against the
+# codec and are left alone so the published 55.2% / 81.6% do not drift; these
+# are the shipping figures.  Session 30 (FINDINGS 62) made the DMAC row's
+# version legal: the palette is a 193rd chain ENTRY, so it costs 512 more
+# delivered bytes and one more entry rather than 256 CPU word writes.
+pal_dma = (PACKED_BPF + PAL_BYTES) * w9 + CHAIN_CLK + B.DMA_CHAIN_CLK + AUD_CLK
+pal_cpu = (PACKED_BPF + PAL_BYTES) * w9 + blit[8] + pal_clk + AUD_CLK
+codec_9 = codec_bpf * w9 + codec_decode + AUD_CLK
+print("\n" + "=" * 78)
+print(f"""THE ANSWER, AT THE ONE RUNG THIS MACHINE CAN BE SHOWN TO RUN (W=9)
+
+  CODEC, gate container          {100*codec_9/FRAME_CLK:6.1f}% of the frame -- DOES NOT FIT
+  free / DMAC->GVRAM / PACKED    {100*free_packed_dma/FRAME_CLK:6.1f}% -- FITS, with {100-100*free_packed_dma/FRAME_CLK:.0f}% to spare
+  free / CPU-painted / PACKED    {100*free_packed_cpu/FRAME_CLK:6.1f}% -- FITS, with {100-100*free_packed_cpu/FRAME_CLK:.0f}% to spare
+
+  WITH THE PER-FRAME PALETTE CHARGED, which is what would ship:
+
+    DMAC-direct, palette on the CHAIN (62)     {100*pal_dma/FRAME_CLK:6.1f}% of the frame, {(PACKED_BPF+PAL_BYTES)*FPS/1024:.0f} KB/s
+    CPU-painted, palette written by the CPU    {100*pal_cpu/FRAME_CLK:6.1f}% of the frame, {(PACKED_BPF+PAL_BYTES)*FPS/1024:.0f} KB/s
+
+  The palette costs the same on the WIRE either way -- {PAL_BYTES} B a frame,
+  +{100*PAL_BYTES/PACKED_BPF:.1f}% -- and the wire is where this design is expensive.  The gap
+  between the two rows is the PAINT, not the palette.
+
+  What session 30 bought is smaller than either and is worth stating exactly:
+  {pal_clk:,.0f} CPU clocks of palette writing replaced by one more chain entry at
+  {B.DMA_CHAIN_CLK} clocks, a net {100*(pal_clk-B.DMA_CHAIN_CLK)/FRAME_CLK:.2f}% of a frame -- plus the structural half,
+  which is that the video path then contains no per-frame PAINT at all.  The
+  CPU still issues the READ(10) and starts the channel, and neither of those
+  is priced anywhere in this tree.
+
+  THE DECODER-FREE PACKED PLAYER FITS THE CLOCK BUDGET THAT THE CODEC MISSES.
+  That is not a small correction to 47.5, it is the reverse of the reason the
+  codec exists.  44.7 said it in advance and on a different cost model: "the
+  codec is not there to save CPU -- it is there to save the wire."  The
+  measured model agrees, and now says the CPU side is not merely affordable
+  but strictly cheaper WITHOUT the codec: at the floor, decoding {codec_bpf:,.0f} bytes
+  costs {100*(codec_bpf*w9+codec_decode)/FRAME_CLK:.0f}% of a frame and NOT decoding {PACKED_BPF:,} costs {100*(PACKED_BPF*w9+blit[8])/FRAME_CLK:.0f}%.
+
+  SO THE QUESTION IS ENTIRELY A MEDIUM QUESTION, and it has two halves:
+
+    1. {PACKED_BPF*FPS/1024:.0f} KB/s SUSTAINED, with no lever to pull.  ROADMAP B1 is
+       unmeasured; the 0.7-1.7 MB/s usually quoted for BlueSCSI on an X68000
+       is folklore with no published benchmark behind it.  {PACKED_BPF*FPS/1024:.0f} KB/s sits
+       inside that range, which is exactly why the range has to be measured
+       rather than cited.  A codec at 327 KB/s survives a slower answer; a
+       literal frame does not degrade, it drops.
+    2. {PACKED_BPF*FPS*GAME_S/1e9:.2f} GB for the whole game, against the codec's {codec_bpf*FPS*GAME_S/1e9:.2f} GB at the gate
+       recipe and ~{327*1024*GAME_S/1e9:.2f} GB at E7's target.  That is a packaging fact (C3),
+       not a performance one.
+
+  AND 47.4 STILL SITS OVER ALL OF IT.  Everything above assumes the layer is
+  visible while it is written.  If it is not, the packed player is a {100*blit[8]/FRAME_CLK:.0f}% duty
+  strobe at best and there is no version of it that is merely expensive.""")

tools/analysis/30_packed_container.py

@@ -0,0 +1,290 @@
+#!/usr/bin/env python3
+"""The PACKED container: does it hold, and what is the picture actually worth?
+
+    python3 tools/analysis/30_packed_container.py [packed.dlxp]
+            [--frames tmp/fr_singe] [--codec tmp/rc_fr_singe_scsi_span.dlx]
+
+ROADMAP K2.  Two jobs, and they are different kinds of claim.
+
+1. THE FORMAT HOLDS.  A packed record is written into the palette registers and
+   GVRAM by a DMA channel with no bounds test anywhere -- the channel has no
+   opinion about what it is copying (FINDINGS 62) -- so "the geometry is right"
+   is not a tidiness check, it is the whole of the container's correctness.
+   Round-trip, sector geometry, and the two reserved indices are gated here.
+
+2. THE PICTURE IS RE-DERIVED, and this is the number session 30 asked for.
+   FINDINGS 61.9 measured the packed player at 34.08 dB against the codec's
+   29.19 and filed TWO caveats: the quantiser was PIL's free 256-colour
+   MEDIANCUT rather than this project's builder, and the figure was quoted in
+   the RGB888 palette domain.  Both are paid here:
+
+     * `vq.frame_palette` is what ships it -- 254 colours, because the packed
+       layout spends index 0 on the transparency key and 255 on black (47.2).
+     * the GRB555+I WORD is charged.  A palette entry in a packed record is
+       already a hardware word; the display renders 5 bits a channel with one
+       shared LSB (23.3).  Every PSNR in this project's encoder is measured
+       upstream of that, so the codec is charged it here too and the comparison
+       stays like for like.
+
+   And the scene-palette CONTROL is built and scored, because "per-frame
+   palettes became legal" is the mechanism 61.9 credits and an unrun control is
+   an assumption.  The codec cannot take this row: every codeword it emits is an
+   index INTO `vq.scene_palette`, so 31.33 dB is its ceiling at any bitrate.
+
+WHAT THIS DOES NOT DO.  It does not put a packed frame on a machine -- that is
+K3, and the layout itself was already rendered pixel-exactly on both emulators
+in 47.2.  It does not price clocks: 29_packed_player.py owns that, off the
+MEASURED blit, and nothing here moves it.  And it settles nothing about the
+medium: 582.0 KB/s is geometry, and whether anything sustains it is B1.
+"""
+import argparse, glob, os, sys
+sys.path.insert(0, "tools/encoder")
+sys.path.insert(0, "tools/bench")
+import numpy as np
+from PIL import Image
+import vq as VQ
+import dlxp as P
+from dlx import DLX
+from dlxload import pack_palette
+
+ap = argparse.ArgumentParser()
+ap.add_argument("packed", nargs="?", default="tmp/packed_singe.dlxp")
+ap.add_argument("--frames", default="tmp/fr_singe")
+ap.add_argument("--mismatch-png", default=None,
+                help="write the 62.5 mismatch as a picture: correct render | the "
+                     "same frame under the NEXT frame's palette | the 24-bit "
+                     "source. A dB is not a look, and this claim is about a look.")
+ap.add_argument("--codec", default="tmp/rc_fr_singe_scsi_span.dlx",
+                help="the shipping container this replaces. Its PSNR is COMPUTED "
+                     "from its own bytes, not transcribed from docs (60.8).")
+a = ap.parse_args()
+
+fail = []
+d = P.DLXP(a.packed)            # every format invariant is checked in here
+print(f"{a.packed}: DLXP{d.version} {d.W}x{d.H} {d.fps}fps {d.nframes} frames")
+print()
+
+# --- 1. the format -----------------------------------------------------------
+print("THE FORMAT, and why each line is a gate and not a courtesy check:")
+print(f"  record {d.rec_bytes:,} B = {d.rec_bytes // P.SECTOR} sectors exactly, "
+      f"palette {d.pal_bytes} B "
+      f"{'LAST' if d.palette_last else 'first'}, picture {d.pic_bytes:,} B")
+print(f"  1.0 B/pixel: {d.pic_bytes} bytes carry {d.W * d.H} pixels "
+      f"(the unpacked path needs {2 * d.W * d.H:,})")
+
+zero = black = 0
+for f in range(d.nframes):
+    idx = d.indices(f)
+    # The channel copies bytes; a container whose interleave is a byte out does
+    # not fail, it paints. So the round trip is the assertion that the bytes in
+    # the record ARE the picture, in the order GVRAM wants them.
+    if P.pack_picture(idx).tobytes() != d._split(f)[1]:
+        fail.append(f"frame {f}: the record does not round-trip through the "
+                    f"interleave -- the container is not what it says it is")
+        break
+    zero += int((idx == 0).sum())
+    black += int((idx == 255).sum())
+if zero:
+    fail.append(f"index 0 appears in the picture {zero:,} times -- it is the "
+                f"TRANSPARENCY KEY of the top page and must stay unused (47.2)")
+print(f"  round-trip: {d.nframes} records unpack and re-pack byte-identical")
+print(f"  index 0 (transparency key) used {zero} times; "
+      f"index 255 (black) {black:,} times in the picture")
+
+# THE PICTURE'S wire, and it is the one this file is about. DLXP2 puts audio on
+# the same wire at a cadence (65.3, 67) and `d.kbps()` is both; what is asserted
+# here is that the PICTURE's share is still exactly geometry, because that is
+# 61.6's claim and a second stream is exactly the thing that could quietly
+# dilute it.
+kbps = d.video_kbps()
+geom = d.rec_bytes * d.fps / 1024
+if abs(kbps - geom) > 1e-6:
+    fail.append(f"wire {kbps} != geometry {geom}")
+print(f"  wire {kbps:.1f} KB/s = {d.rec_bytes:,} B x {d.fps} fps. FIXED. A codec's "
+      f"bitrate is a lever and a literal frame's is geometry (61.6)"
+      + (f"\n  ...and {d.audio_kbps():.2f} KB/s of audio rides beside it on the "
+         f"F={d.cad_f}/A={d.cad_a} cadence, for {d.kbps():.1f} KB/s total "
+         f"(tools/analysis/34_packed_audio.py)" if d.has_audio else ""))
+print()
+
+# --- 2. the picture ----------------------------------------------------------
+files = sorted(glob.glob(f"{a.frames}/f*.png"))[:d.nframes]
+if len(files) < d.nframes:
+    sys.exit(f"{a.frames}: {len(files)} frames, container has {d.nframes}")
+src = [np.asarray(Image.open(f).convert("RGB")) for f in files]
+
+
+def rendered(pal):
+    """RGB888 as the DISPLAY produces it, from the same maths the loader uses."""
+    return pack_palette(np.asarray(pal, np.uint8))[2]
+
+
+def score(name, pal_rgb, idx_frames, note=""):
+    """Two columns: the palette domain every encoder PSNR in this tree is
+    quoted in, and the hardware word the display actually renders."""
+    ren = rendered(pal_rgb)
+    p_pal = np.mean([VQ.psnr(s, np.asarray(pal_rgb)[i])
+                     for s, i in zip(src, idx_frames)])
+    p_hw = np.mean([VQ.psnr(s, ren[i]) for s, i in zip(src, idx_frames)])
+    print(f"  {name:<44s} {p_pal:6.2f}   {p_hw:6.2f}   {note}")
+    return p_pal, p_hw
+
+
+print("PSNR vs the 24-bit source, mean over frames:")
+print(f"  {'':<44s} {'RGB888':>6}   {'GRB555':>6}")
+
+codec_pal = codec_hw = None
+if os.path.exists(a.codec):
+    c = DLX(a.codec)
+    if c.nframes < d.nframes:
+        print(f"  (the codec container has {c.nframes} frames and this has "
+              f"{d.nframes} -- its row is skipped rather than compared over a "
+              f"different window)")
+    else:
+        # Its rate is printed with it because this is the GATE container -- the
+        # heaviest stream the encoder emits, `--kbps 280 --span-kbps 488
+        # --spans all` (check.sh) -- and NOT the 496.7 KB/s / 29.19 dB "current
+        # encode" of the README. Two containers, two numbers; a row that named
+        # neither would invite the difference to be read as a drift.
+        ckbps = sum(c.record_lengths()) * c.fps / c.nframes / 1024
+        codec_pal, codec_hw = score("CODEC, the GATE container", c.pal,
+                                    c.decode_all()[:d.nframes],
+                                    f"{ckbps:.1f} KB/s, "
+                                    f"{os.path.basename(a.codec)}")
+else:
+    print(f"  (no codec container at {a.codec} -- its row is skipped)")
+
+# The codec's CEILING: 256 colours, one palette for the scene, no VQ loss at
+# all. Not a rival, a bound -- no bitrate takes the codec past this row.
+ref, spal = VQ.scene_palette(src, reserve_black=True)
+sidx = VQ.palettise(src, ref)
+ceil_pal, ceil_hw = score("256c SCENE palette -- the CODEC'S CEILING",
+                          spal, sidx, "no bitrate crosses this")
+
+# The control for the mechanism 61.9 credits: same LAYOUT and the same 254
+# picture colours, one palette for the scene instead of one per frame.  It is
+# built to 255 with black reserved and then black is MOVED from 0 to 255, which
+# is the packed layout's convention (47.2) rather than the codec's -- so the
+# only variable between this row and the container's is per-frame vs scene-wide.
+cref, c255 = VQ.scene_palette(src, colors=255, reserve_black=True)
+cpal = np.vstack([np.zeros((1, 3), np.uint8), c255[1:],
+                  np.zeros((1, 3), np.uint8)])
+cidx = [np.where(i == 0, np.uint8(255), i)
+        for i in VQ.palettise(src, cref)]
+ctl_pal, ctl_hw = score("PACKED, 254c SCENE palette  [the CONTROL]", cpal, cidx)
+
+# And the container itself.  The right-hand column is read out of the CONTAINER'S
+# OWN BYTES -- `DLXP.render` unpacks the GRB555 words the record carries -- and
+# the left-hand one is recomputed from the encoder, because a packed record has
+# no RGB888 palette in it to score.  The two are tied together by a gate rather
+# than by trust: the palettes the encoder builds here must reproduce the
+# container's indices exactly, or the left column is describing a different file.
+pk_idx, pk_pal_rgb, mismatch, palbad = [], [], 0, 0
+for n, s in enumerate(src):
+    pal, idx = VQ.frame_palette(s)
+    if not np.array_equal(idx, d.indices(n)):
+        mismatch += 1
+    # And the WORD.  The encoder packed GRB555+I with `dlxload.pack_palette` and
+    # `DLXP.palette_rgb` unpacks it: two separate pieces of maths over the same
+    # 23.3 rule, and a container is the only place they meet. Required to agree,
+    # not assumed to -- a wrong shared LSB is a 1.96 dB bug that still renders.
+    if not np.array_equal(rendered(pal), d.palette_rgb(n)):
+        palbad += 1
+    pk_idx.append(idx)
+    pk_pal_rgb.append(pal)
+if palbad:
+    fail.append(f"{palbad} of {d.nframes} records carry palette words that do "
+                f"not unpack to the RGB the encoder packed -- pack_palette and "
+                f"DLXP.palette_rgb disagree about GRB555+I")
+if mismatch:
+    fail.append(f"{mismatch} of {d.nframes} frames re-quantise to different "
+                f"indices than the container holds -- the RGB888 column would "
+                f"be scoring a file that is not this one")
+pk_pal = np.mean([VQ.psnr(s, p[i]) for s, p, i in zip(src, pk_pal_rgb, pk_idx)])
+pk_hw = np.mean([VQ.psnr(s, d.render(f)) for f, s in enumerate(src)])
+print(f"  {'PACKED CONTAINER, 254c PER-FRAME':<44s} {pk_pal:6.2f}   {pk_hw:6.2f}   "
+      f"GRB555 read out of {os.path.basename(a.packed)}")
+print()
+print("  The right-hand column is the PLAYER'S number. Every PSNR this project")
+print("  has quoted -- 29.19, 31.33, 34.08 -- lives in the left one, upstream of")
+print("  the 5-bit hardware word (23.3), and 61.9's 34.08 is directly comparable")
+print("  to the packed row's left-hand entry and to nothing else.")
+print()
+
+# --- 3. what it means --------------------------------------------------------
+if codec_hw is not None:
+    print(f"  packed vs the codec gate container, as the DISPLAY renders both: "
+          f"{pk_hw - codec_hw:+.2f} dB")
+print(f"  packed vs the codec's CEILING:                             "
+      f"{pk_hw - ceil_hw:+.2f} dB")
+print(f"  what the PER-FRAME palette is worth (vs the control):      "
+      f"{pk_hw - ctl_hw:+.2f} dB")
+print(f"  what the GRB555 word costs the ceiling row:                "
+      f"{ceil_hw - ceil_pal:+.2f} dB")
+# 60.3 measured ONE reserved entry at 0.04 dB; the packed layout spends two.
+# Scored here at scene scale, where the control makes it a clean subtraction.
+print(f"  what the packed layout's TWO reserved entries cost:         "
+      f"{ctl_pal - ceil_pal:+.4f} dB  (256c -> 254c, scene palette, RGB888)")
+print()
+
+# The three claims 61.9 makes, restated as gates. A tree where any of these
+# flipped has a different answer to ROADMAP K and should say so out loud.
+if codec_hw is not None and pk_hw <= codec_hw:
+    fail.append(f"the packed container is {pk_hw:.2f} dB and the codec it "
+                f"replaces is {codec_hw:.2f} -- 61.9's headline is inverted")
+if pk_hw <= ceil_hw:
+    fail.append(f"the packed container is {pk_hw:.2f} dB and the codec's own "
+                f"CEILING is {ceil_hw:.2f} -- the per-frame palette bought "
+                f"nothing, and 61.9's reason for building this branch is gone")
+if pk_hw <= ctl_hw:
+    fail.append(f"per-frame {pk_hw:.2f} dB is not better than the SCENE-palette "
+                f"control {ctl_hw:.2f} -- the mechanism 61.9 credits is absent")
+
+# --- 4. FINDINGS 62.5, which needed this encoder to exist ---------------------
+# 62.5 filed the chain's order -- palette first or 193rd -- as a free choice with
+# a visible consequence, and said the severity "depends on how much the palette
+# moves between consecutive frames, which is a property of the encoder K2 has
+# not been written yet". It is written now, so the number exists.
+#
+# The mismatch is a WIPE, not a flash: rows arrive top to bottom, so at any
+# instant part of the screen is right. What is bounded here is the WORST
+# instant of each order -- the whole screen wrong -- which is the start of the
+# transfer for palette-first and the end of it for palette-last. The mean over
+# the transfer is about half of each, because the wipe is linear in rows.
+print("FINDINGS 62.5 PRICED -- palette FIRST vs LAST, at the worst instant of each:")
+churn = np.mean([int((d.palette_words(n) != d.palette_words(n - 1)).sum())
+                 for n in range(1, d.nframes)])
+first = np.mean([VQ.psnr(src[n - 1], d.palette_rgb(n)[d.indices(n - 1)])
+                 for n in range(1, d.nframes)])
+last = np.mean([VQ.psnr(src[n], d.palette_rgb(n - 1)[d.indices(n)])
+                for n in range(1, d.nframes)])
+correct = np.mean([VQ.psnr(src[n], d.render(n)) for n in range(1, d.nframes)])
+print(f"  palette entries that CHANGE frame to frame: {churn:.1f} of 256 "
+      f"({100 * churn / 256:.0f}%) -- a per-frame palette is not a small delta")
+print(f"  palette FIRST, old rows under the new palette: {first:6.2f} dB "
+      f"({first - correct:+.2f} against the correct pairing)")
+print(f"  palette LAST,  new rows under the old palette: {last:6.2f} dB "
+      f"({last - correct:+.2f})")
+print(f"  the container is currently palette "
+      f"{'LAST' if d.palette_last else 'FIRST'} (dlxp.py, --palette-last)")
+if a.mismatch_png:
+    # The frame whose mismatch is CLOSEST TO THE MEAN, so the picture is not an
+    # outlier picked to make the point look worse than the number.
+    mis = np.array([VQ.psnr(src[n - 1], d.palette_rgb(n)[d.indices(n - 1)])
+                    for n in range(1, d.nframes)])
+    n = int(np.argmin(np.abs(mis - mis.mean()))) + 1
+    z = lambda x: np.repeat(np.repeat(x, 2, 0), 2, 1)
+    gap = np.full((d.H * 2, 6, 3), 30, np.uint8)
+    Image.fromarray(np.concatenate(
+        [z(d.render(n - 1)), gap, z(d.palette_rgb(n)[d.indices(n - 1)]), gap,
+         z(src[n - 1])], axis=1)).save(a.mismatch_png)
+    print(f"  wrote {a.mismatch_png}: frame {n-1} correct | frame {n-1} under "
+          f"frame {n}'s palette ({mis[n-1]:.2f} dB) | the 24-bit source")
+print("  Both are one paint, and both are MOOT if buffer mode blanks the layer")
+print("  (47.4/B2). This bounds the cost of being wrong; it does not decide it,")
+print("  because dB over a whole frame is not what an eye sees in a wipe.")
+print()
+
+for x in fail:
+    print("FAIL " + x)
+sys.exit(1 if fail else 0)

tools/analysis/31_display_duty.py

@@ -0,0 +1,215 @@
+#!/usr/bin/env python3
+"""HOW LONG IS THE PICTURE ACTUALLY ON SCREEN?  ROADMAP K3, FINDINGS 64.
+
+    python3 tools/analysis/31_display_duty.py [container.dlxp] [--rate KB/s ...]
+
+THE QUESTION NOTHING IN THIS TREE HAD ASKED.  Every budget in docs/FINDINGS.md
+asks what a frame COSTS -- clocks, bus cycles, bytes on the wire.  Session 32
+built the packed player and ran it (src/player/packed.s), and the run reported a
+number no budget has a column for: the write window was open on 99.5% of the
+host frames, so the graphics layer was DARK for 99.5% of the scene.  Every frame
+was pixel-exact and almost none of them was visible.
+
+WHY THAT IS ARITHMETIC AND NOT AN EMULATOR ARTEFACT.  256-colour GVRAM masks the
+high byte of every write unless CRTC R20 bit 11 is set (46.5/47.1), and the
+packed layout's whole 1.0 B/pixel claim is that one word carries two pixels --
+so a packed write REQUIRES the bit.  If buffer mode blanks the layer while the
+bit is set (47.4/ROADMAP B2 -- MAME says it does, and 48.1's prior leans that
+way), then the layer is dark for exactly as long as the window is open, and for
+a DMAC-direct player the window is open for the whole transfer.  There is no
+second page to hide behind: the packed layout SPENDS both 256-colour pages,
+which is the same fact that made a frame one channel start (FINDINGS 62).
+
+  dark fraction of a slot  =  record bytes / (data-phase rate x slot)
+
+AND THE RATE IN THAT EXPRESSION IS THE BURST RATE, NOT THE SUSTAINED ONE.  This
+is the correction the session had to make to itself.  The container's 582.0 KB/s
+is a SUSTAINED requirement -- it decides whether record i arrives before slot i.
+The dark fraction is set by how fast bytes move DURING THE DATA PHASE, which for
+a drive with a read-ahead cache can be several times the sustained figure.  The
+two are independent, and a medium can pass one and fail the other:
+
+  sustained >= 582.0 KB/s   or frames arrive late           (B1, known)
+  data phase >> 582.0 KB/s  or the frame is never displayed (NEW, and B1 has
+                                                             no test for it)
+
+THE OTHER PLAYER IN THE FAMILY DOES NOT HAVE THIS PROPERTY.  A packed player
+that DMAs the record into RAM and paints it with the CPU opens the window only
+for the paint -- tools/bench/blit.s V8, MEASURED, not assumed -- which is a
+fixed share of the slot no matter what the medium does.  It costs more clocks
+and 49 KB of RAM and it buys a picture that is on screen.  FINDINGS 61.5 already
+priced both in CLOCKS and ranked DMAC-direct first; this file is the column that
+was missing from that table, and it reverses the ranking under B2-blanks.
+"""
+import argparse, os, re, sys
+sys.path.insert(0, "tools/encoder")
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+from dlxp import DLXP
+import buscost as B
+
+CPUHZ = 10e6                       # stock X68000, MAME 0.277 x68k.cpp:1133
+
+ap = argparse.ArgumentParser()
+ap.add_argument("container", nargs="?", default="tmp/packed_singe.dlxp")
+ap.add_argument("--blit-log", default="tmp/blit_v8.log",
+                help="tools/bench/blit.lua's log -- the MEASURED packed paint. "
+                     "Not a constant in this file: 47.6.1 filed the movem shape "
+                     "as an assumption and session 29 measured it, and a second "
+                     "copy of a measured number is how one of them goes stale.")
+ap.add_argument("--rate", type=float, nargs="*", default=None,
+                help="data-phase rates to price, KB/s. REQUIRED to mean "
+                     "anything: this project has no delivery figure and will "
+                     "not default to one (FINDINGS 50).")
+ap.add_argument("--run-log", default="tmp/packed_free_steal.log",
+                help="a free-running tools/bench/packed.lua log, for the "
+                     "measured corroboration section")
+a = ap.parse_args()
+
+d = DLXP(a.container)
+SLOT_S = 1.0 / d.fps
+FRAME_CLK = CPUHZ * SLOT_S
+wire = d.kbps()                                    # KB/s, and fixed by geometry
+
+blit = {}
+if os.path.exists(a.blit_log):
+    for line in open(a.blit_log, errors="replace"):
+        m = re.search(r"V(\d+)\s+(\d+) cyc", line)
+        if m:
+            blit[int(m.group(1))] = int(m.group(2))
+if 8 not in blit:
+    sys.exit(f"{a.blit_log} has no V8 result. The packed paint is a MEASUREMENT "
+             f"(tools/bench/blit.lua) and this tool will not substitute a "
+             f"constant for it -- run the blit bench, or point --blit-log at "
+             f"its log.")
+PAINT_CLK = blit[8]
+PAINT_FRAC = PAINT_CLK / FRAME_CLK
+
+print(f"""THE CONTAINER (tools/encoder/dlxp.py)
+  {a.container}: {d.W}x{d.H} {d.fps} fps, {d.nframes} frames
+  record {d.rec_bytes:,} B = {d.rec_bytes // 512} sectors, palette \
+{'LAST' if d.palette_last else 'FIRST'}
+  slot {SLOT_S*1000:.2f} ms = {FRAME_CLK:,.0f} clocks
+  wire {wire:.1f} KB/s -- FIXED by geometry. A codec's bitrate is a lever and a
+       literal frame's is not (61.6), so nothing an encoder does moves this.
+
+THE TWO PACKED PLAYERS, and the difference is WHEN the write window is open
+
+  A. DMAC-DIRECT (src/player/packed.s, ROADMAP K3, and the one that is built).
+     One channel start, 193 destinations, the CPU halted or nearly. The window
+     must be open for the WHOLE data phase, because the channel writes when the
+     bytes arrive and the CPU cannot know when that is -- and a packed write
+     that lands with the bit clear is masked to its low byte and silently wrong.
+  B. DMA-TO-RAM + CPU PAINT. The record lands in RAM with the window shut; the
+     68000 then paints it with the packed movem blit. The window is open for the
+     PAINT and nothing else: {PAINT_CLK:,} clocks, {100*PAINT_FRAC:.1f}% of a slot,
+     MEASURED by tools/bench/blit.lua (V8), and INDEPENDENT of the medium.
+
+  Under 47.4-blanks the dark interval IS the open window, so B is on screen for
+  {100*(1-PAINT_FRAC):.1f}% of every slot at ANY rate that delivers the record at all,
+  and A's visibility is a function of the rate.
+""")
+
+rates = a.rate
+if not rates:
+    print("""NO RATES GIVEN, so no table. This project retired its delivery
+constant outright (FINDINGS 50, USER DECISION) and every tool requires an
+explicit rate; a default here would be the same mistake in a new place. Pass
+--rate with the figures you want priced. The three thresholds already derived
+elsewhere, for reference and NOT as defaults:
+  453.6 KB/s  the DLX5 codec gate container needs no prefill (49.5/60)
+  576.0 KB/s  a packed container with no per-frame palette (61.5)
+  582.0 KB/s  THIS container, palette included (63)
+and note that all three are SUSTAINED figures. The dark fraction below is set by
+the DATA-PHASE rate, which is a different measurement nothing has taken.""")
+    sys.exit(0)
+
+print("A's VISIBILITY, against the DATA-PHASE rate\n")
+print(f"  {'data phase':>12} | {'transfer':>9} | {'window open':>11} | "
+      f"{'PICTURE ON SCREEN':>17} | vs B")
+print(f"  {'KB/s':>12} | {'ms':>9} | {'% of slot':>11} | "
+      f"{'% of slot':>17} |")
+print("  " + "-"*12 + "-+-" + "-"*9 + "-+-" + "-"*11 + "-+-" + "-"*17 + "-+-----")
+for R in sorted(rates):
+    t_ms = d.rec_bytes / (R * 1024) * 1000
+    openf = min(1.0, t_ms / (SLOT_S * 1000))
+    vis = 1.0 - openf
+    verdict = ("A wins" if vis > 1 - PAINT_FRAC else
+               "B wins" if vis < 1 - PAINT_FRAC else "equal")
+    late = " LATE" if R < wire else ""
+    print(f"  {R:>12.1f} | {t_ms:>9.2f} | {100*openf:>11.1f} | "
+          f"{100*vis:>17.1f} | {verdict}{late}")
+
+# The crossover, stated as a rate rather than left to be read off the table: it
+# is the one number in here a hardware acceptance test can be written against.
+cross = d.rec_bytes / (PAINT_FRAC * SLOT_S) / 1024
+print(f"""
+  A and B show the picture for the same share of the slot at a data-phase rate
+  of {cross:,.0f} KB/s. Below that, THE PLAYER WITH THE CPU IN THE LOOP IS ON
+  SCREEN LONGER than the one without it -- which is the reverse of FINDINGS
+  61.5's ranking, and 61.5 is not wrong: it ranked them in CLOCKS, and this is
+  the column that table does not have.
+
+  {cross:,.0f} KB/s is {cross/wire:.1f}x the container's own wire. So a medium that exactly
+  meets the sustained requirement puts the DMAC-direct player's picture on
+  screen for {100*max(0.0, 1-wire/wire):.0f}% of every slot: it delivers every frame, on time,
+  pixel-exact, and displays none of them.""")
+
+print(f"""
+THE CPU SIDE, so the trade is priced on both axes (FINDINGS 61.5's ladder)
+
+  W is clocks stolen per delivered byte. Only the dual-address rungs have a code
+  path on this machine (59.2), and 9 is the floor: a 4-clock read of the device
+  plus a 5-clock write to memory.
+""")
+print(f"  {'W':>3} | {'A: DMAC-direct':>15} | {'B: DMA + CPU paint':>19}")
+print("  " + "-"*3 + "-+-" + "-"*15 + "-+-" + "-"*19)
+AUDIO = B.ADPCM_BYTES_PER_S / d.fps * B.ADPCM_CLK_BYTE_BEST
+for W in (5, 9, 12, 16, 19):
+    xfer = d.rec_bytes * W
+    ca = (xfer + AUDIO) / FRAME_CLK
+    cb = (xfer + AUDIO + PAINT_CLK) / FRAME_CLK
+    print(f"  {W:>3} | {100*ca:>14.1f}% | {100*cb:>18.1f}%")
+print(f"""
+  Both include the audio DMA at {AUDIO:,.0f} clocks a frame ({100*AUDIO/FRAME_CLK:.2f}%), charged from
+  the IPL ROM's own channel-3 setup (21_iplrom_dmac.py, 52.5). Neither includes
+  a decoder, because neither has one.
+
+  So B costs the paint -- {100*PAINT_FRAC:.1f}% of a frame -- and TWO record buffers,
+  {2*d.rec_bytes:,} B of RAM. Two and not one: at any rate near the wire the delivery
+  of record i+1 occupies most of the slot the paint of record i happens in, so
+  they overlap by construction. On a 2 MB machine that is {200*d.rec_bytes/(2*1024*1024):.1f}% of memory and
+  it is the resource this design has spare -- the ring the packed branch deleted
+  was 256 KB (FINDINGS 49). That is what a visible picture costs if 47.4 blanks.""")
+
+# ---- the measured corroboration.  It is a SEPARATE section and it is bounded
+# on purpose: MAME's device models carry no transfer timing (docs/BENCHMARK.md,
+# 42.5), so the run cannot supply a rate for the table above -- what it can do
+# is show that the mechanism is real and that the arithmetic predicts it.
+if os.path.exists(a.run_log):
+    txt = open(a.run_log, errors="replace").read()
+    m_rate = re.search(r"record lands in ([\d.]+) ms, i\.e\. ([\d.]+) KB/s", txt)
+    m_open = re.search(r"WRITE WINDOW OPEN on (\d+) of (\d+) host frames", txt)
+    if m_rate and m_open:
+        ms, kbps = float(m_rate.group(1)), float(m_rate.group(2))
+        op, tot = int(m_open.group(1)), int(m_open.group(2))
+        pred = min(1.0, d.rec_bytes / (kbps * 1024) / SLOT_S)
+        print(f"""
+MEASURED, on the emulated machine (tools/bench/packed_run.sh, free-running)
+
+  {a.run_log}: a {d.rec_bytes:,} B record landed in {ms:.2f} ms = {kbps:.1f} KB/s, and the
+  write window was open on {op} of {tot} host frames = {100*op/tot:.1f}%.
+  The expression above predicts {100*pred:.1f}% at that rate.
+
+  THIS IS NOT A RATE MEASUREMENT AND {kbps:.0f} KB/s IS NOT A MEDIUM. MAME's
+  device models carry no transfer timing (42.5); the figure is a property of the
+  apparatus. What the run DOES establish is that the mechanism is the one the
+  arithmetic describes -- and one thing more that no arithmetic could have
+  given: the DMAC CONFIGURATION DOES NOT MOVE IT. Held and stealing delivered
+  the same record within 0.5% of each other, so what a channel configuration
+  buys is who owns the CPU, not when the picture appears.""")
+else:
+    print(f"""
+NO MEASURED SECTION: {a.run_log} is absent. Run
+tools/bench/packed_run.sh to produce it. The arithmetic above stands without
+it -- it is geometry -- but the run is what showed the effect was there to be
+derived at all.""")

tools/analysis/32_audio_wire.py

@@ -0,0 +1,227 @@
+#!/usr/bin/env python3
+"""WHAT DOES AUDIO DO TO THE CONTAINER?  ROADMAP P6, the half that is not the bus.
+
+    python3 tools/analysis/32_audio_wire.py [packed.dlxp] [--audio tmp/au_singe.raw]
+                                            [--rate KB/s ...]
+
+Session 20 (FINDINGS 52) closed the bus half of P6: a second DMA consumer at
+7,812.5 B/s is 1.25%..1.48% of a frame, about 4% of what the decoder leaves, and
+the 7.8 kB/s figure survived with a unit correction.  ROADMAP P6 then says, in
+as many words, that EVERYTHING ELSE in the item is open: extraction, an encoder,
+the container interleave, and what a second stream does to `wire` and therefore
+to `pipe - wire` and therefore to 51.3's refill climb.
+
+This file is the container interleave and the wire.  It is arithmetic over the
+real container's real geometry -- no MAME run, no board.
+
+THE THING THAT MAKES IT INTERESTING, and it is a property of DLXP1 rather than
+of audio: **a packed container has no index and cannot have one.**  A record's
+address is `LBA0 + i*97` because a literal frame's length is geometry (FINDINGS
+63, 64.1).  Audio is a stream at a rate that has nothing to do with the frame
+rate, so the naive interleave -- give record i the audio bytes belonging to slot
+i -- makes records VARIABLE LENGTH, and the moment records are variable length
+the format needs an index and stops being the format.
+
+So the interleave has to be a FIXED CADENCE: every F frames, A whole sectors of
+audio, placed between records.  Then
+
+    LBA(i) = LBA0 + i*RECSEC + floor(i/F)*A
+
+which is still two multiplies and a divide -- arithmetic, no index, nothing
+walked -- and the only cost is that A*512 must be at least F frames' worth of
+audio, so the padding is whatever A*512 exceeds it by.  Choosing (F, A) is a
+rational-approximation problem and the answer is NOT the obvious cadence.
+"""
+import argparse, os, sys
+from fractions import Fraction
+sys.path.insert(0, "tools/encoder")
+sys.path.insert(0, "tools/analysis")
+import buscost as B
+from dlxp import DLXP, SECTOR
+
+ap = argparse.ArgumentParser()
+ap.add_argument("container", nargs="?", default="tmp/packed_singe.dlxp")
+ap.add_argument("--audio", default="tmp/au_singe.raw",
+                help="raw s16le mono at the chip rate, from extract_audio.py")
+ap.add_argument("--codec", default="tmp/rc_fr_singe_scsi_span.dlx",
+                help="the codec container, for the same arithmetic on the other branch")
+ap.add_argument("--rate", type=float, nargs="*",
+                default=[453.6, 500.0, 582.0, 600.0, 650.0, 700.0],
+                help="explicit sustained delivery rates, KB/s")
+a = ap.parse_args()
+
+d = DLXP(a.container)
+SLOT_S = 1.0 / d.fps
+FRAME_CLK = B.CPU_HZ * SLOT_S if hasattr(B, "CPU_HZ") else 10_000_000 * SLOT_S
+RECSEC = d.rec_bytes // SECTOR
+
+print(f"""
+=== THE STREAM =========================================================
+The chip is an MSM6258V on an 8 MHz clock and it has three rates and no
+others.  Every budget in this tree is written against the first one.""")
+
+RATES = {512: 15625.0, 768: 8_000_000/768, 1024: 7812.5}
+print(f"\n  {'divisor':>8} {'samples/s':>11} {'bytes/s':>10} {'B per 1/%d s slot' % d.fps:>19}  exact?")
+for div, hz in RATES.items():
+    bps = hz / 2
+    per = bps / d.fps
+    print(f"  8MHz/{div:<4} {hz:11,.1f} {bps:10,.1f} {per:19,.4f}  "
+          f"{'yes' if per == int(per) else 'NO -- a remainder, like the frame clock (54)'}")
+
+HZ = 15625.0
+AU_BPS = HZ / 2                       # 4 bits a sample, two samples to a byte
+AU_FRAME = AU_BPS / d.fps             # 651.0416... B, and the point is the dots
+
+print(f"""
+  The shipping rate's per-slot figure is {AU_FRAME:,.4f} B and it is NOT an
+  integer -- 8 MHz / 512 / 2 / {d.fps} has a 12 in the denominator that 2**k
+  cannot clear.  That is the same shape as FINDINGS 54's frame clock: what a
+  player carries is a remainder, not a count, and a container that rounds it
+  either drifts or underruns.""")
+
+if os.path.exists(a.audio):
+    n16 = os.path.getsize(a.audio) // 2
+    secs = n16 / HZ
+    print(f"""
+  MEASURED, on the window this project gates everything on (00223 @539.4s,
+  {secs:.3f} s, tools/encoder/extract_audio.py):
+    {n16:,} samples -> {n16//2:,} B of ADPCM = {n16/2/secs:,.1f} B/s
+    which is {AU_BPS:,.1f} to the byte, so the rate is the rate.""")
+
+print(f"""
+=== THE INTERLEAVE, AND WHY THE OBVIOUS CADENCE IS THE WRONG ONE =======
+A packed record is {d.rec_bytes:,} B = {RECSEC} sectors EXACTLY and its address is
+arithmetic.  Audio rides between records at a fixed cadence -- every F frames,
+A whole sectors -- so that LBA(i) stays arithmetic.  A must satisfy
+
+    A * {SECTOR} >= F * {AU_FRAME:,.4f}          i.e.  A/F >= {Fraction(int(AU_BPS*2), int(2*SECTOR*d.fps))} = {AU_FRAME/SECTOR:.9f}
+
+and everything above that ratio is PADDING that the wire pays for and nothing
+plays.  Here is the whole small-F space, best A for each F:""")
+
+target = Fraction(int(round(AU_BPS * 2)), 2 * SECTOR * d.fps)   # sectors per frame, exact
+
+rows, floor = [], None
+for F in range(1, 241):
+    A = -(-(target.numerator * F) // target.denominator)        # ceil(F * target)
+    have, need = A * SECTOR, F * AU_FRAME
+    waste = (have - need) / need
+    add = have / F * d.fps / 1024        # what the cadence puts on the wire, KB/s
+    rows.append((waste, F, A, have, need, add))
+
+print(f"\n  {'F':>4} {'A':>4} {'A*512 B':>10} {'needs':>12} {'padding':>9} {'waste':>7}"
+      f"  {'wire adds':>10}  {'player RAM':>11}")
+seen = None
+for waste, F, A, have, need, add in rows:
+    show = F <= 4 or seen is None or waste < seen - 1e-12
+    if seen is None or waste < seen: seen = waste
+    if show:
+        print(f"  {F:4d} {A:4d} {have:10,} {need:12,.1f} {have-need:9,.1f} "
+              f"{100*waste:6.2f}% {add:9.2f} KB/s  {have:9,} B")
+
+best = sorted(rows)
+w, F, A, have, need, add = best[0]
+f1 = next(x for x in rows if x[1] == 1)
+print(f"""  THE FLOOR OF THAT SWEEP is F={F}, A={A}: {100*w:.3f}% padding, {add:.2f} KB/s of
+  wire for {AU_BPS/1024:.2f} KB/s of audio.
+
+  THE OBVIOUS CADENCE IS THE WORST ONE.  F=1 -- one audio lump per record, which
+  is what "interleave the audio into the frame" means if nobody does the
+  arithmetic -- needs A={f1[2]} and costs {100*f1[0]:.1f}% padding: {AU_FRAME:,.1f} B rounded up to
+  {f1[3]:,}, so {f1[3]-AU_FRAME:,.1f} B of every record is nothing at all, and the wire pays
+  {f1[5]:.2f} KB/s for {AU_BPS/1024:.2f} KB/s of audio.  That is {f1[5]-add:.2f} KB/s thrown away for
+  no reason but the cadence.
+
+=== WHAT IT DOES TO THE WIRE ===========================================""")
+
+vid_kbs = d.video_kbps()
+for label, cad in (("F=1  (one lump a record)", f1), (f"F={F} (the floor)", best[0])):
+    tot = vid_kbs + cad[5]
+    print(f"  {label:26s} video {vid_kbs:7.1f} + audio {cad[5]:5.2f} = {tot:7.1f} KB/s "
+          f"({100*(tot/vid_kbs-1):+.2f}%)")
+
+print(f"""
+  And this is what B1's acceptance test becomes.  The packed container's
+  sustained requirement was {vid_kbs:.1f} KB/s SILENT (FINDINGS 61.5, 63) and it is
+  {vid_kbs + add:.1f} KB/s with sound.  A literal frame's bitrate is geometry and cannot
+  be talked down; the audio on top of it is {add:.2f} KB/s and can only be talked down
+  by choosing a worse chip rate.""")
+
+f11 = next(x for x in rows if x[1] == 11)
+print(f"""
+  AND THE CADENCE HAS A SECOND PRICE, WHICH IS RAM.  A cadence of F frames means
+  the player is holding F frames of audio, and holding it TWICE -- the channel
+  fills lump n+1 while the chip drains lump n, the same reason K4 needs two
+  record buffers (64.2).  So the floor of the sweep is not the answer:
+
+    F={f1[1]:<3} {f1[3]:>7,} B a lump, {2*f1[3]:>7,} B held   {100*f1[0]:6.2f}% padding   {f1[5]:5.2f} KB/s
+    F={f11[1]:<3} {f11[3]:>7,} B a lump, {2*f11[3]:>7,} B held   {100*f11[0]:6.2f}% padding   {f11[5]:5.2f} KB/s   <- the pick
+    F={F:<3} {have:>7,} B a lump, {2*have:>7,} B held   {100*w:6.2f}% padding   {add:5.2f} KB/s
+
+  F={f11[1]} buys {100*(f1[0]-f11[0]):.1f} points of padding for {2*f11[3]-2*f1[3]:,} B of RAM, and F={F} buys the
+  last {100*(f11[0]-w):.2f} of a point for {2*have-2*f11[3]:,} B more.  On a machine where K4 already
+  wants 99,328 B for two record buffers, the second trade is not one.
+
+=== THE ASYMMETRY: THE CODEC CONTAINER PAYS NONE OF THIS ===============""")
+
+if os.path.exists(a.codec):
+    sys.path.insert(0, "tools/encoder")
+    from dlx import DLX
+    c = DLX(a.codec)
+    lens = c.record_lengths() if callable(getattr(c, "record_lengths", None)) else c.record_lengths
+    cwire = sum(lens) / len(lens) * c.fps / 1024
+    print(f"""  {os.path.basename(a.codec)}: {c.nframes} records, index {'PRESENT' if c.has_index else 'absent'},
+  records already VARIABLE ({min(lens):,}..{max(lens):,} B, mean {sum(lens)/len(lens):,.0f}) and
+  sector-aligned since DLX5 (60.1).  A container that already carries an index
+  and already has variable records can put EXACTLY {AU_FRAME:,.1f} B of audio in record i
+  and pad only to the sector it was going to pad to anyway -- so its audio
+  padding is not 57.3% and not 1.11%, it is ZERO, and its wire goes
+  {cwire:.1f} -> {cwire + AU_BPS/1024:.1f} KB/s ({100*(AU_BPS/1024)/cwire:+.2f}%).
+
+  THAT IS THE FIRST COST THIS PROJECT HAS FOUND FOR THE PACKED BRANCH'S OWN
+  SIMPLIFICATION.  "A record's length is geometry, so there is no index and none
+  can be needed" (63, 64.1) is what makes the packed player a page of arithmetic
+  instead of a parser -- and it is exactly the property that makes a second
+  stream at an unrelated rate cost padding, a cadence, and a buffer.  It is a
+  small cost ({f11[5]-AU_BPS/1024:.2f} KB/s at the pick, {2*f11[3]:,} B of RAM) and it is not zero, and
+  nothing in FINDINGS 61-64 predicted it.""")
+else:
+    print(f"  SKIPPED: no codec container at {a.codec}")
+
+print(f"""
+=== WHAT IT DOES TO SLACK (51.3) =======================================
+Slack is ACCUMULATED out of pipe - wire, so a second consumer does not cost a
+fixed amount -- it costs the accumulation rate, and what a branch point costs is
+set by that (51.3, 55.4).  Silent vs sounded, at explicit rates:
+
+  {'pipe':>8}  {'silent':>14}  {'sounded':>14}   what a second of play banks""")
+for kbps in a.rate:
+    s_sl, a_sl = kbps - vid_kbs, kbps - (vid_kbs + add)
+    def fmt(x): return f"{x:+8.1f} KB/s" if x >= 0 else f"{x:+8.1f} KB/s"
+    print(f"  {kbps:8.1f}  {fmt(s_sl):>14}  {fmt(a_sl):>14}   "
+          + ("both starve" if a_sl < 0 and s_sl < 0
+             else "SOUND IS WHAT BREAKS IT" if s_sl >= 0 > a_sl
+             else f"{a_sl/s_sl*100:.0f}% of the silent rate" if s_sl > 0 else ""))
+
+AUCLK_LO = AU_FRAME * B.ADPCM_CLK_BYTE_BEST
+AUCLK_HI = AU_FRAME * B.ADPCM_CLK_BYTE_WORST
+print(f"""
+=== AND WHAT IT DOES TO THE FRAME (the half session 20 already closed) ==
+  {AU_FRAME:,.1f} B a slot at {B.ADPCM_CLK_BYTE_BEST}..{B.ADPCM_CLK_BYTE_WORST} clocks a byte (the IPL ROM's OWN channel-3
+  configuration, read out of the ROM by 21_iplrom_dmac.py, not chosen here) is
+  {AUCLK_LO:,.0f}..{AUCLK_HI:,.0f} clocks = {100*AUCLK_LO/FRAME_CLK:.2f}%..{100*AUCLK_HI/FRAME_CLK:.2f}% of a {SLOT_S*1000:.2f} ms slot.
+  That reproduces FINDINGS 52 exactly, which is the point of printing it.
+
+  THE INTERACTION 52 COULD NOT HAVE HAD is with 64.2's write window.  A
+  DMAC-direct packed player holds the GVRAM window open for the whole data
+  phase, and an audio channel stealing the bus during that phase makes the phase
+  LONGER -- so audio does not merely cost clocks, it costs DARKNESS:
+
+    extra dark per slot = {100*AUCLK_LO/FRAME_CLK:.2f}%..{100*AUCLK_HI/FRAME_CLK:.2f}% of the slot, on top of
+                          record/(burst x slot), which is already 1.0 at the wire
+
+  It is small against a dark fraction that is already 1.0, and it is not small
+  against K4's {100*227553/FRAME_CLK:.1f}% paint.  For the CPU-painted player the audio steals
+  from the paint and not from the picture, which is the third time this session
+  the two players have ranked differently on a column that is not clocks.
+""")

tools/analysis/33_adpcm_model.py

@@ -0,0 +1,123 @@
+#!/usr/bin/env python3
+"""What the chip's own decoder model costs the encoder.  ROADMAP P6a, after it.
+
+tools/bench/adpcm_run.sh MEASURED four things about the MSM6258 as this machine
+models it, and tools/encoder/adpcm.py had a different value for every one:
+
+    axis      encoder default   the chip      how it was measured
+    feed      both-high-first   both-LOW-first
+    formula   shift             terms         1,678 samples, sample-exact
+    clamp     12-bit            10-bit        one model of sixteen matched
+    init      0                 -2
+
+This file prices them, on the same ten seconds of the same stream every audio
+figure in this project is quoted against (tmp/au_singe.raw, FINDINGS 65).  It
+takes an explicit source file rather than defaulting to one, for the same reason
+every rate in this tree is an explicit argument (FINDINGS 50).
+
+THE ONE THAT IS NOT A UNIT SLIP is the CLAMP.  The other three are conventions:
+get one wrong and the decode is wrong, get it right and nothing is lost.  A
+10-bit accumulator is a smaller container, and it is INSIDE the recursion -- the
+predictor cannot represent what will not fit -- so it costs SNR even when the
+encoder knows about it and encodes for it.  That is a ceiling on this format on
+this machine and it is not recoverable by encoding harder.
+"""
+import math, os, sys
+sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "encoder"))
+import adpcm
+
+RAW = sys.argv[1] if len(sys.argv) > 1 else "tmp/au_singe.raw"
+CHIP = dict(variant="terms", order="low", bits=10, init=-2)
+ENC  = dict(variant="shift", order="high", bits=12, init=0)
+
+
+def snr_db(ref, got):
+    num = sum(float(s) * s for s in ref)
+    den = sum((float(a) - b) ** 2 for a, b in zip(ref, got))
+    if den == 0: return float("inf")
+    return 10.0 * math.log10(num / den) if num else float("-inf")
+
+
+def main():
+    import struct
+    if not os.path.exists(RAW):
+        print(f"no {RAW} -- run tools/encoder/extract_audio.py first")
+        return 2
+    pcm = struct.unpack("<%dh" % (os.path.getsize(RAW) // 2), open(RAW, "rb").read())
+    src12 = [max(-2048, min(2047, x >> 4)) for x in pcm]
+    print(f"{RAW}: {len(src12):,} samples, peak {max(abs(v) for v in src12)} "
+          f"in 12-bit units")
+
+    print()
+    print("1. THE COST OF ENCODING FOR THE WRONG CHIP, all four axes at once")
+    print("   Encode under the encoder's defaults; play it on the chip.  The")
+    print("   nibble ORDER is not a decode parameter -- it decides which nibble")
+    print("   of each byte the chip takes -- so it is applied by re-reading the")
+    print("   encoder's own packed bytes the way the chip reads them.")
+    nib = adpcm.encode(src12, ENC["variant"], init=ENC["init"], bits=ENC["bits"])
+    same = adpcm.decode(nib, ENC["variant"], init=ENC["init"], bits=ENC["bits"])
+    data = adpcm.pack(nib, ENC["order"])
+    asread = list(adpcm.unpack(data, len(nib), CHIP["order"]))
+    cross = adpcm.decode(asread, CHIP["variant"], init=CHIP["init"], bits=CHIP["bits"])
+    print(f"   encoded and decoded on the encoder's model : {snr_db(src12, same):7.2f} dB")
+    print(f"   encoded on the encoder's, played on the chip: {snr_db(src12, cross):7.2f} dB")
+
+    print()
+    print("2. ONE AXIS AT A TIME, so the bill is itemised rather than lumped")
+    for name, key, val in (("nibble order", "order", CHIP["order"]),
+                           ("delta formula", "variant", CHIP["variant"]),
+                           ("clamp", "bits", CHIP["bits"]),
+                           ("initial accumulator", "init", CHIP["init"])):
+        m = dict(ENC); m[key] = val
+        d = adpcm.pack(nib, ENC["order"])
+        rd = list(adpcm.unpack(d, len(nib), m["order"]))
+        got = adpcm.decode(rd, m["variant"], init=m["init"], bits=m["bits"])
+        print(f"   {name:22s} wrong only here: {snr_db(src12, got):7.2f} dB")
+
+    print()
+    print("3. AND THE ONE THAT IS NOT A CONVENTION.  Encode FOR the chip -- the")
+    print("   encoder knows the model and searches against it -- and compare a")
+    print("   10-bit accumulator with a 12-bit one on the same seconds.")
+    for bits in (12, 10):
+        n = adpcm.encode(src12, CHIP["variant"], init=CHIP["init"], bits=bits)
+        r = adpcm.decode(n, CHIP["variant"], init=CHIP["init"], bits=bits)
+        clip = sum(1 for v in r if v in adpcm.clamp_bounds(bits))
+        print(f"   encoded and decoded at {bits}-bit: {snr_db(src12, r):7.2f} dB"
+              f"   ({clip:,} of {len(r):,} samples sit ON the clamp)")
+
+    print()
+    print("4. WHAT THE LEVEL DOES NOW, and it did nothing before (65.1).")
+    print("   At 12 bits the disc's -13.4 dBFS peak had headroom to spare and")
+    print("   normalising bought 0.00 dB.  A 10-bit accumulator is 4x smaller,")
+    print("   so the same signal is no longer comfortably inside it.")
+    peak = max(abs(v) for v in src12)
+    for name, g in (("as recorded", 1.0),
+                    ("scaled to fit 10 bits", 500.0 / peak),
+                    ("half of that", 250.0 / peak)):
+        sc = [max(-512, min(511, int(round(v * g)))) for v in src12]
+        n = adpcm.encode(sc, CHIP["variant"], init=CHIP["init"], bits=CHIP["bits"])
+        r = adpcm.decode(n, CHIP["variant"], init=CHIP["init"], bits=CHIP["bits"])
+        print(f"   {name:24s} x{g:5.2f}  peak {max(abs(v) for v in sc):4d}  "
+              f"{snr_db(sc, r):7.2f} dB")
+    print()
+    print("5. THE HEADROOM, which is the part of this that will bite later.")
+    hd = 20 * math.log10(511.0 / peak)
+    print(f"   This window peaks at {peak} of the 10-bit accumulator's 511, so it")
+    print(f"   has {hd:.1f} dB of headroom left -- and it is a QUIET passage: the")
+    print("   disc peaks at -13.4 dBFS here (65.1).  A 10-bit accumulator is")
+    print(f"   {20*math.log10(2047.0/511.0):.1f} dB smaller than the 12-bit word the encoder was")
+    print("   clamping to, so a passage only a few dB louder than this one does")
+    print("   not fit and the predictor CLIPS inside the recursion.  Nothing in")
+    print("   this project has measured the loudest passage on the disc; until")
+    print("   something does, the audio level is an OPEN choice and not a")
+    print("   settled one, and 65.1's `the level is not a lever` is now wrong")
+    print("   in one direction: it is not a lever UPWARD.")
+    print()
+    print("   The rows in 4 are NOT comparable as absolute quality")
+    print("   -- each is scored against its OWN scaled reference, so what they")
+    print("   compare is how well the format tracks a signal of that size.")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

tools/analysis/34_packed_audio.py

@@ -0,0 +1,217 @@
+#!/usr/bin/env python3
+"""DLXP2's GATE: a packed container with sound in it.  ROADMAP P6b.
+
+    python3 tools/analysis/34_packed_audio.py [packed.dlxp] [--audio tmp/au_singe.raw]
+
+WHAT THIS IS FOR.  FINDINGS 65.3 did the arithmetic of putting audio in the
+packed container and wrote no byte of one; 66 measured which of sixteen decoder
+models the machine's chip runs and priced the axes at up to 25.7 dB.  This is
+the container those two produce, and the reason it needs a gate of its own is
+that NOTHING PARSES A PACKED CONTAINER.  A DMA channel copies bytes and has no
+opinion about them (62), so a container whose lump is one sector out does not
+fail -- it plays 512 B of picture as audio and 512 B of audio as picture, both
+of which are things, and a gate that only checked for errors would pass it.
+
+THE FOUR CLAIMS, and each is checked against something that is not the writer:
+
+  1. THE FILE IS ITS OWN ARITHMETIC.  Every byte of the container is accounted
+     for by `off_frm + i*rec + (i//F)*A*512` and `off_aud + k*(F*rec + A*512)`
+     with no byte left over and no byte claimed twice.  A per-record read cannot
+     catch an off-by-one that shifts everything after it; a partition can.
+  2. THE PICTURE DID NOT MOVE.  Interleaving a second stream into a format whose
+     whole claim is "record i is at LBA0 + i*97" is exactly the change that can
+     break that claim, so every record is compared against a re-encode of the
+     same frames with `--audio` off.  The silent container is the control.
+  3. THE BYTES ARE THE ENCODER'S.  The lumps, concatenated, are byte-exact
+     against `adpcm.encode` run again on the same PCM with the same four axes.
+  4. THE HEADER'S AXES ARE LOAD-BEARING.  The stream decodes to the source at
+     the SNR the encoder reported, and flipping any ONE of the four axes the
+     header carries collapses it.  A header field nothing would notice being
+     wrong is a comment.
+
+AND THE FINDING IT REPORTS (FINDINGS 67).  The padding is not where a reader of
+65.3 would put it.  A lump is A*512 B of SPACE; F frames of audio is
+F*hz/(2*fps) B, which at F=11 is 7,161.4583..., so the PAYLOAD alternates 7,161
+and 7,162 and the sector run is 7,168 either way.  A player that handed the chip
+the whole lump -- the obvious implementation, and the one the phrase "14 sectors
+of audio every 11 frames" invites -- would be feeding it 6.54 B a group too
+much.  That is not waste, which is what padding usually is.  It is DRIFT.
+"""
+import argparse, math, os, struct, sys
+sys.path.insert(0, "tools/encoder")
+sys.path.insert(0, "tools/analysis")
+import adpcm
+import dlxp as P
+from dlxp import DLXP, SECTOR
+
+ap = argparse.ArgumentParser()
+ap.add_argument("container", nargs="?", default="tmp/packed_singe.dlxp")
+ap.add_argument("--audio", default="tmp/au_singe.raw")
+ap.add_argument("--silent", default="tmp/packed_singe_silent.dlxp",
+                help="the control: the same frames with --audio off. Built by "
+                     "check.sh; skipped rather than faked when absent")
+ap.add_argument("--game-min", type=float, default=22.8,
+                help="the game's running length, for what the drift comes to")
+a = ap.parse_args()
+
+fails = []
+def ck(ok, msg):
+    print(("  OK   " if ok else "  FAIL ") + msg)
+    if not ok:
+        fails.append(msg)
+
+d = DLXP(a.container)                    # every format invariant is checked here
+print(f"{a.container}: DLXP{d.version} {d.W}x{d.H} {d.fps}fps {d.nframes} frames, "
+      f"{'AUDIO' if d.has_audio else 'SILENT'}")
+if not d.has_audio:
+    sys.exit(f"{a.container} carries no audio -- this gate has nothing to check. "
+             f"Build it with tools/encoder/pack.py --audio")
+
+grp = d.cad_f * d.aud_hz / (2 * d.fps)
+print(f"""
+=== THE LAYOUT =========================================================
+  record {d.rec_bytes:,} B = {d.rec_bytes//SECTOR} sectors, lump {d.cad_a*SECTOR:,} B = {d.cad_a} sectors,
+  cadence F={d.cad_f} A={d.cad_a}, {d.n_lumps} lumps, {d.aud_bytes:,} B of ADPCM at {d.aud_hz:,} Hz
+
+    record i  =  {d.off_frm:,} + i*{d.rec_bytes:,} + (i//{d.cad_f})*{d.cad_a*SECTOR:,}
+    lump   k  =  {d.off_aud:,} + k*{d.cad_f*d.rec_bytes + d.cad_a*SECTOR:,}
+
+  and NEITHER of those is a lookup.  A packed record's length is geometry and a
+  lump's is a cadence, so DLXP2 still has no index and still needs none.""")
+
+# --- 1. the file is its own arithmetic ------------------------------------
+# Every byte, partitioned. Not "does record 7 read back" -- an off-by-one that
+# shifts the whole stream reads back fine one record at a time.
+spans = [(d.frame_off(i), d.rec_bytes, f"record {i}") for i in range(d.nframes)]
+spans += [(d.lump_off(k), d.cad_a * SECTOR, f"lump {k}") for k in range(d.n_lumps)]
+spans.sort()
+pos, overlap, gap = SECTOR, [], []
+for off, n, what in spans:
+    if off < pos:   overlap.append(what)
+    elif off > pos: gap.append((pos, off, what))
+    pos = max(pos, off + n)
+ck(not overlap, f"nothing overlaps ({len(spans)} spans: {d.nframes} records "
+                f"+ {d.n_lumps} lumps)" + (f" -- {overlap[:3]}" if overlap else ""))
+ck(not gap, "no byte between the header and the end belongs to nothing"
+            + (f" -- {gap[:3]}" if gap else ""))
+ck(pos == len(d.raw), f"the arithmetic ends at {pos:,} and the file is "
+                      f"{len(d.raw):,} B")
+ck(all(off % SECTOR == 0 for off, _, _ in spans),
+   "every record and every lump starts on a 512 B sector -- 58.3/60.1's "
+   "precondition survives the interleave")
+
+# --- 1b. and the cadence term is load-bearing -----------------------------
+# THE FAILURE MODE THIS FORMAT HAS AND THE CODEC'S DOES NOT.  A DLX record is
+# found through an index and a player that read the wrong entry gets a length
+# word that does not parse.  A packed record is found by ARITHMETIC and nothing
+# parses it, so a player that drops the `(i//F)*A` term reads 97 sectors
+# starting 14 sectors early and paints them: the last 14 sectors of the previous
+# record, then 83 of this one, shifted down the screen.  It is a picture.  Here
+# is what the gate would be comparing if the term were missing, and it is only
+# WRONG from frame F on -- the first group is exempt, which is how an off-by-one
+# like this survives a rig that checks frame 0.
+blind = [i for i in range(d.nframes)
+         if d.raw[d.off_frm + i*d.rec_bytes:d.off_frm + (i+1)*d.rec_bytes]
+            != d.record(i)]
+ck(blind == list(range(d.cad_f, d.nframes)),
+   f"a cadence-blind player reads the wrong bytes for {len(blind)} of "
+   f"{d.nframes} records, first at frame {blind[0] if blind else '-'} -- and "
+   f"frames 0..{d.cad_f-1} are IDENTICAL either way, so frame 0 proves nothing")
+
+# --- 2. the picture did not move ------------------------------------------
+if os.path.exists(a.silent):
+    q = DLXP(a.silent)
+    same = (q.nframes == d.nframes
+            and all(q.record(i) == d.record(i) for i in range(d.nframes)))
+    ck(same, f"all {d.nframes} records byte-exact against the SILENT control "
+             f"({os.path.basename(a.silent)}) -- interleaving audio moved no "
+             f"picture byte")
+    ck(q.has_audio is False and q.off_frm == SECTOR,
+       "and the control really is silent: no audio flag, record 0 at sector 1")
+else:
+    print(f"  SKIPPED: no silent control at {a.silent}")
+
+# --- 3. the bytes are the encoder's ---------------------------------------
+if os.path.exists(a.audio):
+    raw = open(a.audio, "rb").read()
+    pcm = struct.unpack("<%dh" % (len(raw) // 2), raw)
+    src = [max(-2048, min(2047, x >> 4)) for x in pcm]
+    axes = d.decoder()
+    ck(axes == adpcm.CHIP, f"the header's four axes ARE adpcm.CHIP: {axes}")
+    nib = adpcm.encode(src, variant=axes["variant"], init=axes["init"],
+                       bits=axes["bits"])
+    want = adpcm.pack(nib, order=axes["order"])[:d.aud_bytes]
+    got = d.audio()
+    ck(got == want, f"the {len(got):,} B the lumps carry are byte-exact against "
+                    f"adpcm.encode on the same PCM")
+    ck(all(d.lump(k, padding=True)[len(d.lump(k)):] == b"\0" * (
+            d.cad_a * SECTOR - len(d.lump(k))) for k in range(d.n_lumps)),
+       "and every lump's padding is zero, so a player that overruns the payload "
+       "feeds the chip silence rather than the next lump's first sample")
+
+    # --- 4. the header's axes are load-bearing ----------------------------
+    def snr(axes_):
+        rec = adpcm.decode(adpcm.unpack(got, len(src), order=axes_["order"]),
+                           variant=axes_["variant"], init=axes_["init"],
+                           bits=axes_["bits"])
+        n = min(len(rec), len(src))
+        e = sum((x - y) ** 2 for x, y in zip(src[:n], rec[:n]))
+        s = sum(x * x for x in src[:n])
+        return 10 * math.log10(s / e) if e else float("inf")
+
+    right = snr(axes)
+    ck(right > 20.0, f"decoded on the axes the header names: {right:.2f} dB")
+    print(f"\n  AND EVERY AXIS IS A NEGATIVE CONTROL -- flip ONE and this is what\n"
+          f"  a player that ignored the header would hear:\n")
+    print(f"    {'axis':<12} {'header':>8} {'flipped to':>11} {'SNR':>9}   cost")
+    flips = [("order", "high" if axes["order"] == "low" else "low"),
+             ("variant", "terms" if axes["variant"] == "shift" else "shift"),
+             ("bits", 12 if axes["bits"] == 10 else 10),
+             ("init", 0 if axes["init"] else -2)]
+    for k, v in flips:
+        w = dict(axes); w[k] = v
+        s2 = snr(w)
+        print(f"    {k:<12} {str(axes[k]):>8} {str(v):>11} {s2:9.2f} dB   "
+              f"{s2-right:+.2f} dB")
+        if k in ("order", "variant"):
+            ck(s2 < right - 2.0, f"axis '{k}' is load-bearing: {s2-right:+.2f} dB")
+else:
+    print(f"  SKIPPED: no PCM at {a.audio} -- the bytes were not re-derived")
+
+# --- the finding ----------------------------------------------------------
+per = d.cad_a * SECTOR - grp
+print(f"""
+=== THE PAYLOAD IS NOT THE LUMP (FINDINGS 67) ==========================
+  A lump is {d.cad_a*SECTOR:,} B of SPACE.  {d.cad_f} frames of audio is {grp:,.4f} B, so the
+  PAYLOAD is {P.lump_bytes(0, d.cad_f, d.fps, d.aud_hz):,} or {P.lump_bytes(2, d.cad_f, d.fps, d.aud_hz):,} -- the same remainder FINDINGS 54's frame
+  clock carries, one dimension over -- and the last {per:.4f} B are zero.
+
+  A PLAYER THAT FED THE CHIP THE WHOLE LUMP would hand it {per:.2f} B a group it
+  should not have.  At {d.aud_hz:,} Hz that is {2*per/d.aud_hz*1000:.2f} ms of audio every
+  {d.cad_f/d.fps:.4f} s, which is {100*per/grp:.3f}% -- and it does not average out, it ACCUMULATES:""")
+for mins in (1.0, a.game_min):
+    print(f"    {mins:5.1f} min of play -> {mins*60*(per/grp):.2f} s of lip-sync error")
+print(f"""  so the cadence's {100*per/grp:.3f}% is not the waste figure 65.3 called it and left
+  at that.  It is waste ON THE WIRE and DRIFT IN THE PLAYER, and the second is
+  the expensive one: {a.game_min:.1f} minutes is {a.game_min*60*(per/grp):.2f} s, which is a scene of dialogue
+  arriving after the mouth that spoke it.
+
+  WHAT A PLAYER CARRIES INSTEAD IS ONE ACCUMULATOR, and it is three
+  instructions rather than a table:
+
+      acc += {d.cad_f}*{d.aud_hz:,}          ; = {d.cad_f*d.aud_hz:,}
+      n    = acc // {2*d.fps}          ; the MTC for this lump's channel
+      acc %= {2*d.fps}
+
+  which is exactly clock.i's shape (54) and for exactly the same reason: a rate
+  with a denominator of {2*d.fps} cannot be a count, so it is a remainder.
+
+=== THE WIRE ===========================================================
+  video {d.video_kbps():7.1f} KB/s   FIXED by geometry
+  audio {d.audio_kbps():7.2f} KB/s   the CADENCE's, padding included -- the disc moves
+                          whole sectors and the wire pays for the zero ones
+  total {d.kbps():7.1f} KB/s   ({100*(d.kbps()/d.video_kbps()-1):+.2f}%), and 65.3 predicted {589.6:.1f}
+""")
+
+print(f"{'FAIL' if fails else 'OK'} 34_packed_audio: {len(fails)} failure(s)")
+sys.exit(1 if fails else 0)

tools/analysis/35_audio_level.py

@@ -0,0 +1,496 @@
+#!/usr/bin/env python3
+"""HOW LOUD IS THE DISC?  ROADMAP P6, the item FINDINGS 66.3 reopened.
+
+    python3 tools/analysis/35_audio_level.py [--streams 00000-00201] [--json out]
+
+FINDINGS 66 asked MAME's MSM6258 which decoder it is and got four axes back.
+The one with a bill attached is the CLAMP: the chip's accumulator is **10 bits**
+and it clamps INSIDE the recursion, so the reachable set of reconstructed
+samples is [-512, 511] in the 12-bit units everything in this project counts in
+-- a quarter of the 12-bit word `adpcm.py` used to clamp at.
+
+`pack.py` hands the encoder `s16 >> 4`, i.e. it maps the disc's full scale onto
+the 12-bit word, and 66.3 measured the Singe window peaking at **435 of 511**.
+That fit with 1.4 dB to spare, and it fit BY ACCIDENT: the window is a -13.4
+dBFS passage. Any passage more than 1.4 dB louder does not merely distort at the
+top, it drives the predictor -- a clamped accumulator is a WRONG STATE that the
+next nibble is applied to, so the error outlives the loud sample.
+
+So the level cannot be chosen from the ten seconds this project gates on. It has
+to be chosen from the loudest thing the game will ever play, and this file
+measures that: every stream of the unique scene footage -- `00000`-`00201`,
+1366.6 s, FINDINGS 32.1 -- through the SAME chain `extract_audio.py` uses (AC-3
+5.1, ffmpeg's default downmix matrix, mono, 15,625 Hz), because a level measured
+through a different resampler is a level for a different encoder.
+
+Two statistics, and the difference between them is the whole argument:
+
+  PEAK       max |x| over the disc. What must fit under 511 for NOTHING to clamp.
+  PASSAGE    the loudest ~1 s window's peak and RMS. What the ear gets. A single
+             sample 6 dB above everything else is a click and costs one clamp; a
+             passage 6 dB above the gate window is where the recursion lives for
+             fifteen thousand samples.
+
+It prints the attenuation each choice implies, in dB and as the shift `pack.py`
+would have to make, and it does NOT choose. Choosing needs the other half --
+what attenuation costs at the quiet end, where the OKI step table's floor of 16
+(12-bit units) does not scale with the signal -- and that is `--ladder`, which
+encodes real passages at real gains with `adpcm.CHIP` and reports the SNR.
+"""
+import argparse, getpass, json, os, subprocess, sys
+import numpy as np
+
+sys.path.insert(0, "tools/encoder")
+import adpcm
+
+BDROM = os.environ.get("DLX_BDROM") or f"/media/{getpass.getuser()}/BDROM"
+STREAM_DIR = f"{BDROM}/BDMV/STREAM"
+
+HZ = 15625                      # the chip's rate, and the only one budgeted for
+FPS = 12
+LUMP_FRAMES = 11                # FINDINGS 65.3's cadence: 11 frames of audio
+WIN = LUMP_FRAMES * HZ // FPS   # 14,322 samples ~ 0.917 s -- one audio lump
+HOP = HZ // 4                   # 0.25 s blocks; the window is 4 of them (rounded)
+
+CLAMP_LO, CLAMP_HI = adpcm.clamp_bounds(adpcm.CHIP["bits"])   # -512, 511
+FULL12 = 2048                   # what `s16 >> 4` maps full scale to
+
+
+def db(x, ref=FULL12):
+    return -np.inf if x <= 0 else 20 * np.log10(x / ref)
+
+
+_PCM_CACHE = {}
+
+
+def pcm12(stream, start=None, dur=None):
+    """One stream as 12-bit signed samples, through extract_audio.py's chain.
+
+    Cached, because the scan, the census and the event walk are three passes
+    over the same 20 million samples and the whole game is 40 MB of int16.
+    """
+    ck = (stream, start, dur)
+    if ck in _PCM_CACHE:
+        return _PCM_CACHE[ck]
+    cmd = ["ffmpeg", "-v", "error"]
+    if start is not None: cmd += ["-ss", str(start)]
+    if dur is not None:   cmd += ["-t", str(dur)]
+    cmd += ["-i", f"{STREAM_DIR}/{stream}.m2ts", "-vn", "-ac", "1",
+            "-ar", str(HZ), "-f", "s16le", "-acodec", "pcm_s16le", "-"]
+    p = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+    if p.returncode:
+        # 00176 is 3.0 s of mpeg2video with NO AUDIO TRACK AT ALL. That is a
+        # fact about the disc, not a failure here, so it is reported rather
+        # than swallowed -- but anything else is a real error.
+        if b"does not contain any stream" not in p.stderr:
+            raise SystemExit(f"ffmpeg failed on {stream}: "
+                             f"{p.stderr.decode(errors='replace')[:400]}")
+        _PCM_CACHE[ck] = None
+        return None
+    x = np.frombuffer(p.stdout, "<i2").astype(np.int32)
+    # The SAME requantisation pack.py makes. It is a shift and not a divide, so
+    # it floors toward -inf, and that asymmetry is real: -1 >> 4 is -1.
+    out = np.clip(x >> 4, -FULL12, FULL12 - 1).astype(np.int16)
+    _PCM_CACHE[ck] = out
+    return out
+
+
+def scan(streams):
+    """Per-stream peak and loudest-passage statistics, in 12-bit units."""
+    rows, mute = [], []
+    for s in streams:
+        x = pcm12(s)
+        if x is None:
+            mute.append(s)
+            continue
+        if x.size == 0:
+            continue
+        a = np.abs(x).astype(np.float64)
+        nb = a.size // HOP
+        if nb:
+            bmax = a[:nb * HOP].reshape(nb, HOP).max(1)
+            bsq = (a[:nb * HOP].reshape(nb, HOP) ** 2).sum(1)
+            k = max(1, round(WIN / HOP))
+            if nb >= k:
+                # sliding sum over k blocks == the ~1 s lump window
+                cs = np.concatenate(([0.0], np.cumsum(bsq)))
+                wrms = np.sqrt((cs[k:] - cs[:-k]) / (k * HOP))
+                wpk = np.array([bmax[i:i + k].max() for i in range(nb - k + 1)])
+            else:
+                wrms = np.array([np.sqrt((a ** 2).mean())])
+                wpk = np.array([a.max()])
+        else:
+            wrms = np.array([np.sqrt((a ** 2).mean())])
+            wpk = np.array([a.max()])
+        ipk = int(np.argmax(a))
+        irms = int(np.argmax(wrms))
+        rows.append(dict(stream=s, n=int(x.size), secs=x.size / HZ,
+                         peak=float(a.max()), peak_t=ipk / HZ,
+                         rms=float(np.sqrt((a ** 2).mean())),
+                         wpeak=float(wpk.max()),
+                         wrms=float(wrms.max()), wrms_t=irms * HOP / HZ))
+    return rows, mute
+
+
+def report(rows, mute):
+    tot = sum(r["secs"] for r in rows)
+    peak = max(rows, key=lambda r: r["peak"])
+    loud = max(rows, key=lambda r: r["wrms"])
+    disc_peak = peak["peak"]
+
+    print(f"=== THE DISC, {len(rows)} streams, {tot:,.1f} s = {tot/60:.1f} min "
+          f"(FINDINGS 32.1 says 1,366.6) ===\n")
+    if mute:
+        print(f"  {len(mute)} stream(s) carry NO AUDIO TRACK: {', '.join(mute)}"
+              f"  -- a fact about the disc, and a case a shipping encoder has\n"
+              f"  to have an answer for (silence of the right length).\n")
+    print(f'{"stream":>8}{"secs":>8}{"peak":>7}{"dBFS":>8}{"passage pk":>12}'
+          f'{"passage rms":>13}{"dBFS":>8}   at')
+    for r in sorted(rows, key=lambda r: -r["wrms"])[:12]:
+        print(f'{r["stream"]:>8}{r["secs"]:8.1f}{r["peak"]:7.0f}{db(r["peak"]):8.2f}'
+              f'{r["wpeak"]:12.0f}{r["wrms"]:13.1f}{db(r["wrms"]):8.2f}'
+              f'   {r["wrms_t"]:6.2f} s')
+    print("   (the twelve loudest PASSAGES; the table is sorted by the window "
+          "RMS, not the peak)\n")
+
+    print(f"  DISC PEAK       {disc_peak:.0f} of {FULL12} = {db(disc_peak):.2f} dBFS"
+          f"   ({peak['stream']} @ {peak['peak_t']:.2f} s)")
+    print(f"  LOUDEST PASSAGE peak {loud['wpeak']:.0f}, rms {loud['wrms']:.1f}"
+          f" = {db(loud['wrms']):.2f} dBFS   ({loud['stream']} @ {loud['wrms_t']:.2f} s)")
+    print(f"  THE CLAMP       +{CLAMP_HI} / {CLAMP_LO} (it is not symmetric), "
+          f"{db(CLAMP_HI):.2f} dBFS in the same units\n")
+
+    need = disc_peak / CLAMP_HI
+    print("=== WHAT THAT COSTS, AS A LEVEL ===\n")
+    print(f"  `s16 >> 4` is what pack.py does today and it puts the disc's own")
+    print(f"  peak at {disc_peak:.0f} against a clamp of {CLAMP_HI}: "
+          f"{'OVER by' if need > 1 else 'under by'} {abs(20*np.log10(need)):.2f} dB.")
+    print(f"  Fitting the whole disc under the clamp with no sample clamped at")
+    print(f"  all needs a gain of {1/need:.4f} = {-20*np.log10(need):.2f} dB, i.e.")
+    for sh in (4, 5, 6, 7):
+        pk = disc_peak / (1 << (sh - 4))
+        mark = "  <- fits" if pk <= CLAMP_HI else ""
+        # ~ because a further right shift floors again and this halves; the
+        # difference is one count and the column is a signpost, not a spec.
+        print(f"      s16 >> {sh}   disc peak ~{pk:7.1f}   "
+              f"{'clamps' if pk > CLAMP_HI else 'clear':>6} by "
+              f"{abs(20*np.log10(pk/CLAMP_HI)):5.2f} dB{mark}")
+    print()
+
+    # How much of the disc is actually above the clamp at today's level: the
+    # number that decides whether this is a level question or a limiter question.
+    return dict(rows=rows, disc_peak=disc_peak, peak_stream=peak["stream"],
+                peak_t=peak["peak_t"], loud_stream=loud["stream"],
+                loud_t=loud["wrms_t"], loud_rms=loud["wrms"],
+                loud_wpeak=loud["wpeak"], clamp=CLAMP_HI, mute=mute)
+
+
+def clip_census(rows, streams, gains):
+    """At each candidate gain, how many samples of the WHOLE DISC clamp?
+
+    A peak is one number and this is the distribution behind it. A gain that
+    clamps 12 samples in 22 minutes is a different object from one that clamps
+    thousands, and the peak alone cannot tell them apart.
+    """
+    print("=== THE CENSUS: how much of the disc is ABOVE the clamp, by gain ===\n")
+    print(f'{"gain":>8}{"dB":>8}{"samples over":>14}{"of":>12}{"share":>10}'
+          f'{"worst over":>12}')
+    tot = 0
+    over = {g: 0 for g in gains}
+    worst = {g: 0.0 for g in gains}
+    for s in streams:
+        x = pcm12(s)
+        if x is None:
+            continue
+        a = np.abs(x).astype(np.float64)
+        tot += a.size
+        for g in gains:
+            # ROUNDED, exactly as the ladder and pack.py requantise. Comparing
+            # the float product instead makes 511/946 report one sample over
+            # its own clamp, which is arithmetic about floats and not about
+            # the disc.
+            v = np.round(a * g)
+            m = v > CLAMP_HI
+            over[g] += int(m.sum())
+            if m.any():
+                worst[g] = max(worst[g], float(v.max() / CLAMP_HI))
+    for g in gains:
+        w = f"{20*np.log10(worst[g]):.2f} dB" if worst[g] else "-"
+        print(f'{g:8.4f}{20*np.log10(g):8.2f}{over[g]:14,}{tot:12,}'
+              f'{100*over[g]/tot:9.4f}%{w:>12}')
+    print()
+    return dict(total=tot, over={f"{g:.4f}": over[g] for g in gains})
+
+
+def clamp_events(streams, gain=1.0):
+    """WHERE the over-clamp samples are, not just how many.
+
+    687 isolated samples in 22 minutes and one sustained 44 ms burst are the
+    same census row and completely different sounds, and a clamp inside a
+    recursion is not a clipped sample -- it is a wrong predictor state that the
+    next nibble is applied to. So the run lengths are the statistic.
+    """
+    runs = []
+    for st in streams:
+        x = pcm12(st)
+        if x is None:
+            continue
+        m = np.round(np.abs(x).astype(np.float64) * gain) > CLAMP_HI
+        if not m.any():
+            continue
+        d = np.diff(np.concatenate(([0], m.view(np.int8), [0])))
+        beg = np.where(d == 1)[0]
+        end = np.where(d == -1)[0]
+        for b, e in zip(beg, end):
+            runs.append((int(e - b), st, b / HZ))
+    runs.sort(reverse=True)
+    n = sum(r[0] for r in runs)
+    print(f"=== WHERE THE CLAMPS ARE at gain {gain:.4f} "
+          f"({len(runs)} events, {n:,} samples = {1000*n/HZ:.1f} ms) ===\n")
+    print(f'{"run":>6}{"ms":>8}   stream    at')
+    for r, st, t in runs[:10]:
+        print(f'{r:6}{1000*r/HZ:8.2f}   {st}   {t:7.2f} s')
+    if runs:
+        print(f"   longest run {runs[0][0]} samples = {1000*runs[0][0]/HZ:.2f} ms; "
+              f"median run {sorted(r[0] for r in runs)[len(runs)//2]}")
+    print()
+    return dict(events=len(runs), samples=n,
+                longest=runs[0][0] if runs else 0)
+
+
+def ladder(where, gains, dur, label):
+    """Encode a real passage at each gain with adpcm.CHIP and report the SNR.
+
+    This is the half a peak measurement cannot do. Attenuation buys headroom at
+    the top and spends resolution at the bottom, because the OKI step table's
+    floor is a constant 16 in 12-bit units and does not scale with the signal.
+    The SNR is reported against the SCALED source, which is the honest
+    comparison: the encoder's job is to reproduce what it was handed, and the
+    listener's volume knob is not this project's problem.
+    """
+    stream, start = where
+    x = pcm12(stream, start, dur)
+    print(f"=== THE LADDER: {label} -- {stream} @ {start:.2f} s, {dur:.2f} s, "
+          f"{x.size:,} samples ===\n")
+    print(f'{"gain":>8}{"dB":>8}{"src peak":>10}{"clamped":>9}{"SNR dB":>9}'
+          f'{"vs 1.0":>8}')
+    base = None
+    out = []
+    for g in gains:
+        src = np.clip(np.round(x * g), -FULL12, FULL12 - 1).astype(int).tolist()
+        nib = adpcm.encode(src, variant=adpcm.CHIP["variant"],
+                           init=adpcm.CHIP["init"], bits=adpcm.CHIP["bits"])
+        rec = adpcm.decode(nib, variant=adpcm.CHIP["variant"],
+                           init=adpcm.CHIP["init"], bits=adpcm.CHIP["bits"])
+        s = np.array(src, dtype=np.float64)
+        r = np.array(rec, dtype=np.float64)
+        e = ((s - r) ** 2).sum()
+        snr = 10 * np.log10((s ** 2).sum() / e) if e else np.inf
+        nclamp = int((np.abs(s) > CLAMP_HI).sum())
+        if base is None:
+            base = snr
+        print(f'{g:8.4f}{20*np.log10(g):8.2f}{np.abs(s).max():10.0f}{nclamp:9,}'
+              f'{snr:9.2f}{snr-base:+8.2f}')
+        out.append(dict(gain=g, snr=snr, clamped=nclamp,
+                        peak=float(np.abs(s).max())))
+    print()
+    return out
+
+
+def survey(rows, gains, n, dur, seed=20260825):
+    """THE DISC, not three passages of it.
+
+    Three hand-picked passages can be argued with; a sample cannot. `n` windows
+    are drawn uniformly over the game's own timeline -- weighted by stream
+    length, so a 24 s stream gets twenty times the draws of a 1.2 s one -- and
+    every one is encoded at every gain with `adpcm.CHIP`. What is reported is
+    the distribution: the mean SNR is what the level costs on average, and the
+    WORST window is what it costs where it matters, because a level is chosen
+    for the passage it fails on.
+    """
+    rng = np.random.default_rng(seed)
+    pool = [r for r in rows if r["secs"] >= dur]
+    w = np.array([r["secs"] for r in pool], dtype=np.float64)
+    w /= w.sum()
+    picks = []
+    for _ in range(n):
+        r = pool[int(rng.choice(len(pool), p=w))]
+        t = float(rng.uniform(0, r["secs"] - dur))
+        picks.append((r["stream"], t))
+    print(f"=== THE SURVEY: {n} windows of {dur:.1f} s drawn over the whole "
+          f"{sum(r['secs'] for r in rows)/60:.1f} min, encoded at every gain ===\n")
+    src = [pcm12(st, t, dur) for st, t in picks]
+    print(f'{"gain":>8}{"dB":>8}{"mean SNR":>10}{"median":>9}{"WORST":>8}'
+          f'{"windows w/ clamp":>18}{"samples":>9}')
+    out = []
+    for g in gains:
+        snrs, nclamp, ncw = [], 0, 0
+        for x in src:
+            v = np.clip(np.round(x * g), -FULL12, FULL12 - 1).astype(int)
+            k = int((np.abs(v) > CLAMP_HI).sum())
+            nclamp += k
+            ncw += 1 if k else 0
+            nib = adpcm.encode(v.tolist(), variant=adpcm.CHIP["variant"],
+                               init=adpcm.CHIP["init"], bits=adpcm.CHIP["bits"])
+            rec = np.array(adpcm.decode(nib, variant=adpcm.CHIP["variant"],
+                                        init=adpcm.CHIP["init"],
+                                        bits=adpcm.CHIP["bits"]), dtype=np.float64)
+            f = v.astype(np.float64)
+            e = ((f - rec) ** 2).sum()
+            snrs.append(10 * np.log10((f ** 2).sum() / e) if e else np.inf)
+        a = np.array(snrs)
+        print(f'{g:8.4f}{20*np.log10(g):8.2f}{a.mean():10.2f}'
+              f'{np.median(a):9.2f}{a.min():8.2f}{ncw:14} of {len(src)}{nclamp:9,}',
+              flush=True)
+        out.append(dict(gain=g, mean=float(a.mean()), median=float(np.median(a)),
+                        worst=float(a.min()), clamped=nclamp, windows=ncw))
+    print()
+    return out
+
+
+def recover(stream, gain, control, K=64):
+    """DOES A CLAMP OUTLIVE THE SAMPLE IT HAPPENS ON?  66.3 said it would.
+
+    The worry was exact and it is the right worry for a recursive codec: a
+    clamped accumulator is a WRONG STATE and the next nibble is applied to it,
+    so the error should persist after the loud sample has gone.  Measuring the
+    error after a clamp run does show it elevated -- and that is not evidence,
+    because the samples after a clamp run are LOUD samples, where the step is
+    large and the error is large anyway.
+
+    So the control is the same window at the gain that never clamps, rescaled
+    to the same units and read at the SAME sample indices.  What the ratio
+    isolates is the clamp and nothing else.
+    """
+    x = pcm12(stream).astype(np.float64)
+
+    def enc(g):
+        src = np.clip(np.round(x * g), -FULL12, FULL12 - 1).astype(int)
+        nib = adpcm.encode(src.tolist(), variant=adpcm.CHIP["variant"],
+                           init=adpcm.CHIP["init"], bits=adpcm.CHIP["bits"])
+        rec = adpcm.decode(nib, variant=adpcm.CHIP["variant"],
+                           init=adpcm.CHIP["init"], bits=adpcm.CHIP["bits"])
+        return src.astype(np.float64), np.array(rec, dtype=np.float64)
+
+    s1, r1 = enc(gain)
+    s2, r2 = enc(control)
+    e1 = np.abs(s1 - r1)
+    e2 = np.abs(s2 - r2) / control * gain      # the control, in gain's units
+    m = np.abs(s1) > CLAMP_HI
+    d = np.diff(np.concatenate(([0], m.view(np.int8), [0])))
+    ends = [e for e in np.where(d == -1)[0] if e + K <= e1.size]
+    p1 = np.array([e1[e:e + K] for e in ends], dtype=np.float64).mean(0)
+    p2 = np.array([e2[e:e + K] for e in ends], dtype=np.float64).mean(0)
+
+    print(f"=== DOES THE CLAMP OUTLIVE THE SAMPLE?  {stream}, gain {gain:g} "
+          f"against a control at {control:g} ===\n")
+    print(f"  {int(m.sum())} samples clamp in {len(ends)} runs; the profile is "
+          f"the mean |error| at each\n  offset after a run ENDS, in 12-bit units, "
+          f"against the same offsets of a\n  window that never clamps at all.\n")
+    print(f'{"after":>7}{"clamped":>10}{"control":>10}{"ratio":>8}')
+    for i in (0, 1, 2, 4, 8, 16, 32, K - 1):
+        print(f'{"+" + str(i):>7}{p1[i]:10.2f}{p2[i]:10.2f}{p1[i]/p2[i]:8.2f}')
+    off = ~m
+    print(f'\n  off-clamp mean |err|   {e1[off].mean():.2f} vs {e2[off].mean():.2f}')
+    print(f'  whole-window mean |err| {e1.mean():.2f} vs {e2.mean():.2f}')
+    print(f'  worst ratio over the {K} offsets: {(p1/p2).max():.2f}\n')
+    return dict(stream=stream, gain=gain, control=control,
+                runs=len(ends), clamped=int(m.sum()),
+                worst_ratio=float((p1 / p2).max()),
+                mean_err=float(e1.mean()), mean_err_control=float(e2.mean()))
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--first", type=int, default=0)
+    ap.add_argument("--last", type=int, default=201,
+                    help="the unique scene footage is 00000-00201 (FINDINGS 32.1); "
+                         "00215/00216/00223 are compilations of the same material")
+    ap.add_argument("--ladder", action="store_true",
+                    help="also encode the loudest and a quiet passage at each gain")
+    ap.add_argument("--dur", type=float, default=4.0, help="ladder passage seconds")
+    ap.add_argument("--survey", type=int, default=0,
+                    help="encode N windows drawn over the whole game at each gain")
+    ap.add_argument("--survey-dur", type=float, default=2.0)
+    ap.add_argument("--recover", action="store_true",
+                    help="does a clamp outlive its sample? 66.3 said it would")
+    ap.add_argument("--gate", action="store_true",
+                    help="assert FINDINGS 69's headline numbers, exit 1 if not")
+    ap.add_argument("--json")
+    a = ap.parse_args()
+
+    streams = [f"{i:05d}" for i in range(a.first, a.last + 1)]
+    streams = [s for s in streams if os.path.exists(f"{STREAM_DIR}/{s}.m2ts")]
+    if not streams:
+        sys.exit(f"no streams under {STREAM_DIR} -- is the Blu-ray mounted? "
+                 f"(DLX_BDROM)")
+
+    rows, mute = scan(streams)
+    summary = report(rows, mute)
+
+    # The odd one is not a round number and is not meant to be: it is
+    # CLAMP/disc peak, the gain at which the disc's own loudest sample lands
+    # EXACTLY on the clamp, computed from the scan rather than typed in.
+    exact = round(CLAMP_HI / summary["disc_peak"], 4)
+    gains = sorted({1.0, 0.7071, exact, 0.5, 0.3536, 0.25}, reverse=True)
+    summary["exact_gain"] = exact
+    summary["census"] = clip_census(rows, streams, gains)
+    summary["events"] = clamp_events(streams, 1.0)
+
+    if a.ladder:
+        # Three passages, because they answer three different questions.
+        #   PEAK   what CLAMPING costs, since this is the only place on the disc
+        #          that clamps at today's level.
+        #   LOUD   the loudest sustained window that is long enough to encode.
+        #   QUIET  what ATTENUATION costs, which is the other end of the same
+        #          decision and the reason -15 dB is not free.
+        long = [r for r in rows if r["secs"] >= a.dur]
+        pk = max(rows, key=lambda r: r["peak"])   # the DISC peak, however short
+        loud = max(long, key=lambda r: r["wrms"])
+        quiet = min(long, key=lambda r: r["wrms"])
+        at = lambda r, t: (r["stream"], min(max(0.0, t - a.dur / 2),
+                                            max(0.0, r["secs"] - a.dur)))
+        pkdur = min(a.dur, pk["secs"])
+        summary["ladder_peak"] = ladder(
+            (pk["stream"], min(max(0.0, pk["peak_t"] - pkdur / 2),
+                               max(0.0, pk["secs"] - pkdur))),
+            gains, pkdur, "THE DISC PEAK ITSELF")
+        summary["ladder_loud"] = ladder(at(loud, loud["wrms_t"]), gains, a.dur,
+                                        "THE LOUDEST SUSTAINED PASSAGE")
+        summary["ladder_quiet"] = ladder(at(quiet, quiet["wrms_t"]), gains, a.dur,
+                                         "A QUIET PASSAGE, for the other end")
+
+    if a.survey:
+        summary["survey"] = survey(rows, gains, a.survey, a.survey_dur)
+
+    if a.recover:
+        summary["recover"] = recover(summary["peak_stream"], 1.0, exact)
+
+    if a.gate:
+        expect = dict(disc_peak=946.0, peak_stream="00200", clamp=511,
+                      events=402, over=687)
+        bad = []
+        for k, v in expect.items():
+            got = (summary["events"]["events"] if k == "events" else
+                   summary["events"]["samples"] if k == "over" else summary[k])
+            if got != v:
+                bad.append(f"{k}: expected {v}, measured {got}")
+        if bad:
+            print("LEVEL GATE RED -- the disc does not measure as FINDINGS 69 "
+                  "recorded it:")
+            for b in bad:
+                print("   " + b)
+            print("   (a different pressing is a legitimate cause; a different "
+                  "ffmpeg downmix is not)")
+            sys.exit(1)
+        print("LEVEL GATE GREEN: disc peak 946 of 2048 at 00200, 5.35 dB over "
+              "the chip's 511,\n  687 samples in 402 events = 44.0 ms of the "
+              "game's 21.5 min of audio.")
+
+    if a.json:
+        json.dump(summary, open(a.json, "w"), indent=1)
+        print(f"-> {a.json}")
+
+
+if __name__ == "__main__":
+    main()

tools/analysis/36_branch_audio.py

@@ -0,0 +1,440 @@
+#!/usr/bin/env python3
+"""51.3's REFILL CLIMB WITH A SECOND CONSUMER, THROUGH A REAL BRANCH POINT.
+
+    python3 tools/analysis/36_branch_audio.py --kbps R [R ...]
+                                              [--ring KB ...] [--gate]
+
+The oldest open item in ROADMAP P6, named by FINDINGS 65.6 and again by 67.6:
+
+    "the slack table is here, but 51.3's refill climb with a second consumer
+     through a real branch point is not."
+
+Everything it needs already exists and none of it has ever been put in the same
+room:
+
+  * 51.3 -- slack is ACCUMULATED out of `pipe - wire`, at `pipe - wire` bytes a
+    second, and a seek spends all of it.  `tools/analysis/20_seek_slack.py`.
+  * 56.3 -- where the branch points ARE: 612 distinct transitions into a seek
+    over the arcade's own graph, worst gap 0.000 s, median 3.473 s.
+    `tools/analysis/25_scene_graph.py`, reading only DLXSCENE1.
+  * 56.4 -- the climb against that distribution, for the codec container.  It
+    charged audio as `ratectl.AUDIO_KBPS`, a flat 7.8 KB/s placeholder that
+    predates any of the audio work.
+  * 65.3/67.1 -- what a second consumer ACTUALLY costs a container: a fixed
+    cadence of F frames per A sectors, because a packed record's address is
+    arithmetic and cannot be an index.  `tools/analysis/32_audio_wire.py`.
+  * 68 -- the player that holds both streams at once, and its buffers.
+
+Three questions, and the tree has never asked any of them:
+
+  1. What does the SECOND CONSUMER do to the climb?  Not to the wire -- 32
+     answered that and it is 1.3% -- but to `pipe - wire`, which is a small
+     difference of two large numbers and is the thing the climb is made of.
+
+  2. What is the climb on the PACKED branch?  This is the branch the player
+     runs (68) and the one B1's acceptance is written against.
+
+  3. What does the CADENCE do at a branch point?  A group is `lump k, then F
+     records`, so lump k sits at a LOWER address than every record in its group
+     but the first.  A seek to record i lands inside a group whose audio is
+     BEHIND it.  Nobody has ever priced entering a group off-boundary, and the
+     game's own seek targets say how often it happens.
+
+THE FRAME INDEX OF A SEEK TARGET IS A DESIGN ASSUMPTION AND IS LABELLED ONE.
+DLXSCENE1 carries positions on the laserdisc timeline in ms.  This tree's design
+puts ONE CONTAINER PER SCENE -- 53 and 55.1 charge a scene change 6,164 header
+bytes, and 56.3 counts 203 of the 612 transitions as container changes for
+exactly that reason -- so a seek target's frame index inside its container is
+`(target start - the scene's own earliest start) * fps / 1000`.  If the design
+ever puts one container per SEQUENCE instead, every seek lands on frame 0, the
+group offset is always zero and section 3 collapses to nothing.  That is the
+assumption, said out loud, in the one place the answer depends on it.
+"""
+import sys, os, json, argparse, importlib.util
+
+sys.path.insert(0, "tools/encoder")
+sys.path.insert(0, "tools/analysis")
+
+TABLE = os.environ.get("DLX_SCENEGRAPH", "tmp/scenegraph.json")
+FPS = 12
+CHIP_HZ = 15625.0                 # MSM6258V, 8 MHz / 512 (FINDINGS 32, 65)
+AU_BPS = CHIP_HZ / 2              # 4 bits a sample, two samples to a byte
+AU_FRAME = AU_BPS / FPS           # 651.0416... B a slot, and the dots are 65.3
+SECTOR = 512
+
+
+def load(path, name):
+    spec = importlib.util.spec_from_file_location(name, path)
+    m = importlib.util.module_from_spec(spec)
+    spec.loader.exec_module(m)
+    return m
+
+
+def cadence(F):
+    """Best A for this F: the fewest whole sectors that hold F frames of chip."""
+    num, den = int(round(AU_BPS * 2)), 2 * SECTOR * FPS
+    A = -(-(num * F) // den)                       # ceil(F * bytes/sector)
+    return A, A * SECTOR
+
+
+def climb_s(records, rec_b, pipe_kbps, wire_kbps):
+    """Seconds of play to accumulate `records` records of lookahead (51.3)."""
+    surplus = (pipe_kbps - wire_kbps) * 1024.0
+    return float("inf") if surplus <= 0 else records * rec_b / surplus
+
+
+# --------------------------------------------------------------- the branches
+
+def branch_points(doc, nodes, sg):
+    """The 612 transitions into a seek, plus each one's target frame index.
+
+    Returns (gaps_s, within, changes).  `within` entries carry the frame index
+    the seek lands on inside its container; `changes` are scene changes, which
+    land on frame 0 of a new one.
+    """
+    gaps, _ = sg.worst_gap(nodes)
+    play = [g for g in gaps if g[5] != "attract_mode"]
+    scene_start = {}
+    for scene, seqs in doc["scenes"].items():
+        st = [s["start_ms"] for s in seqs.values() if s["start_ms"] >= 0]
+        scene_start[scene] = min(st) if st else None
+
+    within, changes = [], []
+    for g, src, tgt, kind, ends, scene in play:
+        if ends:
+            changes.append((g / 1000.0, src))
+            continue
+        n = nodes.get(f"{scene}.{tgt}")
+        if n is None or not n.seeks or scene_start[scene] is None:
+            continue
+        i = int(round((n.start - scene_start[scene]) / 1000.0 * FPS))
+        within.append((g / 1000.0, max(0, i), f"{scene}.{tgt}"))
+    return sorted(g[0] / 1000.0 for g in play), within, changes
+
+
+def silences(within, F):
+    """Ms of silence entering each branch's group off-boundary, at cadence F.
+
+    A group is `lump k, then F records`.  Seek to record i, take the next lump
+    that lies AHEAD of the read point -- lump k+1, which arrives at frame
+    (k+1)*F -- and the frames from i to (k+1)*F-1 have no audio.  The chip's
+    second is a real second (15,625 samples, 2 to a byte), so the missing
+    time is exactly `(F - i mod F) mod F` frames of 1/12 s and none of
+    FINDINGS 54's frame-clock remainder gets into it.
+    """
+    return sorted(((F - (i % F)) % F) / FPS * 1000.0 for _, i, _ in within)
+
+
+def pct(xs, p):
+    return xs[min(len(xs) - 1, int(p * len(xs)))] if xs else float("nan")
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--kbps", type=float, nargs="+",
+                    help="delivered pipe rates, KB/s. REQUIRED (FINDINGS 50) "
+                         "unless --gate, which carries its own.")
+    ap.add_argument("--ring", type=float, nargs="+", default=[256, 512])
+    ap.add_argument("--pkbps", type=float, nargs="+",
+                    default=[589.6, 600.0, 650.0, 700.0, 900.0],
+                    help="pipe rates for the PACKED table.  Its own list, "
+                         "because a packed record is 1.3x the codec's and "
+                         "every rate that serves one starves the other.")
+    ap.add_argument("--table", default=TABLE)
+    ap.add_argument("--packed", default="tmp/packed_singe.dlxp")
+    ap.add_argument("--codec", default="tmp/rc_fr_singe_scsi_span.dlx")
+    ap.add_argument("--fsweep", type=int, default=12,
+                    help="highest cadence F in the pick table")
+    ap.add_argument("--gate", action="store_true",
+                    help="check.sh mode: fixed rates, and assert the structural "
+                         "results rather than print the essay")
+    a = ap.parse_args()
+    if a.gate and not a.kbps:
+        a.kbps = [451.4, 488.0, 600.0]
+    if not a.kbps:
+        ap.error("--kbps is required and has no default (FINDINGS 50)")
+
+    if not os.path.exists(a.table):
+        print(f"no scene table at {a.table} -- run:\n"
+              f"  python3 tools/import/scenegraph.py")
+        return 2
+    doc = json.load(open(a.table))
+    if doc.get("format") != "DLXSCENE1":
+        print(f"{a.table}: not a DLXSCENE1 table")
+        return 2
+
+    sg = load("tools/analysis/25_scene_graph.py", "scene_graph")
+    ss = load("tools/analysis/20_seek_slack.py", "seek_slack")
+    from dlxp import DLXP
+    import ratectl as RC
+
+    nodes = sg.build_graph(doc["scenes"])
+    gaps, within, changes = branch_points(doc, nodes, sg)
+    med = gaps[len(gaps) // 2]
+
+    d = DLXP(a.packed)
+    P_REC = d.rec_bytes
+    P_VID = P_REC * FPS / 1024
+    F0, A0 = d.cad_f, d.cad_a
+    P_AUD = A0 * SECTOR / F0 * FPS / 1024
+
+    print(f"""
+=== WHAT EACH BRANCH ACTUALLY HOLDS ====================================
+51.3's climb is a statement about an ACCUMULATOR.  The two branches of this
+project do not have the same one, and one of them does not have one at all.
+
+  codec  ({os.path.basename(a.codec)})
+         a {a.ring[0]:.0f} KB ring of variable records with an index in front of it;
+         lookahead is whole records and the ceiling is what 20_seek_slack.py
+         simulates.
+  packed ({os.path.basename(a.packed)})
+         a record is {P_REC:,} B of literal picture and the channel puts it
+         STRAIGHT INTO GVRAM (FINDINGS 61, 62, 64).  There is no record
+         buffer, so the VIDEO lookahead is ZERO records and there is nothing
+         to climb.  The only consumer on that branch with any lookahead at
+         all is the AUDIO one: {d.cad_a * SECTOR:,} B a lump, PG_ANBUF={3} slots and
+         PG_APRE prefilled at {2} for the run 68 measured (it is a mailbox,
+         not a constant: src/player/packed.s, 68.6).
+         That is {2 * F0 / FPS:.3f} s of sound held against {0.0:.3f} s of picture.
+
+  The CPU-painted packed variant (64.2's column B) is the one that holds two
+  record buffers and 99,328 B, and it is the only packed configuration the
+  word "climb" applies to.  Both are priced below.
+
+=== THE BRANCH POINTS, OUT OF THE ARCADE'S OWN GRAPH ====================
+  {len(gaps)} transitions into a seek (attract mode excluded, 56.3)
+  worst {gaps[0]:.3f} s   p10 {pct(gaps,.10):.3f}   median {med:.3f}   p90 {pct(gaps,.90):.3f}
+  {len(changes)} of them END THE SCENE and are container changes;
+  {len(within)} land INSIDE a container, at frame indices 0..{max(i for _, i, _ in within)}""")
+
+    # ---------------------------------------------------------------- 1
+    dc, rec = ss.records(a.codec)
+    C_REC = float(rec.mean())
+    C_VID = C_REC * FPS / 1024
+    print(f"""
+=== 1. THE CLIMB WITH THE SECOND CONSUMER (the codec branch) ============
+Audio is {AU_BPS/1024:.3f} KB/s and this container pays no padding for it (65.4: it
+already has an index and already has variable records).  Against a video wire
+of {C_VID:.1f} KB/s that is {100*(AU_BPS/1024)/C_VID:+.2f}% -- and the climb is not built out of the wire,
+it is built out of `pipe - wire`, so that is not the number that matters.
+
+  {'ring':>5} {'pipe':>7} {'ceil':>5} {'climb SILENT':>13} {'SOUNDED':>9} {'x':>6}"""
+          f"  {'under, silent':>13} {'under, sounded':>14}")
+    tab1 = []
+    for ring_kb in a.ring:
+        ring = int(ring_kb * 1024)
+        for kbps in a.kbps:
+            fill = (kbps - AU_BPS / 1024) * 1024 / FPS
+            lo, hi, ring_ref, rate_ref = ss.paced_sim(rec, ring, fill)
+            ceil = int(hi.max())
+            cs = climb_s(ceil, C_REC, kbps, C_VID)
+            ca = climb_s(ceil, C_REC, kbps, C_VID + AU_BPS / 1024)
+            us = sum(1 for g in gaps if g < cs)
+            ua = sum(1 for g in gaps if g < ca)
+            ratio = ca / cs if cs not in (0.0, float("inf")) else float("inf")
+            tab1.append((ring_kb, kbps, ceil, cs, ca, ratio, us, ua))
+            print(f"  {ring_kb:5.0f} {kbps:7.1f} {ceil:5d} {cs:12.2f}s "
+                  f"{ca:8.2f}s {ratio:5.2f}x  {f'{us}/{len(gaps)}':>13} "
+                  f"{f'{ua}/{len(gaps)}':>14}")
+
+    worst = max(tab1, key=lambda r: r[5])
+    print(f"""
+  THE SECOND CONSUMER IS {100*(AU_BPS/1024)/C_VID:.1f}% OF THE WIRE AND UP TO {worst[5]:.2f}x OF THE CLIMB.
+  At {worst[1]:.1f} KB/s in a {worst[0]:.0f} KB ring the climb goes {worst[3]:.2f} s -> {worst[4]:.2f} s and the
+  branch points that arrive under it go {worst[6]}/{len(gaps)} -> {worst[7]}/{len(gaps)}
+  ({100*worst[6]/len(gaps):.0f}% -> {100*worst[7]/len(gaps):.0f}%).  Nothing about audio got bigger; the
+  DIFFERENCE it is subtracted from got smaller, and the climb is made of the
+  difference.  This is why 51.4's rate/ring distinction matters more with a
+  second consumer than without one, and why quoting audio as a share of the
+  wire (32, and every budget before it) understates it at every rate close to
+  the wire.
+
+  A CORRECTION TO 56.4, and it is small: that table charged audio at
+  ratectl.AUDIO_KBPS = {RC.AUDIO_KBPS} KB/s, which is {AU_BPS:.1f} B/s expressed in decimal
+  kB (21_iplrom_dmac.py says so).  In binary KB the figure is {AU_BPS/1024:.4f}, i.e.
+  {100*(RC.AUDIO_KBPS-AU_BPS/1024)/(AU_BPS/1024):+.2f}%.  Every column of 56.4 moves in the flattering direction by
+  less than one part in six hundred of the wire.  It is recorded because a
+  placeholder that turns out to be right is still a placeholder.""")
+
+    # ---------------------------------------------------------------- 2
+    print(f"""
+=== 2. THE PACKED BRANCH: THERE IS NO CLIMB, AND THAT IS THE FINDING ====
+  video {P_VID:.1f} + audio {P_AUD:.4f} (F={F0}, A={A0}, {A0*SECTOR:,} B a lump) = {P_VID+P_AUD:.1f} KB/s,
+  which is B1's acceptance figure and is where it comes from.
+
+  DMAC-direct (what src/player/packed.s runs, FINDINGS 64/68):
+    video lookahead 0 records.  The climb does not exist, the ceiling does not
+    exist, and the {len(gaps)} gaps buy it NOTHING -- there is no accumulator for
+    play to fill.  Its acceptance is a PER-FRAME deadline: {P_REC:,} B must land
+    inside every slot, and a rate that averages {P_VID+P_AUD:.1f} KB/s over a second is
+    not the same claim.  56.4's alarming column -- most branch points arrive
+    with less lookahead than the one before them -- does not apply to it,
+    because every frame arrives with less lookahead than the one before it.
+
+  CPU-painted (64.2 column B, 99,328 B, two record buffers -> 1 record of
+  lookahead):
+    {'pipe':>7} {'climb SILENT':>13} {'SOUNDED':>9} {'x':>6}  {'under, silent':>13} {'under, sounded':>14}""")
+    tab2 = []
+    def secs(x, w):
+        return f"{x:{w}.2f}s" if x != float("inf") else f"{'never':>{w+1}}"
+    for kbps in a.pkbps:
+        cs = climb_s(1, P_REC, kbps, P_VID)
+        ca = climb_s(1, P_REC, kbps, P_VID + P_AUD)
+        us = sum(1 for g in gaps if g < cs)
+        ua = sum(1 for g in gaps if g < ca)
+        r = ca / cs if cs not in (0.0, float("inf")) else float("inf")
+        tab2.append((kbps, cs, ca, r, us, ua))
+        print(f"    {kbps:7.1f} {secs(cs,12)} {secs(ca,8)} "
+              f"{(f'{r:5.2f}x' if r != float('inf') else '  inf ')}  "
+              f"{f'{us}/{len(gaps)}':>13} {f'{ua}/{len(gaps)}':>14}")
+    print(f"    -- and EVERY rate in section 1's table is below {P_VID:.1f} KB/s, so "
+          f"none of\n       them serves this container at all.")
+    print(f"""
+  READ THE FIRST ROW. At {P_VID+P_AUD:.1f} KB/s -- the acceptance figure this project
+  quotes -- the SILENT container still climbs its one record in {tab2[0][1]:.2f} s and
+  the SOUNDED one NEVER DOES, because {P_VID+P_AUD:.1f} is where its surplus is exactly
+  zero.  The acceptance figure is the rate at which the sounded container has
+  no lookahead at any amount of play, which is a different thing from the rate
+  at which it plays.
+
+  A packed record is 1.3x the codec's mean record and the packed wire is 1.3x
+  the codec's, so a rate that is generous to one is tight for the other and the
+  same audio debit costs the packed climb more.  ONE RECORD of lookahead is
+  1/12 s of tolerance and it takes seconds of play to earn.
+
+  THE TWO BRANCHES DIFFER HERE ON A COLUMN THAT IS NOT CLOCKS, which is the
+  third time (61.9, 64.2, and this).  The packed branch spent its ring to
+  delete a decoder; what it bought with the RAM is a player with no tolerance
+  for a slow record at ANY time, not merely after a branch.""")
+
+    # ---------------------------------------------------------------- 3
+    sil = silences(within, F0)
+    free = sum(1 for x in sil if x == 0.0)
+    P_PIPE = a.pkbps[1] if len(a.pkbps) > 1 else a.pkbps[0]
+    lump_ms = A0 * SECTOR / (P_PIPE * 1024) * 1000
+    behind = (F0 - 1) * P_REC + A0 * SECTOR
+    print(f"""
+=== 3. THE COST NOBODY HAD COUNTED: entering a group off-boundary =======
+A DLXP2 group is `lump k, then F records` (dlxp.py), so lump k is at a LOWER
+address than every record of its group except the first.  Reading forward from
+record i, the next lump to arrive is k+1, and it carries frame (k+1)*F.  The
+frames from i to (k+1)*F-1 therefore have picture and no sound.
+
+  Measured on the {len(within)} within-container seek targets of the arcade's own
+  graph, at the shipped cadence F={F0}:
+
+    mean   {sum(sil)/len(sil):7.1f} ms of silence entering the branch
+    median {pct(sil,.50):7.1f}    p90 {pct(sil,.90):7.1f}    worst {sil[-1]:7.1f}
+    free   {free}/{len(within)} land on a group boundary and cost nothing
+
+  The other {len(changes)} branch points -- the scene changes -- are FREE, and by
+  construction: lump 0 sits at sector 1 and record 0 at {d.off_frm:,}, so a
+  container's own first bytes are header, lump, record and a scene change reads
+  them in one forward pass.  **The container's start is the one branch point
+  the cadence costs nothing at, and it is the only one anybody had looked at.**
+
+  THE FIX IS A SECOND READ AND NOBODY HAS ONE.  Lump k is {behind:,} B behind
+  record i at worst, so it cannot be picked up by reading early -- it is a
+  separate command at a separate LBA, of {A0*SECTOR:,} B, which at {P_PIPE:.1f} KB/s is
+  {lump_ms:.1f} ms against a mean {sum(sil)/len(sil):.0f} ms of silence -- {sum(sil)/len(sil)/lump_ms:.0f}x cheaper in TIME,
+  one more command per branch, and the command overhead is B1's and unmeasured.
+  src/player/packed.s starts PG_AK and PG_AKF at lump 0 and has no audio seek
+  path at all; the player that branches needs one.
+
+=== 4. THE CADENCE PICK, WITH THE THIRD COLUMN IT DID NOT HAVE ==========
+32_audio_wire.py chose F={F0} on two columns, padding and RAM.  Here is the same
+sweep with the branch column, measured on the game's own seek targets rather
+than assumed uniform:
+
+  {'F':>3} {'A':>3} {'lump B':>8} {'pad%':>7} {'aud KB/s':>9} {'RAM x2':>8} {'mean sil':>9} {'p90':>8} {'worst':>8} {'free':>10} {'vs uniform':>11}""")
+    for F in range(1, a.fsweep + 1):
+        A, lump = cadence(F)
+        need = F * AU_FRAME
+        s = silences(within, F)
+        uni = (F - 1) / 2 / FPS * 1000.0
+        mean = sum(s) / len(s)
+        mark = "  <- shipped" if F == F0 else ""
+        print(f"  {F:3d} {A:3d} {lump:8,} {100*(lump-need)/need:6.2f}% "
+              f"{lump/F*FPS/1024:8.3f} {2*lump:8,} {mean:8.1f} "
+              f"{pct(s,.90):8.1f} {s[-1]:8.1f} "
+              f"{f'{sum(1 for x in s if x == 0)}/{len(s)}':>10} "
+              f"{(mean/uni if uni else 1.0):10.2f}x{mark}")
+
+    ratios = []
+    for F in range(2, a.fsweep + 1):
+        sF_ = silences(within, F)
+        ratios.append((sum(sF_) / len(sF_)) / ((F - 1) / 2 / FPS * 1000.0))
+    min_r, max_r = min(ratios), max(ratios)
+    A1, l1 = cadence(1)
+    AF, lF = cadence(F0)
+    s1, sF = silences(within, 1), silences(within, F0)
+    print(f"""
+  F=1 -- "one lump a record", the cadence 32 called THE WORST ONE -- has no
+  group to enter off-boundary, no second read, no audio seek path and 2,048 B
+  of held lump instead of {2*lF:,}.  It costs {l1/1*FPS/1024 - lF/F0*FPS/1024:+.3f} KB/s of wire, which is
+  {100*(l1/1*FPS/1024 - lF/F0*FPS/1024)/(P_VID+P_AUD):+.2f}% of the packed acceptance figure, and it BUYS BACK {2*lF-2*l1:,} B
+  of RAM on the branch whose whole argument is that RAM is what it has spare.
+
+  THE PICK IS THEREFORE REOPENED, and it is a real trade rather than an error:
+  padding is what F={F0} minimises and padding is not the only thing F sets.
+  A player that gets its audio seek right is indifferent; a player that does
+  not pays a mean {sum(sF)/len(sF):.0f} ms of silence at {len(within)} of the game's {len(gaps)} branch
+  points.  Nothing here decides it -- the deciding number is the SCSI command
+  overhead of the extra read, and that is B1's.
+
+  AND THE CONTENT IS NOT UNIFORM MOD F.  A uniform assumption would put the
+  mean at (F-1)/2 frames; the arcade's seek targets land where they land, and
+  the ratio column above runs {min_r:.2f}x..{max_r:.2f}x over the sweep, so a design
+  that assumed uniform would be out by a quarter at F=3.  At the shipped F={F0} it is
+  {(sum(sF)/len(sF))/((F0-1)/2/FPS*1000):.2f}x, which is a coincidence and is reported as one.
+
+=== 5. WHAT THIS DOES NOT ESTABLISH ====================================
+  1. NO RATE HERE IS MEASURED.  Every pipe column is a sensitivity (FINDINGS
+     50), and B1 -- sustained AND data-phase burst -- is still the user's.
+  2. THE FRAME INDEX OF A SEEK TARGET IS A DESIGN ASSUMPTION.  One container
+     per SCENE (53, 55.1, 56.3).  One container per SEQUENCE makes section 3
+     zero and section 4 moot; nothing else in the file changes.
+  3. NOTHING RAN ON THE MACHINE.  This is arithmetic over a scene table, two
+     containers and a player's own constants.  68's player has never seeked.
+  4. THE MECHANICAL SEEK IS STILL UNMODELLED (51.7.5) and is charged on top of
+     every millisecond here.
+  5. THE SILENCE IS A CONTAINER PROPERTY, NOT A CHIP ONE.  What the MSM6258
+     does when it is not fed -- hold the last sample, or click -- is a board
+     question and belongs with session 34's fifth hardware item.""")
+
+    if a.gate:
+        # Structural assertions.  Not the milliseconds -- those move with the
+        # scene table -- but the ORDER and the SIGNS, which are the finding.
+        ok = True
+
+        def check(cond, msg):
+            nonlocal ok
+            print(f"  {'OK  ' if cond else 'FAIL'} {msg}")
+            ok = ok and bool(cond)
+
+        print("\n=== GATE ===============================================")
+        check(len(gaps) == 612, f"612 transitions into a seek, got {len(gaps)}")
+        check(len(within) + len(changes) == len(gaps),
+              f"{len(within)} within + {len(changes)} scene changes = {len(gaps)}")
+        check(all(r[5] >= 1.0 for r in tab1),
+              "audio never SHORTENS the codec climb")
+        check(max(r[5] for r in tab1) > 1.5,
+              f"and at some rate it more than 1.5x's it "
+              f"({max(r[5] for r in tab1):.2f}x)")
+        check(all(r[7] >= r[6] for r in tab1),
+              "and never lowers the count of branch points under the climb")
+        check(free < len(within) // 2,
+              f"most within-container branches enter a group off-boundary "
+              f"({len(within)-free}/{len(within)})")
+        check(silences(within, 1) == [0.0] * len(within),
+              "F=1 has no off-boundary case at all")
+        check(sum(sil) / len(sil) > 10 * lump_ms,
+              f"the silence F={F0} costs is >10x the lump read that removes it "
+              f"({sum(sil)/len(sil):.0f} ms vs {lump_ms:.1f} ms)")
+        print("  " + ("BRANCH-AUDIO GATE GREEN" if ok else "BRANCH-AUDIO GATE RED"))
+        return 0 if ok else 1
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

tools/analysis/37_audio_seek.py

@@ -0,0 +1,238 @@
+#!/usr/bin/env python3
+"""What a BRANCH costs the chip, and what an encoder could do about it.
+
+    python3 tools/analysis/37_audio_seek.py [container.dlxp] [--raw au.raw]
+    python3 tools/analysis/37_audio_seek.py --gate          # the check.sh stage
+
+FINDINGS 71.  Session 39 put an audio seek path in src/player/packed.s and ran
+it: 132,162 B across a real branch, every byte accounted for in MAME's own
+capture.  That settled the BYTES.  This settles what is left, which is a
+property of the codec rather than of the player and which no counter in the
+player can reach.
+
+THE MSM6258'S ACCUMULATOR HAS NO LEAKAGE TERM.  It is a pure integrator of
+deltas, clamped, and nothing pulls it back toward zero.  So a branch that hands
+the chip bytes chosen for a state it is not in does not produce a transient with
+a time constant -- it produces a DC OFFSET THAT NEVER DECAYS.  The machine run
+measures both designs at one branch point; this measures the CENSUS, over every
+frame boundary of the container, and prices the only fix that is worth anything,
+which is in the encoder and not in the player.
+
+  * PLAY THROUGH the branch: the chip keeps whatever accumulator and step index
+    the previous scene's audio left it in.  Unbounded, and its decay is the
+    signal's own clamping rather than the recursion forgetting.
+
+  * STOP and re-PLAY: the accumulator goes to the container's `init` and the
+    step index to 0 -- a state this script knows exactly, so the error is
+    EXACTLY `init - acc(target)`, constant, forever.
+
+  * ...and the third option is the ENCODER'S: encode the stream with the
+    predictor RESET at every point a branch can land on.  Then a re-PLAYing
+    player is not close, it is exact.  What that costs is a codec question and
+    is measured below.
+
+NAME THE LAYER.  Everything here is host arithmetic over one container and its
+source PCM.  The chip's four axes are the ones FINDINGS 66 measured on the
+machine and 67.3 put in the header; the branch behaviour is the one session 39
+ran.  No emulator is involved and no rate is claimed.
+"""
+import argparse, math, os, statistics, sys
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+sys.path.insert(0, os.path.join(HERE, "..", "encoder"))
+import adpcm
+from dlxp import DLXP
+
+
+def acc_trajectory(nibbles, dec):
+    """The decoder's accumulator after every sample.  This IS the encoder's
+    assumed state, because adpcm.encode runs its decoder inside its own search
+    loop -- the encoder cannot hold a state the decoder will not reach."""
+    lo, hi = adpcm.clamp_bounds(dec["bits"])
+    sig, idx = dec["init"], 0
+    out = []
+    for n in nibbles:
+        sig += adpcm.delta(n, adpcm.STEP[idx], dec["variant"])
+        sig = lo if sig < lo else (hi if sig > hi else sig)
+        idx += adpcm.INDEX_ADJUST[n & 7]
+        idx = 0 if idx < 0 else (48 if idx > 48 else idx)
+        out.append((sig, idx))
+    return out
+
+
+def encode_reset(src12, dec, period):
+    """adpcm.encode with the predictor forced back to (init, 0) every `period`
+    samples.  period=None is the ordinary encode.
+
+    THIS CHANGES THE BYTES, so it is a container property and not a flag a
+    player can set.  It is written here rather than in tools/encoder/adpcm.py
+    because nothing has decided to ship it: 71.5 is the trade and the deciding
+    number is a hardware one."""
+    if period is None:
+        return adpcm.encode(src12, variant=dec["variant"], init=dec["init"],
+                            bits=dec["bits"])
+    out = bytearray()
+    for i in range(0, len(src12), period):
+        out += adpcm.encode(src12[i:i + period], variant=dec["variant"],
+                            init=dec["init"], bits=dec["bits"])
+    return bytes(out)
+
+
+def snr(ref, got):
+    n = min(len(ref), len(got))
+    sig = sum(x * x for x in ref[:n])
+    err = sum((ref[i] - got[i]) ** 2 for i in range(n))
+    if err == 0:
+        return float("inf")
+    return 10 * math.log10(sig / err) if sig else float("-inf")
+
+
+def decode_reset(nib, dec, period):
+    if period is None:
+        return list(adpcm.decode_state(nib, variant=dec["variant"],
+                                       init=dec["init"], bits=dec["bits"])[0])
+    out = []
+    for i in range(0, len(nib), period):
+        out += list(adpcm.decode_state(nib[i:i + period], variant=dec["variant"],
+                                       init=dec["init"], bits=dec["bits"])[0])
+    return out
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("container", nargs="?", default="tmp/packed_singe.dlxp")
+    ap.add_argument("--raw", default="tmp/au_singe.raw")
+    ap.add_argument("--gate", action="store_true")
+    a = ap.parse_args()
+
+    d = DLXP(a.container)
+    if not d.has_audio:
+        sys.exit("this container is silent -- there is no branch to price")
+    dec = d.decoder()
+    lo, hi = adpcm.clamp_bounds(dec["bits"])
+    data = d.audio()
+    nib = adpcm.unpack(data, len(data) * 2, order=dec["order"])
+    traj = acc_trajectory(nib, dec)
+
+    fails = []
+    def ck(ok, msg):
+        print(("OK   " if ok else "FAIL ") + msg)
+        if not ok:
+            fails.append(msg)
+
+    # ---- 1. THE CENSUS.  What a re-PLAY costs at every frame boundary the
+    # container has, which is every point a branch in this design can land on:
+    # 56.3's targets are frame indices and this player seeks to a frame.
+    den = 2 * d.fps
+    # THE INDEX IS A NIBBLE INDEX AND THE POSITION IS A BYTE ONE, which is the
+    # one conversion in this file and it is worth the line: getting it wrong
+    # reads the trajectory at HALF the target and produces a census that is
+    # entirely plausible -- a distribution of the right shape over the wrong
+    # instants.  The cross-check against the machine's own branch point below is
+    # what caught it.
+    pos = lambda f: 2 * (f * d.aud_hz // den) - 1
+    frames = [f for f in range(1, d.nframes) if pos(f) < len(traj)]
+    dcs = [abs(dec["init"] - traj[pos(f)][0]) for f in frames]
+    idxs = [traj[pos(f)][1] for f in frames]
+    dcs_s = sorted(dcs)
+    p = lambda q: dcs_s[min(len(dcs_s) - 1, int(q * len(dcs_s)))]
+    print(f"--- 1. A RE-PLAYED BRANCH COSTS `init - acc(target)`, EXACTLY AND "
+          f"FOREVER.  {len(frames)} frame boundaries of {a.container}:")
+    print(f"     |DC| against the {hi}-unit clamp: mean {statistics.mean(dcs):.1f} "
+          f"({statistics.mean(dcs)*100/hi:.1f}%), median {statistics.median(dcs):.0f}, "
+          f"p90 {p(0.90)}, worst {max(dcs)} ({max(dcs)*100/hi:.1f}%)")
+    print(f"     ...and the step index the encoder assumed at those points runs "
+          f"{min(idxs)}..{max(idxs)} of 48, median {statistics.median(idxs):.0f} "
+          f"-- a re-PLAY sets it to 0, so a branch into a LOUD passage gets the "
+          f"offset AND a step index that has to climb back")
+    # The machine run's own branch, so the two layers are checked against each
+    # other rather than merely agreeing in prose.
+    F37 = 37
+    dc37 = dec["init"] - traj[pos(F37)][0]
+    print(f"     frame {F37}, the branch tools/bench/packed_run.sh runs on the "
+          f"machine: DC {dc37} -- and MAME's capture measured the chip at "
+          f"exactly that, constant over 62,500 samples (FINDINGS 71.3)")
+    ck(abs(dc37) == 65,
+       f"the host's arithmetic for the machine's own branch point is {abs(dc37)} "
+       f"and the capture said 65 -- one number, two layers")
+
+    # ---- 2. THE DECAY THAT ISN'T.  A re-PLAY's error is constant BY
+    # CONSTRUCTION -- same step index, same nibbles, one offset -- and playing
+    # through is not, because the step indices differ too.  The point of
+    # measuring it here is that the constancy is a PROPERTY OF THE PREDICTOR
+    # and not of the ten seconds this container happens to hold.
+    print(f"--- 2. AND IT DOES NOT DECAY.  The accumulator is an integrator with "
+          f"no leak: a re-PLAY changes the STARTING value and nothing else, so "
+          f"the same nibbles produce the same deltas and the offset is carried "
+          f"to the end of the stream. The machine agrees -- AC 0.00 over four "
+          f"seconds (FINDINGS 71.3). Playing THROUGH the branch is worse and is "
+          f"not constant, because the step index differs as well: -355 falling "
+          f"to -108 over four seconds, which is clamping and not forgetting.")
+
+    # ---- 3. THE ENCODER'S FIX, PRICED.  Reset the predictor where a branch can
+    # land and a re-PLAYing player is EXACT rather than close.
+    if not os.path.exists(a.raw):
+        print(f"     (no {a.raw}: the encoder trade below needs the source PCM)")
+        return 1 if fails else 0
+    import struct
+    pcm = struct.unpack("<%dh" % (os.path.getsize(a.raw) // 2),
+                        open(a.raw, "rb").read())
+    src12 = [max(-2048, min(2047, x >> 4)) for x in pcm][:len(nib)]
+    per_frame = d.aud_hz // den * 2            # samples in one frame slot
+    print(f"--- 3. THE ONLY FIX THAT MAKES A BRANCH FREE IS THE ENCODER'S, and "
+          f"here is its bill.  Reset the predictor every N frames when encoding; "
+          f"a player that re-PLAYs at a branch landing on one of those points is "
+          f"then EXACT, not close:")
+    print(f"       {"reset every":>24}  {'SNR dB':>8}  {'vs shipped':>10}  "
+          f"{'branch points made free':>24}")
+    base = None
+    rows = []
+    for label, period in [("never (shipped)", None),
+                          (f"{d.cad_f} frames (the cadence)", d.cad_f * per_frame),
+                          ("1 frame", per_frame)]:
+        nb = encode_reset(src12, dec, period)
+        got = decode_reset(nb, dec, period)
+        v = snr(src12, got)
+        if base is None:
+            base = v
+        free = (0 if period is None
+                else (len(frames) // d.cad_f if period != per_frame
+                      else len(frames)))
+        rows.append((label, v, v - base, free))
+        print(f"       {label:>24}  {v:8.2f}  {v-base:+10.2f}  "
+              f"{free:>15} of {len(frames)}")
+    # THE ASSERTION IS THE ORDER AND THE SIGN, not the decibel: the source PCM
+    # is a property of the disc and the encoder is greedy, so the exact figures
+    # move with the window. What must not move is that resetting COSTS SNR and
+    # that resetting more often costs more -- if it ever came out free, the
+    # predictor would not be doing anything and the codec would be pointless.
+    ck(rows[1][1] <= rows[0][1] + 1e-9 and rows[2][1] <= rows[1][1] + 1e-9,
+       f"resetting the predictor costs SNR, and resetting it more often costs "
+       f"more: {rows[0][1]:.2f} -> {rows[1][1]:.2f} -> {rows[2][1]:.2f} dB")
+    ck(rows[2][1] > rows[0][1] - 3.0,
+       f"...and a reset EVERY FRAME is {rows[0][1]-rows[2][1]:.2f} dB, which is "
+       f"the price of making all {len(frames)} of this container's branch points "
+       f"exact. The step table's floor is a constant 16 and the recursion "
+       f"re-converges in a few samples, which is why twelve resets a second is "
+       f"not twelve times anything")
+
+    print(f"--- 4. WHAT THIS DOES NOT SETTLE.")
+    print(f"     * Nothing here is a rate and nothing here ran on silicon. The "
+          f"branch behaviour is MAME's okim6258 -- PLAY sets the accumulator to "
+          f"-2, the step index to 0 and the nibble select to 0 -- which is the "
+          f"model FINDINGS 66 fitted to the machine and NOT a measurement of an "
+          f"MSM6258V. It joins session 34's fifth hardware item.")
+    print(f"     * The census is ONE container, ten seconds, one passage at "
+          f"-13.4 dBFS (FINDINGS 69). The offset a re-PLAY costs is the signal's "
+          f"own value at the cut, so a louder passage costs more, up to the "
+          f"clamp -- and the disc peaks at 946 of 2048 (69.2).")
+    print(f"     * The reset-every-frame encode is NOT in tools/encoder. It is "
+          f"a container change (a DLXP3), it costs bytes nothing and SNR "
+          f"something, and what decides it is whether a branch is allowed to "
+          f"land anywhere or only on frames the encoder was told about.")
+    print("AUDIO SEEK GATE " + ("GREEN" if not fails else f"RED: {len(fails)}"))
+    return 1 if fails else 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

tools/analysis/mc68450.py

@@ -0,0 +1,59 @@
+"""MC68450 / HD63450 register field layouts, in ONE copy.
+
+Read by tools/analysis/21_iplrom_dmac.py, which decodes what the X68000's IPL
+ROM programs into the DMAC, and by tools/analysis/27_dmac_config.py, which
+decodes what src/player/dma.i programs into it.  The two exist to be COMPARED
+-- the ROM's own disk channel costs 16..19 clocks a byte (FINDINGS 52.5) and
+the player's job is to be cheaper -- and a comparison between two decodings
+that used two copies of these tables would not be one.  This tree has already
+paid twice for a transform with two copies of itself (FINDINGS 49.7.5).
+
+SOURCED: MC68450 Direct Memory Access Controller, Motorola, Jul 1989
+(bitsavers) -- the same document FINDINGS 39 cites for the transfer timings in
+tools/analysis/buscost.py.
+"""
+# --- MC68450 register map, by offset inside a channel's 0x40 block ----------
+REG = {0x00: "CSR", 0x01: "CER", 0x04: "DCR", 0x05: "OCR", 0x06: "SCR",
+       0x07: "CCR", 0x0A: "MTC", 0x0C: "MAR", 0x14: "DAR", 0x1A: "BTC",
+       0x1C: "BAR", 0x25: "NIV", 0x27: "EIV", 0x29: "MFC", 0x2D: "CPR",
+       0x31: "DFC", 0x39: "BFC"}
+
+XRM  = {0: "burst",
+        1: "UNDEFINED",
+        2: "cycle steal WITHOUT hold  (bus released between operands)",
+        3: "cycle steal with hold"}
+DTYP = {0: "68000-compatible, EXPLICITLY addressed  -> DUAL ADDRESS",
+        1: "6800-compatible, EXPLICITLY addressed   -> DUAL ADDRESS",
+        2: "device with ACK, implicitly addressed   -> SINGLE ADDRESS",
+        3: "device with ACK and RDY, implicit       -> SINGLE ADDRESS"}
+DPS  = {0: "8-bit port", 1: "16-bit port"}
+PCL  = {0: "status input", 1: "status input with interrupt",
+        2: "start pulse", 3: "abort input"}
+SIZE = {0: "byte", 1: "word", 2: "long word", 3: "byte, unpacked"}
+CHAIN= {0: "none", 1: "UNDEFINED", 2: "array", 3: "linked array"}
+REQG = {0: "auto-request at limited rate", 1: "auto-request at max rate",
+        2: "EXTERNAL request  (one operand per device request)",
+        3: "auto-request first operand, external thereafter"}
+
+
+def dcr(v):
+    return [f"XRM  = {v>>6&3:02b}  {XRM[v>>6&3]}",
+            f"DTYP = {v>>4&3:02b}  {DTYP[v>>4&3]}",
+            f"DPS  = {v>>3&1:b}   {DPS[v>>3&1]}",
+            f"PCL  = {v&3:02b}  {PCL[v&3]}"]
+
+
+def ocr(v):
+    return [f"DIR  = {v>>7&1:b}   " +
+            ("device -> memory (read)" if v & 0x80 else "memory -> device (write)"),
+            f"SIZE = {v>>4&3:02b}  {SIZE[v>>4&3]}",
+            f"CHAIN= {v>>2&3:02b}  {CHAIN[v>>2&3]}",
+            f"REQG = {v&3:02b}  {REQG[v&3]}"]
+
+
+def scr(v):
+    m = {0: "no count", 1: "increment", 2: "decrement", 3: "UNDEFINED"}
+    return [f"MAC  = {v>>2&3:02b}  memory address {m[v>>2&3]}",
+            f"DAC  = {v&3:02b}  device address {m[v&3]}"]
+
+

tools/bench/adpcm.lua

@@ -0,0 +1,88 @@
+-- Play one buffer of ADPCM nibbles on the emulated MSM6258V, from 68000 code
+-- (ROADMAP P6a).  The Lua here does what Lua is allowed to do in this tree:
+-- push bytes in, start the CPU, read the mailbox out.  It is NOT in the feed
+-- path -- session 33's probe was, and a host that writes the data register at
+-- host-frame rate is not feeding a chip that consumes at 15,625 Hz (65.5).
+--
+-- WHAT THE MEASUREMENT IS.  MAME's -wavwrite capture, at a sample rate chosen
+-- to EQUAL the chip's stream rate so nothing resamples it, is the chip's own
+-- output.  tools/bench/verify_adpcm_chip.py reads the four model axes out of it.
+M = manager.machine
+SP = M.devices[":maincpu"].spaces["program"]
+
+local META = loadfile("adpcm_meta.lua")()
+local AD_FLAG, AD_BUF, AD_LEN = 0x18600, 0x18604, 0x18608
+local AD_MTC0, AD_CSRF, AD_CERF = 0x1860C, 0x18610, 0x18614
+local AD_MTCF, AD_MARF, AD_SPIN, AD_STAT = 0x18618, 0x1861C, 0x18620, 0x18624
+
+local code do local f=io.open("adpcmgate.bin","rb"); code=f:read("a"); f:close() end
+local data do local f=io.open("adpcm_data.bin","rb"); data=f:read("a"); f:close() end
+
+local function P(s) print("[ADP] "..s) end
+local function T() local t=M.time; return t.seconds + t.attoseconds/1e18 end
+
+local st, t0, tplay = "boot", nil, nil
+SUB = emu.add_machine_frame_notifier(function()
+  local ok, err = pcall(function()
+    local t = T()
+    if st == "boot" then
+      if t < 3.0 then return end
+      for i = 1, #data do SP:write_u8(META.buf + i - 1, string.byte(data, i)) end
+      for i = 1, #code do SP:write_u8(0x10000 + i - 1, string.byte(code, i)) end
+      SP:write_u32(AD_FLAG, 0)
+      SP:write_u32(AD_BUF, META.buf)
+      SP:write_u32(AD_LEN, META.nbytes)
+      local cpu = M.devices[":maincpu"]
+      cpu.state["SR"].value = 0x2700      -- supervisor, ALL interrupts masked
+      cpu.state["SP"].value = 0x8000
+      cpu.state["PC"].value = 0x10000
+      P(string.format("pushed %d B of code and %d B of nibbles at 0x%X",
+                      #code, META.nbytes, META.buf))
+      -- THE CAPTURE'S OWN CLOCK.  The wav starts at t=0 of the run, so the host
+      -- has to know when PLAY happened to find the stream in it -- but it is
+      -- NOT used as the alignment: the verifier searches a small window around
+      -- it, because a host frame is 17.6 ms and a sample is 64 us.
+      st, t0 = "running", t
+      return
+    end
+    if st == "running" then
+      local fl = SP:read_u32(AD_FLAG)
+      if fl == 2 and not tplay then
+        tplay = t
+        P(string.format("PLAY at t=%.4f s, chip status $%02X (bit7 clear = playing), "
+                        .."MTC then = %d of %d", t, SP:read_u32(AD_STAT),
+                        SP:read_u32(AD_MTC0), META.nbytes))
+      end
+      if fl == 0xFF or fl == 0xEE then
+        P(string.format("channel finished: CSR=$%02X CER=$%02X MTC=%d MAR=$%06X "
+                        .."spin=%d", SP:read_u32(AD_CSRF), SP:read_u32(AD_CERF),
+                        SP:read_u32(AD_MTCF), SP:read_u32(AD_MARF),
+                        SP:read_u32(AD_SPIN)))
+        local dt = t - (tplay or t)
+        P(string.format("%d bytes took %.4f s = %.1f B/s "
+                        .."(15,625 nibbles/s wants 7,812.5)",
+                        META.nbytes, dt, META.nbytes/dt))
+        if fl == 0xEE then P("ERROR: the gate flagged a channel error or a timeout") end
+        local f = io.open("adpcm_run.lua", "w")
+        f:write(string.format("return { tplay = %.9f, ok = %s, nbytes = %d,\n"
+                              .."  csr = %d, cer = %d, mtc = %d, spin = %d }\n",
+                              tplay or -1, tostring(fl == 0xFF), META.nbytes,
+                              SP:read_u32(AD_CSRF), SP:read_u32(AD_CERF),
+                              SP:read_u32(AD_MTCF), SP:read_u32(AD_SPIN)))
+        f:close()
+        st = "drain"; t0 = t
+        return
+      end
+      if t - t0 > 60 then P("TIMEOUT flag="..string.format("%08X", fl)); M:exit() end
+      return
+    end
+    if st == "drain" then
+      -- let the capture run past the end of the stream, so a truncated wav is
+      -- never mistaken for a short stream
+      if t - t0 < 0.3 then return end
+      P("done")
+      M:exit()
+    end
+  end)
+  if not ok then print("[ADP] LUA ERROR: "..tostring(err)); M:exit() end
+end)

tools/bench/adpcm_run.sh

@@ -0,0 +1,56 @@
+#!/bin/bash
+# ROADMAP P6a: ask the machine's own MSM6258V which decoder it is.
+#
+#   tools/bench/adpcm_run.sh
+#
+# WHAT A GREEN RUN MEANS: 68000 code programmed HD63450 channel 3 exactly as the
+# IPL ROM programs it -- dual address, 8-bit port, cycle steal, EXTERNAL request
+# -- fed the chip a designed nibble stream at the chip's own pace, and exactly
+# ONE of sixteen candidate decoder models reproduces MAME's capture of the
+# result SAMPLE-EXACT, with every one of the four axes shown to matter.
+#
+# WHAT IT DOES NOT MEAN: anything about an MSM6258.  This is MAME's device model
+# measured end to end through the machine's real transport.  It settles the RIG.
+# The silicon stays on the hardware list.
+set -e
+cd "$(dirname "$0")/../.."
+
+tools/vasm/vasmm68k_mot -Fbin -o tmp/adpcmgate.bin src/player/adpcmgate.s > /dev/null
+python3 tools/bench/prep_adpcm.py
+
+# -samplerate 15625 is not a preference: it is the chip's own stream rate
+# (8 MHz / 512), and equal rates are what keep MAME's resampler from filtering
+# the thing being measured.  The first cut of this ran at the default 48000 and
+# every reconstructed sample arrived as an interpolated pair.
+( cd tmp && SDL_VIDEODRIVER=dummy stdbuf -oL timeout -k 5 300 \
+    mame x68000 -bios ipl10 -ramsize 2M -video soft -window \
+    -samplerate 15625 -wavwrite adpcm.wav -nothrottle -plugins \
+    -autoboot_script ../tools/bench/adpcm.lua \
+    -seconds_to_run 12 > adpcm_run.log 2>&1 )
+grep -aq "^\[ADP\] done" tmp/adpcm_run.log || {
+  echo "FAIL: the ADPCM gate did not finish -- no completion marker."
+  tail -8 tmp/adpcm_run.log; exit 1; }
+grep -a "^\[ADP\]" tmp/adpcm_run.log | sed 's/^\[ADP\] /  /'
+
+fail() { echo "FAIL: $1"; exit 1; }
+if grep -aq "^\[ADP\] ERROR" tmp/adpcm_run.log; then
+  fail "the channel reported an error or the gate timed out -- see CSR/CER above."
+fi
+grep -aq "bit7 clear = playing" tmp/adpcm_run.log || \
+  fail "the chip never reported itself playing."
+# THE FEED RATE IS A GATE, not a note.  The chip is the pacemaker: one byte per
+# #DRQ3 and #DRQ3 at half the sample rate.  If the bytes went out at some other
+# rate then the channel was NOT being paced by the device, and every sample
+# below is of a stream that arrived faster or slower than it was consumed --
+# which is precisely the failure session 33 hit from Lua.
+RATE=$(sed -n 's/.*= \([0-9.]*\) B\/s .*/\1/p' tmp/adpcm_run.log | head -1)
+python3 - "$RATE" <<'PY' || fail "the feed was not paced by the chip (see above)."
+import sys
+r = float(sys.argv[1])
+want = 7812.5
+print(f"  feed rate {r:,.1f} B/s against the chip's own {want:,.1f} B/s "
+      f"({100*(r-want)/want:+.2f}%)")
+sys.exit(0 if abs(r - want) / want < 0.02 else 1)
+PY
+
+python3 tools/bench/verify_adpcm_chip.py tmp/adpcm.wav tmp/adpcm_seq.json

tools/bench/blit.lua

@@ -11,7 +11,8 @@
 -- a LOWER BOUND, not a prediction.  Interrupts are masked (SR=$2700) so the
 -- IPL's timer and VBL handlers cannot steal cycles into the measurement.
 --
--- Timing resolution is one video frame (1/55.46 s = 18.03 ms), because Lua
+-- Timing resolution is one video frame (1/56.69 s = 17.64 ms -- MAME's, not
+-- the hardware's 55.46; see crtc_mode.lua), because Lua
 -- gets no cycle counter -- luaengine.cpp exposes machine.time and nothing
 -- from device_execute_interface.  Each variant therefore loops enough times
 -- to run ~4 emulated seconds, putting the granularity error near 0.4%.
@@ -28,7 +29,7 @@ end
 local MODE = load_mode()
 
 local FLAG, VAR, ITER = 0x18000, 0x18004, 0x18008
-local SRCW, SRCB      = 0x60000, 0x80000
+local SRCW, SRCB, SRCP = 0x60000, 0x80000, 0x90000
 local GVRAM, GPAL     = 0xC00000, 0xE82000
 local CPUHZ           = 10000000            -- x68k.cpp:1133, 40_MHz_XTAL/4
 local FRAME12         = CPUHZ / 12          -- 833333 cycles at 12 fps
@@ -39,6 +40,16 @@ local PLAN = {
   {var=2, iter= 50, name="V2 naive byte-source expansion (move.b/move.w per pixel)"},
   {var=3, iter=200, name="V3 write-only floor (no source read at all)"},
   {var=4, iter= 60, name="V4 same 96KB of writes, issued in 4x4 BLOCK order (decoder access pattern)"},
+  -- V8 is V1 with R20 bit 11's packing: 48KB read + 48KB write for the SAME
+  -- 49,152 pixels.  It is the per-frame work of a decoder-free packed player
+  -- (FINDINGS 44.7 / 46.6 / 47.5), and 47.6.1 filed its `movem` shape as an
+  -- ASSUMPTION -- this is the measurement that assumption was standing in for.
+  {var=8, iter=200, name="V8 PACKED movem.l blit (48KB read + 48KB write, same 49,152 pixels as V1)"},
+  -- V9/V10 are the two ways a BLOCK decoder could survive the packed layout
+  -- (47.6.4, open since session 16): sixteen move.b at stride 2 per block, or
+  -- pair the blocks 128 columns apart in the encoder and get V4's movem back.
+  {var= 9, iter= 40, name="V9 PACKED block order, 16 move.b at stride 2 per 4x4 block"},
+  {var=10, iter=120, name="V10 PACKED block order, blocks PAIRED so a movem writes whole words"},
 }
 
 local code do
@@ -96,6 +107,18 @@ local function setup()
       SP:write_u8 (SRCB + y*256 + x,   px)
     end
   end
+  -- SRCP: the PACKED frame, interleaved the way tools/bench/show_frame256_packed.lua
+  -- lays it out -- word i of a row is (column i+128) << 8 | (column i), because
+  -- page 0 is the low byte at screen column i and page 1 the high byte at i+128.
+  -- Only V8 reads it, and only its SIZE (128 words a row) affects the timing;
+  -- the interleave is written correctly so the buffer is the real artefact and
+  -- not a same-sized stand-in.
+  for y = 0, H-1 do
+    local row = PIX0 + y*W
+    for i = 0, (W//2)-1 do
+      SP:write_u16(SRCP + y*(W//2)*2 + i*2, (B(row+i+W//2) << 8) | B(row+i))
+    end
+  end
   for i = 1, #code do SP:write_u8(0x10000+i-1, string.byte(code,i)) end
   P(string.format("loaded blit.bin=%d bytes, source frame %dx%d at yoff=%d", #code, W, H, YOFF))
 end

tools/bench/blit.s

@@ -104,6 +104,9 @@ ITER    = $18008                ; iteration count, written by Lua
 SPTR    = $1800C                ; V5 span stream pointer, written by Lua
 SRCW    = $60000                ; word-expanded frame  192*512 = 96KB
 SRCB    = $80000                ; byte-per-pixel frame 192*256 = 48KB
+SRCP    = $90000                ; PACKED frame 192*256 = 48KB (V8): two picture
+                                ; bytes per word, already interleaved by the
+                                ; encoder, so the blit is a straight copy
 DST0    = $C08000               ; GVRAM + 32*1024   (first picture row)
 DSTE    = $C38000               ; GVRAM + 224*1024  (one past last)
 ROWS    = 192                   ; picture rows a V5 stream describes
@@ -130,6 +133,12 @@ start:
         beq     v6
         cmp.l   #7,d0
         beq     v7
+        cmp.l   #8,d0
+        beq     v8
+        cmp.l   #9,d0
+        beq     v9
+        cmp.l   #10,d0
+        beq     v10
         bra     v3
 
 ; ---------------------------------------------------------------- V1
@@ -372,5 +381,142 @@ v7fh:
         bne     v7
         bra     done
 
+; ---------------------------------------------------------------- V8
+; THE PACKED FULL-FRAME BLIT (FINDINGS 46.6/47.2).  Identical in shape to V1 --
+; a row-linear movem.l chain out of a RAM frame into GVRAM -- and different in
+; exactly one thing: a row is 128 WORDS, not 256, because R20 bit 11 lets one
+; word carry two picture bytes.  256 = 5*48 + 16, so five 12-register bursts
+; and a 4-register tail, against V1's ten and one.
+;
+; TIMING ONLY, and it does not set bit 11.  MAME's gvram_w carries no timing in
+; either arm (blit.lua's header), so the bit cannot move a cycle here; what it
+; moves is the PICTURE, and the picture is what tools/bench/show_frame256_packed.lua
+; and tools/bench/gvpack already verify pixel-exactly.  Setting it here would
+; make this variant's snapshot right and its measurement no different, and
+; would put a display-mode change inside a timing loop for no gain.
+;
+; The source is PRE-INTERLEAVED by the host, which is the honest half of the
+; claim: the packing is an encoder-side transform (46.3's argument for the text
+; plane, and the same one here), so the decoder-free player's per-frame work is
+; this copy and nothing else.  If the interleave had to happen at run time this
+; variant would be V2, not V1.
+v8:     lea     SRCP,a0
+        lea     DST0,a1
+        lea     DSTE,a6
+v8row:  movem.l (a0)+,d0-d7/a2-a5
+        movem.l d0-d7/a2-a5,(a1)
+        movem.l (a0)+,d0-d7/a2-a5
+        movem.l d0-d7/a2-a5,48(a1)
+        movem.l (a0)+,d0-d7/a2-a5
+        movem.l d0-d7/a2-a5,96(a1)
+        movem.l (a0)+,d0-d7/a2-a5
+        movem.l d0-d7/a2-a5,144(a1)
+        movem.l (a0)+,d0-d7/a2-a5
+        movem.l d0-d7/a2-a5,192(a1)
+        movem.l (a0)+,d0-d3
+        movem.l d0-d3,240(a1)
+        lea     1024(a1),a1
+        cmpa.l  a6,a1
+        bne     v8row
+        subq.l  #1,ITER.l
+        bne     v8
+        bra     done
+
+; ---------------------------------------------------------------- V9
+; WHAT THE PACKED LAYOUT COSTS A BLOCK DECODER (FINDINGS 47.6.4, open).
+;
+; V4 is the access pattern of a decoder that writes 4x4 codewords straight into
+; GVRAM: 4 rows of 8 contiguous bytes at a 1024-byte stride, so each row is one
+; `movem.l` of two registers.  Under the packed layout that pattern is GONE.
+; A block at columns x..x+3 owns the LOW bytes of four consecutive words -- four
+; bytes at STRIDE 2 -- and the high bytes of those same words belong to the
+; block 128 columns away.  There is no burst that writes every other byte, so
+; the block is sixteen `move.b`s.
+;
+; V9 does the pair together, low block then high block off one base, so it
+; writes every byte it touches and covers the same 49,152 pixels V1/V4/V8 do.
+; It is the HONEST version of "keep the codec and pack the screen": the mode
+; map is unchanged, SKIP still works per block, and the writes go byte at a
+; time.  V10 below is the other option, and the comparison is the point.
+v9:     lea     SRCB,a0
+        lea     DST0,a3
+        lea     DSTE,a4
+v9brow: move.l  a3,a1
+        lea     256(a3),a5              ; 32 block PAIRS * 8 bytes
+v9blk:
+        move.b  (a0)+,(a1)
+        move.b  (a0)+,2(a1)
+        move.b  (a0)+,4(a1)
+        move.b  (a0)+,6(a1)
+        move.b  (a0)+,1024(a1)
+        move.b  (a0)+,1026(a1)
+        move.b  (a0)+,1028(a1)
+        move.b  (a0)+,1030(a1)
+        move.b  (a0)+,2048(a1)
+        move.b  (a0)+,2050(a1)
+        move.b  (a0)+,2052(a1)
+        move.b  (a0)+,2054(a1)
+        move.b  (a0)+,3072(a1)
+        move.b  (a0)+,3074(a1)
+        move.b  (a0)+,3076(a1)
+        move.b  (a0)+,3078(a1)
+        move.b  (a0)+,1(a1)
+        move.b  (a0)+,3(a1)
+        move.b  (a0)+,5(a1)
+        move.b  (a0)+,7(a1)
+        move.b  (a0)+,1025(a1)
+        move.b  (a0)+,1027(a1)
+        move.b  (a0)+,1029(a1)
+        move.b  (a0)+,1031(a1)
+        move.b  (a0)+,2049(a1)
+        move.b  (a0)+,2051(a1)
+        move.b  (a0)+,2053(a1)
+        move.b  (a0)+,2055(a1)
+        move.b  (a0)+,3073(a1)
+        move.b  (a0)+,3075(a1)
+        move.b  (a0)+,3077(a1)
+        move.b  (a0)+,3079(a1)
+        addq.l  #8,a1
+        cmpa.l  a5,a1
+        bne     v9blk
+        lea     4096(a3),a3
+        cmpa.l  a4,a3
+        bne     v9brow
+        subq.l  #1,ITER.l
+        bne     v9
+        bra     done
+
+; ---------------------------------------------------------------- V10
+; THE OTHER OPTION: PAIR THE BLOCKS IN THE ENCODER.  If the codec codes the
+; block at x and the block at x+128 as ONE unit, the destination is whole words
+; again and V4's `movem.l` shape comes straight back -- the same instructions,
+; the same 32 bytes of source per unit, and TWICE the pixels, because a word now
+; carries two of them.  So V10 is V4's inner loop run half as many times.
+;
+; WHAT IT COSTS IS NOT IN THIS MEASUREMENT.  A pair skips only if BOTH of its
+; blocks skip, and the two are 128 columns apart with nothing in the picture
+; relating them.  That is a CONTAINER question -- what fraction of the mode map
+; survives pairing -- and 08_mode_map.py has the data to answer it.  V10 prices
+; the paint; it does not price the SKIPs the pairing loses.
+v10:    lea     SRCP,a0
+        lea     DST0,a3
+        lea     DSTE,a4
+v10brow: move.l a3,a1
+        lea     256(a3),a5              ; 32 block PAIRS * 8 bytes
+v10blk: movem.l (a0)+,d0-d7             ; 32 bytes = one PAIR of 4x4 blocks
+        movem.l d0-d1,(a1)
+        movem.l d2-d3,1024(a1)
+        movem.l d4-d5,2048(a1)
+        movem.l d6-d7,3072(a1)
+        addq.l  #8,a1
+        cmpa.l  a5,a1
+        bne.s   v10blk
+        lea     4096(a3),a3
+        cmpa.l  a4,a3
+        bne     v10brow
+        subq.l  #1,ITER.l
+        bne     v10
+        bra     done
+
 done:   move.l  #$FF,FLAG.l             ; timer stops here
 halt:   bra.s   halt

tools/bench/c68k/harness.c

@@ -55,6 +55,12 @@ void p6logd(const char *fmt, ...) { (void)fmt; }
 #define GV_HI     0xC80000u
 
 #define FLAG      0x18000u
+#define LFLAG     0x18040u              /* src/player/load.i's control block */
+#define LHDR      0x18044u
+#define LDARK     0x18048u
+#define LMODE     0x18054u
+#define LITER     0x18058u
+#define GPAL      0xE82000u
 #define ITER      0x18008u
 #define NFR       0x1800Cu
 #define FPTR      0x18010u
@@ -106,9 +112,16 @@ static void wr8(unsigned int a, unsigned char d)
  * momentarily reads back as $FF again.  Without in_exec that transient
  * recorded a run's stop cycle before the run had started, and every frame
  * after the first came out as the whole slice. */
+/* Which flag word the run watches.  decode.s and stream.s use FLAG; the
+ * load-time transforms of src/player/load.i use their own, so that a player
+ * could eventually contain both without one clearing the other's state.  The
+ * VALUES mean the same thing in both (1 running, $FF done, $EE failed), which
+ * is why one hook serves both. */
+static unsigned int flag_adr = FLAG;
+
 static void note_flag(void)
 {
-    unsigned int v = rd32(FLAG);
+    unsigned int v = rd32(flag_adr);
     long long now = slice - C68K.ICount;
     if (!in_exec) return;
     if (v == 1 && cyc_start < 0) cyc_start = now;
@@ -124,7 +137,7 @@ static void wr16(unsigned int a, unsigned short d)
     a &= ADRMASK;
     if (a >= GV_LO && a < GV_HI) { buf[a] = (unsigned char)d; buf[a+1] = 0; return; }
     buf[a] = (unsigned char)d; buf[a+1] = (unsigned char)(d >> 8);
-    if (a >= FLAG && a < FLAG + 4) note_flag();
+    if (a >= flag_adr && a < flag_adr + 4) note_flag();
 }
 
 static void wr32(unsigned int a, unsigned int d){ wr16(a, (unsigned short)(d >> 16)); wr16(a+2, (unsigned short)d); }
@@ -185,12 +198,104 @@ static long long run(unsigned int off, unsigned int nfr, unsigned int iter)
     return cyc_stop - cyc_start;
 }
 
+/* ---- the load-time transforms (ROADMAP P1+P2, FINDINGS 53) --------------
+ * The same question this harness asks of the decoder, asked of the loader: does
+ * a SECOND 68000 core, with its own cycle table and its own memory model,
+ * produce the same bytes and agree about what they cost?  It also counts BUS
+ * cycles, which MAME cannot report -- and the bus is the resource this project
+ * established is the binding one (FINDINGS 38).
+ */
+static int run_load(const char *fcode, const char *fraw, const char *dump,
+                    unsigned int mode, unsigned int iter,
+                    unsigned int cb1_len, unsigned int cb4_len)
+{
+    size_t nc, nr;
+    unsigned char *code = slurp(fcode, &nc), *raw = slurp(fraw, &nr);
+    push(STREAM, raw, nr);                      /* the RAW container header */
+    push(CODE, code, nc);
+    /* Poison every destination, so that a transform which writes NOTHING
+     * cannot pass by leaving the harness's own zeros in place. */
+    for (unsigned int a = CB1; a < CB1 + cb1_len; a += 2) wr16(a, 0xDEAD);
+    for (unsigned int a = CB4; a < CB4 + cb4_len; a += 2) wr16(a, 0xDEAD);
+    for (unsigned int c = 0; c < 256; c++) wr16(GPAL + c*2, 0xDEAD);
+    wr32(LDARK, 0xFFFFFFFFu);
+    /* The three scratch tables are poisoned only before a run that claims to
+     * build them.  A run that only PACKS the palette is entitled to find them
+     * already built -- that is the point of pricing it separately -- so when
+     * this process is asked for one, it does the boot pass first, untimed,
+     * exactly as a player would have done at boot.  Without that the pack runs
+     * on zeros: every entry then takes the same branch and the darkest entry
+     * comes out 0, which is a measurement of nothing. */
+    if (mode & 4)
+        for (unsigned int a = 0x19000; a < 0x19340; a += 2) wr16(a, 0xDEAD);
+
+    flag_adr = LFLAG;
+    if ((mode & 2) && !(mode & 4)) {
+        cyc_start = cyc_stop = -1; desync = 0;
+        wr32(LFLAG, 0); wr32(LHDR, STREAM); wr32(LMODE, 4); wr32(LITER, 1);
+        C68k_Reset(&C68K);
+        C68k_Set_Reg(&C68K, C68K_SR, 0x2700);
+        C68k_Set_Reg(&C68K, C68K_A7, STACK);
+        C68k_Set_Reg(&C68K, C68K_PC, CODE);
+        slice = 2000000000LL; in_exec = 1;
+        C68k_Exec(&C68K, (INT32)slice);
+        in_exec = 0;
+        if (cyc_stop < 0) { fprintf(stderr, "TIMEOUT in the table pre-pass\n"); return 4; }
+    }
+    cyc_start = cyc_stop = -1; desync = 0; bus_r = bus_w = 0;
+    wr32(LFLAG, 0); wr32(LHDR, STREAM); wr32(LMODE, mode); wr32(LITER, iter);
+    C68k_Reset(&C68K);
+    C68k_Set_Reg(&C68K, C68K_SR, 0x2700);
+    C68k_Set_Reg(&C68K, C68K_A7, STACK);
+    C68k_Set_Reg(&C68K, C68K_PC, CODE);
+    slice = 2000000000LL;
+    in_exec = 1;
+    C68k_Exec(&C68K, (INT32)slice);
+    in_exec = 0;
+    if (cyc_stop < 0) { fprintf(stderr, "TIMEOUT -- loader never set LFLAG\n"); return 4; }
+    if (desync)       { fprintf(stderr, "BAD HEADER -- load.i found no 'DLX3' magic\n"); return 5; }
+
+    long long cyc = (cyc_stop - cyc_start) / (iter ? iter : 1);
+    fprintf(stderr, "[C68K] load mode %u: %lld cyc/pass (%.2f ms at 10MHz, "
+            "%.1f%% of a 12fps frame), dark=%u\n", mode, cyc, cyc / 10000.0,
+            100.0 * cyc / (10000000.0 / 12), rd32(LDARK));
+    /* A 68000 bus cycle is 4 clocks.  Prefetch is not counted (C68K reads
+     * opcodes straight through the fetch pointer), so this is a LOWER bound on
+     * occupancy and the headroom it implies is an UPPER bound -- same caveat as
+     * the decoder's figure above. */
+    {
+        double slots = (double)cyc / 4.0;
+        double used = (double)(bus_r + bus_w) / (iter ? iter : 1);
+        fprintf(stderr, "[C68K]   data bus: %.0f reads + %.0f writes = %.0f of "
+                "%.0f cycles = %.1f%% occupied (prefetch NOT counted)\n",
+                (double)bus_r / iter, (double)bus_w / iter, used, slots,
+                100.0 * used / slots);
+    }
+    if (dump) {
+        FILE *g = fopen(dump, "wb");
+        if (!g) { perror(dump); return 2; }
+        for (unsigned int a = CB1; a < CB1 + cb1_len; a++) { unsigned char b = rd8(a); fwrite(&b,1,1,g); }
+        for (unsigned int a = CB4; a < CB4 + cb4_len; a++) { unsigned char b = rd8(a); fwrite(&b,1,1,g); }
+        for (unsigned int c = 0; c < 256; c++) {
+            unsigned short w = rd16(GPAL + c*2);
+            unsigned char b[2] = { (unsigned char)(w >> 8), (unsigned char)w };
+            fwrite(b, 1, 2, g);
+        }
+        fclose(g);
+        fprintf(stderr, "[C68K] load output dumped to %s (%u B)\n",
+                dump, cb1_len + cb4_len + 512);
+    }
+    return 0;
+}
+
 int main(int argc, char **argv)
 {
     const char *fcode = "tmp/decode.bin", *fdata = "tmp/decode_data.bin", *dump = NULL;
     unsigned int cb1_len=0, cb4_len=0, pal_len=0, stream_len=0, nframes=0, H=192, W=256, fps=12;
     unsigned int dark = 255;
     unsigned int anch[32]; int nanch = 0;
+    const char *fraw = NULL, *loaddump = NULL;
+    unsigned int loadmode = 7, loaditer = 1;
     for (int i = 1; i < argc; i++) {
         if      (!strcmp(argv[i], "--code"))   fcode = argv[++i];
         else if (!strcmp(argv[i], "--data"))   fdata = argv[++i];
@@ -204,10 +309,15 @@ int main(int argc, char **argv)
         else if (!strcmp(argv[i], "--H"))      H = (unsigned)atoi(argv[++i]);
         else if (!strcmp(argv[i], "--fps"))    fps = (unsigned)atoi(argv[++i]);
         else if (!strcmp(argv[i], "--dark"))   dark = (unsigned)atoi(argv[++i]);
+        else if (!strcmp(argv[i], "--loadraw"))  fraw = argv[++i];
+        else if (!strcmp(argv[i], "--loaddump")) loaddump = argv[++i];
+        else if (!strcmp(argv[i], "--loadmode")) loadmode = (unsigned)atoi(argv[++i]);
+        else if (!strcmp(argv[i], "--loaditer")) loaditer = (unsigned)atoi(argv[++i]);
         else if (!strcmp(argv[i], "--anchor")) { if (nanch < 32) anch[nanch++] = (unsigned)strtoul(argv[++i], NULL, 10); }
         else { fprintf(stderr, "unknown arg %s\n", argv[i]); return 2; }
     }
-    if (!nframes || !stream_len) { fprintf(stderr, "need --nframes and --stream (from decode_meta.lua)\n"); return 2; }
+    if (!fraw && (!nframes || !stream_len)) {
+        fprintf(stderr, "need --nframes and --stream (from decode_meta.lua)\n"); return 2; }
 
     /* MAP_32BIT: C68K keeps its fetch base in a UINT32, so the arena must live
      * below 4 GB or every opcode fetch reads a truncated pointer. */
@@ -216,6 +326,17 @@ int main(int argc, char **argv)
     if (buf == MAP_FAILED) { perror("mmap MAP_32BIT"); return 2; }
     fprintf(stderr, "[C68K] arena at %p\n", (void *)buf);
 
+    if (fraw) {
+        C68k_Init(&C68K);
+        C68k_Set_ReadB (&C68K, rd8);
+        C68k_Set_ReadW (&C68K, rd16);
+        C68k_Set_WriteB(&C68K, wr8);
+        C68k_Set_WriteW(&C68K, wr16);
+        C68k_Set_Fetch (&C68K, 0x000000, 0xFFFFFF, (UINT32)(unsigned long)buf);
+        return run_load(fcode, fraw, loaddump, loadmode, loaditer,
+                        cb1_len ? cb1_len : 8192, cb4_len ? cb4_len : 2048);
+    }
+
     size_t nc, nd;
     unsigned char *code = slurp(fcode, &nc), *data = slurp(fdata, &nd);
     size_t need = (size_t)cb1_len + cb4_len + pal_len + stream_len;

tools/bench/check.sh

@@ -270,4 +270,638 @@ grep -q "ceiling 8 frames" tmp/pace_check.log || {
 grep -q "^OK" tmp/pace_check.log || { echo "FAIL: paced pass not pixel-exact";
   tail -4 tmp/pace_check.log; exit 1; }
 
+echo "--- session 21: the 68000 builds its own codebooks and palette (FINDINGS 53) ---"
+# ROADMAP P1+P2. Until now tools/bench/dlxload.py expanded the codebooks and
+# packed the palette HOST-SIDE and the rigs pushed the result into emulated RAM.
+# A player has no host. src/player/load.i does both on the 68000, out of the RAW
+# container header, and this gates it byte-for-byte against dlxload.py -- which
+# stays the reference, because what changed is where the transforms RUN, not
+# what they produce.
+#
+# Byte-for-byte and not "close enough": a wrong codebook byte is a wrong colour
+# in every block that uses that codeword, and a wrong shared LSB is a slightly
+# wrong colour that looks like a codec artefact rather than a loader bug.
+# The palette half is read back out of the PALETTE REGISTERS at $E82000, so
+# "the words reached the hardware" is part of what passes.
+#
+# NOT gated on the cycle counts, and the reason is NOT the one blit.s has. These
+# are emulated time and reproduce exactly run to run; what they are not is
+# sharp, because MAME samples them on a 1/56.69 s clock and the job takes
+# milliseconds. Nothing in the tree's cost models depends on them either. A
+# change in them is a re-derivation in FINDINGS 53, not a red light here.
+bash tools/bench/load_run.sh "$DLX" > tmp/load_gate.log 2>&1 || {
+  echo "FAIL: the load-time transforms did not pass."; tail -12 tmp/load_gate.log
+  exit 1; }
+grep -aE "^ *OK|both CPU cores|SCENE CHANGE" tmp/load_gate.log | sed 's/^ *//;s/^/  /'
+
+echo "--- session 22: the 68000 keeps its own frame clock (FINDINGS 54) ---"
+# ROADMAP P3. Until now the 12 fps tick came from tools/bench/stream.lua -- a
+# host writing a word into emulated RAM. A player has no host. src/player/
+# clock.i derives the tick from the CRTC's own V-DISP output through the MFP,
+# with a remainder-keeping divider whose two constants are READ OUT OF THE CRTC
+# at init, so the clock and the raster it counts cannot disagree.
+#
+# WHAT IS GATED, and it is deliberately structural rather than numeric:
+#   * the interrupt count equals the raster frame count -- the tick IS the
+#     raster, not something that merely resembles it;
+#   * the divider does not accumulate drift, stated in TICKS (a remainder can
+#     hold back at most one) rather than in ppm, which would let a longer
+#     window advertise a tighter clock for free;
+#   * every frame tick waits 4 or 5 refreshes and nothing else, which is what a
+#     remainder-keeping divider can produce and a broken one cannot.
+# The interrupt COST is printed and not gated, for the same reason FINDINGS 53's
+# cycle counts are not: it is a measurement, and a change in it is a
+# re-derivation in FINDINGS 54 rather than a red light here.
+bash tools/bench/clock_run.sh 3000 12 > tmp/clock_gate.log 2>&1 || {
+  echo "FAIL: the frame clock did not pass."; tail -12 tmp/clock_gate.log
+  exit 1; }
+grep -aE "INTERRUPT:|PER FRAME:|DRIFT:|CADENCE:" tmp/clock_gate.log
+grep -q "V-DISP interrupts 3000" tmp/clock_gate.log || {
+  echo "FAIL: the tick is not the raster -- the interrupt count and the frame"
+  echo "      count disagree. Everything else in FINDINGS 54 rests on that."
+  exit 1; }
+
+echo "--- session 22: 120 frames decoded on the machine's own clock (FINDINGS 54) ---"
+# The strongest form of the claim: the same pixel-exact 120-frame decode out of
+# the same 256 KB ring, with NOTHING outside the machine deciding when a frame
+# may start. The pace gate in src/player/stream.s is byte-for-byte the one
+# FINDINGS 51 measured -- it cannot tell a host-written tick from a machine-
+# written one, which is why this is a test of the clock and not of a new rig.
+DLX_PACE=2 bash tools/bench/pace_run.sh 256 0 > tmp/selfpace_check.log 2>&1 || {
+  echo "FAIL: the self-paced pass did not complete."; tail -8 tmp/selfpace_check.log
+  exit 1; }
+grep -aE "decoder SELF-PACED|FRAME CLOCK|UNDERRUNS|NO IDLE" tmp/selfpace_check.log
+grep -q "UNDERRUNS: 0/120" tmp/selfpace_check.log || {
+  echo "FAIL: the self-paced decoder underran."; exit 1; }
+grep -q "^OK" tmp/selfpace_check.log || {
+  echo "FAIL: the self-paced pass was not pixel-exact. The clock changed WHEN"
+  echo "      frames start; if it changed WHAT they draw, the interrupt is"
+  echo "      corrupting decoder state."; tail -4 tmp/selfpace_check.log; exit 1; }
+
+echo "--- session 23: the 68000 fills its own ring (FINDINGS 55) ---"
+# ROADMAP P5. Until now the RING was filled by tools/bench/stream.lua: the host
+# held the record index, chose where each record went, wrote the descriptor and
+# advertised it. A player has no host. src/player/ring.i does all of that on the
+# 68000, out of the DLX4 record index in the scene header, and this script keeps
+# only the part that is not the CPU's -- a transport that answers one request at
+# a time at a modelled rate.
+#
+# WHAT IS GATED:
+#   * pixel-exact, which is the only test that can see a wrong placement: the
+#     block loop reads with a monotonically increasing a0 and no bounds check,
+#     so a record placed over one the decoder has not finished corrupts pixels
+#     rather than faulting (49.2);
+#   * the host AUDITS every placement against its own index and its own list of
+#     live records, and refuses the run on the first disagreement;
+#   * the wrap policy still produces the SAME 18 wraps the
+#     host producer produced in FINDINGS 49.4 -- a third independent
+#     implementation of `aligned` landing on the same tiling;
+#   * zero underruns at a two-deep request queue, which is the finding: a
+#     one-deep queue leaves the channel idle between records and underran 59 of
+#     120 frames on this same container and rate.
+DLX_PACE=2 DLX_RINGOWN=1 DLX_QDEPTH=2 bash tools/bench/pace_run.sh 256 488 \
+  > tmp/ringown_check.log 2>&1 || {
+  echo "FAIL: the machine-owned ring pass did not complete."
+  tail -10 tmp/ringown_check.log; exit 1; }
+grep -aE "MACHINE-OWNED|PREFILL:|CHANNEL IDLE|UNDERRUNS|SEEK SLACK" tmp/ringown_check.log \
+  | sed "s/\[STR\] /  /"
+grep -q "MISPLACED" tmp/ringown_check.log && {
+  echo "FAIL: the 68000 placed a record over one the decoder still owned."
+  exit 1; }
+grep -q "UNDERRUNS: 0/120" tmp/ringown_check.log || {
+  echo "FAIL: the machine-owned ring underran at a two-deep queue. That is the"
+  echo "      configuration FINDINGS 55 says keeps the channel busy; if it no"
+  echo "      longer does, the poll site in src/player/stream.s moved."
+  exit 1; }
+grep -q "ring: 18 wraps" tmp/ringown_check.log || {
+  echo "FAIL: the machine's own \`aligned\` no longer tiles this container the"
+  echo "      way FINDINGS 49.4's host producer did (18 wraps). The policy is"
+  echo "      meant to be the SAME policy in a different place."
+  exit 1; }
+grep -q "^OK" tmp/ringown_check.log || {
+  echo "FAIL: the machine-owned ring pass was not pixel-exact -- a record was"
+  echo "      placed or described wrongly."; tail -4 tmp/ringown_check.log
+  exit 1; }
+
+echo "--- session 23: a seek, and the decode after it (FINDINGS 55) ---"
+# The branch point rehearsed. A second pass over the scene begins with a real
+# seek in src/player/ring.i: the channel is waited quiet, the ring is declared
+# empty, the disc address of record 0 comes out of the index rather than from a
+# walk, and the whole lookahead 51.3 says takes seconds of play to accumulate is
+# thrown away and rebuilt from the prefill. What is gated afterwards is the one
+# thing that can see a wrong seek: the last frame of the SECOND pass has to be
+# pixel-exact, and a SKIP block is a claim about the previous frame, so it is
+# only right if every frame after the seek was.
+DLX_PACE=2 DLX_RINGOWN=1 DLX_QDEPTH=2 DLX_ITER=2 \
+  bash tools/bench/pace_run.sh 256 488 > tmp/ringseek_check.log 2>&1 || {
+  echo "FAIL: the seek pass did not complete."; tail -10 tmp/ringseek_check.log
+  exit 1; }
+grep -aE "SEEK PASS|CHANNEL IDLE|UNDERRUNS" tmp/ringseek_check.log | sed "s/\[STR\] /  /"
+grep -q "SEEK PASS 2" tmp/ringseek_check.log || {
+  echo "FAIL: no second pass -- the seek never happened, so this gated nothing."
+  exit 1; }
+grep -q "UNDERRUNS: 0/120" tmp/ringseek_check.log || {
+  echo "FAIL: the pass after the seek underran."; exit 1; }
+grep -q "^OK" tmp/ringseek_check.log || {
+  echo "FAIL: the decode after the seek was not pixel-exact."
+  tail -4 tmp/ringseek_check.log; exit 1; }
+
+echo "--- session 25: the 68000 reads the disc itself (FINDINGS 57) ---"
+# ROADMAP P4, first half. Until now every byte the player consumed was placed in
+# emulated RAM by a host: decode.lua preloaded a container, stream.lua answered a
+# mailbox at a modelled rate. A player has no host. src/player/scsi.i selects a
+# SCSI target on a real MB89352 and issues READ(10) itself.
+#
+# Session 21 recorded this as blocked -- "MAME's x68000 has no MB89352 path" --
+# and that was wrong: `-exp1 cz6bs1` instantiates one, and FINDINGS 32.4 had
+# already read its DMA glue in session 9. What was actually missing was the
+# card's 8 KB boot ROM, which MAME requires to instantiate the device and which
+# the player never executes. scsi_run.sh supplies a blank one on its own rompath.
+#
+# WHAT IS GATED: the register window (60 of 64 addresses -- the two holes ARE the
+# MB89352's missing TMOD and EXBF, and they are what put DREG at $EA0015), and
+# two READ(10)s verified byte-for-byte against the host's copy of the same image,
+# one of them at a NON-ZERO LBA. Nothing here is gated on rate, and nothing here
+# can be: MAME's device models are functional, not transfer-timing accurate.
+# Skipped rather than failed when chdman is absent -- it ships with mame-tools.
+if command -v chdman > /dev/null; then
+  bash tools/bench/scsi_run.sh "$DLX" > tmp/scsi_gate.log 2>&1 || {
+    echo "FAIL: the 68000 could not read the disc."; tail -14 tmp/scsi_gate.log
+    exit 1; }
+  grep -aE "ANSWERED|READ\(10\) OK" tmp/scsi_gate.log
+else
+  echo "  SKIPPED: no chdman (ships with mame-tools) -- cannot build the volume"
+fi
+
+echo "--- session 26: the ring is filled off a real SCSI volume (FINDINGS 58) ---"
+# ROADMAP P4b. The stage above shows the 68000 can READ the disc. This shows it
+# can RUN off it: src/player/xfer.i sits behind src/player/ring.i's XF_* mailbox
+# in place of tools/bench/stream.lua's modelled transport, and the same 120
+# frames are decoded out of the same 256 KB ring with NOTHING outside the
+# machine in the transfer path -- no host file, no modelled rate, no synthesised
+# ack.
+#
+# WHAT IS GATED, and it is correctness rather than rate on purpose:
+#   * pixel-exact, which is the only test that can see a wrong record: the
+#     window in scsi.i decides which of a sector's bytes reach the ring, and a
+#     window off by one byte desyncs the bitstream rather than faulting (49.2);
+#   * the SAME 18 wraps -- ring.i's placement policy must not be able to tell
+#     which transport answered it, and this is the assertion that says it could
+#     not.  The WRAP COUNT is gated and the mean hole is only reported: DLX5's
+#     records are up to 511 B longer than DLX4's, so the hole moved (14.7 KB ->
+#     13.5 KB) while the tiling did not.  Gating a number that the container's
+#     record lengths move would gate the container, not the policy;
+#   * every record accounted for: 120 READ(10)s, and the bytes into the ring
+#     EQUAL to the bytes off the disc -- both read out of the container rather
+#     than written here. Under DLX4 they differed by 1.34% because a record was
+#     not a sector (58.3); DLX5 aligns records to sectors and the covering-sector
+#     read disappears, so the gate is now their IDENTITY;
+#   * a real mid-stream SEEK with the real transport, in the second pass. This
+#     is the one path that could not exist before: ring_seek waits for the
+#     channel to go quiet, and with the transport INSIDE the machine the only
+#     thing that can retire an outstanding request is that wait loop itself.
+#
+# NOTHING HERE IS GATED ON RATE and nothing here can be. What the run DOES cost
+# is printed by tools/bench/xfer_cost.sh and recorded in FINDINGS 58.2; it is a
+# measurement, and a change in it is a re-derivation there rather than a red
+# light here. Skipped rather than failed when chdman is absent.
+if command -v chdman > /dev/null; then
+  DLX_PACE=0 DLX_RINGOWN=1 DLX_QDEPTH=2 DLX_XFER=scsi \
+    bash tools/bench/pace_run.sh 256 0 > tmp/p4b_check.log 2>&1 || {
+    echo "FAIL: the 68000 could not run the ring off a real SCSI volume."
+    tail -12 tmp/p4b_check.log; exit 1; }
+  grep -aE "REAL TRANSPORT:|SECTOR OVERHEAD|ring: " tmp/p4b_check.log \
+    | sed "s/^ *//;s/^/  /"
+  grep -aq "TRANSPORT FAILED" tmp/p4b_check.log && {
+    echo "FAIL: a record's READ(10) reported an error."; exit 1; }
+  # THE BYTE COUNTS COME OUT OF THE CONTAINER, not out of this file.  They were
+  # two hardcoded constants fitted to the DLX4 gate container, and session 28's
+  # re-encode went red on both of them for the right reason -- the container had
+  # changed and the expectation had not.  A gate whose expected value is a
+  # literal tests the literal.
+  EXPECT_B=$(python3 -c "
+import sys; sys.path.insert(0, 'tools/encoder')
+from dlx import DLX
+print(sum(DLX('$DLX').record_lengths()))")
+  grep -aq "REAL TRANSPORT: 120 READ(10)s by the 68000, $EXPECT_B B into the ring" \
+    tmp/p4b_check.log || {
+    echo "FAIL: the 68000 did not fetch all 120 records, or did not fetch"
+    echo "      $EXPECT_B B of them. A short record is a desync, not a shortfall."
+    exit 1; }
+  # DLX5 MAKES THESE THE SAME NUMBER, and that identity IS the finding (59.4,
+  # and 58.3 option C): a sector-aligned container has no covering-sector read,
+  # so the disc moves exactly the records and nothing else.  Under DLX4 they
+  # differed by 1.34% and both were gated so neither could drift into the other;
+  # under DLX5 the gate is that they are EQUAL.  If a windowed read ever came
+  # back -- a container that was not aligned, or a layout that lost the
+  # alignment -- the disc figure would exceed the ring figure and this goes red.
+  grep -aq "SECTOR OVERHEAD: $EXPECT_B B off the disc for $EXPECT_B B of record = 0.00%" \
+    tmp/p4b_check.log || {
+    echo "FAIL: the disc no longer moves EXACTLY the records. On a sector-"
+    echo "      aligned container (DLX5) there is no covering-sector read at"
+    echo "      all, so these two counts must be the same $EXPECT_B B. If they"
+    echo "      differ, either the container lost its alignment or scsi.i is"
+    echo "      windowing again -- and a DMA channel cannot window (59.4)."
+    exit 1; }
+  grep -aq "ring: 18 wraps" tmp/p4b_check.log || {
+    echo "FAIL: the placement policy tiled this container differently with a"
+    echo "      real transport behind it than with a modelled one. ring.i is"
+    echo "      not supposed to be able to tell them apart."; exit 1; }
+  grep -aq "^OK" tmp/p4b_check.log || {
+    echo "FAIL: the pass off the SCSI volume was not pixel-exact."
+    tail -4 tmp/p4b_check.log; exit 1; }
+  DLX_PACE=2 DLX_RINGOWN=1 DLX_QDEPTH=2 DLX_ITER=2 DLX_XFER=scsi \
+    DLX_SECONDS=240 bash tools/bench/pace_run.sh 256 0 \
+    > tmp/p4b_seek_check.log 2>&1 || {
+    echo "FAIL: the seek pass off the SCSI volume did not complete."
+    tail -12 tmp/p4b_seek_check.log; exit 1; }
+  grep -aE "SEEK PASS|IS VACUOUS" tmp/p4b_seek_check.log | sed "s/^ *//;s/^/  /"
+  grep -aq "SEEK PASS 2" tmp/p4b_seek_check.log || {
+    echo "FAIL: no real seek -- the second pass never threw its ring away, so"
+    echo "      ring_seek's quiet-wait was never asked to retire an outstanding"
+    echo "      transfer and this gated nothing."; exit 1; }
+  grep -aq "^OK" tmp/p4b_seek_check.log || {
+    echo "FAIL: the decode after a seek off the SCSI volume was not pixel-exact."
+    tail -4 tmp/p4b_seek_check.log; exit 1; }
+else
+  echo "  SKIPPED: no chdman (ships with mame-tools) -- cannot build the volume"
+fi
+
+echo "--- session 27: the DMAC drives the data phase, and holds the bus (FINDINGS 59) ---"
+# ROADMAP P4a, the last item before M2. The two stages above have the CPU moving
+# every byte itself, at the 87.28 clocks per delivered byte FINDINGS 58.2
+# measured -- 391.8% of a 12 fps frame. This one hands the DATA IN phase to the
+# HD63450 and gates on the thing 57.3 said would be hard to show: that the DMAC,
+# and not the CPU, is driving it.
+#
+# IT IS GATED WITHOUT LOOKING AT $EA0015, and that is the design. With the
+# DMAC's OWN asserted -- which it is at idle here -- MAME cannot distinguish a
+# CPU-driven byte at that address from a DMAC-driven one, so watching it proves
+# nothing. What is gated instead is THE CPU'S OWN PROGRESS:
+#   * the same 2,048 B off the disc three ways -- PIO, held, stealing -- all
+#     three byte-exact against the host's copy, so the configuration is being
+#     compared against a delivery that works and not against nothing;
+#   * MTC sampled by the INSTRUCTION AFTER the one that starts the channel: zero
+#     in the held configuration (the whole transfer happened between two
+#     instructions, because the 68000 did not execute in between) and the full
+#     count in the stealing one;
+#   * the CPU's own trip count round its wait loop: 1 against hundreds. A
+#     counter that CANNOT come out different is 58.3's vacuous "UNDERRUNS: 0/120"
+#     again, so the run asserts the contrast and not just the held value;
+#   * the channel's own CSR/CER/MTC/MAR, which must say it moved every byte
+#     without error;
+#   * THE PALETTE REGISTERS AT $E82000 (session 30, ROADMAP K1): the same
+#     transfer aimed at the palette, byte-exact into 256 register words read
+#     back by the 68000; the SAME transfer aimed 20 KB away leaving the palette
+#     as the CPU poisoned it, which is what attributes the first run to the
+#     channel's MAR; and ONE array-chained start crossing from the registers
+#     into GVRAM, which is the shape of a whole frame -- a palette entry and
+#     192 row entries, started once. What this does NOT settle is the board:
+#     MAME models the palette as a generic palette_device over memory_array,
+#     whose write16 is a plain COMBINE_DATA, so it cannot tell a register file
+#     that takes byte writes from one that does not (FINDINGS 62.4);
+#   * and a WINDOWED read through the channel REFUSED. 117 of 120 records start
+#     part way into a sector (58.3); a channel writes a contiguous run and cannot
+#     drop the bytes in front of one, so it would write the neighbouring records
+#     into the ring with no bounds check to catch it (49.2). The refusal is what
+#     makes "sector-aligned container" a precondition the transport states.
+#
+# NOT GATED ON RATE, and it cannot be: MAME's DMAC runs on wall-clock attotimes
+# (42.5) and models a held bus by HALTING the CPU rather than by charging it
+# cycles per operand. `W` is untouched. tools/analysis/28_autorequest_cost.py
+# prices what this configuration costs, from the datasheet and an explicit rate.
+# Skipped rather than failed when chdman is absent.
+if command -v chdman > /dev/null; then
+  bash tools/bench/dma_run.sh "$DLX" > tmp/dma_gate.log 2>&1 || {
+    echo "FAIL: the DMAC did not drive the SCSI data phase."
+    tail -16 tmp/dma_gate.log; exit 1; }
+  grep -aE "BYTES OK|MTC one instruction|trips round|REFUSED|PALETTE|ONE START" \
+    tmp/dma_gate.log \
+    | sed 's/^ *//;s/^/  /'
+else
+  echo "  SKIPPED: no chdman (ships with mame-tools) -- cannot build the volume"
+fi
+
+echo "--- session 24: the scene graph, and the gap between branch points (FINDINGS 56) ---"
+# The arcade scene graph is not in this repo and is not redistributable from
+# here. tools/import/scenegraph.py is the ONE file in the tree that knows the
+# outside projects exist; it writes tmp/scenegraph.json in this project's own
+# DLXSCENE1 schema and everything downstream reads only that.
+# What is gated is the IMPORT, not the numbers: 516 sequences and 906 input
+# windows, and the four timing helpers still being the formulas the importer
+# evaluates. Skipped when the checkout is absent.
+DIRKSIMPLE=${DLX_DIRKSIMPLE:-tmp/scenegraph/DirkSimple}
+if [ -f "$DIRKSIMPLE/data/games/lair/game.lua" ]; then
+  DLX_DIRKSIMPLE="$DIRKSIMPLE" python3 tools/import/scenegraph.py \
+    -o tmp/scenegraph.json > tmp/scenegraph_import.log 2>&1 \
+    || { cat tmp/scenegraph_import.log; exit 1; }
+  sed "s/^/  /" tmp/scenegraph_import.log
+  grep -q "516 sequences, 906 input windows" tmp/scenegraph_import.log || {
+    echo "FAIL: the scene graph did not import to 516/906 -- upstream changed,"
+    echo "      or the parser silently dropped branches."; exit 1; }
+  python3 tools/analysis/25_scene_graph.py --kbps 488 --ring 256 \
+    > tmp/scenegraph_check.log 2>&1 \
+    || { tail -20 tmp/scenegraph_check.log; exit 1; }
+  grep -aE "^  WORST |^  ZERO-PLAY|^  BRANCH STRUCTURE" tmp/scenegraph_check.log
+else
+  echo "  SKIPPED: no DirkSimple checkout at $DIRKSIMPLE"
+  echo "           (git clone --depth 1 https://github.com/icculus/DirkSimple)"
+fi
+
+echo "--- session 29: the packed paint, and what it does to the codec (FINDINGS 61) ---"
+# tools/bench/blit.s gained V8/V9/V10 -- the packed full-frame blit, and the two
+# ways a 4x4 BLOCK decoder could survive the packed layout. 47.6.1 had filed the
+# packed paint's `movem` shape as an ASSUMPTION since session 16; this measures
+# it, in the same run as V1/V3/V4 so it is quoted against numbers that have not
+# moved since session 9.
+#
+# WHAT IS GATED IS STRUCTURAL, not numeric, for the reason the load stage gives:
+# MAME samples these on a 1/56.69 s clock and no cost model in the tree depends
+# on their exact value. What DOES depend on them is the ORDER, and the order is
+# the whole of FINDINGS 61:
+#   V8 < V1   packing halves the full-frame literal paint
+#   V9 > V4   packing makes a BLOCK decoder DEARER, not cheaper
+#   V10 < V4  unless the blocks are paired, which costs SKIPs instead
+# A tree where any of those flipped has a different answer to 44.7 and should
+# say so out loud rather than let 29_packed_player.py narrate the old one.
+python3 tools/bench/prep_frame.py tmp/fr_00020 tmp/frame256.bin 0 --reserve-black
+rm -f tmp/blit_v8.log
+( cd tmp && SDL_VIDEODRIVER=dummy stdbuf -oL timeout -k 5 300 mame x68000 \
+    -bios ipl10 -ramsize 2M -video soft -window -sound none -nothrottle -plugins \
+    -autoboot_script ../tools/bench/blit.lua -seconds_to_run 120 \
+    > blit_v8.log 2>&1 )
+grep -aq "summary (instruction cycles only" tmp/blit_v8.log || {
+  echo "FAIL: the blit timing run produced no summary -- it did not finish."
+  tail -8 tmp/blit_v8.log; exit 1; }
+python3 - <<'EOF' || exit 1
+import re, sys
+v = {}
+for line in open("tmp/blit_v8.log", errors="replace"):
+    m = re.search(r"V(\d+)\s+(\d+) cyc", line)
+    if m: v[int(m.group(1))] = int(m.group(2))
+need = (1, 2, 3, 4, 8, 9, 10)
+missing = [n for n in need if n not in v]
+if missing: sys.exit(f"FAIL: blit.lua reported no V{missing} -- run incomplete.")
+for a, op, b, why in ((8, "<", 1, "packing did not halve the literal paint"),
+                      (9, ">", 4, "packed BLOCK order came out CHEAPER than "
+                                  "unpacked -- 61.3's conclusion is inverted"),
+                      (10, "<", 4, "pairing the blocks did not buy back the "
+                                   "movem shape")):
+    ok = v[a] < v[b] if op == "<" else v[a] > v[b]
+    if not ok:
+        sys.exit(f"FAIL: V{a} {v[a]:,} is not {op} V{b} {v[b]:,} -- {why}.")
+print(f"  V1 {v[1]:,} / V8 PACKED {v[8]:,} = {100*v[8]/v[1]:.0f}% -- "
+      f"and V3, the unpacked WRITE-ONLY floor, is {v[3]:,}")
+print(f"  V4 {v[4]:,} / V9 packed-block {v[9]:,} = {100*v[9]/v[4]:.0f}% -- "
+      f"packing costs a BLOCK decoder {100*v[9]/v[4]-100:.0f}%")
+print(f"  V10 paired blocks {v[10]:,} = {100*v[10]/v[4]:.0f}% of V4, and pairing "
+      f"is paid for in SKIPs")
+EOF
+python3 tools/analysis/29_packed_player.py "$DLX" > tmp/packed_player.log 2>&1 \
+  || { tail -20 tmp/packed_player.log; exit 1; }
+grep -aE "SKIP block PAIRS|free / DMAC->GVRAM / PACKED|^  CODEC, gate" \
+  tmp/packed_player.log
+
+echo "--- session 31: the PACKED container, and the picture re-derived (FINDINGS 63) ---"
+# ROADMAP K2. The container is REBUILT every run rather than reused when
+# present, the way the codec's gate container is: a packed encode is 3 seconds
+# because there is no k-means in it, so there is no reason to let a stale file
+# stand between the encoder and the gate.
+# The window's audio has to exist before the container can carry it, so the
+# extraction that used to live in session 33's stage moves up here. Same seconds
+# as the frames, and that is not a convenience: an audio stream that is not the
+# same seconds as the picture is not this project's audio.
+[ -f tmp/au_singe.raw ] || python3 tools/encoder/extract_audio.py 00223 tmp/au_singe.raw 15625 539.4 10.0
+python3 tools/encoder/pack.py tmp/fr_singe tmp/packed_singe.dlxp \
+  --nframes "$NF" --audio tmp/au_singe.raw > tmp/pack_encode.log 2>&1 \
+  || { cat tmp/pack_encode.log; exit 1; }
+grep -aE "^  (record|wire|DLXP2|the four axes|lump payload)" tmp/pack_encode.log
+# ...and the SILENT control beside it, which is what says the interleave moved no
+# picture byte. It is the same encode with one flag off, and a packed encode is
+# four seconds because there is no k-means in it, so the control is cheap enough
+# to build every run rather than reason about.
+python3 tools/encoder/pack.py tmp/fr_singe tmp/packed_singe_silent.dlxp \
+  --nframes "$NF" > tmp/pack_encode_silent.log 2>&1 \
+  || { cat tmp/pack_encode_silent.log; exit 1; }
+# WHAT IS GATED. Four format invariants that a DMA channel cannot check for
+# itself -- it copies bytes and has no opinion about them (FINDINGS 62) -- and
+# the three quality claims FINDINGS 61.9 rests the whole packed branch on:
+#   round-trip, sector geometry, index 0 unused, palette words agree
+#   packed > the shipping codec / > the codec's CEILING / > a SCENE-palette control
+# A tree where any of the last three flipped has a different answer to ROADMAP K
+# and should say so out loud rather than let the branch keep building.
+python3 tools/analysis/30_packed_container.py tmp/packed_singe.dlxp \
+  --codec "$DLX" > tmp/packed_container.log 2>&1 \
+  || { cat tmp/packed_container.log; exit 1; }
+sed -n '/^PSNR/,$p' tmp/packed_container.log | grep -aE "RGB888|CODEC|CEILING|CONTROL|PER-FRAME|packed vs|worth|costs"
+# And the container's OWN BYTES through px68k's real gvram.c, with the harness
+# computing no interleave -- the only test that can catch an encoder whose byte
+# order is wrong, because the container round-trips against its own inverse
+# either way. Same skip-not-fail rule as the C68K stage: px68k is not in here.
+if [ -f "$PX68K/x68k/gvram.c" ]; then
+  make -s -C tools/bench/gvpack PX68K="$PX68K"
+  # Frame 0 carries the two negative controls; the other two are there because
+  # one frame rendering does not say the container's 120th record is placed
+  # right, and record placement is arithmetic this format has no index to check.
+  python3 tools/bench/gvpack/verify_dlxp.py tmp/packed_singe.dlxp 0 --controls || exit 1
+  for f in $((NF / 2)) $((NF - 1)); do
+    # NOT piped into head: `set -e` reads a pipeline's status from its LAST
+    # command, so a piped verifier that failed would be reported by head's zero.
+    python3 tools/bench/gvpack/verify_dlxp.py tmp/packed_singe.dlxp "$f" \
+      > "tmp/dlxp_f$f.log" 2>&1 || { cat "tmp/dlxp_f$f.log"; exit 1; }
+    head -1 "tmp/dlxp_f$f.log"
+  done
+else
+  echo "  SKIPPED: no px68k at $PX68K -- the container's bytes were not rendered"
+fi
+
+echo "--- session 32: the PACKED PLAYER, end to end off the disc (FINDINGS 64) ---"
+# ROADMAP K3. src/player/packed.s brings up its own display, builds its own
+# 193-entry DMA chain, keeps its own frame clock off V-DISP and fetches every
+# record itself with READ(10) off a CZ-6BS1. This script writes no picture byte
+# and no palette entry.
+#
+# WHY EVERY FRAME IS COMPARED AND THE CODEC'S GATE COMPARES ONE. The codec is
+# temporally recursive -- a SKIP block is a claim about the previous frame -- so
+# its last frame audits all 120. A packed frame is a LITERAL and frame 119 says
+# nothing about frame 60. The simplification that deleted the ring also deleted
+# the gate's free lunch.
+#
+# AND WHY IT IS PACED AT HALF RATE. The write window has to be OPEN for the
+# whole transfer and buffer mode blanks the graphics layer, so at the
+# container's own 12 fps there is no instant at which a complete frame is
+# displayable and there is nothing to snapshot (FINDINGS 64.2 -- that is the
+# session's finding, not a rig limitation being worked around). Half rate opens
+# a display interval without changing one byte of the transfer.
+#
+# GATE ONLY: runs 2-4 of packed_run.sh measure the apparatus rather than gate
+# it, and they are three more MAME jobs for numbers that cannot change unless
+# MAME does. tools/bench/packed_run.sh with no DLX_PK_GATE_ONLY runs all four.
+if [ -f "$HOME/mame/roms/x68000.zip" ] || [ -d "$HOME/mame/roms/x68000" ]; then
+  DLX_PK_GATE_ONLY=1 DLX_PK_NFR="$NF" bash tools/bench/packed_run.sh \
+    tmp/packed_singe.dlxp > tmp/packed_gate_stage.log 2>&1 \
+    || { cat tmp/packed_gate_stage.log; exit 1; }
+  # SPLIT AT RUN 5, because the audio run below is in the same log and its
+  # verifier emits OK lines too -- one grep over the whole file would print
+  # session 36's result under session 31's heading.
+  sed -n '1,/--- 5. THE AUDIO/p' tmp/packed_gate_stage.log | \
+    grep -aE "^  (FLAG|array|chain|frame clock|late frames|WRITE WINDOW)|^OK "
+else
+  echo "  SKIPPED: no x68000 romset -- the player was not run"
+fi
+
+echo "--- session 36: THE CONTAINER'S OWN AUDIO, OUT OF THE CHIP (FINDINGS 68) ---"
+# ROADMAP P6c. Every piece of this existed before this stage did and none of it
+# was joined up: the container has carried the lumps since 67, the transport is
+# the IPL ROM's channel-3 configuration from 66, and what was missing was the
+# lump buffer and the remainder accumulator in a player. This is the run where
+# a byte of the container reaches the speaker.
+#
+# WHY THE GATE IS A WAV AND NOT A COUNTER. src/player/packed.s reports lumps
+# armed, lumps fetched, payload bytes and starves, and EVERY ONE OF THOSE CAN BE
+# RIGHT WHILE THE SOUND IS WRONG -- which is not hypothetical, it is the bug
+# this session shipped: the refill ran one lump too far ahead and overwrote the
+# buffer channel 3 was reading out of, and the player's account of it was
+# 11 of 11 armed, 11 fetched, 78,125 B, no starve. Nothing parses a packed
+# container (FINDINGS 67.4), so a wrong byte is not an error, it is a sound.
+# tools/bench/verify_packed_audio.py accounts for all 78,125 bytes against
+# MAME's own capture, one delivered byte at a time.
+#
+# The run is part of packed_run.sh's gate half, so DLX_PK_GATE_ONLY takes it.
+if [ -f tmp/packed_gate_stage.log ] && \
+   grep -aq -- "--- 5. THE AUDIO" tmp/packed_gate_stage.log; then
+  sed -n '/--- 5. THE AUDIO/,/--- 7. THE SEEK/p' tmp/packed_gate_stage.log | \
+    grep -aE "^(OK|FAIL) |^  AUDIO:|^ {5}(nibbles per|worst|payload|-> |the player)"
+else
+  echo "  SKIPPED: no x68000 romset, or the container is silent"
+fi
+
+echo "--- session 33: AUDIO -- the encoder, and what it does to the wire (FINDINGS 65) ---"
+# ROADMAP P6, everything in it except the bus half session 20 closed. The audio
+# is the SAME WINDOW as the frames -- 00223 from 539.4 s for 10 s -- because an
+# audio stream that is not the same seconds as the picture is not this project's
+# audio, and a gate that lets the two drift apart would never say so.
+# tmp/au_singe.raw was extracted by session 31's stage, which needs it to build
+# the container.
+# There is NO ffmpeg encoder for this format -- adpcm_ima_oki is decode-only --
+# so the encoder cannot be checked against a reference. What is checked is that
+# the decoder our encoder runs in its own loop IS ffmpeg's, sample for sample.
+# An encoder that agrees with its own wrong decoder is the failure this catches.
+python3 tools/bench/verify_adpcm.py tmp/au_singe.raw || exit 1
+# And the container arithmetic. The interesting line is the padding: a packed
+# record has no index BY DESIGN, so audio has to ride a fixed cadence, and the
+# obvious cadence throws away a third of every audio sector.
+python3 tools/analysis/32_audio_wire.py tmp/packed_singe.dlxp \
+  > tmp/audio_wire.log 2>&1 || { cat tmp/audio_wire.log; exit 1; }
+grep -aE "^  (   1|  11|  81) |THE FLOOR|F=1 |F=11|is ZERO|SOUND IS WHAT" tmp/audio_wire.log
+
+echo "--- session 34: THE CHIP'S OWN DECODER, off the machine (FINDINGS 66) ---"
+# ROADMAP P6a. 68000 code programs HD63450 channel 3 exactly as the IPL ROM
+# programs it and feeds the MSM6258 a designed nibble stream at the chip's own
+# pace; ONE of sixteen candidate decoder models reproduces MAME's capture
+# sample-exact, and every axis has a negative control. The encoder disagreed
+# with the chip on ALL FOUR axes, and the largest of them is not the delta
+# formula 65 named -- it is the NIBBLE ORDER, at -25.7 dB.
+bash tools/bench/adpcm_run.sh > tmp/adpcm_gate.log 2>&1 || {
+  cat tmp/adpcm_gate.log; exit 1; }
+grep -aE "^(OK|FAIL) |^     (feed|nibbles|delta|clamp|accumulator)" tmp/adpcm_gate.log \
+  | sed 's/^/  /'
+# And the bill, on the same ten seconds every other audio figure is quoted on.
+python3 tools/analysis/33_adpcm_model.py tmp/au_singe.raw > tmp/adpcm_model.log 2>&1 \
+  || { cat tmp/adpcm_model.log; exit 1; }
+grep -aE "wrong only here|played on the chip|decoded on the encoder|headroom left" \
+  tmp/adpcm_model.log
+
+echo "--- session 35: DLXP2 -- a packed container with sound in it (FINDINGS 67) ---"
+# ROADMAP P6b. 65.3 did the arithmetic and wrote no byte; 66 measured which of
+# sixteen decoder models the chip runs. This is the container both produce, and
+# it needs a gate of its own because NOTHING PARSES A PACKED CONTAINER: a lump
+# one sector out does not fail, it paints 512 B of audio and plays 512 B of
+# picture, and a gate that only looked for errors would pass it.
+#
+# The picture side of it is gated twice over: here against a SILENT control
+# built from the same frames, and above by the packed player itself, which now
+# carries the `(i//F)*A` term and still gets all 120 frames pixel-exact off a
+# real volume.
+python3 tools/analysis/34_packed_audio.py tmp/packed_singe.dlxp \
+  > tmp/packed_audio.log 2>&1 || { cat tmp/packed_audio.log; exit 1; }
+grep -aE "^  (OK|FAIL) |^    (order|variant|bits|init) |min of play ->" \
+  tmp/packed_audio.log
+
+echo "--- session 37: THE AUDIO LEVEL, measured off the whole disc (FINDINGS 69) ---"
+# ROADMAP P6, the item 66.3 reopened and two sessions deferred. The chip clamps
+# its accumulator at 10 bits INSIDE the recursion, and the ten seconds every
+# audio figure in this tree is quoted on peak at 435 of 511 -- which fits, and
+# fits BY ACCIDENT, because that window is a -13.4 dBFS passage.
+#
+# So the level is measured against the loudest thing the game can play, which
+# means every stream of the unique scene footage (00000-00201, FINDINGS 32.1)
+# through extract_audio.py's own chain. The gate asserts the disc's peak and
+# the census behind it; a different pressing is a legitimate reason for it to
+# go red, a different ffmpeg downmix is not.
+#
+# ~18 s, and it needs the Blu-ray mounted like every other stage here.
+python3 tools/analysis/35_audio_level.py --gate > tmp/audio_level.log 2>&1 \
+  || { cat tmp/audio_level.log; exit 1; }
+grep -aE "DISC PEAK|LOUDEST PASSAGE|THE CLAMP|LEVEL GATE|NO AUDIO TRACK|^  1\.0000|^  0\.5" \
+  tmp/audio_level.log
+
+echo "--- session 39: THE SEEK, on the machine, with sound across it (FINDINGS 71) ---"
+# Run 7 of packed_run.sh, split out of the same log for the same reason session
+# 36's stage is split out of it: three verifiers in one file all emit OK lines,
+# and one grep over the whole thing prints this session's result under session
+# 31's heading.
+if [ -f tmp/packed_gate_stage.log ] && \
+   grep -aq -- "--- 7. THE SEEK" tmp/packed_gate_stage.log; then
+  sed -n '/--- 7. THE SEEK/,$p' tmp/packed_gate_stage.log | \
+    grep -aE "^(OK|FAIL) |^ {4}-- the chip|^ {5}(the chip was|error against|-> )|DC +-?[0-9.]+ +AC"
+else
+  echo "  SKIPPED: no x68000 romset, or the container is silent"
+fi
+
+echo "--- session 39: the audio seek path, run across a real branch (FINDINGS 71) ---"
+# ROADMAP P6d, and it is the item session 38's handoff put first: 70.3 named
+# what was missing -- "src/player/packed.s starts PG_AK/PG_AKF at lump 0 and has
+# no audio seek path at all" -- and priced its absence at a mean 416.5 ms of
+# silence over the arcade's 409 within-container seek targets.
+#
+# The MACHINE half of this is run 7 of tools/bench/packed_run.sh, above: two
+# passes over the container with the second starting at frame 37, which is four
+# frames into lump 3 and therefore NOT on a group boundary, so the byte offset
+# inside the lump is load-bearing.  It runs in both chip configurations.
+#
+# THIS STAGE IS THE HOST HALF, and what it adds is the census the one branch
+# point on the machine cannot give. The chip's accumulator is an integrator with
+# NO LEAKAGE TERM, so the state error a branch creates is a DC offset that never
+# decays -- and the only fix that makes a branch free is in the ENCODER, not the
+# player. Both are priced here, and section 1 checks the host's arithmetic for
+# the machine's own branch point against what MAME's capture measured.
+python3 tools/analysis/37_audio_seek.py > tmp/audio_seek.log 2>&1 \
+  || { cat tmp/audio_seek.log; exit 1; }
+grep -aE "^(OK|FAIL) |RE-PLAYED BRANCH|\|DC\| against|step index the encoder|frame 37, the branch|reset every|never \(shipped\)|frames \(the cadence\)|1 frame |ONLY FIX" \
+  tmp/audio_seek.log
+
+echo "--- session 38: the refill climb with a second consumer, through a real branch point (FINDINGS 70) ---"
+# ROADMAP P6, the oldest item in it: 65.6 and 67.6 both recorded that the slack
+# table existed and that 51.3's climb had never met a branch point with audio on
+# the wire.  This is that run.  It needs the scene graph, so it skips with the
+# session-24 stage when there is no checkout.
+#
+# WHAT IS GATED IS STRUCTURAL, and deliberately not the milliseconds: the
+# silences move with the scene table and with the one-container-per-scene
+# assumption the tool prints in its own section 5.  What must not move is the
+# ORDER and the SIGNS -- audio never shortens a climb, the shipped cadence
+# strands most within-container branch points off a group boundary, F=1 strands
+# none, and the lump read that removes the silence is an order of magnitude
+# cheaper than the silence.  A tree where any of those flipped has a different
+# answer to the cadence pick.
+if [ -f "$DIRKSIMPLE/data/games/lair/game.lua" ]; then
+  python3 tools/analysis/36_branch_audio.py --gate \
+    > tmp/branch_audio.log 2>&1 || { cat tmp/branch_audio.log; exit 1; }
+  grep -aE "^  (OK|FAIL) |SECOND CONSUMER IS|BRANCH-AUDIO GATE|^    mean |^    free " \
+    tmp/branch_audio.log
+else
+  echo "  SKIPPED: no DirkSimple checkout at $DIRKSIMPLE"
+fi
+
 echo "ALL GREEN"

tools/bench/clock.lua

@@ -0,0 +1,240 @@
+-- Drive src/player/clockgate.s: measure the 68000's own FRAME CLOCK.
+-- ROADMAP P3.
+--
+-- Two things are being measured and they need different instruments.
+--
+-- THE RATE AND THE CADENCE are counted, not timed.  The clock's tick is a
+-- V-DISP interrupt, and MAME's Lua sees the machine once per screen frame --
+-- which is once per V-DISP.  So the host's sampling granularity is exactly the
+-- clock's own granularity, and the cadence comes out as integers: how many
+-- refreshes each frame tick waited.  There is no timing error to argue about
+-- in a count of 4s and 5s.
+--
+-- THE COST IS TIMED BY THE 68000, because the host cannot.  1/55.46 s of host
+-- granularity is 18 ms and the interrupt costs microseconds.  So the 68000 runs
+-- a one-instruction loop for a window of thousands of refreshes and the host
+-- reads the iteration count at both ends; the interrupt cost falls out of the
+-- difference between a run with the clock armed and one without.  See the head
+-- of src/player/clockgate.s for the arithmetic.  This script emits the raw
+-- counts; tools/bench/clock_cost.py does the subtraction, so that the two runs
+-- it needs can be separate MAME invocations.
+--
+-- MEASUREMENT SCOPE.  This is MAME 0.277's emulated X68000, not real hardware.
+-- What is being priced is the interrupt sequence of MAME's cycle-accurate
+-- M68000 core (src/devices/cpu/m68000, the `M68000` device x68k.cpp:1133 asks
+-- for) against zero-wait-state RAM.  Real DRAM adds wait states to the six bus
+-- cycles of the exception and the four of the handler alike, so this is a LOWER
+-- BOUND in the same way every other 68000 figure in this project is.
+--
+-- Env:
+--   DLX_CLK_ON   1 = arm the frame clock, 0 = leave it off (the calibration
+--                run).  REQUIRED -- the two runs are not interchangeable and a
+--                default would let one be reported as the other.
+--   DLX_CLK_FPS  frame rate to ask clk_init for (default 12)
+--   DLX_CLK_WIN  measurement window, in raster frames (default 3000 = 54.1 s)
+--   DLX_CLK_OUT  where to write the raw counts (default tmp/clock_run.txt)
+
+M = manager.machine
+SP = M.devices[":maincpu"].spaces["program"]
+
+local function findfile(n)
+  for _,p in ipairs{"../tools/bench/"..n, "tools/bench/"..n, n} do
+    local f = io.open(p,"rb"); if f then f:close(); return p end
+  end
+  error(n.." not found")
+end
+local MODE = loadfile(findfile("crtc_mode.lua"))()
+
+local CGFLAG, CGON, CGCNT     = 0x18070, 0x18074, 0x18078
+local CLK_PACE                = 0x18034
+local CLK_ACC, CLK_INCR       = 0x18060, 0x18062
+local CLK_VDISP, CLK_FPS      = 0x18064, 0x18068
+local CLK_ERR                 = 0x1806C
+local CPUHZ                   = 10000000
+
+local ONS = os.getenv("DLX_CLK_ON")
+local FPS = tonumber(os.getenv("DLX_CLK_FPS") or "") or 12
+local WIN = tonumber(os.getenv("DLX_CLK_WIN") or "") or 3000
+local OUT = os.getenv("DLX_CLK_OUT") or "clock_run.txt"
+
+local function P(s) print("[CLK] "..s) end
+
+if ONS ~= "0" and ONS ~= "1" then
+  P("DLX_CLK_ON must be 0 (calibration, clock off) or 1 (clock armed). The "
+    .."cost figure is the DIFFERENCE between the two runs, so neither is "
+    .."meaningful alone and neither gets to be the default.")
+  M:exit()
+  return
+end
+local ON = (ONS == "1")
+
+local code do local f=io.open("clockgate.bin","rb"); code=f:read("a"); f:close() end
+
+local function T() local t=M.time; return t.seconds + t.attoseconds/1e18 end
+
+-- Settling frames between the gate reporting `running` and the window opening.
+-- The CPU may still be inside clk_init when the host first sees CGFLAG=1, and
+-- the first V-DISP edge after arming lands wherever the raster happens to be.
+-- Two frames puts the window entirely inside the steady state.
+local SETTLE = 2
+
+local st, n = "boot", 0
+local f_ready, f0, f1 = nil, nil, nil
+local c0, c1, v0, v1, p0, p1, t0, t1
+-- Cadence: refreshes between consecutive frame ticks.  Recorded as a histogram
+-- and as the raw first few, because the interesting claim is not the mean (the
+-- divider makes that exact by construction) but that the SPREAD is only ever
+-- the two values either side of fps*VTOTAL/HFREQ.
+local last_pace, last_pace_f, cad, seen_tick = nil, nil, {}, false
+
+SUB = emu.add_machine_frame_notifier(function()
+  local ok, err = pcall(function()
+    n = n + 1
+    if st == "boot" then
+      if T() < 3.0 then return end
+      MODE.apply(SP)
+      for i = 1, #code do SP:write_u8(0x10000+i-1, string.byte(code,i)) end
+      SP:write_u32(CGFLAG, 0)
+      SP:write_u32(CGON, ON and 1 or 0)
+      SP:write_u32(CLK_FPS, FPS)
+      local cpu = M.devices[":maincpu"]
+      cpu.state["SR"].value = 0x2700    -- clk_init lowers it to $2500 itself
+      cpu.state["SP"].value = 0x8000
+      cpu.state["PC"].value = 0x10000
+      P(string.format("clockgate.bin=%d B, clock %s, asking for %d fps, "
+                      .."window %d raster frames", #code,
+                      ON and "ARMED" or "OFF (calibration run)", FPS, WIN))
+      st = "wait"; return
+    end
+    if st == "wait" then
+      local fl = SP:read_u32(CGFLAG)
+      if fl == 0xEE then
+        local e = SP:read_u32(CLK_ERR)
+        P("clk_init REFUSED: CLK_ERR="..e..(e == 1 and
+          " (CRTC is not in a 31.5 kHz mode, so HFREQ=31500 would be wrong)" or
+          e == 2 and " (fps*VTOTAL does not fit the 16-bit accumulator)" or ""))
+        M:exit(); return
+      end
+      if fl ~= 1 then
+        if T() > 60 then P("TIMEOUT: the gate never started"); M:exit() end
+        return
+      end
+      f_ready = n; st = "settle"; return
+    end
+    if st == "settle" then
+      if n < f_ready + SETTLE then return end
+      f0, t0 = n, T()
+      c0 = SP:read_u32(CGCNT)
+      v0 = SP:read_u32(CLK_VDISP)
+      p0 = SP:read_u32(CLK_PACE)
+      last_pace, last_pace_f = p0, n
+      if ON then
+        P(string.format("armed: incr=%d (fps*VTOTAL), acc=%d, first tick "
+                        .."pending", SP:read_u16(CLK_INCR),
+                        SP:read_u16(CLK_ACC)))
+      end
+      st = "run"; return
+    end
+    if st == "run" then
+      if ON then
+        local pc = SP:read_u32(CLK_PACE)
+        if pc ~= last_pace then
+          -- The FIRST change is dropped.  Its interval runs from the window
+          -- opening rather than from a tick, so it measures where the window
+          -- happened to start and would show up as a spurious short bucket.
+          if seen_tick then
+            -- More than one tick in a single refresh would mean fps above the
+            -- raster rate; give it its own bucket rather than averaging it in.
+            local gap = n - last_pace_f
+            if pc - last_pace > 1 then gap = 0 end
+            cad[gap] = (cad[gap] or 0) + 1
+          end
+          seen_tick = true
+          last_pace, last_pace_f = pc, n
+        end
+      end
+      if n < f0 + WIN then return end
+      f1, t1 = n, T()
+      c1 = SP:read_u32(CGCNT)
+      v1 = SP:read_u32(CLK_VDISP)
+      p1 = SP:read_u32(CLK_PACE)
+      st = "done"
+
+      local frames  = f1 - f0
+      local secs    = t1 - t0
+      local clocks  = secs * CPUHZ
+      local iters   = c1 - c0
+      local ints    = v1 - v0
+      local ticks   = p1 - p0
+      P(string.format("window: %d raster frames, %.6f s emulated -> %.0f "
+                      .."68000 clocks", frames, secs, clocks))
+      -- THE INSTRUMENT IS 2.22% FAST AND IT IS WORTH SAYING SO EVERY RUN.
+      -- The CRTC registers describe a 31,500 lines/s raster of VTOTAL lines.
+      -- MAME does not run it at that rate: x68k_crtc.cpp refresh_mode()
+      -- computes the frame period as (scr.max_x * scr.max_y) dots with
+      -- scr.max_x = m_htotal - 8, one character cell short and an INCLUSIVE
+      -- rectangle bound used as a count.  So the emulated raster is fast by
+      -- htotal/(htotal-8) -- 368/360 in this mode -- and every rate derived
+      -- from it here is fast by the same factor.  The divider under test is
+      -- built on the registers, so its HARDWARE rate is the asked-for one and
+      -- what this rig can check is that it tracks whatever raster it is given.
+      local vtotal = SP:read_u16(0xE80008) + 1
+      local htotal = (SP:read_u16(0xE80000) + 1) * 8
+      local hw_hz  = 31500 / vtotal
+      local skew   = htotal / (htotal - 8)
+      P(string.format("        raster period %.4f ms = %.4f Hz", 1000*secs/frames,
+                      frames/secs))
+      P(string.format("        the CRTC registers describe 31500/%d = %.4f Hz; "
+                      .."MAME is fast by htotal/(htotal-8) = %d/%d = %.4f",
+                      vtotal, hw_hz, htotal, htotal-8, skew))
+      P(string.format("        loop iterations %d", iters))
+      if ON then
+        P(string.format("        V-DISP interrupts %d, frame ticks %d", ints,
+                        ticks))
+        -- The self-check that makes the rest of it worth reading: the interrupt
+        -- count and the host's screen-frame count are supposed to be the SAME
+        -- clock seen from two sides.  If they disagree by more than the one
+        -- edge the window boundaries can straddle, the tick is not the raster.
+        if math.abs(ints - frames) > 1 then
+          P(string.format("FAIL: %d V-DISP interrupts over %d raster frames. "
+                          .."The tick is not coming from the raster.", ints,
+                          frames))
+          M:exit(); return
+        end
+        -- Two numbers, and confusing them is the whole trap.  The measured rate
+        -- is against MAME's fast raster; dividing the skew out gives the rate
+        -- the same code produces on a machine whose raster matches its own
+        -- registers, which is the number the player is judged on.
+        local meas = ticks/secs
+        P(string.format("        measured rate %.6f fps against MAME's raster "
+                        .."(%+.0f ppm vs the asked %d)", meas,
+                        1e6*(meas/FPS - 1), FPS))
+        P(string.format("        de-skewed    %.6f fps  ->  %+.1f ppm from %d, "
+                        .."which is the tick quantisation of %d ticks and not "
+                        .."drift", meas/skew, 1e6*(meas/skew/FPS - 1), FPS,
+                        ticks))
+        local ks = {}
+        for k in pairs(cad) do ks[#ks+1] = k end
+        table.sort(ks)
+        local s = ""
+        for _,k in ipairs(ks) do
+          s = s .. string.format("%d:%d ", k, cad[k])
+        end
+        P("        cadence, refreshes per frame tick: "..s)
+      end
+
+      local fh = io.open(OUT, "w")
+      fh:write(string.format("on %d\nfps %d\nframes %d\nsecs %.15g\n"
+                             .."clocks %.15g\niters %d\nints %d\nticks %d\n"
+                             .."vtotal %d\nhtotal %d\nhw_hz %.15g\nskew %.15g\n",
+                             ON and 1 or 0, FPS, frames, secs, clocks, iters,
+                             ints, ticks, vtotal, htotal, hw_hz, skew))
+      for k, v in pairs(cad) do fh:write(string.format("cad %d %d\n", k, v)) end
+      fh:close()
+      P("counts -> "..OUT)
+      P("done")
+      M:exit(); return
+    end
+  end)
+  if not ok then print("[CLK] LUA ERROR: "..tostring(err)); M:exit() end
+end)

tools/bench/clock_cost.py

@@ -0,0 +1,132 @@
+"""What the 68000's own frame clock costs, out of the two clock.lua runs.
+
+ROADMAP P3.  Usage:  clock_cost.py <off-run.txt> <on-run.txt>
+
+THE SUBTRACTION.  Both runs execute the same one-instruction loop over a window
+of the same number of raster frames, so the window is the same number of 68000
+clocks in both.  With the clock off, every clock in the window went into loop
+iterations:
+
+    L = clocks / iters_off                     clocks per iteration
+
+With it armed, the interrupts took some of them:
+
+    H = (clocks - iters_on * L) / ints         clocks per V-DISP interrupt
+
+L is CALIBRATED rather than looked up.  That is the point: this project's cost
+model (tools/analysis/buscost.py) says a 68000 bus cycle is 4 clocks and an
+instruction costs 4 * (instruction words + data accesses), and the whole reason
+to measure is to avoid scoring the clock against the table the table is meant to
+be checked by.  L falling on a whole number of clocks is therefore a RESULT, not
+an assumption, and it is reported as one.
+
+WHAT THE FIGURE IS PER FRAME.  Not H -- the interrupt fires once per refresh and
+a frame is several refreshes.  On the hardware raster that is 31500/VTOTAL over
+fps interrupts per frame, and the de-skewed rate is the one to use: MAME's
+raster is fast by htotal/(htotal-8) (see tools/bench/clock.lua), and charging
+the player the emulator's extra interrupts would overstate the cost by that
+same 2.2%.
+"""
+import sys
+
+
+def read(path):
+    d, cad = {}, {}
+    for line in open(path):
+        f = line.split()
+        if f[0] == "cad":
+            cad[int(f[1])] = int(f[2])
+        else:
+            d[f[0]] = float(f[1])
+    d["cad"] = cad
+    return d
+
+
+def main(off_path, on_path):
+    off, on = read(off_path), read(on_path)
+    if off["on"] != 0 or on["on"] != 1:
+        sys.exit("FAIL: expected the calibration run first and the armed run "
+                 "second; got on=%d then on=%d" % (off["on"], on["on"]))
+    for k in ("frames", "clocks", "fps", "vtotal"):
+        if off[k] != on[k]:
+            sys.exit("FAIL: the two runs do not share a window: %s is %g in "
+                     "the calibration run and %g in the armed one"
+                     % (k, off[k], on[k]))
+
+    clocks = off["clocks"]
+    L = clocks / off["iters"]
+    ints = on["ints"]
+    H = (clocks - on["iters"] * L) / ints
+
+    # The self-check that licenses the subtraction: the interrupt count must be
+    # the raster frame count.  clock.lua already fails on this, restated here
+    # because this file is also read on its own.
+    if abs(ints - on["frames"]) > 1:
+        sys.exit("FAIL: %d interrupts over %g raster frames -- not the raster"
+                 % (ints, on["frames"]))
+
+    fps, skew = on["fps"], on["skew"]
+    hw_hz = on["hw_hz"]
+    per_frame_ints = hw_hz / fps
+    per_frame = H * per_frame_ints
+    FRAME_CLK = 10e6 / fps
+
+    print("  calibration: %.6f clocks per loop iteration over %d iterations"
+          % (L, off["iters"]))
+    print("               (%s a whole number of clocks -- the loop is one "
+          "`addq.l #1,abs.l` at 7 bus cycles plus a `bra.s`)"
+          % ("lands on" if abs(L - round(L)) < 1e-3 else "does NOT land on"))
+    print("  INTERRUPT:   %.2f clocks per V-DISP, measured over %d of them"
+          % (H, ints))
+    print("  PER FRAME:   %.2f interrupts x %.2f = %.0f clocks = %.4f%% of a "
+          "%g fps frame" % (per_frame_ints, H, per_frame,
+                            100 * per_frame / FRAME_CLK, fps))
+    print("               (%.4f refreshes per frame on the HARDWARE raster of "
+          "31500/%d = %.4f Hz, not on MAME's, which is %.4fx fast)"
+          % (per_frame_ints, on["vtotal"], hw_hz, skew))
+
+    # THE DRIFT GATE, and it is stated in TICKS rather than in ppm on purpose.
+    # A remainder-keeping divider emits floor() or ceil() of the exact tick
+    # count over any window and never accumulates -- so the only honest
+    # tolerance is one tick, and any ppm figure is that one tick divided by
+    # however long the window happened to be.  Quoting ppm would let a longer
+    # window advertise a tighter clock for no reason.
+    want = on["frames"] * fps * on["vtotal"] / 31500.0
+    ticks = on["ticks"]
+    print("  DRIFT:       %d ticks over %d refreshes; exact is %.4f, so the "
+          "error is %+.4f ticks" % (ticks, on["frames"], want, ticks - want))
+    if abs(ticks - want) > 1.0:
+        sys.exit("FAIL: %d ticks where %.4f were due -- off by %.2f, which is "
+                 "more than the one tick a remainder can hold back. The "
+                 "divider is accumulating drift." % (ticks, want, ticks - want))
+
+    cad = on["cad"]
+    tot = sum(cad.values())
+    if tot:
+        # Refreshes per frame is 31500 / (fps * VTOTAL) exactly -- the divider's
+        # own ratio, upside down.  A remainder-keeping divider can only ever
+        # emit the two whole numbers either side of it, so anything else in the
+        # histogram is a bug in the divider and not a rounding taste.
+        rpf = 31500.0 / (fps * on["vtotal"])
+        lo, hi = int(rpf), int(rpf) + 1
+        print("  CADENCE:     %s  (%d intervals; %.4f refreshes per frame, so "
+              "only %d and %d are possible)"
+              % (", ".join("%dx%d (%.1f%%)" % (k, v, 100.0 * v / tot)
+                           for k, v in sorted(cad.items())), tot, rpf, lo, hi))
+        for k in cad:
+            if k not in (lo, hi):
+                sys.exit("FAIL: a frame tick waited %d refreshes, which a "
+                         "remainder-keeping divider cannot produce" % k)
+        # The mix is forced too: lo*a + hi*b = refreshes, a + b = ticks.
+        b = tot * rpf - lo * tot
+        print("               expected %d:%d split %.1f%% / %.1f%%, got "
+              "%.1f%% / %.1f%%"
+              % (lo, hi, 100 * (tot - b) / tot, 100 * b / tot,
+                 100.0 * cad.get(lo, 0) / tot, 100.0 * cad.get(hi, 0) / tot))
+    return 0
+
+
+if __name__ == "__main__":
+    if len(sys.argv) != 3:
+        sys.exit(__doc__)
+    sys.exit(main(sys.argv[1], sys.argv[2]))

tools/bench/clock_run.sh

@@ -0,0 +1,36 @@
+#!/bin/bash
+# One frame-clock run: the 68000 derives its own 12 fps tick from the raster
+# (ROADMAP P3, FINDINGS 54).
+#
+#   tools/bench/clock_run.sh [window-in-raster-frames] [fps]
+#
+# TWO MAME INVOCATIONS, and they are not interchangeable.  The first leaves the
+# clock off and calibrates the cost of the gate's own loop; the second arms it.
+# The interrupt cost is the difference, so a run that reported only the second
+# would be reporting a number it cannot compute.  See src/player/clockgate.s.
+#
+# Only MAME can run this: the frame clock is an MFP interrupt driven by the
+# CRTC's V-DISP output, and tools/bench/c68k has neither device.  That is why
+# this stage has no second-core half, unlike load_run.sh.
+set -e
+cd "$(dirname "$0")/../.."
+WIN=${1:-3000}
+FPS=${2:-12}
+
+tools/vasm/vasmm68k_mot -Fbin -o tmp/clockgate.bin src/player/clockgate.s > /dev/null
+
+for ON in 0 1; do
+  # stdbuf -oL: without it a long MAME run is unobservable until it exits, and
+  # a run that is merely finishing looks exactly like one that is wedged (34.1).
+  ( cd tmp && DLX_CLK_ON=$ON DLX_CLK_FPS=$FPS DLX_CLK_WIN=$WIN \
+      DLX_CLK_OUT=clock_$ON.txt SDL_VIDEODRIVER=dummy stdbuf -oL \
+      timeout -k 5 600 mame x68000 -bios ipl10 -ramsize 2M -video soft -window \
+      -sound none -nothrottle -plugins -autoboot_script ../tools/bench/clock.lua \
+      -seconds_to_run 240 > clock_$ON.log 2>&1 )
+  # A run that never reached the counts must fail as that, not as bad arithmetic.
+  grep -q "^\[CLK\] done" tmp/clock_$ON.log || {
+    echo "FAIL: the clock rig did not finish run ON=$ON -- no completion marker."
+    tail -8 tmp/clock_$ON.log; exit 1; }
+done
+grep -a "^\[CLK\]" tmp/clock_1.log | sed -n '/window:/,/cadence/p' | sed 's/\[CLK\] /  /'
+python3 tools/bench/clock_cost.py tmp/clock_0.txt tmp/clock_1.txt

tools/bench/crtc_mode.lua

@@ -23,6 +23,26 @@
 --   Total blanking time is identical to the 768 mode (112 dots @ 11.592MHz =
 --   336 dots @ 34.776MHz = 9.66us), which is what a real monitor needs.
 --
+-- THE EMULATOR DOES NOT RUN THE RASTER THESE REGISTERS DESCRIBE, and every
+-- rig in this tree samples the machine at ITS rate, not at the hardware's.
+-- x68k_crtc.cpp refresh_mode() builds the frame period as
+--
+--     (scr.max_x * scr.max_y) dots / dotclock,  scr.max_x = m_htotal - 8
+--
+-- which is one character cell short AND uses an inclusive rectangle bound as a
+-- count.  So MAME's refresh is fast by htotal/(htotal-8) = 368/360 = 1.02222:
+-- 56.6901 Hz where the registers say 55.4577.  MEASURED, not read off the
+-- source alone -- tools/bench/clock.lua reports both every run, and they agree
+-- to six digits (FINDINGS 54.5).
+--
+-- It matters in exactly two places and is harmless in the rest.  Anything timed
+-- by counting host frames has 1/56.69 s of granularity, not 1/55.46; and
+-- anything PACED by the raster runs 2.22% fast under MAME.  It does NOT touch
+-- 68000 cycle figures: the CPU clock is 40 MHz/4 and has nothing to do with the
+-- screen.  Do not "correct" the 55.4577 below to match a measurement -- it is
+-- the hardware's, derived from the dot clocks above, and it is what
+-- src/player/clock.i builds its divider on.
+--
 -- VERTICAL registers are NOT halved. The CRTC still generates a 568-line
 -- 31.5kHz raster (31500/568 = 55.46 Hz); "256 lines" is a graphics-layer
 -- double-scan (draw_gfx() halves gfxrect, x68k_v.cpp:401). Halving them would

tools/bench/decode.lua

@@ -93,7 +93,7 @@ end
 
 -- The plan: one sequential correctness pass, then the cost anchors, then a
 -- full pass timed.  Iteration counts target ~4 emulated seconds each so the
--- 1/55.46 s timing granularity costs under 0.5%.
+-- 1/56.69 s timing granularity (crtc_mode.lua) costs under 0.5%.
 -- DLX_VERIFY_ONLY=1 drops the cost anchors and runs only the correctness pass,
 -- so tools/bench/check.sh can gate the decoder without paying for ~2 minutes of
 -- timing runs that would make the green light sensitive to host load anyway.

tools/bench/dlxload.py

@@ -36,11 +36,18 @@ def pack_palette(d):
     which is the point: the verifier and the loader must agree or a colour bug
     reads as a decoder bug.
 
-    Returns (palette bytes 256x2 big-endian, index of the darkest entry).  The
-    encoder does not yet reserve a black entry (docs/STATUS.md, encoder gaps),
-    so the letterbox gets the closest thing to black the palette has.
+    Returns (palette bytes 256x2 big-endian, index of the darkest entry, and the
+    RGB888 the hardware actually RENDERS from those words).  The encoder does not
+    reserve a black entry in the CODEC container (docs/STATUS.md, encoder gaps),
+    so the letterbox gets the closest thing to black the palette has; the PACKED
+    container does reserve one (tools/encoder/dlxp.py, index 255).
+
+    `d` is a DLX container OR a bare (256,3) uint8 palette.  The packed path has
+    no codebooks and so no DLX object to carry a palette on, and this had to stay
+    the ONE copy of the GRB555+I maths -- the verifier, the loader and now the
+    packed encoder all have to agree or a colour bug reads as a decoder bug.
     """
-    pal = d.pal.astype(int)
+    pal = (d if isinstance(d, np.ndarray) else d.pal).astype(int)
     p6 = lambda v: ((v << 2) | (v >> 4)) & 0xFF
     f = pal >> 3
     render = lambda I: p6((f << 1) | I[:, None])

tools/bench/dma.lua

@@ -0,0 +1,312 @@
+-- Drive src/player/dmagate.s: does the HD63450 drive the SCSI data phase, and
+-- does it HOLD THE BUS?  (ROADMAP P4a)
+--
+-- THE APPARATUS is tools/bench/scsi_run.sh's, unchanged and stated again
+-- because it is two substitutions deep: `x68000 -exp1 cz6bs1` (the board 42.5
+-- says to benchmark, never x68ksupr, whose internal SCSI is PIO-only in MAME),
+-- and a ZERO-FILLED scsiexrom.bin on a private rompath, which is honest only
+-- because the player drives the SPC registers directly and never executes a
+-- byte of that ROM.
+--
+-- WHAT THIS RIG DOES NOT DO, and it is the point of the whole design: it never
+-- looks at $EA0015.  57.3 showed that address cannot answer the question --
+-- with the DMAC's OWN asserted MAME cannot tell a CPU-driven byte there from a
+-- DMAC-driven one.  What separates the two configurations below is whether the
+-- 68000 EXECUTED ANYTHING while the bytes were arriving, which is a fact about
+-- the CPU and is read out of the DMAC's own registers plus a counter the
+-- machine incremented itself.
+--
+-- AND IT IS NOT A RATE.  MAME's DMAC is configured in wall-clock attotimes
+-- (42.5); its burst mode halts the CPU outright rather than charging it cycles
+-- per operand.  `W` is untouched here and still wants a board.
+local M  = manager.machine
+local SP = M.devices[":maincpu"].spaces["program"]
+local function P(s) print("[DMA] "..s) end
+local function T() local t=M.time; return t.seconds + t.attoseconds/1e18 end
+
+local DGFLAG, DGREC, DGREC_SZ = 0x18600, 0x18800, 32
+local DGWIN, DGWERR, DGR20, DGR20N, DGR20C = 0x18700, 0x18704, 0x18708, 0x1870C, 0x18710
+local DGR20P = 0x18714
+local CHROW, CHN, CHBASE = 256, 8, 0xC10000
+local R20OF -- filled in after the mailbox addresses are known
+local GV = 0xC00000
+local DGLBA, DGBLK  = 1000, 4
+-- the palette runs (ROADMAP K1).  PS7/PS8/PS9 are the SNAPSHOTS dmagate.s takes
+-- by reading $E82000 back with the 68000 after each run; the registers
+-- themselves hold only the last of the three by the time the host looks.
+local PAL, PALN, PALB, POIS = 0xE82000, 256, 512, 0xA500
+local PS7, PS8, PS9 = 0x1A000, 0x1A200, 0x1A400
+local CHROW2, CH2BASE, CHN2ROWS = 256, 0xC14000, 6
+local DST = {0x20000, 0x24000, 0x28000, 0xC08000, 0xC0C000, 0xC10000,
+             PAL, 0x2C000, CH2BASE}
+local NAME = {"PIO            (the path FINDINGS 58 measured)",
+              "DMA, BUS HELD  (DCR $00 burst, OCR $81 max rate)",
+              "DMA, STEALING  (DCR $80 cycle steal, OCR $80 limited)",
+              "DMA -> GVRAM   (bus held, R20 bit 11 = BUFFER MODE)  [47.6.2]",
+              "DMA -> GVRAM   (the SAME, bit 11 CLEAR -- NEGATIVE CONTROL)",
+              "DMA -> GVRAM   (ARRAY CHAINED, 8 rows at the 1024 B line stride)",
+              "DMA -> PALETTE (bus held, 512 B into $E82000)        [K1, 61.9]",
+              "DMA -> RAM     (the SAME read aimed elsewhere -- NEGATIVE CONTROL:"
+              .." the palette must still read poison)",
+              "DMA -> PALETTE + SIX ROWS (ONE array-chained start across two"
+              .." kinds of destination)"}
+local SHORT = {"pio", "held", "steal", "gvram", "masked", "chain",
+               "pal", "palctl", "palchain"}
+local LENOF = {[6]=PALB, [7]=PALB}      -- everything else is DGBLK*512
+local ERRNAME = {[0]="OK", "SELECTION TIMEOUT -- no target answered",
+                 "UNEXPECTED PHASE", "POLL TIMEOUT -- a phase never arrived",
+                 "NON-ZERO SCSI STATUS",
+                 "WINDOWED READ REFUSED -- a channel cannot drop bytes"}
+R20OF = {[3]=DGR20, [4]=DGR20N, [5]=DGR20C, [8]=DGR20P}
+local DISK = os.getenv("DLX_SCSI_IMG") or "dlxdisk.img"
+
+local code do local f=io.open("dmagate.bin","rb"); code=f:read("a"); f:close() end
+
+-- the disc's own bytes, once, for all three comparisons
+local want do
+  local f = io.open(DISK, "rb")
+  if f then f:seek("set", DGLBA*512); want = f:read(DGBLK*512); f:close() end
+end
+
+local st = "boot"
+SUB = emu.add_machine_frame_notifier(function()
+  local ok, err = pcall(function()
+    if st == "boot" then
+      if T() < 3.0 then return end
+      for i = 1, #code do SP:write_u8(0x10000+i-1, string.byte(code,i)) end
+      SP:write_u32(DGFLAG, 0)
+      local cpu = M.devices[":maincpu"]
+      cpu.state["SR"].value = 0x2700
+      cpu.state["SP"].value = 0x8000
+      cpu.state["PC"].value = 0x10000
+      P(string.format("dmagate.bin=%d B loaded at $10000; reading LBA %d, %d B, "
+                      .."three ways, then once more into GVRAM",
+                      #code, DGLBA, DGBLK*512))
+      st = "wait"; return
+    end
+    if st == "wait" then
+      if SP:read_u32(DGFLAG) ~= 1 then
+        if T() > 60 then P("TIMEOUT: the gate never finished"); P("done"); M:exit() end
+        return
+      end
+      if not want then P("no "..DISK.." to check against"); P("done"); M:exit(); return end
+      for i = 0, 8 do
+        local LEN = LENOF[i] or DGBLK*512
+        local b = DGREC + i*DGREC_SZ
+        local rc   = SP:read_u32(b)
+        local e    = SP:read_u32(b+4)
+        local mtc0 = SP:read_u32(b+8)
+        local spin = SP:read_u32(b+12)
+        local csr  = SP:read_u32(b+16)
+        local cer  = SP:read_u32(b+20)
+        local mtcf = SP:read_u32(b+24)
+        local marf = SP:read_u32(b+28)
+        P(NAME[i+1])
+        if rc ~= 0 then
+          P(string.format("  FAILED: err=%d (%s)", e, ERRNAME[e] or "?"))
+        else
+          local bad, first = 0, nil
+          -- The GVRAM run is read back a WORD at a time and split by hand.
+          -- SP:read_u8 on $C00000 goes through gvram_r, which in buffer mode
+          -- returns the whole word; asking for one byte of it would hand back
+          -- whichever half MAME's address space happens to hand over, and the
+          -- question here is precisely WHICH HALF each disc byte landed in.
+          -- Even disc byte -> high half (page 1), odd -> low half (page 0),
+          -- because the 68000 is big-endian and an even address is the MS byte.
+          local pg1, pg0, bad_hi, bad_lo = 0, 0, 0, 0
+          for k = 1, LEN do
+            local got
+            -- `a` is set for every destination that has to be read a WORD at a
+            -- time and split by hand -- GVRAM in buffer mode, and the palette
+            -- snapshots, whose words are what the 68000 read back out of
+            -- $E82000. Where it stays nil the destination is plain RAM.
+            local off, a
+            if i == 5 then
+              -- The chained run's destination is not linear: byte k of the
+              -- transfer is byte k%256 of row k//256, and the rows are a full
+              -- 1024 B line stride apart. If the channel had ignored the array
+              -- and run contiguously, every byte past the first row would be
+              -- in the wrong place and this comparison would say so.
+              off = (k-1) % CHROW
+              a = CHBASE + ((k-1) // CHROW) * 1024 + (off & ~1)
+            elseif i == 6 then
+              off = (k-1) % 2
+              a = PS7 + ((k-1) & ~1)
+            elseif i == 8 then
+              -- ONE transfer across two kinds of destination: the first sector
+              -- is the palette, the rest is six picture rows at the line
+              -- stride. The split is the array's, and this walks it the same
+              -- way the channel was told to.
+              if k <= PALB then
+                off = (k-1) % 2
+                a = PS9 + ((k-1) & ~1)
+              else
+                local idx = k - PALB - 1
+                off = idx % 2
+                a = CH2BASE + (idx // CHROW2) * 1024 + ((idx % CHROW2) & ~1)
+              end
+            elseif i >= 3 and i ~= 7 then
+              off = (k-1) % 2
+              a = DST[i+1] + ((k-1) & ~1)
+            end
+            if a then
+              local w = SP:read_u16(a)
+              if (off % 2) == 0 then got = (w >> 8) & 0xff; pg1 = pg1 + 1
+              else                   got = w & 0xff;        pg0 = pg0 + 1 end
+            else
+              got = SP:read_u8(DST[i+1]+k-1)
+            end
+            if got ~= string.byte(want, k) then
+              bad = bad + 1; first = first or (k-1)
+              if ((k-1) % 2) == 0 then bad_hi = bad_hi + 1
+              else                     bad_lo = bad_lo + 1 end
+            end
+          end
+          if R20OF[i] then
+            P(string.format("  R20 during the run = $%04X (bit 11 %s); %d bytes "
+                            .."read back out of the HIGH half of a destination "
+                            .."word and %d out of the LOW half",
+                            SP:read_u32(R20OF[i]),
+                            ((SP:read_u32(R20OF[i]) & 0x0800) ~= 0)
+                              and "SET" or "CLEAR",
+                            pg1, pg0))
+          end
+          -- THE PALETTE RUNS' OWN VACUITY CHECK.  Run 7's destination was
+          -- poisoned by the 68000 first, so "it matches the disc" cannot be
+          -- satisfied by a channel that did nothing -- but only if the poison
+          -- and the disc actually differ everywhere they are compared. That is
+          -- a property of THIS record and is counted rather than assumed.
+          if i == 6 then
+            local diff = 0
+            for j = 0, PALN-1 do
+              local w = (POIS | j) & 0xffff
+              if ((w >> 8) & 0xff) ~= string.byte(want, 2*j+1) then diff = diff + 1 end
+              if (w & 0xff) ~= string.byte(want, 2*j+2) then diff = diff + 1 end
+            end
+            P(string.format("  PALETTE POISON IS A DISCRIMINATOR: %d of %d "
+                            .."positions differ from the disc's bytes -- a "
+                            .."channel that wrote nothing could not have passed "
+                            .."in those.", diff, PALB))
+            P(string.format("  %d bytes read back out of the HIGH half of a "
+                            .."palette word (G and the top of R) and %d out of "
+                            .."the LOW half", pg1, pg0))
+          end
+          if bad == 0 then
+            P(string.format("  BYTES OK: %d B from LBA %d match %s byte for byte "
+                            .."[%s]", LEN, DGLBA, DISK, SHORT[i+1]))
+            if i == 3 then
+              P("  A CHANNEL FILLS THE PACKED LAYOUT: every disc byte landed in "
+                .."its own half of a GVRAM word, with the CPU halted -- so a "
+                .."stream interleaved (right<<8)|left goes from disc to screen "
+                .."with no CPU in the loop (47.6.2, first half).")
+            end
+            if i == 5 then
+              P(string.format("  THE CHANNEL WALKED THE ARRAY ITSELF: %d rows of "
+                .."%d B landed at a %d B line stride from ONE start, CPU halted "
+                .."throughout. A frame is %d such entries; the CPU does not "
+                .."restart the channel per row.", CHN, CHROW, 1024, 192))
+            end
+            if i == 4 then
+              P("  CONTROL DID NOT FAIL: the masked write path delivered every "
+                .."byte too, so the run above is not evidence about R20 bit 11.")
+            end
+            if i == 6 then
+              P("  A CHANNEL WRITES THE PALETTE REGISTERS: 512 B off the disc "
+                .."became 256 palette words, read back OUT OF $E82000 by the "
+                .."68000 itself, with the CPU halted for the transfer. Each "
+                .."disc byte landed in its own half of a register word, so a "
+                .."per-frame palette needs no CPU (61.9, ROADMAP K1).")
+            end
+            if i == 8 then
+              P(string.format("  ONE START PAINTED THE PALETTE AND %d ROWS: a "
+                .."single array-chained transfer crossed from device registers "
+                .."at $%06X into GVRAM at $%06X, %d B in %d entries, CPU halted "
+                .."throughout. A frame is that shape with %d row entries "
+                .."instead of %d.", CHN2ROWS, PAL, CH2BASE, LEN, CHN2ROWS+1,
+                192, CHN2ROWS))
+            end
+          elseif i == 6 or i == 8 then
+            P(string.format("  PALETTE WRONG [%s]: %d of %d differ, first at "
+                            .."+%d -- %d at EVEN offsets (the HIGH half of a "
+                            .."word), %d at ODD.",
+                            SHORT[i+1], bad, LEN, first, bad_hi, bad_lo))
+          elseif i == 4 then
+            -- THE CLAIM IS NOT "half the bytes differ".  In masked 256-colour
+            -- mode gvram_w takes `data & 0x00ff` and ignores mem_mask, so a byte
+            -- written to an EVEN address is never stored and the high half keeps
+            -- whatever it held; some of those stale halves match the disc by
+            -- coincidence, and this record is full of pad, so a lot of them do.
+            -- The mechanism's signature is WHERE the damage is, not how much:
+            -- every ODD byte must survive and only EVEN ones may be lost.
+            P(string.format("  BYTES LOST [masked]: %d of %d differ (first at "
+                            .."+%d) -- %d at EVEN offsets, %d at ODD.",
+                            bad, LEN, first, bad_hi, bad_lo))
+            if bad_lo == 0 and bad_hi > 0 then
+              P(string.format("  EXACTLY THE MECHANISM: all %d survivors of the "
+                              .."high half are stale GVRAM that happens to match "
+                              .."(this record is mostly pad); not one of the %d "
+                              .."ODD bytes was harmed. Bit 11 is what carried the "
+                              .."even ones in the run above.", LEN//2 - bad_hi, LEN//2))
+            end
+          else
+            P(string.format("  BYTES WRONG [%s]: %d of %d differ, first at +%d",
+                            SHORT[i+1], bad, LEN, first))
+          end
+        end
+        if i == 7 then
+          -- THE ATTRIBUTION CONTROL'S SECOND CLAIM, and the one that makes run
+          -- 7 mean something: the same transfer aimed 20 KB away leaves the
+          -- palette exactly as the 68000 poisoned it. If this comes back with
+          -- the disc's bytes in it, something other than the channel's MAR
+          -- decides what reaches $E82000 and run 7 measured that instead.
+          local stale, first_s = 0, nil
+          for j = 0, PALN-1 do
+            if SP:read_u16(PS8 + 2*j) ~= ((POIS | j) & 0xffff) then
+              stale = stale + 1; first_s = first_s or j
+            end
+          end
+          if stale == 0 then
+            P(string.format("  PALETTE UNTOUCHED BY THE CONTROL: %d of %d words "
+                            .."still read the poison the 68000 wrote, so the "
+                            .."bytes in run 7 got there because the channel's "
+                            .."MAR pointed at $%06X.", PALN, PALN, PAL))
+          else
+            P(string.format("  CONTROL DID NOT FAIL [palctl]: %d of %d palette "
+                            .."words are no longer poison (first at entry %d) "
+                            .."-- the palette changed without a channel aimed "
+                            .."at it.", stale, PALN, first_s))
+          end
+        end
+        if i > 0 then
+          -- THE DISCRIMINATOR.  MTC as the instruction after START saw it, and
+          -- the number of times the CPU went round its own wait loop.
+          P(string.format("  MTC one instruction after START: %d of %d  -> the "
+                          .."CPU %s while the transfer ran [%s]",
+                          mtc0, LEN,
+                          (mtc0 == 0) and "NEVER EXECUTED" or "kept executing",
+                          SHORT[i+1]))
+          P(string.format("  CPU trips round the wait loop: %d [%s]", spin, SHORT[i+1]))
+          P(string.format("  channel: CSR=$%02X (%s%s%s) CER=$%02X MTC=%d "
+                          .."MAR=$%06X (+%d) [%s]",
+                          csr,
+                          ((csr & 0x80) ~= 0) and "COC " or "",
+                          ((csr & 0x10) ~= 0) and "ERR " or "",
+                          ((csr & 0x08) ~= 0) and "ACT" or "idle",
+                          cer, mtcf, marf, marf - DST[i+1], SHORT[i+1]))
+        end
+      end
+      -- The refusal.  Expected to fail, and the run is only green if it did.
+      local w, we = SP:read_u32(DGWIN), SP:read_u32(DGWERR)
+      if w == 0xFFFFFFFF and we == 5 then
+        P("WINDOWED DMA READ REFUSED, as it must be: a channel writes a "
+          .."contiguous run and cannot drop the 300 B in front of the record "
+          .."(58.3). P4a's precondition is a SECTOR-ALIGNED container.")
+      else
+        P(string.format("WINDOW NOT REFUSED: rc=%d err=%d -- the transport would "
+                        .."have written the neighbours' bytes into the ring.", w, we))
+      end
+      P("done"); M:exit(); return
+    end
+  end)
+  if not ok then P("LUA ERROR: "..tostring(err)); P("done"); M:exit() end
+end)

tools/bench/dma_run.sh

@@ -0,0 +1,154 @@
+#!/bin/bash
+# One HD63450 data-phase run: does the DMAC drive the SCSI data phase, and does
+# it HOLD THE BUS?  (ROADMAP P4a, the last item before M2.)
+#
+#   tools/bench/dma_run.sh [container.dlx]
+#
+# The apparatus is tools/bench/scsi_run.sh's -- `x68000 -exp1 cz6bs1` and a
+# zero-filled scsiexrom.bin on a private rompath -- and the volume is
+# tools/bench/mkvol.sh's, the same bytes the host-file ring rig reads.
+#
+# WHAT A GREEN RUN MEANS: the same 2,048 B came off the disc three ways -- PIO,
+# the channel with the bus held, the channel stealing cycles -- all three
+# byte-exact against the host's copy; and in the held configuration THE WHOLE
+# TRANSFER HAPPENED BETWEEN TWO INSTRUCTIONS, which is what holding the bus
+# means and is not a claim about $EA0015 (57.3).
+#
+# WHAT IT DOES NOT MEAN: anything about `W`.  MAME's DMAC runs on wall-clock
+# attotimes (42.5) and models a held bus by HALTING the CPU rather than by
+# charging it cycles per operand.  This settles which configuration works.
+set -e
+cd "$(dirname "$0")/../.."
+DLX=${1:-tmp/rc_fr_singe_scsi_span.dlx}
+
+bash tools/bench/mkvol.sh "$DLX"
+
+tools/vasm/vasmm68k_mot -Fbin -o tmp/dmagate.bin src/player/dmagate.s > /dev/null
+
+# What the player will program, decoded out of the same constants it programs.
+python3 tools/analysis/27_dmac_config.py
+
+# stdbuf -oL: without it a long MAME run is unobservable until it exits, and a
+# run that is merely finishing looks exactly like one that is wedged (34.1).
+( cd tmp && SDL_VIDEODRIVER=dummy stdbuf -oL timeout -k 5 300 \
+    mame x68000 -bios ipl10 -exp1 cz6bs1 \
+    -rompath "$HOME/mame/roms;./p4roms" -hard dlxdisk.chd \
+    -ramsize 2M -video soft -window -sound none -nothrottle -plugins \
+    -autoboot_script ../tools/bench/dma.lua \
+    -seconds_to_run 90 > dma_run.log 2>&1 )
+grep -aq "^\[DMA\] done" tmp/dma_run.log || {
+  echo "FAIL: the DMA gate did not finish -- no completion marker."
+  tail -8 tmp/dma_run.log; exit 1; }
+grep -a "^\[DMA\]" tmp/dma_run.log | sed 's/^\[DMA\] /  /'
+
+# THE ASSERTIONS.  Printing a result and gating on it are different things.
+fail() { echo "FAIL: $1"; exit 1; }
+grep -aq "BYTES OK: 2048 B from LBA 1000 .*\[pio\]"   tmp/dma_run.log || \
+  fail "the PIO reference read did not match -- nothing below is about the DMAC."
+grep -aq "BYTES OK: 2048 B from LBA 1000 .*\[held\]"  tmp/dma_run.log || \
+  fail "the bus-held DMA read did not deliver the disc's bytes."
+grep -aq "BYTES OK: 2048 B from LBA 1000 .*\[steal\]" tmp/dma_run.log || \
+  fail "the cycle-stealing DMA read did not deliver the disc's bytes."
+grep -aq "MTC one instruction after START: 0 of 2048 .*NEVER EXECUTED .*\[held\]" \
+  tmp/dma_run.log || \
+  fail "the bus was NOT held: the CPU executed while the channel ran, so this is
+      not the configuration ROADMAP P4a asks for. That MTC is the whole of the
+      evidence that does not come from watching \$EA0015 (57.3)."
+grep -aq "CPU trips round the wait loop: 1 \[held\]" tmp/dma_run.log || \
+  fail "the held configuration's CPU went round its wait loop more than once --
+      it was running, so the bus was not held for the whole transfer."
+# A NEGATIVE ASSERTION IS WRITTEN AS AN `if`, not as `grep ... && fail`: under
+# `set -e` a failing grep in an AND-list takes the whole script's exit status
+# with it, so the run would report the failure it was looking for as a pass.
+SPIN=$(sed -n 's/.*CPU trips round the wait loop: \([0-9]*\) \[steal\].*/\1/p' \
+       tmp/dma_run.log)
+[ -n "$SPIN" ] && [ "$SPIN" -ge 100 ] || \
+  fail "the cycle-stealing configuration did not leave the CPU running (spin
+      = ${SPIN:-none}) -- the two configurations are meant to DIFFER in exactly
+      that, and a contrast of one against one is not a contrast."
+if grep -aq "MTC one instruction after START: 0 of 2048 .*\[steal\]" tmp/dma_run.log
+then
+  fail "the cycle-stealing configuration also finished between two instructions,
+      so the comparison has no contrast in it and the discriminator is measuring
+      something other than bus ownership."
+fi
+grep -aq "COC .*CER=\$00 MTC=0 .*(+2048) \[held\]" tmp/dma_run.log || \
+  fail "the held channel did not report a clean completion of every byte."
+grep -aq "COC .*CER=\$00 MTC=0 .*(+2048) \[steal\]" tmp/dma_run.log || \
+  fail "the stealing channel did not report a clean completion of every byte."
+# ---- the GVRAM run and its control (47.6.2). A channel that writes GVRAM in
+# buffer mode is the decoder-free packed player's entire per-frame path, and a
+# run with no control is 58.3's vacuous "UNDERRUNS: 0/120" again -- the IPL
+# leaves R20 = $0B16, bit 11 ALREADY SET, so the first cut of this test could
+# not have failed.
+grep -aq "BYTES OK: 2048 B from LBA 1000 .*\[gvram\]" tmp/dma_run.log || \
+  fail "the channel did not fill GVRAM in buffer mode -- a device->GVRAM
+      transfer is the whole of the decoder-free packed player's frame."
+grep -aq "R20 during the run = \$0916 (bit 11 SET)" tmp/dma_run.log || \
+  fail "the GVRAM run did not run in buffer mode with a KNOWN R20."
+grep -aq "R20 during the run = \$0116 (bit 11 CLEAR)" tmp/dma_run.log || \
+  fail "the negative control did not run with bit 11 clear."
+if grep -aq "BYTES OK: 2048 B from LBA 1000 .*\[masked\]" tmp/dma_run.log
+then
+  fail "the MASKED control delivered every byte, so the run above is not a
+      measurement of R20 bit 11 -- it is a measurement of nothing."
+fi
+grep -aq "EXACTLY THE MECHANISM" tmp/dma_run.log || \
+  fail "the masked control lost bytes at ODD offsets too, or lost none at all.
+      The claim is not a COUNT -- stale GVRAM matches the disc by coincidence
+      wherever the record is pad -- it is a PLACE: gvram_w's 256-colour arm
+      drops what the channel wrote to EVEN addresses and stores what it wrote
+      to odd ones. Damage anywhere else is a different mechanism."
+
+grep -aq "BYTES OK: 2048 B from LBA 1000 .*\[chain\]" tmp/dma_run.log || \
+  fail "the array-chained run did not put the bytes at the row bases its array
+      named. A picture row is 256 B of a 1024 B line stride, so a frame is 192
+      destinations; if the channel cannot walk them the CPU has to restart it
+      per row and the decoder-free path costs a per-row front end."
+grep -aq "THE CHANNEL WALKED THE ARRAY ITSELF" tmp/dma_run.log || \
+  fail "the chained run did not report walking its own array."
+
+# ---- THE PALETTE (ROADMAP K1, FINDINGS 61.9). If the registers at $E82000 take
+# a byte-wide DMA the way GVRAM does in buffer mode, a per-frame palette is a
+# 193rd array-chain entry and ONE channel start paints a whole frame; if they do
+# not, the CPU writes 256 words a frame and the architecture still stands. The
+# run is poisoned first and controlled twice -- once by aiming the same transfer
+# elsewhere, once by counting how many of the 512 positions the poison and the
+# disc actually differ in.
+grep -aq "BYTES OK: 512 B from LBA 1000 .*\[pal\]" tmp/dma_run.log || \
+  fail "the channel did not write the palette registers at \$E82000 -- so a
+      per-frame palette costs the CPU 256 word writes and cannot ride the
+      frame's array chain (61.9). That is a RESULT, not a broken run: check the
+      PALETTE WRONG line above for whether the bytes were dropped or misplaced."
+DIFF=$(sed -n 's/.*PALETTE POISON IS A DISCRIMINATOR: \([0-9]*\) of 512.*/\1/p' \
+       tmp/dma_run.log)
+[ -n "$DIFF" ] && [ "$DIFF" -ge 500 ] || \
+  fail "the poison and the disc's bytes agree in ${DIFF:-?} of 512 positions, so
+      the palette run could have passed without a channel writing anything --
+      this is run 4's could-not-fail trap in a new place. Change DGPOIS."
+grep -aq "BYTES OK: 512 B from LBA 1000 .*\[palctl\]" tmp/dma_run.log || \
+  fail "the ATTRIBUTION control's read did not land in RAM, so its palette claim
+      is about a transfer that did not happen."
+grep -aq "PALETTE UNTOUCHED BY THE CONTROL: 256 of 256 words" tmp/dma_run.log || \
+  fail "the palette changed during a transfer aimed 20 KB away from it. Then
+      what reached \$E82000 in the run above was not decided by the channel's
+      MAR, and that run measured something else."
+if grep -aq "CONTROL DID NOT FAIL \[palctl\]" tmp/dma_run.log
+then
+  fail "the control reported its own failure -- see the line above it."
+fi
+grep -aq "BYTES OK: 2048 B from LBA 1000 .*\[palchain\]" tmp/dma_run.log || \
+  fail "ONE array-chained start could not cross from the palette registers into
+      GVRAM. A frame is one palette entry and 192 row entries; if the two kinds
+      of destination cannot share a chain, the CPU is back in the video path
+      once a frame to start the second half of it."
+grep -aq "ONE START PAINTED THE PALETTE AND 6 ROWS" tmp/dma_run.log || \
+  fail "the palette+rows run did not report the crossing it exists to show."
+
+grep -aq "WINDOWED DMA READ REFUSED" tmp/dma_run.log || \
+  fail "a WINDOWED read through the channel was not refused. 117 of 120 records
+      start part way into a sector (58.3), and a channel cannot drop the bytes
+      in front of one -- so it would write the neighbouring records into the
+      ring, over data the decoder has not finished with, with no bounds check
+      to catch it (49.2)."
+exit 0

tools/bench/gvpack/harness.c

@@ -97,6 +97,17 @@ int main(int argc, char **argv)
     int iw = (d[4] << 8) | d[5], ih = (d[6] << 8) | d[7];
     if (n < (size_t)(8 + 768 + iw * ih)) { fprintf(stderr, "short blob\n"); return 2; }
     const BYTE *pix = d + 8 + 768;
+    /* 'DLXQ' -- the blob is PRE-INTERLEAVED: `pix` is already the bytes a DLXP1
+     * record carries, in GVRAM order.  The ordinary 'DLXR' path computes the
+     * interleave here, which tests the LAYOUT; this path tests the CONTAINER,
+     * by writing its bytes verbatim and asking px68k's own gvram.c what they
+     * display as.  The two agreeing is the claim ROADMAP K2 has to make: the
+     * encoder's byte order is the one 47.2 verified as a picture. */
+    int prepacked = (d[3] == 'Q');
+    if (prepacked && !packed) {
+        fprintf(stderr, "a pre-interleaved blob has no unpacked form\n");
+        return 2;
+    }
     int yoff = (H - ih) / 2;
     const BYTE BLACK = 255;
 
@@ -121,8 +132,23 @@ int main(int argc, char **argv)
         for (int y = 0; y < H; y++) {
             DWORD base = 0xC00000 + y * 1024;
             for (int i = 128; i < 512; i++) wr16(base + i * 2, 0);
-            for (int i = 0; i < 128; i++)
-                wr16(base + i * 2, (WORD)((PIX(y, i + 128) << 8) | PIX(y, i)));
+            for (int i = 0; i < 128; i++) {
+                WORD w;
+                if (prepacked) {
+                    /* The letterbox rows are STATIC SETUP and are not in a
+                     * record (dlxp.py), so they are supplied here, the way a
+                     * player's scene setup supplies them: both halves BLACK. */
+                    if (y < yoff || y >= yoff + ih)
+                        w = (WORD)((BLACK << 8) | BLACK);
+                    else {
+                        const BYTE *row = pix + (y - yoff) * iw;
+                        w = (WORD)((row[i * 2] << 8) | row[i * 2 + 1]);
+                    }
+                } else {
+                    w = (WORD)((PIX(y, i + 128) << 8) | PIX(y, i));
+                }
+                wr16(base + i * 2, w);
+            }
         }
         if (!keepbuf) set_r20(R20_DISPLAY);       /* back to display */
     } else {

tools/bench/gvpack/verify_dlxp.py

@@ -0,0 +1,87 @@
+#!/usr/bin/env python3
+"""The PACKED CONTAINER's own bytes, through px68k's real GVRAM code.
+
+    python3 tools/bench/gvpack/verify_dlxp.py [packed.dlxp] [frame] [--controls]
+
+`verify_gvpack.py` checks the LAYOUT: it hands the harness a picture and lets
+the harness compute the interleave, so what it proves is that FINDINGS 47.2's
+scheme renders.  This checks the CONTAINER: it writes a DLXP1 record's bytes
+into GVRAM VERBATIM -- no interleave computed anywhere in the harness -- and
+asks px68k what they display as.  That is the only way to test a format whose
+whole design is that nothing parses it (dlxp.py): if the encoder's byte order
+were wrong, every check upstream of the display would still pass, because the
+container round-trips against its own inverse.
+
+It is the same second-emulator argument tools/bench/c68k makes for cycles: the
+address decode, the R20 bit-11 write path, the page-byte selection, the scroll
+wrap and the index-0 transparency test are px68k's own `x68k/gvram.c`.
+
+Two negative controls, because a test that cannot fail proves nothing, and both
+are mechanisms this container depends on rather than decoration:
+  --nobuffer   R20 bit 11 CLEAR -- the high byte of every word is masked away,
+               so page 1 (columns 128..255) never gets written
+  --noscroll   page 1 unscrolled -- its storage sits under the wrong columns
+"""
+import os, struct, subprocess, sys
+sys.path.insert(0, "tools/encoder")
+import numpy as np
+from dlxp import DLXP
+
+args = [x for x in sys.argv[1:] if not x.startswith("--")]
+path = args[0] if args else "tmp/packed_singe.dlxp"
+frame = int(args[1]) if len(args) > 1 else 0
+controls = "--controls" in sys.argv
+
+d = DLXP(path)
+_, pic = d._split(frame)
+blob = b"DLXQ" + struct.pack(">HH", d.W, d.H) + b"\0" * 768 + pic
+open("tmp/dlxp_gvpack.bin", "wb").write(blob)
+
+BIN = "tools/bench/gvpack/gvpack"
+if not os.path.exists(BIN):
+    sys.exit(f"{BIN} not built -- make -C tools/bench/gvpack PX68K=...")
+
+
+def run(extra=None):
+    cmd = [BIN, "tmp/dlxp_gvpack.bin", "tmp/dlxp_gvpack.raw", "--packed", "0x02"]
+    if extra:
+        cmd.append(extra)
+    subprocess.run(cmd, check=True, stderr=subprocess.DEVNULL)
+    g = np.frombuffer(open("tmp/dlxp_gvpack.raw", "rb").read(), np.uint8)
+    return g.reshape(256, 256)
+
+
+want = d.indices(frame)
+yoff = (256 - d.H) // 2
+g = run()
+act = g[yoff:yoff + d.H]
+
+fail = []
+if not np.array_equal(act, want):
+    bad = act != want
+    fail.append(f"{bad.sum()} px differ (left half {bad[:, :128].sum()}, "
+                f"right half {bad[:, 128:].sum()})")
+bars = np.concatenate([g[:yoff], g[yoff + d.H:]])
+if bars.size and (bars != 255).any():
+    fail.append(f"letterbox is not index 255: {(bars != 255).sum()} px")
+if (act == 0).any():
+    fail.append(f"index 0 reached the screen: {(act == 0).sum()} px")
+
+if controls and not fail:
+    for flag, why in (("--nobuffer", "R20 bit 11 clear"),
+                      ("--noscroll", "page 1 unscrolled")):
+        c = run(flag)[yoff:yoff + d.H]
+        n = int((c != want).sum())
+        print(f"    control {flag:<11s} ({why}): {n:,} px differ"
+              + ("" if n else "   <-- IT DID NOT FAIL"))
+        if not n:
+            fail.append(f"control {flag} passed -- the test cannot fail on it")
+
+for x in fail:
+    print("FAIL " + x)
+if fail:
+    sys.exit(1)
+print(f"OK  {os.path.basename(path)} frame {frame}: px68k's own gvram.c renders "
+      f"the container's {d.pic_bytes:,} bytes index-exact over {d.W}x{d.H},")
+print(f"    letterbox on the reserved black, and the transparency key never "
+      f"reaches the screen. The harness computed no interleave.")

tools/bench/load.lua

@@ -0,0 +1,178 @@
+-- Time and verify src/player/load.i on the emulated 68000 (ROADMAP P1+P2).
+--
+-- Two questions, one run, exactly as decode.lua asks them of the decoder:
+--   1. CORRECTNESS.  Does the 68000 produce, out of the RAW container header,
+--      byte for byte what tools/bench/dlxload.py produces host-side?  The
+--      expanded codebooks are read back out of RAM and the palette out of the
+--      PALETTE REGISTERS -- not out of a RAM shadow, because "the words reached
+--      $E82000" is the claim being tested.  tools/bench/verify_load.py does the
+--      comparison against dlxload.py, so the ground truth stays in one place.
+--   2. COST.  How long does it take, split into the codebook expansion and the
+--      palette pack, and what is that as a fraction of a 12 fps frame -- the
+--      only unit this project prices anything in.
+--
+-- Nothing here is pre-chewed: the blob pushed into RAM is the first 5,920 bytes
+-- of the container as they come off the disc.  That is the whole point of the
+-- exercise, and it is also, not incidentally, exactly the read a player has to
+-- complete at a scene change before it can draw a single frame.
+--
+-- MEASUREMENT SCOPE, unchanged from decode.lua: MAME's memory carries no wait
+-- states, so these are pure 68000 instruction cycles -- a LOWER BOUND on real
+-- hardware.  Interrupts are masked (SR=$2700).  The host clock has 1/56.69 s
+-- granularity and the job takes milliseconds, so each configuration is repeated
+-- LITER times and divided; repeating is honest because do_load is not
+-- temporally recursive -- every pass rewrites what the last one wrote, from the
+-- same source bytes.
+
+M = manager.machine
+SP = M.devices[":maincpu"].spaces["program"]
+
+local function findfile(n)
+  for _,p in ipairs{"../tools/bench/"..n, "tools/bench/"..n, n} do
+    local f = io.open(p,"rb"); if f then f:close(); return p end
+  end
+  error(n.." not found")
+end
+local MODE = loadfile(findfile("crtc_mode.lua"))()
+local META = loadfile("load_meta.lua")()
+
+local LFLAG, LHDR, LDARK = 0x18040, 0x18044, 0x18048
+local LK1, LK4, LMODE, LITER = 0x1804C, 0x18050, 0x18054, 0x18058
+local CB1, CB4, RAW = 0x20000, 0x22000, 0x30000
+local GPAL           = 0xE82000
+local CPUHZ          = 10000000        -- x68k.cpp:1133, 40_MHz_XTAL/4
+local FPS            = 12
+local FRAME12        = CPUHZ / FPS
+local ITER           = tonumber(os.getenv("DLX_LOAD_ITER") or "40")
+
+local code do local f=io.open("loadgate.bin","rb"); code=f:read("a"); f:close() end
+local data do local f=io.open("load_data.bin","rb"); data=f:read("a"); f:close() end
+
+local function T() local t=M.time; return t.seconds + t.attoseconds/1e18 end
+local function P(s) print("[LOD] "..s) end
+
+local function push(addr, s, from, len)
+  local i, n = from, len
+  while n >= 4 do
+    SP:write_u32(addr, (string.unpack(">I4", s, i)))
+    addr, i, n = addr+4, i+4, n-4
+  end
+  while n > 0 do
+    SP:write_u8(addr, string.byte(s,i)); addr, i, n = addr+1, i+1, n-1
+  end
+end
+
+-- Poison every destination before each run.  Without this a stage that wrote
+-- NOTHING would still compare equal to the previous stage's output, and the
+-- palette-only run would "pass" the codebook check for free.
+--
+-- The three scratch tables are poisoned only before a run that CLAIMS to build
+-- them (mode bit 2).  They are scene-independent, so the palette-entry stage is
+-- entitled to find them already there -- that is the whole point of measuring
+-- it separately -- but a stage that says it builds them must be shown to.
+local P6TAB, TABEND = 0x19000, 0x19340
+local function poison(mode)
+  for a = CB1, CB1 + META.cb1_len - 2, 2 do SP:write_u16(a, 0xDEAD) end
+  for a = CB4, CB4 + META.cb4_len - 2, 2 do SP:write_u16(a, 0xDEAD) end
+  for c = 0, 255 do SP:write_u16(GPAL + c*2, 0xDEAD) end
+  SP:write_u32(LDARK, 0xFFFFFFFF)
+  if mode & 4 ~= 0 then
+    for a = P6TAB, TABEND - 2, 2 do SP:write_u16(a, 0xDEAD) end
+  end
+end
+
+local function setup()
+  MODE.apply(SP)
+  push(RAW, data, 1, META.raw_len)
+  for i = 1, #code do SP:write_u8(0x10000+i-1, string.byte(code,i)) end
+  P(string.format("loaded loadgate.bin=%d B, raw container header %d B at 0x%X",
+                  #code, META.raw_len, RAW))
+end
+
+local function launch(mode, iter)
+  poison(mode)
+  SP:write_u32(LFLAG, 0)
+  SP:write_u32(LHDR,  RAW)
+  SP:write_u32(LMODE, mode)
+  SP:write_u32(LITER, iter)
+  local cpu = M.devices[":maincpu"]
+  cpu.state["SR"].value = 0x2700          -- supervisor, ALL interrupts masked
+  cpu.state["SP"].value = 0x8000
+  cpu.state["PC"].value = 0x10000
+end
+
+-- Written after the mode-3 run, and only after it: it is the output of ONE
+-- do_load call over the whole header, which is what the player does.
+local function dump()
+  local out = io.open("load_out.bin", "wb")
+  for a = CB1, CB1 + META.cb1_len - 1 do out:write(string.char(SP:read_u8(a))) end
+  for a = CB4, CB4 + META.cb4_len - 1 do out:write(string.char(SP:read_u8(a))) end
+  for c = 0, 255 do out:write(string.pack(">I2", SP:read_u16(GPAL + c*2) & 0xFFFF)) end
+  out:close()
+  P(string.format("dumped %d B of 68000 output to tmp/load_out.bin", 
+                  META.cb1_len + META.cb4_len + 512))
+  P(string.format("DARK=%d (host-side dlxload.py says %d), K1=%d K4=%d",
+                  SP:read_u32(LDARK), META.dark, SP:read_u32(LK1), SP:read_u32(LK4)))
+end
+
+-- Order matters: the scratch tables are built by the first stage and the
+-- palette-entry stage runs on them, which is exactly how a player would be
+-- arranged.  The two stages that stand for real player events -- boot, and a
+-- scene change -- come last, and the dump the verifier checks is taken from the
+-- BOOT one, so the path that is proved correct is the one that builds
+-- everything from nothing.
+local PLAN = {
+  {name="scratch tables only (boot, once)",           mode=4, iter=ITER},
+  {name="codebook expansion only (P1)",               mode=1, iter=ITER},
+  {name="palette entries only (P2)",                  mode=2, iter=ITER},
+  {name="BOOT: tables + codebooks + palette",         mode=7, iter=ITER, dump=true},
+  {name="SCENE CHANGE: codebooks + palette",          mode=3, iter=ITER},
+}
+
+local step, st, t0 = 0, "boot", nil
+local results = {}
+
+SUB = emu.add_machine_frame_notifier(function()
+  local ok, err = pcall(function()
+    local t = T()
+    if st == "boot" then
+      if t < 3.0 then return end
+      setup(); step = 1; launch(PLAN[1].mode, PLAN[1].iter)
+      st, t0 = "running", nil; return
+    end
+    if st == "running" then
+      local fl = SP:read_u32(LFLAG)
+      if fl == 1 and not t0 then t0 = t; return end
+      if fl == 0xEE then
+        P("BAD HEADER -- load.i did not find the 'DLX3' magic at LHDR")
+        M:exit(); return
+      end
+      if fl == 0xFF then
+        local p = PLAN[step]
+        local dt = t - (t0 or t)
+        local cyc = dt * CPUHZ / p.iter
+        results[#results+1] = {name=p.name, cyc=cyc}
+        P(string.format("%s: %d passes in %.4f s -> %.0f cycles = %.1f%% of a "
+                        .."%dfps frame (%.2f ms)", p.name, p.iter, dt, cyc,
+                        100*cyc/FRAME12, FPS, 1000*cyc/CPUHZ))
+        if p.dump then dump() end
+        step = step + 1
+        if PLAN[step] then launch(PLAN[step].mode, PLAN[step].iter); st, t0 = "running", nil
+        else st = "finish" end
+        return
+      end
+      if t > 400 then P("TIMEOUT flag="..string.format("%08X",fl)); M:exit() end
+      return
+    end
+    if st == "finish" then
+      P("---- summary (instruction cycles only; real RAM adds wait states) ----")
+      for _,r in ipairs(results) do
+        P(string.format("   %-44s %8.0f cyc  %5.1f%% of a frame  %6.2f ms",
+                        r.name, r.cyc, 100*r.cyc/FRAME12, 1000*r.cyc/CPUHZ))
+      end
+      P("done")
+      M:exit()
+    end
+  end)
+  if not ok then print("[LOD] LUA ERROR: "..tostring(err)); M:exit() end
+end)

tools/bench/load_run.sh

@@ -0,0 +1,54 @@
+#!/bin/bash
+# One load-time transform run: the 68000 builds its own codebooks and palette
+# out of the RAW container header, on both CPU cores (ROADMAP P1+P2, FINDINGS
+# 53).
+#
+#   tools/bench/load_run.sh [container]
+#
+# Both instruments run the same loadgate.bin over the same header bytes:
+#   * MAME, which is the only one of the two with real PALETTE REGISTERS -- the
+#     packed words are read back out of $E82000, not out of a RAM shadow, so
+#     "the words reached the hardware" is part of what passes.
+#   * px68k's C68K, which is exact to the cycle and counts BUS cycles, and is a
+#     second opinion on the cost from a separately written cycle table.
+# Both outputs are compared byte-for-byte against tools/bench/dlxload.py, which
+# stays the reference: this code replaces where those transforms RUN, not what
+# they produce.
+set -e
+cd "$(dirname "$0")/../.."
+DLX=${1:-tmp/rc_fr_singe_scsi_span.dlx}
+PX68K=${PX68K:-$HOME/src/px68k}
+ITER=${DLX_LOAD_ITER:-40}
+
+tools/vasm/vasmm68k_mot -Fbin -o tmp/loadgate.bin src/player/loadgate.s > /dev/null
+python3 tools/bench/prep_load.py "$DLX" > tmp/prep_load.log
+cat tmp/prep_load.log
+
+# stdbuf -oL: without it a long MAME run is unobservable until it exits, and a
+# run that is merely finishing looks exactly like one that is wedged (34.1).
+( cd tmp && DLX_LOAD_ITER=$ITER SDL_VIDEODRIVER=dummy stdbuf -oL timeout -k 5 180 \
+    mame x68000 -bios ipl10 -ramsize 2M -video soft -window -sound none \
+    -nothrottle -plugins -autoboot_script ../tools/bench/load.lua \
+    -seconds_to_run 30 > load_check.log 2>&1 )
+# A run that never reached the dump must fail as that, not as a byte mismatch.
+grep -q "^\[LOD\] done" tmp/load_check.log || {
+  echo "FAIL: the load rig did not finish -- no completion marker."
+  tail -6 tmp/load_check.log; exit 1; }
+grep -a "^\[LOD\]" tmp/load_check.log | sed -n '/summary/,$p' | sed 's/\[LOD\] /  /'
+python3 tools/bench/verify_load.py "$DLX"
+
+if [ -f "$PX68K/m68000/c68k.c" ]; then
+  make -s -C tools/bench/c68k PX68K="$PX68K" 2>/dev/null
+  for M in 4 1 2 7 3; do
+    tools/bench/c68k/c68k_bench --code tmp/loadgate.bin --loadraw tmp/load_data.bin \
+      --loadmode $M --loaditer 1 --cb1 8192 --cb4 2048 \
+      $([ $M = 3 ] && echo "--loaddump tmp/load_c68k.bin") 2>&1 >/dev/null \
+      | grep -av arena | sed 's/\[C68K\] /  /'
+  done
+  # The second core's bytes are held to the same standard as the first's.
+  cmp -s tmp/load_c68k.bin tmp/load_out.bin || {
+    echo "FAIL: the two CPU cores produced DIFFERENT load-time output."; exit 1; }
+  echo "  OK  both CPU cores produced the same $(stat -c%s tmp/load_out.bin) B"
+else
+  echo "  SKIPPED: no px68k at $PX68K (set PX68K= to point at a checkout)"
+fi

tools/bench/mkvol.sh

@@ -0,0 +1,76 @@
+#!/bin/bash
+# Build the SCSI VOLUME the P4 rigs read, and the blank card ROM MAME needs to
+# instantiate the card.  Sourced-by-calling from tools/bench/scsi_run.sh and
+# tools/bench/pace_run.sh so there is ONE copy of the layout.
+#
+#   tools/bench/mkvol.sh [container.dlx | container.dlxp]
+#
+# TWO CONTAINERS, ONE VOLUME BUILDER.  A DLX volume is tools/bench/prep_stream.py's
+# disk image -- the codec's records, laid down from sector 0.  A DLXP volume
+# (ROADMAP K2/K3) needs no preparation at all: the container is ALREADY a
+# sector-aligned image of itself -- a 512 B header, then fixed 97-sector records
+# -- so the file IS the volume and copying it is the whole build.  That is not a
+# convenience, it is the format's central claim (tools/encoder/dlxp.py) arriving
+# at the disc, and a builder that transformed it on the way would be hiding the
+# claim rather than testing it.
+#
+# The two get DIFFERENT CHD NAMES.  Alternating between the packed rig and the
+# codec rig would otherwise rebuild the volume on every run, and -- much worse --
+# a stale CHD under the name the other rig expected would serve one container's
+# bytes to the other's gate, which reads as a decode failure and is not one.
+#
+# ONE COPY, ON PURPOSE.  The volume is tmp/stream_disk.bin -- byte for byte the
+# file the host-file ring rig reads -- laid out as 512 B sectors.  If two scripts
+# each built it, a difference between the SCSI rig and the modelled-transport rig
+# could be a difference in what they were reading, and the whole value of running
+# both is that it cannot be.  This tree has already paid twice for a transform
+# with two copies of itself (FINDINGS 49.7.5, and check.sh's dlxload note).
+#
+# THE BLANK BOOT ROM is the substitution session 25 argued for and it is
+# unchanged: MAME refuses to instantiate the CZ-6BS1 without an 8 KB
+# `scsiexrom.bin` (CRC 7be488de) that the player never executes, so a zero-filled
+# placeholder goes on a SEPARATE rompath and the user's romset is untouched.
+# MAME prints WRONG CHECKSUMS, as it should.  DO NOT reuse this rompath for
+# anything that boots from the card or calls SCSI IOCS -- those DO execute it.
+set -e
+cd "$(dirname "$0")/../.."
+DLX=${1:-tmp/rc_fr_singe_scsi_span.dlx}
+
+case "$DLX" in
+  *.dlxp) SRC="$DLX"
+          IMG=tmp/dlxpdisk.img; CHD=tmp/dlxpdisk.chd ;;
+  *)      SRC=tmp/stream_disk.bin
+          IMG=tmp/dlxdisk.img;  CHD=tmp/dlxdisk.chd
+          [ -f "$SRC" ] || python3 tools/bench/prep_stream.py "$DLX" > /dev/null ;;
+esac
+
+if [ ! -f "$CHD" ] || [ "$SRC" -nt "$CHD" ]; then
+  SRC="$SRC" IMG="$IMG" python3 - <<'PY'
+import os
+src, img = os.environ["SRC"], os.environ["IMG"]
+d = open(src, "rb").read()
+n = (len(d) + 511) // 512
+open(img, "wb").write(d + b"\0" * (n * 512 - len(d)))
+print(f"  disc image: {len(d)} B of records -> {n} sectors")
+PY
+  rm -f "$CHD"
+  # -c none IS LOAD-BEARING, and it was found by a gate rather than by taste.
+  # Session 28, on the DLX5 volume: with the default (lzma/zlib/huff/flac) MAME
+  # 0.277 served the CHD FILE'S OWN BYTES as sector data -- the destination
+  # buffer after READ(10) at LBA 0 was byte-for-byte the first 4,096 bytes of
+  # dlxdisk.chd, starting "MComprHD" -- while `chdman verify` reported both SHA1s
+  # correct.  Uncompressed, the identical image reads byte-exact.  The trigger is
+  # the image's CONTENT: the same 8,768-sector length that works for the DLX4
+  # volume fails for the DLX5 one, a conventional 16x63 geometry fails too, and
+  # `-c zlib` alone fails as well.  The MAME-side cause is NOT diagnosed; what is
+  # measured is that compression decides it and uncompressed is sound.
+  # Costs 4.5 MB in tmp/ against 1.6 MB.  DO NOT restore compression to save the
+  # disc space: the failure is SILENT at the transport layer -- every READ(10)
+  # reports success and returns the wrong bytes -- and only the byte comparison
+  # in tools/bench/scsi.lua catches it.
+  chdman createhd -i "$IMG" -o "$CHD" -ss 512 -c none > /dev/null 2>&1
+fi
+
+mkdir -p tmp/p4roms/x68k_cz6bs1
+[ -f tmp/p4roms/x68k_cz6bs1/scsiexrom.bin ] || \
+  head -c 8192 /dev/zero > tmp/p4roms/x68k_cz6bs1/scsiexrom.bin

tools/bench/pace_run.sh

@@ -15,12 +15,81 @@
 #
 # kbps 0 = unlimited pipe. There is no default rate anywhere in this tree
 # (FINDINGS 50) and there is none here either.
+#
+# DLX_RINGOWN=1 hands the RING to the 68000 as well (ROADMAP P5,
+# src/player/ring.i): this script's Lua stops placing records and becomes a
+# transport that answers one request at a time. DLX_PREFILL_FR is then the
+# prefill policy, in whole records. It needs a DLX4 container, because the
+# machine cannot learn a record's length by walking a stream it has not fetched.
+#
+# DLX_ITER=2 runs the scene TWICE, which under DLX_RINGOWN means a real seek
+# between the passes: the channel goes quiet, the ring is declared empty and the
+# whole accumulated lookahead is thrown away and rebuilt from the prefill. It
+# needs DLX_PACE=2, because rebasing the frame clock across a pass is the
+# machine's to do and a host-written tick would carry on counting.
+#
+# DLX_XFER=scsi replaces the MODELLED transport with a real one (ROADMAP P4b,
+# src/player/xfer.i): the machine gets a CZ-6BS1 and the same volume the SCSI
+# gate reads, this script stops moving bytes altogether, and every record is
+# fetched by the 68000 with READ(10). It needs DLX_RINGOWN=1 -- the mailbox it
+# answers is ring.i's -- and it FORBIDS a modelled rate, because there is no
+# longer anything for one to model.
+#
+# DLX_PACE selects WHO KEEPS THE TIME: 1 (default) is the host writing the tick,
+# 2 is the 68000 writing it off the CRTC's V-DISP (ROADMAP P3, FINDINGS 54).
+# Everything else about the run is identical, which is the whole point -- the
+# pace gate in src/player/stream.s cannot tell them apart, so a difference in
+# the result is a difference in the CLOCK and not in the rig.
 set -e
 cd "$(dirname "$0")/../.."
 RING=${1:?ring KB}; KBPS=${2:?pipe KB/s, or 0 for unlimited}
 CUT_AT=$3; CUT_FR=${4:-1}
 DLX=${DLX:-tmp/rc_fr_singe_scsi_span.dlx}
+PACE=${DLX_PACE:-1}
+OWN=${DLX_RINGOWN:-0}
+ITERS=${DLX_ITER:-1}
+XFER=${DLX_XFER:-model}
+# EMULATED seconds the run is allowed. A pass that is cut short compares a
+# half-drawn screen and reads as a wrap bug, so this is raised deliberately
+# rather than left to a timeout: a DLX_XFER=scsi pass costs ~46 s of emulated
+# time against the modelled transport's ~7, because the CPU moves every byte
+# itself (FINDINGS 58.2), and two of them do not fit in 90.
+SECS=${DLX_SECONDS:-90}
+if [ "$XFER" = scsi ]; then
+  [ "$OWN" = 1 ] || { echo "DLX_XFER=scsi needs DLX_RINGOWN=1: the transport in"
+    echo "src/player/xfer.i answers src/player/ring.i's mailbox, and with the"
+    echo "host owning the ring there is no mailbox to answer."; exit 2; }
+  # A rate is not merely ignored here, it is REFUSED. The bytes now arrive on
+  # the emulated machine's own time, and a run labelled "488 KB/s" that did not
+  # deliver at 488 KB/s is exactly the kind of number this project has twice
+  # paid for. There is no rate in a DLX_XFER=scsi run, and the log says so.
+  [ "$KBPS" = 0 ] || { echo "DLX_XFER=scsi takes kbps 0. The transport is real,"
+    echo "so nothing here delivers at a modelled rate -- and MAME's device"
+    echo "models are functional, not transfer-timing accurate, so the rate it"
+    echo "DOES deliver at is not a measurement either (docs/BENCHMARK.md)."
+    exit 2; }
+  command -v chdman > /dev/null || { echo "DLX_XFER=scsi needs chdman (ships"
+    echo "with mame-tools) to build the volume."; exit 2; }
+  bash tools/bench/mkvol.sh "$DLX"
+fi
+if [ "$OWN" = 1 ] && [ "$ITERS" != 1 ] && [ "$PACE" != 2 ]; then
+  echo "DLX_ITER>1 needs DLX_PACE=2: the frame clock is rebased per pass by"
+  echo "src/player/stream.s, and a host-written tick would go on counting"
+  echo "through the seek and open every slot of the second pass at once."
+  exit 2
+fi
 TAG="r${RING}_k${KBPS}${CUT_AT:+_cut${CUT_AT}x${CUT_FR}}"
+# The default tag is left ALONE when the host keeps the time: tools/bench/
+# pace_sweep.sh reads tmp/pace_r<ring>_k<kbps>.log by name, and renaming the
+# host-paced logs would break a sweep that has nothing to do with this option.
+if [ "$PACE" != 1 ]; then TAG="${TAG}_p$PACE"; fi
+if [ "$OWN" = 1 ]; then TAG="${TAG}_own"; fi
+if [ "$ITERS" != 1 ]; then TAG="${TAG}_x$ITERS"; fi
+if [ "$XFER" != model ]; then TAG="${TAG}_$XFER"; fi
+MAMEX=()
+if [ "$XFER" = scsi ]; then
+  MAMEX=(-exp1 cz6bs1 -rompath "$HOME/mame/roms;./p4roms" -hard dlxdisk.chd)
+fi
 
 tools/vasm/vasmm68k_mot -Fbin -o tmp/stream.bin src/player/stream.s > /dev/null
 [ -f tmp/stream_disk.bin ] || python3 tools/bench/prep_stream.py "$DLX" > tmp/prep_stream.log
@@ -29,12 +98,16 @@ mkdir -p "tmp/snap_pace_$TAG"; rm -f "tmp/snap_pace_$TAG/x68000"/*.png
 # of a prefix is not an assignment token, so bash takes the next word as the
 # command and the run dies with "SDL_VIDEODRIVER=dummy: command not found".
 CUTENV=(); [ -n "$CUT_AT" ] && CUTENV=(DLX_CUT_AT="$CUT_AT" DLX_CUT_FR="$CUT_FR")
-( cd tmp && env DLX_PACE=1 DLX_RING_KB=$RING DLX_STREAM_KBPS=$KBPS \
+( cd tmp && env DLX_PACE=$PACE DLX_RING_KB=$RING DLX_STREAM_KBPS=$KBPS \
+    DLX_RINGOWN=$OWN DLX_ITER=$ITERS DLX_XFER=$XFER \
+    ${DLX_PREFILL_FR:+DLX_PREFILL_FR=$DLX_PREFILL_FR} \
     "${CUTENV[@]}" DLX_SLACK_CSV="slack_$TAG.csv" \
     SDL_VIDEODRIVER=dummy stdbuf -oL timeout -k 5 900 \
-    mame x68000 -bios ipl10 -ramsize 2M -video soft -window -sound none \
+    mame x68000 -bios ipl10 "${MAMEX[@]}" -ramsize 2M -video soft -window \
+    -sound none \
     -nothrottle -plugins -autoboot_script ../tools/bench/stream.lua \
-    -snapshot_directory "./snap_pace_$TAG" -snapview native -seconds_to_run 90 \
+    -snapshot_directory "./snap_pace_$TAG" -snapview native \
+    -seconds_to_run $SECS \
     > "pace_$TAG.log" 2>&1 )
 # The completion marker is not optional: a run killed mid-decode compares a
 # half-drawn screen and reads as a wrap bug rather than as a truncated run.
@@ -42,6 +115,6 @@ grep -q "snapshot taken" "tmp/pace_$TAG.log" || {
   echo "FAIL($TAG): no snapshot marker -- the pass did not complete."
   tail -6 "tmp/pace_$TAG.log"; exit 1; }
 echo "=== $TAG"
-grep -aE "decoder (PACED|FREE)|ring: |UNDERRUNS|SEEK SLACK|RING-BOUND|RATE-BOUND|BUILD TIME|PIPE CUT|DEADLINE|REQUIRED" \
+grep -aE "decoder (SELF-PACED|PACED|FREE)|FRAME CLOCK|ring: |UNDERRUNS|NO IDLE|SEEK SLACK|RING-BOUND|RATE-BOUND|BUILD TIME|PIPE CUT|DEADLINE|REQUIRED|MACHINE-OWNED|PREFILL:|CHANNEL IDLE|MISPLACED|SEEK PASS|REAL TRANSPORT|SECTOR OVERHEAD|TRANSPORT FAILED|IS VACUOUS" \
   "tmp/pace_$TAG.log" | sed "s/\[STR\] /  /"
 python3 tools/bench/verify_decode.py "$DLX" --snap "tmp/snap_pace_$TAG" | tail -2

tools/bench/packed.lua

@@ -0,0 +1,509 @@
+-- Drive src/player/packed.s: THE DECODER-FREE PACKED PLAYER, END TO END,
+-- OFF A REAL VOLUME.  ROADMAP K3.
+--
+-- WHAT THIS SCRIPT DOES NOT DO IS THE POINT OF IT.  tools/bench/stream.lua
+-- pushes expanded codebooks and a packed palette into RAM, plays a transport at
+-- a modelled byte rate, and writes the frame tick.  This one pushes 2,898 bytes
+-- of 68000 code and eleven mailbox words, and then READS.  It moves no picture
+-- byte, models no rate, sets no CRTC register and writes no palette entry: the
+-- machine brings up its own display, builds its own chain, keeps its own clock
+-- off V-DISP and fetches every record itself with READ(10) off a CZ-6BS1.
+--
+-- SO THE GATE IS NOT THE ONE THE CODEC USES, AND IT HAD TO CHANGE.
+-- tools/bench/verify_decode.py checks ONE frame -- the last -- and that audits
+-- the whole run because the codec is temporally recursive: a SKIP block is a
+-- claim about the previous frame still being on screen, so the final frame is
+-- only right if all 120 were.  A packed frame is a LITERAL.  Frame 119 being
+-- pixel-exact says nothing whatever about frame 60.  This script therefore
+-- snapshots EVERY frame and tools/bench/verify_packed.py compares all of them;
+-- the simplification that deleted the ring also deleted the gate's free lunch.
+--
+-- WHEN A SNAPSHOT IS TAKEN, and why not on the frame it changed.  PG_SHOWN is
+-- bumped by the 68000 after it clears R20 bit 11, so a change means "a complete
+-- frame is now displayable".  But MAME's screen bitmap for the host frame in
+-- progress was drawn partly before that instant, so snapshotting immediately
+-- would sample the write window -- which BLANKS the graphics layer -- for part
+-- of the picture.  A 12 fps frame lasts 4 or 5 host refreshes at 56.69 Hz, so
+-- waiting SNAP_DELAY whole host frames is safely inside the slot and safely
+-- after the window closed.
+--
+-- Env:
+--   DLX_PK_HELD    1 = the channel HOLDS THE BUS (burst, max rate), 0 = it
+--                  steals cycles.  Not two speeds of one thing: 59.3 showed an
+--                  auto-requested channel is charged by TIME, so held is the
+--                  68000 stopped for as long as the record takes to arrive.
+--                  Default 1.
+--   DLX_PK_PACE    1 = the machine holds itself to the container's fps off
+--                  V-DISP (default). 0 free-runs, which tests the CHAIN with
+--                  the clock out of the way.
+--   DLX_PK_ITER    passes over the scene (default 1). >1 exercises the SEEK.
+--   DLX_PK_SEEK    the frame passes after the first start at (default 0, i.e.
+--                  a replay).  On the video path the seek IS arithmetic and
+--                  nothing else; on the audio path it is a second read at a
+--                  separate LBA, because a DLXP2 group puts lump k in FRONT of
+--                  its records and a branch lands `f mod F` frames into it
+--                  (FINDINGS 70.3).  Setting this to a frame that is NOT a
+--                  multiple of the cadence is the point: 36 of the arcade's
+--                  409 within-container seek targets land on a boundary and
+--                  373 do not.
+--   DLX_PK_NFR     play only the first N frames (default: all of them)
+--   DLX_PK_FPS     pace at this rate instead of the container's.  NOT a
+--                  cosmetic knob and not a way to make a number look better:
+--                  under MAME the emulated transport takes about a whole 12 fps
+--                  slot to deliver a 49,664 B record, and the write window has
+--                  to be OPEN for all of it -- so at 12 fps there is no instant
+--                  at which a complete frame is displayable and nothing can be
+--                  snapshotted.  Pacing slower opens a display interval without
+--                  changing one byte of the transfer, which is what lets the
+--                  PIXEL-EXACTNESS of all 120 frames be gated separately from
+--                  the RATE the emulated transport happens to run at.  The two
+--                  are different questions and this is the knob that separates
+--                  them.
+--   DLX_PK_CSV     write the per-frame arrival series here
+
+M = manager.machine
+SP = M.devices[":maincpu"].spaces["program"]
+
+local function findfile(n)
+  for _,p in ipairs{"../tools/bench/"..n, "tools/bench/"..n, n} do
+    local f = io.open(p,"rb"); if f then f:close(); return p end
+  end
+  error(n.." not found")
+end
+local META = loadfile("packed_meta.lua")()
+
+-- src/player/packed.s.  Inputs first, then outputs; the split is the file's.
+local PG_FLAG, PG_NFR, PG_FPS, PG_LBA0 = 0x18900, 0x18904, 0x18908, 0x1890C
+local PG_RECS, PG_PALL, PG_HELD        = 0x18910, 0x18914, 0x18918
+local PG_PACEON, PG_ITER               = 0x1891C, 0x18920
+local PG_CADF, PG_CADA, PG_AUDON       = 0x18924, 0x18928, 0x1892C
+local PG_SHOWN, PG_ERR, PG_ERRAT       = 0x18930, 0x18934, 0x18938
+local PG_LATE, PG_LATE1, PG_LATEM      = 0x1893C, 0x18940, 0x18944
+local PG_VDISP, PG_VD0, PG_TSPIN       = 0x18948, 0x1894C, 0x18950
+local PG_GSPIN, PG_LOSTV, PG_ARRN      = 0x18954, 0x18958, 0x1895C
+-- ROADMAP P6c: the audio path's inputs and its own account of what it did.
+local PG_AFPS, PG_AHZ, PG_ALBA0        = 0x18960, 0x18964, 0x18968
+local PG_NLUMP, PG_ABYTES, PG_APRE     = 0x1896C, 0x18970, 0x18974
+local PG_SEEKF, PG_ARST                = 0x18978, 0x1897C
+local PG_APOS, PG_ASKIP                = 0x189C8, 0x189CC
+local PG_ASKN, PG_ASKB, PG_TSEQ        = 0x189D0, 0x189D4, 0x189DC
+local PG_AARM, PG_AFET, PG_ABYT        = 0x18980, 0x18984, 0x18988
+local PG_ADRY, PG_ASEAM, PG_ASRV       = 0x1898C, 0x18990, 0x18994
+local PG_ACSR, PG_ACER, PG_ALATE       = 0x18998, 0x1899C, 0x189A0
+local PG_AMTC0, PG_AK, PG_AKF, PG_AACC = 0x189A4, 0x189B0, 0x189B4, 0x189B8
+local PG_AFERR, PG_AFERA               = 0x189A8, 0x189AC
+local PG_ARR                           = 0x1B000
+-- src/player/clock.i and src/player/scsi.i, read for diagnosis only.
+local CLK_PACE, CLK_VDISP, CLK_ERR     = 0x18034, 0x18064, 0x1806C
+local SC_ERR                           = 0x18200
+local CRTC_R20                         = 0xE80028
+
+local HELD    = (os.getenv("DLX_PK_HELD") or "1") == "1"
+local PACED   = (os.getenv("DLX_PK_PACE") or "1") == "1"
+local ITERS   = tonumber(os.getenv("DLX_PK_ITER") or "") or 1
+local SEEKF   = tonumber(os.getenv("DLX_PK_SEEK") or "") or 0
+-- ROADMAP P6d.  1 = STOP and re-PLAY the chip at a branch.  Not a tidiness
+-- knob: the chip's accumulator has no leak, so the two settings are a large
+-- decaying error against a small permanent one (FINDINGS 71.3).
+local ARST    = (os.getenv("DLX_PK_ARST") or "0") == "1"
+local NFR     = tonumber(os.getenv("DLX_PK_NFR") or "") or META.nframes
+local FPS     = tonumber(os.getenv("DLX_PK_FPS") or "") or META.fps
+local CSV     = os.getenv("DLX_PK_CSV")
+-- ROADMAP P6c.  OFF by default and it is not a convenience: a run with the chip
+-- silent is the CONTROL this one is read against, and every gate that existed
+-- before session 36 is that control.  DLX_PK_APRE is the lumps fetched before
+-- frame 0 -- see src/player/packed.s on why the answer is not 1.
+local AUDIO   = (os.getenv("DLX_PK_AUD") or "0") == "1"
+local APRE    = tonumber(os.getenv("DLX_PK_APRE") or "") or 2
+local AJSON   = os.getenv("DLX_PK_AJSON")
+local SNAP_DELAY = 2
+
+local SCERRNAME = {[0]="OK", "SELECTION TIMEOUT -- no target answered",
+                   "UNEXPECTED PHASE", "POLL TIMEOUT -- a phase never arrived",
+                   "NON-ZERO SCSI STATUS",
+                   "WINDOWED READ REFUSED -- a channel cannot drop bytes"}
+
+local code do local f=assert(io.open("packed.bin","rb")); code=f:read("a"); f:close() end
+
+local function T() local t=M.time; return t.seconds + t.attoseconds/1e18 end
+local function P(s) print("[PK] "..s) end
+
+local function setup()
+  for i = 1, #code do SP:write_u8(0x10000+i-1, string.byte(code,i)) end
+  SP:write_u32(PG_FLAG, 0)
+  SP:write_u32(PG_NFR, NFR)
+  SP:write_u32(PG_FPS, FPS)
+  SP:write_u32(PG_LBA0, META.lba0)
+  SP:write_u32(PG_RECS, META.rec_sectors)
+  SP:write_u32(PG_PALL, META.palette_last)
+  SP:write_u32(PG_HELD, HELD and 1 or 0)
+  SP:write_u32(PG_PACEON, PACED and 1 or 0)
+  SP:write_u32(PG_ITER, ITERS)
+  SP:write_u32(PG_SEEKF, SEEKF)
+  SP:write_u32(PG_ARST, ARST and 1 or 0)
+  -- DLXP2's cadence. Zero for a silent container, and the 68000 branches on the
+  -- zero: a player told the wrong cadence does not fail, it reads an audio lump
+  -- as a record and paints it.
+  SP:write_u32(PG_CADF, META.cad_f or 0)
+  SP:write_u32(PG_CADA, META.cad_a or 0)
+  SP:write_u32(PG_SHOWN, 0)
+  -- P6c. AUDON is separate from the cadence on purpose: the LBA arithmetic has
+  -- to skip the lumps whether or not a chip is being fed, and a run that skips
+  -- them without playing them is the control this one is measured against.
+  SP:write_u32(PG_AUDON, (AUDIO and (META.has_audio or 0) == 1) and 1 or 0)
+  SP:write_u32(PG_AFPS, META.fps)          -- the CONTAINER's, NOT the pace
+  SP:write_u32(PG_AHZ, META.aud_hz or 0)
+  SP:write_u32(PG_ALBA0, META.lba_aud or 0)
+  SP:write_u32(PG_NLUMP, META.n_lumps or 0)
+  SP:write_u32(PG_ABYTES, META.aud_bytes or 0)
+  SP:write_u32(PG_APRE, APRE)
+  if ITERS > 1 then
+    local grp = (META.cad_f or 0) > 0 and (SEEKF % META.cad_f) or 0
+    P(string.format("SEEK: %d passes, and passes 2..%d start at FRAME %d%s",
+                    ITERS, ITERS, SEEKF,
+                    (META.cad_f or 0) == 0 and " (silent container)"
+                    or string.format(" -- lump %d, %d frame(s) into its group "
+                       .."of %d, so the audio needs a second read and a byte "
+                       .."offset (FINDINGS 70.3)%s",
+                       SEEKF // META.cad_f, grp, META.cad_f,
+                       grp == 0 and " -- ON a group boundary, the free case"
+                                or "")))
+    P(ARST and "     the chip is STOPPED and re-PLAYED at the branch: its "
+               .."accumulator goes to the container's own init and its step "
+               .."index to 0"
+            or "     the chip PLAYS THROUGH the branch: it keeps the predictor "
+               .."state the previous scene's audio left it in")
+  end
+  P(string.format("packed.bin=%d B, %dx%d %d fps, %d of %d frames, %d passes",
+                  #code, META.W, META.H, META.fps, NFR, META.nframes, ITERS))
+  local cad = ""
+  if (META.cad_f or 0) > 0 then
+    cad = string.format(" + (i//%d)*%d", META.cad_f, META.cad_a)
+  end
+  P(string.format("record %d B = %d sectors at LBA %d + i*%d%s, palette %s",
+                  META.rec_bytes, META.rec_sectors, META.lba0,
+                  META.rec_sectors, cad,
+                  META.palette_last == 1 and "LAST" or "FIRST"))
+  if (META.cad_f or 0) > 0 then
+    P(string.format("DLXP2: %d B of ADPCM at %d Hz rides in %d lumps of %d sectors, "
+                    .."one in front of every %d records -- the third term above is the "
+                    .."whole cost of it on the video path",
+                    META.aud_bytes, META.aud_hz, META.n_lumps, META.cad_a, META.cad_f))
+  end
+  if AUDIO and (META.has_audio or 0) == 1 then
+    P(string.format("AUDIO ON: lump k at LBA %d + k*%d, payload 11*%d/24 B a "
+                    .."group -- the PAYLOAD and not the %d B lump (FINDINGS "
+                    .."67.2). Decoder from the header: %s/%s, %d-bit clamp, "
+                    .."accumulator %d at PLAY. Prefill %d lumps of %d.",
+                    META.lba_aud, META.cad_f*META.rec_sectors + META.cad_a,
+                    META.aud_hz, META.cad_a*512, META.aud_variant,
+                    META.aud_order, META.aud_bits, META.aud_init,
+                    APRE, META.n_lumps))
+  elseif (META.has_audio or 0) == 1 then
+    P("audio present in the container and NOT played -- this is the silent "
+      .."control (DLX_PK_AUD=1 plays it)")
+  end
+  P(string.format("channel: %s, %s",
+                  HELD and "BUS HELD (burst, max rate)" or "CYCLE STEALING",
+                  PACED and ("SELF-PACED at "..FPS.." fps off V-DISP"
+                             ..(FPS ~= META.fps and (" -- NOT the container's "
+                               ..META.fps..", see DLX_PK_FPS") or ""))
+                        or "FREE-RUNNING (tests the chain, not the clock)"))
+  P("this script writes NO picture byte, NO palette entry and NO CRTC register: "
+    .."the machine brings up its own display and fetches its own records.")
+end
+
+local function launch()
+  local cpu = M.devices[":maincpu"]
+  cpu.state["SR"].value = 0x2700          -- supervisor, all interrupts masked;
+  cpu.state["SP"].value = 0x8000          -- clk_init lowers it to $2500 itself
+  cpu.state["PC"].value = 0x10000
+end
+
+local st, t0 = "boot", nil
+local shown, pending, snaps = 0, nil, 0
+local arrive, hostfr, missed = {}, 0, 0
+-- WHICH FRAME EACH SNAPSHOT IS.  MAME numbers snapshots 0000, 0001, ... in the
+-- order they were taken, and a frame that could not be sampled leaves no gap in
+-- that sequence -- so the file name is NOT the frame index and a verifier that
+-- assumed it was would compare frame 61 against record 60 and report a codec
+-- bug that is really a bookkeeping one.  This is the map, written out for it.
+local snapfr = {}
+local r20seen = {}
+
+SUB = emu.add_machine_frame_notifier(function()
+  local ok, err = pcall(function()
+    local t = T()
+    if st == "boot" then
+      if t < 3.0 then return end
+      setup(); launch(); t0 = t; st = "running"; return
+    end
+    if st ~= "running" then return end
+    hostfr = hostfr + 1
+
+    -- WHAT THE SCREEN MODE WAS, sampled every host frame.  R20 bit 11 blanks
+    -- the graphics layer, so this is the only way to see the shutter the player
+    -- is running: the fraction of host frames that found the window OPEN is the
+    -- fraction of the scene the display spent dark, and it is a MEASUREMENT of
+    -- 47.4's cost under MAME rather than a restatement of the prior.
+    local r20 = SP:read_u16(CRTC_R20)
+    r20seen[#r20seen+1] = ((r20 >> 11) & 1)
+
+    local s = SP:read_u32(PG_SHOWN)
+    if s > shown then
+      -- Only the LAST change matters if several landed in one host frame; that
+      -- cannot happen at 12 fps on a 56.69 Hz raster, and if it ever does the
+      -- gate below catches it as a missing snapshot rather than a wrong one.
+      arrive[#arrive+1] = {n = s, t = t - t0}
+      shown = s
+      pending = SNAP_DELAY
+    end
+    if pending then
+      pending = pending - 1
+      if pending <= 0 then
+        pending = nil
+        if (SP:read_u16(CRTC_R20) >> 11) & 1 == 0 then
+          M.video:snapshot(); snaps = snaps + 1
+          snapfr[#snapfr+1] = shown - 1
+        else
+          -- The window was open again when the delay expired: the frame we
+          -- meant to sample is being overwritten.  COUNTED, NOT PRINTED -- when
+          -- the transfer is longer than the slot EVERY frame misses, and 119
+          -- identical lines bury the four numbers the run exists to report.
+          -- The count is reported once at the end and the gate reads it there.
+          missed = missed + 1
+        end
+      end
+    end
+
+    local flag = SP:read_u32(PG_FLAG)
+    if flag ~= 1 and pending then
+      -- THE LAST FRAME IS STILL PENDING.  packed.s spins in pg_hold with the
+      -- window CLOSED once the scene is over, so the delay can simply run out;
+      -- returning here rather than reporting is what stops the final frame
+      -- being the one frame the gate never sees.
+      return
+    end
+    if flag ~= 1 then
+      st = "done"
+      local wall = t - t0
+      P(string.format("FLAG=$%02X after %.3f s, %d frames shown, %d snapshots, "
+                      .."%d frames NOT SAMPLED (the write window had reopened "
+                      .."-- the transfer is longer than the display interval)",
+                      flag, wall, shown, snaps, missed))
+      local err = SP:read_u32(PG_ERR)
+      if err ~= 0 then
+        P(string.format("TRANSPORT FAILED on frame %d: %s",
+                        SP:read_u32(PG_ERRAT), SCERRNAME[err] or ("code "..err)))
+      end
+      P(string.format("array: the 68000 built %d entries (the container wants "
+                      .."%d)", SP:read_u32(PG_ARRN), META.entries))
+      -- The first entry, read back out of the machine's own RAM.  It is the one
+      -- place palette-first and palette-last are visible as a FACT rather than
+      -- as a flag the rig passed in and the rig read back.
+      P(string.format("chain[0] = MAR $%06X MTC %d ; chain[1] = MAR $%06X MTC %d",
+                      SP:read_u32(PG_ARR), SP:read_u16(PG_ARR+4),
+                      SP:read_u32(PG_ARR+6), SP:read_u16(PG_ARR+10)))
+      if PACED then
+        local vd = SP:read_u32(PG_VDISP) - SP:read_u32(PG_VD0)
+        -- THE CLOCK, AGAINST THE RASTER THAT DROVE IT.  CLK_VDISP counts the
+        -- edges the 68000's ISR SAW.  hostfr counts the frames MAME actually
+        -- drew.  A held channel halts the CPU, and the MFP's pending bit is one
+        -- bit, so an edge that falls inside a transfer long enough to span two
+        -- of them is an edge the machine can never count.  Nothing in this
+        -- project has ever run a transfer and a clock at once, so nothing could
+        -- have seen this before.
+        P(string.format("frame clock: PACE=%d ticks, V-DISP edges SEEN=%d, "
+                        .."host frames drawn=%d -> %d edges LOST (%.1f%%)",
+                        SP:read_u32(CLK_PACE), vd, hostfr, hostfr - vd,
+                        hostfr > 0 and (hostfr-vd)*100/hostfr or 0))
+        local nlate = SP:read_u32(PG_LATE)
+        P(string.format("late frames (tick already past at the gate): %d%s",
+                        nlate, nlate > 0 and string.format(", first %d, worst "
+                          .."%d ticks", SP:read_u32(PG_LATE1),
+                          SP:read_u32(PG_LATEM)) or ""))
+        -- AND WHY `late = 0` IS NOT `on time`.  The gate compares the frame
+        -- index against PACE, and PACE is advanced by the ISR that the held
+        -- channel stops the CPU from running. A clock that loses edges loses
+        -- them from BOTH sides of the comparison, so a player whose own clock
+        -- has halved still reports every frame early. The LOST figure above is
+        -- the only thing in this run that can contradict it, and it comes from
+        -- the host's raster count rather than from the machine.
+        if hostfr - vd > 0 then
+          P(string.format("            ...and %d of those ticks were never "
+                          .."issued, so `late=%d` is measured against a clock "
+                          .."running at %.1f%% of the raster. The player "
+                          .."believes it is at %d fps and the screen is at "
+                          .."%.2f.", hostfr - vd, nlate, vd*100/hostfr, FPS,
+                          FPS * vd / hostfr))
+        end
+      end
+      P(string.format("CPU: %d trips round the TRANSFER wait in total, %d round "
+                      .."the PACE gate", SP:read_u32(PG_TSPIN),
+                      SP:read_u32(PG_GSPIN)))
+      if AUDIO and (META.has_audio or 0) == 1 then
+        local armed, fet = SP:read_u32(PG_AARM), SP:read_u32(PG_AFET)
+        local byt, dry   = SP:read_u32(PG_ABYT), SP:read_u32(PG_ADRY)
+        local seam, srv  = SP:read_u32(PG_ASEAM), SP:read_u32(PG_ASRV)
+        local late, acc  = SP:read_u32(PG_ALATE), SP:read_u32(PG_AACC)
+        P(string.format("AUDIO: %d of %d lumps armed, %d fetched, %d B of "
+                        .."payload handed to the chip (the stream is %d B)",
+                        armed, META.n_lumps, fet, byt, META.aud_bytes))
+        -- THE PADDING, CHARGED.  A player that fed the chip the whole lump
+        -- would have handed it n_lumps*A*512 B; the difference is the drift
+        -- FINDINGS 67.2 priced at 1.25 s over the game, and printing both
+        -- numbers is the only way the accumulator is visible from outside.
+        -- FULL groups only.  The last lump is short when the scene's frame
+        -- count is not a multiple of F, and averaging that in reports the
+        -- scene's TAIL as though it were the cadence -- which is a different
+        -- number from the drift and looks like a worse one.  The percentage
+        -- lives in verify_packed_audio.py, which knows each lump's payload.
+        local nfull = META.n_lumps
+        if NFR % META.cad_f ~= 0 then nfull = nfull - 1 end
+        local lumpb = nfull * META.cad_a * 512
+        P(string.format("        %d whole groups: %d B of lump space for the "
+                        .."payload the accumulator asked for. The whole-lump "
+                        .."player feeds the chip that space, and the excess is "
+                        .."DRIFT and not waste (67.2). Accumulator left at "
+                        .."%d/%d.", nfull, lumpb, acc, 2*META.fps))
+        P(string.format("        service: %d calls, %d found the channel "
+                        .."counted out, %d of those had NO lump ready (a "
+                        .."STARVE -- the chip replays its last byte)",
+                        srv, seam, dry))
+        P(string.format("        re-arms with MTC still non-zero: %d (bytes "
+                        .."fetched and never played; 0 is the correct value)",
+                        late))
+        -- THE SEEK'S OWN ACCOUNT.  PG_ASKN is the second reads and PG_ASKB the
+        -- bytes they skipped at the head of a lump: a seek path with no offset
+        -- term would report the first and zero for the second, and would be
+        -- indistinguishable from a correct one on any target that happened to
+        -- land on a group boundary.
+        local skn, skb = SP:read_u32(PG_ASKN), SP:read_u32(PG_ASKB)
+        if skn > 0 then
+          P(string.format("        SEEK: %d audio seek(s), %d B skipped into "
+                          .."the head of a lump. Stream position ended at %d B "
+                          .."and the chip was handed %d -- they differ BY the "
+                          .."skip, which is the whole reason they are two cells "
+                          .."(FINDINGS 71).", skn, skb, SP:read_u32(PG_APOS),
+                          byt))
+        end
+        local ferr = SP:read_u32(PG_AFERR)
+        if ferr ~= 0 then
+          P(string.format("        A LUMP FETCH FAILED on lump %d: %s -- the "
+                          .."picture is unaffected and the sound is gone, which "
+                          .."is why this has its own error word",
+                          SP:read_u32(PG_AFERA), SCERRNAME[ferr] or ("code "..ferr)))
+        end
+        P(string.format("        channel 3 at the end: CSR=$%02X CER=$%02X, "
+                        .."MTC one instruction after the first START = %d",
+                        SP:read_u32(PG_ACSR), SP:read_u32(PG_ACER),
+                        SP:read_u32(PG_AMTC0)))
+        -- WHAT HOLDING THE BUS COSTS A SECOND CONSUMER, and it is this line.
+        -- Stealing, pg_aserv runs from inside dma.i's transfer wait as well as
+        -- twice a frame; held, the 68000 is HALTED for the whole transfer and
+        -- the two frame-loop calls are all it gets.  The ratio is the audio's
+        -- half of FINDINGS 64.3.
+        P(string.format("        -> %.1f service calls per frame shown. %s",
+                        shown > 0 and srv/shown or 0,
+                        HELD and ("BUS HELD: the 68000 is halted for the whole "
+                          .."transfer, so DM_HOOK never runs and this is the "
+                          .."two frame-loop calls and nothing else.")
+                             or ("CYCLE STEALING: DM_HOOK ran from inside the "
+                          .."transfer wait, which is where a 68000 driving this "
+                          .."video path has any time at all.")))
+        if AJSON then
+          local f = io.open(AJSON, "w")
+          f:write(string.format('{"armed":%d,"fetched":%d,"bytes":%d,'
+                  ..'"starve":%d,"seam":%d,"serv":%d,"late":%d,"acc":%d,'
+                  ..'"csr":%d,"cer":%d,"held":%s,"fps":%d,"shown":%d,'
+                  ..'"seekn":%d,"seekb":%d,"pos":%d,"seekf":%d,"iters":%d,'
+                  ..'"arst":%s}\n',
+                  armed, fet, byt, dry, seam, srv, late, acc,
+                  SP:read_u32(PG_ACSR), SP:read_u32(PG_ACER),
+                  HELD and "true" or "false", FPS, shown,
+                  SP:read_u32(PG_ASKN), SP:read_u32(PG_ASKB),
+                  SP:read_u32(PG_APOS), SEEKF, ITERS,
+                  ARST and "true" or "false"))
+          f:close()
+          P("audio counters -> "..AJSON)
+        end
+      end
+      local open = 0
+      for _,v in ipairs(r20seen) do open = open + v end
+      P(string.format("WRITE WINDOW OPEN on %d of %d host frames (%.1f%%) -- "
+                      .."buffer mode blanks the graphics layer, so that is the "
+                      .."share of the scene the display spent DARK under MAME",
+                      open, #r20seen, #r20seen > 0 and open*100/#r20seen or 0))
+      if #arrive >= 2 then
+        local dts, first, last = {}, arrive[1].t, arrive[#arrive].t
+        for i = 2, #arrive do dts[#dts+1] = arrive[i].t - arrive[i-1].t end
+        table.sort(dts)
+        -- CADENCE, AND THE GRANULARITY IT IS MEASURED AT.  PG_SHOWN is sampled
+        -- once per host frame, so a single inter-frame figure is quantised to
+        -- 1/56.69 s = 17.6 ms and the min/median/max below are multiples of it.
+        -- The MEAN over the whole run is not: the quantisation error is bounded
+        -- by one host frame at each END, so over n-1 intervals it is 35 ms/(n-1)
+        -- -- 0.30 ms a frame over 120.  Read the mean; the spread is the
+        -- sampler's, not the player's.
+        local mean = (last-first)/(#arrive-1)
+        P(string.format("cadence: %d frames in %.3f s = %.3f fps, mean "
+                        .."%.2f ms/frame (+/- %.2f ms, the sampler's); "
+                        .."inter-frame min %.1f median %.1f max %.1f ms",
+                        #arrive, last-first, (#arrive-1)/(last-first),
+                        mean*1000, 35.3/(#arrive-1),
+                        dts[1]*1000, dts[math.ceil(#dts/2)]*1000,
+                        dts[#dts]*1000))
+        -- THE TRANSPORT'S TIME IS ONLY READABLE OFF A FREE-RUNNING RUN.
+        -- Paced, the mean inter-frame IS THE PACE PERIOD: the player waits for
+        -- its tick and the record's delivery hides inside the slot, so dividing
+        -- the record by that mean reports the clock rather than the transport
+        -- and reports it as a rate. The first cut of this script printed
+        -- "297.4 KB/s" off a 6 fps gate run, which is the pace and not the
+        -- disc. Free-running there is no gate and the loop is transfer-bound,
+        -- so the mean is the transfer.
+        if not PACED then
+          P(string.format("            -> FREE-RUNNING, so the mean IS the "
+                          .."transport: a %d B record lands in %.2f ms, i.e. "
+                          .."%.1f KB/s and %.1f%% of a %d fps slot. MAME's "
+                          .."device models carry no transfer timing "
+                          .."(docs/BENCHMARK.md, 42.5), so this is a property "
+                          .."of the APPARATUS -- it is not W and it is not a "
+                          .."measurement of any medium.",
+                          META.rec_bytes, mean*1000,
+                          META.rec_bytes/mean/1024,
+                          mean*META.fps*100, META.fps))
+        else
+          -- What a paced run CAN say about the transfer, and it says it from
+          -- the display rather than from the clock: the window is open for
+          -- exactly as long as the record takes, so the open fraction times the
+          -- slot is the transfer time, sampled at the host's frame rate.
+          P(string.format("            -> PACED, so this mean is the PACE and "
+                          .."NOT the transport. What the run does bound is the "
+                          .."transfer: the window was open %.1f%% of a "
+                          .."%.2f ms slot = %.1f ms a record.",
+                          open*100/#r20seen, mean*1000,
+                          open/#r20seen*mean*1000))
+        end
+      end
+      if CSV then
+        local f = io.open(CSV, "w")
+        f:write("frame,t_s\n")
+        for _,a in ipairs(arrive) do f:write(string.format("%d,%.6f\n", a.n-1, a.t)) end
+        f:close()
+        P("arrivals -> "..CSV)
+      end
+      do
+        local f = assert(io.open("packed_snaps.csv", "w"))
+        f:write("snapshot,frame\n")
+        for i, fr in ipairs(snapfr) do
+          f:write(string.format("%04d,%d\n", i-1, fr))
+        end
+        f:close()
+        P(string.format("%d snapshots -> tmp/packed_snaps.csv", #snapfr))
+      end
+      P("done")
+      M:exit()
+    end
+  end)
+  if not ok then print("[PK] LUA ERROR: "..tostring(err)); M:exit() end
+end)

tools/bench/packed_run.sh

@@ -0,0 +1,290 @@
+#!/bin/bash
+# THE PACKED PLAYER, END TO END, OFF A REAL VOLUME.  ROADMAP K3.
+#
+#   tools/bench/packed_run.sh [container.dlxp]
+#
+# Four runs of src/player/packed.s, and each answers a different question.  They
+# are separate runs because the questions interfere: the write window has to be
+# OPEN for the whole transfer and buffer mode blanks the graphics layer, so at
+# the container's own 12 fps there is no instant at which a complete frame is
+# displayable and the pixel gate has nothing to sample.  Pacing slower opens a
+# display interval without changing one byte of the transfer.  Reporting the
+# rate off the gate run instead would have been the flattering shortcut, and it
+# reports the PACE rather than the disc -- the first cut of packed.lua did
+# exactly that and printed 297 KB/s off a 6 fps run.
+#
+#   1. GATE, stealing, paced at half rate: 120 records, 120 snapshots, every one
+#      compared. A packed frame is a LITERAL, so unlike the codec's gate the last
+#      frame audits nothing and all 120 have to be checked (verify_packed.py).
+#   2. RATE, stealing, FREE-RUNNING: the loop is transfer-bound, so the mean
+#      inter-frame IS the emulated transport's time for a record.
+#   3. RATE, held, FREE-RUNNING: the same, with the bus held.
+#   4. CLOCK, held, paced at the container's fps: what holding the bus does to a
+#      frame clock built on counting V-DISP interrupts.
+#
+# DLX_PK_GATE_ONLY=1 runs 1 alone.  That is what tools/bench/check.sh takes: the
+# green light's job is to catch a regression in the PLAYER, and runs 2-4 measure
+# the apparatus rather than gate it -- three more MAME jobs for numbers that
+# cannot change unless MAME does.
+#
+# THE APPARATUS is tools/bench/dma_run.sh's -- `x68000 -exp1 cz6bs1` and a
+# zero-filled scsiexrom.bin on a private rompath -- and the volume is
+# tools/bench/mkvol.sh's, which for a DLXP container is the container itself.
+#
+# WHAT NO RUN HERE MEASURES: `W`, and any rate a real medium would deliver.
+# MAME's device models carry no transfer timing (docs/BENCHMARK.md, 42.5). What
+# is measured is the SHAPE -- one channel start, 193 destinations, 120 times,
+# on a clock the machine keeps itself, with every frame pixel-exact.
+set -e
+cd "$(dirname "$0")/../.."
+DLXP=${1:-tmp/packed_singe.dlxp}
+NFR=${DLX_PK_NFR:-120}
+GATE_FPS=${DLX_PK_GATE_FPS:-6}
+
+bash tools/bench/mkvol.sh "$DLXP"
+python3 tools/bench/prep_packed.py "$DLXP"
+# WHICH ORDER THIS CONTAINER USES, read out of the container rather than
+# assumed.  FINDINGS 62.5/63.4 priced palette-first and palette-last at -12.8 dB
+# for one paint apiece and could not choose between them, so the format records
+# it (dlxp.py flags bit 1) and BOTH have to pass this gate.  The chain assertion
+# below is the only place the difference is visible from outside the machine,
+# and hard-coding either order there would turn "K3 ran both" into "K3 ran one
+# and the other could not have failed".
+PALLAST=$(sed -n 's/.*palette_last = \([01]\),.*/\1/p' tmp/packed_meta.lua)
+if [ "$PALLAST" = "1" ]; then
+  CHAIN0='chain\[0\] = MAR \$C08000 MTC 256'
+  ORDER="palette LAST -- the 193rd entry"
+else
+  CHAIN0='chain\[0\] = MAR \$E82000 MTC 512 ; chain\[1\] = MAR \$C08000 MTC 256'
+  ORDER="palette FIRST -- entry 0, then 192 rows"
+fi
+echo "  container order: $ORDER"
+tools/vasm/vasmm68k_mot -Fbin -o tmp/packed.bin src/player/packed.s > /dev/null
+
+# One run.  $1 names the log, the rest are environment.
+run() {
+  local tag=$1; shift
+  rm -rf "tmp/snap_packed_$tag"; mkdir -p "tmp/snap_packed_$tag"
+  # stdbuf -oL: without it a long MAME run is unobservable until it exits, and a
+  # run that is merely finishing looks exactly like one that is wedged (34.1).
+  ( cd tmp && env SDL_VIDEODRIVER=dummy "$@" stdbuf -oL timeout -k 5 900 \
+      mame x68000 -bios ipl10 -exp1 cz6bs1 \
+      -rompath "$HOME/mame/roms;./p4roms" -hard dlxpdisk.chd \
+      -ramsize 2M -video soft -window $SOUNDARGS -nothrottle -plugins \
+      -autoboot_script ../tools/bench/packed.lua \
+      -snapshot_directory "./snap_packed_$tag" -snapview native \
+      -seconds_to_run "$SECS" > "packed_$tag.log" 2>&1 )
+  grep -aq "^\[PK\] done" "tmp/packed_$tag.log" || {
+    echo "FAIL: the $tag run did not finish -- no completion marker."
+    tail -12 "tmp/packed_$tag.log"; exit 1; }
+  grep -a "^\[PK\]" "tmp/packed_$tag.log" | sed 's/^\[PK\] /  /'
+}
+fail() { echo "FAIL: $1"; exit 1; }
+# -sound none for every run that is not about sound, which is all of them until
+# run 5.  15,625 is not a preference there: it is the chip's own stream rate
+# (8 MHz / 512), and equal rates are what keep MAME's resampler from filtering
+# the thing being measured (FINDINGS 66, adpcm_run.sh).
+SOUNDARGS="-sound none"
+
+echo "--- 1. THE GATE: $NFR records, paced at $GATE_FPS fps, channel stealing ---"
+SECS=$(( NFR / GATE_FPS + 25 ))
+run gate DLX_PK_HELD=0 DLX_PK_PACE=1 DLX_PK_FPS=$GATE_FPS DLX_PK_NFR=$NFR
+cp tmp/packed_snaps.csv tmp/packed_snaps_gate.csv
+
+# THE ASSERTIONS.  Printing a result and gating on it are different things.
+grep -aq "^\[PK\] FLAG=\$FF" tmp/packed_gate.log || \
+  fail "the player did not reach the end of the scene. FLAG=\$E1 is a CRTC mode
+      the frame clock cannot divide, \$E2 is a transport failure -- and the
+      TRANSPORT FAILED line above names which."
+grep -aq "array: the 68000 built 193 entries (the container wants 193)" \
+  tmp/packed_gate.log || \
+  fail "the 68000 built a chain of the wrong length. One entry short delivers a
+      picture with its last row missing, which looks like a decode bug and is a
+      layout bug; the container's geometry and the player's arithmetic are two
+      independent statements of one number and they have to agree."
+grep -aq "$CHAIN0" tmp/packed_gate.log || \
+  fail "the chain does not have the shape this container asks for ($ORDER).
+      The crossing from the palette registers into GVRAM IS the packed frame
+      (FINDINGS 62) -- a palette entry and 192 row entries, one start, the CPU
+      halted throughout -- and an array built the other way round from the
+      record feeding it does not fail: it paints 192 rows of picture into the
+      palette registers and 512 B of palette across the top of the screen."
+grep -aq "late frames (tick already past at the gate): 0$" tmp/packed_gate.log || \
+  fail "a frame missed its slot in the GATE run, which is paced at half rate on
+      purpose. That is not a rate result -- it means the transfer did not fit in
+      a slot twice as long as the container's, and the pixel comparison below is
+      then sampling frames the player was still overwriting."
+
+grep -aq "0 frames NOT SAMPLED" tmp/packed_gate.log || \
+  fail "the gate run could not sample every frame: the write window reopened
+      before the snapshot on at least one. At half the container's rate the
+      transfer must fit inside the display interval with room to spare, and if
+      it does not the comparison below is checking frames the player was still
+      overwriting."
+
+python3 tools/bench/verify_packed.py "$DLXP" --snap tmp/snap_packed_gate \
+        --map tmp/packed_snaps_gate.csv --min-frames "$NFR"
+
+# ---------------------------------------------------------------------------
+# 5. THE AUDIO.  ROADMAP P6c: the container's own bytes, out of channel 3,
+# beside the video channel.  Paced at the CONTAINER's rate rather than the
+# gate's half rate, because the audio was cut at 12 fps and a 6 fps run would
+# starve the chip for half of every group -- the picture can be slowed down and
+# a crystal cannot.  Nothing is snapshotted; the instrument is the WAV, and
+# tools/bench/verify_packed_audio.py accounts for every byte of the stream in
+# it.  The tag is `aud` and not `audio` because tmp/packed_audio.log is
+# tools/analysis/34_packed_audio.py's, in check.sh.
+#
+# CYCLE STEALING, and that is a result rather than a setting -- run 6 below is
+# the same run with the bus held and it is a CONTRAST, not a gate.
+AUDIO_ON=$(sed -n 's/.*has_audio = \([01]\),.*/\1/p' tmp/packed_meta.lua)
+CFPS=$(sed -n 's/^  fps = \([0-9]*\),.*/\1/p' tmp/packed_meta.lua)
+if [ "$AUDIO_ON" = "1" ]; then
+  echo
+  echo "--- 5. THE AUDIO: the container's own lumps, out of channel 3, while"
+  echo "       the video channel is on the same bus (ROADMAP P6c) ---"
+  SECS=$(( NFR / 8 + 30 ))
+  SOUNDARGS="-samplerate 15625 -wavwrite packed_aud.wav"
+  run aud DLX_PK_HELD=0 DLX_PK_PACE=1 DLX_PK_FPS="$CFPS" DLX_PK_NFR=$NFR \
+          DLX_PK_AUD=1 DLX_PK_AJSON=packed_aud.json
+  SOUNDARGS="-sound none"
+  grep -aq "^\[PK\] FLAG=\$FF" tmp/packed_aud.log || \
+    fail "the audio run did not reach the end of the scene."
+  python3 tools/bench/verify_packed_audio.py "$DLXP" tmp/packed_aud.wav \
+          tmp/packed_aud.json || \
+    fail "the chip did not play the container. The counters above can all be
+      right while this fails -- nothing parses a packed container, so a lump
+      fetched into a buffer that is still being read is not an error, it is a
+      sound (FINDINGS 67.4, and it is the bug session 36 shipped and caught)."
+fi
+
+# ---------------------------------------------------------------------------
+# 7. THE SEEK, WITH SOUND ON IT.  FINDINGS 70.3 asked for this and named what
+# was missing: "src/player/packed.s starts PG_AK/PG_AKF at lump 0 and has no
+# audio seek path at all".  This is that path, run.
+#
+# THE FRAME IS CHOSEN NOT TO BE A MULTIPLE OF THE CADENCE, and that is the whole
+# design of the run.  A DLXP2 group is `lump k, then F records`, so a branch
+# that lands on a group boundary needs no offset and no second read -- and 36 of
+# the arcade's 409 within-container seek targets do land on one.  A run that
+# picked one of those would exercise the arithmetic that was already there and
+# report success.  DLX_PK_SEEK defaults below to a frame `f mod F != 0`, so the
+# byte offset inside the lump is load-bearing: get it wrong and the chip is fed
+# a stream that starts up to F frames early, which is not an error, it is a
+# rate, and only verify_packed_audio.py's spliced walk can see it.
+#
+# AND THE SECOND RESULT IS ONE NO COUNTER CAN REACH: the chip's predictor does
+# not seek.  Every byte can arrive, in order, exactly -- and the samples still
+# be wrong, because the encoder chose them for a state a continuous play would
+# have been in.  The verifier measures that against its own control.
+if [ "$AUDIO_ON" = "1" ] && [ "${DLX_PK_NOSEEK:-0}" != "1" ]; then
+  SEEKF=${DLX_PK_SEEK:-37}
+  CADF=$(sed -n 's/^  cad_f = \([0-9]*\),.*/\1/p' tmp/packed_meta.lua)
+  [ $((SEEKF % CADF)) -ne 0 ] || \
+    fail "the seek frame $SEEKF is a multiple of the cadence $CADF, so it lands
+      ON a group boundary -- the one case that needs no byte offset and would
+      pass with the offset arithmetic deleted (FINDINGS 70.3)."
+  echo
+  echo "--- 7. THE SEEK: two passes, the second starting at frame $SEEKF --"
+  echo "       lump $((SEEKF / CADF)), $((SEEKF % CADF)) frame(s) into its group"
+  echo "       of $CADF (FINDINGS 70.3/71) ---"
+  TOTFR=$(( NFR + NFR - SEEKF ))
+  SECS=$(( TOTFR / 8 + 35 ))
+  # BOTH CONFIGURATIONS, and they are not two speeds of one thing.  The chip's
+  # accumulator is an integrator with no leak, so what a branch costs is set by
+  # the state it lands in: play THROUGH and the chip keeps whatever the previous
+  # scene left it in; STOP and re-PLAY and it goes to the container's own `init`
+  # with the step index at 0.  Neither is zero and they are 5.5x apart, so the
+  # run measures both and FINDINGS 71.3 chooses.
+  for M in 0 1; do
+    TAG=seek; [ "$M" = 1 ] && TAG=seek_rst
+    echo "    -- the chip $([ "$M" = 1 ] && echo 'STOPPED and re-PLAYED' \
+                                       || echo 'PLAYING THROUGH') the branch"
+    SOUNDARGS="-samplerate 15625 -wavwrite packed_$TAG.wav"
+    run $TAG DLX_PK_HELD=0 DLX_PK_PACE=1 DLX_PK_FPS="$CFPS" DLX_PK_NFR=$NFR \
+             DLX_PK_AUD=1 DLX_PK_AJSON=packed_$TAG.json \
+             DLX_PK_ITER=2 DLX_PK_SEEK=$SEEKF DLX_PK_ARST=$M
+    SOUNDARGS="-sound none"
+    grep -aq "^\[PK\] FLAG=\$FF" "tmp/packed_$TAG.log" || \
+      fail "the $TAG run did not reach the end of the second pass."
+    python3 tools/bench/verify_packed_audio.py "$DLXP" "tmp/packed_$TAG.wav" \
+            "tmp/packed_$TAG.json" --seek "$SEEKF" --iters 2 || \
+      fail "the player did not play the container ACROSS A BRANCH ($TAG).
+      Nothing here parses anything (FINDINGS 67.4): a seek that fetched the
+      wrong lump plays 7,168 B of the wrong part of the scene, and one that
+      dropped the byte offset plays the right lump from up to $((CADF-1))
+      frames too early -- neither is an error and both are a sound."
+  done
+fi
+
+if [ "${DLX_PK_GATE_ONLY:-0}" = "1" ]; then exit 0; fi
+
+echo
+echo "--- 2/3. THE RATE: free-running, both channel configurations ---"
+SECS=$(( NFR / 8 + 25 ))
+run free_steal DLX_PK_HELD=0 DLX_PK_PACE=0 DLX_PK_NFR=$NFR
+run free_held  DLX_PK_HELD=1 DLX_PK_PACE=0 DLX_PK_NFR=$NFR
+for t in free_steal free_held; do
+  grep -aq "FREE-RUNNING, so the mean IS the transport" "tmp/packed_$t.log" || \
+    fail "the $t run did not report a transport time, so there is no rate here
+        to read -- and a rate read off a PACED run is the pace."
+done
+
+echo
+echo "--- 4. THE CLOCK: held, paced at the container's own rate ---"
+SECS=$(( NFR / 12 + 25 ))
+run held_paced DLX_PK_HELD=1 DLX_PK_PACE=1 DLX_PK_NFR=$NFR
+# THE FINDING THIS RUN EXISTS FOR, asserted rather than admired.  A held channel
+# halts the 68000, and the frame clock is an INTERRUPT off V-DISP whose pending
+# bit is ONE BIT -- so every edge that falls inside a transfer spanning two of
+# them is an edge the machine can never count.  If this ever comes back at zero,
+# either the transfer got short enough to fit between two rasters or the held
+# configuration stopped halting the CPU, and both change what the run means.
+LOST=$(sed -n 's/.*-> \([0-9]*\) edges LOST.*/\1/p' tmp/packed_held_paced.log | head -1)
+[ -n "$LOST" ] && [ "$LOST" -gt 0 ] || \
+  fail "the held run lost no V-DISP edges (${LOST:-none}). Either the bus is no
+      longer being held for the transfer, or the transfer now fits between two
+      rasters -- and the comparison with the stealing run below is then a
+      comparison of two configurations that do the same thing."
+LOSTS=$(sed -n 's/.*-> \([0-9]*\) edges LOST.*/\1/p' tmp/packed_gate.log | head -1)
+
+# ---------------------------------------------------------------------------
+# 6. THE SAME AUDIO WITH THE BUS HELD, and this is the interaction ROADMAP P6c
+# said neither half had met.  A burst channel HALTS the 68000 for the whole
+# 88 ms record, so the audio service cannot run during it -- src/player/dma.i's
+# DM_HOOK is never reached and the chip is looked at twice a frame instead of
+# two thousand times.  The bytes are the same bytes either way; what changes is
+# WHEN the next lump is armed, and the chip has no starvation state: what it
+# does in between is replay the byte pair it is holding.
+#
+# NOT A GATE.  Both configurations play the container byte for byte and the
+# verifier passes on both; the difference is entirely in the seams, and a seam
+# is a design cost rather than a correctness one.
+if [ "$AUDIO_ON" = "1" ]; then
+  echo
+  echo "--- 6. THE AUDIO AGAIN, WITH THE BUS HELD (the contrast, not a gate) ---"
+  SECS=$(( NFR / 6 + 30 ))
+  SOUNDARGS="-samplerate 15625 -wavwrite packed_aud_held.wav"
+  run aud_held DLX_PK_HELD=1 DLX_PK_PACE=1 DLX_PK_FPS="$CFPS" DLX_PK_NFR=$NFR \
+               DLX_PK_AUD=1 DLX_PK_AJSON=packed_aud_held.json
+  SOUNDARGS="-sound none"
+  python3 tools/bench/verify_packed_audio.py "$DLXP" tmp/packed_aud_held.wav \
+          tmp/packed_aud_held.json || fail "the held run did not play the
+      container. The bytes are not what holding the bus was expected to cost."
+  echo
+  echo "  THE SEAM, STEALING AGAINST HELD -- audio does not merely cost clocks:"
+  for t in aud aud_held; do
+    printf '    %-9s ' "$t"
+    python3 tools/bench/verify_packed_audio.py "$DLXP" "tmp/packed_$t.wav" \
+      | sed -n 's/^     worst \(.*\)$/\1/p' | head -1
+  done
+  echo "  Stealing, the 68000 sees the channel from inside dma.i's transfer"
+  echo "  wait. Held, it is halted for the whole record and cannot look at all."
+fi
+
+echo
+echo "  V-DISP edges lost: $LOST held at 12 fps, $LOSTS stealing at $GATE_FPS fps."
+echo "  A player keeps a clock, reads a stick and feeds ADPCM. Which of the two"
+echo "  configurations can do any of that is a DESIGN question, and it is the"
+echo "  one this run answers; neither figure is W."
+exit 0

tools/bench/prep_adpcm.py

@@ -0,0 +1,141 @@
+#!/usr/bin/env python3
+"""Build the nibble stream that asks the MSM6258 which decoder it is.
+ROADMAP P6a.
+
+WHAT HAS TO BE DISCRIMINATED, and it is four things rather than the one
+FINDINGS 65 named:
+
+  1. DELTA FORMULA -- 'shift' (ffmpeg's adpcm_ima_oki) against 'terms' (the
+     datasheet's per-term truncation).  Worth 25 dB (65.2).
+  2. NIBBLE ORDER  -- which half of a byte handed to the data register is played
+     FIRST.  65.1 measured 'high' AGAINST FFMPEG, which is a fact about the VOX
+     file convention and not about a chip's data register.
+  3. THE CLAMP     -- the accumulator saturates somewhere, and where is inside
+     the recursion, so it is not an output scaling that can be undone.
+  4. THE INITIAL ACCUMULATOR at the instant of PLAY.
+
+The stream is in three parts and each part exists for a reason:
+
+  PROLOGUE, 16 zero nibbles.  Nibble 0 moves the step index DOWN, so it stays
+  pinned at 0 and the delta is a constant +2 under every candidate.  That makes
+  the prologue a RAMP that both formulas agree on, which is what absorbs the one
+  thing this rig cannot control: how many times the chip consumes byte 0 before
+  the channel delivers byte 1.  The verifier reads that count off the capture
+  instead of assuming it.
+
+  SEGMENT A, a quiet sine, encoded by tools/encoder/adpcm.py itself.  Amplitude
+  300 keeps it clear of even the 10-bit clamp, so A discriminates the FORMULA
+  and the ORDER without the clamp confounding either.  Using the shipping
+  encoder rather than a hand-written pattern is deliberate: the nibbles the chip
+  is asked about are the kind of nibbles it will be sent.
+
+  SEGMENT B, loud bursts.  It exists ONLY to cross the 10-bit clamp, which
+  segment A is built never to reach, and it is last because a clamp is
+  irreversible state and everything after it would be measuring segment B.
+"""
+import json, math, os, sys
+sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "encoder"))
+import adpcm
+
+PRO_NIB   = 16          # prologue nibbles (byte 0 = $00, so a repeat costs nothing)
+A_SAMPLES = 1500        # segment A, one nibble each
+A_AMP     = 300         # clear of the 10-bit clamp at 511 with room for the ramp
+A_HZ      = 61.0        # ~256 samples a cycle at 15,625 Hz: many step indices
+RATE      = 15625.0
+BUF       = 0x30000     # where the harness pushes the bytes
+OUT_BIN   = "tmp/adpcm_data.bin"
+OUT_META  = "tmp/adpcm_meta.lua"
+OUT_SEQ   = "tmp/adpcm_seq.json"
+
+
+# THE TRIGGER, and it is here because the first cut of this file did not have
+# one and measured ONE differing sample in 1,676.  The two formulas are
+# IDENTICAL whenever the step value is a multiple of 8:
+#
+#     terms - shift  =  b2*floor(r/2) + b3*floor(r/4) - floor((4*b2+2*b3+1)*r/8)
+#
+# with r = step mod 8 and (b1,b2,b3) the nibble's low three bits.  It is zero
+# for r = 0, and the step table STARTS at 16.  A quiet signal never moves the
+# step index off its floor, so a probe made of quiet nibbles asks the chip a
+# question that has the same answer either way.
+#
+#   nibble 4 at step 16: delta 18 under both, and it moves the index to 2
+#   nibble 3 at step 19: shift 16, terms 15  <- the two states part company
+#
+# After that they never rejoin, because the delta is added to a running
+# predictor -- so ONE two-nibble trigger converts the rest of the stream into
+# discriminating evidence.  That is the same recursion 65.2 priced at 25 dB,
+# used deliberately instead of suffered.
+TRIGGER = [4, 3]
+
+
+def segment_a():
+    """The trigger, then a sine encoded by the shipping encoder.  The model the
+    sine is encoded under does not matter for discrimination -- once the trigger
+    has parted the two states, any nibble stream keeps them apart -- so the
+    defaults are used and the choice is recorded rather than tuned.  Using the
+    shipping encoder rather than a hand-written pattern is the point: the
+    nibbles the chip is asked about are the kind of nibbles it will be sent."""
+    sig = [int(round(A_AMP * math.sin(2 * math.pi * A_HZ * i / RATE)))
+           for i in range(A_SAMPLES)]
+    return TRIGGER + list(adpcm.encode(sig, "shift"))
+
+
+def segment_b():
+    """Loud, and alternating in sign so the step index does not simply pin: 40
+    up, 40 down, twice.  Under a 10-bit accumulator this saturates; under a
+    12-bit one it does not, and that difference is the whole point of it."""
+    return ([7] * 40 + [15] * 40) * 2
+
+
+def main():
+    core = segment_a() + segment_b()
+    nibs = [0] * PRO_NIB + core
+    data = adpcm.pack(nibs, "high")     # HIGH first: the encoder's convention,
+                                        # which is one of the things on trial
+    os.makedirs("tmp", exist_ok=True)
+    open(OUT_BIN, "wb").write(data)
+
+    # HOW MUCH DISCRIMINATING POWER IS IN IT, counted rather than asserted.  A
+    # probe that cannot separate two candidates reports a match against both and
+    # a gate that did not count this would call that a result.
+    ref = adpcm.decode(nibs, "shift", init=-2, bits=10)
+    axes = {}
+    for name, kw in (("formula", dict(variant="terms")),
+                     ("order",   dict(order="low")),
+                     ("clamp",   dict(bits=12)),
+                     ("init",    dict(init=0))):
+        order = kw.pop("order", "high")
+        n2 = ([0] * PRO_NIB
+              + list(adpcm.unpack(data, len(nibs), order))[PRO_NIB:]) \
+             if order != "high" else nibs
+        n2 = list(adpcm.unpack(data, len(nibs), order))
+        alt = adpcm.decode(n2, kw.get("variant", "shift"),
+                           init=kw.get("init", -2), bits=kw.get("bits", 10))
+        d = sum(1 for a, b in zip(ref, alt) if a != b)
+        axes[name] = d
+
+    seq = {"nibbles": nibs, "core": core, "pro": PRO_NIB,
+           "bytes": len(data), "buf": BUF, "axes": axes,
+           "a_samples": A_SAMPLES, "a_amp": A_AMP, "a_hz": A_HZ}
+    json.dump(seq, open(OUT_SEQ, "w"))
+
+    with open(OUT_META, "w") as f:
+        f.write("return {\n")
+        f.write(f"  buf = 0x{BUF:X},\n")
+        f.write(f"  nbytes = {len(data)},\n")
+        f.write(f"  nnibs = {len(nibs)},\n")
+        f.write("}\n")
+
+    print(f"  probe stream: {len(nibs)} nibbles = {len(data)} B "
+          f"= {len(nibs)/RATE*1000:.1f} ms at 15,625 Hz")
+    print(f"    prologue {PRO_NIB} zero nibbles, segment A {len(segment_a())} "
+          f"(sine {A_AMP} @ {A_HZ} Hz), segment B {len(segment_b())} (loud)")
+    print("  DISCRIMINATING POWER -- samples that change when ONE axis is "
+          "flipped away from MAME's own model:")
+    for k, v in axes.items():
+        print(f"    {k:8s} {v:5d} of {len(ref)}")
+
+
+if __name__ == "__main__":
+    main()

tools/bench/prep_load.py

@@ -0,0 +1,59 @@
+#!/usr/bin/env python3
+"""Lay out the LOAD-TIME test: raw container header in, expected results out.
+
+    python3 tools/bench/prep_load.py <in.dlx> [--out tmp/load]
+
+src/player/load.i does on the 68000 what tools/bench/dlxload.py has been doing
+host-side since session 1: expand the two codebooks to word-per-pixel form and
+pack the 24-bit palette into GGGGGRRRRRBBBBBI with the shared LSB chosen per
+entry (ROADMAP P1 and P2).  This writes both halves of that comparison.
+
+  <out>_data.bin    the container's HEADER REGION, byte for byte as it comes
+                    off the disc: magic, geometry, the three section offsets,
+                    the 768-byte palette, CB1 and CB4.  Nothing is pre-chewed --
+                    that is the entire point.  It ends where the frame stream
+                    begins, so it is also exactly what a player would have to
+                    read before it could draw anything.
+  <out>_expect.bin  what dlxload.py says the 68000 must produce: expanded CB1,
+                    expanded CB4, then 256 big-endian palette words.
+  <out>_meta.lua    sizes, k1/k4, and the expected darkest-entry index.
+
+The expectation is generated by the SAME module the two decode rigs load
+through, so this cannot pass by agreeing with a second copy of the maths.
+"""
+import sys, argparse
+sys.path.insert(0, "tools/encoder")
+sys.path.insert(0, "tools/bench")
+from dlx import DLX
+import dlxload as DL
+
+ap = argparse.ArgumentParser()
+ap.add_argument("container")
+ap.add_argument("--out", default="tmp/load")
+a = ap.parse_args()
+
+d = DLX(a.container)
+if d.version < 3:
+    sys.exit(f"{a.container} is DLX{d.version}: load.i wants DLX3 or DLX4")
+if d.idx_bytes != 1:
+    sys.exit("2-byte codebook indices: load.i expands one source byte per pixel")
+
+off_frm = int.from_bytes(d.raw[28:32], "big")
+raw = d.raw[:off_frm]
+
+cb1, cb4 = DL.expand_codebooks(d)
+palb, dark, _ = DL.pack_palette(d)
+
+open(a.out + "_data.bin", "wb").write(raw)
+open(a.out + "_expect.bin", "wb").write(cb1.tobytes() + cb4.tobytes() + palb.tobytes())
+
+with open(a.out + "_meta.lua", "w") as fh:
+    fh.write("-- generated by tools/bench/prep_load.py -- do not edit\nreturn {\n")
+    fh.write(f"  k1={d.k1}, k4={d.k4}, dark={dark},\n")
+    fh.write(f"  raw_len={len(raw)}, cb1_len={cb1.nbytes}, cb4_len={cb4.nbytes},\n")
+    fh.write(f"  pal_len={palb.nbytes},\n}}\n")
+
+print(f"{a.container}: k1={d.k1} k4={d.k4}, header region {len(raw)} B "
+      f"(pal 768 + cb1 {d.k1*16} + cb4 {d.k4*4} + 32)")
+print(f"  the 68000 must produce {cb1.nbytes} + {cb4.nbytes} B of expanded "
+      f"codebook and {palb.nbytes} B of palette, darkest entry {dark}")

tools/bench/prep_packed.py

@@ -0,0 +1,79 @@
+#!/usr/bin/env python3
+"""Everything tools/bench/packed.lua needs to know about a DLXP container.
+
+    python3 tools/bench/prep_packed.py <in.dlxp>   -> tmp/packed_meta.lua
+
+THERE IS NO BLOB TO PREPARE, and that is the whole difference from
+`prep_stream.py`.  The codec's rig has to hand the machine expanded codebooks, a
+packed palette and a record index, because a DLX record cannot be found or drawn
+without them; `prep_dlx.py` and `prep_stream.py` exist for that and FINDINGS
+49.7.5 records what it cost to have two copies of one of those transforms.  A
+packed container carries no such thing: record `i` is at sector 1 + i*97 by
+geometry and its bytes are already in the order GVRAM wants them (dlxp.py).  So
+this file emits METADATA ONLY -- six numbers the rig would otherwise have to
+hard-code, every one of them read out of the container's own header.
+
+The volume is the container itself; tools/bench/mkvol.sh copies it.
+"""
+import os, sys
+sys.path.insert(0, os.path.join(os.path.dirname(os.path.abspath(__file__)),
+                                "..", "encoder"))
+from dlxp import DLXP, SECTOR
+
+if len(sys.argv) != 2:
+    sys.exit(__doc__)
+d = DLXP(sys.argv[1])
+
+# The record's sector count, and the array chain's entry count, DERIVED here and
+# asserted by the 68000 (PG_ARRN).  Two independent statements of one geometry
+# is the only way a container and a player can be caught disagreeing about it --
+# a chain one entry short delivers a picture with its last row missing, which
+# looks like a decode bug and is a layout bug.
+recs = d.rec_bytes // SECTOR
+rows = d.H
+entries = rows + (1 if d.has_palette else 0)
+
+out = "tmp/packed_meta.lua"
+with open(out, "w") as fh:
+    fh.write("-- generated by tools/bench/prep_packed.py; do not edit\n")
+    fh.write("return {\n")
+    for k, v in [("W", d.W), ("H", d.H), ("fps", d.fps), ("nframes", d.nframes),
+                 ("rec_bytes", d.rec_bytes), ("rec_sectors", recs),
+                 ("pal_bytes", d.pal_bytes), ("pic_bytes", d.pic_bytes),
+                 ("lba0", d.off_frm // SECTOR),
+                 ("palette_last", int(d.palette_last)),
+                 ("has_palette", int(d.has_palette)),
+                 # DLXP2. Zero in a silent container, and the player branches on
+                 # the zero rather than being built two ways.
+                 ("cad_f", d.cad_f), ("cad_a", d.cad_a),
+                 ("has_audio", int(d.has_audio)),
+                 ("aud_bytes", d.aud_bytes), ("aud_hz", d.aud_hz),
+                 ("n_lumps", d.n_lumps),
+                 # ROADMAP P6c. lba_aud is off_aud/512 and is a HEADER field
+                 # rather than the constant 1 it happens to equal: a shipping
+                 # volume has a filesystem in front of the stream and the
+                 # player adds PG_LBA0's base to neither of them by accident.
+                 ("lba_aud", d.off_aud // SECTOR if d.has_audio else 0),
+                 # the four axes, so the rig can print what the container says
+                 # it was encoded for and the verifier can decode with it
+                 ("aud_variant", '"%s"' % d.decoder()["variant"]
+                  if d.has_audio else '""'),
+                 ("aud_order", '"%s"' % d.decoder()["order"]
+                  if d.has_audio else '""'),
+                 ("aud_bits", d.aud_bits), ("aud_init", d.aud_init),
+                 ("entries", entries)]:
+        fh.write(f"  {k} = {v},\n")
+    fh.write("}\n")
+
+print(f"{sys.argv[1]}: DLXP{d.version} {d.W}x{d.H} {d.fps}fps {d.nframes} frames"
+      + (f", AUDIO F={d.cad_f} A={d.cad_a}" if d.has_audio else ", silent"))
+print(f"  record {d.rec_bytes:,} B = {recs} sectors, palette "
+      f"{'LAST' if d.palette_last else 'FIRST'}, {d.pal_bytes} B")
+cad = (f" + (i//{d.cad_f})*{d.cad_a}" if d.has_audio else "")
+print(f"  record i is at LBA {d.off_frm // SECTOR} + i*{recs}{cad} -- ARITHMETIC. "
+      f"There is no index in this container and none can be needed.")
+print(f"  the chain the 68000 must build: {entries} entries "
+      f"({rows} rows{' + 1 palette' if d.has_palette else ''})")
+print(f"  wire {d.video_kbps():.1f}"
+      + (f" + {d.audio_kbps():.2f} = {d.kbps():.1f}" if d.has_audio else "")
+      + f" KB/s, FIXED by geometry -> {out}")

tools/bench/prep_stream.py

@@ -18,6 +18,10 @@ This writes three files instead:
                   filesystem and feeds it into a bounded ring, so the emulated
                   machine's RAM stops bounding how much of a window can be
                   tested.  A stock 2 MB machine can now run all 120 frames.
+  <out>_idx.bin   the DLX4 RECORD INDEX, nframes u16 big-endian, straight out
+                  of the container's scene header.  This is what the 68000
+                  producer reads (src/player/ring.i); it is not derived here,
+                  because deriving it is precisely what a player cannot do.
   <out>_meta.lua  geometry, and the record index.
 
 THE RECORD INDEX IS NOT A CONVENIENCE.  src/player/stream.s takes each frame's
@@ -66,17 +70,42 @@ disk, index = bytearray(), []
 for (o, n) in d.frames:
     start = len(disk)
     disk += n.to_bytes(4, "big") + d.raw[o:o + n]
-    while len(disk) % 4:
+    # The container's own alignment rule, not this script's copy of it: DLX5
+    # pads to 512 so a DMA channel can read whole sectors into the ring, DLX4
+    # to 4 so `move.l (a0)+` does not take an address error (28.3).
+    while len(disk) % d.rec_align:
         disk += b"\0"
     index.append((start, len(disk) - start))
 open(a.out + "_disk.bin", "wb").write(bytes(disk))
 
+# THE CONTAINER'S OWN INDEX, and it is checked against this layout rather than
+# regenerated from it.  src/player/ring.i walks the disk with a running sum of
+# these lengths and never reads a record's length word before fetching it, so a
+# container index that disagreed with the disk image by one byte would place
+# every later record at the wrong address -- and the block loop reads without a
+# bounds check (49.2), so the symptom would be wrong pixels, not a fault.
+if d.has_index:
+    want = [ln // 4 for _, ln in index]
+    if d.index != want:
+        bad = next(i for i in range(len(want)) if d.index[i] != want[i])
+        sys.exit(f"{a.container}: the DLX4 index disagrees with this disk "
+                 f"layout at record {bad}: {d.index[bad]} vs {want[bad]} "
+                 f"longwords")
+    open(a.out + "_idx.bin", "wb").write(
+        b"".join(q.to_bytes(2, "big") for q in d.index))
+else:
+    if os.path.exists(a.out + "_idx.bin"):
+        os.remove(a.out + "_idx.bin")   # a stale index is worse than none
+
 rec = np.array([n for _, n in index])
 with open(a.out + "_meta.lua", "w") as fh:
     fh.write("-- generated by tools/bench/prep_stream.py -- do not edit\nreturn {\n")
     fh.write(f"  W={d.W}, H={d.H}, fps={d.fps}, nframes={d.nframes}, dark={dark},\n")
     fh.write(f"  cb1_len={cb1.nbytes}, cb4_len={cb4.nbytes}, pal_len={palb.nbytes},\n")
     fh.write(f"  disk_len={len(disk)}, maxrec={int(rec.max())},\n")
+    fh.write(f"  dlx_version={d.version}, "
+             f"has_index={'true' if d.has_index else 'false'},\n")
+    fh.write(f"  padrec={{{','.join(str(ln) for _, ln in index)}}},\n")
     fh.write("  index={\n")
     for off, ln in index:
         fh.write(f"    {{off={off}, len={ln}}},\n")
@@ -89,5 +118,8 @@ print(f"  disk image {len(disk):,} B -> {a.out}_disk.bin  "
       f"(records: min {rec.min():,} median {int(np.median(rec)):,} "
       f"max {rec.max():,})")
 print(f"  wire rate {rec.mean()*d.fps/1024:.1f} KB/s video at {d.fps} fps")
+if d.has_index:
+    print(f"  DLX4 record index: {2*d.nframes:,} B of scene header, and it "
+          f"agrees with the disk image on all {d.nframes} records")
 print(f"  A ring must hold one whole record contiguously: >= {rec.max():,} B "
       f"({rec.max()/1024:.1f} KB) before any policy or prefill.")

tools/bench/probe_adpcm.lua

@@ -0,0 +1,28 @@
+-- Ask MAME what the x68000's ADPCM device is, and where the 68000 reaches it.
+-- FINDINGS 64.4 recorded that no MAME source tree is on this machine; this is
+-- the way to ask the same question without one.
+M = manager.machine
+local done = false
+SUB = emu.add_machine_frame_notifier(function()
+  if done then return end
+  done = true
+  print("[AD] === devices whose tag looks like an ADPCM chip ===")
+  for tag, dev in pairs(M.devices) do
+    local t = tag:lower()
+    if t:find("adpcm") or t:find("oki") or t:find("msm") or t:find("6258") then
+      print(string.format("[AD] DEV %-24s shortname=%s", tag, tostring(dev.shortname)))
+    end
+  end
+  local sp = M.devices[":maincpu"].spaces["program"]
+  print("[AD] === program map, $E90000..$EA0000 ===")
+  local ok, err = pcall(function()
+    for _, e in ipairs(sp.map.entries) do
+      if e.address_start >= 0xE90000 and e.address_start < 0xEA0000 then
+        print(string.format("[AD] MAP %08X-%08X", e.address_start, e.address_end))
+      end
+    end
+  end)
+  if not ok then print("[AD] MAP unavailable: " .. tostring(err)) end
+  print("[AD] done")
+  M:exit()
+end)

tools/bench/probe_adpcm2.lua

@@ -0,0 +1,36 @@
+-- Feed the x68000's OWN okim6258 a known nibble stream and let MAME record what
+-- comes out, so that "which delta formula does the chip use" is a MEASUREMENT
+-- and not a reading of source code that is not on this machine (64.4).
+--
+-- The feed is deliberately SLOW -- a byte every host frame, where real time
+-- wants ~138 -- because the question is not the rate. A starved chip holds its
+-- last sample, so the wave is a STAIRCASE of the reconstructed values, which is
+-- exactly the sequence the two candidate formulas disagree about.
+M = manager.machine
+local sp = M.devices[":maincpu"].spaces["program"]
+local CTRL, DATA = 0xE92001, 0xE92003
+local CMD = tonumber(os.getenv("AD_CMD") or "1")
+-- 12 loud nibbles to climb the step index, then every nibble in turn: the pairs
+-- where the two formulas differ are all at step indices above the floor.
+local nibs = {}
+for i = 1, 12 do nibs[#nibs+1] = 7 end
+for i = 0, 15 do nibs[#nibs+1] = i end
+for i = 0, 15 do nibs[#nibs+1] = i end
+local bytes = {}
+for i = 1, #nibs, 2 do bytes[#bytes+1] = nibs[i] * 16 + nibs[i+1] end
+
+local n, started = 0, false
+SUB = emu.add_machine_frame_notifier(function()
+  n = n + 1
+  if n == 30 then
+    sp:write_u8(CTRL, CMD)
+    started = true
+    print(string.format("[AD] ctrl $%02X written to $%06X", CMD, CTRL))
+  elseif started and n > 30 and (n - 30) <= #bytes then
+    sp:write_u8(DATA, bytes[n - 30])
+  elseif started and (n - 30) == #bytes + 20 then
+    print("[AD] fed " .. #bytes .. " bytes = " .. #nibs .. " nibbles")
+    print("[AD] done")
+    M:exit()
+  end
+end)

tools/bench/probe_adpcm3.lua

@@ -0,0 +1,24 @@
+-- Sweep the PPI's port C -- which is where the X68000 puts ADPCM pan and the
+-- chip's clock divider -- and feed a loud burst under each value, so that the
+-- WAV says which value un-mutes the chip. Nothing here is assumed: the segment
+-- boundaries are printed and the analysis reads the wave against them.
+M = manager.machine
+local sp = M.devices[":maincpu"].spaces["program"]
+local CTRL, DATA, PPIC = 0xE92001, 0xE92003, 0xE9A005
+local SEG = 40            -- host frames per segment
+local n = 0
+SUB = emu.add_machine_frame_notifier(function()
+  n = n + 1
+  if n <= 20 then return end
+  local k = n - 20
+  local seg = math.floor((k - 1) / SEG)
+  local off = (k - 1) % SEG
+  if seg > 15 then print("[AD] done"); M:exit(); return end
+  if off == 0 then
+    sp:write_u8(PPIC, seg)
+    sp:write_u8(CTRL, 1)
+    print(string.format("[AD] seg %2d portC=$%02X starts at host frame %d", seg, seg, n))
+  elseif off <= 30 then
+    sp:write_u8(DATA, 0x77)          -- two loud positive nibbles
+  end
+end)

tools/bench/probe_adpcm4.lua

@@ -0,0 +1,33 @@
+-- SPIKE (session 34).  Not a gate: it exists to find out whether the chip can be
+-- driven at all, what scale its samples arrive at in a -wavwrite capture, and
+-- where it clamps.  It sets EVERY thing session 33's probes left to the IPL:
+--   * YM2151 reg $1B bit1 = 0  -> CT1 = 0 -> ADPCM master clock 8 MHz
+--   * PPI control $92          -> port C is an OUTPUT (without this the i8255's
+--                                 out_pc_callback never fires and the pan and
+--                                 divider writes go nowhere)
+--   * PPI port C  $08          -> pan 00 = BOTH, rate 10 = /512 -> 15,625 Hz
+--   * ctrl $02 = COMMAND_PLAY  -- session 33's probes wrote $01, COMMAND_STOP.
+M = manager.machine
+local sp = M.devices[":maincpu"].spaces["program"]
+local YMA, YMD = 0xE90001, 0xE90003
+local PPIC, PPICTL = 0xE9A005, 0xE9A007
+local CTRL, DATA = 0xE92001, 0xE92003
+local BYTE = tonumber(os.getenv("AD_BYTE") or "0x77")
+local n = 0
+SUB = emu.add_machine_frame_notifier(function()
+  n = n + 1
+  if n == 40 then
+    sp:write_u8(YMA, 0x1B); sp:write_u8(YMD, 0x00)
+  elseif n == 60 then
+    sp:write_u8(YMA, 0x1B); sp:write_u8(YMD, 0x00)
+    sp:write_u8(PPICTL, 0x92)
+    sp:write_u8(PPIC, 0x08)
+    print(string.format("[AD4] portC readback = $%02X", sp:read_u8(PPIC)))
+    sp:write_u8(CTRL, 0x02)
+    sp:write_u8(DATA, BYTE)
+    print(string.format("[AD4] PLAY, data $%02X, status = $%02X",
+                        BYTE, sp:read_u8(CTRL)))
+  elseif n == 90 then
+    print("[AD4] done"); M:exit()
+  end
+end)

tools/bench/scsi.lua

@@ -0,0 +1,170 @@
+-- Drive src/player/scsigate.s: can the 68000 reach the MB89352? (ROADMAP P4)
+--
+-- Bus errors are EXPECTED and are data: the gate probes one address at a time
+-- and survives each fault, so a hole in the decode costs an entry in the map
+-- rather than the rest of the run.  See src/player/scsigate.s.
+--
+-- THE APPARATUS, STATED UP FRONT.  This runs `x68000 -exp1 cz6bs1`, which is
+-- the board FINDINGS 42.5 says to benchmark and never `x68ksupr` (whose
+-- internal SCSI is PIO-only in MAME -- `// TODO: duplicate DMA glue from
+-- CZ-6BS1`).  MAME refuses to instantiate the card without an 8 KB
+-- `scsiexrom.bin`, which is not in this tree; the rig supplies a ZERO-FILLED
+-- placeholder on a SEPARATE rompath so the user's own romset is untouched.
+--
+-- THAT PLACEHOLDER IS HONEST HERE AND WOULD NOT BE EVERYWHERE.  The CZ-6BS1's
+-- boot ROM exists to make the card bootable through IOCS.  This player drives
+-- the SPC registers directly and never executes a byte of it -- that was
+-- already the plan in docs/BENCHMARK.md item 4, long before the ROM was missing
+-- -- so a blank one changes nothing this rig measures.  What it WOULD change is
+-- anything that booted from the card or called SCSI IOCS; do not reuse it for
+-- that.  The run prints the substitution rather than burying it.
+--
+-- WHAT A GREEN RUN MEANS, and what it does not.  It means the 68000 reaches the
+-- SPC and the register map is the one the driver will be written against.  It
+-- says NOTHING about rate: MAME's device models are functional, not
+-- transfer-timing accurate (docs/BENCHMARK.md), and 42.5 shows its DMAC is
+-- configured in wall-clock attotimes rather than per-operand cycles.  `W` is
+-- untouched by anything here.
+local M  = manager.machine
+local SP = M.devices[":maincpu"].spaces["program"]
+local function P(s) print("[SCSI] "..s) end
+local function T() local t=M.time; return t.seconds + t.attoseconds/1e18 end
+
+local SCFLAG, SCN, SCVAL, SCOK, SCTMP, SCTMPOK =
+      0x18080, 64, 0x18100, 0x18140, 0x180D0, 0x180D4
+local SCRD, SCDST, SCBLKS = 0x180D8, 0x20000, 8
+local SC_ERR, SC_STAT, SC_PH = 0x18200, 0x18204, 0x18208
+local ERRNAME = {[0]="OK", "SELECTION TIMEOUT -- no target answered",
+                 "UNEXPECTED PHASE", "POLL TIMEOUT -- a phase never arrived",
+                 "NON-ZERO SCSI STATUS"}
+local DISK = os.getenv("DLX_SCSI_IMG") or "dlxdisk.img"
+
+local code do local f=io.open("scsigate.bin","rb"); code=f:read("a"); f:close() end
+
+local st = "boot"
+SUB = emu.add_machine_frame_notifier(function()
+  local ok, err = pcall(function()
+    if st == "boot" then
+      if T() < 3.0 then return end
+      for i = 1, #code do SP:write_u8(0x10000+i-1, string.byte(code,i)) end
+      SP:write_u32(SCFLAG, 0)
+      local cpu = M.devices[":maincpu"]
+      cpu.state["SR"].value = 0x2700
+      cpu.state["SP"].value = 0x8000
+      cpu.state["PC"].value = 0x10000
+      P(string.format("scsigate.bin=%d B loaded at $10000", #code))
+      st = "wait"; return
+    end
+    if st == "wait" then
+      local fl = SP:read_u32(SCFLAG)
+      if fl ~= 1 then
+        if T() > 30 then P("TIMEOUT: the gate never finished"); P("done"); M:exit() end
+        return
+      end
+      -- THE MAP, address by address.  A dead address is a bus error the
+      -- gate survived, not a run that stopped.
+      local live, dead = {}, {}
+      for i = 0, SCN-1 do
+        if SP:read_u8(SCOK+i) == 1 then live[#live+1] = i else dead[#dead+1] = i end
+      end
+      for row = 0, (SCN/16)-1 do
+        local t = {}
+        for i = 0, 15 do
+          local a = row*16 + i
+          t[#t+1] = (SP:read_u8(SCOK+a) == 1)
+                    and string.format("%02X", SP:read_u8(SCVAL+a)) or "--"
+        end
+        P(string.format("  $EA%04X: %s", row*16, table.concat(t, " ")))
+      end
+      P(string.format("ANSWERED %d of %d addresses; \"--\" is a bus error.",
+                      #live, SCN))
+      -- The lane question, decided by which addresses answer.
+      local odd_live, even_live = 0, 0
+      for _, a in ipairs(live) do
+        if a % 2 == 1 then odd_live = odd_live + 1 else even_live = even_live + 1 end
+      end
+      P(string.format("LANES: %d odd addresses answer, %d even.", odd_live, even_live))
+      if SP:read_u32(SCTMPOK) == 1 then
+        P(string.format("TEMP writeback ($EA0017): wrote $A5, read back $%02X",
+                        SP:read_u32(SCTMP) & 0xFF))
+      else
+        P("TEMP writeback ($EA0017): BUS ERROR -- not a writable register here.")
+      end
+      P(string.format("DREG writeback ($EA0015): wrote $5A, SSTS then $%02X "
+                      .."(FIFO %s), read back $%02X",
+                      SP:read_u32(0x180E0) & 0xFF,
+                      ((SP:read_u32(0x180E0) & 1) == 1) and "EMPTY -- the write "
+                      .."never reached dreg_w" or "has a byte",
+                      SP:read_u32(0x180DC) & 0xFF))
+      -- The trace, whatever happened.
+      local ntr = SP:read_u32(0x18230)
+      if ntr > 0 then
+        P("trace (SSTS PSNS INTS SERR | TCH TCM TCL | where):")
+        for i = 0, ntr-1 do
+          local b = {}
+          for k = 0, 7 do b[k+1] = string.format("%02X", SP:read_u8(0x18240+i*8+k)) end
+          local WH = {[0]="init", "selected", "INTS cleared", "phase loop saw REQ",
+                      "TRANSFER issued (out)", "bytes handed over",
+                      "after xfer wait", "TRANSFER issued (in)",
+                      "bytes taken from FIFO", "after IN xfer wait",
+                      "after bus release"}
+          P(string.format("  %2d: %s %s %s %s | %s %s %s | %s", i,
+            b[1], b[2], b[3], b[4], b[5], b[6], b[7],
+            WH[tonumber(b[8], 16)] or b[8]))
+        end
+      end
+      -- THE READ.  Verified against the host's copy of the same image: a
+      -- transport that returns the wrong bytes without saying so is exactly the
+      -- failure a checksum-free ring cannot survive (49.2).
+      local rd = SP:read_u32(SCRD)
+      local e  = SP:read_u32(0x180E8)
+      if rd ~= 0 or e ~= 0 then
+        P(string.format("scsi_read FAILED: err=%d (%s), status=$%02X, phase=%d",
+                        e, ERRNAME[e] or "?", SP:read_u32(SC_STAT) & 0xFF,
+                        SP:read_u32(SC_PH)))
+      else
+        local f = io.open(DISK, "rb")
+        if not f then
+          P("scsi_read returned OK but "..DISK.." is not here to check it against.")
+        else
+          local want = f:read(SCBLKS * 512); f:close()
+          local bad, first = 0, nil
+          for i = 1, #want do
+            if SP:read_u8(SCDST + i - 1) ~= string.byte(want, i) then
+              bad = bad + 1; first = first or (i-1)
+            end
+          end
+          if bad == 0 then
+            P(string.format("READ(10) OK: %d B from LBA 0 match %s byte for byte.",
+                            #want, DISK))
+          else
+            P(string.format("READ(10) WRONG: %d of %d bytes differ, first at +%d.",
+                            bad, #want, first))
+          end
+        end
+      end
+      -- The second read, at a non-zero LBA.
+      local rd2 = SP:read_u32(0x180E4)
+      if rd2 ~= 0 then
+        P(string.format("second scsi_read FAILED: err=%d (%s)",
+                        SP:read_u32(0x180EC),
+                        ERRNAME[SP:read_u32(0x180EC)] or "?"))
+      else
+        local f = io.open(DISK, "rb")
+        if f then
+          f:seek("set", 1000 * 512)
+          local want = f:read(4 * 512); f:close()
+          local bad = 0
+          for i = 1, #want do
+            if SP:read_u8(0x28000 + i - 1) ~= string.byte(want, i) then bad = bad + 1 end
+          end
+          P(bad == 0
+            and string.format("READ(10) OK: %d B from LBA 1000 match too.", #want)
+            or  string.format("READ(10) WRONG at LBA 1000: %d of %d differ.", bad, #want))
+        end
+      end
+      P("done"); M:exit(); return
+    end
+  end)
+  if not ok then P("LUA ERROR: "..tostring(err)); P("done"); M:exit() end
+end)

tools/bench/scsi_run.sh

@@ -0,0 +1,79 @@
+#!/bin/bash
+# One MB89352 transport run: the 68000 selects a SCSI target and reads the disc
+# itself (ROADMAP P4, first half).
+#
+#   tools/bench/scsi_run.sh [container.dlx]
+#
+# THE APPARATUS, AND ITS TWO SUBSTITUTIONS, BOTH DELIBERATE AND BOTH LABELLED.
+#
+# 1. THE BOARD.  `x68000 -exp1 cz6bs1`, which is what FINDINGS 42.5 says to
+#    benchmark and never `x68ksupr` -- the Super/XVI internal SCSI is PIO-only in
+#    MAME (`// TODO: duplicate DMA glue from CZ-6BS1`), so it would measure a
+#    fallback the real machine does not have.
+#
+# 2. THE BOOT ROM.  MAME refuses to instantiate the card without an 8 KB
+#    `scsiexrom.bin` (CRC 7be488de), which is not in this tree and is not on this
+#    machine.  This script writes a ZERO-FILLED placeholder into its own rompath,
+#    so the user's romset is untouched, and MAME prints WRONG CHECKSUMS as it
+#    should.  That is honest HERE and would not be everywhere: the player drives
+#    the SPC registers directly and never executes a byte of that ROM -- which
+#    was already the plan in docs/BENCHMARK.md item 4, long before the ROM turned
+#    out to be missing.  DO NOT reuse this rompath for anything that boots from
+#    the card or calls SCSI IOCS; those DO execute it.
+#
+# WHAT A GREEN RUN MEANS: the 68000 reached the SPC, selected a target, issued
+# READ(10) twice -- at LBA 0 and at a non-zero LBA -- and both came back byte for
+# byte identical to the host's copy of the same image.  No IOCS, no host in the
+# transfer path.
+#
+# WHAT IT DOES NOT MEAN: anything at all about RATE.  MAME's device models are
+# functional, not transfer-timing accurate (docs/BENCHMARK.md), and 42.5 reads
+# its DMAC configured in wall-clock attotimes rather than per-operand cycles.
+# `W` -- clocks stolen per delivered byte, the project's largest open number --
+# is untouched by this script.
+set -e
+cd "$(dirname "$0")/../.."
+DLX=${1:-tmp/rc_fr_singe_scsi_span.dlx}
+
+# The container's frame records, laid out as a disc, and the blank card ROM.
+# Both are tools/bench/mkvol.sh's, shared with tools/bench/pace_run.sh so the
+# SCSI rig and the host-file ring rig cannot be reading different bytes.
+bash tools/bench/mkvol.sh "$DLX"
+
+tools/vasm/vasmm68k_mot -Fbin -o tmp/scsigate.bin src/player/scsigate.s > /dev/null
+
+# stdbuf -oL: without it a long MAME run is unobservable until it exits, and a
+# run that is merely finishing looks exactly like one that is wedged (34.1).
+( cd tmp && SDL_VIDEODRIVER=dummy stdbuf -oL timeout -k 5 300 \
+    mame x68000 -bios ipl10 -exp1 cz6bs1 \
+    -rompath "$HOME/mame/roms;./p4roms" -hard dlxdisk.chd \
+    -ramsize 2M -video soft -window -sound none -nothrottle -plugins \
+    -autoboot_script ../tools/bench/scsi.lua \
+    -seconds_to_run 60 > scsi_run.log 2>&1 )
+# A run that never reached the report must fail as that, not as a missing line.
+grep -aq "^\[SCSI\] done" tmp/scsi_run.log || {
+  echo "FAIL: the SCSI gate did not finish -- no completion marker."
+  tail -8 tmp/scsi_run.log; exit 1; }
+grep -a "^\[SCSI\]" tmp/scsi_run.log | grep -avE "^\[SCSI\] +[0-9]+:" \
+  | sed 's/^\[SCSI\] /  /'
+
+# THE ASSERTIONS.  Printing a result and gating on it are different things, and
+# this project has already paid once for a stage that printed.
+grep -aq "ANSWERED 60 of 64" tmp/scsi_run.log || {
+  echo "FAIL: the card's register window is no longer 60 of 64 addresses."
+  echo "      The MB89352 omits TMOD (index 3) and EXBF (index 15) where the"
+  echo "      MB87030 has them, and MAME leaves HOLES rather than shifting the"
+  echo "      later indices down -- which is what puts DREG at \$EA0015. If this"
+  echo "      moved, src/player/scsi.i's whole register map moved with it."
+  exit 1; }
+grep -aq "READ(10) OK: 4096 B from LBA 0" tmp/scsi_run.log || {
+  echo "FAIL: the 68000 did not read LBA 0 off the SCSI volume byte-exact."
+  exit 1; }
+grep -aq "READ(10) OK: 2048 B from LBA 1000" tmp/scsi_run.log || {
+  echo "FAIL: the read at a NON-ZERO LBA did not match. A driver that emits a"
+  echo "      malformed LBA field still passes LBA 0, because zero is what a"
+  echo "      malformed field usually is -- so this is the half that matters."
+  exit 1; }
+grep -aq "scsi_read FAILED" tmp/scsi_run.log && {
+  echo "FAIL: a read reported an error."; exit 1; }
+exit 0

tools/bench/span.lua

@@ -107,7 +107,8 @@ end
 local function launch(cfg)
   push(STREAM, blob, cfg.off+1, cfg.len)
   clear_picture()
-  -- ~4 emulated seconds per config: 1/55.46 s granularity costs under 0.5%.
+  -- ~4 emulated seconds per config: 1/56.69 s granularity (crtc_mode.lua)
+  -- costs under 0.5%.
   local est = cfg.nspans*(cfg.var == 5 and 60 or (cfg.var == 6 and 50 or 70))
               + cfg.npix*10
   cfg.iter = math.max(4, math.floor(4*CPUHZ/est))

tools/bench/stream.lua

@@ -52,8 +52,29 @@
 --                    no provenance that was never a bus measurement at all
 --                    (FINDINGS 42.1). A default is how a folklore number ends up
 --                    silently underneath a table nobody restates it in.
---   DLX_PREFILL_KB   bytes to deliver before releasing the CPU (default 0)
---   DLX_PACE         1 = hold the decoder to META.fps (default 0 = free-run)
+--   DLX_PREFILL_KB   bytes to deliver before releasing the CPU (default 0).
+--                    Host-owned ring only: with DLX_RINGOWN the machine does
+--                    its own prefill and DLX_PREFILL_FR is the knob.
+--   DLX_RINGOWN      1 = THE 68000 OWNS THE RING (ROADMAP P5,
+--                    src/player/ring.i).  This script stops placing records and
+--                    becomes a TRANSPORT: it answers one request at a time,
+--                    delivering XF_LEN bytes from XF_OFF to XF_DST in
+--                    len/rate seconds, and the machine decides which record,
+--                    where in the ring, and when.  It also AUDITS every
+--                    placement the machine makes against its own index.
+--   DLX_PREFILL_FR   whole records the machine's prefill waits for before it
+--                    releases the decoder (default 2; see ring_prefill)
+--   DLX_PACE         1 = hold the decoder to META.fps, with the HOST writing
+--                    the tick (default 0 = free-run)
+--                    2 = hold it to META.fps with the 68000 writing its own
+--                    tick, off the CRTC's V-DISP (src/player/clock.i, ROADMAP
+--                    P3).  Same gate, same ring, same deadlines; what changes
+--                    is that nothing outside the machine decides when a frame
+--                    may start.  Deadlines are then taken from the ticks the
+--                    machine actually emitted rather than from a host model of
+--                    12 fps -- which matters, because MAME's raster runs 2.22%
+--                    fast (see tools/bench/clock.lua) and a host model would
+--                    quietly grade every arrival against the wrong clock.
 --   DLX_CUT_AT       frame tick at which the pipe stops dead (a seek).  Needs
 --                    DLX_PACE; unset = no cut.
 --   DLX_CUT_FR       how many frame times the cut lasts (default 1)
@@ -88,7 +109,33 @@ local META = loadfile("stream_meta.lua")()
 local FLAG, ITER, NFR         = 0x18000, 0x18008, 0x1800C
 local RD_PTR, FR_HEAD, FR_TAIL = 0x18020, 0x18024, 0x18028
 local STALLS, SPINS, DESC      = 0x1802C, 0x18030, 0x18100
-local PACE, PACEON             = 0x18034, 0x18038
+local PACE, PACEON, CLKON      = 0x18034, 0x18038, 0x1803C
+local CLK_VDISP, CLK_FPS, CLK_ERR = 0x18064, 0x18068, 0x1806C
+local LATEFR, LATEMAX, LATE1ST = 0x18080, 0x18084, 0x18088
+-- src/player/ring.i.  The transport mailbox, then the producer's state and its
+-- instruments.  Nothing below 0x18300 changed: the decoder's handshake with the
+-- ring (FR_HEAD/FR_TAIL/DESC) is the same one FINDINGS 49 and 51 measured, which
+-- is what makes a self-filled run a test of ring.i and not of a new rig.
+local XF_SLOT, XF_SLSZ         = 0x18300, 16
+local XF_GO, XF_ACK, XF_QD     = 0x18320, 0x18324, 0x18328
+local A_RINGOWN, A_RNG_B, A_RNG_SZ, A_IDX_B = 0x1832C, 0x18330, 0x18334, 0x18338
+local A_RQ_NEXT, A_WCUR, A_RCUR = 0x1833C, 0x18340, 0x18344
+local A_NHOLE, A_NHOLEB, A_NFULL = 0x18354, 0x18358, 0x1835C
+local A_NPOLL, A_NISSUE          = 0x18360, 0x18364
+local A_SLKMIN, A_SLKAT          = 0x18368, 0x1836C
+local A_PFREC, A_PFDONE          = 0x18370, 0x18374
+local A_NSEEK, A_SKWAIT          = 0x18378, 0x1837C
+-- src/player/xfer.i, the transport that lives in the machine.
+local A_XFSCSI, A_XSLBA0         = 0x18380, 0x18384
+local A_XSNXFER, A_XSNBYTE       = 0x18388, 0x1838C
+local A_XSNWIRE, A_XSERR, A_XSERRAT = 0x18390, 0x18394, 0x18398
+-- The scene header's record index, pushed into RAM before the CPU is launched.
+-- It sits ABOVE the codebooks rather than in the low RAM around ring.i's own
+-- tables: setup() runs one raster frame before launch() and the IPL is still
+-- executing in that gap, so anything written below ~$20000 can be overwritten
+-- before the 68000 ever sees it. That cost an hour: the machine read a table of
+-- zeroes and asked the transport for a 0-byte record.
+local IDXRAM                     = 0x30000   -- the DLX4 index, in RAM
 local DESCN                    = 64
 local CB1, CB4                 = 0x20000, 0x22000
 local RING                     = 0x40000
@@ -108,10 +155,45 @@ if KBPS == nil then
   return
 end
 local PREFILL   = (tonumber(os.getenv("DLX_PREFILL_KB") or "") or 0) * 1024
-local PACED     = (os.getenv("DLX_PACE") == "1")
+local SELFCLK   = (os.getenv("DLX_PACE") == "2")
+local PACED     = (os.getenv("DLX_PACE") == "1") or SELFCLK
 local CUT_AT    = tonumber(os.getenv("DLX_CUT_AT") or "")
 local CUT_FR    = tonumber(os.getenv("DLX_CUT_FR") or "") or 1
 local SLACK_CSV = os.getenv("DLX_SLACK_CSV")
+local RINGOWN   = (os.getenv("DLX_RINGOWN") == "1")
+local PREFILL_FR = tonumber(os.getenv("DLX_PREFILL_FR") or "") or 2
+-- Passes over the scene.  With DLX_RINGOWN each pass after the first begins
+-- with a REAL seek in src/player/stream.s: the channel is waited quiet, the
+-- ring is declared empty and the lookahead 51.3 says takes seconds of play to
+-- accumulate is thrown away and rebuilt from the prefill.  The check that
+-- matters afterwards is the one this rig always makes -- the last frame of the
+-- last pass has to be pixel-exact, and a SKIP block is a claim about the
+-- previous frame, so it is only right if every frame after the seek was.
+local ITERS     = tonumber(os.getenv("DLX_ITER") or "") or 1
+-- How many requests the player may have outstanding at once. 1 is the obvious
+-- loop and leaves the channel idle from every completion until the next poll;
+-- 2 keeps the next request queued so it never stops. Both are real designs and
+-- the difference between them is what this rig measures.
+local QDEPTH    = tonumber(os.getenv("DLX_QDEPTH") or "") or 2
+-- WHO MOVES THE BYTES.  "model" is this script: a transport that delivers
+-- XF_LEN bytes in XF_LEN/rate seconds and acks from emulated time, which is
+-- what FINDINGS 49/51/55 were all measured through.  "scsi" is ROADMAP P4b --
+-- src/player/xfer.i and src/player/scsi.i on the 68000, a real MB89352, a real
+-- volume, and NOTHING of this script in the transfer path.
+--
+-- The two are not interchangeable and the log must never let them look it.  The
+-- modelled transport OVERLAPS with the CPU, which is what a DMAC channel does;
+-- the real one here does not, because 57.3 leaves the CPU moving every byte
+-- itself.  So a "scsi" run measures CORRECTNESS off a real volume and the CPU
+-- cost of a PIO transport, and it measures NO rate: every rate-shaped number in
+-- this script's report is suppressed rather than printed against a transport
+-- that has no model behind it.
+local XFER      = os.getenv("DLX_XFER") or "model"
+local SCSIX     = (XFER == "scsi")
+-- src/player/scsi.i's SCE_* codes, so a failure names itself.
+local SCERRNAME = {[0]="OK", "SELECTION TIMEOUT -- no target answered",
+                   "UNEXPECTED PHASE", "POLL TIMEOUT -- a phase never arrived",
+                   "NON-ZERO SCSI STATUS"}
 -- One snapshot per frame tick instead of one at the end of the run. This is a
 -- DOCUMENTATION artefact -- 120 PNGs of a paced player, for a recording -- and
 -- it is deliberately not on any path tools/bench/check.sh takes. Needs
@@ -127,6 +209,16 @@ local BPS       = (KBPS > 0) and (KBPS - AUDIO_KBPS) * 1024 or math.huge
 local code do local f=io.open("stream.bin","rb"); code=f:read("a"); f:close() end
 local cb   do local f=io.open("stream_cb.bin","rb"); cb=f:read("a"); f:close() end
 local DISK = assert(io.open("stream_disk.bin","rb"))
+local idxblob do
+  local f = io.open("stream_idx.bin","rb")
+  if f then idxblob = f:read("a"); f:close() end
+end
+if RINGOWN and not idxblob then
+  print("[STR] DLX_RINGOWN needs the DLX4 record index (stream_idx.bin). "
+        .."Re-run tools/bench/prep_stream.py on a DLX4 container: the machine "
+        .."cannot derive record lengths by walking a stream it has not fetched.")
+  manager.machine:exit(); return
+end
 
 local YOFF = (MODE.height - META.H) // 2
 local function T() local t=M.time; return t.seconds + t.attoseconds/1e18 end
@@ -251,6 +343,202 @@ local function produce(now)
   end
 end
 
+-- -------------------------------------------------------------- transport
+-- THE OTHER HALF OF ROADMAP P5.  With DLX_RINGOWN the producer above is not
+-- used at all: src/player/ring.i decides which record to fetch, where to put it
+-- and when it is safe, and this becomes the part that is genuinely not the
+-- CPU's -- an SPC and one DMAC channel moving bytes at a rate.
+--
+-- IT IS BUILT OUT OF MEMORY TAPS, not out of the machine-frame notifier, and
+-- that is a measurement decision rather than a stylistic one.  A notifier sees
+-- the machine once per raster frame, 17.64 ms (54.5), and the thing being
+-- measured here is the gap between a transfer COMPLETING and the CPU issuing
+-- the next one -- which is a fraction of a frame slot.  Sampling it at 17.64 ms
+-- would have quantised the very quantity in question, and in the flattering
+-- direction if the ack were early or the pessimistic one if late.
+--
+--   write tap on XF_GO   fires inside the 68000's write, so the issue time is
+--                        exact.  The bytes are copied in there and then: the
+--                        CPU cannot observe them before its ack anyway, because
+--                        it does not advertise the record until it retires.
+--   read tap on XF_ACK   synthesises the completion word from emulated time --
+--                        `done` iff now >= t_done -- so the CPU learns of the
+--                        completion on the exact cycle it happens, the way it
+--                        would from a status register.
+-- A 68000 bus access is 16 bits, so a longword is two tap calls: the write is
+-- acted on at the LOW half (written second, so the whole value is there) and
+-- the read latches its answer at the HIGH half so the two halves cannot
+-- straddle a completion.
+local xf = {go = 0, ack = 0, busy = 0.0, gap = 0.0, gapmax = 0.0, ngap = 0,
+            latch = 0, tfree = 0, copylate = 0.0, ncopylate = 0}
+-- q[n] = the n-th request (n counts from 1): when the CPU issued it, when the
+-- transport will have finished it, and whether its bytes have been moved yet.
+local q = {}
+local npass = 1
+local function xf_deliver(off, dst, len, now)
+  DISK:seek("set", off)
+  push(dst, DISK:read(len), 1, len)
+end
+
+-- The host as AUDITOR rather than as producer.  Every placement the machine
+-- makes is checked against this script's own record index and its own list of
+-- records the decoder has not finished with -- the same overlap test the host
+-- producer used to make its decisions with, now used only to grade them.  A
+-- wrong placement corrupts pixels rather than faulting (49.2), so it has to be
+-- caught where it is made.
+local function audit(idx, off, dst, len)
+  local rec = META.index[idx + 1]
+  if not rec then return string.format("record %d does not exist", idx) end
+  if off ~= rec.off then
+    return string.format("record %d: machine asked for disc offset %d, the "
+                         .."index says %d", idx, off, rec.off) end
+  if len ~= rec.len then
+    return string.format("record %d: machine asked for %d B, the index says %d",
+                         idx, len, rec.len) end
+  local w = dst - RING
+  if w < 0 or w + len > RINGSZ then
+    return string.format("record %d: placed at ring offset %d..%d, outside a "
+                         .."%d B ring", idx, w, w + len, RINGSZ) end
+  local tail = SP:read_u32(FR_TAIL)
+  local i = 1
+  while i <= #live do
+    if live[i].idx < tail then table.remove(live, i) else i = i + 1 end
+  end
+  for _, r in ipairs(live) do
+    if w < r.off + r.len and r.off < w + len then
+      return string.format("record %d at %d..%d overlaps record %d at %d..%d, "
+                           .."which the decoder has not consumed",
+                           idx, w, w + len, r.idx, r.off, r.off + r.len) end
+  end
+  live[#live+1] = {idx = idx, off = w, len = len}
+  return nil
+end
+
+-- THE TAP OBJECTS ARE KEPT ALIVE HERE ON PURPOSE.  install_*_tap returns a
+-- handle and the tap dies with it: dropping it on the floor leaves the taps
+-- working until Lua's collector next runs, and then the mailbox silently stops
+-- answering. That looks exactly like a wedged producer -- the machine polled
+-- 2.4 million times for an ack that had already been computed.
+-- THE TAP OBJECTS ARE KEPT ALIVE HERE ON PURPOSE.  install_*_tap returns a
+-- handle and the tap dies with it: dropping it on the floor leaves the taps
+-- working until Lua's collector next runs, and then the mailbox silently stops
+-- answering. That looks exactly like a wedged producer -- the machine polled
+-- 2.4 million times for an ack that had already been computed.
+local TAPS = {}
+
+local function install_transport()
+  -- NOTHING IN A TAP CALLBACK TOUCHES THE MEMORY SPACE.  MAME 0.277 segfaults
+  -- if a tap reads or writes the space it was triggered from -- a nested access
+  -- part way through the CPU's own -- and it does it intermittently, which is
+  -- the worst way to find out.  So the taps do pure Lua: the write tap stamps
+  -- the EXACT emulated time the 68000 issued a request, the read tap answers
+  -- the completion count from those stamps, and every actual memory access is
+  -- done from the machine-frame notifier.
+  --
+  -- What that buys is the thing worth having: issue and completion times are
+  -- both exact, so the gap between a transfer finishing and the next one
+  -- starting -- the disc standing still because nobody has asked it for
+  -- anything -- is measured at cycle resolution rather than at 17.64 ms.
+  TAPS[#TAPS+1] = SP:install_write_tap(XF_GO, XF_GO + 3, "dlx_xf_go",
+  function(offset, data, mask)
+    if offset ~= XF_GO + 2 then return data end     -- low half, written second
+    -- ...and it must be the INCREMENT, not any other write to the word.
+    -- ring_init clears the mailbox at boot, which lands on this address with
+    -- the request fields still empty; without this test the transport answered
+    -- that clear as if it were a request and audited a 0-byte record 0.
+    if data ~= ((xf.go + 1) & 0xFFFF) then return data end
+    xf.go = xf.go + 1
+    q[xf.go] = {tissue = T(), served = false}
+    return data
+  end)
+  TAPS[#TAPS+1] = SP:install_read_tap(XF_ACK, XF_ACK + 3, "dlx_xf_ack",
+  function(offset, data, mask)
+    if offset == XF_ACK then
+      -- Latched on the high half so the two halves of one longword read cannot
+      -- straddle a completion.  Requests complete IN ORDER: one channel, one
+      -- transfer at a time, however many are queued.
+      local now = T()
+      while true do
+        local r = q[xf.ack + 1]
+        if not (r and r.served and now >= r.tdone) then break end
+        xf.ack = xf.ack + 1
+      end
+      xf.latch = xf.ack
+      return (xf.latch >> 16) & 0xFFFF
+    end
+    return xf.latch & 0xFFFF
+  end)
+end
+
+-- Service every request the machine has issued but this script has not yet
+-- looked at: read what was asked for, grade the placement, move the bytes, and
+-- work out when the channel will have finished it.  Called from the
+-- machine-frame notifier, which is where this script may touch memory.
+local function transport_service(t)
+  local n = xf.ack + 1
+  while q[n] do
+    local r = q[n]
+    if r.served then n = n + 1; goto continue end
+    local slot = XF_SLOT + ((n - 1) % 2) * XF_SLSZ
+    local off, dst = SP:read_u32(slot), SP:read_u32(slot + 4)
+    local len, idx = SP:read_u32(slot + 8), SP:read_u32(slot + 12)
+    if idx == 0 and n > 1 then
+      -- A new pass: everything the auditor knows about the ring is about the
+      -- scene we just left. The decoder has not consumed the new records and
+      -- the old ones are no longer anybody's.
+      live, nsent, arrival = {}, 0, {}
+      npass = npass + 1
+      P(string.format("SEEK PASS %d: the machine seeked back to record 0 at "
+                      .."%.3f s and is refilling from empty", npass, r.tissue))
+    end
+    local err = audit(idx, off, dst, len)
+    if err then
+      P("MISPLACED: "..err)
+      P("  The machine owns the ring in this run, so this is a policy bug in "
+        .."src/player/ring.i, not a rig one -- and it would have shown up as "
+        .."wrong pixels, because the block loop reads without a bounds check.")
+      M:exit(); return
+    end
+    -- ONE CHANNEL: a transfer starts when the channel is free AND the request
+    -- exists, so a queued request starts the instant its predecessor lands and
+    -- a late one starts when it is issued.  The difference is the gap, and the
+    -- gap is the player's, not the medium's -- no host-filled run could see it,
+    -- because the host producer placed records whenever it liked.
+    local startt = math.max(r.tissue, xf.tfree)
+    if xf.tfree > 0 then
+      local g = startt - xf.tfree
+      if g > 0 and nsent < META.nframes then
+        xf.gap, xf.ngap = xf.gap + g, xf.ngap + 1
+        if g > xf.gapmax then xf.gapmax = g end
+      end
+    end
+    local dur = (BPS == math.huge) and 0 or (len / BPS)
+    local done = startt + dur
+    -- A cut is a seek: the transfer freezes for its duration rather than
+    -- carrying on invisibly.  Freezing is the conservative reading and the one
+    -- 51.6 settled on for the host producer -- a drive that is repositioning is
+    -- not banking bytes it will burst on arrival.
+    if cut_t0 and done > cut_t0 and startt < cut_t1 then
+      done = done + (cut_t1 - math.max(startt, cut_t0))
+    end
+    xf_deliver(off, dst, len, t)
+    r.tdone, r.served = done, true
+    xf.tfree = done
+    xf.busy = xf.busy + dur
+    nsent = math.max(nsent, idx + 1)
+    arrival[idx + 1] = done          -- resident when the last byte lands
+    delivered = delivered + len
+    -- The one place this rig is coarser than the machine: a record whose
+    -- modelled transfer was shorter than the wait for this notifier was already
+    -- "done" by the time the bytes could be moved.  Counted, not absorbed.
+    if t > done then
+      xf.copylate, xf.ncopylate = xf.copylate + (t - done), xf.ncopylate + 1
+    end
+    n = n + 1
+    ::continue::
+  end
+end
+
 -- ------------------------------------------------------------------ setup
 local function setup()
   MODE.apply(SP)
@@ -267,17 +555,62 @@ local function setup()
   end
   for i = 1, #code do SP:write_u8(0x10000+i-1, string.byte(code,i)) end
   SP:write_u32(FLAG, 0); SP:write_u32(FR_HEAD, 0); SP:write_u32(FR_TAIL, 0)
-  SP:write_u32(ITER, 1); SP:write_u32(NFR, META.nframes)
+  SP:write_u32(ITER, ITERS); SP:write_u32(NFR, META.nframes)
   SP:write_u32(PACE, 0); SP:write_u32(PACEON, PACED and 1 or 0)
+  SP:write_u32(CLKON, SELFCLK and 1 or 0)
+  SP:write_u32(CLK_FPS, META.fps)
+  SP:write_u32(A_RINGOWN, RINGOWN and 1 or 0)
+  if RINGOWN then
+    -- The scene header the machine reads: the DLX4 record index, exactly the
+    -- bytes the container carries, pushed into RAM the way a loaded scene
+    -- header would be.  ring_init turns it into the disc-offset table a seek
+    -- needs; nothing here derives a record boundary for the machine.
+    push(IDXRAM, idxblob, 1, #idxblob)
+    SP:write_u32(A_IDX_B, IDXRAM)
+    SP:write_u32(A_RNG_B, RING)
+    SP:write_u32(A_RNG_SZ, RINGSZ)
+    SP:write_u32(A_PFREC, PREFILL_FR)
+    SP:write_u32(XF_QD, QDEPTH)
+    if SCSIX then
+      -- The stream begins at LBA 0 of the volume, because tools/bench/mkvol.sh
+      -- lays tmp/stream_disk.bin down from sector 0 and nothing else is on it.
+      -- It is a WORD the machine reads rather than a constant in xfer.i: a
+      -- shipping volume has a filesystem in front of the stream, and the base
+      -- LBA is the one number that changes when it does.
+      SP:write_u32(A_XFSCSI, 1)
+      SP:write_u32(A_XSLBA0, 0)
+    else
+      install_transport()
+    end
+  end
   P(string.format("stream.bin=%d B, codebooks %d+%d B, disk %d B, %d frames",
                   #code, META.cb1_len, META.cb4_len, META.disk_len, META.nframes))
-  P(string.format("decoder %s%s", PACED and ("PACED at "..META.fps.." fps")
+  P(string.format("decoder %s%s", SELFCLK
+                  and ("SELF-PACED at "..META.fps.." fps off V-DISP")
+                  or PACED and ("PACED at "..META.fps.." fps by the host")
                   or "FREE-RUNNING (tests wrap, not buffering -- 49.7.2)",
                   CUT_AT and string.format(", pipe cut at tick %d for %.2f fr",
                                            CUT_AT, CUT_FR) or ""))
-  P(string.format("ring %d KB at %06X, pipe %s, prefill %d KB, maxrec %d B",
+  P(string.format("ring %d KB at %06X, pipe %s, prefill %s, maxrec %d B",
                   RING_KB, RING, (KBPS > 0) and (KBPS.." KB/s") or "unlimited",
-                  PREFILL // 1024, META.maxrec))
+                  RINGOWN and (PREFILL_FR.." records (the machine's own)")
+                          or ((PREFILL // 1024).." KB"), META.maxrec))
+  if RINGOWN then
+    if SCSIX then
+      P(string.format("REAL TRANSPORT (ROADMAP P4b, src/player/xfer.i): the "
+                      .."68000 fetches every record itself with READ(10) off a "
+                      .."CZ-6BS1. This script moves NO bytes and models NO "
+                      .."rate. %d B of DLX4 index at %06X.", #idxblob, IDXRAM))
+      P("  the queue depth cannot deepen anything here: the transport is "
+        .."SYNCHRONOUS (xfer.i), so a queued request is drained by the same "
+        .."instruction stream that would be decoding. FINDINGS 55.3's 1-vs-2 "
+        .."result is about a transport that OVERLAPS; this one does not.")
+    else
+      P(string.format("ring OWNED BY THE 68000 (src/player/ring.i): this script "
+                      .."is a transport with a %d-deep request queue, %d B of "
+                      .."DLX4 index at %06X", QDEPTH, #idxblob, IDXRAM))
+    end
+  end
   if META.maxrec > RINGSZ then
     P("RING TOO SMALL: one record does not fit. stream.s needs a whole record "
       .."contiguous."); M:exit()
@@ -292,20 +625,66 @@ local function launch()
 end
 
 local st, t0, t_rel = "boot", nil, 0
+-- tick_t[i+1] = emulated time of frame tick i, filled in as they are observed.
+-- Under a self-clock this replaces the host's `t_rel + i/fps` as the deadline:
+-- the machine's clock is the one the player is actually held to, and MAME's
+-- raster is 2.22% fast, so grading arrivals against a host model of 12 fps
+-- would flatter every record by that much.
+local tick_t = {}
 local pace, min_ahead, min_at = -1, math.huge, -1
+-- DLX_XFER=scsi: the machine's seek counter, and the transfer count it stood at
+-- when that seek happened.  -1 means "not sampled yet", so the first sample
+-- rebases without announcing a pass that did not happen.
+local xs_nseek, xs_base = -1, 0
 local sum_ahead, n_ahead = 0, 0
 local slack_series = {}
 
 SUB = emu.add_machine_frame_notifier(function()
   local ok, err = pcall(function()
     local t = T()
+    if RINGOWN and not SCSIX then transport_service(t) end
+    if SCSIX and st ~= "boot" then
+      -- The machine's own count of completed transfers, which is what `nsent`
+      -- means in a run where this script sends nothing.  Sampled rather than
+      -- derived, so the slack window still closes when the producer runs out of
+      -- records to place instead of running on into the drain tail.
+      --
+      -- REBASED AT EVERY SEEK, and it has to be: XS_NXFER is cumulative over
+      -- the whole run, so on a second pass it is already >= nframes and the
+      -- slack sampling below -- gated on `nsent < META.nframes` -- would never
+      -- fire again.  It did not, and the second pass reported a ceiling of 0
+      -- frames and a build time of -1 ticks, which is an empty series printing
+      -- as a result.  ring.i's own seek counter is the exact place to rebase:
+      -- a seek discards every outstanding request by definition.
+      local ns = SP:read_u32(A_NSEEK)
+      if ns ~= xs_nseek then
+        -- ANNOUNCE ONLY A SEEK WITH PLAY BEHIND IT.  A pass opens with two
+        -- seeks that are not branch points -- ring_init's, and stream.s's at
+        -- the top of `outer` -- and both happen before a single record has been
+        -- fetched.  Announcing those numbered the first pass "SEEK PASS 2" and
+        -- the real one "3".  XS_NXFER separates them exactly: a seek taken with
+        -- transfers already behind it is the one that threw a ring away.
+        if xs_nseek >= 0 and SP:read_u32(A_XSNXFER) > 0 then
+          npass = npass + 1
+          P(string.format("SEEK PASS %d: the machine seeked at %.3f s and is "
+                          .."refilling from empty", npass, t))
+        end
+        xs_nseek, xs_base = ns, SP:read_u32(A_XSNXFER)
+      end
+      nsent = SP:read_u32(A_XSNXFER) - xs_base
+      if nsent < 0 then nsent = 0 end
+    end
     if st == "boot" then
       if t < 3.0 then return end
       setup(); last_t, pf_t0 = t, t; st = "prefill"; return
     end
     if st == "prefill" then
-      produce(t)
-      if delivered >= PREFILL then
+      if not RINGOWN then produce(t) end
+      -- RINGOWN: there is nothing for the host to prefill.  The machine issues
+      -- its own requests, so it has to be running first; ring_prefill then
+      -- holds the decoder until PF_REC records are resident, which is the same
+      -- policy in the place a player would keep it.
+      if RINGOWN or delivered >= PREFILL then
         P(string.format("prefill done: %.1f KB in %.3f s, releasing the CPU",
                         delivered/1024, t - pf_t0))
         launch(); t_rel = t; st = "running"
@@ -314,10 +693,29 @@ SUB = emu.add_machine_frame_notifier(function()
     end
     if st == "running" then
       if PACED then
-        local tick = math.floor((t - t_rel) * META.fps)
+        -- WHO WRITES THE TICK.  Host-paced, the tick is a host model of
+        -- META.fps and this script advances it.  Self-paced, the 68000 has
+        -- already advanced it off the raster and this script only READS it --
+        -- the frame gate in src/player/stream.s cannot tell the two apart,
+        -- which is the point: the same bytes are gated either way.
+        local tick = SELFCLK and SP:read_u32(PACE)
+                     or math.floor((t - t_rel) * META.fps)
+        if tick < pace then
+          -- src/player/stream.s rebased the clock for a new pass: tick 0 is the
+          -- instant the decoder was released after the seek's prefill, not the
+          -- start of the run. Everything sampled per tick starts again with it.
+          pace, tick_t = -1, {}
+          min_ahead, min_at = math.huge, -1
+          sum_ahead, n_ahead, slack_series = 0, 0, {}
+        end
         if tick > pace then
           pace = tick
-          SP:write_u32(PACE, pace)
+          if not SELFCLK then SP:write_u32(PACE, pace) end
+          -- The emulated time this tick actually happened, which is what a
+          -- record's deadline is measured against under a self-clock.  Ticks
+          -- can arrive more than one apart if the host misses a frame, so the
+          -- whole run is filled rather than just the newest.
+          for k = #tick_t, tick do tick_t[k+1] = t end
           -- Taken BEFORE this tick's frame is decoded, so snapshot n is the
           -- finished picture of frame n-1. tick 0 is skipped: nothing has been
           -- drawn yet and it would record a black screen as a decoded frame.
@@ -338,7 +736,12 @@ SUB = emu.add_machine_frame_notifier(function()
           if nsent < META.nframes then
             if ahead < min_ahead then min_ahead, min_at = ahead, tick end
             sum_ahead, n_ahead = sum_ahead + ahead, n_ahead + 1
-            slack_series[#slack_series+1] = {tick, ahead, n_ring, n_rate}
+            -- Column 3 is "has the producer been refused for space yet",
+            -- which is what makes a later minimum meaningful: before the first
+            -- refusal the ring is still filling. With the ring owned by the
+            -- machine that counter is ring.i's, not this script's.
+            slack_series[#slack_series+1] =
+              {tick, ahead, RINGOWN and SP:read_u32(A_NFULL) or n_ring, n_rate}
           end
           if CUT_AT and tick >= CUT_AT and not cut_t0 then
             cut_t0, cut_t1 = t, t + CUT_FR / META.fps
@@ -348,16 +751,55 @@ SUB = emu.add_machine_frame_notifier(function()
           end
         end
       end
-      produce(t)
+      if not RINGOWN then produce(t) end
+      if RINGOWN and os.getenv("DLX_RINGDBG") == "1" then
+        dbg_n = (dbg_n or 0) + 1
+        if dbg_n % 30 == 0 then
+          P(string.format("DBG t=%.2f flag=%X go=%d ack=%d tfree=%.3f queued=%s "
+                          .."head=%d tail=%d rq=%d poll=%d full=%d wcur=%d rcur=%d",
+                          t, SP:read_u32(FLAG), xf.go, xf.ack, xf.tfree,
+                          tostring(q[xf.ack+1] ~= nil), SP:read_u32(FR_HEAD),
+                          SP:read_u32(FR_TAIL), SP:read_u32(A_RQ_NEXT),
+                          SP:read_u32(A_NPOLL), SP:read_u32(A_NFULL),
+                          SP:read_u32(A_WCUR), SP:read_u32(A_RCUR)))
+        end
+      end
       if cut_t0 and not cut_done and t >= cut_t1 then cut_done = true end
       local fl = SP:read_u32(FLAG)
       if fl == 1 and not t0 then t0 = t; return end
       if fl == 0xEE then
         P("BITSTREAM DESYNC -- the decoder consumed the wrong number of bytes.")
+        if RINGOWN then
+          local tail, head = SP:read_u32(FR_TAIL), SP:read_u32(FR_HEAD)
+          P(string.format("  tail=%d head=%d rq=%d wcur=%d rcur=%d go=%d ack=%d",
+                          tail, head, SP:read_u32(A_RQ_NEXT), SP:read_u32(A_WCUR),
+                          SP:read_u32(A_RCUR), xf.go, xf.ack))
+          P(string.format("  DESC[%d]=%08X, the index says record %d is at ring "
+                          .."offset ? len %d", tail,
+                          SP:read_u32(DESC + (tail % 64)*4), tail,
+                          META.index[tail+1] and META.index[tail+1].len or -1))
+          local ls = {}
+          for _, r in ipairs(live) do
+            ls[#ls+1] = string.format("%d@%d+%d", r.idx, r.off, r.len) end
+          P("  host thinks live: "..table.concat(ls, " "))
+        end
         P("  Under a ring that is the whole point: it means a record was placed "
           .."or described wrongly, not that the codec changed.")
         M:exit(); return
       end
+      if fl == 0xE2 then
+        local e = SP:read_u32(CLK_ERR)
+        P("FRAME CLOCK REFUSED: CLK_ERR="..e..(e == 1 and
+          " (the CRTC is not in a 31.5 kHz mode, so the divider's 31500 "
+          .."lines/s would be wrong)" or e == 2 and
+          " (fps*VTOTAL does not fit the 16-bit accumulator)" or ""))
+        M:exit(); return
+      end
+      if fl == 0xE3 then
+        P("RING INIT REFUSED: the record index has more entries than "
+          .."src/player/ring.i's disc-offset table (ROFFMAX) can hold.")
+        M:exit(); return
+      end
       if fl == 0xE1 then
         P("PRODUCER STALLED OUT -- stream.s spun SPINMAX times with no new "
           .."record. Delivered "..nsent.."/"..META.nframes..".")
@@ -382,10 +824,117 @@ SUB = emu.add_machine_frame_notifier(function()
                           META.nframes, dt, dt*CPUHZ/META.nframes,
                           100*(dt*CPUHZ/META.nframes)/FRAME12, META.fps))
         end
+        if SELFCLK then
+          -- The clock's own report, in the run where it actually paced a
+          -- decoder rather than a busy loop.  The rate is against MAME's fast
+          -- raster; tools/bench/clock_run.sh is where it gets de-skewed.
+          local vd = SP:read_u32(CLK_VDISP)
+          local vt = SP:read_u16(0xE80008) + 1   -- VTOTAL, as clk_init read it
+          -- DELIBERATELY NOT A RATE.  120 ticks is far too short a window to
+          -- quote fps from: the run's start and end each straddle a tick, so
+          -- +/-1 on 119 intervals is +/-8000 ppm and would read as drift the
+          -- clock does not have. What IS exact here is a count of refreshes
+          -- against a count of ticks. The rate figure comes from
+          -- tools/bench/clock_run.sh, over thousands of them.
+          P(string.format("FRAME CLOCK: %d V-DISP interrupts drove %d ticks "
+                          .."(%.4f refreshes/frame; the divider's own ratio is "
+                          .."31500/(%d*%d) = %.4f), %.0f clocks/frame of "
+                          .."interrupt at FINDINGS 54's 181.35 each",
+                          vd, pace + 1, vd/(pace + 1), META.fps, vt,
+                          31500/(META.fps*vt), 181.35*vd/(pace+1)))
+        end
+        local rholes, rhole_b = holes, hole_bytes
+        if RINGOWN then
+          rholes, rhole_b = SP:read_u32(A_NHOLE), SP:read_u32(A_NHOLEB)
+        end
         P(string.format("ring: %d wraps, %d B of hole (mean %.1f KB, %.1f%% of "
-                        .."the ring)", holes, hole_bytes,
-                        holes > 0 and hole_bytes/holes/1024 or 0,
-                        100*(holes > 0 and hole_bytes/holes or 0)/RINGSZ))
+                        .."the ring)", rholes, rhole_b,
+                        rholes > 0 and rhole_b/rholes/1024 or 0,
+                        100*(rholes > 0 and rhole_b/rholes or 0)/RINGSZ))
+        if RINGOWN then
+          -- The producer's own account of itself.  These are read out of the
+          -- machine's RAM, not kept here: the point of the run is that this
+          -- script did not make any of these decisions.
+          local pf   = SP:read_u32(A_PFDONE)
+          local npoll, nissue = SP:read_u32(A_NPOLL), SP:read_u32(A_NISSUE)
+          local nfull = SP:read_u32(A_NFULL)
+          local slk, slkat = SP:read_u32(A_SLKMIN), SP:read_u32(A_SLKAT)
+          local nseek, skw = SP:read_u32(A_NSEEK), SP:read_u32(A_SKWAIT)
+          P(string.format("MACHINE-OWNED RING: %d records placed by the 68000, "
+                          .."%d polls, %d refused for space (ring-bound), "
+                          .."%d seeks", nissue, npoll, nfull, nseek))
+          P(string.format("PREFILL: released the decoder at %d records "
+                          .."(policy: %d); LEAST SLACK %d records at frame %d",
+                          pf, PREFILL_FR, slk, slkat))
+          -- THE NUMBER THIS RIG EXISTS TO PRODUCE.  The channel only moves
+          -- bytes while a request is outstanding and only the CPU can issue the
+          -- next one, so the disc stands still between the completion of one
+          -- record and the issue of the next.  That gap is a property of the
+          -- PLAYER's loop, not of the medium, and no host-filled run could see
+          -- it -- the host producer placed records whenever it liked.
+          local span = (arrival[nissue] or t) - (t0 or t)
+          if SCSIX then
+            -- WHAT A REAL TRANSPORT CAN AND CANNOT BE ASKED HERE.  The modelled
+            -- transport knew when every byte landed because it decided; this one
+            -- does not report, and MAME's device models are functional rather
+            -- than transfer-timing accurate, so even if it did the number would
+            -- not be a rate (docs/BENCHMARK.md, 42.5).  So CHANNEL IDLE,
+            -- DEADLINE and REQUIRED PREFILL are not printed at all rather than
+            -- printed as zeros -- a zero here reads as "the channel never
+            -- stopped", which would be a claim about a medium this rig has
+            -- never timed.
+            local nb, nw = SP:read_u32(A_XSNBYTE), SP:read_u32(A_XSNWIRE)
+            local nx = SP:read_u32(A_XSNXFER)
+            P(string.format("REAL TRANSPORT: %d READ(10)s by the 68000, %d B "
+                            .."into the ring", nx, nb))
+            -- THE COST THE CONTAINER PAYS FOR NOT BEING SECTOR-ALIGNED.  A
+            -- record starts wherever the previous one ended, rounded up to 4;
+            -- a target answers in 512 B blocks.  So the command covers the
+            -- sectors the record lies in and src/player/scsi.i's window drops
+            -- the rest.  In PIO those bytes cost nothing but wire; they are
+            -- still bytes the disc moved that no frame contains.
+            P(string.format("SECTOR OVERHEAD: %d B off the disc for %d B of "
+                            .."record = %.2f%% the medium moved and no frame "
+                            .."contains", nw, nb,
+                            nb > 0 and 100*(nw - nb)/nb or 0))
+            local e, eat = SP:read_u32(A_XSERR), SP:read_u32(A_XSERRAT)
+            if e ~= 0 then
+              P(string.format("TRANSPORT FAILED: SC_ERR=%d on request %d (%s)",
+                              e, eat, SCERRNAME[e] or "see src/player/scsi.i"))
+            end
+            if skw > 0 then
+              P(string.format("  the last seek waited %d polls for the channel "
+                              .."to go quiet before it could start", skw))
+            end
+            goto ringdone
+          end
+          P(string.format("CHANNEL IDLE: %.1f ms over %d gaps (worst %.1f ms) = "
+                          .."%.1f%% of the %.2f s the transport was needed for; "
+                          .."busy %.1f ms",
+                          xf.gap*1000, xf.ngap, xf.gapmax*1000,
+                          span > 0 and 100*xf.gap/span or 0, span,
+                          xf.busy*1000))
+          -- THE RIG'S OWN RESOLUTION, PRINTED RATHER THAN LEFT IMPLICIT.
+          -- The transport's timing is exact (memory taps), but the BYTES are
+          -- moved from the machine-frame notifier, so a record whose modelled
+          -- transfer is shorter than the wait for that notifier is acked later
+          -- than the model says it landed. Every millisecond here is a
+          -- millisecond the decoder may have waited that the medium would not
+          -- have made it wait.
+          if xf.ncopylate > 0 then
+            P(string.format("RIG RESOLUTION: %d of %d transfers were acked late "
+                            .."because the bytes are moved on the notifier, by "
+                            .."%.1f ms in total (mean %.1f ms) -- an artefact of "
+                            .."this rig, not of the player",
+                            xf.ncopylate, nissue, xf.copylate*1000,
+                            xf.copylate*1000/xf.ncopylate))
+          end
+          if skw > 0 then
+            P(string.format("  the last seek waited %d polls for the channel "
+                            .."to go quiet before it could start", skw))
+          end
+          ::ringdone::
+        end
         -- The decoder's own spin counter, kept for what it is: evidence that
         -- stream.s outran the pipe, NOT evidence of an underrun.  See the note
         -- on `arrival` above.
@@ -395,6 +944,38 @@ SUB = emu.add_machine_frame_notifier(function()
           -- opposite thing, so the two are never printed in the same words.
           P(string.format("UNDERRUNS: %d/%d frames waited past their %d fps slot "
                           .."(%d polls)", stalls, META.nframes, META.fps, spins))
+          if SCSIX then
+            -- AND IT IS VACUOUS HERE, WHICH IS WORTH MORE THAN THE ZERO IS.
+            -- A synchronous transport cannot underrun by construction: the
+            -- decoder does not go on until the record is in its hand, because
+            -- it fetched the record itself.  What it does instead is MISS THE
+            -- SLOT, and that is the NO IDLE line above -- 442 whole ticks of
+            -- overrun is not a late present, it is a player running at a
+            -- fraction of its frame rate.  Quoting "0 underruns" from a
+            -- DLX_XFER=scsi run without this line would be the most flattering
+            -- possible reading of the least flattering result in the tree.
+            local ticks = pace + 1
+            P(string.format("  0 IS VACUOUS: the transport is synchronous, so a "
+                            .."frame cannot start before its record has landed "
+                            .."-- the decoder IS the transport. The honest rate "
+                            .."is %d frames in %d slots of %d fps = %.2f fps.",
+                            META.nframes, ticks, META.fps,
+                            ticks > 0 and META.fps*META.nframes/ticks or 0))
+          end
+          -- The other way a paced frame can go wrong, and it is not the same
+          -- failure.  An UNDERRUN is the pipe: the record was not there.  This
+          -- is the CPU: the record was there, the slot was already open, and
+          -- the decoder had no idle left in the frame before.  Under a
+          -- host-written tick every slot is 83.33 ms; under the machine's own
+          -- clock they alternate 72.13 and 90.16 ms, 37.9% of them short, and
+          -- this is what the short ones cost (FINDINGS 54).
+          local late, latemax = SP:read_u32(LATEFR), SP:read_u32(LATEMAX)
+          local late1 = SP:read_u32(LATE1ST)
+          P(string.format("NO IDLE: %d/%d frames found their slot already open "
+                          .."-- the frame before used all of it; worst overrun "
+                          .."%d whole tick%s, first at frame %d", late,
+                          META.nframes, latemax, latemax == 1 and "" or "s",
+                          late1 == 0xFFFFFFFF and -1 or late1))
           -- WHAT THE MINIMUM OVER A RUN IS, AND IS NOT.  At release the ring
           -- holds only what the prefill put there, so the early ticks report a
           -- buffer that has not been built yet, not a ring or a pipe that
@@ -415,15 +996,22 @@ SUB = emu.add_machine_frame_notifier(function()
           for _,e in ipairs(slack_series) do
             if e[3] > 0 and e[2] < ss_min then ss_min, ss_at = e[2], e[1] end
           end
+          if ss_min == math.huge then ss_min, ss_at = -1, -1 end
           P(string.format("SEEK SLACK: ceiling %d frames (%.0f ms), first "
                           .."reached at tick %d; mean %.1f over the window",
                           ceiling, 1000*ceiling/META.fps, ceil_at,
                           sum_ahead/math.max(1,n_ahead)))
-          if n_ring > 0 then
+          -- WHOSE REFUSAL COUNT.  With the ring owned by the machine the host
+          -- producer's counters are not merely stale, they are zero -- it never
+          -- placed anything -- so the classification has to come from
+          -- src/player/ring.i's own N_FULL. Reading n_ring here in a RINGOWN
+          -- run would have reported RATE-BOUND on every run by construction.
+          local ring_ref = RINGOWN and SP:read_u32(A_NFULL) or n_ring
+          if ring_ref > 0 then
             P(string.format("  RING-BOUND: the ring filled (%d refusals). Once "
                             .."full it survives delivery stopped dead for %d "
                             .."frames = %.0f ms; min after first fill %d "
-                            .."(tick %d)", n_ring, ceiling,
+                            .."(tick %d)", ring_ref, ceiling,
                             1000*ceiling/META.fps, ss_min, ss_at))
           else
             P(string.format("  RATE-BOUND: the ring NEVER filled in %d frames. "
@@ -451,11 +1039,35 @@ SUB = emu.add_machine_frame_notifier(function()
         end
 
         -- The delivery result.  Deadline for record i is release + i/fps.
+        -- Only this script can compute it, because only this script knows when
+        -- a byte landed -- and in a DLX_XFER=scsi run it does not.  Printing it
+        -- from an empty arrival table would report "0/120 records late", which
+        -- is the most flattering possible reading of no data at all.
+        if SCSIX then
+          P("DEADLINE: not measured. The transport is in the machine and does "
+            .."not report arrival times, and MAME's SCSI is not transfer-timing "
+            .."accurate, so there is no honest deadline to grade against here. "
+            .."The sharp instrument that DOES survive is the decoder's own "
+            .."UNDERRUNS count above -- the machine takes it, and it is exact.")
+          st = "settle"; return
+        end
         local misses, worst, prefill_s = 0, 0.0, 0.0
         for i = 0, META.nframes-1 do
           local a = arrival[i+1]
           if a then
-            local late = a - (t_rel + i / META.fps)
+            -- INSTRUMENT NOTE, and it only bites under the self-clock.
+            -- tick_t[] is when this script OBSERVED the machine's tick, which
+            -- is the next machine-frame boundary after it -- up to 17.64 ms
+            -- late (54.5). So these deadlines are generous by that much and
+            -- this count UNDERSTATES lateness. The sharp instrument for the
+            -- same question is the decoder's own stall counter, which is exact
+            -- because the machine takes it: at 488 KB/s with a one-deep request
+            -- queue this reported 6 records late where the 68000 counted 59
+            -- frames that had to wait. Host-paced (DLX_PACE=1) the deadline is
+            -- a host model and is exact, so FINDINGS 49.6's table is unaffected.
+            local due = SELFCLK and tick_t[i+1] or (t_rel + i / META.fps)
+            if not due then due = t_rel + i / META.fps end
+            local late = a - due
             if late > 0 then
               misses = misses + 1
               if late > worst then worst = late end
@@ -473,14 +1085,28 @@ SUB = emu.add_machine_frame_notifier(function()
                         .."at %s", prefill_s*1000, prefill_s*META.fps,
                         (KBPS > 0) and (prefill_s * BPS / 1024) or 0.0,
                         (KBPS > 0) and (KBPS.." KB/s") or "an unlimited pipe"))
-        M.video:snapshot()
-        P("snapshot taken after the sequential pass -- last frame, decoded "
-          .."entirely out of a "..RING_KB.." KB ring")
-        st = "snapped"; return
+        -- ONE FRAME OF SETTLE BEFORE THE CAPTURE, and it is not cosmetic.
+        -- MAME renders a screen scanline by scanline and the machine-frame
+        -- notifier fires at the END of that frame, so a bitmap for a frame in
+        -- which GVRAM changed holds some lines drawn before the change and some
+        -- after. Snapshotting it captures a TEAR -- and the symptom is a
+        -- pixel-exactness failure in the bottom blocks plus a broken double-scan
+        -- pairing, which reads exactly like a decoder bug and is not one. It
+        -- only bites when the decoder finishes its last frame LATE in a screen
+        -- frame, which is why it never showed up until a run with underruns in
+        -- it. Waiting one whole frame renders the finished picture with nothing
+        -- writing to GVRAM.
+        st = "settle"; return
       end
       if t > 900 then P("TIMEOUT flag="..string.format("%08X",fl)); M:exit() end
       return
     end
+    if st == "settle" then
+      M.video:snapshot()
+      P("snapshot taken after the sequential pass -- last frame, decoded "
+        .."entirely out of a "..RING_KB.." KB ring")
+      st = "snapped"; return
+    end
     if st == "snapped" then M:exit(); return end
   end)
   if not ok then print("[STR] LUA ERROR: "..tostring(err)); M:exit() end

tools/bench/verify_adpcm.py

@@ -0,0 +1,140 @@
+#!/usr/bin/env python3
+"""Gate tools/encoder/adpcm.py against the only independent decoder on this
+machine: ffmpeg's `adpcm_ima_oki`.
+
+THIS FILE IS NOT ABOUT THE X68000's CHIP and after session 34 that distinction
+is load-bearing. It checks this implementation against an independent one, so
+its parameters stay ffmpeg's -- variant 'shift', high nibble first, a 12-bit
+clamp, accumulator from 0. What the MACHINE's MSM6258 does is measured by
+tools/bench/adpcm_run.sh and it is a different set of four values on all four
+axes (adpcm.CHIP, FINDINGS 66). Do not "fix" the defaults here to match it: a
+reference check whose reference has been adjusted to agree is not a check.
+
+There is no ffmpeg ENCODER for this format -- `adpcm_ima_oki` is decode-only --
+so the encoder here cannot be checked against a reference implementation. What
+CAN be checked, and is, is that the decoder our encoder runs in its own loop is
+byte-for-byte the decoder that ships in ffmpeg. An encoder that agrees with its
+own wrong decoder is exactly the failure this catches.
+
+Usage: verify_adpcm.py [wav_or_raw12 ...]
+"""
+import os, struct, subprocess, sys, random, math
+sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "encoder"))
+import adpcm
+
+TMP = "tmp/adpcm_gate"
+
+# The step table as it is printed in the OKI datasheet and in every
+# implementation of this format. adpcm.py BUILDS its table from 16*1.1**k; if
+# the two ever disagree, one of them is a typo and this says which.
+CANON = [16,17,19,21,23,25,28,31,34,37,41,45,50,55,60,66,73,80,88,97,107,118,
+         130,143,157,173,190,209,230,253,279,307,337,371,408,449,494,544,598,
+         658,724,796,876,963,1060,1166,1282,1411,1552]
+
+fails = []
+def ck(ok, msg):
+    print(("OK  " if ok else "FAIL ") + msg)
+    if not ok: fails.append(msg)
+
+
+def ffmpeg_decode(data, rate=15625):
+    """Decode packed OKI ADPCM through ffmpeg, by wrapping it in a WAV whose
+    format tag is 0x0010 (WAVE_FORMAT_OKI_ADPCM). Returns 16-bit samples."""
+    os.makedirs(TMP, exist_ok=True)
+    fmt = struct.pack("<HHIIHHH", 0x0010, 1, rate, rate, 1, 4, 0)
+    body = (b"WAVE" + b"fmt " + struct.pack("<I", len(fmt)) + fmt
+                    + b"data" + struct.pack("<I", len(data)) + data)
+    w = f"{TMP}/probe.wav"
+    open(w, "wb").write(b"RIFF" + struct.pack("<I", len(body)) + body)
+    raw = subprocess.check_output(
+        ["ffmpeg", "-v", "error", "-i", w, "-f", "s16le", "-acodec", "pcm_s16le", "-"])
+    return list(struct.unpack("<%dh" % (len(raw) // 2), raw))
+
+
+def snr_db(ref, got):
+    """Signal-to-noise over the 12-bit sample word."""
+    num = sum(float(s) * s for s in ref)
+    den = sum((float(a) - b) ** 2 for a, b in zip(ref, got))
+    if den == 0: return float("inf")
+    return 10.0 * math.log10(num / den) if num else float("-inf")
+
+
+print("--- the step table ---")
+ck(adpcm.STEP == CANON, f"49 entries, built = published (16*1.1**k), {adpcm.STEP[0]}..{adpcm.STEP[-1]}")
+
+print("--- our decoder vs ffmpeg's adpcm_ima_oki ---")
+random.seed(1234)
+nibs = ([7]*12 + [i % 16 for i in range(256)]
+        + [random.randrange(16) for _ in range(4000)])
+data = adpcm.pack(nibs)
+ff = ffmpeg_decode(data)
+ours = [v * 16 for v in adpcm.decode(adpcm.unpack(data, len(nibs)), "shift")]
+ck(len(ff) == len(ours), f"sample count {len(ff)} = {len(ours)}")
+ck(ff == ours, f"variant 'shift' is SAMPLE-EXACT vs ffmpeg over {len(nibs)} nibbles")
+
+# NEGATIVE CONTROL. A gate that passes whatever it is handed proves nothing;
+# reading the nibbles the other way round has to go red, or "high nibble first"
+# is an assertion rather than a measurement.
+lowfirst = [adpcm.unpack(data, len(nibs))[i ^ 1] for i in range(len(nibs))]
+bad = [v * 16 for v in adpcm.decode(lowfirst, "shift")]
+ndiff = sum(1 for a, b in zip(ff, bad) if a != b)
+ck(ndiff > 0, f"low-nibble-first DISAGREES on {ndiff}/{len(ff)} -- so what ffmpeg reads is measured, not assumed")
+# AND IT IS A FACT ABOUT A FILE FORMAT, NOT ABOUT A CHIP. Session 33 recorded
+# this line as "nibble order: HIGH FIRST, measured", which it is -- of the VOX
+# convention ffmpeg implements. Session 34 asked the machine's own MSM6258 the
+# same question through HD63450 channel 3 and got the OTHER answer: the chip
+# takes the LOW nibble of a delivered byte first (FINDINGS 66), and encoding
+# for the wrong one of the two costs -25.7 dB on the Singe window. The two
+# claims do not conflict; they are about different things, and only one of them
+# is about the machine this is being ported to.
+
+print("--- and the second variant is not the same decoder ---")
+terms = [v * 16 for v in adpcm.decode(adpcm.unpack(data, len(nibs)), "terms")]
+d = [abs(a - b) // 16 for a, b in zip(ff, terms)]
+nd = sum(1 for x in d if x)
+ck(nd > 0, f"variant 'terms' differs on {nd}/{len(d)} samples, max {max(d)} in 12-bit units"
+           " -- and 'terms' is the one the machine runs (adpcm_run.sh, FINDINGS 66)")
+
+print("--- and getting the variant wrong is NOT a rounding error ---")
+# THE MEASUREMENT THAT CHANGED THIS FROM A FOOTNOTE INTO AN OPEN ITEM. The two
+# variants differ by at most 3 in 12-bit units PER SAMPLE, which reads like
+# something nobody could hear. ADPCM is RECURSIVE -- the delta is added to a
+# running predictor and the nibble also moves the step index -- so the
+# disagreement does not stay where it happens. It is the same shape as the
+# codec's temporal recursion (64.1), one dimension down.
+if os.path.exists("tmp/au_singe.raw"):
+    raw = open("tmp/au_singe.raw", "rb").read()
+    pcm = struct.unpack("<%dh" % (len(raw) // 2), raw)
+    ref = [max(-2048, min(2047, x >> 4)) for x in pcm]
+    nib = adpcm.encode(ref, "shift")
+    same, cross = adpcm.decode(nib, "shift"), adpcm.decode(nib, "terms")
+    err = [abs(x - y) for x, y in zip(same, cross)]
+    print(f"    encoded 'shift', decoded 'shift': SNR {snr_db(ref, same):6.2f} dB")
+    print(f"    encoded 'shift', decoded 'terms': SNR {snr_db(ref, cross):6.2f} dB"
+          f"   <- the noise is LOUDER THAN THE SIGNAL")
+    print(f"    per-sample disagreement over {len(err):,} samples: max {max(err)}, "
+          f"mean {sum(err)/len(err):.1f} in 12-bit units")
+    ck(snr_db(ref, cross) < 0,
+       "a 3-LSB formula disagreement costs ~25 dB, because ADPCM is RECURSIVE")
+else:
+    print("    SKIPPED: no tmp/au_singe.raw (tools/encoder/extract_audio.py)")
+
+print("--- the encoder, through the gated decoder ---")
+for path in (sys.argv[1:] or []):
+    raw = open(path, "rb").read()
+    if raw[:4] == b"RIFF":
+        raw = subprocess.check_output(["ffmpeg", "-v", "error", "-i", path,
+              "-f", "s16le", "-ac", "1", "-ar", "15625", "-"])
+    pcm16 = struct.unpack("<%dh" % (len(raw) // 2), raw)
+    src = [max(-2048, min(2047, s >> 4)) for s in pcm16]
+    nib = adpcm.encode(src, "shift")
+    packed = adpcm.pack(nib)
+    rec_ff = [v // 16 for v in ffmpeg_decode(packed)][:len(src)]
+    rec_us = adpcm.decode(nib, "shift")
+    ck(rec_ff == rec_us,
+       f"{os.path.basename(path)}: encoder's own reconstruction = ffmpeg's, {len(src)} samples")
+    print(f"    {len(src)} samples, {len(packed)} B, SNR {snr_db(src, rec_us):.2f} dB "
+          f"(12-bit word; the source is already quantised to it)")
+
+print("ADPCM GATE " + ("GREEN" if not fails else f"RED: {len(fails)} failed"))
+sys.exit(1 if fails else 0)

tools/bench/verify_adpcm_chip.py

@@ -0,0 +1,176 @@
+#!/usr/bin/env python3
+"""Read the MSM6258's own decoder out of a MAME capture.  ROADMAP P6a.
+
+FIVE THINGS ARE UNKNOWN, not the one FINDINGS 65 named:
+
+    feed     which nibble(s) of a delivered BYTE the chip actually plays
+    variant  'shift' | 'terms'   the delta formula        (65.2, worth 25 dB)
+    bits      12     | 10        where the accumulator clamps
+    init       0     | -2        the accumulator at PLAY
+    (and the capture's own decimation, below)
+
+`feed` is on the list because the run MEASURED it and it is not what anybody
+assumed.  Fed by HD63450 channel 3 in the IPL ROM's own configuration, this
+machine plays ONE nibble per delivered byte -- the prologue is 16 zero nibbles
+in 8 bytes and the accumulator climbs by 8 steps, not 16.  A probe that had
+assumed two would have found no model that fit and reported a broken rig.  So
+the hypothesis is enumerated with the others and the capture picks.
+
+THE CAPTURE'S DECIMATION is enumerated for the same reason.  MAME resamples the
+chip's stream to the wav's rate, and a filtered 2x upsample is not recognisable
+from a single sample -- on a slow ramp it looks like an exact repeat and on a
+step it does not.  So (factor, phase) is searched over {1x, 2x phase 0, 2x phase
+1}, and the SCALE RESIDUAL is then checked on the decimated stream: MAME's
+okim6258 puts `signal << 4` into a stream scaled to 32768 and the machine routes
+it to the speaker at gain 0.50, so a chip sample is `signal * 8`.  If the
+winning decimation does not land within a couple of counts of a multiple of 8 on
+every sample, it is not the chip's own stream and the run says so instead of
+rounding to the nearest story.
+
+WHAT THIS DOES NOT SETTLE.  It measures MAME's device model driven through the
+machine's real transport.  It settles the RIG -- an emulated audio test encoded
+against the wrong model is 25 dB of nothing -- and it leaves the silicon where
+it was: needing a board or a datasheet.
+"""
+import json, os, struct, sys, wave
+
+sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "encoder"))
+import adpcm
+
+WAV = sys.argv[1] if len(sys.argv) > 1 else "tmp/adpcm.wav"
+SEQ = sys.argv[2] if len(sys.argv) > 2 else "tmp/adpcm_seq.json"
+SCALE = 8                # okim6258's <<4, times the machine's 0.50 speaker route
+PRO_MAX = 40             # zero nibbles the prologue is allowed to have grown by
+SKEW = 8                 # samples of slack on where PLAY lands in the capture
+DECIM = ((1, 0), (2, 0), (2, 1))
+FEEDS = ("both-high-first", "both-low-first", "low-only", "high-only")
+
+fails = []
+def ck(ok, msg):
+    print(("OK   " if ok else "FAIL ") + msg)
+    if not ok: fails.append(msg)
+
+
+def nibbles_for(data, feed):
+    """The nibble sequence the chip is hypothesised to have PLAYED, given the
+    bytes the channel delivered."""
+    if feed == "both-high-first":
+        return list(adpcm.unpack(data, None, "high"))
+    if feed == "both-low-first":
+        return list(adpcm.unpack(data, None, "low"))
+    if feed == "low-only":
+        return [b & 15 for b in data]
+    return [b >> 4 for b in data]
+
+
+def main():
+    seq = json.load(open(SEQ))
+    pro, core = seq["pro"], seq["core"]
+    data = adpcm.pack([0] * pro + core, "high")
+    pro_bytes = pro // 2                 # bytes of prologue, all $00
+
+    w = wave.open(WAV)
+    rate = w.getframerate()
+    n, ch = w.getnframes(), w.getnchannels()
+    s = struct.unpack("<%dh" % (n * ch), w.readframes(n))
+    left, right = list(s[0::ch]), list(s[1::ch])
+    ck(left == right, "both speakers carry the same samples (pan 00 = BOTH)")
+    ck(any(left), "the capture contains a signal at all")
+    if not any(left):
+        return 1
+
+    # ---- search: (decimation) x (feed) x (variant, bits, init) x (prologue)
+    results = {}
+    for fac, ph in DECIM:
+        rec = [round(v / SCALE) for v in left[ph::fac]]
+        nz = next((i for i, v in enumerate(rec) if v), None)
+        if nz is None:
+            continue
+        lo, hi = max(0, nz - SKEW), nz + 1
+        for feed in FEEDS:
+            base = nibbles_for(data, feed)
+            # a repeat of byte 0 costs whole nibbles under 'both' and one nibble
+            # under 'low-only'/'high-only'; either way it is zeros
+            for extra in range(PRO_MAX):
+                for variant in ("shift", "terms"):
+                    for bits in (12, 10):
+                        for init in (0, -2):
+                            want = adpcm.decode([0] * extra + base, variant,
+                                                init=init, bits=bits)
+                            for off in range(lo, hi):
+                                if rec[off:off + len(want)] == want:
+                                    results.setdefault(
+                                        (feed, variant, bits, init),
+                                        (fac, ph, extra, off, len(want)))
+    print("--- candidates that reproduce the capture SAMPLE-EXACT ---")
+    for k, v in results.items():
+        print(f"     feed={k[0]:<15s} variant={k[1]:<5s} bits={k[2]} init={k[3]:<2d}"
+              f"   decimation {v[0]}x phase {v[1]}, prologue +{v[2]}, "
+              f"{v[4]:,} samples")
+    ck(len(results) == 1,
+       f"exactly one model reproduces the capture ({len(results)} did)")
+    if len(results) != 1:
+        return 1
+
+    model, (fac, ph, extra, off, ln) = next(iter(results.items()))
+    feed, variant, bits, init = model
+
+    # ---- the scale residual, on the stream the winner actually matched
+    seg = left[ph::fac][off:off + ln]
+    worst = max(abs(v - SCALE * round(v / SCALE)) for v in seg)
+    ck(worst <= 2,
+       f"every matched sample is within {worst} of a multiple of {SCALE} -- so "
+       f"`signal = round(sample/{SCALE})` is a recovery and not a rounding")
+
+    print("--- THE CHIP, AS THIS MACHINE MODELS IT ---")
+    print(f"     nibbles played  {feed}")
+    print(f"     delta formula   {variant}")
+    print(f"     clamp           {bits}-bit accumulator "
+          f"{adpcm.clamp_bounds(bits)}")
+    print(f"     accumulator at PLAY  {init}")
+    print(f"     matched {ln:,} consecutive samples, "
+          f"capture decimated {fac}x at phase {ph}")
+    print(f"     chip stream rate = {rate}/{fac} = {rate/fac:,.1f} Hz, and the "
+          f"channel delivered {len(data):,} B")
+
+    # ---- THE NEGATIVE CONTROLS.  Flip one axis alone; the match must die.
+    # Without these an axis the probe is BLIND to reads exactly like an axis it
+    # has settled, which is 58.3's vacuous-counter trap in a new place.
+    print("--- and every axis was actually asked (flip one, the match dies) ---")
+    rec = [round(v / SCALE) for v in left[ph::fac]]
+    flips = {"feed":    [f for f in FEEDS if f != feed],
+             "formula": ["terms" if variant == "shift" else "shift"],
+             "clamp":   [12 if bits == 10 else 10],
+             "init":    [0 if init == -2 else -2]}
+    for name, alts in flips.items():
+        worst_axis = None
+        for a in alts:
+            m = dict(feed=feed, variant=variant, bits=bits, init=init)
+            m[{"feed": "feed", "formula": "variant",
+               "clamp": "bits", "init": "init"}[name]] = a
+            want = adpcm.decode([0] * extra + nibbles_for(data, m["feed"]),
+                                m["variant"], init=m["init"], bits=m["bits"])
+            got = rec[off:off + len(want)]
+            d = sum(1 for x, y in zip(got, want) if x != y)
+            worst_axis = d if worst_axis is None else min(worst_axis, d)
+        ck(worst_axis > 0,
+           f"{name:8s} flipped: the closest alternative still disagrees on "
+           f"{worst_axis:,} of {ln:,} samples")
+
+    print("--- against what tools/encoder/adpcm.py DEFAULTS to ---")
+    cur = {"feed": "both-high-first", "formula": "shift", "clamp": 12, "init": 0}
+    got = {"feed": feed, "formula": variant, "clamp": bits, "init": init}
+    for k in cur:
+        print(f"     {k:8s} encoder {str(cur[k]):<15s} chip {str(got[k]):<15s}"
+              f" {'agree' if cur[k] == got[k] else 'DISAGREE'}")
+
+    json.dump({"feed": feed, "variant": variant, "bits": bits, "init": init,
+               "decimation": fac, "phase": ph, "extra": extra,
+               "matched": ln, "chip_rate": rate / fac},
+              open("tmp/adpcm_model.json", "w"))
+    print("ADPCM CHIP GATE " + ("GREEN" if not fails else f"RED: {len(fails)} failed"))
+    return 1 if fails else 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

tools/bench/verify_load.py

@@ -0,0 +1,73 @@
+#!/usr/bin/env python3
+"""Check the 68000's load-time output against tools/bench/dlxload.py, byte for byte.
+
+    python3 tools/bench/verify_load.py <in.dlx> [--out tmp/load]
+
+The 68000 ran src/player/load.i over the RAW container header; tools/bench/
+load.lua read the results back out of emulated RAM and out of the PALETTE
+REGISTERS.  This compares them with what the host-side transforms produce.
+
+Byte-for-byte and not "close enough", for both halves:
+
+  * the codebooks are indices, so a single wrong byte is a wrong COLOUR in
+    every block that uses that codeword, in every frame of the scene.
+  * the palette words carry the shared LSB the encoder's 1.96 dB (FINDINGS
+    23.3) depends on, and a wrong choice of it is invisible in a diff of the
+    picture's SHAPE -- it is a slightly wrong colour, which is exactly the sort
+    of thing that gets attributed to the codec.
+
+The darkest-entry index is checked too: it is what the letterbox is filled
+with until the encoder reserves a black entry (23.4, still open), and it comes
+out of an argmin whose tie-break has to match numpy's -- first index wins.
+"""
+import sys, argparse
+sys.path.insert(0, "tools/encoder")
+sys.path.insert(0, "tools/bench")
+from dlx import DLX
+import dlxload as DL
+
+ap = argparse.ArgumentParser()
+ap.add_argument("container")
+ap.add_argument("--out", default="tmp/load")
+ap.add_argument("--log", default="tmp/load_check.log",
+                help="the rig's log, for the DARK= line it printed")
+a = ap.parse_args()
+
+d = DLX(a.container)
+cb1, cb4 = DL.expand_codebooks(d)
+palb, dark, _ = DL.pack_palette(d)
+want = cb1.tobytes() + cb4.tobytes() + palb.tobytes()
+got = open(a.out + "_out.bin", "rb").read()
+
+if len(got) != len(want):
+    sys.exit(f"FAIL: the 68000 produced {len(got)} B, expected {len(want)}")
+
+n1, n4 = cb1.nbytes, cb4.nbytes
+sections = (("CB1", 0, n1), ("CB4", n1, n1 + n4), ("palette", n1 + n4, len(want)))
+bad = 0
+for name, lo, hi in sections:
+    diff = [i for i in range(lo, hi) if got[i] != want[i]]
+    if diff:
+        bad += len(diff)
+        i = diff[0]
+        print(f"FAIL: {name}: {len(diff)}/{hi-lo} bytes differ; first at "
+              f"+{i-lo} (68000 {got[i]:#04x}, dlxload {want[i]:#04x})")
+    else:
+        print(f"  OK  {name}: {hi-lo} B identical to dlxload.py")
+
+# The rig prints the index the 68000 chose; parse it rather than re-deriving,
+# so a rig that failed to read LDARK cannot pass by silence.
+got_dark = None
+for line in open(a.log, "rb").read().decode("utf-8", "replace").splitlines():
+    if "DARK=" in line:
+        got_dark = int(line.split("DARK=")[1].split()[0].rstrip(","))
+if got_dark is None:
+    sys.exit("FAIL: the rig printed no DARK= line -- it did not reach the dump")
+if got_dark != dark:
+    sys.exit(f"FAIL: darkest palette entry: 68000 says {got_dark}, dlxload says {dark}")
+print(f"  OK  darkest entry {dark}, chosen by the same argmin tie-break")
+
+if bad:
+    sys.exit(f"FAIL: {bad} bytes differ in total")
+print(f"OK  the 68000 reproduced all {len(want)} B of load-time output exactly "
+      f"(P1 codebooks, P2 palette)")

tools/bench/verify_packed.py

@@ -0,0 +1,114 @@
+#!/usr/bin/env python3
+"""Is EVERY frame the packed player put on screen pixel-exact?  ROADMAP K3.
+
+    python3 tools/bench/verify_packed.py <in.dlxp> [--snap tmp/snap_packed]
+                                         [--map tmp/packed_snaps.csv]
+                                         [--min-frames N]
+
+WHY THIS CHECKS ALL OF THEM AND tools/bench/verify_decode.py CHECKS ONE.  The
+codec is temporally recursive: a SKIP block is a claim that the previous frame is
+still in GVRAM, so the last frame of a sequential run is only correct if every
+frame before it was, and one comparison audits 120.  A packed frame is a
+LITERAL -- 192 rows of picture and a whole new palette, written over whatever
+was there.  Frame 119 being right says nothing at all about frame 60.  The
+simplification that deleted the ring, the codebooks and the decoder also deleted
+the gate's free lunch, and this is the bill.
+
+WHAT IS COMPARED.  MAME's own screen, through MAME's own video code: the
+snapshot is what the display produced out of GVRAM and the palette REGISTERS.
+Nothing here re-implements the packed interleave -- that is deliberate and it is
+the same rule tools/bench/gvpack/verify_dlxp.py was built on, because a
+container round-trips against its own inverse whether or not its byte order is
+the one the hardware wants.  The reference is dlxp.render(i), which is the
+palette in the record applied to the indices in the record.
+
+THE LETTERBOX IS CHECKED TOO, and it is not padding.  The picture is 192 rows of
+a 256-row screen; the other 64 rows are STATIC SETUP the 68000 wrote once at
+scene start (packed.s pg_static) and the channel never touches again.  If they
+were wrong -- or if they decayed as the per-frame palette moved under them --
+the picture would still be pixel-exact and the screen would not be.  Index 255
+is black in every frame's palette by construction (vq.frame_palette), so this
+also gates that reservation across all 120 records.
+"""
+import argparse, csv, os, sys
+sys.path.insert(0, "tools/encoder")
+import numpy as np
+from PIL import Image
+from dlxp import DLXP
+
+ap = argparse.ArgumentParser()
+ap.add_argument("container")
+ap.add_argument("--snap", default="tmp/snap_packed")
+ap.add_argument("--map", default="tmp/packed_snaps.csv")
+ap.add_argument("--min-frames", type=int, default=1,
+                help="fail if fewer than this many frames were sampled -- a "
+                     "run that displayed nothing must not pass as a run with "
+                     "no mismatches in it")
+a = ap.parse_args()
+
+d = DLXP(a.container)
+if not d.has_palette:
+    # A --no-palette container leaves the palette registers holding whatever the
+    # scene setup put there, and this rig's player writes none -- so there is no
+    # reference for what the screen should show.  Say so rather than compare
+    # against an assumption.
+    sys.exit(f"{a.container} carries no palette; this gate has no reference "
+             f"for what the display should have produced.")
+
+with open(a.map) as fh:
+    pairs = [(r["snapshot"], int(r["frame"])) for r in csv.DictReader(fh)]
+if len(pairs) < a.min_frames:
+    print(f"FAIL 0. only {len(pairs)} frames were sampled, --min-frames is "
+          f"{a.min_frames}. A player whose write window never closed displays "
+          f"nothing, and an empty comparison is not a pass.")
+    sys.exit(1)
+
+SCRH, SCRW = 256, 256
+YOFF = (SCRH - d.H) // 2
+fails, checked = [], 0
+for name, fr in pairs:
+    path = f"{a.snap}/x68000/{name}.png"
+    if not os.path.exists(path):
+        fails.append(f"snapshot {name} (frame {fr}) is missing from {a.snap}")
+        continue
+    s = np.asarray(Image.open(path).convert("RGB")).astype(int)
+    if s.shape[:2] != (2 * SCRH, SCRW):
+        fails.append(f"frame {fr}: geometry {s.shape[1]}x{s.shape[0]}, "
+                     f"expected {SCRW}x{2*SCRH}")
+        continue
+    if not all(np.array_equal(s[i], s[i + 1]) for i in range(1, s.shape[0] - 1, 2)):
+        fails.append(f"frame {fr}: double-scan pairing (1,2),(3,4),... broken")
+        continue
+    g = s[0::2]
+    pal = d.palette_rgb(fr)
+    exp = np.empty((SCRH, SCRW, 3), int)
+    exp[:] = pal[255]                       # the letterbox, and the reservation
+    exp[YOFF:YOFF + d.H] = d.render(fr)
+    checked += 1
+    if np.array_equal(g, exp):
+        continue
+    bad = (g != exp).any(2)
+    by, bx = np.where(bad)
+    inpic = ((by >= YOFF) & (by < YOFF + d.H)).sum()
+    fails.append(f"frame {fr} (snapshot {name}): {bad.sum()} px differ "
+                 f"({inpic} in the picture, {bad.sum()-inpic} in the "
+                 f"letterbox), first at y={by[0]} x={bx[0]}, maxdiff "
+                 f"{abs(g-exp).max()}")
+
+for f in fails[:12]:
+    print("FAIL " + f)
+if len(fails) > 12:
+    print(f"FAIL ... and {len(fails)-12} more")
+if fails:
+    print(f"     {checked-len([f for f in fails])} of {len(pairs)} sampled "
+          f"frames compared clean")
+    sys.exit(1)
+lo, hi = min(f for _, f in pairs), max(f for _, f in pairs)
+print(f"OK  {checked} frames of {a.container} pixel-exact on the emulated "
+      f"68000, frames {lo}..{hi} of {d.nframes}")
+print(f"    every one of them a LITERAL: no decoder, no codebook, no ring. "
+      f"Screen {SCRW}x{SCRH}, picture {d.W}x{d.H} at y={YOFF}, letterbox on "
+      f"the reserved index 255.")
+print(f"    palette {'LAST' if d.palette_last else 'FIRST'} in the record, "
+      f"{d.pal_bytes} B, compared as the DISPLAY renders it (GRB555+I out of "
+      f"the palette registers)")

tools/bench/verify_packed_audio.py

@@ -0,0 +1,598 @@
+#!/usr/bin/env python3
+"""The container's own audio, READ BACK OFF THE SPEAKER.  ROADMAP P6c.
+
+    python3 tools/bench/verify_packed_audio.py <in.dlxp> <capture.wav> \
+            [counters.json] [--seek FRAME --iters N]
+
+WHY THIS READS THE CAPTURE AND NOT THE PLAYER'S COUNTERS.  src/player/packed.s
+reports how many lumps it armed and how many payload bytes it handed the chip,
+and every one of those numbers can be right while the sound is wrong.  Nothing
+in this format parses anything (FINDINGS 67.4): a lump fetched one sector out is
+not an error, it is 7,168 B of noise played at the right length; a payload one
+byte long is not an error either, it is a rate.  The only instrument that can
+tell those apart from a correct run is the stream the chip actually produced.
+
+WHAT IS CHECKED, and it is the whole scene rather than a sample of it:
+
+  1. every lump's payload, decoded with the FOUR AXES OUT OF THE CONTAINER'S OWN
+     HEADER (FINDINGS 66/67.3), appears in the capture SAMPLE-EXACT and in
+     order.  Not "close": the recursion is exact arithmetic and MAME's okim6258
+     puts `signal << 4` into a stream the machine routes to the speaker at gain
+     0.50, so a chip sample is `signal * 8` and recovering it is a division and
+     not a rounding.  The residual is asserted.
+
+  2. the PAYLOAD lengths are the accumulator's and not the lump's.  A player
+     that fed the chip the whole A*512 B lump runs 0.09% fast -- 1.25 s of
+     lip-sync over the game (67.2) -- and the difference between the two is 6.54
+     B a group, which is 13 samples.  So this is checked by LENGTH: lump k's run
+     of matched samples must be exactly 2*lump_bytes(k), and that alternates
+     14,322 / 14,324 rather than being 14,336 every time.
+
+  3. THE SEAMS, measured rather than assumed.  Channel 3 counts out at the end
+     of a lump and the chip has no FIFO and no starvation state -- it goes on
+     decoding whatever byte its data register still holds, alternating that
+     byte's low and high nibbles, until the CPU arms the next lump.  Those
+     samples are NOT silence, they are the recursion running on a repeated byte,
+     and the state they leave behind is what lump k+1 decodes from.  So the
+     search below carries the state across the seam and reports its LENGTH,
+     which is the audible cost of every design decision on the video path.
+
+  4. THE SEEK, when the run made one (--seek).  A branch is a frame index and
+     a DLXP2 group puts lump k in FRONT of its records, so the player has to
+     issue a second read and enter the lump `f mod F` frames in (FINDINGS 70.3).
+     The stream the chip should then have been fed is the container's own bytes
+     SPLICED -- everything, then everything from byte floor(f*hz/(2*fps)) on --
+     and the walk above accounts for it as one continuous run, which is the
+     check: a player that dropped the byte offset feeds a stream that starts up
+     to F frames early and NOTHING ABOUT IT IS AN ERROR.  It is a rate.
+
+     AND THE THING THE PLAYER CANNOT SEE.  An MSM6258 has no seek: its
+     accumulator and step index are the product of every nibble since PLAY, so a
+     seek hands it bytes the encoder chose for a state it is not in.  The
+     samples are then wrong while the recursion re-converges, and they are wrong
+     WITHOUT ANY BYTE BEING WRONG -- every counter in the player stays right and
+     the walk above still passes, because the walk tracks the chip rather than
+     the intent.  This measures it: the chip's own samples across the splice
+     against the samples the ENCODER meant, which are the same bytes decoded
+     from the state a continuous play would have been in.
+
+THE NEGATIVE CONTROL IS BUILT IN.  A seam is found by searching for the repeat
+count that makes the next lump match; if the player had fed the wrong bytes, no
+repeat count would make it match and the run fails rather than sliding.
+"""
+import json, os, struct, sys, wave
+
+sys.path.insert(0, os.path.join(os.path.dirname(os.path.abspath(__file__)),
+                                "..", "encoder"))
+import adpcm
+from dlxp import DLXP, lump_bytes as _lump_bytes
+
+
+def dlxp_lump_bytes(d, k):
+    """What the CADENCE gives lump k, before the stream's end
+    truncates it -- so a lump that is short because the scene ran
+    out can be told from one that is short because the remainder
+    arithmetic said so."""
+    return _lump_bytes(k, d.cad_f, d.fps, d.aud_hz)
+
+SCALE = 8            # okim6258's `signal << 4` into a 32768 stream, times the
+                     # machine's 0.50 speaker route.  verify_adpcm_chip.py's.
+RESID = 2            # counts of slack on that recovery, as 66 measured it
+LOOK = 32            # samples of continuation a candidate run has to survive
+MAXRUN = 60000       # nibbles one delivered byte may be stretched over: 3.8 s
+                     # at 15,625 Hz, far past any seam a working player makes.
+                     # A bound is what makes a failure say "not found" rather
+                     # than run until the host is bored.
+LEAD = 400000        # samples of silence before PLAY
+
+
+def stepper(dec):
+    """One sample of the recursion, as a closure over the container's own four
+    axes.  adpcm.decode_state is the same arithmetic and is what the whole-lump
+    paths use; this exists because the walk below needs it ONE NIBBLE AT A TIME
+    and a function call per sample over 78,125 bytes is the difference between
+    a gate that runs in seconds and one that does not."""
+    lo, hi = adpcm.clamp_bounds(dec["bits"])
+    variant, step, adj = dec["variant"], adpcm.STEP, adpcm.INDEX_ADJUST
+
+    def one(st, n):
+        sig, idx = st
+        sig += adpcm.delta(n, step[idx], variant)
+        sig = lo if sig < lo else (hi if sig > hi else sig)
+        idx += adj[n & 7]
+        idx = 0 if idx < 0 else (48 if idx > 48 else idx)
+        return sig, (sig, idx)
+    return one
+
+
+def walk(rec, pos, st, data, dec):
+    """Account for EVERY byte the player handed the chip, in order.
+
+    WHY A WALK AND NOT A COMPARISON.  A whole-stream `decode(data) == capture`
+    is the check this wanted to be and it does not survive contact with the
+    machine.  The MSM6258 has no FIFO and no handshake at all: the DMA channel
+    writes a byte into the data register whenever #DRQ3 asks, and the chip
+    decodes a nibble out of whatever is in there on every sample tick.  Those
+    are two clocks -- 7,812.5 B/s and 15,625 Hz -- and MAME's okim6258 data_w
+    RESETS the nibble select on every write.  So a byte is normally played as
+    two nibbles, and near a boundary it can be played as one (the high nibble
+    dropped) or as three or more (the low/high pair repeated) purely from where
+    the write lands inside a sound-stream slice.
+
+    THE MODEL IS THEREFORE ONE LINE: byte b was played as `c` nibbles taken from
+    the cycle (b&15, b>>4), c >= 1.  This walk finds `c` for every byte, and the
+    HISTOGRAM of c is the result -- c=2 everywhere is a chip being fed exactly
+    at its own rate, and every c>2 is the chip replaying a byte while the 68000
+    was somewhere else, which is what a SEAM is.
+
+    It is not a loose check.  Every one of the `c` samples has to be exactly
+    right, the run has to be followed by LOOK samples that are exactly right,
+    and a byte the player never sent leaves no c at all.  Returns
+    (fail_index, pos, runs); fail_index is None on success.
+    """
+    one = stepper(dec)
+    runs = []
+    n = len(data)
+    pos_ = pos
+    for i in range(n):
+        b = data[i]
+        pair = (b & 15, b >> 4)
+        nxt = data[i + 1] & 15 if i + 1 < n else None
+        # DEEP FIRST, THEN SHALLOW.  LOOK samples of continuation is what tells
+        # a real seam from a coincidence -- a repeated pair can agree with the
+        # next lump's first nibble for one sample and does not for thirty-two.
+        # But the window is 16 bytes wide and a SECOND stretched byte inside it
+        # rejects the true answer as well as the false ones, so a byte that no
+        # candidate survives is retried with a shallow window rather than
+        # failing -- and the retry window is TWO samples, the next byte's own
+        # pair, because a four-sample one reaches into the stretched byte
+        # itself.  The cost of resolving it wrong is a failed walk, not a pass.
+        got = _pick(rec, pos_, st, pair, nxt, data, i, one, LOOK)
+        if got is None:
+            got = _pick(rec, pos_, st, pair, nxt, data, i, one, 2)
+        if got is None:
+            return i, pos_, runs
+        c, st = got
+        pos_ += c
+        runs.append(c)
+    return None, pos_, runs
+
+
+def _pick(rec, pos, st, pair, nxt, data, i, one, look):
+    """The run length for one byte: every candidate `c` whose samples are exact
+    and whose continuation survives `look`, resolved to c=2 where c=2 is one of
+    them.  Two nibbles a byte is what the two clocks agree on; anything else is
+    an event and an event needs the evidence, which is what `look` is."""
+    cands = []
+    s2, c = st, 0
+    while c < MAXRUN and pos + c < len(rec):
+        smp, s3 = one(s2, pair[c & 1])
+        if rec[pos + c] != smp:
+            break
+        c += 1
+        s2 = s3
+        if nxt is None:
+            cands.append((c, s2))
+            break
+        smp2, _ = one(s2, nxt)
+        if pos + c < len(rec) and rec[pos + c] == smp2 \
+           and _look(rec, pos + c, s2, data, i + 1, one, look):
+            cands.append((c, s2))
+            if c > 2 and len(cands) >= 2:
+                break
+    if not cands:
+        return None
+    for cc in cands:
+        if cc[0] == 2:
+            return cc
+    return cands[0]
+
+
+def _look(rec, pos, st, data, i, one, look=LOOK):
+    """`look` samples of continuation, assuming two nibbles a byte from here.
+
+    This is what tells a real seam from a coincidence.  At a lump boundary the
+    repeated pair can happen to agree with the next lump's first nibble for one
+    sample; it does not go on agreeing for thirty-two.
+    """
+    n, k = len(data), 0
+    while k < look and i < n:
+        b = data[i]
+        for nib in (b & 15, b >> 4):
+            if pos >= len(rec):
+                return True
+            smp, st = one(st, nib)
+            if rec[pos] != smp:
+                return False
+            pos += 1
+            k += 1
+        i += 1
+    return True
+
+
+def stream_pos(d, frame):
+    """The byte of the ADPCM stream frame `frame` starts at.  Exact, and the
+    exactness is the same one FINDINGS 67.2 is about: 15,625 samples a second
+    over 24 half-frames does not divide, and a seek that rounded it would be a
+    rate error rather than a byte error."""
+    return frame * d.aud_hz // (2 * d.fps)
+
+
+def spliced(d, seek, iters):
+    """The stream the player should have fed, as (label, bytes) chunks.
+
+    One pass is the container's lumps in order.  A pass after a SEEK starts at
+    stream byte stream_pos(seek): the tail of lump k = seek//F, then every lump
+    after it.  The chunks are kept separate because their BOUNDARIES are what
+    the stretched-byte assertion is measured against -- the chip replays its
+    last byte at a lump end and at a seek, and nowhere else.
+    """
+    out = [(f"lump {k}", d.lump(k)) for k in range(d.n_lumps)]
+    if iters <= 1 or seek is None:
+        return out
+    b0 = stream_pos(d, seek)
+    for it in range(1, iters):
+        off = 0
+        for k in range(d.n_lumps):
+            lb = d.lump(k)
+            if off + len(lb) <= b0:
+                off += len(lb)
+                continue
+            cut = max(0, b0 - off)
+            out.append((f"pass {it+1} lump {k}" + (f" +{cut} B" if cut else ""),
+                        lb[cut:]))
+            off += len(lb)
+    return out
+
+
+def walk_reset(rec, start, data, dec, sp):
+    """The walk, SPLIT AT A CHIP RESET.
+
+    A STOP/PLAY at the branch is a discontinuity the continuous model cannot
+    cross: the accumulator, the step index and the nibble select all go back to
+    the container's own start state, so no repeat count of the last byte of
+    pass 1 can be followed by pass 2's first sample.  The walk would report
+    "byte 78,124 does not account for capture sample N", which is true and is
+    the wrong complaint.
+
+    So pass 1 is walked without its final byte, the RESTART is searched for --
+    the first capture position at which pass 2 walks cleanly from `init` -- and
+    pass 2 is walked from there.  The final byte's run length is then the gap,
+    which is exactly what it is: the chip replaying it while the 68000 fetched
+    a lump and stopped the chip.
+
+    The search is cheap because the first post-reset sample is DETERMINED: one
+    nibble of a known byte from a known state.  Only positions carrying that
+    value are tried at all.
+    """
+    one = stepper(dec)
+    bad, pos, runs = walk(rec, start, (dec["init"], 0), data[:sp - 1], dec)
+    if bad is not None:
+        return bad, pos, runs, None
+    first, _ = one((dec["init"], 0), data[sp] & 15)
+    K = 64                                  # bytes of pass 2 a candidate must
+    for p in range(pos, min(pos + MAXRUN, len(rec))):   # survive before the
+        if rec[p] != first:                             # full walk is run
+            continue
+        b2, _, _ = walk(rec, p, (dec["init"], 0), data[sp:sp + K], dec)
+        if b2 is None:
+            b3, pos3, runs3 = walk(rec, p, (dec["init"], 0), data[sp:], dec)
+            if b3 is None:
+                return None, pos3, runs + [p - pos] + runs3, p
+    return sp - 1, pos, runs, None
+
+
+def main():
+    argv = [a for a in sys.argv[1:]]
+    seek, iters = None, 1
+    while "--seek" in argv:
+        i = argv.index("--seek"); seek = int(argv[i+1]); del argv[i:i+2]
+    while "--iters" in argv:
+        i = argv.index("--iters"); iters = int(argv[i+1]); del argv[i:i+2]
+    if len(argv) < 2:
+        sys.exit(__doc__)
+    sys.argv = [sys.argv[0]] + argv
+    d = DLXP(sys.argv[1])
+    counters = json.load(open(sys.argv[3])) if len(sys.argv) > 3 else None
+    if not d.has_audio:
+        sys.exit("this container is silent -- there is nothing to have heard")
+
+    dec = d.decoder()
+    fails = []
+    def ck(ok, msg):
+        print(("OK   " if ok else "FAIL ") + msg)
+        if not ok:
+            fails.append(msg)
+
+    w = wave.open(sys.argv[2])
+    n, ch, rate = w.getnframes(), w.getnchannels(), w.getframerate()
+    s = struct.unpack("<%dh" % (n * ch), w.readframes(n))
+    left, right = s[0::ch], s[1::ch]
+    ck(rate == d.aud_hz,
+       f"the capture is at {rate:,} Hz and the chip's stream is {d.aud_hz:,} -- "
+       f"equal rates are what keep MAME's resampler out of the measurement")
+    ck(list(left) == list(right), "both speakers carry the same samples (pan 00)")
+    ck(any(left), "the capture contains a signal at all")
+    if not any(left) or rate != d.aud_hz:
+        return 1
+    rec = [round(v / SCALE) for v in left]
+    start = next(i for i, v in enumerate(rec) if v)
+    ck(start < LEAD, f"the chip starts playing {start/rate:.2f} s in")
+
+    print(f"--- {sys.argv[1]}: {d.n_lumps} lumps, decoder {dec['variant']}/"
+          f"{dec['order']}, {dec['bits']}-bit clamp, accumulator {dec['init']} "
+          f"at PLAY -- ALL FOUR out of the header (67.3)")
+
+    # ---- THE STREAM, ACCOUNTED FOR BYTE BY BYTE.  d.audio() is the container's
+    # own lumps reassembled BY PAYLOAD -- 67.2's accumulator, not the padded
+    # sector runs -- so a player that fed the chip whole lumps does not merely
+    # score worse here, it fails to walk: the 6.54 B of zero at the end of a
+    # lump are nibbles that are not in this stream.
+    chunks = spliced(d, seek, iters)
+    data = b"".join(c for _, c in chunks)
+    if seek is not None and iters > 1:
+        print(f"--- THE SEEK: {iters} passes, passes 2..{iters} start at frame "
+              f"{seek} = stream byte {stream_pos(d, seek):,}, which is "
+              f"{seek % d.cad_f} frame(s) into lump {seek // d.cad_f} "
+              f"(FINDINGS 70.3). The stream below is SPLICED and is "
+              f"{len(data):,} B against the container's {d.aud_bytes:,}.")
+    replay = bool(counters and counters.get("arst"))
+    if replay and seek is not None and iters > 1:
+        sp0 = sum(len(c) for _, c in chunks[:d.n_lumps])
+        bad, pos, runs, rp = walk_reset(rec, start, data, dec, sp0)
+        if bad is None:
+            print(f"     the chip was STOPPED and re-PLAYED at the branch: the "
+                  f"walk is SPLIT there, and pass 2 restarts at capture sample "
+                  f"{rp:,}, {runs[sp0-1]:,} samples after the last byte of pass "
+                  f"1 was handed over ({runs[sp0-1]/rate*1000:.0f} ms of "
+                  f"replayed byte and stopped chip)")
+    else:
+        bad, pos, runs = walk(rec, start, (dec["init"], 0), data, dec)
+    ck(bad is None,
+       f"all {len(data):,} bytes of the container's audio reached the chip, in "
+       f"order, and every sample the chip produced from them is exact"
+       + ("" if bad is None else f" -- byte {bad:,} of {len(data):,} does not "
+          f"account for capture sample {pos:,}"))
+    if bad is not None:
+        k = bad * 2 * d.fps // (d.cad_f * d.aud_hz)
+        print(f"     that is inside lump {k}, {bad - sum(len(d.lump(j)) for j in range(k)):,} B in")
+        return 1
+
+    matched = pos - start
+    worst = max(abs(v - SCALE * round(v / SCALE))
+                for v in left[start:start + matched])
+    ck(worst <= RESID,
+       f"every one of {matched:,} matched samples is within {worst} of a "
+       f"multiple of {SCALE} -- so `signal = sample/{SCALE}` recovers the chip's "
+       f"own stream rather than rounding to the nearest story")
+
+    # ---- 1. THE FEED.  c=2 is a chip being fed at exactly its own rate.
+    hist = {}
+    for c in runs:
+        hist[c] = hist.get(c, 0) + 1
+    two = hist.get(2, 0)
+    ck(two * 1000 >= len(runs) * 999,
+       f"{two:,} of {len(runs):,} bytes ({two*100/len(runs):.3f}%) were played "
+       f"as exactly two nibbles -- the chip was paced by its own #DRQ3 and not "
+       f"by the CPU")
+    print(f"     nibbles per delivered byte: "
+          + ", ".join(f"{c}x{v:,}" for c, v in sorted(hist.items())))
+
+    # ---- 2. THE SEAMS, which is what every c > 2 is.  A lump's channel counts
+    # out and the chip goes on replaying the last byte until the CPU arms the
+    # next one; the excess nibbles ARE that interval, measured in the only place
+    # it exists, which is the sound.
+    seams = [(i, c - 2) for i, c in enumerate(runs) if c > 2]
+    print(f"--- THE SEAMS: {len(seams)} byte(s) were stretched, out of "
+          f"{d.n_lumps - 1} lump boundaries")
+    if seams:
+        ex = sum(c for _, c in seams)
+        print(f"     worst {max(c for _, c in seams)} samples = "
+              f"{max(c for _, c in seams)/rate*1000:.2f} ms; total {ex} samples "
+              f"= {ex/rate*1000:.2f} ms of replayed byte over "
+              f"{matched/rate:.2f} s of audio ({ex*100/matched:.4f}%)")
+    # and every stretched byte must BE a lump boundary -- a stretch anywhere
+    # else is the CPU losing the chip in the middle of a buffer.
+    ends = set()
+    off = 0
+    for _, c in chunks:
+        off += len(c)
+        ends.add(off - 1)
+    stray = [i for i, _ in seams if i not in ends]
+    ck(not stray,
+       f"every stretched byte is the LAST byte of a lump ({len(stray)} were not)"
+       + ("" if not stray else f" -- first at byte {stray[0]:,}, which is the "
+          f"chip running dry in the middle of a buffer"))
+
+    # ---- 3. THE PAYLOAD IS THE ACCUMULATOR'S (FINDINGS 67.2).  The walk
+    # already proves it -- a whole-lump player's stream contains the padding and
+    # would not walk -- so what is left is to price what was avoided.  The LAST
+    # lump is left out: 120 frames is not a multiple of F=11, so it carries ten
+    # frames of audio and is short for an arithmetic reason and not a rate one.
+    full = [k for k in range(d.n_lumps)
+            if len(d.lump(k)) == dlxp_lump_bytes(d, k)]
+    ck(len(full) >= d.n_lumps - 1,
+       f"{len(full)} of {d.n_lumps} lumps carry a whole group of audio")
+    if len(full) > 1:
+        pad = d.cad_a * 512
+        got = sum(len(d.lump(k)) for k in full)
+        over = pad * len(full) - got
+        secs = got * 2 / rate
+        print(f"--- THE PADDING IS DRIFT (FINDINGS 67.2), over the {len(full)} "
+              f"lumps that carry a whole group")
+        print(f"     payload {got:,} B against {pad*len(full):,} B of lump "
+              f"space: {over:,} B more, {over*100/got:.3f}%, "
+              f"{over*2/rate*1000:.2f} ms over {secs:.2f} s of audio")
+        print(f"     -> {over*2/rate/secs*22.8*60:.2f} s of lip-sync over the "
+              f"game's 22.8 min, and the accumulator in pg_apay is the three "
+              f"lines that do not spend it")
+
+    # ---- 4. WHAT A SEEK COSTS THE CHIP, and it is the thing no counter in the
+    # player can reach.  The walk above proves every BYTE arrived; this asks
+    # whether the SAMPLES the chip made out of them are the ones the encoder
+    # meant.  They are not, and they cannot be: the MSM6258's accumulator is a
+    # pure integrator with NO LEAKAGE TERM, so a state mismatch at a branch is a
+    # DC offset that does not decay -- it is not a transient with a time
+    # constant, and calling it one would be the flattering reading.
+    #
+    # THE REFERENCE IS THE ENCODER'S OWN STATE, not a fresh one.  adpcm.py
+    # encoded the stream in one pass, so the state it chose byte B(f)'s nibbles
+    # for is the state a CONTINUOUS play reaches at B(f).  The FRESH control is
+    # the other design -- STOP the chip and PLAY it again at the branch, which
+    # is what DLX_PK_ARST does -- and the two numbers are what choose between
+    # them.  Under --replay the fresh series is not a control at all: it is the
+    # prediction, and it has to be sample-exact.
+    if seek is not None and iters > 1 and bad is None:
+        one = stepper(dec)
+        sp = sum(len(c) for _, c in chunks[:d.n_lumps])
+        b0 = stream_pos(d, seek)
+        whole = d.audio()
+
+        def run_from(st, i0, nmax):
+            """The samples the chip WOULD have made from byte i0 on, had it been
+            in state `st` -- with the run lengths it actually used, so the two
+            series are sample-aligned across a seam as well as a byte."""
+            out, i = [], i0
+            while i < len(data) and len(out) < nmax:
+                b, c = data[i], runs[i]
+                for j in range(c):
+                    smp, st = one(st, (b & 15) if j % 2 == 0 else (b >> 4))
+                    out.append(smp)
+                i += 1
+            return out
+
+        if replay:
+            st = (dec["init"], 0)                  # by construction: PLAY sets
+        else:                                      # both, and the run above
+            st = (dec["init"], 0)                  # proved it sample-exact
+            for i in range(sp):                    # ...otherwise the chip is
+                b, c = data[i], runs[i]            # wherever the previous
+                for j in range(c):                 # scene's audio left it,
+                    _, st = one(st, (b & 15) if j % 2 == 0 else (b >> 4))
+        pos0 = start + sum(runs[:sp])
+        N = min(4 * rate, len(rec) - pos0)
+
+        ref = (dec["init"], 0)                     # ...and the ENCODER's state
+        for by in whole[:b0]:                      # at the SAME stream byte,
+            for nib in (by & 15, by >> 4):         # reached continuously
+                _, ref = one(ref, nib)
+
+        got = rec[pos0:pos0 + N]
+        want = run_from(ref, sp, N)
+        fresh = run_from((dec["init"], 0), sp, N)
+        n = min(len(got), len(want), len(fresh))
+        e = [got[i] - want[i] for i in range(n)]
+
+        def band(lo, hi):
+            seg = e[lo:min(hi, n)]
+            if not seg:
+                return None
+            m = sum(seg) / len(seg)
+            ac = (sum((x - m) ** 2 for x in seg) / len(seg)) ** 0.5
+            return m, ac, max(abs(x) for x in seg)
+
+        lo_c, hi_c = adpcm.clamp_bounds(dec["bits"])
+        print(f"--- THE PREDICTOR DOES NOT SEEK (FINDINGS 71). The chip's state "
+              f"at the branch is accumulator {st[0]}, step index {st[1]}; the "
+              f"encoder chose byte {b0:,}'s nibbles for accumulator {ref[0]}, "
+              f"step index {ref[1]}.")
+        print(f"     error against the encoder's intent, DC and AC separately "
+              f"(full scale is {hi_c}):")
+        for lo, hi, lab in [(0, 100, "0-6 ms"), (100, 1000, "6-64 ms"),
+                            (1000, 5000, "64-320 ms"), (5000, rate, "0.3-1.0 s"),
+                            (rate, 2*rate, "1-2 s"), (2*rate, 4*rate, "2-4 s")]:
+            r = band(lo, hi)
+            if r:
+                print(f"       {lab:>10}  DC {r[0]:8.1f}   AC {r[1]:7.2f}   "
+                      f"|max| {r[2]}")
+        if replay:
+            # THE STRONGEST FORM THIS CAN TAKE.  A STOP/PLAY puts the chip in a
+            # state this script knows exactly, so the prediction is not "close",
+            # it is every sample.  A run that claimed to reset and did not fails
+            # here and passes everything else on the page.
+            diff = [i for i in range(n) if got[i] != fresh[i]]
+            ck(not diff,
+               f"the chip was STOPPED and re-PLAYED at the branch, so all "
+               f"{n:,} post-seek samples are EXACTLY a decode from the "
+               f"container's own accumulator ({dec['init']}) and step index 0"
+               + ("" if not diff else f" -- {len(diff):,} differ, first at "
+                  f"sample {diff[0]}"))
+            dc = [x for x in e]
+            const = len(set(dc)) == 1
+            ck(const,
+               f"...and the whole error against the encoder's intent is the "
+               f"SINGLE CONSTANT {dc[0]}"
+               + ("" if const else f" -- it takes {len(set(dc))} values, so the "
+                  f"step indices differ too and this is distortion, not offset")
+               )
+            if const:
+                print(f"     -> a re-PLAYED branch costs a PERMANENT DC offset "
+                      f"of {dc[0]} = {abs(dc[0])*100/hi_c:.1f}% of full scale "
+                      f"and {abs(dc[0])*100/511:.1f}% of the 10-bit clamp's "
+                      f"headroom. It is inaudible as a tone and it is not free: "
+                      f"it is headroom, and it clicks once at the branch.")
+        else:
+            r0, r4 = band(0, 100), band(2*rate, 4*rate)
+            ck(band(1000, 5000)[1] < abs(band(1000, 5000)[0]),
+               f"the error at the branch is an OFFSET and not distortion: over "
+               f"64-320 ms its DC is {band(1000,5000)[0]:.1f} and its AC is "
+               f"{band(1000,5000)[1]:.2f}, so the chip is decoding the right "
+               f"shape from the wrong ground")
+            if r4:
+                print(f"     -> playing THROUGH the branch, the offset is still "
+                      f"{r4[0]:.0f} four seconds later ({abs(r4[0])*100/hi_c:.0f}%"
+                      f" of full scale). There is no leakage term in this "
+                      f"predictor; what decay there is comes from the signal's "
+                      f"own clamping, not from the recursion forgetting.")
+        # AND THE CONTROL THAT MAKES EITHER READING MEAN ANYTHING.
+        ck(any(want[i] != fresh[i] for i in range(n)),
+           f"the two references are distinguishable over these {n:,} samples, "
+           f"so 'carry the predictor' and 'reset it' are different runs and this "
+           f"measurement has a subject")
+
+    if counters:
+        ck(counters["late"] == 0,
+           f"the player never re-armed a channel that still had bytes to send "
+           f"({counters['late']} did)")
+        ck(counters["starve"] == 0,
+           f"the player never found the channel counted out with no lump ready "
+           f"({counters['starve']} times it did)")
+        ck(counters["bytes"] == len(data),
+           f"the player's own byte count ({counters['bytes']:,}) is the whole "
+           f"stream it should have fed ({len(data):,})")
+        if seek is not None and iters > 1:
+            # THE TWO CELLS, ASSERTED APART.  A player that kept one counter for
+            # "where the stream is" and "what the chip got" reports a number
+            # that is right for neither, and the symptom is a LAST LUMP that is
+            # long by the skip -- which is not an error, it is a rate.
+            want_skip = (iters - 1) * (stream_pos(d, seek)
+                                       - stream_pos(d, seek - seek % d.cad_f))
+            ck(counters.get("seekn") == iters - 1,
+               f"the player made {counters.get('seekn')} audio seek(s) for "
+               f"{iters-1} branch point(s) -- the second read FINDINGS 70.3 "
+               f"asks for, at a separate LBA")
+            ck(counters.get("seekb") == want_skip,
+               f"it skipped {counters.get('seekb')} B into the head of a lump "
+               f"and the cadence says {want_skip} -- the byte offset is what "
+               f"makes a branch land on its own frame instead of up to "
+               f"{d.cad_f-1} frames early")
+            ck(counters["pos"] == d.aud_bytes,
+               f"the stream POSITION ended at {counters['pos']:,} B, the "
+               f"container's whole stream ({d.aud_bytes:,}) -- while the chip "
+               f"was handed {counters['bytes']:,}. TWO CELLS FOR TWO FACTS: the "
+               f"position is where the container has got to and is what the "
+               f"last lump's length is measured against; the byte count is what "
+               f"the capture has to account for, and a seek moves one and not "
+               f"the other")
+        print(f"     the player: {counters['armed']} lumps armed, "
+              f"{counters['fetched']} fetched, {counters['serv']:,} service "
+              f"calls over {counters['shown']} frames "
+              f"({'BUS HELD' if counters['held'] else 'CYCLE STEALING'})")
+
+    print("PACKED AUDIO GATE " + ("GREEN" if not fails
+                                  else f"RED: {len(fails)} failed"))
+    return 1 if fails else 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

tools/bench/xfer_cost.sh

@@ -0,0 +1,112 @@
+#!/bin/bash
+# What the PIO transport costs the 68000, by subtraction (ROADMAP P4b,
+# FINDINGS 58.2).
+#
+#   tools/bench/xfer_cost.sh [ring_kb]
+#
+# THE MEASUREMENT IS AN A/B AND HAS TO BE.  Nothing in the machine can time
+# itself finely enough: src/player/clock.i counts V-DISP at 56.69 Hz and the
+# thing being priced is a per-BYTE cost.  So the same 120 frames are decoded
+# twice, from the same ring, by the same src/player/stream.s, with the same
+# src/player/ring.i placing every record -- and the ONLY difference is which
+# side of the XF_* mailbox answers:
+#
+#   model   tools/bench/stream.lua, an unlimited modelled pipe.  Bytes appear
+#           in the ring for free; the emulated time is decode plus ring_poll.
+#   scsi    src/player/xfer.i and src/player/scsi.i, a real MB89352 and a real
+#           volume.  The CPU moves every byte itself.
+#
+# Both runs are FREE-RUNNING (DLX_PACE=0). A paced run would measure the pace,
+# not the work, and the difference this script exists for would vanish into the
+# wait loop.
+#
+# WHAT THE SUBTRACTION IS HONEST ABOUT, and it is worth stating twice: this is a
+# count of 68000 CLOCKS, not a delivery rate. It says what the player pays to
+# move a byte with its own instruction stream. It says nothing about how fast
+# the medium can supply one -- MAME's device models are functional, not
+# transfer-timing accurate (docs/BENCHMARK.md, 42.5), and `W`, the clocks a DMAC
+# steals per delivered byte, is untouched by every line here.
+#
+# The two run boundaries are sampled on the machine-frame notifier, so each end
+# is up to 17.64 ms coarse (54.5). Against a difference of tens of seconds that
+# is under a tenth of a percent, and the analytic cross-check below is what
+# actually establishes the figure -- the subtraction only has to agree with it.
+set -e
+cd "$(dirname "$0")/../.."
+RING=${1:-256}
+command -v chdman > /dev/null || { echo "needs chdman (mame-tools)"; exit 2; }
+
+echo "=== modelled transport, unlimited pipe"
+DLX_PACE=0 DLX_RINGOWN=1 DLX_QDEPTH=2 bash tools/bench/pace_run.sh "$RING" 0 \
+  > tmp/xfer_cost_model.log 2>&1
+echo "=== real transport, a CZ-6BS1 and a real volume"
+DLX_PACE=0 DLX_RINGOWN=1 DLX_QDEPTH=2 DLX_XFER=scsi \
+  bash tools/bench/pace_run.sh "$RING" 0 > tmp/xfer_cost_scsi.log 2>&1
+
+python3 - <<'PY'
+import re, sys
+
+def one(tag, log):
+    t = open(log, "rb").read().decode("latin1")
+    m = re.search(r"decoded (\d+) frames in ([0-9.]+) s emulated", t)
+    if not m: sys.exit(f"{log}: no completion line -- the pass did not finish")
+    return int(m.group(1)), float(m.group(2)), t
+
+nfr, t_model, _  = one("model", "tmp/pace_r256_k0_p0_own.log")
+nfr2, t_scsi, ts = one("scsi",  "tmp/pace_r256_k0_p0_own_scsi.log")
+assert nfr == nfr2, "the two runs decoded different frame counts"
+nb = int(re.search(r"REAL TRANSPORT: \d+ READ\(10\)s by the 68000, (\d+) B", ts).group(1))
+nw = int(re.search(r"SECTOR OVERHEAD: (\d+) B off the disc", ts).group(1))
+
+CPUHZ = 10_000_000
+FPS   = 12
+budget = CPUHZ / FPS
+d = (t_scsi - t_model) * CPUHZ
+
+print(f"  {nfr} frames, {nb:,} B of record, {nw:,} B off the disc")
+print(f"  decode + ring_poll alone   {t_model:8.4f} s emulated  "
+      f"= {100*t_model*CPUHZ/nfr/budget:5.1f}% of a {FPS} fps frame")
+print(f"  ...with the real transport {t_scsi:8.4f} s emulated  "
+      f"= {100*t_scsi*CPUHZ/nfr/budget:5.1f}%")
+print(f"  TRANSPORT COST: {d:,.0f} clocks = {d/nb:.2f} per delivered byte, "
+      f"{d/nw:.2f} per byte off the FIFO")
+print(f"                  {d/nfr:,.0f} clk/frame = "
+      f"{100*d/nfr/budget:.1f}% of a {FPS} fps frame")
+
+# ---- THE CROSS-CHECK, and it is the half that makes the number portable.
+# If the measured cost is the 68000's own instruction stream, it must equal the
+# 68000's cycle table for the loop in src/player/scsi.i. If it does NOT, the
+# difference is time spent waiting on MAME's SPC model -- which is emulator
+# behaviour and would not survive contact with a board.
+LOOP = [("move.l #SC_PATIENCE,d3", 12), ("move.b SC_SSTS,d0  (xxx).L->Dn", 16),
+        ("btst #0,d0", 10), ("beq.s taken", 10),
+        ("move.b SC_DREG,(a1)+  (xxx).L->(An)+", 20),
+        ("subq.l #1,d7", 8), ("bne.s taken", 10)]
+loop = sum(c for _, c in LOOP)
+print(f"\n  the keep loop in src/player/scsi.i, from the 68000's cycle table:")
+for n, c in LOOP: print(f"    {c:3d}  {n}")
+pred = loop * nw / nb
+print(f"    ---  {loop} clk per byte off the FIFO, and the FIFO carries the "
+      f"dropped\n         window bytes too: {pred:.2f} per DELIVERED byte")
+err = 100 * abs(pred - d/nb) / (d/nb)
+print(f"  MEASURED {d/nb:.2f} vs PREDICTED {pred:.2f} -- {err:.1f}% apart, so "
+      f"the cost is the\n  instruction stream and not MAME's SPC: it is a "
+      f"figure a real board would also pay.")
+resid = d/nb - pred
+print(f"  residual {resid:+.2f} clk/B = {resid*nb/nfr:,.0f} clk/record of "
+      f"select, CDB, status,\n  message and xf_service -- the per-COMMAND cost, "
+      f"which is the part that does not\n  scale with the record.")
+
+print(f"\n  AGAINST THE LADDER, in the same units (clocks charged to the CPU "
+      f"per delivered\n  byte). W is the DMAC's steal, 52.5's bracket:")
+for W, name in [(5, "single address, bus HELD"), (9, "dual address, held"),
+                (12, "single address, arbitrated"),
+                (19, "dual address, arbitrated -- the IPL ROM's own disk ch")]:
+    print(f"    W = {W:2d}  {name:52s} {100*W*nb/nfr/budget:5.1f}% of the frame")
+print(f"    PIO   {d/nb:.0f}  this rig, measured                    "
+      f"                {100*d/nfr/budget:15.1f}%")
+print(f"\n  So the PIO transport is {d/nb/19:.1f}x the WORST DMA configuration "
+      f"this project has\n  found and {d/nb/5:.1f}x the best. P4a is not an "
+      f"optimisation of this; it is the\n  difference between a player and a "
+      f"slideshow.")
+PY

tools/encoder/adpcm.py

@@ -0,0 +1,186 @@
+#!/usr/bin/env python3
+"""MSM6258 (OKI/Dialogic) 4-bit ADPCM -- encoder, decoder, and the fact that
+there are TWO decoders and they are not the same one.
+
+The X68000's ADPCM is an OKI MSM6258V clocked at 8 MHz, dividing to 15,625 /
+10,417 / 7,812.5 samples a second, 4 bits each, two samples to a byte
+(FINDINGS 52, buscost.ADPCM_SAMPLE_HZ). The sample word is 12 bits signed.
+
+WHY THIS FILE HAS TWO DECODERS. Nothing in this repo can be trusted to say what
+the chip does, and the two references available on this machine DISAGREE:
+
+  VARIANT 'shift'  delta = ((2*(n&7) + 1) * step) >> 3
+                   This is ffmpeg's `adpcm_ima_oki`, and `gate_vs_ffmpeg()`
+                   reproduces it SAMPLE-EXACT, so it is not a reading of source
+                   code -- it is a measurement of the decoder that ships.
+
+  VARIANT 'terms'  delta = step/8 + (n&4 ? step : 0) + (n&2 ? step/2 : 0)
+                            + (n&1 ? step/4 : 0),  each term truncated
+                   This is the OKI datasheet's own form, the one an ADPCM chip
+                   can actually build out of shifts and adds, and it is what
+                   MAME's okim6258 is understood to compute. NOT VERIFIED HERE:
+                   no MAME source tree is on this machine (FINDINGS 64.4).
+
+They differ on 445 of 2,268 sampled nibbles, by up to 4 in 12-bit units --
+small, and small is not zero. Which one the machine runs is an open question
+with an experiment attached: MAME's x68000 HAS an okim6258, so it can be asked
+rather than argued about.
+
+Nibble order is HIGH NIBBLE FIRST within a byte -- measured, not assumed, by the
+same gate: reading low-first mismatches ffmpeg on 1,728 of 2,268 samples.
+"""
+
+# The 49-entry OKI step table. floor(16 * 1.1**k) for k in 0..48 -- built rather
+# than pasted, so a transcription slip is not one of the things that can be
+# wrong here.
+STEP = [int(16 * 1.1**k) for k in range(49)]
+
+# The nibble magnitude's effect on the step index. Four quiet nibbles walk it
+# down one, four loud ones walk it up by more.
+INDEX_ADJUST = (-1, -1, -1, -1, 2, 4, 6, 8)
+
+SAMPLE_MIN, SAMPLE_MAX = -2048, 2047     # the 12-bit DAC word
+
+VARIANTS = ("shift", "terms")
+
+# ---------------------------------------------------------------------------
+# THE THREE AXES THAT WERE FIXED CONSTANTS UNTIL SESSION 34, and every one of
+# them turned out to be a real choice that a decoder can get wrong.  FINDINGS 65
+# priced the `variant` axis at 25 dB and left the other three unnamed; MAME's
+# okim6258 disagrees with this file on ALL THREE, so they are parameters now and
+# tools/bench/adpcm_run.sh measures which values the emulated chip runs.
+#
+#   ORDER  which nibble of a byte is played FIRST.  'high' is the Dialogic VOX
+#          file convention and is what ffmpeg's adpcm_ima_oki reads, which is
+#          what 65.1 measured.  That is a fact about a FILE FORMAT.  What the
+#          chip does with a byte handed to its data register is a different
+#          question and MAME answers it 'low'.
+#   INIT   the accumulator at the instant the chip is told to PLAY.  This file
+#          started it at 0; MAME's okim6258 resets it to -2.
+#   BITS   where the accumulator CLAMPS.  This file clamped at the 12-bit ADPCM
+#          word; the MSM6258's own D/A is 10-bit and MAME clamps there, INSIDE
+#          the recursion, so it is not a post-hoc output scaling.
+#
+# Defaults are unchanged, so tools/bench/verify_adpcm.py still measures exactly
+# what it measured in session 33: ffmpeg's decoder, high nibble first.
+ORDERS = ("high", "low")
+
+# WHAT THE MACHINE'S OWN CHIP DOES, MEASURED -- tools/bench/adpcm_run.sh, one
+# model of sixteen reproducing 1,678 consecutive samples of a MAME capture
+# sample-exact, with a negative control on every axis (FINDINGS 66).  It is a
+# measurement of MAME's device model driven through the real transport, not of
+# an MSM6258; the silicon is still a hardware item.
+#
+# THE DEFAULTS ABOVE ARE DELIBERATELY *NOT* THESE.  The defaults are ffmpeg's
+# adpcm_ima_oki, because tools/bench/verify_adpcm.py's whole value is that it
+# checks this file against an independent implementation, and a default that
+# had drifted to match the thing under test would end that.  Anything that
+# ENCODES FOR THE MACHINE passes CHIP explicitly.
+CHIP = dict(variant="terms", order="low", bits=10, init=-2)
+
+
+def clamp_bounds(bits):
+    """The accumulator's clamp, as MAME's okim6258 computes it: max = 2^(b-1)-1,
+    min = -2^(b-1).  Note it is NOT symmetric, and the asymmetry is load-bearing
+    on a signal that saturates."""
+    return -(1 << (bits - 1)), (1 << (bits - 1)) - 1
+
+
+def delta(nibble, step, variant):
+    """The reconstruction step for one nibble, in 12-bit units."""
+    if variant == "shift":
+        d = ((2 * (nibble & 7) + 1) * step) >> 3
+    elif variant == "terms":
+        d = step // 8
+        if nibble & 4: d += step
+        if nibble & 2: d += step // 2
+        if nibble & 1: d += step // 4
+    else:
+        raise ValueError(f"unknown variant {variant!r}")
+    return -d if nibble & 8 else d
+
+
+def decode_state(nibbles, variant="shift", state=None, init=0, bits=12):
+    """The recursion, with its STATE in and out.  (samples, (signal, idx)).
+
+    `decode` is this with the state thrown away, and it is written this way
+    round rather than duplicated because a stream that STOPS and RESUMES is not
+    a hypothetical here: an MSM6258 fed by a DMA channel goes on decoding
+    whatever byte its data register still holds when the channel counts out
+    (MAME okim6258 sound_stream_update reads m_data_in unconditionally while
+    PLAYING), so the samples between one lump and the next are the recursion
+    continuing over a repeated byte.  tools/bench/verify_packed_audio.py has to
+    carry that state across the seam to check the lump on the far side of it,
+    and a second copy of the loop is a second place the clamp can drift.
+    """
+    lo, hi = clamp_bounds(bits)
+    signal, idx = state if state is not None else (init, 0)
+    out = []
+    for n in nibbles:
+        signal += delta(n, STEP[idx], variant)
+        signal = lo if signal < lo else (hi if signal > hi else signal)
+        idx += INDEX_ADJUST[n & 7]
+        idx = 0 if idx < 0 else (48 if idx > 48 else idx)
+        out.append(signal)
+    return out, (signal, idx)
+
+
+def decode(nibbles, variant="shift", init=0, bits=12):
+    """Nibbles -> signed samples. State is (signal, step index); the step index
+    is 0 at the start of a stream and `init` is where the accumulator starts."""
+    return decode_state(nibbles, variant, None, init, bits)[0]
+
+
+def encode(samples, variant="shift", init=0, bits=12):
+    """12-bit signed samples -> nibbles.
+
+    The nibble is chosen by EXHAUSTIVE SEARCH over all sixteen, minimising the
+    reconstruction error of this sample. That is greedy rather than optimal --
+    a nibble also moves the step index, so a locally worse choice can pay later
+    -- but it is what a chip-matched encoder is expected to do and it costs
+    nothing offline. The decoder is run INSIDE the loop, so the encoder can
+    never drift away from what the decoder will reconstruct.
+    """
+    lo, hi = clamp_bounds(bits)
+    signal, idx, out = init, 0, bytearray()
+    for s in samples:
+        step = STEP[idx]
+        best, best_err = 0, None
+        for n in range(16):
+            v = signal + delta(n, step, variant)
+            v = lo if v < lo else (hi if v > hi else v)
+            err = (v - s) ** 2
+            if best_err is None or err < best_err:
+                best, best_err = n, err
+        signal += delta(best, step, variant)
+        signal = lo if signal < lo else (hi if signal > hi else signal)
+        idx += INDEX_ADJUST[best & 7]
+        idx = 0 if idx < 0 else (48 if idx > 48 else idx)
+        out.append(best)
+    return bytes(out)
+
+
+def pack(nibbles, order="high"):
+    """Nibbles -> bytes. `order` names which nibble of a byte is played FIRST;
+    'high' is the VOX file convention. An odd count pads with a 0 nibble, which
+    is the quietest one the format has (delta = step/8)."""
+    if order not in ORDERS:
+        raise ValueError(f"unknown nibble order {order!r}")
+    n = bytes(nibbles)
+    if len(n) & 1:
+        n += b"\0"
+    if order == "high":
+        return bytes((n[i] << 4) | n[i + 1] for i in range(0, len(n), 2))
+    return bytes((n[i + 1] << 4) | n[i] for i in range(0, len(n), 2))
+
+
+def unpack(data, count=None, order="high"):
+    if order not in ORDERS:
+        raise ValueError(f"unknown nibble order {order!r}")
+    out = bytearray()
+    for b in data:
+        if order == "high":
+            out.append(b >> 4); out.append(b & 15)
+        else:
+            out.append(b & 15); out.append(b >> 4)
+    return bytes(out[:count] if count is not None else out)

tools/encoder/dlx.py

@@ -29,6 +29,13 @@ import spans as SP
 MODE_SKIP, MODE_V1, MODE_V4, MODE_RAW = 0, 1, 2, 3
 
 
+# A SCSI target answers in 512-byte blocks and a record is not a sector: on the
+# DLX4 gate container 117 of 120 records start part way into one.  DLX5 makes
+# the container agree with the medium instead of making the transport reconcile
+# them (tools/analysis/26_sector_align.py prices all three ways).
+SECTOR = 512
+
+
 class DLX:
     def __init__(self, path):
         self.raw = open(path, "rb").read()
@@ -38,14 +45,25 @@ class DLX:
         # (FINDINGS 28.3), so the padding is part of the format, not a loader
         # convenience -- but DLX1 containers stay readable, because every
         # measurement in FINDINGS 28-31 was taken on one.
-        if b[:4] not in (b"DLX1", b"DLX2", b"DLX3"):
+        if b[:4] not in (b"DLX1", b"DLX2", b"DLX3", b"DLX4", b"DLX5"):
             raise ValueError(f"{path}: not a DLX container")
         self.version = int(b[3:4])
         self.aligned = self.version >= 2
         self.has_spans = self.version >= 3
+        self.has_index = self.version >= 4
+        # DLX5: every record starts on a 512-BYTE SECTOR boundary, and so does
+        # the frame stream itself.  That is not a tidier version of DLX4's
+        # 4-byte rule -- it is what lets a DMA channel read a record as whole
+        # sectors straight into the ring, with no window and no bounce copy
+        # (FINDINGS 58.3 option C, and 59.4 made it a precondition: sc_in_data
+        # REFUSES a windowed read when the data phase is the channel's).
+        self.sector_aligned = self.version >= 5
+        self.rec_align = SECTOR if self.sector_aligned else (4 if self.aligned
+                                                             else 1)
         (self.W, self.H, self.fps, self.nframes,
          self.k1, self.k4) = struct.unpack(">HHHHHH", b[4:16])
         off_pal, off_cb1, off_cb4, off_frm = struct.unpack(">IIII", b[16:32])
+        off_idx = struct.unpack(">I", b[32:36])[0] if self.has_index else None
 
         self.pal = np.frombuffer(b, np.uint8, 256 * 3, off_pal).reshape(256, 3)
         self.cb1 = np.frombuffer(b, np.uint8, self.k1 * 16,
@@ -59,24 +77,69 @@ class DLX:
         self.mode_bytes = (self.nb * 2 + 7) // 8
 
         # frame directory: (offset of the mode header, payload length)
-        if self.aligned and off_frm % 4:
-            raise ValueError(f"{path}: DLX2 frame stream starts at {off_frm}, "
-                             f"which is not 4-byte aligned")
+        if off_frm % self.rec_align:
+            raise ValueError(f"{path}: DLX{self.version} frame stream starts at "
+                             f"{off_frm}, which is not {self.rec_align}-byte "
+                             f"aligned")
         self.frames = []
         p = off_frm
         for _ in range(self.nframes):
             (n,) = struct.unpack(">I", b[p:p + 4])
             self.frames.append((p + 4, n))
             p += 4 + n
-            if self.aligned:
-                p += -p % 4          # skip the pad to the next record
-        # The writer does not pad after the LAST record -- nothing follows it --
-        # so `p` may have advanced past the end by up to 3 bytes there.
+            p += -p % self.rec_align    # skip the pad to the next record
+        # DLX2/DLX3 do not pad after the LAST record -- nothing follows it --
+        # so `p` may have advanced past the end by up to 3 bytes there.  DLX4
+        # and DLX5 DO pad it, because a producer that trusts the index fetches
+        # a whole padded record for the last frame like any other.
         slack = len(b) - p
-        if not (slack == 0 or (self.aligned and -3 <= slack < 0)):
+        if not (slack == 0 or (self.aligned and not self.has_index
+                               and -(self.rec_align - 1) <= slack < 0)):
             raise ValueError(f"{path}: {slack} trailing bytes after "
                              f"{self.nframes} frames")
 
+        # DLX4's record index, and it is CHECKED rather than trusted.  The
+        # walk above is what every reader in this tree did before there was an
+        # index -- read a record's length word to find the next one -- and it is
+        # exactly what a player streaming off a disc cannot do, because the
+        # length word of record i+1 is one of the bytes it has not fetched.  So
+        # the two are computed independently here and required to agree: the
+        # index is the producer's only source of record geometry, and an index
+        # that disagrees with the stream places records at wrong addresses,
+        # which the block loop reads without a bounds check (49.2).
+        self.index = None
+        if self.has_index:
+            self.index = list(struct.unpack(
+                f">{self.nframes}H", b[off_idx:off_idx + 2 * self.nframes]))
+            walked = [self._padded(n) // 4 for _, n in self.frames]
+            if self.index != walked:
+                bad = next(i for i in range(self.nframes)
+                           if self.index[i] != walked[i])
+                raise ValueError(
+                    f"{path}: record index disagrees with the frame stream at "
+                    f"frame {bad}: index says {self.index[bad]} longwords, the "
+                    f"stream is {walked[bad]}")
+            if off_frm + 4 * sum(self.index) != len(b):
+                raise ValueError(
+                    f"{path}: the index accounts for "
+                    f"{off_frm + 4 * sum(self.index)} bytes and the file is "
+                    f"{len(b)} -- a producer trusting it would run off the end")
+
+    def _padded(self, n):
+        """Bytes one record of `n` payload bytes occupies, pad included."""
+        ln = 4 + n
+        return ln + (-ln % self.rec_align)
+
+    def record_lengths(self):
+        """Padded record lengths in BYTES, in stream order.
+
+        The one place the container's alignment rule is applied.  Every caller
+        that used to write `4 + n + (-(4+n) % 4)` was carrying its own copy of
+        that rule, which is exactly the kind of duplication that made DLX5 a
+        multi-file change instead of a one-line one.
+        """
+        return [self._padded(n) for _, n in self.frames]
+
     def modes(self, f):
         o, _ = self.frames[f]
         h = np.frombuffer(self.raw, np.uint8, self.mode_bytes, o)

tools/encoder/dlxp.py

@@ -0,0 +1,463 @@
+#!/usr/bin/env python3
+"""DLXP1 -- the PACKED container, and the one place its layout rule is written.
+
+    from dlxp import DLXP, pack_picture, unpack_picture, write
+
+THIS CONTAINER HAS NO DECODER.  That is the point of it (FINDINGS 61): a record
+is the bytes a DMA channel puts straight into the palette registers and GVRAM,
+so the "reference decoder" here is not a decoder at all -- it is an assertion
+about where each byte lands.  `dlx.py` exists because a 68000 has to PARSE the
+codec's container and can get it wrong; this file exists because a DMA channel
+must NOT have to parse anything, and the format is what makes that true.
+
+The layout, all of it fixed, all of it verified as a picture in FINDINGS 47.2:
+
+  * 256-colour GVRAM normally throws away the high byte of every word a CPU
+    writes, so a picture byte costs two disc bytes.  CRTC R20 bit 11 turns the
+    masking off (46.5/47.1), and with the two 256-colour pages scrolled apart by
+    384 one word carries TWO pixels: word `i` of a row is
+        (pix[y][i+128] << 8) | pix[y][i]
+    -- page 1 (X-scrolled 384, transparent top) shows columns 128..255, page 0
+    (unscrolled, opaque bottom) shows columns 0..127.
+  * so a row is 128 words = 256 BYTES for 256 pixels, and big-endian storage
+    makes the byte order `pix[128], pix[0], pix[129], pix[1], ...`.  That byte
+    order is not a serialisation choice: it is what the 68000's bus puts on the
+    high half of the word, and the channel copies bytes.
+  * 192 rows -> 49,152 B of picture, 1.0 B/pixel against the unpacked 2.0.
+  * index 0 is the TRANSPARENCY KEY and never appears; black is 255, which is
+    what the letterbox rows display (`vq.frame_palette`).
+  * words 128..511 of each row, and the letterbox rows themselves, are STATIC
+    SETUP -- written once at scene setup, never per frame -- so they are not in
+    the container.  FINDINGS 47.2 lists them; keeping them out is what makes the
+    per-frame payload exactly the picture.
+  * the GVRAM line stride is 1,024 B and a row is 256 B, so the container's rows
+    are CONTIGUOUS and the 1,024 B step is the channel's, walked by array
+    chaining from one start (FINDINGS 62).  A container that carried the stride
+    would be 4x the size and would say nothing extra.
+
+  header, 64 bytes, big-endian, then zero pad to the first sector:
+    0   'DLXP'
+    4   u16 version (2)
+    6   u16 flags        bit 0: a per-frame palette is present
+                         bit 1: the palette is at the END of the record
+                         bit 2: an audio stream is interleaved (DLXP2)
+    8   u16 width, u16 height
+    12  u16 fps,   u16 nframes
+    16  u32 record bytes         fixed, and a whole number of 512 B sectors
+    20  u32 palette bytes        512 (256 GRB555+I words), or 0
+    24  u32 picture bytes        49,152
+    28  u32 frames offset        offset of RECORD 0, lump 0 already skipped
+    -- DLXP2 adds, and every field is zero in a silent container:
+    32  u32 audio offset         offset of LUMP 0, 512 B, i.e. sector 1
+    36  u32 audio sample Hz      15,625 -- the chip's, not the disc's
+    40  u32 audio bytes          the ADPCM payload, padding NOT counted
+    44  u16 cadence F            frames between one lump and the next
+    46  u16 cadence A            whole sectors in a lump
+    48  u16 audio format         bit 0: LOW nibble first.  bit 1: the OKI
+                                 datasheet's per-term delta ('terms')
+    50  u16 audio clamp bits     where the accumulator saturates, 10 or 12
+    52  i16 audio init           the accumulator at PLAY, -2 on this chip
+    54  u16 reserved (0)
+    56  u32 reserved (0)
+    60  u32 reserved (0)
+
+  then, from sector 1, GROUPS: one audio lump of A sectors, then F records.
+  A record is 49,664 B = 97 sectors EXACTLY and a lump is A*512 B EXACTLY, so
+
+    record i  is at  off_frm + i*rec_bytes + (i//F)*A*512
+    lump   k  is at  off_aud + k*(F*rec_bytes + A*512)
+
+  -- still arithmetic, still no index, still nothing walked.
+
+WHY AUDIO IS A CADENCE AND NOT A FIELD IN THE RECORD (FINDINGS 65.3).  15,625
+samples a second, two to a byte, is 651.0416... B per 12 fps slot, and the dots
+are the whole problem: put slot i's audio in record i and records become
+VARIABLE LENGTH, which needs an index, which ends the format.  A fixed cadence
+of F frames per A sectors keeps `LBA(i)` arithmetic and pays instead in padding,
+and the padding is a rational-approximation problem whose answer is not the
+obvious cadence: F=1 wastes 57.3% of every audio sector and F=11, A=14 wastes
+0.09%.
+
+AND THE PADDING IS NOT WHERE THE BYTES ARE (FINDINGS 67).  A lump is A*512 B of
+SPACE and it does NOT carry A*512 B of audio: F frames need F*15625/24 B, which
+is 7,161.4583... at F=11, so a lump's PAYLOAD alternates 7,161 and 7,162 by the
+same remainder arithmetic FINDINGS 54's frame clock carries, and the rest of the
+sector is zero.  A player that fed the chip the whole lump would be handing it
+6.54 B a group it should not have -- 0.09% too much audio, which is not waste,
+it is DRIFT: 0.83 ms a group, 1.2 s of lip-sync over the game's 22 minutes.  So
+`lump_bytes(k)` below is the format, not a convenience, and a player computes it
+with one accumulator: `acc += F*aud_hz; n = acc // (2*fps); acc %= 2*fps`.
+
+THE FOUR ADPCM AXES ARE IN THE HEADER BECAUSE GETTING ONE WRONG COSTS 25 dB
+(FINDINGS 66).  Nibble order, delta formula, clamp width and the accumulator's
+value at PLAY are properties of the DECODER, and a container encoded for one
+decoder and played on another comes out with the noise louder than the signal.
+They are four fields rather than a version number so that a mismatch is legible
+in a hexdump rather than inferred from a container's age.
+
+THERE IS NO RECORD INDEX AND NO LENGTH WORD, and that is the difference DLX4's
+index was invented for (49.3): a codec record's length is content-dependent, so
+a producer cannot know where record i+1 starts without being told.  A packed
+record's length is GEOMETRY -- 192 rows of 256 B plus a palette -- so record `i`
+is at `off_frm + i * rec_bytes` and a seek is arithmetic.  Nothing in this
+format has to be walked, which is also why the packed player has no ring
+(ROADMAP K3): there is no variable-length thing to keep contiguous.
+
+THE PALETTE ORDER IS A DECISION, NOT AN ACCIDENT (FINDINGS 62.5).  Palette first
+or 193rd is visible on screen for one paint -- old rows under the new palette, or
+new rows under the old one -- and it is moot if buffer mode blanks the layer
+(47.4/B2).  It is a CONTAINER property here, chosen at encode time by
+`--palette-last` and recorded in flags bit 1, so K3 can measure both without a
+re-encode being an argument about which one the format assumed.
+"""
+import struct
+import numpy as np
+
+MAGIC = b"DLXP"
+VERSION = 2
+SECTOR = 512                 # same rule and the same reason as dlx.SECTOR
+PAL_BYTES = 512              # 256 entries, one GRB555+I word each
+HDR_BYTES = 64
+FLAG_PALETTE = 1 << 0
+FLAG_PALETTE_LAST = 1 << 1
+FLAG_AUDIO = 1 << 2
+
+# The two ADPCM axes that are booleans.  The other two -- the clamp width and
+# the accumulator at PLAY -- are numbers and get their own fields, because
+# encoding them as flags would mean this file deciding which values are legal.
+AFMT_ORDER_LOW = 1 << 0
+AFMT_VARIANT_TERMS = 1 << 1
+
+# The default cadence, and it is a MEASUREMENT rather than a taste (FINDINGS
+# 65.3): the sweep's floor is F=81 and costs 91,136 B more of player RAM for the
+# last 0.09 of a point of padding, on a machine where two record buffers already
+# want 99,328 B.
+CADENCE_F = 11
+
+
+def audio_format(variant, order, bits, init):
+    """adpcm.py's four axes -> the three header fields that carry them.
+
+    This file does not import adpcm.py and must not: a container format that
+    depended on an encoder would be a format that could not be read without one.
+    What it carries is the DESCRIPTION, and `tools/analysis/34_packed_audio.py`
+    is what checks the description against the encoder that wrote the bytes.
+    """
+    if variant not in ("shift", "terms") or order not in ("high", "low"):
+        raise ValueError(f"unknown ADPCM decoder ({variant!r}, {order!r})")
+    fmt = ((AFMT_ORDER_LOW if order == "low" else 0)
+           | (AFMT_VARIANT_TERMS if variant == "terms" else 0))
+    return fmt, int(bits), int(init)
+
+
+def audio_decoder(fmt, bits, init):
+    """The inverse: what a player, or a gate, has to run to hear the bytes."""
+    return dict(variant="terms" if fmt & AFMT_VARIANT_TERMS else "shift",
+                order="low" if fmt & AFMT_ORDER_LOW else "high",
+                bits=int(bits), init=int(init))
+
+
+def cadence(fps, hz, F=CADENCE_F):
+    """(F, A): F frames of audio rounded UP to whole sectors.
+
+    A*512 must cover F frames or the chip runs dry, so A is a ceiling and the
+    excess is padding the wire pays for and nothing plays.  Integer arithmetic
+    throughout: the whole point of 65.3 is that this ratio has a remainder, and
+    a float here would hide the case where it does not.
+    """
+    num, den = F * hz, 2 * fps                 # bytes per group = num/den
+    return F, -(-num // (den * SECTOR))
+
+
+def lump_bytes(k, F, fps, hz, total=None):
+    """The PAYLOAD of lump k -- what is handed to the chip, padding excluded.
+
+    Exact, and exactness is the finding (FINDINGS 67): floor((k+1)*F*hz/(2*fps))
+    - floor(k*F*hz/(2*fps)) alternates 7,161 and 7,162 at F=11, and a player
+    that fed the chip the whole A*512 B lump instead would run 0.09% fast --
+    1.2 s of lip-sync over 22 minutes.
+    """
+    den = 2 * fps
+    n = ((k + 1) * F * hz) // den - (k * F * hz) // den
+    if total is not None:                      # the last lump is short, not padded
+        n = max(0, min(n, total - (k * F * hz) // den))
+    return n
+
+
+def n_lumps(nframes, F):
+    return -(-nframes // F)
+
+
+def pack_picture(idx):
+    """(H,W) palette indices -> the bytes GVRAM wants, in GVRAM order.
+
+    The ONE place the interleave rule is applied, for the same reason
+    `dlx.record_lengths` is the one place the alignment rule is: every caller
+    that carries its own copy of a layout rule is a place the layout can drift.
+    """
+    H, W = idx.shape
+    if W % 2:
+        raise ValueError(f"packed layout needs an even width, got {W}")
+    half = W // 2
+    left, right = idx[:, :half], idx[:, half:]
+    out = np.empty((H, half, 2), np.uint8)
+    out[:, :, 0] = right          # high byte of the word -> page 1 -> col i+128
+    out[:, :, 1] = left           # low  byte             -> page 0 -> col i
+    return out.reshape(H, half * 2)
+
+
+def unpack_picture(buf, W, H):
+    """The inverse, and the assertion that `pack_picture` is reversible."""
+    b = np.frombuffer(buf, np.uint8, W * H).reshape(H, W // 2, 2)
+    idx = np.empty((H, W), np.uint8)
+    idx[:, W // 2:] = b[:, :, 0]
+    idx[:, :W // 2] = b[:, :, 1]
+    return idx
+
+
+def record_bytes(W, H, palette=True):
+    n = W * H + (PAL_BYTES if palette else 0)
+    if n % SECTOR:
+        raise ValueError(f"a {W}x{H} packed record is {n} B, which is not a "
+                         f"whole number of {SECTOR} B sectors")
+    return n
+
+
+def write(path, W, H, fps, frames, palette_last=False, audio=None):
+    """`frames` is a sequence of (palette_words_bytes | None, picture_bytes).
+
+    `audio`, when given, is a dict: `data` the packed ADPCM bytes, `hz` the
+    chip's sample rate, `variant`/`order`/`bits`/`init` the four axes FINDINGS
+    66 measured, and optionally `F`.  The stream is CUT UP HERE and nowhere
+    else, by `lump_bytes`, for the same reason `pack_picture` is the one place
+    the interleave lives: a second copy of a layout rule is a place the layout
+    can drift.
+    """
+    frames = list(frames)
+    pal_b = PAL_BYTES if frames and frames[0][0] is not None else 0
+    pic_b = W * H
+    rec_b = record_bytes(W, H, palette=bool(pal_b))
+    flags = ((FLAG_PALETTE if pal_b else 0)
+             | (FLAG_PALETTE_LAST if palette_last and pal_b else 0)
+             | (FLAG_AUDIO if audio else 0))
+
+    if audio:
+        hz = int(audio["hz"])
+        F, A = cadence(fps, hz, audio.get("F", CADENCE_F))
+        au = audio["data"]
+        fmt, bits, init = audio_format(audio["variant"], audio["order"],
+                                       audio["bits"], audio["init"])
+        lumps = []
+        got = 0
+        for k in range(n_lumps(len(frames), F)):
+            n = lump_bytes(k, F, fps, hz, total=len(au))
+            if n > A * SECTOR:
+                raise ValueError(f"lump {k} needs {n} B and the cadence gives "
+                                 f"{A*SECTOR}")
+            lumps.append(au[got:got + n].ljust(A * SECTOR, b"\0"))
+            got += n
+        # A container whose audio runs out before its pictures do is a container
+        # that goes silent part way through, which is exactly the failure a
+        # writer should refuse rather than a player discover.
+        if got < len(au):
+            raise ValueError(f"{len(au)-got} B of audio have no lump to ride in "
+                             f"-- {len(frames)} frames hold {got} B")
+        if got < min(len(au), (len(frames) * hz) // (2 * fps)):
+            raise ValueError(f"the audio stream is short: {len(au)} B for "
+                             f"{len(frames)} frames at {fps} fps")
+        off_aud, off_frm = SECTOR, SECTOR + A * SECTOR
+        atail = struct.pack(">IIIHHHHhHII", off_aud, hz, len(au), F, A,
+                            fmt, bits, init, 0, 0, 0)
+    else:
+        F = A = 0
+        lumps = []
+        off_aud, off_frm = 0, SECTOR
+        atail = struct.pack(">IIIHHHHhHII", 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0)
+
+    hdr = (MAGIC + struct.pack(">HHHHHH", VERSION, flags, W, H, fps, len(frames))
+           + struct.pack(">IIII", rec_b, pal_b, pic_b, off_frm) + atail)
+    assert len(hdr) == HDR_BYTES, len(hdr)
+    with open(path, "wb") as fh:
+        fh.write(hdr + b"\0" * (SECTOR - HDR_BYTES))
+        for i, (pw, pic) in enumerate(frames):
+            if len(pic) != pic_b or (pal_b and len(pw) != pal_b):
+                raise ValueError(f"frame {i}: record parts are the wrong size")
+            # The lump goes BEFORE the group it feeds, which is the one place
+            # this format makes a choice audio forced on it.  65.3's formula put
+            # it after; a stream is read forwards, so bytes that arrive after
+            # the slot they belong to are bytes a player has to have fetched
+            # early anyway.  Placing it first makes the fetch order the play
+            # order and costs one lump of offset in the arithmetic.
+            if audio and i % F == 0:
+                fh.write(lumps[i // F])
+            rec = pic if not pal_b else (pic + pw if palette_last else pw + pic)
+            fh.write(rec)
+    return rec_b
+
+
+class DLXP:
+    """Reader, and every invariant the format claims, CHECKED rather than read.
+
+    The checks are not defensive coding.  A packed record is written into GVRAM
+    and the palette registers with no bounds test anywhere -- the channel has no
+    opinion about what it is copying -- so a container whose geometry is a byte
+    wrong does not fail, it paints.
+    """
+
+    def __init__(self, path):
+        b = self.raw = open(path, "rb").read()
+        if b[:4] != MAGIC:
+            raise ValueError(f"{path}: not a DLXP container")
+        (self.version, self.flags, self.W, self.H, self.fps,
+         self.nframes) = struct.unpack(">HHHHHH", b[4:16])
+        (self.rec_bytes, self.pal_bytes,
+         self.pic_bytes, self.off_frm) = struct.unpack(">IIII", b[16:32])
+        if self.version != VERSION:
+            raise ValueError(f"{path}: DLXP version {self.version}")
+        self.has_palette = bool(self.flags & FLAG_PALETTE)
+        self.palette_last = bool(self.flags & FLAG_PALETTE_LAST)
+        self.has_audio = bool(self.flags & FLAG_AUDIO)
+        (self.off_aud, self.aud_hz, self.aud_bytes, self.cad_f, self.cad_a,
+         self.aud_fmt, self.aud_bits, self.aud_init,
+         _r0, _r1, _r2) = struct.unpack(">IIIHHHHhHII", b[32:64])
+        if not self.has_audio:
+            if any((self.off_aud, self.aud_hz, self.aud_bytes, self.cad_f,
+                    self.cad_a, self.aud_fmt, self.aud_bits, self.aud_init)):
+                raise ValueError(f"{path}: silent container with audio fields set")
+        else:
+            if (self.cad_f, self.cad_a) != cadence(self.fps, self.aud_hz,
+                                                   self.cad_f):
+                raise ValueError(f"{path}: cadence F={self.cad_f} A={self.cad_a} "
+                                 f"does not cover {self.cad_f} frames of "
+                                 f"{self.aud_hz} Hz audio at {self.fps} fps")
+            if self.off_aud != SECTOR or self.off_aud % SECTOR:
+                raise ValueError(f"{path}: audio starts at {self.off_aud}")
+            if self.aud_bits not in (10, 12):
+                raise ValueError(f"{path}: ADPCM clamp is {self.aud_bits} bits")
+            # The stream has to fill the groups it is cut into, or the last
+            # frames of the scene play in silence and nothing says so.
+            want = sum(lump_bytes(k, self.cad_f, self.fps, self.aud_hz)
+                       for k in range(n_lumps(self.nframes, self.cad_f)))
+            if not (want - self.cad_f * self.aud_hz // (2 * self.fps)
+                    <= self.aud_bytes <= want):
+                raise ValueError(f"{path}: {self.aud_bytes} B of audio for "
+                                 f"{self.nframes} frames, geometry wants {want}")
+        if self.pic_bytes != self.W * self.H:
+            raise ValueError(f"{path}: picture is {self.pic_bytes} B for "
+                             f"{self.W}x{self.H} -- the packed layout is 1.0 B/px")
+        if self.pal_bytes != (PAL_BYTES if self.has_palette else 0):
+            raise ValueError(f"{path}: palette section is {self.pal_bytes} B")
+        if self.rec_bytes != self.pal_bytes + self.pic_bytes:
+            raise ValueError(f"{path}: record is {self.rec_bytes} B, parts are "
+                             f"{self.pal_bytes} + {self.pic_bytes}")
+        # The whole reason for DLX5 (58.3/59.4), and here it is free rather than
+        # a re-encode: the record is a fixed multiple of a sector by geometry.
+        if self.off_frm % SECTOR or self.rec_bytes % SECTOR:
+            raise ValueError(f"{path}: not sector-aligned -- stream at "
+                             f"{self.off_frm}, record {self.rec_bytes}")
+        if self.off_frm != (SECTOR + (self.cad_a * SECTOR if self.has_audio
+                                      else 0)):
+            raise ValueError(f"{path}: record 0 is at {self.off_frm}, and a "
+                             f"{self.cad_a}-sector lump comes before it")
+        want = (self.off_frm + self.nframes * self.rec_bytes
+                + (n_lumps(self.nframes, self.cad_f) - 1) * self.cad_a * SECTOR
+                if self.has_audio else
+                self.off_frm + self.nframes * self.rec_bytes)
+        if len(b) != want:
+            raise ValueError(f"{path}: {len(b)} bytes, geometry says {want}")
+
+    def frame_off(self, i):
+        """The arithmetic, and the ONE place it is written on the host side --
+        src/player/packed.s is the other, in six instructions, and
+        tools/analysis/34_packed_audio.py is what makes the two agree."""
+        if not (0 <= i < self.nframes):
+            raise IndexError(i)
+        o = self.off_frm + i * self.rec_bytes
+        return o + (i // self.cad_f) * self.cad_a * SECTOR if self.has_audio else o
+
+    def lump_off(self, k):
+        if not self.has_audio or not (0 <= k < self.n_lumps):
+            raise IndexError(k)
+        return self.off_aud + k * (self.cad_f * self.rec_bytes
+                                   + self.cad_a * SECTOR)
+
+    @property
+    def n_lumps(self):
+        return n_lumps(self.nframes, self.cad_f) if self.has_audio else 0
+
+    def lump(self, k, padding=False):
+        """Lump k's PAYLOAD -- what the chip is fed.  `padding=True` returns the
+        whole A*512 B sector run instead, which is what the disc moves and what
+        a player must NOT hand to the chip (FINDINGS 67)."""
+        o = self.lump_off(k)
+        if padding:
+            return self.raw[o:o + self.cad_a * SECTOR]
+        n = lump_bytes(k, self.cad_f, self.fps, self.aud_hz, total=self.aud_bytes)
+        return self.raw[o:o + n]
+
+    def audio(self):
+        """The whole ADPCM stream, reassembled from its lumps."""
+        return b"".join(self.lump(k) for k in range(self.n_lumps))
+
+    def decoder(self):
+        """The four axes the bytes were encoded for, as adpcm.decode's kwargs."""
+        return audio_decoder(self.aud_fmt, self.aud_bits, self.aud_init)
+
+    def record(self, i):
+        o = self.frame_off(i)
+        return self.raw[o:o + self.rec_bytes]
+
+    def _split(self, i):
+        r = self.record(i)
+        if not self.has_palette:
+            return None, r
+        if self.palette_last:
+            return r[self.pic_bytes:], r[:self.pic_bytes]
+        return r[:self.pal_bytes], r[self.pal_bytes:]
+
+    def palette_words(self, i):
+        pw, _ = self._split(i)
+        if pw is None:
+            raise ValueError("this container carries no palette -- it was made "
+                             "with --no-palette, and the palette a player would "
+                             "display is not in the file to be read back")
+        return np.frombuffer(pw, ">u2").astype(np.uint16)
+
+    def indices(self, i):
+        _, pic = self._split(i)
+        return unpack_picture(pic, self.W, self.H)
+
+    def palette_rgb(self, i):
+        """(256,3) uint8 -- what the DISPLAY produces, not what the encoder meant.
+
+        The palette in a record is already a GRB555+I word, so this is where the
+        5-bit hardware quantisation gets charged.  Everything upstream of the
+        container is in RGB888 and 61.9's +4.89 dB was quoted there; a player's
+        number has to come from here.  Same maths as `dlxload.pack_palette` and
+        `tools/bench/verify_frame256.py` -- the shared LSB `I` is a bit in the
+        word, so unpacking it needs no choice made.
+        """
+        w = self.palette_words(i).astype(int)
+        f = np.stack([(w >> 6) & 31, (w >> 11) & 31, (w >> 1) & 31], 1)   # R,G,B
+        p6 = lambda v: ((v << 2) | (v >> 4)) & 0xFF
+        return p6((f << 1) | (w & 1)[:, None]).astype(np.uint8)
+
+    def render(self, i):
+        """(H,W,3) uint8 -- the frame as the display produces it."""
+        return self.palette_rgb(i)[self.indices(i)]
+
+    def video_kbps(self):
+        """The picture on the wire.  FIXED by geometry -- there is no lever."""
+        return self.rec_bytes * self.fps / 1024
+
+    def audio_kbps(self):
+        """What the CADENCE costs, padding included, which is the honest figure:
+        the disc moves whole sectors and the wire pays for the ones that are
+        zero as well as the ones that are audio (FINDINGS 65.3)."""
+        if not self.has_audio:
+            return 0.0
+        return self.cad_a * SECTOR / self.cad_f * self.fps / 1024
+
+    def kbps(self):
+        """Both, over the scene's own duration."""
+        return self.video_kbps() + self.audio_kbps()

tools/encoder/encode.py

@@ -15,7 +15,7 @@ lets quiet frames spend the whole allowance and lands the mean ON target.
 Container (little-endian is WRONG here -- the 68000 is big-endian, so every
 multi-byte field is big-endian and the decoder can read it with a plain move.w):
 
-  header, 32 bytes
+  header, 32 bytes ('DLX4': 36 -- one more offset, see below)
     0   'DLX2'                 magic ('DLX1' = the same, unaligned; still read)
     4   u16 width, u16 height
     8   u16 fps,   u16 nframes
@@ -25,6 +25,23 @@ multi-byte field is big-endian and the decoder can read it with a plain move.w):
     20  u32 cb1 offset         (k1 * 16 bytes of palette indices)
     24  u32 cb4 offset         (k4 *  4 bytes)
     28  u32 frames offset
+    32  u32 record index offset    DLX4 ONLY.  nframes * u16, each the length of
+                                   that frame's PADDED record in LONGWORDS --
+                                   i.e. (4 + payload + pad) / 4, the whole thing
+                                   the ring producer must place contiguously.
+  THE INDEX IS NOT A CONVENIENCE, AND IT IS NOT DERIVABLE ON THE MACHINE.  The
+  `aligned` wrap policy (FINDINGS 49.3) requires the producer to know how long
+  the next record is BEFORE it fetches it, because that is what decides whether
+  it fits before the end of the ring or leaves a hole and restarts at the base.
+  Every reader in this tree up to DLX3 learned record boundaries by WALKING the
+  frame stream -- reading each record's length word to find the next -- which a
+  host with the whole file mapped can do and a player streaming off a disc
+  cannot: the length word of record i+1 is exactly one of the bytes it has not
+  fetched yet.  A branching game needs the same structure a second time, to seek
+  to a branch point without reading what lies between.
+  Lengths are stored rather than offsets: 2 bytes a frame instead of 4, and the
+  offsets are a running sum the player builds once at scene load (u16 caps a
+  record at 262,140 B, asserted at write time).
   then, per frame, each record starting on a 4-BYTE BOUNDARY (0-3 zero pad
   bytes before it; a 68000 takes an address error, not a slow read, on an odd
   `move.l` -- FINDINGS 28.3):
@@ -49,6 +66,7 @@ import argparse, struct, sys, os
 sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
 import numpy as np
 import vq as VQ, vq_hybrid as H, ratectl as RC, spans as SP
+from dlx import SECTOR
 
 # Measured on the emulated 68000, FINDINGS 24. Instruction cycles against
 # zero-wait-state memory, so these are floors, not hardware predictions.
@@ -117,8 +135,12 @@ def build_records(m, enc, span_mode):
     return out
 
 
-def write_container(path, m, frames, fps, k1, k4, span_mode):
-    """Write the whole container.  Returns (total bytes, video bytes, pad)."""
+def write_container(path, m, frames, fps, k1, k4, span_mode, sector=True):
+    """Write the whole container.  Returns (total bytes, video bytes, pad).
+
+    `sector` selects DLX5's 512-byte record alignment over DLX4's 4-byte one.
+    """
+    align = SECTOR if sector else 4
     palette = m["pal"][:256]
     if len(palette) < 256:
         palette = np.vstack([palette, np.zeros((256 - len(palette), 3), np.uint8)])
@@ -126,31 +148,63 @@ def write_container(path, m, frames, fps, k1, k4, span_mode):
     cb1_b = m["cb1"].astype(np.uint8).tobytes()
     cb4_b = m["cb4"].astype(np.uint8).tobytes()
 
-    off_pal = 32
+    off_pal = 36 if span_mode else 32
+    if not span_mode:
+        sector = False          # DLX2 has no index and no sector rule
+        align = 4
     off_cb1 = off_pal + len(pal_b)
     off_cb4 = off_cb1 + len(cb1_b)
-    off_frm = off_cb4 + len(cb4_b)
+    # DLX4: the record index sits with the palette and the codebooks, ahead of
+    # the frame stream, because it is part of what has to ARRIVE before frame 0
+    # can be decoded -- FINDINGS 53.5's scene header, and this adds to it.
+    idx_b = b""
+    if span_mode:
+        qlens = []
+        for i, rec in enumerate(frames):
+            n = 4 + len(rec)
+            n += -n % align
+            q = n // 4
+            assert q <= 0xFFFF, (f"frame {i} is {n} B: a u16 longword count "
+                                 f"caps a record at 262,140 B")
+            qlens.append(q)
+        idx_b = struct.pack(f">{len(qlens)}H", *qlens)
+
+    off_idx = off_cb4 + len(cb4_b)
+    off_frm = off_idx + len(idx_b)
     # DLX2: every frame record starts on a 4-byte boundary, including the
     # first. Payload lengths are arbitrary, so end-to-end records land on odd
     # addresses -- and `move.l (a0)+` at an odd address is an ADDRESS ERROR on
     # a 68000, not a slow read. It vectors into the IPL and looks exactly like
     # an infinite loop (FINDINGS 28.3). tools/bench/prep_dlx.py has been
     # realigning at load time; the container now carries it.
-    tbl_pad = -off_frm % 4
+    # DLX5 aligns the frame stream itself as well as the records inside it, so
+    # the whole container can be laid on a volume at a sector boundary and every
+    # record lands on one.  Aligning the records to each other and not the run
+    # they sit in would leave 117 of 120 of them off-sector again the moment the
+    # scene header changed length by a byte.
+    tbl_pad = -off_frm % align
     off_frm += tbl_pad
-    hdr = ((b"DLX3" if span_mode else b"DLX2")
+    magic = (b"DLX5" if sector else b"DLX4") if span_mode else b"DLX2"
+    hdr = (magic
            + struct.pack(">HHHHHH", m["W"], m["H"], fps, len(frames), k1, k4)
            + struct.pack(">IIII", off_pal, off_cb1, off_cb4, off_frm))
-    assert len(hdr) == 32, len(hdr)
+    if span_mode:
+        hdr += struct.pack(">I", off_idx)
+    assert len(hdr) == (36 if span_mode else 32), len(hdr)
 
     frm_pad = 0
     with open(path, "wb") as fh:
         fh.write(hdr); fh.write(pal_b); fh.write(cb1_b); fh.write(cb4_b)
-        fh.write(b"\0" * tbl_pad)
+        fh.write(idx_b); fh.write(b"\0" * tbl_pad)
         for i, rec in enumerate(frames):
             fh.write(struct.pack(">I", len(rec))); fh.write(rec)
-            if i + 1 < len(frames):          # nothing follows the last record
-                n = -(4 + len(rec)) % 4
+            # DLX2/3 skipped the pad after the LAST record because nothing
+            # followed it. DLX4's index describes PADDED records, and a producer
+            # that trusts the index fetches that many bytes -- so the last
+            # record is padded too, and the file ends where the index says it
+            # does rather than up to 3 bytes short of it.
+            if span_mode or i + 1 < len(frames):
+                n = -(4 + len(rec)) % align
                 fh.write(b"\0" * n); frm_pad += n
     total = os.path.getsize(path)
     return total, sum(len(r) + 4 for r in frames) + frm_pad, frm_pad
@@ -212,6 +266,13 @@ def main():
                          "Default OFF: measured, it is worth one frame of 120 "
                          "at --spans need and a 2%% regression at --spans all "
                          "(FINDINGS 44)")
+    ap.add_argument("--joint-spans", action="store_true",
+                    help="re-run the lam search with the bytes the span pass "
+                         "freed, then re-span (E3, FINDINGS 39.3 item 5). The "
+                         "span pass removes the block payload of every block "
+                         "it covers, so without this the frame lands under its "
+                         "allowance and the blocks that were NOT spanned were "
+                         "priced as if those bytes were still needed.")
     ap.add_argument("--no-cpu-fit", action="store_true",
                     help="drop the per-frame 68000 decode ceiling (session 7 "
                          "behaviour: 31%% of frames on hard content do not fit)")
@@ -219,6 +280,13 @@ def main():
                     help="how full the player's buffer is assumed to be at "
                          "scene start, as a fraction of the bucket (0 = cold "
                          "buffer after a seek, the conservative assumption)")
+    ap.add_argument("--no-reserve-black", action="store_true",
+                    help="let the scene palette spend all 256 entries on the "
+                         "picture. The default RESERVES index 0 as true black "
+                         "(FINDINGS 23.4), because GVRAM cleared to zero shows "
+                         "entry 0 and the 256x192 picture sits in a 256x256 "
+                         "mode -- so a free palette letterboxes the frame in "
+                         "whatever colour mediancut happened to put first.")
     ap.add_argument("--preview")
     a = ap.parse_args()
 
@@ -239,6 +307,7 @@ def main():
     if a.disk_clk_byte is not None:
         RC.DISK_CLK_BYTE = a.disk_clk_byte
     RC.JOINT_DECIDE = a.joint_decide
+    RC.JOINT_SPANS = a.joint_spans
     RC.JOINT_BUCKET = a.joint_bucket
     cyc_budget = None if a.no_cpu_fit else RC.FRAME_CYCLES
     span_mode = None if (a.spans == "off" or not rc) else a.spans
@@ -261,8 +330,13 @@ def main():
     else:
         print(f"  target {prof['kbps']} KB/s, FIXED lam={lam} (no rate control)")
     print(f"  k1={k1} k4={k4}, {_IDX_BYTES}-byte indices")
+    print(f"  palette: " + ("255 picture colours, index 0 RESERVED as true "
+          "black for the letterbox (23.4)" if not a.no_reserve_black
+          else "all 256 entries to the picture (--no-reserve-black); index 0 "
+               "is whatever mediancut put there, and the letterbox with it"))
 
-    m = H.build(a.frames_dir, k1=k1, k4=k4, iters=a.iters)
+    m = H.build(a.frames_dir, k1=k1, k4=k4, iters=a.iters,
+                reserve_black=not a.no_reserve_black)
     if rc:
         enc = RC.encode_rate_controlled(m, prof["kbps"], fps=a.fps,
                                         bucket_frames=a.bucket_frames,

tools/encoder/extract_audio.py

@@ -0,0 +1,45 @@
+#!/usr/bin/env python3
+"""Extract the audio of a Blu-ray window as mono PCM at an MSM6258 sample rate.
+
+The video side of this window is tools/encoder/extract.py; the arguments mean
+the same things and are meant to be given the same values, because an audio
+stream that is not the same seconds as the frames is not this project's audio.
+
+The disc is AC-3 5.1 at 48 kHz. The arcade original is MONO, so this downmixes
+-- ffmpeg's default matrix, dialogue from the centre channel included -- and
+resamples to the chip's rate. Nothing here shapes, gates or normalises the
+level: what the ADPCM encoder is handed is what the disc has, so that the SNR
+it reports is the codec's and not a gain stage's.
+"""
+import getpass, os, subprocess, sys
+
+BDROM = os.environ.get("DLX_BDROM") or f"/media/{getpass.getuser()}/BDROM"
+STREAM_DIR = f"{BDROM}/BDMV/STREAM"
+
+# 8 MHz / {512, 768, 1024}. The chip has no other rates and 15,625 is the one
+# every budget in this project is written against (FINDINGS 52).
+RATES = {15625: 512, 10417: 768, 7813: 1024}
+
+
+def extract(stream, out, rate=15625, start=None, dur=None):
+    if rate not in RATES:
+        raise SystemExit(f"{rate} is not an MSM6258 rate: {sorted(RATES)}")
+    src = f"{STREAM_DIR}/{stream}.m2ts"
+    cmd = ["ffmpeg", "-v", "error"]
+    if start is not None: cmd += ["-ss", str(start)]
+    if dur is not None:   cmd += ["-t", str(dur)]
+    cmd += ["-i", src, "-vn", "-ac", "1", "-ar", str(rate),
+            "-f", "s16le", "-acodec", "pcm_s16le", out, "-y"]
+    subprocess.check_call(cmd)
+    n = os.path.getsize(out) // 2
+    print(f"{stream}: {n} samples @ {rate} Hz mono = {n/rate:.3f} s -> {out}")
+    return n
+
+
+if __name__ == "__main__":
+    # extract_audio.py <stream> <out.raw> [rate] [start_s] [dur_s]
+    stream, out = sys.argv[1], sys.argv[2]
+    rate  = int(sys.argv[3])   if len(sys.argv) > 3 else 15625
+    start = float(sys.argv[4]) if len(sys.argv) > 4 else None
+    dur   = float(sys.argv[5]) if len(sys.argv) > 5 else None
+    extract(stream, out, rate, start, dur)

tools/encoder/pack.py

@@ -0,0 +1,195 @@
+#!/usr/bin/env python3
+"""Encode one scene to the PACKED container -- ROADMAP K2.
+
+    python3 tools/encoder/pack.py <frames_dir> <out.dlxp> [--fps 12]
+                                  [--nframes N] [--palette-last] [--no-palette]
+                                  [--scene-palette] [--audio tmp/au_singe.raw]
+
+WHAT IS NOT HERE IS THE POINT.  No VQ, no codebooks, no mode map, no rate
+control, no `lam`, no leaky bucket, no span geometry -- `encode.py` is 452 lines
+and 95% of its wall clock is k-means.  A packed frame is a palette and a
+picture, and both are geometry.  FINDINGS 61: at the 9 clk/B dual-address floor
+the codec is 110.4% of a 12 fps frame and this is 55.2%, so the thing that
+replaces the codec is also the thing that is simpler than it.
+
+THERE IS NO RATE CONTROL BECAUSE THERE IS NO RATE LEVER.  A codec's bitrate is
+adjustable; a literal frame's is geometry.  The wire cost of this container is
+fixed by W, H and fps and nothing an encoder does can move it, which is exactly
+why FINDINGS 61.6 says the medium question decides which player exists.  A
+`--kbps` argument here would be a lie of the shape FINDINGS 50 removed from the
+rest of the tree.
+
+THE QUANTISER IS THIS PROJECT'S, NOT PIL'S DEFAULT PATH.  61.9's +4.89 dB was
+measured with `18_text_plane_16col.py`'s free 256-colour MEDIANCUT and was filed
+as "a direction, not the player's number" (risk 2 in the session 30 handoff).
+`vq.frame_palette` is what actually ships it: 254 colours, index 0 held free for
+the transparency key, black at 255.  `tools/analysis/30_packed_container.py`
+re-derives the figure against this and charges the GRB555 word on top.
+
+AND `--audio` MAKES IT A DLXP2, WHICH IS THE ONE PLACE THE FOUR ADPCM AXES ARE
+CHOSEN.  It encodes for `adpcm.CHIP` -- the datasheet's per-term delta, the LOW
+nibble of a byte first, a 10-bit clamp, the accumulator at -2 -- because that is
+the set FINDINGS 66 measured out of the machine's own chip through the machine's
+own DMA channel, and encoding for any other set costs up to 25.7 dB.  It does
+NOT use adpcm.py's module defaults, which are ffmpeg's on purpose so that
+tools/bench/verify_adpcm.py stays a check against an independent implementation.
+The axes go in the header, so a player never has to be told.
+"""
+import argparse, glob, math, os, sys
+import numpy as np
+from PIL import Image
+
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+sys.path.insert(0, os.path.join(os.path.dirname(os.path.abspath(__file__)),
+                                "..", "bench"))
+import vq as VQ
+import dlxp as P
+import adpcm
+from dlxload import pack_palette
+
+ap = argparse.ArgumentParser()
+ap.add_argument("frames_dir")
+ap.add_argument("out")
+ap.add_argument("--fps", type=int, default=12)
+ap.add_argument("--nframes", type=int, default=None,
+                help="encode only the first N frames (the gate window is 120)")
+ap.add_argument("--palette-last", action="store_true",
+                help="put the palette after the picture in every record. "
+                     "FINDINGS 62.5 -- a design choice, not a default to inherit")
+ap.add_argument("--no-palette", action="store_true",
+                help="picture only, 49,152 B a record. NOT a shipping option: "
+                     "it re-imposes the scene palette the codec is capped by")
+ap.add_argument("--scene-palette", action="store_true",
+                help="one palette for the whole scene, repeated in every record "
+                     "-- the CONTROL for 61.9's per-frame claim")
+ap.add_argument("--audio", default=None,
+                help="raw s16le mono at --audio-hz, from extract_audio.py. "
+                     "Makes the output a DLXP2: the stream is encoded for "
+                     "adpcm.CHIP and interleaved on the 65.3 cadence")
+ap.add_argument("--audio-hz", type=int, default=15625,
+                help="the CHIP's rate, and the rate the .raw was resampled to")
+ap.add_argument("--cadence", type=int, default=P.CADENCE_F,
+                help="frames between audio lumps (65.3's sweep picks 11)")
+ap.add_argument("--audio-gain", type=float, default=1.0,
+                help="LEVEL, applied before the 12-bit requantisation. 1.0 is "
+                     "s16>>4, the disc's own level, and it is the MEASURED "
+                     "choice: FINDINGS 69 encoded windows drawn over the whole "
+                     "game at six gains and every attenuation that buys "
+                     "headroom under the chip's 10-bit clamp costs more SNR "
+                     "than the clamping it avoids. Below 1.0 is a fallback")
+a = ap.parse_args()
+
+files = sorted(glob.glob(f"{a.frames_dir}/f*.png"))
+if a.nframes:
+    files = files[:a.nframes]
+if not files:
+    sys.exit(f"no f*.png in {a.frames_dir}")
+rgb = [np.asarray(Image.open(f).convert("RGB")) for f in files]
+H, W = rgb[0].shape[:2]
+if any(r.shape[:2] != (H, W) for r in rgb):
+    sys.exit("frames are not all the same size")
+
+# The scene-palette control shares ONE palette across every record, which is the
+# constraint the codec cannot escape (61.9) and this format merely chooses not
+# to inherit.  It is built by the same routine the codec uses, with black at 0
+# moved to 255 and index 0 vacated, so the only variable between the two runs is
+# per-frame versus scene-wide.
+scene = None
+if a.scene_palette:
+    # 255, not 256: reserve_black spends one entry on black and the packed
+    # layout needs the OTHER end free too, so the picture gets 254 either way.
+    ref, spal = VQ.scene_palette(rgb, colors=255, reserve_black=True)
+    sidx = VQ.palettise(rgb, ref)
+    # 0 -> 255: the packed layout needs index 0 free and black displayed at 255.
+    spal = np.vstack([np.zeros((1, 3), np.uint8), spal[1:],
+                      np.zeros((1, 3), np.uint8)])
+    scene = (spal, [np.where(i == 0, np.uint8(255), i) for i in sidx])
+
+recs, psnr_pal = [], []
+for n, src in enumerate(rgb):
+    if scene is not None:
+        pal, idx = scene[0], scene[1][n]
+    else:
+        pal, idx = VQ.frame_palette(src)
+    if (idx == 0).any():
+        sys.exit(f"frame {n}: index 0 is the transparency key and got used")
+    palw = None
+    if not a.no_palette:
+        palb, _dark, rendered = pack_palette(pal)
+        palw = palb.tobytes()
+    psnr_pal.append(VQ.psnr(src, pal[idx]))
+    recs.append((palw, P.pack_picture(idx).tobytes()))
+
+# THE AUDIO, AND IT IS ENCODED FOR THE CHIP AND NOT FOR ffmpeg.  The source is
+# s16le because that is what ffmpeg resamples to; the chip's word is 12 bits
+# signed, so the shift is a requantisation and not a format conversion, and it
+# is the same one tools/bench/verify_adpcm.py makes.
+audio = None
+if a.audio:
+    import struct as _struct
+    raw = open(a.audio, "rb").read()
+    pcm = _struct.unpack("<%dh" % (len(raw) // 2), raw)
+    # THE LEVEL.  `>> 4` maps the disc's full scale onto the 12-bit word and is
+    # what every container in this tree has been encoded at; the gain is a
+    # multiply BEFORE it, so gain 1.0 is byte-identical to what shipped.  The
+    # chip clamps at 10 bits INSIDE the recursion (adpcm.CHIP), so anything the
+    # gain puts above 511 is unreachable -- and FINDINGS 69 measured that the
+    # attenuation which avoids that costs more than the clamping does.
+    g = a.audio_gain
+    src12 = [max(-2048, min(2047, int(math.floor(x * g)) >> 4)) for x in pcm]
+    lo12, hi12 = adpcm.clamp_bounds(adpcm.CHIP["bits"])
+    nclamp = sum(1 for v in src12 if v > hi12 or v < lo12)
+    need = len(files) * a.audio_hz // (2 * a.fps)
+    nib = adpcm.encode(src12, variant=adpcm.CHIP["variant"],
+                       init=adpcm.CHIP["init"], bits=adpcm.CHIP["bits"])
+    data = adpcm.pack(nib, order=adpcm.CHIP["order"])[:need]
+    # What the encoder thinks it wrote, checked by DECODING it with the same
+    # four axes -- the encoder runs its decoder inside its own loop, so this is
+    # not circular in the way it looks: it is the PACKED bytes going back
+    # through unpack(), which is where a nibble-order slip would land.
+    back = adpcm.decode(adpcm.unpack(data, len(src12), order=adpcm.CHIP["order"]),
+                        variant=adpcm.CHIP["variant"], init=adpcm.CHIP["init"],
+                        bits=adpcm.CHIP["bits"])
+    ref = src12[:len(back)]
+    e = [(x - y) ** 2 for x, y in zip(ref, back)]
+    sig = sum(x * x for x in ref)
+    snr = 10 * np.log10(sig / sum(e)) if sum(e) else float("inf")
+    audio = dict(data=data, hz=a.audio_hz, F=a.cadence, **adpcm.CHIP)
+
+rec_b = P.write(a.out, W, H, a.fps, recs, palette_last=a.palette_last,
+                audio=audio)
+d = P.DLXP(a.out)                        # re-read: every invariant is checked
+if d.nframes != len(recs):
+    sys.exit("writer and reader disagree about the frame count")
+
+kind = ("scene palette" if a.scene_palette else "per-frame palette")
+if a.no_palette:
+    kind += ", NONE in the record"
+print(f"{a.out}: DLXP{P.VERSION} {W}x{H} {a.fps}fps {d.nframes} frames, {kind}"
+      f"{', palette LAST' if a.palette_last else ''}")
+print(f"  record {rec_b:,} B = {rec_b // P.SECTOR} sectors exactly, "
+      f"file {os.path.getsize(a.out):,} B")
+if d.has_audio:
+    print(f"  DLXP2: audio {d.aud_bytes:,} B at {d.aud_hz:,} Hz, SNR {snr:.2f} dB, "
+          f"cadence F={d.cad_f} A={d.cad_a} ({d.n_lumps} lumps)")
+    print(f"  level gain {a.audio_gain:g}, source peak {max(abs(v) for v in src12)}"
+          f" of the chip's {hi12}: {nclamp:,} of {len(src12):,} samples "
+          f"({100*nclamp/len(src12):.4f}%) are above the clamp and cannot be "
+          f"reached (FINDINGS 69)")
+    print(f"  the four axes, in the header: "
+          + ", ".join(f"{k}={v}" for k, v in d.decoder().items()))
+    # STEADY STATE, not the file: the last lump of a 120-frame window feeds 4
+    # frames out of 11 and occupies 14 sectors either way, so a whole-file
+    # padding figure is a boundary effect of the WINDOW and would change with
+    # its length.  65.3's 0.09% is the cadence's, and the cadence is the thing.
+    grp = d.cad_f * d.aud_hz / (2 * d.fps)
+    print(f"  lump payload {P.lump_bytes(0, d.cad_f, d.fps, d.aud_hz):,}.."
+          f"{P.lump_bytes(2, d.cad_f, d.fps, d.aud_hz):,} B of {d.cad_a*P.SECTOR:,} "
+          f"-- {100*(d.cad_a*P.SECTOR-grp)/grp:.3f}% padding steady state, and the "
+          f"payload is NOT the lump (FINDINGS 67)")
+    print(f"  wire {d.video_kbps():.1f} + {d.audio_kbps():.2f} = {d.kbps():.1f} KB/s "
+          f"-- FIXED by geometry, there is no lever")
+else:
+    print(f"  wire {d.kbps():.1f} KB/s -- FIXED by geometry, there is no lever")
+print(f"  palette-domain PSNR vs the 24-bit source: "
+      f"{np.mean(psnr_pal):.2f} dB (min {np.min(psnr_pal):.2f})")

tools/encoder/ratectl.py

@@ -157,6 +157,23 @@ JOINT_DECIDE = False
 # `--joint-bucket` turns it on.
 JOINT_BUCKET = False
 
+# E3 / FINDINGS 39.3 item 5: SPAN SELECTION IS GREEDY AFTER `lam`, and this is
+# the switch that makes the two joint.  The lam bisection picks a mode map
+# against a byte allowance, and the span pass then REMOVES the block payload of
+# every block it covers -- so the frame lands under the allowance by exactly
+# the bytes the spans freed, and the blocks that were NOT spanned were priced
+# at a lam chosen as if those bytes were still needed.  Joint mode hands the
+# freed bytes back to the lam search and re-spans the result, to a fixed point
+# or two rounds, whichever comes first.
+#
+# It is a REFINEMENT, not a different objective: lam can only fall (the
+# allowance only grows), so the un-spanned blocks can only improve, and a round
+# is kept only if the frame still fits both ceilings it was already fitting.
+# Default OFF until measured, which is 44.3's lesson -- ask whether the lever
+# is loaded before pulling it.
+JOINT_SPANS = False
+JOINT_SPAN_ROUNDS = 2
+
 
 def _byte_clk():
     """The debit the mode decision is allowed to see (0 = the old decision)."""
@@ -287,6 +304,39 @@ def _fit_spans(m, ctx, mode, sz, room, cyc_budget, span_mode, ib):
     return nmode, nsz, H.cycles(nmode) + sel["clocks"], sel
 
 
+def _refit_joint(m, ctx, allow, span_allow, lam_lo, lam_hi, cyc_budget,
+                 span_mode, ib, mode_pre, mode, sz, cyc, sel, mu=0.0):
+    """Give the lam search back the bytes the span pass freed, then re-span.
+
+    `mode_pre` is the mode map BEFORE spanning and `mode` the one after, so the
+    difference in frame_bytes is exactly what the spans made unnecessary.  The
+    ceiling the result is judged against is the one _fit_spans was already
+    working to, so a kept round is never a frame that grew past a budget it was
+    inside.
+    """
+    ceiling = span_allow if span_allow is not None else allow
+    for _ in range(JOINT_SPAN_ROUNDS):
+        if sel is None:
+            break
+        freed = (H.frame_bytes(mode_pre, ctx["nb"], ib)
+                 - H.frame_bytes(mode, ctx["nb"], ib))
+        if freed <= 0:
+            break
+        lam2, mode2, sz2, _ = _search_lam(ctx, allow + freed, lam_lo, lam_hi, mu=mu)
+        if sz2 <= H.frame_bytes(mode_pre, ctx["nb"], ib):
+            break                     # lam did not move: already at the floor
+        n_pre, n_mode, n_sz, n_cyc, n_sel = (
+            mode2, *_fit_spans(m, ctx, mode2, sz2, span_allow if span_allow
+                               is not None else allow, cyc_budget, span_mode, ib))
+        if n_sel is None or n_sz > ceiling:
+            break
+        if cyc_budget is not None and n_cyc + DISK_CLK_BYTE * n_sz > cyc_budget \
+           and cyc + DISK_CLK_BYTE * sz <= cyc_budget:
+            break                     # round 1 made the deadline and this does not
+        mode_pre, mode, sz, cyc, sel = n_pre, n_mode, n_sz, n_cyc, n_sel
+    return mode_pre, mode, sz, cyc, sel
+
+
 def encode_rate_controlled(m, target_kbps, fps=12, bucket_frames=8,
                            lam_lo=1.0, lam_hi=LAM_CLIFF, prefill=0.0,
                            steps=None, verbose=False, cycle_budget=None,
@@ -364,6 +414,10 @@ def encode_rate_controlled(m, target_kbps, fps=12, bucket_frames=8,
             mode_pre = mode
             mode, sz, cyc, sel = _fit_spans(m, ctx, mode, sz, span_allow,
                                             cycle_budget, span_mode, ib)
+            if JOINT_SPANS:
+                mode_pre, mode, sz, cyc, sel = _refit_joint(
+                    m, ctx, allow, span_budget and span_allow, lam_lo, lam_hi,
+                    cycle_budget, span_mode, ib, mode_pre, mode, sz, cyc, sel)
         if cycle_budget is not None and cyc + DISK_CLK_BYTE * sz > cycle_budget:
             # The byte allowance could not buy the frame's deadline, so fall
             # back to the controller that pays in picture -- and then offer
@@ -374,6 +428,11 @@ def encode_rate_controlled(m, target_kbps, fps=12, bucket_frames=8,
                 mode_pre = mode
                 mode, sz, cyc, sel = _fit_spans(m, ctx, mode, sz, span_allow,
                                                 cycle_budget, span_mode, ib)
+                if JOINT_SPANS:
+                    mode_pre, mode, sz, cyc, sel = _refit_joint(
+                        m, ctx, allow, span_budget and span_allow, lam_lo,
+                        lam_hi, cycle_budget, span_mode, ib, mode_pre, mode,
+                        sz, cyc, sel, mu=mu)
                 late = cyc + DISK_CLK_BYTE * sz > cycle_budget
         # Paint from the mode map as it was BEFORE spanning.  A spanned run's
         # blocks read SKIP in the emitted header, but SKIP means "hold the

tools/encoder/vq.py

@@ -28,15 +28,78 @@ def load_frames(d):
     return [np.asarray(Image.open(f).convert("RGB")) for f in fs]
 
 
-def scene_palette(rgb, colors=256, stride=3):
-    """One shared palette for the whole scene, no dithering (cel art is flat)."""
+def scene_palette(rgb, colors=256, stride=3, reserve_black=True):
+    """One shared palette for the whole scene, no dithering (cel art is flat).
+
+    `reserve_black` puts TRUE BLACK at index 0 and quantises the picture into
+    the other 255 entries.  It is not a cosmetic default (FINDINGS 23.4): the
+    picture is 256x192 inside a 256x256 mode, GVRAM cleared to zero displays
+    palette entry 0, and a free mediancut palette puts a real image colour
+    there -- on 00020 f0001 it was (206,192,176), used by 210 image pixels, so
+    the 64 blank rows of letterbox came out beige.  Entry 0 also needs `I = 0`
+    in the X68000's GRB555 word or the bars sit at RGB (4,4,4) (23.3); that
+    half is `dlxload.pack_palette`'s and it needs no special case, because a
+    (0,0,0) entry picks I=0 by its own minimum-squared-error rule.
+
+    Black is RESERVED, not withheld: the mapper may still spend index 0 on
+    genuinely black pixels, which is the entry it would have wanted anyway.
+    What the reservation buys is that index 0 is black REGARDLESS of what the
+    scene contains, which is what the letterbox needs and what a free palette
+    cannot promise.
+    """
     samp = np.concatenate([r.reshape(-1, 3) for r in rgb[::stride]])
-    ref = Image.fromarray(samp.reshape(-1, 1, 3)).quantize(
-        colors=colors, method=Image.MEDIANCUT, dither=Image.NONE)
-    pal = np.array(ref.getpalette()[:colors * 3], dtype=np.uint8).reshape(-1, 3)
+    n = colors - 1 if reserve_black else colors
+    q = Image.fromarray(samp.reshape(-1, 1, 3)).quantize(
+        colors=n, method=Image.MEDIANCUT, dither=Image.NONE)
+    pal = np.array(q.getpalette()[:n * 3], dtype=np.uint8).reshape(-1, 3)
+    if not reserve_black:
+        return q, pal
+    pal = np.vstack([np.zeros((1, 3), np.uint8), pal])
+    # The quantiser above cannot be reused as the mapping reference: its
+    # palette is the 255 it chose, at the wrong indices.  A P-mode image
+    # carrying the FINAL palette is what every frame is then mapped against,
+    # so the indices in the container and the entries in the container's
+    # palette section are the same table by construction.
+    ref = Image.new("P", (1, 1))
+    ref.putpalette(pal.tobytes().ljust(768, b"\0"))
     return ref, pal
 
 
+def frame_palette(rgb1, colors=254):
+    """`scene_palette`'s sibling, for the PACKED layout: ONE FRAME, 254 colours.
+
+    The codec cannot have this.  Every codeword it emits is an index INTO
+    `scene_palette`, so its palette is shared scene-wide and 31.33 dB is a
+    ceiling no bitrate crosses (FINDINGS 61.9).  A literal frame has no
+    codebooks, so nothing forces a shared palette on it.
+
+    The layout spends TWO entries where `--reserve-black` spends one (47.2):
+    index 0 is the TRANSPARENCY KEY of the top graphics page and must never
+    appear in the picture, and black therefore lives at 255 for the letterbox.
+    So the picture gets 254.
+
+    The +1 shift is the whole mechanism, and it is why this does NOT go through
+    a P-mode reference image the way `scene_palette` does.  `palettise` maps
+    against the FINAL 256-entry table, and that table has (0,0,0) at both 0 and
+    255 -- a nearest-colour mapper is free to pick either, and there is no way to
+    forbid the one that must stay unused.  Quantising to 254 and shifting keeps
+    index 0 free BY CONSTRUCTION rather than by hoping the mapper agrees, and it
+    is still exact: `pal[idx]` reproduces the quantiser's own rendering.
+
+    Returns (pal (256,3) uint8, idx (H,W) uint8 in 1..254).
+    """
+    q = Image.fromarray(rgb1).quantize(colors=colors, method=Image.MEDIANCUT,
+                                       dither=Image.NONE)
+    raw = q.getpalette()
+    if len(raw) < colors * 3:
+        raise ValueError(f"quantiser returned {len(raw)//3} entries, wanted {colors}")
+    pal = np.array(raw[:colors * 3], dtype=np.uint8).reshape(-1, 3)
+    pal = np.vstack([np.zeros((1, 3), np.uint8), pal, np.zeros((1, 3), np.uint8)])
+    idx = np.asarray(q, dtype=np.uint8) + np.uint8(1)
+    if idx.min() < 1 or idx.max() > colors:
+        raise ValueError("index 0 (transparency key) or 255 (black) got used")
+    return pal, idx
+
 def palettise(rgb, ref):
     return [np.asarray(Image.fromarray(r).quantize(palette=ref, dither=Image.NONE),
                        dtype=np.uint8) for r in rgb]

tools/encoder/vq_hybrid.py

@@ -113,10 +113,10 @@ def blocks_of(idx, pal, bw, bh):
     return VQ.blockify(idx, pal, bw, bh)
 
 
-def build(frames_dir, k1=256, k4=256, iters=16, lam=0.0):
+def build(frames_dir, k1=256, k4=256, iters=16, lam=0.0, reserve_black=True):
     rgb = VQ.load_frames(frames_dir)
     H, W = rgb[0].shape[:2]
-    ref, pal = VQ.scene_palette(rgb)
+    ref, pal = VQ.scene_palette(rgb, reserve_black=reserve_black)
     idx = VQ.palettise(rgb, ref)
 
     # --- two codebooks, trained on the whole scene ---

tools/import/scenegraph.py

@@ -0,0 +1,436 @@
+"""The ONE file in this tree that knows anything about somebody else's source.
+
+Everything downstream -- `tools/analysis/25_scene_graph.py` and whatever the
+game-logic layer eventually becomes -- reads the neutral table this writes and
+knows nothing about where it came from.  That is deliberate.  Two outside
+projects are read here, neither is vendored, neither ships in this repo, and
+their file layouts, table names, timing formulas and magic constants are wired
+into THIS file and nowhere else.  When one of them moves, one file breaks.
+
+    python3 tools/import/scenegraph.py [-o tmp/scenegraph.json]
+
+Sources, both permissively licensed and both cloned by the reader:
+
+  icculus/DirkSimple -- zlib, Ryan C. Gordon.  `data/games/lair/game.lua` holds
+    the arcade scene graph, transcribed from the arcade ROM's own data table.
+    THE source; the measurement is made on this.
+      DLX_DIRKSIMPLE=/path (default tmp/scenegraph/DirkSimple)
+
+  astrobleem/SNES-SuperDragonsLairArcade -- MIT, Chad Doebelin.
+    `data/events/*.xml` holds 516 chapter definitions.  Read as a CROSS-CHECK
+    only, and a weak one: their own README states the chapters are "derived
+    from DirkSimple game data", so this is a second COPY, not a second
+    transcription (FINDINGS 56.2).  Optional; skipped when absent.
+      DLX_SNESLAIR=/path (default tmp/scenegraph/SNES-SuperDragonsLairArcade)
+
+WHAT COMES OUT is `DLXSCENE1`, our own schema, in one JSON file:
+
+    {"format": "DLXSCENE1",
+     "sources": [{"name", "url", "licence", "holder", "role"} ...],
+     "scenes": {scene: {sequence: {
+         "start_ms":   float, absolute position on the medium, or -1 for
+                       "keep playing from where the disc already is",
+         "timeout_ms": float, how long the clip runs unbranched,
+         "kills", "single_frame": bool,
+         "timeout_next": sequence name or null,
+         "actions": [{"input", "from_ms", "to_ms", "next"}]}}},
+     "rows": [[scene, scene, scene] ...],     the arcade's 13x3 scene order
+     "counts": {"scenes", "sequences", "windows"},
+     "crosscheck": {scene: {sequence: {"start_ms", "end_ms", "actions": [...],
+                                       "chapter"}}}    or null}
+
+Nothing in the schema is a copy of either project's structure: it is what this
+project's own analysis needs, which is a clip's length, its exits and the
+earliest instant each of them can fire.
+
+If a scene table is ever COMMITTED to this repo rather than regenerated into
+gitignored tmp/, it becomes redistribution of derived data and the attribution
+in "sources" has to travel with it.  FINDINGS 16, 56.5.
+"""
+import sys, os, re, json, argparse
+import xml.etree.ElementTree as ET
+
+DIRK = os.environ.get("DLX_DIRKSIMPLE", "tmp/scenegraph/DirkSimple")
+SNES = os.environ.get("DLX_SNESLAIR", "tmp/scenegraph/SNES-SuperDragonsLairArcade")
+
+# DirkSimple's own constants, from data/games/lair/game.lua.  Restated here
+# because this file evaluates its table; both are gated below against the text
+# of the file they came from, so they cannot go stale silently.
+LD_FPS = 23.976          # laserdisc_frame_to_ms
+ROM_OFFSET_MS = 6297.0   # time_laserdisc_frame's "magic millisecond offset"
+
+SOURCES = [
+    {"name": "icculus/DirkSimple",
+     "url": "https://github.com/icculus/DirkSimple",
+     "licence": "zlib", "holder": "Ryan C. Gordon",
+     "role": "the scene graph itself, transcribed from the arcade ROM"},
+    {"name": "astrobleem/SNES-SuperDragonsLairArcade",
+     "url": "https://github.com/astrobleem/SNES-SuperDragonsLairArcade",
+     "licence": "MIT", "holder": "Chad Doebelin",
+     "role": "cross-check only; itself derived from DirkSimple (FINDINGS 56.2)"},
+]
+
+# ---------------------------------------------------------------- Lua subset
+
+class LuaParser:
+    """Just enough Lua to read game.lua's `scenes` table constructor.
+
+    Not a Lua interpreter and not trying to be: the grammar is table
+    constructors, string/number/boolean/nil literals, bare identifiers (used
+    only for function references like `interrupt=game_over_complete`), calls to
+    four known helpers, and + / - between them.  Anything else is a parse error
+    rather than a silent skip, and the caller asserts the whole table was
+    consumed, so a change upstream shows up as a failure and not as a smaller
+    scene graph.
+    """
+
+    def __init__(self, text, helpers):
+        self.s, self.i, self.helpers = text, 0, helpers
+
+    def error(self, msg):
+        line = self.s.count("\n", 0, self.i) + 1
+        raise SyntaxError(f"{msg} at line {line}: {self.s[self.i:self.i+60]!r}")
+
+    def ws(self):
+        while self.i < len(self.s):
+            c = self.s[self.i]
+            if c in " \t\r\n":
+                self.i += 1
+            elif self.s.startswith("--", self.i):
+                self.i = self.s.find("\n", self.i)
+                if self.i < 0:
+                    self.i = len(self.s)
+            else:
+                return
+
+    def take(self, lit):
+        self.ws()
+        if self.s.startswith(lit, self.i):
+            self.i += len(lit)
+            return True
+        return False
+
+    def expect(self, lit):
+        if not self.take(lit):
+            self.error(f"expected {lit!r}")
+
+    def name(self):
+        self.ws()
+        m = re.compile(r"[A-Za-z_][A-Za-z0-9_]*").match(self.s, self.i)
+        if not m:
+            return None
+        self.i = m.end()
+        return m.group(0)
+
+    def value(self):
+        self.ws()
+        if self.take("{"):
+            return self.table()
+        m = re.compile(r"-?\d+(\.\d+)?").match(self.s, self.i)
+        if m:
+            self.i = m.end()
+            v = float(m.group(0))
+            return self.arith(int(v) if v.is_integer() else v)
+        if self.s[self.i] in "\"'":
+            q = self.s[self.i]
+            j = self.s.index(q, self.i + 1)
+            v = self.s[self.i + 1:j]
+            self.i = j + 1
+            return v
+        n = self.name()
+        if n is None:
+            self.error("expected a value")
+        if n == "nil":
+            return None
+        if n in ("true", "false"):
+            return n == "true"
+        if self.take("("):                       # a call
+            args = []
+            if not self.take(")"):
+                while True:
+                    args.append(self.value())
+                    if self.take(")"):
+                        break
+                    self.expect(",")
+            if n not in self.helpers:
+                self.error(f"unknown helper {n!r}")
+            return self.arith(self.helpers[n](*args))
+        return Symbol(n)                          # a function reference
+
+    def arith(self, left):
+        """+ and - between numbers.  Present in the source and load-bearing:
+        e.g. `time_laserdisc_frame(1823) - laserdisc_frame_to_ms(2)`."""
+        while True:
+            self.ws()
+            if self.i < len(self.s) and self.s[self.i] in "+-":
+                op = self.s[self.i]
+                self.i += 1
+                right = self.value()
+                left = left + right if op == "+" else left - right
+            else:
+                return left
+
+    def table(self):
+        d, arr = {}, []
+        while True:
+            self.ws()
+            if self.take("}"):
+                break
+            save = self.i
+            k = self.name()
+            if k is not None and self.take("="):
+                d[k] = self.value()
+            else:
+                self.i = save
+                arr.append(self.value())
+            if not (self.take(",") or self.take(";")):
+                self.expect("}")
+                break
+        if d and arr:
+            self.error("mixed array/hash table")
+        return arr if arr or not d else d
+
+
+class Symbol(str):
+    """A bare Lua identifier used as a value (a function reference)."""
+
+
+def load_dirksimple(path):
+    """Parse lair/game.lua's `scenes` and `scene_manager` tables.
+
+    Gates the four timing helpers against the text that defines them, so this
+    tool cannot keep evaluating a formula upstream has changed.
+    """
+    src = open(os.path.join(path, "data/games/lair/game.lua"), encoding="utf-8").read()
+
+    def gate(pat, what):
+        if not re.search(pat, src):
+            raise AssertionError(f"DirkSimple's {what} is not what this tool "
+                                 f"evaluates any more (looked for {pat!r})")
+    gate(r"frame\s*/\s*23\.976", "laserdisc_frame_to_ms")
+    gate(r"-\s*6297\.0", "time_laserdisc_frame's ROM offset")
+    gate(r"time_laserdisc_noseek.*?\n\s*return -1", "time_laserdisc_noseek")
+    gate(r"\(seconds \* 1000\) \+ ms", "time_to_ms")
+
+    helpers = {
+        "laserdisc_frame_to_ms": lambda f: (f / LD_FPS) * 1000.0,
+        "time_laserdisc_frame": lambda f: (f / LD_FPS) * 1000.0 - ROM_OFFSET_MS,
+        "time_laserdisc_noseek": lambda: -1,
+        # time_to_ms takes (seconds, ms); the source calls it with a third
+        # argument exactly once, which Lua discards.
+        "time_to_ms": lambda s, ms=0, *_: s * 1000 + ms,
+    }
+
+    out = {}
+    for var in ("scenes", "scene_manager"):
+        m = re.search(rf"^{var} = \{{", src, re.M)
+        if not m:
+            raise AssertionError(f"{var} table not found in game.lua")
+        p = LuaParser(src, helpers)
+        p.i = m.end()
+        out[var] = p.table()
+        # The parser must have consumed the WHOLE constructor: table() returns
+        # with the cursor just past the matching close brace, and in this file
+        # every top-level table ends on a `}` in column 0.  A parser that
+        # stopped early would return a smaller graph, which is the direction
+        # that flatters the measurement, so this is checked rather than assumed.
+        if src[p.i - 1] != "}":
+            raise AssertionError(f"{var}: parser did not end on a close brace")
+        if src.rfind("\n", 0, p.i) != p.i - 2:
+            raise AssertionError(f"{var}: the constructor ended mid-line at "
+                                 f"{p.i}, so the parse stopped early")
+    return out["scenes"], out["scene_manager"]
+
+
+# ------------------------------------------------- the SNES cross-check
+
+def load_snes(path):
+    """Parse the SNES project's chapter XMLs into {chapter: (start_ms, end_ms,
+    [(input, from_ms, to_ms, target)])}, plus the provenance line."""
+    ev = os.path.join(path, "data/events")
+    readme = os.path.join(ev, "README.md")
+    prov = None
+    if os.path.exists(readme):
+        txt = open(readme, encoding="utf-8").read()
+        if "derived from DirkSimple" in txt:
+            prov = "derived from DirkSimple game data (data/events/README.md)"
+
+    def ms(e):
+        return (int(e.get("min", 0)) * 60000 + int(e.get("second", 0)) * 1000
+                + int(e.get("ms", 0)))
+
+    chapters = {}
+    for f in sorted(os.listdir(ev)):
+        if not f.endswith(".xml"):
+            continue
+        root = ET.parse(os.path.join(ev, f)).getroot()
+        tl = root.find("timeline")
+        s = tl.find("timestart")
+        e = tl.find("timeend")
+        acts = []
+        for evt in root.findall("./events/event"):
+            if evt.get("type") != "direction":
+                continue
+            t = evt.find("timeline")
+            p = evt.find("./params/str[@key='type']")
+            r = evt.find("./result/playchapter")
+            acts.append((p.get("value") if p is not None else "?",
+                         ms(t.find("timestart")),
+                         ms(t.find("timeend")) if t.find("timeend") is not None else -1,
+                         r.get("name") if r is not None else None))
+        chapters[f[:-4]] = (ms(s) if s is not None else -1,
+                            ms(e) if e is not None else -1, acts)
+    return chapters, prov
+
+
+def map_abbrevs(chapters, scenes):
+    """Match each SNES scene abbreviation to a DirkSimple scene from the DATA,
+    not from a hand-written table, so a wrong match is visible rather than
+    assumed.  Three signals, in order:
+
+      1. the abbreviation's letters must be a SUBSEQUENCE of the scene name
+         (`snkr` is snake_room, and cannot be black_knight however similar
+         their `seqN` names are -- overlap alone got that one wrong);
+      2. the Jaccard overlap of the two sequence-name sets;
+      3. the reversed-scene rule: Dragon's Lair mirrors thirteen scenes, and
+         where BOTH `x` and `xr` exist as abbreviations, `xr` is the
+         `_reversed` scene and `x` is not.  Their sequence names are identical,
+         so nothing else separates them.
+
+    Assignment is 1:1 and greedy on that ranking.
+    """
+    def subseq(a, b):
+        it = iter(b)
+        return all(c in it for c in a)
+
+    by_abbr = {}
+    for ch in chapters:
+        abbr, _, seq = ch.partition("_")
+        by_abbr.setdefault(abbr, set()).add(seq)
+
+    pairs = []
+    for abbr, seqs in by_abbr.items():
+        mirror = abbr.endswith("r") and abbr[:-1] in by_abbr
+        for scene, s in scenes.items():
+            if not subseq(abbr, scene.replace("_", "")):
+                continue
+            union = len(seqs | set(s))
+            j = len(seqs & set(s)) / union if union else 0.0
+            rev = scene.endswith("_reversed")
+            pairs.append((j, 1 if rev == mirror else 0, -len(scene),
+                          abbr, scene, len(seqs)))
+    pairs.sort(reverse=True)
+
+    mapping, used = {}, set()
+    for j, _, _, abbr, scene, n in pairs:
+        if abbr in mapping or scene in used:
+            continue
+        mapping[abbr] = (scene, j, n)
+        used.add(scene)
+    unmatched = [(a, None, 0.0, len(s)) for a, s in by_abbr.items()
+                 if a not in mapping]
+    return mapping, unmatched
+
+
+
+
+# ------------------------------------------------------------------- emit
+
+def to_neutral(scenes, mgr):
+    """DirkSimple's tables -> DLXSCENE1.  The one place their shape is read."""
+    out = {}
+    for scene, seqs in scenes.items():
+        s = out.setdefault(scene, {})
+        for name, seq in seqs.items():
+            t = seq["timeout"]
+            s[name] = {
+                "start_ms": float(seq["start_time"]),
+                "timeout_ms": float(t["when"]),
+                "timeout_next": t.get("nextsequence"),
+                "kills": bool(seq.get("kills_player")),
+                "single_frame": bool(seq.get("is_single_frame")),
+                "actions": [{"input": a["input"],
+                             "from_ms": float(a["from"]),
+                             "to_ms": float(a["to"]),
+                             "next": a.get("nextsequence")}
+                            for a in seq.get("actions", [])],
+            }
+    rows = [list(r) for r in mgr["rows"]]
+    return out, rows
+
+
+def crosscheck_neutral(chapters, scenes):
+    """The SNES chapters, keyed the way OUR table is keyed.
+
+    Their `abbr_sequence` naming is resolved to a scene here, from the data
+    (see map_abbrevs), so nothing downstream has to know an abbreviation
+    exists.  Chapters the mapping cannot place are dropped and counted.
+    """
+    mapping, unmatched = map_abbrevs(chapters, scenes)
+    out, dropped = {}, 0
+    for ch, (s_ms, e_ms, acts) in chapters.items():
+        abbr, _, seq = ch.partition("_")
+        if abbr not in mapping:
+            dropped += 1
+            continue
+        scene = mapping[abbr][0]
+        pre = abbr + "_"
+        out.setdefault(scene, {})[seq] = {
+            "chapter": ch,
+            "start_ms": float(s_ms),
+            "end_ms": float(e_ms),
+            "actions": [{"input": i, "from_ms": float(f), "to_ms": float(t2),
+                         "next": (n[len(pre):] if n and n.startswith(pre) else n)}
+                        for i, f, t2, n in acts],
+        }
+    weak = sorted((round(j, 3), ab, sc) for ab, (sc, j, _n) in mapping.items()
+                  if j < 1.0)
+    return out, {"chapters": len(chapters), "dropped": dropped,
+                 "scenes_mapped": len(mapping),
+                 "weakest_mappings": weak[:6]}
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("-o", "--out", default="tmp/scenegraph.json")
+    a = ap.parse_args()
+
+    if not os.path.isdir(DIRK):
+        sys.exit(f"no DirkSimple checkout at {DIRK} -- set DLX_DIRKSIMPLE, or\n"
+                 f"  git clone --depth 1 https://github.com/icculus/DirkSimple")
+    scenes, mgr = load_dirksimple(DIRK)
+    neutral, rows = to_neutral(scenes, mgr)
+    nseq = sum(len(s) for s in neutral.values())
+    nwin = sum(len(q["actions"]) for s in neutral.values() for q in s.values())
+
+    cross, cross_meta = None, None
+    if os.path.isdir(SNES):
+        chapters, prov = load_snes(SNES)
+        cross, cross_meta = crosscheck_neutral(chapters, scenes)
+        cross_meta["provenance"] = prov
+        # 56.2: this is the sentence the whole "second source" claim turned on.
+        # If upstream ever removes it, say so rather than silently promoting a
+        # copy back to an independent transcription.
+        if not prov:
+            cross_meta["provenance"] = ("UNSTATED -- data/events/README.md no "
+                                        "longer says where the chapters came "
+                                        "from; re-check before trusting this "
+                                        "as anything.")
+
+    doc = {"format": "DLXSCENE1", "sources": SOURCES,
+           "scenes": neutral, "rows": rows,
+           "counts": {"scenes": len(neutral), "sequences": nseq,
+                      "windows": nwin},
+           "crosscheck": cross, "crosscheck_meta": cross_meta}
+    os.makedirs(os.path.dirname(a.out) or ".", exist_ok=True)
+    with open(a.out, "w") as f:
+        json.dump(doc, f, indent=1, sort_keys=True)
+    print(f"{a.out}: {len(neutral)} scenes, {nseq} sequences, {nwin} input "
+          f"windows, {len(rows)} rows"
+          + (f", cross-check {cross_meta['chapters']} chapters"
+             if cross_meta else ", no cross-check (SNES tree absent)"))
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

tools/media/make_packed_media.py

@@ -0,0 +1,235 @@
+#!/usr/bin/env python3
+"""The README still for the PACKED player -- ROADMAP K3, FINDINGS 64.
+
+    python3 tools/media/make_packed_media.py [container.dlxp]
+            [--snap tmp/snap_packed_gate] [--map tmp/packed_snaps_gate.csv]
+            [--src tmp/fr_singe] [--frame N] [--out docs/img/packed-player.png]
+
+Blu-ray source | what the emulated 68000 actually put on screen. The right-hand
+panel is MAME's own snapshot, de-double-scanned and cropped to the picture -- not
+a re-render, not `dlxp.render`. It is the same rule the codec's still is built on
+(tools/media/make_readme_media.py) and it is the only reason the picture is worth
+printing: an encoder can be checked against its own inverse, and a screen cannot.
+
+THE FRAME IS CHOSEN, NOT PICKED. --frame defaults to the one whose PSNR against
+the source is CLOSEST TO THE MEAN over the whole gated window, so the still is
+representative rather than flattering. The chosen frame and its distance from the
+mean are printed, so a reader can see it was not the best one.
+"""
+import argparse, csv, os, shutil, struct, subprocess, sys, wave
+sys.path.insert(0, "tools/encoder")
+import numpy as np
+from PIL import Image, ImageDraw
+from dlxp import DLXP
+
+ap = argparse.ArgumentParser()
+ap.add_argument("container", nargs="?", default="tmp/packed_singe.dlxp")
+ap.add_argument("--snap", default="tmp/snap_packed_gate")
+ap.add_argument("--map", default="tmp/packed_snaps_gate.csv")
+ap.add_argument("--src", default="tmp/fr_singe")
+ap.add_argument("--frame", type=int, default=None)
+ap.add_argument("--out", default="docs/img/packed-player.png")
+ap.add_argument("--webm", default="docs/img/packed-player.webm")
+ap.add_argument("--wav", default="tmp/packed_aud.wav",
+                help="MAME's own -wavwrite capture from run 5 of "
+                     "packed_run.sh; --no-audio drops it")
+ap.add_argument("--no-audio", action="store_true")
+ap.add_argument("--no-webm", action="store_true")
+ap.add_argument("--fps", type=float, default=None,
+                help="clip rate; defaults to the container's own")
+a = ap.parse_args()
+
+d = DLXP(a.container)
+SNAP_W, SNAP_H = 256, 512
+with open(a.map) as fh:
+    shot = {int(r["frame"]): r["snapshot"] for r in csv.DictReader(fh)}
+if not shot:
+    sys.exit(f"{a.map} is empty -- run tools/bench/packed_run.sh first")
+
+
+def screen(fr):
+    """The 256x192 picture out of one MAME native snapshot."""
+    p = f"{a.snap}/x68000/{shot[fr]}.png"
+    s = np.asarray(Image.open(p).convert("RGB"))
+    if s.shape[:2] != (SNAP_H, SNAP_W):
+        sys.exit(f"{p}: expected {SNAP_W}x{SNAP_H}, got {s.shape[1]}x{s.shape[0]}")
+    g = s[0::2]                                   # undo the double scan
+    y = (g.shape[0] - d.H) // 2                   # the picture is centred
+    return g[y:y + d.H]
+
+
+def source(fr):
+    p = f"{a.src}/f{fr+1:04d}.png"
+    if not os.path.exists(p):
+        sys.exit(f"missing {p} -- re-extract the frames the container was built "
+                 f"from, or point --src at them")
+    return np.asarray(Image.open(p).convert("RGB"))
+
+
+def psnr(x, y):
+    e = ((x.astype(float) - y.astype(float)) ** 2).mean()
+    return float("inf") if e == 0 else 10 * np.log10(255.0 ** 2 / e)
+
+
+frames = sorted(shot)
+scores = {f: psnr(source(f), screen(f)) for f in frames}
+mean = float(np.mean(list(scores.values())))
+if a.frame is None:
+    pick = min(scores, key=lambda f: abs(scores[f] - mean))
+else:
+    pick = a.frame
+    if pick not in scores:
+        sys.exit(f"frame {pick} was not sampled by that run")
+
+# THE PANEL IS GATED, not just drawn. A still of the player is a claim that the
+# player drew it, and the snapshot has to still be pixel-exact against the
+# container for that claim to hold -- verify_packed.py checks all of them and
+# this checks the one being printed, so the picture cannot outlive the result.
+ref = d.render(pick)
+if not np.array_equal(screen(pick), ref):
+    sys.exit(f"frame {pick} is NOT pixel-exact against {a.container}. The still "
+             f"is not being written: it would be a picture of a failure with a "
+             f"caption saying otherwise.")
+
+Z, BAR = 2, 22
+
+
+def captioned(img, text):
+    up = np.repeat(np.repeat(img, Z, 0), Z, 1)
+    out = Image.new("RGB", (up.shape[1], up.shape[0] + BAR), (16, 16, 18))
+    out.paste(Image.fromarray(up), (0, BAR))
+    ImageDraw.Draw(out).text((6, 6), text, fill=(190, 190, 196))
+    return out
+
+
+left = captioned(source(pick), "Blu-ray source, cropped 256x192")
+right = captioned(screen(pick),
+                  "emulated 68000, MAME's own snapshot, no decoder")
+out = Image.new("RGB", (left.width + right.width + 8, left.height), (16, 16, 18))
+out.paste(left, (0, 0))
+out.paste(right, (left.width + 8, 0))
+os.makedirs(os.path.dirname(a.out), exist_ok=True)
+out.save(a.out)
+print(f"{a.out}: frame {pick} of {d.nframes}, {scores[pick]:.2f} dB against the "
+      f"24-bit source")
+print(f"  chosen as the frame CLOSEST TO THE MEAN ({mean:.2f} dB over "
+      f"{len(frames)} gated frames), {abs(scores[pick]-mean):.3f} dB from it -- "
+      f"best in the window is {max(scores.values()):.2f}, worst "
+      f"{min(scores.values()):.2f}")
+print(f"  and it is pixel-exact against {a.container}, checked before writing")
+
+
+# ---------------------------------------------------------------------------
+# THE CLIP.  ROADMAP K3's result is a SEQUENCE -- 120 records off a real volume,
+# every one of them a literal -- and a still cannot show the one property that
+# distinguishes this branch from the codec's: there is no recursion here, so
+# frame 119 says nothing about frame 60 and every frame has to be its own claim.
+# This writes all of them, and gates all of them before writing any (64.1).
+#
+# WHAT IT IS, EXACTLY, because two runs of packed_run.sh are in it:
+#   picture   run 1, the GATE run -- paced at half the container's rate so the
+#             snapshot lands inside the write window, cycle stealing, no sound.
+#             Every panel is MAME's own snapshot, 2x nearest, no filtering.
+#   sound     run 5, the AUDIO run -- the same container at 12 fps with the
+#             MSM6258 on channel 3, captured by MAME's -wavwrite off the
+#             speaker. It is the chip's stream, not the encoder's.
+# They are two runs because they have to be: the gate run is at 6 fps and audio
+# cut at 12 fps played at 6 is not this scene. The clip is therefore a
+# COMPOSITE, and saying so is the point -- what it is NOT is a real-time capture
+# of the shipping configuration, which at the container's own burst rate would
+# show a blank layer for 99.5% of every slot (FINDINGS 64.2).
+if not a.no_webm:
+    if not shutil.which("ffmpeg"):
+        sys.exit("ffmpeg not found -- needed for the webm (--no-webm skips it)")
+    fps = a.fps or d.fps
+
+    # EVERY frame gated, not the printed one. A packed frame is a literal: the
+    # codec's last-frame test audits 120 through its own recursion and nothing
+    # here does, so a clip of 120 frames is 120 separate claims.
+    bad = [f for f in frames if not np.array_equal(screen(f), d.render(f))]
+    if bad:
+        sys.exit(f"{len(bad)} of {len(frames)} frames are NOT pixel-exact "
+                 f"against {a.container} (first {bad[0]}). The clip is not "
+                 f"being written: it would be a recording of a failure.")
+
+    aud = None
+    if not a.no_audio and d.has_audio and os.path.exists(a.wav):
+        # THE ALIGNMENT IS FOUND, NOT ASSUMED, and then CHECKED. The capture
+        # opens with the machine booting, so the stream starts at the first
+        # non-zero sample -- and "first non-zero" is exactly the kind of thing
+        # that is off by one byte forever with every counter in the player
+        # agreeing (FINDINGS 71.5). So lump 0 is decoded with the FOUR AXES OUT
+        # OF THE CONTAINER'S OWN HEADER and required to be sample-exact from
+        # there. Only lump 0: past it the seams need the walk in
+        # tools/bench/verify_packed_audio.py, which is what gates the whole
+        # stream in check.sh. This gates the cut.
+        sys.path.insert(0, "tools/encoder")
+        import adpcm
+        SCALE = 8                        # okim6258's signal<<4 at gain 0.50
+        w = wave.open(a.wav)
+        nfr, ch, rate, sw = (w.getnframes(), w.getnchannels(),
+                             w.getframerate(), w.getsampwidth())
+        if rate != d.aud_hz or sw != 2:
+            sys.exit(f"{a.wav}: {rate} Hz / {sw*8}-bit -- the capture has to be "
+                     f"the chip's own {d.aud_hz} Hz or a resampler is in the "
+                     f"measurement")
+        raw = w.readframes(nfr)
+        left = struct.unpack("<%dh" % (nfr * ch), raw)[0::ch]
+        rec = [round(v / SCALE) for v in left]
+        start = next((i for i, v in enumerate(rec) if v), None)
+        if start is None:
+            sys.exit(f"{a.wav} is silent -- run 5 of packed_run.sh writes it")
+        dec = d.decoder()
+        l0 = d.lump(0)
+        want = adpcm.decode(adpcm.unpack(l0, order=dec["order"]),
+                            variant=dec["variant"], init=dec["init"],
+                            bits=dec["bits"])
+        got = rec[start:start + len(want)]
+        if got != list(want):
+            n = sum(x != y for x, y in zip(got, want))
+            sys.exit(f"the cut at capture sample {start:,} does not decode as "
+                     f"lump 0: {n:,} of {len(want):,} samples differ. The clip "
+                     f"is not being written -- the sound would be the right "
+                     f"scene from the wrong byte.")
+        n_out = int(round(len(frames) / fps * rate))
+        aud = "tmp/_packed_media_audio.wav"
+        ow = wave.open(aud, "wb")
+        ow.setnchannels(ch); ow.setsampwidth(sw); ow.setframerate(rate)
+        ow.writeframes(raw[start * ch * sw:(start + n_out) * ch * sw])
+        ow.close()
+        print(f"  audio: {a.wav}, cut at sample {start:,} "
+              f"({start/rate:.2f} s of boot dropped), {n_out/rate:.2f} s -- "
+              f"lump 0 sample-exact against the container's own axes "
+              f"({dec['variant']}/{dec['order']}, {dec['bits']}-bit, "
+              f"init {dec['init']})")
+    elif not a.no_audio:
+        print(f"  no audio: {a.wav} is missing or the container is silent")
+
+    tmpd = "tmp/_packed_media_frames"
+    shutil.rmtree(tmpd, ignore_errors=True)
+    os.makedirs(tmpd)
+    for n, f in enumerate(frames):
+        l = captioned(source(f), "Blu-ray source, cropped 256x192")
+        r = captioned(screen(f), "emulated 68000, MAME's own snapshot, "
+                                 "no decoder in the machine")
+        im = Image.new("RGB", (l.width + r.width + 8, l.height), (16, 16, 18))
+        im.paste(l, (0, 0)); im.paste(r, (l.width + 8, 0))
+        im.save(f"{tmpd}/{n:04d}.png")
+
+    # VP9 near-lossless: this is 256x192 palettised pixel art scaled by an
+    # integer, and a codec that smooths a colour boundary would be editorialising
+    # about the one thing the picture is evidence of.
+    cmd = ["ffmpeg", "-y", "-loglevel", "error", "-framerate", f"{fps:g}",
+           "-i", f"{tmpd}/%04d.png"]
+    if aud:
+        cmd += ["-i", aud, "-c:a", "libopus", "-b:a", "96k", "-shortest"]
+    cmd += ["-c:v", "libvpx-vp9", "-crf", "12", "-b:v", "0",
+            "-pix_fmt", "yuv444p", "-row-mt", "1", a.webm]
+    subprocess.run(cmd, check=True)
+    shutil.rmtree(tmpd)
+    if aud:
+        os.remove(aud)
+    print(f"{a.webm}: {len(frames)} frames @ {fps:g} fps, "
+          f"{os.path.getsize(a.webm)/1024:.0f} KB, every frame pixel-exact "
+          f"against {a.container}"
+          + (" -- with the chip's own audio" if aud else " -- silent"))