# Status & next-session handoff — session 8 (2026-08-23) ## Where this stands The decoder exists, it is pixel-exact, and **it does not fit**: mean 81.7% of a 12fps frame on the worst sustained window at `sasi`, 31% of frames over budget (`scsi`: 94.9% median, 42% miss). FINDINGS 28. CPU is the binding constraint. Session 7 proposed two levers and session 8 measured the cheap one first. **Lever A — spend bandwidth to buy cycles — is real, and it is an encoder format.** A row-linear span of word-expanded literals measures **43.7 cycles per span + 9.152 per pixel** (FINDINGS 30, `tools/bench/span.sh`), which is what FINDINGS 29 assumed — but only when the *encoder* hands the decoder an absolute GVRAM address and a jump displacement into an unrolled copy chain. The obvious decoder, handed `(x, npix)` and left to work the copy out, is 97.9 + 10.46 and 2.2x dearer on a short span. Re-priced against the unchanged mode maps: | | today | 29 (derived) | **30 (measured)** | |---|---:|---:|---:| | `sasi` median frame | 74.4% | 43.0% | **52.0%** | | `sasi` worst frame | 136.2% | 106.2% | **108.7%** | | `sasi` frames missing | 37/120 | 8/120 | **10/120** | | bitrate | 101.7 KB/s | 453.2 | **448.0 KB/s** (bus 488) | Break-even moved with it: a run beats all-V1 **from 4 blocks up**, not 2. And 29.4 survives — a scene cut needs `x >= 0.196` of the frame as spans and the bus allows `x <= 0.373`, so it fits at 12fps. **Lever B — stop buying modes the CPU cannot afford — is untouched.** `vq_hybrid.decide()` still minimises `D + lam*R`, distortion against BYTES, on a machine whose binding budget is CYCLES: | mode | payload bytes | cycles | cycles per byte | |---|---:|---:|---:| | SKIP | 0 | 13 (clustered) | — | | V1 | 1 | 300 | 300 | | V4 | 4 | 448 | 112 | | RAW | 16 | 400 | 25 | | **span, per 4x4 block in a run of L** | **32** | **1053/L, floor 154** | **~5** | V4 is 25% of blocks and 50% of the cycles. The lagrangian charges it 4x a V1 block; the CPU charges it 1.49x. ## The work, in order 1. **Add a cycle term to the mode decision.** `decide()` already builds a cost matrix of `error + lam * bytes` per mode per block; add `+ mu * cycles`, with the per-mode cycles measured in FINDINGS 28.2. **SKIP is not a constant and this is the one trap here.** A SKIP block costs 13.25 cycles when all four blocks in its header byte are SKIP (one `tst.b` clears the group) and ~45 when it sits in a mixed byte — so SKIP's price depends on its *neighbours*, which a per-block lagrangian cannot see. Do not pick one number and move on: 45 overcharges clustered SKIPs and pushes the encoder away from the mode that saves cycles, 13.25 undercharges isolated ones and lets frames overrun. The way out is that the **budget check does not have to use the same cost function as the mode decision** — score frames with the exact clustered cost (`cycles()` in `tools/analysis/11_cpu_budget.py`, validated to 1 point against the 68000) and let the bisection converge on that, while the per-block term uses a constant purely to *rank* modes. 2. **Then bisect `mu` per frame against the 833,333-cycle budget**, exactly as session 6 bisects `lam` against the byte budget. The machinery is already there and already gated: `ratectl.encode_rate_controlled` is frame-driven and feeds back the frame it emitted. **But cycles have NO bucket.** Bytes can be banked in the ring buffer; a frame that misses its decode deadline is just late, because there is no double buffer to decode ahead into. So this is a hard per-frame ceiling, not a leaky bucket — simpler than rate control, and the two controllers have to run together (raising `mu` moves blocks to SKIP and V1, which also *lowers* the bitrate, so the byte controller must see it). 3. **Measure the quality cost.** Everything session 6 did for bytes: what does fitting 100% of frames in the CPU budget cost in dB, and does any frame hit a cliff? `tools/analysis/11_cpu_budget.py` scores a container without needing MAME, so the search loop is cheap; confirm the winner on the 68000 with `tools/bench/decode.lua`. 3b. **Know which misses are yours to fix before starting.** Re-coding every non-SKIP block as V1 is the floor any mode assignment *of the current mode set* can reach, and it still misses 11 frames at `sasi` and 12 at `scsi` — every frame above ~90% non-SKIP. So the cost-aware decision can reach about three quarters of the misses (26 of 37 at `sasi`) and the rest need item 4. FINDINGS 28.7. 3c. **Buy RAW, not V4, wherever the bytes allow.** RAW is 400 cycles against V4's 448 *and* is pixel-exact, so on the CPU axis V4 is strictly dominated — the byte lagrangian's preference inverts. `scsi` can take that escape and `sasi` cannot afford it, so expect the cycle ceiling to cost `sasi` more quality even though it costs `sasi` fewer cycles. FINDINGS 28.8. 4. **Put spans in the bitstream** — the mode is measured and nothing implements it. This is a container change (`encode.py`, `dlx.py`, `decode.s`), a mode decision that can see runs rather than blocks, and the 24-pixel quantisation and row-overrun rules of FINDINGS 30.2. It subsumes item 4 of session 7's plan: with spans, a scene cut fits. 5. **The three things 30.7 leaves open, now more load-bearing than before**, because the span design runs at 448 KB/s of a 488 KB/s pipe: re-run the ring-buffer simulation at that rate (FINDINGS 21 was established at 110 and 280), confirm the provenance of the 4 Mbps figure, and **confirm DMA rather than PIO** — a PIO fallback puts a 448 KB/s transfer back on the CPU this whole lever exists to relieve. The DMA check is the cheapest of the three. **Do not start by hand-optimising `decode.s`.** The hand-derived timings agree with the measurements to 0.5% on V1 and 1% on RAW (FINDINGS 28.4), so the inner loop is close to what the instruction set allows; the plausible wins are single-digit percentages against a 36-point gap. The V4 write pattern is the one place worth a look afterwards — pairing sub-block rows into `movem.l d0/d2,(a4)` saves ~16 of 448 cycles. --- ## What session 8 settled 1. **The span is measured: 43.7 cycles/span + 9.152/pixel, fitted to 0.3% over eleven span lengths.** `tools/bench/blit.s` v5/v6, `prep_spans.py`, `span.lua`, driven by `tools/bench/span.sh` (~25 s, not in `check.sh` because it is a wall timing). FINDINGS 30. 2. **Only in an encoder-assisted format.** `{u32 absolute GVRAM address, u16 jump displacement}` into an unrolled chain, versus `(x, npix)` and a decoder that works it out: 43.7 + 9.152 against 97.9 + 10.46. All the arithmetic a span decoder would do per frame is known at encode time. FINDINGS 30.2. 3. **The per-pixel cost is a function of register pressure**, which FINDINGS 24 could not have shown: 9.08 was a fixed blit with 12 registers free, v5 can spare 8 and pays 10.46, v6 gets 12 back by making the encoder hold the state. 4. **Short spans die in the remainder path, and the fix is padding.** A 12-pixel span costs more than a 16-pixel one in v5. v6 has no remainder path: lengths are multiples of 24 pixels, padding is free of everything but bytes, and an overrun past the visible 256 lands in the invisible half of the 1024-byte line stride. FINDINGS 30.3. 5. **Odd-`x` alignment is free** (259.0 vs 261.8 cycles/span) — expected on a 16-bit bus, now measured rather than assumed. 6. **The trade is smaller than 29 derived but the conclusion holds**, including 29.4's reopening of the scene cut. All 23 timing configs also drew a pixel-exact frame, so nothing here was timed against a decoder that skipped work. FINDINGS 30.5/30.6. --- ## What session 7 settled 1. **68000 code parses a bitstream and draws frames, pixel-exact.** `src/player/decode.s` + `tools/bench/decode.lua`. 120 frames of the Singe window decoded in sequence, all four block modes, verified against the new reference decoder `tools/encoder/dlx.py`. Because SKIP blocks are claims about the previous frame, the last frame is only right if all 120 were. In `check.sh` now. **FINDINGS 28.** 2. **It does not fit.** Mean 81.7% of a 12fps frame, p90 116.4%, worst 135.8%; 31% of frames miss at `sasi`, 42% at `scsi`. Zero-wait-state floor, as ever. 3. **The dual-display-path plan (FINDINGS 24.5/25.6) is withdrawn as incoherent — the sixth false premise this project has caught.** The compose path needs a RAM copy of the previous reconstruction; the direct path's whole selling point is that it keeps none. Mixing them displays stale pixels on **70 of 120 frames**, worst frame 18.8% of the screen. Every coherent repair is worse than not mixing. `tools/analysis/10_pathmix_drift.py`, kept runnable as a counterexample and gated in `check.sh`. FINDINGS 28.1. 4. **24.5 also compared a copy against a copy.** Its 53.6% and 76.6% both come from `blit.s` and neither includes decoding. Compose = decode-into-RAM *plus* the 53.6% blit, so it is strictly dearer than decoding into GVRAM. There was never a crossover. The player has **one path and no reference frame**, which also gives back 96 KB. 5. **The four block modes cost 300 / 448 / 400 cycles, not one number.** V4 is 1.49x a V1 block while the mode decision charges it 4x the bytes. The 24.5 model is 2.03x optimistic at the median. `tools/analysis/11_cpu_budget.py` reproduces all four frames timed on the 68000 to within 1 point. FINDINGS 28.2. 6. **The container is big-endian but not aligned, and on a 68000 that is an address error, not a slow read.** Frame records are variable-length and laid end to end, so their boundaries land on odd addresses. Frame 0 decoded perfectly, then the length read for frame 1 vectored into the IPL and sat there for 59 emulated seconds looking like an infinite loop. Found by dumping PC and the address registers — the code was right, the data layout was not. FINDINGS 28.3. **Encoder gap: `encode.py` should pad records to 4 bytes.** Measured cost 1.66 B/frame = 20 B/s against 110 KB/s. 7. **A full frame does not fit at 12fps in any mode.** All-V1 is 110.5%, all-V4 165.2%, all-RAW 147.6%. At most ~88% of the screen can change in one frame however cheaply it is coded, and scene cuts change 100%. FINDINGS 28.5. --- ## What session 6 settled 1. **Rate control works, is wired in, and is ON by default.** `encode.py` bisects lam per frame under a leaky bucket; `--fixed-lam` restores session 5 behaviour. FINDINGS 27. 2. **Both overshoots are closed for under 1 dB.** On the Singe window, totals including audio: `sasi` 137.4 -> **109.5 KB/s** (target 110) for -0.60 dB, `scsi` 381.6 -> **280.0 KB/s** (target 280) for -0.91 dB. Zero frames hit the lam=800 cliff at either profile. FINDINGS 27.2. 3. **The FINDINGS 26 desync is gone by construction, not by tuning.** The encoder is frame-drivable (`vq_hybrid.frame_ctx` / `decide` / `paint`) and rate control feeds back the frame it actually emitted. The regression test `tools/analysis/09_ratectl_drift.py` goes 111/120 drifting frames -> **0**, and it is now part of `./tools/bench/check.sh`. FINDINGS 27.1. 4. **Rate control makes the display path cheaper.** Raising lam moves blocks to SKIP and V1, so there is less to write: `scsi`'s median display cost drops 53.6% -> 47.1%. The decoder conclusion of 25.6 is unaffected. FINDINGS 27.3. 5. **FINDINGS 26.5 was wrong in both halves, and this is the fifth false premise this project has caught.** `_paint` was not the bottleneck (14% of a frame) and the ladder was never "minutes" (~18 s; the minutes were k-means in `build`). Vectorising it was still right — 17.1x — but what actually makes per-frame rate control affordable is that `VQ.assign` depends on neither `lam` nor `prev`, so it is cached: a 12-step search over 120 frames costs **0.31 s** against 49.1 s. FINDINGS 27.6. 6. **`--prefill` is a trap and defaults to 0.** It buys a permission to overshoot of exactly bucket/nframes; at prefill=1.0 the Singe window goes to 116.3 KB/s against a 110 ceiling, and on a 14-frame clip it disables the controller outright. FINDINGS 27.4. 7. **Fixed-lam `sasi` was already 5% over target on 00020**, the clip everyone called easy — nothing noticed because the profile table quotes PSNR, not bitrate. FINDINGS 27.5. 8. **1.2-second clips cannot be used to judge rate control.** The bucket's startup transient is bucket/nframes: 6% on a 10 s window, 20% on 00020. Same lesson as FINDINGS 25.3, different costume. --- ## Start here: is the tree still green? ``` ./tools/bench/check.sh ``` ~3 min, needs the Blu-ray mounted. From source media it re-runs both display regression tests, the rate-control drift test (session 6), the display-path coherency counterexample and a **120-frame 68000 decode** (session 7), then prints `ALL GREEN`. Verified green at end of session 7. If it fails, fix that before doing anything else — everything downstream assumes the display path is pixel-exact. The two session-7 stages are worth knowing the shape of before they fail on you: - `10_pathmix_drift.py` is expected to exit **non-zero**; `check.sh` fails if it ever starts passing, because that would mean the counterexample behind the one-path decoder had stopped reproducing. - the decode stage needs `tmp/rc_fr_singe_sasi_rcprofile.dlx` and will spend ~55 s encoding it if it is missing, nearly all of that k-means in `H.build`. ## Decisions locked | decision | value | why | |---|---|---| | Target CPU | 68000 @ 10MHz (stock) | hardest honest constraint | | Display mode | 256 colors, 256x192 in 256x256 CRTC mode | every mode is 1 word-access/pixel, so 256c is free vs 16c | | Double buffer | **none** — page 1 sacrificed | enables `movem.l` 24px bursts; delta coding needs a RAM reference frame anyway | | **Codec** | **hybrid VQ: SKIP / V1 4x4 / V4 four-2x2 / RAW, per-block rate-distortion** | flat 4x4 VQ was measured and rejected — see FINDINGS 9-10 | | **Quality modes** | **two: `sasi` and `scsi`** (USER DECISION, session 2) | one codec, one decoder, one bitstream; only `lam` differs | | Profile axis | **I/O bandwidth only** | the profiles say nothing about CPU; **both target the same stock 10MHz 68000**, and the Super has SCSI at 10MHz. FINDINGS 28.7 | | Framerate | 12 fps, **explicit decimation** | source has zero duplicate frames; no free "twos" win | | Emulator | MAME 0.277 x68000 | accurate enough that measured cycles mean something | | SNES project reuse | **MIT — cleared** | `data/events/` scene graph is reusable with attribution | ### The SASI/SCSI question is RESOLVED Session 1 left "which machine do we target" open. The user's answer: **ship both**, as two quality profiles. This is now implemented rather than hypothetical — the bitrate ceiling is a build parameter in `tools/encoder/ratectl.py`: | profile | target | lam | quality (00020 / 00146) | machine | |---|---|---|---|---| | `sasi` | 110 KB/s | 60 (floor) | 36.9 / 29.6 dB | stock 10MHz ACE/EXPERT | | `scsi` | 280 KB/s | 10 (floor) | 39.4 / 32.3 dB | Super/XVI, or CZ-6BS1 board | **That "machine" column is about the BUS, not the CPU.** The profiles are an I/O-bandwidth axis and say nothing about clock speed: the X68000 Super has built-in SCSI at 10 MHz (`x68k.cpp:1194`, `40_MHz_XTAL/4`, same as the base machine), and only the XVI is faster. **Both profiles target the same stock 10 MHz 68000**, so both must fit the same 833,333-cycle frame budget — and as of session 7 neither does. FINDINGS 28.7. **As of session 6 `lam` is a floor, not a setting.** The target is a ceiling and the encoder bisects lam per frame to stay under it; the profile's lam is the best quality it is allowed to spend on a quiet frame. On the worst sustained window that takes `sasi` from 137.4 to 109.5 KB/s and `scsi` from 381.6 to 280.0 KB/s, for -0.60 and -0.91 dB. FINDINGS 27.2. Sized against the user's working figure of **4 Mbps = 488 KB/s sustained**, on SD-backed SCSI (BlueSCSI / SCSI2SD) — so that rate is a bus-limited **constant**, not an average over seek latency. **Both profiles fit with room.** Ring-buffer simulation on the real per-frame sizes gives **zero required prefill** for every scene at both profiles: the fill delivers 40.69 KB per frame time and only one measured frame (42.10 KB) exceeds that, recovered by the next. A 256 KB buffer carries ~1 s of stall tolerance, far more than an SD-backed seek needs. FINDINGS 21. An earlier warning here said `scsi` did not fit because a frame peaked at 96.4% of the pipe. That compared instantaneous demand to a sustained rate as if they had to match frame-by-frame; with a buffer the test is cumulative, and it passes. `scsi` is now within **0.5 dB of the palette ceiling** on 00020. These were initially set at 45 / 75 KB/s, which was 12% / 7% bus utilisation — read off the RD curve rather than derived from the hardware. See FINDINGS 17. Codebooks are **k=256 with 1-byte indices** in both profiles. k=1024 was measured and rejected — see FINDINGS 14, it was a false-good result from a rate model that undercharged the index. Do not ship past `lam~800`; FINDINGS 15 has the cliff. Because of the RAW escape mode, `lam=0` is **pixel-exact** against the palettised frame (measured 0.00 dB loss). The profiles are two points on one continuous rate-distortion curve, not two codecs. --- ## What session 3 settled 1. **The display path works and is verified end to end.** First real frame on an emulated X68000 screen: `docs/images/x68k_first_frame_compare.png`. Full write-up in **FINDINGS 22**. Everything before this session was Python-side or a headless `-video none` run, which cannot snapshot at all. 2. **The render is pixel-exact, not merely close.** With monitor contrast at 15, all 256 palette entries render exactly as `GGGGGRRRRRBBBBBI` + `pal6bit` predicts. That exactness is the regression test — see `tools/bench/verify_frame.py`, which exits non-zero if it ever drifts. 3. **Three hardware facts that were previously assumed are now confirmed from MAME 0.277 source**, not folklore: the palette word format, the 1024-byte GVRAM line stride, and the 256-colour page aliasing in `HARDWARE.md`. All three were already written down correctly; they are now cited. 4. ~~**A new quality ceiling was measured** — the 15-bit+I palette alone costs 38.88 dB.~~ **Superseded by session 4:** that figure assumed the shared LSB `I` is always 1. Chosen per entry, the ceiling is **40.81 dB**. FINDINGS 23.3. 5. **Two shell traps that wedged session 2's background jobs** are documented in the working-setup section below. They cost ~1.5 h of wall clock and a wedged CPU core, and one of them was hit again this session. ## What session 5 settled 1. **68000 code drew a frame, and the blit was measured.** `tools/bench/blit.s` + `blit.lua`. The snapshot passes `verify_frame256.py` unchanged — pixel-exact in the real 256x256 mode. **FINDINGS 23.5 is closed**: no longer "proven from Lua only". 2. **The 38% full-frame blit estimate is dead. It is 53.6%.** And that is a zero-wait-state floor — MAME models no GVRAM wait states, so real hardware is worse. FINDINGS 24. Every variant was hand-derived from the MC68000 timing tables before being measured and the two agree to 0.006-0.43%, so this is not another MAME artefact. 3. **Reading the source frame is exactly half the blit cost** (V1 53.6% vs a write-only floor V3 of 27.1%). That is what makes the architecture question below live. 4. **That number is now measured, and the answer is "implement both paths".** On the worst sustained window found on the disc, 30% of frames (`sasi`) to 53% (`scsi`) sit above the 70% crossover and want the flat blit; the rest want direct-to-GVRAM. A player that picks per frame — the mode headers are parsed before any pixel is written, so the count is free — pays a **median 37.0%** and is **capped at 53.6%**. FINDINGS 25.6. 5. **The sustained action sequence exists, was found by measurement, and breaks both profiles.** `tools/analysis/07_motion_survey.py` scans a whole stream for the hottest sliding window; on 00223 it is t=539.4s, the Singe endgame, at 2.01x the stream mean. There, fixed-lam `sasi` overshoots 110 -> 129.6 KB/s (+18%) and `scsi` 280 -> 373.8 KB/s (+34%). **Rate control is no longer insurance — it is required.** FINDINGS 25.3. 6. **The two largest streams on the disc are bonus material, not game footage.** 00216 is the feature with a burned-in commentary PiP; 00215 is the commentary itself. **00223 (9.4 min) is the clean one.** A size-ranked survey would have encoded live action. FINDINGS 25.1. 7. **Rate control is unsound as written, caught before wiring it up.** The lam-ladder in `ratectl.py` picks frames from independent temporal chains, so SKIP blocks reference reconstructions the decoder never saw: 111 of 120 frames drift, worst frame 43.4%, reported PSNR overstated 0.36 dB. Regression test `tools/analysis/09_ratectl_drift.py`. FINDINGS 26. 8. **On hard content the scene palette, not the display, is the binding ceiling** — 31.33 dB on the Singe window against 39.90 dB on 00020 and 40.81 dB for the X68000 display. `scsi` is already within 0.51 dB of it. FINDINGS 25.4. ### Superseded within session 5 4a. **The decoder architecture hinged on one unmeasured number.** Writing codewords straight into GVRAM costs 76.6% of the frame budget for a *full* frame (V4 — the 1024-byte stride kills the `movem.l` burst), but scales with the non-SKIP block fraction and needs **no RAM reference frame at all**, because the previous frame is already in GVRAM. Compose-then-blit is a flat 53.6%. **They cross at 70% of blocks changed.** FINDINGS 24.5. --- ## What session 4 settled 1. **A real 256x256 CRTC mode exists and is verified.** `crtc_mode.lua`, derived from `x68k_crtc.cpp`'s divisor ladder rather than recalled — the derivation is self-checking (368 = 1104/3 exactly, so the horizontal registers divide by three with no remainder). Snapshot is native 256x512, active area pixel-exact, letterbox true black. FINDINGS 23. The x=512 wrap of FINDINGS 22.5 is gone. 2. **The palette ceiling was wrong by 2 dB, in our favour.** The shared LSB `I` must be chosen **per palette entry**, not hardcoded to 1. Doing so lifts the display ceiling from 38.85 to **40.81 dB** and is the only way to get true black at all (`pal6bit(1) = 4`). 102 of 256 entries want `I = 0`. This supersedes FINDINGS 22.4 and gives `scsi` ~2 dB more headroom than believed. **The encoder does not do this yet** — see the encoder-gaps list. 3. **Letterboxing costs one palette entry.** 255 colours + a reserved black at index 0, with `I = 0` on it. `prep_frame.py --reserve-black`. FINDINGS 23.4. 4. **MAME's graphics double-scan is phase-shifted one raster line** — pairs are (1,2),(3,4),..., not (0,1), because `get_gfx_pixel` halves the *absolute* scanline and `vbegin = 41` is odd. Cost a false failure. FINDINGS 23.2. --- ## What session 2 settled 1. **The critical-path question is answered.** "Does VQ soften Bluth's linework unacceptably?" — **flat 4x4 VQ: yes, badly. The hybrid (SKIP/V1/V4/RAW): no.** Verified by eye, not just PSNR. See `docs/FINDINGS.md` 9-11 and the two images in `docs/images/`. Both profiles use **k=256**; see item 2b. 2. **Session 1's 12fps bitrate was wrong** (183 KB/s claimed, 340 KB/s measured). Halving the framerate does not halve the bitrate. FINDINGS 8. 2b. **A fourth false-good result was produced and caught this session** — k=1024 codebooks looked like a +2.4 dB free win because the rate model charged 1 byte for a 10-bit index. FINDINGS 14. The k=256 configuration ships. 3. **The 256-colour palettised frame is the real quality ceiling** and it looks excellent. Judge the codec against that, not against 1080p. 4. Encoder exists and produces a real bitstream: `tools/encoder/`. --- ## Encoder — working ``` python3 tools/encoder/extract.py 00020 /tmp/fr_00020 12 crop python3 tools/encoder/encode.py /tmp/fr_00020 out.dlx --profile sasi --preview p.png ``` | file | role | |---|---| | `extract.py` | .m2ts -> 256x192 PNGs, 12fps, spatial-only denoise | | `vq.py` | palette, blockify, hand-rolled k-means (no sklearn on this box), PSNR | | `vq_hybrid.py` | the codec: 4 block modes + lagrangian mode decision | | `ratectl.py` | SASI/SCSI profiles, leaky-bucket rate control | | `encode.py` | CLI + `DLX1` container writer | `DLX1` container layout is documented in the `encode.py` docstring. All multi-byte fields are **big-endian** so the 68000 reads them with a plain `move`. ### Known encoder gaps - ~~Rate control is written but not yet wired into `encode.py`.~~ **DONE, session 6.** It is on by default; `--fixed-lam` restores the old behaviour. Gated by `tools/analysis/09_ratectl_drift.py`, which is now in `check.sh`. - **Payload is deliberately NOT entropy-coded** — deflate decode does not fit in the 68000's frame budget (FINDINGS 17.2). Do not "optimise" this later. - **Frame records are not aligned.** They must be padded to a 4-byte boundary: unaligned is an ADDRESS ERROR on a 68000, not a slow read (FINDINGS 28.3). `prep_dlx.py` repairs it at load time, which a player streaming from disc cannot do. The pad is real bytes on disc, so it belongs inside the rate controller's accounting. 1.66 B/frame, 20 B/s. - **The mode decision is blind to CPU cost.** It charges V4 four payload bytes and ignores that it costs 1.49x a V1 block to draw. This is the top item at the head of this file. FINDINGS 28.2. - **Palette packing is not implemented in the encoder.** It still emits 24-bit palettes; the X68000 word packing happens Lua-side. Whatever writes real palette words must pick `I` per entry by minimum squared error (FINDINGS 23.3, worth 1.96 dB) and reserve index 0 as black with `I = 0` (FINDINGS 23.4). - Codebooks are per-scene and rebuilt from scratch; no inter-scene reuse. - ~~`_paint` is a Python per-block loop.~~ **DONE, session 6** — vectorised, 17.1x. It was never the bottleneck, though: `VQ.assign` is 78% of a frame and `H.build`'s k-means is 51 s of a 55 s run. **That k-means is now the thing to attack before the full-disc survey**, not anything in the per-frame path. FINDINGS 27.6. --- ## Working setup (unchanged from session 1, re-verified) **MAME ROMs** — `~/mame/roms/x68000.zip`. Must pass **`-bios ipl10`**. ``` mame x68000 -bios ipl10 -video none -sound none -nothrottle -seconds_to_run 3 ``` **Assembler** — `tools/vasm/vasmm68k_mot -Fbin -o out.bin in.s` **Blu-ray** — `udisksctl loop-setup -r -f DRAGONS_LAIR.iso` -> `/media/reala-misaki/BDROM` (still mounted as of end of session 2). **MAME Lua harness** — `tools/bench/*.lua`, working. Three gotchas (retain the notifier subscription in a global; the stack register is `SP` not `A7`; `autoboot_script` fires at PC=0 before boot) are documented in FINDINGS. **Two shell traps, both hit again this session:** - piping MAME (or any long job) through `grep` block-buffers — write to a file. - `pkill -f ` matches your own shell and kills it (exit 144). Use `pkill -x` or kill by PID. - **`pgrep -f | xargs kill` kills your own shell too — exit 144.** Same root cause as the `pkill -f` trap above: the shell's own command line contains the pattern. **Hit again in session 5**, which makes it four times across three sessions. Kill by PID captured at launch (`$!`), or use `pkill -x`. - **`until ! pgrep -f foo.py; do sleep; done` watcher loops never exit.** The watching shell's own command line contains the string `foo.py`, so `pgrep -f` matches the watcher itself and the loop spins forever. Session 2 left 11 of these wedged for over an hour. Wait on the PID (`while kill -0 $PID`) or on a sentinel file the job touches when it finishes -- never on a `-f` name match. - **`timeout N mame ...` does not kill MAME.** MAME catches SIGTERM and, with an autoboot script blocked waiting on a flag that never arrives, never reaches its shutdown path. `timeout` without `-k` then waits forever while MAME burns a full core at `-nothrottle`. Always `timeout -k 5 N`. --- ## Disk throughput benchmark — still blocked, no longer gating `IOCS _B_READ` returns -1 uniformly. Full diagnosis and the four untested hypotheses are in session 1's notes (git history of this file, commit 65112b9); the ordered plan for retrying is in **`docs/BENCHMARK.md`**. **Status changed twice this session — read this rather than the git history.** It was briefly promoted to critical-path while the working bandwidth figure was misread as 4 MB/s. With the correct figure (**4 Mbps = 488 KB/s**) and the ring-buffer simulation showing **zero required prefill** for both profiles (FINDINGS 21), the design no longer hangs on it. Pixel-exact on SCSI is **not** available at 4 Mbps — it needs 92-97% of the pipe — so there is no longer a "measure it and maybe ship transparent" decision waiting. What the benchmark is still worth doing for: - **Confirming the 4 Mbps figure.** It is user-supplied and its provenance is not recorded. Every profile hangs off it. - **Confirming DMA is actually used.** If transfers fall back to PIO the CPU cost rises far above the ~12-15% cycle-steal estimate and CPU becomes the binding constraint. This is the worst plausible outcome and the cheapest to check — do it first. **Do not try to get the bandwidth number out of MAME.** Its SCSI/SASI devices are functional models, not timing-accurate; a KB/s figure from MAME measures the emulator's scheduler. `docs/BENCHMARK.md` covers the three-tier approach (MAME validates the path, derivation bounds it, real hardware settles it). ## Display path — VERIFIED (session 3), in a real mode (session 4), by 68000 code (session 5), by a 68000 DECODER (session 7). The first real frame is on screen: `docs/images/x68k_first_frame_compare.png`. **Session 7 went from copying a frame to parsing one.** `src/player/decode.s` reads DLX1, dispatches all four block modes and writes straight into GVRAM; 120 frames decoded in sequence are pixel-exact against `tools/encoder/dlx.py` (`tools/bench/verify_decode.py`, in `check.sh`). The blit numbers below are still correct for what they measured — a *copy* — but they are no longer the display-path budget: the decoder costs 300/448/400 cycles per V1/V4/RAW block and misses the 12fps budget on 31% of frames. FINDINGS 28. **Session 5 closed the gap this paragraph used to describe.** GVRAM is now filled by 68000 instructions and the result is still pixel-exact, and the blit cost is measured rather than estimated: **53.6% of a 12fps frame**, not 38% (FINDINGS 24). The remaining caveat is different and narrower: MAME models **no GVRAM wait states**, so 53.6% is a floor and real hardware is worse. Full write-up in **FINDINGS 22**. Harness: `tools/bench/show_frame.lua` + `tools/bench/prep_frame.py`. Three facts the player MUST honour, none of which were guessable: | what | where | value | |---|---|---| | **Un-hide the graphics layer** | CRTC R20 `$E80028` | clear bit 11 ("G-VRAM set to buffer"); IPL leaves `0x0B16` | | Colour setup (256c) | CRTC R20 bits 9-8 | `0x0100` | | **Monitor contrast** | `$E8E001` bits 3-0 | IPL leaves **14**; write **15** or everything renders 7% dark | The `R20 = 0x0116` value quoted here in session 3 is the **768-wide IPL timing** with the gate cleared. The shipping value is **`R20 = 0x0110`** — see the mode table in `tools/bench/crtc_mode.lua`, which is now the single source of truth for all of R00-R08 and R20. Bit 11 is the one that cost the most time: GVRAM writes land and read back correctly while the layer is invisible, so the video controller looks guilty and is not. Contrast `0` blanks the screen — free fade-to-black for transitions. Palette format is now **confirmed from MAME source**, not assumed: `GGGGGRRRRRBBBBBI` (G 15:11, R 10:6, B 5:1, shared LSB I), expanded as `pal6bit((field<<1)|I)`. With contrast at 15 the render is **pixel-exact**. Ceiling: the 15-bit+I palette costs **40.81 dB** against the 24-bit palettised source, once `I` is chosen per entry (FINDINGS 23.3 — session 3's 38.88 dB assumed `I = 1`). Still the same order as the `scsi` profile's own codec error (39.4 dB), so `scsi` remains near display-transparent, with ~2 dB more headroom than session 3 thought. Snapshot recipe that works (`-video none` CANNOT snapshot): ``` SDL_VIDEODRIVER=dummy mame x68000 -bios ipl10 -video soft -window \ -sound none -nothrottle -plugins -autoboot_script