Session 1: hardware research, content measurement, codec decision, MAME harness

Verified GVRAM is one word-access per pixel in ALL color modes; chose 256-color
256x192 with movem.l bursts (page 1 sacrificed as double-buffer).

Measured 8 scenes from the Blu-ray source: blit costs under 8% of the 12fps
cycle budget, so I/O is the bottleneck, not CPU. Naive delta+RLE reaches only
3.2:1 (365 KB/s, 470MB) -> decision to use 4x4 vector quantization (~30 KB/s).

"Shot on twos" assumption failed: the transfer has zero duplicate frames, so
12fps requires explicit decimation.

Documents three false measurement results and their root causes (per-frame
Floyd-Steinberg dithering, temporal denoise, exact-match dedupe on noisy source).

MAME Lua injection harness works and is reusable for cycle-cost measurement;
the IOCS _B_READ disk benchmark is blocked returning -1.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6

docs/FINDINGS.md

@@ -0,0 +1,190 @@
+# Findings — session 1 (2026-08-23)
+
+All numbers here are MEASURED unless marked ESTIMATE or FOLKLORE.
+
+---
+
+## 1. Source material
+
+`DRAGONS_LAIR.iso` — 16 GB, UDF 2.x, **decrypted** (no AACS dir).
+Loop-mounted read-only at `/media/reala-misaki/BDROM` via `udisksctl loop-setup -r -f`.
+(7-Zip cannot read UDF 2.x; use the loop mount.)
+
+- **224 `.m2ts` streams**, 1920x1080, **MPEG-2, progressive, 23.976 fps**
+- Size histogram: 47 <5MB, 138 5-50MB, 22 50-150MB, 14 150-400MB, 3 >400MB
+- The 185 sub-50MB streams are the **arcade branching scenes already split into
+  individual clips** — we get scene boundaries for free.
+- Big streams are full-feature playthroughs: 00215 (1376s), 00216 (1151s), 00223 (566s)
+- Typical scene clip ~60s (00203/00205/00199), some ~100s (00164/00212)
+
+**Gotcha:** clip durations vary wildly. Always read `format=duration` and seek
+relative to it. Seeking to a fixed offset silently yields 0 frames on short clips.
+
+---
+
+## 2. GVRAM layout  [verified — see HARDWARE.md for source]
+
+**One 16-bit word per pixel position in EVERY color mode.** Bit depth does not
+change VRAM bandwidth; it only subdivides the word.
+
+`addr = page_base + y*1024 + x*2`  — adjacent pixels are 2 bytes apart in all modes.
+
+Consequence: low bit depth buys **no speed**. 16-color mode is strictly worse than
+256-color (same bus traffic, 1/16 the palette). Page-alias writes are hardware
+auto-masked, so 16-color needs no software read-modify-write — but it's still
+one word-access per pixel.
+
+**Chosen: 256 colors, 256x192 active area.**
+In 256-color mode P0=low byte, P1=high byte of each word. Sacrificing page 1 as a
+double-buffer lets a `move.l` cover two pixel positions, enabling `movem.l` bursts
+(12 regs = 48 bytes = 24 pixels). Identical blit cost to 65536-color mode but
+**half the on-disk data**.
+
+---
+
+## 3. Content measurements (8 scenes sampled, 5s each at 40% into each clip)
+
+| metric | mean | p90 |
+|---|---|---|
+| pixels changed / frame | 20.1% | 30.2% |
+| **blit cost** | **~64k cycles** | **~97k cycles** |
+| naive delta+RLE frame size | 15.5 KB | 19.6 KB |
+
+Budget is **833,333 cycles/frame** @ 12fps on a 10MHz 68000.
+
+### => THE CPU IS NOT THE BOTTLENECK. I/O IS.
+Blit uses **under 8%** of budget. The naive row-span+RLE codec achieves only
+**3.2:1**, giving **365 KB/s / 470 MB** at 24fps (~183 KB/s / 235 MB at 12fps).
+
+Per-scene variance is extreme: static dialogue ~30 KB/s, action ~700 KB/s.
+Any codec needs a hard bitrate ceiling, not just a good average.
+
+### "Shot on twos" — ASSUMPTION FAILED
+Dedupe found **zero** duplicate frames across all 8 scenes (`uniq=120/120`,
+24.0 fps effective). This Blu-ray is a restoration where every frame is unique.
+We do NOT get halved data for free. **Decimation to 12fps must be explicit.**
+
+A weak alternation signature does exist (even-index pairs 40.7% vs odd 27.5%,
+ratio 1.5x, with occasional true-duplicate pairs at 0.03-0.19%), but it is
+irregular — Bluth mixed ones and twos; action is animated on ones.
+
+---
+
+## 4. MEASUREMENT TRAPS — read before trusting any pipeline number
+
+Three separate false results were produced and caught this session. All three
+looked plausible. Guard against them:
+
+1. **Per-frame Floyd-Steinberg dithering destroys temporal coherence.**
+   Error diffusion is chaotic: a +/-1 input change cascades across the row and
+   produces a completely different index pattern. First run reported 31.5% pixels
+   changed with near-zero variance (median 31.6, p90 32.3, max 32.7) while source
+   mean-abs-diff was 0.09 — i.e. visually identical frames. That flat variance is
+   the tell: **real animation has scene-dependent variance; noise does not.**
+   Use no dithering (cel art is flat) or ordered/Bayer (spatially fixed, temporally stable).
+
+2. **Temporal denoise smears motion.** `hqdn3d=4:3:6:4` — the `6:4` are temporal
+   params. It flattened real motion, which then measured as "no motion" and
+   produced an absurd 0.8 fps / 4 MB result. **Use spatial-only: `hqdn3d=4:3:0:0`.**
+
+3. **Exact-match dedupe fails on a noisy source.** MPEG-2 grain means near-duplicate
+   frames differ by +/-1 and are never bit-exact. Use a threshold on
+   "% pixels differing by more than N levels", and pick the threshold from the
+   observed distribution, not a guess. A 2% threshold ate genuine animation when
+   mean consecutive change was only 0.9%.
+
+**Sanity rule: if a result has suspiciously low variance, or is suspiciously
+good, it is probably an artifact of the measurement, not a property of the content.**
+
+Scripts kept in `tools/analysis/` — 01 and 02 are marked BROKEN deliberately as
+regression references; 03 and 04 are the correct ones.
+
+---
+
+## 5. Storage interface — the SASI/SCSI split
+
+[Yasuma, X68030 internal SCSI controller]
+
+- Interface: **SCSI-1**, 50-pin, 5 MB/s bus spec
+- Controller: **Fujitsu MB89352** SPC
+- Transfer mode: **DMA** (via **HD63450** DMAC)
+- Bus: X68000 original bus, **16-bit @ 10MHz**
+
+**Even on the X68030, SCSI runs at 10MHz 16-bit DMA.** Storage bandwidth does
+NOT scale with CPU — the controller sits on the original bus. HD63450's 12.5MHz
+official ceiling is why the X68030 runs at 25MHz. An "HSCSI" TSR forces PIO/FIFO
+transfer instead of DMA but was marginal even at 25MHz.
+
+Because it's DMA, **streaming costs essentially no CPU** — this stacks with the
+8% blit utilisation. The 68000 really is nearly idle.
+
+### Model split — IMPORTANT
+**The 10MHz models (original X68000, ACE, PRO, EXPERT) use SASI, not SCSI.**
+Built-in SCSI starts at the X68000 **Super** (1990) and continues through XVI,
+Compact, X68030. SCSI on earlier machines needs the **Sharp CZ-6BS1** board
+in an I/O slot (MAME models this: `-exp1 cz6bs1`).
+
+| target | bandwidth | naive codec (365 KB/s) | VQ codec (~30 KB/s) |
+|---|---|---|---|
+| SASI (stock ACE/EXPERT) | ~300-500 KB/s FOLKLORE | infeasible | comfortable |
+| SCSI (Super+, or CZ-6BS1) | ~1 MB/s FOLKLORE | tight but viable | trivial |
+
+Derived bounds (ESTIMATE): 16-bit @10MHz with 4-clock bus cycle = 5 MB/s absolute
+ceiling; HD63450 single-address DMA ~8 clocks/word => ~2.5 MB/s practical ceiling,
+before SCSI-1 async handshake and drive latency.
+
+**No measured benchmark was obtained — see STATUS.md.** The ~300-500 KB/s and
+~1 MB/s figures are folklore-grade; I could not find a primary measurement.
+
+---
+
+## 6. Codec decision: vector quantization (Cinepak-style)
+
+Given ~8x CPU headroom and an I/O ceiling, spend CPU to buy bandwidth.
+
+- Split frame into 4x4 blocks, encode each as a 1-byte index into a per-scene codebook
+- Decode = 16-byte copy from a lookup table: nearly free
+- A **full** frame = 256*192/16 = **3,072 bytes** — a hard 16:1 floor before delta
+- Add block-level delta on top; action scenes ~2-3 KB/frame
+- => roughly **30 KB/s, ~40 MB total**, with a *deterministic* bitrate ceiling
+
+Divergence from the SNES project (below): use a **per-scene codebook with delta
+updates**, not a per-frame rebuild. We trade adaptivity for bandwidth because we
+have 2MB RAM to keep a codebook resident and CPU to spare.
+
+**Risk not yet evaluated:** 4x4 VQ with a 256-entry codebook will visibly soften
+detail. Bluth's fine ink linework is what suffers. Prototype and eyeball before committing.
+
+---
+
+## 7. Comparison: astrobleem/SNES-SuperDragonsLairArcade
+
+Reached the **same core architecture independently** — "512 tiles per frame" is
+vector quantization (8x8 codebook + tilemap). Good validation.
+
+But: the SNES PPU has **no bitmap mode**, so tiles are forced on them by display
+hardware. The X68000 has a real linear framebuffer, so VQ is a *compression
+choice* we can tune or drop per-scene.
+
+**MSU-1 is a bandwidth cheat we don't have.** It's a modern flash-cart coprocessor
+giving memory-mapped streaming the real SNES never had. Their budget: 512 tiles x
+32 bytes (4bpp 8x8) + tilemap ~= 18 KB/frame => **~430 KB/s** at 23.976fps.
+That's *higher* than the 365 KB/s we'd reject on SASI. (ESTIMATE: my arithmetic on
+their stated tile budget, not a measured figure.)
+
+Where we're ahead: 256 simultaneous colors from a 65536 palette vs their 4bpp
+sub-palettes needing a tile-aware palette optimizer plus a spatial smoothing pass
+to hide 8x8 palette seams. That problem doesn't exist for us. Plus 68000@10MHz
+vs 65816@3.58MHz, and 2MB vs 128KB.
+
+**Most valuable thing in that repo is NOT the codec — it's `data/events/`:**
+516 chapter definitions across 29 scenes as XML, plus
+`data/chapter_event_inventory.md`. That's the arcade scene graph and input-timing
+structure, entirely hardware-independent — the whole game-logic layer we'd
+otherwise reverse-engineer from the arcade ROM.
+
+**TODO: check their license before planning to reuse it.**
+Their 516 chapters are finer-grained than our 224 Blu-ray streams, so mapping
+their event table onto our footage means subdividing streams by timecode.
+
+Caveat: all of the above is from README/repo-tree summaries, not their source.