Files

T

reala-misaki 65112b9305 Session 1: hardware research, content measurement, codec decision, MAME harness

Verified GVRAM is one word-access per pixel in ALL color modes; chose 256-color
256x192 with movem.l bursts (page 1 sacrificed as double-buffer).

Measured 8 scenes from the Blu-ray source: blit costs under 8% of the 12fps
cycle budget, so I/O is the bottleneck, not CPU. Naive delta+RLE reaches only
3.2:1 (365 KB/s, 470MB) -> decision to use 4x4 vector quantization (~30 KB/s).

"Shot on twos" assumption failed: the transfer has zero duplicate frames, so
12fps requires explicit decimation.

Documents three false measurement results and their root causes (per-frame
Floyd-Steinberg dithering, temporal denoise, exact-match dedupe on noisy source).

MAME Lua injection harness works and is reusable for cycle-cost measurement;
the IOCS _B_READ disk benchmark is blocked returning -1.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6

2026-08-23 11:23:49 -07:00

8.7 KiB

Raw Blame History

Findings — session 1 (2026-08-23)

All numbers here are MEASURED unless marked ESTIMATE or FOLKLORE.

1. Source material

DRAGONS_LAIR.iso — 16 GB, UDF 2.x, decrypted (no AACS dir). Loop-mounted read-only at /media/reala-misaki/BDROM via udisksctl loop-setup -r -f. (7-Zip cannot read UDF 2.x; use the loop mount.)

224 .m2ts streams, 1920x1080, MPEG-2, progressive, 23.976 fps
Size histogram: 47 <5MB, 138 5-50MB, 22 50-150MB, 14 150-400MB, 3 >400MB
The 185 sub-50MB streams are the arcade branching scenes already split into individual clips — we get scene boundaries for free.
Big streams are full-feature playthroughs: 00215 (1376s), 00216 (1151s), 00223 (566s)
Typical scene clip ~60s (00203/00205/00199), some ~100s (00164/00212)

Gotcha: clip durations vary wildly. Always read format=duration and seek relative to it. Seeking to a fixed offset silently yields 0 frames on short clips.

2. GVRAM layout [verified — see HARDWARE.md for source]

One 16-bit word per pixel position in EVERY color mode. Bit depth does not change VRAM bandwidth; it only subdivides the word.

addr = page_base + y*1024 + x*2 — adjacent pixels are 2 bytes apart in all modes.

Consequence: low bit depth buys no speed. 16-color mode is strictly worse than 256-color (same bus traffic, 1/16 the palette). Page-alias writes are hardware auto-masked, so 16-color needs no software read-modify-write — but it's still one word-access per pixel.

Chosen: 256 colors, 256x192 active area. In 256-color mode P0=low byte, P1=high byte of each word. Sacrificing page 1 as a double-buffer lets a move.l cover two pixel positions, enabling movem.l bursts (12 regs = 48 bytes = 24 pixels). Identical blit cost to 65536-color mode but half the on-disk data.

3. Content measurements (8 scenes sampled, 5s each at 40% into each clip)

metric	mean	p90
pixels changed / frame	20.1%	30.2%
blit cost	~64k cycles	~97k cycles
naive delta+RLE frame size	15.5 KB	19.6 KB

Budget is 833,333 cycles/frame @ 12fps on a 10MHz 68000.

=> THE CPU IS NOT THE BOTTLENECK. I/O IS.

Blit uses under 8% of budget. The naive row-span+RLE codec achieves only 3.2:1, giving 365 KB/s / 470 MB at 24fps (~183 KB/s / 235 MB at 12fps).

Per-scene variance is extreme: static dialogue ~30 KB/s, action ~700 KB/s. Any codec needs a hard bitrate ceiling, not just a good average.

"Shot on twos" — ASSUMPTION FAILED

Dedupe found zero duplicate frames across all 8 scenes (uniq=120/120, 24.0 fps effective). This Blu-ray is a restoration where every frame is unique. We do NOT get halved data for free. Decimation to 12fps must be explicit.

A weak alternation signature does exist (even-index pairs 40.7% vs odd 27.5%, ratio 1.5x, with occasional true-duplicate pairs at 0.03-0.19%), but it is irregular — Bluth mixed ones and twos; action is animated on ones.

4. MEASUREMENT TRAPS — read before trusting any pipeline number

Three separate false results were produced and caught this session. All three looked plausible. Guard against them:

Per-frame Floyd-Steinberg dithering destroys temporal coherence. Error diffusion is chaotic: a +/-1 input change cascades across the row and produces a completely different index pattern. First run reported 31.5% pixels changed with near-zero variance (median 31.6, p90 32.3, max 32.7) while source mean-abs-diff was 0.09 — i.e. visually identical frames. That flat variance is the tell: real animation has scene-dependent variance; noise does not. Use no dithering (cel art is flat) or ordered/Bayer (spatially fixed, temporally stable).
Temporal denoise smears motion. hqdn3d=4:3:6:4 — the 6:4 are temporal params. It flattened real motion, which then measured as "no motion" and produced an absurd 0.8 fps / 4 MB result. Use spatial-only: hqdn3d=4:3:0:0.
Exact-match dedupe fails on a noisy source. MPEG-2 grain means near-duplicate frames differ by +/-1 and are never bit-exact. Use a threshold on "% pixels differing by more than N levels", and pick the threshold from the observed distribution, not a guess. A 2% threshold ate genuine animation when mean consecutive change was only 0.9%.

Sanity rule: if a result has suspiciously low variance, or is suspiciously good, it is probably an artifact of the measurement, not a property of the content.

Scripts kept in tools/analysis/ — 01 and 02 are marked BROKEN deliberately as regression references; 03 and 04 are the correct ones.

5. Storage interface — the SASI/SCSI split

[Yasuma, X68030 internal SCSI controller]

Interface: SCSI-1, 50-pin, 5 MB/s bus spec
Controller: Fujitsu MB89352 SPC
Transfer mode: DMA (via HD63450 DMAC)
Bus: X68000 original bus, 16-bit @ 10MHz

Even on the X68030, SCSI runs at 10MHz 16-bit DMA. Storage bandwidth does NOT scale with CPU — the controller sits on the original bus. HD63450's 12.5MHz official ceiling is why the X68030 runs at 25MHz. An "HSCSI" TSR forces PIO/FIFO transfer instead of DMA but was marginal even at 25MHz.

Because it's DMA, streaming costs essentially no CPU — this stacks with the 8% blit utilisation. The 68000 really is nearly idle.

Model split — IMPORTANT

The 10MHz models (original X68000, ACE, PRO, EXPERT) use SASI, not SCSI. Built-in SCSI starts at the X68000 Super (1990) and continues through XVI, Compact, X68030. SCSI on earlier machines needs the Sharp CZ-6BS1 board in an I/O slot (MAME models this: -exp1 cz6bs1).

target	bandwidth	naive codec (365 KB/s)	VQ codec (~30 KB/s)
SASI (stock ACE/EXPERT)	~300-500 KB/s FOLKLORE	infeasible	comfortable
SCSI (Super+, or CZ-6BS1)	~1 MB/s FOLKLORE	tight but viable	trivial

Derived bounds (ESTIMATE): 16-bit @10MHz with 4-clock bus cycle = 5 MB/s absolute ceiling; HD63450 single-address DMA ~8 clocks/word => ~2.5 MB/s practical ceiling, before SCSI-1 async handshake and drive latency.

No measured benchmark was obtained — see STATUS.md. The ~300-500 KB/s and ~1 MB/s figures are folklore-grade; I could not find a primary measurement.

6. Codec decision: vector quantization (Cinepak-style)

Given ~8x CPU headroom and an I/O ceiling, spend CPU to buy bandwidth.

Split frame into 4x4 blocks, encode each as a 1-byte index into a per-scene codebook
Decode = 16-byte copy from a lookup table: nearly free
A full frame = 256*192/16 = 3,072 bytes — a hard 16:1 floor before delta
Add block-level delta on top; action scenes ~2-3 KB/frame
=> roughly 30 KB/s, ~40 MB total, with a deterministic bitrate ceiling

Divergence from the SNES project (below): use a per-scene codebook with delta updates, not a per-frame rebuild. We trade adaptivity for bandwidth because we have 2MB RAM to keep a codebook resident and CPU to spare.

Risk not yet evaluated: 4x4 VQ with a 256-entry codebook will visibly soften detail. Bluth's fine ink linework is what suffers. Prototype and eyeball before committing.

7. Comparison: astrobleem/SNES-SuperDragonsLairArcade

Reached the same core architecture independently — "512 tiles per frame" is vector quantization (8x8 codebook + tilemap). Good validation.

But: the SNES PPU has no bitmap mode, so tiles are forced on them by display hardware. The X68000 has a real linear framebuffer, so VQ is a compression choice we can tune or drop per-scene.

MSU-1 is a bandwidth cheat we don't have. It's a modern flash-cart coprocessor giving memory-mapped streaming the real SNES never had. Their budget: 512 tiles x 32 bytes (4bpp 8x8) + tilemap ~= 18 KB/frame => ~430 KB/s at 23.976fps. That's higher than the 365 KB/s we'd reject on SASI. (ESTIMATE: my arithmetic on their stated tile budget, not a measured figure.)

Where we're ahead: 256 simultaneous colors from a 65536 palette vs their 4bpp sub-palettes needing a tile-aware palette optimizer plus a spatial smoothing pass to hide 8x8 palette seams. That problem doesn't exist for us. Plus 68000@10MHz vs 65816@3.58MHz, and 2MB vs 128KB.

Most valuable thing in that repo is NOT the codec — it's data/events/: 516 chapter definitions across 29 scenes as XML, plus data/chapter_event_inventory.md. That's the arcade scene graph and input-timing structure, entirely hardware-independent — the whole game-logic layer we'd otherwise reverse-engineer from the arcade ROM.

TODO: check their license before planning to reuse it. Their 516 chapters are finer-grained than our 224 Blu-ray streams, so mapping their event table onto our footage means subdividing streams by timecode.

Caveat: all of the above is from README/repo-tree summaries, not their source.

8.7 KiB Raw Blame History