reala-misaki
65112b9305
Verified GVRAM is one word-access per pixel in ALL color modes; chose 256-color 256x192 with movem.l bursts (page 1 sacrificed as double-buffer). Measured 8 scenes from the Blu-ray source: blit costs under 8% of the 12fps cycle budget, so I/O is the bottleneck, not CPU. Naive delta+RLE reaches only 3.2:1 (365 KB/s, 470MB) -> decision to use 4x4 vector quantization (~30 KB/s). "Shot on twos" assumption failed: the transfer has zero duplicate frames, so 12fps requires explicit decimation. Documents three false measurement results and their root causes (per-frame Floyd-Steinberg dithering, temporal denoise, exact-match dedupe on noisy source). MAME Lua injection harness works and is reusable for cycle-cost measurement; the IOCS _B_READ disk benchmark is blocked returning -1. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
191 lines
8.7 KiB
Markdown
191 lines
8.7 KiB
Markdown
# Findings — session 1 (2026-08-23)
|
|
|
|
All numbers here are MEASURED unless marked ESTIMATE or FOLKLORE.
|
|
|
|
---
|
|
|
|
## 1. Source material
|
|
|
|
`DRAGONS_LAIR.iso` — 16 GB, UDF 2.x, **decrypted** (no AACS dir).
|
|
Loop-mounted read-only at `/media/reala-misaki/BDROM` via `udisksctl loop-setup -r -f`.
|
|
(7-Zip cannot read UDF 2.x; use the loop mount.)
|
|
|
|
- **224 `.m2ts` streams**, 1920x1080, **MPEG-2, progressive, 23.976 fps**
|
|
- Size histogram: 47 <5MB, 138 5-50MB, 22 50-150MB, 14 150-400MB, 3 >400MB
|
|
- The 185 sub-50MB streams are the **arcade branching scenes already split into
|
|
individual clips** — we get scene boundaries for free.
|
|
- Big streams are full-feature playthroughs: 00215 (1376s), 00216 (1151s), 00223 (566s)
|
|
- Typical scene clip ~60s (00203/00205/00199), some ~100s (00164/00212)
|
|
|
|
**Gotcha:** clip durations vary wildly. Always read `format=duration` and seek
|
|
relative to it. Seeking to a fixed offset silently yields 0 frames on short clips.
|
|
|
|
---
|
|
|
|
## 2. GVRAM layout [verified — see HARDWARE.md for source]
|
|
|
|
**One 16-bit word per pixel position in EVERY color mode.** Bit depth does not
|
|
change VRAM bandwidth; it only subdivides the word.
|
|
|
|
`addr = page_base + y*1024 + x*2` — adjacent pixels are 2 bytes apart in all modes.
|
|
|
|
Consequence: low bit depth buys **no speed**. 16-color mode is strictly worse than
|
|
256-color (same bus traffic, 1/16 the palette). Page-alias writes are hardware
|
|
auto-masked, so 16-color needs no software read-modify-write — but it's still
|
|
one word-access per pixel.
|
|
|
|
**Chosen: 256 colors, 256x192 active area.**
|
|
In 256-color mode P0=low byte, P1=high byte of each word. Sacrificing page 1 as a
|
|
double-buffer lets a `move.l` cover two pixel positions, enabling `movem.l` bursts
|
|
(12 regs = 48 bytes = 24 pixels). Identical blit cost to 65536-color mode but
|
|
**half the on-disk data**.
|
|
|
|
---
|
|
|
|
## 3. Content measurements (8 scenes sampled, 5s each at 40% into each clip)
|
|
|
|
| metric | mean | p90 |
|
|
|---|---|---|
|
|
| pixels changed / frame | 20.1% | 30.2% |
|
|
| **blit cost** | **~64k cycles** | **~97k cycles** |
|
|
| naive delta+RLE frame size | 15.5 KB | 19.6 KB |
|
|
|
|
Budget is **833,333 cycles/frame** @ 12fps on a 10MHz 68000.
|
|
|
|
### => THE CPU IS NOT THE BOTTLENECK. I/O IS.
|
|
Blit uses **under 8%** of budget. The naive row-span+RLE codec achieves only
|
|
**3.2:1**, giving **365 KB/s / 470 MB** at 24fps (~183 KB/s / 235 MB at 12fps).
|
|
|
|
Per-scene variance is extreme: static dialogue ~30 KB/s, action ~700 KB/s.
|
|
Any codec needs a hard bitrate ceiling, not just a good average.
|
|
|
|
### "Shot on twos" — ASSUMPTION FAILED
|
|
Dedupe found **zero** duplicate frames across all 8 scenes (`uniq=120/120`,
|
|
24.0 fps effective). This Blu-ray is a restoration where every frame is unique.
|
|
We do NOT get halved data for free. **Decimation to 12fps must be explicit.**
|
|
|
|
A weak alternation signature does exist (even-index pairs 40.7% vs odd 27.5%,
|
|
ratio 1.5x, with occasional true-duplicate pairs at 0.03-0.19%), but it is
|
|
irregular — Bluth mixed ones and twos; action is animated on ones.
|
|
|
|
---
|
|
|
|
## 4. MEASUREMENT TRAPS — read before trusting any pipeline number
|
|
|
|
Three separate false results were produced and caught this session. All three
|
|
looked plausible. Guard against them:
|
|
|
|
1. **Per-frame Floyd-Steinberg dithering destroys temporal coherence.**
|
|
Error diffusion is chaotic: a +/-1 input change cascades across the row and
|
|
produces a completely different index pattern. First run reported 31.5% pixels
|
|
changed with near-zero variance (median 31.6, p90 32.3, max 32.7) while source
|
|
mean-abs-diff was 0.09 — i.e. visually identical frames. That flat variance is
|
|
the tell: **real animation has scene-dependent variance; noise does not.**
|
|
Use no dithering (cel art is flat) or ordered/Bayer (spatially fixed, temporally stable).
|
|
|
|
2. **Temporal denoise smears motion.** `hqdn3d=4:3:6:4` — the `6:4` are temporal
|
|
params. It flattened real motion, which then measured as "no motion" and
|
|
produced an absurd 0.8 fps / 4 MB result. **Use spatial-only: `hqdn3d=4:3:0:0`.**
|
|
|
|
3. **Exact-match dedupe fails on a noisy source.** MPEG-2 grain means near-duplicate
|
|
frames differ by +/-1 and are never bit-exact. Use a threshold on
|
|
"% pixels differing by more than N levels", and pick the threshold from the
|
|
observed distribution, not a guess. A 2% threshold ate genuine animation when
|
|
mean consecutive change was only 0.9%.
|
|
|
|
**Sanity rule: if a result has suspiciously low variance, or is suspiciously
|
|
good, it is probably an artifact of the measurement, not a property of the content.**
|
|
|
|
Scripts kept in `tools/analysis/` — 01 and 02 are marked BROKEN deliberately as
|
|
regression references; 03 and 04 are the correct ones.
|
|
|
|
---
|
|
|
|
## 5. Storage interface — the SASI/SCSI split
|
|
|
|
[Yasuma, X68030 internal SCSI controller]
|
|
|
|
- Interface: **SCSI-1**, 50-pin, 5 MB/s bus spec
|
|
- Controller: **Fujitsu MB89352** SPC
|
|
- Transfer mode: **DMA** (via **HD63450** DMAC)
|
|
- Bus: X68000 original bus, **16-bit @ 10MHz**
|
|
|
|
**Even on the X68030, SCSI runs at 10MHz 16-bit DMA.** Storage bandwidth does
|
|
NOT scale with CPU — the controller sits on the original bus. HD63450's 12.5MHz
|
|
official ceiling is why the X68030 runs at 25MHz. An "HSCSI" TSR forces PIO/FIFO
|
|
transfer instead of DMA but was marginal even at 25MHz.
|
|
|
|
Because it's DMA, **streaming costs essentially no CPU** — this stacks with the
|
|
8% blit utilisation. The 68000 really is nearly idle.
|
|
|
|
### Model split — IMPORTANT
|
|
**The 10MHz models (original X68000, ACE, PRO, EXPERT) use SASI, not SCSI.**
|
|
Built-in SCSI starts at the X68000 **Super** (1990) and continues through XVI,
|
|
Compact, X68030. SCSI on earlier machines needs the **Sharp CZ-6BS1** board
|
|
in an I/O slot (MAME models this: `-exp1 cz6bs1`).
|
|
|
|
| target | bandwidth | naive codec (365 KB/s) | VQ codec (~30 KB/s) |
|
|
|---|---|---|---|
|
|
| SASI (stock ACE/EXPERT) | ~300-500 KB/s FOLKLORE | infeasible | comfortable |
|
|
| SCSI (Super+, or CZ-6BS1) | ~1 MB/s FOLKLORE | tight but viable | trivial |
|
|
|
|
Derived bounds (ESTIMATE): 16-bit @10MHz with 4-clock bus cycle = 5 MB/s absolute
|
|
ceiling; HD63450 single-address DMA ~8 clocks/word => ~2.5 MB/s practical ceiling,
|
|
before SCSI-1 async handshake and drive latency.
|
|
|
|
**No measured benchmark was obtained — see STATUS.md.** The ~300-500 KB/s and
|
|
~1 MB/s figures are folklore-grade; I could not find a primary measurement.
|
|
|
|
---
|
|
|
|
## 6. Codec decision: vector quantization (Cinepak-style)
|
|
|
|
Given ~8x CPU headroom and an I/O ceiling, spend CPU to buy bandwidth.
|
|
|
|
- Split frame into 4x4 blocks, encode each as a 1-byte index into a per-scene codebook
|
|
- Decode = 16-byte copy from a lookup table: nearly free
|
|
- A **full** frame = 256*192/16 = **3,072 bytes** — a hard 16:1 floor before delta
|
|
- Add block-level delta on top; action scenes ~2-3 KB/frame
|
|
- => roughly **30 KB/s, ~40 MB total**, with a *deterministic* bitrate ceiling
|
|
|
|
Divergence from the SNES project (below): use a **per-scene codebook with delta
|
|
updates**, not a per-frame rebuild. We trade adaptivity for bandwidth because we
|
|
have 2MB RAM to keep a codebook resident and CPU to spare.
|
|
|
|
**Risk not yet evaluated:** 4x4 VQ with a 256-entry codebook will visibly soften
|
|
detail. Bluth's fine ink linework is what suffers. Prototype and eyeball before committing.
|
|
|
|
---
|
|
|
|
## 7. Comparison: astrobleem/SNES-SuperDragonsLairArcade
|
|
|
|
Reached the **same core architecture independently** — "512 tiles per frame" is
|
|
vector quantization (8x8 codebook + tilemap). Good validation.
|
|
|
|
But: the SNES PPU has **no bitmap mode**, so tiles are forced on them by display
|
|
hardware. The X68000 has a real linear framebuffer, so VQ is a *compression
|
|
choice* we can tune or drop per-scene.
|
|
|
|
**MSU-1 is a bandwidth cheat we don't have.** It's a modern flash-cart coprocessor
|
|
giving memory-mapped streaming the real SNES never had. Their budget: 512 tiles x
|
|
32 bytes (4bpp 8x8) + tilemap ~= 18 KB/frame => **~430 KB/s** at 23.976fps.
|
|
That's *higher* than the 365 KB/s we'd reject on SASI. (ESTIMATE: my arithmetic on
|
|
their stated tile budget, not a measured figure.)
|
|
|
|
Where we're ahead: 256 simultaneous colors from a 65536 palette vs their 4bpp
|
|
sub-palettes needing a tile-aware palette optimizer plus a spatial smoothing pass
|
|
to hide 8x8 palette seams. That problem doesn't exist for us. Plus 68000@10MHz
|
|
vs 65816@3.58MHz, and 2MB vs 128KB.
|
|
|
|
**Most valuable thing in that repo is NOT the codec — it's `data/events/`:**
|
|
516 chapter definitions across 29 scenes as XML, plus
|
|
`data/chapter_event_inventory.md`. That's the arcade scene graph and input-timing
|
|
structure, entirely hardware-independent — the whole game-logic layer we'd
|
|
otherwise reverse-engineer from the arcade ROM.
|
|
|
|
**TODO: check their license before planning to reuse it.**
|
|
Their 516 chapters are finer-grained than our 224 Blu-ray streams, so mapping
|
|
their event table onto our footage means subdividing streams by timecode.
|
|
|
|
Caveat: all of the above is from README/repo-tree summaries, not their source.
|