Session 2: hybrid VQ codec, two quality profiles, three corrections
Answers session 1's critical-path question. Flat 4x4 VQ at k=256 was prototyped and REJECTED by eye: Dirk's face disintegrates and ink outlines break into 4-pixel stair-steps. The 256-colour palettised frame is excellent, so the palette was never the problem -- block VQ was. Replaced it with a Cinepak-style hybrid: each 4x4 block is SKIP, one 4x4 codeword, four 2x2 codewords, or RAW literal pixels, chosen per block by rate-distortion. The RAW escape makes lam=0 pixel-exact (measured 0.00 dB loss), so the quality knob spans lossless to heavily-compressed in one bitstream. Per the user's decision, ships TWO quality profiles from that one codec, one decoder and one bitstream -- only the rate knob differs: sasi 45 KB/s lam=300 34.8 dB stock 10MHz ACE/EXPERT scsi 75 KB/s lam=100 35.9 dB Super/XVI or CZ-6BS1 Three corrections to earlier numbers: 1. Session 1's "183 KB/s at 12fps" was a bad extrapolation. Halving the framerate does not halve the bitrate -- decimation roughly doubles the per-frame delta. Re-measured directly: 340 KB/s for session 1's own RLE, 247 KB/s for changed-spans+deflate. The lossless floor is 319 MB. 2. A FOURTH false-good result, same family as the three in FINDINGS 4: k=1024 codebooks appeared to buy +2.4 dB free, because the rate model charged 1 byte for a 10-bit index. Charging the true cost reverses the verdict -- k=256 wins at every matched bitrate, and by 5 dB at the low end where the SASI profile lives. k=256 ships. 3. Stream inventory: the ~3-5MB clips are 1.2-1.7s, not ~60s, and some 60s streams are menus, not content. Any survey must classify before averaging. Also cleared both candidate sources for the game-logic layer: the SNES project is MIT and DirkSimple is zlib, so the arcade scene graph can be imported and the two transcriptions diffed against each other. Encoder is working end-to-end: extract.py -> vq/vq_hybrid/ratectl -> encode.py, emitting a big-endian DLX1 container the 68000 can parse with plain moves. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
This commit is contained in:
@@ -188,3 +188,185 @@ Their 516 chapters are finer-grained than our 224 Blu-ray streams, so mapping
|
||||
their event table onto our footage means subdividing streams by timecode.
|
||||
|
||||
Caveat: all of the above is from README/repo-tree summaries, not their source.
|
||||
|
||||
---
|
||||
---
|
||||
|
||||
# Findings — session 2 (2026-08-23)
|
||||
|
||||
## 8. CORRECTION to session 1: halving the framerate does NOT halve the bitrate
|
||||
|
||||
Session 1 measured 365 KB/s for naive delta+RLE at 24 fps and wrote
|
||||
"(~183 KB/s at 12fps)". **That extrapolation is wrong.** Decimating to 12 fps
|
||||
roughly doubles the per-frame delta, so the *rate* stays nearly flat.
|
||||
|
||||
Re-measured directly on 12 fps decimated frames (4 scenes, 66 frames):
|
||||
|
||||
| codec (all LOSSLESS w.r.t. the 256-colour frame) | B/frame | KB/s @12 | 22 min | ratio |
|
||||
|---|---|---|---|---|
|
||||
| raw 8bpp 256x192 | 49152 | 576 | 743 MB | 1.0:1 |
|
||||
| session 1 row-span + RLE | 29055 | 340 | 439 MB | 1.7:1 |
|
||||
| XOR vs prev + deflate | 30196 | 354 | 456 MB | 1.6:1 |
|
||||
| **changed-spans + deflate** | **21110** | **247** | **319 MB** | **2.3:1** |
|
||||
| changed-spans + LZMA | 18759 | 220 | 283 MB | 2.6:1 |
|
||||
|
||||
Session 1's own RLE re-measured at 12 fps gives **340 KB/s, not 183**.
|
||||
Any plan that assumed 183 KB/s was based on a bad number.
|
||||
|
||||
Deflate-class entropy coding on top of the span payload is worth **1.4x** over
|
||||
hand-rolled RLE, and LZ decode is cheap on a 68000 (byte copies), so the
|
||||
lossless floor is ~247 KB/s / 319 MB. That is **infeasible on SASI** and
|
||||
**tight but real on SCSI**.
|
||||
|
||||
## 9. Flat 4x4 VQ at k=256 is NOT acceptable — confirmed by eye
|
||||
|
||||
The risk flagged in 6 is real. At k=256, 4x4:
|
||||
|
||||
| scene | palette-only PSNR | after VQ | VQ loss |
|
||||
|---|---|---|---|
|
||||
| 00010 | 38.35 | 29.68 | 8.67 dB |
|
||||
| 00020 | 39.90 | 32.67 | 7.22 dB |
|
||||
| 00146 | 35.25 | 29.35 | 5.89 dB |
|
||||
| 00181 | 41.92 | 32.87 | 9.05 dB |
|
||||
|
||||
Visually: Dirk's face disintegrates, teeth and eyes turn to mush, ink outlines
|
||||
break into 4-pixel stair-steps, colour bleeds across block boundaries.
|
||||
|
||||

|
||||
*Left: 1080p source. Middle: 256-colour palettised 256x192 — the quality ceiling,
|
||||
and it is excellent. Right: flat 4x4 VQ at k=256. This is the result that killed
|
||||
the flat-VQ architecture.*
|
||||
|
||||
**Crucially, the 256-colour palettised frame itself looks excellent.** Flat cel
|
||||
art with a per-scene median-cut palette and no dithering is near-transparent
|
||||
(35-42 dB). So the palette is not the problem and 256 colours is not the
|
||||
problem — **block VQ is**. The quality ceiling we should hold ourselves to is
|
||||
the palettised frame, not the 1080p source.
|
||||
|
||||
## 10. Hybrid VQ (Cinepak V1/V4 + SKIP) — this is the codec
|
||||
|
||||
Per 4x4 block, choose by rate-distortion: SKIP (reuse previous frame),
|
||||
V1 (one 4x4 codeword, 1 byte), or V4 (four 2x2 codewords, 4 bytes),
|
||||
with a 2-bit-per-block mode header. `lam` is the lagrangian rate knob.
|
||||
|
||||
Measured, k1=k4=256, 4 scenes (mean of the per-scene table in the session log):
|
||||
|
||||
| lam | PSNR | loss vs palette | SKIP% | V1% | V4% | B/frame | KB/s @12 |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| 0 (max quality) | 33.9 | 4.9 | 30.8 | 18.5 | 50.8 | 7574 | 88.8 |
|
||||
| 200 | 31.9 | 5.9 | 44.0 | 37.6 | 18.4 | 4183 | 49.0 |
|
||||
| 1000 | 31.6 | 7.3 | 47.4 | 47.7 | 4.9 | 2841 | 33.3 |
|
||||
| 5000 | 25.5 | 13.3 | 55.6 | 44.4 | 0.0 | 2134 | 25.0 |
|
||||
|
||||
At a **matched ~30 KB/s** the hybrid beats flat 4x4 VQ by ~1 dB, and unlike flat
|
||||
VQ it keeps scaling: at 89 KB/s it reaches within **4.9 dB of the palette
|
||||
ceiling**, which flat VQ cannot reach at any bitrate.
|
||||
|
||||
Note V4% collapses to 0 at lam=5000 — that is the knob doing exactly what it
|
||||
should: under a hard ceiling, detail blocks are the first thing sacrificed.
|
||||
|
||||
## 11. Codebook size sweep (flat 4x4, for reference)
|
||||
|
||||
| block | k | PSNR | loss | key B | changed% | KB/s @12 | codebook RAM |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| 4x4 | 256 | 30.46 | 8.39 | 3072 | 52.7 | 28.5 | 8K |
|
||||
| 4x4 | 1024 | 32.89 | 5.96 | 3840 | 56.6 | 35.6 | 32K |
|
||||
|
||||
+2.4 dB for 24K more RAM and 7 KB/s. With 2 MB of RAM, a 1024-entry codebook is
|
||||
cheap and clearly worth it. (RAM figure is the word-expanded form the blitter
|
||||
wants: k * 16 px * 2 bytes.)
|
||||
|
||||
## 12. Source framing — OPEN
|
||||
|
||||
The Blu-ray is **full-frame 1920x1080 16:9 with no pillarboxing**. The arcade
|
||||
original is 4:3. The extractor currently centre-crops 1440x1080, which is the
|
||||
arcade-faithful choice but discards image the 2006 remaster added. Options are
|
||||
`crop` (default), `squash`, `wide` in `tools/encoder/extract.py`.
|
||||
**Not yet decided; needs an eyeball comparison against arcade reference.**
|
||||
|
||||
## 13. Stream inventory correction
|
||||
|
||||
Session 1 said "typical scene clip ~60s". Sampled directly: the ~3-5 MB streams
|
||||
are **1.2-1.7 s** clips — these are the individual arcade death/action moments,
|
||||
which is exactly the granularity the game logic needs. Some 60 s streams
|
||||
(e.g. 00203) are **menu screens, not content**. Any survey must classify
|
||||
menu vs content before averaging, or the bitrate numbers are diluted by static
|
||||
menus.
|
||||
|
||||
## 14. A FOURTH false-good result — and the correction
|
||||
|
||||
Add this to the 4 list. The mechanism was new but the shape was identical.
|
||||
|
||||
**The false result:** flat and hybrid VQ both showed **+2.4 dB for k=1024 over
|
||||
k=256** at an apparently similar bitrate, which made a 1024-entry codebook look
|
||||
like an obvious win. The k=1024 quality ladder rendered from that run looked
|
||||
great at "45 KB/s".
|
||||
|
||||
**The bug:** the rate-distortion model in `vq_hybrid.encode()` charged **1 byte**
|
||||
per codebook index unconditionally. A 1024-entry codebook needs a **10-bit index,
|
||||
stored as 2 bytes**. So every k=1024 measurement understated the V1 and V4
|
||||
payload by exactly 2x, *and* the lagrangian mode decision was choosing V4 on the
|
||||
belief that four codewords cost 4 bytes when they cost 8.
|
||||
|
||||
**After charging the true index cost** (`idx_bytes` is now explicit and defaults
|
||||
from the codebook size), matched-bitrate comparison on scene 00020:
|
||||
|
||||
| KB/s | k=256 (1-byte idx) | k=1024 (2-byte idx) |
|
||||
|---|---|---|
|
||||
| ~32-42 | **33.87 dB** @ 32.5 | 28.91 dB @ 42.3 |
|
||||
| ~44-52 | **34.80 dB** @ 44.1 | 35.13 dB @ 52.5 |
|
||||
| ~72-86 | **35.87 dB** @ 72.2 | 36.51 dB @ 86.0 |
|
||||
|
||||
k=1024 buys +0.3 to +0.6 dB for +19% bitrate — a wash at best — and at the low
|
||||
end where the SASI profile lives it is **5 dB worse**, because the 2-byte index
|
||||
floor dominates once V4 is priced out.
|
||||
|
||||
**k=256 with 1-byte indices is the shipping choice.** It is also the better
|
||||
decoder: a plain `move.b` index with no alignment case, and an 8 KB codebook
|
||||
instead of 32 KB.
|
||||
|
||||
**The general lesson, again:** the comparison was not wrong about VQ, it was
|
||||
wrong about *cost*. When a knob looks like a free win, check that the rate model
|
||||
is charging for it. Same failure family as 4.1-4.3: a plausible number produced
|
||||
by a pipeline that was not measuring what it claimed to measure.
|
||||
|
||||
## 15. Rate-distortion curve of the shipping codec (k=256, corrected)
|
||||
|
||||
Scene 00020 (Dirk screaming, close-up face — the hardest case for linework),
|
||||
and 00146. Includes the 2-bit-per-block mode header. No entropy coding yet.
|
||||
|
||||
| lam | 00020 PSNR | 00020 KB/s | 00146 PSNR | 00146 KB/s | SKIP | V1 | V4 | RAW |
|
||||
|---|---|---|---|---|---|---|---|---|
|
||||
| 25 | 38.68 | 182.2 | 31.04 | 193.5 | ~37% | ~24% | ~13% | ~26% |
|
||||
| 100 | 35.87 | 72.2 | 29.04 | 72.5 | ~41% | ~34% | ~21% | ~4% |
|
||||
| 300 | 34.80 | 44.1 | 28.28 | 44.4 | ~44% | ~42% | ~14% | 0% |
|
||||
| 800 | 33.87 | 32.5 | 27.77 | 36.1 | ~46% | ~48% | ~5% | 0% |
|
||||
| 2000 | 27.57 | 25.5 | 24.88 | 30.2 | ~50% | ~49% | ~1% | 0% |
|
||||
|
||||
Palette ceilings: 00020 = 39.90 dB, 00146 = 35.25 dB.
|
||||
|
||||

|
||||
*The shipping codec across the rate knob. Top: source, palette ceiling, lam=25.
|
||||
Bottom: lam=100 (`scsi` profile), lam=300 (`sasi` profile), lam=800.
|
||||
Both shipping profiles hold Bluth's linework; the failure only starts past lam=800.*
|
||||
|
||||
Two things to read off this table:
|
||||
- **The cliff is between lam=800 and lam=2000.** That is where V4 is priced out
|
||||
entirely and detail blocks have nowhere to go. Do not ship past lam~800.
|
||||
- **RAW is doing real work at high bitrate** (26% of blocks at lam=25) and
|
||||
vanishes by lam=300. It is what makes the top of the curve reach the palette
|
||||
ceiling, and it costs the decoder nothing — RAW is the cheapest mode to blit.
|
||||
|
||||
## 16. Licences cleared for the game-logic layer
|
||||
|
||||
Both checked this session:
|
||||
|
||||
- **astrobleem/SNES-SuperDragonsLairArcade — MIT**, "Copyright (c) 2026 Chad
|
||||
Doebelin". `data/events/` holds 516 XML chapter definitions with timing and
|
||||
event data. Reusable with attribution.
|
||||
- **icculus/DirkSimple — zlib.** Independent from-scratch reimplementation of
|
||||
the game logic in Lua, scene/timing tables in `game.lua`. Also permissive.
|
||||
|
||||
Having **two independent permissively-licensed transcriptions** of the arcade
|
||||
scene graph is better than one: they can be diffed against each other to catch
|
||||
transcription errors before any of it is committed to 68000 tables.
|
||||
|
||||
+104
-103
@@ -1,4 +1,4 @@
|
||||
# Status & next-session handoff — end of session 1 (2026-08-23)
|
||||
# Status & next-session handoff — end of session 2 (2026-08-23)
|
||||
|
||||
## Decisions locked
|
||||
|
||||
@@ -7,133 +7,134 @@
|
||||
| Target CPU | 68000 @ 10MHz (stock) | hardest honest constraint |
|
||||
| Display mode | 256 colors, 256x192 in 256x256 CRTC mode | every mode is 1 word-access/pixel, so 256c is free vs 16c |
|
||||
| Double buffer | **none** — page 1 sacrificed | enables `movem.l` 24px bursts; delta coding needs a RAM reference frame anyway |
|
||||
| Codec | 4x4 vector quantization, per-scene codebook + block delta | CPU is idle, I/O is the ceiling — spend cycles to buy bandwidth |
|
||||
| **Codec** | **hybrid VQ: SKIP / V1 4x4 / V4 four-2x2 / RAW, per-block rate-distortion** | flat 4x4 VQ was measured and rejected — see FINDINGS 9-10 |
|
||||
| **Quality modes** | **two: `sasi` and `scsi`** (USER DECISION, session 2) | one codec, one decoder, one bitstream; only `lam` differs |
|
||||
| Framerate | 12 fps, **explicit decimation** | source has zero duplicate frames; no free "twos" win |
|
||||
| Medium | SCSI HDD image (.hds) | but see SASI/SCSI split below |
|
||||
| Emulator | MAME 0.277 x68000 | accurate enough that measured cycles mean something |
|
||||
| SNES project reuse | **MIT — cleared** | `data/events/` scene graph is reusable with attribution |
|
||||
|
||||
**OPEN QUESTION for the user:** stock 10MHz machines are **SASI**, not SCSI.
|
||||
Three options, not yet chosen:
|
||||
1. Stock 10MHz + SASI (purist) — VQ becomes mandatory
|
||||
2. Stock 10MHz + CZ-6BS1 SCSI board — relieves I/O, keeps CPU honest
|
||||
3. Super/XVI baseline — built-in SCSI, still a 10MHz 68000
|
||||
### The SASI/SCSI question is RESOLVED
|
||||
Session 1 left "which machine do we target" open. The user's answer: **ship both**,
|
||||
as two quality profiles. This is now implemented rather than hypothetical — the
|
||||
bitrate ceiling is a build parameter in `tools/encoder/ratectl.py`:
|
||||
|
||||
Recommendation: make the codec's bitrate ceiling a **build parameter**, so one
|
||||
encoder serves all three and the target is chosen at package time.
|
||||
| profile | target | lam | quality (00020 / 00146) | machine |
|
||||
|---|---|---|---|---|
|
||||
| `sasi` | 45 KB/s | 300 | 34.8 / 28.3 dB | stock 10MHz ACE/EXPERT |
|
||||
| `scsi` | 75 KB/s | 100 | 35.9 / 29.0 dB | Super/XVI, or CZ-6BS1 board |
|
||||
|
||||
Codebooks are **k=256 with 1-byte indices** in both profiles. k=1024 was measured
|
||||
and rejected — see FINDINGS 14, it was a false-good result from a rate model
|
||||
that undercharged the index. Do not ship past `lam~800`; FINDINGS 15 has the cliff.
|
||||
|
||||
Because of the RAW escape mode, `lam=0` is **pixel-exact** against the palettised
|
||||
frame (measured 0.00 dB loss). The profiles are two points on one continuous
|
||||
rate-distortion curve, not two codecs.
|
||||
|
||||
---
|
||||
|
||||
## Working setup
|
||||
## What session 2 settled
|
||||
|
||||
**MAME ROMs** — `~/mame/roms/x68000.zip` (present, working).
|
||||
Must pass **`-bios ipl10`**; the default BIOS is `cz600ce`, whose split
|
||||
even/odd IPL halves (`rh-ix0897cezz.ic12` / `rh-ix0898cezz.ic11`) are absent.
|
||||
`-verifyroms` will still report those two as missing — this is expected and harmless.
|
||||
1. **The critical-path question is answered.** "Does VQ soften Bluth's linework
|
||||
unacceptably?" — **flat 4x4 k=256 VQ: yes, badly. Hybrid VQ with k=1024: no.**
|
||||
Verified by eye, not just PSNR. See `docs/FINDINGS.md` 9-11.
|
||||
2. **Session 1's 12fps bitrate was wrong** (183 KB/s claimed, 340 KB/s measured).
|
||||
Halving the framerate does not halve the bitrate. FINDINGS 8.
|
||||
2b. **A fourth false-good result was produced and caught this session** — k=1024
|
||||
codebooks looked like a +2.4 dB free win because the rate model charged 1 byte
|
||||
for a 10-bit index. FINDINGS 14. The k=256 configuration ships.
|
||||
3. **The 256-colour palettised frame is the real quality ceiling** and it looks
|
||||
excellent. Judge the codec against that, not against 1080p.
|
||||
4. Encoder exists and produces a real bitstream: `tools/encoder/`.
|
||||
|
||||
Boots headless at ~430-480% speed:
|
||||
---
|
||||
|
||||
## Encoder — working
|
||||
|
||||
```
|
||||
python3 tools/encoder/extract.py 00020 /tmp/fr_00020 12 crop
|
||||
python3 tools/encoder/encode.py /tmp/fr_00020 out.dlx --profile sasi --preview p.png
|
||||
```
|
||||
|
||||
| file | role |
|
||||
|---|---|
|
||||
| `extract.py` | .m2ts -> 256x192 PNGs, 12fps, spatial-only denoise |
|
||||
| `vq.py` | palette, blockify, hand-rolled k-means (no sklearn on this box), PSNR |
|
||||
| `vq_hybrid.py` | the codec: 4 block modes + lagrangian mode decision |
|
||||
| `ratectl.py` | SASI/SCSI profiles, leaky-bucket rate control |
|
||||
| `encode.py` | CLI + `DLX1` container writer |
|
||||
|
||||
`DLX1` container layout is documented in the `encode.py` docstring. All
|
||||
multi-byte fields are **big-endian** so the 68000 reads them with a plain `move`.
|
||||
|
||||
### Known encoder gaps
|
||||
- **Rate control is written but not yet wired into `encode.py`** — the CLI uses a
|
||||
fixed `lam` from the profile. `ratectl.encode_rate_controlled()` exists and
|
||||
builds a lam-ladder per frame; it needs hooking up and validating.
|
||||
- **Payload is not entropy-coded.** Deflate on the payload should buy ~1.4x
|
||||
(measured on the lossless path, FINDINGS 8). LZ decode is cheap on a 68000.
|
||||
- Codebooks are per-scene and rebuilt from scratch; no inter-scene reuse.
|
||||
- `_paint` is a Python per-block loop — fine for prototyping, slow for a full
|
||||
disc encode. Vectorise before the 224-stream run.
|
||||
|
||||
---
|
||||
|
||||
## Working setup (unchanged from session 1, re-verified)
|
||||
|
||||
**MAME ROMs** — `~/mame/roms/x68000.zip`. Must pass **`-bios ipl10`**.
|
||||
```
|
||||
mame x68000 -bios ipl10 -video none -sound none -nothrottle -seconds_to_run 3
|
||||
```
|
||||
**Assembler** — `tools/vasm/vasmm68k_mot -Fbin -o out.bin in.s`
|
||||
|
||||
**Assembler** — vasm built from source, binary at `tools/vasm/vasmm68k_mot`
|
||||
(source tarball alongside it). Verified correct 68000 output.
|
||||
```
|
||||
tools/vasm/vasmm68k_mot -Fbin -o out.bin in.s
|
||||
```
|
||||
**Blu-ray** — `udisksctl loop-setup -r -f DRAGONS_LAIR.iso` -> `/media/reala-misaki/BDROM`
|
||||
(still mounted as of end of session 2).
|
||||
|
||||
**Blu-ray** — mount with:
|
||||
```
|
||||
udisksctl loop-setup -r -f DRAGONS_LAIR.iso # -> /media/reala-misaki/BDROM
|
||||
```
|
||||
NOTE: this loop mount is still active from session 1. Re-mount if the machine rebooted.
|
||||
**MAME Lua harness** — `tools/bench/*.lua`, working. Three gotchas (retain the
|
||||
notifier subscription in a global; the stack register is `SP` not `A7`;
|
||||
`autoboot_script` fires at PC=0 before boot) are documented in FINDINGS.
|
||||
|
||||
**Two shell traps, both hit again this session:**
|
||||
- piping MAME (or any long job) through `grep` block-buffers — write to a file.
|
||||
- `pkill -f <pattern>` matches your own shell and kills it (exit 144).
|
||||
Use `pkill -x` or kill by PID.
|
||||
|
||||
---
|
||||
|
||||
## MAME Lua harness — WORKING, reusable
|
||||
## STILL BLOCKED: disk throughput benchmark
|
||||
|
||||
`tools/bench/*.lua` inject 68000 machine code straight into emulated RAM and time
|
||||
it against the emulated clock. No bootable disk or OS required. This is the
|
||||
measurement rig for all future cycle-cost work (blit timing, decoder benchmarks).
|
||||
Unchanged from session 1 — `IOCS _B_READ` returns -1 uniformly. Full diagnosis
|
||||
and the four untested hypotheses are in session 1's notes (git history of this
|
||||
file, commit 65112b9).
|
||||
|
||||
Pattern:
|
||||
```
|
||||
mame x68000 -bios ipl10 -video none -sound none -nothrottle \
|
||||
-seconds_to_run 30 -plugins -autoboot_script yourscript.lua
|
||||
```
|
||||
|
||||
### Three MAME Lua gotchas — all cost real time, all now solved
|
||||
1. **Retain the notifier subscription.** `emu.add_machine_frame_notifier()` returns
|
||||
a token; if you drop it into a chunk-local it is garbage-collected and the
|
||||
callback **silently stops firing**. Assign it to a **global** (`SUB = ...`).
|
||||
2. **The stack pointer is `SP`, not `A7`** in `cpu.state[...]`.
|
||||
Full list: A0-A6, D0-D7, PC, SP, SR, USP, CURPC, CURFLAGS, IR.
|
||||
3. **`autoboot_script` fires at time=0, before boot** (PC=0). Wait until
|
||||
`machine.time` >= ~5s before injecting, or IOCS is not yet initialised.
|
||||
|
||||
Also: piping MAME through `grep` block-buffers output — write raw to a file when
|
||||
backgrounding, or you will see an empty log and assume a hang.
|
||||
And never `pkill -f 'mame x68000'` — the pattern matches your own shell and kills it
|
||||
(exit 144). Use `pkill -x mame`.
|
||||
|
||||
---
|
||||
|
||||
## BLOCKED: disk throughput benchmark
|
||||
|
||||
**Goal:** measure real SASI/SCSI KB/s to replace the folklore figures in FINDINGS.md §5.
|
||||
|
||||
**Status:** harness fully working; the IOCS call itself fails.
|
||||
|
||||
`IOCS _B_READ` ($46 via `TRAP #15`; d1.hb=PDA, d2.l=position, d3.l=bytes, a1=buffer)
|
||||
returns **`FFFFFFFF` (-1), zero reads**, uniformly across:
|
||||
- all 16 PDA values $80-$8F
|
||||
- both d1 encodings (PDA in bits 31-24 and bits 15-8)
|
||||
- image sizes 10MB / 20MB / 40MB
|
||||
|
||||
The uniformity is the diagnostic: calls are **dispatched and cleanly rejected**,
|
||||
so `TRAP #15` and IOCS are reachable. MAME does mount the image
|
||||
(`:x68k_hdc: opened image file bench.hdf`).
|
||||
|
||||
**Untested hypotheses, in rough order of likelihood:**
|
||||
1. The raw image has no X68000 SASI format, so the IPL's boot scan never registered
|
||||
a usable drive and IOCS refuses. Would need Human68k to format one — **we have
|
||||
no Human68k image on this system.**
|
||||
2. MAME's `x68k_hdc` SASI implementation may be too partial for IOCS-level reads.
|
||||
3. `SP=$8000` may put the injected stack on top of the IOCS work area in low RAM.
|
||||
Try a much higher stack.
|
||||
4. The **SCSI path was never tried** — this is the obvious next move and is more
|
||||
relevant to the target anyway:
|
||||
`-exp1 cz6bs1 -hard disk.chd` with `exp1:cz6bs1:scsi:0 harddisk`
|
||||
(`-listmedia` gains a `harddisk` slot accepting .chd/.hd/.hdv/.2mg/.hdi).
|
||||
|
||||
**Honest assessment: this benchmark is NOT on the critical path.** The VQ codec
|
||||
(~30 KB/s) is correct whether SASI does 300 or 600 KB/s. Do not let it block the
|
||||
encoder. Its real value is deciding whether the *simpler* row-span codec could
|
||||
have sufficed.
|
||||
|
||||
Caveat if resumed: MAME idealizes drive seek latency. That's acceptable because the
|
||||
realistic deployment is BlueSCSI/SCSI2SD (SD-backed, no mechanical seek), so what
|
||||
gets measured is the bus/DMAC/controller path — the genuine ceiling. The caveat
|
||||
only bites for a real period spinning drive.
|
||||
**This now matters more than session 1 thought.** Session 1 dismissed it because
|
||||
"VQ at 30 KB/s is correct whether SASI does 300 or 600 KB/s". But we now ship
|
||||
*two profiles*, and the profile bitrates (45 / 120 KB/s) are set against
|
||||
**folklore** bandwidth figures. A real measurement would let us set them
|
||||
honestly instead of conservatively. Next move is the untried SCSI path:
|
||||
`-exp1 cz6bs1 -hard disk.chd`.
|
||||
|
||||
---
|
||||
|
||||
## Next steps, in priority order
|
||||
|
||||
1. **Build the VQ encoder** (`tools/encoder/`) — 4x4 blocks, per-scene codebook,
|
||||
block delta. Emit sample PNGs for visual evaluation. **The open question is
|
||||
whether VQ softens Bluth's ink linework unacceptably — decide by eye before
|
||||
committing to the architecture.**
|
||||
2. **Full-disc survey** — all 224 streams, not 5s samples, to firm up bitrate
|
||||
(current numbers are +/-30%) and map streams onto the arcade scene graph.
|
||||
3. **Check the SNES project's license**, then evaluate reusing `data/events/`
|
||||
(516 chapters / 29 scenes) as the scene-graph and input-timing layer.
|
||||
4. Resolve the SASI/SCSI target question with the user.
|
||||
5. Optionally unblock the disk benchmark via the SCSI path.
|
||||
6. 68000 player skeleton: CRTC init for 256x192x256c, `movem.l` blitter,
|
||||
ADPCM via HD63450 DMA.
|
||||
1. **Wire rate control into `encode.py`** and validate that the hard ceiling
|
||||
actually holds on an action scene (the whole point of choosing VQ).
|
||||
2. **Entropy-code the payload** (deflate) — ~1.4x for cheap 68000 decode cost.
|
||||
3. **68000 decoder skeleton**: parse `DLX1`, expand codebooks to word-per-pixel,
|
||||
blit V1/V4/RAW/SKIP. Measure real cycles with the existing MAME Lua harness —
|
||||
this is the first time the harness gets used for its actual purpose.
|
||||
4. **Full-disc survey** — classify menu vs content first (FINDINGS 13), then
|
||||
measure bitrate across all 224 streams per profile.
|
||||
5. **Resolve the framing question** (FINDINGS 12: crop vs squash vs wide).
|
||||
6. Unblock the disk benchmark via the SCSI path, then re-set profile bitrates.
|
||||
7. Import the SNES project's `data/events/` (MIT, cleared) as the scene graph.
|
||||
Cross-check against DirkSimple (zlib) which has the same data independently.
|
||||
8. ADPCM audio: MSM6258, 15.6kHz mono, 7.8 KB/s — already budgeted in `ratectl`,
|
||||
not yet extracted or encoded.
|
||||
|
||||
## Not yet started
|
||||
- Any 68000 player code
|
||||
- ADPCM audio extraction/encoding (MSM6258, 15.6kHz mono, ~7.8 KB/s, ~10MB for 22min)
|
||||
- Disk image packaging / container format
|
||||
- ADPCM audio extraction/encoding
|
||||
- Disk image packaging
|
||||
- Game logic (scene branching, input windows, death clips)
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 680 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 1.3 MiB |
Reference in New Issue
Block a user