drwily/Dragon-s-Lair-X68k
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

The 68000 decoder draws pixel-exact frames, and does not fit

Browse Source 
src/player/decode.s parses DLX1 and decodes straight into GVRAM. Verified
pixel-exact over a 120-frame sequential run of the worst sustained window on
the disc -- all four block modes, full temporal recursion, so the last frame
is only right if all 120 were. In check.sh.

It costs a mean of 81.7% of a 12fps frame budget, and 31% of frames exceed
100% (42% at scsi). CPU is now the binding constraint. FINDINGS 28.

Three things that were believed and are not true:

- The dual-display-path plan of FINDINGS 24.5/25.6 is incoherent. The compose
  path needs a RAM copy of the previous reconstruction; the direct path's
  selling point is that it keeps none. Mixing them shows stale pixels on 70 of
  120 frames, worst frame 18.8% of the screen. Every coherent repair is dearer
  than not mixing, and 24.5's two figures were both copies with no decode in
  either, so there was never a crossover to find. One path ships, and the 96KB
  reference frame is gone. tools/analysis/10_pathmix_drift.py keeps the
  counterexample runnable; check.sh asserts it still reproduces.

- The four block modes do not cost the same. V1 300, V4 448, RAW 400 cycles
  against the old model's flat 207.8. V4 is 25% of blocks and 50% of the
  cycles, and the mode decision charges it bytes it does not charge cycles for.
  tools/analysis/11_cpu_budget.py reproduces all four frames timed on the
  68000 to within 1 point. Hand-derived timings agree to 0.5% on V1.

- The container is big-endian but not aligned. Variable-length records laid end
  to end put frame 1's length field at an odd address, and move.l (a0)+ there
  is an address error: frame 0 decoded perfectly and then vectored into the
  IPL for 59 emulated seconds looking like a hang. Found by dumping PC, not by
  reading the source.

Also: an all-V1 frame, the cheapest possible full redraw, is 110.5% of budget.
No mode assignment fits a scene cut at 12fps. That one needs a decision, not a
measurement.

Next: charge cycles in the mode decision and bisect against 833,333 per frame,
the way session 6 bisects lam against bytes -- but with no bucket, because a
late frame cannot be banked.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
This commit is contained in:
prosolis
2026-08-23 15:04:38 -07:00
parent 497f88b945
commit e1aa26bb57
11 changed files with 1276 additions and 38 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/STATUS.md
+161 -33
View File
@@ -1,29 +1,102 @@
# Status & next-session handoff — end of session 6 (2026-08-23)
# Status & next-session handoff — end of session 7 (2026-08-23)
## NEXT SESSION: the 68000 decoder skeleton
## NEXT SESSION: make the mode decision cost-aware
Rate control is done and gated (below). `src/player/` is still empty, and it is
now the only thing between this project and an answer to "does the CPU path
work". Everything it needs has been measured:
The decoder exists, it is pixel-exact, and **it does not fit**. On the worst
sustained window at the shipping `sasi` profile it costs a mean of **81.7% of a
12fps frame budget** and **31% of frames exceed 100%** (`scsi`: 94.9% median,
42% of frames miss). FINDINGS 28. CPU is now the binding constraint — the first
time in this project that it has been.
1. **Inner loop: implement BOTH display paths and pick per frame.** FINDINGS
25.6, re-measured under rate control in 27.3. Compose-in-RAM-then-blit is a
flat 53.6% of the 12fps budget; decode-direct-to-GVRAM is 76.6% x the
non-SKIP block fraction. They cross at 70% of blocks changed. The mode
headers are parsed before any pixel is written, so counting non-SKIP blocks
to choose is free. Median cost 36.6% (`sasi`) / 47.1% (`scsi`), capped 53.6%.
2. **Copy the harness pattern from `tools/bench/blit.s` + `blit.lua`** — it
already loads code, masks interrupts, times a loop against a flag, and
snapshots for `verify_frame256.py`. Assemble with
`tools/vasm/vasmm68k_mot -Fbin -o out.bin in.s`.
3. **Parse `DLX1`** (layout in the `encode.py` docstring, all fields big-endian),
expand the codebooks once at load, then dispatch per block on the 2-bit mode.
4. **Budget against 53.6%, not 38%.** Still not done — see priority 2a below.
The blit alone eats over half the frame before any decoding, and MAME models
no GVRAM wait states, so it is a floor.
The fix is not assembly micro-optimisation. It is that **`vq_hybrid.decide()`
minimises `D + lam*R` — distortion against BYTES — on a machine where the
binding budget is CYCLES**, and the two are not proportional:
Feed it `tmp/rc_fr_singe_sasi_rcprofile.dlx` — the worst sustained window on the
disc, at the shipping profile. If the decoder fits there it fits everywhere.
| mode | payload bytes | measured cycles | cycles per byte |
|---|---:|---:|---:|
| SKIP | 0 | 13 (clustered) | — |
| V1 | 1 | 300 | 300 |
| V4 | 4 | 448 | 112 |
| RAW | 16 | 400 | 25 |
V4 is **25% of blocks and 50% of the cycles**. The lagrangian charges it 4x a V1
block; the CPU charges it 1.49x. So the encoder currently buys V4 whenever it is
worth 4 bytes, with no idea what it costs to draw.
**The work, in order:**
1. **Add a cycle term to the mode decision.** `decide()` already builds a cost
matrix of `error + lam * bytes` per mode per block; add `+ mu * cycles`, with
the cycles vector `[13, 300, 448, 400]` measured in FINDINGS 28.2. One extra
row of arithmetic in a function that is already vectorised.
2. **Then bisect `mu` per frame against the 833,333-cycle budget**, exactly as
session 6 bisects `lam` against the byte budget. The machinery is already
there and already gated: `ratectl.encode_rate_controlled` is frame-driven and
feeds back the frame it emitted. **But cycles have NO bucket.** Bytes can be
banked in the ring buffer; a frame that misses its decode deadline is just
late, because there is no double buffer to decode ahead into. So this is a
hard per-frame ceiling, not a leaky bucket — simpler than rate control, and
the two controllers have to run together (raising `mu` moves blocks to SKIP
and V1, which also *lowers* the bitrate, so the byte controller must see it).
3. **Measure the quality cost.** Everything session 6 did for bytes: what does
fitting 100% of frames in the CPU budget cost in dB, and does any frame hit a
cliff? `tools/analysis/11_cpu_budget.py` scores a container without needing
MAME, so the search loop is cheap; confirm the winner on the 68000 with
`tools/bench/decode.lua`.
4. **28.5 may not be solvable by the encoder at all.** An all-V1 frame — the
cheapest possible full redraw — is **110.5%** of the budget. A scene cut
changes 100% of the screen, so *no* mode assignment fits one at 12fps. Decide
deliberately: allow one late frame at a cut (the outgoing content is
unrelated, so it may be invisible), spread a cut over two frame times, or
drop to 10fps where an all-V1 frame fits. This is a design decision, not a
measurement, and it needs the user.
**Do not start by hand-optimising `decode.s`.** The hand-derived timings agree
with the measurements to 0.5% on V1 and 1% on RAW (FINDINGS 28.4), so the
inner loop is close to what the instruction set allows; the plausible wins are
single-digit percentages against a 36-point gap. The V4 write pattern is the one
place worth a look afterwards — pairing sub-block rows into `movem.l d0/d2,(a4)`
saves ~16 of 448 cycles.
---
## What session 7 settled
1. **68000 code parses a bitstream and draws frames, pixel-exact.**
`src/player/decode.s` + `tools/bench/decode.lua`. 120 frames of the Singe
window decoded in sequence, all four block modes, verified against the new
reference decoder `tools/encoder/dlx.py`. Because SKIP blocks are claims
about the previous frame, the last frame is only right if all 120 were.
In `check.sh` now. **FINDINGS 28.**
2. **It does not fit.** Mean 81.7% of a 12fps frame, p90 116.4%, worst 135.8%;
31% of frames miss at `sasi`, 42% at `scsi`. Zero-wait-state floor, as ever.
3. **The dual-display-path plan (FINDINGS 24.5/25.6) is withdrawn as incoherent
— the sixth false premise this project has caught.** The compose path needs a
RAM copy of the previous reconstruction; the direct path's whole selling
point is that it keeps none. Mixing them displays stale pixels on **70 of 120
frames**, worst frame 18.8% of the screen. Every coherent repair is worse
than not mixing. `tools/analysis/10_pathmix_drift.py`, kept runnable as a
counterexample and gated in `check.sh`. FINDINGS 28.1.
4. **24.5 also compared a copy against a copy.** Its 53.6% and 76.6% both come
from `blit.s` and neither includes decoding. Compose = decode-into-RAM *plus*
the 53.6% blit, so it is strictly dearer than decoding into GVRAM. There was
never a crossover. The player has **one path and no reference frame**, which
also gives back 96 KB.
5. **The four block modes cost 300 / 448 / 400 cycles, not one number.** V4 is
1.49x a V1 block while the mode decision charges it 4x the bytes. The 24.5
model is 2.03x optimistic at the median. `tools/analysis/11_cpu_budget.py`
reproduces all four frames timed on the 68000 to within 1 point. FINDINGS 28.2.
6. **The container is big-endian but not aligned, and on a 68000 that is an
address error, not a slow read.** Frame records are variable-length and laid
end to end, so their boundaries land on odd addresses. Frame 0 decoded
perfectly, then the length read for frame 1 vectored into the IPL and sat
there for 59 emulated seconds looking like an infinite loop. Found by dumping
PC and the address registers — the code was right, the data layout was not.
FINDINGS 28.3. **Encoder gap: `encode.py` should pad records to 4 bytes.**
Measured cost 1.66 B/frame = 20 B/s against 110 KB/s.
7. **A full frame does not fit at 12fps in any mode.** All-V1 is 110.5%, all-V4
165.2%, all-RAW 147.6%. At most ~88% of the screen can change in one frame
however cheaply it is coded, and scene cuts change 100%. FINDINGS 28.5.
---
@@ -265,6 +338,14 @@ multi-byte fields are **big-endian** so the 68000 reads them with a plain `move`
Gated by `tools/analysis/09_ratectl_drift.py`, which is now in `check.sh`.
- **Payload is deliberately NOT entropy-coded** — deflate decode does not fit in
the 68000's frame budget (FINDINGS 17.2). Do not "optimise" this later.
- **Frame records are not aligned.** They must be padded to a 4-byte boundary:
unaligned is an ADDRESS ERROR on a 68000, not a slow read (FINDINGS 28.3).
`prep_dlx.py` repairs it at load time, which a player streaming from disc
cannot do. The pad is real bytes on disc, so it belongs inside the rate
controller's accounting. 1.66 B/frame, 20 B/s.
- **The mode decision is blind to CPU cost.** It charges V4 four payload bytes
and ignores that it costs 1.49x a V1 block to draw. This is the top item at
the head of this file. FINDINGS 28.2.
- **Palette packing is not implemented in the encoder.** It still emits 24-bit
palettes; the X68000 word packing happens Lua-side. Whatever writes real
palette words must pick `I` per entry by minimum squared error (FINDINGS 23.3,
@@ -414,15 +495,25 @@ SDL_VIDEODRIVER=dummy mame x68000 -bios ipl10 -video soft -window \
it needs a genuinely quiet scene to decide, and it is a quality-per-byte
judgement rather than a correctness one.
2. **68000 decoder skeleton**, with the inner loop chosen by (1). **This is now
the agreed next session's work — see the block at the top of this file.** Parse `DLX1`,
expand codebooks, blit per block mode. The display path is verified *by 68000
code* now (FINDINGS 24) and the harness pattern is `tools/bench/blit.s` +
`blit.lua`, which already loads code, masks interrupts, times a loop against
a flag, and snapshots the result for `verify_frame256.py`. Copy that.
Assembler: `tools/vasm/vasmm68k_mot -Fbin -o out.bin in.s`.
2. ~~**68000 decoder skeleton.**~~ **DONE, session 7.** `src/player/decode.s`,
pixel-exact over 120 frames, gated in `check.sh`. It answered the question it
was written to answer, and the answer is no: **it does not fit** — mean 81.7%
of a 12fps frame, 31% of frames over 100%. FINDINGS 28. The follow-on is
priority 0 at the top of this file.
2a. **Re-budget everything against 53.6%, not 38%.** Several downstream figures
2b. **Pad frame records to 4 bytes in `encode.py`.** Not optional: unaligned
records are an address error on a 68000 (FINDINGS 28.3), and `prep_dlx.py`
currently repairs it at load time, which the shipping player streaming from
disc cannot do. The padding is real bytes on disc, so it has to be inside
the rate controller's accounting, not added after it. 20 B/s at 12fps.
2a. **Re-budget everything against the MEASURED per-mode costs**, not 53.6% and
not 38%. Session 7 replaced the model twice over (FINDINGS 28.2): the display
path is not one number times a block fraction, and the median frame is 74.4%
rather than 36.6%. The original note is kept below because its warning about
downstream figures derived from a dead estimate is exactly what happened
again.
~~Re-budget everything against 53.6%, not 38%.~~ Several downstream figures
were derived from the old estimate. The blit alone now eats over half the
frame at 12fps in the compose-then-blit design, before any decode, and MAME
models no GVRAM wait states so that is a floor. This may reopen questions
@@ -465,9 +556,13 @@ SDL_VIDEODRIVER=dummy mame x68000 -bios ipl10 -video soft -window \
- ~~Flat 4x4 VQ.~~ Rejected by eye (FINDINGS 9).
## Not yet started
- **Any 68000 player code.** `src/player/` is still empty. 68000 code has now
drawn a frame, but it lives in `tools/bench/blit.s` as a benchmark, not in a
player: it does no bitstream parsing, no mode dispatch, no codebook expansion.
- **A player, as opposed to a decoder.** `src/player/decode.s` parses DLX1,
dispatches all four block modes and draws pixel-exact frames, but it decodes
from RAM that Lua pre-loaded. There is no disc streaming, no ring buffer, no
audio, no timing against the VBL, and no scene branching.
- **Codebook expansion on the 68000.** `prep_dlx.py` does it host-side because
it is a load-time cost and including it would flatter or damn the inner loop.
The player must do it: 8 KB + 2 KB per scene.
- ADPCM audio extraction/encoding
- Disk image packaging
- Game logic (scene branching, input windows, death clips)
@@ -548,6 +643,39 @@ V1's output. To check that snapshot is still pixel-exact:
Not added to `check.sh`: `check.sh` asserts pixel-exactness, and asserting wall
timings there would make the green-light check sensitive to host load.
## Reproducing the decoder result (session 7)
```
python3 tools/encoder/encode.py tmp/fr_singe tmp/rc_fr_singe_sasi_rcprofile.dlx --profile sasi
python3 tools/bench/prep_dlx.py tmp/rc_fr_singe_sasi_rcprofile.dlx
tools/vasm/vasmm68k_mot -Fbin -o tmp/decode.bin src/player/decode.s
mkdir -p tmp/snap_decode && cd tmp && SDL_VIDEODRIVER=dummy timeout -k 5 900 mame x68000 \
-bios ipl10 -ramsize 2M -video soft -window -sound none -nothrottle -plugins \
-autoboot_script ../tools/bench/decode.lua \
-snapshot_directory ./snap_decode -snapview native -seconds_to_run 150
cd .. && python3 tools/bench/verify_decode.py tmp/rc_fr_singe_sasi_rcprofile.dlx
```
~90 s wall. Prints cycles/frame and % of a 12fps budget for four real frames
spanning the non-SKIP distribution, four synthetic single-mode frames, and one
full 120-frame pass; then verifies the last frame is pixel-exact. Expected:
median 73.8%, p90 116.4%, max 135.8%, mean 81.7%; V1 299.9 / V4 448.2 / RAW
400.4 cycles per block.
`-ramsize 2M` matters — MAME defaults to 4M and the locked target is a stock 2MB
machine. `DLX_VERIFY_ONLY=1` drops the timing anchors, which is how `check.sh`
runs it.
Score a container against the measured costs without touching MAME:
```
python3 tools/analysis/11_cpu_budget.py tmp/rc_fr_singe_scsi_rcprofile.dlx
```
And re-demonstrate why there is only one display path (exits non-zero **by
design** — it is the counterexample):
```
python3 tools/analysis/10_pathmix_drift.py # 70/120 frames corrupt
python3 tools/analysis/10_pathmix_drift.py --fix direct # clean, and cheapest
```
## Reproducing the rate-control result (session 6)
```