drwily/Dragon-s-Lair-X68k
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Build v7 into the player, and find the cost model 18% wrong on the block it made commonest

Browse Source 
src/player/decode.s now paints v7 literal spans, pixel-exact under MAME and
px68k's C68K core over a container where every frame carries 128-216 spans
covering up to 38% of the picture. The span pass is blit.s v7 verbatim: the
66.0/9.143/9.978 fit was measured on that instruction sequence.

The container is DLX3 -- a span section between the mode header and the block
payload, since that is the only place the 68000 can reach without first parsing
something of variable length. 16_span_roundtrip.py gates it in check.sh, and
asserts it emitted enough spans to have tested anything.

Two synthetic all-SPAN anchors price v7 inside decode.s at 151.2 and 225.6
clocks per 4x4 block, against FINDINGS 40's table of 151 and 226 -- 0.2% on
both emulators. The measured mode costs what it was said to cost.

Two things that were not on the list:

TWO BYTE BUDGETS. FINDINGS 40's 18/120 was scored against the 488 KB/s PIPE,
not the 280 KB/s profile, and at the profile rate the lam search has already
spent the allowance -- spans fired on 5 frames of 120 and looked like a
regression. The profile is a chosen quality rate point; the pipe is hardware.
--kbps and --span-kbps are now separate and spans run before mu, because a span
pays in bytes and mu pays in picture. Delivered: 86/120 over budget without
spans, 77/120 at the profile budget, 34/120 on the pipe for +0.36 dB.

C_SKIP_MIXED WAS NEVER MEASURED, and it was 18% low -- 45.0, now 55.0. It is
the one constant in the table that came from a derivation, because the
synthetic frame that would measure it cannot exist: a byte needs a coded block
for its SKIP to be mixed. Four bracketing anchors measure it on both emulators
with the header byte rotated through all four positions, and the partner mode
solves back to its own anchored value to 0.2%. With it corrected the model
predicts a real spanned decode to -0.06% mean / 0.09% worst, against -2.99% /
4.30%. It matters because a span marks its run SKIP, so mixed SKIPs dominate
exactly the frames spans are judged on.

Also: the rig had been writing its synthetic timing frames 26 KB past the top
of a 2 MB machine, and got away with it because the modes it overran are
data-independent. A span's jump displacements come out of the stream, so it is
not. And frames-over-budget is no longer a safe headline -- the controller aims
at the deadline, so 55 of 120 frames sit within 5% of it and a 1% cost shift
moves 22 frames.

FINDINGS 41. check.sh ALL GREEN, now gating on a span-heavy DLX3 container.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
This commit is contained in:
prosolis
2026-08-23 20:02:03 -07:00
parent c520a89e14
commit b49bbdc939
16 changed files with 1342 additions and 203 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/FINDINGS.md
+163
View File
@@ -2532,3 +2532,166 @@ fine**. `span.sh` now runs at `-seconds_to_run 200`, measured at 30 s wall for
all 36 configs, and asserts the snapshot count against the number of configs in
the generated metadata rather than a literal 23 -- so adding a config can no
longer silently weaken the pixel-exactness gate.
## 41. v7 is in the player, and the model it is scored by was 18% wrong (session 12)
FINDINGS 40 measured v7 in `tools/bench/blit.s` and left it there. This builds it
into `src/player/decode.s`, defines the container that carries it, and scores
what the encoder actually delivers rather than what a selection model predicts.
Three things came out of it that were not on the list.
### 41.1 The decoder, and the format
`src/player/decode.s` gains `paint_spans`, which is `blit.s` v7 verbatim -- the
same instruction sequence, deliberately, because the 66.0/9.143/9.978 fit was
measured on that sequence and a tidier rewrite would silently invalidate it.
The container is **DLX3**: the span section sits between the 768-byte mode
header and the block payload, because that is the only place the 68000 can
reach without first parsing something of variable length.
```
u32 payload length
768 B mode header spanned blocks read SKIP
u16 nspans
nspans * { u32 GVRAM address, u16 coarse disp, c*48 B,
u16 fine disp, f*4 B }
block payload V1 -> 1 B, V4 -> 4 B, RAW -> 16 B
```
Every span record is a multiple of 4 bytes (4+2+48c+2+4f), so the section needs
no internal padding and the block payload starts aligned. `a1`, the mode-header
cursor, is one of v7's twelve payload registers, so it goes on the stack across
the pass: two long accesses a frame, against the 24 pixels a register buys per
chain unit.
**Pixel-exact under both CPU cores on the first run**, over a container where
every frame carries 128-216 spans painting up to 38% of the picture, with full
temporal recursion. `tools/analysis/16_span_roundtrip.py` is the new gate and it
is in `check.sh`: encode, write the container, read it back with the reference
decoder, compare to what the encoder recorded. It asserts it emitted enough
spans to have tested anything -- a round-trip over a span-less container is
green by vacuity, which is FINDINGS 40.6's lesson about the snapshot count.
### 41.2 There are TWO byte budgets, and conflating them hides the whole win
The first measured span encode looked like a regression: at the `scsi` profile
spans fired on 5 of 120 frames and bought almost nothing. The cause is not the
codec. **The lam search had already spent the byte allowance**, so the span pass
inherited a few hundred bytes of room.
FINDINGS 40's 18/120 was never scored at 280 KB/s. `14_dmac_chain.py` defaults
to `--bus 488` -- the PIPE -- and gives each frame 40,977 bytes. The profile's
is 23,228. **Those are two different budgets and only one of them is hardware.**
The profile is a chosen quality rate point; the pipe is a ceiling. Bytes between
the two buy a better picture if spent on `lam`, the 68000's deadline if spent on
spans, and nothing at all if left unspent.
So the encoder now takes both: `--kbps` sets the quality target and
`--span-kbps` the ceiling the span pass may draw on, flat per frame and not
banked, because a pipe cannot be saved up. The quality bucket is credited with
the BLOCK payload only -- charging it the span bytes drives it to its floor on
the first spanned frame and starves every later frame of quality for a budget
the spans were never drawing on.
Spans also run BEFORE `mu`, and that ordering is the point. Both controllers
make a frame decode in time; `mu` pays in quality and a span pays in bytes, and
a span carries literal source pixels so it *removes* that run's quantisation
error. Spending bytes we already have beats spending picture.
| 120-frame `scsi` window | KB/s | over budget | PSNR |
|---|---:|---:|---:|
| no spans | 278.3 | 86/120 | 29.27 dB |
| spans, profile budget only | 280.0 | 77/120 | 29.23 dB |
| **spans on the 488 KB/s pipe** | 487.7 | **34/120** | **29.63 dB** |
Scored with `17_span_delivered.py`, which reads the emitted span section and
prices exactly those spans -- no selection model at all -- in 14's additive
model: block decode + span painting + disk DMA.
### 41.3 The blit.s fit transfers into the player, to 0.2%
`prep_dlx.py` gained two synthetic all-SPAN frames (full-row runs, and 4-block
runs at the break-even). They price v7 inside `decode.s` against the constants
`span.sh` fitted in `blit.s`:
| | predicted | MAME | C68K | error |
|---|---:|---:|---:|---:|
| all-SPAN-64 (192 spans x 256 px) | 505,636 | 506,533 | 506,824 | +0.18% |
| all-SPAN-4 (3072 spans x 16 px) | 734,193 | 735,133 | 735,304 | +0.13% |
Per 4x4 block that is **151.2 and 225.6 clocks, against FINDINGS 40's table of
151 and 226**. The mode costs what it was said to cost, in the real decoder, on
two emulators.
### 41.4 The rig had been writing past the top of RAM
`prep_dlx.py` truncated the real frames to a RAM budget and then appended its
synthetic timing frames ON TOP, 26 KB past the 0x200000 top of a 2 MB machine.
Survivable while it lasted, because the modes it overran are data-independent:
reading junk payload costs a V1 or a RAW block exactly what reading pixels
costs, so the anchors timed correctly by luck.
**A span is not data-independent.** Its two jump displacements come out of the
stream, so an out-of-RAM span record jumps into open bus. The synthetic frames
are now built first and their size comes out of the budget, with an assertion
that the stream ends below the top of RAM.
### 41.5 C_SKIP_MIXED was never measured, and it was 18% low
Chasing a 3% gap between the model and the measured decode turned up the one
constant in `vq_hybrid`'s cost table that came from a derivation rather than a
measurement: **the cost of a SKIP block sharing its header byte with a coded
block.** It was 45.0 from session 7 to session 12. It is **55.0**.
Every other constant comes from a synthetic frame of a single mode, and there
was no such frame for a mixed SKIP, *because one cannot exist* -- the byte has
to hold a coded block for the SKIP to be mixed at all. So `prep_dlx.py` now
emits four frames that bracket it, (3 SKIP + 1 V1), (1 SKIP + 3 V1), and the
same pair with RAW, each pair solving for the SKIP cost and its partner's
together:
| | MAME | C68K |
|---|---:|---:|
| mixed SKIP, from the V1 pair | 55.03 | 56.50 |
| mixed SKIP, from the RAW pair | 55.83 | 56.50 |
| V1, solved back out | 300.66 | 300.50 |
The partner solves back to its own anchored value to 0.2%, which is what says
the pair is measuring the SKIP rather than absorbing it. 55.0 is taken because
every other constant in the table is MAME's.
**The header bytes ROTATE through all four positions, and that is load-bearing.**
`decode.s` reaches a block's mode bits with `lsr.b #6/#4/#2` and no shift at all
for the last one, so a block costs 52/48/44/34 clocks of dispatch depending on
where in its byte it sits. A fixed pattern like 0x01 pins every SKIP to the
three expensive slots and every V1 to the free one, and solving two such
equations returns a number that describes no real frame. The first attempt did
exactly that, and a single-parameter fit against real frames then "confirmed"
87.7 -- collinear with the span term, and wrong.
With the constant corrected the model predicts the measured decode of a real
spanned container to **-0.06% on the mean and 0.09% worst frame**, against
-2.99% and 4.30% as it stood.
It matters more than 10 clocks a block sounds, because **a span marks its run
SKIP**: a spanned container is made largely of mixed SKIPs, so this is the
dominant population in exactly the frames spans are judged on. It is imported
now, not copied, in `spans.py` and `14_dmac_chain.py`.
*Incidental, and it resolved a false lead:* RAW and V4 read 3.2-3.5% higher on
C68K than on MAME, on mixed and pure frames alike. That is FINDINGS 37's known
table spread, not a property of mixed bytes -- but comparing a C68K-derived
solve against a MAME-derived anchor made it look like one for an hour.
### 41.6 Frames-over-budget is not a safe headline any more
The delivered 34/120 against 14's simulated 18/120 is a **1.4% difference in
mean frame cost** (786,381 clocks against 774,356). The metric is that
sensitive because the rate controller *aims* at the deadline: 55 of 120 frames
land within 5% of it, and shifting every frame by 1% moves the count from 25 to
47.
| every frame shifted by | -3% | -2% | -1% | 0 | +1% | +2% | +3% |
|---|---:|---:|---:|---:|---:|---:|---:|
| frames over budget | 18 | 21 | 25 | **31** | 47 | 61 | 63 |
This was a fair metric when nothing controlled to the budget. It is now a
measurement of where the controller aims, and any cost-model error is amplified
into a large count change -- which is how 41.5's 18% error stayed invisible.
**Report the cost distribution; quote the count only with its sensitivity.**
Add this to the §4 measurement traps.
docs/STATUS.md
+81 -80
View File
@@ -1,107 +1,108 @@
# Status & next-session handoff — end of session 11 (2026-08-23)
# Status & next-session handoff — end of session 12 (2026-08-23)
## Where this stands
Session 11 measured the one item session 10 left at the top of the list, and it
paid: **`blit.s` v7, the literal span with a fine tail, is MEASURED and takes
the `scsi` window from 84/120 frames over budget to 18/120.** FINDINGS 40.
Session 12 built v7 into the player. **`src/player/decode.s` paints v7 literal
spans, and it is pixel-exact under both CPU cores** over a container where every
frame carries 128-216 spans covering up to 38% of the picture. FINDINGS 41.
```
v7: cycles = 66.0 per span + 9.143 per COARSE pixel + 9.978 per FINE pixel
13 span lengths, all fitted to within 0.2%, all pixel-exact
```
The container is **DLX3**: a span section between the mode header and the block
payload, `{u32 GVRAM address, u16 coarse disp}` per span with the fine
displacement mid-stream. `tools/analysis/16_span_roundtrip.py` gates it and is
in `check.sh`.
| | frames over budget, 120-frame `scsi` window |
|---|---:|
| today (no spans) | 84/120 |
| v6 span as built | 55/120 |
| **v7, measured** | **18/120** |
| DMAC chain (datasheet) | 12/120 |
**The measured cost transfers.** Two synthetic all-SPAN anchors price v7 inside
`decode.s` at **151.2 and 225.6 clocks per 4x4 block**, against FINDINGS 40's
table of 151 and 226 — 0.2% on both emulators.
**The DMAC stays dropped, and now on a measurement rather than an argument.**
v7 takes back 37 of the 43 frames the DMAC chain would, with no reserved
channel, no two-region container, and no transfer timing neither emulator here
can verify. FINDINGS 39.1 still holds if that ever changes: a chain array entry
and a v6/v7 span record are the same six bytes.
### The two things that were not on the list
**Break-even against all-V1 moves from L=4 blocks to L=2.** 39.4 predicted L=3.
**1. There are TWO byte budgets, and FINDINGS 40's 18/120 was scored at the
wrong one.** The `scsi` profile is 280 KB/s; `14_dmac_chain.py` scores spans
against the 488 KB/s PIPE, which is 40,977 B/frame against 23,228. At the
profile rate the lam search has already spent the allowance and spans fire on 5
frames of 120. The profile is a chosen quality rate point; the pipe is hardware.
`--kbps` and `--span-kbps` are now separate, and spans run before `mu` because a
span pays in bytes and `mu` pays in picture. FINDINGS 41.2.
**18/120 is exactly what 39.4 derived, and that is a coincidence of two
cancelling errors** — worth knowing before the next derived figure gets trusted
for landing on its measurement. 39.4 assumed a 2-register `movem` tail at 14.0
clocks/pixel (the real tail is 9.978, 29% cheaper) and assumed the second chain
entry costs nothing per span (it costs 22.3 clocks). The two nearly cancel over
this window. FINDINGS 40.2.
| 120-frame `scsi` window | KB/s | over budget | PSNR |
|---|---:|---:|---:|
| no spans | 278.3 | 86/120 | 29.27 dB |
| spans, profile budget only | 280.0 | 77/120 | 29.23 dB |
| **spans on the 488 KB/s pipe** | 487.7 | **34/120** | **29.63 dB** |
**The tail instruction the derivation should have picked is `move.l (a0)+,(a2)+`.**
A 2-register `movem` pays two instruction words and a `lea` to move what two
post-incrementing `move.l`s move: 14 bus cycles against 10 for the same 4
pixels. Taking the plain instruction also makes the padding quantum **2 pixels**
instead of 4 — and a span is a run of 4x4 blocks, so **its padding is exactly
zero**. FINDINGS 40.3.
**2. `C_SKIP_MIXED` was never measured, and it was 18% low — 45.0, now 55.0.**
It is the one constant in the cost table that came from a derivation, because
the synthetic frame that would measure it cannot exist (a byte needs a coded
block for its SKIP to be mixed). Four new bracketing anchors measure it on both
emulators, and with it corrected the model predicts a real spanned decode to
**-0.06% mean / 0.09% worst**, against -2.99% / 4.30% before. It matters here
because **a span marks its run SKIP**, so mixed SKIPs are the dominant
population in exactly the frames spans are judged on. FINDINGS 41.5.
**The fine displacement lives in the STREAM, not in the span record**, after the
coarse pixels and before the fine ones: the coarse chain falls out into
`move.w (a0)+,d0 / jmp`, where `d0` is dead payload and `a0` already points at
it. That is what lets v7 keep all 12 payload registers, which is the entire
reason v6's unit is 24 pixels. The container costs 2 more bytes a span.
FINDINGS 40.4.
**And the metric everything has been quoted in is unstable.** 34/120 delivered
against 14's simulated 18/120 is a 1.4% difference in mean frame cost. 55 of 120
frames sit within 5% of the deadline because the rate controller aims there, so
a 1% cost shift moves 22 frames. Quote the distribution, not the count.
FINDINGS 41.6.
**One process note.** `span.sh` ran 13 minutes producing an empty log and zero
snapshots; the same command with a shorter `-seconds_to_run` did the identical
work in 30 s, and the wedge never reproduced. The cause is unidentified. What
resolved it was not chasing the hang but **shrinking the stimulus**:
`tmp/spans_meta.lua` holds byte offsets into a blob `prep_spans.py` writes once,
so deleting lines from the metadata runs any subset in seconds against the same
stream file. Keep that trick. `span.sh` is now `-seconds_to_run 200` (30 s wall
for all 36 configs) and takes its expected snapshot count from the metadata
instead of a literal 23. FINDINGS 40.6.
Green light: `./tools/bench/check.sh` **ALL GREEN** at the end of this session.
Green light: `./tools/bench/check.sh` **ALL GREEN**, now gating on a span-heavy
DLX3 container.
## NEXT SESSION, in order
0. **Green light first.** `./tools/bench/check.sh` (~5 min, Blu-ray mounted).
Verified green at end of session 11.
0. **Green light first.** `./tools/bench/check.sh` (~6 min, Blu-ray mounted).
Verified green at end of session 12. The gate container is now
`tmp/rc_fr_singe_scsi_span.dlx` (scsi modes, spans on the full pipe) and the
rig fits 37 of 120 frames in a 2 MB machine.
1. **Build v7 into `src/player/decode.s`.** This is now the largest thing
standing between the measured decoder and the budget: 84/120 to 18/120, the
format is fully specified (FINDINGS 30.2, costs 40.1), the executor is
written and pixel-exact in `blit.s`, and the encoder side is
`prep_spans.py`'s v7 emitter. The container is `{u32 absolute GVRAM address,
u16 coarse displacement}` per span plus one `u16` fine displacement carried
mid-stream — see FINDINGS 40.4 before changing that layout, the register
pressure is the reason for it.
1. **Decide the rate point, because the span result now depends on it.** This is
the user's call and it is the first real fork since the profile was set:
spans only pay if the stream is allowed to run near the pipe (487.7 KB/s
delivered, 34/120 over budget) rather than at the 280 KB/s profile (77/120).
That is a delivery-medium question — FINDINGS 32 dropped SASI on capacity and
parked the 110 KB/s point for CD-ROM, and 488 KB/s is 93% of a 4 Mbps figure
whose provenance is still unconfirmed (FINDINGS 29.5 item 3). **Do not spend
another session optimising against a budget nobody has chosen.**
2. **Then re-run `14_dmac_chain.py` and `13_cpu_ratectl.py` against a container
the encoder actually emits with spans in it.** Every span figure so far is
scored against mode maps chosen without spans available, which FINDINGS 39.3
flags as a lower bound on what a span-aware encoder would find.
2. **Re-derive the span selection jointly with lam, not after it.** The encoder
picks modes at one budget and then spans what is left, which FINDINGS 39.3
already called a lower bound. A frame that misses its deadline would often do
better raising lam to free room for spans than lowering it — spans are
pixel-exact, so the quality trade is not what it looks like. Bisecting a
span reserve fraction inside the existing search is the tractable version.
3. **Make sure the player actually gets DMA** — unchanged from session 10, and
still not an optimisation. Benchmark `x68000 -exp1 cz6bs1`, **never
`x68ksupr`**; MAME's internal SCSI has no DMA glue (`// TODO: duplicate DMA
glue from CZ-6BS1`) and would measure a PIO fallback the real machine does
not have.
3. **Re-run `13_cpu_ratectl.py` against a DLX3 container.** 14 and 15 are done
(15 now counts span bus traffic and still reproduces the C68K measurement to
0.04%); 13 has not been re-run since the constant changed.
4. **Re-decide the framerate.** 10 fps absorbs the DMA steal on current
estimates. Still the user's call, and now cheaper to defer: v7 buys back
enough of the budget that 12 fps is no longer obviously out of reach.
4. **Make sure the player actually gets DMA** — unchanged from sessions 10-11,
and still not an optimisation. Benchmark `x68000 -exp1 cz6bs1`, **never
`x68ksupr`**; MAME's internal SCSI has no DMA glue and would measure a PIO
fallback the real machine does not have. This is now more load-bearing, not
less: the delivered stream is 487.7 KB/s and the disk debit is 163,798
clocks a frame, 20% of the budget.
5. **Re-run the ring-buffer simulation at the surviving rate** and confirm the
488 KB/s figure's provenance (FINDINGS 29.5/30.7, still open).
5. **Re-run the ring-buffer simulation at the surviving rate** (FINDINGS
29.5/30.7, still open) and confirm the 488 KB/s figure's provenance.
**Do not start by hand-optimising `decode.s`.** Unchanged and still true: the
hand-derived timings agree with the measurements to 0.5% on V1 and 1% on RAW
(FINDINGS 28.4), 34 confirms the model on a second container, and the cycles to
be won are in the budget, not the loop.
**Do not start by hand-optimising `decode.s`.** Unchanged and still true. The
cycles to be won are in the budget, not the loop — and session 12 is the second
demonstration that the *model* of the budget is where the errors live.
**Always `stdbuf -oL` a MAME job that prints progress — and do not trust it.**
Session 11 added the case where even that is not enough (40.6). If a run is not
producing observable output, shrink the stimulus rather than waiting.
**A new trap, worth reading before quoting any figure:** the rig had been
writing its synthetic timing frames 26 KB past the top of a 2 MB machine, and
got away with it because the modes it overran are data-independent. A span is
not — its jump displacements come out of the stream. FINDINGS 41.4.
---
## What session 11 settled
Session 11 measured v7 in `blit.s` and left it there; session 12 built it into
the player. Items 0 and 1 of session 11's list are done (FINDINGS 40, 41) and
the rest are carried forward in the list above.
## What session 10 settled
Session 10 cross-checked the whole cycle model against a second emulator, then