drwily/Dragon-s-Lair-X68k
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Move the loader onto the 68000, and find 5,920 bytes nobody counted

Browse Source 
src/player/load.i expands both codebooks to word-per-pixel form and packs the
palette to GGGGGRRRRRBBBBBI out of the RAW container header, byte-exact against
tools/bench/dlxload.py on both CPU cores.  The palette half is gated on words
read back out of the palette registers at $E82000, so "the words reached the
hardware" is part of what passes.  ROADMAP P1 is done; P2's encoder half (a
reserved black entry, 23.4) is not, and is a re-encode rather than an edit.

A scene change costs 18.96 ms of 68000 time, 22.8% of one 12 fps frame; boot
costs 24.70 ms.  The scratch tables describe the CRTC, not the scene, so
pal_tables is a separate entry point built once at boot -- 5.29 ms off every
scene change.

The one that moves something: the scene header is 5,920 B that no rate table in
this tree included, because it belongs to no frame record.  In FINDINGS 51.3's
currency it is divided by the surplus pipe - wire, so it is hypersensitive:
138 ms of extra refill climb at 488 KB/s and 1.099 s at 451.4 KB/s, for the
same bytes.  tools/analysis/22_scene_load.py prices it across explicit rates.

Recorded as open: the two CPU cores agree to <3% on every stage but the table
build, where they differ by 16.4%.  px68k's C68K charges a flat 50 clocks for
MULU/MULS (c68kmacro.h:1869) where the 68000 charges 38+2n, which explains
4,608 of the 8,703 clock gap.  4,095 clocks are unexplained.  Nothing else in
src/player/ multiplies, so no figure in FINDINGS 24-52 is affected.

decode.s and stream.s are untouched; decode.bin is still 1,296 B at the same
MD5.  check.sh gains a stage that gates byte-exactness on both cores and
deliberately does not gate the cycle counts -- MAME's clock is 1/55.46 s and a
wall timing would make the green light host-sensitive.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
This commit is contained in:
prosolis
2026-08-24 20:20:40 -07:00
parent ed172c2da2
commit 7179339bd2
13 changed files with 1189 additions and 18 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/FINDINGS.md
+141
View File
@@ -4477,3 +4477,144 @@ Relevant to 51's underrun analysis, which models delivery as a smooth rate.
6. **The frame-period accounting assumes the DMAC does not overlap the CPU**
(`buscost.DMA_OVERLAPS = False`), which is FINDINGS 35's premise: no cache,
a two-word prefetch queue. A stolen bus cycle is a stopped 68000.
---
## 53. The loader moves onto the 68000, and a scene change finally has a price (session 21)
ROADMAP P1 and P2. Since session 1 the two load-time transforms have been done
**host-side**, in `tools/bench/dlxload.py`, with the rigs pushing the *result*
into emulated RAM: the codebooks expanded to word-per-pixel form (CB1 to 32 B an
entry, CB4 to 8 B) and the 24-bit palette packed to `GGGGGRRRRRBBBBBI` with the
shared LSB chosen per entry. That was the right call while the inner loop was
what was being measured — charging a once-per-scene cost to the per-frame path
would have flattered or damned it for no reason — but **a player has no host.**
`src/player/load.i` does both on the 68000, out of the raw container header as
it comes off the disc. `tools/bench/loadgate.s` is its front-end, the way
`decode.s` is `frame.i`'s.
**NAME THE LAYER.** Everything here is **emulated**: MAME 0.277 `x68000`,
`-bios ipl10`, stock 10 MHz / 2 MB, cross-checked on px68k's C68K core. Nothing
has run on real hardware.
### 53.1 It reproduces dlxload.py exactly, on both cores
`dlxload.py` stays the reference — what changed is **where the transforms run,
not what they produce** — so the gate is byte-for-byte, not "close enough":
- **CB1 8,192 B, CB4 2,048 B, palette 512 B: identical.** A wrong codebook byte
is a wrong colour in every block that uses that codeword, in every frame of
the scene, and a wrong shared LSB is a *slightly* wrong colour, which is
exactly the sort of defect that gets attributed to the codec.
- **The palette half is read back out of the palette registers at `$E82000`**,
not out of a RAM shadow, so "the words reached the hardware" is part of what
passes.
- **The darkest-entry index agrees too** (255 on the gate container). It comes
out of an `argmin` whose tie-break has to match numpy's — first index at the
minimum wins — and it is what the letterbox is filled with.
- **Both CPU cores produced the same 10,752 bytes**, and the same as the host.
`tools/bench/load_run.sh` runs it and `check.sh` gates it.
### 53.2 What it costs, measured on two cores
| stage | MAME clocks | C68K clocks | Δ | data bus (C68K) |
|---|---|---|---|---|
| scratch tables (boot only) | 52,919 | 61,622 | +16.4% | 6.5% |
| **P1** codebook expansion | 92,609 | 95,304 | +2.9% | **43.2%** |
| **P2** palette entries | 97,019 | 97,348 | +0.4% | 14.3% |
| BOOT: all three | 246,957 | 253,614 | +2.7% | 23.2% |
| **SCENE CHANGE: P1 + P2** | **189,627** | 192,322 | +1.4% | 28.6% |
**A scene change costs 18.96 ms of 68000 time — 22.8% of one 12 fps frame.**
Boot costs 24.70 ms. Bus occupancy is data accesses only (C68K does not see
prefetch), so it is a lower bound; the expansion is the bus-heaviest thing here
because it is a pure copy, and it still runs alone.
**The stages are exactly additive on the exact core.** `P1 + P2 - SCENE = 330`
clocks, and `tables + P1 + P2 - 2x330 = 253,614 = BOOT`, to the clock — 330 is
the front-end's own per-pass overhead. On MAME the same identity closes to 1.8%,
which is one tick of its 1/55.46 s clock over the 0.99 s run. Two instruments,
two granularities, one arithmetic.
### 53.3 The scratch tables are scene-independent, so they are not in the scene path
Packing a palette entry needs the squared error of both choices of the shared
LSB, per channel. That is three table reads and a sign test here, out of three
tables — the 6-bit-to-8-bit rendering the CRTC performs, its square, and the
per-channel error difference — and **not one of them describes the scene.** They
describe the machine. `pal_tables` is therefore a separate entry point from
`pal_pack`, built once at boot: **5.29 ms saved on every scene change**, 22% of
what a naive port of `dlxload.py` would have charged per scene.
### 53.4 The two cores disagree only where the multiplies are, and C68K is wrong in kind
The table build is the only code in this tree that multiplies, and it is the
only stage where the two cores disagree by more than 3%. **px68k's C68K charges
a flat 50 clocks for `MULU` and `MULS`** regardless of the operand
(`c68kmacro.h:1869/1883`, `RET(50 + EA_CLOCKS_...)`); the 68000 charges
**38 + 2n**, n counting bits in the source. For the 576 multiplies this code
executes, the real total is 24,192 clocks against C68K's 28,800: **the flat rate
explains 4,608 of the 8,703 clock gap, and 4,095 clocks — 7.7% of the stage —
are NOT explained.** Recorded as open rather than rounded away; the residual is
somewhere else in the two cycle tables and this stage is not worth the hunt.
**The consequence is general and belongs in the reader's head:** where a future
measurement contains multiplies, C68K over-charges them, and it is the second
opinion this tree leans on for every cycle figure. Nothing else in
`src/player/` multiplies — index scaling is `lsl.w #5`/`#3` by construction —
so no figure in FINDINGS 24-52 is affected.
### 53.5 Where the cost actually lands: the scene change, priced
`tools/analysis/22_scene_load.py`, cycle counts parsed out of the rig's own log
rather than pasted in as constants. Three costs in three units, and **the third
is the one that compounds**:
- **BYTES.** The header region is **5,920 B** (palette 768 + CB1 4,096 + CB4
1,024 + 32) and it must arrive before frame 0 can be decoded. It is not part
of any frame record, so **no rate table in this tree has ever counted it.**
- **CLOCKS.** 189,627, from 53.2.
- **ACCUMULATED SLACK.** Those bytes are bytes the pipe did not spend filling
the ring, so they cost play-time at the surplus rate `pipe - wire` — the
currency FINDINGS 51.3 established a branch point spends.
| pipe KB/s | header ms | + load ms | total | frame slots | surplus KB/s | slack cost |
|---|---|---|---|---|---|---|
| 451.4 | 12.81 | 18.96 | 31.77 | 0.38 | 5.3 | **1.099 s** |
| 488 | 11.85 | 18.96 | 30.81 | 0.37 | 41.9 | 0.138 s |
| 513.2 | 11.27 | 18.96 | 30.23 | 0.36 | 67.1 | 0.086 s |
| 600 | 9.64 | 18.96 | 28.60 | 0.34 | 153.9 | 0.038 s |
(Rates are explicit arguments with no default, FINDINGS 50. `wire` is 446.1
KB/s on the gate container, audio included.)
**Two readings, and the second is the finding.** First: the whole fixed cost of
a scene change is **about a third of one frame slot** — it is not what makes a
branch point expensive, the seek and the refill climb are. Second: **the slack
cost is hypersensitive to the rate**, because it is divided by a surplus that
goes to zero. At 488 KB/s the header lengthens the climb by 138 ms; at 451.4
KB/s — the *arrival-deadline* rate for this same container, 49.5 — the same
5,920 bytes cost **1.1 seconds of play**. The header is cheap only where the
pipe already has room, which is the same place everything else in this project
is cheap.
### 53.6 The alternative that was not taken
The encoder could ship the codebooks pre-expanded and P1 would not exist. That
trades **9.26 ms of 68000 time for 5,120 more bytes in every scene header** —
10.5 ms of pipe at 488 KB/s, and 5,120 bytes that lengthen the climb again by
the arithmetic above. **Derived, not measured**, from the two figures either
side of it. It is close to a wash in milliseconds and it is not a wash in
*kind*: the CPU is idle during a seek and the pipe is the resource this project
is short of. The transform stays on the 68000.
### 53.7 What is still open in P2
**The encoder still does not reserve a black entry** (23.4), so the letterbox
gets the palette's closest thing to black — index 255 here — rather than a true
black with `I = 0`. That half of P2 is encoder-side, it changes the container,
and it moves every constant fitted to the gate container, so it is a re-encode
plus a re-measurement rather than an edit. `load.i` is ready for it: it reads
whatever the palette section holds and reports the darkest index either way.
docs/ROADMAP.md
+27 -11
View File
@@ -1,6 +1,7 @@
# Roadmap — remaining work to a completion target
Written end of session 19 (2026-08-24), against a tree that is ALL GREEN.
Amended end of session 21: P1 done, P2 half done (FINDINGS 53).
**THE COMPLETION TARGET IS M3, THE VERTICAL SLICE** (USER DECISION): one scene
tree — a decision point, two outcomes, a death clip — with audio, streaming from
@@ -96,17 +97,32 @@ out of a bounded ring fed by a host file on a paced clock. Neither is a player.
**Exit criterion: boots from a real SCSI volume on a stock 2 MB X68000, plays
one scene at 12 fps from disc, no host-file pipe, no Lua in the loop. Silent.**
**P1. Codebook expansion on the 68000.** `dlxload.py:19` expands CB1 to 32 B per
entry and CB4 to 8 B, host-side, because at the time it was a load-time cost that
would have flattered or damned the inner loop. The player must do it: **8 KB +
2 KB per scene**. Note where that lands — *at a scene change, when the ring is
empty because of the seek*. It compounds with 51.3 and should be priced against
the refill climb, not treated as free setup.
~~**P1. Codebook expansion on the 68000.**~~ **DONE, session 21 — FINDINGS 53.**
`src/player/load.i` expands both codebooks out of the raw container header,
byte-exact against `dlxload.py` on both CPU cores. **9.26 ms**, and it was
priced where it lands rather than treated as free setup: the scene header is
**5,920 B that no rate table in this tree counted**, and in the currency of
51.3 — accumulated slack — those bytes lengthen the refill climb by 138 ms at
488 KB/s and by **1.099 s at 451.4 KB/s**, because the surplus they are divided
by goes to zero. The whole fixed cost of a scene change is about a third of one
frame slot; what makes a branch point expensive is still the seek and the climb.
Shipping the codebooks pre-expanded was considered and refused: it trades
9.26 ms of CPU for 5,120 more header bytes, which is a wash in milliseconds and
not a wash in kind (53.6).
**P2. Palette packing on the 68000.** The encoder still emits RGB888; the X68000
word packing is Lua-side. Whatever writes real palette words must pick `I` per
entry by minimum squared error (**1.96 dB**, FINDINGS 23.3) and reserve index 0
as black with `I = 0` (23.4).
**P2. Palette packing on the 68000. HALF DONE, session 21 — FINDINGS 53.**
~~The encoder still emits RGB888; the X68000 word packing is Lua-side.~~ The
packing is on the 68000: `pal_pack` writes 256 words straight into `$E82000`
with `I` chosen per entry by minimum squared error (**1.96 dB**, 23.3), gated on
the words read back **out of the palette registers**. 9.70 ms per scene, plus
5.29 ms of scene-independent table build hoisted to boot (53.3).
**What is left is the other half of the sentence: reserve index 0 as black with
`I = 0` (23.4).** That is ENCODER-side, it changes the container, and it moves
every constant fitted to the gate container, so it is a re-encode plus a
re-measurement rather than an edit. Until then the letterbox gets the palette's
closest thing to black (index 255 on the gate container); `load.i` reports
whichever index that is and needs no change when it becomes 0.
**P3. A real frame clock.** `stream.s` has `PACE`/`PACEON` (`$18034`/`$18038`)
but the 12 fps tick comes from the Lua producer. Needs MFP timer or VBL. Keep
@@ -229,7 +245,7 @@ Listed for completeness; past M3 these are scope, not risk.
B1 seek+rate ─┐
B3 DTYP ──────┴─> P4 transport ─┐
├─> M2 ─> M3 (COMPLETION TARGET) ─> M4
P1 P2 P3 P5 P7 ─────────────────┘ ^
P1 P2(half) P3 P5 P7 ───────────┘ ^
│
P6 (bus cost DONE, 52) ──────────────────┤
G1 scene graph (fetch, do early) ─────────┘
docs/STATUS.md
+90
View File
@@ -1,3 +1,93 @@
# Status & next-session handoff — end of session 21 (2026-08-24)
## Session 21: the loader moves onto the 68000, and a scene change gets a price
**Green light first and last: `./tools/bench/check.sh` was ALL GREEN before any
of this and ALL GREEN after**, 120/120 on both cores, no `TRUNCATED`, plus a new
load-time stage.
**ROADMAP P1 is DONE and P2 is half done. FINDINGS 53.** Session 20's handoff
named P4 as the item that decides the project, and P4 is **blocked in this
tree** — re-checked, not assumed: there is still no `scsiexrom.bin` anywhere on
this machine (`~/mame/roms/x68000.zip` holds six files, four IPLs, a cgrom and
an sram), MAME's `x68000` has no MB89352 path, and `hd63450.cpp` decodes no
DTYP. **Nothing here can measure W.** P1+P2 was the M2 item that could be built
here, and it is the one that touches an already-measured number: it lands at a
scene change, where FINDINGS 51.3's refill climb is.
**1. The transforms are on the 68000 and they are byte-exact.** `src/player/
load.i` expands both codebooks to word-per-pixel form and packs the palette to
`GGGGGRRRRRBBBBBI` with the shared LSB chosen per entry, out of the RAW
container header. Gated **byte-for-byte against `tools/bench/dlxload.py`**,
which stays the reference — what changed is where the transforms run, not what
they produce. The palette half is read back **out of the palette registers at
`$E82000`**, so "the words reached the hardware" is part of what passes. Both
CPU cores emit the same 10,752 B. 53.1.
**2. A scene change costs 18.96 ms of 68000 time, 22.8% of one 12 fps frame.**
Boot costs 24.70 ms. Split: codebooks 92,609 clocks, palette entries 97,019,
scratch tables 52,919. Cross-checked on px68k's C68K, which agrees to 1.4% on
the scene-change figure. 53.2.
**3. The scratch tables describe the machine, not the scene.** They are the
CRTC's 6-to-8-bit rendering, its square, and the per-channel error difference —
so `pal_tables` is a separate entry point, built once at boot. **5.29 ms off
every scene change**, 22% of what a naive port of `dlxload.py` would have cost
per scene. 53.3.
**4. THE ONE THAT MOVES SOMETHING: the scene header is 5,920 bytes nothing has
ever counted.** Palette + CB1 + CB4, and it must arrive before frame 0 can be
decoded. It is not part of any frame record, so no rate table in this tree
includes it. `tools/analysis/22_scene_load.py` prices it across explicit rates.
The whole fixed cost of a scene change is about **a third of one frame slot** —
but its cost in FINDINGS 51.3's currency, accumulated slack, is divided by the
surplus `pipe - wire` and so is hypersensitive: **138 ms of extra climb at 488
KB/s, and 1.099 s at 451.4 KB/s**, for the same 5,920 bytes. 53.5.
**5. An instrument disagreement worth carrying forward.** The two CPU cores
agree to <3% on every stage except the table build, where they differ by 16.4%
— and that is the only code in this tree that multiplies. **px68k's C68K charges
a flat 50 clocks for `MULU`/`MULS`** regardless of operand (`c68kmacro.h:1869`),
where the 68000 charges 38+2n. That explains 4,608 of the 8,703 clock gap;
**4,095 clocks are not explained and are recorded as open.** Nothing in
`src/player/` outside these three instructions multiplies (checked), so no
figure in FINDINGS 24-52 is affected — but the second opinion this tree leans on
over-charges multiplies, and a future measurement containing one must not be
taken from it uncorrected. 53.4.
**6. Shipping pre-expanded codebooks was considered and refused.** It would
trade 9.26 ms of 68000 time for 5,120 more bytes in every scene header — 10.5 ms
of pipe at 488 KB/s, and bytes that lengthen the climb. Close to a wash in
milliseconds, not a wash in kind: the CPU is idle during a seek and the pipe is
what this project is short of. **Derived, not measured.** 53.6.
**New in the tree:** `src/player/load.i` (the transforms) and
`src/player/loadgate.s` (its front-end, 488 B); `tools/bench/prep_load.py`,
`load.lua`, `verify_load.py`, `load_run.sh` (the rig, both cores);
`tools/analysis/22_scene_load.py` (the pricing). `tools/bench/c68k/harness.c`
gains a `--loadraw` mode, which also makes its flag-watch address a variable
rather than a constant. `check.sh` gains a stage that gates byte-exactness on
both cores, and deliberately does **not** gate the cycle counts — MAME's clock
is 1/55.46 s and a wall timing would make the green light host-sensitive, the
same reason `blit.s` and `span.sh` are not in it.
**`decode.s` and `stream.s` are unchanged.** Nothing in the per-frame path was
touched; `decode.bin` is still 1,296 B at the same MD5.
**Still open in P2:** the encoder does not reserve a black entry (23.4), so the
letterbox still gets the palette's closest thing to black (index 255 here). That
half is encoder-side, it changes the container, and it moves every constant
fitted to the gate container — a re-encode plus a re-measurement, not an edit.
**Next:** P3 (a real frame clock from the MFP or VBL) and P5 (per-record index,
prefill policy, the accumulated-slack rule in the player rather than the rig)
are both buildable here. G1 (import the scene graph) is the one that would let
this tree ask a question it currently cannot: what is the worst gap between
consecutive decision points, and does the refill climb survive it. P4 still
decides the project and still cannot be measured here.
---
# Status & next-session handoff — end of session 20 (2026-08-24)
## Session 20: the DMAC configuration was in the IPL ROM the whole time