Move the loader onto the 68000, and find 5,920 bytes nobody counted

src/player/load.i expands both codebooks to word-per-pixel form and packs the
palette to GGGGGRRRRRBBBBBI out of the RAW container header, byte-exact against
tools/bench/dlxload.py on both CPU cores.  The palette half is gated on words
read back out of the palette registers at $E82000, so "the words reached the
hardware" is part of what passes.  ROADMAP P1 is done; P2's encoder half (a
reserved black entry, 23.4) is not, and is a re-encode rather than an edit.

A scene change costs 18.96 ms of 68000 time, 22.8% of one 12 fps frame; boot
costs 24.70 ms.  The scratch tables describe the CRTC, not the scene, so
pal_tables is a separate entry point built once at boot -- 5.29 ms off every
scene change.

The one that moves something: the scene header is 5,920 B that no rate table in
this tree included, because it belongs to no frame record.  In FINDINGS 51.3's
currency it is divided by the surplus pipe - wire, so it is hypersensitive:
138 ms of extra refill climb at 488 KB/s and 1.099 s at 451.4 KB/s, for the
same bytes.  tools/analysis/22_scene_load.py prices it across explicit rates.

Recorded as open: the two CPU cores agree to <3% on every stage but the table
build, where they differ by 16.4%.  px68k's C68K charges a flat 50 clocks for
MULU/MULS (c68kmacro.h:1869) where the 68000 charges 38+2n, which explains
4,608 of the 8,703 clock gap.  4,095 clocks are unexplained.  Nothing else in
src/player/ multiplies, so no figure in FINDINGS 24-52 is affected.

decode.s and stream.s are untouched; decode.bin is still 1,296 B at the same
MD5.  check.sh gains a stage that gates byte-exactness on both cores and
deliberately does not gate the cycle counts -- MAME's clock is 1/55.46 s and a
wall timing would make the green light host-sensitive.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
This commit is contained in:
prosolis
2026-08-24 20:20:40 -07:00
parent ed172c2da2
commit 7179339bd2
13 changed files with 1189 additions and 18 deletions
+141
View File
@@ -4477,3 +4477,144 @@ Relevant to 51's underrun analysis, which models delivery as a smooth rate.
6. **The frame-period accounting assumes the DMAC does not overlap the CPU**
(`buscost.DMA_OVERLAPS = False`), which is FINDINGS 35's premise: no cache,
a two-word prefetch queue. A stolen bus cycle is a stopped 68000.
---
## 53. The loader moves onto the 68000, and a scene change finally has a price (session 21)
ROADMAP P1 and P2. Since session 1 the two load-time transforms have been done
**host-side**, in `tools/bench/dlxload.py`, with the rigs pushing the *result*
into emulated RAM: the codebooks expanded to word-per-pixel form (CB1 to 32 B an
entry, CB4 to 8 B) and the 24-bit palette packed to `GGGGGRRRRRBBBBBI` with the
shared LSB chosen per entry. That was the right call while the inner loop was
what was being measured — charging a once-per-scene cost to the per-frame path
would have flattered or damned it for no reason — but **a player has no host.**
`src/player/load.i` does both on the 68000, out of the raw container header as
it comes off the disc. `tools/bench/loadgate.s` is its front-end, the way
`decode.s` is `frame.i`'s.
**NAME THE LAYER.** Everything here is **emulated**: MAME 0.277 `x68000`,
`-bios ipl10`, stock 10 MHz / 2 MB, cross-checked on px68k's C68K core. Nothing
has run on real hardware.
### 53.1 It reproduces dlxload.py exactly, on both cores
`dlxload.py` stays the reference — what changed is **where the transforms run,
not what they produce** — so the gate is byte-for-byte, not "close enough":
- **CB1 8,192 B, CB4 2,048 B, palette 512 B: identical.** A wrong codebook byte
is a wrong colour in every block that uses that codeword, in every frame of
the scene, and a wrong shared LSB is a *slightly* wrong colour, which is
exactly the sort of defect that gets attributed to the codec.
- **The palette half is read back out of the palette registers at `$E82000`**,
not out of a RAM shadow, so "the words reached the hardware" is part of what
passes.
- **The darkest-entry index agrees too** (255 on the gate container). It comes
out of an `argmin` whose tie-break has to match numpy's — first index at the
minimum wins — and it is what the letterbox is filled with.
- **Both CPU cores produced the same 10,752 bytes**, and the same as the host.
`tools/bench/load_run.sh` runs it and `check.sh` gates it.
### 53.2 What it costs, measured on two cores
| stage | MAME clocks | C68K clocks | Δ | data bus (C68K) |
|---|---|---|---|---|
| scratch tables (boot only) | 52,919 | 61,622 | +16.4% | 6.5% |
| **P1** codebook expansion | 92,609 | 95,304 | +2.9% | **43.2%** |
| **P2** palette entries | 97,019 | 97,348 | +0.4% | 14.3% |
| BOOT: all three | 246,957 | 253,614 | +2.7% | 23.2% |
| **SCENE CHANGE: P1 + P2** | **189,627** | 192,322 | +1.4% | 28.6% |
**A scene change costs 18.96 ms of 68000 time — 22.8% of one 12 fps frame.**
Boot costs 24.70 ms. Bus occupancy is data accesses only (C68K does not see
prefetch), so it is a lower bound; the expansion is the bus-heaviest thing here
because it is a pure copy, and it still runs alone.
**The stages are exactly additive on the exact core.** `P1 + P2 - SCENE = 330`
clocks, and `tables + P1 + P2 - 2x330 = 253,614 = BOOT`, to the clock — 330 is
the front-end's own per-pass overhead. On MAME the same identity closes to 1.8%,
which is one tick of its 1/55.46 s clock over the 0.99 s run. Two instruments,
two granularities, one arithmetic.
### 53.3 The scratch tables are scene-independent, so they are not in the scene path
Packing a palette entry needs the squared error of both choices of the shared
LSB, per channel. That is three table reads and a sign test here, out of three
tables — the 6-bit-to-8-bit rendering the CRTC performs, its square, and the
per-channel error difference — and **not one of them describes the scene.** They
describe the machine. `pal_tables` is therefore a separate entry point from
`pal_pack`, built once at boot: **5.29 ms saved on every scene change**, 22% of
what a naive port of `dlxload.py` would have charged per scene.
### 53.4 The two cores disagree only where the multiplies are, and C68K is wrong in kind
The table build is the only code in this tree that multiplies, and it is the
only stage where the two cores disagree by more than 3%. **px68k's C68K charges
a flat 50 clocks for `MULU` and `MULS`** regardless of the operand
(`c68kmacro.h:1869/1883`, `RET(50 + EA_CLOCKS_...)`); the 68000 charges
**38 + 2n**, n counting bits in the source. For the 576 multiplies this code
executes, the real total is 24,192 clocks against C68K's 28,800: **the flat rate
explains 4,608 of the 8,703 clock gap, and 4,095 clocks — 7.7% of the stage —
are NOT explained.** Recorded as open rather than rounded away; the residual is
somewhere else in the two cycle tables and this stage is not worth the hunt.
**The consequence is general and belongs in the reader's head:** where a future
measurement contains multiplies, C68K over-charges them, and it is the second
opinion this tree leans on for every cycle figure. Nothing else in
`src/player/` multiplies — index scaling is `lsl.w #5`/`#3` by construction —
so no figure in FINDINGS 24-52 is affected.
### 53.5 Where the cost actually lands: the scene change, priced
`tools/analysis/22_scene_load.py`, cycle counts parsed out of the rig's own log
rather than pasted in as constants. Three costs in three units, and **the third
is the one that compounds**:
- **BYTES.** The header region is **5,920 B** (palette 768 + CB1 4,096 + CB4
1,024 + 32) and it must arrive before frame 0 can be decoded. It is not part
of any frame record, so **no rate table in this tree has ever counted it.**
- **CLOCKS.** 189,627, from 53.2.
- **ACCUMULATED SLACK.** Those bytes are bytes the pipe did not spend filling
the ring, so they cost play-time at the surplus rate `pipe - wire` — the
currency FINDINGS 51.3 established a branch point spends.
| pipe KB/s | header ms | + load ms | total | frame slots | surplus KB/s | slack cost |
|---|---|---|---|---|---|---|
| 451.4 | 12.81 | 18.96 | 31.77 | 0.38 | 5.3 | **1.099 s** |
| 488 | 11.85 | 18.96 | 30.81 | 0.37 | 41.9 | 0.138 s |
| 513.2 | 11.27 | 18.96 | 30.23 | 0.36 | 67.1 | 0.086 s |
| 600 | 9.64 | 18.96 | 28.60 | 0.34 | 153.9 | 0.038 s |
(Rates are explicit arguments with no default, FINDINGS 50. `wire` is 446.1
KB/s on the gate container, audio included.)
**Two readings, and the second is the finding.** First: the whole fixed cost of
a scene change is **about a third of one frame slot** — it is not what makes a
branch point expensive, the seek and the refill climb are. Second: **the slack
cost is hypersensitive to the rate**, because it is divided by a surplus that
goes to zero. At 488 KB/s the header lengthens the climb by 138 ms; at 451.4
KB/s — the *arrival-deadline* rate for this same container, 49.5 — the same
5,920 bytes cost **1.1 seconds of play**. The header is cheap only where the
pipe already has room, which is the same place everything else in this project
is cheap.
### 53.6 The alternative that was not taken
The encoder could ship the codebooks pre-expanded and P1 would not exist. That
trades **9.26 ms of 68000 time for 5,120 more bytes in every scene header** —
10.5 ms of pipe at 488 KB/s, and 5,120 bytes that lengthen the climb again by
the arithmetic above. **Derived, not measured**, from the two figures either
side of it. It is close to a wash in milliseconds and it is not a wash in
*kind*: the CPU is idle during a seek and the pipe is the resource this project
is short of. The transform stays on the 68000.
### 53.7 What is still open in P2
**The encoder still does not reserve a black entry** (23.4), so the letterbox
gets the palette's closest thing to black — index 255 here — rather than a true
black with `I = 0`. That half of P2 is encoder-side, it changes the container,
and it moves every constant fitted to the gate container, so it is a re-encode
plus a re-measurement rather than an edit. `load.i` is ready for it: it reads
whatever the palette section holds and reports the darkest index either way.