Move the loader onto the 68000, and find 5,920 bytes nobody counted
src/player/load.i expands both codebooks to word-per-pixel form and packs the palette to GGGGGRRRRRBBBBBI out of the RAW container header, byte-exact against tools/bench/dlxload.py on both CPU cores. The palette half is gated on words read back out of the palette registers at $E82000, so "the words reached the hardware" is part of what passes. ROADMAP P1 is done; P2's encoder half (a reserved black entry, 23.4) is not, and is a re-encode rather than an edit. A scene change costs 18.96 ms of 68000 time, 22.8% of one 12 fps frame; boot costs 24.70 ms. The scratch tables describe the CRTC, not the scene, so pal_tables is a separate entry point built once at boot -- 5.29 ms off every scene change. The one that moves something: the scene header is 5,920 B that no rate table in this tree included, because it belongs to no frame record. In FINDINGS 51.3's currency it is divided by the surplus pipe - wire, so it is hypersensitive: 138 ms of extra refill climb at 488 KB/s and 1.099 s at 451.4 KB/s, for the same bytes. tools/analysis/22_scene_load.py prices it across explicit rates. Recorded as open: the two CPU cores agree to <3% on every stage but the table build, where they differ by 16.4%. px68k's C68K charges a flat 50 clocks for MULU/MULS (c68kmacro.h:1869) where the 68000 charges 38+2n, which explains 4,608 of the 8,703 clock gap. 4,095 clocks are unexplained. Nothing else in src/player/ multiplies, so no figure in FINDINGS 24-52 is affected. decode.s and stream.s are untouched; decode.bin is still 1,296 B at the same MD5. check.sh gains a stage that gates byte-exactness on both cores and deliberately does not gate the cycle counts -- MAME's clock is 1/55.46 s and a wall timing would make the green light host-sensitive. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
This commit is contained in:
prosolis
@@ -103,14 +103,26 @@ the disk it is where nothing fits at any container size. The ROM drives SASI
|
||||
rather than the MB89352, so it does not settle W, but a cheap configuration is
|
||||
now the thing that has to be shown rather than assumed (FINDINGS 52).
|
||||
|
||||
**The player builds its own codebooks and palette now.** The two load-time
|
||||
transforms — codebooks to word-per-pixel form, palette to `GGGGGRRRRRBBBBBI`
|
||||
with the shared LSB picked per entry — ran host-side until session 21 and now
|
||||
run on the 68000, out of the raw container header, byte-exact against the host
|
||||
implementation on both CPU cores and with the palette read back out of the
|
||||
hardware registers. A scene change costs **18.96 ms**, a third of one 12fps
|
||||
frame slot. The finding underneath it is a cost nothing had counted: a scene
|
||||
header is **5,920 bytes** that must arrive before frame 0, and in the currency
|
||||
of seek slack those bytes lengthen the refill climb by 138 ms at 488 KB/s and by
|
||||
**1.099 s at 451.4 KB/s**, because the surplus they are divided by goes to zero
|
||||
(FINDINGS 53).
|
||||
|
||||
**Current encode:** 496.7 KB/s at 29.19 dB, 1 frame of 120 over the 12fps
|
||||
budget, and that one is frame 0, the intra frame, late on purpose.
|
||||
|
||||
**Green-light check:** `./tools/bench/check.sh` (~3 min, needs the Blu-ray
|
||||
**Green-light check:** `./tools/bench/check.sh` (~4 min, needs the Blu-ray
|
||||
mounted) re-runs both display regression tests, the rate-control drift gate, the
|
||||
display-path coherency counterexample, a 120-frame 68000 decode on two CPU
|
||||
cores, the ring and paced-ring passes and the DMAC configuration gate, then
|
||||
prints `ALL GREEN`.
|
||||
cores, the ring and paced-ring passes, the DMAC configuration gate and the
|
||||
load-time transforms on both cores, then prints `ALL GREEN`.
|
||||
|
||||
## Reproducing this
|
||||
|
||||
@@ -255,6 +267,10 @@ tools/analysis/ measurement scripts, numbered in the order they were written.
|
||||
model, sharing no code with the Lua producer it checks.
|
||||
21 decodes the IPL ROM's HD63450 configuration and gates on the
|
||||
bytes being where it says they are.
|
||||
22 prices a scene change: header bytes, load-time clocks and
|
||||
what both cost in accumulated seek slack, across explicit
|
||||
rates. Its cycle counts are PARSED out of the rig's log, not
|
||||
pasted in, so they cannot go stale silently.
|
||||
buscost.py is the shared bus-cycle table. The per-block
|
||||
constants live in tools/encoder/vq_hybrid.py and are imported,
|
||||
never copied.
|
||||
@@ -275,7 +291,12 @@ tools/bench/ MAME Lua injection harness and 68000 benchmark sources.
|
||||
through a bounded ring at a modelled pipe rate, so the rig is
|
||||
not bounded by the emulated machine's RAM and a stock 2 MB
|
||||
machine runs the whole window. dlxload.py holds the
|
||||
codebook/palette load-time maths both preps share.
|
||||
codebook/palette load-time maths both preps share -- and
|
||||
the reference src/player/load.i is gated against.
|
||||
prep_load.py/load.lua/verify_load.py/load_run.sh run those
|
||||
transforms ON the 68000 and compare all 10,752 output bytes
|
||||
with dlxload.py's, palette words read back out of the palette
|
||||
registers rather than a RAM shadow.
|
||||
tools/bench/c68k/ headless px68k C68K harness, a SECOND emulator for every
|
||||
68000 cycle figure. Links only px68k's CPU core: no SDL, no
|
||||
ROMs, no emulated machine. `make PX68K=~/src/px68k` then
|
||||
@@ -296,6 +317,11 @@ tools/encoder/ hybrid VQ encoder and DLX3 container writer.
|
||||
dlx.py is the reference DECODER, ground truth for the 68000.
|
||||
src/player/ decode.s is the 68000 DLX3 decoder with a preloaded-stream
|
||||
front-end. stream.s is the same decoder behind a bounded ring.
|
||||
load.i is the LOAD-time half: codebook expansion and palette
|
||||
packing, out of the raw container header, with loadgate.s as
|
||||
its rig front-end. Its three scratch tables describe the
|
||||
machine rather than the scene, so they are a separate entry
|
||||
point a player calls once at boot.
|
||||
Both include frame.i (the block loop and span chain) and
|
||||
geom.i (the constants), so there is exactly ONE copy of the
|
||||
bytes every cycle constant is fitted to. The span pass is
|
||||
|
||||
@@ -4477,3 +4477,144 @@ Relevant to 51's underrun analysis, which models delivery as a smooth rate.
|
||||
6. **The frame-period accounting assumes the DMAC does not overlap the CPU**
|
||||
(`buscost.DMA_OVERLAPS = False`), which is FINDINGS 35's premise: no cache,
|
||||
a two-word prefetch queue. A stolen bus cycle is a stopped 68000.
|
||||
|
||||
---
|
||||
|
||||
## 53. The loader moves onto the 68000, and a scene change finally has a price (session 21)
|
||||
|
||||
ROADMAP P1 and P2. Since session 1 the two load-time transforms have been done
|
||||
**host-side**, in `tools/bench/dlxload.py`, with the rigs pushing the *result*
|
||||
into emulated RAM: the codebooks expanded to word-per-pixel form (CB1 to 32 B an
|
||||
entry, CB4 to 8 B) and the 24-bit palette packed to `GGGGGRRRRRBBBBBI` with the
|
||||
shared LSB chosen per entry. That was the right call while the inner loop was
|
||||
what was being measured — charging a once-per-scene cost to the per-frame path
|
||||
would have flattered or damned it for no reason — but **a player has no host.**
|
||||
|
||||
`src/player/load.i` does both on the 68000, out of the raw container header as
|
||||
it comes off the disc. `tools/bench/loadgate.s` is its front-end, the way
|
||||
`decode.s` is `frame.i`'s.
|
||||
|
||||
**NAME THE LAYER.** Everything here is **emulated**: MAME 0.277 `x68000`,
|
||||
`-bios ipl10`, stock 10 MHz / 2 MB, cross-checked on px68k's C68K core. Nothing
|
||||
has run on real hardware.
|
||||
|
||||
### 53.1 It reproduces dlxload.py exactly, on both cores
|
||||
|
||||
`dlxload.py` stays the reference — what changed is **where the transforms run,
|
||||
not what they produce** — so the gate is byte-for-byte, not "close enough":
|
||||
|
||||
- **CB1 8,192 B, CB4 2,048 B, palette 512 B: identical.** A wrong codebook byte
|
||||
is a wrong colour in every block that uses that codeword, in every frame of
|
||||
the scene, and a wrong shared LSB is a *slightly* wrong colour, which is
|
||||
exactly the sort of defect that gets attributed to the codec.
|
||||
- **The palette half is read back out of the palette registers at `$E82000`**,
|
||||
not out of a RAM shadow, so "the words reached the hardware" is part of what
|
||||
passes.
|
||||
- **The darkest-entry index agrees too** (255 on the gate container). It comes
|
||||
out of an `argmin` whose tie-break has to match numpy's — first index at the
|
||||
minimum wins — and it is what the letterbox is filled with.
|
||||
- **Both CPU cores produced the same 10,752 bytes**, and the same as the host.
|
||||
|
||||
`tools/bench/load_run.sh` runs it and `check.sh` gates it.
|
||||
|
||||
### 53.2 What it costs, measured on two cores
|
||||
|
||||
| stage | MAME clocks | C68K clocks | Δ | data bus (C68K) |
|
||||
|---|---|---|---|---|
|
||||
| scratch tables (boot only) | 52,919 | 61,622 | +16.4% | 6.5% |
|
||||
| **P1** codebook expansion | 92,609 | 95,304 | +2.9% | **43.2%** |
|
||||
| **P2** palette entries | 97,019 | 97,348 | +0.4% | 14.3% |
|
||||
| BOOT: all three | 246,957 | 253,614 | +2.7% | 23.2% |
|
||||
| **SCENE CHANGE: P1 + P2** | **189,627** | 192,322 | +1.4% | 28.6% |
|
||||
|
||||
**A scene change costs 18.96 ms of 68000 time — 22.8% of one 12 fps frame.**
|
||||
Boot costs 24.70 ms. Bus occupancy is data accesses only (C68K does not see
|
||||
prefetch), so it is a lower bound; the expansion is the bus-heaviest thing here
|
||||
because it is a pure copy, and it still runs alone.
|
||||
|
||||
**The stages are exactly additive on the exact core.** `P1 + P2 - SCENE = 330`
|
||||
clocks, and `tables + P1 + P2 - 2x330 = 253,614 = BOOT`, to the clock — 330 is
|
||||
the front-end's own per-pass overhead. On MAME the same identity closes to 1.8%,
|
||||
which is one tick of its 1/55.46 s clock over the 0.99 s run. Two instruments,
|
||||
two granularities, one arithmetic.
|
||||
|
||||
### 53.3 The scratch tables are scene-independent, so they are not in the scene path
|
||||
|
||||
Packing a palette entry needs the squared error of both choices of the shared
|
||||
LSB, per channel. That is three table reads and a sign test here, out of three
|
||||
tables — the 6-bit-to-8-bit rendering the CRTC performs, its square, and the
|
||||
per-channel error difference — and **not one of them describes the scene.** They
|
||||
describe the machine. `pal_tables` is therefore a separate entry point from
|
||||
`pal_pack`, built once at boot: **5.29 ms saved on every scene change**, 22% of
|
||||
what a naive port of `dlxload.py` would have charged per scene.
|
||||
|
||||
### 53.4 The two cores disagree only where the multiplies are, and C68K is wrong in kind
|
||||
|
||||
The table build is the only code in this tree that multiplies, and it is the
|
||||
only stage where the two cores disagree by more than 3%. **px68k's C68K charges
|
||||
a flat 50 clocks for `MULU` and `MULS`** regardless of the operand
|
||||
(`c68kmacro.h:1869/1883`, `RET(50 + EA_CLOCKS_...)`); the 68000 charges
|
||||
**38 + 2n**, n counting bits in the source. For the 576 multiplies this code
|
||||
executes, the real total is 24,192 clocks against C68K's 28,800: **the flat rate
|
||||
explains 4,608 of the 8,703 clock gap, and 4,095 clocks — 7.7% of the stage —
|
||||
are NOT explained.** Recorded as open rather than rounded away; the residual is
|
||||
somewhere else in the two cycle tables and this stage is not worth the hunt.
|
||||
|
||||
**The consequence is general and belongs in the reader's head:** where a future
|
||||
measurement contains multiplies, C68K over-charges them, and it is the second
|
||||
opinion this tree leans on for every cycle figure. Nothing else in
|
||||
`src/player/` multiplies — index scaling is `lsl.w #5`/`#3` by construction —
|
||||
so no figure in FINDINGS 24-52 is affected.
|
||||
|
||||
### 53.5 Where the cost actually lands: the scene change, priced
|
||||
|
||||
`tools/analysis/22_scene_load.py`, cycle counts parsed out of the rig's own log
|
||||
rather than pasted in as constants. Three costs in three units, and **the third
|
||||
is the one that compounds**:
|
||||
|
||||
- **BYTES.** The header region is **5,920 B** (palette 768 + CB1 4,096 + CB4
|
||||
1,024 + 32) and it must arrive before frame 0 can be decoded. It is not part
|
||||
of any frame record, so **no rate table in this tree has ever counted it.**
|
||||
- **CLOCKS.** 189,627, from 53.2.
|
||||
- **ACCUMULATED SLACK.** Those bytes are bytes the pipe did not spend filling
|
||||
the ring, so they cost play-time at the surplus rate `pipe - wire` — the
|
||||
currency FINDINGS 51.3 established a branch point spends.
|
||||
|
||||
| pipe KB/s | header ms | + load ms | total | frame slots | surplus KB/s | slack cost |
|
||||
|---|---|---|---|---|---|---|
|
||||
| 451.4 | 12.81 | 18.96 | 31.77 | 0.38 | 5.3 | **1.099 s** |
|
||||
| 488 | 11.85 | 18.96 | 30.81 | 0.37 | 41.9 | 0.138 s |
|
||||
| 513.2 | 11.27 | 18.96 | 30.23 | 0.36 | 67.1 | 0.086 s |
|
||||
| 600 | 9.64 | 18.96 | 28.60 | 0.34 | 153.9 | 0.038 s |
|
||||
|
||||
(Rates are explicit arguments with no default, FINDINGS 50. `wire` is 446.1
|
||||
KB/s on the gate container, audio included.)
|
||||
|
||||
**Two readings, and the second is the finding.** First: the whole fixed cost of
|
||||
a scene change is **about a third of one frame slot** — it is not what makes a
|
||||
branch point expensive, the seek and the refill climb are. Second: **the slack
|
||||
cost is hypersensitive to the rate**, because it is divided by a surplus that
|
||||
goes to zero. At 488 KB/s the header lengthens the climb by 138 ms; at 451.4
|
||||
KB/s — the *arrival-deadline* rate for this same container, 49.5 — the same
|
||||
5,920 bytes cost **1.1 seconds of play**. The header is cheap only where the
|
||||
pipe already has room, which is the same place everything else in this project
|
||||
is cheap.
|
||||
|
||||
### 53.6 The alternative that was not taken
|
||||
|
||||
The encoder could ship the codebooks pre-expanded and P1 would not exist. That
|
||||
trades **9.26 ms of 68000 time for 5,120 more bytes in every scene header** —
|
||||
10.5 ms of pipe at 488 KB/s, and 5,120 bytes that lengthen the climb again by
|
||||
the arithmetic above. **Derived, not measured**, from the two figures either
|
||||
side of it. It is close to a wash in milliseconds and it is not a wash in
|
||||
*kind*: the CPU is idle during a seek and the pipe is the resource this project
|
||||
is short of. The transform stays on the 68000.
|
||||
|
||||
### 53.7 What is still open in P2
|
||||
|
||||
**The encoder still does not reserve a black entry** (23.4), so the letterbox
|
||||
gets the palette's closest thing to black — index 255 here — rather than a true
|
||||
black with `I = 0`. That half of P2 is encoder-side, it changes the container,
|
||||
and it moves every constant fitted to the gate container, so it is a re-encode
|
||||
plus a re-measurement rather than an edit. `load.i` is ready for it: it reads
|
||||
whatever the palette section holds and reports the darkest index either way.
|
||||
|
||||
+27
-11
@@ -1,6 +1,7 @@
|
||||
# Roadmap — remaining work to a completion target
|
||||
|
||||
Written end of session 19 (2026-08-24), against a tree that is ALL GREEN.
|
||||
Amended end of session 21: P1 done, P2 half done (FINDINGS 53).
|
||||
|
||||
**THE COMPLETION TARGET IS M3, THE VERTICAL SLICE** (USER DECISION): one scene
|
||||
tree — a decision point, two outcomes, a death clip — with audio, streaming from
|
||||
@@ -96,17 +97,32 @@ out of a bounded ring fed by a host file on a paced clock. Neither is a player.
|
||||
**Exit criterion: boots from a real SCSI volume on a stock 2 MB X68000, plays
|
||||
one scene at 12 fps from disc, no host-file pipe, no Lua in the loop. Silent.**
|
||||
|
||||
**P1. Codebook expansion on the 68000.** `dlxload.py:19` expands CB1 to 32 B per
|
||||
entry and CB4 to 8 B, host-side, because at the time it was a load-time cost that
|
||||
would have flattered or damned the inner loop. The player must do it: **8 KB +
|
||||
2 KB per scene**. Note where that lands — *at a scene change, when the ring is
|
||||
empty because of the seek*. It compounds with 51.3 and should be priced against
|
||||
the refill climb, not treated as free setup.
|
||||
~~**P1. Codebook expansion on the 68000.**~~ **DONE, session 21 — FINDINGS 53.**
|
||||
`src/player/load.i` expands both codebooks out of the raw container header,
|
||||
byte-exact against `dlxload.py` on both CPU cores. **9.26 ms**, and it was
|
||||
priced where it lands rather than treated as free setup: the scene header is
|
||||
**5,920 B that no rate table in this tree counted**, and in the currency of
|
||||
51.3 — accumulated slack — those bytes lengthen the refill climb by 138 ms at
|
||||
488 KB/s and by **1.099 s at 451.4 KB/s**, because the surplus they are divided
|
||||
by goes to zero. The whole fixed cost of a scene change is about a third of one
|
||||
frame slot; what makes a branch point expensive is still the seek and the climb.
|
||||
Shipping the codebooks pre-expanded was considered and refused: it trades
|
||||
9.26 ms of CPU for 5,120 more header bytes, which is a wash in milliseconds and
|
||||
not a wash in kind (53.6).
|
||||
|
||||
**P2. Palette packing on the 68000.** The encoder still emits RGB888; the X68000
|
||||
word packing is Lua-side. Whatever writes real palette words must pick `I` per
|
||||
entry by minimum squared error (**1.96 dB**, FINDINGS 23.3) and reserve index 0
|
||||
as black with `I = 0` (23.4).
|
||||
**P2. Palette packing on the 68000. HALF DONE, session 21 — FINDINGS 53.**
|
||||
~~The encoder still emits RGB888; the X68000 word packing is Lua-side.~~ The
|
||||
packing is on the 68000: `pal_pack` writes 256 words straight into `$E82000`
|
||||
with `I` chosen per entry by minimum squared error (**1.96 dB**, 23.3), gated on
|
||||
the words read back **out of the palette registers**. 9.70 ms per scene, plus
|
||||
5.29 ms of scene-independent table build hoisted to boot (53.3).
|
||||
|
||||
**What is left is the other half of the sentence: reserve index 0 as black with
|
||||
`I = 0` (23.4).** That is ENCODER-side, it changes the container, and it moves
|
||||
every constant fitted to the gate container, so it is a re-encode plus a
|
||||
re-measurement rather than an edit. Until then the letterbox gets the palette's
|
||||
closest thing to black (index 255 on the gate container); `load.i` reports
|
||||
whichever index that is and needs no change when it becomes 0.
|
||||
|
||||
**P3. A real frame clock.** `stream.s` has `PACE`/`PACEON` (`$18034`/`$18038`)
|
||||
but the 12 fps tick comes from the Lua producer. Needs MFP timer or VBL. Keep
|
||||
@@ -229,7 +245,7 @@ Listed for completeness; past M3 these are scope, not risk.
|
||||
B1 seek+rate ─┐
|
||||
B3 DTYP ──────┴─> P4 transport ─┐
|
||||
├─> M2 ─> M3 (COMPLETION TARGET) ─> M4
|
||||
P1 P2 P3 P5 P7 ─────────────────┘ ^
|
||||
P1 P2(half) P3 P5 P7 ───────────┘ ^
|
||||
│
|
||||
P6 (bus cost DONE, 52) ──────────────────┤
|
||||
G1 scene graph (fetch, do early) ─────────┘
|
||||
|
||||
@@ -1,3 +1,93 @@
|
||||
# Status & next-session handoff — end of session 21 (2026-08-24)
|
||||
|
||||
## Session 21: the loader moves onto the 68000, and a scene change gets a price
|
||||
|
||||
**Green light first and last: `./tools/bench/check.sh` was ALL GREEN before any
|
||||
of this and ALL GREEN after**, 120/120 on both cores, no `TRUNCATED`, plus a new
|
||||
load-time stage.
|
||||
|
||||
**ROADMAP P1 is DONE and P2 is half done. FINDINGS 53.** Session 20's handoff
|
||||
named P4 as the item that decides the project, and P4 is **blocked in this
|
||||
tree** — re-checked, not assumed: there is still no `scsiexrom.bin` anywhere on
|
||||
this machine (`~/mame/roms/x68000.zip` holds six files, four IPLs, a cgrom and
|
||||
an sram), MAME's `x68000` has no MB89352 path, and `hd63450.cpp` decodes no
|
||||
DTYP. **Nothing here can measure W.** P1+P2 was the M2 item that could be built
|
||||
here, and it is the one that touches an already-measured number: it lands at a
|
||||
scene change, where FINDINGS 51.3's refill climb is.
|
||||
|
||||
**1. The transforms are on the 68000 and they are byte-exact.** `src/player/
|
||||
load.i` expands both codebooks to word-per-pixel form and packs the palette to
|
||||
`GGGGGRRRRRBBBBBI` with the shared LSB chosen per entry, out of the RAW
|
||||
container header. Gated **byte-for-byte against `tools/bench/dlxload.py`**,
|
||||
which stays the reference — what changed is where the transforms run, not what
|
||||
they produce. The palette half is read back **out of the palette registers at
|
||||
`$E82000`**, so "the words reached the hardware" is part of what passes. Both
|
||||
CPU cores emit the same 10,752 B. 53.1.
|
||||
|
||||
**2. A scene change costs 18.96 ms of 68000 time, 22.8% of one 12 fps frame.**
|
||||
Boot costs 24.70 ms. Split: codebooks 92,609 clocks, palette entries 97,019,
|
||||
scratch tables 52,919. Cross-checked on px68k's C68K, which agrees to 1.4% on
|
||||
the scene-change figure. 53.2.
|
||||
|
||||
**3. The scratch tables describe the machine, not the scene.** They are the
|
||||
CRTC's 6-to-8-bit rendering, its square, and the per-channel error difference —
|
||||
so `pal_tables` is a separate entry point, built once at boot. **5.29 ms off
|
||||
every scene change**, 22% of what a naive port of `dlxload.py` would have cost
|
||||
per scene. 53.3.
|
||||
|
||||
**4. THE ONE THAT MOVES SOMETHING: the scene header is 5,920 bytes nothing has
|
||||
ever counted.** Palette + CB1 + CB4, and it must arrive before frame 0 can be
|
||||
decoded. It is not part of any frame record, so no rate table in this tree
|
||||
includes it. `tools/analysis/22_scene_load.py` prices it across explicit rates.
|
||||
The whole fixed cost of a scene change is about **a third of one frame slot** —
|
||||
but its cost in FINDINGS 51.3's currency, accumulated slack, is divided by the
|
||||
surplus `pipe - wire` and so is hypersensitive: **138 ms of extra climb at 488
|
||||
KB/s, and 1.099 s at 451.4 KB/s**, for the same 5,920 bytes. 53.5.
|
||||
|
||||
**5. An instrument disagreement worth carrying forward.** The two CPU cores
|
||||
agree to <3% on every stage except the table build, where they differ by 16.4%
|
||||
— and that is the only code in this tree that multiplies. **px68k's C68K charges
|
||||
a flat 50 clocks for `MULU`/`MULS`** regardless of operand (`c68kmacro.h:1869`),
|
||||
where the 68000 charges 38+2n. That explains 4,608 of the 8,703 clock gap;
|
||||
**4,095 clocks are not explained and are recorded as open.** Nothing in
|
||||
`src/player/` outside these three instructions multiplies (checked), so no
|
||||
figure in FINDINGS 24-52 is affected — but the second opinion this tree leans on
|
||||
over-charges multiplies, and a future measurement containing one must not be
|
||||
taken from it uncorrected. 53.4.
|
||||
|
||||
**6. Shipping pre-expanded codebooks was considered and refused.** It would
|
||||
trade 9.26 ms of 68000 time for 5,120 more bytes in every scene header — 10.5 ms
|
||||
of pipe at 488 KB/s, and bytes that lengthen the climb. Close to a wash in
|
||||
milliseconds, not a wash in kind: the CPU is idle during a seek and the pipe is
|
||||
what this project is short of. **Derived, not measured.** 53.6.
|
||||
|
||||
**New in the tree:** `src/player/load.i` (the transforms) and
|
||||
`src/player/loadgate.s` (its front-end, 488 B); `tools/bench/prep_load.py`,
|
||||
`load.lua`, `verify_load.py`, `load_run.sh` (the rig, both cores);
|
||||
`tools/analysis/22_scene_load.py` (the pricing). `tools/bench/c68k/harness.c`
|
||||
gains a `--loadraw` mode, which also makes its flag-watch address a variable
|
||||
rather than a constant. `check.sh` gains a stage that gates byte-exactness on
|
||||
both cores, and deliberately does **not** gate the cycle counts — MAME's clock
|
||||
is 1/55.46 s and a wall timing would make the green light host-sensitive, the
|
||||
same reason `blit.s` and `span.sh` are not in it.
|
||||
|
||||
**`decode.s` and `stream.s` are unchanged.** Nothing in the per-frame path was
|
||||
touched; `decode.bin` is still 1,296 B at the same MD5.
|
||||
|
||||
**Still open in P2:** the encoder does not reserve a black entry (23.4), so the
|
||||
letterbox still gets the palette's closest thing to black (index 255 here). That
|
||||
half is encoder-side, it changes the container, and it moves every constant
|
||||
fitted to the gate container — a re-encode plus a re-measurement, not an edit.
|
||||
|
||||
**Next:** P3 (a real frame clock from the MFP or VBL) and P5 (per-record index,
|
||||
prefill policy, the accumulated-slack rule in the player rather than the rig)
|
||||
are both buildable here. G1 (import the scene graph) is the one that would let
|
||||
this tree ask a question it currently cannot: what is the worst gap between
|
||||
consecutive decision points, and does the refill climb survive it. P4 still
|
||||
decides the project and still cannot be measured here.
|
||||
|
||||
---
|
||||
|
||||
# Status & next-session handoff — end of session 20 (2026-08-24)
|
||||
|
||||
## Session 20: the DMAC configuration was in the IPL ROM the whole time
|
||||
|
||||
@@ -0,0 +1,245 @@
|
||||
; ---------------------------------------------------------------------------
|
||||
; load.i -- the two LOAD-TIME transforms, on the 68000 itself. ROADMAP P1+P2.
|
||||
;
|
||||
; Until now both of these were done host-side, in tools/bench/dlxload.py, and
|
||||
; the rigs pushed the RESULT into emulated RAM. That was the right call while
|
||||
; the inner loop was the thing being measured -- charging a once-per-scene cost
|
||||
; to the per-frame path would have flattered or damned it for no reason -- but
|
||||
; a player has no host. These are the bytes that replace it.
|
||||
;
|
||||
; The reference is tools/bench/dlxload.py and it stays the reference: this code
|
||||
; is gated BYTE-FOR-BYTE against it (tools/bench/verify_load.py), palette words
|
||||
; and darkest-entry index included. If the two ever disagree, the symptom in a
|
||||
; rig would be wrong colours rather than a crash, which is exactly the class of
|
||||
; bug the split was made to prevent.
|
||||
;
|
||||
; WHAT IT READS. The RAW container as it comes off the disc. The DLX header is
|
||||
; fixed-layout and big-endian (tools/encoder/dlx.py):
|
||||
; +0 magic 'DLX3' +12 k1 u16 +16 off_pal u32
|
||||
; +4 W u16 +14 k4 u16 +20 off_cb1 u32
|
||||
; +6 H u16 +24 off_cb4 u32
|
||||
; +8 fps u16 +28 off_frm u32
|
||||
; +10 nframes u16
|
||||
; The three offsets are container-relative, so every one of them is an add of
|
||||
; the base the loader was handed. Nothing here parses a frame record.
|
||||
;
|
||||
; WHAT IT WRITES. CB1 (8 KB) and CB4 (2 KB) expanded to one WORD per pixel at
|
||||
; the addresses geom.i names, and 256 packed palette words straight into the
|
||||
; graphics palette at $E82000. It also reports the darkest entry, which is what
|
||||
; the letterbox is filled with until the encoder reserves a black one (23.4,
|
||||
; still open).
|
||||
;
|
||||
; WHY WORD-PER-PIXEL. The block loop movems codebook entries straight into
|
||||
; GVRAM with no unpacking, and the high byte of a GVRAM word write is discarded
|
||||
; by the hardware, so the high byte is left zero and never has to be cleared.
|
||||
; It also makes index scaling a shift rather than a multiply (lsl.w #5 / #3).
|
||||
;
|
||||
; SCRATCH. Three tables, built here and dead the moment the palette is packed:
|
||||
; P6TAB 64 B 6-bit level -> the 8-bit value the hardware renders it as
|
||||
; SQTAB 256 B the square of that, so the darkest-entry search has no muls
|
||||
; DTAB 512 B err(v, I=0) - err(v, I=1) per 8-bit channel value, signed
|
||||
; DTAB is what turns P2's per-entry minimum-squared-error choice of the shared
|
||||
; LSB into three table reads and a sign test. Choosing I per entry rather than
|
||||
; fixing it is worth 1.96 dB (FINDINGS 23.3), and it is a per-ENTRY decision
|
||||
; across three channels, so it cannot be folded into a per-channel table alone.
|
||||
; ---------------------------------------------------------------------------
|
||||
|
||||
LFLAG = $18040 ; 0 idle / 1 running / $FF done / $EE bad header
|
||||
LHDR = $18044 ; -> raw container base
|
||||
LDARK = $18048 ; <- index of the darkest palette entry
|
||||
LK1 = $1804C ; <- k1, as the 68000 read it out of the header
|
||||
LK4 = $18050 ; <- k4
|
||||
LMODE = $18054 ; bit0 codebooks, bit1 palette entries,
|
||||
; bit2 the three scratch tables
|
||||
LITER = $18058 ; repeat count, so a 55 Hz host clock can time it
|
||||
|
||||
P6TAB = $19000 ; 64 bytes
|
||||
SQTAB = $19040 ; 64 longs
|
||||
DTAB = $19140 ; 256 words
|
||||
GPAL = $E82000 ; graphics palette, 256 words
|
||||
|
||||
; ---------------------------------------------------------------- do_load
|
||||
; in: a0 = container base, d1 = mode bits: 1 codebooks, 2 palette entries,
|
||||
; 4 the scratch tables. A player builds the tables ONCE at boot (they
|
||||
; describe the hardware's colour rendering and nothing about the scene) and
|
||||
; then loads each scene with 3.
|
||||
; out: d0 = 0 ok, -1 not a DLX3 container. a0-a4 clobbered, a5 = base.
|
||||
do_load:
|
||||
movea.l a0,a5
|
||||
cmpi.l #$444C5833,(a5) ; 'DLX3'
|
||||
bne .bad
|
||||
move.w 12(a5),d0
|
||||
ext.l d0
|
||||
move.l d0,LK1.l
|
||||
move.w 14(a5),d0
|
||||
ext.l d0
|
||||
move.l d0,LK4.l
|
||||
|
||||
btst #2,d1
|
||||
beq.s .notab
|
||||
move.l d1,-(sp)
|
||||
bsr pal_tables
|
||||
move.l (sp)+,d1
|
||||
.notab:
|
||||
btst #0,d1
|
||||
beq.s .nocb
|
||||
moveq #0,d2 ; the count is built as a LONG and the
|
||||
move.w 12(a5),d2 ; high word must not carry junk into it
|
||||
lsl.l #4,d2 ; k1 entries x 16 source bytes
|
||||
movea.l 20(a5),a0
|
||||
adda.l a5,a0
|
||||
lea CB1,a1
|
||||
bsr expand
|
||||
moveq #0,d2
|
||||
move.w 14(a5),d2
|
||||
lsl.l #2,d2 ; k4 entries x 4 source bytes
|
||||
movea.l 24(a5),a0
|
||||
adda.l a5,a0
|
||||
lea CB4,a1
|
||||
bsr expand
|
||||
.nocb:
|
||||
btst #1,d1
|
||||
beq.s .nopal
|
||||
bsr pal_pack
|
||||
.nopal:
|
||||
moveq #0,d0
|
||||
rts
|
||||
.bad: moveq #-1,d0
|
||||
rts
|
||||
|
||||
; ---------------------------------------------------------------- expand
|
||||
; One source byte -> one destination word, high byte zero.
|
||||
; in: a0 src, a1 dst, d2 = source byte count. Always a multiple of 4: CB1 is
|
||||
; k1*16 and CB4 is k4*4, so no remainder case can exist and none is written.
|
||||
; A junk high word here is not a slow path, it is a WRONG one: `lsr.l #2` walks
|
||||
; two of its bits down into the low word and the dbra count comes out long.
|
||||
expand:
|
||||
lsr.l #2,d2
|
||||
subq.l #1,d2 ; k<=256, so the count fits a dbra
|
||||
moveq #0,d0
|
||||
.e1: move.b (a0)+,d0
|
||||
move.w d0,(a1)+
|
||||
move.b (a0)+,d0
|
||||
move.w d0,(a1)+
|
||||
move.b (a0)+,d0
|
||||
move.w d0,(a1)+
|
||||
move.b (a0)+,d0
|
||||
move.w d0,(a1)+
|
||||
dbra d2,.e1
|
||||
rts
|
||||
|
||||
; ---------------------------------------------------------------- pal_tables
|
||||
; The three scratch tables. SCENE-INDEPENDENT, every one of them: they describe
|
||||
; how the CRTC renders a 5-bit channel plus the shared LSB, which is a property
|
||||
; of the machine. A player builds them once at boot and never again, which is
|
||||
; why they are a separate entry point rather than the head of pal_pack -- see
|
||||
; FINDINGS 53.3 for what that is worth.
|
||||
pal_tables:
|
||||
; -- P6TAB[x] = ((x<<2)|(x>>4)) & $FF, and SQTAB[x] = P6TAB[x]^2
|
||||
lea P6TAB,a0
|
||||
lea SQTAB,a1
|
||||
moveq #0,d1
|
||||
.p1: move.w d1,d0
|
||||
lsl.w #2,d0
|
||||
move.w d1,d2
|
||||
lsr.w #4,d2
|
||||
or.w d2,d0
|
||||
andi.w #$FF,d0
|
||||
move.b d0,(a0)+
|
||||
move.w d0,d2
|
||||
mulu d2,d2
|
||||
move.l d2,(a1)+
|
||||
addq.w #1,d1
|
||||
cmpi.w #64,d1
|
||||
bne.s .p1
|
||||
|
||||
; -- DTAB[v] = (render(v,0)-v)^2 - (render(v,1)-v)^2, signed
|
||||
lea P6TAB,a0
|
||||
lea DTAB,a1
|
||||
moveq #0,d1
|
||||
.p2: move.w d1,d2
|
||||
lsr.w #2,d2
|
||||
andi.w #$3E,d2 ; x0 = (v>>3)<<1
|
||||
moveq #0,d3
|
||||
move.b 0(a0,d2.w),d3
|
||||
sub.w d1,d3
|
||||
muls d3,d3
|
||||
moveq #0,d4
|
||||
move.b 1(a0,d2.w),d4
|
||||
sub.w d1,d4
|
||||
muls d4,d4
|
||||
sub.l d4,d3
|
||||
move.w d3,(a1)+
|
||||
addq.w #1,d1
|
||||
cmpi.w #256,d1
|
||||
bne.s .p2
|
||||
rts
|
||||
|
||||
; ---------------------------------------------------------------- pal_pack
|
||||
; 24-bit RGB -> GGGGGRRRRRBBBBBI, the shared LSB chosen per entry by minimum
|
||||
; squared error, written to the palette registers. Identical arithmetic to
|
||||
; dlxload.pack_palette, including its tie-breaks: I stays 0 when the two errors
|
||||
; are equal, and the darkest entry is the FIRST index at the minimum.
|
||||
; in: a5 = container base, and pal_tables already run.
|
||||
pal_pack:
|
||||
movea.l 16(a5),a0
|
||||
adda.l a5,a0 ; -> 256 x RGB888
|
||||
lea GPAL,a1
|
||||
lea DTAB,a2
|
||||
lea SQTAB,a4 ; P6TAB is not needed here: the rendered
|
||||
; value is only ever wanted SQUARED
|
||||
move.l #$7FFFFFFF,d6
|
||||
clr.l LDARK.l
|
||||
moveq #0,d7
|
||||
.p3: moveq #0,d1
|
||||
move.b (a0)+,d1 ; R
|
||||
moveq #0,d2
|
||||
move.b (a0)+,d2 ; G
|
||||
moveq #0,d3
|
||||
move.b (a0)+,d3 ; B
|
||||
move.w d1,d0
|
||||
add.w d0,d0
|
||||
move.w 0(a2,d0.w),d4
|
||||
move.w d2,d0
|
||||
add.w d0,d0
|
||||
add.w 0(a2,d0.w),d4
|
||||
move.w d3,d0
|
||||
add.w d0,d0
|
||||
add.w 0(a2,d0.w),d4 ; sum of err0-err1 over the three
|
||||
moveq #0,d5
|
||||
tst.w d4
|
||||
ble.s .p4
|
||||
moveq #1,d5 ; I=1 only when it is STRICTLY better
|
||||
.p4: lsr.w #3,d1 ; fR
|
||||
lsr.w #3,d2 ; fG
|
||||
lsr.w #3,d3 ; fB
|
||||
move.w d2,d4
|
||||
lsl.w #5,d4
|
||||
or.w d1,d4
|
||||
lsl.w #6,d4 ; (fG<<11)|(fR<<6)
|
||||
move.w d3,d0
|
||||
add.w d0,d0
|
||||
or.w d0,d4
|
||||
or.w d5,d4
|
||||
move.w d4,(a1)+ ; -> the palette register
|
||||
|
||||
add.w d1,d1 ; x = (f<<1)|I, per channel
|
||||
or.w d5,d1
|
||||
add.w d2,d2
|
||||
or.w d5,d2
|
||||
add.w d3,d3
|
||||
or.w d5,d3
|
||||
lsl.w #2,d1 ; SQTAB holds longs
|
||||
move.l 0(a4,d1.w),d0
|
||||
lsl.w #2,d2
|
||||
add.l 0(a4,d2.w),d0
|
||||
lsl.w #2,d3
|
||||
add.l 0(a4,d3.w),d0 ; squared distance from black
|
||||
cmp.l d6,d0
|
||||
bge.s .p5
|
||||
move.l d0,d6
|
||||
move.l d7,LDARK.l ; first index at the minimum wins
|
||||
.p5: addq.w #1,d7
|
||||
cmpi.w #256,d7
|
||||
bne .p3
|
||||
rts
|
||||
@@ -0,0 +1,38 @@
|
||||
; Front-end for the load-time transforms (ROADMAP P1+P2), for the rig.
|
||||
;
|
||||
; It is to load.i what decode.s is to frame.i: a timing and control wrapper that
|
||||
; does nothing the shipping player would not do, so that the bytes being
|
||||
; measured are the bytes that will ship. The player's own boot path will call
|
||||
; do_load once with the mode bits set to 3; this repeats it LITER times so a
|
||||
; host clock with 1/55.46 s granularity can time a job that takes milliseconds,
|
||||
; and splits it by LMODE so the codebook expansion and the palette pack can be
|
||||
; priced apart. A player calls do_load with mode 7 once at boot -- the three
|
||||
; scratch tables describe the machine, not the scene -- and with mode 3 at every
|
||||
; scene change after that.
|
||||
;
|
||||
; Repeating is honest here in a way it would not be for a frame: nothing in
|
||||
; do_load is temporally recursive. Pass n writes exactly what pass n-1 wrote,
|
||||
; over the top of it, out of the same source bytes.
|
||||
|
||||
include "src/player/geom.i"
|
||||
|
||||
org $10000
|
||||
start:
|
||||
move.l LMODE.l,d1
|
||||
move.l LITER.l,d3
|
||||
move.l #1,LFLAG.l ; timer starts here
|
||||
loop:
|
||||
movem.l d1/d3,-(sp)
|
||||
movea.l LHDR.l,a0
|
||||
bsr do_load
|
||||
movem.l (sp)+,d1/d3
|
||||
tst.l d0
|
||||
bne.s bad
|
||||
subq.l #1,d3
|
||||
bne.s loop
|
||||
move.l #$FF,LFLAG.l ; timer stops here
|
||||
hold: bra.s hold
|
||||
bad: move.l #$EE,LFLAG.l
|
||||
bra.s hold
|
||||
|
||||
include "src/player/load.i"
|
||||
@@ -0,0 +1,106 @@
|
||||
#!/usr/bin/env python3
|
||||
"""What a scene change costs, now that the loader runs on the 68000 (FINDINGS 53).
|
||||
|
||||
python3 tools/analysis/22_scene_load.py [container ...] --kbps R [R ...]
|
||||
|
||||
ROADMAP P1 asked for the codebook expansion to be priced "against the refill
|
||||
climb, not treated as free setup", and that is the whole job of this file. A
|
||||
scene change is the one moment where every cost in this project lands at once:
|
||||
the ring is empty because of the seek, the header has to arrive before a single
|
||||
frame can be drawn, and the 68000 cannot decode anything until it has expanded
|
||||
the codebooks out of that header.
|
||||
|
||||
THREE COSTS, IN THREE DIFFERENT UNITS, and they are not interchangeable:
|
||||
|
||||
* BYTES. The container's header region -- palette, CB1, CB4 -- must be
|
||||
delivered before frame 0 can be decoded. It is not part of any frame
|
||||
record, so no rate table in this tree has ever counted it.
|
||||
* CLOCKS. What src/player/load.i costs to turn that header into what the
|
||||
block loop reads, MEASURED on the emulated 68000 by tools/bench/load.lua
|
||||
and parsed out of its log rather than copied in here as a constant.
|
||||
* ACCUMULATED SLACK. The bytes above are bytes the pipe did not spend
|
||||
filling the ring, so they cost play-time at the surplus rate (pipe - wire),
|
||||
which is the currency FINDINGS 51.3 established a branch point spends.
|
||||
This is the one that compounds: it is charged on top of the seek itself.
|
||||
|
||||
`--kbps` is REQUIRED and takes no default, for the reason FINDINGS 50 gives.
|
||||
Rates are decimal-KB per the rest of the tree's tooling; sizes are KiB.
|
||||
"""
|
||||
import sys, os, re, argparse
|
||||
sys.path.insert(0, "tools/encoder")
|
||||
import numpy as np
|
||||
from dlx import DLX
|
||||
import ratectl as RC
|
||||
|
||||
FPS = 12
|
||||
CPUHZ = 10_000_000
|
||||
|
||||
|
||||
def rig_cycles(path):
|
||||
"""The measured per-stage cost, out of tools/bench/load.lua's own log.
|
||||
|
||||
Parsed rather than pasted: a constant copied in here would go stale the
|
||||
first time load.i changed, and it would go stale SILENTLY -- the arithmetic
|
||||
below would keep working and keep being wrong.
|
||||
"""
|
||||
if not os.path.exists(path):
|
||||
sys.exit(f"no rig log at {path} -- run tools/bench/load_run.sh first")
|
||||
out = {}
|
||||
for line in open(path, "rb").read().decode("utf-8", "replace").splitlines():
|
||||
m = re.search(r"^\[LOD\]\s+(\S.*?)\s{2,}(\d+) cyc", line)
|
||||
if m:
|
||||
out[m.group(1).strip()] = int(m.group(2))
|
||||
need = ("SCENE CHANGE: codebooks + palette", "scratch tables only (boot, once)")
|
||||
for k in need:
|
||||
if k not in out:
|
||||
sys.exit(f"{path} has no '{k}' line -- is it a load.lua summary?")
|
||||
return out
|
||||
|
||||
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("containers", nargs="*", default=["tmp/rc_fr_singe_scsi_span.dlx"])
|
||||
ap.add_argument("--kbps", type=float, nargs="+", required=True,
|
||||
help="delivered pipe rates, KB/s. REQUIRED, no default (FINDINGS 50)")
|
||||
ap.add_argument("--log", default="tmp/load_check.log",
|
||||
help="tools/bench/load.lua's log, for the measured cycle counts")
|
||||
a = ap.parse_args()
|
||||
|
||||
cyc = rig_cycles(a.log)
|
||||
scene_cyc = cyc["SCENE CHANGE: codebooks + palette"]
|
||||
boot_cyc = cyc["scratch tables only (boot, once)"]
|
||||
frame_cyc = CPUHZ / FPS
|
||||
|
||||
print(f"measured on the emulated 68000 ({a.log}):")
|
||||
print(f" per scene change {scene_cyc:>8,} clocks = {1000*scene_cyc/CPUHZ:6.2f} ms "
|
||||
f"= {100*scene_cyc/frame_cyc:.1f}% of one {FPS}fps frame")
|
||||
print(f" once at boot {boot_cyc:>8,} clocks = {1000*boot_cyc/CPUHZ:6.2f} ms "
|
||||
f" (the three scratch tables: scene-independent)")
|
||||
|
||||
for path in a.containers:
|
||||
d = DLX(path)
|
||||
hdr = int.from_bytes(d.raw[28:32], "big")
|
||||
rec = np.array([4 + n + (-(4 + n) % 4) for _, n in d.frames], np.int64)
|
||||
wire = rec.mean() * FPS / 1024 + RC.AUDIO_KBPS
|
||||
print(f"\n=== {path}: header region {hdr:,} B "
|
||||
f"(pal 768 + cb1 {d.k1*16:,} + cb4 {d.k4*4:,} + 32), wire {wire:.1f} KB/s")
|
||||
print(f"{'pipe':>6} {'header ms':>10} {'+load ms':>9} {'total':>7} "
|
||||
f"{'frames':>7} {'surplus':>9} {'slack s':>9}")
|
||||
for kbps in a.kbps:
|
||||
hdr_ms = 1000 * hdr / (kbps * 1024)
|
||||
load_ms = 1000 * scene_cyc / CPUHZ
|
||||
total = hdr_ms + load_ms
|
||||
surplus = kbps - wire
|
||||
# What the header costs in the currency of 51.3: play-time at the
|
||||
# surplus rate. A negative surplus means the container does not fit the
|
||||
# pipe at all and no amount of play buys the bytes back.
|
||||
slack = f"{hdr/(surplus*1024):8.3f}" if surplus > 0 else " NEVER"
|
||||
print(f"{kbps:>6.0f} {hdr_ms:>10.2f} {load_ms:>9.2f} {total:>7.2f} "
|
||||
f"{total/(1000/FPS):>7.2f} {surplus:>9.1f} {slack:>9}")
|
||||
|
||||
print("""
|
||||
Reading it. The 'frames' column is the scene change's FIXED cost in 12fps
|
||||
frame slots, before the ring has been given a single frame of lookahead -- so it
|
||||
is a floor under the black gap at a branch point, not the gap itself. The
|
||||
'slack s' column is the one that compounds with FINDINGS 51.3: the header's
|
||||
bytes are bytes that did not go into the ring, so they lengthen the climb back
|
||||
to the seek-slack ceiling by that much play-time, every time.""")
|
||||
+124
-3
@@ -55,6 +55,12 @@ void p6logd(const char *fmt, ...) { (void)fmt; }
|
||||
#define GV_HI 0xC80000u
|
||||
|
||||
#define FLAG 0x18000u
|
||||
#define LFLAG 0x18040u /* src/player/load.i's control block */
|
||||
#define LHDR 0x18044u
|
||||
#define LDARK 0x18048u
|
||||
#define LMODE 0x18054u
|
||||
#define LITER 0x18058u
|
||||
#define GPAL 0xE82000u
|
||||
#define ITER 0x18008u
|
||||
#define NFR 0x1800Cu
|
||||
#define FPTR 0x18010u
|
||||
@@ -106,9 +112,16 @@ static void wr8(unsigned int a, unsigned char d)
|
||||
* momentarily reads back as $FF again. Without in_exec that transient
|
||||
* recorded a run's stop cycle before the run had started, and every frame
|
||||
* after the first came out as the whole slice. */
|
||||
/* Which flag word the run watches. decode.s and stream.s use FLAG; the
|
||||
* load-time transforms of src/player/load.i use their own, so that a player
|
||||
* could eventually contain both without one clearing the other's state. The
|
||||
* VALUES mean the same thing in both (1 running, $FF done, $EE failed), which
|
||||
* is why one hook serves both. */
|
||||
static unsigned int flag_adr = FLAG;
|
||||
|
||||
static void note_flag(void)
|
||||
{
|
||||
unsigned int v = rd32(FLAG);
|
||||
unsigned int v = rd32(flag_adr);
|
||||
long long now = slice - C68K.ICount;
|
||||
if (!in_exec) return;
|
||||
if (v == 1 && cyc_start < 0) cyc_start = now;
|
||||
@@ -124,7 +137,7 @@ static void wr16(unsigned int a, unsigned short d)
|
||||
a &= ADRMASK;
|
||||
if (a >= GV_LO && a < GV_HI) { buf[a] = (unsigned char)d; buf[a+1] = 0; return; }
|
||||
buf[a] = (unsigned char)d; buf[a+1] = (unsigned char)(d >> 8);
|
||||
if (a >= FLAG && a < FLAG + 4) note_flag();
|
||||
if (a >= flag_adr && a < flag_adr + 4) note_flag();
|
||||
}
|
||||
|
||||
static void wr32(unsigned int a, unsigned int d){ wr16(a, (unsigned short)(d >> 16)); wr16(a+2, (unsigned short)d); }
|
||||
@@ -185,12 +198,104 @@ static long long run(unsigned int off, unsigned int nfr, unsigned int iter)
|
||||
return cyc_stop - cyc_start;
|
||||
}
|
||||
|
||||
/* ---- the load-time transforms (ROADMAP P1+P2, FINDINGS 53) --------------
|
||||
* The same question this harness asks of the decoder, asked of the loader: does
|
||||
* a SECOND 68000 core, with its own cycle table and its own memory model,
|
||||
* produce the same bytes and agree about what they cost? It also counts BUS
|
||||
* cycles, which MAME cannot report -- and the bus is the resource this project
|
||||
* established is the binding one (FINDINGS 38).
|
||||
*/
|
||||
static int run_load(const char *fcode, const char *fraw, const char *dump,
|
||||
unsigned int mode, unsigned int iter,
|
||||
unsigned int cb1_len, unsigned int cb4_len)
|
||||
{
|
||||
size_t nc, nr;
|
||||
unsigned char *code = slurp(fcode, &nc), *raw = slurp(fraw, &nr);
|
||||
push(STREAM, raw, nr); /* the RAW container header */
|
||||
push(CODE, code, nc);
|
||||
/* Poison every destination, so that a transform which writes NOTHING
|
||||
* cannot pass by leaving the harness's own zeros in place. */
|
||||
for (unsigned int a = CB1; a < CB1 + cb1_len; a += 2) wr16(a, 0xDEAD);
|
||||
for (unsigned int a = CB4; a < CB4 + cb4_len; a += 2) wr16(a, 0xDEAD);
|
||||
for (unsigned int c = 0; c < 256; c++) wr16(GPAL + c*2, 0xDEAD);
|
||||
wr32(LDARK, 0xFFFFFFFFu);
|
||||
/* The three scratch tables are poisoned only before a run that claims to
|
||||
* build them. A run that only PACKS the palette is entitled to find them
|
||||
* already built -- that is the point of pricing it separately -- so when
|
||||
* this process is asked for one, it does the boot pass first, untimed,
|
||||
* exactly as a player would have done at boot. Without that the pack runs
|
||||
* on zeros: every entry then takes the same branch and the darkest entry
|
||||
* comes out 0, which is a measurement of nothing. */
|
||||
if (mode & 4)
|
||||
for (unsigned int a = 0x19000; a < 0x19340; a += 2) wr16(a, 0xDEAD);
|
||||
|
||||
flag_adr = LFLAG;
|
||||
if ((mode & 2) && !(mode & 4)) {
|
||||
cyc_start = cyc_stop = -1; desync = 0;
|
||||
wr32(LFLAG, 0); wr32(LHDR, STREAM); wr32(LMODE, 4); wr32(LITER, 1);
|
||||
C68k_Reset(&C68K);
|
||||
C68k_Set_Reg(&C68K, C68K_SR, 0x2700);
|
||||
C68k_Set_Reg(&C68K, C68K_A7, STACK);
|
||||
C68k_Set_Reg(&C68K, C68K_PC, CODE);
|
||||
slice = 2000000000LL; in_exec = 1;
|
||||
C68k_Exec(&C68K, (INT32)slice);
|
||||
in_exec = 0;
|
||||
if (cyc_stop < 0) { fprintf(stderr, "TIMEOUT in the table pre-pass\n"); return 4; }
|
||||
}
|
||||
cyc_start = cyc_stop = -1; desync = 0; bus_r = bus_w = 0;
|
||||
wr32(LFLAG, 0); wr32(LHDR, STREAM); wr32(LMODE, mode); wr32(LITER, iter);
|
||||
C68k_Reset(&C68K);
|
||||
C68k_Set_Reg(&C68K, C68K_SR, 0x2700);
|
||||
C68k_Set_Reg(&C68K, C68K_A7, STACK);
|
||||
C68k_Set_Reg(&C68K, C68K_PC, CODE);
|
||||
slice = 2000000000LL;
|
||||
in_exec = 1;
|
||||
C68k_Exec(&C68K, (INT32)slice);
|
||||
in_exec = 0;
|
||||
if (cyc_stop < 0) { fprintf(stderr, "TIMEOUT -- loader never set LFLAG\n"); return 4; }
|
||||
if (desync) { fprintf(stderr, "BAD HEADER -- load.i found no 'DLX3' magic\n"); return 5; }
|
||||
|
||||
long long cyc = (cyc_stop - cyc_start) / (iter ? iter : 1);
|
||||
fprintf(stderr, "[C68K] load mode %u: %lld cyc/pass (%.2f ms at 10MHz, "
|
||||
"%.1f%% of a 12fps frame), dark=%u\n", mode, cyc, cyc / 10000.0,
|
||||
100.0 * cyc / (10000000.0 / 12), rd32(LDARK));
|
||||
/* A 68000 bus cycle is 4 clocks. Prefetch is not counted (C68K reads
|
||||
* opcodes straight through the fetch pointer), so this is a LOWER bound on
|
||||
* occupancy and the headroom it implies is an UPPER bound -- same caveat as
|
||||
* the decoder's figure above. */
|
||||
{
|
||||
double slots = (double)cyc / 4.0;
|
||||
double used = (double)(bus_r + bus_w) / (iter ? iter : 1);
|
||||
fprintf(stderr, "[C68K] data bus: %.0f reads + %.0f writes = %.0f of "
|
||||
"%.0f cycles = %.1f%% occupied (prefetch NOT counted)\n",
|
||||
(double)bus_r / iter, (double)bus_w / iter, used, slots,
|
||||
100.0 * used / slots);
|
||||
}
|
||||
if (dump) {
|
||||
FILE *g = fopen(dump, "wb");
|
||||
if (!g) { perror(dump); return 2; }
|
||||
for (unsigned int a = CB1; a < CB1 + cb1_len; a++) { unsigned char b = rd8(a); fwrite(&b,1,1,g); }
|
||||
for (unsigned int a = CB4; a < CB4 + cb4_len; a++) { unsigned char b = rd8(a); fwrite(&b,1,1,g); }
|
||||
for (unsigned int c = 0; c < 256; c++) {
|
||||
unsigned short w = rd16(GPAL + c*2);
|
||||
unsigned char b[2] = { (unsigned char)(w >> 8), (unsigned char)w };
|
||||
fwrite(b, 1, 2, g);
|
||||
}
|
||||
fclose(g);
|
||||
fprintf(stderr, "[C68K] load output dumped to %s (%u B)\n",
|
||||
dump, cb1_len + cb4_len + 512);
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
int main(int argc, char **argv)
|
||||
{
|
||||
const char *fcode = "tmp/decode.bin", *fdata = "tmp/decode_data.bin", *dump = NULL;
|
||||
unsigned int cb1_len=0, cb4_len=0, pal_len=0, stream_len=0, nframes=0, H=192, W=256, fps=12;
|
||||
unsigned int dark = 255;
|
||||
unsigned int anch[32]; int nanch = 0;
|
||||
const char *fraw = NULL, *loaddump = NULL;
|
||||
unsigned int loadmode = 7, loaditer = 1;
|
||||
for (int i = 1; i < argc; i++) {
|
||||
if (!strcmp(argv[i], "--code")) fcode = argv[++i];
|
||||
else if (!strcmp(argv[i], "--data")) fdata = argv[++i];
|
||||
@@ -204,10 +309,15 @@ int main(int argc, char **argv)
|
||||
else if (!strcmp(argv[i], "--H")) H = (unsigned)atoi(argv[++i]);
|
||||
else if (!strcmp(argv[i], "--fps")) fps = (unsigned)atoi(argv[++i]);
|
||||
else if (!strcmp(argv[i], "--dark")) dark = (unsigned)atoi(argv[++i]);
|
||||
else if (!strcmp(argv[i], "--loadraw")) fraw = argv[++i];
|
||||
else if (!strcmp(argv[i], "--loaddump")) loaddump = argv[++i];
|
||||
else if (!strcmp(argv[i], "--loadmode")) loadmode = (unsigned)atoi(argv[++i]);
|
||||
else if (!strcmp(argv[i], "--loaditer")) loaditer = (unsigned)atoi(argv[++i]);
|
||||
else if (!strcmp(argv[i], "--anchor")) { if (nanch < 32) anch[nanch++] = (unsigned)strtoul(argv[++i], NULL, 10); }
|
||||
else { fprintf(stderr, "unknown arg %s\n", argv[i]); return 2; }
|
||||
}
|
||||
if (!nframes || !stream_len) { fprintf(stderr, "need --nframes and --stream (from decode_meta.lua)\n"); return 2; }
|
||||
if (!fraw && (!nframes || !stream_len)) {
|
||||
fprintf(stderr, "need --nframes and --stream (from decode_meta.lua)\n"); return 2; }
|
||||
|
||||
/* MAP_32BIT: C68K keeps its fetch base in a UINT32, so the arena must live
|
||||
* below 4 GB or every opcode fetch reads a truncated pointer. */
|
||||
@@ -216,6 +326,17 @@ int main(int argc, char **argv)
|
||||
if (buf == MAP_FAILED) { perror("mmap MAP_32BIT"); return 2; }
|
||||
fprintf(stderr, "[C68K] arena at %p\n", (void *)buf);
|
||||
|
||||
if (fraw) {
|
||||
C68k_Init(&C68K);
|
||||
C68k_Set_ReadB (&C68K, rd8);
|
||||
C68k_Set_ReadW (&C68K, rd16);
|
||||
C68k_Set_WriteB(&C68K, wr8);
|
||||
C68k_Set_WriteW(&C68K, wr16);
|
||||
C68k_Set_Fetch (&C68K, 0x000000, 0xFFFFFF, (UINT32)(unsigned long)buf);
|
||||
return run_load(fcode, fraw, loaddump, loadmode, loaditer,
|
||||
cb1_len ? cb1_len : 8192, cb4_len ? cb4_len : 2048);
|
||||
}
|
||||
|
||||
size_t nc, nd;
|
||||
unsigned char *code = slurp(fcode, &nc), *data = slurp(fdata, &nd);
|
||||
size_t need = (size_t)cb1_len + cb4_len + pal_len + stream_len;
|
||||
|
||||
@@ -270,4 +270,28 @@ grep -q "ceiling 8 frames" tmp/pace_check.log || {
|
||||
grep -q "^OK" tmp/pace_check.log || { echo "FAIL: paced pass not pixel-exact";
|
||||
tail -4 tmp/pace_check.log; exit 1; }
|
||||
|
||||
echo "--- session 21: the 68000 builds its own codebooks and palette (FINDINGS 53) ---"
|
||||
# ROADMAP P1+P2. Until now tools/bench/dlxload.py expanded the codebooks and
|
||||
# packed the palette HOST-SIDE and the rigs pushed the result into emulated RAM.
|
||||
# A player has no host. src/player/load.i does both on the 68000, out of the RAW
|
||||
# container header, and this gates it byte-for-byte against dlxload.py -- which
|
||||
# stays the reference, because what changed is where the transforms RUN, not
|
||||
# what they produce.
|
||||
#
|
||||
# Byte-for-byte and not "close enough": a wrong codebook byte is a wrong colour
|
||||
# in every block that uses that codeword, and a wrong shared LSB is a slightly
|
||||
# wrong colour that looks like a codec artefact rather than a loader bug.
|
||||
# The palette half is read back out of the PALETTE REGISTERS at $E82000, so
|
||||
# "the words reached the hardware" is part of what passes.
|
||||
#
|
||||
# NOT gated on the cycle counts, and the reason is NOT the one blit.s has. These
|
||||
# are emulated time and reproduce exactly run to run; what they are not is
|
||||
# sharp, because MAME samples them on a 1/55.46 s clock and the job takes
|
||||
# milliseconds. Nothing in the tree's cost models depends on them either. A
|
||||
# change in them is a re-derivation in FINDINGS 53, not a red light here.
|
||||
bash tools/bench/load_run.sh "$DLX" > tmp/load_gate.log 2>&1 || {
|
||||
echo "FAIL: the load-time transforms did not pass."; tail -12 tmp/load_gate.log
|
||||
exit 1; }
|
||||
grep -aE "^ *OK|both CPU cores|SCENE CHANGE" tmp/load_gate.log | sed 's/^ *//;s/^/ /'
|
||||
|
||||
echo "ALL GREEN"
|
||||
|
||||
@@ -0,0 +1,178 @@
|
||||
-- Time and verify src/player/load.i on the emulated 68000 (ROADMAP P1+P2).
|
||||
--
|
||||
-- Two questions, one run, exactly as decode.lua asks them of the decoder:
|
||||
-- 1. CORRECTNESS. Does the 68000 produce, out of the RAW container header,
|
||||
-- byte for byte what tools/bench/dlxload.py produces host-side? The
|
||||
-- expanded codebooks are read back out of RAM and the palette out of the
|
||||
-- PALETTE REGISTERS -- not out of a RAM shadow, because "the words reached
|
||||
-- $E82000" is the claim being tested. tools/bench/verify_load.py does the
|
||||
-- comparison against dlxload.py, so the ground truth stays in one place.
|
||||
-- 2. COST. How long does it take, split into the codebook expansion and the
|
||||
-- palette pack, and what is that as a fraction of a 12 fps frame -- the
|
||||
-- only unit this project prices anything in.
|
||||
--
|
||||
-- Nothing here is pre-chewed: the blob pushed into RAM is the first 5,920 bytes
|
||||
-- of the container as they come off the disc. That is the whole point of the
|
||||
-- exercise, and it is also, not incidentally, exactly the read a player has to
|
||||
-- complete at a scene change before it can draw a single frame.
|
||||
--
|
||||
-- MEASUREMENT SCOPE, unchanged from decode.lua: MAME's memory carries no wait
|
||||
-- states, so these are pure 68000 instruction cycles -- a LOWER BOUND on real
|
||||
-- hardware. Interrupts are masked (SR=$2700). The host clock has 1/55.46 s
|
||||
-- granularity and the job takes milliseconds, so each configuration is repeated
|
||||
-- LITER times and divided; repeating is honest because do_load is not
|
||||
-- temporally recursive -- every pass rewrites what the last one wrote, from the
|
||||
-- same source bytes.
|
||||
|
||||
M = manager.machine
|
||||
SP = M.devices[":maincpu"].spaces["program"]
|
||||
|
||||
local function findfile(n)
|
||||
for _,p in ipairs{"../tools/bench/"..n, "tools/bench/"..n, n} do
|
||||
local f = io.open(p,"rb"); if f then f:close(); return p end
|
||||
end
|
||||
error(n.." not found")
|
||||
end
|
||||
local MODE = loadfile(findfile("crtc_mode.lua"))()
|
||||
local META = loadfile("load_meta.lua")()
|
||||
|
||||
local LFLAG, LHDR, LDARK = 0x18040, 0x18044, 0x18048
|
||||
local LK1, LK4, LMODE, LITER = 0x1804C, 0x18050, 0x18054, 0x18058
|
||||
local CB1, CB4, RAW = 0x20000, 0x22000, 0x30000
|
||||
local GPAL = 0xE82000
|
||||
local CPUHZ = 10000000 -- x68k.cpp:1133, 40_MHz_XTAL/4
|
||||
local FPS = 12
|
||||
local FRAME12 = CPUHZ / FPS
|
||||
local ITER = tonumber(os.getenv("DLX_LOAD_ITER") or "40")
|
||||
|
||||
local code do local f=io.open("loadgate.bin","rb"); code=f:read("a"); f:close() end
|
||||
local data do local f=io.open("load_data.bin","rb"); data=f:read("a"); f:close() end
|
||||
|
||||
local function T() local t=M.time; return t.seconds + t.attoseconds/1e18 end
|
||||
local function P(s) print("[LOD] "..s) end
|
||||
|
||||
local function push(addr, s, from, len)
|
||||
local i, n = from, len
|
||||
while n >= 4 do
|
||||
SP:write_u32(addr, (string.unpack(">I4", s, i)))
|
||||
addr, i, n = addr+4, i+4, n-4
|
||||
end
|
||||
while n > 0 do
|
||||
SP:write_u8(addr, string.byte(s,i)); addr, i, n = addr+1, i+1, n-1
|
||||
end
|
||||
end
|
||||
|
||||
-- Poison every destination before each run. Without this a stage that wrote
|
||||
-- NOTHING would still compare equal to the previous stage's output, and the
|
||||
-- palette-only run would "pass" the codebook check for free.
|
||||
--
|
||||
-- The three scratch tables are poisoned only before a run that CLAIMS to build
|
||||
-- them (mode bit 2). They are scene-independent, so the palette-entry stage is
|
||||
-- entitled to find them already there -- that is the whole point of measuring
|
||||
-- it separately -- but a stage that says it builds them must be shown to.
|
||||
local P6TAB, TABEND = 0x19000, 0x19340
|
||||
local function poison(mode)
|
||||
for a = CB1, CB1 + META.cb1_len - 2, 2 do SP:write_u16(a, 0xDEAD) end
|
||||
for a = CB4, CB4 + META.cb4_len - 2, 2 do SP:write_u16(a, 0xDEAD) end
|
||||
for c = 0, 255 do SP:write_u16(GPAL + c*2, 0xDEAD) end
|
||||
SP:write_u32(LDARK, 0xFFFFFFFF)
|
||||
if mode & 4 ~= 0 then
|
||||
for a = P6TAB, TABEND - 2, 2 do SP:write_u16(a, 0xDEAD) end
|
||||
end
|
||||
end
|
||||
|
||||
local function setup()
|
||||
MODE.apply(SP)
|
||||
push(RAW, data, 1, META.raw_len)
|
||||
for i = 1, #code do SP:write_u8(0x10000+i-1, string.byte(code,i)) end
|
||||
P(string.format("loaded loadgate.bin=%d B, raw container header %d B at 0x%X",
|
||||
#code, META.raw_len, RAW))
|
||||
end
|
||||
|
||||
local function launch(mode, iter)
|
||||
poison(mode)
|
||||
SP:write_u32(LFLAG, 0)
|
||||
SP:write_u32(LHDR, RAW)
|
||||
SP:write_u32(LMODE, mode)
|
||||
SP:write_u32(LITER, iter)
|
||||
local cpu = M.devices[":maincpu"]
|
||||
cpu.state["SR"].value = 0x2700 -- supervisor, ALL interrupts masked
|
||||
cpu.state["SP"].value = 0x8000
|
||||
cpu.state["PC"].value = 0x10000
|
||||
end
|
||||
|
||||
-- Written after the mode-3 run, and only after it: it is the output of ONE
|
||||
-- do_load call over the whole header, which is what the player does.
|
||||
local function dump()
|
||||
local out = io.open("load_out.bin", "wb")
|
||||
for a = CB1, CB1 + META.cb1_len - 1 do out:write(string.char(SP:read_u8(a))) end
|
||||
for a = CB4, CB4 + META.cb4_len - 1 do out:write(string.char(SP:read_u8(a))) end
|
||||
for c = 0, 255 do out:write(string.pack(">I2", SP:read_u16(GPAL + c*2) & 0xFFFF)) end
|
||||
out:close()
|
||||
P(string.format("dumped %d B of 68000 output to tmp/load_out.bin",
|
||||
META.cb1_len + META.cb4_len + 512))
|
||||
P(string.format("DARK=%d (host-side dlxload.py says %d), K1=%d K4=%d",
|
||||
SP:read_u32(LDARK), META.dark, SP:read_u32(LK1), SP:read_u32(LK4)))
|
||||
end
|
||||
|
||||
-- Order matters: the scratch tables are built by the first stage and the
|
||||
-- palette-entry stage runs on them, which is exactly how a player would be
|
||||
-- arranged. The two stages that stand for real player events -- boot, and a
|
||||
-- scene change -- come last, and the dump the verifier checks is taken from the
|
||||
-- BOOT one, so the path that is proved correct is the one that builds
|
||||
-- everything from nothing.
|
||||
local PLAN = {
|
||||
{name="scratch tables only (boot, once)", mode=4, iter=ITER},
|
||||
{name="codebook expansion only (P1)", mode=1, iter=ITER},
|
||||
{name="palette entries only (P2)", mode=2, iter=ITER},
|
||||
{name="BOOT: tables + codebooks + palette", mode=7, iter=ITER, dump=true},
|
||||
{name="SCENE CHANGE: codebooks + palette", mode=3, iter=ITER},
|
||||
}
|
||||
|
||||
local step, st, t0 = 0, "boot", nil
|
||||
local results = {}
|
||||
|
||||
SUB = emu.add_machine_frame_notifier(function()
|
||||
local ok, err = pcall(function()
|
||||
local t = T()
|
||||
if st == "boot" then
|
||||
if t < 3.0 then return end
|
||||
setup(); step = 1; launch(PLAN[1].mode, PLAN[1].iter)
|
||||
st, t0 = "running", nil; return
|
||||
end
|
||||
if st == "running" then
|
||||
local fl = SP:read_u32(LFLAG)
|
||||
if fl == 1 and not t0 then t0 = t; return end
|
||||
if fl == 0xEE then
|
||||
P("BAD HEADER -- load.i did not find the 'DLX3' magic at LHDR")
|
||||
M:exit(); return
|
||||
end
|
||||
if fl == 0xFF then
|
||||
local p = PLAN[step]
|
||||
local dt = t - (t0 or t)
|
||||
local cyc = dt * CPUHZ / p.iter
|
||||
results[#results+1] = {name=p.name, cyc=cyc}
|
||||
P(string.format("%s: %d passes in %.4f s -> %.0f cycles = %.1f%% of a "
|
||||
.."%dfps frame (%.2f ms)", p.name, p.iter, dt, cyc,
|
||||
100*cyc/FRAME12, FPS, 1000*cyc/CPUHZ))
|
||||
if p.dump then dump() end
|
||||
step = step + 1
|
||||
if PLAN[step] then launch(PLAN[step].mode, PLAN[step].iter); st, t0 = "running", nil
|
||||
else st = "finish" end
|
||||
return
|
||||
end
|
||||
if t > 400 then P("TIMEOUT flag="..string.format("%08X",fl)); M:exit() end
|
||||
return
|
||||
end
|
||||
if st == "finish" then
|
||||
P("---- summary (instruction cycles only; real RAM adds wait states) ----")
|
||||
for _,r in ipairs(results) do
|
||||
P(string.format(" %-44s %8.0f cyc %5.1f%% of a frame %6.2f ms",
|
||||
r.name, r.cyc, 100*r.cyc/FRAME12, 1000*r.cyc/CPUHZ))
|
||||
end
|
||||
P("done")
|
||||
M:exit()
|
||||
end
|
||||
end)
|
||||
if not ok then print("[LOD] LUA ERROR: "..tostring(err)); M:exit() end
|
||||
end)
|
||||
Executable
+54
@@ -0,0 +1,54 @@
|
||||
#!/bin/bash
|
||||
# One load-time transform run: the 68000 builds its own codebooks and palette
|
||||
# out of the RAW container header, on both CPU cores (ROADMAP P1+P2, FINDINGS
|
||||
# 53).
|
||||
#
|
||||
# tools/bench/load_run.sh [container]
|
||||
#
|
||||
# Both instruments run the same loadgate.bin over the same header bytes:
|
||||
# * MAME, which is the only one of the two with real PALETTE REGISTERS -- the
|
||||
# packed words are read back out of $E82000, not out of a RAM shadow, so
|
||||
# "the words reached the hardware" is part of what passes.
|
||||
# * px68k's C68K, which is exact to the cycle and counts BUS cycles, and is a
|
||||
# second opinion on the cost from a separately written cycle table.
|
||||
# Both outputs are compared byte-for-byte against tools/bench/dlxload.py, which
|
||||
# stays the reference: this code replaces where those transforms RUN, not what
|
||||
# they produce.
|
||||
set -e
|
||||
cd "$(dirname "$0")/../.."
|
||||
DLX=${1:-tmp/rc_fr_singe_scsi_span.dlx}
|
||||
PX68K=${PX68K:-$HOME/src/px68k}
|
||||
ITER=${DLX_LOAD_ITER:-40}
|
||||
|
||||
tools/vasm/vasmm68k_mot -Fbin -o tmp/loadgate.bin src/player/loadgate.s > /dev/null
|
||||
python3 tools/bench/prep_load.py "$DLX" > tmp/prep_load.log
|
||||
cat tmp/prep_load.log
|
||||
|
||||
# stdbuf -oL: without it a long MAME run is unobservable until it exits, and a
|
||||
# run that is merely finishing looks exactly like one that is wedged (34.1).
|
||||
( cd tmp && DLX_LOAD_ITER=$ITER SDL_VIDEODRIVER=dummy stdbuf -oL timeout -k 5 180 \
|
||||
mame x68000 -bios ipl10 -ramsize 2M -video soft -window -sound none \
|
||||
-nothrottle -plugins -autoboot_script ../tools/bench/load.lua \
|
||||
-seconds_to_run 30 > load_check.log 2>&1 )
|
||||
# A run that never reached the dump must fail as that, not as a byte mismatch.
|
||||
grep -q "^\[LOD\] done" tmp/load_check.log || {
|
||||
echo "FAIL: the load rig did not finish -- no completion marker."
|
||||
tail -6 tmp/load_check.log; exit 1; }
|
||||
grep -a "^\[LOD\]" tmp/load_check.log | sed -n '/summary/,$p' | sed 's/\[LOD\] / /'
|
||||
python3 tools/bench/verify_load.py "$DLX"
|
||||
|
||||
if [ -f "$PX68K/m68000/c68k.c" ]; then
|
||||
make -s -C tools/bench/c68k PX68K="$PX68K" 2>/dev/null
|
||||
for M in 4 1 2 7 3; do
|
||||
tools/bench/c68k/c68k_bench --code tmp/loadgate.bin --loadraw tmp/load_data.bin \
|
||||
--loadmode $M --loaditer 1 --cb1 8192 --cb4 2048 \
|
||||
$([ $M = 3 ] && echo "--loaddump tmp/load_c68k.bin") 2>&1 >/dev/null \
|
||||
| grep -av arena | sed 's/\[C68K\] / /'
|
||||
done
|
||||
# The second core's bytes are held to the same standard as the first's.
|
||||
cmp -s tmp/load_c68k.bin tmp/load_out.bin || {
|
||||
echo "FAIL: the two CPU cores produced DIFFERENT load-time output."; exit 1; }
|
||||
echo " OK both CPU cores produced the same $(stat -c%s tmp/load_out.bin) B"
|
||||
else
|
||||
echo " SKIPPED: no px68k at $PX68K (set PX68K= to point at a checkout)"
|
||||
fi
|
||||
@@ -0,0 +1,59 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Lay out the LOAD-TIME test: raw container header in, expected results out.
|
||||
|
||||
python3 tools/bench/prep_load.py <in.dlx> [--out tmp/load]
|
||||
|
||||
src/player/load.i does on the 68000 what tools/bench/dlxload.py has been doing
|
||||
host-side since session 1: expand the two codebooks to word-per-pixel form and
|
||||
pack the 24-bit palette into GGGGGRRRRRBBBBBI with the shared LSB chosen per
|
||||
entry (ROADMAP P1 and P2). This writes both halves of that comparison.
|
||||
|
||||
<out>_data.bin the container's HEADER REGION, byte for byte as it comes
|
||||
off the disc: magic, geometry, the three section offsets,
|
||||
the 768-byte palette, CB1 and CB4. Nothing is pre-chewed --
|
||||
that is the entire point. It ends where the frame stream
|
||||
begins, so it is also exactly what a player would have to
|
||||
read before it could draw anything.
|
||||
<out>_expect.bin what dlxload.py says the 68000 must produce: expanded CB1,
|
||||
expanded CB4, then 256 big-endian palette words.
|
||||
<out>_meta.lua sizes, k1/k4, and the expected darkest-entry index.
|
||||
|
||||
The expectation is generated by the SAME module the two decode rigs load
|
||||
through, so this cannot pass by agreeing with a second copy of the maths.
|
||||
"""
|
||||
import sys, argparse
|
||||
sys.path.insert(0, "tools/encoder")
|
||||
sys.path.insert(0, "tools/bench")
|
||||
from dlx import DLX
|
||||
import dlxload as DL
|
||||
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("container")
|
||||
ap.add_argument("--out", default="tmp/load")
|
||||
a = ap.parse_args()
|
||||
|
||||
d = DLX(a.container)
|
||||
if d.version < 3:
|
||||
sys.exit(f"{a.container} is DLX{d.version}: load.i checks for the 'DLX3' magic")
|
||||
if d.idx_bytes != 1:
|
||||
sys.exit("2-byte codebook indices: load.i expands one source byte per pixel")
|
||||
|
||||
off_frm = int.from_bytes(d.raw[28:32], "big")
|
||||
raw = d.raw[:off_frm]
|
||||
|
||||
cb1, cb4 = DL.expand_codebooks(d)
|
||||
palb, dark, _ = DL.pack_palette(d)
|
||||
|
||||
open(a.out + "_data.bin", "wb").write(raw)
|
||||
open(a.out + "_expect.bin", "wb").write(cb1.tobytes() + cb4.tobytes() + palb.tobytes())
|
||||
|
||||
with open(a.out + "_meta.lua", "w") as fh:
|
||||
fh.write("-- generated by tools/bench/prep_load.py -- do not edit\nreturn {\n")
|
||||
fh.write(f" k1={d.k1}, k4={d.k4}, dark={dark},\n")
|
||||
fh.write(f" raw_len={len(raw)}, cb1_len={cb1.nbytes}, cb4_len={cb4.nbytes},\n")
|
||||
fh.write(f" pal_len={palb.nbytes},\n}}\n")
|
||||
|
||||
print(f"{a.container}: k1={d.k1} k4={d.k4}, header region {len(raw)} B "
|
||||
f"(pal 768 + cb1 {d.k1*16} + cb4 {d.k4*4} + 32)")
|
||||
print(f" the 68000 must produce {cb1.nbytes} + {cb4.nbytes} B of expanded "
|
||||
f"codebook and {palb.nbytes} B of palette, darkest entry {dark}")
|
||||
@@ -0,0 +1,73 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Check the 68000's load-time output against tools/bench/dlxload.py, byte for byte.
|
||||
|
||||
python3 tools/bench/verify_load.py <in.dlx> [--out tmp/load]
|
||||
|
||||
The 68000 ran src/player/load.i over the RAW container header; tools/bench/
|
||||
load.lua read the results back out of emulated RAM and out of the PALETTE
|
||||
REGISTERS. This compares them with what the host-side transforms produce.
|
||||
|
||||
Byte-for-byte and not "close enough", for both halves:
|
||||
|
||||
* the codebooks are indices, so a single wrong byte is a wrong COLOUR in
|
||||
every block that uses that codeword, in every frame of the scene.
|
||||
* the palette words carry the shared LSB the encoder's 1.96 dB (FINDINGS
|
||||
23.3) depends on, and a wrong choice of it is invisible in a diff of the
|
||||
picture's SHAPE -- it is a slightly wrong colour, which is exactly the sort
|
||||
of thing that gets attributed to the codec.
|
||||
|
||||
The darkest-entry index is checked too: it is what the letterbox is filled
|
||||
with until the encoder reserves a black entry (23.4, still open), and it comes
|
||||
out of an argmin whose tie-break has to match numpy's -- first index wins.
|
||||
"""
|
||||
import sys, argparse
|
||||
sys.path.insert(0, "tools/encoder")
|
||||
sys.path.insert(0, "tools/bench")
|
||||
from dlx import DLX
|
||||
import dlxload as DL
|
||||
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("container")
|
||||
ap.add_argument("--out", default="tmp/load")
|
||||
ap.add_argument("--log", default="tmp/load_check.log",
|
||||
help="the rig's log, for the DARK= line it printed")
|
||||
a = ap.parse_args()
|
||||
|
||||
d = DLX(a.container)
|
||||
cb1, cb4 = DL.expand_codebooks(d)
|
||||
palb, dark, _ = DL.pack_palette(d)
|
||||
want = cb1.tobytes() + cb4.tobytes() + palb.tobytes()
|
||||
got = open(a.out + "_out.bin", "rb").read()
|
||||
|
||||
if len(got) != len(want):
|
||||
sys.exit(f"FAIL: the 68000 produced {len(got)} B, expected {len(want)}")
|
||||
|
||||
n1, n4 = cb1.nbytes, cb4.nbytes
|
||||
sections = (("CB1", 0, n1), ("CB4", n1, n1 + n4), ("palette", n1 + n4, len(want)))
|
||||
bad = 0
|
||||
for name, lo, hi in sections:
|
||||
diff = [i for i in range(lo, hi) if got[i] != want[i]]
|
||||
if diff:
|
||||
bad += len(diff)
|
||||
i = diff[0]
|
||||
print(f"FAIL: {name}: {len(diff)}/{hi-lo} bytes differ; first at "
|
||||
f"+{i-lo} (68000 {got[i]:#04x}, dlxload {want[i]:#04x})")
|
||||
else:
|
||||
print(f" OK {name}: {hi-lo} B identical to dlxload.py")
|
||||
|
||||
# The rig prints the index the 68000 chose; parse it rather than re-deriving,
|
||||
# so a rig that failed to read LDARK cannot pass by silence.
|
||||
got_dark = None
|
||||
for line in open(a.log, "rb").read().decode("utf-8", "replace").splitlines():
|
||||
if "DARK=" in line:
|
||||
got_dark = int(line.split("DARK=")[1].split()[0].rstrip(","))
|
||||
if got_dark is None:
|
||||
sys.exit("FAIL: the rig printed no DARK= line -- it did not reach the dump")
|
||||
if got_dark != dark:
|
||||
sys.exit(f"FAIL: darkest palette entry: 68000 says {got_dark}, dlxload says {dark}")
|
||||
print(f" OK darkest entry {dark}, chosen by the same argmin tie-break")
|
||||
|
||||
if bad:
|
||||
sys.exit(f"FAIL: {bad} bytes differ in total")
|
||||
print(f"OK the 68000 reproduced all {len(want)} B of load-time output exactly "
|
||||
f"(P1 codebooks, P2 palette)")
|
||||
Reference in New Issue
Block a user