Measure the blit on the 68000: the 38% estimate was 53.6%
First 68000 instructions in this project to draw a pixel. Everything before this was GVRAM filled from Lua, which costs zero 68000 cycles, so the blit figure the whole CPU budget rests on had never been validated. Four variants of a full-frame 256x192 paint, timed in MAME and each also hand-derived from the MC68000 timing tables beforehand; the two agree to 0.006-0.43%, which is what makes the result trustworthy after this project's history of false-good measurements. V1 movem.l blit from a word-expanded RAM frame 446,286 cyc 53.6% V2 naive move.b/move.w per pixel 1,284,174 cyc 154.1% V3 write-only floor, no source read 225,789 cyc 27.1% V4 same writes in 4x4 block order 637,971 cyc 76.6% Scope: MAME's gvram_w/gvram_r carry no timing at all, so these are instruction cycles against zero-wait-state memory -- a floor, not a hardware prediction. V1's output snapshots pixel-exact through verify_frame256.py, closing FINDINGS 23.5. The V1/V3 gap shows reading the source frame is exactly half the cost, which makes the architecture question live: decode-direct-to-GVRAM needs no RAM reference frame and scales with the non-SKIP block fraction, crossing compose-then-blit at 70% of blocks changed. That fraction is now the top priority and is already a by-product of vq_hybrid.py's mode decision. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
This commit is contained in:
prosolis
@@ -792,3 +792,101 @@ and the 38% full-frame blit estimate underpinning the CPU budget remains
|
||||
unvalidated. What this section adds is that the *target mode* is now real, so
|
||||
68000 code has a defined geometry to write into: 256 words per visible row, a
|
||||
1024-byte line stride, and rows 32..223 of a 256-row page.
|
||||
|
||||
---
|
||||
|
||||
## 24. The blit, measured on the 68000 — the 38% estimate was wrong (session 5)
|
||||
|
||||
**The first 68000 instructions in this project to draw a pixel.** Everything in
|
||||
22 and 23 was GVRAM filled from Lua, which costs zero 68000 cycles. This section
|
||||
replaces the estimate that the whole CPU budget rested on with a measurement.
|
||||
|
||||
Harness: `tools/bench/blit.s` + `tools/bench/blit.lua`. Four variants of a
|
||||
full-frame 256x192 paint, each looped to run ~4 emulated seconds, timed from
|
||||
`machine.time` between two flag writes by the 68000 itself.
|
||||
|
||||
| variant | what it does | cycles/frame | % of a 12fps frame |
|
||||
|---|---|---:|---:|
|
||||
| **V1** | `movem.l` blit from a word-expanded RAM frame (96KB read + 96KB write) | **446,286** | **53.6%** |
|
||||
| V2 | naive `move.b`/`move.w` per pixel from a byte source | 1,284,174 | 154.1% |
|
||||
| **V3** | write-only floor — registers preloaded, no source read at all | **225,789** | **27.1%** |
|
||||
| **V4** | the same 96KB of writes issued in **4x4 block order** | **637,971** | **76.6%** |
|
||||
|
||||
The 12fps budget is 833,333 cycles (10.0 MHz confirmed from `x68k.cpp:1133`,
|
||||
`40_MHz_XTAL / 4`).
|
||||
|
||||
### 24.1 The numbers are cross-checked against hand-derived cycle counts
|
||||
Every variant was predicted from the MC68000 timing tables *before* the run
|
||||
(`MOVEM.L` M->R `(An)+` = 12+8n, `(d16,An)` = 16+8n; R->M `(An)` = 8+8n,
|
||||
`(d16,An)` = 12+8n) and then measured:
|
||||
|
||||
| | predicted | measured | error |
|
||||
|---|---:|---:|---:|
|
||||
| V1 | 447,744 | 446,286 | 0.33% |
|
||||
| V2 | 1,284,096 | 1,284,174 | 0.006% |
|
||||
| V3 | 225,792 | 225,789 | 0.001% |
|
||||
| V4 | 640,704 | 637,971 | 0.43% |
|
||||
|
||||
This agreement is the point. A MAME timing number on its own would be worth
|
||||
little given how many false-good results this project has produced (FINDINGS 4);
|
||||
two independent derivations landing within half a percent is worth something.
|
||||
The residual error is the frame-granularity of the measurement — Lua gets no
|
||||
cycle counter (`luaengine.cpp` exposes `machine.time` and nothing from
|
||||
`device_execute_interface`), so timing resolution is one video frame, 18.03 ms.
|
||||
|
||||
### 24.2 SCOPE: these are instruction cycles, and therefore a LOWER BOUND
|
||||
MAME's `gvram_w`/`gvram_r` (`x68k_crtc.cpp:501,595`) contain **no timing at
|
||||
all** — no wait states, no `adjust_icount`. GVRAM in MAME is as fast as main
|
||||
RAM. Real X68000 GVRAM stalls the CPU on access, so every figure above is a
|
||||
floor, not a prediction. **Do not quote these as hardware numbers.** Interrupts
|
||||
were masked (`SR = $2700`) so the IPL's timer and VBL handlers could not steal
|
||||
cycles into the measurement; a real player will take interrupts on top.
|
||||
|
||||
### 24.3 The 38% estimate is dead — a full-frame blit is 53.6%
|
||||
The realistic "decode into a RAM frame, then blit it" design costs **53.6% of
|
||||
the frame budget before decoding a single block**, and that is the zero-wait-
|
||||
state floor. The estimate the CPU budget has been carrying since session 1 was
|
||||
38%. It was optimistic by 41%.
|
||||
|
||||
The cause is visible in the V1/V3 gap: **reading the source frame is exactly
|
||||
half the total cost** (221,952 of 446,286 cycles). The 68000 pays 8 cycles per
|
||||
longword read and 8 per longword written, and in 256-colour mode a pixel
|
||||
occupies a whole word of address space, so a frame is 96KB of traffic in each
|
||||
direction rather than 48KB.
|
||||
|
||||
### 24.4 The high byte of every GVRAM write is discarded — confirmed from source
|
||||
`gvram_w` case `0x0100` writes `data & 0x00ff` with `mem_mask 0x00ff`. So in
|
||||
256-colour mode the CPU cannot pack two pixels into one word, and the odd bytes
|
||||
of a word-expanded source frame never need clearing — V1 exploits this by
|
||||
leaving them uninitialised. This is why 96KB, not 48KB, is the irreducible
|
||||
write traffic.
|
||||
|
||||
### 24.5 The architecture question, and where it turns over
|
||||
V4 prices the access pattern a decoder that writes codewords **straight into
|
||||
GVRAM** actually has: 4 rows of 8 bytes at a 1024-byte stride per 4x4 block. The
|
||||
same 96KB of writes costs **76.6%** in block order versus 53.6% row-linear — the
|
||||
stride destroys the `movem.l` burst, 208 cycles per block against a theoretical
|
||||
best of ~150.
|
||||
|
||||
But a decoder never writes every block: SKIP blocks cost **nothing at all**, and
|
||||
the previous frame is already sitting in GVRAM, so **no RAM reference frame is
|
||||
needed for SKIP to work**. So the two designs scale differently:
|
||||
|
||||
- **compose-in-RAM then blit** — flat 53.6%, independent of how much changed
|
||||
- **decode-direct-to-GVRAM** — 76.6% x (fraction of non-SKIP blocks)
|
||||
|
||||
**They cross at 70% of blocks changed.** Below that, writing straight into GVRAM
|
||||
wins, and it also drops the 96KB RAM reference frame entirely. Above it, the
|
||||
flat blit wins.
|
||||
|
||||
**This makes the non-SKIP block fraction the single most important unmeasured
|
||||
number in the project.** It is already computable from the encoder — it is a
|
||||
by-product of the mode decision in `vq_hybrid.py` — and it has never been
|
||||
reported. Measure it before writing any decoder inner loop, because it selects
|
||||
which inner loop to write.
|
||||
|
||||
### 24.6 The frame the 68000 drew is pixel-exact
|
||||
V1's output was snapshotted and passes `verify_frame256.py` unchanged: `256x512
|
||||
native, double-scan exact, active 256x192 pixel-exact, letterbox true black`,
|
||||
40.81 dB. So 68000 code drives the mode of FINDINGS 23 correctly, and 23.5 is
|
||||
now closed.
|
||||
|
||||
+86
-45
@@ -1,4 +1,4 @@
|
||||
# Status & next-session handoff — end of session 4 (2026-08-23)
|
||||
# Status & next-session handoff — end of session 5 (2026-08-23)
|
||||
|
||||
## Start here: is the tree still green?
|
||||
|
||||
@@ -82,6 +82,29 @@ rate-distortion curve, not two codecs.
|
||||
the working-setup section below. They cost ~1.5 h of wall clock and a wedged
|
||||
CPU core, and one of them was hit again this session.
|
||||
|
||||
## What session 5 settled
|
||||
|
||||
1. **68000 code drew a frame, and the blit was measured.** `tools/bench/blit.s`
|
||||
+ `blit.lua`. The snapshot passes `verify_frame256.py` unchanged — pixel-exact
|
||||
in the real 256x256 mode. **FINDINGS 23.5 is closed**: no longer "proven from
|
||||
Lua only".
|
||||
2. **The 38% full-frame blit estimate is dead. It is 53.6%.** And that is a
|
||||
zero-wait-state floor — MAME models no GVRAM wait states, so real hardware is
|
||||
worse. FINDINGS 24. Every variant was hand-derived from the MC68000 timing
|
||||
tables before being measured and the two agree to 0.006-0.43%, so this is not
|
||||
another MAME artefact.
|
||||
3. **Reading the source frame is exactly half the blit cost** (V1 53.6% vs a
|
||||
write-only floor V3 of 27.1%). That is what makes the architecture question
|
||||
below live.
|
||||
4. **The decoder architecture now hinges on one unmeasured number.** Writing
|
||||
codewords straight into GVRAM costs 76.6% of the frame budget for a *full*
|
||||
frame (V4 — the 1024-byte stride kills the `movem.l` burst), but scales with
|
||||
the non-SKIP block fraction and needs **no RAM reference frame at all**,
|
||||
because the previous frame is already in GVRAM. Compose-then-blit is a flat
|
||||
53.6%. **They cross at 70% of blocks changed.** FINDINGS 24.5.
|
||||
|
||||
---
|
||||
|
||||
## What session 4 settled
|
||||
|
||||
1. **A real 256x256 CRTC mode exists and is verified.** `crtc_mode.lua`, derived
|
||||
@@ -212,16 +235,15 @@ functional models, not timing-accurate; a KB/s figure from MAME measures the
|
||||
emulator's scheduler. `docs/BENCHMARK.md` covers the three-tier approach
|
||||
(MAME validates the path, derivation bounds it, real hardware settles it).
|
||||
|
||||
## Display path — VERIFIED (session 3), in a real mode (session 4). CPU path — still unproven.
|
||||
## Display path — VERIFIED (session 3), in a real mode (session 4), by 68000 code (session 5).
|
||||
|
||||
The first real frame is on screen: `docs/images/x68k_first_frame_compare.png`.
|
||||
|
||||
**What this does and does not mean.** The video hardware is genuinely emulated
|
||||
and the render is bit-exact. But GVRAM was filled by a MAME Lua script, not by
|
||||
68000 code — no 68000 instruction has drawn a pixel yet. Lua writes cost zero
|
||||
68000 cycles, so the 38% full-frame blit estimate underpinning the whole CPU
|
||||
budget is still unvalidated. "Verified end to end" applies to the *display*
|
||||
path only. See FINDINGS 22 scope note.
|
||||
**Session 5 closed the gap this paragraph used to describe.** GVRAM is now
|
||||
filled by 68000 instructions and the result is still pixel-exact, and the blit
|
||||
cost is measured rather than estimated: **53.6% of a 12fps frame**, not 38%
|
||||
(FINDINGS 24). The remaining caveat is different and narrower: MAME models
|
||||
**no GVRAM wait states**, so 53.6% is a floor and real hardware is worse.
|
||||
Full write-up in **FINDINGS 22**. Harness: `tools/bench/show_frame.lua` +
|
||||
`tools/bench/prep_frame.py`.
|
||||
|
||||
@@ -262,53 +284,53 @@ SDL_VIDEODRIVER=dummy mame x68000 -bios ipl10 -video soft -window \
|
||||
|
||||
## Next steps, in priority order
|
||||
|
||||
1. **Full-disc survey.** Only 4 clips of 1.2-1.7 s out of 224 streams have been
|
||||
1. **Measure the non-SKIP block fraction.** *(new top priority, session 5)*
|
||||
FINDINGS 24.5: compose-in-RAM-then-blit costs a flat 53.6% of the frame
|
||||
budget; decode-direct-to-GVRAM costs 76.6% x (fraction of blocks that are not
|
||||
SKIP) and needs no RAM reference frame. **They cross at 70%.** Which side of
|
||||
70% the content sits on decides which decoder inner loop to write, so this
|
||||
must come before writing one.
|
||||
**It needs no new machinery** — the mode decision in `vq_hybrid.py` already
|
||||
computes it per frame and simply never reports it. Add the histogram
|
||||
(SKIP / V1 / V4 / RAW counts per frame) to `encode.py` output and run it over
|
||||
the clips already extracted. Report the *distribution*, not the mean: a
|
||||
scene-cut frame is ~100% non-SKIP and a held frame near 0%, and the mean of
|
||||
those two is a number describing no actual frame.
|
||||
|
||||
2. **68000 decoder skeleton**, with the inner loop chosen by (1). Parse `DLX1`,
|
||||
expand codebooks, blit per block mode. The display path is verified *by 68000
|
||||
code* now (FINDINGS 24) and the harness pattern is `tools/bench/blit.s` +
|
||||
`blit.lua`, which already loads code, masks interrupts, times a loop against
|
||||
a flag, and snapshots the result for `verify_frame256.py`. Copy that.
|
||||
Assembler: `tools/vasm/vasmm68k_mot -Fbin -o out.bin in.s`.
|
||||
|
||||
2a. **Re-budget everything against 53.6%, not 38%.** Several downstream figures
|
||||
were derived from the old estimate. The blit alone now eats over half the
|
||||
frame at 12fps in the compose-then-blit design, before any decode, and MAME
|
||||
models no GVRAM wait states so that is a floor. This may reopen questions
|
||||
that were closed against the 38% number — check FINDINGS 17.2's entropy-coding
|
||||
rejection, which was argued as "54% LZ4 with no room beside a 38% blit". The
|
||||
conclusion gets *stronger*, not weaker, but the arithmetic should be restated.
|
||||
|
||||
3. **Full-disc survey.** Only 4 clips of 1.2-1.7 s out of 224 streams have been
|
||||
measured, and 00146 already runs 23% hotter than 00020. A *sustained* action
|
||||
sequence is the one thing that could still break the bitrate. Classify menu
|
||||
vs content first (FINDINGS 13) or the averages are diluted by static menus.
|
||||
**Vectorise `_paint` before this run** — it is a Python per-block loop.
|
||||
2. **68000 decoder skeleton.** Parse `DLX1`, expand codebooks to word-per-pixel,
|
||||
blit SKIP/V1/V4/RAW. Measure real cycles with the existing MAME Lua harness.
|
||||
**Fully unblocked** — the display path is verified (FINDINGS 22) AND the
|
||||
target CRTC mode is now real (FINDINGS 23), so 68000 code has a defined
|
||||
geometry to write into: 256 words per row, 1024-byte line stride, picture in
|
||||
rows 32..223 of a 256-row page. `tools/bench/show_frame256.lua` gives a
|
||||
known-good reference image to diff the 68000's output against. This is what
|
||||
validates the 38% full-frame blit estimate the whole CPU budget rests on.
|
||||
**This is now the top priority** — it is the only remaining unknown that can
|
||||
still invalidate the design.
|
||||
Pairs naturally with (1): the same run produces both numbers.
|
||||
|
||||
**Concrete first step, deliberately smaller than "write the decoder":** do
|
||||
not start by parsing `DLX1`. Start by making 68000 code do the dumbest
|
||||
possible full-frame blit — copy 256x192 bytes from RAM to GVRAM through the
|
||||
mode set by `crtc_mode.lua` — and time it with the existing Lua harness.
|
||||
That single number either confirms or kills the 38% estimate, and it needs
|
||||
no bitstream, no codebooks, and no container parsing. `show_frame256.lua`
|
||||
already produces the exact reference image to diff the result against, and
|
||||
`verify_frame256.py` already knows how to check it. Only once that number is
|
||||
in hand is it worth writing the mode dispatch.
|
||||
|
||||
Assembler: `tools/vasm/vasmm68k_mot -Fbin -o out.bin in.s`. The harness
|
||||
pattern for loading and running 68000 code is in `tools/bench/one.lua` and
|
||||
`tools/bench/bench.lua` (working, from session 1).
|
||||
2a. ~~CRTC mode table for 256x192-in-256x256.~~ **DONE, session 4.** Derived from
|
||||
the CRTC divisor ladder (not recalled), verified by snapshot, pixel-exact.
|
||||
`tools/bench/crtc_mode.lua`; write-up in FINDINGS 23; regression test
|
||||
`tools/bench/verify_frame256.py`. Untested on real hardware, but the blanking
|
||||
timing is identical to the IPL's 768 mode, which is what a monitor cares about.
|
||||
|
||||
3. **Wire rate control into `encode.py`.** No longer a blocker (FINDINGS 21), but
|
||||
4. **Wire rate control into `encode.py`.** No longer a blocker (FINDINGS 21), but
|
||||
it is what gives a deterministic ceiling over content not yet measured, which
|
||||
was the original reason for choosing VQ. Insurance, not a fix. Pairs with (1).
|
||||
4. **Confirm DMA vs PIO in MAME** (see the benchmark section above) — cheap, and
|
||||
5. **Confirm DMA vs PIO in MAME** (see the benchmark section above) — cheap, and
|
||||
the only thing that could still move CPU into the binding position.
|
||||
5. **Resolve the framing question** (FINDINGS 12: crop vs squash vs wide).
|
||||
6. **Resolve the framing question** (FINDINGS 12: crop vs squash vs wide).
|
||||
Needs an eyeball against arcade reference, not a measurement.
|
||||
6. **Import the scene graph.** SNES project `data/events/` (MIT, cleared),
|
||||
7. **Import the scene graph.** SNES project `data/events/` (MIT, cleared),
|
||||
cross-checked against DirkSimple (zlib) which transcribed the same data
|
||||
independently — diff them to catch transcription errors before committing
|
||||
any of it to 68000 tables.
|
||||
7. **ADPCM audio.** MSM6258, 15.6kHz mono, 7.8 KB/s — already budgeted in
|
||||
8. **ADPCM audio.** MSM6258, 15.6kHz mono, 7.8 KB/s — already budgeted in
|
||||
`ratectl.py`, not yet extracted or encoded.
|
||||
|
||||
### Explicitly abandoned — do not re-propose
|
||||
@@ -321,8 +343,9 @@ SDL_VIDEODRIVER=dummy mame x68000 -bios ipl10 -video soft -window \
|
||||
- ~~Flat 4x4 VQ.~~ Rejected by eye (FINDINGS 9).
|
||||
|
||||
## Not yet started
|
||||
- **Any 68000 player code.** `src/player/` is still empty. The display path is
|
||||
proven, but proven *from Lua* — no 68000 instruction has yet drawn a pixel.
|
||||
- **Any 68000 player code.** `src/player/` is still empty. 68000 code has now
|
||||
drawn a frame, but it lives in `tools/bench/blit.s` as a benchmark, not in a
|
||||
player: it does no bitstream parsing, no mode dispatch, no codebook expansion.
|
||||
- ADPCM audio extraction/encoding
|
||||
- Disk image packaging
|
||||
- Game logic (scene branching, input windows, death clips)
|
||||
@@ -384,3 +407,21 @@ with `extract.py`; the earlier ones lived in `/tmp` and do not survive a reboot.
|
||||
Stern's scene boundaries the way `DRAGONS_LAIR.iso` is, so the footage would
|
||||
have to be sourced and cut to match. Not to be started until the CPU path is
|
||||
proven — it changes nothing about whether this design works.
|
||||
|
||||
## Reproducing the blit measurement (session 5)
|
||||
|
||||
```
|
||||
python3 tools/encoder/extract.py 00020 tmp/fr_00020 12 crop
|
||||
python3 tools/bench/prep_frame.py tmp/fr_00020 tmp/frame256.bin 0 --reserve-black
|
||||
tools/vasm/vasmm68k_mot -Fbin -o tmp/blit.bin tools/bench/blit.s
|
||||
mkdir -p tmp/snap_blit && cd tmp && SDL_VIDEODRIVER=dummy timeout -k 5 900 mame x68000 \
|
||||
-bios ipl10 -video soft -window -sound none -nothrottle -plugins \
|
||||
-autoboot_script ../tools/bench/blit.lua \
|
||||
-snapshot_directory ./snap_blit -snapview native -seconds_to_run 120
|
||||
```
|
||||
~25 s wall. Prints cycles/frame and % of a 12fps budget for V1-V4, and snapshots
|
||||
V1's output. To check that snapshot is still pixel-exact:
|
||||
`sed 's|snap256|snap_blit|' tools/bench/verify_frame256.py | python3 -`
|
||||
|
||||
Not added to `check.sh`: `check.sh` asserts pixel-exactness, and asserting wall
|
||||
timings there would make the green-light check sensitive to host load.
|
||||
|
||||
Reference in New Issue
Block a user