drwily/Dragon-s-Lair-X68k
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Measure the span: the mode survives, and it is an encoder format

Browse Source 
FINDINGS 29 priced a literal-span mode at 4*(50 + 4L*9.08) cycles and labelled
the whole section DERIVED. Session 8 step 0 was to measure it before optimising
over the mode set it implies. Two variants in blit.s, one stream per span length
from prep_spans.py, timed by span.lua, driven by span.sh in ~25 s:

  v5, handed (x, npix) and left to work the copy out:  97.9/span + 10.459/px
  v6, handed an address and a jump displacement:       43.7/span +  9.152/px
  29 assumed                                           50.0/span +  9.080/px

So 29's arithmetic was right about a format nobody had written. The difference
is not tuning: v5 spends ~122 cycles a span computing a destination, dividing
npix into bursts and handling a 0..15 remainder, all of which the encoder knows
at build time. v6's record is {u32 absolute GVRAM address, u16 jump
displacement} into an unrolled chain of 24-pixel copy units -- no loop, no
remainder, no arithmetic -- and it fits 11 span lengths to 0.3%.

Three things that measurement showed and derivation could not:

  - The per-pixel cost is a function of REGISTER PRESSURE. FINDINGS 24's 9.08
    was a fixed blit with 12 registers free; v5 can spare 8 and pays 10.46; v6
    gets 12 back only because the encoder holds the state.
  - Short spans die in the remainder path -- a 12-pixel span costs MORE than a
    16-pixel one -- and the fix is padding, not avoidance.
  - Odd-x alignment is free (259.0 vs 261.8 cycles/span), as a 16-bit bus
    implies but nobody had checked.

Re-priced against the unchanged mode maps, sasi: median 74.4% -> 52.0% (29 said
43.0), misses 37 -> 10/120 (29 said 8), 448.0 KB/s. Break-even moved from runs
of 2 blocks to runs of 4. 29.4 survives: a scene cut needs x >= 0.196 of the
frame as spans and the bus allows x <= 0.373, so it fits at 12fps.

All 23 timing configs are also checked pixel-exact, so none of this was timed
against a decoder that quietly skipped work.

FINDINGS 30. Next: lever B, the cost-aware mode decision.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
This commit is contained in:
prosolis
2026-08-23 15:45:51 -07:00
parent 3641f37e28
commit 29eb78a599
9 changed files with 780 additions and 73 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/FINDINGS.md
+148 -4
View File
@@ -1428,10 +1428,17 @@ without closing it.
## 29. Trading bytes for cycles: the bus has 4x the headroom the CPU has (session 7)
> **STATUS: DERIVED, NOT MEASURED.** No 68000 has executed a span decoder. The
> per-pixel figure it rests on *is* measured (FINDINGS 24 V1) but at full row
> width; the per-span overhead is hand-derived. Treat every number below as a
> hypothesis with a test attached, not as a result. FINDINGS 4 is why.
> **SUPERSEDED IN PART BY 30, which measured it.** The mode survives and the
> conclusion holds, but every number in this section moved: a span costs 43.7
> cycles + 9.152/pixel *only* in an encoder-assisted format (the obvious
> decoder is 97.9 + 10.46), spans beat V1 from runs of 4 blocks and not 2, and
> the re-priced trade-off is 52.0% median / 10 misses, not 43.0% / 8. Read 30's
> tables over 29.3's. 29.5's other three items are still open, and 29.6 stands.
>
> **STATUS AT THE TIME: DERIVED, NOT MEASURED.** No 68000 had executed a span
> decoder. The per-pixel figure it rests on *is* measured (FINDINGS 24 V1) but
> at full row width; the per-span overhead was hand-derived. FINDINGS 4 is why
> it was labelled and then tested rather than believed.
FINDINGS 28 leaves the project CPU-bound while the **bus sits 4x idle**: `sasi`
spends 110 KB/s of a 488 KB/s pipe. That asymmetry is exploitable, because the
@@ -1534,3 +1541,140 @@ It cannot be settled in MAME: like the SCSI/SASI devices (BENCHMARK.md), the
HD63450 is a functional model, so a timing number out of it would measure the
emulator's scheduler. It needs hand-derivation against the datasheet plus real
hardware — the same three-tier approach the disk benchmark already documents.
## 30. The span, measured: the mode survives, and it is an encoder format (session 8)
FINDINGS 29 priced a new decoder mode at `4 * (50 + 4L*9.08)` cycles and marked
the whole section DERIVED. This is the measurement. `tools/bench/blit.s` gained
two span variants, `tools/bench/prep_spans.py` generates one stream per span
length, `tools/bench/span.lua` times them, and `tools/bench/span.sh` runs the
lot, and the whole thing takes about 25 seconds.
Same scope as every 68000 figure since FINDINGS 24: instruction cycles against
MAME's zero-wait-state GVRAM, interrupts masked. A **lower bound**, not a
prediction.
### 30.1 What was measured
Twelve `v5` configs and eleven `v6` configs, each cutting the **same** 256x192
frame into spans of a different length, so the work differs only in how finely
it is cut. Regressing `cycles = A*spans + B*pixels` over a set reads the
per-span overhead and the per-pixel cost straight off.
Every config draws the whole picture, the picture is cleared before each run and
snapshotted after, and all 23 snapshots are checked pixel-exact by
`tools/bench/verify_frame256.py`. A config cannot time fast by writing nothing.
| | per span | per pixel | fit error |
|---|---:|---:|---:|
| **v5** — decoder handed `(x, npix)`, works the copy out | **97.9** | **10.459** | ±1.4%, and only on spans that are a whole number of bursts |
| **v6** — encoder hands it an address and a jump | **43.7** | **9.152** | **±0.3% over all 11 lengths** |
| *29's assumption* | *50.0* | *9.080* | — |
**29's arithmetic was right about a format nobody had written yet.** v6 hits it
almost exactly; v5 — the obvious decoder, and the one 29 was describing — is
2.24x dearer per span and 14% dearer per pixel.
### 30.2 Why the difference is a format difference, not an optimisation
v5's record is `(x, npix)`, so the decoder computes the destination, divides
`npix` into 16-pixel bursts, and handles the 0..15 remainder: about 122 cycles
of arithmetic and branching per span before a single pixel moves. All of it is
known at encode time.
v6's record is `{u32 absolute GVRAM address, u16 jump displacement}` and nothing
else. The displacement jumps into an unrolled chain of eleven 24-pixel copy
units, so a span of any supported length is straight-line code with no loop, no
remainder, and no address arithmetic — `move.l (a0)+,a2` / `move.w (a0)+,d0` /
`jmp v6ch(pc,d0.w)`, then `movem.l` pairs. GVRAM is at $C00000 on every X68000,
so absolute destinations are a legitimate thing to bake into a stream.
Two consequences of that format, both cheap:
- **Span lengths are multiples of 24 pixels** and a run pads up to it. The
padding costs bytes and its own pixels, nothing else, and it is *correct on
screen*: a literal span carries true pixels of the current frame, so painting
a clean neighbour is a no-op visually.
- **A span may overrun the visible 256 pixels of its row by up to 23.** Free:
the line stride is 1024 bytes and only the first 512 are displayed, so the
overrun lands in the invisible half of the line.
### 30.3 The remainder path is where a short span actually dies
v5's cost per span, measured, against its length:
| span | 4 px | 8 px | 12 px | 16 px | 20 px | 24 px | 32 px |
|---|---:|---:|---:|---:|---:|---:|---:|
| cycles/span | 180.3 | 240.9 | 296.3 | 261.8 | 347.7 | 401.9 | 430.7 |
| cycles/pixel | 45.08 | 30.11 | 25.46 | **16.36** | 17.65 | 17.27 | **13.46** |
A 12-pixel span costs *more* than a 16-pixel one. Everything below the 16-pixel
burst width goes through `move.l`/`move.w` at roughly 10 cycles a pixel plus the
per-span overhead, and 29's warning that "short spans are flattered" was
correct — but the fix is to pad them up to a burst, not to avoid them. v6 has no
remainder path at all, which is most of why its fit is linear to 0.3%.
### 30.4 Registers are the reason the per-pixel cost moved
FINDINGS 24's 9.08 cycles/pixel came from a fixed blit with 12 registers free
for `movem.l` and no live state. A span decoder keeps a stream pointer, a
destination and counters live, so v5 can spare only 8 registers per burst — 32
bytes instead of 48 — and pays 10.46 cycles/pixel for it. v6 gets back to 12
registers precisely because the encoder holds the state instead, and lands at
9.152. **The per-pixel figure is a function of how much the decoder has to
remember**, which is not something the FINDINGS 24 measurement could have shown.
Two smaller results, both cheap and both worth having on the record:
- **Odd-`x` alignment is free.** Spans starting at an odd pixel run their bursts
at `addr mod 4 == 2` and cost 259.0 cycles/span against 261.8 aligned — inside
the timing granularity. The 68000's 16-bit bus does not care, as expected;
now it is measured rather than assumed.
- **A full-row span is 154 cycles per 4x4 block**, the floor this mode can
reach, against V1's measured 299.9.
### 30.5 Re-pricing: the trade holds, and it is smaller
`tools/analysis/12_span_tradeoff.py` now runs on measured constants. Same greedy
(buy the best cycles-saved-per-byte until the bus budget is gone), same
unmodified mode maps, same Singe window:
| | today | 29 (derived) | **30 (measured)** |
|---|---:|---:|---:|
| `sasi` median frame | 74.4% | 43.0% | **52.0%** |
| `sasi` worst frame | 136.2% | 106.2% | **108.7%** |
| `sasi` frames missing | 37/120 | 8/120 | **10/120** |
| `sasi` bitrate | 101.7 KB/s | 453.2 | **448.0 KB/s** |
| `scsi` median frame | 94.9% | 69.4% | **74.6%** |
| `scsi` frames missing | 51/120 | 18/120 | **25/120** |
And the break-even moved. Cycles per 4x4 block in a run of L blocks, v6, with
each of the run's 4 spans padded to a whole 24-pixel unit:
| L | 1 | 2 | 4 | 8 | 16 | 64 |
|---|---:|---:|---:|---:|---:|---:|
| cycles/block | 1053 | 527 | **263** | 242 | 176 | 154 |
So a run beats all-V1 (299.9) **from L=4 up**, not from L=2 as 29.3 claimed, and
runs of 1-3 blocks all cost the same 1053 cycles because they pad to the same
single unit. A cost-aware mode decision should not offer a span below 4 blocks
at all.
### 30.6 29.4 survives: a scene cut still fits at 12fps
Mixing a fraction `x` of a 100%-changed frame as full-row spans against V1 for
the rest, on measured costs (154 cycles and 33.4 bytes per block):
- CPU needs `x >= 0.196`
- the 40,977 B/frame bus budget allows `x <= 0.373`
The interval is not empty — narrower than 29.4's 0.19..0.39, same conclusion.
FINDINGS 28.5's "a scene cut cannot fit" was a ceiling of the bitstream, not of
the machine, and that now rests on a measurement. `12_span_tradeoff.py` prints
this arithmetic and will say so if it ever stops being true.
### 30.7 What this does NOT settle
The three remaining items of 29.5 are unchanged and are now **more** load-bearing,
because the measured design runs at 448 KB/s of a 488 KB/s pipe rather than 453:
re-run the ring-buffer simulation at that rate, confirm the 4 Mbps figure's
provenance, and confirm DMA rather than PIO. A PIO fallback would put a 448 KB/s
transfer back on the CPU this mode exists to relieve.
Also unmeasured: **the parse cost of a span-heavy stream**. Every figure here
times the copy. The 68000 also has to read the mode map and dispatch: the
re-priced `sasi` stream buys 8773 spans across 120 frames, a mean of 73 a frame,
and each one's three-instruction dispatch is inside the fitted 43.7 — but the
mode-map walk that decides a span exists is not. `decode.s` does not implement
spans yet.
docs/STATUS.md
+74 -55
View File
@@ -1,60 +1,48 @@
# Status & next-session handoff — end of session 7 (2026-08-23)
# Status & next-session handoff — session 8 (2026-08-23)
## NEXT SESSION: measure a span, then make the mode decision cost-aware
## Where this stands
The decoder exists, it is pixel-exact, and **it does not fit**. On the worst
sustained window at `sasi` it costs a mean of **81.7% of a 12fps frame** and
**31% of frames exceed 100%** (`scsi`: 94.9% median, 42% miss). FINDINGS 28.
CPU is the binding constraint now — the first time in this project.
The decoder exists, it is pixel-exact, and **it does not fit**: mean 81.7% of a
12fps frame on the worst sustained window at `sasi`, 31% of frames over budget
(`scsi`: 94.9% median, 42% miss). FINDINGS 28. CPU is the binding constraint.
**Two levers, and the cheap one has to be measured first.**
Session 7 proposed two levers and session 8 measured the cheap one first.
*Lever A — spend bandwidth to buy cycles.* The bus sits 4x idle: `sasi` uses 110
KB/s of 488. Every codec decision was made when bytes were scarce, so each one
trades cycles to save them, and the cheapest thing a 68000 can be handed is the
most expensive thing to store — **word-expanded pixels in row-linear runs**.
Adding one mode, a per-row span of literal words `movem.l`-ed straight from the
stream buffer into GVRAM, prices out at (FINDINGS 29, `12_span_tradeoff.py`):
**Lever A — spend bandwidth to buy cycles — is real, and it is an encoder
format.** A row-linear span of word-expanded literals measures **43.7 cycles per
span + 9.152 per pixel** (FINDINGS 30, `tools/bench/span.sh`), which is what
FINDINGS 29 assumed — but only when the *encoder* hands the decoder an absolute
GVRAM address and a jump displacement into an unrolled copy chain. The obvious
decoder, handed `(x, npix)` and left to work the copy out, is 97.9 + 10.46 and
2.2x dearer on a short span. Re-priced against the unchanged mode maps:
| | today | + literal spans |
|---|---:|---:|
| median frame | 74.4% | **43.0%** |
| worst frame | 136.2% | **106.2%** |
| frames missing | **37/120** | **8/120** |
| bitrate | 101.7 KB/s | 453.2 KB/s (bus 488) |
| | today | 29 (derived) | **30 (measured)** |
|---|---:|---:|---:|
| `sasi` median frame | 74.4% | 43.0% | **52.0%** |
| `sasi` worst frame | 136.2% | 106.2% | **108.7%** |
| `sasi` frames missing | 37/120 | 8/120 | **10/120** |
| bitrate | 101.7 KB/s | 453.2 | **448.0 KB/s** (bus 488) |
**This is DERIVED, not measured, and it is load-bearing — so measure it first.**
Extend `tools/bench/blit.s` with a span variant and time it against run length.
The 9.08 cycles/pixel it rests on is real (FINDINGS 24 V1) but was measured at
full row width with 12-register bursts; short and oddly-aligned spans cannot
burst as well and are flattered by the model. If spans come in near the derived
figure, the whole mode set changes and lever B optimises over different modes —
which is exactly why this goes first. FINDINGS 29.5 lists the other three things
that have to hold, of which **confirming DMA vs PIO is the cheapest and now the
most consequential**: at 453 KB/s a PIO fallback puts the transfer back on the
CPU this is trying to relieve.
Break-even moved with it: a run beats all-V1 **from 4 blocks up**, not 2. And
29.4 survives — a scene cut needs `x >= 0.196` of the frame as spans and the bus
allows `x <= 0.373`, so it fits at 12fps.
*Lever B — stop buying modes the CPU cannot afford.* `vq_hybrid.decide()`
minimises `D + lam*R` — distortion against BYTES — on a machine whose binding
budget is CYCLES, and the two are not proportional:
**Lever B — stop buying modes the CPU cannot afford — is untouched.**
`vq_hybrid.decide()` still minimises `D + lam*R`, distortion against BYTES, on a
machine whose binding budget is CYCLES:
| mode | payload bytes | measured cycles | cycles per byte |
| mode | payload bytes | cycles | cycles per byte |
|---|---:|---:|---:|
| SKIP | 0 | 13 (clustered) | — |
| V1 | 1 | 300 | 300 |
| V4 | 4 | 448 | 112 |
| RAW | 16 | 400 | 25 |
| *word-expanded literal block* | *32* | *~240 (derived)* | *7.5* |
| **span, per 4x4 block in a run of L** | **32** | **1053/L, floor 154** | **~5** |
V4 is **25% of blocks and 50% of the cycles**. The lagrangian charges it 4x a V1
block; the CPU charges it 1.49x. Note the last row: a literal block is cheaper
than **every** codebook mode, and pixel-exact — the codebook is a byte
optimisation that now costs cycles (FINDINGS 29.2).
V4 is 25% of blocks and 50% of the cycles. The lagrangian charges it 4x a V1
block; the CPU charges it 1.49x.
**The work, in order:**
0. **Measure the span cost on the 68000** (lever A above). Cheap, and everything
below optimises over whatever mode set it leaves.
## The work, in order
1. **Add a cycle term to the mode decision.** `decide()` already builds a cost
matrix of `error + lam * bytes` per mode per block; add `+ mu * cycles`,
@@ -86,10 +74,11 @@ optimisation that now costs cycles (FINDINGS 29.2).
`tools/bench/decode.lua`.
3b. **Know which misses are yours to fix before starting.** Re-coding every
non-SKIP block as V1 is the floor any mode assignment can reach, and it
still misses 11 frames at `sasi` and 12 at `scsi` — every frame above ~90%
non-SKIP. So the cost-aware decision can reach about three quarters of the
misses (26 of 37 at `sasi`) and the rest are item 4. FINDINGS 28.7.
non-SKIP block as V1 is the floor any mode assignment *of the current mode
set* can reach, and it still misses 11 frames at `sasi` and 12 at `scsi` —
every frame above ~90% non-SKIP. So the cost-aware decision can reach about
three quarters of the misses (26 of 37 at `sasi`) and the rest need item 4.
FINDINGS 28.7.
3c. **Buy RAW, not V4, wherever the bytes allow.** RAW is 400 cycles against
V4's 448 *and* is pixel-exact, so on the CPU axis V4 is strictly dominated —
@@ -97,15 +86,18 @@ optimisation that now costs cycles (FINDINGS 29.2).
`sasi` cannot afford it, so expect the cycle ceiling to cost `sasi` more
quality even though it costs `sasi` fewer cycles. FINDINGS 28.8.
4. **Scene cuts: 28.5 said impossible, 29.4 reopened it.** An all-V1 frame —
the cheapest full redraw the *current* mode set allows — is 110.5% of budget,
so no mode assignment fits a 100%-changed frame. With literal spans the
arithmetic changes: CPU needs at least 19% of the frame sent as spans, the
bus allows up to 39%, **and that interval is not empty**. So 28.5 was a
ceiling of the bitstream, not of the machine — *if* lever A measures out.
If it does not, this is still a design decision that needs the user: one late
frame at each cut (the outgoing content is unrelated, so it may be
invisible), a cut spread over two frame times, or 10fps.
4. **Put spans in the bitstream** — the mode is measured and nothing implements
it. This is a container change (`encode.py`, `dlx.py`, `decode.s`), a mode
decision that can see runs rather than blocks, and the 24-pixel quantisation
and row-overrun rules of FINDINGS 30.2. It subsumes item 4 of session 7's
plan: with spans, a scene cut fits.
5. **The three things 30.7 leaves open, now more load-bearing than before**,
because the span design runs at 448 KB/s of a 488 KB/s pipe: re-run the
ring-buffer simulation at that rate (FINDINGS 21 was established at 110 and
280), confirm the provenance of the 4 Mbps figure, and **confirm DMA rather
than PIO** — a PIO fallback puts a 448 KB/s transfer back on the CPU this
whole lever exists to relieve. The DMA check is the cheapest of the three.
**Do not start by hand-optimising `decode.s`.** The hand-derived timings agree
with the measurements to 0.5% on V1 and 1% on RAW (FINDINGS 28.4), so the
@@ -116,6 +108,33 @@ saves ~16 of 448 cycles.
---
## What session 8 settled
1. **The span is measured: 43.7 cycles/span + 9.152/pixel, fitted to 0.3% over
eleven span lengths.** `tools/bench/blit.s` v5/v6, `prep_spans.py`,
`span.lua`, driven by `tools/bench/span.sh` (~25 s, not in `check.sh`
because it is a wall timing). FINDINGS 30.
2. **Only in an encoder-assisted format.** `{u32 absolute GVRAM address, u16
jump displacement}` into an unrolled chain, versus `(x, npix)` and a decoder
that works it out: 43.7 + 9.152 against 97.9 + 10.46. All the arithmetic a
span decoder would do per frame is known at encode time. FINDINGS 30.2.
3. **The per-pixel cost is a function of register pressure**, which FINDINGS 24
could not have shown: 9.08 was a fixed blit with 12 registers free, v5 can
spare 8 and pays 10.46, v6 gets 12 back by making the encoder hold the state.
4. **Short spans die in the remainder path, and the fix is padding.** A 12-pixel
span costs more than a 16-pixel one in v5. v6 has no remainder path: lengths
are multiples of 24 pixels, padding is free of everything but bytes, and an
overrun past the visible 256 lands in the invisible half of the 1024-byte
line stride. FINDINGS 30.3.
5. **Odd-`x` alignment is free** (259.0 vs 261.8 cycles/span) — expected on a
16-bit bus, now measured rather than assumed.
6. **The trade is smaller than 29 derived but the conclusion holds**, including
29.4's reopening of the scene cut. All 23 timing configs also drew a
pixel-exact frame, so nothing here was timed against a decoder that skipped
work. FINDINGS 30.5/30.6.
---
## What session 7 settled
1. **68000 code parses a bitstream and draws frames, pixel-exact.**