drwily/Dragon-s-Lair-X68k
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Build v7 into the player, and find the cost model 18% wrong on the block it made commonest

Browse Source 
src/player/decode.s now paints v7 literal spans, pixel-exact under MAME and
px68k's C68K core over a container where every frame carries 128-216 spans
covering up to 38% of the picture. The span pass is blit.s v7 verbatim: the
66.0/9.143/9.978 fit was measured on that instruction sequence.

The container is DLX3 -- a span section between the mode header and the block
payload, since that is the only place the 68000 can reach without first parsing
something of variable length. 16_span_roundtrip.py gates it in check.sh, and
asserts it emitted enough spans to have tested anything.

Two synthetic all-SPAN anchors price v7 inside decode.s at 151.2 and 225.6
clocks per 4x4 block, against FINDINGS 40's table of 151 and 226 -- 0.2% on
both emulators. The measured mode costs what it was said to cost.

Two things that were not on the list:

TWO BYTE BUDGETS. FINDINGS 40's 18/120 was scored against the 488 KB/s PIPE,
not the 280 KB/s profile, and at the profile rate the lam search has already
spent the allowance -- spans fired on 5 frames of 120 and looked like a
regression. The profile is a chosen quality rate point; the pipe is hardware.
--kbps and --span-kbps are now separate and spans run before mu, because a span
pays in bytes and mu pays in picture. Delivered: 86/120 over budget without
spans, 77/120 at the profile budget, 34/120 on the pipe for +0.36 dB.

C_SKIP_MIXED WAS NEVER MEASURED, and it was 18% low -- 45.0, now 55.0. It is
the one constant in the table that came from a derivation, because the
synthetic frame that would measure it cannot exist: a byte needs a coded block
for its SKIP to be mixed. Four bracketing anchors measure it on both emulators
with the header byte rotated through all four positions, and the partner mode
solves back to its own anchored value to 0.2%. With it corrected the model
predicts a real spanned decode to -0.06% mean / 0.09% worst, against -2.99% /
4.30%. It matters because a span marks its run SKIP, so mixed SKIPs dominate
exactly the frames spans are judged on.

Also: the rig had been writing its synthetic timing frames 26 KB past the top
of a 2 MB machine, and got away with it because the modes it overran are
data-independent. A span's jump displacements come out of the stream, so it is
not. And frames-over-budget is no longer a safe headline -- the controller aims
at the deadline, so 55 of 120 frames sit within 5% of it and a 1% cost shift
moves 22 frames.

FINDINGS 41. check.sh ALL GREEN, now gating on a span-heavy DLX3 container.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
This commit is contained in:
prosolis
2026-08-23 20:02:03 -07:00
parent c520a89e14
commit b49bbdc939
16 changed files with 1342 additions and 203 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tools/bench/prep_dlx.py
+104 -10
View File
@@ -28,6 +28,7 @@ import sys, os, argparse
sys.path.insert(0, "tools/encoder")
import numpy as np
from dlx import DLX
import spans as SP
# The harness loads the WHOLE container into emulated RAM at STREAM=0x30000 and
# the target is a stock 2 MB machine, so there is a hard ceiling on how much of
@@ -54,6 +55,14 @@ a = ap.parse_args()
d = DLX(a.container)
if d.idx_bytes != 1:
sys.exit("2-byte codebook indices: decode.s assumes 1 (k<=256)")
# decode.s reads a u16 span count out of every frame record (FINDINGS 41), so a
# DLX2 container is not merely span-less to it -- the first two bytes of the
# block payload would be read as a count and the frame would decode as garbage.
# Fail here rather than there.
if not d.has_spans:
sys.exit(f"{a.container} is DLX{d.version}: src/player/decode.s expects the "
f"DLX3 span section. Re-encode (tools/encoder/encode.py emits DLX3 "
f"by default) or pass --spans off and use an older decoder.")
# --- codebooks, expanded to one WORD per pixel (high byte is discarded by
# gvram_w, so it is left zero and never has to be cleared)
@@ -72,6 +81,69 @@ palb = np.zeros((256, 2), np.uint8)
palb[:, 0], palb[:, 1] = words >> 8, words & 0xFF
dark = int(((render(I).astype(int)) ** 2).sum(1).argmin())
def build_synth(d):
"""The synthetic timing frames, as record bodies.
No real frame is all one mode, but the mix is exactly what the "76.6% x
non-SKIP fraction" model of FINDINGS 24.5 assumes away: it prices every
non-SKIP block as one V1-style burst. These price the modes separately,
which is the only way to see which one is expensive.
Every record carries the DLX3 span section, empty or not -- decode.s reads a
u16 count out of all of them, and a synthetic frame that omitted it would
desync the bitstream exactly where the harness is least likely to look.
The last two are the mode the block loop cannot express: a frame that is ALL
SPAN, its mode header entirely SKIP. Two run lengths, because a span costs
per-span plus per-pixel and one length cannot separate them --
all-SPAN-64 full-row runs, the floor of the mode (154 clocks/block,
FINDINGS 30.4)
all-SPAN-4 4-block runs, the break-even against V1 (FINDINGS 40.1)
They price v7 INSIDE decode.s against the constants tools/bench/span.sh
fitted in blit.s. Agreement cross-checks both; disagreement means the
decoder's span pass is not the sequence that was measured.
"""
out = {}
empty = SP.serialise([])
for name, mo, per in (("all-SKIP", 0, 0), ("all-V1", 1, 1),
("all-V4", 2, 4), ("all-RAW", 3, 16)):
out[name] = (bytes([mo * 0x55] * d.mode_bytes) + empty
+ bytes(d.nb * per))
# MIXED-SKIP frames. Every other synthetic frame here is a pure population,
# which is exactly why none of them prices the commonest block in a real
# container: a SKIP that shares its header byte with a coded block, and so
# cannot take the all-SKIP fast path. vq_hybrid's C_SKIP_MIXED has never
# been measured -- it was derived -- and a spanned container is made mostly
# of them, because a spanned block reads SKIP. FINDINGS 41.5.
#
# Two mixes per coded mode, because one equation cannot separate the SKIP
# cost from the cost of the block it shares a group with.
#
# THE HEADER BYTES ROTATE, and that is not decoration. decode.s reaches a
# block's 2 mode bits with `lsr.b #6/#4/#2` and no shift at all for the last
# one, so a block costs 52/48/44/34 clocks of dispatch depending on WHERE in
# its header byte it sits. A fixed byte like 0x01 puts every SKIP at the
# three expensive positions and every V1 at the free one, and solving two
# such equations returns a number that describes no real frame. Cycling the
# byte through the four rotations puts each mode at each position equally,
# which is what a real mode map does.
for nm, bys, per in (("mix-3SKIP-V1", (0x01, 0x04, 0x10, 0x40), 1),
("mix-1SKIP-3V1", (0x54, 0x51, 0x45, 0x15), 1),
("mix-3SKIP-RAW", (0x03, 0x0C, 0x30, 0xC0), 16),
("mix-1SKIP-3RAW", (0xFC, 0xF3, 0xCF, 0x3F), 16)):
hdr = bytes(bys[i % 4] for i in range(d.mode_bytes))
ncoded = sum(bin(b).count("1") and
sum(1 for k in range(4) if (b >> (2 * k)) & 3) for b in hdr[:1])
ncoded = sum(sum(1 for k in range(4) if (b >> (2 * k)) & 3) for b in hdr)
out[nm] = hdr + empty + bytes(ncoded * per)
pat = np.tile(np.arange(d.W, dtype=np.uint8), (d.H, 1))
for name, blocks in (("all-SPAN-64", d.W // 4), ("all-SPAN-4", 4)):
sp = [(y, x, pat[y, x:x + blocks * 4])
for y in range(d.H) for x in range(0, d.W, blocks * 4)]
out[name] = bytes(d.mode_bytes) + SP.serialise(sp)
return out
# --- frame stream: [u32 len][modes][payload] per frame, each record start
# rounded up to a 4-byte boundary.
#
@@ -85,6 +157,17 @@ dark = int(((render(I).astype(int)) ** 2).sum(1).argmin())
budget = a.ram - STREAM_BASE - MARGIN
stream, rec_off, pad = bytearray(), [], 0
dropped = 0
# The synthetic timing frames are built FIRST, so their size comes out of the
# RAM budget rather than being appended past it. It used to be appended: the
# stream ran 26 KB beyond the top of a 2 MB machine, which was survivable only
# because the modes it overran are data-independent -- their cost is in the
# mode header, and reading junk payload costs the same as reading pixels. A
# span is not: its two jump DISPLACEMENTS come out of the stream, so an
# out-of-RAM span record jumps into open bus. FINDINGS 41.4.
SYNTH = build_synth(d)
budget -= sum(4 + len(b) + 3 for b in SYNTH.values())
for (o, n) in d.frames:
while len(stream) % 4:
stream += b"\0"; pad += 1
@@ -101,21 +184,26 @@ if dropped:
f" This is the TEST RIG's limit, not the player's -- the player "
f"streams into a ring buffer.")
# Synthetic single-mode frames. No real frame is all one mode, but the mix is
# exactly what the "76.6% x non-SKIP fraction" model of FINDINGS 24.5 assumes
# away: it prices every non-SKIP block as one V1-style burst. These four price
# the modes separately, which is the only way to see which one is expensive.
# Append the synthetic frames the budget above already reserved.
synth = {}
for name, mo, per in (("all-SKIP", 0, 0), ("all-V1", 1, 1),
("all-V4", 2, 4), ("all-RAW", 3, 16)):
for name, body in SYNTH.items():
while len(stream) % 4:
stream += b"\0"; pad += 1
synth[name] = len(stream)
hdr = bytes([mo * 0x55] * d.mode_bytes)
stream += (d.mode_bytes + d.nb * per).to_bytes(4, "big") + hdr + bytes(d.nb * per)
stream += len(body).to_bytes(4, "big") + body
assert STREAM_BASE + len(stream) <= a.ram, (
f"stream ends at 0x{STREAM_BASE+len(stream):X}, past the 0x{a.ram:X} top "
f"of RAM -- the budget arithmetic above is wrong")
# --- timing anchors: the distribution, not its mean (FINDINGS 25.6's lesson)
#
# A spanned block reads SKIP here, so this fraction is the BLOCK-LOOP workload
# and no longer the frame's whole cost: the span section is the rest of it. The
# anchors still pick out the extremes of the block loop, which is what they are
# for, but a frame's total decode time now has two terms.
ns = np.array([100 * (d.modes(i) != 0).mean() for i in range(NFRAMES)])
nsp = np.array([len(d.spans(i)[0]) for i in range(NFRAMES)])
spx = np.array([sum(len(p) for _, _, p in d.spans(i)[0]) for i in range(NFRAMES)])
order = np.argsort(ns)
pick = {
"min non-SKIP %.1f%%" % ns[order[0]]: int(order[0]),
@@ -124,9 +212,11 @@ pick = {
"max non-SKIP %.1f%%" % ns[order[-1]]: int(order[-1]),
}
anchors = [(n, rec_off[i], float(ns[i])) for n, i in pick.items()]
for name in ("all-SKIP", "all-V1", "all-V4", "all-RAW"):
for name in ("all-SKIP", "all-V1", "all-V4", "all-RAW",
"all-SPAN-64", "all-SPAN-4", "mix-3SKIP-V1", "mix-1SKIP-3V1",
"mix-3SKIP-RAW", "mix-1SKIP-3RAW"):
anchors.append((f"synthetic {name}", synth[name],
0.0 if name == "all-SKIP" else 100.0))
0.0 if name.startswith(("all-SKIP", "all-SPAN")) else 100.0))
blob = cb1.tobytes() + cb4.tobytes() + palb.tobytes() + bytes(stream)
open(a.out + "_data.bin", "wb").write(blob)
@@ -147,6 +237,10 @@ print(f" cb1 {cb1.nbytes} B + cb4 {cb4.nbytes} B expanded, palette {palb.nbytes
f"stream {len(stream)} B -> {a.out}_data.bin ({len(blob)} B)")
print(f" non-SKIP blocks/frame: median {np.median(ns):.1f}% "
f"p90 {np.percentile(ns,90):.1f}% max {ns.max():.1f}%")
print(f" v7 spans/frame: median {np.median(nsp):.0f} max {nsp.max()} "
f"({int((nsp>0).sum())}/{NFRAMES} frames); pixels painted by one: "
f"median {100*np.median(spx)/(d.W*d.H):.1f}% "
f"max {100*spx.max()/(d.W*d.H):.1f}% of the picture")
print(f" darkest palette entry: index {dark} -> {tuple(render(I)[dark])}")
# A DLX2 container already carries this padding (FINDINGS 28.3 closed, session
# 9), so the realignment above re-derives bytes that were already there and the