Build v7 into the player, and find the cost model 18% wrong on the block it made commonest
src/player/decode.s now paints v7 literal spans, pixel-exact under MAME and px68k's C68K core over a container where every frame carries 128-216 spans covering up to 38% of the picture. The span pass is blit.s v7 verbatim: the 66.0/9.143/9.978 fit was measured on that instruction sequence. The container is DLX3 -- a span section between the mode header and the block payload, since that is the only place the 68000 can reach without first parsing something of variable length. 16_span_roundtrip.py gates it in check.sh, and asserts it emitted enough spans to have tested anything. Two synthetic all-SPAN anchors price v7 inside decode.s at 151.2 and 225.6 clocks per 4x4 block, against FINDINGS 40's table of 151 and 226 -- 0.2% on both emulators. The measured mode costs what it was said to cost. Two things that were not on the list: TWO BYTE BUDGETS. FINDINGS 40's 18/120 was scored against the 488 KB/s PIPE, not the 280 KB/s profile, and at the profile rate the lam search has already spent the allowance -- spans fired on 5 frames of 120 and looked like a regression. The profile is a chosen quality rate point; the pipe is hardware. --kbps and --span-kbps are now separate and spans run before mu, because a span pays in bytes and mu pays in picture. Delivered: 86/120 over budget without spans, 77/120 at the profile budget, 34/120 on the pipe for +0.36 dB. C_SKIP_MIXED WAS NEVER MEASURED, and it was 18% low -- 45.0, now 55.0. It is the one constant in the table that came from a derivation, because the synthetic frame that would measure it cannot exist: a byte needs a coded block for its SKIP to be mixed. Four bracketing anchors measure it on both emulators with the header byte rotated through all four positions, and the partner mode solves back to its own anchored value to 0.2%. With it corrected the model predicts a real spanned decode to -0.06% mean / 0.09% worst, against -2.99% / 4.30%. It matters because a span marks its run SKIP, so mixed SKIPs dominate exactly the frames spans are judged on. Also: the rig had been writing its synthetic timing frames 26 KB past the top of a 2 MB machine, and got away with it because the modes it overran are data-independent. A span's jump displacements come out of the stream, so it is not. And frames-over-budget is no longer a safe headline -- the controller aims at the deadline, so 55 of 120 frames sit within 5% of it and a 1% cost shift moves 22 frames. FINDINGS 41. check.sh ALL GREEN, now gating on a span-heavy DLX3 container. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
This commit is contained in:
+83
-4
@@ -1,5 +1,5 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Reference DLX1 reader/decoder -- the ground truth the 68000 player is checked against.
|
||||
"""Reference DLX reader/decoder -- the ground truth the 68000 player is checked against.
|
||||
|
||||
This is deliberately a *decoder*, not a re-run of the encoder: it parses the
|
||||
container byte-for-byte the way `src/player/` must, so that any disagreement
|
||||
@@ -12,9 +12,19 @@ Everything is big-endian (see the `encode.py` docstring). Block raster order,
|
||||
|
||||
V4 sub-block order is (sub_y, sub_x) row-major -- TL, TR, BL, BR -- matching
|
||||
`vq_hybrid.paint`'s reshape(-1,2,2,2,2).transpose(0,1,3,2,4).
|
||||
|
||||
DLX3 adds the v7 LITERAL SPAN section between the mode header and the block
|
||||
payload (FINDINGS 40, tools/encoder/spans.py). A spanned block reads SKIP in
|
||||
the mode header and is painted by a span instead, so a reader that ignores the
|
||||
section does not merely lose the spans -- it displays stale pixels wherever one
|
||||
was. The section is at a KNOWN offset (header end) rather than behind the
|
||||
block payload precisely so that the 68000 can paint it before it has parsed
|
||||
anything of variable length.
|
||||
"""
|
||||
import struct
|
||||
import os, sys, struct
|
||||
import numpy as np
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
import spans as SP
|
||||
|
||||
MODE_SKIP, MODE_V1, MODE_V4, MODE_RAW = 0, 1, 2, 3
|
||||
|
||||
@@ -28,10 +38,11 @@ class DLX:
|
||||
# (FINDINGS 28.3), so the padding is part of the format, not a loader
|
||||
# convenience -- but DLX1 containers stay readable, because every
|
||||
# measurement in FINDINGS 28-31 was taken on one.
|
||||
if b[:4] not in (b"DLX1", b"DLX2"):
|
||||
if b[:4] not in (b"DLX1", b"DLX2", b"DLX3"):
|
||||
raise ValueError(f"{path}: not a DLX container")
|
||||
self.version = int(b[3:4])
|
||||
self.aligned = self.version >= 2
|
||||
self.has_spans = self.version >= 3
|
||||
(self.W, self.H, self.fps, self.nframes,
|
||||
self.k1, self.k4) = struct.unpack(">HHHHHH", b[4:16])
|
||||
off_pal, off_cb1, off_cb4, off_frm = struct.unpack(">IIII", b[16:32])
|
||||
@@ -72,6 +83,60 @@ class DLX:
|
||||
m = np.stack([(h >> 6) & 3, (h >> 4) & 3, (h >> 2) & 3, h & 3], axis=1)
|
||||
return m.reshape(-1)[:self.nb].copy()
|
||||
|
||||
def spans(self, f):
|
||||
"""The frame's literal spans: (list of (y, x, pixels), payload offset).
|
||||
|
||||
Layout, big-endian, at the end of the mode header (DLX3 only):
|
||||
u16 nspans
|
||||
nspans * { u32 GVRAM address, u16 coarse disp, c*48 B pixels,
|
||||
u16 fine disp, f*4 B pixels }
|
||||
The displacements are JUMP offsets into the decoder's two unrolled copy
|
||||
chains, so the pixel counts are read back out of them -- which is the
|
||||
strongest available check that the encoder and `blit.s` agree about the
|
||||
chain geometry, because a wrong displacement lands mid-chain and paints
|
||||
the wrong number of pixels rather than failing loudly.
|
||||
"""
|
||||
o, n = self.frames[f]
|
||||
p = o + self.mode_bytes
|
||||
if not self.has_spans:
|
||||
return [], p
|
||||
b = self.raw
|
||||
(ns,) = struct.unpack(">H", b[p:p + 2])
|
||||
p += 2
|
||||
out = []
|
||||
for _ in range(ns):
|
||||
addr, cd = struct.unpack(">IH", b[p:p + 6])
|
||||
p += 6
|
||||
c = SP.COARSE_N - cd // SP.COARSE_CODE
|
||||
if cd % SP.COARSE_CODE or not 0 <= c <= SP.COARSE_N:
|
||||
raise ValueError(f"frame {f}: coarse displacement {cd} is not "
|
||||
f"an entry point in an {SP.COARSE_N}-unit chain")
|
||||
px = list(np.frombuffer(b, ">u2", c * SP.COARSE_PX, p) & 0xFF)
|
||||
p += c * SP.COARSE_PX * 2
|
||||
(fd,) = struct.unpack(">H", b[p:p + 2])
|
||||
p += 2
|
||||
fu = SP.FINE_N - fd // SP.FINE_CODE
|
||||
if fd % SP.FINE_CODE or not 0 <= fu <= SP.FINE_N:
|
||||
raise ValueError(f"frame {f}: fine displacement {fd} is not "
|
||||
f"an entry point in a {SP.FINE_N}-unit chain")
|
||||
px += list(np.frombuffer(b, ">u2", fu * SP.FINE_PX, p) & 0xFF)
|
||||
p += fu * SP.FINE_PX * 2
|
||||
a = addr - SP.GVRAM
|
||||
y, x = divmod(a, SP.STRIDE)
|
||||
y -= SP.YOFF
|
||||
if x % 2 or not (0 <= y < self.H) or not (0 <= x // 2 < self.W):
|
||||
raise ValueError(f"frame {f}: span destination {addr:#x} is "
|
||||
f"not a pixel of the {self.W}x{self.H} picture")
|
||||
# blit.s tolerates a span running past the visible 256 pixels (the
|
||||
# line stride is 1024 bytes and only the first 512 are displayed),
|
||||
# but nothing an encoder emits should need to: a span is a run of
|
||||
# whole blocks. numpy would truncate it here in silence.
|
||||
if x // 2 + len(px) > self.W:
|
||||
raise ValueError(f"frame {f}: span at ({x//2},{y}) of "
|
||||
f"{len(px)} px overruns the picture width")
|
||||
out.append((y, x // 2, np.array(px, np.uint8)))
|
||||
return out, p
|
||||
|
||||
def blocks(self, f):
|
||||
"""Decoded 4x4 palette-index blocks for the non-SKIP blocks of frame f.
|
||||
|
||||
@@ -82,7 +147,8 @@ class DLX:
|
||||
"""
|
||||
mode = self.modes(f)
|
||||
o, n = self.frames[f]
|
||||
p, end = o + self.mode_bytes, o + n
|
||||
_, p = self.spans(f)
|
||||
end = o + n
|
||||
ib, out = self.idx_bytes, {}
|
||||
b = self.raw
|
||||
for i, mo in enumerate(mode):
|
||||
@@ -115,6 +181,19 @@ class DLX:
|
||||
for i, blk in blks.items():
|
||||
by, bx = divmod(i, self.nbx)
|
||||
canvas[by * 4:by * 4 + 4, bx * 4:bx * 4 + 4] = blk
|
||||
sp, _ = self.spans(f)
|
||||
for y, x, pix in sp:
|
||||
# A span paints blocks the mode header calls SKIP. If it ever
|
||||
# overlaps a coded block the two disagree about the same pixels and
|
||||
# the 68000's answer depends on which it does last -- so this is a
|
||||
# format invariant, not a courtesy check.
|
||||
b0, b1 = x // 4, -(-(x + len(pix)) // 4)
|
||||
bad = [b for b in range(b0, b1)
|
||||
if mode[(y // 4) * self.nbx + b] != MODE_SKIP]
|
||||
if bad:
|
||||
raise ValueError(f"frame {f}: span at ({x},{y}) covers "
|
||||
f"non-SKIP block(s) {bad} of block row {y//4}")
|
||||
canvas[y, x:x + len(pix)] = pix
|
||||
return mode
|
||||
|
||||
def decode_all(self):
|
||||
|
||||
Reference in New Issue
Block a user