prosolis
b49bbdc939
src/player/decode.s now paints v7 literal spans, pixel-exact under MAME and px68k's C68K core over a container where every frame carries 128-216 spans covering up to 38% of the picture. The span pass is blit.s v7 verbatim: the 66.0/9.143/9.978 fit was measured on that instruction sequence. The container is DLX3 -- a span section between the mode header and the block payload, since that is the only place the 68000 can reach without first parsing something of variable length. 16_span_roundtrip.py gates it in check.sh, and asserts it emitted enough spans to have tested anything. Two synthetic all-SPAN anchors price v7 inside decode.s at 151.2 and 225.6 clocks per 4x4 block, against FINDINGS 40's table of 151 and 226 -- 0.2% on both emulators. The measured mode costs what it was said to cost. Two things that were not on the list: TWO BYTE BUDGETS. FINDINGS 40's 18/120 was scored against the 488 KB/s PIPE, not the 280 KB/s profile, and at the profile rate the lam search has already spent the allowance -- spans fired on 5 frames of 120 and looked like a regression. The profile is a chosen quality rate point; the pipe is hardware. --kbps and --span-kbps are now separate and spans run before mu, because a span pays in bytes and mu pays in picture. Delivered: 86/120 over budget without spans, 77/120 at the profile budget, 34/120 on the pipe for +0.36 dB. C_SKIP_MIXED WAS NEVER MEASURED, and it was 18% low -- 45.0, now 55.0. It is the one constant in the table that came from a derivation, because the synthetic frame that would measure it cannot exist: a byte needs a coded block for its SKIP to be mixed. Four bracketing anchors measure it on both emulators with the header byte rotated through all four positions, and the partner mode solves back to its own anchored value to 0.2%. With it corrected the model predicts a real spanned decode to -0.06% mean / 0.09% worst, against -2.99% / 4.30%. It matters because a span marks its run SKIP, so mixed SKIPs dominate exactly the frames spans are judged on. Also: the rig had been writing its synthetic timing frames 26 KB past the top of a 2 MB machine, and got away with it because the modes it overran are data-independent. A span's jump displacements come out of the stream, so it is not. And frames-over-budget is no longer a safe headline -- the controller aims at the deadline, so 55 of 120 frames sit within 5% of it and a 1% cost shift moves 22 frames. FINDINGS 41. check.sh ALL GREEN, now gating on a span-heavy DLX3 container. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
108 lines
4.9 KiB
Python
108 lines
4.9 KiB
Python
#!/usr/bin/env python3
|
|
"""GATE for the DLX3 span container: does the reference decoder reproduce the
|
|
encoder's own reconstruction, from the emitted bytes?
|
|
|
|
python3 tools/analysis/16_span_roundtrip.py [frames_dir] [--kbps 488]
|
|
|
|
Exits non-zero if any frame differs by a single pixel.
|
|
|
|
WHY THIS EXISTS SEPARATELY FROM 09. `09_ratectl_drift.py` replays SKIP
|
|
semantics in Python against the mode maps the encoder returned; it never reads
|
|
a container. A span breaks exactly that shortcut: a spanned block reads SKIP
|
|
in the mode header and is painted by the span section instead, so a replay that
|
|
knows only about mode maps reports drift where there is none, and -- far worse
|
|
-- a container whose span section is malformed would still pass, because 09
|
|
never parses one. This gate closes that: encode, WRITE THE CONTAINER, read it
|
|
back with tools/encoder/dlx.py (the byte-for-byte reference decoder the 68000
|
|
is checked against), and compare to what ratectl recorded.
|
|
|
|
It also has to prove it tested something. A round-trip over a container with
|
|
no spans in it is green by vacuity, which is the failure mode FINDINGS 40.6
|
|
named for the snapshot count: a gate must take its expected work from the
|
|
generated artefact, not from an assumption. So the thresholds below are
|
|
asserted, not printed.
|
|
|
|
The `--kbps` default is the BUS rate, not the `scsi` profile's 280: spans are
|
|
bought with bytes, and 14_dmac_chain.py scores them against the 488 KB/s pipe.
|
|
At the profile rate the lam search has already spent the allowance and there is
|
|
nothing left to buy a span with -- which is a real finding about the encoder
|
|
(FINDINGS 41.2), not a reason for the gate to test nothing.
|
|
"""
|
|
import argparse, os, pickle, sys, time
|
|
sys.path.insert(0, "tools/encoder")
|
|
import numpy as np
|
|
import vq_hybrid as H, ratectl as RC, encode as E
|
|
from dlx import DLX
|
|
|
|
ap = argparse.ArgumentParser()
|
|
ap.add_argument("frames_dir", nargs="?", default="tmp/fr_singe")
|
|
ap.add_argument("--kbps", type=float, default=488.0)
|
|
ap.add_argument("--out", default="tmp/s12_roundtrip")
|
|
ap.add_argument("--cache", default=None)
|
|
a = ap.parse_args()
|
|
|
|
cache = a.cache or f"tmp/model_{os.path.basename(a.frames_dir.rstrip('/'))}.pkl"
|
|
if os.path.exists(cache):
|
|
m = pickle.load(open(cache, "rb"))
|
|
print(f"model from {cache}")
|
|
else:
|
|
t = time.time()
|
|
m = H.build(a.frames_dir, k1=256, k4=256, iters=16)
|
|
pickle.dump(m, open(cache, "wb"))
|
|
print(f"built model in {time.time()-t:.0f} s -> {cache}")
|
|
|
|
bad = 0
|
|
for span_mode in ("need", "all"):
|
|
print(f"\n=== spans={span_mode}, {a.kbps:g} KB/s ===")
|
|
m.pop("_sym", None)
|
|
enc = RC.encode_rate_controlled(m, target_kbps=a.kbps, lam_lo=1.0,
|
|
cycle_budget=RC.FRAME_CYCLES,
|
|
span_mode=span_mode)
|
|
recs = E.build_records(m, enc, span_mode)
|
|
path = f"{a.out}_{span_mode}.dlx"
|
|
total, vid, _ = E.write_container(path, m, recs, 12, m["k1"], m["k4"],
|
|
span_mode)
|
|
|
|
nsp = sum(len(x) for x in enc["spans"])
|
|
nfr = sum(1 for x in enc["spans"] if x)
|
|
px = sum(len(p) for x in enc["spans"] for _, _, p in x)
|
|
print(f"{path}: {total:,} B, {len(recs)} frames, "
|
|
f"{nsp:,} spans on {nfr} frames, {px:,} pixels painted by one "
|
|
f"({100*px/(len(recs)*m['H']*m['W']):.1f}% of all pixels)")
|
|
|
|
d = DLX(path)
|
|
if d.version != 3:
|
|
print(f"FAIL: container is DLX{d.version}, not DLX3"); bad += 1; continue
|
|
|
|
# The decoder's own walk of the span section must land exactly where the
|
|
# block payload starts, and blocks() already raises if the payload does not
|
|
# consume the record -- so this reads the spans back through the same code
|
|
# path the 68000 is modelled on rather than trusting the writer.
|
|
got = d.decode_all()
|
|
diff = np.array([(g != r).sum() for g, r in zip(got, enc["recon"])])
|
|
print(f"pixels differing from the encoder's reconstruction: "
|
|
f"{diff.sum()} total, worst frame {diff.max()}, "
|
|
f"frames with any: {int((diff>0).sum())}/{len(diff)}")
|
|
if diff.sum():
|
|
f = int(np.argmax(diff))
|
|
ys, xs = np.where(got[f] != enc["recon"][f])
|
|
print(f"FAIL: frame {f} differs at {diff[f]} px, first (x={xs[0]}, "
|
|
f"y={ys[0]}), block (bx={xs[0]//4}, by={ys[0]//4}), "
|
|
f"mode there = {d.modes(f)[(ys[0]//4)*d.nbx + xs[0]//4]}")
|
|
bad += 1
|
|
|
|
# A green round-trip over a container with no spans in it proves nothing.
|
|
if span_mode == "all":
|
|
if nsp < 1000:
|
|
print(f"FAIL: only {nsp} spans emitted -- this gate did not "
|
|
f"exercise the span path"); bad += 1
|
|
if not (px and max(len(x) for x in enc["spans"]) > 50):
|
|
print(f"FAIL: no frame carries a substantial span table"); bad += 1
|
|
|
|
print()
|
|
if bad:
|
|
print(f"FAILED: {bad} check(s)")
|
|
sys.exit(1)
|
|
print("OK the DLX3 span container round-trips: the reference decoder rebuilds "
|
|
"the\n encoder's reconstruction exactly, from the emitted bytes.")
|