drwily/Dragon-s-Lair-X68k
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Build v7 into the player, and find the cost model 18% wrong on the block it made commonest

Browse Source 
src/player/decode.s now paints v7 literal spans, pixel-exact under MAME and
px68k's C68K core over a container where every frame carries 128-216 spans
covering up to 38% of the picture. The span pass is blit.s v7 verbatim: the
66.0/9.143/9.978 fit was measured on that instruction sequence.

The container is DLX3 -- a span section between the mode header and the block
payload, since that is the only place the 68000 can reach without first parsing
something of variable length. 16_span_roundtrip.py gates it in check.sh, and
asserts it emitted enough spans to have tested anything.

Two synthetic all-SPAN anchors price v7 inside decode.s at 151.2 and 225.6
clocks per 4x4 block, against FINDINGS 40's table of 151 and 226 -- 0.2% on
both emulators. The measured mode costs what it was said to cost.

Two things that were not on the list:

TWO BYTE BUDGETS. FINDINGS 40's 18/120 was scored against the 488 KB/s PIPE,
not the 280 KB/s profile, and at the profile rate the lam search has already
spent the allowance -- spans fired on 5 frames of 120 and looked like a
regression. The profile is a chosen quality rate point; the pipe is hardware.
--kbps and --span-kbps are now separate and spans run before mu, because a span
pays in bytes and mu pays in picture. Delivered: 86/120 over budget without
spans, 77/120 at the profile budget, 34/120 on the pipe for +0.36 dB.

C_SKIP_MIXED WAS NEVER MEASURED, and it was 18% low -- 45.0, now 55.0. It is
the one constant in the table that came from a derivation, because the
synthetic frame that would measure it cannot exist: a byte needs a coded block
for its SKIP to be mixed. Four bracketing anchors measure it on both emulators
with the header byte rotated through all four positions, and the partner mode
solves back to its own anchored value to 0.2%. With it corrected the model
predicts a real spanned decode to -0.06% mean / 0.09% worst, against -2.99% /
4.30%. It matters because a span marks its run SKIP, so mixed SKIPs dominate
exactly the frames spans are judged on.

Also: the rig had been writing its synthetic timing frames 26 KB past the top
of a 2 MB machine, and got away with it because the modes it overran are
data-independent. A span's jump displacements come out of the stream, so it is
not. And frames-over-budget is no longer a safe headline -- the controller aims
at the deadline, so 55 of 120 frames sit within 5% of it and a 1% cost shift
moves 22 frames.

FINDINGS 41. check.sh ALL GREEN, now gating on a span-heavy DLX3 container.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
This commit is contained in:
prosolis
2026-08-23 20:02:03 -07:00
parent c520a89e14
commit b49bbdc939
16 changed files with 1342 additions and 203 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tools/analysis/14_dmac_chain.py
+5 -2
View File
@@ -56,8 +56,11 @@ import buscost as B
FRAME_CYC = 833333.0
AUDIO_KBPS = 7.8
C_V1, C_V4, C_RAW = 299.9, 448.2, 400.4 # FINDINGS 28.2 (MEASURED)
C_SKIP_CLUSTERED, C_SKIP_MIXED = 13.25, 45.0
import vq_hybrid as _H
C_V1, C_V4, C_RAW = _H.C_V1, _H.C_V4, _H.C_RAW # FINDINGS 28.2 (MEASURED)
# 45.0 until session 12 measured it at 55.0 (FINDINGS 41.5) -- imported now, so
# the correction cannot be undone by a stale copy.
C_SKIP_CLUSTERED, C_SKIP_MIXED = _H.C_SKIP_CLUSTERED, _H.C_SKIP_MIXED
SPAN_BYTES_PX, SPAN_HDR = 2, 6
ap = argparse.ArgumentParser()
tools/analysis/15_bus_occupancy.py
+13
View File
@@ -25,8 +25,11 @@ A 68000 bus cycle is 4 clocks, so a frame of C clocks holds C/4 bus slots.
"""
import sys, os, argparse, csv
sys.path.insert(0, "tools/encoder")
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import numpy as np
from dlx import DLX
import buscost as B
from buscost import V7_FRAME_PREF, V7_FRAME_DATA
BUS_CLK = 4
@@ -90,6 +93,16 @@ for f in range(NF):
pw, pd = BODY[b]
pref += DISPATCH[b] + pw + SK_TAIL
data += 1 + pd
# The span section is bus traffic too, and it is most of the frame's data
# accesses in a span-heavy container: 48 per 24-pixel chain unit. Leaving it
# out would not merely understate the total -- it would break the CHECK
# below, which is the whole licence for the prefetch figure.
sp, _ = d.spans(f)
if sp:
pref += V7_FRAME_PREF; data += V7_FRAME_DATA
for _, _, px in sp:
sp_p, sp_d = B.v7_span_split(len(px))
pref += sp_p; data += sp_d
pref_t.append(pref); data_t.append(data)
cyc_t.append(meas.get(f, (0, 0))[0])
tools/analysis/16_span_roundtrip.py
+107
View File
@@ -0,0 +1,107 @@
#!/usr/bin/env python3
"""GATE for the DLX3 span container: does the reference decoder reproduce the
encoder's own reconstruction, from the emitted bytes?
python3 tools/analysis/16_span_roundtrip.py [frames_dir] [--kbps 488]
Exits non-zero if any frame differs by a single pixel.
WHY THIS EXISTS SEPARATELY FROM 09. `09_ratectl_drift.py` replays SKIP
semantics in Python against the mode maps the encoder returned; it never reads
a container. A span breaks exactly that shortcut: a spanned block reads SKIP
in the mode header and is painted by the span section instead, so a replay that
knows only about mode maps reports drift where there is none, and -- far worse
-- a container whose span section is malformed would still pass, because 09
never parses one. This gate closes that: encode, WRITE THE CONTAINER, read it
back with tools/encoder/dlx.py (the byte-for-byte reference decoder the 68000
is checked against), and compare to what ratectl recorded.
It also has to prove it tested something. A round-trip over a container with
no spans in it is green by vacuity, which is the failure mode FINDINGS 40.6
named for the snapshot count: a gate must take its expected work from the
generated artefact, not from an assumption. So the thresholds below are
asserted, not printed.
The `--kbps` default is the BUS rate, not the `scsi` profile's 280: spans are
bought with bytes, and 14_dmac_chain.py scores them against the 488 KB/s pipe.
At the profile rate the lam search has already spent the allowance and there is
nothing left to buy a span with -- which is a real finding about the encoder
(FINDINGS 41.2), not a reason for the gate to test nothing.
"""
import argparse, os, pickle, sys, time
sys.path.insert(0, "tools/encoder")
import numpy as np
import vq_hybrid as H, ratectl as RC, encode as E
from dlx import DLX
ap = argparse.ArgumentParser()
ap.add_argument("frames_dir", nargs="?", default="tmp/fr_singe")
ap.add_argument("--kbps", type=float, default=488.0)
ap.add_argument("--out", default="tmp/s12_roundtrip")
ap.add_argument("--cache", default=None)
a = ap.parse_args()
cache = a.cache or f"tmp/model_{os.path.basename(a.frames_dir.rstrip('/'))}.pkl"
if os.path.exists(cache):
m = pickle.load(open(cache, "rb"))
print(f"model from {cache}")
else:
t = time.time()
m = H.build(a.frames_dir, k1=256, k4=256, iters=16)
pickle.dump(m, open(cache, "wb"))
print(f"built model in {time.time()-t:.0f} s -> {cache}")
bad = 0
for span_mode in ("need", "all"):
print(f"\n=== spans={span_mode}, {a.kbps:g} KB/s ===")
m.pop("_sym", None)
enc = RC.encode_rate_controlled(m, target_kbps=a.kbps, lam_lo=1.0,
cycle_budget=RC.FRAME_CYCLES,
span_mode=span_mode)
recs = E.build_records(m, enc, span_mode)
path = f"{a.out}_{span_mode}.dlx"
total, vid, _ = E.write_container(path, m, recs, 12, m["k1"], m["k4"],
span_mode)
nsp = sum(len(x) for x in enc["spans"])
nfr = sum(1 for x in enc["spans"] if x)
px = sum(len(p) for x in enc["spans"] for _, _, p in x)
print(f"{path}: {total:,} B, {len(recs)} frames, "
f"{nsp:,} spans on {nfr} frames, {px:,} pixels painted by one "
f"({100*px/(len(recs)*m['H']*m['W']):.1f}% of all pixels)")
d = DLX(path)
if d.version != 3:
print(f"FAIL: container is DLX{d.version}, not DLX3"); bad += 1; continue
# The decoder's own walk of the span section must land exactly where the
# block payload starts, and blocks() already raises if the payload does not
# consume the record -- so this reads the spans back through the same code
# path the 68000 is modelled on rather than trusting the writer.
got = d.decode_all()
diff = np.array([(g != r).sum() for g, r in zip(got, enc["recon"])])
print(f"pixels differing from the encoder's reconstruction: "
f"{diff.sum()} total, worst frame {diff.max()}, "
f"frames with any: {int((diff>0).sum())}/{len(diff)}")
if diff.sum():
f = int(np.argmax(diff))
ys, xs = np.where(got[f] != enc["recon"][f])
print(f"FAIL: frame {f} differs at {diff[f]} px, first (x={xs[0]}, "
f"y={ys[0]}), block (bx={xs[0]//4}, by={ys[0]//4}), "
f"mode there = {d.modes(f)[(ys[0]//4)*d.nbx + xs[0]//4]}")
bad += 1
# A green round-trip over a container with no spans in it proves nothing.
if span_mode == "all":
if nsp < 1000:
print(f"FAIL: only {nsp} spans emitted -- this gate did not "
f"exercise the span path"); bad += 1
if not (px and max(len(x) for x in enc["spans"]) > 50):
print(f"FAIL: no frame carries a substantial span table"); bad += 1
print()
if bad:
print(f"FAILED: {bad} check(s)")
sys.exit(1)
print("OK the DLX3 span container round-trips: the reference decoder rebuilds "
"the\n encoder's reconstruction exactly, from the emitted bytes.")
tools/analysis/17_span_delivered.py
+100
View File
@@ -0,0 +1,100 @@
#!/usr/bin/env python3
"""What do the spans the ENCODER actually emitted cost, and what do they buy?
python3 tools/analysis/17_span_delivered.py a.dlx [b.dlx ...] [--bus 488]
Every span figure before this one -- FINDINGS 29 through 40, and
tools/analysis/12 and 14 -- was scored by SIMULATING span selection over mode
maps that were chosen without spans available. FINDINGS 39.3 flagged that as a
lower bound on what a span-aware encoder would find, and docs/STATUS.md's item 2
asks for the figures to be re-run "against a container the encoder actually
emits with spans in it". This is that script: it reads the span section out of
a DLX3 container and prices exactly those spans, with no selection model at all.
THE MODEL IS 14_dmac_chain.py's, deliberately unchanged, so the columns are
comparable:
frame clocks = block decode + span painting + disk DMA
additive, because a 68000 has no cache and a two-word prefetch queue and stalls
the moment another master takes the bus (FINDINGS 38.3). Block cost is
vq_hybrid.cycles(), which reads a spanned block as SKIP -- correct, because the
span section is what paints it, and its cost is the second term.
The span term is the MEASURED v7 fit (FINDINGS 40), and as of session 12 that
fit is confirmed inside src/player/decode.s itself rather than only in
tools/bench/blit.s: the synthetic all-SPAN anchors of tools/bench/prep_dlx.py
reproduce it to 0.23% on both emulators (FINDINGS 41.3).
"""
import argparse, os, sys
sys.path.insert(0, "tools/encoder")
sys.path.insert(0, "tools/analysis")
import numpy as np
import vq_hybrid as H
import spans as SP
import buscost as B
from dlx import DLX
FRAME_CYC = 833333.0
AUDIO_KBPS = 7.8
ap = argparse.ArgumentParser()
ap.add_argument("containers", nargs="+")
ap.add_argument("--bus", type=float, default=488.0, help="SCSI pipe, KB/s")
ap.add_argument("--fps", type=float, default=12.0)
ap.add_argument("--disk-clk-word", type=float, default=8.0,
help="clocks the SCSI DMA steals per word (FINDINGS 39.7 "
"brackets it at 5..12; 8 is the midpoint)")
a = ap.parse_args()
def score(path):
d = DLX(path)
rows = []
for f in range(d.nframes):
mode = d.modes(f)
sp, _ = d.spans(f)
_, n = d.frames[f]
blk = H.cycles(mode)
spc = sum(SP.clocks(len(p)) for _, _, p in sp)
disk = n / 2.0 * a.disk_clk_word
rows.append((blk, spc, disk, n, len(sp),
sum(len(p) for _, _, p in sp)))
return d, np.array(rows).T
print(f"{'container':<34}{'KB/s':>8}{'spans':>9}{'span px':>9}"
f"{'median':>9}{'worst':>9}{'over':>9}")
print(f"{'':<34}{'':>8}{'/frame':>9}{'%':>9}"
f"{'% frame':>9}{'% frame':>9}{'budget':>9}")
for path in a.containers:
if not os.path.exists(path):
print(f"{path:<34} missing"); continue
d, r = score(path)
blk, spc, disk, byt, nsp, spx = r
tot = blk + spc + disk
kbps = byt.mean() * a.fps / 1024 + AUDIO_KBPS
print(f"{os.path.basename(path):<34}{kbps:>8.1f}{nsp.mean():>9.0f}"
f"{100*spx.mean()/(d.W*d.H):>9.1f}"
f"{100*np.median(tot)/FRAME_CYC:>9.1f}"
f"{100*tot.max()/FRAME_CYC:>9.1f}"
f"{int((tot > FRAME_CYC).sum()):>6}/{d.nframes:<3}")
print(f"\n ADDITIVE: frame = block decode + span painting + disk DMA, the model"
f"\n of 14_dmac_chain.py. Disk debited at {a.disk_clk_word:g} clocks/word "
f"over the\n container's own byte count; CPU budget {FRAME_CYC:,.0f} "
f"clocks at {a.fps:g} fps.")
# The decomposition is the point: a span moves work out of the block loop and
# into the span section, and it pays for it in bytes -- which the disk term
# then charges back. A design that only counted the CPU would show a win that
# the I/O it created takes away again (docs/FINDINGS.md 33).
print(f"\nWHERE EACH FRAME'S CLOCKS GO, mean over the container")
print(f" {'container':<34}{'blocks':>12}{'spans':>12}{'disk':>12}{'total':>12}")
for path in a.containers:
if not os.path.exists(path):
continue
d, r = score(path)
blk, spc, disk = r[0], r[1], r[2]
print(f" {os.path.basename(path):<34}{blk.mean():>12,.0f}{spc.mean():>12,.0f}"
f"{disk.mean():>12,.0f}{(blk+spc+disk).mean():>12,.0f}")
tools/analysis/buscost.py
+28 -1
View File
@@ -154,8 +154,35 @@ def v7_span(npix):
def v7_span_bus(npix):
"""Bus CYCLES a v7 span occupies -- instruction words plus data accesses."""
p, d = v7_span_split(npix)
return p + d
def v7_span_split(npix):
"""(instruction words, data accesses) for one v7 span, separately.
15_bus_occupancy.py needs the two apart, because the DATA half is what the
C68K harness can check and the PREFETCH half is what rides on that check.
per span move.l (a0)+,a2 1 word + 2 reads
move.w (a0)+,d0 1 word + 1 read (coarse displacement)
jmp (pc,d0.w) 2 words
move.w (a0)+,d0 1 word + 1 read (fine, from mid-stream)
jmp (pc,d0.w) 2 words
dbra 2 words -> 9 words, 4 accesses
per coarse 2 movem.l of 12 + lea = 6 words, 24 reads + 24 writes
per fine move.l (a0)+,(a2)+ = 1 word, 2 reads + 2 writes
"""
k, r = divmod(pad2(npix), V6_UNIT_PX)
return V7_SPAN_BUS + k * V6_UNIT_BUS + (r // V7_FINE_PX) * V7_FINE_BUS
f = r // V7_FINE_PX
return (9 + k * 6 + f * 1,
4 + k * 48 + f * 4)
# Per FRAME, decode.s's paint_spans entry and exit: the span count read, the
# guard branch, and the push/pop of a1 that buys back a twelfth payload
# register. Two long accesses a frame against 24 pixels a chain unit.
V7_FRAME_PREF, V7_FRAME_DATA = 7, 7
def v6_span_bus(npix):