Price cycles in the mode decision: 37 misses become 1, for 0.26 dB
The decoder has been CPU-bound since FINDINGS 28 while the mode decision
minimised D + lam*R -- distortion against BYTES. decide() now minimises
D + lam*bytes + mu*cycles, and ratectl bisects mu per frame against the
833,333-cycle budget with the lam bisection nested inside it. On the worst
sustained window:
sasi 27.22 -> 26.95 dB, 109.5 -> 109.4 KB/s, 37/120 misses -> 1
scsi 29.90 -> 29.27 dB, 280.0 -> 278.6 KB/s, 51/120 misses -> 1
Bitrate does not move: the byte controller still binds, and mu changes WHICH
modes are bought. V4 is what it stops buying -- 25.2 -> 20.3% of blocks at sasi
and 15.0 -> 5.3% at scsi, where RAW takes it. That is 28.8's inversion in
practice: RAW is dearer in bytes and cheaper in cycles, so only the byte-rich
profile can buy its way out of V4.
Three things worth knowing beyond the headline:
- The one frame that still misses, at both profiles, is FRAME 0 -- no previous
reconstruction, so 100% changed by definition, which is also what a scene
cut is. It comes out at the all-V1 floor of 110.6% and is emitted late on
purpose. Freezing a cut to make a deadline is the worse failure.
- 28.7's "11 frames are impossible" was too pessimistic. That floor held the
SKIP set fixed and asked how cheaply the drawn blocks could be drawn; the
real decision can also MOVE a block to SKIP, which above ~90% non-SKIP is
the only lever left.
- SKIP's price depends on its neighbours (13.25 cycles clustered, 45 mixed),
which a per-block lagrangian cannot see. The way out is that the two uses
need not share a cost function: a ranking constant inside decide(), the
exact clustered rule for the frame-level bisection. vq_hybrid.cycles() is
now the one definition of that rule and 11_cpu_budget.py imports it.
Gated: 09_ratectl_drift.py runs both controllers, both 0/120 drifting frames.
The cost-aware container decodes pixel-exact on the 68000 (120 frames). ON by
default in encode.py; --no-cpu-fit restores session 7. check.sh ALL GREEN.
Still a model, not a measurement, for THIS container: FINDINGS 31's cycle
figures come from vq_hybrid.cycles (within 1 point of the 68000 on four frames
of the session-7 container). Timing this one on the machine is step 1 of the
next session -- it was started and killed for time, and it is slow.
FINDINGS 31. tools/analysis/13_cpu_ratectl.py.
Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
This commit is contained in:
prosolis
@@ -19,6 +19,10 @@ paint), so drift is zero by construction rather than by tuning.
|
||||
This replays what a real decoder does -- SKIP copies the ACTUALLY EMITTED
|
||||
previous frame -- and compares it to the reconstruction ratectl recorded.
|
||||
|
||||
Session 8 added a SECOND controller (mu, the per-frame 68000 decode ceiling)
|
||||
that also varies the mode map frame to frame, so it is exposed to exactly the
|
||||
same failure and is tested here too. Both configurations must show zero drift.
|
||||
|
||||
Needs tmp/fr_singe (see docs/STATUS.md, reproducing the sustained-action
|
||||
result). ~55 s, nearly all of it the k-means in H.build; the rate-controlled
|
||||
encode of 120 frames is ~2 s.
|
||||
@@ -29,49 +33,63 @@ import numpy as np
|
||||
import vq as VQ, vq_hybrid as H, ratectl as RC
|
||||
|
||||
m = H.build("tmp/fr_singe", k1=256, k4=256, iters=16)
|
||||
# lam_lo=1.0: let quiet frames spend the whole allowance, which is the
|
||||
# harder case for this test -- it maximises how often lam moves frame to frame.
|
||||
enc = RC.encode_rate_controlled(m, target_kbps=110, lam_lo=1.0)
|
||||
|
||||
lam = enc["lam"]
|
||||
sw = int((np.diff(lam) != 0).sum())
|
||||
print(f"frames={len(lam)} distinct lam used={len(set(lam.tolist()))} "
|
||||
f"lam changes frame-to-frame={sw} "
|
||||
f"overruns={int(enc['overrun'].sum())}")
|
||||
|
||||
pal, nbx = m["pal"], m["W"] // 4
|
||||
emitted = []
|
||||
drift_px, drift_db = [], []
|
||||
for f, (rec, mode) in enumerate(zip(enc["recon"], enc["modes"])):
|
||||
out = rec.copy()
|
||||
if f > 0:
|
||||
prev_true = emitted[-1]
|
||||
for b in np.flatnonzero(mode == 0): # SKIP blocks
|
||||
by, bx = divmod(int(b), nbx)
|
||||
y, x = by*4, bx*4
|
||||
out[y:y+4, x:x+4] = prev_true[y:y+4, x:x+4]
|
||||
emitted.append(out)
|
||||
d = (out != rec).sum()
|
||||
drift_px.append(d)
|
||||
drift_db.append(VQ.psnr(pal[rec], pal[out]))
|
||||
def check(label, cycle_budget):
|
||||
"""Encode, replay as a decoder would, and return the drift in pixels."""
|
||||
print(f"\n=== {label} ===")
|
||||
m.pop("_sym", None)
|
||||
# lam_lo=1.0: let quiet frames spend the whole allowance, which is the
|
||||
# harder case for this test -- it maximises how often lam moves frame to
|
||||
# frame.
|
||||
enc = RC.encode_rate_controlled(m, target_kbps=110, lam_lo=1.0,
|
||||
cycle_budget=cycle_budget)
|
||||
lam = enc["lam"]
|
||||
sw = int((np.diff(lam) != 0).sum())
|
||||
print(f"frames={len(lam)} distinct lam used={len(set(lam.tolist()))} "
|
||||
f"lam changes frame-to-frame={sw} "
|
||||
f"overruns={int(enc['overrun'].sum())}")
|
||||
|
||||
drift_px = np.array(drift_px)
|
||||
print(f"pixels differing from what the encoder recorded:")
|
||||
print(f" frames with ANY drift: {int((drift_px>0).sum())}/{len(drift_px)}")
|
||||
print(f" max {drift_px.max()} px ({100*drift_px.max()/(m['H']*m['W']):.1f}% of frame)")
|
||||
print(f" mean {drift_px.mean():.0f} px")
|
||||
fin = [d for d in drift_db if np.isfinite(d)]
|
||||
if fin:
|
||||
print(f" encoder-vs-decoder agreement: min {min(fin):.1f} dB "
|
||||
f"(inf = identical on {len(drift_db)-len(fin)} frames)")
|
||||
pal, nbx = m["pal"], m["W"] // 4
|
||||
emitted = []
|
||||
drift_px, drift_db = [], []
|
||||
for f, (rec, mode) in enumerate(zip(enc["recon"], enc["modes"])):
|
||||
out = rec.copy()
|
||||
if f > 0:
|
||||
prev_true = emitted[-1]
|
||||
for b in np.flatnonzero(mode == 0): # SKIP blocks
|
||||
by, bx = divmod(int(b), nbx)
|
||||
y, x = by*4, bx*4
|
||||
out[y:y+4, x:x+4] = prev_true[y:y+4, x:x+4]
|
||||
emitted.append(out)
|
||||
d = (out != rec).sum()
|
||||
drift_px.append(d)
|
||||
drift_db.append(VQ.psnr(pal[rec], pal[out]))
|
||||
|
||||
drift_px = np.array(drift_px)
|
||||
print(f"pixels differing from what the encoder recorded:")
|
||||
print(f" frames with ANY drift: {int((drift_px>0).sum())}/{len(drift_px)}")
|
||||
print(f" max {drift_px.max()} px ({100*drift_px.max()/(m['H']*m['W']):.1f}% of frame)")
|
||||
print(f" mean {drift_px.mean():.0f} px")
|
||||
fin = [d for d in drift_db if np.isfinite(d)]
|
||||
if fin:
|
||||
print(f" encoder-vs-decoder agreement: min {min(fin):.1f} dB "
|
||||
f"(inf = identical on {len(drift_db)-len(fin)} frames)")
|
||||
|
||||
r = RC.summarise(m, enc, 110)
|
||||
print(f"\nratectl reports PSNR {r['psnr']:.2f} dB, {r['kbps']:.1f} KB/s "
|
||||
f"(target 110), {r['over']:.0f}% of frames over budget")
|
||||
tp = np.mean([VQ.psnr(o, pal[e]) for o, e in zip(m["rgb"], emitted)])
|
||||
print(f"what a decoder actually reconstructs: {tp:.2f} dB "
|
||||
f"-> overstated by {r['psnr']-tp:.2f} dB")
|
||||
return drift_px
|
||||
|
||||
r = RC.summarise(m, enc, 110)
|
||||
print(f"\nratectl reports PSNR {r['psnr']:.2f} dB, {r['kbps']:.1f} KB/s "
|
||||
f"(target 110), {r['over']:.0f}% of frames over budget")
|
||||
tp = np.mean([VQ.psnr(o, pal[e]) for o, e in zip(m["rgb"], emitted)])
|
||||
print(f"what a decoder actually reconstructs: {tp:.2f} dB "
|
||||
f"-> overstated by {r['psnr']-tp:.2f} dB")
|
||||
|
||||
# Acceptance criterion for the fix: a decoder replaying the emitted stream must
|
||||
# reconstruct exactly what the encoder recorded.
|
||||
sys.exit(1 if (drift_px > 0).any() else 0)
|
||||
# reconstruct exactly what the encoder recorded -- under either controller.
|
||||
bad = 0
|
||||
for label, cb in (("bytes only (session 6)", None),
|
||||
("bytes + CPU ceiling (session 8)", RC.FRAME_CYCLES)):
|
||||
d = check(label, cb)
|
||||
bad += int((d > 0).any())
|
||||
sys.exit(1 if bad else 0)
|
||||
|
||||
@@ -22,6 +22,7 @@ import sys, os, argparse
|
||||
sys.path.insert(0, "tools/encoder")
|
||||
import numpy as np
|
||||
from dlx import DLX
|
||||
import vq_hybrid as H
|
||||
|
||||
# Machine clocks, confirmed from MAME 0.277 src/mame/sharp/x68k.cpp:1133/1194/
|
||||
# 1200 -- not recalled. x68000 and x68ksupr are BOTH 40_MHz_XTAL/4 = 10 MHz;
|
||||
@@ -30,10 +31,13 @@ from dlx import DLX
|
||||
CLOCKS = {"stock": 10.0, "super": 10.0, "xvi": 33.33 / 2, "x68030": 25.0}
|
||||
FPS = 12
|
||||
|
||||
# cycles per block, measured on the emulated 68000 (synthetic single-mode frames)
|
||||
C_V1, C_V4, C_RAW = 299.9, 448.2, 400.4
|
||||
C_SKIP_FAST = 53.0 / 4 # all-SKIP header byte: one tst.b for 4
|
||||
C_SKIP_MIXED = 45.0 # a SKIP block inside a mixed byte
|
||||
# Cycles per block, measured on the emulated 68000 (synthetic single-mode
|
||||
# frames). Defined in tools/encoder/vq_hybrid.py, which is where the mode
|
||||
# decision needs them too -- one copy, not two, so a re-measurement cannot
|
||||
# leave the encoder and the scorer disagreeing.
|
||||
C_V1, C_V4, C_RAW = H.C_V1, H.C_V4, H.C_RAW
|
||||
C_SKIP_FAST, C_SKIP_MIXED = H.C_SKIP_CLUSTERED, H.C_SKIP_MIXED
|
||||
cycles = H.cycles
|
||||
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("container", nargs="?",
|
||||
@@ -50,17 +54,6 @@ if not os.path.exists(a.container):
|
||||
|
||||
d = DLX(a.container)
|
||||
|
||||
def cycles(mode):
|
||||
g = mode.reshape(-1, 4) # one header byte = four blocks
|
||||
allskip = (g == 0).all(1)
|
||||
c = allskip.sum() * 4 * C_SKIP_FAST
|
||||
m = g[~allskip]
|
||||
c += (m == 0).sum() * C_SKIP_MIXED
|
||||
c += (m == 1).sum() * C_V1
|
||||
c += (m == 2).sum() * C_V4
|
||||
c += (m == 3).sum() * C_RAW
|
||||
return c
|
||||
|
||||
modes = [d.modes(f) for f in range(d.nframes)]
|
||||
cyc = np.array([cycles(m) for m in modes])
|
||||
pct = 100 * cyc / FRAME
|
||||
|
||||
@@ -0,0 +1,101 @@
|
||||
#!/usr/bin/env python3
|
||||
"""What does fitting the CPU budget cost in quality? (session 8, lever B)
|
||||
|
||||
python3 tools/analysis/13_cpu_ratectl.py [frames_dir] [--profiles sasi,scsi]
|
||||
|
||||
Session 6 made the BYTE budget a ceiling by bisecting `lam` per frame. FINDINGS
|
||||
28 then showed the binding budget is CYCLES, not bytes, and that the mode
|
||||
decision cannot see them: it minimises `D + lam*R` on a machine that charges V4
|
||||
1.49x a V1 block while the lagrangian charges it 4x.
|
||||
|
||||
`ratectl.encode_rate_controlled(cycle_budget=...)` adds the second controller --
|
||||
`mu` bisected per frame against 833,333 cycles, with the lam bisection nested
|
||||
inside it. This measures what that costs: PSNR, bitrate, and how many frames
|
||||
still miss, against the same encode with the ceiling off.
|
||||
|
||||
The cycle budget is HARD, not a bucket. Bytes bank in the player's ring buffer;
|
||||
there is no double buffer to decode ahead into, so a frame that misses its
|
||||
decode deadline is simply late (FINDINGS 28).
|
||||
|
||||
Both controllers score frames with the exact clustered cost `vq_hybrid.cycles`,
|
||||
validated to 1 point against the 68000 (FINDINGS 28.2) -- not with the per-block
|
||||
ranking constant the mode decision uses. See vq_hybrid's note on SKIP.
|
||||
"""
|
||||
import argparse, os, pickle, sys, time
|
||||
sys.path.insert(0, "tools/encoder")
|
||||
import numpy as np
|
||||
import vq as VQ, vq_hybrid as H, ratectl as RC
|
||||
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("frames_dir", nargs="?", default="tmp/fr_singe")
|
||||
ap.add_argument("--profiles", default="sasi,scsi")
|
||||
ap.add_argument("--fps", type=int, default=12)
|
||||
ap.add_argument("--cache", default=None, help="pickle of H.build (auto by dir)")
|
||||
a = ap.parse_args()
|
||||
if not os.path.isdir(a.frames_dir):
|
||||
sys.exit(f"missing {a.frames_dir} -- see tools/bench/check.sh for extraction")
|
||||
|
||||
BUDGET = RC.FRAME_CYCLES
|
||||
|
||||
# H.build is ~55 s, nearly all k-means, and it does not depend on the profile:
|
||||
# both ship k1=k4=256. One build, cached, serves every row of the table.
|
||||
cache = a.cache or f"tmp/model_{os.path.basename(a.frames_dir.rstrip('/'))}.pkl"
|
||||
if os.path.exists(cache):
|
||||
m = pickle.load(open(cache, "rb"))
|
||||
print(f"model from {cache}")
|
||||
else:
|
||||
t = time.time()
|
||||
m = H.build(a.frames_dir, k1=256, k4=256, iters=16)
|
||||
pickle.dump(m, open(cache, "wb"))
|
||||
print(f"built model in {time.time()-t:.0f} s -> {cache}")
|
||||
print(f"{a.frames_dir}: {len(m['idx'])} frames, {m['nb']} blocks, "
|
||||
f"budget {BUDGET:,.0f} cycles/frame at {a.fps}fps\n")
|
||||
|
||||
|
||||
def run(prof_name, cycle_budget):
|
||||
p = RC.PROFILES[prof_name]
|
||||
m.pop("_sym", None) # the frame-symbol cache holds one frame
|
||||
t = time.time()
|
||||
enc = RC.encode_rate_controlled(m, p["kbps"], fps=a.fps, lam_lo=p["lam"],
|
||||
cycle_budget=cycle_budget)
|
||||
s = RC.summarise(m, enc, p["kbps"], fps=a.fps)
|
||||
s["secs"] = time.time() - t
|
||||
s["ns"] = float(np.mean([100*(mm != 0).mean() for mm in enc["modes"]]))
|
||||
return s, enc
|
||||
|
||||
|
||||
rows = []
|
||||
for name in a.profiles.split(","):
|
||||
for label, cb in (("bytes only", None), ("bytes + cycles", BUDGET)):
|
||||
s, enc = run(name, cb)
|
||||
rows.append((name, label, s))
|
||||
print(f"{name:5s} {label:<15s} {s['secs']:5.1f} s "
|
||||
f"PSNR {s['psnr']:.2f} dB {s['kbps']:6.1f} KB/s "
|
||||
f"CPU med {100*s['cyc_med']/BUDGET:5.1f}% p90 "
|
||||
f"{100*s['cyc_p90']/BUDGET:5.1f}% max {100*s['cyc_max']/BUDGET:5.1f}% "
|
||||
f"miss {s['cpu_miss']:3d} late {s['late']:2d} "
|
||||
f"mu med {s['mu_med']:.4f} max {s['mu_max']:.3f}")
|
||||
|
||||
print()
|
||||
hdr = f"{'':<22}{'PSNR':>8}{'KB/s':>9}{'CPU med':>10}{'CPU max':>10}{'miss':>7}"
|
||||
for name in a.profiles.split(","):
|
||||
r = {lab: s for n, lab, s in rows if n == name}
|
||||
b, c = r["bytes only"], r["bytes + cycles"]
|
||||
print(f"--- {name} (target {RC.PROFILES[name]['kbps']} KB/s) ---")
|
||||
print(hdr)
|
||||
for lab, s in (("bytes only", b), ("bytes + cycles", c)):
|
||||
print(f" {lab:<20}{s['psnr']:>7.2f} {s['kbps']:>8.1f} "
|
||||
f"{100*s['cyc_med']/BUDGET:>9.1f}%{100*s['cyc_max']/BUDGET:>9.1f}%"
|
||||
f"{s['cpu_miss']:>6d}")
|
||||
print(f" {'cost of fitting':<20}{c['psnr']-b['psnr']:>+7.2f} dB, "
|
||||
f"{c['kbps']-b['kbps']:+.1f} KB/s, "
|
||||
f"{b['cpu_miss']-c['cpu_miss']} fewer misses, "
|
||||
f"{c['late']} frames unfixable at mu={RC.MU_CLIFF:g}")
|
||||
print(f" {'modes % (b/c)':<20}SKIP {b['skip']:.1f}/{c['skip']:.1f} "
|
||||
f"V1 {b['v1']:.1f}/{c['v1']:.1f} V4 {b['v4']:.1f}/{c['v4']:.1f} "
|
||||
f"RAW {b['raw']:.1f}/{c['raw']:.1f}")
|
||||
print()
|
||||
|
||||
print("FINDINGS 28.7: re-coding every non-SKIP block as V1 is the floor the "
|
||||
"CURRENT mode set\nallows, and it still misses 11 frames at sasi / 12 at "
|
||||
"scsi. Misses above that floor\nare item 4 (spans), not item 1.")
|
||||
Reference in New Issue
Block a user