Price cycles in the mode decision: 37 misses become 1, for 0.26 dB
The decoder has been CPU-bound since FINDINGS 28 while the mode decision
minimised D + lam*R -- distortion against BYTES. decide() now minimises
D + lam*bytes + mu*cycles, and ratectl bisects mu per frame against the
833,333-cycle budget with the lam bisection nested inside it. On the worst
sustained window:
sasi 27.22 -> 26.95 dB, 109.5 -> 109.4 KB/s, 37/120 misses -> 1
scsi 29.90 -> 29.27 dB, 280.0 -> 278.6 KB/s, 51/120 misses -> 1
Bitrate does not move: the byte controller still binds, and mu changes WHICH
modes are bought. V4 is what it stops buying -- 25.2 -> 20.3% of blocks at sasi
and 15.0 -> 5.3% at scsi, where RAW takes it. That is 28.8's inversion in
practice: RAW is dearer in bytes and cheaper in cycles, so only the byte-rich
profile can buy its way out of V4.
Three things worth knowing beyond the headline:
- The one frame that still misses, at both profiles, is FRAME 0 -- no previous
reconstruction, so 100% changed by definition, which is also what a scene
cut is. It comes out at the all-V1 floor of 110.6% and is emitted late on
purpose. Freezing a cut to make a deadline is the worse failure.
- 28.7's "11 frames are impossible" was too pessimistic. That floor held the
SKIP set fixed and asked how cheaply the drawn blocks could be drawn; the
real decision can also MOVE a block to SKIP, which above ~90% non-SKIP is
the only lever left.
- SKIP's price depends on its neighbours (13.25 cycles clustered, 45 mixed),
which a per-block lagrangian cannot see. The way out is that the two uses
need not share a cost function: a ranking constant inside decide(), the
exact clustered rule for the frame-level bisection. vq_hybrid.cycles() is
now the one definition of that rule and 11_cpu_budget.py imports it.
Gated: 09_ratectl_drift.py runs both controllers, both 0/120 drifting frames.
The cost-aware container decodes pixel-exact on the 68000 (120 frames). ON by
default in encode.py; --no-cpu-fit restores session 7. check.sh ALL GREEN.
Still a model, not a measurement, for THIS container: FINDINGS 31's cycle
figures come from vq_hybrid.cycles (within 1 point of the 68000 on four frames
of the session-7 container). Timing this one on the machine is step 1 of the
next session -- it was started and killed for time, and it is slow.
FINDINGS 31. tools/analysis/13_cpu_ratectl.py.
Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
This commit is contained in:
prosolis
@@ -19,6 +19,10 @@ paint), so drift is zero by construction rather than by tuning.
|
||||
This replays what a real decoder does -- SKIP copies the ACTUALLY EMITTED
|
||||
previous frame -- and compares it to the reconstruction ratectl recorded.
|
||||
|
||||
Session 8 added a SECOND controller (mu, the per-frame 68000 decode ceiling)
|
||||
that also varies the mode map frame to frame, so it is exposed to exactly the
|
||||
same failure and is tested here too. Both configurations must show zero drift.
|
||||
|
||||
Needs tmp/fr_singe (see docs/STATUS.md, reproducing the sustained-action
|
||||
result). ~55 s, nearly all of it the k-means in H.build; the rate-controlled
|
||||
encode of 120 frames is ~2 s.
|
||||
@@ -29,49 +33,63 @@ import numpy as np
|
||||
import vq as VQ, vq_hybrid as H, ratectl as RC
|
||||
|
||||
m = H.build("tmp/fr_singe", k1=256, k4=256, iters=16)
|
||||
# lam_lo=1.0: let quiet frames spend the whole allowance, which is the
|
||||
# harder case for this test -- it maximises how often lam moves frame to frame.
|
||||
enc = RC.encode_rate_controlled(m, target_kbps=110, lam_lo=1.0)
|
||||
|
||||
lam = enc["lam"]
|
||||
sw = int((np.diff(lam) != 0).sum())
|
||||
print(f"frames={len(lam)} distinct lam used={len(set(lam.tolist()))} "
|
||||
f"lam changes frame-to-frame={sw} "
|
||||
f"overruns={int(enc['overrun'].sum())}")
|
||||
|
||||
pal, nbx = m["pal"], m["W"] // 4
|
||||
emitted = []
|
||||
drift_px, drift_db = [], []
|
||||
for f, (rec, mode) in enumerate(zip(enc["recon"], enc["modes"])):
|
||||
out = rec.copy()
|
||||
if f > 0:
|
||||
prev_true = emitted[-1]
|
||||
for b in np.flatnonzero(mode == 0): # SKIP blocks
|
||||
by, bx = divmod(int(b), nbx)
|
||||
y, x = by*4, bx*4
|
||||
out[y:y+4, x:x+4] = prev_true[y:y+4, x:x+4]
|
||||
emitted.append(out)
|
||||
d = (out != rec).sum()
|
||||
drift_px.append(d)
|
||||
drift_db.append(VQ.psnr(pal[rec], pal[out]))
|
||||
def check(label, cycle_budget):
|
||||
"""Encode, replay as a decoder would, and return the drift in pixels."""
|
||||
print(f"\n=== {label} ===")
|
||||
m.pop("_sym", None)
|
||||
# lam_lo=1.0: let quiet frames spend the whole allowance, which is the
|
||||
# harder case for this test -- it maximises how often lam moves frame to
|
||||
# frame.
|
||||
enc = RC.encode_rate_controlled(m, target_kbps=110, lam_lo=1.0,
|
||||
cycle_budget=cycle_budget)
|
||||
lam = enc["lam"]
|
||||
sw = int((np.diff(lam) != 0).sum())
|
||||
print(f"frames={len(lam)} distinct lam used={len(set(lam.tolist()))} "
|
||||
f"lam changes frame-to-frame={sw} "
|
||||
f"overruns={int(enc['overrun'].sum())}")
|
||||
|
||||
drift_px = np.array(drift_px)
|
||||
print(f"pixels differing from what the encoder recorded:")
|
||||
print(f" frames with ANY drift: {int((drift_px>0).sum())}/{len(drift_px)}")
|
||||
print(f" max {drift_px.max()} px ({100*drift_px.max()/(m['H']*m['W']):.1f}% of frame)")
|
||||
print(f" mean {drift_px.mean():.0f} px")
|
||||
fin = [d for d in drift_db if np.isfinite(d)]
|
||||
if fin:
|
||||
print(f" encoder-vs-decoder agreement: min {min(fin):.1f} dB "
|
||||
f"(inf = identical on {len(drift_db)-len(fin)} frames)")
|
||||
pal, nbx = m["pal"], m["W"] // 4
|
||||
emitted = []
|
||||
drift_px, drift_db = [], []
|
||||
for f, (rec, mode) in enumerate(zip(enc["recon"], enc["modes"])):
|
||||
out = rec.copy()
|
||||
if f > 0:
|
||||
prev_true = emitted[-1]
|
||||
for b in np.flatnonzero(mode == 0): # SKIP blocks
|
||||
by, bx = divmod(int(b), nbx)
|
||||
y, x = by*4, bx*4
|
||||
out[y:y+4, x:x+4] = prev_true[y:y+4, x:x+4]
|
||||
emitted.append(out)
|
||||
d = (out != rec).sum()
|
||||
drift_px.append(d)
|
||||
drift_db.append(VQ.psnr(pal[rec], pal[out]))
|
||||
|
||||
drift_px = np.array(drift_px)
|
||||
print(f"pixels differing from what the encoder recorded:")
|
||||
print(f" frames with ANY drift: {int((drift_px>0).sum())}/{len(drift_px)}")
|
||||
print(f" max {drift_px.max()} px ({100*drift_px.max()/(m['H']*m['W']):.1f}% of frame)")
|
||||
print(f" mean {drift_px.mean():.0f} px")
|
||||
fin = [d for d in drift_db if np.isfinite(d)]
|
||||
if fin:
|
||||
print(f" encoder-vs-decoder agreement: min {min(fin):.1f} dB "
|
||||
f"(inf = identical on {len(drift_db)-len(fin)} frames)")
|
||||
|
||||
r = RC.summarise(m, enc, 110)
|
||||
print(f"\nratectl reports PSNR {r['psnr']:.2f} dB, {r['kbps']:.1f} KB/s "
|
||||
f"(target 110), {r['over']:.0f}% of frames over budget")
|
||||
tp = np.mean([VQ.psnr(o, pal[e]) for o, e in zip(m["rgb"], emitted)])
|
||||
print(f"what a decoder actually reconstructs: {tp:.2f} dB "
|
||||
f"-> overstated by {r['psnr']-tp:.2f} dB")
|
||||
return drift_px
|
||||
|
||||
r = RC.summarise(m, enc, 110)
|
||||
print(f"\nratectl reports PSNR {r['psnr']:.2f} dB, {r['kbps']:.1f} KB/s "
|
||||
f"(target 110), {r['over']:.0f}% of frames over budget")
|
||||
tp = np.mean([VQ.psnr(o, pal[e]) for o, e in zip(m["rgb"], emitted)])
|
||||
print(f"what a decoder actually reconstructs: {tp:.2f} dB "
|
||||
f"-> overstated by {r['psnr']-tp:.2f} dB")
|
||||
|
||||
# Acceptance criterion for the fix: a decoder replaying the emitted stream must
|
||||
# reconstruct exactly what the encoder recorded.
|
||||
sys.exit(1 if (drift_px > 0).any() else 0)
|
||||
# reconstruct exactly what the encoder recorded -- under either controller.
|
||||
bad = 0
|
||||
for label, cb in (("bytes only (session 6)", None),
|
||||
("bytes + CPU ceiling (session 8)", RC.FRAME_CYCLES)):
|
||||
d = check(label, cb)
|
||||
bad += int((d > 0).any())
|
||||
sys.exit(1 if bad else 0)
|
||||
|
||||
@@ -22,6 +22,7 @@ import sys, os, argparse
|
||||
sys.path.insert(0, "tools/encoder")
|
||||
import numpy as np
|
||||
from dlx import DLX
|
||||
import vq_hybrid as H
|
||||
|
||||
# Machine clocks, confirmed from MAME 0.277 src/mame/sharp/x68k.cpp:1133/1194/
|
||||
# 1200 -- not recalled. x68000 and x68ksupr are BOTH 40_MHz_XTAL/4 = 10 MHz;
|
||||
@@ -30,10 +31,13 @@ from dlx import DLX
|
||||
CLOCKS = {"stock": 10.0, "super": 10.0, "xvi": 33.33 / 2, "x68030": 25.0}
|
||||
FPS = 12
|
||||
|
||||
# cycles per block, measured on the emulated 68000 (synthetic single-mode frames)
|
||||
C_V1, C_V4, C_RAW = 299.9, 448.2, 400.4
|
||||
C_SKIP_FAST = 53.0 / 4 # all-SKIP header byte: one tst.b for 4
|
||||
C_SKIP_MIXED = 45.0 # a SKIP block inside a mixed byte
|
||||
# Cycles per block, measured on the emulated 68000 (synthetic single-mode
|
||||
# frames). Defined in tools/encoder/vq_hybrid.py, which is where the mode
|
||||
# decision needs them too -- one copy, not two, so a re-measurement cannot
|
||||
# leave the encoder and the scorer disagreeing.
|
||||
C_V1, C_V4, C_RAW = H.C_V1, H.C_V4, H.C_RAW
|
||||
C_SKIP_FAST, C_SKIP_MIXED = H.C_SKIP_CLUSTERED, H.C_SKIP_MIXED
|
||||
cycles = H.cycles
|
||||
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("container", nargs="?",
|
||||
@@ -50,17 +54,6 @@ if not os.path.exists(a.container):
|
||||
|
||||
d = DLX(a.container)
|
||||
|
||||
def cycles(mode):
|
||||
g = mode.reshape(-1, 4) # one header byte = four blocks
|
||||
allskip = (g == 0).all(1)
|
||||
c = allskip.sum() * 4 * C_SKIP_FAST
|
||||
m = g[~allskip]
|
||||
c += (m == 0).sum() * C_SKIP_MIXED
|
||||
c += (m == 1).sum() * C_V1
|
||||
c += (m == 2).sum() * C_V4
|
||||
c += (m == 3).sum() * C_RAW
|
||||
return c
|
||||
|
||||
modes = [d.modes(f) for f in range(d.nframes)]
|
||||
cyc = np.array([cycles(m) for m in modes])
|
||||
pct = 100 * cyc / FRAME
|
||||
|
||||
@@ -0,0 +1,101 @@
|
||||
#!/usr/bin/env python3
|
||||
"""What does fitting the CPU budget cost in quality? (session 8, lever B)
|
||||
|
||||
python3 tools/analysis/13_cpu_ratectl.py [frames_dir] [--profiles sasi,scsi]
|
||||
|
||||
Session 6 made the BYTE budget a ceiling by bisecting `lam` per frame. FINDINGS
|
||||
28 then showed the binding budget is CYCLES, not bytes, and that the mode
|
||||
decision cannot see them: it minimises `D + lam*R` on a machine that charges V4
|
||||
1.49x a V1 block while the lagrangian charges it 4x.
|
||||
|
||||
`ratectl.encode_rate_controlled(cycle_budget=...)` adds the second controller --
|
||||
`mu` bisected per frame against 833,333 cycles, with the lam bisection nested
|
||||
inside it. This measures what that costs: PSNR, bitrate, and how many frames
|
||||
still miss, against the same encode with the ceiling off.
|
||||
|
||||
The cycle budget is HARD, not a bucket. Bytes bank in the player's ring buffer;
|
||||
there is no double buffer to decode ahead into, so a frame that misses its
|
||||
decode deadline is simply late (FINDINGS 28).
|
||||
|
||||
Both controllers score frames with the exact clustered cost `vq_hybrid.cycles`,
|
||||
validated to 1 point against the 68000 (FINDINGS 28.2) -- not with the per-block
|
||||
ranking constant the mode decision uses. See vq_hybrid's note on SKIP.
|
||||
"""
|
||||
import argparse, os, pickle, sys, time
|
||||
sys.path.insert(0, "tools/encoder")
|
||||
import numpy as np
|
||||
import vq as VQ, vq_hybrid as H, ratectl as RC
|
||||
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("frames_dir", nargs="?", default="tmp/fr_singe")
|
||||
ap.add_argument("--profiles", default="sasi,scsi")
|
||||
ap.add_argument("--fps", type=int, default=12)
|
||||
ap.add_argument("--cache", default=None, help="pickle of H.build (auto by dir)")
|
||||
a = ap.parse_args()
|
||||
if not os.path.isdir(a.frames_dir):
|
||||
sys.exit(f"missing {a.frames_dir} -- see tools/bench/check.sh for extraction")
|
||||
|
||||
BUDGET = RC.FRAME_CYCLES
|
||||
|
||||
# H.build is ~55 s, nearly all k-means, and it does not depend on the profile:
|
||||
# both ship k1=k4=256. One build, cached, serves every row of the table.
|
||||
cache = a.cache or f"tmp/model_{os.path.basename(a.frames_dir.rstrip('/'))}.pkl"
|
||||
if os.path.exists(cache):
|
||||
m = pickle.load(open(cache, "rb"))
|
||||
print(f"model from {cache}")
|
||||
else:
|
||||
t = time.time()
|
||||
m = H.build(a.frames_dir, k1=256, k4=256, iters=16)
|
||||
pickle.dump(m, open(cache, "wb"))
|
||||
print(f"built model in {time.time()-t:.0f} s -> {cache}")
|
||||
print(f"{a.frames_dir}: {len(m['idx'])} frames, {m['nb']} blocks, "
|
||||
f"budget {BUDGET:,.0f} cycles/frame at {a.fps}fps\n")
|
||||
|
||||
|
||||
def run(prof_name, cycle_budget):
|
||||
p = RC.PROFILES[prof_name]
|
||||
m.pop("_sym", None) # the frame-symbol cache holds one frame
|
||||
t = time.time()
|
||||
enc = RC.encode_rate_controlled(m, p["kbps"], fps=a.fps, lam_lo=p["lam"],
|
||||
cycle_budget=cycle_budget)
|
||||
s = RC.summarise(m, enc, p["kbps"], fps=a.fps)
|
||||
s["secs"] = time.time() - t
|
||||
s["ns"] = float(np.mean([100*(mm != 0).mean() for mm in enc["modes"]]))
|
||||
return s, enc
|
||||
|
||||
|
||||
rows = []
|
||||
for name in a.profiles.split(","):
|
||||
for label, cb in (("bytes only", None), ("bytes + cycles", BUDGET)):
|
||||
s, enc = run(name, cb)
|
||||
rows.append((name, label, s))
|
||||
print(f"{name:5s} {label:<15s} {s['secs']:5.1f} s "
|
||||
f"PSNR {s['psnr']:.2f} dB {s['kbps']:6.1f} KB/s "
|
||||
f"CPU med {100*s['cyc_med']/BUDGET:5.1f}% p90 "
|
||||
f"{100*s['cyc_p90']/BUDGET:5.1f}% max {100*s['cyc_max']/BUDGET:5.1f}% "
|
||||
f"miss {s['cpu_miss']:3d} late {s['late']:2d} "
|
||||
f"mu med {s['mu_med']:.4f} max {s['mu_max']:.3f}")
|
||||
|
||||
print()
|
||||
hdr = f"{'':<22}{'PSNR':>8}{'KB/s':>9}{'CPU med':>10}{'CPU max':>10}{'miss':>7}"
|
||||
for name in a.profiles.split(","):
|
||||
r = {lab: s for n, lab, s in rows if n == name}
|
||||
b, c = r["bytes only"], r["bytes + cycles"]
|
||||
print(f"--- {name} (target {RC.PROFILES[name]['kbps']} KB/s) ---")
|
||||
print(hdr)
|
||||
for lab, s in (("bytes only", b), ("bytes + cycles", c)):
|
||||
print(f" {lab:<20}{s['psnr']:>7.2f} {s['kbps']:>8.1f} "
|
||||
f"{100*s['cyc_med']/BUDGET:>9.1f}%{100*s['cyc_max']/BUDGET:>9.1f}%"
|
||||
f"{s['cpu_miss']:>6d}")
|
||||
print(f" {'cost of fitting':<20}{c['psnr']-b['psnr']:>+7.2f} dB, "
|
||||
f"{c['kbps']-b['kbps']:+.1f} KB/s, "
|
||||
f"{b['cpu_miss']-c['cpu_miss']} fewer misses, "
|
||||
f"{c['late']} frames unfixable at mu={RC.MU_CLIFF:g}")
|
||||
print(f" {'modes % (b/c)':<20}SKIP {b['skip']:.1f}/{c['skip']:.1f} "
|
||||
f"V1 {b['v1']:.1f}/{c['v1']:.1f} V4 {b['v4']:.1f}/{c['v4']:.1f} "
|
||||
f"RAW {b['raw']:.1f}/{c['raw']:.1f}")
|
||||
print()
|
||||
|
||||
print("FINDINGS 28.7: re-coding every non-SKIP block as V1 is the floor the "
|
||||
"CURRENT mode set\nallows, and it still misses 11 frames at sasi / 12 at "
|
||||
"scsi. Misses above that floor\nare item 4 (spans), not item 1.")
|
||||
@@ -157,6 +157,11 @@ SUB = emu.add_machine_frame_notifier(function()
|
||||
M.video:snapshot()
|
||||
P("snapshot taken after the sequential pass -- last frame, 68000-decoded")
|
||||
step = step + 1
|
||||
-- DLX_VERIFY_ONLY leaves nothing after the correctness pass, and this
|
||||
-- used to walk off the end of PLAN and raise a Lua error AFTER the
|
||||
-- snapshot was already on disk -- harmless to check.sh, and exactly the
|
||||
-- kind of thing that gets mistaken for a decoder failure later.
|
||||
if not PLAN[step] then st = "finish"; return end
|
||||
launch(PLAN[step].off, PLAN[step].nfr, PLAN[step].iter)
|
||||
st, t0 = "running", nil; return
|
||||
end
|
||||
|
||||
+37
-15
@@ -93,6 +93,9 @@ def main():
|
||||
help="quality floor for rate control")
|
||||
ap.add_argument("--bucket-frames", type=int, default=8,
|
||||
help="leaky-bucket depth, in frame budgets")
|
||||
ap.add_argument("--no-cpu-fit", action="store_true",
|
||||
help="drop the per-frame 68000 decode ceiling (session 7 "
|
||||
"behaviour: 31%% of frames on hard content do not fit)")
|
||||
ap.add_argument("--prefill", type=float, default=0.0,
|
||||
help="how full the player's buffer is assumed to be at "
|
||||
"scene start, as a fraction of the bucket (0 = cold "
|
||||
@@ -108,12 +111,19 @@ def main():
|
||||
# An explicit --lam is a request for that lam, so it implies --fixed-lam.
|
||||
rc = not (a.fixed_lam or a.lam is not None)
|
||||
lam_lo = lam if a.rc_floor == "profile" else 1.0
|
||||
# The CPU ceiling is hardware, not taste: without it 31%% of frames on the
|
||||
# worst sustained window do not decode in time on a stock 68000, and with
|
||||
# it that is one frame -- the intra frame -- for 0.26 dB. FINDINGS 31.
|
||||
cyc_budget = None if a.no_cpu_fit else RC.FRAME_CYCLES
|
||||
|
||||
print(f"profile {a.profile}: {prof['desc']}")
|
||||
if rc:
|
||||
print(f" target {prof['kbps']} KB/s CEILING, rate-controlled: "
|
||||
f"lam bisected per frame in [{lam_lo:g}, {RC.LAM_CLIFF:g}], "
|
||||
f"{a.bucket_frames}-frame bucket")
|
||||
print(f" CPU ceiling: " + (f"mu bisected per frame against "
|
||||
f"{RC.FRAME_CYCLES:,.0f} cycles (12fps, stock 68000)"
|
||||
if cyc_budget else "OFF (--no-cpu-fit)"))
|
||||
else:
|
||||
print(f" target {prof['kbps']} KB/s, FIXED lam={lam} (no rate control)")
|
||||
print(f" k1={k1} k4={k4}, {_IDX_BYTES}-byte indices")
|
||||
@@ -122,7 +132,8 @@ def main():
|
||||
if rc:
|
||||
enc = RC.encode_rate_controlled(m, prof["kbps"], fps=a.fps,
|
||||
bucket_frames=a.bucket_frames,
|
||||
lam_lo=lam_lo, prefill=a.prefill)
|
||||
lam_lo=lam_lo, prefill=a.prefill,
|
||||
cycle_budget=cyc_budget)
|
||||
else:
|
||||
enc = H.encode(m, lam=lam)
|
||||
r = H.evaluate(m, enc, fps=a.fps)
|
||||
@@ -186,23 +197,34 @@ def main():
|
||||
print(f" frames that could not fit even at the lam={RC.LAM_CLIFF:g} "
|
||||
f"cliff: {rr['overrun']}/{len(lm)}")
|
||||
|
||||
# PER-FRAME non-SKIP distribution. The mean above cannot answer the
|
||||
# decoder-architecture question (FINDINGS 24.5): decode-direct-to-GVRAM
|
||||
# costs 76.6% of a 12fps frame budget x (non-SKIP fraction), while
|
||||
# compose-in-RAM-then-blit is a flat 53.6% regardless. They cross at 70%,
|
||||
# and that is a decision taken FRAME BY FRAME -- a scene cut is ~100%
|
||||
# non-SKIP and a held frame near 0%, so their mean describes no real frame.
|
||||
# PER-FRAME DECODE COST, from the measured per-mode block costs
|
||||
# (FINDINGS 28.2, vq_hybrid.cycles). The mean cannot answer this: a scene
|
||||
# cut is ~100% non-SKIP and a held frame near 0%, so their mean describes
|
||||
# no real frame. What matters is how many frames MISS, and by how much.
|
||||
#
|
||||
# This replaces the per-frame blit-vs-direct path choice that used to be
|
||||
# printed here. That plan is withdrawn -- mixing the two paths displays
|
||||
# stale pixels on 70 of 120 frames, and there was never a crossover to
|
||||
# begin with, because the compose path pays the blit ON TOP of decoding.
|
||||
# FINDINGS 28.1/28.4. The player has one path and no reference frame.
|
||||
ns = np.array([100 * (mm != 0).mean() for mm in enc["modes"]])
|
||||
over = int((ns > CROSSOVER_PCT).sum())
|
||||
cyc = np.array([H.cycles(mm) for mm in enc["modes"]])
|
||||
pct = 100 * cyc / RC.FRAME_CYCLES
|
||||
miss = int((pct > 100).sum())
|
||||
print(f" non-SKIP blocks/frame: median {np.median(ns):.1f}% "
|
||||
f"p90 {np.percentile(ns, 90):.1f}% max {ns.max():.1f}%")
|
||||
print(f" frames above the {CROSSOVER_PCT:.0f}% blit crossover: "
|
||||
f"{over}/{len(ns)} ({100*over/len(ns):.1f}%) -> "
|
||||
f"{'compose+blit wins on those' if over else 'direct-to-GVRAM wins throughout'}")
|
||||
cost = np.minimum(BLIT_PCT, DIRECT_PCT * ns / 100)
|
||||
print(f" display cost if the player picks the cheaper path per frame: "
|
||||
f"median {np.median(cost):.1f}% p90 {np.percentile(cost, 90):.1f}% "
|
||||
f"max {cost.max():.1f}% of a 12fps frame")
|
||||
print(f" decode cost: median {np.median(pct):.1f}% "
|
||||
f"p90 {np.percentile(pct, 90):.1f}% max {pct.max():.1f}% "
|
||||
f"of a {a.fps}fps frame")
|
||||
print(f" frames that do NOT decode in time: {miss}/{len(pct)} "
|
||||
f"({100*miss/len(pct):.0f}%)"
|
||||
+ (f" -- worst {pct.max():.1f}%" if miss else ""))
|
||||
if rc and cyc_budget:
|
||||
rr2 = RC.summarise(m, enc, prof["kbps"], fps=a.fps)
|
||||
print(f" mu: median {rr2['mu_med']:.4f} max {rr2['mu_max']:.3f} "
|
||||
f"frames needing any mu at all: {int((enc['mu'] > 0).sum())}/{len(pct)}")
|
||||
print(f" frames that cannot fit even at mu={RC.MU_CLIFF:g} "
|
||||
f"(emitted late on purpose): {rr2['late']}")
|
||||
|
||||
if a.preview:
|
||||
from PIL import Image
|
||||
|
||||
+101
-10
@@ -81,6 +81,27 @@ PROFILES = {
|
||||
# instead (FINDINGS 26.2). The old ladder ran to lam=2e5, 250x past shippable.
|
||||
LAM_CLIFF = 800.0
|
||||
|
||||
# Ceiling on the CYCLE search. mu prices a cycle in the same units lam prices a
|
||||
# byte, so the scale that matters is set by their ratio: at the `sasi` floor of
|
||||
# lam=60, mu=0.2 makes a V1 block's 300 cycles cost what its 1 payload byte
|
||||
# costs. MU_CLIFF=100 is three decades past that: a V1 block priced at 30,000
|
||||
# distortion units.
|
||||
#
|
||||
# It does NOT freeze the picture, and that is the point. At MU_CLIFF a block
|
||||
# only becomes SKIP if holding the previous reconstruction costs less than
|
||||
# 28,665 units of distortion, so a frame with nothing on screen to hold -- the
|
||||
# first frame of a stream, or a scene cut -- stays fully coded and comes out at
|
||||
# the all-V1 floor of 110.6% (FINDINGS 28.5). Such a frame is emitted LATE on
|
||||
# purpose, exactly as a frame that will not fit at LAM_CLIFF is emitted over
|
||||
# budget. Freezing a cut to make the deadline would be the worse failure.
|
||||
MU_CLIFF = 100.0
|
||||
MU_FLOOR = 1e-4 # bisection is geometric, so lo must be > 0
|
||||
|
||||
# The hard per-frame decode budget. NOT a bucket: bytes can be banked in the
|
||||
# player's ring buffer, but there is no double buffer to decode ahead into, so
|
||||
# a frame that misses its deadline is simply late. FINDINGS 28.
|
||||
FRAME_CYCLES = 10_000_000 / 12.0
|
||||
|
||||
AUDIO_KBPS = 7.8 # MSM6258 ADPCM 15.6kHz mono -- comes out of the same budget
|
||||
|
||||
|
||||
@@ -89,7 +110,7 @@ def frame_budget(kbps, fps=12, audio=AUDIO_KBPS):
|
||||
return (kbps - audio) * 1024.0 / fps
|
||||
|
||||
|
||||
def _search_lam(ctx, allow, lam_lo, lam_hi, iters=12):
|
||||
def _search_lam(ctx, allow, lam_lo, lam_hi, iters=12, mu=0.0):
|
||||
"""Smallest lam (=> best quality) whose frame fits `allow` bytes.
|
||||
|
||||
Payload size is non-increasing in lam -- raising lam can only move a block
|
||||
@@ -100,17 +121,17 @@ def _search_lam(ctx, allow, lam_lo, lam_hi, iters=12):
|
||||
not fit: that frame is emitted over budget on purpose. Past the FINDINGS 15
|
||||
cliff a frame is not rate-controlled, it is destroyed, so a visible overrun
|
||||
is the better failure (FINDINGS 26.2)."""
|
||||
mode, sz = H.decide(ctx, lam_lo)
|
||||
mode, sz = H.decide(ctx, lam_lo, mu)
|
||||
if sz <= allow:
|
||||
return lam_lo, mode, sz, False
|
||||
mode_hi, sz_hi = H.decide(ctx, lam_hi)
|
||||
mode_hi, sz_hi = H.decide(ctx, lam_hi, mu)
|
||||
if sz_hi > allow:
|
||||
return lam_hi, mode_hi, sz_hi, True
|
||||
lo, hi = lam_lo, lam_hi # lo does not fit, hi does
|
||||
best = (lam_hi, mode_hi, sz_hi)
|
||||
for _ in range(iters):
|
||||
mid = float(np.sqrt(lo * hi))
|
||||
mode_m, sz_m = H.decide(ctx, mid)
|
||||
mode_m, sz_m = H.decide(ctx, mid, mu)
|
||||
if sz_m <= allow:
|
||||
hi = mid; best = (mid, mode_m, sz_m)
|
||||
else:
|
||||
@@ -118,9 +139,55 @@ def _search_lam(ctx, allow, lam_lo, lam_hi, iters=12):
|
||||
return best[0], best[1], best[2], False
|
||||
|
||||
|
||||
def _search_mu(ctx, allow, lam_lo, lam_hi, cyc_budget, iters=10):
|
||||
"""Smallest mu whose frame fits BOTH budgets: `allow` bytes and
|
||||
`cyc_budget` 68000 cycles.
|
||||
|
||||
Two controllers, one nested inside the other, because the constraints are
|
||||
not separable. Raising mu moves blocks to cheaper-to-DECODE modes, which
|
||||
usually also shrinks the frame -- but not always: RAW is 400 cycles against
|
||||
V4's 448 and 16 bytes against 4, so mu can buy cycles by SPENDING bytes
|
||||
(FINDINGS 28.8). So every mu step re-runs the lam bisection and the byte
|
||||
budget is enforced at the mu that is actually chosen.
|
||||
|
||||
Cost is scored with H.cycles(), the exact clustered rule, NOT with the
|
||||
per-block ranking constant the decision uses -- see vq_hybrid's note on
|
||||
SKIP. The controller therefore converges on what the 68000 will really do.
|
||||
|
||||
Monotonicity: at a fixed lam, raising mu can only move a block to a mode
|
||||
that costs no more cycles, and it can only ADD to a SKIP cluster, so frame
|
||||
cycles are non-increasing in mu. The nested lam re-search can perturb that
|
||||
at the margin (a smaller frame permits a smaller lam, which buys quality
|
||||
back and can cost a few cycles), so the bisection keeps the best FEASIBLE
|
||||
point it has actually seen rather than trusting the invariant.
|
||||
|
||||
Returns (mu, lam, mode, size, cyc, over_bytes, over_cycles)."""
|
||||
lam, mode, sz, ovr = _search_lam(ctx, allow, lam_lo, lam_hi, mu=0.0)
|
||||
cyc = H.cycles(mode)
|
||||
if cyc <= cyc_budget:
|
||||
return 0.0, lam, mode, sz, cyc, ovr, False
|
||||
|
||||
lam_h, mode_h, sz_h, ovr_h = _search_lam(ctx, allow, lam_lo, lam_hi, mu=MU_CLIFF)
|
||||
cyc_h = H.cycles(mode_h)
|
||||
if cyc_h > cyc_budget: # cannot fit even frozen: emit late
|
||||
return MU_CLIFF, lam_h, mode_h, sz_h, cyc_h, ovr_h, True
|
||||
|
||||
lo, hi = MU_FLOOR, MU_CLIFF # lo overruns, hi fits
|
||||
best = (MU_CLIFF, lam_h, mode_h, sz_h, cyc_h, ovr_h)
|
||||
for _ in range(iters):
|
||||
mid = float(np.sqrt(lo * hi))
|
||||
lam_m, mode_m, sz_m, ovr_m = _search_lam(ctx, allow, lam_lo, lam_hi, mu=mid)
|
||||
cyc_m = H.cycles(mode_m)
|
||||
if cyc_m <= cyc_budget:
|
||||
hi = mid; best = (mid, lam_m, mode_m, sz_m, cyc_m, ovr_m)
|
||||
else:
|
||||
lo = mid
|
||||
return (*best, False)
|
||||
|
||||
|
||||
def encode_rate_controlled(m, target_kbps, fps=12, bucket_frames=8,
|
||||
lam_lo=1.0, lam_hi=LAM_CLIFF, prefill=0.0,
|
||||
steps=None, verbose=False):
|
||||
steps=None, verbose=False, cycle_budget=None):
|
||||
"""Per-frame lam search under a leaky bucket, driving the encoder ONE FRAME
|
||||
AT A TIME and feeding back the frame actually emitted.
|
||||
|
||||
@@ -149,6 +216,12 @@ def encode_rate_controlled(m, target_kbps, fps=12, bucket_frames=8,
|
||||
109.5 to 116.3 KB/s against a 110 ceiling, and on a 14-frame clip it
|
||||
disables rate control entirely because the bucket is larger than the clip.
|
||||
|
||||
`cycle_budget` adds the SECOND controller (session 8): a hard per-frame
|
||||
68000 decode ceiling, bisected on `mu` inside the lam search. None (the
|
||||
default) leaves it off and reproduces session 6 exactly, which is what
|
||||
keeps tools/analysis/09_ratectl_drift.py comparable. Pass
|
||||
FRAME_CYCLES for the 12fps stock-68000 budget.
|
||||
|
||||
`steps` is accepted and ignored -- there is no ladder any more.
|
||||
"""
|
||||
if steps is not None and verbose:
|
||||
@@ -156,25 +229,35 @@ def encode_rate_controlled(m, target_kbps, fps=12, bucket_frames=8,
|
||||
budget = frame_budget(target_kbps, fps)
|
||||
cap = bucket_frames * budget
|
||||
bucket = prefill * cap # banked bytes; bounded by the player's buffer both ways
|
||||
out = dict(recon=[], modes=[], sizes=[], lam=[], l1=[], l4g=[], overrun=[])
|
||||
out = dict(recon=[], modes=[], sizes=[], lam=[], l1=[], l4g=[], overrun=[],
|
||||
mu=[], cycles=[], late=[])
|
||||
prev = None
|
||||
for f in range(len(m["idx"])):
|
||||
ctx = H.frame_ctx(m, f, prev)
|
||||
allow = budget + bucket
|
||||
lam, mode, sz, ovr = _search_lam(ctx, allow, lam_lo, lam_hi)
|
||||
if cycle_budget is None:
|
||||
lam, mode, sz, ovr = _search_lam(ctx, allow, lam_lo, lam_hi)
|
||||
mu, cyc, late = 0.0, H.cycles(mode), False
|
||||
else:
|
||||
mu, lam, mode, sz, cyc, ovr, late = _search_mu(
|
||||
ctx, allow, lam_lo, lam_hi, cycle_budget)
|
||||
rec = H.paint(m, ctx, mode)
|
||||
bucket = float(np.clip(bucket + budget - sz, -cap, cap))
|
||||
out["recon"].append(rec); out["modes"].append(mode)
|
||||
out["sizes"].append(sz); out["lam"].append(lam); out["overrun"].append(ovr)
|
||||
out["mu"].append(mu); out["cycles"].append(cyc); out["late"].append(late)
|
||||
out["l1"].append(ctx["sym"]["l1"]); out["l4g"].append(ctx["sym"]["l4g"])
|
||||
prev = rec
|
||||
if verbose:
|
||||
print(f" f{f:04d} lam={lam:8.2f} {sz:7.0f} B "
|
||||
f"(allow {allow:7.0f}){' OVER' if ovr else ''}")
|
||||
print(f" f{f:04d} lam={lam:8.2f} mu={mu:8.4f} {sz:7.0f} B "
|
||||
f"(allow {allow:7.0f}) {100*cyc/FRAME_CYCLES:5.1f}% cpu"
|
||||
f"{' OVER' if ovr else ''}{' LATE' if late else ''}")
|
||||
return dict(recon=out["recon"], modes=out["modes"],
|
||||
sizes=np.array(out["sizes"]), lam=np.array(out["lam"]),
|
||||
l1=out["l1"], l4g=out["l4g"], overrun=np.array(out["overrun"]),
|
||||
nb=m["nb"], budget=budget, cap=cap)
|
||||
mu=np.array(out["mu"]), cycles=np.array(out["cycles"]),
|
||||
late=np.array(out["late"]),
|
||||
nb=m["nb"], budget=budget, cap=cap, cycle_budget=cycle_budget)
|
||||
|
||||
|
||||
def summarise(m, enc, target_kbps, fps=12):
|
||||
@@ -192,6 +275,14 @@ def summarise(m, enc, target_kbps, fps=12):
|
||||
over=100.0 * np.mean(sz > enc.get("budget", np.inf)),
|
||||
skip=100 * (mo == 0).mean(), v1=100 * (mo == 1).mean(),
|
||||
v4=100 * (mo == 2).mean(), raw=100 * (mo == 3).mean())
|
||||
if "cycles" in enc:
|
||||
cy = np.asarray(enc["cycles"])
|
||||
d.update(cyc_med=float(np.median(cy)), cyc_max=float(cy.max()),
|
||||
cyc_p90=float(np.percentile(cy, 90)),
|
||||
cpu_miss=int((cy > FRAME_CYCLES).sum()),
|
||||
mu_med=float(np.median(enc["mu"])),
|
||||
mu_max=float(np.asarray(enc["mu"]).max()),
|
||||
late=int(np.asarray(enc.get("late", [])).sum()))
|
||||
if "lam" in enc:
|
||||
lam = enc["lam"]
|
||||
d.update(lam_med=float(np.median(lam)), lam_max=float(lam.max()),
|
||||
|
||||
+60
-10
@@ -42,6 +42,49 @@ LUMA = VQ.LUMA
|
||||
_HDR_BYTES_PER_BLOCK = 2 / 8.0
|
||||
RAW_BYTES = 16.0 # literal palette bytes, never indices
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# CYCLE cost of each mode, per block, MEASURED on the 68000 (FINDINGS 28.2,
|
||||
# tools/bench/decode.lua). This is the other axis: `lam` prices bytes, `mu`
|
||||
# prices cycles, and the two are not proportional -- V4 is 4x a V1 block in
|
||||
# bytes and 1.49x in cycles.
|
||||
#
|
||||
# SKIP IS NOT A CONSTANT, and it is the one trap in here. A SKIP block costs
|
||||
# 13.25 cycles when all four blocks sharing its header byte are SKIP (one
|
||||
# `tst.b` clears the group) and ~45 when it sits in a mixed byte -- so its
|
||||
# price depends on its NEIGHBOURS, which a per-block lagrangian cannot see.
|
||||
# The way out is that the two uses do not need the same number:
|
||||
# * `decide` uses C_SKIP_RANK purely to RANK modes within a block. SKIP is
|
||||
# the cheapest mode either way, so the choice only scales the incentive:
|
||||
# the V1-SKIP gap moves 12% between the two candidates.
|
||||
# * `cycles()` scores a WHOLE frame with the exact clustered rule, and that
|
||||
# is what the rate controller bisects against. Nothing downstream of the
|
||||
# mode decision uses the ranking constant.
|
||||
C_V1, C_V4, C_RAW = 299.9, 448.2, 400.4
|
||||
C_SKIP_CLUSTERED = 53.0 / 4 # all-SKIP header byte: one tst.b for four
|
||||
C_SKIP_MIXED = 45.0 # a SKIP block inside a mixed byte
|
||||
C_SKIP_RANK = C_SKIP_CLUSTERED # ranking only -- see above
|
||||
MODE_CYCLES = np.array([C_SKIP_RANK, C_V1, C_V4, C_RAW], dtype=np.float64)
|
||||
|
||||
FRAME_CYCLES_12FPS = 10_000_000 / 12.0 # 833,333, x68k.cpp:1133
|
||||
|
||||
|
||||
def cycles(mode):
|
||||
"""Exact decode cost of one frame's mode map, in 68000 cycles.
|
||||
|
||||
Single source of truth: tools/analysis/11_cpu_budget.py imports this, and
|
||||
it reproduces the four frames timed on the 68000 to within 1 point
|
||||
(FINDINGS 28.2). Instruction cycles against zero-wait-state memory, so a
|
||||
LOWER BOUND like every 68000 figure since FINDINGS 24."""
|
||||
g = np.asarray(mode).reshape(-1, 4) # one header byte = four blocks
|
||||
allskip = (g == 0).all(1)
|
||||
c = allskip.sum() * 4 * C_SKIP_CLUSTERED
|
||||
mm = g[~allskip]
|
||||
c += (mm == 0).sum() * C_SKIP_MIXED
|
||||
c += (mm == 1).sum() * C_V1
|
||||
c += (mm == 2).sum() * C_V4
|
||||
c += (mm == 3).sum() * C_RAW
|
||||
return float(c)
|
||||
|
||||
|
||||
def blocks_of(idx, pal, bw, bh):
|
||||
return VQ.blockify(idx, pal, bw, bh)
|
||||
@@ -149,17 +192,24 @@ def frame_ctx(m, f, prev, idx_bytes=None):
|
||||
idx_bytes=default_idx_bytes(m) if idx_bytes is None else idx_bytes)
|
||||
|
||||
|
||||
def decide(ctx, lam):
|
||||
"""Lagrangian mode decision at one lam. Returns (mode, payload bytes).
|
||||
def decide(ctx, lam, mu=0.0):
|
||||
"""Lagrangian mode decision at one lam and one mu. Returns (mode, bytes).
|
||||
|
||||
Cheap by design: no painting, no image-sized work. A lam search calls this
|
||||
a dozen times per frame and paints once."""
|
||||
Minimises `distortion + lam*bytes + mu*cycles` per block. `mu=0` is the
|
||||
byte-only decision every session before 8 made; the machine's binding
|
||||
budget is cycles, and bytes and cycles do not rank the modes the same way
|
||||
(V4 is 4x V1 in bytes, 1.49x in cycles; RAW is dearer than V4 in bytes and
|
||||
CHEAPER in cycles, so mu inverts that preference -- FINDINGS 28.8).
|
||||
|
||||
Cheap by design: no painting, no image-sized work. A search calls this a
|
||||
dozen times per lam step and paints once."""
|
||||
ib = ctx["idx_bytes"]
|
||||
s = ctx["sym"]
|
||||
cost = np.stack([ctx["eS"],
|
||||
s["e1"] + lam * (1.0 * ib),
|
||||
s["e4"] + lam * (4.0 * ib),
|
||||
np.full(ctx["nb"], lam * RAW_BYTES)])
|
||||
mc = mu * MODE_CYCLES
|
||||
cost = np.stack([ctx["eS"] + mc[0],
|
||||
s["e1"] + lam * (1.0 * ib) + mc[1],
|
||||
s["e4"] + lam * (4.0 * ib) + mc[2],
|
||||
np.full(ctx["nb"], lam * RAW_BYTES + mc[3])])
|
||||
mode = np.argmin(cost, axis=0).astype(np.uint8)
|
||||
return mode, frame_bytes(mode, ctx["nb"], ib)
|
||||
|
||||
@@ -193,10 +243,10 @@ def paint(m, ctx, mode):
|
||||
return from_blocks(ob, nbx, nby)
|
||||
|
||||
|
||||
def encode_frame(m, f, prev, lam, idx_bytes=None):
|
||||
def encode_frame(m, f, prev, lam, idx_bytes=None, mu=0.0):
|
||||
"""One frame at one lam against one previous reconstruction."""
|
||||
ctx = frame_ctx(m, f, prev, idx_bytes)
|
||||
mode, sz = decide(ctx, lam)
|
||||
mode, sz = decide(ctx, lam, mu)
|
||||
return dict(recon=paint(m, ctx, mode), mode=mode, size=sz,
|
||||
l1=ctx["sym"]["l1"], l4g=ctx["sym"]["l4g"], ctx=ctx)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user