Handoff: rate control is next, and it is unsound as written

Session 5 handoff. The user has chosen rate control as the next session's work,
so this reads ratectl.py properly before that session starts rather than
discovering the problem mid-implementation.

FINDINGS 26: encode_rate_controlled() is not sound. H.encode() is temporally
recursive -- SKIP blocks copy the previous RECONSTRUCTION -- but rate control
builds a ladder of independent whole-sequence encodes and picks each frame from
whichever rung fits the budget. Frames then reference reconstructions the
decoder never saw. Measured on the Singe window: 67 rung switches, 111 of 120
frames drift, worst frame 43.4% of pixels, reported PSNR overstated by 0.36 dB.
It would have wired up cleanly and reported a plausible wrong answer.

Two further defects in the same function: the lam ladder runs to 2e5, 250x past
the FINDINGS 15 cliff, so a frame that only fits up there is destroyed rather
than rate-controlled; and with 5 rungs only two are ever chosen, straddling the
operating point by 7.5x. The docstring describes a per-frame binary search,
which is the right design -- the implementation is a fixed ladder. The leaky
bucket does work and should be kept: 109.1 KB/s against a 110 target.

tools/analysis/09_ratectl_drift.py is the regression test and the acceptance
criterion: it exits non-zero until zero frames drift.

Also corrected the stale 38% blit figure in ratectl.py's profile commentary,
which session 5 measured at 53.6% (FINDINGS 24), and recorded the pgrep -f
self-kill trap again -- four times across three sessions now.

check.sh ALL GREEN.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
This commit is contained in:
prosolis
2026-08-23 14:10:07 -07:00
parent e00264a058
commit 145753c0bf
5 changed files with 218 additions and 16 deletions
+22 -6
View File
@@ -11,6 +11,12 @@ without that, quiet frames waste budget and action frames stay ugly.
The ceiling is HARD: the 68000 streams at a fixed rate off the disk, and a frame
that overruns is a dropped frame, not a slow frame.
STATUS, session 5: this module is written but STILL NOT WIRED INTO encode.py,
and FINDINGS 25.3 measured both profiles overshooting their targets by 18% and
34% on the worst sustained window because of that. Before wiring it up, read
the correctness note on encode_rate_controlled() -- the lam-ladder approach it
uses is not sound against a temporally recursive encoder.
"""
import numpy as np
import vq_hybrid as H
@@ -28,9 +34,16 @@ import vq_hybrid as H
# What actually bounds the high end:
# - Bus: unmeasured. ~300-500 KB/s SASI / ~1 MB/s SCSI, both FOLKLORE.
# This is the binding unknown and the reason the disk benchmark matters.
# - CPU: a FULL-frame blit is only 38% of the 12fps budget, and VQ decode is
# table copies (RAW, the mode that dominates at high rate, is the CHEAPEST
# to blit). So raising the bitrate is nearly free on CPU.
# - CPU: a full-frame blit is **53.6%** of the 12fps budget -- MEASURED on the
# emulated 68000, session 5, FINDINGS 24. This line previously said 38%,
# which was an estimate and was wrong by 41%. And 53.6% is a floor: MAME
# models no GVRAM wait states, so real hardware is worse.
# "Raising the bitrate is nearly free on CPU" survives but is now much
# tighter. It rests on RAW being the cheapest mode to blit, which is still
# true, but the display path alone now eats over half the frame before any
# decoding happens. The per-frame path choice of FINDINGS 25.6 (blit vs
# direct-to-GVRAM, whichever is cheaper for that frame) brings the median
# back to ~37% and caps the worst case at 53.6%.
# - Entropy coding is NOT the way to buy headroom here: deflate decode is
# ~216% of the frame budget on a 68000 and even LZ4 is ~54%. See FINDINGS 17.
# The rates below are therefore RAW payload, no entropy coding.
@@ -39,15 +52,18 @@ import vq_hybrid as H
PROFILES = {
"sasi": dict(kbps=110, lam=60.0, k1=256, k4=256,
desc="stock 10MHz ACE/EXPERT, SASI",
quality="36.9 dB on 00020 / 29.6 dB on 00146",
quality="36.9 dB on 00020 / 29.6 dB on 00146 / 27.8 dB on the "
"Singe window, where it overshoots to 129.6 KB/s",
util="~105 KB/s = 35% of the pessimistic 300 KB/s SASI figure"),
"scsi": dict(kbps=280, lam=10.0, k1=256, k4=256,
desc="Super/XVI, or CZ-6BS1 board in a 10MHz machine",
quality="39.4 dB on 00020 / 32.3 dB on 00146",
quality="39.4 dB on 00020 / 32.3 dB on 00146 / 30.8 dB on the "
"Singe window, where it overshoots to 373.8 KB/s",
util="~275 KB/s = 28% of the 1 MB/s SCSI folklore figure"),
}
# lam=0 is PIXEL-EXACT against the palettised frame (0.00 dB loss) at ~450 KB/s
# of raw payload, and costs only 38% of the CPU budget. If the blocked disk
# of raw payload, and costs 53.6% of the CPU budget (not the 38% written here
# before session 5 -- FINDINGS 24). If the blocked disk
# benchmark confirms SCSI sustains >=800 KB/s, the `scsi` profile should become
# lam=0 and the port ships transparent video. That decision is waiting on a
# measurement, not on a design choice.