FINDINGS 26 stopped the session-5 rate controller before it shipped: it built a
lam-ladder of independent whole-sequence encodes and picked frames off it, so
SKIP blocks referenced reconstructions the decoder never saw -- 111 of 120
frames drifted. The fix is the structural one 26.1 said it had to be.
vq_hybrid is now frame-drivable -- frame_ctx / decide / paint -- and encode() is
a thin loop over it. Rate control drives the same three calls, bisects lam per
frame under the leaky bucket, and feeds back the frame it actually emitted. The
desync has no way to occur, and 09_ratectl_drift.py goes 111/120 -> 0/120. That
test is now part of check.sh, which is ~2 min rather than ~40 s.
Both overshoots on the worst sustained window are closed for under 1 dB, totals
including audio: sasi 137.4 -> 109.5 KB/s (-0.60 dB), scsi 381.6 -> 280.0 KB/s
(-0.91 dB). Zero frames hit the lam=800 cliff, so nothing was destroyed to get
there. Rate control also makes the display path cheaper -- scsi's median drops
53.6% -> 47.1% -- because raising lam moves blocks to SKIP and V1.
Two knobs measured rather than guessed. --rc-floor is worth 0.00 dB on that
window and defaults to the profile lam, so rate control cannot regress content
that already fits. --prefill defaults to 0 and is documented as a trap: it buys
a permission to overshoot of exactly bucket/nframes, and on a 14-frame clip it
disables the controller outright.
FINDINGS 26.5 was wrong in both halves and 27.6 records it. _paint was not the
bottleneck (14% of a frame, though vectorising it was still right at 17.1x) and
the ladder was never "minutes" -- those were k-means in build(). What makes
per-frame rate control affordable is that VQ.assign depends on neither lam nor
prev, so it is cached one frame deep: a 12-step search over 120 frames costs
0.31 s against 49.1 s.
Also caught: fixed-lam sasi was already 5% over target on 00020, the clip
everyone called easy. Nothing noticed because the profile table quotes PSNR and
not bitrate.
check.sh: ALL GREEN.
Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
Session 5 handoff. The user has chosen rate control as the next session's work,
so this reads ratectl.py properly before that session starts rather than
discovering the problem mid-implementation.
FINDINGS 26: encode_rate_controlled() is not sound. H.encode() is temporally
recursive -- SKIP blocks copy the previous RECONSTRUCTION -- but rate control
builds a ladder of independent whole-sequence encodes and picks each frame from
whichever rung fits the budget. Frames then reference reconstructions the
decoder never saw. Measured on the Singe window: 67 rung switches, 111 of 120
frames drift, worst frame 43.4% of pixels, reported PSNR overstated by 0.36 dB.
It would have wired up cleanly and reported a plausible wrong answer.
Two further defects in the same function: the lam ladder runs to 2e5, 250x past
the FINDINGS 15 cliff, so a frame that only fits up there is destroyed rather
than rate-controlled; and with 5 rungs only two are ever chosen, straddling the
operating point by 7.5x. The docstring describes a per-frame binary search,
which is the right design -- the implementation is a fixed ladder. The leaky
bucket does work and should be kept: 109.1 KB/s against a 110 target.
tools/analysis/09_ratectl_drift.py is the regression test and the acceptance
criterion: it exits non-zero until zero frames drift.
Also corrected the stale 38% blit figure in ratectl.py's profile commentary,
which session 5 measured at 53.6% (FINDINGS 24), and recorded the pgrep -f
self-kill trap again -- four times across three sessions now.
check.sh ALL GREEN.
Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
The open risk since session 2 was "a sustained action sequence could still
break the bitrate", with every clip measured so far being 1.2-1.7 s. Closed by
measurement rather than by sampling clips by hand.
07_motion_survey.py scans a whole stream at 96x72 for the hottest sliding
window of inter-frame difference. On 00223 the spread between the quietest and
hottest sustained 10 s windows is 10.6x, which is the argument for not eyeballing
it. Hottest is t=539.4s, the Singe endgame.
There, with the fixed lam the CLI uses, sasi overshoots 110 -> 129.6 KB/s (+18%)
and scsi 280 -> 373.8 KB/s (+34%). Rate control moves from "insurance, not a
fix" to required, and is promoted above the full-disc survey. The bus is not
broken -- 381.6 KB/s still fits the 488 KB/s figure -- so FINDINGS 21 survives,
at 78% of the pipe instead of a comfortable margin.
Three further corrections fall out:
- The two largest streams on the disc are bonus material. 00216 is the feature
with a burned-in commentary PiP; 00215 is the commentary. 00223 is the clean
9.4 min. A size-ranked survey would have encoded live action.
- On hard content the 256-colour scene palette (31.33 dB) binds well before the
X68000 display (40.81 dB); scsi is already within 0.51 dB of it.
- FINDINGS 24.5's architecture question resolves to "both paths, chosen per
frame": 30-53% of frames sit above the 70% crossover. Picking per frame costs
a median 37.0% of the frame budget and caps at 53.6%. Reporting for this is
wired into encode.py, which previously only printed a mean over all frames --
the one statistic that cannot answer a per-frame question.
extract.py takes optional start/dur; 08_mode_map.py renders source | decoded |
block-mode map to .webm.
Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
Session 2 reversed several of its own conclusions. The docs are append-only, so
a reader could land on a superseded section and act on it. This pass makes the
repo internally consistent.
Defects found and fixed in STATUS.md:
- claimed "Hybrid VQ with k=1024: no" as the answer to the linework question,
directly contradicting FINDINGS 14, which rejected k=1024. Both profiles are
k=256.
- malformed profile table (six column separators, five columns).
- next-steps list had two items numbered 3 and listed the full-disc survey
twice.
- the disk-benchmark section still read CRITICAL-PATH with "if SCSI sustains
>=800 KB/s, ship pixel-exact". That was written while the bandwidth figure
was misread as 4 MB/s. At 4 Mbps pixel-exact needs 92-97% of the pipe and is
not available, and the ring-buffer result means the design no longer hangs on
the benchmark at all. Rewritten with what it IS still worth doing: confirming
the 4 Mbps provenance, and confirming DMA is used rather than PIO.
FINDINGS now carries supersession blockquotes on 5, 8, 11, 17 and 18 pointing
at the sections that correct them. 18 is the dangerous one -- its peak-vs-
sustained test is reversed by 21 -- so it is marked DO NOT ACT ON THIS SECTION
while noting the per-frame data itself remains valid.
profile_gen.py had the same problem in code: it defaulted to the superseded
peak sizing and returned lam=25 where the docs say lam=10. The buffered test is
now the default and peak sizing is behind --size-for-peak as a bound only. A
tool that contradicts the findings is worse than no tool.
Also preserves the five measurement scripts that produced this session's
numbers as tools/analysis/05-09, following the session 1 precedent, and adds an
"explicitly abandoned -- do not re-propose" list to STATUS covering entropy
coding, k=1024 codebooks and flat 4x4 VQ.
Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
Verified GVRAM is one word-access per pixel in ALL color modes; chose 256-color
256x192 with movem.l bursts (page 1 sacrificed as double-buffer).
Measured 8 scenes from the Blu-ray source: blit costs under 8% of the 12fps
cycle budget, so I/O is the bottleneck, not CPU. Naive delta+RLE reaches only
3.2:1 (365 KB/s, 470MB) -> decision to use 4x4 vector quantization (~30 KB/s).
"Shot on twos" assumption failed: the transfer has zero duplicate frames, so
12fps requires explicit decimation.
Documents three false measurement results and their root causes (per-frame
Floyd-Steinberg dithering, temporal denoise, exact-match dedupe on noisy source).
MAME Lua injection harness works and is reusable for cycle-cost measurement;
the IOCS _B_READ disk benchmark is blocked returning -1.
Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
prosolis