Session 5 handoff. The user has chosen rate control as the next session's work,
so this reads ratectl.py properly before that session starts rather than
discovering the problem mid-implementation.
FINDINGS 26: encode_rate_controlled() is not sound. H.encode() is temporally
recursive -- SKIP blocks copy the previous RECONSTRUCTION -- but rate control
builds a ladder of independent whole-sequence encodes and picks each frame from
whichever rung fits the budget. Frames then reference reconstructions the
decoder never saw. Measured on the Singe window: 67 rung switches, 111 of 120
frames drift, worst frame 43.4% of pixels, reported PSNR overstated by 0.36 dB.
It would have wired up cleanly and reported a plausible wrong answer.
Two further defects in the same function: the lam ladder runs to 2e5, 250x past
the FINDINGS 15 cliff, so a frame that only fits up there is destroyed rather
than rate-controlled; and with 5 rungs only two are ever chosen, straddling the
operating point by 7.5x. The docstring describes a per-frame binary search,
which is the right design -- the implementation is a fixed ladder. The leaky
bucket does work and should be kept: 109.1 KB/s against a 110 target.
tools/analysis/09_ratectl_drift.py is the regression test and the acceptance
criterion: it exits non-zero until zero frames drift.
Also corrected the stale 38% blit figure in ratectl.py's profile commentary,
which session 5 measured at 53.6% (FINDINGS 24), and recorded the pgrep -f
self-kill trap again -- four times across three sessions now.
check.sh ALL GREEN.
Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
The profiles shipped in e4062ed were set far too low. 45 KB/s (sasi) and
75 KB/s (scsi) are 12% and 7% of the respective folklore bus figures. They had
been read off the knee of the rate-distortion curve and then presented as
though bandwidth-derived, which they were not.
Raised to sasi 110 KB/s (lam=60) and scsi 280 KB/s (lam=10) -- 35% and 28%
utilisation. scsi is now within 0.52 dB of the palette ceiling on scene 00020.
Checking the CPU side, which nobody had done for the decode path, produces a
second and more important result. Against the 833k cycle/frame budget at 12fps:
full-frame blit, every frame 319k 38% affordable
LZ4/LZSS decode ~30KB/frame 450k 54%
deflate decode ~30KB/frame 1800k 216% infeasible
So raising the VQ bitrate is nearly free -- RAW, the mode that dominates at
high rate, is the cheapest mode to blit -- but entropy coding is not viable at
all. That demotes the "247 KB/s lossless changed-spans+deflate" figure from
FINDINGS 8 to a compression upper bound rather than a shippable design, and
removes entropy coding from the roadmap. VQ is the right architecture precisely
because its decode is a table copy.
Also confirms the architecture unifies: the hybrid at lam=0 lands within 3% of
the purpose-built lossless coder, so there is no separate lossless path.
Consequence for planning: the blocked disk benchmark is now critical-path, not
optional. If SCSI sustains >=800 KB/s the correct scsi profile is lam=0 --
pixel-exact video at ~450 KB/s and 38% CPU. Whether this port ships transparent
or lossy on SCSI is waiting on one measurement.
Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
Answers session 1's critical-path question. Flat 4x4 VQ at k=256 was prototyped
and REJECTED by eye: Dirk's face disintegrates and ink outlines break into
4-pixel stair-steps. The 256-colour palettised frame is excellent, so the
palette was never the problem -- block VQ was.
Replaced it with a Cinepak-style hybrid: each 4x4 block is SKIP, one 4x4
codeword, four 2x2 codewords, or RAW literal pixels, chosen per block by
rate-distortion. The RAW escape makes lam=0 pixel-exact (measured 0.00 dB loss),
so the quality knob spans lossless to heavily-compressed in one bitstream.
Per the user's decision, ships TWO quality profiles from that one codec, one
decoder and one bitstream -- only the rate knob differs:
sasi 45 KB/s lam=300 34.8 dB stock 10MHz ACE/EXPERT
scsi 75 KB/s lam=100 35.9 dB Super/XVI or CZ-6BS1
Three corrections to earlier numbers:
1. Session 1's "183 KB/s at 12fps" was a bad extrapolation. Halving the
framerate does not halve the bitrate -- decimation roughly doubles the
per-frame delta. Re-measured directly: 340 KB/s for session 1's own RLE,
247 KB/s for changed-spans+deflate. The lossless floor is 319 MB.
2. A FOURTH false-good result, same family as the three in FINDINGS 4:
k=1024 codebooks appeared to buy +2.4 dB free, because the rate model
charged 1 byte for a 10-bit index. Charging the true cost reverses the
verdict -- k=256 wins at every matched bitrate, and by 5 dB at the low end
where the SASI profile lives. k=256 ships.
3. Stream inventory: the ~3-5MB clips are 1.2-1.7s, not ~60s, and some 60s
streams are menus, not content. Any survey must classify before averaging.
Also cleared both candidate sources for the game-logic layer: the SNES project
is MIT and DirkSimple is zlib, so the arcade scene graph can be imported and
the two transcriptions diffed against each other.
Encoder is working end-to-end: extract.py -> vq/vq_hybrid/ratectl -> encode.py,
emitting a big-endian DLX1 container the 68000 can parse with plain moves.
Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
prosolis