Handoff: rate control is next, and it is unsound as written
Session 5 handoff. The user has chosen rate control as the next session's work, so this reads ratectl.py properly before that session starts rather than discovering the problem mid-implementation. FINDINGS 26: encode_rate_controlled() is not sound. H.encode() is temporally recursive -- SKIP blocks copy the previous RECONSTRUCTION -- but rate control builds a ladder of independent whole-sequence encodes and picks each frame from whichever rung fits the budget. Frames then reference reconstructions the decoder never saw. Measured on the Singe window: 67 rung switches, 111 of 120 frames drift, worst frame 43.4% of pixels, reported PSNR overstated by 0.36 dB. It would have wired up cleanly and reported a plausible wrong answer. Two further defects in the same function: the lam ladder runs to 2e5, 250x past the FINDINGS 15 cliff, so a frame that only fits up there is destroyed rather than rate-controlled; and with 5 rungs only two are ever chosen, straddling the operating point by 7.5x. The docstring describes a per-frame binary search, which is the right design -- the implementation is a fixed ladder. The leaky bucket does work and should be kept: 109.1 KB/s against a 110 target. tools/analysis/09_ratectl_drift.py is the regression test and the acceptance criterion: it exits non-zero until zero frames drift. Also corrected the stale 38% blit figure in ratectl.py's profile commentary, which session 5 measured at 53.6% (FINDINGS 24), and recorded the pgrep -f self-kill trap again -- four times across three sessions now. check.sh ALL GREEN. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
This commit is contained in:
prosolis
+53
-9
@@ -1,5 +1,44 @@
|
||||
# Status & next-session handoff — end of session 5 (2026-08-23)
|
||||
|
||||
## NEXT SESSION: wire rate control into `encode.py`
|
||||
|
||||
Decided with the user at the end of session 5. Everything needed is below; read
|
||||
**FINDINGS 26** in full before editing `ratectl.py`, because the module does not
|
||||
work the way its docstring says it does.
|
||||
|
||||
**Why it is now top of the list.** FINDINGS 25.3: on the worst sustained window
|
||||
on the disc, the fixed-`lam` CLI overshoots both shipping targets — `sasi`
|
||||
110 -> 129.6 KB/s (+18%), `scsi` 280 -> 373.8 KB/s (+34%). This item sat at
|
||||
priority 4 marked "insurance, not a fix" for three sessions; that was true of the
|
||||
1.2-1.7 s clips it was judged on and is not true of a sustained action sequence.
|
||||
|
||||
**Do NOT just call `encode_rate_controlled()` from `encode.py`.** It is unsound
|
||||
as written (FINDINGS 26.1), and it fails quietly — it returns a plausible PSNR
|
||||
for a reconstruction no decoder will ever produce.
|
||||
|
||||
Order of work:
|
||||
|
||||
1. **Vectorise `_paint`** (`vq_hybrid.py:122`, a Python per-block loop). Not
|
||||
cosmetic: rate control runs the encoder once per lam rung, so everything
|
||||
below is minutes-per-experiment until this is done. FINDINGS 26.5.
|
||||
2. **Make `H.encode()` frame-drivable** — take `prev` and one lam, return one
|
||||
frame. The current whole-sequence signature is *why* the broken ladder
|
||||
exists. This is the actual fix for 26.1.
|
||||
3. **Replace the fixed ladder with the per-frame binary search** the docstring
|
||||
already promises, feeding back the frame actually emitted. Cap `lam_hi` at
|
||||
**800**, not 2e5 — past the FINDINGS 15 cliff a frame is not rate-controlled,
|
||||
it is destroyed (26.2). Keep the leaky bucket; it works (26.4).
|
||||
4. **Gate on the regression test**: `python3 tools/analysis/09_ratectl_drift.py`
|
||||
exits non-zero while the desync is present and must report **zero** drifting
|
||||
frames after the fix. It currently reports 111/120. Needs `tmp/fr_singe`.
|
||||
5. Then re-measure the Singe window at both profiles and confirm they land on
|
||||
target rather than 18%/34% over.
|
||||
|
||||
Only after that is the full-disc survey worth running — otherwise it measures an
|
||||
encoder nobody will ship.
|
||||
|
||||
---
|
||||
|
||||
## Start here: is the tree still green?
|
||||
|
||||
```
|
||||
@@ -112,7 +151,12 @@ rate-distortion curve, not two codecs.
|
||||
00216 is the feature with a burned-in commentary PiP; 00215 is the commentary
|
||||
itself. **00223 (9.4 min) is the clean one.** A size-ranked survey would have
|
||||
encoded live action. FINDINGS 25.1.
|
||||
7. **On hard content the scene palette, not the display, is the binding
|
||||
7. **Rate control is unsound as written, caught before wiring it up.** The
|
||||
lam-ladder in `ratectl.py` picks frames from independent temporal chains,
|
||||
so SKIP blocks reference reconstructions the decoder never saw: 111 of 120
|
||||
frames drift, worst frame 43.4%, reported PSNR overstated 0.36 dB. Regression
|
||||
test `tools/analysis/09_ratectl_drift.py`. FINDINGS 26.
|
||||
8. **On hard content the scene palette, not the display, is the binding
|
||||
ceiling** — 31.33 dB on the Singe window against 39.90 dB on 00020 and 40.81
|
||||
dB for the X68000 display. `scsi` is already within 0.51 dB of it.
|
||||
FINDINGS 25.4.
|
||||
@@ -218,6 +262,10 @@ notifier subscription in a global; the stack register is `SP` not `A7`;
|
||||
- piping MAME (or any long job) through `grep` block-buffers — write to a file.
|
||||
- `pkill -f <pattern>` matches your own shell and kills it (exit 144).
|
||||
Use `pkill -x` or kill by PID.
|
||||
- **`pgrep -f <name> | xargs kill` kills your own shell too — exit 144.** Same
|
||||
root cause as the `pkill -f` trap above: the shell's own command line contains
|
||||
the pattern. **Hit again in session 5**, which makes it four times across three
|
||||
sessions. Kill by PID captured at launch (`$!`), or use `pkill -x`.
|
||||
- **`until ! pgrep -f foo.py; do sleep; done` watcher loops never exit.** The
|
||||
watching shell's own command line contains the string `foo.py`, so `pgrep -f`
|
||||
matches the watcher itself and the loop spins forever. Session 2 left 11 of
|
||||
@@ -323,14 +371,10 @@ SDL_VIDEODRIVER=dummy mame x68000 -bios ipl10 -video soft -window \
|
||||
scene-cut frame is ~100% non-SKIP and a held frame near 0%, and the mean of
|
||||
those two is a number describing no actual frame.
|
||||
|
||||
1b. **Wire rate control into `encode.py`. NOW REQUIRED (was priority 4).**
|
||||
FINDINGS 25.3: on the worst sustained window both profiles overshoot their
|
||||
targets with the fixed `lam` the CLI uses — `sasi` by 18%, `scsi` by 34%.
|
||||
The note that used to sit here, "no longer a blocker (FINDINGS 21)", was true
|
||||
of the 1.2-1.7 s clips measured at the time and is not true of this one.
|
||||
`ratectl.encode_rate_controlled()` already builds a per-frame lam ladder; it
|
||||
has never been hooked up. Do this before the full-disc survey or the survey
|
||||
measures an encoder nobody will ship.
|
||||
1b. **Wire rate control into `encode.py`. NOW REQUIRED, and it is the agreed
|
||||
next session's work** — see the **NEXT SESSION** block at the top of this
|
||||
file for the ordered plan, and FINDINGS 26 for why
|
||||
`encode_rate_controlled()` cannot simply be called as it stands.
|
||||
|
||||
2. **68000 decoder skeleton**, with the inner loop chosen by (1). Parse `DLX1`,
|
||||
expand codebooks, blit per block mode. The display path is verified *by 68000
|
||||
|
||||
Reference in New Issue
Block a user