drwily/Dragon-s-Lair-X68k
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Rate control: rebuilt per-frame, wired in, and gated at zero drift

Browse Source 
FINDINGS 26 stopped the session-5 rate controller before it shipped: it built a
lam-ladder of independent whole-sequence encodes and picked frames off it, so
SKIP blocks referenced reconstructions the decoder never saw -- 111 of 120
frames drifted. The fix is the structural one 26.1 said it had to be.

vq_hybrid is now frame-drivable -- frame_ctx / decide / paint -- and encode() is
a thin loop over it. Rate control drives the same three calls, bisects lam per
frame under the leaky bucket, and feeds back the frame it actually emitted. The
desync has no way to occur, and 09_ratectl_drift.py goes 111/120 -> 0/120. That
test is now part of check.sh, which is ~2 min rather than ~40 s.

Both overshoots on the worst sustained window are closed for under 1 dB, totals
including audio: sasi 137.4 -> 109.5 KB/s (-0.60 dB), scsi 381.6 -> 280.0 KB/s
(-0.91 dB). Zero frames hit the lam=800 cliff, so nothing was destroyed to get
there. Rate control also makes the display path cheaper -- scsi's median drops
53.6% -> 47.1% -- because raising lam moves blocks to SKIP and V1.

Two knobs measured rather than guessed. --rc-floor is worth 0.00 dB on that
window and defaults to the profile lam, so rate control cannot regress content
that already fits. --prefill defaults to 0 and is documented as a trap: it buys
a permission to overshoot of exactly bucket/nframes, and on a 14-frame clip it
disables the controller outright.

FINDINGS 26.5 was wrong in both halves and 27.6 records it. _paint was not the
bottleneck (14% of a frame, though vectorising it was still right at 17.1x) and
the ladder was never "minutes" -- those were k-means in build(). What makes
per-frame rate control affordable is that VQ.assign depends on neither lam nor
prev, so it is cached one frame deep: a 12-step search over 120 frames costs
0.31 s against 49.1 s.

Also caught: fixed-lam sasi was already 5% over target on 00020, the clip
everyone called easy. Nothing noticed because the profile table quotes PSNR and
not bitrate.

check.sh: ALL GREEN.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
This commit is contained in:
prosolis
2026-08-23 14:36:45 -07:00
parent 145753c0bf
commit 497f88b945
9 changed files with 653 additions and 194 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/FINDINGS.md
+143
View File
@@ -1073,3 +1073,146 @@ Each rung is a full-sequence encode and `_paint` is still a Python per-block
loop, so a 5-rung run over 120 frames takes minutes. **Vectorise `_paint`
first** — it is already on the list for the full-disc survey and it makes the
rate-control work practical rather than merely faster.
## 27. Rate control, rebuilt and wired in (session 6)
FINDINGS 26 stopped the session-5 rate controller before it shipped: it picked
frames out of independently-encoded whole-sequence runs, so 111 of 120 frames
referenced reconstructions the decoder would never see. The fix was structural,
as 26.1 said it had to be. It is now wired into `encode.py` and **on by
default** for a profile.
### 27.1 The encoder is frame-drivable, and the drift is zero by construction
`vq_hybrid` now exposes one frame at a time — `frame_ctx(m, f, prev)` /
`decide(ctx, lam)` / `paint(m, ctx, mode)` — and `encode()` is a thin loop over
that API. Rate control drives the same three calls and feeds back **the frame it
actually emitted** as the next frame's `prev`. There is no ladder to pick from,
so the desync has no way to occur.
`tools/analysis/09_ratectl_drift.py`, unchanged in what it asserts:
| | session 5 | session 6 |
|---|---|---|
| frames whose emitted output differs from what the encoder recorded | 111 / 120 | **0 / 120** |
| worst frame | 21,339 px (43.4%) | **0 px** |
| reported PSNR overstatement | 0.36 dB | **0.00 dB** |
This is the harder case for that test on purpose: it runs with `lam_lo=1.0`, so
lam moves on 117 of 119 frame boundaries. Under the old ladder, 67 rung switches
were enough to corrupt 111 frames.
### 27.2 Both overshoots are closed, and they cost under 1 dB
The Singe window (FINDINGS 25.3), which is the worst sustained window on the
disc. Totals include the 7.8 KB/s ADPCM allowance:
| profile | target | fixed lam (session 5) | rate-controlled | quality cost |
|---|---|---|---|---|
| `sasi` | 110 KB/s | 137.4 KB/s (**+25%**) | **109.5 KB/s** | 27.82 → 27.22 dB (−0.60) |
| `scsi` | 280 KB/s | 381.6 KB/s (**+36%**) | **280.0 KB/s** | 30.81 → 29.90 dB (−0.91) |
Zero frames hit the lam=800 cliff at either profile, so nothing was destroyed to
get there (26.2's failure mode did not trigger). `sasi` needed lam to reach 183
at worst against a floor of 60; `scsi` reached 58.7 against 10. The controller
is working an order of magnitude below the cliff, which is where the search
range being capped at 800 rather than 2e5 stops mattering at all — and that is
the point: a range that never needs its top is a range you can trust.
`scsi` still sits **1.43 dB** from the scene palette ceiling of 31.33 dB
(FINDINGS 25.4), against 0.51 dB before. The ceiling, not the codec, is still
what bounds this content.
The percentages differ from 25.3's +18%/+34% because those compared video
payload against the total target; the table above compares like with like
(total against total). The payload figures are unchanged: 129.6 and 373.8 KB/s.
### 27.3 Rate control makes the display path cheaper, not dearer
The decoder-architecture numbers of FINDINGS 25.6 were measured on the
fixed-lam encoder. Re-measured under rate control, on the same window, with the
player picking the cheaper of compose-then-blit and direct-to-GVRAM per frame:
| profile | median display cost | frames above the 70% crossover |
|---|---|---|
| `sasi` fixed → RC | 37.0% → **36.6%** | 30.0% → 26.7% |
| `scsi` fixed → RC | 53.6% → **47.1%** | 53.3% → 35.8% |
Raising lam moves blocks to SKIP and V1, which is fewer blocks to write. The
"implement both paths, pick per frame" conclusion is unaffected and the cap is
still 53.6%.
### 27.4 The quality floor barely matters; the prefill matters, wrongly
Two knobs were measured rather than guessed.
**`--rc-floor`** decides whether a quiet frame may spend more than the fixed-lam
profile would. On the Singe window it is worth nothing — 109.5 vs 110.0 KB/s and
**0.00 dB** — because no frame on that window is quiet enough for the bucket to
saturate. The default is `profile` (never spend more than session 5 would), so
rate control cannot regress content that already fits.
**`--prefill`** models how full the player's buffer is at scene start. It is
tempting and it is a trap, so it defaults to 0:
| clip | prefill 0.0 | 0.5 | 1.0 | target |
|---|---|---|---|---|
| Singe, 120 fr, `sasi` | 109.5 | 112.9 | **116.3** | 110 |
| Singe, 120 fr, `scsi` | 280.0 | 289.1 | **298.2** | 280 |
| 00020, 14 fr, `sasi` | 92.0 | **115.8** | **115.8** | 110 |
| 00020, 14 fr, `scsi` | 224.8 | **255.9** | **255.9** | 280 |
(`scsi` on 00020 is the one cell where prefill looks harmless: the clip fits
under 280 either way. That is the content being easy, not the knob being safe.)
Prefill buys a permission to overshoot of exactly `bucket / nframes`. At 8
frames of bucket over 120 frames that is 6.2% — measured — and on a 14-frame
clip the bucket is larger than the clip, so rate control switches itself off and
reproduces fixed-lam exactly (lam never leaves its floor: min = median = max =
60). **A prefill that makes a target look met has disabled the controller.**
### 27.5 The 00020 undershoot is a clip-length artefact, not a bug
At prefill 0 the 14-frame 00020 clip lands at 92.0 KB/s against a 110 ceiling —
0.66 dB given away for nothing. That is the leaky bucket's startup transient:
the first `bucket_frames` frames cannot draw on a bank they have not accumulated.
It is bounded by `bucket / nframes`, so it is 6% on a 10-second window and 20%
on a 1.2-second one.
The lesson is the one FINDINGS 25.3 already taught in a different costume: **a
1.2-second clip cannot be used to judge rate control.** Real scenes are tens of
seconds. Do not tune the bucket against 00020.
Worth recording separately: fixed-lam `sasi` on 00020 delivers 115.8 KB/s — the
supposedly easy clip was **already 5% over its target**, which nothing had
noticed because the profile table quotes its PSNR and not its bitrate.
### 27.6 FINDINGS 26.5's cost premise was wrong in both halves
26.5 said a rate-control experiment was minutes because `_paint` is a Python
per-block loop, and told the next session to vectorise it first. Vectorising it
was correct and it is **17.1x faster**, but it was never the bottleneck, and the
ladder was never minutes. Measured per frame, 256x192:
| | ms |
|---|---|
| `VQ.assign` x2 — codeword search | **22.83** |
| SKIP error against `prev` | 1.40 |
| `decide` — argmin at one lam | 0.06 |
| `paint`, vectorised | 0.29 |
| `paint`, old per-block loop | 4.93 |
`_paint` was 14% of a frame. A 5-rung ladder over 120 frames was ~18 s of
encoding, not minutes — the "few minutes" in the drift test's docstring was
`H.build`'s k-means (51 s), which no amount of vectorising `_paint` would have
touched.
What actually makes per-frame rate control affordable is that `VQ.assign`'s
output depends on **neither `lam` nor `prev`**, so it is computed once per frame
and a lam search only re-runs the 0.06 ms argmin:
| | |
|---|---|
| 12-step per-frame lam search, 120 frames, symbols cached | **0.31 s** |
| the same search by re-running whole-sequence encodes | 49.10 s |
That is a 158x difference, and it is the reason the controller can afford a real
bisection instead of a 5-rung ladder — which was the actual defect in 26.3.
The cache holds **one frame**. At ~133 KB of intermediates per frame, caching
the sequence would cost 900 MB on a 9.4-minute stream to save nothing: every
caller works a frame at a time.
docs/STATUS.md
+113 -47
View File
@@ -1,41 +1,66 @@
# Status & next-session handoff — end of session 5 (2026-08-23)
# Status & next-session handoff — end of session 6 (2026-08-23)
## NEXT SESSION: wire rate control into `encode.py`
## NEXT SESSION: the 68000 decoder skeleton
Decided with the user at the end of session 5. Everything needed is below; read
**FINDINGS 26** in full before editing `ratectl.py`, because the module does not
work the way its docstring says it does.
Rate control is done and gated (below). `src/player/` is still empty, and it is
now the only thing between this project and an answer to "does the CPU path
work". Everything it needs has been measured:
**Why it is now top of the list.** FINDINGS 25.3: on the worst sustained window
on the disc, the fixed-`lam` CLI overshoots both shipping targets — `sasi`
110 -> 129.6 KB/s (+18%), `scsi` 280 -> 373.8 KB/s (+34%). This item sat at
priority 4 marked "insurance, not a fix" for three sessions; that was true of the
1.2-1.7 s clips it was judged on and is not true of a sustained action sequence.
1. **Inner loop: implement BOTH display paths and pick per frame.** FINDINGS
25.6, re-measured under rate control in 27.3. Compose-in-RAM-then-blit is a
flat 53.6% of the 12fps budget; decode-direct-to-GVRAM is 76.6% x the
non-SKIP block fraction. They cross at 70% of blocks changed. The mode
headers are parsed before any pixel is written, so counting non-SKIP blocks
to choose is free. Median cost 36.6% (`sasi`) / 47.1% (`scsi`), capped 53.6%.
2. **Copy the harness pattern from `tools/bench/blit.s` + `blit.lua`** — it
already loads code, masks interrupts, times a loop against a flag, and
snapshots for `verify_frame256.py`. Assemble with
`tools/vasm/vasmm68k_mot -Fbin -o out.bin in.s`.
3. **Parse `DLX1`** (layout in the `encode.py` docstring, all fields big-endian),
expand the codebooks once at load, then dispatch per block on the 2-bit mode.
4. **Budget against 53.6%, not 38%.** Still not done — see priority 2a below.
The blit alone eats over half the frame before any decoding, and MAME models
no GVRAM wait states, so it is a floor.
**Do NOT just call `encode_rate_controlled()` from `encode.py`.** It is unsound
as written (FINDINGS 26.1), and it fails quietly — it returns a plausible PSNR
for a reconstruction no decoder will ever produce.
Feed it `tmp/rc_fr_singe_sasi_rcprofile.dlx` — the worst sustained window on the
disc, at the shipping profile. If the decoder fits there it fits everywhere.
Order of work:
---
1. **Vectorise `_paint`** (`vq_hybrid.py:122`, a Python per-block loop). Not
cosmetic: rate control runs the encoder once per lam rung, so everything
below is minutes-per-experiment until this is done. FINDINGS 26.5.
2. **Make `H.encode()` frame-drivable** — take `prev` and one lam, return one
frame. The current whole-sequence signature is *why* the broken ladder
exists. This is the actual fix for 26.1.
3. **Replace the fixed ladder with the per-frame binary search** the docstring
already promises, feeding back the frame actually emitted. Cap `lam_hi` at
**800**, not 2e5 — past the FINDINGS 15 cliff a frame is not rate-controlled,
it is destroyed (26.2). Keep the leaky bucket; it works (26.4).
4. **Gate on the regression test**: `python3 tools/analysis/09_ratectl_drift.py`
exits non-zero while the desync is present and must report **zero** drifting
frames after the fix. It currently reports 111/120. Needs `tmp/fr_singe`.
5. Then re-measure the Singe window at both profiles and confirm they land on
target rather than 18%/34% over.
## What session 6 settled
Only after that is the full-disc survey worth running — otherwise it measures an
encoder nobody will ship.
1. **Rate control works, is wired in, and is ON by default.** `encode.py`
bisects lam per frame under a leaky bucket; `--fixed-lam` restores session 5
behaviour. FINDINGS 27.
2. **Both overshoots are closed for under 1 dB.** On the Singe window, totals
including audio: `sasi` 137.4 -> **109.5 KB/s** (target 110) for -0.60 dB,
`scsi` 381.6 -> **280.0 KB/s** (target 280) for -0.91 dB. Zero frames hit the
lam=800 cliff at either profile. FINDINGS 27.2.
3. **The FINDINGS 26 desync is gone by construction, not by tuning.** The
encoder is frame-drivable (`vq_hybrid.frame_ctx` / `decide` / `paint`) and
rate control feeds back the frame it actually emitted. The regression test
`tools/analysis/09_ratectl_drift.py` goes 111/120 drifting frames -> **0**,
and it is now part of `./tools/bench/check.sh`. FINDINGS 27.1.
4. **Rate control makes the display path cheaper.** Raising lam moves blocks to
SKIP and V1, so there is less to write: `scsi`'s median display cost drops
53.6% -> 47.1%. The decoder conclusion of 25.6 is unaffected. FINDINGS 27.3.
5. **FINDINGS 26.5 was wrong in both halves, and this is the fifth false premise
this project has caught.** `_paint` was not the bottleneck (14% of a frame)
and the ladder was never "minutes" (~18 s; the minutes were k-means in
`build`). Vectorising it was still right — 17.1x — but what actually makes
per-frame rate control affordable is that `VQ.assign` depends on neither
`lam` nor `prev`, so it is cached: a 12-step search over 120 frames costs
**0.31 s** against 49.1 s. FINDINGS 27.6.
6. **`--prefill` is a trap and defaults to 0.** It buys a permission to overshoot
of exactly bucket/nframes; at prefill=1.0 the Singe window goes to 116.3 KB/s
against a 110 ceiling, and on a 14-frame clip it disables the controller
outright. FINDINGS 27.4.
7. **Fixed-lam `sasi` was already 5% over target on 00020**, the clip everyone
called easy — nothing noticed because the profile table quotes PSNR, not
bitrate. FINDINGS 27.5.
8. **1.2-second clips cannot be used to judge rate control.** The bucket's
startup transient is bucket/nframes: 6% on a 10 s window, 20% on 00020. Same
lesson as FINDINGS 25.3, different costume.
---
@@ -44,8 +69,9 @@ encoder nobody will ship.
```
./tools/bench/check.sh
```
~40 s, needs the Blu-ray mounted. Re-runs both display regression tests from
source media and prints `ALL GREEN`. Verified green at end of session 4.
~2 min, needs the Blu-ray mounted. Re-runs both display regression tests from
source media **and the rate-control drift test** (session 6), then prints
`ALL GREEN`. Verified green at end of session 6.
If it fails, fix that before doing anything else — everything downstream assumes
the display path is pixel-exact.
@@ -69,8 +95,14 @@ bitrate ceiling is a build parameter in `tools/encoder/ratectl.py`:
| profile | target | lam | quality (00020 / 00146) | machine |
|---|---|---|---|---|
| `sasi` | 110 KB/s | 60 | 36.9 / 29.6 dB | stock 10MHz ACE/EXPERT |
| `scsi` | 280 KB/s | 10 | 39.4 / 32.3 dB | Super/XVI, or CZ-6BS1 board |
| `sasi` | 110 KB/s | 60 (floor) | 36.9 / 29.6 dB | stock 10MHz ACE/EXPERT |
| `scsi` | 280 KB/s | 10 (floor) | 39.4 / 32.3 dB | Super/XVI, or CZ-6BS1 board |
**As of session 6 `lam` is a floor, not a setting.** The target is a ceiling and
the encoder bisects lam per frame to stay under it; the profile's lam is the
best quality it is allowed to spend on a quiet frame. On the worst sustained
window that takes `sasi` from 137.4 to 109.5 KB/s and `scsi` from 381.6 to
280.0 KB/s, for -0.60 and -0.91 dB. FINDINGS 27.2.
Sized against the user's working figure of **4 Mbps = 488 KB/s sustained**, on
SD-backed SCSI (BlueSCSI / SCSI2SD) — so that rate is a bus-limited **constant**,
@@ -228,9 +260,9 @@ python3 tools/encoder/encode.py /tmp/fr_00020 out.dlx --profile sasi --preview
multi-byte fields are **big-endian** so the 68000 reads them with a plain `move`.
### Known encoder gaps
- **Rate control is written but not yet wired into `encode.py`** — the CLI uses a
fixed `lam` from the profile. `ratectl.encode_rate_controlled()` exists and
builds a lam-ladder per frame; it needs hooking up and validating.
- ~~Rate control is written but not yet wired into `encode.py`.~~ **DONE,
session 6.** It is on by default; `--fixed-lam` restores the old behaviour.
Gated by `tools/analysis/09_ratectl_drift.py`, which is now in `check.sh`.
- **Payload is deliberately NOT entropy-coded** — deflate decode does not fit in
the 68000's frame budget (FINDINGS 17.2). Do not "optimise" this later.
- **Palette packing is not implemented in the encoder.** It still emits 24-bit
@@ -238,8 +270,11 @@ multi-byte fields are **big-endian** so the 68000 reads them with a plain `move`
palette words must pick `I` per entry by minimum squared error (FINDINGS 23.3,
worth 1.96 dB) and reserve index 0 as black with `I = 0` (FINDINGS 23.4).
- Codebooks are per-scene and rebuilt from scratch; no inter-scene reuse.
- `_paint` is a Python per-block loop — fine for prototyping, slow for a full
disc encode. Vectorise before the 224-stream run.
- ~~`_paint` is a Python per-block loop.~~ **DONE, session 6** — vectorised,
17.1x. It was never the bottleneck, though: `VQ.assign` is 78% of a frame and
`H.build`'s k-means is 51 s of a 55 s run. **That k-means is now the thing to
attack before the full-disc survey**, not anything in the per-frame path.
FINDINGS 27.6.
---
@@ -371,12 +406,16 @@ SDL_VIDEODRIVER=dummy mame x68000 -bios ipl10 -video soft -window \
scene-cut frame is ~100% non-SKIP and a held frame near 0%, and the mean of
those two is a number describing no actual frame.
1b. **Wire rate control into `encode.py`. NOW REQUIRED, and it is the agreed
next session's work** — see the **NEXT SESSION** block at the top of this
file for the ordered plan, and FINDINGS 26 for why
`encode_rate_controlled()` cannot simply be called as it stands.
1b. ~~**Wire rate control into `encode.py`.**~~ **DONE, session 6.** FINDINGS 27.
Both overshoots closed for under 1 dB, drift test at zero, `check.sh` gates
it. The remaining rate-control question is not a defect: whether `--rc-floor
open` is worth taking on quiet content. It measured as worth **0.00 dB** on
the Singe window (no frame there is quiet enough to saturate the bucket), so
it needs a genuinely quiet scene to decide, and it is a quality-per-byte
judgement rather than a correctness one.
2. **68000 decoder skeleton**, with the inner loop chosen by (1). Parse `DLX1`,
2. **68000 decoder skeleton**, with the inner loop chosen by (1). **This is now
the agreed next session's work — see the block at the top of this file.** Parse `DLX1`,
expand codebooks, blit per block mode. The display path is verified *by 68000
code* now (FINDINGS 24) and the harness pattern is `tools/bench/blit.s` +
`blit.lua`, which already loads code, masks interrupts, times a loop against
@@ -400,8 +439,10 @@ SDL_VIDEODRIVER=dummy mame x68000 -bios ipl10 -video soft -window \
- Run `tools/analysis/07_motion_survey.py` per stream first; it is cheap
(96x72 greyscale) and gives a hot-window shortlist so the expensive encode
only runs where it matters.
- **Vectorise `_paint` before this run** — it is a Python per-block loop.
- Do it **after** rate control (1b), or it measures an encoder nobody ships.
- ~~Vectorise `_paint` before this run.~~ Done. The cost to attack now is
`H.build`'s k-means: 51 s of a 55 s run, and it runs once per scene.
- ~~Do it after rate control (1b), or it measures an encoder nobody ships.~~
Rate control is in, so the survey now measures the shipping encoder.
5. **Confirm DMA vs PIO in MAME** (see the benchmark section above) — cheap, and
the only thing that could still move CPU into the binding position.
@@ -507,6 +548,31 @@ V1's output. To check that snapshot is still pixel-exact:
Not added to `check.sh`: `check.sh` asserts pixel-exactness, and asserting wall
timings there would make the green-light check sensitive to host load.
## Reproducing the rate-control result (session 6)
```
python3 tools/encoder/extract.py 00223 tmp/fr_singe 12 crop 539.4 10.0
for prof in sasi scsi; do
python3 tools/encoder/encode.py tmp/fr_singe tmp/rc_$prof.dlx --profile $prof --fixed-lam
python3 tools/encoder/encode.py tmp/fr_singe tmp/rc_$prof.dlx --profile $prof
done
python3 tools/analysis/09_ratectl_drift.py # must exit 0, zero drifting frames
```
Expected, totals including the 7.8 KB/s audio allowance: `sasi` 137.4 -> 109.5
KB/s and 27.82 -> 27.22 dB; `scsi` 381.6 -> 280.0 KB/s and 30.81 -> 29.90 dB;
zero frames at the lam=800 cliff in either. ~55 s per encode, nearly all of it
k-means in `H.build`.
The block-mode map now renders the rate-controlled encoder by default:
```
python3 tools/analysis/08_mode_map.py tmp/fr_singe tmp/singe_modes_rc.webm \
--profile sasi --scale 2 # add --fixed-lam to compare
```
**Do not judge rate control on `tmp/fr_00020`.** It is 14 frames; the leaky
bucket's startup transient is bucket/nframes, so it lands 18% under target there
for reasons that have nothing to do with the content. FINDINGS 27.5.
## Reproducing the sustained-action result (session 5)
```