Correct the peak analysis: both profiles fit 4 Mbps with zero prefill

The previous commit warned that scsi "does not fit 4 Mbps" because a frame
peaked at 96.4% of the sustained rate. That was the wrong test, and the user
was right to push back on it. It compared instantaneous frame demand against a
sustained rate as if they had to match frame-by-frame; the disk keeps filling
during the frame, and a ring buffer absorbs any shortfall.

Correct test is cumulative demand vs cumulative supply. Simulated on the real
per-frame sizes at a constant 488 KB/s fill:

  required prefill = 0.0 KB for EVERY scene at both profiles

Fill delivers 40.69 KB per frame time; only one measured frame (00146,
42.10 KB) exceeds it and the next frame recovers it. A 256 KB buffer -- 12.5%
of RAM -- carries ~1 second of stall tolerance, orders of magnitude more than
an SD-backed seek needs. scsi at lam=10 stands; the hardest sampled scene runs
313 KB/s mean, 64% of the pipe.

Also carries through a consequence of SD-backed deployment that session 1 noted
as a caveat but never applied: with BlueSCSI/SCSI2SD the sustained rate is a
bus-limited CONSTANT, not an average over variable seek latency, so there is no
long tail to leave margin for and we can size much closer to the ceiling.
Conservative margins here were cargo-culted from a constraint this deployment
does not have. The SASI/SCSI split remains correct because it is about bus
protocol, not media.

Rate control drops from load-bearing to insurance: intra-scene peaks are a
non-problem, but only 4 clips of 1.2-1.7s out of 224 streams have been measured
and 00146 already runs 23% hotter than 00020. The full-disc survey is now the
highest-value measurement, ahead of wiring up ratectl.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
This commit is contained in:
prosolis
2026-08-23 12:23:08 -07:00
parent 48e912de8b
commit fb8a1462b0
2 changed files with 76 additions and 10 deletions
+21 -10
View File
@@ -23,11 +23,19 @@ bitrate ceiling is a build parameter in `tools/encoder/ratectl.py`:
| `sasi` | 110 KB/s | 60 | 36.9 / 29.6 dB | stock 10MHz ACE/EXPERT |
| `scsi` | 280 KB/s | 10 | 39.4 / 32.3 dB | Super/XVI, or CZ-6BS1 board |
**WARNING — `scsi` does not currently fit 4 Mbps.** The user's working bandwidth
figure is **4 Mbps = 488 KB/s**. `scsi` means 52% of that but **peaks at 96.4%**
(FINDINGS 18), and a late frame is a dropped frame. Until `ratectl.py` is wired
into `encode.py`, `scsi` must either drop to `lam=25` (194 KB/s mean) or not
ship. Derive profiles with `tools/encoder/profile_gen.py --bw-mbps 4`, not by eye.
Sized against the user's working figure of **4 Mbps = 488 KB/s sustained**, on
SD-backed SCSI (BlueSCSI / SCSI2SD) — so that rate is a bus-limited **constant**,
not an average over seek latency.
**Both profiles fit with room.** Ring-buffer simulation on the real per-frame
sizes gives **zero required prefill** for every scene at both profiles: the fill
delivers 40.69 KB per frame time and only one measured frame (42.10 KB) exceeds
that, recovered by the next. A 256 KB buffer carries ~1 s of stall tolerance,
far more than an SD-backed seek needs. FINDINGS 21.
An earlier warning here said `scsi` did not fit because a frame peaked at 96.4%
of the pipe. That compared instantaneous demand to a sustained rate as if they
had to match frame-by-frame; with a buffer the test is cumulative, and it passes.
`scsi` is now within **0.5 dB of the palette ceiling** on 00020. These were
initially set at 45 / 75 KB/s, which was 12% / 7% bus utilisation — read off the
@@ -132,11 +140,14 @@ Next move is the untried SCSI path: `-exp1 cz6bs1 -hard disk.chd`.
## Next steps, in priority order
1. **Wire rate control into `encode.py`** — now the highest-value work in the
repo, not a loose end. It is worth a full step on the quality ladder
(lam=25 -> lam=10, +0.7/+1.2 dB) because it lets us size for the mean
instead of the peak. Validate the bucket holds on an action scene.
2. ~~Entropy-code the payload~~**ABANDONED, see FINDINGS 17.2.** Deflate
1. **Full-disc survey** — we have measured 4 clips of 1.2-1.7 s out of 224
streams, and 00146 already runs 23% hotter than 00020. A sustained action
sequence is the one thing that could still break the bitrate. This is now
the highest-value measurement, ahead of rate control.
2. **Wire rate control into `encode.py`** — no longer a blocker (FINDINGS 21),
but it is what gives a deterministic ceiling over content we have not
measured, which was the original reason for choosing VQ. Insurance, not a fix.
3. ~~Entropy-code the payload~~**ABANDONED, see FINDINGS 17.2.** Deflate
decode is ~216% of the frame budget on a 68000 and LZ4 is ~54%; there is no
room beside a 38% blit. All bitrates are raw payload. This also demotes the
"247 KB/s lossless" figure in FINDINGS 8 to a compression bound, not a design.