diff --git a/docs/FINDINGS.md b/docs/FINDINGS.md index eba283b..4746317 100644 --- a/docs/FINDINGS.md +++ b/docs/FINDINGS.md @@ -511,3 +511,58 @@ the bandwidth question — is in `docs/BENCHMARK.md`. The 4 Mbps figure itself is **user-supplied and its provenance is not recorded**. Every profile now hangs off it, so it is worth pinning down. + +## 21. Correction to 18 — the peak test was the wrong test + +18 flagged that `scsi` "does not fit 4 Mbps" because a frame peaked at 96.4% of +the sustained rate. **That was the wrong comparison**, and the user was right to +push back. It measured instantaneous frame demand against a sustained rate as if +they had to match frame-by-frame. They do not: the disk keeps filling *during* +the frame, and any shortfall is absorbed by a ring buffer. + +The correct test is whether **cumulative** demand ever outruns cumulative supply. +Simulated at a constant 488 KB/s fill, 12 fps, using the real per-frame sizes: + +| scene | lam | mean KB/s | worst frame | **required prefill** | stall tolerance @256KB | +|---|---|---|---|---|---| +| 00010 | 10 | 198.9 | 23.67 KB | **0.0 KB** | 15.4 frames | +| 00020 | 10 | 255.9 | 39.23 KB | **0.0 KB** | 12.0 frames | +| 00146 | 10 | **313.0** | 42.10 KB | **0.0 KB** | 9.8 frames | +| 00181 | 10 | 211.1 | 25.25 KB | **0.0 KB** | 14.6 frames | +| (all) | 60 | 95-116 | 11-19 KB | **0.0 KB** | 26-32 frames | + +Fill delivers **40.69 KB per frame time**. Only one measured frame exceeds that +(00146, 42.10 KB) and it is recovered by the following frame. **No scene needs +any prefill at all**, and a 256 KB buffer — 12.5% of RAM — carries ~1 second of +stall tolerance at `lam=10`, which is orders of magnitude more than an SD-backed +seek requires. + +`scsi` at `lam=10` stands. The hardest scene sampled (00146) runs 313 KB/s mean, +64% of the pipe, with zero underrun risk. + +### Why SD-backed changes the sizing rule +The deployment target is BlueSCSI / SCSI2SD, not a period spinning drive. That +was noted as a caveat in 5 but its consequence was not carried through: + +- The sustained rate is a **bus-limited constant**, not an average over variable + seek latency. There is no long tail to leave margin for. +- Seek is ~microseconds, so branch-point stalls are a non-issue against a buffer + measured in whole seconds. +- Therefore we can size much closer to the ceiling than spinning-disk practice + would suggest. Conservative margins here are cargo-culted from a constraint + this deployment does not have. + +**The SASI/SCSI split is about BUS PROTOCOL, not media.** SD emulation removes +seek latency from both, but a SASI bus is still slower than a SCSI one. Two +profiles remain the right design; both are now predictable constants rather than +distributions. + +### What rate control is actually for now +Its value drops from load-bearing to **insurance**. Intra-scene peaks are a +non-problem. But we have measured **4 clips of 1.2-1.7s** out of 224 streams, and +00146 already runs 23% hotter than 00020. A sustained action sequence could +plausibly exceed the pipe where a 1.7s clip does not. Rate control gives a +*deterministic ceiling* across content we have not measured yet — which was the +original reason for choosing VQ over a lossless delta in the first place. + +Still worth wiring in. No longer a blocker for shipping `scsi` at `lam=10`. diff --git a/docs/STATUS.md b/docs/STATUS.md index 7d8780f..37dc6f5 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -23,11 +23,19 @@ bitrate ceiling is a build parameter in `tools/encoder/ratectl.py`: | `sasi` | 110 KB/s | 60 | 36.9 / 29.6 dB | stock 10MHz ACE/EXPERT | | `scsi` | 280 KB/s | 10 | 39.4 / 32.3 dB | Super/XVI, or CZ-6BS1 board | -**WARNING — `scsi` does not currently fit 4 Mbps.** The user's working bandwidth -figure is **4 Mbps = 488 KB/s**. `scsi` means 52% of that but **peaks at 96.4%** -(FINDINGS 18), and a late frame is a dropped frame. Until `ratectl.py` is wired -into `encode.py`, `scsi` must either drop to `lam=25` (194 KB/s mean) or not -ship. Derive profiles with `tools/encoder/profile_gen.py --bw-mbps 4`, not by eye. +Sized against the user's working figure of **4 Mbps = 488 KB/s sustained**, on +SD-backed SCSI (BlueSCSI / SCSI2SD) — so that rate is a bus-limited **constant**, +not an average over seek latency. + +**Both profiles fit with room.** Ring-buffer simulation on the real per-frame +sizes gives **zero required prefill** for every scene at both profiles: the fill +delivers 40.69 KB per frame time and only one measured frame (42.10 KB) exceeds +that, recovered by the next. A 256 KB buffer carries ~1 s of stall tolerance, +far more than an SD-backed seek needs. FINDINGS 21. + +An earlier warning here said `scsi` did not fit because a frame peaked at 96.4% +of the pipe. That compared instantaneous demand to a sustained rate as if they +had to match frame-by-frame; with a buffer the test is cumulative, and it passes. `scsi` is now within **0.5 dB of the palette ceiling** on 00020. These were initially set at 45 / 75 KB/s, which was 12% / 7% bus utilisation — read off the @@ -132,11 +140,14 @@ Next move is the untried SCSI path: `-exp1 cz6bs1 -hard disk.chd`. ## Next steps, in priority order -1. **Wire rate control into `encode.py`** — now the highest-value work in the - repo, not a loose end. It is worth a full step on the quality ladder - (lam=25 -> lam=10, +0.7/+1.2 dB) because it lets us size for the mean - instead of the peak. Validate the bucket holds on an action scene. -2. ~~Entropy-code the payload~~ — **ABANDONED, see FINDINGS 17.2.** Deflate +1. **Full-disc survey** — we have measured 4 clips of 1.2-1.7 s out of 224 + streams, and 00146 already runs 23% hotter than 00020. A sustained action + sequence is the one thing that could still break the bitrate. This is now + the highest-value measurement, ahead of rate control. +2. **Wire rate control into `encode.py`** — no longer a blocker (FINDINGS 21), + but it is what gives a deterministic ceiling over content we have not + measured, which was the original reason for choosing VQ. Insurance, not a fix. +3. ~~Entropy-code the payload~~ — **ABANDONED, see FINDINGS 17.2.** Deflate decode is ~216% of the frame budget on a 68000 and LZ4 is ~54%; there is no room beside a 38% blit. All bitrates are raw payload. This also demotes the "247 KB/s lossless" figure in FINDINGS 8 to a compression bound, not a design.