Correct the peak analysis: both profiles fit 4 Mbps with zero prefill

The previous commit warned that scsi "does not fit 4 Mbps" because a frame
peaked at 96.4% of the sustained rate. That was the wrong test, and the user
was right to push back on it. It compared instantaneous frame demand against a
sustained rate as if they had to match frame-by-frame; the disk keeps filling
during the frame, and a ring buffer absorbs any shortfall.

Correct test is cumulative demand vs cumulative supply. Simulated on the real
per-frame sizes at a constant 488 KB/s fill:

  required prefill = 0.0 KB for EVERY scene at both profiles

Fill delivers 40.69 KB per frame time; only one measured frame (00146,
42.10 KB) exceeds it and the next frame recovers it. A 256 KB buffer -- 12.5%
of RAM -- carries ~1 second of stall tolerance, orders of magnitude more than
an SD-backed seek needs. scsi at lam=10 stands; the hardest sampled scene runs
313 KB/s mean, 64% of the pipe.

Also carries through a consequence of SD-backed deployment that session 1 noted
as a caveat but never applied: with BlueSCSI/SCSI2SD the sustained rate is a
bus-limited CONSTANT, not an average over variable seek latency, so there is no
long tail to leave margin for and we can size much closer to the ceiling.
Conservative margins here were cargo-culted from a constraint this deployment
does not have. The SASI/SCSI split remains correct because it is about bus
protocol, not media.

Rate control drops from load-bearing to insurance: intra-scene peaks are a
non-problem, but only 4 clips of 1.2-1.7s out of 224 streams have been measured
and 00146 already runs 23% hotter than 00020. The full-disc survey is now the
highest-value measurement, ahead of wiring up ratectl.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
This commit is contained in:
prosolis
2026-08-23 12:23:08 -07:00
parent 48e912de8b
commit fb8a1462b0
2 changed files with 76 additions and 10 deletions
+55
View File
@@ -511,3 +511,58 @@ the bandwidth question — is in `docs/BENCHMARK.md`.
The 4 Mbps figure itself is **user-supplied and its provenance is not recorded**.
Every profile now hangs off it, so it is worth pinning down.
## 21. Correction to 18 — the peak test was the wrong test
18 flagged that `scsi` "does not fit 4 Mbps" because a frame peaked at 96.4% of
the sustained rate. **That was the wrong comparison**, and the user was right to
push back. It measured instantaneous frame demand against a sustained rate as if
they had to match frame-by-frame. They do not: the disk keeps filling *during*
the frame, and any shortfall is absorbed by a ring buffer.
The correct test is whether **cumulative** demand ever outruns cumulative supply.
Simulated at a constant 488 KB/s fill, 12 fps, using the real per-frame sizes:
| scene | lam | mean KB/s | worst frame | **required prefill** | stall tolerance @256KB |
|---|---|---|---|---|---|
| 00010 | 10 | 198.9 | 23.67 KB | **0.0 KB** | 15.4 frames |
| 00020 | 10 | 255.9 | 39.23 KB | **0.0 KB** | 12.0 frames |
| 00146 | 10 | **313.0** | 42.10 KB | **0.0 KB** | 9.8 frames |
| 00181 | 10 | 211.1 | 25.25 KB | **0.0 KB** | 14.6 frames |
| (all) | 60 | 95-116 | 11-19 KB | **0.0 KB** | 26-32 frames |
Fill delivers **40.69 KB per frame time**. Only one measured frame exceeds that
(00146, 42.10 KB) and it is recovered by the following frame. **No scene needs
any prefill at all**, and a 256 KB buffer — 12.5% of RAM — carries ~1 second of
stall tolerance at `lam=10`, which is orders of magnitude more than an SD-backed
seek requires.
`scsi` at `lam=10` stands. The hardest scene sampled (00146) runs 313 KB/s mean,
64% of the pipe, with zero underrun risk.
### Why SD-backed changes the sizing rule
The deployment target is BlueSCSI / SCSI2SD, not a period spinning drive. That
was noted as a caveat in 5 but its consequence was not carried through:
- The sustained rate is a **bus-limited constant**, not an average over variable
seek latency. There is no long tail to leave margin for.
- Seek is ~microseconds, so branch-point stalls are a non-issue against a buffer
measured in whole seconds.
- Therefore we can size much closer to the ceiling than spinning-disk practice
would suggest. Conservative margins here are cargo-culted from a constraint
this deployment does not have.
**The SASI/SCSI split is about BUS PROTOCOL, not media.** SD emulation removes
seek latency from both, but a SASI bus is still slower than a SCSI one. Two
profiles remain the right design; both are now predictable constants rather than
distributions.
### What rate control is actually for now
Its value drops from load-bearing to **insurance**. Intra-scene peaks are a
non-problem. But we have measured **4 clips of 1.2-1.7s** out of 224 streams, and
00146 already runs 23% hotter than 00020. A sustained action sequence could
plausibly exceed the pipe where a 1.7s clip does not. Rate control gives a
*deterministic ceiling* across content we have not measured yet — which was the
original reason for choosing VQ over a lossless delta in the first place.
Still worth wiring in. No longer a blocker for shipping `scsi` at `lam=10`.