Size against 4 Mbps: peaks break the scsi profile; DMA steal is not free

User clarified the bandwidth figure is 4 Mbps (488 KB/s), not 4 MB/s -- ~8x
tighter than the previous commit reasoned against. Two consequences, plus a
correction to session 1.

1. The scsi profile committed in f0f2f80 DOES NOT FIT. Its mean is a
   comfortable 52% of the pipe but it PEAKS at 96.4% (470.8 KB/s on scene
   00020), and a frame that arrives late is a dropped frame, not a slow one.
   Peak/mean is 1.4-1.9x even on 1.2-1.7s clips. Sizing a real-time stream on
   the mean was the error. Flagged in STATUS rather than silently retuned,
   because the fix is rate control, not a lower lam.

   This promotes ratectl.py -- written in session 2, never wired into
   encode.py -- from a loose end to the highest-value work in the repo. It is
   worth a full step on the quality ladder (lam=25 -> lam=10, +0.7/+1.2 dB)
   because it allows sizing for the mean instead of the peak.

2. Pixel-exact is off the table at this bandwidth: lam=0 needs 92-97% of the
   pipe. The previous commit's "if SCSI sustains >=800 KB/s, ship transparent"
   conclusion only applies at roughly double the user's figure.

3. FINDINGS 5 said that because transfers are DMA, streaming "costs essentially
   no CPU" and the 68000 is "nearly idle". That is wrong. The HD63450 steals
   ~8 clocks per 16-bit word: 10-20% of the machine at the rates the profiles
   now use, on top of a 38% full-frame blit. Bandwidth and CPU are one budget.

Adds tools/encoder/profile_gen.py, which derives lam FROM a bandwidth figure
(accounting for audio, peak/mean and DMA steal) instead of reading it off the
knee of the RD curve, and docs/BENCHMARK.md covering how to actually measure
the subsystem -- including why MAME cannot answer the bandwidth question and
would be the same class of error as the FINDINGS 4 traps.

The 4 Mbps figure is user-supplied and its provenance is not recorded; every
profile now hangs off it.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
This commit is contained in:
prosolis
2026-08-23 12:12:34 -07:00
parent f0f2f807a4
commit 48e912de8b
4 changed files with 324 additions and 5 deletions
+13 -5
View File
@@ -18,10 +18,16 @@ Session 1 left "which machine do we target" open. The user's answer: **ship both
as two quality profiles. This is now implemented rather than hypothetical — the
bitrate ceiling is a build parameter in `tools/encoder/ratectl.py`:
| profile | target | lam | quality (00020 / 00146) | bus utilisation | machine |
| profile | target | lam | quality (00020 / 00146) | machine |
|---|---|---|---|---|---|
| `sasi` | 110 KB/s | 60 | 36.9 / 29.6 dB | 35% of 300 KB/s | stock 10MHz ACE/EXPERT |
| `scsi` | 280 KB/s | 10 | 39.4 / 32.3 dB | 28% of 1 MB/s | Super/XVI, or CZ-6BS1 board |
| `sasi` | 110 KB/s | 60 | 36.9 / 29.6 dB | stock 10MHz ACE/EXPERT |
| `scsi` | 280 KB/s | 10 | 39.4 / 32.3 dB | Super/XVI, or CZ-6BS1 board |
**WARNING — `scsi` does not currently fit 4 Mbps.** The user's working bandwidth
figure is **4 Mbps = 488 KB/s**. `scsi` means 52% of that but **peaks at 96.4%**
(FINDINGS 18), and a late frame is a dropped frame. Until `ratectl.py` is wired
into `encode.py`, `scsi` must either drop to `lam=25` (194 KB/s mean) or not
ship. Derive profiles with `tools/encoder/profile_gen.py --bw-mbps 4`, not by eye.
`scsi` is now within **0.5 dB of the palette ceiling** on 00020. These were
initially set at 45 / 75 KB/s, which was 12% / 7% bus utilisation — read off the
@@ -126,8 +132,10 @@ Next move is the untried SCSI path: `-exp1 cz6bs1 -hard disk.chd`.
## Next steps, in priority order
1. **Wire rate control into `encode.py`** and validate that the hard ceiling
actually holds on an action scene (the whole point of choosing VQ).
1. **Wire rate control into `encode.py`** — now the highest-value work in the
repo, not a loose end. It is worth a full step on the quality ladder
(lam=25 -> lam=10, +0.7/+1.2 dB) because it lets us size for the mean
instead of the peak. Validate the bucket holds on an action scene.
2. ~~Entropy-code the payload~~**ABANDONED, see FINDINGS 17.2.** Deflate
decode is ~216% of the frame budget on a 68000 and LZ4 is ~54%; there is no
room beside a 38% blit. All bitrates are raw payload. This also demotes the