prosolis
2f9f5cc995
Two sessions that were never separated in the working tree, so they land as one commit. check.sh ALL GREEN before and after both. SESSION 19 -- the ring rig gets a frame clock (FINDINGS 51). src/player/stream.s had no frame clock: it asked for record i the instant it finished i-1, outran any finite pipe, and never let the ring back up. The 49.1 sweep passing at 48 KB was therefore a wrap-correctness result and nothing else. PACE/PACEON ($18034/$18038) hold the decoder to 12 fps, so FR_HEAD-FR_TAIL finally means what it reads as: whole frames the decoder could still draw with delivery stopped dead. PACEON=0 free-runs and is what the wrap gate still uses, so every figure in 49 is unmoved. Paced, on the gate container: 64 KB holds 2 frames, 256 KB holds 7-8, 512 KB holds 14-15, all pixel-exact. Tolerance is ceiling-1, measured by cutting the pipe: 256 KB buys 500 ms of dead pipe, not 583. SLACK IS ACCUMULATED, NOT OWNED. It is built out of pipe-wire and a seek spends all of it. At 488 KB/s a 256 KB ring needs 4.83 s of play to reach its ceiling from empty; 512 KB needs 8.42 s to reach 14. A bigger ring raises the ceiling AND lengthens the climb, so a branch point does not ask "is the buffer big enough" but "has there been enough play since the last one" -- and Dragon's Lair's decision points are seconds apart. The rig now also says WHICH resource is binding: at 460 KB/s every ring from 192 KB to 512 KB is rate-bound at ceiling 4 and never fills, so larger rings are dead RAM in that scene. 20_seek_slack.py is the same model rewritten in Python from record sizes, sharing no code with the Lua producer: 35/35 ceilings inside its bracket. SESSION 20 -- the DMAC configuration was in the IPL ROM the whole time (FINDINGS 52). ROADMAP's "do this first" was to put the ADPCM stream on the bus. That needs a clocks-per-byte figure for the audio channel, and 11_cpu_budget.py was charging audio the DISK's rate -- 5 clk/B, its own help text calling it "single-address, bus held". Audio was being charged the favourable end of B3, a 242 KB/s open question. It never had to be a guess. The IPL ROM programs all four HD63450 channels itself and MAME boots the rig with it, so 21_iplrom_dmac.py reads the configuration out of the image and decodes the MC68450 fields. Eight (address, expected bytes, meaning) sites; a mismatch or an unknown revision exits non-zero. In check.sh, no emulator, milliseconds. ch3 DCR=$80, OCR=$32: dual address, 8-bit port, cycle steal WITHOUT hold, REQG=10 external request. The DMAC arbitrates once per byte with no burst to amortise the 5..8 + 2 over, so an audio byte is 16..19 clocks, not 5 -- the old debit was 3.2x..3.8x small. And on the bus it is still nothing: 651 B/frame is 1.25%..1.48% of a frame, about 4% of what the decoder leaves. P6's bus risk does not materialise. The unit worry was worth checking and nearly right: 15.6 kHz is 8 MHz/512 = 15,625 samples/s, two 4-bit samples to a byte = 7,812.5 B/s exactly, and AUDIO_KBPS=7.8 is that in decimal kB while the tool multiplied by 1024. THE DISK CHANNEL IS PROGRAMMED IDENTICALLY. ch1 (SASI) is DCR=$80 too, and so is ch0. That is 16..19 clocks per delivered byte, where 42.4 brackets W at 5..12 and 42.5 has W=8 already missing 47/120 frames. The only worked example of a disk DMA configuration on this machine sits above the entire bracket, and at that price nothing fits at any container size. It is not scsiexrom.bin so B3 stays open -- what changed is that a cheap configuration is now the thing that has to be SHOWN. W <= 12 is a requirement on the player's DMAC programming, not a range the hardware hands us, and it is now the largest open number in the project, ahead of the rate. An unforced cross-check fell out: 15_bus_occupancy.py's new W sweep puts W=8 at 105.7% of the frame, agreeing with 42.5's 47/120, from mode histograms and bus clocks respectively, two models sharing no code. Also: ADPCM outranks the disk at the arbiter (CPR 1 against 2), so an audio byte never waits and a video byte does -- relevant to 51's smooth-rate delivery model. README MEDIA. stream.lua gains DLX_SNAP_EVERY=1 (needs DLX_PACE, off by default, on no path check.sh takes) and tools/media/make_readme_media.py turns the PNGs into docs/img/. The stills and both clips are MAME's own screen pixels. Building it turned up something worth recording. 116 of 119 captured frames are pixel-exact against dlx.py; three are TORN -- frame n on top, frame n-1 below the tear line -- because MAME captured the screen while the block loop was partway down it. decode.s writes straight to the displayed page (one display path, 28.1), so a real player tears the same way, and this is the first time that consequence has been visible rather than argued. The script ASSERTS the tear and refuses to build otherwise, rather than trimming three frames and reporting "every frame I kept is exact". Second correction the capture forced: the snapshot fires before frame n is decoded, so the obvious reading is that it holds frame n-1 -- it does not, because MAME renders the screen at the end of the machine frame, by which time the 68000 has finished frame n. 11_cpu_budget.py's "validated to within 1 pt" line is also corrected: the model reads 2..10 pt HIGH and by more as the frame gets harder, which was already true before either session. src/player/decode.s is unchanged; decode.bin is still 1,296 B at the same MD5. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
160 lines
7.7 KiB
Markdown
160 lines
7.7 KiB
Markdown
# Benchmarking the storage subsystem, and deriving profiles from it
|
|
|
|
Written session 2, in answer to "how do we benchmark the SCSI subsystem itself
|
|
and base our performance profiles around that?"
|
|
|
|
## The short answer
|
|
|
|
**You cannot set a bitrate profile from MAME.** MAME's `x68k_hdc` (SASI) and
|
|
`mb89352`/`cz6bs1` (SCSI) are *functional* models — they move the right bytes
|
|
and raise the right interrupts, but they are not transfer-timing accurate. A
|
|
throughput number out of MAME measures how fast the emulator's device model
|
|
hands over a buffer, which is an artefact of MAME's scheduling, not of a
|
|
Fujitsu MB89352 on a 10MHz bus.
|
|
|
|
So split the question in two, because they need different instruments:
|
|
|
|
| question | instrument | what it settles |
|
|
|---|---|---|
|
|
| does our read path work at all? | MAME | correctness, IOCS vs direct SPC, DMA setup |
|
|
| what rate does the hardware sustain? | derivation + real hardware | the profile bitrates |
|
|
|
|
Using MAME for the second is the same class of error as FINDINGS 4: a number
|
|
that looks like a measurement but is an artefact of the apparatus.
|
|
|
|
## Tier 1 — MAME: validate the path, not the speed
|
|
|
|
This is what `tools/bench/` already does, and what is currently blocked
|
|
(`IOCS _B_READ` returns -1 uniformly). Its value is that it proves the
|
|
request/DMA/completion loop is correct before any of it is burned into 68000
|
|
player code.
|
|
|
|
Next moves, in order — the SCSI path was never tried and is more relevant to
|
|
the target anyway:
|
|
|
|
1. **SCSI instead of SASI.** `-exp1 cz6bs1 -hard disk.chd`, with
|
|
`exp1:cz6bs1:scsi:0 harddisk`. Use IOCS `_S_READ` ($F5) rather than
|
|
`_B_READ` ($46).
|
|
2. **Move the stack.** `SP=$8000` may sit on top of the IOCS work area in low
|
|
RAM; put it at $200000+ (hypothesis 3 from session 1).
|
|
3. **Format the image.** Hypothesis 1 — a raw image has no X68000 partition
|
|
structure, so the IPL's boot scan never registers a drive and IOCS refuses.
|
|
Needs a Human68k image, which this machine does not have.
|
|
4. **Bypass IOCS entirely** and drive the MB89352 SPC registers directly. This
|
|
is what the shipping player will do anyway, since we want DMA straight into
|
|
a ring buffer with no OS in the path. If direct SPC works while IOCS does
|
|
not, that is a complete answer to the blocker and we simply skip IOCS.
|
|
|
|
Record from MAME: bytes transferred, completion status, and whether DMA or PIO
|
|
was used. **Do not record KB/s and treat it as a hardware figure.**
|
|
|
|
## Tier 2 — derivation: the defensible ceiling
|
|
|
|
Already partly in FINDINGS 5. Bounds worth tightening from datasheets:
|
|
|
|
- 68000 bus cycle: 4 clocks @ 10MHz, 16-bit => **5 MB/s** absolute ceiling
|
|
- HD63450 single-address DMA, **5 clocks/BYTE** => **2.0 MB/s** practical ceiling
|
|
(dual-address is 9 clocks/byte => 1.11 MB/s). CORRECTED session 14: this line
|
|
read "~8 clocks/word => ~2.5 MB/s", which charged a byte-wide SPC per word.
|
|
FINDINGS 43.
|
|
- SCSI-1 asynchronous REQ/ACK handshake per byte, plus MB89352 FIFO depth
|
|
=> the real limiter, and the number we do not have from a primary source
|
|
|
|
**RETIRED, session 18 (USER DECISION).** This document used to name a working
|
|
figure of "4 Mbps" here and note that every profile hung off it. It was never a
|
|
bus measurement — user-supplied, no provenance, and 10% of SCSI-1's asynchronous
|
|
rating (FINDINGS 42.1). It has been removed as a default from every analysis
|
|
tool and from `tools/bench/stream.lua`; the tools now REQUIRE an explicit rate,
|
|
so nothing can be scored against a figure the scorer never restates.
|
|
|
|
**There is no working delivery figure. That is the honest state, and it is the
|
|
point:** the rate is a property of the medium, the medium is a BlueSCSI, and it
|
|
has not been measured. `tools/analysis/19_ring_stream.py` reports the
|
|
**zero-prefill pipe** — the rate a medium must clear for a given container to
|
|
need no prefill at all — which is the threshold a measurement should be taken
|
|
against. For the session-14 candidate that is **513.2 KB/s** (FINDINGS 49.5).
|
|
|
|
One place still carries the old number: `GATE_SPAN_KBPS` in
|
|
`tools/bench/check.sh`, because the gate container was *encoded* with it and
|
|
every per-block and span constant in FINDINGS 41/43/45/49 is fitted to that
|
|
container. It is a container recipe, not a claim about any medium.
|
|
|
|
### The coupling nobody had counted
|
|
Cycle-stealing DMA is not free DMA. At ~8 clocks per 16-bit word:
|
|
|
|
| stream | CPU stolen | + full-frame blit (38.3%) |
|
|
|---|---|---|
|
|
| 110 KB/s | 4.5% | 42.8% |
|
|
| 250 KB/s | 10.2% | 48.5% |
|
|
| 450 KB/s | 18.4% | 56.7% |
|
|
| ~490 KB/s | 20.0% | 58.3% |
|
|
|
|
FINDINGS 5 concluded that because transfers are DMA, "streaming costs
|
|
essentially no CPU". **That is wrong.** It costs up to a fifth of the machine at
|
|
the rates we now care about. Bandwidth and CPU are one budget, not two.
|
|
|
|
## Tier 3 — real hardware: the only thing that settles it
|
|
|
|
An X68000 (ACE/EXPERT for SASI, Super/XVI or a CZ-6BS1-equipped 10MHz machine
|
|
for SCSI) with a **BlueSCSI or SCSI2SD**, which is the realistic deployment
|
|
anyway and removes mechanical seek from the measurement.
|
|
|
|
The benchmark must measure **what the player actually does**, not a synthetic
|
|
bulk read:
|
|
|
|
1. Sequential read into a ring buffer, in the chunk size the player will use.
|
|
2. **With the decoder running** — so DMA/CPU contention is included. An idle-CPU
|
|
bulk read will overstate the sustained rate by roughly the blit percentage.
|
|
3. Timed with the machine's own timer (MFP timer-C or the 1/100s system clock),
|
|
not a stopwatch.
|
|
4. Reported as sustained KB/s over >=30s, plus the worst 1-second window. The
|
|
worst window is what the profile must survive, since a frame that arrives
|
|
late is a dropped frame.
|
|
|
|
Deliverable: a `.x` executable and its source in `tools/bench/`, runnable on
|
|
real hardware and reporting a single number.
|
|
|
|
## Feeding the result back into the profiles
|
|
|
|
`tools/encoder/profile_gen.py` inverts the dependency — give it a bandwidth and
|
|
it returns the lam that fits, from the MEASURED rate-distortion points in
|
|
FINDINGS 17.4:
|
|
|
|
```
|
|
python3 tools/encoder/profile_gen.py --bw-mbps 4 --name scsi
|
|
```
|
|
|
|
It accounts for what eats the pipe before video sees any of it: audio
|
|
(7.8 KB/s), the buffering condition, and it reports the DMA cycle-steal so the
|
|
CPU coupling stays visible.
|
|
|
|
At 4 Mbps it returns:
|
|
|
|
| sizing rule | lam | mean | 00020 | 00146 | CPU |
|
|
|---|---|---|---|---|---|
|
|
| **buffered (default, FINDINGS 21)** | **10** | 305 KB/s | -0.52 dB | -2.98 dB | 51% |
|
|
| `--size-for-peak` (FINDINGS 18, superseded) | 25 | 194 KB/s | -1.22 dB | -4.21 dB | 46% |
|
|
|
|
The default is the buffered test: cumulative demand vs cumulative supply.
|
|
Ring-buffer simulation gives **zero required prefill** for every measured scene,
|
|
so `lam=10` ships without rate control. `--size-for-peak` reproduces the earlier
|
|
pessimistic sizing and is kept only as a bound.
|
|
|
|
**Rate control is therefore insurance, not a fix.** Its value is a deterministic
|
|
ceiling over the 220 streams not yet measured — see the survey caveat below.
|
|
|
|
## What would change the design
|
|
|
|
- **If sustained is much below 4 Mbps** (say 2 Mbps / 244 KB/s): `scsi`
|
|
collapses toward today's `sasi`, and the two profiles stop being meaningfully
|
|
different. At that point reconsider 10 fps, or a narrower active area.
|
|
- **If sustained is much above** (>=8 Mbps / 976 KB/s): `lam=0` fits with
|
|
margin and the port ships **pixel-exact** video on SCSI. At the current
|
|
4 Mbps figure this is NOT available — `lam=0` needs 92-97% of the pipe.
|
|
- **If the full-disc survey finds a sustained action sequence hotter than
|
|
00146** (313 KB/s mean, the worst of 4 clips sampled): that is the scenario
|
|
rate control exists for, and the reason to wire it up before the survey run.
|
|
- **If DMA cannot be used** and transfers fall back to PIO, the CPU cost rises
|
|
from ~15% to something far larger and CPU becomes the binding constraint.
|
|
This is the single worst outcome and is worth checking early in Tier 1.
|