diff --git a/README.md b/README.md index c7ee990..90e595a 100644 --- a/README.md +++ b/README.md @@ -6,6 +6,56 @@ This is fundamentally a **video codec problem**, not a game-logic problem: the game logic is a scene table with branching input windows; the difficulty is pushing ~22 minutes of Don Bluth animation through a 10MHz 68000. +--- + +## What it looks like + +![Blu-ray source next to the 68000's output](docs/img/source-vs-decoded.png) + +Left, the Blu-ray frame cropped to 256x192. Right, **the same frame as the +emulated 68000 actually drew it** — 256 colours out of the X68000's 65536, one +16-colour-per-4x4-block codebook, decoded by `src/player/decode.s` from the +container. Not a re-render: these are the pixels MAME had on screen, extracted +from its own snapshot. 2x nearest-neighbour, no filtering. + +**The player, running.** 119 frames out of a **256 KB ring buffer on an emulated +stock 2 MB X68000**, paced to a 12 fps frame clock, streamed from a host file at +488 KB/s — `src/player/stream.s`, no Lua in the decode path. Source on the left, +the machine's screen on the right. + + + +[`docs/img/player.webm`](docs/img/player.webm) — 119 frames, 12 fps, VP9 + +116 of those 119 frames are **pixel-exact** against `tools/encoder/dlx.py`'s +reference reconstruction. The other three are **torn** — the top of the picture +is frame *n* and the bottom still holds frame *n-1*, because MAME captured the +screen while the block loop was partway down it. That is not a rig artefact: +`decode.s` writes straight to the displayed page (one display path, FINDINGS +28.1 — the dual-path plan is kept runnable as a counterexample precisely because +it corrupts frames), so a real player tears the same way. +`tools/media/make_readme_media.py` asserts the tear rather than trimming it — +every differing pixel has to come from the previous frame, or it refuses to +build. + +**What the decoder is actually doing.** The same window with the block-mode map +beside it: **black = SKIP** (costs nothing, draws nothing — the previous frame +stands), **blue = V1** (one codebook index for a whole 4x4 block), **amber = V4** +(four indices), **red = RAW** (sixteen bytes verbatim). The mode mix is what +every cost table in `docs/FINDINGS.md` is really about — V4 costs 1.5x V1, and +since session 8 the mode decision is charged both bytes *and* cycles, which is +why a byte-rich profile buys its way out to RAW instead of V4. + + + +[`docs/img/modes.webm`](docs/img/modes.webm) — the same 119 frames with the mode map + +**Name the layer:** everything above is **emulated** (MAME 0.277 `x68000`, +`-bios ipl10`, stock 10 MHz / 2 MB), cross-checked frame-for-frame on a second +CPU core (px68k's C68K). Nothing in this project has run on real hardware yet. + +--- + **And the binding resource is the 68000's local BUS, not its clock.** The decoder occupies 86.7% of it once instruction prefetch is counted, and 52 of the 53 frames that miss the 12fps budget miss it on the bus, not the CPU @@ -15,11 +65,60 @@ The **literal span with a fine tail** (v7) is now IN the player: `decode.s` paints it, pixel-exact under both CPU cores, and it costs inside the decoder what `blit.s` said it would to 0.2% (FINDINGS 41). -**It only pays if the stream is allowed to run near the pipe.** Spans buy the -68000's deadline with bytes, and at the 280 KB/s profile the mode decision has -already spent them: 77/120 frames over budget against 86 without spans. Given -the full 488 KB/s pipe it is **34/120, and 0.36 dB better** — so the next -decision is a rate point, not an optimisation (FINDINGS 41.2, docs/STATUS.md). +**The delivery path is built and tested too** (FINDINGS 49). `src/player/stream.s` +decodes the whole 120-frame window **out of a 256 KB ring on a stock 2 MB +machine**, final frame pixel-exact, with the container in a host file rather than +preloaded into RAM. The constraint turned out to be **contiguity, not byte +count** — the block loop reads with a monotonically increasing `a0` and no bounds +check, so the ring needs the whole next record resident *and contiguous*, which +is a condition no byte-counting buffer simulation can see. + +**And building it caught a live defect**, then cost the project a constant. +The shipping candidate is 496.7 KB/s; the pipe figure the design had been +simulated against since session 2 was smaller, and nothing in the tree was +comparing the two — the rate controller binds on clocks and has no pipe term at +all, while the buffer sizing kept standing on a constant the design had stopped +enforcing. + +**So the pipe constant is retired (session 18, USER DECISION).** It was never a +bus measurement — user-supplied, no provenance, 10% of SCSI-1's asynchronous +rating (FINDINGS 42.1). It is gone as a default from every analysis tool and +from `stream.lua`; `--bus` / `--kbps` / `DLX_STREAM_KBPS` are now **required +arguments**, so no table can be scored against a rate its own output does not +state. **There is no working delivery figure, and that is the honest state.** + +What replaces it is a requirement rather than a constant: +`tools/analysis/19_ring_stream.py` reports the **zero-prefill pipe**, the rate a +medium must clear for a container to need no prefill. For the candidate that is +**513.2 KB/s** — a hardware acceptance test to measure a BlueSCSI against. + +**Bytes are not free, and the number that said they were was in the wrong +unit.** Session 13 found the pipe figure the design was built against was never +a bus figure (SCSI-1 is 1.5 MB/s asynchronous) and concluded the span pass +saturates at ~837 KB/s, 0/120 frames over budget. Session 14 found the disk +debit behind that was charged **per word of stream to a byte-wide port** — the +MB89352 is an 8-bit SPC, so the DMAC pays per BYTE, and the debit is 2x every +table since FINDINGS 5. No 68000 bus cycle is shorter than four clocks, so the +old figure was below a physical floor. + +**What survives, re-encoded honestly: 496.7 KB/s at 29.19 dB, 1 frame of 120 +over the 12fps budget** — and that one is frame 0, the intra frame, late on +purpose. The remaining lever is not ours: **whether the CZ-6BS1 wires the SPC's +DACK to the bus's `#EXACK`**, which decides 5 clocks/byte against 9, and with +it 242 KB/s and 0.69 dB. Read FINDINGS 43 before quoting any rate figure. + +**And session 20 read the answer the machine already had.** The X68000's IPL ROM +programs all four HD63450 channels itself, and MAME boots the rig with it, so +`tools/analysis/21_iplrom_dmac.py` decodes the configuration straight out of the +image and gates on the bytes still being there. The audio channel is +dual-address, 8-bit port, cycle steal **without hold**, one external request per +byte: **16..19 clocks a byte, not 5** — which prices the ADPCM stream at +1.25%..1.48% of a frame and closes ROADMAP's "do this first" item. The disk +channel is programmed **identically**. That is 16..19 clocks per delivered byte, +above the whole 5..12 bracket the project costs the transport in, and at that +price nothing fits. It is SASI and not the MB89352, so it does not settle the +question — but **a cheap configuration is now the thing that has to be shown, +not the thing assumed.** FINDINGS 52. **Green-light check:** `./tools/bench/check.sh` (~3 min, needs the Blu-ray mounted) re-runs both display regression tests, the rate-control drift test, the @@ -33,6 +132,9 @@ display-path coherency counterexample and a 120-frame 68000 decode, then prints - **`docs/STATUS.md`** — current state, working setup, blockers, next steps. **Start here.** It also lists what has been explicitly abandoned, so old ideas do not get re-proposed. +- **`docs/ROADMAP.md`** — the remaining work to a completion target, and which + milestone that target is. Read it with STATUS, not instead of it: STATUS holds + the measurements, ROADMAP holds the shape and goes stale first. - **`docs/BENCHMARK.md`** — how to measure the storage subsystem, and why a bandwidth figure out of MAME would be meaningless. - **`docs/HARDWARE.md`** — X68000 GVRAM/CRTC reference. @@ -70,6 +172,17 @@ tools/analysis/ measurement scripts, numbered in the order they were written emitted too few spans to have tested anything. 17 prices the spans the encoder ACTUALLY emitted, with no selection model, which is what 12 and 14 could only simulate. + 18 measures what a 16-colour text-plane literal would cost in + dB, and closes that direction (FINDINGS 46.3). + 19 is the RING-BUFFER simulation, and it supersedes 09_buffer_ + sim.py's question rather than repeating it: it models the ring's + ADDRESSES, because src/player/ needs each record contiguous and + not merely resident. It reports the ZERO-PREFILL PIPE -- the + rate a medium must clear for a container to need no prefill -- + and warns explicitly when demand exceeds supply on the MEAN, + where a "required prefill" figure would flatter a sustained + overrun. Its wrap count and hole size match tools/bench/ + stream.lua's, measured on a real 68000, to the digit. buscost.py is the shared bus-cycle table both import; the per-BLOCK constants live in tools/encoder/vq_hybrid.py and are imported, never copied (session 12 corrected one of them). @@ -90,6 +203,12 @@ tools/bench/ MAME Lua injection harness + 68000 benchmark sources. and R20 — do not write CRTC values anywhere else. `prep_dlx.py`/`decode.lua`/`verify_decode.py` load, time and verify `src/player/decode.s`; the verify pass is in check.sh. + `prep_stream.py`/`stream.lua` do the same for `stream.s`, but + lay the container out as a DISK in a host file and feed it + through a bounded ring at a modelled pipe rate -- so the rig is + no longer bounded by the emulated machine's RAM, and a stock + 2 MB machine runs the whole window. `dlxload.py` holds the + codebook/palette load-time maths both preps share. tools/bench/c68k/ headless px68k C68K harness -- a SECOND emulator for every 68000 cycle figure (FINDINGS 37). Links only px68k's CPU core: no SDL, no ROMs, no emulated machine. `make PX68K=~/src/px68k` @@ -107,7 +226,13 @@ tools/encoder/ hybrid VQ encoder + DLX3 container writer (working). DLX2 4-byte-aligns every frame record: an odd `move.l` is an ADDRESS ERROR on a 68000, not a slow read (FINDINGS 28.3). dlx.py is the reference DECODER -- ground truth for the 68000. -src/player/ decode.s: the 68000 DLX3 decoder. Pixel-exact under MAME and +src/player/ decode.s: the 68000 DLX3 decoder, PRELOADED-stream front-end. + stream.s: the same decoder behind a bounded RING (FINDINGS 49). + Both include frame.i (the block loop and span chain) and geom.i + (the constants) so there is exactly ONE copy of the bytes every + cycle constant in FINDINGS 24/30/40/41 is fitted to. check.sh + asserts decode.s still assembles to the same 1,296 bytes. + decode.s: the 68000 DLX3 decoder. Pixel-exact under MAME and px68k's C68K core, blocks and v7 literal spans both. The span pass is blit.s v7 verbatim -- the same instruction sequence the 66.0/9.143/9.978 fit was measured on, so do not tidy it. @@ -140,6 +265,18 @@ The profile bitrate is a **ceiling**: lam is bisected per frame under a leaky bucket, so the profile's `lam` is a quality floor rather than a setting (`--fixed-lam` opts out). +**But at `--spans all` none of that binds.** A 32-frame bucket emits the same +container byte for byte as an 8-frame one, and `lam` never leaves its floor on +any frame of the reference window: the rate is set by the span pass and by `mu`, +not by `--kbps` or the bucket (FINDINGS 44.3). Two known unit inconsistencies on +that side are implemented and default OFF because they measure as a wash -- +`--joint-decide` (the per-block lagrangian prices a byte at `lam + mu*c` rather +than `lam`) and `--joint-bucket` (the bucket may not lend clocks it cannot +repay). FINDINGS 44. + +An encode is ~95% k-means; a 120-frame window is ~29 s, of which ~22 s is +training the two codebooks (FINDINGS 44.5). + There are **two** ceilings, on two different axes. The second is the 68000's decode budget: `mu` is bisected per frame against 833,333 cycles so the frame also *decodes* in time, which takes the worst sustained window from 37 frames diff --git a/docs/BENCHMARK.md b/docs/BENCHMARK.md index 3a5e1de..3d8dc3d 100644 --- a/docs/BENCHMARK.md +++ b/docs/BENCHMARK.md @@ -53,13 +53,31 @@ was used. **Do not record KB/s and treat it as a hardware figure.** Already partly in FINDINGS 5. Bounds worth tightening from datasheets: - 68000 bus cycle: 4 clocks @ 10MHz, 16-bit => **5 MB/s** absolute ceiling -- HD63450 single-address DMA, ~8 clocks/word => **~2.5 MB/s** practical ceiling +- HD63450 single-address DMA, **5 clocks/BYTE** => **2.0 MB/s** practical ceiling + (dual-address is 9 clocks/byte => 1.11 MB/s). CORRECTED session 14: this line + read "~8 clocks/word => ~2.5 MB/s", which charged a byte-wide SPC per word. + FINDINGS 43. - SCSI-1 asynchronous REQ/ACK handshake per byte, plus MB89352 FIFO depth => the real limiter, and the number we do not have from a primary source -The user's working figure is **4 Mbps = 488 KB/s**, which sits sensibly between -the derived DMA ceiling and observed period-drive rates. **Provenance not yet -recorded — worth pinning down, because every profile now hangs off it.** +**RETIRED, session 18 (USER DECISION).** This document used to name a working +figure of "4 Mbps" here and note that every profile hung off it. It was never a +bus measurement — user-supplied, no provenance, and 10% of SCSI-1's asynchronous +rating (FINDINGS 42.1). It has been removed as a default from every analysis +tool and from `tools/bench/stream.lua`; the tools now REQUIRE an explicit rate, +so nothing can be scored against a figure the scorer never restates. + +**There is no working delivery figure. That is the honest state, and it is the +point:** the rate is a property of the medium, the medium is a BlueSCSI, and it +has not been measured. `tools/analysis/19_ring_stream.py` reports the +**zero-prefill pipe** — the rate a medium must clear for a given container to +need no prefill at all — which is the threshold a measurement should be taken +against. For the session-14 candidate that is **513.2 KB/s** (FINDINGS 49.5). + +One place still carries the old number: `GATE_SPAN_KBPS` in +`tools/bench/check.sh`, because the gate container was *encoded* with it and +every per-block and span constant in FINDINGS 41/43/45/49 is fitted to that +container. It is a container recipe, not a claim about any medium. ### The coupling nobody had counted Cycle-stealing DMA is not free DMA. At ~8 clocks per 16-bit word: @@ -69,7 +87,7 @@ Cycle-stealing DMA is not free DMA. At ~8 clocks per 16-bit word: | 110 KB/s | 4.5% | 42.8% | | 250 KB/s | 10.2% | 48.5% | | 450 KB/s | 18.4% | 56.7% | -| 488 KB/s | 20.0% | 58.3% | +| ~490 KB/s | 20.0% | 58.3% | FINDINGS 5 concluded that because transfers are DMA, "streaming costs essentially no CPU". **That is wrong.** It costs up to a fifth of the machine at diff --git a/docs/FINDINGS.md b/docs/FINDINGS.md index 13b5178..851abb9 100644 --- a/docs/FINDINGS.md +++ b/docs/FINDINGS.md @@ -4,6 +4,28 @@ All numbers here are MEASURED unless marked ESTIMATE or FOLKLORE. --- + +> **THE DELIVERY RATE HAS NO WORKING FIGURE — retired session 18 (USER +> DECISION).** Sections below written before session 18 name a "4 Mbps" pipe +> constant and score tables against it. **Read every one of those as history.** +> It was never a bus measurement: user-supplied, no provenance, 10% of SCSI-1's +> asynchronous rating (FINDINGS 42.1), and FINDINGS 49.5 caught the shipping +> candidate exceeding it while nothing in the tree was comparing the two. +> +> It is now gone as a default from every analysis tool and from +> `tools/bench/stream.lua` — `--bus` / `--kbps` / `DLX_STREAM_KBPS` are +> **required arguments** with no fallback, so no table can be scored against a +> rate its own output does not state. The one survivor is `GATE_SPAN_KBPS` in +> `tools/bench/check.sh`, which is a **container recipe**, not a delivery claim: +> the gate container was encoded with it and every per-block and span constant +> in FINDINGS 41/43/45/49 is fitted to that container, so changing it is a +> re-encode plus a re-measurement, not an edit. +> +> **What to use instead:** `tools/analysis/19_ring_stream.py` reports the +> **zero-prefill pipe** — the rate a medium must clear for a container to need no +> prefill. That is a requirement to measure a BlueSCSI against, not a constant to +> design on. For the session-14 candidate it is **513.2 KB/s**. + ## 1. Source material `DRAGONS_LAIR.iso` — 16 GB, UDF 2.x, **decrypted** (no AACS dir). @@ -105,7 +127,11 @@ regression references; 03 and 04 are the correct ones. > **SUPERSEDED IN PART.** The claim below that DMA means streaming "costs > essentially no CPU" is **wrong** — see 19. The bandwidth figures here are -> folklore; the working figure is now **4 Mbps = 488 KB/s** (21). +> folklore; the working figure was **4 Mbps = 488 KB/s** (21), and **42.1 +> retires that too**: it was never a bus figure. The 50-pin SCSI-1 5 MB/s below +> is the correct bus rating. What binds is not the pipe but `W`, the clocks the +> DMA steals per word — and this section's 8 clk/word is a bracket midpoint, +> not a measurement (39.7, 42.6). [Yasuma, X68030 internal SCSI controller] @@ -387,9 +413,15 @@ transcription errors before any of it is committed to 68000 tables. > **PARTLY SUPERSEDED.** 17.1's diagnosis (the profiles were not derived from > hardware) and 17.2's CPU analysis both stand. But 17 reasoned against a -> misread bandwidth of 4 MB/s; the correct figure is **4 Mbps = 488 KB/s**, so -> the "ship pixel-exact if SCSI sustains >=800 KB/s" conclusion in 17.5 is -> **not available**. See 18 and 21. +> misread bandwidth of 4 MB/s; the correct figure was taken as **4 Mbps = +> 488 KB/s**, so the "ship pixel-exact if SCSI sustains >=800 KB/s" conclusion +> in 17.5 was withdrawn. See 18 and 21. +> +> **17.5 IS REINSTATED BY 42.3.** The 488 KB/s figure that withdrew it was +> itself unsourced, and the delivered stream is 837.4 KB/s at 0.29 dB off the +> palette ceiling — 17.5's threshold and 17.5's conclusion, arrived at from the +> other end five sessions later. Its reasoning was sound; only its bandwidth +> number was wrong, and it was wrong in the direction that made it look wrong. Prompted by the user asking why the SCSI profile was only 75 KB/s. It should not have been. Two separate errors, one of them serious. @@ -2535,6 +2567,12 @@ longer silently weaken the pixel-exactness gate. ## 41. v7 is in the player, and the model it is scored by was 18% wrong (session 12) +> **41.2's FRAMING IS SUPERSEDED BY 42.** This section ends by naming the rate +> point as a fork the user must choose. There is no fork: the span pass +> saturates at ~837 KB/s on its own, and the 488 KB/s ceiling it is scored +> against here was never a bus figure (42.1). The measurements below stand; the +> "two byte budgets" mechanism of 41.2 is what made 42 findable. + FINDINGS 40 measured v7 in `tools/bench/blit.s` and left it there. This builds it into `src/player/decode.s`, defines the container that carries it, and scores what the encoder actually delivers rather than what a selection model predicts. @@ -2695,3 +2733,1728 @@ measurement of where the controller aims, and any cost-model error is amplified into a large count change -- which is how 41.5's 18% error stayed invisible. **Report the cost distribution; quote the count only with its sensitivity.** Add this to the §4 measurement traps. + +## 42. The rate point dissolves: bytes are nearly free, and one constant decides everything (session 13) + +Session 12 ended by naming the rate point as "the first real fork since the +profile was set" and asking the user to choose one. **The question turned out to +be mis-posed, and the user is what posed it correctly**: challenged on where the +488 KB/s constant came from, the answer is that it never came from the SCSI bus +at all, and the bus has roughly eight times that in it. Once bytes are that +cheap the rate point is not a choice, it is a saturation point the encoder finds +on its own -- and what actually decides whether the game runs is **the number of +clocks the SCSI DMA steals per word**, which is still unmeasured and now worth +more frames than every optimisation since FINDINGS 24 combined. + +### 42.1 The 488 KB/s constant was never a bus figure +FINDINGS 18/21 recorded it as a user-supplied "4 Mbps" with no provenance, and +29.5 item 3 has carried "confirm the 4 Mbps figure" as an open item ever since. +Checked against the standard: **SCSI-1 (ANSI X3.131-1986) is an 8-bit bus at +~1.5 MB/s asynchronous and 5 MB/s synchronous** -- MB/s, not Mbps. The working +constant is 10% of the asynchronous rating and 3% of the deployment target's +(FINDINGS 21.2 committed to SD-backed SCSI in session 2). + +So the pipe was never the binding resource on the I/O side. What binds is that +**a delivered byte is charged to the 68000's frame budget**, because the DMAC +stalls the CPU rather than overlapping with it (38.3). The pipe's units are +KB/s; the real currency is clocks. + +### 42.2 The cheapest way to put a pixel on this screen is to not code it +Per pixel, in frame-budget clocks, with `W` the SCSI DMA's clocks per word: + +| | wire bytes | paint | disk debit | total | +|---|---:|---:|---:|---| +| v7 literal span | 2 | 9.143 | `W` | `9.143 + W` | +| RAW block | 1 | 25.03 | `W/2` | `25.03 + 0.5W` | +| V1 block | 0.25 | 18.74 | `W/8` | `18.74 + 0.125W` | + +A span pixel costs **two** wire bytes, not one: X68000 GVRAM in 256-colour mode +is one pixel per WORD with the high byte discarded (`spans.py:44`, and it is the +memory model both emulators are pixel-exact against), so the span path is a +straight `movem` copy that cannot pack. That 2x on bytes is exactly what buys +9.143 clocks/pixel. + +**A span beats a RAW block for any `W < 31.7`**, i.e. everywhere in the 5..12 +bracket of 39.7, and by a factor of nearly 2. The codec's expensive modes exist +to save bytes, and bytes have stopped being the scarce thing. + +The budget is 833,333 clocks for 49,152 pixels = **16.95 clocks/pixel**. A +full-frame literal therefore needs `9.143 + W` to fit under ~16.1 after span and +all-SKIP overhead: **lossless fits iff `W` is about 6.5 or less.** + +### 42.3 Measured: the span pass saturates at ~935 KB/s, 0.13 dB off the display +Encoded with `--kbps 280` held fixed and `--span-kbps` swept, so every column +below has the same quality target and differs only in what the span pass was +allowed to draw on. Scored with `17_span_delivered.py` over the emitted span +sections: + +| container | KB/s | span px | PSNR | over @5 | @8 | @12 | +|---|---:|---:|---:|---:|---:|---:| +| `s12_280_off` | 278.6 | 0% | 29.27 dB | 76/120 | 86/120 | 109/120 | +| `rc_fr_singe_scsi_span` (the gate) | 487.7 | 26.6% | 29.63 dB | 11/120 | 34/120 | 78/120 | +| `s13_280p700` | 693.8 | 48.3% | 30.28 dB | 16/120 | 28/120 | 63/120 | +| `s13_280p1000` | 815.4 | 60.2% | 30.93 dB | 7/120 | 20/120 | 64/120 | +| `s13_280p1500` | 837.4 | 62.3% | 31.04 dB | **0/120** | 21/120 | 63/120 | +| `s13_lossless` (`--kbps 2000`) | 934.6 | 70.4% | **31.19 dB** | **0/120** | 47/120 | -- | + +**`s13_280p1500` and `s13_280p2200` are byte-identical files.** The span pass +stops finding spans worth taking at ~837 KB/s; asking for more bytes returns the +same container. The rate point is not a choice between quality and deadline any +more, it is a saturation the encoder reaches by itself. + +Three results worth separating out: + +1. **0/120 is the first time anything in this project has fitted 12fps on every + frame.** It is also the best picture yet: 31.19 dB against a **31.33 dB + palette ceiling**, 0.13 dB off exact for this display. +2. **`mu` is never spent.** At span budgets >= 1000 KB/s, 0/120 frames need the + CPU-fit lagrangian at all. The 0.62 dB FINDINGS 31 paid to make frames decode + in time is refunded in full -- spans buy the deadline with bytes, and a span + is pixel-exact, so the trade is quality-positive in both directions. +3. **Quality and deadline stopped competing.** Every earlier section on this + project trades one against the other. Above ~800 KB/s more bytes improve + both at once, until the span pass runs out of runs worth spanning. + +### 42.4 So the whole result now hangs on `W`, and only on `W` +Read the columns of 42.3 sideways rather than down. At 934.6 KB/s: + +| `W`, clocks/word | median frame | worst | over budget | +|---:|---:|---:|---:| +| 5 | 83.0% | 91.0% | **0/120** | +| 6 | 87.6% | 97.0% | **0/120** | +| 6.5 | -- | -- | crossover | +| 7 | 92.2% | 103.0% | 21/120 | +| 8 | 96.8% | 109.0% | 47/120 | + +Moving `W` across its datasheet bracket costs more frames than moving the rate +from 280 to 935 KB/s wins. **`W` is now the most load-bearing unmeasured number +in the project**, displacing the 4 Mbps figure it just retired. + +### 42.5 MAME settles which end of the bracket applies, and cannot settle the number +STATUS has carried "benchmark `x68000 -exp1 cz6bs1`, never `x68ksupr`" since +session 10, and the note reads as a hardware claim. **It is not one, and the +user was right to challenge it.** Both machines use the same MB89352 SPC -- +`x68k.cpp:834` maps it at `$E96020`, `x68k_scsiext.cpp:83` at `$EA0000` -- and +neither needs a driver because SCSI IOCS is in ROM. The difference is an +emulation gap: `x68k.cpp:1176` is literally `// TODO: duplicate DMA glue from +CZ-6BS1`. + +The external board's glue is modelled, and reading it answers the mode question: + +- `x68k_scsiext.cpp:110-136`: a transfer is a DMA cycle when `m_slot->exown()` + -- the HD63450's **OWN** -- is asserted, so the DMAC holds the bus and the + 68000 is off it. It single-address reads the data register at `$EA0015` and + **negates `#DTACK` whenever `DRQ` is not ready**, stalling rather than + arbitrating away. +- `x68k.cpp:1114-1115, 1122-1123`: the slot's DTACK feeds `hd63450::dtack_w` + and the DMAC's `own()` feeds back to the slot. + +So the modelled CZ-6BS1 path is **cycle-steal with the bus held** -- the 5 +clocks/word end of 39.7's bracket, not the ~12 arbitrated end. That is the end +where everything above fits. + +**But MAME cannot give the constant, and asking it for one would be reading back +a hand-set table.** `x68k.cpp:1047-1048` configures the DMAC with +`set_clocks(attotime::from_usec(2), from_nsec(450), from_usec(4), ...)` and +`set_burst_clocks(...)` -- wall-clock attotimes, not the MC68450's per-operand +cycle counts. Same lesson as FINDINGS 39: the datasheet supplies constants, the +emulator supplies structure. Use MAME to confirm which handshake the player's +code actually provokes; use Fig 4-25 for what it costs. + +### 42.6 Where the wait actually comes from, and why flash is the right premise +With DTACK gating the cost decomposes: + + W = 5 clocks (MC68450 single-address write, Fig 4-25 sheet 3) + + however long the drive makes the DMAC wait for DRQ + +The 5 is silicon and fixed. **The rest is the device**, and it is where 8 and 12 +came from -- 8 was taken as a bracket midpoint in FINDINGS 5, not measured. A +period spinning SCSI-1 drive supplies a real wait; SD-backed SCSI with a modern +controller collapses it toward zero and leaves the floor. The deployment target +has been SD since session 2 (21.2), so **the favourable end of the bracket is +the one the actual hardware is on** -- which is what makes 42.3's 0/120 worth +taking seriously rather than filing as a best case. + +This is a better-founded version of the user's argument than the bus rating was. +The bus rating is true and irrelevant; the wait term is what their premise +actually buys. + +### 42.7 What is NOT established +Stated plainly, because 42.3 is the most favourable table this project has ever +produced and that is exactly when it should be distrusted: + +1. **No 68000 has decoded a 70%-span container.** Every figure in 42.3 is the + additive cost model. That model is validated to -0.06% mean / 0.09% worst on + spanned containers (41.5), but the heaviest one ever *run* is the gate's + 26.6%. The rig loads the stream into a 2 MB machine, so a 934.6 KB/s stream + is about **5 frames** -- getting coverage back needs a different rig + (stream-in-chunks, or a larger machine as a deliberately-labelled non-target), + not a longer pass. This is FINDINGS 36's lesson pointing at a new wall. +2. **`W` is unmeasured**, and 42.4 is the whole result's sensitivity to it. +3. **The ring buffer has never been simulated near this rate.** 98 KB/frame of + span payload against 2 MB with no double buffer; `09_buffer_sim.py` last ran + at 110 and 280 KB/s (29.5/30.7). +4. **Capacity is fine but should be stated:** 934.6 KB/s x 1366.6 s = **1.22 GB** + for the whole game. Irrelevant on SD, fatal on anything period. This closes + CD-ROM permanently rather than parking it (32.3). +5. **Span selection is still greedy after `lam`** (STATUS item 2, 39.3). At + saturation that probably leaves something on the table rather than costing + anything, but it has not been checked at this rate. +6. **The `--kbps 2000` container is not a shippable configuration**, it is a + probe: it lets `lam` fall to its floor and asks what the span pass does with + an unbounded budget. `s13_280p1500` at 837.4 KB/s is the honest candidate. + +### 42.8 FINDINGS 17.5 was right, and was withdrawn for the wrong reason +Session 4 concluded: **"ship pixel-exact if SCSI sustains >=800 KB/s."** Session +5 withdrew it, because 17 had reasoned against a misread 4 MB/s and the +correction to 4 Mbps = 488 KB/s put 800 out of reach. + +The correction was to the wrong direction of the error. 4 MB/s was indeed a +misread of the bus, but 488 KB/s was not a measurement either -- and the bus +really does have ~1.5-5 MB/s in it (42.1). **The delivered stream is 837.4 KB/s +at 0.29 dB off the palette ceiling**: 17.5's threshold, and 17.5's conclusion, +reached from the opposite end five sessions later by a route that never cited +it. + +Worth recording as a methodology result rather than a curiosity. The project's +habit of appending corrections rather than editing history is what made this +recoverable at all -- but 17.5 spent five sessions marked "not available" +because a number nobody had sourced was allowed to retire a conclusion that had +been reasoned properly. **A correction is only as good as the constant behind +it**, and this is the second time that constant is the one at fault (compare +39: bus arithmetic retired the DMAC's first costing, and the datasheet retired +the retirement). + +## 43. The disk debit was denominated per word, and the SPC is a byte-wide port (session 14) + +Session 13 made `W` -- the clocks the SCSI DMA steals -- the one binding unknown +in the project, and 42.4 showed the whole result swinging on where in a 5..12 +bracket it landed. **The bracket was in the wrong unit.** `W` was charged per +WORD of delivered stream, and the MB89352 is an 8-bit port: the DMAC pays per +BYTE. The debit is 2x what every table since FINDINGS 5 has charged, the +favourable end of the bracket was never physically reachable, and 42.3's 0/120 +does not survive. + +The result is not lost, but it is re-anchored 41% lower in rate and 1.85 dB +lower in picture, and it had to be re-encoded rather than re-scored -- because +the encoder was making its decisions in the wrong units too (43.6). + +### 43.1 The floor argument needs no datasheet +A 68000 bus cycle is four clocks minimum. The SPC delivers one byte per bus +cycle. Therefore **no DMA of this device can cost less than 4 clocks/byte = 8 +clocks/word**, before a single clock of DMAC overhead, the write cycle that +puts the byte in RAM, or any wait the drive imposes. + +The retired bracket's own midpoint, 8 clocks/word, is that floor exactly; its +favourable end, 5 clocks/word, is **2.5 clocks/byte -- 62% of a single bus +cycle**. It implied a 4 MB/s DMA on a 10 MHz bus, on a SCSI-1 link 42.1 had +just established runs at 1.5 MB/s asynchronous. The number that retired the +4 Mbps figure should have retired this one in the same paragraph. + +Independent cross-check, of the kind this project files as folklore rather than +measurement: BlueSCSI-class throughput on an X68000 is discussed at 0.7-1.7 +MB/s. 5 clocks/byte is a 2 MB/s ceiling and 9 is 1.11 MB/s; 2.5 clocks/byte is +4 MB/s, which no one has ever reported on this machine. + +### 43.2 The datasheet, per byte, for the transfer this actually is +Device-to-memory, 8-bit device, MC68450 Fig 4-25, all while the DMAC owns the +bus (note 2: reads 4 clocks, writes 5): + +| how the DMAC is programmed | clocks per BYTE | source | +|---|---:|---| +| single address, D->M | **5.0** | sheet 2 | +| dual address, byte, no packing | **9.0** | sheet 4, the note on the 9-clock word case | +| dual address, byte packed to word writes | **16.5** | sheet 3: 14 clocks then 19, per two bytes | + +Byte packing is *dearer*, not cheaper: the FIFO path carries 10 clocks of +inter-cycle overhead per operand where the unpacked path carries none. + +To this, per period of bus ownership, add front-end overhead of **5 clocks best +case (Fig 4-23) or 8 worst (Fig 4-24)** and back-end of **2 (4.5.2.2)**. +Amortised over a sector that is noise; taken per operand it would add 7-10 +clocks to every byte. **That is what actually rules out arbitrating per byte** +-- not the OWN pin (43.3). + +**Which row applies is a wiring question, and it is worth 4 clocks on every +byte of the game.** Single-address needs the SPC's DACK driven from the bus's +`#EXACK` (pin B37, and the bus really does have it, `x68kexp.h`). MAME models +the dual-address row: an `exown`-gated byte read of `$EA0015` written to memory +with `space.write_byte` (`x68k_scsiext.cpp:110-121`, `hd63450.cpp:383-385`), and +it has no DACK path at all, so **MAME cannot settle this one either way.** + +### 43.3 42.5's reading of the OWN pin was over-read +42.5 concluded from `x68k_scsiext.cpp`'s `m_slot->exown()` gate that the modelled +path is "cycle steal with the bus held", i.e. the 5-clock end. It does not say +that. `hd63450.cpp:366,447` asserts `m_own(0)` before **every** `single_transfer` +and negates it after, in every request-generation mode, and `x68kexp.h:131` +inverts it. So `exown()` distinguishes **a DMAC access from a CPU access** -- +DMA versus PIO -- and says nothing about hold versus arbitrate. The check was +answering a different question than the one asked of it. + +Two things about the board do survive, and they are the useful half: + +1. **The glue's flow control is a stalled bus cycle, not a released bus.** When + `DRQ` is low the board negates `#DTACK` and the DMAC waits mid-cycle + (`hd63450.cpp:449`, `if (!m_dtack) return;` -- the operand does not advance). + There is no path by which it hands the bus back inside a word. So whatever + the drive makes it wait, it waits **holding the bus**, which is why the wait + term of 42.6 is charged to the frame budget in full. +2. **MAME has no DRQ line from the expansion slot to the DMAC.** `x68kexp.cpp` + has no such callback; DMAC channel 0 is wired to the FDC and channel 3 to the + ADPCM (`x68k.cpp:1052-1053,159`). The real bus has `#EXREQ`/`#EXACK` + (B36/B37), so external-request modes exist on hardware and are simply absent + from the model. **Any request-generation experiment run in MAME would be + measuring the gap, not the board.** + +### 43.4 What it costs, on the containers session 13 already had +`17_span_delivered.py` and `14_dmac_chain.py` now take `--disk-clk-byte`; +`--disk-clk-word` is kept and halves it, so session 13's tables reproduce +exactly. Frames over the 12fps budget, 120-frame `singe` window: + +| container | KB/s | 2.5 c/B (s13) | 5 c/B | 9 c/B | 16.5 c/B | +|---|---:|---:|---:|---:|---:| +| `s12_280_off` | 278.6 | 76/120 | 100/120 | 118/120 | 120/120 | +| `rc_fr_singe_scsi_span` (the gate) | 487.7 | 11/120 | 71/120 | 119/120 | 120/120 | +| `s13_280p1000` | 815.4 | 7/120 | 45/120 | 120/120 | 120/120 | +| **`s13_280p1500`** (s13's candidate) | 837.4 | **0/120** | 44/120 | 120/120 | 120/120 | +| `s13_lossless` | 934.6 | **0/120** | 87/120 | 120/120 | 120/120 | + +**Nothing session 13 emitted fits at any point in the real bracket.** The +mechanism is 42.2's own table read at the right price. Per pixel, with `c` the +clocks per delivered byte: + + v7 literal span 2 bytes 9.143 + 2c + RAW block 1 byte 25.03 + c + V1 block 0.25 B 18.74 + 0.25c + +A span beats RAW for `c < 15.9` -- still true everywhere real. But **a span +beats a V1 block only for `c < 5.48`**, and that crossover sits between the +single-address row and the dual-address row of 43.2. The span pass is not +robust to the wiring question; it is decided by it. The encodes in 43.6 +confirm the prediction: spans paint 30.7% of the picture at `c=5` and 3.9% at +`c=9`. + +### 43.5 A frame's cost is now, to three figures, its byte count +On `s13_280p1500` at 5 clocks/byte, `corr(bytes, total frame clocks) = 0.989`, +and the disk term's p10..p90 spread (281k..464k clocks) is most of the frame +total's (713k..966k). Frames that miss carry 87,203 bytes on average against +61,295 for frames that fit. + +**Which breaks the rate controller's bucket.** It banks bytes across 8 frames +because the player's ring buffer can hold them -- true, and irrelevant now: +those bytes are also clocks, and FINDINGS 28 established there is no double +buffer to decode ahead into, so **a frame that borrows bytes from the bucket +borrows clocks it cannot bank.** Byte smoothing was free when bytes were free. +It is now a direct source of deadline misses. + +### 43.6 The encoder was never told a byte costs anything either +This is why the session could not just rescore. `ratectl.py` bisected `mu` +against 833,333 cycles of DECODE, with no disk term, and `spans.select()` +admitted a run "only if the span beats the blocks it replaces on cycles ALONE" +-- explicitly ignoring the bytes it adds. Both now work in one currency: + +- `ratectl.DISK_CLK_BYTE` (default **5.0**, `encode.py --disk-clk-byte`) is + charged inside the CPU ceiling, so every fit test is `decode + c*bytes`. +- `spans.select()` admits and ranks on **net** clocks, `(clocks won) - c*(bytes + added)`. At `c=0` both reduce exactly to the old rules, and + `--disk-clk-byte 0` re-emits `s13_280p1500` byte for byte (8,501,948 B, + 31.04 dB) -- so the change is the price, not the codec. + +Re-encoded honestly, 120-frame `singe` window, 12 fps: + +| | KB/s | PSNR | loss vs ceiling | over budget | span px | +|---|---:|---:|---:|---:|---:| +| s13's claim (2.5 c/B) | 837.4 | 31.04 dB | 0.29 dB | 0/120 | 62.3% | +| **`s14_d5_all1500`, single address** | **496.7** | **29.19 dB** | 2.14 dB | **1/120** | 30.7% | +| `s14_d9_all1500`, dual address | 255.0 | 28.50 dB | 2.83 dB | **1/120** | 3.9% | + +**The one frame over is frame 0** in both -- the intra frame, which FINDINGS +28.5/31 already established is emitted late on purpose because there is nothing +on screen to hold. **Every other frame lands at or under 100.0%**: the +controller now binds exactly on the joint budget. `17_span_delivered.py`, which +shares no code with the encoder's own accounting, reproduces both rows to the +digit. + +Two smaller results fall out: + +1. **`--spans all` is now the better rule, and `--spans need` the worse one** + (275.8 KB/s, 28.92 dB, 2/120). `need` stops as soon as the frame fits and + leaves profitable clock savings unbought; once profitability is measured in + one currency, spending every profitable byte *is* the optimum. My + recommendation is to make `all` the default; that is my inference, not a + measurement. +2. **`--span-kbps` has stopped binding.** 700 and 1500 produce identical files. + Session 13's saturation was the byte ceiling running out; this one is the + span pass running out of runs that pay, which is a property of the codec + rather than of a chosen number. + +### 43.7 What this withdraws +- **42.3's 0/120 and "quality and deadline stopped competing".** They compete + again, and harder than before: a byte now buys picture and spends deadline. +- **42.3's "`mu` is never spent"** and the refund of FINDINGS 31's 0.62 dB. + `mu` is spent on 103 of 120 frames at `c=5` and 118 at `c=9`. +- **42.8's revival of 17.5** ("ship pixel-exact if SCSI sustains >=800 KB/s"). + The stream that fits is 496.7 KB/s and 2.14 dB off the palette ceiling, so + 17.5's threshold is not met and its conclusion does not return. The + methodology point in 42.8 stands and now applies to itself. +- **`docs/BENCHMARK.md`'s "~8 clocks/word => ~2.5 MB/s practical ceiling".** + The ceiling is 2 MB/s at best (5 clocks/byte) and 1.11 MB/s dual-address. +- **FINDINGS 5's 8 clocks/word**, retroactively, wherever it was used: every + I/O debit in the project before this section was charged at half rate. + +### 43.8 What is NOT established +1. **Single-address versus dual-address is a hardware fact this tree cannot + check.** It is worth 242 KB/s and 0.69 dB, and MAME models only the dual + row (43.2). It needs the CZ-6BS1 schematic or a real board -- and note it is + the *board's* wiring, not our code, so unlike every previous item on this + list it is not a design decision we get to make. +2. **The 5 and the 9 are datasheet floors with the drive wait set to zero.** + 42.6's argument that SD-backed SCSI collapses that term is unchanged and + still unmeasured. +3. **Front-end and back-end overhead are excluded**, which assumes at least a + sector per period of bus ownership. If the player ends up taking the bus per + operand, add 7-10 clocks to every byte and nothing fits at all. +4. **Still no 68000 has decoded any of these containers** (42.7 item 1). The + new ones are lighter in spans than the gate, so that gap is narrower than it + was, but it is the same gap. +5. **The bucket has not been fixed**, only diagnosed (43.5). + +### 43.9 The unit was never written down +The error is one substitution -- bytes/2 for bytes -- and it survived from +FINDINGS 5 through nine sessions, two cost-model rewrites, a datasheet reading +that corrected the *value* of the same constant (39.7), and a section devoted +to distrusting the table it produced (42.7, which lists six things 42.3 did not +establish and does not list its own denominator). + +What let it hide: `W` was carried as "clocks per word" in three tools and a +half-dozen tables, and **the device's port width was never in the same sentence +as it.** The 8-bit-ness of the MB89352 was known -- it is visible in +`install_device(..., 0x00ff00ff)` and in every register map in the project -- +but it lived in the I/O notes while `W` lived in the budget arithmetic. + +The rule this project already had (FINDINGS 33: a design that counts only CPU +shows a win the I/O it created takes away) needed one more clause: **check what +the unit is denominated in, on the device that supplies it.** A per-word debit +for a byte-wide port is a factor of two, and a factor of two is the difference +between this game running and not. + +--- + +## 44. The byte-side rate controller is inert, and both inconsistencies in it are worth under 2% (session 15) + +STATUS's item 2 asked for the leaky bucket to be fixed: 43.5 diagnosed it as +banking bytes that are now clocks, across a player with no double buffer to +bank clocks in, and called it "a direct source of deadline misses". The +diagnosis is correct as a mechanism. **It is not a source of anything at the +operating point this project actually recommends**, and neither is the second, +larger-looking inconsistency found next to it. Both are now implemented, both +measure as a wash or a regression, and **both ship off by default** -- +`--joint-decide` and `--joint-bucket` turn them on. + +Everything below is the 120-frame `singe` window, `--profile scsi --kbps 280 +--span-kbps 1500`, `c = 5` clocks/byte, 12 fps, scored both by the encoder and +by `17_span_delivered.py`, which shares no code with it. + +### 44.1 The mode decision was the last place a byte was free +43.6 charged the disk inside the rate controller's fit test and inside +`spans.select()`, but `vq_hybrid.decide()` still minimised `D + lam*bytes + +mu*cycles` with `cycles` meaning DECODE cycles only. So while `mu` was +enforcing a joint budget from above, the per-block lagrangian underneath it +still believed delivery was free. + +**That inverts FINDINGS 28.8.** RAW is 400.4 cycles against V4's 448.2, so with +a free byte, raising `mu` buys cycles by moving V4 -> RAW -- which is exactly +what 28.8 observed and what session 8's `0c` recorded as V4 collapsing. Priced +per delivered byte, a RAW block costs `400.4 + 16c` and a V4 block `448.2 + 4c`: + +| c, clocks/byte | V1 | V4 | RAW | +|---:|---:|---:|---:| +| 0 | 299.9 | 448.2 | **400.4** | +| 2.5 (the retired 5 clk/word) | 302.4 | 458.2 | **440.4** | +| **3.98** | 303.9 | **464.3** | **464.3** | +| 5 (single address) | 304.9 | **468.2** | 480.4 | +| 9 (dual address) | 308.9 | **484.2** | 544.4 | + +**The crossover is c = 3.98 and 43.1's floor argument is c >= 4.** A 68000 bus +cycle is four clocks and the SPC hands over one byte per cycle, so RAW's cycle +advantage does not exist on any real machine: it was spending 12 clocks of bus +to save 47.8 of CPU. The escape hatch was an artefact of the same free byte +that FINDINGS 43 found everywhere else. + +### 44.2 And correcting it changes almost nothing +`decide()` now takes `byte_clk` and prices a payload byte at `lam + mu*byte_clk`. +At `byte_clk = 0` it is the old decision exactly, and the containers of session +14 re-encode to the same MD5. + +| `--spans all`, c=5 | KB/s | PSNR | mean frame clocks | over budget | worst | +|---|---:|---:|---:|---:|---:| +| the shipped decision | 496.7 | **29.19** | **740,049** | 1/120 | 112.9% | +| `--joint-decide` | 482.5 | 29.17 | 745,438 | 1/120 | 112.9% | + +It **buys 6,058 clocks of disk with 17,207 clocks of block decode** -- a net +5,389 clocks a frame in the wrong direction. Scored at c=4 and c=9 it is the +marginally worse container at every price, so this is not a bet on which row of +43.2 wins. The reason is scale, not sign: at the `mu` the controller actually +settles on (median 0.74-0.91) the added byte price is `mu*c` ~ 4 against a +`lam` floor of 10, and the RAW/V4 decision is dominated by V4's distortion term +rather than by either. RAW moves 16.0% -> 15.7% of blocks. + +**The inconsistency was real, the correction is right, and the effect is 0.02 dB.** + +### 44.3 The bucket does not bind, and at `--spans all` nothing on the byte side does +Before fixing the bucket, measure whether it is loaded. It is not: + +| `--spans all`, c=5 | KB/s | PSNR | over budget | container | +|---|---:|---:|---:|---| +| `--bucket-frames 8` (shipped) | 496.7 | 29.19 | 1/120 | baseline | +| `--bucket-frames 32` | 496.7 | 29.19 | 1/120 | **byte-identical** | +| `--bucket-frames 1` (no banking at all) | 498.0 | 29.19 | 1/120 | +1.3 KB/s | +| `--rc-floor open` (lam floor 1.0) | 503.7 | 29.21 | 1/120 | +0.02 dB | + +**`lam` never leaves its floor of 10.0 on any of 120 frames**, in any of these, +and quadrupling the bucket emits the same bytes. With `--spans off` at 259.3 +KB/s under a 280 KB/s target, a 1-frame bucket and an 8-frame bucket are again +byte-identical. The block coder at the profile floor simply lands under the +per-frame byte budget, so there is nothing for the bucket to lend and nothing +for the lam bisection to do. **The rate this project reports is set by the span +pass and by `mu`; `--kbps` and the bucket are not the levers.** + +The one place the bucket does cost something is the mode 43.6.1 recommends +abandoning: at `--spans need` it is worth exactly one frame of 120. + +### 44.4 The cap, and why capping only half a frame is worse than not capping +`--joint-bucket` caps what the bucket may lend at what the frame's clock budget +can still absorb after its own block decode -- `(cycle_budget - cycles(mode0)) +/ c`, priced at the mode map the un-banked budget buys, and never below that +budget. The bucket then smooths only what is left after the disk is paid, which +is what STATUS item 2 asked for. + +| c=5 | KB/s | PSNR | median frame | over budget | +|---|---:|---:|---:|---:| +| `need`, shipped | 275.8 | 28.92 | 99.8% | 2/120 | +| **`need --joint-bucket`** | 302.1 | 28.91 | 99.9% | **1/120** | +| `all`, shipped | **496.7** | **29.19** | **87.9%** | 1/120 | +| `all --joint-bucket` | 506.4 | 29.18 | 89.6% | 1/120 | +| `all --joint-bucket`, cap extended to the span section | 273.7 | 28.88 | 99.4% | 1/120 | + +**It RAISES the bitrate at `--spans all`**, which is the tell: capping the block +payload does not remove those bytes, it moves them into the span section, which +draws on its own flat pipe (41.2) and is not under the cap at all. So the third +row is the honest reading of "the per-frame ceiling should be joint and hard" -- +and it is the worst container here, losing 0.30 dB and 233 KB/s for no change in +the frames-over count, because the cap starves the pass that was buying the +deadline in the first place. + +There is no per-frame ceiling on the whole frame record anywhere in this +encoder, and **44.3 is why adding one has not been urgent**: the joint budget is +enforced after the span pass by `mu`, which is a controller that pays in +picture, and the byte-side ceiling it would replace is not binding. + +### 44.5 An encode is 95% k-means, and that is now 2.7x faster, exactly +Prompted by the user observing that this should not take a minute a scene. +Profiled, a 120-frame encode was 60.6 s of which 60.8 s was two k-means runs +(the rest of the encoder, rate control and the span pass included, is about a +second). `VQ.assign` was all of it, and three things were on the floor, none of +which changes a label: + +| | 1,474,560 2x2 blocks, k=256 | +|---|---:| +| as written | 1.83 s | +| `C.T` materialised once (a view makes BLAS re-copy it per chunk) | 1.02 s | +| chunk 8192 -> 2048 (the `(chunk,k)` temporary, cache not memory; 32768 is 2.80 s) | 0.81 s | +| 8 threads over the chunk loop (numpy releases the GIL in both matmul and argmin) | **0.31 s** | + +Partitioning rows cannot change an `argmin`, so the labels are bit-identical to +the serial ones and every container this encoder emits still hashes the same -- +which is the assertion, not the hope: `s14_d5_all1500` re-encodes to +`d13d142b...` on both sides of the change. Whole encode 60.6 s -> 29.4 s, +`H.build` 60.9 -> 22.2 s. What is left is `np.add.at` in the centroid update +(~5 s of 22), and taking it costs the bit-exactness, so it stays. + +At 0.18 s/frame the 22.8 minutes of unique scene footage is ~50 minutes of +codebook training, single machine, single pass. + +### 44.6 What this does NOT establish +1. **It does not re-open the c=5 vs c=9 question** (43.8.1), which is still the + largest open number and still a hardware fact this tree cannot check. +2. **The 1/120 is the same 1/120** -- frame 0, the intra frame, emitted late on + purpose. Nothing here moved it, and 43.8's five caveats all stand. +3. **`--spans all` remains a recommendation, not a measurement** (43.6.1), and + 44.3 sharpens why it matters: it is the only lever on this side of the + encoder that does anything. +4. **The pixel-exact gate now covers 37 of 120 frames**, down from 80 in session + 10 (FINDINGS 36) -- the span-heavy container outgrew the 2 MB machine. The + strongest test in the tree audits under a third of the window it names. That + makes STATUS item 3's chunk-streaming rig load-bearing rather than optional. +5. **Two fixes, both correct, both defaulted off.** The pattern worth keeping is + that the second one was found by asking whether the lever was loaded before + pulling it, and 44.3 took four encodes to establish -- against a session that + could have been spent making a bucket cap work. + +### 44.7 No decoder at all: what a literal frame costs, and where it dies +Asked by the user: does streaming raw preprocessed frames straight into video +memory save the CPU? On clocks, yes -- completely. It dies on the medium, and +the arithmetic is worth writing down because it also retires the last line of +42.2 still standing in the wrong units. + +**Two versions, and the split is the answer.** A frame is 256x192 = 49,152 +pixels; 256-colour GVRAM is one pixel per WORD with the high byte discarded +(42.2, `spans.py:44`, and the memory model both emulators are pixel-exact +against), so a literal frame is **98,304 bytes of GVRAM writes** against a +budget of 833,333 clocks = **16.95 clocks/pixel**. + +*1. The CPU paints it.* This is what a v7 span already is -- literal GRB555 +words, preprocessed offline, copied by `movem` -- at `9.143 + 2c` clocks/pixel: + +| | clocks/pixel | vs 16.95 | +|---|---:|---| +| c = 3.906 | 16.95 | breakeven | +| **c = 4.0**, the 43.1 floor | **17.14** | **misses by 1.1%** | +| c = 5.0, single address | 19.14 | misses by 13% | +| c = 9.0, dual address | 27.14 | misses by 60% | + +**A CPU-painted full-frame literal does not fit at any physically reachable +price**, and it misses at the floor itself. This supersedes 42.2's "lossless +fits iff `W` is about 6.5 or less", which was denominated per WORD: 6.5 +clocks/word is 3.25 clocks/byte, below the 4-clock floor, so that threshold was +never reachable either. FINDINGS 43 withdrew the tables downstream of the unit +error but not this line; it is withdrawn here. + +*2. The DMAC writes device -> GVRAM and the CPU is not in the loop.* The 9.143 +disappears and a pixel costs only its two wire bytes: + +| c | clocks/frame | % of budget | wire | 22.8 min of game | +|---:|---:|---:|---:|---:| +| 4.0 | 393,216 | 47.2% | 1,152 KB/s | 1.61 GB | +| **5.0** single address | **491,520** | **59.0%** | 1,152 KB/s | 1.61 GB | +| 9.0 dual address (the row MAME models) | 884,736 | **106.2%** | 1,152 KB/s | 1.61 GB | +| 16.5 packed | 1,622,016 | 194.6% | 1,152 KB/s | 1.61 GB | + +**At c=5 it fits, with 41% of the frame budget left and no decoder at all.** It +is defeated by delivery, not by the 68000: 1,152 KB/s is ~79% of SCSI-1 +asynchronous (42.1) with nothing left for audio or seeks, and 1.61 GB is 2.3x +the 0.70 GB the shipping 496.7 KB/s container already needs -- which is itself +already past a CD-ROM. At c=9 it does not fit the clocks either. + +**The waste is specific and it is the same one the span path pays.** Because the +high byte is discarded, half of every byte pulled off the disk is thrown away by +the hardware on arrival: two bytes of disk per byte of picture. That 2:1 is why +a span pixel costs `2c` and why 43.4's span-vs-V1 crossover sits at c=5.48. The +codec is not there to save CPU -- 44.7 shows the CPU can be removed outright -- +it is there to save the wire. + +**The one thing that would change this answer** is a packed write path into +256-colour GVRAM, two pixels per word. That halves the wire to 576 KB/s and +806 MB and puts a decoder-free player back in play. **It is NOT established +either way.** The tree's basis for one-pixel-per-word is that MAME and px68k are +both pixel-exact against that model, which is evidence about two emulators, not +about the CRTC and the palette hardware. It is the same class of question as +43.8.1 -- a service manual or a real board settles it, an emulator cannot, and +both emulators here would model a packed path identically wrong if it exists. + +Note this is a different transfer from the one FINDINGS 39 costed and rejected. +39 priced **memory -> GVRAM** (the DMAC replacing v6's `movem` chain out of a +RAM stream buffer) and it lost by a clock. This is **device -> GVRAM**, with no +RAM staging and no CPU, and it loses to the disk instead. + +## 45. The strongest test in the tree was short by 83 frames, and the fix was the rig's memory (session 16) + +STATUS item 1 said the pixel-exact gate "needs the chunk-streaming rig, not a +longer pass." It needed neither. It needed a bigger emulated machine, and the +reason that was not obvious is worth more than the fix. + +### 45.1 The constraint was the rig's, and the gate does not measure timing +`prep_dlx.py` preloads the whole container into emulated RAM at `STREAM=0x30000` +and `check.sh` ran the machine at `-ramsize 2M`, so the span-heavy gate container +-- 5,261,814 B of stream, ending at 0x534BF6 -- was truncated to the prefix that +fit. That was **37 of 120 frames**, down from 80 in session 10 (FINDINGS 36) as +the container grew. + +The 2 MB was carried over from the timing rig, where it is correct: the locked +target is a stock 2 MB machine. But `check.sh` runs this gate under +`DLX_VERIFY_ONLY=1`, which **drops the cost anchors entirely** and asserts only +pixel-exactness. A verify-only pass makes no claim about the target's memory, so +it was never the 2 MB that was load-bearing here -- and preloading a whole +container is unlike the shipping player at *any* size, because the player streams +from disk into a ring buffer and never holds a window at once. + +`RIG_RAM=6` in `check.sh` (MAME's x68000 accepts 1M-12M) covers all 120 frames. + +### 45.2 The raise is licensed by measurement, not by convenience +Raising the emulated RAM to make a test pass is exactly the move that should be +distrusted, so it was checked rather than asserted. The full timing pass -- not +verify-only -- was run at `-ramsize 2M` and `-ramsize 6M`, and the **five +synthetic anchors come out bit-identical**: + +| synthetic frame | 2M | 6M | +|---|---:|---:| +| all-SKIP | 40,729 | 40,729 | +| all-V1 | 921,187 | 921,187 | +| all-V4 | 1,376,881 | 1,376,881 | +| all-RAW | 1,229,883 | 1,229,883 | +| all-SPAN-64 | 506,533 | 506,533 | + +They sit at **different addresses** in the two layouts -- the synthetic block is +placed after a 37-frame stream in one and a 120-frame stream in the other -- and +still cost the same, so MAME's cycle model does not depend on `-ramsize` over +this range. Every per-block constant in FINDINGS 24/30/41 is measured from these +frames and is therefore unmoved by the change. + +### 45.3 The 37-frame prefix was a biased sample, and the quiet end was missing +The gate is now 120/120 pixel-exact on **both** cores -- MAME's 68000 and px68k's +C68K. (The C68K harness never had a RAM ceiling at all: its arena is 16 MB and +`RAMTOP` is defined but unused, so it was short only because it reads the blob +`prep_dlx.py` truncated.) What the extra 83 frames show is that the prefix was +not representative: + +| anchor | 37-frame prefix | full 120 | +|---|---:|---:| +| min non-SKIP | 25.1% of blocks, 61.9% of budget | **15.2%, 53.6%** | +| median | 45.4%, 79.7% | **41.1%, 81.1%** | +| p90 | 52.3%, 93.1% | **48.5%, 91.1%** | +| max non-SKIP | 62.5%, 91.8% | 62.5%, 91.8% (same frame) | +| C68K sequential-pass mean | 693,886 cyc, **83.3%** | **641,444 cyc, 77.0%** | + +The prefix **overstated the mean cost of the window by 8.2%**. It caught the +worst frame -- the max is the same frame in both -- but it never saw the quiet +end: the true minimum is 15.2% non-SKIP against the prefix's 25.1%, and 53.6% of +budget against 61.9%. The distribution the anchors exist to sample was cut off at +one end, which is the failure mode FINDINGS 25.6 warned about in a different +guise: a prefix is not a sample. + +Note the direction. The prefix was **pessimistic**, so nothing downstream of it +was flattered, and no headroom claim in this tree was resting on the missing +frames. That is luck, not design. + +### 45.4 What this does NOT establish +1. **The streaming path is still untested, at either RAM size.** The rig + preloads; the player streams into a ring buffer. This gate proves the decoder + is pixel-exact over a whole window, and says nothing about the ring buffer, + the chunk boundaries, or the disk. The chunk-streaming rig STATUS item 1 + called for is still unbuilt -- it was just never what the 37/120 needed. +2. **The 2 MB target is unchanged.** `RIG_RAM` is the emulated machine's memory + for a verify-only pass. `span.sh` and the session-7 timing reproduction still + run at 2M, and the shipping player's memory budget is untouched. +3. **The truncation guard stays.** `prep_dlx.py` still truncates and announces + it, and `check.sh` still reads the count back and greps for `TRUNCATED`. A + heavier container, or a lowered `RIG_RAM`, brings it straight back. +4. **The moved anchors are a re-measurement, not a regression.** No constant + changed; the frames the anchors point at did. + +## 46. The 256-colour mask is defeatable, and a packed path is back in play (session 16) + +STATUS item 2 asked whether 256-colour GVRAM has a PACKED write path -- two +pixels per word rather than one pixel per word with the high byte discarded. It +**The first answer was no and it was wrong** -- or rather, it was right about the +default write path and missed the register that turns the masking off. 46.1-46.3 +record what was established and what was measured, and are kept as written. +**46.5 is the correction and it is the important part of this section.** + +### 46.1 The answer is no, and the sub-word fields are PAGES, not pixels +Three independent lines, none of them a service manual (see 46.4 on evidence +class): + +1. **A community hardware guide** (x68000-dev-guide, `docs/graphics.md`): "each + pixel occupies exactly one word (2 bytes), regardless of the color mode." In + 256-colour mode the word packs **2 pages** -- page 0 is mask `$00FF` at the + `$C00000` alias, page 1 is mask `$FF00` at `$C80000`. +2. **A Japanese retro-computing writeup** (wizforest), independently: 1 word = 1 + dot in every display mode, and in 256-colour mode only the lower 8 bits of a + 1 MB region are valid. It states the masking as the well-known nuisance it + was, at a *different* depth: "16色モードでは 16bit を書き込んでもハード的に + マスクされてしまって 4bit しか書きこまれない" -- in 16-colour mode a 16-bit + write is hardware-masked down to 4 bits. +3. **px68k's write path**, which is the mechanism the other two describe: + +```c +case 1: /* 256 colors */ + if ( adr<0x100000 ) + if ( !(adr&1) ) { /* the other byte of the word: discarded */ + ... + if (adr&0x80000) adr+=1; /* the $C80000 alias IS the other byte */ + adr &= 0x7ffff; + GVRAM[adr] = data; +``` + +**The decisive detail is that both pages derive `line` identically**, from +`(adr&0x7ffff)>>10` -- 1024 bytes per scanline for 512 pixels. Page 0 and page 1 +are the two bytes of one word **at the same screen coordinate**. The same holds +in 16-colour mode, where `page = (adr>>17)&0x0c` selects one of four nibbles of +the word: four pages, one coordinate. There is **no graphics mode in which one +16-bit word holds two horizontally adjacent pixels.** + +### 46.2 The near-miss is worth writing down so it is not re-derived +Page 1 has its own scroll register, so scrolling it one pixel relative to page 0 +would put its byte at screen x+1 while page 0's sits at x -- two adjacent screen +pixels from one word. It does not work, and the arithmetic is why: page 1 stores +a byte for **every** coordinate, not every other one, so to let page 0 show +through on alternate columns you must write page 1's transparent index there. +That is the same byte count. 1024 bytes per row still buys ~512 screen pixels. +The 2:1 is structural, not an addressing accident. + +**So FINDINGS 44.7 stands unchanged.** The decoder-free player still needs +1,152 KB/s and 1.61 GB, and is still killed by the medium. The `2c` span pixel, +the c=5.48 span-vs-V1 crossover, and every figure resting on two disk bytes per +picture byte are unmoved. + +### 46.3 But the text plane is 4bpp PLANAR, and that is 4x denser on the wire +The tax is a property of the *graphics* planes. The X68000's text plane is not +laid out that way at all -- px68k's `TVRAM_Write` addresses **four planes at +`0x20000` stride**, each 0x20000 = 131,072 B = 1024x1024 **bits**. That is 4bpp +planar: **0.5 bytes per pixel**, against 2.0 for 256-colour graphics. + +The wire arithmetic, and it is only arithmetic: + +| surface | B/pixel | frame (256x192) | at 12fps | 22.8 min of game | +|---|---:|---:|---:|---:| +| 256-colour graphics, literal (44.7) | 2.0 | 98,304 | 1,152 KB/s | 1.61 GB | +| **shipping DLX3 container, compressed** | -- | -- | **496.7 KB/s** | **0.70 GB** | +| **4bpp planar text plane, literal** | **0.5** | **24,576** | **288.0 KB/s** | **0.40 GB** | + +**An UNCOMPRESSED 16-colour frame is 42% cheaper on the wire than this project's +compressed 256-colour stream**, and it needs no decoder at all -- the planar +conversion is an encoder-side transform, so the disk delivers plane words that +go straight out. On clocks it is not close either: 24,576 bytes/frame is +`24,576c`, which at c=5 is 122,880 clocks = **14.7% of a 12fps budget**. + +**It was measured the same session, and it is dead.** `tools/analysis/18_text_plane_16col.py` +over the 120-frame `singe` window, PSNR against the 24-bit source, generous to +the 16-colour side on every axis the hardware allows -- per-frame palettes (the +text palette is 16 entries; reloading it is 16 words a frame, nothing against +833,333 clocks) which the 256-colour path *cannot* use, because its codebooks +are indices into a scene-wide palette: + +| | mean PSNR | min | max | +|---|---:|---:|---:| +| 256 colours, scene palette (the tree's) | 31.33 | 27.08 | 34.08 | +| 256 colours, per-frame palette | 34.08 | 32.19 | 38.32 | +| 16 colours, scene palette | 23.17 | 17.36 | 25.87 | +| **16 colours, per-frame palette** | **25.49** | 22.99 | 28.48 | + +**256 -> 16 costs 5.84 dB at each side's best.** Against the shipping container's +**29.19 dB at 496.7 KB/s**, a 16-colour literal delivers **25.49 dB at 288.0 +KB/s** -- **3.70 dB worse for 58% of the bitrate.** A codec that buys 3.70 dB for +1.72x the bytes is doing its job; the wire saving does not pay for the colours. + +**Closed.** The user's call was to drop the 16-colour direction outright, and the +number agrees with it, so the text plane is not pursued. FINDINGS 7's 256-colour +claim stands, and it now stands on a measurement rather than on preference. + +Two honest notes on that measurement: +- **The dither row is void.** A Floyd-Steinberg run was included and came out + *bit-identical* to the undithered one, which means PIL ignored `dither=` under + `MEDIANCUT` rather than that dither is free. It is left out of the table. It + does not change the conclusion -- dither lowers PSNR by construction, and the + lead was already 3.70 dB short. +- **Everything else about the path stayed unestablished and now stays that way**: + TVRAM wait states (every `c` in this tree is a GVRAM figure), the text plane's + geometry and priority against the graphics planes, and the fact that a planar + word spans 16 pixels of one bitplane, so VQ blocks and v7 spans -- both chunky + -- would not survive the change unaltered. None of it was worth measuring once + the colour cost came in. + +### 46.4 Evidence class, stated plainly +46.1 is **secondary documentation plus an emulator's mechanism**, not primary. No +service manual, CRTC databook, or real board was consulted. What changed since +44.7 is the *kind* of evidence: 44.7 rested on MAME and px68k both being +pixel-exact against one-pixel-per-word, which is evidence about two emulators +that could be identically wrong. Now two independent documents describe the same +mechanism -- sub-word fields are pages sharing a coordinate -- and px68k's code +implements exactly that mechanism, including the `$C80000` alias landing on the +adjacent byte. Agreement on a mechanism is much harder to get wrong by accident +than agreement on an output. + +It is still not a board. STATUS item 3 (single- vs dual-address, 43.2/43.3) is +unaffected by any of this and remains the largest open hardware fact. + +### 46.5 CORRECTION: CRTC R20 bit 11 turns the masking off +46.1 concluded there is no packed write path. That conclusion was drawn from +px68k's `GVRAM_Write` and two documents describing the default behaviour, and it +**missed a register bit that both emulators implement**. MAME's +`x68k_crtc_device::gvram_w` shows it first: + +```c +if (m_reg[20] & 0x0800) /* "G-VRAM set to buffer" */ +{ + if (offset < 0x40000) + m_gvram_write_callback(offset, data, mem_mask); /* FULL WORD, unmasked */ +} +else switch (m_reg[20] & 0x0300) +{ + case 0x0100: /* 256 colour */ + if (offset < 0x40000) + m_gvram_write_callback(offset, data & 0x00ff, 0x00ff); + else if (offset < 0x80000) + m_gvram_write_callback(offset - 0x40000, (data & 0x00ff) << 8, 0xff00); +``` + +The `case 0x0100` arm confirms 46.1 exactly -- `offset - 0x40000` is the **same +word, other byte**, and `data & 0x00ff` throws the CPU's high byte away on both +aliases. But the `m_reg[20] & 0x0800` arm bypasses the depth switch entirely and +writes the full 16 bits. + +**px68k has the identical bit**, which is what makes this a mechanism and not a +MAME quirk: `if (CRTC_Regs[0x28]&8)`, commented `65536モードのVRAMアクセス +（Nemesis用）` -- "65536-mode VRAM access, for Nemesis". px68k's `CRTC_Regs` is +byte-indexed, so `[0x28]` is the **high byte of R20** and bit 3 of it is bit 11 +of the register. Same bit, same effect, two independent implementations, and a +named shipping game that used it. + +**So the 2:1 tax is a property of the default write path, not of the memory.** + +### 46.6 A contiguous packed layout, derived and NOT yet tested +The second thing 46.1 missed is that the two 256-colour pages have **independent +scroll registers**. px68k's `Grp_DrawLine8(int page, int opaq)` indexes +`GrphScrollX[page*8]` and `GrphScrollY[page*8]`, selects the byte within the word +with `add esi, ecx` (page 0 -> low, page 1 -> high), and takes an `opaq` flag -- +so the pages composite with transparency and can be offset from each other. + +46.2 dismissed interleaving on byte count, and that dismissal assumed a 1-pixel +scroll. **Scrolling by 128 instead makes the used words contiguous**, which is +the whole difference: + +- GVRAM row stride is fixed at 1024 bytes = 512 words (`shl esi, 10`). +- Write words 0..127 of each row, **unmasked, full 16 bits** (R20 bit 11). +- Page 0, unscrolled: `page0[i]` displays at screen column `i` -> columns 0..127. +- Page 1, X-scrolled by **+128**: `page1[i]` displays at column `i+128` -> + columns 128..255. +- Page 1 opaque and above page 0, so page 0's stale storage at 128..511 is + covered; page 1's storage at 128..511 displays at 256..383, which is off the + edge of the **real 256x256 mode this project already uses** (FINDINGS 23). + +**128 contiguous words carry 256 pixels: 1.0 byte per pixel, against 2.0.** No +transparency mask to maintain, no stride for a DMAC to skip, and the writes are +`movem`-shaped. If it holds, it is exactly the halving 44.7 named: + +| | B/pixel | frame | at 12fps | 22.8 min | +|---|---:|---:|---:|---:| +| 256-colour, default masked path | 2.0 | 98,304 | 1,152 KB/s | 1.61 GB | +| **packed via R20 bit 11 + page scroll** | **1.0** | **49,152** | **576 KB/s** | **0.81 GB** | + +**This is a derivation, not a result.** What is CONFIRMED is the register bit +(both emulators, plus a named game) and the per-page scroll and opacity (px68k's +draw path). What is DERIVED and untested is the layout above: whether R20 bit 11 +coexists with 256-colour *display* rather than forcing the 65536-colour +interpretation, whether the video controller's priority and transparency +registers can put page 1 over page 0 the way this needs, and what the 256-wide +screen does to page 1's off-edge storage. + +**Unlike 43.2 and 43.8.1, this one the tree CAN answer.** It is a display-model +question, both emulators implement the mechanism, and the rig already screenshots +and compares pixel-exactly (`tools/bench/verify_frame256.py`). It is a register +setup and a snapshot, not a service manual -- and if it holds it halves the wire +for every path in this project, the shipping codec included. + +## 47. The packed layout works on both emulators, and they disagree about what it costs (session 16) + +46.6 derived a 1.0 byte/pixel layout and did not test it. It has now been built +and run on **both** emulators. **It renders correctly on both.** The two +disagree on two register semantics, and one of those disagreements decides +whether the thing is usable. + +### 47.1 The write path, measured directly +`tools/bench/probe_packed.lua` writes one word and reads the two page aliases +back, under MAME: + +| R20 | wrote | raw word | page 0 (low byte) | page 1 (high byte) | +|---|---|---:|---:|---:| +| 0x0110, bit 11 = 0 | `AB5C` | `005C` | `5C` | `00` | +| **0x0910, bit 11 = 1** | `AB5C` | `AB5C` | `5C` | **`AB`** | + +**That is the 2:1 tax, and its off switch, in one table.** Masked, the CPU's high +byte is destroyed. In buffer mode one word write lands two picture bytes. +46.5's reading of `gvram_w` is confirmed by experiment, not just by code. + +### 47.2 The packed layout renders correctly, on both +`tools/bench/show_frame256_packed.lua` (MAME) and `tools/bench/gvpack` (px68k): + +- page 0, opaque bottom, unscrolled -> screen columns 0..127 +- page 1, transparent top, X-scroll **384** -> columns 128..255, because column + `c` fetches `page1[(c+384) & 511]` +- words 0..127 of each row carry both halves: `(right << 8) | left` +- words 128..511 zeroed **once** -- page 1's storage at 384..511 sits under + columns 0..127 and must read 0 so the opaque page 0 shows through. Static + setup, not per-frame payload. +- the blob is built `--pack-transparent`: index 0 is the transparency key, so it + never appears in the picture and black lives at 255. + +| | result | palette ceiling | +|---|---|---:| +| MAME, `verify_frame256.py` | **256x192 pixel-exact, letterbox true black** | 40.83 dB | +| px68k, `verify_gvpack.py` | **256x192 index-exact, letterbox on 255** | 40.83 dB | + +**Per-frame payload: 128 words/row x 192 rows = 24,576 words = 49,152 bytes for +49,152 pixels. 1.0 B/pixel, against 2.0.** + +`tools/bench/gvpack` links px68k's real `x68k/gvram.c` -- the address decode, the +bit-11 write path, the page-byte selection, the scroll wrap and the index-0 +transparency test are px68k's own code, the way `tools/bench/c68k` links its CPU +core. The only mirrored part is `windraw.c`'s twelve-line page-ordering +dispatch, which is SDL-bound; it is quoted verbatim in `pick_order()`. + +**Four negative controls, because a test that cannot fail proves nothing:** + +| control | expected | got | +|---|---|---| +| ordinary unpacked 2.0 B/px path | pass | pass | +| packed, **bit 11 OFF** | fail | fail: right half is 24,576 px of index 0 | +| packed, **page-1 scroll removed** | fail | fail: 49,073 px differ | +| packed, priority `vc1=0x00` | fail (per MAME) | **PASS on px68k** -- see 47.3 | + +### 47.3 Disagreement 1: the priority register, when the fields are equal +Video controller R1 (`0xE82500`) decides which page composites on top. + +| `vc1` | MAME | px68k | +|---|---|---| +| `0x0000` | page 1 **not shown** -- right half black, 24,576 px differ | page 0 on top **transparently** -- renders correctly | +| `0x0002` | page 1 on top, **correct** | page 1 on top, **correct** | + +They agree at `0x0002` and that is what the layout uses, so the result stands on +a setting both model identically. But the packed layout's correctness rests on a +register the two emulators do not model the same way, and neither is authority. + +### 47.4 Disagreement 2: does buffer mode BLANK the display? -- and this one decides it +`tools/bench/probe_bit11_blank.lua` is the known-good 256-colour test with **one +line added**, setting bit 11: + +- **MAME: the screen goes fully black.** Max channel 0, zero non-black pixels. + Buffer mode is a write *window*, not a display mode -- which is why the working + test clears bit 11 after painting. +- **px68k: it does not blank.** `Grp_DrawLine8` never reads `CRTC_Regs[0x28]`, + and `gvpack --keepbuffer` renders the frame correctly with the bit still set. + +**This is the question the packed path lives or dies on.** If MAME is right, the +graphics layer is blanked for the whole time the CPU or DMAC is painting, and a +12fps full-frame player would show black for whatever fraction of each frame the +paint takes. If px68k is right, the packing is free. Both are plausible readings +of "G-VRAM set to buffer", and px68k's own comment -- `65536モードのVRAMアクセス +（Nemesis用）` -- says the bit exists for a game that blasted graphics through it, +which is at least consistent with the write-window reading. + +**It is a hardware fact, and it is now the cheapest high-value one outstanding** +-- cheaper than 43.2, because a single real board plus the two-line probe above +settles it, and because the answer moves more numbers. + +### 47.5 What it would be worth, DERIVED +Arithmetic on measured constants, not measurements. **Every line below is void if +47.4 goes MAME's way**, and the mode-decision cost model (44.3, 43.1) would need +re-deriving from scratch either way. + +| | default masked | **packed** | +|---|---:|---:| +| bytes/frame into GVRAM | 98,304 | **49,152** | +| word writes/frame | 49,152 | **24,576** | +| wire at 12fps | 1,152 KB/s | **576 KB/s** | +| 22.8 min of game | 1.61 GB | **0.81 GB** | +| DMAC device->GVRAM, c=5 | 59.0% of budget | **29.5%** | +| CPU-painted literal, c=5 | 19.14 clocks/px (**misses** by 13%) | **9.57 (fits, 44% spare)** | +| CPU-painted literal, c=9 | 27.14 (**misses** by 60%) | **13.57 (fits)** | + +The CPU-painted row is the one that overturns something. 44.7 concluded "a +CPU-painted full-frame literal does not fit at any physically reachable price, +and it misses at the floor itself." Packed, one `movem` word carries two pixels, +so the per-pixel cost becomes `(9.143 + 2c)/2 = 4.571 + c` -- and it fits at c=5 +**and** at c=9. That conclusion is withdrawn, conditionally on 47.4. + +### 47.6 What this does NOT establish +1. **Correctness was tested, cost was not.** Both harnesses write GVRAM + directly -- MAME through Lua's address space, `gvpack` by calling + `GVRAM_Write`. Neither runs 68000 instructions, so no clock in 47.5 is + measured. The `movem` shape of the packed writes is an assumption. +2. **The DMAC has not been near this.** 44.7's device->GVRAM transfer in buffer + mode is untested, and 43.2's single- vs dual-address question sits underneath + every `c` in 47.5. +3. **One frame, not a stream.** A static frame was painted and snapshotted. + Nothing here exercises per-frame toggling of bit 11, and if MAME is right + about blanking, that toggling is the whole problem. +4. **The codec was not considered.** VQ blocks and v7 spans address chunky + pixels; under the packed layout a word spans two columns 128 apart. Whether + the existing codec survives that is untouched -- 47 is about a literal frame. +5. **Two of 46.6's three assumptions held, one was wrong.** R20 bit 11 does + coexist with 256-colour display (after clearing it), and the off-edge storage + behaves. The priority guess was wrong: 46.6 assumed page 0 on top; it is + page 1. + +## 48. The blanking disagreement is not symmetric, and the paper trail favours MAME (session 17) + +47.4 filed the blanking question as two emulators disagreeing, and called both +readings of "G-VRAM set to buffer" equally plausible. **They are not equally +plausible, and the two implementations are not making the same kind of +statement.** Nothing here is a board, so 47.4 is not closed -- but the prior +moves, and it moves against the packed layout. + +### 48.1 px68k is silent, not dissenting +`grep -a` matters here: `x68k/gvram.c` carries EUC-JP comments, so a plain +`grep` treats it as binary and reports **no matches at all** for any pattern. +Read with `-a`, R20's high byte appears in exactly one file: + + $ for f in x68k/*.c x11/*.c; do n=$(grep -ac "CRTC_Regs\[0x28\]" $f); ... + x68k/gvram.c: 6 + +and all six are the address decode -- lines 150/153/158 inside `GVRAM_Read`, +211/221/225 inside `GVRAM_Write`, where `CRTC_Regs[0x28]&8` is R20 bit 11 and +line 211 carries the comment `65536モードのVRAMアクセス（Nemesis用）`. + +**No px68k display code reads the bit anywhere.** `Grp_DrawLine8` and +`x11/windraw.c` never consult it. So px68k does not model buffer mode as +non-blanking; it does not model the display side of buffer mode **at all**. +Its "no" is an omission. + +MAME's "yes" is the opposite -- a deliberate, commented claim, made twice: + + x68k_v.cpp:407 if(m_crtc->gfx_layer_buffer()) // if graphic layers are set + return false; // to buffer, they aren't visible + x68k_v.cpp:766 if((m_video.gfx_pri == priority) && !m_crtc->gfx_layer_buffer() && ... + +and the parallel bit 12 is modelled the same way for the text layer +(`x68k_v.cpp:756`, `!m_crtc->text_layer_buffer()`). Someone implemented a +semantic on purpose. **An assertion and a silence are not a tie.** + +### 48.2 The register table names the bit "for display / for buffer" +Sharp's own R20 bit map, transcribed on Data Crystal's X68k IOMAP: + +| bit | name | %0 | %1 | +|---|---|---|---| +| 12 | T-MEM | `表示用` | `バッファ用` | +| **11** | **G-MEM** | **`表示用`** | **`バッファ用`(bit 10〜8 は無効)**
`(G-VRAM が 65536 色表示時と同じ構造になる)` | +| 10 | SIZE | 512x512 | 1024x1024 | +| 9-8 | COL | %00 16 / %01 256 / %11 65536 | | + +Two things in that one row: + +1. **The bit is named "for display" against "for buffer"** -- not "16-bit write" + or "unmasked access". The naming is MAME's reading. +2. **`bit 10〜8 は無効` -- the colour-mode field goes invalid.** COL is what the + display side decodes a plane structure from. A layer whose colour mode is + undefined has nothing to render with, which is a *mechanism* for blanking + rather than a restatement of it. + +The counter-reading survives, and it is the parenthetical: the doc says the +structure becomes the 65536-colour one, and does **not** say the screen goes +dark. That is why this is a shifted prior and not a result. + +### 48.3 The cost of the MAME branch, DERIVED, and it is not a partial blank +47.4 left "measure what fraction of a frame the paint takes" as the fallback. +Bit 11 only has to be set across the GVRAM writes, so the blank interval is the +paint, not the frame. The measured blit is **53.6% of the frame budget** +(session 9) unpacked; packed halves the word count, so the floor is ~27% and the +ceiling ~54% depending on how much of the blit is stores. + +**Either end is fatal for this content.** The graphics layer would be visible +between roughly half and three-quarters of each frame at 12fps, with the black +interval locked to frame rate -- a 12 Hz strobe over the whole picture, not a +tear or a partial band. And the packed layout has **no page left to flip to**: +both 256-colour pages carry picture, which is the entire point of it. There is +no version of the MAME branch where the packing is merely expensive. + +### 48.4 px68k cannot testify about SCSI at all (bears on 43.2) +Checked while looking for a second opinion on single- vs dual-address. **px68k +does not emulate the MB89352.** `x68k/scsi.c` is 81 lines: it synthesises a +64-byte fake CZ-6BS1 boot ROM at `$EA0020` (the `Human68k` signature, the IOCS +`$F5` vector, `move.b d1,$e9f800`) and traps the IOCS call on the host. Its own +header says so -- `SCSI IOCS を特殊処理で対応。SPCはエミュレートしない`. + +So the "second emulator" method that settled 46/47 **is not available for +43.2**, and never was. MAME models the dual-address row and has no DACK path; +px68k models no SPC. Item 2 needs the CZ-6BS1's `scsiexrom.bin` (8 KB, CRC +`7be488de`, absent here) disassembled for its DMAC DCR programming -- or a +schematic. It does not need another emulator. + +**And the field to read is now pinned to the primary source.** MC68450 +datasheet (Motorola ADI1216, Jul-89 printing), section 3.6.1 -- the DCR field +order is `XRM | DTYP | DPS | PCL`, MSB to LSB, and DTYP is the answer in one +two-bit field: + +| DTYP | datasheet text, verbatim | addressing | +|---|---|---| +| `00` | `M68000 Compatible, Explicitly Addressed` | **dual** | +| `01` | `M6800 Compatible, Explicitly Addressed` | **dual** | +| `10` | `Device with ACK, Implicitly Addressed` | **single** | +| `11` | `Device with ACK and RDY, Implicitly Addressed` | **single** | + +and 3.6.1.2 states the equivalence outright rather than leaving it to be +inferred: + +> For M68000 type devices, the DMAC will use a **dual address transfer +> protocol** by running M68000 type bus cycles to transfer data to or from the +> device registers and a second bus cycle to complete the operand transfer from +> or to memory. [...] In the remaining two device protocols, the DMAC asserts +> the acknowledge signal to **implicitly address the device during a single +> address transfer** while it is explicitly addressing a memory location. + +So 43.2's 5.0-vs-9.0 clocks/byte is decided by two bits in one byte the +CZ-6BS1's boot ROM writes at init. (Bit *positions* within DCR are the +conventional 7-6/5-4/3/2-0 split; the datasheet's own bit-number row did not +survive OCR, so treat the positions as unconfirmed and the field order and +encodings as quoted.) + +**MAME confirms the negative half of this from the other side:** +`hd63450.cpp` contains **no DTYP handling at all** -- the only device-shape +field it decodes is `ocr & 0x30` (operand size). It cannot express an +implicitly-addressed device, which is why 43.2 was right that MAME settles +nothing here. + +### 48.5 One thing the hunt did confirm independently +px68k's `kaiseki.txt` (the author's own analysis notes, 2014/2/14), on GVRAM +structure, unprompted and predating any of this: + + 256色の場合は、Page0の(0,0), Page1の(0,0), Page0の(1,0), Page1の(1,0)... と交互に並ぶ。 + +Page 0 and page 1 bytes alternate within the word at the *same* coordinate. +That is 46.1's page-not-pixel finding from a third source, and it is the +premise the packed layout is built on. **The layout's premise is solid; only +its visibility is in doubt.** + +## 49. The streaming path, built and run: contiguity is the constraint, and the shipping rate does not fit the pipe (session 18) + +STATUS item 3 has been open since session 7, and 45.4.1 stated the gap in as +many words: **the gate proves the decoder is pixel-exact over a whole window and +says nothing about how the bytes get to it.** `tools/bench/decode.lua` preloads +5,261,814 B of container into emulated RAM and lets `a0` walk through all of it. +The shipping player never holds a window at once. + +That rig is now built. `src/player/stream.s`, `tools/bench/prep_stream.py` and +`tools/bench/stream.lua` decode the gate container **out of a bounded ring**, +with a modelled SCSI pipe as the producer, and the container living in a HOST +file rather than in emulated RAM. + +### 49.1 The result: pixel-exact from a ring one twentieth the size of the stream + +| ring | machine | wraps | mean hole | result | +|---|---|---:|---:|---| +| 256 KB | **stock 2 MB** | 18 | 14.7 KB (5.7%) | **120/120, final frame pixel-exact** | +| 128 KB | stock 2 MB | 37 | 10.6 KB (8.3%) | 120/120, pixel-exact | +| 96 KB | stock 2 MB | 54 | 15.6 KB (16.2%) | 120/120, pixel-exact | +| 80 KB | stock 2 MB | 60 | 8.9 KB (11.2%) | 120/120, pixel-exact | +| 64 KB | stock 2 MB | 107 | 23.4 KB (36.6%) | 120/120, pixel-exact | +| 48 KB | stock 2 MB | 111 | 8.9 KB (18.5%) | 120/120, pixel-exact | + +Verified by `tools/bench/verify_decode.py`, the same comparison that gates +`decode.s`: the last frame against `tools/encoder/dlx.py`'s reconstruction, and +because a SKIP block is a claim about the previous frame still being in GVRAM, +the last frame is only right if all 120 were. + +**A side effect worth naming: the rig's RAM ceiling is gone.** FINDINGS 44.6.4 +audited 37/120 frames because the container did not fit 2 MB, and 45 raised +`RIG_RAM` to 6 MB to fix it. The streaming rig holds ~256 KB of stream and +reads the rest from the host, so **a stock 2 MB machine runs the whole window** — +and the machine it runs on is now the machine the player targets, rather than a +rig-shaped one. + +### 49.2 The constraint is CONTIGUITY, and a byte-counting simulation cannot see it +`09_buffer_sim.py` asked whether cumulative supply ever falls behind cumulative +demand, in bytes, and FINDINGS 21 answered "zero required prefill". That test is +necessary and not sufficient. The block loop and the span chain read the stream +with a monotonically increasing `a0` and **no bounds check anywhere** — +`move.l (a0)+,d0`, `lea MODEB(a0),a0`, eleven unrolled `movem.l (a0)+`, a +`move.b (a0)+` per block index. None of it survives an address that wraps +mid-record. + +So the ring needs the **whole next record resident AND contiguous**, not merely +enough bytes by the deadline. Those are different conditions and only the +second one is a byte count. `tools/analysis/19_ring_stream.py` models the ring's +addresses rather than its occupancy. + +### 49.3 `aligned` beats `split`, and it is not close +Two policies can give the reader a contiguous record. Both columns below are for +**`s14_d5_all1500`**, the shipping candidate, in a 256 KB ring — the costs scale +with the container's record sizes, so they must be quoted per container: + +| policy | mechanism | RAM cost | CPU cost | +|---|---|---:|---:| +| **`aligned`** | producer refuses to start a record it cannot finish; leaves a hole, restarts at 0 | 23.4 KB mean hole = **9.1%** of the ring | **0** | +| `split` | records wrap; ring's first MAXREC bytes mirrored into a shadow past its end | 0 | **46,394 clk/frame = 5.57% of the frame budget, forever** | + +For the lighter gate container `rc_fr_singe_scsi_span` the same trade is +**5.7% of the ring against 3.64% of the frame budget** — same direction, same +verdict, smaller numbers. + +The decoder already spends **77.0% of the budget on the mean frame and 91.1% at +p90** (FINDINGS 45). `split` puts p90 at **96.7%**. RAM is the thing this machine +has 2 MB of; clocks are the thing it has none of. + +`aligned` also needs a per-record **index** on the fill side — and a branching +laserdisc game needs one anyway to seek to a branch point. **The policy that +costs no clocks reuses a structure the player cannot avoid.** + +The third option — teach the block loop to wrap its own reads — is the expensive +one, and not because of the branch. A bounds test lands *inside* the instruction +sequences FINDINGS 30.4 and 40 fitted their constants to, so it does not cost a +compare: it costs **every span and per-block figure in the tree being +re-measured**. + +### 49.4 Two independent implementations agree exactly +The Python simulation predicts the ring's behaviour from record sizes alone; the +Lua producer drives a real 68000 through MAME. They agree to the digit, and at +two ring sizes rather than one -- so it is not a coincidence of a single tiling: + +| ring | | wraps | mean hole | usable ring | +|---|---|---:|---:|---:| +| 256 KB | `19_ring_stream.py` (from record sizes) | 18 | 14.7 KB | 94.3% | +| 256 KB | `stream.lua` (driving the 68000) | **18** | **14.7 KB** | **94.3%** | +| 128 KB | `19_ring_stream.py` | 37 | 10.6 KB | 91.7% | +| 128 KB | `stream.lua` | **37** | **10.6 KB** | **91.7%** | + +They share no code. This is the same class of check as FINDINGS 46's — agreement +on a *mechanism* rather than on an output. + +### 49.5 The shipping candidate does not fit the 488 KB/s pipe, and nothing was checking +`s14_d5_all1500` is the session-14 candidate: **29.19 dB at 496.7 KB/s** (43.6). +The pipe constant this tree has simulated against since session 2 is +**488 KB/s**. Those two numbers have never been put side by side. + + wire demand 496.7 KB/s - pipe 488.0 KB/s = 8.7 KB/s OVER, on the MEAN + +This is **not a burst a ring absorbs**. The deficit grows 744 B per frame for as +long as the scene runs — 87 KB over the 120-frame window, **523 KB per minute of +play**. No ring size fixes a sustained overrun, and quoting the window's 118.4 KB +"required prefill" for it would be the most flattering possible way to state it. + +**Why it was never caught.** 42.1 established that 488 was never a bus figure and +that the binding resource is *clocks*, not KB/s — so the rate controller was +built to bind on `decode + c*bytes` and has **no pipe term at all**. That was a +defensible decision. What was not decided is that FINDINGS 21's buffer sizing, +and its "zero required prefill", would keep standing on a constant the design had +stopped enforcing. Item 4 has been open since session 7 for exactly this reason. + +**The useful output is a requirement on the medium, not a verdict.** Since 488 is +unmeasured folklore and the intent is to measure a BlueSCSI directly, the tool +reports the threshold to measure *against*: + +| container | wire | **zero-prefill pipe** | vs 488 | +|---|---:|---:|---:| +| `s14_d5_all1500` (the candidate) | 496.7 KB/s | **513.2 KB/s** | +5% | +| `rc_fr_singe_scsi_span` (the gate) | 446.1 KB/s | **451.4 KB/s** | -8% | + +**513.2 KB/s is now a hardware acceptance test**, and it is 33% of SCSI-1's +asynchronous rating and 10% of its synchronous one (42.1). It is very likely met; +it has never been shown to be met. + +### 49.6 The rig measures ARRIVAL, and the first version of it measured the wrong thing +`stream.s` has no frame clock — it asks for the next record the instant it +finishes the last. So it outruns any finite pipe, and its spin counter reports +**91 of 120 frames "stalled" at a pipe the same run shows is fast enough**. A +shipping player waits for vblank and spends that same time idle. Reporting that +count as an underrun would have been a false finding of exactly the shape this +project keeps filing. + +The rig now records the emulated time at which each record becomes **resident** +and checks it against a 12 fps deadline, which is a question about arrival alone +and does not need the decoder paced: + +| pipe | decoder waited | **records late** | worst | **required prefill** | +|---|---:|---:|---:|---:| +| unlimited | 0/120 | **0/120** | — | 0 | +| 520 KB/s | 85/120 | **0/120** | — | 0 | +| 488 KB/s | 91/120 | **1/120** | 4.9 ms (0.06 fr) | 2.3 KB | +| 460 KB/s | 97/120 | **2/120** | 9.7 ms (0.12 fr) | 4.4 KB | + +The 2.3 KB at 488 against the simulation's 0.0 KB is a modelling difference, not +a disagreement: the Lua producer delivers in **whole records**, the simulation in +**512-byte sectors**, and whole-record granularity is the more conservative of +the two by up to one record's worth of latency. + +### 49.7 What this does NOT establish +1. **The DMAC's clock debit is not modelled.** The pipe here is a constant byte + rate on the emulated clock. It is honest about *arrival order and residency*, + which is what a ring manages, and says nothing about the clocks the DMAC + steals from the 68000 while it delivers (43.2, and `W` is still undecided). + **A zero-late result from this rig means "the bytes were in time", NOT "the + frame fits".** +2. **Buffer stall-tolerance at a branch point is still untested.** Because the + decoder free-runs, the ring never backs up, so the ring-size sweep in 49.1 + tests *wrap correctness* at each size and **not** the buffering that a seek + needs. At 64 KB the mean hole is 36.6% and effective capacity is one record — + it is single-buffered, and it passes anyway. Do not read 48 KB as a viable + player buffer. +3. **The producer is not an MB89352.** No sector-level command overhead, no + arbitration, no seek. `19_ring_stream.py` quantises to 512 B; the rig does not. +4. **The hole is a tiling effect and is not monotonic in ring size** — 96 KB + wastes more than 80 KB does. It depends on how record sizes tile the ring, so + a ring should be sized against the *measured* hole for the container it will + carry, not against a fraction. +5. **An unexplained cross-emulator gap, left open rather than explained away.** + The ring pass decodes the gate container at **561,532 cycles/frame under + MAME**. FINDINGS 45's figure for the same container is **641,444 cycles/frame, + and it is px68k's C68K core**, not MAME's -- so the two are not comparable and + the 12% between them is not evidence of anything yet. Getting MAME's own + full-pass number for this container needs a `decode.lua` timing run that did + not complete inside 25 minutes on this host; it was killed rather than left to + race another MAME job, which is how session 18 lost its first attempt at it. + **No figure in this section rests on the comparison.** Whoever picks it up: + run `decode.lua` on `rc_fr_singe_scsi_span.dlx` with nothing else touching + `tmp/`, and compare its "full 120-frame pass" against 561,532. If they agree, + the C68K/MAME delta is the thing to explain; if they do not, `stream.s`'s + frame loop is. +6. **`decode.s` is unchanged, and provably.** The block loop and span chain were + moved to `src/player/frame.i` and the constants to `geom.i` so both + front-ends assemble from literally the same bytes; `decode.s` still assembles + to the same **1,296 bytes** it did before the split, and `prep_dlx.py` still + emits a byte-identical blob after the loader maths moved to + `tools/bench/dlxload.py`. Both are asserted in `check.sh`. + +## 50. The pipe constant is retired (session 18, USER DECISION) + +**USER DECISION, after 49.5: remove the delivery-rate constant from the repo as +a live number.** 42.1 established in session 13 that it was never a bus figure — +a user-supplied "4 Mbps" with no recorded provenance, 10% of SCSI-1's +asynchronous rating — and 29.5 item 3 had carried "confirm the 4 Mbps figure" as +an open item since session 7. It was never confirmed. It was also never removed, +and 49.5 is what that cost: the shipping candidate ran 8.7 KB/s over it for five +sessions with nothing in the tree comparing the two. + +### 50.1 What was actually wrong was the DEFAULT, not the number +The number being unmeasured was known and written down. What kept it load-bearing +was that **six tools defaulted to it** — `12_span_tradeoff.py`, +`14_dmac_chain.py`, `16_span_roundtrip.py`, `17_span_delivered.py`, +`19_ring_stream.py` and `tools/bench/stream.lua`. A default is how a figure gets +into a table without appearing in the sentence that reports the table. Every +span figure in FINDINGS 30–41 was scored against it; none of them had to say so. + +**All six now take a REQUIRED argument with no fallback.** A tool that cannot run +without being told the rate cannot quietly assume one, and a result that had to +name its rate to exist is a result whose provenance travels with it. + +A small confirmation of the same point fell out of doing it: writing the new +help text broke `--help` on four of the tools -- the text said "10%" and argparse +read it as a format spec -- and **nobody would have noticed**, because none of +these tools had ever been run with `--help` in this tree. The removal was the +first thing that made them state their own arguments out loud. + +That is the transferable part: **an unmeasured constant is not made safe by +documenting that it is unmeasured.** 42.1 documented it perfectly and it went on +silently underwriting tables for five more sessions. It is made safe by deleting +the default. + +### 50.2 What survives, and why it is not the same thing +`GATE_SPAN_KBPS` in `tools/bench/check.sh`. The gate container was **encoded** +with it, and every per-block and span constant in FINDINGS 41/43/45/49 is fitted +to that container. Changing it is a re-encode plus a re-measurement of all of +them, not an edit. + +It is a **container recipe, not a delivery claim**, and check.sh says so at the +point of use. The distinction is the whole reason it can stay: nothing reads a +medium's throughput out of it. + +### 50.3 What replaces it: a requirement, not a constant +`19_ring_stream.py` reports the **zero-prefill pipe** — the rate a medium must +clear for a given container to need no prefill at all: + +| container | wire demand | **zero-prefill pipe** | +|---|---:|---:| +| `s14_d5_all1500` (the candidate) | 496.7 KB/s | **513.2 KB/s** | +| `rc_fr_singe_scsi_span` (the gate) | 446.1 KB/s | **451.4 KB/s** | + +Its rate sweep is now anchored to each container's own wire demand (0.90x to +2.00x) rather than to a fixed list of absolute rates, so it privileges no +constant and stays meaningful for any container. + +**This is a number to MEASURE A MEDIUM AGAINST, not one to design on**, which is +the difference that mattered. The BlueSCSI has never been benchmarked on this +machine and the intent has always been to measure it directly; it now has a +threshold to be measured against. + +### 50.4 What this does NOT do +1. **It does not measure anything.** The delivery rate remains unknown. The + tree is now honest about that rather than carrying a placeholder — which is + a smaller claim than it sounds, and the right one. +2. **It does not re-derive the tables that were scored against the old figure.** + FINDINGS 30–41's span figures stand as measured *at that rate*; what changed + is that nothing new can be scored there without saying so. Anything that + needs a delivery rate to mean something is now waiting on a measurement, and + should be. +3. **It does not touch the gate container**, so the green light and every + constant fitted to it are unmoved. `./tools/bench/check.sh` is **ALL GREEN** + after the removal. + +## 51. Pacing the decoder: seek slack is accumulated, not owned (session 19) + +STATUS item 4, open since session 7 in one form or another and sharpened by +49.7.2. `tools/bench/stream.lua` decodes 120 frames out of a bounded ring and +the pass is pixel-exact at every ring size down to 48 KB — and that result could +not be read as a statement about buffering, because `src/player/stream.s` has no +frame clock. It asks for record *i* the instant it finishes record *i-1*, so it +outruns any finite pipe, the ring never backs up, and the producer's overlap +test is never the thing that refuses a placement. **A ring-size sweep under +those conditions tests wrap correctness at each size and nothing else.** + +The rig now has a frame clock. `PACE`/`PACEON` (`$18034`/`$18038`) are written +by the producer — vblank or an MFP timer in the player — and frame *i* may not +start before tick *i*. `PACEON=0` leaves the loop free-running and is what the +green light's wrap gate still uses, so 49's figures are unmoved. + +### 51.1 The ceiling: what a ring is worth once it is full +Every cell below is a full 120-frame decode on MAME's 68000, pixel-verified +against `tools/encoder/dlx.py`, on the gate container (`rc_fr_singe_scsi_span`, +446.1 KB/s wire, 36.5 KB mean record). "Ceiling" is the largest number of whole +records resident and unconsumed, i.e. **the frames the decoder could still draw +with delivery stopped dead**. Pipe `0` is unlimited, which isolates the ring's +own capacity from the rate. + +| ring | unlimited | 460 KB/s | 488 KB/s | 520 KB/s | 600 KB/s | +|---:|---:|---:|---:|---:|---:| +| 64 KB | 2 | 2 | 2 | 2 | 2 | +| 96 KB | 3 | 3 | 3 | 3 | 3 | +| 128 KB | 5 | 4 | 4 | 4 | 4 | +| 192 KB | 6 | 4 | 5 | 5 | 5 | +| 256 KB | 8 | 4 | 7 | 7 | 7 | +| 384 KB | 11 | 4 | 10 | 11 | 11 | +| 512 KB | 15 | 4 | 11 | 14 | 14 | + +**64 KB carries two frames and 96 KB carries three.** 49.7.2 warned that 48 KB +was single-buffered and passing anyway; this is the number, and it says the +small end of that sweep was measuring nothing about delivery. + +### 51.2 Tolerance is `ceiling - 1`, and it was falsified rather than asserted +`DLX_CUT_AT`/`DLX_CUT_FR` stop the pipe dead at a chosen tick, as a seek does. +At 256 KB and 520 KB/s, with **7 records resident**: + +| cut | underruns | +|---:|---:| +| 2 frame times | 0 | +| 6 | **0** | +| 7 | **1** | +| 8 | 1 | +| 10 | 1 | + +**Seven resident records buy six frame times, not seven** — 500 ms, not 583. +The last one is spent covering the pipe's restart: a 36.5 KB record takes ~0.9 +frame times to place at 512 KB/s of video, so the record due immediately after +the cut is still arriving when its slot opens. A design that reads the resident +count as its stall budget is over by one record, every time. + +Every cut run stayed **pixel-exact**, including the ones that underran. That is +the expected shape and worth stating: under-delivery makes a frame LATE, not +wrong — `waitrec` spins and the decode is byte-identical when it resumes. +A rig that reported corruption here would be reporting its own bug. + +### 51.3 The result that bears on a branching game +Slack is not a property the buffer has. It is **accumulated out of the surplus +between the pipe and the wire demand**, and a seek spends all of it. + +| ring | pipe | ceiling | play needed to reach it, from empty | +|---:|---:|---:|---:| +| 256 KB | 488 KB/s | 7 fr (583 ms) | **4.83 s** | +| 256 KB | 520 KB/s | 7 fr | 2.83 s | +| 256 KB | 600 KB/s | 7 fr | 1.67 s | +| 512 KB | 520 KB/s | 14 fr (1.17 s) | **8.42 s** | +| 512 KB | 600 KB/s | 14 fr | 8.42 s | + +**A bigger ring raises the ceiling and lengthens the climb to it.** The fill +rate is `pipe - wire`, which is set by the encoder and the medium; the ring only +sets where the climb stops. So the question a branch point asks is not "is the +buffer big enough" but "has there been enough play since the last branch point +to refill it" — and at 488 KB/s with a 256 KB ring the answer is **4.83 seconds +of play**. Two branch decisions closer together than that and the second one has +no buffer to spend, at any ring size. Dragon's Lair's decision points are +seconds apart. + +This is the first statement in this tree about back-to-back branches, and it is +a consequence of the delivery model, not of the decoder. + +### 51.4 Which resource is binding, said out loud +The producer now counts its refusals separately. A **rate** refusal (no credit +for a whole record) means a bigger ring buys nothing; a **ring** refusal (the +decoder still owns those bytes) means a faster pipe buys nothing. From the +decoder's side the two are identical — "no new record" — and they have opposite +fixes. + +**At 460 KB/s every ring from 192 KB to 512 KB reports RATE-BOUND with a ceiling +of 4 and never fills inside 120 frames.** 460 is 13.9 KB/s over the container's +446.1 wire demand; that surplus fills 146 KB in the whole 10-second window. Any +ring larger than that is dead RAM in this scene. The zero-prefill pipe for this +container is 451.4 (49.5) — **clearing the arrival deadline and being able to +absorb a seek are different requirements, and the gap between them is large.** + +### 51.5 An independent implementation agrees, within one record +`tools/analysis/20_seek_slack.py` is the same model written from record sizes in +Python, sharing no code with the Lua producer — the 49.4 pattern. Over all 35 +cells of the grid: + +- **35/35 of the rig's ceilings fall inside the sim's bracket**, and 33/35 sit + at the top of it. + +The bracket is one record wide and is reported as a range rather than a number, +deliberately. At these rates the pipe delivers almost exactly one record per +frame slot, so "records resident at slot *i*" differs by one depending on +whether the sample is taken before or after that slot's delivery. Sampled after, +the sim matched the rig's ceiling in 33/35 cells; sampled before, it was exactly +one low in 33/35. Picking the sampling point that matched, and then reporting +the match as a cross-check, would have been fitting the model to the +measurement. Both are returned and the caller prints the range. + +### 51.6 Two rig defects the pacing exposed +Both were in the producer, both were invisible while the decoder free-ran, and +both would have made the first paced result wrong in a plausible direction. + +1. **The `RD_PTR` cross-check was really a test of how often `reap()` ran.** It + asserted, for every record it retired, that the decoder's released-to pointer + equalled that record's end. `RD_PTR` is a single pointer and names the end of + record `tail-1`; retiring several records in one pass — normal the moment the + decoder is paced and the pipe stops — made it fire a false MISMATCH on the + earlier ones. It now asserts on `tail-1` only, which is the invariant that + actually holds. +2. **`reap()` was skipped for the duration of a cut.** The cut returned early + from `produce()`, so the ring looked full for the whole seek and the producer + could not restart against space the decoder had long since released. A seek + stops delivery; it does not stop the decoder. Only credit and placement stop + now. + +### 51.7 What this does NOT establish +1. **The pipe is still a model and the medium is still unmeasured.** Every rate + in 51.1 is a chosen input, not a measurement — FINDINGS 50 stands, and + `pace_run.sh` requires the rate for the same reason. The columns are a + sensitivity table; none of them is a claim about a BlueSCSI. +2. **The DMAC's clock debit is still not modelled** (49.7.1). A paced decoder + makes the ring's behaviour honest and does nothing about the clocks the DMAC + steals while it fills. "Survives a 500 ms cut" means the bytes were there, + not that the frames fit. +3. **One container, one scene.** The ceilings are in whole records, so they move + with record size; a scene with heavier frames has a lower ceiling in the same + ring. Nothing here is a universal per-KB figure and the tools take the + container as an argument for that reason. +4. **The pace gate costs the rig a `tst.l`/`beq.s` per frame** — `stream.s` is + 1,418 bytes against 1,396. It is outside `src/player/frame.i`, so every + per-block and span constant in FINDINGS 24/30/40/41 is untouched, and + `decode.bin` is still 1,296 bytes at the same MD5 (asserted in `check.sh`). +5. **Seek TIME itself is not modelled.** The cut is a chosen duration. What a + real seek costs on the target medium is part of the same unmeasured + question as the rate, and it is the other half of what item 1 buys. + +--- + +## 52. The DMAC configuration was never a mystery: it is in the IPL ROM (session 20) + +ROADMAP called the ADPCM stream "the largest unpriced risk left in the project" +and asked for one cheap thing first — **put the audio DMA on the bus and see +what it does to the 86.7%.** Doing that needs a clocks-per-byte figure for the +audio channel, and the tree did not have one: `11_cpu_budget.py` charged audio +bytes `--dma-clocks-per-byte`, the *disk's* rate, defaulting to 5 and described +in its own help text as "single-address, bus held, no drive wait". That is a +description of the SCSI channel, and it is the favourable end of ROADMAP B3, an +open question worth 242 KB/s. **Audio was being charged the disk's guess.** + +It did not have to be a guess for either of them. The X68000's IPL ROM programs +all four HD63450 channels itself, and **the ROM is on this machine** — MAME +boots the player rig with `-bios ipl10`. `tools/analysis/21_iplrom_dmac.py` +reads the configuration out of the image and decodes the MC68450 register +fields. It is a gate, not a report: every value is (address, expected bytes, +meaning), eight sites, and a mismatch or an unrecognised ROM revision exits +non-zero rather than decoding some other code. It is in `check.sh`, needs no +emulator, and runs in milliseconds. + +**NAME THE LAYER.** This is not a measurement of a running machine and not real +hardware. It is the shipping ROM image (IPL 1.0, md5 `7fd4caab…`, 131,072 B), +read statically — evidence about what Sharp's engineers configured this board +to do, from the vendor, for these exact devices. Field layouts are SOURCED from +MC68450, Motorola, Jul 1989, the document `buscost.py` already cites. + +### 52.1 What the ROM programs + +Boot, `$FF0CCA`–`$FF0C58`: a ten-pair table at `$FF0D8E` initialises channels 0 +and 1, then two inline runs do channels 2 and 3. + +| ch | device | DAR | DCR | decoded | +|---|---|---|---|---| +| 0 | FDC | `$E94003` | `$80` | dual address, 8-bit port, cycle steal **without hold** | +| 1 | **SASI** | `$E96001` | `$80` | dual address, 8-bit port, cycle steal without hold | +| 2 | IOCS `_DMAMOVE` | per call | `$08` | dual address, **16-bit** port, burst | +| 3 | **ADPCM** | `$E92003` | `$80` | dual address, 8-bit port, cycle steal without hold | + +Per transfer, `$FF9A82` (ADPCM play) and `$FF9944` (SASI): **OCR = `$32`** for +memory→device, `$B2` for device→memory. Both decode to `SIZE = 11` (byte), +`CHAIN = 00` (none) and — the load-bearing field — **`REQG = 10`, external +request: one operand per device request.** + +### 52.2 Audio is dual-address and cannot hold the bus. 16..19 clocks a byte. +`DTYP = 00` is *explicitly addressed*, so every ADPCM byte is a memory read +followed by a device write: 4 + 5 = **9 clocks** (Fig 4-25 sheet 4, note 2 — +already in `buscost.py` for the span work). `XRM = 10` is cycle steal without +hold and `REQG = 10` is external request, so **the DMAC arbitrates for the bus +once per byte and hands it straight back.** There is no burst to amortise the +front-end (5..8 clocks, §4.5.2.1) and back-end (2, §4.5.2.2) over. The audio +byte costs **16 clocks best case, 19 worst** — not 5. + +### 52.3 The byte rate, derived rather than restated +`ratectl.AUDIO_KBPS = 7.8` had no derivation next to it, and ROADMAP flagged it +as exactly the kind of figure that cost the project a 2x error in FINDINGS 43. +It survives, with a correction of units: 15.6 kHz is the MSM6258V's 8 MHz clock +÷512 = **15,625 samples/s**, 4 bits each, two to a byte = **7,812.5 B/s**. The +7.8 is that in *decimal* kB; `11_cpu_budget.py` was multiplying it by 1024, so +it read 2.4% high. Harmless, and now derived from the sample rate in +`buscost.ADPCM_BYTES_PER_S` instead of typed in. + +**The request count does not halve.** 7,812.5 B/s is 7,812.5 DMA requests/s, +because the port is 8 bits and the operand is a byte. That is the FINDINGS 43 +trap in the other stream, and it does not spring: nobody had denominated audio +per word. + +### 52.4 So: what the audio does to the bus. Almost nothing. +651.0 B/frame at 12 fps × 16..19 clocks = **10,417..12,370 clocks of 833,333 — +1.25% to 1.48% of the frame.** The decoder's measured mean is 68.5% of the frame +period on the gate container, so audio takes about **4% of what the decoder +leaves**, and about 1.7% of it on the worst frame (which is already at 110.8% +and misses with or without audio). + +**P6's bus risk does not materialise.** The concern was sound and the answer is +that a second DMA consumer at 7.8 kB/s is not what a bus at 88% occupancy is +short of. The debit was 3.2x..3.8x understated, and it is still small. + +### 52.5 THE ONE THAT MOVES SOMETHING: the disk channel is programmed the same +ch1, SASI, `DCR = $80`, `OCR = $B2` — **dual address, 8-bit port, cycle steal +without hold, external request.** Byte by byte, full arbitration each time. ch0 +(FDC) too. Sharp programs every explicitly-addressed 8-bit device on this board +identically, and by 52.2's arithmetic that is **16..19 clocks per delivered +byte**. + +FINDINGS 42.4–42.6 brackets W, the clocks stolen per delivered byte, at **5..12**, +and reports that **W ≤ 6 fits 0/120 frames while W = 8 misses 47/120**. The only +worked example of a disk DMA configuration on this machine sits **above the +entire bracket.** `15_bus_occupancy.py` now sweeps it on the gate container +(37,403 B/frame, measured mean decode 570,958 clocks): + +| W clk/B | video clk/frame | % of frame | CPU + audio + video | +|---:|---:|---:|---:| +| 5 | 187,017 | 22.4% | 92.2% | +| 8 | 299,228 | 35.9% | 105.7% | +| 12 | 448,842 | 53.9% | 123.6% | +| **16** | 598,455 | 71.8% | **141.6%** | +| **19** | 710,666 | 85.3% | **155.0%** | + +The W = 8 row agreeing with 42.5's "misses 47/120" is a cross-check, not a new +result — two models of the same machine, one built from mode histograms and one +from bus clocks, landing in the same place. + +**This does not close B3.** `scsiexrom.bin` drives an MB89352, not the SASI +port, and a different ROM may configure it differently. What changed is the +prior and the framing: 42.6 says the handshake "is ours to choose, not to +receive", and that is still true — but **nothing in this tree has shown a +cheaper configuration is reachable for an explicitly-addressed 8-bit port, and +the vendor's own answer is the expensive one.** W ≤ 12 is a *requirement on the +player's DMAC programming*, not a range the hardware hands us. It is now the +largest open number in the project, ahead of the rate. + +### 52.6 Audio outranks the disk at the arbiter +CPR: FDC 0, **ADPCM 1**, SASI 2, `_DMAMOVE` 3 — lower is higher priority. With +the ROM's arrangement, when both channels want the bus in the same slot ADPCM is +served first. An audio byte is never the thing that waits; **a video byte is.** +Relevant to 51's underrun analysis, which models delivery as a smooth rate. + +### 52.7 What this does NOT establish +1. **Static read of a ROM image, not a running machine.** No emulator executed + this code for the purpose; the claim is about bytes in the shipping image. + Real hardware would confirm the registers, not the timings. +2. **The timings are datasheet, not measured.** 9 clocks for the transfer and + 5..8 / 2 for the arbitration come from MC68450 Fig 4-25 and §4.5.2. They are + the same source `buscost.py` already rests on for the span work, and they + have never been checked against a board. +3. **IPL 1.0 is a pre-SCSI machine.** ch1 is SASI. B3 stands. +4. **Our player is not obliged to copy the ROM.** It programs these registers + itself. 52.5 is a prior and a warning, not a measured ceiling — and the + experiment that would settle it is P4, not another reading. +5. **Nothing here is an audio implementation.** P6's other risks — extraction, + encode, container interleave, the second stream's effect on `wire` and hence + on 51.3's refill climb — are untouched. Only the bus question is answered. +6. **The frame-period accounting assumes the DMAC does not overlap the CPU** + (`buscost.DMA_OVERLAPS = False`), which is FINDINGS 35's premise: no cache, + a two-word prefetch queue. A stolen bus cycle is a stopped 68000. diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md new file mode 100644 index 0000000..16c40a9 --- /dev/null +++ b/docs/ROADMAP.md @@ -0,0 +1,244 @@ +# Roadmap — remaining work to a completion target + +Written end of session 19 (2026-08-24), against a tree that is ALL GREEN. + +**THE COMPLETION TARGET IS M3, THE VERTICAL SLICE** (USER DECISION): one scene +tree — a decision point, two outcomes, a death clip — with audio, streaming from +a real SCSI volume on a stock 2 MB machine, playable. That is the point at which +every layer of this design has been shown to work at once. M4 is listed because +it is real work, but past M3 it is content grinding rather than open questions. + +`docs/STATUS.md` remains the session-by-session record and the handoff. This file +is the shape of what is left; where the two disagree about what is done, STATUS +is the one with the measurements and this one is the one that goes stale. Both +were wrong about two encoder gaps until this file was written — see "What was +already done" below. + +--- + +## Status of the four resources + +The project's own framing, restated because every item below is priced in one of +these units: + +| resource | state | +|---|---| +| **68000 local bus** | the binding one. Decoder occupies 86.7%; 52 of 53 missed frames miss on the bus, not the clock (FINDINGS 38). | +| **68000 clocks** | measured, and the rate controller binds on them. | +| **Delivery rate** | **no working figure, deliberately** (FINDINGS 50, USER DECISION). Every tool REQUIRES an explicit rate. | +| **Seek time** | **no figure at all, and never had one.** 51.3/51.4 made it matter. | +| **W, clocks stolen per delivered byte** | bracketed 5..12 (42.4); the IPL ROM's own disk channel is **16..19** (52.5). **The largest open number in the project.** | + +--- + +## What was already done, and was still on the list + +Found while inventorying for this file. Both had been closed in code for several +sessions and were still listed as open gaps in `docs/STATUS.md`: + +- **4-byte record padding.** `DLX2`, `encode.py:139-156`, inside rate-control + accounting, reported per frame and per second. +- **CPU cost in the mode decision.** `vq_hybrid.py:218`, priced against measured + per-mode cycles with the exact clustered SKIP rule. + +Both entries are now struck in STATUS. **The lesson is procedural: a gap list +that is only ever appended to manufactures phantom work.** Anything crossed off +below should be crossed off in STATUS in the same sitting. + +--- + +## Blocked on hardware this tree does not have + +None of these block M2 or M3 software work, because session 18 forced every rate +to be an explicit argument. They set constants, and two of them decide how much +headroom the finished player has. + +**B1. Measure the BlueSCSI — throughput AND seek time.** +Throughput has an acceptance test already derived from real record sizes: +**513.2 KB/s** for the session-14 candidate, **451.4 KB/s** for the gate +container (`19_ring_stream.py`, FINDINGS 49.5). Seek time has nothing. +51.3/51.4 is why the second half matters: slack is *accumulated* out of +`pipe - wire`, so what a branch point costs is set by the rate and the time since +the last branch, not by the ring size. At 460 KB/s every ring from 192 KB to +512 KB is rate-bound and never fills. **Do not substitute a guess** — run at +several explicit rates and report the sensitivity. That is exactly how the +retired pipe constant survived five sessions after 42.1 called it folklore. + +**B2. Does buffer mode blank the display?** `probe_bit11_blank.lua` is written +and settles it in minutes on a real board. FINDINGS 48 shifted the prior toward +MAME and toward "unusable" — **do not pre-build on 1.0 B/pixel**. Same sitting: +the priority register `0xE82500` at `0x0000` (47.3). + +**B3. Single-address vs dual-address DMA.** 242 KB/s and 0.69 dB. Needs +`scsiexrom.bin` (8 KB, CRC `7be488de`) sourced, then its DMAC init disassembled +for DCR's DTYP: `10`/`11` = single (5.0 clk/B), `00`/`01` = dual (9.0). +FINDINGS 48.4. Not on this machine (checked, session 18). +**This is also P4's input** — the handshake the player drives is the same +question from the software side. + +> **Session 20 moved the prior hard, and it moved the wrong way (FINDINGS 52.5).** +> The IPL ROM *is* on this machine, and `tools/analysis/21_iplrom_dmac.py` reads +> its HD63450 setup: the on-board disk channel (ch1, SASI) is `DCR = $80` — +> **dual address, 8-bit port, cycle steal WITHOUT hold**, with `REQG = 10` +> external request, i.e. a full bus arbitration per byte. That is **16..19 +> clocks per delivered byte**, above the whole 5..12 bracket 42.4 costs P4 in. +> Same vendor, same DMAC, same class of 8-bit port — but it is *not* +> `scsiexrom.bin`, so B3 stays open. What it changes is that a cheap +> configuration is now the thing that has to be **shown**, not assumed. + +--- + +## M2 — a player, as opposed to a decoder + +`decode.s` draws pixel-exact frames from RAM Lua pre-loaded; `stream.s` decodes +out of a bounded ring fed by a host file on a paced clock. Neither is a player. + +**Exit criterion: boots from a real SCSI volume on a stock 2 MB X68000, plays +one scene at 12 fps from disc, no host-file pipe, no Lua in the loop. Silent.** + +**P1. Codebook expansion on the 68000.** `dlxload.py:19` expands CB1 to 32 B per +entry and CB4 to 8 B, host-side, because at the time it was a load-time cost that +would have flattered or damned the inner loop. The player must do it: **8 KB + +2 KB per scene**. Note where that lands — *at a scene change, when the ring is +empty because of the seek*. It compounds with 51.3 and should be priced against +the refill climb, not treated as free setup. + +**P2. Palette packing on the 68000.** The encoder still emits RGB888; the X68000 +word packing is Lua-side. Whatever writes real palette words must pick `I` per +entry by minimum squared error (**1.96 dB**, FINDINGS 23.3) and reserve index 0 +as black with `I = 0` (23.4). + +**P3. A real frame clock.** `stream.s` has `PACE`/`PACEON` (`$18034`/`$18038`) +but the 12 fps tick comes from the Lua producer. Needs MFP timer or VBL. Keep +`PACEON=0` free-run working — the wrap gate uses it and every FINDINGS 49 figure +depends on it. + +**P4. Real transport.** Drive the MB89352 instead of a host file. The `W` +handshake — clocks stolen per delivered byte, bracketed 5..12 by MC68450 Fig +4-25 — is listed in "Decisions locked" as UNDECIDED and as the thing that +decides the project: `W<=6` fits 0/120 frames, `W=8` misses 47/120. It is a +property of how the player drives the SPC, **so it is ours to choose, not to +receive** (FINDINGS 42.4-42.6). B3 informs it. + +**Session 20 promoted this to the project's biggest open number.** FINDINGS 52.5 +found the only worked example of a disk DMA configuration on this machine — the +IPL ROM's own — sitting at 16..19 clk/B, outside the bracket entirely, where the +whole design fails at any container size (`15_bus_occupancy.py` sweeps it). +`W <= 12` is now a **requirement on the player's DMAC programming**, not a range +the hardware hands us, and demonstrating a configuration that meets it is P4's +first job rather than its last. + +**P5. Seek and branch.** Per-record index (the `aligned` producer needs one +anyway, 49.3), prefill policy, and the accumulated-slack rule from 51.3 made +explicit in the player rather than implied by the rig. + +**P7. Boot.** The player as an executable loading from the SCSI volume. + +--- + +## M3 — the vertical slice, and the completion target + +**Exit criterion: one decision point, two outcomes, a death clip, with audio, +playing from disc on stock hardware.** + +**P6. Audio — and it is the largest unpriced risk left in the project.** +MSM6258 ADPCM, 15.6 kHz mono, **7.8 KB/s**. That figure is in `ratectl.py`'s +budget and nowhere else: not extracted, not encoded, not interleaved into the +container, and **never priced on the bus**. Two reasons to treat it as a risk +rather than a task: + +1. A second DMA consumer attacks **the bus** — the resource this project already + established is the binding one, at 86.7% occupied. Clock headroom says + nothing about whether it fits. +2. 7.8 KB/s is a *byte* figure. The last time a byte/word unit error went + unexamined in a delivery budget it cost the project a 2x error in every table + since FINDINGS 5 (session 14, the MB89352 being an 8-bit SPC). + +~~**Price it before writing it**: add the ADPCM DMA stream to `15_bus.py` and see +what it does to the 86.7%.~~ **DONE, session 20 — FINDINGS 52.** It is in +`15_bus_occupancy.py` and the answer is **1.25%..1.48% of the frame**, about 4% +of what the decoder leaves. The per-byte cost is no longer a guess borrowed from +the disk: `tools/analysis/21_iplrom_dmac.py` reads the IPL ROM's own HD63450 +configuration and finds ch3 dual-address, 8-bit port, cycle steal without hold, +external request — **16..19 clocks per byte**, where `11_cpu_budget.py` had been +charging audio the disk's 5. Both worries above resolve: + +1. **The bus concern does not materialise.** A second DMA consumer at 7.8 kB/s + is not what a bus at 88% occupancy is short of. +2. **The unit was checked and is nearly right.** 15.6 kHz = 8 MHz ÷ 512 = + 15,625 samples/s, 4 bits each, two to a byte = **7,812.5 B/s exactly**. The + 7.8 was decimal kB being multiplied by 1024; 2.4% high, now derived from the + sample rate in `buscost.ADPCM_BYTES_PER_S`. + +**What is still open in P6 is everything except the bus:** extraction, encode, +container interleave, and what a second stream does to `wire` — and therefore to +`pipe - wire`, and therefore to 51.3's refill climb. That last one is the +interaction to price next, and it is E2's question with a second consumer in it. + +**E6. Container v2** — audio interleave, per-record index, scene table. Depends +on P6's answer and on P5's index. + +**G1. Import the scene graph — early, because it is a measurement input.** +SNES project `data/events/` (MIT, cleared) diffed against DirkSimple (zlib), +which transcribed the same data independently, to catch transcription errors +before anything reaches 68000 tables. **Neither is on this box** — both need +fetching. + +The reason to pull this ahead of the game logic that consumes it: 51.3 says +4.83 s of play to refill a 256 KB ring at 488 KB/s, and Dragon's Lair's decision +points are seconds apart. **Nothing in this tree can currently say what the worst +gap between consecutive decision points is** — only the scene table knows, and +until it is imported, whether this design survives a back-to-back branch is an +open question nobody is able to ask. + +--- + +## M4 — the whole game + +Listed for completeness; past M3 these are scope, not risk. + +- **C1. Full-disc survey**, 22.8 minutes. Classify **content / menu / bonus** — + not menu vs content: the two largest streams are bonus material and look like + content by size, duration and bitrate alike (25.1). Run + `07_motion_survey.py` per stream first for a hot-window shortlist. + **Gated by E4.** +- **E4. `H.build` k-means**, 51 s of a 55 s run, once per scene. The thing to + attack before C1, and not anything in the per-frame path (27.6). +- **E2. `--spans all` as default.** Still a recommendation, not a measurement + (43.6.1), and the only loaded lever on the encoder's byte side (44.3). **It + spends every profitable byte, which raises `wire`, which shrinks `pipe - wire`, + which lengthens the refill climb after every branch.** That interaction is not + priced, and M3 is where it becomes measurable. +- **E3. Re-derive span selection jointly with `lam`** (39.3). +- **C2. Framing** — crop vs squash vs wide (FINDINGS 12). Needs an eyeball + against arcade reference, not a measurement. Cheap; blocks only final encodes. +- **C3. Disk image packaging**, ~1.09 GiB at the candidate rate. +- **G2/G3.** Branching, input windows, death clips, attract mode; playtest. + +--- + +## Dependency summary + +``` +B1 seek+rate ─┐ +B3 DTYP ──────┴─> P4 transport ─┐ + ├─> M2 ─> M3 (COMPLETION TARGET) ─> M4 +P1 P2 P3 P5 P7 ─────────────────┘ ^ + │ +P6 (bus cost DONE, 52) ──────────────────┤ +G1 scene graph (fetch, do early) ─────────┘ +B2 blanking ─> (page 1; do not pre-build on it) +``` + +## Standing rules that apply to all of it + +- **Green light first and last.** `./tools/bench/check.sh`, ALL GREEN, before and + after. **Never two MAME jobs at once** — session 18 did it, two `decode.lua` + runs shared a log file, and it produced a 0-byte log and 15 wasted minutes. +- **Name the layer.** Emulated, or real hardware. Every progress claim. +- **Label measured / estimated / folklore.** A rate with no provenance is + folklore even when it is plausible, and this project has already paid for that + twice. +- **No new default constants.** Rates stay explicit arguments. If a measurement + is not available, report the sensitivity across several rates rather than + picking one. diff --git a/docs/STATUS.md b/docs/STATUS.md index b48c784..12890cf 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -1,6 +1,825 @@ -# Status & next-session handoff — end of session 12 (2026-08-23) +# Status & next-session handoff — end of session 20 (2026-08-24) -## Where this stands +## Session 20: the DMAC configuration was in the IPL ROM the whole time + +**Green light first and last: `./tools/bench/check.sh` was ALL GREEN before any +of this and ALL GREEN after**, 120/120 on both cores, no `TRUNCATED`, plus a new +ROM-evidence stage. + +**ROADMAP's "do this first" item is DONE. FINDINGS 52.** It asked for the cheap +thing — put the ADPCM DMA stream on the bus and see what it does to the 86.7% +— and doing it needed a clocks-per-byte figure for the audio channel that the +tree did not have. `11_cpu_budget.py` was charging audio bytes the *disk's* +rate, 5 clk/B, described in its own help text as "single-address, bus held". +**Audio was being charged the favourable end of ROADMAP B3, an open question +worth 242 KB/s.** + +**1. It never had to be a guess.** The X68000 IPL ROM programs all four HD63450 +channels itself, and **the ROM is on this machine** — MAME boots the rig with +`-bios ipl10`. `tools/analysis/21_iplrom_dmac.py` reads the configuration out of +the image and decodes the MC68450 fields. Eight (address, expected bytes, +meaning) sites; a mismatch or an unknown ROM revision exits non-zero. Needs no +emulator, runs in milliseconds, and is now in `check.sh`. **Layer: a static read +of a shipping ROM image, not a running machine and not real hardware.** + +**2. Audio is dual-address and cannot hold the bus: 16..19 clk/B.** ch3 +`DCR = $80`, `OCR = $32`: explicitly addressed (9 clocks, read 4 + write 5), +cycle steal **without hold**, `REQG = 10` external request — the DMAC +arbitrates once per byte with no burst to amortise the 5..8 + 2 arbitration +over. The old debit was 3.2x..3.8x too small. 52.2. + +**3. And on the bus it is still nothing.** 651.0 B/frame x 16..19 = +**1.25%..1.48% of the frame**, about 4% of what the decoder leaves. **P6's bus +risk does not materialise.** The unit worry was worth checking and nearly +right: 15.6 kHz = 8 MHz / 512 = 15,625 samples/s, two 4-bit samples to a byte = +**7,812.5 B/s exactly**; `AUDIO_KBPS = 7.8` is that in decimal kB and was being +multiplied by 1024. 2.4% high, now derived from the sample rate. 52.3/52.4. + +**4. THE ONE THAT MOVES SOMETHING: the disk channel is programmed identically.** +ch1 (SASI) is `DCR = $80` too, and so is ch0 (FDC) — Sharp programs every +explicitly-addressed 8-bit device on this board the same way. By the same +arithmetic that is **16..19 clocks per delivered byte**, where FINDINGS 42.4 +brackets W at **5..12** and 42.5 reports `W = 8` already missing 47/120 frames. +**The only worked example of a disk DMA configuration on this machine sits above +the entire bracket**, and at that price the design does not fit at any container +size. It is not `scsiexrom.bin`, so **B3 stays open** — what changed is that a +cheap configuration is now the thing that has to be *shown*. `W <= 12` is a +requirement on the player's DMAC programming, not a range the hardware hands us. +**This is now the largest open number in the project, ahead of the rate.** 52.5. + +**5. An unforced cross-check fell out.** `15_bus_occupancy.py` now sweeps W over +the whole frame period. Its `W = 8` row lands at 105.7% of the frame — agreeing +with 42.5's "misses 47/120", from mode histograms and bus clocks respectively, +two models that share no code. 52.5. + +**6. Audio outranks the disk at the arbiter.** CPR: FDC 0, ADPCM 1, SASI 2, +`_DMAMOVE` 3. When both want the bus, ADPCM is served first — an audio byte +never waits, a video byte does. Relevant to 51's smooth-rate delivery model. +52.6. + +**New in the tree:** `tools/analysis/21_iplrom_dmac.py` (the ROM decoder/gate); +`15_bus_occupancy.py` gains a "THE OTHER TWO MASTERS" section pricing both DMA +consumers against the frame period; `buscost.py` gains the ADPCM constants and +the split read/write DMAC timings; `11_cpu_budget.py` charges audio and video +separately, with `--adpcm-clocks-per-byte`. Its stale "validated to within 1 pt" +line is corrected — the model reads 2..10 pt HIGH and by more as the frame gets +harder, which was already true at HEAD. + +**Also: the README now shows the thing working.** `tools/bench/stream.lua` +gains `DLX_SNAP_EVERY=1` (needs `DLX_PACE`; off by default and on no path +`check.sh` takes), which snapshots every frame tick instead of once at the end, +and `tools/media/make_readme_media.py` turns those PNGs into `docs/img/`. The +stills and both clips are **MAME's own screen pixels**, not a re-render. + +Building it turned up something worth recording. 116 of 119 captured frames are +pixel-exact against `tools/encoder/dlx.py`; **three are torn** — frame *n* on +top, frame *n-1* below the tear line — because MAME captured the screen while +the block loop was partway down it. `decode.s` writes straight to the displayed +page (one display path, 28.1), so **a real player tears the same way**; this is +the first time that consequence has been visible rather than argued. The script +ASSERTS the tear (every differing pixel must come from the previous frame) and +refuses to build otherwise, rather than trimming the frames and quietly +reporting "every frame I kept is exact". + +Second correction the capture forced: the snapshot fires at the tick, *before* +frame *n* is decoded, so the obvious reading is that snapshot *n* holds frame +*n-1*. It does not — MAME renders the screen at the end of the machine frame, by +which time the 68000 has finished frame *n*. Checked, not reasoned about. + +**`decode.s` and `stream.s` are unchanged.** Nothing in `src/player/` was +touched; `decode.bin` is still 1,296 B at the same MD5. + +**Next:** P4 is the item that decides the project now, and 52.5 says so with a +number. Everything else in M2 (P1, P2, P3, P5, P7) is still buildable here. + +--- + +# Status & next-session handoff — end of session 19 (2026-08-24) + +## Session 19: the ring rig gets a frame clock, and a branch point costs 4.83 s of play + +**Green light first and last: `./tools/bench/check.sh` was ALL GREEN before any +of this and ALL GREEN after**, 120/120 on both cores, no `TRUNCATED`, plus a new +paced-ring stage. + +**Item 4 is DONE. FINDINGS 51.** Items 1, 2 and 3 were all still blocked on +hardware this tree does not have — a real board for 1 and 2, `scsiexrom.bin` +for 3 — and none of them moved. Item 4 was the one that could be built here. + +`src/player/stream.s` had no frame clock, so it asked for record *i* the instant +it finished *i-1*, outran any finite pipe, and never let the ring back up. That +is why the 49.1 ring sweep passed at 48 KB. It now has `PACE`/`PACEON` +(`$18034`/`$18038`) and the producer supplies a 12 fps tick, so +`FR_HEAD-FR_TAIL` finally means what it is read to mean: **whole frames the +decoder could still draw with delivery stopped dead.** `PACEON=0` free-runs and +is what the wrap gate still uses, so every figure in FINDINGS 49 is unmoved. + +**1. Small rings carry almost nothing.** Paced, on the gate container: 64 KB +holds **2 frames**, 96 KB holds 3, 128 KB holds 4–5, 256 KB holds 7–8, 512 KB +holds 14–15. All pixel-exact, which is exactly why the unpaced sweep passed +them. 51.1. + +**2. Tolerance is `ceiling - 1`, measured by cutting the pipe.** With 7 records +resident, a 6-frame-time cut underruns nothing and a 7-frame-time cut underruns. +The last record pays for the pipe's restart (~0.9 frame times to place 36.5 KB). +**256 KB buys 500 ms of dead pipe, not 583.** 51.2. + +**3. THE ONE THAT MOVES SOMETHING: slack is accumulated, not owned.** It is +built out of `pipe - wire` and a seek spends all of it. At 488 KB/s a 256 KB ring +takes **4.83 s of play** to reach its 7-frame ceiling from empty; 512 KB takes +8.42 s to reach 14. **A bigger ring raises the ceiling AND lengthens the climb.** +So a branch point does not ask "is the buffer big enough", it asks "has there +been enough play since the last one" — and Dragon's Lair's decision points are +seconds apart. First statement in this tree about back-to-back branches. 51.3. + +**4. The rig now says which resource is binding.** Rate refusals and ring +refusals are counted separately: at 460 KB/s every ring from 192 KB to 512 KB is +**RATE-BOUND**, ceiling 4, and never fills in 120 frames — larger rings are dead +RAM in that scene. Clearing the arrival deadline (451.4 KB/s, 49.5) and being +able to absorb a seek are **different requirements**, and the gap is large. 51.4. + +**5. Independent agreement, honestly bracketed.** `tools/analysis/20_seek_slack.py` +is the model rewritten in Python from record sizes, sharing no code with the Lua +producer. **35/35 of the rig's ceilings fall inside its bracket, 33/35 at the +top.** The bracket is one record wide because the pipe delivers ~one record per +slot, so the answer depends on sampling before or after that slot's delivery. +Both are reported rather than picking the one that matched — that would have +been fitting the model and calling it a cross-check. 51.5. + +**6. Pacing exposed two producer defects, both invisible free-running.** The +`RD_PTR` cross-check was really testing how often `reap()` ran (it asserted per +retired record; `RD_PTR` names only `tail-1`), and `reap()` was skipped for the +whole duration of a cut, so the ring looked full through a seek. 51.6. + +**New in the tree:** `tools/bench/pace_run.sh` (one paced run, pixel-verified), +`tools/bench/pace_sweep.sh` (ring x pipe grid), `tools/analysis/20_seek_slack.py` +(the independent sim). `check.sh` gains a paced stage that gates pixel-exactness, +zero underruns, and the 256 KB ceiling — the last one because a change in it is +a change in what a branch point can afford, and that should not slip past as a +log line. + +**`decode.s` is unchanged and still asserted:** 1,296 bytes, same MD5. The pace +gate is in `stream.s` only (1,396 -> 1,418 B) and outside `src/player/frame.i`, +so no per-block or span constant moves. + +--- + +# Session 18 and earlier + +## Session 18: the streaming path exists, and the shipping rate does not fit the pipe + +> **THE DELIVERY RATE HAS NO WORKING FIGURE — retired session 18 (USER +> DECISION).** Sections below written before session 18 name a "4 Mbps" pipe +> constant and score tables against it. **Read every one of those as history.** +> It was never a bus measurement: user-supplied, no provenance, 10% of SCSI-1's +> asynchronous rating (FINDINGS 42.1), and FINDINGS 49.5 caught the shipping +> candidate exceeding it while nothing in the tree was comparing the two. +> +> It is now gone as a default from every analysis tool and from +> `tools/bench/stream.lua` — `--bus` / `--kbps` / `DLX_STREAM_KBPS` are +> **required arguments** with no fallback, so no table can be scored against a +> rate its own output does not state. The one survivor is `GATE_SPAN_KBPS` in +> `tools/bench/check.sh`, which is a **container recipe**, not a delivery claim: +> the gate container was encoded with it and every per-block and span constant +> in FINDINGS 41/43/45/49 is fitted to that container, so changing it is a +> re-encode plus a re-measurement, not an edit. +> +> **What to use instead:** `tools/analysis/19_ring_stream.py` reports the +> **zero-prefill pipe** — the rate a medium must clear for a container to need no +> prefill. That is a requirement to measure a BlueSCSI against, not a constant to +> design on. For the session-14 candidate it is **513.2 KB/s**. + + +**Green light re-run first: `./tools/bench/check.sh` was ALL GREEN** before any +of this, 120/120 on both cores, no `TRUNCATED`. + +**Items 1 and 2 were both blocked in this tree, and checked rather than +assumed.** Item 1 needs a real board. Item 2 needs the CZ-6BS1's +`scsiexrom.bin`: `~/mame/roms/x68000.zip` holds six files — `cgrom`, four IPLs, +`sram` — and there is no SCSI ext ROM anywhere on this machine. MAME's +`hd63450.cpp` decodes no DTYP, so the emulator cannot stand in (48.4). **USER +DECISION: build item 3.** + +**Item 3 is DONE, and item 4 folded into it. FINDINGS 49.** +`src/player/stream.s` + `tools/bench/prep_stream.py` + `tools/bench/stream.lua` +decode the gate container **out of a bounded ring**, container in a host file: + +| ring | machine | result | +|---|---|---| +| **256 KB** | **stock 2 MB** | **120/120, final frame pixel-exact** | +| 128 / 96 / 80 / 64 / 48 KB | stock 2 MB | 120/120, pixel-exact | + +**The rig's RAM ceiling is gone.** 45 raised `RIG_RAM` to 6 MB because 5,261,814 B +of stream did not fit 2 MB. The streaming rig holds ~256 KB and reads the rest +from the host, so the machine it runs on is now **the machine the player +targets**. + +**1. The constraint is CONTIGUITY, not byte count, and 09_buffer_sim could not +see it.** The block loop and span chain read with a monotonically increasing +`a0` and no bounds check anywhere, so the ring needs the whole next record +**resident AND contiguous**. 49.2. + +**2. `aligned` beats `split` and it is not close.** Producer refuses to start a +record it cannot finish, leaves a hole: **9.1% of a 256 KB ring, zero clocks**. +Letting records wrap and shadowing the ring head costs **5.57% of the frame +budget, forever** — and the decoder is already at 91.1% at p90. (Both are +`s14_d5_all1500`'s; the gate container makes it 5.7% of the ring against 3.64% +of the budget. The costs are per container, not universal.) `aligned` also +needs a per-record index, which a branching laserdisc game needs anyway. 49.3. + +**3. Two independent implementations agree exactly.** The Python sim (from record +sizes) and the Lua producer (driving a real 68000) both give **18 wraps, 14.7 KB +mean hole, 94.3% usable**. They share no code. 49.4. + +**4. THE ONE THAT MOVES SOMETHING: the shipping candidate does not fit the pipe.** +`s14_d5_all1500` is **496.7 KB/s**; the pipe this tree has simulated against +since session 2 is **488 KB/s**. Those two numbers had never been put side by +side. It is **8.7 KB/s over on the MEAN — not a burst a ring absorbs.** The +deficit grows **523 KB per minute of play**; no ring size fixes it. + +It was never caught because 42.1 retired the pipe as the binding resource and +built the rate controller to bind on clocks, with **no pipe term at all** — a +defensible decision. What was not decided is that FINDINGS 21's buffer sizing +and its "zero required prefill" would keep standing on a constant the design had +stopped enforcing. **That is what item 4 has been open since session 7 for.** + +**The output is a requirement on the medium, not a verdict**, because 488 is +unmeasured folklore: + +| container | wire | **zero-prefill pipe** | +|---|---:|---:| +| `s14_d5_all1500` (the candidate) | 496.7 KB/s | **513.2 KB/s** | +| `rc_fr_singe_scsi_span` (the gate) | 446.1 KB/s | **451.4 KB/s** | + +**513.2 KB/s is now a hardware acceptance test** — 33% of SCSI-1 async, 10% of +sync. Very likely met; never shown to be met. 49.5. + +**5. The rig's first version measured the wrong thing, and it is worth knowing +why.** `stream.s` has no frame clock, so it outruns any finite pipe and reported +**91 of 120 frames "stalled" at a pipe the same run shows is fast enough**. It +now records when each record becomes RESIDENT and checks that against a 12 fps +deadline: **1/120 late by 4.9 ms at 488, 0/120 at 520**. 49.6. + +**`decode.s` is unchanged, and provably.** The block loop and span chain moved to +`src/player/frame.i` and the constants to `geom.i` so both front-ends assemble +from literally the same bytes — every cycle constant in FINDINGS 24/30/40/41 is +fitted to those bytes. `decode.s` still assembles to **1,296 bytes**, same MD5, +and `prep_dlx.py` still emits a byte-identical blob after the loader maths moved +to `tools/bench/dlxload.py`. **Both are now asserted in `check.sh`**, along with +the ring pass itself. + +--- + +# Session 17 and earlier + +## Session 17: the blanking question is not a tie, and it leans the wrong way + +**Green light re-run first: `./tools/bench/check.sh` is ALL GREEN**, 120/120 on +both cores, no `TRUNCATED`. Nothing in session 16's uncommitted tree has drifted. + +**FINDINGS 48. Session 16 filed 47.4 as "two emulators disagree, both readings +plausible." That framing was too generous to the outcome we want.** + +1. **px68k is silent, not dissenting.** Read with `grep -a` — `gvram.c` is + EUC-JP, so a plain `grep` silently reports *nothing* — R20's high byte + (`CRTC_Regs[0x28]&8`) appears in **one file and six places, all address + decode**: three in `GVRAM_Read`, three in `GVRAM_Write`. **No px68k display + code reads the bit anywhere.** It does not model buffer mode as + non-blanking; it does not model the display side of buffer mode at all. + MAME's blanking is a deliberate commented claim made twice, and it models + bit 12 the same way for the text layer. **An assertion and a silence are not + a tie.** + +2. **Sharp's own register table names the bit MAME's way.** R20 bit 11 G-MEM: + `%0 表示用 / %1 バッファ用` — "for display" / "for buffer" — and + `bit 10〜8 は無効`, i.e. the colour-mode field goes *invalid* in buffer mode. + COL is what the display side decodes a plane structure from. That is a + mechanism for blanking, not just a name for it. The counter-reading survives + only on the parenthetical (`G-VRAM が 65536 色表示時と同じ構造になる`), which + describes the structure and does not say the screen goes dark. + +3. **The MAME branch has no survivable partial-blank.** The blank interval is + the paint, not the frame — but the measured blit is 53.6% of budget, packed + halves the words, so the picture is dark for ~27–54% of every frame at 12fps. + That is a 12 Hz strobe over the whole image, and the packed layout has **no + page left to flip to**, because both 256-colour pages carry picture. There is + no version where the packing is merely expensive. + +**This is a shifted prior, not a result. It still needs the board.** But it +should be read before anyone spends a session building on 1.0 B/pixel. + +**Item 2 lost its cheap method, and the loss is informative.** px68k **does not +emulate the MB89352 at all** — `x68k/scsi.c` is 81 lines that synthesise a fake +64-byte CZ-6BS1 boot ROM and trap IOCS `$F5` on the host (`SPCはエミュレートしない`, +its own header). So the "second emulator agrees" method that carried 46/47 was +never available for single- vs dual-address. What item 2 actually wants is the +**CZ-6BS1's `scsiexrom.bin`** (8 KB, CRC `7be488de`, not present here) +disassembled for the DCR it writes. **The MC68450 datasheet pins the field: +DTYP `00`/`01` are "Explicitly Addressed" (dual), `10`/`11` are "Implicitly +Addressed" (single), and 3.6.1.2 names the two protocols "dual address" and +"single address" in so many words.** 43.2's 5.0-vs-9.0 clocks/byte is two bits +in one byte the boot ROM writes at init. Not another emulator — and MAME's +`hd63450.cpp` decodes no DTYP at all, so it could never have answered. +FINDINGS 48.4. + +**One thing was confirmed rather than doubted.** px68k's `kaiseki.txt` — the +author's own 2014 analysis notes, predating all of this — states the 256-colour +word interleave (`Page0の(0,0), Page1の(0,0), Page0の(1,0)...`) independently. +The packed layout's *premise* is solid from a third source; only its visibility +is in question. FINDINGS 48.5. + +--- + +## Where session 16 left it + +**The strongest test in the tree now covers the whole window, on both cores.** +FINDINGS 45. The pixel-exact gate was auditing 37 of 120 frames; it is now +**120/120 on MAME's 68000 and on px68k's C68K**, and `./tools/bench/check.sh` +is **ALL GREEN** with no `TRUNCATED` line. + +Session 15 handed this over as "it needs the chunk-streaming rig, not a longer +pass." **That was wrong, and cheaply so — it needed neither.** The constraint +was the *rig's* memory, not the player's: `prep_dlx.py` preloads the whole +container into emulated RAM and `check.sh` ran the machine at `-ramsize 2M`, so +a 5,261,814 B stream was truncated to the prefix that fit. The gate runs under +`DLX_VERIFY_ONLY=1`, which drops the cost anchors entirely and asserts only +pixel-exactness, so the 2 MB was never load-bearing *here* — and preloading a +whole container is unlike the shipping player at any size, because the player +streams into a ring buffer. `RIG_RAM=6` in `check.sh` covers all 120 frames. + +**The raise is licensed by measurement, not by convenience.** Run the full +timing pass at 2M and at 6M and the five synthetic anchors come out +**bit-identical** — 40,729 / 921,187 / 1,376,881 / 1,229,883 / 506,533 cycles — +despite sitting at different addresses in the two layouts. MAME's cycle model +does not depend on `-ramsize` over this range, so every per-block constant in +FINDINGS 24/30/41 is unmoved. + +**And the 37-frame prefix was a biased sample.** It overstated the mean cost of +the window by 8.2%, and it never saw the quiet end: + +| anchor | 37-frame prefix | full 120 | +|---|---:|---:| +| min non-SKIP | 25.1% of blocks, 61.9% of budget | **15.2%, 53.6%** | +| median | 45.4%, 79.7% | **41.1%, 81.1%** | +| p90 | 52.3%, 93.1% | **48.5%, 91.1%** | +| max non-SKIP | 62.5%, 91.8% | 62.5%, 91.8% (same frame) | +| C68K sequential-pass mean | 693,886 cyc, **83.3%** | **641,444 cyc, 77.0%** | + +The direction is lucky rather than designed: the prefix was **pessimistic**, so +nothing downstream was flattered and no headroom claim was resting on the +missing frames. + +**Item 2 is answered, and the answer is no.** FINDINGS 46. There is no packed +256-colour write path — no graphics mode on this machine puts two horizontally +adjacent pixels in one 16-bit word. The sub-word fields are **pages, not +pixels**: page 0 and page 1 are the two bytes of one word *at the same screen +coordinate* (px68k derives `line` identically for both, and the `$C80000` alias +lands on the adjacent byte), exactly as two independent documents describe. The +near-miss — scroll page 1 by one pixel and interleave — fails on byte count, not +on addressing, and 46.2 records why so it is not re-derived. **FINDINGS 44.7 +stands unchanged**: the decoder-free player still needs 1,152 KB/s and 1.61 GB +and is still killed by the medium. + +**Chasing it opened one more surface, and the same session closed it.** The 2:1 +tax is a property of the *graphics* planes; the **text plane is 4bpp planar** — +four planes at `0x20000` stride, 1024x1024 bits each — so 0.5 bytes/pixel against +2.0, and an **uncompressed** 16-colour frame is 288.0 KB/s against the shipping +compressed 256-colour stream's 496.7. 42% cheaper on the wire, no decoder. + +**It costs 5.84 dB and that kills it.** `tools/analysis/18_text_plane_16col.py`, +120 frames, generous to the 16-colour side (per-frame palettes, which the +256-colour path cannot use because its codebooks index a scene-wide palette): + +| | mean PSNR | +|---|---:| +| 256 colours, scene palette (the tree's) | 31.33 | +| 256 colours, per-frame palette | 34.08 | +| 16 colours, scene palette | 23.17 | +| **16 colours, per-frame palette** | **25.49** | + +Against the shipping container's **29.19 dB at 496.7 KB/s**, a 16-colour literal +is **25.49 dB at 288.0 KB/s — 3.70 dB worse for 58% of the bitrate.** The wire +saving does not pay for the colours. **The user's call was to drop the 16-colour +direction outright and the number agrees**, so it is closed, not parked. +FINDINGS 7's 256-colour claim now rests on a measurement. (One caveat recorded +in 46.3: a Floyd-Steinberg row came out bit-identical to the undithered one — +PIL ignored `dither=` under `MEDIANCUT` — so that row is void and excluded.) + +**Then the user said to keep chasing it, and the "no" turned out to be wrong.** +FINDINGS 46.5. The masking is defeatable: **CRTC R20 bit 11** ("G-VRAM set to +buffer") bypasses the depth switch and writes the **full 16 bits** unmasked. +MAME's `gvram_w` and px68k's `GVRAM_Write` both implement it — px68k's comment +names the shipping game that used it, `65536モードのVRAMアクセス（Nemesis用）` — +so it is a mechanism, not an emulator quirk. **The 2:1 tax is a property of the +default write path, not of the memory.** + +**And the two 256-colour pages have independent scroll**, which 46.1 also missed: +px68k's `Grp_DrawLine8(int page, int opaq)` indexes `GrphScrollX[page*8]`, +selects the byte within the word by page, and takes an opacity flag. 46.2 +dismissed interleaving assuming a 1-pixel scroll; **scrolling by 128 makes the +used words contiguous**: + +- write words 0..127 of each row, unmasked, full 16 bits +- page 0 unscrolled → screen columns 0..127 +- page 1 X-scrolled +128, opaque, above → screen columns 128..255 +- page 1's storage past 128 lands off the edge of the real 256x256 mode already + in use (FINDINGS 23) + +**128 contiguous words carry 256 pixels: 1.0 B/pixel against 2.0** — 576 KB/s and +0.81 GB, exactly the halving 44.7 said would reopen the whole design. `movem`-shaped, +no stride, no transparency mask to maintain. + +**This is a derivation, not a result** — see 46.6 for the three things untested. +But unlike items that need a board, **the tree can answer this one**: it is a +register setup and a snapshot, and `verify_frame256.py` already compares +pixel-exactly. + +**The packed layout was then built and run on both emulators. It works.** +FINDINGS 47. 46.6 was a derivation; it is now a result: + +| | result | palette ceiling | +|---|---|---:| +| MAME, `verify_frame256.py` | **256x192 pixel-exact, letterbox true black** | 40.83 dB | +| px68k, `verify_gvpack.py` | **256x192 index-exact** | 40.83 dB | + +**Per-frame payload: 128 words/row x 192 rows = 49,152 bytes for 49,152 pixels — +1.0 B/pixel against 2.0.** The write path was measured directly: masked, writing +`AB5C` leaves page0=`5C` page1=`00` (the high byte destroyed); with R20 bit 11 +set it leaves page0=`5C` page1=`AB`. That is the 2:1 tax and its off switch in +one table. + +Four negative controls behave — bit 11 off, scroll removed, and the unpacked +control. `tools/bench/gvpack` links px68k's real `x68k/gvram.c`, the way +`tools/bench/c68k` links its CPU core. + +**But the two emulators disagree twice, and the second one decides everything.** + +1. *Priority register `0xE82500`.* At `0x0000` MAME hides page 1 (right half + black) while px68k puts page 0 on top transparently and renders correctly. + They agree at `0x0002`, which is what the layout uses — so the result stands, + but on a register they model differently. +2. **Does buffer mode BLANK the display?** `probe_bit11_blank.lua` is the + known-good 256-colour test with *one line added*. **MAME: the screen goes + fully black.** **px68k: it does not blank** — `Grp_DrawLine8` never reads that + bit. If MAME is right, the graphics layer is blanked for the whole time the + CPU or DMAC is painting, and a 12fps player shows black for whatever fraction + of each frame the paint takes. If px68k is right, the packing is free. + +**That is now the cheapest high-value hardware fact outstanding** — cheaper than +single-vs-dual-address, because one real board plus a two-line probe settles it +and it moves more numbers. + +**What it is worth if it goes px68k's way (DERIVED, not measured):** wire 1,152 → +**576 KB/s**, game 1.61 → **0.81 GB**, DMAC device→GVRAM 59.0% → **29.5%** of the +clock budget. And one reversal: 44.7 concluded "a CPU-painted full-frame literal +does not fit at any physically reachable price." Packed, one `movem` word carries +two pixels, so a pixel costs `(9.143 + 2c)/2 = 4.571 + c` — **9.57 clocks at c=5, +against a 16.95 budget. It fits, at c=5 and at c=9.** Withdrawn conditionally on +the blanking question. + +Reproduce: +``` +python3 tools/bench/prep_frame.py tmp/fr_00020 tmp/frame256p.bin 0 --pack-transparent +( cd tmp && SDL_VIDEODRIVER=dummy mame x68000 -bios ipl10 -ramsize 2M -video soft \ + -window -sound none -nothrottle -plugins -seconds_to_run 8 \ + -autoboot_script ../tools/bench/show_frame256_packed.lua \ + -snapshot_directory ./snap_pack -snapview native ) +python3 tools/bench/verify_frame256.py tmp/snap_pack/x68000/0000.png tmp/frame256p.bin +make -s -C tools/bench/gvpack && tools/bench/gvpack/gvpack tmp/frame256p.bin tmp/gvpack_px68k.raw +python3 tools/bench/gvpack/verify_gvpack.py tmp/gvpack_px68k.raw tmp/frame256p.bin +``` +Not wired into `check.sh` yet: it is a capability probe, not something the +shipping player depends on. It should join the green light the moment anything +in the tree starts relying on the packed layout. + +Green light: `./tools/bench/check.sh` **ALL GREEN**, before and after. + +## NEXT SESSION, in order + +**Item 4 is CLOSED (session 19, FINDINGS 51).** The list below is otherwise +unchanged, and items 1-3 are all still blocked on hardware this machine does not +have. What session 19 adds to item 1 is a SECOND number to measure the medium +against, and it is the harder one: + +- **451.4 KB/s** is the zero-prefill pipe for the gate container (49.5) — enough + to arrive on time in a straight line. +- **Absorbing a seek is a different requirement.** At 460 KB/s every ring from + 192 KB to 512 KB is rate-bound and never fills. The rate that makes a branch + point affordable is set by `pipe - wire` and how long the scene runs between + branches, not by the buffer — so measuring the BlueSCSI's *seek time* matters + as much as its throughput, and neither is known. 51.3/51.4. + +Item 5 (`--spans all`) now has a second reason to be careful: it spends every +profitable byte, which raises `wire`, which shrinks `pipe - wire`, which +lengthens the climb back to a full ring after every branch. That interaction is +not priced. + +## The list + + +0. **Green light first.** `./tools/bench/check.sh`. It is slower again: it now + also runs the 120-frame **ring-buffer** pass on a 2 MB machine, re-derives + `prep_dlx.py`'s blob, and asserts `decode.bin`'s MD5. + **Do not run two MAME jobs at once** — session 18 did, and two `decode.lua` + runs sharing one log file produced a 0-byte log and 15 wasted minutes. The + warning is already in this document; it is there because it keeps happening. + +1. **Measure the BlueSCSI. It is now the only thing that unblocks the I/O side.** + FINDINGS 50 retired the pipe constant outright (USER DECISION): it is gone as + a default from every analysis tool and from `stream.lua`, which now all + REQUIRE an explicit rate. **There is no working delivery figure in this repo + any more, and that is deliberate.** + + The consequence is that anything needing a delivery rate to mean something is + now waiting on a measurement — which is the honest state, and it is why this + is item 1. `19_ring_stream.py` gives the threshold to measure against: + **513.2 KB/s** for the session-14 candidate, **451.4 KB/s** for the gate + container. Zero-prefill pipes, per container, from real record sizes. + + **Do not substitute a guess.** If the measurement is not available this + session, run the tools at several explicit rates and report the sensitivity, + rather than picking one and letting it become the next constant. That is + precisely how the last one survived five sessions after 42.1 called it + folklore. + + If it clears 513.2, the 8.7 KB/s overrun of 49.5 is not one, and the rate + controller can go on binding purely on clocks *with that fact written down*. + If it does not, the two levers are giving the rate controller a pipe term + (it has none — it binds on `decode + c*bytes`) or re-encoding lower; the + first is more informative and neither should be priced before the + measurement exists. + +2. **Settle whether buffer mode blanks the display.** Unchanged, and still + blocked on a real board. **Read FINDINGS 48 first — session 17 shifted the + prior toward MAME and toward "unusable."** `tools/bench/probe_bit11_blank.lua` + is written and settles it in minutes. Second, smaller, same sitting: the + priority register `0xE82500` at `0x0000` (47.3). **Do not pre-build on + 1.0 B/pixel.** + +3. **Settle single-address vs dual-address.** Unchanged, still the largest open + number: 242 KB/s and 0.69 dB. **Session 18 confirmed the artefact is not on + this machine**: `~/mame/roms/x68000.zip` has `cgrom`, four IPLs and `sram`, + and no SCSI ext ROM anywhere on the box. So this needs `scsiexrom.bin` + (8 KB, CRC `7be488de`) sourced, then its DMAC init disassembled for **DCR's + DTYP** field: `10`/`11` = single (5.0 clk/B), `00`/`01` = dual (9.0). + FINDINGS 48.4. A board or schematic is the fallback, not the first move. + +4. **DONE, session 19 — FINDINGS 51.** ~~Pace the decoder, then measure + branch-point stall tolerance.~~ The rig has a frame clock, the ceilings are + measured at seven ring sizes and five pipe rates, the `ceiling - 1` tolerance + was falsified by cutting the pipe, and the result that matters is that slack + is accumulated: 4.83 s of play to refill 256 KB at 488 KB/s. The original + statement of the gap, kept because it is still the right description of what + was wrong: FINDINGS 49.7.2 is the sharpest gap the new rig leaves: because `stream.s` free-runs, + the ring never backs up, so the ring-size sweep tests **wrap correctness** at + each size and **not buffering**. 48 KB passes and is single-buffered — do not + read it as a viable player buffer. A paced decoder (MFP timer, or Lua gating) + turns the rig into the seek-tolerance test Dragon's Lair actually needs, and + it is the last piece of the delivery story. + +5. **Make `--spans all` the default.** Still a recommendation rather than a + measurement (43.6.1), and 44.3 sharpens it: it is the only lever on the + encoder's byte side that changes anything. **Note it interacts with item 1** — + `all` spends every profitable byte, which is what put the candidate over the + pipe in the first place. + +6. **Re-derive span selection jointly with `lam`** (39.3). The selector ranks on + net clocks; `lam` still prices bytes against a KB/s target rather than + against 5 clocks each — and if item 1 goes route (b), `lam` gets a real KB/s + ceiling back and this question changes shape. + +## What session 15 settled + +**Session 14's item 2 was aimed at a lever that is not loaded.** FINDINGS 44. +The bucket diagnosis of 43.5 is correct as a mechanism and worth one frame of +120 in practice, because at `--spans all` the byte side of the rate controller +does not bind at all: + +| `--spans all`, c=5, the 120-frame `singe` window | KB/s | PSNR | over budget | +|---|---:|---:|---:| +| **shipped, `--bucket-frames 8`** | **496.7** | **29.19** | **1/120** | +| `--bucket-frames 32` | 496.7 | 29.19 | 1/120 | +| `--bucket-frames 1` — no banking at all | 498.0 | 29.19 | 1/120 | +| `--rc-floor open` — lam floor 1.0 | 503.7 | 29.21 | 1/120 | + +A 32-frame bucket emits the **same container byte for byte** as an 8-frame one, +and `lam` never leaves its floor of 10.0 on any of 120 frames. The rate this +project reports is set by the span pass and by `mu`; `--kbps` and the bucket +are not the levers. Measure whether the lever is loaded before pulling it. + +**Two real unit inconsistencies were found, fixed, and defaulted OFF on +measurement** — `--joint-decide` and `--joint-bucket` turn them on: + +| c=5, `--spans all` | KB/s | PSNR | mean frame clocks | over | +|---|---:|---:|---:|---:| +| **shipped** | **496.7** | **29.19** | **740,049** | 1/120 | +| `--joint-decide` — the lagrangian sees the disk | 482.5 | 29.17 | 745,438 | 1/120 | +| `--joint-bucket` — the bucket may not lend clocks | 506.4 | 29.18 | 754,429 | 1/120 | + +The first is the more interesting one even though it does nothing: priced per +delivered byte, a RAW block costs `400.4 + 16c` and a V4 block `448.2 + 4c`, +which **cross at c = 3.98 — just under 43.1's hard floor of 4.** So `mu`'s +V4 -> RAW escape hatch, FINDINGS 28.8 and session 8's `0c`, never existed on +real hardware: it spends 12 clocks of bus to save 47.8 of CPU. Correcting it +moves 0.3% of blocks and 0.02 dB. + +Defaults are unchanged in effect: `s14_d5_all1500` re-encodes to the same MD5. + +**An encode is 95% k-means, and it is now 2.7x faster, exactly.** 60.6 s -> 29.4 s +for a 120-frame window, from three fixes to `VQ.assign` (a materialised `C.T`, a +cache-sized chunk, and a thread pool over the chunk loop). Bit-identical labels, +so every container still hashes the same. FINDINGS 44.5. + +**A decoder-free player fits the clocks and dies on the medium.** Streaming raw +preprocessed frames straight into video memory — DMAC device -> GVRAM, no +decoder in the loop — fits at c=5 with 41% of the frame to spare, but needs +1,152 KB/s sustained (~79% of SCSI-1 async, nothing left for audio or seeks) and +1.61 GB for the game, against the 0.70 GB the shipping container needs. The +cause is that 256-colour GVRAM is one pixel per word with the high byte +discarded, so half of every byte pulled off the disk is thrown away by the +hardware on arrival. FINDINGS 44.7. **Session 16 confirmed that layout is not +escapable on the graphics planes (FINDINGS 46) — so this stands.** + +## What session 14 settled + +**Session 13's binding unknown `W` was in the wrong unit, and correcting it +costs the project 41% of its rate and 1.85 dB.** FINDINGS 43. + +`W` was charged per WORD of delivered stream. The MB89352 is an **8-bit** SPC, +so the DMAC pays per BYTE. Every I/O debit in the project since FINDINGS 5 has +been charged at half rate, and the favourable end of 39.7's 5..12 bracket was +never physically reachable: **a 68000 bus cycle is four clocks and the SPC +hands over one byte per cycle, so nothing can cost less than 4 clocks/byte.** +5 clocks/word is 2.5 clocks/byte — 62% of one bus cycle, and a 4 MB/s DMA on a +link that runs at 1.5 MB/s asynchronous. + +The datasheet, per byte, device-to-memory, bus held (MC68450 Fig 4-25): + +| how the DMAC is programmed | clocks/byte | +|---|---:| +| single address, D->M (sheet 2) | **5.0** | +| dual address, byte, no packing (sheet 4) | **9.0** | +| dual address, byte packed (sheet 3) | 16.5 | + +**Nothing session 13 emitted fits at any of these.** `s13_280p1500`, the +0/120 candidate, is 44/120 at 5 clocks/byte and 120/120 at 9. + +### What survives, and it had to be re-encoded rather than re-scored + +The encoder was making its decisions in the same wrong units: `ratectl` +bisected `mu` against 833,333 cycles of *decode* with no disk term, and +`spans.select()` admitted a run only if it beat the blocks **on cycles alone**, +explicitly ignoring the bytes it added. Both now work in one currency — clocks — +and `--disk-clk-byte 0` re-emits session 13's container byte for byte, so the +change is the price and not the codec. + +| | KB/s | PSNR | over budget | span px | +|---|---:|---:|---:|---:| +| s13's claim (2.5 clk/B) | 837.4 | 31.04 dB | 0/120 | 62.3% | +| **`s14_d5_all1500` — single address** | **496.7** | **29.19 dB** | **1/120** | 30.7% | +| `s14_d9_all1500` — dual address | 255.0 | 28.50 dB | **1/120** | 3.9% | + +**The one frame over is frame 0 in both** — the intra frame, which 28.5/31 +established is emitted late on purpose. Every other frame lands at or under +100.0%. `17_span_delivered.py` shares no code with the encoder's accounting and +reproduces both rows to the digit. + +**The remaining lever is not ours to pull.** A span pixel costs `9.143 + 2c` +clocks and a V1 block pixel `18.74 + 0.25c`, so **a span beats a block only for +`c < 5.48`** — which falls between the two rows above. Single-address needs the +CZ-6BS1 to drive the SPC's DACK from the bus's `#EXACK` (pin B37, which the bus +does have). That is the *board's* wiring, not our code, and MAME models only the +dual-address row. + +Green light: `./tools/bench/check.sh` **ALL GREEN**, re-run after the encoder +change — 0/120 rate-control drift under the joint cost, DLX3 round-trip exact. + +### session 14's list + +Items 0, 2 and 4 are answered by FINDINGS 44; the rest carry forward, +reordered, in the session-15 list at the top of this file. + + +0. **Green light first.** `./tools/bench/check.sh` (~6 min, Blu-ray mounted). + +1. **Settle single-address vs dual-address.** It is worth 242 KB/s and 0.69 dB + and it is the largest open number in the project. It is a **hardware** fact: + the CZ-6BS1 schematic, or a real board, or a period service manual. MAME + cannot answer it (`x68k_scsiext.cpp` has no DACK path and no DRQ line from + the slot to the DMAC at all, though the real bus has `#EXREQ`/`#EXACK`), so + do not spend a session in the emulator on it. FINDINGS 43.2/43.3. + +2. **Fix the rate controller's bucket.** Diagnosed, not fixed (43.5): it banks + bytes across 8 frames, and bytes are now clocks, which cannot be banked + because there is no double buffer. `corr(bytes, frame clocks) = 0.989`. The + per-frame ceiling should be joint and hard; the bucket should smooth only + what is left after the disk is paid. + +3. **Get a 68000 to decode one of the new containers.** Unchanged from session + 13's item 3 and still the strongest claim resting on the weakest coverage — + but *narrower* now: `s14_d5_all1500` is 30.7% span coverage against the + gate's 26.6%, where session 13's candidate was 62.3%. This is the cheapest + it will ever be to close. A 496.7 KB/s stream is ~10 frames in a 2 MB + machine, so it still wants a chunk-streaming rig, not a longer pass. + +4. **Make `--spans all` the default** (my recommendation, not a measurement): + with a joint cost, `need` stops early and leaves profitable clock savings + unbought — 275.8 KB/s / 28.92 dB / 2 frames over, against `all`'s 496.7 / + 29.19 / 1. FINDINGS 43.6. + +5. **Re-run the ring-buffer simulation at ~497 KB/s** (29.5/30.7, open since + session 7). Cheaper than the 837 KB/s it was going to have to survive. + +6. **Re-derive span selection jointly with `lam`** (39.3). Half-done: the + selector now ranks on net clocks, but `lam` still prices bytes against a + KB/s target rather than against 5 clocks each. + +**Do not start by hand-optimising `decode.s`.** Five sessions running, and more +true than ever: at `c=5` the disk is 26.6% of the median frame and the decoder +is not what misses. + +**Do not treat 1/120 as settled either.** FINDINGS 43.8 lists five things it +does not establish, and 43.9 is about how the last favourable table survived a +section written specifically to distrust it. + +--- + +## What session 13 settled + +Session 12 ended by asking the user to choose a rate point. The user instead +challenged the constant the question rested on — and it does not hold. The +488 KB/s figure was never a SCSI bus figure: **SCSI-1 is 1.5 MB/s asynchronous +and 5 MB/s synchronous** (ANSI X3.131-1986), so the design has been running +against 10% of the bus. Once bytes are that cheap the span pass simply +saturates, and it does so on its own at ~837 KB/s. + +| container | KB/s | span px | PSNR | over @5 | @8 | @12 | +|---|---:|---:|---:|---:|---:|---:| +| the session-12 gate | 487.7 | 26.6% | 29.63 dB | 11/120 | 34/120 | 78/120 | +| **`s13_280p1500` — the candidate** | **837.4** | **62.3%** | **31.04 dB** | **0/120** | 21/120 | 63/120 | +| `s13_lossless` — a probe, not shippable | 934.6 | 70.4% | 31.19 dB | **0/120** | 47/120 | — | + +**0/120 is the first time anything here has fitted 12fps on every frame**, and +it is simultaneously the best picture yet — 0.29 dB off a 31.33 dB palette +ceiling, i.e. off exact for this display. `mu` is never spent at these rates, so +FINDINGS 31's 0.62 dB CPU-fit cost is refunded. **Quality and deadline have +stopped competing**, which was true of no earlier session. + +The mechanism: a v7 span pixel costs 2 wire bytes and **9.143 clocks**, a RAW +block pixel costs 1 byte and **25.03**. Spans win for any `W < 31.7`. The budget +is 16.95 clocks/pixel, so **a full-frame literal fits iff `W <= ~6.5`.** + +> **Withdrawn, session 15.** That threshold is per WORD: 6.5 clocks/word is +> 3.25 clocks/byte, under the 4-clock floor, so it was never reachable. In byte +> units the breakeven is **c = 3.906** against a floor of 4.0 — a CPU-painted +> full-frame literal misses at every real price, by 1.1% at the floor itself. +> FINDINGS 44.7. + +### Which makes `W` the whole result + +| `W` clocks/word | 934.6 KB/s stream | over budget | +|---:|---|---:| +| 5 | median 83.0%, worst 91.0% | **0/120** | +| 6 | median 87.6%, worst 97.0% | **0/120** | +| 7 | median 92.2%, worst 103.0% | 21/120 | +| 8 | median 96.8%, worst 109.0% | 47/120 | + +Moving `W` across FINDINGS 39.7's datasheet bracket costs more frames than +moving the rate from 280 to 935 KB/s wins. It has displaced the 4 Mbps figure as +the most load-bearing unmeasured number in the project. + +**Two things are known about `W` and they point the good way.** MAME's CZ-6BS1 +glue is cycle-steal **with the bus held** — `x68k_scsiext.cpp:110-136` gates +`#DTACK` on `DRQ` while the DMAC's OWN is asserted — which is the 5 clk/word end +of the bracket, not the ~12 arbitrated end. And `W` decomposes into `5 clocks of +MC68450 + however long the drive makes the DMAC wait`; the deployment target has +been SD-backed SCSI since session 2 (FINDINGS 21.2), which collapses the wait +term. FINDINGS 42.5, 42.6. + +**Also corrected: "never `x68ksupr`" was never a hardware claim.** Internal and +external are the same MB89352 and neither needs a driver (IOCS is in ROM); the +difference is that MAME has not implemented the internal DMA glue +(`x68k.cpp:1176`, `// TODO`). The external board **is** the way to test the DMA +path in MAME, and that is what item 1 below does. + +Green light: `./tools/bench/check.sh` **ALL GREEN** at the start of session 13, +unchanged since. Nothing in session 13 touched the encoder, the decoder or the +container — this session moved numbers and documents only. + +**Session 13's list is superseded** — its items 1 and 2 are done and its +answer did not survive them (FINDINGS 43); the rest are carried forward, +reordered, in the session-14 list at the top of this file. + +**And its headline is withdrawn.** The 0/120 above, the 31.04 dB, the +saturation at 837 KB/s and "quality and deadline have stopped competing" +were all scored with the disk debited per WORD to a byte-wide port. The +reasoning in this section about the SCSI bus rating (42.1) stands; every +number downstream of it was charged at half rate. Read FINDINGS 43. + + +--- + +## What session 12 settled Session 12 built v7 into the player. **`src/player/decode.s` paints v7 literal spans, and it is pixel-exact under both CPU cores** over a container where every @@ -49,52 +868,6 @@ FINDINGS 41.6. Green light: `./tools/bench/check.sh` **ALL GREEN**, now gating on a span-heavy DLX3 container. -## NEXT SESSION, in order - -0. **Green light first.** `./tools/bench/check.sh` (~6 min, Blu-ray mounted). - Verified green at end of session 12. The gate container is now - `tmp/rc_fr_singe_scsi_span.dlx` (scsi modes, spans on the full pipe) and the - rig fits 37 of 120 frames in a 2 MB machine. - -1. **Decide the rate point, because the span result now depends on it.** This is - the user's call and it is the first real fork since the profile was set: - spans only pay if the stream is allowed to run near the pipe (487.7 KB/s - delivered, 34/120 over budget) rather than at the 280 KB/s profile (77/120). - That is a delivery-medium question — FINDINGS 32 dropped SASI on capacity and - parked the 110 KB/s point for CD-ROM, and 488 KB/s is 93% of a 4 Mbps figure - whose provenance is still unconfirmed (FINDINGS 29.5 item 3). **Do not spend - another session optimising against a budget nobody has chosen.** - -2. **Re-derive the span selection jointly with lam, not after it.** The encoder - picks modes at one budget and then spans what is left, which FINDINGS 39.3 - already called a lower bound. A frame that misses its deadline would often do - better raising lam to free room for spans than lowering it — spans are - pixel-exact, so the quality trade is not what it looks like. Bisecting a - span reserve fraction inside the existing search is the tractable version. - -3. **Re-run `13_cpu_ratectl.py` against a DLX3 container.** 14 and 15 are done - (15 now counts span bus traffic and still reproduces the C68K measurement to - 0.04%); 13 has not been re-run since the constant changed. - -4. **Make sure the player actually gets DMA** — unchanged from sessions 10-11, - and still not an optimisation. Benchmark `x68000 -exp1 cz6bs1`, **never - `x68ksupr`**; MAME's internal SCSI has no DMA glue and would measure a PIO - fallback the real machine does not have. This is now more load-bearing, not - less: the delivered stream is 487.7 KB/s and the disk debit is 163,798 - clocks a frame, 20% of the budget. - -5. **Re-run the ring-buffer simulation at the surviving rate** (FINDINGS - 29.5/30.7, still open) and confirm the 488 KB/s figure's provenance. - -**Do not start by hand-optimising `decode.s`.** Unchanged and still true. The -cycles to be won are in the budget, not the loop — and session 12 is the second -demonstration that the *model* of the budget is where the errors live. - -**A new trap, worth reading before quoting any figure:** the rig had been -writing its synthetic timing frames 26 KB past the top of a 2 MB machine, and -got away with it because the modes it overran are data-independent. A span is -not — its jump displacements come out of the stream. FINDINGS 41.4. - --- ## What session 11 settled @@ -410,7 +1183,8 @@ The two session-7 stages are worth knowing the shape of before they fail on you: | Double buffer | **none** — page 1 sacrificed | enables `movem.l` 24px bursts; delta coding needs a RAM reference frame anyway | | **Codec** | **hybrid VQ: SKIP / V1 4x4 / V4 four-2x2 / RAW, per-block rate-distortion** | flat 4x4 VQ was measured and rejected — see FINDINGS 9-10 | | **Quality modes** | **one: `scsi`** (USER DECISION, session 9 — `sasi` dropped) | a SASI volume is 40 MB and the game is 146 MiB at the LOWEST rate this codec makes. FINDINGS 32 | -| Delivery medium | **SD-backed SCSI** (BlueSCSI / SCSI2SD), as locked in session 2 | capacity does not bind on SD at any rate this codec makes; it is what killed SASI, and it is what rules CD-ROM OUT (650 MiB with spans, and 1x CD is ~150 KB/s). FINDINGS 32.3 | +| Delivery medium | **SD-backed SCSI** (BlueSCSI / SCSI2SD), as locked in session 2 | capacity does not bind on SD at any rate this codec makes; it is what killed SASI, and it is what rules CD-ROM OUT (**1.09 GiB** at the session-13 candidate rate, against a CD's ~620 MiB and ~150 KB/s at 1x). FINDINGS 32.3, 42.7 | +| **SCSI DMA handshake** | **UNDECIDED — and it decides the project** | `W`, clocks stolen per word, bracketed 5..12 by MC68450 Fig 4-25. `W<=6` fits 0/120 frames; `W=8` misses 47/120. It is a property of how the player drives the MB89352, so it is ours to choose, not to receive. FINDINGS 42.4-42.6 | | Profile axis | **I/O bandwidth only** | the profiles say nothing about CPU; **both target the same stock 10MHz 68000**, and the Super has SCSI at 10MHz. FINDINGS 28.7 | | Framerate | 12 fps, **explicit decimation** | source has zero duplicate frames; no free "twos" win | | Emulator | MAME 0.277 x68000 | accurate enough that measured cycles mean something | @@ -432,10 +1206,24 @@ ships: | ~~`sasi`~~ | ~~110 KB/s~~ | ~~60~~ | ~~36.9 / 29.6 dB~~ | **RETIRED session 9** | | `scsi` | 280 KB/s | 10 (floor) | 39.4 / 32.3 dB | Super/XVI, or CZ-6BS1 board | -The 110 KB/s **rate point** is not necessarily gone with the interface: a 1x +~~The 110 KB/s **rate point** is not necessarily gone with the interface: a 1x CD-ROM sustains ~150 KB/s and CD-ROM is the only period medium with room for the whole game (374 MiB at `scsi`, 600 MiB with spans). The user's call was to ship -one profile now and settle the medium when the pipe is measured. FINDINGS 32.3. +one profile now and settle the medium when the pipe is measured.~~ FINDINGS 32.3. + +**SUPERSEDED by FINDINGS 42.** There is no rate point left to choose: the span +pass saturates at ~837 KB/s and the candidate container is **`s13_280p1500`, +837.4 KB/s = 1.09 GiB for the whole game**. That is unremarkable on SD and +impossible anywhere period, so **CD-ROM is closed, not parked**. `--kbps` is now +a quality knob whose value barely matters above ~800 KB/s of span budget; the +number that matters is `W`. + +| what | KB/s | whole game (1366.6 s) | +|---|---:|---:| +| `scsi` profile as set | 280.0 | 0.36 GiB | +| session-12 gate | 487.7 | 0.64 GiB | +| **candidate `s13_280p1500`** | **837.4** | **1.09 GiB** | +| probe `s13_lossless` | 934.6 | 1.22 GiB | **That "machine" column is about the BUS, not the CPU.** The profiles are an I/O-bandwidth axis and say nothing about clock speed: the X68000 Super has @@ -611,14 +1399,19 @@ multi-byte fields are **big-endian** so the 68000 reads them with a plain `move` Gated by `tools/analysis/09_ratectl_drift.py`, which is now in `check.sh`. - **Payload is deliberately NOT entropy-coded** — deflate decode does not fit in the 68000's frame budget (FINDINGS 17.2). Do not "optimise" this later. -- **Frame records are not aligned.** They must be padded to a 4-byte boundary: - unaligned is an ADDRESS ERROR on a 68000, not a slow read (FINDINGS 28.3). - `prep_dlx.py` repairs it at load time, which a player streaming from disc - cannot do. The pad is real bytes on disc, so it belongs inside the rate - controller's accounting. 1.66 B/frame, 20 B/s. -- **The mode decision is blind to CPU cost.** It charges V4 four payload bytes - and ignores that it costs 1.49x a V1 block to draw. This is the top item at - the head of this file. FINDINGS 28.2. +- ~~**Frame records are not aligned.**~~ **DONE.** The container is `DLX2` and + `write_container` (`encode.py:139-156`) pads both the frame table and every + record to a 4-byte boundary, inside the rate controller's accounting rather + than after it, and `encode.py` reports the cost in B/frame and B/s. Unaligned + is an ADDRESS ERROR on a 68000, not a slow read (FINDINGS 28.3); `DLX1` is + still read, so the old unaligned containers have not been invalidated. +- ~~**The mode decision is blind to CPU cost.**~~ **DONE**, sessions 12-19. + `vq_hybrid.decide()` (`vq_hybrid.py:218`) minimises + `distortion + lam*bytes + mu*(decode cycles + byte_clk*bytes)` against the + MEASURED per-mode cycle costs, and `cycles()` scores a whole frame with the + exact clustered SKIP rule rather than a per-block constant. This entry sat in + the gap list for several sessions after it was closed; **the list is not + self-maintaining, and stale entries here become phantom work in the roadmap.** - **Palette packing is not implemented in the encoder.** It still emits 24-bit palettes; the X68000 word packing happens Lua-side. Whatever writes real palette words must pick `I` per entry by minimum squared error (FINDINGS 23.3, @@ -938,6 +1731,51 @@ V1's output. To check that snapshot is still pixel-exact: Not added to `check.sh`: `check.sh` asserts pixel-exactness, and asserting wall timings there would make the green-light check sensitive to host load. +## Reproducing the streaming result (session 18) + +The ring-buffer pass, on a **stock 2 MB machine**, container in a host file: + +``` +DLX=tmp/rc_fr_singe_scsi_span.dlx +tools/vasm/vasmm68k_mot -Fbin -o tmp/stream.bin src/player/stream.s +python3 tools/bench/prep_stream.py "$DLX" +mkdir -p tmp/snap_stream && rm -f tmp/snap_stream/x68000/*.png +( cd tmp && DLX_STREAM_KBPS=0 SDL_VIDEODRIVER=dummy stdbuf -oL \ + timeout -k 5 600 mame x68000 -bios ipl10 -ramsize 2M -video soft -window \ + -sound none -nothrottle -plugins -autoboot_script ../tools/bench/stream.lua \ + -snapshot_directory ./snap_stream -snapview native -seconds_to_run 90 \ + > stream_check.log 2>&1 ) +python3 tools/bench/verify_decode.py "$DLX" --snap tmp/snap_stream +``` + +Env knobs: `DLX_RING_KB` (default 256), `DLX_STREAM_KBPS` (**REQUIRED, no +default**; `0` = unlimited, which isolates the WRAP question from the DELIVERY +one), `DLX_PREFILL_KB` (default 0). + +`19_ring_stream.py`'s `--kbps` is required too, as are `--bus`/`--kbps` on +12/14/16/17. None of them has a default any more — see the retirement note at +the top of this document. + +`DLX_STREAM_KBPS=0` is the right setting for a **correctness** gate: the wrap +policy is what is being tested and an unlimited pipe removes delivery as a +variable. Use a finite rate to get the `DEADLINE`/`REQUIRED PREFILL` lines. + +The ring simulation, which needs no emulator and takes a second: + +``` +python3 tools/analysis/19_ring_stream.py # both containers +python3 tools/analysis/19_ring_stream.py --kbps 512 --ring 128 +``` + +It reports the **zero-prefill pipe** — the threshold a medium has to clear for +the container to need no prefill at all — which is the number to measure a +BlueSCSI against. It also warns explicitly when demand exceeds supply on the +MEAN, because a "required prefill" figure for a sustained overrun is the most +flattering possible way to state one. + +**Do not run this alongside another MAME job.** They share `tmp/` logs and +snapshot directories and the second run silently truncates the first. + ## Reproducing the decoder result (session 7) ``` @@ -956,9 +1794,18 @@ full 120-frame pass; then verifies the last frame is pixel-exact. Expected: median 73.8%, p90 116.4%, max 135.8%, mean 81.7%; V1 299.9 / V4 448.2 / RAW 400.4 cycles per block. -`-ramsize 2M` matters — MAME defaults to 4M and the locked target is a stock 2MB -machine. `DLX_VERIFY_ONLY=1` drops the timing anchors, which is how `check.sh` -runs it. +`-ramsize 2M` here matters for a different reason than it used to. MAME defaults +to 4M and the locked *target* is a stock 2 MB machine, so a timing reproduction +quoting the target's memory should use it. But the rig preloads the whole +container, so at 2M this run is truncated to the prefix that fits — 37 of 120 +frames on the current span-heavy container — and its real-frame anchors are +therefore a biased sample (session 16 measured the bias at 8.2% on the mean; +FINDINGS 45.3). The **synthetic** anchors are unaffected: they come out +bit-identical at 2M and 6M, which is what licenses `check.sh` running the +verify-only gate at `RIG_RAM=6`. For anchors over the whole window, run this +with `--ram 0x600000` and `-ramsize 6M`. + +`DLX_VERIFY_ONLY=1` drops the timing anchors, which is how `check.sh` runs it. Score a container against the measured costs without touching MAME: ``` diff --git a/docs/img/decoded-frame.png b/docs/img/decoded-frame.png new file mode 100644 index 0000000..c3c59f1 Binary files /dev/null and b/docs/img/decoded-frame.png differ diff --git a/docs/img/modes.webm b/docs/img/modes.webm new file mode 100644 index 0000000..70f00bf Binary files /dev/null and b/docs/img/modes.webm differ diff --git a/docs/img/player.webm b/docs/img/player.webm new file mode 100644 index 0000000..18f93d6 Binary files /dev/null and b/docs/img/player.webm differ diff --git a/docs/img/source-vs-decoded.png b/docs/img/source-vs-decoded.png new file mode 100644 index 0000000..7b589af Binary files /dev/null and b/docs/img/source-vs-decoded.png differ diff --git a/src/player/decode.s b/src/player/decode.s index dcf5aae..1fa418b 100644 --- a/src/player/decode.s +++ b/src/player/decode.s @@ -88,18 +88,8 @@ FPTR = $18010 ; -> first frame record SCR_N = $18014 ; frames remaining this pass SCR_END = $18018 ; expected end of the current payload -CB1 = $20000 ; expanded 4x4 codebook -CB4 = $22000 ; expanded 2x2 codebook + include "src/player/geom.i" -DST0 = $C08000 ; GVRAM + 32*1024 (first picture row) -DSTE = $C38000 ; GVRAM + 224*1024 (one past last) -BROW = 4096 ; bytes per block row (4 picture rows) -ROWLEN = 512 ; bytes per block row of blocks (64 * 8) -MODEB = 768 ; packed mode header, 3072 blocks * 2 bits -SPCU = 12 ; bytes of code per COARSE span unit (24 px) -SPCN = 11 ; coarse units: 11*24 = 264 px >= one row -SPFU = 2 ; bytes of code per FINE span unit (2 px) -SPFN = 11 ; fine units: 11*2 = 22 px > one coarse unit org $10000 start: @@ -130,186 +120,4 @@ hold: bra.s hold desync: move.l #$EE,FLAG.l bra.s hold -; ---------------------------------------------------------------- one block -; \1 = right-shift needed to bring this block's 2 mode bits to bits 1-0. -BLOCK macro - move.b (a1),d0 - ifne \1 - lsr.b #\1,d0 - endc - and.w #3,d0 - beq .sk\@ ; 00 SKIP -- the median block - subq.w #1,d0 - beq .v1\@ ; 01 V1 - subq.w #1,d0 - bne .rw\@ ; 11 RAW, else 10 V4 - -; -- V4: four 2x2 codewords, sub-block order TL TR BL BR (vq_hybrid.paint) - moveq #0,d0 - move.b (a0)+,d0 - lsl.w #3,d0 - movem.l (a3,d0.w),d0-d1 - move.l d0,(a4) - move.l d1,1024(a4) - moveq #0,d0 - move.b (a0)+,d0 - lsl.w #3,d0 - movem.l (a3,d0.w),d0-d1 - move.l d0,4(a4) - move.l d1,1028(a4) - moveq #0,d0 - move.b (a0)+,d0 - lsl.w #3,d0 - movem.l (a3,d0.w),d0-d1 - move.l d0,2048(a4) - move.l d1,3072(a4) - moveq #0,d0 - move.b (a0)+,d0 - lsl.w #3,d0 - movem.l (a3,d0.w),d0-d1 - move.l d0,2052(a4) - move.l d1,3076(a4) - bra .sk\@ - -; -- V1: one 4x4 codeword, 32 bytes, straight out of the expanded codebook -.v1\@: - moveq #0,d0 - move.b (a0)+,d0 - lsl.w #5,d0 - movem.l (a2,d0.w),d0-d7 ; EA is resolved before the load - movem.l d0-d1,(a4) - movem.l d2-d3,1024(a4) - movem.l d4-d5,2048(a4) - movem.l d6-d7,3072(a4) - bra .sk\@ - -; -- RAW: 16 literal palette indices. Two indices are assembled into one long -; via swap, so each pair of pixels costs one write instead of two; the high -; byte of each word is left as zero because the hardware discards it anyway. -.rw\@: - RAWPAIR 0 - RAWPAIR 4 - RAWPAIR 1024 - RAWPAIR 1028 - RAWPAIR 2048 - RAWPAIR 2052 - RAWPAIR 3072 - RAWPAIR 3076 -.sk\@: - addq.l #8,a4 - endm - -RAWPAIR macro - moveq #0,d0 - move.b (a0)+,d0 - swap d0 - move.b (a0)+,d0 - move.l d0,\1(a4) - endm - -; ------------------------------------------------------- the span section -; in: a0 = span section, a1 = mode header (preserved across the call) -; out: a0 = one past the section, i.e. the block payload -; -; This is tools/bench/blit.s v7 verbatim, and deliberately so: the 66.0 clocks -; per span + 9.143 per coarse pixel + 9.978 per fine pixel of FINDINGS 40 were -; measured on exactly this instruction sequence, over thirteen span lengths, and -; a "tidier" rewrite here would silently invalidate every span figure in -; FINDINGS 39/40 and in tools/analysis/14_dmac_chain.py. -; -; The fine chain is entered by FALLING OUT of the coarse one, so a span with no -; coarse units enters at v7cx with d0 already reloaded -- which is why the -; coarse displacement for c=0 is SPCN*SPCU, one past the last coarse unit, -; rather than a special case. -paint_spans: - move.w (a0)+,d7 ; spans in this frame - subq.w #1,d7 - bmi spnone ; a frame may legitimately have none (the - ; chain is far past a short branch) - move.l a1,-(sp) ; a1 is a payload register below -spspan: move.l (a0)+,a2 ; absolute GVRAM destination - move.w (a0)+,d0 ; (SPCN - coarse) * SPCU - jmp spch(pc,d0.w) -spch: - movem.l (a0)+,d0-d6/a1/a3-a6 - movem.l d0-d6/a1/a3-a6,(a2) - lea 48(a2),a2 - movem.l (a0)+,d0-d6/a1/a3-a6 - movem.l d0-d6/a1/a3-a6,(a2) - lea 48(a2),a2 - movem.l (a0)+,d0-d6/a1/a3-a6 - movem.l d0-d6/a1/a3-a6,(a2) - lea 48(a2),a2 - movem.l (a0)+,d0-d6/a1/a3-a6 - movem.l d0-d6/a1/a3-a6,(a2) - lea 48(a2),a2 - movem.l (a0)+,d0-d6/a1/a3-a6 - movem.l d0-d6/a1/a3-a6,(a2) - lea 48(a2),a2 - movem.l (a0)+,d0-d6/a1/a3-a6 - movem.l d0-d6/a1/a3-a6,(a2) - lea 48(a2),a2 - movem.l (a0)+,d0-d6/a1/a3-a6 - movem.l d0-d6/a1/a3-a6,(a2) - lea 48(a2),a2 - movem.l (a0)+,d0-d6/a1/a3-a6 - movem.l d0-d6/a1/a3-a6,(a2) - lea 48(a2),a2 - movem.l (a0)+,d0-d6/a1/a3-a6 - movem.l d0-d6/a1/a3-a6,(a2) - lea 48(a2),a2 - movem.l (a0)+,d0-d6/a1/a3-a6 - movem.l d0-d6/a1/a3-a6,(a2) - lea 48(a2),a2 - movem.l (a0)+,d0-d6/a1/a3-a6 - movem.l d0-d6/a1/a3-a6,(a2) - lea 48(a2),a2 -spcx: move.w (a0)+,d0 ; (SPFN - fine) * SPFU, from mid-stream - jmp spfh(pc,d0.w) -spfh: - move.l (a0)+,(a2)+ - move.l (a0)+,(a2)+ - move.l (a0)+,(a2)+ - move.l (a0)+,(a2)+ - move.l (a0)+,(a2)+ - move.l (a0)+,(a2)+ - move.l (a0)+,(a2)+ - move.l (a0)+,(a2)+ - move.l (a0)+,(a2)+ - move.l (a0)+,(a2)+ - move.l (a0)+,(a2)+ - dbra d7,spspan - move.l (sp)+,a1 -spnone: rts - -; ------------------------------------------------------------- one frame -; in: a0 = payload, a1 = packed mode header -; out: a0 = one past the last payload byte consumed -decode_frame: - lea CB1,a2 - lea CB4,a3 - lea DST0,a6 -rowloop: - move.l a6,a4 - lea ROWLEN(a6),a5 -byteloop: - tst.b (a1) ; four SKIPs in one test - beq allskip - BLOCK 6 - BLOCK 4 - BLOCK 2 - BLOCK 0 - addq.l #1,a1 - cmpa.l a5,a4 - bne byteloop - bra rowdone -allskip: - addq.l #1,a1 - lea 32(a4),a4 - cmpa.l a5,a4 - bne byteloop -rowdone: - lea BROW(a6),a6 - cmpa.l #DSTE,a6 - bne rowloop - rts + include "src/player/frame.i" \ No newline at end of file diff --git a/src/player/frame.i b/src/player/frame.i new file mode 100644 index 0000000..385ffb9 --- /dev/null +++ b/src/player/frame.i @@ -0,0 +1,183 @@ +; ---------------------------------------------------------------- one block +; \1 = right-shift needed to bring this block's 2 mode bits to bits 1-0. +BLOCK macro + move.b (a1),d0 + ifne \1 + lsr.b #\1,d0 + endc + and.w #3,d0 + beq .sk\@ ; 00 SKIP -- the median block + subq.w #1,d0 + beq .v1\@ ; 01 V1 + subq.w #1,d0 + bne .rw\@ ; 11 RAW, else 10 V4 + +; -- V4: four 2x2 codewords, sub-block order TL TR BL BR (vq_hybrid.paint) + moveq #0,d0 + move.b (a0)+,d0 + lsl.w #3,d0 + movem.l (a3,d0.w),d0-d1 + move.l d0,(a4) + move.l d1,1024(a4) + moveq #0,d0 + move.b (a0)+,d0 + lsl.w #3,d0 + movem.l (a3,d0.w),d0-d1 + move.l d0,4(a4) + move.l d1,1028(a4) + moveq #0,d0 + move.b (a0)+,d0 + lsl.w #3,d0 + movem.l (a3,d0.w),d0-d1 + move.l d0,2048(a4) + move.l d1,3072(a4) + moveq #0,d0 + move.b (a0)+,d0 + lsl.w #3,d0 + movem.l (a3,d0.w),d0-d1 + move.l d0,2052(a4) + move.l d1,3076(a4) + bra .sk\@ + +; -- V1: one 4x4 codeword, 32 bytes, straight out of the expanded codebook +.v1\@: + moveq #0,d0 + move.b (a0)+,d0 + lsl.w #5,d0 + movem.l (a2,d0.w),d0-d7 ; EA is resolved before the load + movem.l d0-d1,(a4) + movem.l d2-d3,1024(a4) + movem.l d4-d5,2048(a4) + movem.l d6-d7,3072(a4) + bra .sk\@ + +; -- RAW: 16 literal palette indices. Two indices are assembled into one long +; via swap, so each pair of pixels costs one write instead of two; the high +; byte of each word is left as zero because the hardware discards it anyway. +.rw\@: + RAWPAIR 0 + RAWPAIR 4 + RAWPAIR 1024 + RAWPAIR 1028 + RAWPAIR 2048 + RAWPAIR 2052 + RAWPAIR 3072 + RAWPAIR 3076 +.sk\@: + addq.l #8,a4 + endm + +RAWPAIR macro + moveq #0,d0 + move.b (a0)+,d0 + swap d0 + move.b (a0)+,d0 + move.l d0,\1(a4) + endm + +; ------------------------------------------------------- the span section +; in: a0 = span section, a1 = mode header (preserved across the call) +; out: a0 = one past the section, i.e. the block payload +; +; This is tools/bench/blit.s v7 verbatim, and deliberately so: the 66.0 clocks +; per span + 9.143 per coarse pixel + 9.978 per fine pixel of FINDINGS 40 were +; measured on exactly this instruction sequence, over thirteen span lengths, and +; a "tidier" rewrite here would silently invalidate every span figure in +; FINDINGS 39/40 and in tools/analysis/14_dmac_chain.py. +; +; The fine chain is entered by FALLING OUT of the coarse one, so a span with no +; coarse units enters at v7cx with d0 already reloaded -- which is why the +; coarse displacement for c=0 is SPCN*SPCU, one past the last coarse unit, +; rather than a special case. +paint_spans: + move.w (a0)+,d7 ; spans in this frame + subq.w #1,d7 + bmi spnone ; a frame may legitimately have none (the + ; chain is far past a short branch) + move.l a1,-(sp) ; a1 is a payload register below +spspan: move.l (a0)+,a2 ; absolute GVRAM destination + move.w (a0)+,d0 ; (SPCN - coarse) * SPCU + jmp spch(pc,d0.w) +spch: + movem.l (a0)+,d0-d6/a1/a3-a6 + movem.l d0-d6/a1/a3-a6,(a2) + lea 48(a2),a2 + movem.l (a0)+,d0-d6/a1/a3-a6 + movem.l d0-d6/a1/a3-a6,(a2) + lea 48(a2),a2 + movem.l (a0)+,d0-d6/a1/a3-a6 + movem.l d0-d6/a1/a3-a6,(a2) + lea 48(a2),a2 + movem.l (a0)+,d0-d6/a1/a3-a6 + movem.l d0-d6/a1/a3-a6,(a2) + lea 48(a2),a2 + movem.l (a0)+,d0-d6/a1/a3-a6 + movem.l d0-d6/a1/a3-a6,(a2) + lea 48(a2),a2 + movem.l (a0)+,d0-d6/a1/a3-a6 + movem.l d0-d6/a1/a3-a6,(a2) + lea 48(a2),a2 + movem.l (a0)+,d0-d6/a1/a3-a6 + movem.l d0-d6/a1/a3-a6,(a2) + lea 48(a2),a2 + movem.l (a0)+,d0-d6/a1/a3-a6 + movem.l d0-d6/a1/a3-a6,(a2) + lea 48(a2),a2 + movem.l (a0)+,d0-d6/a1/a3-a6 + movem.l d0-d6/a1/a3-a6,(a2) + lea 48(a2),a2 + movem.l (a0)+,d0-d6/a1/a3-a6 + movem.l d0-d6/a1/a3-a6,(a2) + lea 48(a2),a2 + movem.l (a0)+,d0-d6/a1/a3-a6 + movem.l d0-d6/a1/a3-a6,(a2) + lea 48(a2),a2 +spcx: move.w (a0)+,d0 ; (SPFN - fine) * SPFU, from mid-stream + jmp spfh(pc,d0.w) +spfh: + move.l (a0)+,(a2)+ + move.l (a0)+,(a2)+ + move.l (a0)+,(a2)+ + move.l (a0)+,(a2)+ + move.l (a0)+,(a2)+ + move.l (a0)+,(a2)+ + move.l (a0)+,(a2)+ + move.l (a0)+,(a2)+ + move.l (a0)+,(a2)+ + move.l (a0)+,(a2)+ + move.l (a0)+,(a2)+ + dbra d7,spspan + move.l (sp)+,a1 +spnone: rts + +; ------------------------------------------------------------- one frame +; in: a0 = payload, a1 = packed mode header +; out: a0 = one past the last payload byte consumed +decode_frame: + lea CB1,a2 + lea CB4,a3 + lea DST0,a6 +rowloop: + move.l a6,a4 + lea ROWLEN(a6),a5 +byteloop: + tst.b (a1) ; four SKIPs in one test + beq allskip + BLOCK 6 + BLOCK 4 + BLOCK 2 + BLOCK 0 + addq.l #1,a1 + cmpa.l a5,a4 + bne byteloop + bra rowdone +allskip: + addq.l #1,a1 + lea 32(a4),a4 + cmpa.l a5,a4 + bne byteloop +rowdone: + lea BROW(a6),a6 + cmpa.l #DSTE,a6 + bne rowloop + rts diff --git a/src/player/geom.i b/src/player/geom.i new file mode 100644 index 0000000..980bf40 --- /dev/null +++ b/src/player/geom.i @@ -0,0 +1,27 @@ +; Geometry and codebook constants shared by every front-end in src/player/. +; +; Split out of decode.s in session 18 so that decode.s (the preloaded-stream +; rig, gated by tools/bench/check.sh) and stream.s (the ring-buffer streaming +; rig, FINDINGS 49) assemble from LITERALLY THE SAME BYTES for the block loop +; and the span chain. Those bytes are not incidental: the 66.0 clocks/span, +; 9.143 clocks/coarse pixel and 9.978 clocks/fine pixel of FINDINGS 40, and +; every per-block constant in FINDINGS 24/30/41, are fitted to this exact +; instruction sequence. Two hand-maintained copies of it would drift, and the +; drift would be invisible -- both would still decode correctly, and only the +; cost model would be wrong. +; +; The split is a no-op by construction: tools/bench/check.sh asserts that +; decode.s still assembles to the same 1,296 bytes it did before it. + +CB1 = $20000 ; expanded 4x4 codebook +CB4 = $22000 ; expanded 2x2 codebook + +DST0 = $C08000 ; GVRAM + 32*1024 (first picture row) +DSTE = $C38000 ; GVRAM + 224*1024 (one past last) +BROW = 4096 ; bytes per block row (4 picture rows) +ROWLEN = 512 ; bytes per block row of blocks (64 * 8) +MODEB = 768 ; packed mode header, 3072 blocks * 2 bits +SPCU = 12 ; bytes of code per COARSE span unit (24 px) +SPCN = 11 ; coarse units: 11*24 = 264 px >= one row +SPFU = 2 ; bytes of code per FINE span unit (2 px) +SPFN = 11 ; fine units: 11*2 = 22 px > one coarse unit diff --git a/src/player/stream.s b/src/player/stream.s new file mode 100644 index 0000000..c2f8b88 --- /dev/null +++ b/src/player/stream.s @@ -0,0 +1,184 @@ +; DLX3 frame decoder, RING-BUFFER front-end -- STATUS item 3, FINDINGS 49. +; +; src/player/decode.s decodes a stream that is ALREADY WHOLLY IN RAM: the rig +; preloads 5,261,814 bytes at $30000 and walks a0 forward through all of it. +; That gate proves the decoder is pixel-exact over a 120-frame window +; (FINDINGS 45) and says NOTHING about how the bytes got there. The shipping +; player never holds a window at once; it streams from a SCSI disk into a ring +; a fraction of the size. Nothing in this tree has ever tested that path. +; +; WHAT IS ACTUALLY HARD ABOUT IT. The block loop and the span chain read the +; stream with a monotonically increasing a0 and no bounds check anywhere -- +; `move.l (a0)+,d0`, `lea MODEB(a0),a0`, eleven unrolled `movem.l (a0)+`, a +; `move.b (a0)+` per block index. None of it can survive an address that wraps +; mid-record. So the ring does not merely need ENOUGH BYTES resident by the +; deadline -- 09_buffer_sim.py's question, and FINDINGS 21's answer -- it needs +; the WHOLE NEXT RECORD resident and CONTIGUOUS. +; +; THE WRAP POLICY IS `aligned`, and it was chosen on measurement, not taste +; (tools/analysis/19_ring_stream.py). The producer refuses to start a record it +; cannot finish before the end of the ring: it leaves a hole and restarts at 0. +; +; aligned costs RAM -- a mean hole of 23.4 KB in a 256 KB ring, 9.1% of it -- +; and ZERO CPU. +; split lets records wrap and mirrors the ring's first MAXREC bytes into a +; shadow past its end, so any record start reads linearly. Costs +; zero RAM and 46,394 clocks/frame of memcpy -- 5.57% of the frame +; budget, forever. +; +; Both figures are for s14_d5_all1500, the shipping candidate, in a 256 KB ring; +; they scale with record size, so they are per container, not universal. The +; lighter gate container makes it 5.7% of the ring against 3.64% of the budget -- +; same direction, same verdict. +; +; The decoder already spends 77.0% of the budget on the mean frame and 91.1% at +; p90 (FINDINGS 45). 5.57% more puts p90 at 96.7%. RAM is the resource this +; machine has 2 MB of and clocks are the one it has none of, so the trade is not +; close. `aligned` also needs a per-record INDEX on the fill side, which a +; BRANCHING laserdisc game needs anyway to seek to a branch point -- so the +; policy that costs no clocks also reuses a structure the player cannot avoid. +; +; The third option -- teach the block loop to wrap its own reads -- is the +; expensive one and not because of the branch. A bounds test lands INSIDE the +; instruction sequences FINDINGS 30.4 and 40 fitted their constants to, so it +; does not cost a compare, it costs every span and per-block figure in the tree +; being re-measured. +; +; THE PRODUCER IS OUTSIDE THIS FILE. Here it is tools/bench/stream.lua playing +; a SCSI disk at a modelled byte rate; in the player it is the MB89352 and a +; DMAC channel. The handshake is deliberately the same either way: +; +; producer -> FR_HEAD count of records made wholly resident (monotonic) +; DESC[] ring of record base addresses, DESCN entries +; decoder -> FR_TAIL count of records consumed (monotonic) +; RD_PTR one past the last byte read; everything below is free +; +; Two monotonic counters and a released-to pointer -- no lock, no shared cursor, +; single reader and single writer, so it is correct on a 68000 with no atomics +; provided each side only ever writes its own words. That is why FR_TAIL is +; the decoder's and FR_HEAD is the producer's rather than one shared index. +; +; STALLS ARE COUNTED, NOT HIDDEN. A frame whose record is not resident when the +; decoder wants it spins in `waitrec`, and STALLS counts the FRAMES that had to +; wait at all (not the polls). A rig that silently absorbed an underrun would +; report a pixel-exact decode of a stream that arrived late, which is precisely +; the failure this front-end exists to make visible. The spin is bounded: +; SPINMAX polls without progress sets FLAG=$E1, so a wedged producer fails as a +; wedged producer instead of as a MAME timeout with no diagnosis (FINDINGS 34.1). + +FLAG = $18000 ; 0 idle / 1 running / $FF done / $EE desync + ; / $E1 producer stalled out +ITER = $18008 ; outer repeat count, written by Lua +NFR = $1800C ; frames per pass +SCR_N = $18014 ; frames remaining this pass +SCR_END = $18018 ; expected end of the current payload +RD_PTR = $18020 ; decoder -> producer: released up to here +FR_HEAD = $18024 ; producer -> decoder: records resident +FR_TAIL = $18028 ; decoder -> producer: records consumed +STALLS = $1802C ; frames that had to wait for their record +SPINS = $18030 ; total poll iterations spent waiting +PACE = $18034 ; producer -> decoder: frame ticks elapsed since + ; release. Frame i may not START before tick i. +PACEON = $18038 ; 1 = obey PACE. 0 leaves the loop free-running, + ; byte for byte the loop FINDINGS 49 measured. +DESC = $18100 ; DESCN x u32, record base addresses + +DESCN = 64 ; power of two; the index is masked, not compared +DESCM = (DESCN-1)*4 ; mask for a BYTE offset into DESC + +SPINMAX = 2000000 ; polls with no progress before giving up + + include "src/player/geom.i" + + org $10000 +start: + move.l #1,FLAG.l ; timer starts here +outer: + move.l NFR.l,SCR_N.l + clr.l FR_TAIL.l + clr.l STALLS.l + clr.l SPINS.l +frameloop: +; ---- PACE GATE (FINDINGS 49.7.2, and it is the whole point of this session). +; Free-running, this loop asks for record i the instant it finishes record i-1, +; so it outruns any finite pipe, the ring NEVER backs up, and the producer's +; overlap test never refuses a placement. A ring-size sweep under those +; conditions tests WRAP CORRECTNESS at each size and nothing about BUFFERING: +; 48 KB passes while holding one record. A shipping player does not do this -- +; it draws frame i, waits for its slot, and spends the rest of the frame time +; idle while the disk fills the ring behind it. +; +; So the rig gets a frame clock. PACE is bumped by the producer (in the player, +; vblank or an MFP timer) and frame i is forbidden to start before tick i. With +; the decoder held to 12 fps the ring fills, the producer starts hitting its own +; overlap test, and FR_HEAD-FR_TAIL becomes what it claims to be: the number of +; whole frames the decoder could run on if delivery stopped dead -- which is the +; branch-point seek question stated in frames. +; +; It also makes STALLS mean something. Free-running, a stall is EARLINESS +; (49.6); paced, a frame that has to wait for its record is a real underrun. + tst.l PACEON.l + beq.s nopace +pacewait: + move.l PACE.l,d0 + cmp.l FR_TAIL.l,d0 ; d0 - FR_TAIL; carry = tick not reached + bcs.s pacewait +nopace: +; ---- wait until the producer has made this record wholly resident. +; d1 counts polls for this frame; a nonzero d1 on exit means the frame stalled. + moveq #0,d1 + move.l FR_TAIL.l,d2 +waitrec: + move.l FR_HEAD.l,d0 + cmp.l d2,d0 + bhi.s gotrec ; HEAD > TAIL: at least one record ready + addq.l #1,d1 + cmp.l #SPINMAX,d1 + bcs.s waitrec + move.l #$E1,FLAG.l ; producer never delivered + bra hold +gotrec: + tst.l d1 + beq.s nostall + addq.l #1,STALLS.l + add.l d1,SPINS.l +nostall: +; ---- pop the descriptor. DESCN is a power of two, so the wrap is an and. + move.l d2,d0 + lsl.l #2,d0 + and.w #DESCM,d0 + lea DESC,a1 ; DESC is absolute; (d0.w) needs a base + move.l (a1,d0.w),a0 ; a1 is reloaded from a0 two lines below + +; ---- from here to the release, byte for byte what decode.s does. a0 is +; inside the ring rather than inside a preloaded blob, and the block loop +; cannot tell the difference -- which is the whole claim being tested. + move.l (a0)+,d0 ; u32 payload length, big-endian + lea 0(a0,d0.l),a1 + move.l a1,SCR_END.l ; where the payload must end + move.l a0,a1 ; a1 = packed mode header + lea MODEB(a0),a0 ; a0 = span section + bsr paint_spans ; -> a0 = block payload, a1 preserved + bsr decode_frame + cmpa.l SCR_END.l,a0 ; bitstream desync is silent otherwise + bne desync + +; ---- release. Round up to 4 the same way decode.s does: the producer lays +; records on 4-byte boundaries, so the byte one past this record's padded +; end is the first byte the producer may reuse. + move.l a0,d0 + addq.l #3,d0 + and.b #$FC,d0 + move.l d0,RD_PTR.l + addq.l #1,FR_TAIL.l + + subq.l #1,SCR_N.l + bne frameloop + subq.l #1,ITER.l + bne outer + move.l #$FF,FLAG.l ; timer stops here +hold: bra.s hold +desync: move.l #$EE,FLAG.l + bra.s hold + + include "src/player/frame.i" diff --git a/tools/analysis/08_peaks.py b/tools/analysis/08_peaks.py index 64fc76e..6338a8f 100644 --- a/tools/analysis/08_peaks.py +++ b/tools/analysis/08_peaks.py @@ -1,4 +1,4 @@ -"""At a 488 KB/s (4 Mbps) ceiling and ~52% mean utilisation, the mean is not the +"""At any fixed delivery ceiling and ~52% mean utilisation, the mean is not the risk -- the peaks are. Measure per-frame peak-to-mean, then check whether the leaky-bucket rate controller actually holds the ceiling.""" import sys; sys.path.insert(0,'tools/encoder') diff --git a/tools/analysis/11_cpu_budget.py b/tools/analysis/11_cpu_budget.py index 64e3782..21cb450 100644 --- a/tools/analysis/11_cpu_budget.py +++ b/tools/analysis/11_cpu_budget.py @@ -24,7 +24,13 @@ import numpy as np from dlx import DLX import vq_hybrid as H import ratectl as RC -RC_AUDIO_BPS = RC.AUDIO_KBPS * 1024 +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +import buscost as B +# The audio byte rate is now DERIVED, not restated: 15.6 kHz mono MSM6258V is +# 15,625 4-bit samples/s, two to a byte. RC.AUDIO_KBPS's 7.8 is that figure in +# DECIMAL kB, and was being multiplied by 1024 here -- a 2.4% overstatement, +# harmless, but it hid which unit the constant was in. +RC_AUDIO_BPS = B.ADPCM_BYTES_PER_S # Machine clocks, confirmed from MAME 0.277 src/mame/sharp/x68k.cpp:1133/1194/ # 1200 -- not recalled. x68000 and x68ksupr are BOTH 40_MHz_XTAL/4 = 10 MHz; @@ -58,8 +64,26 @@ ap.add_argument("--dma-clocks-per-word", type=float, default=8.0, help="HD63450 cycle-steal. ESTIMATE from FINDINGS 5, NEVER " "MEASURED, and the most load-bearing unmeasured number " "in the project (FINDINGS 35.3)") +ap.add_argument("--dma-clocks-per-byte", type=float, default=5.0, + help="what the SCSI DMA costs per DELIVERED BYTE. The MB89352 " + "is an 8-bit port, so the DMAC pays per byte and the " + "per-word denominator of FINDINGS 5/39.7 was half the " + "real debit (FINDINGS 43). 5 = single-address, bus held, " + "no drive wait; 9 = dual-address") ap.add_argument("--pio-clocks-per-byte", type=float, default=12.0, help="hand-derived floor for a 68000 register-to-RAM copy") +# Audio is NOT the disk, and charging it the disk's rate was charging it the +# favourable side of an open question. tools/analysis/21_iplrom_dmac.py reads +# the IPL ROM's own HD63450 setup: channel 3 is dual address, 8-bit port, cycle +# steal WITHOUT hold, external request -- one full arbitration per byte, no +# burst to amortise it over. 16 is the datasheet best case, 19 the worst. +ap.add_argument("--adpcm-clocks-per-byte", type=float, + default=B.ADPCM_CLK_BYTE_BEST, + help="what an ADPCM byte costs. READ OUT OF THE IPL ROM's DMAC " + "configuration (21_iplrom_dmac.py), not assumed: dual " + "address + per-byte arbitration = 16 best, 19 worst. The " + "audio stream always DMAs, whatever --io says about the " + "disk") a = ap.parse_args() CPUHZ = CLOCKS[a.machine] * 1e6 FPS = a.fps @@ -72,13 +96,15 @@ d = DLX(a.container) # --- what the transfer costs, from the container's own byte rate vid_bps = sum(n + 4 for (_, n) in d.frames) / d.nframes * d.fps io_bps = vid_bps + RC_AUDIO_BPS +aud_cycles_per_s = RC_AUDIO_BPS * a.adpcm_clocks_per_byte if a.io == "dma": - io_cycles_per_s = (io_bps / 2) * a.dma_clocks_per_word + io_cycles_per_s = vid_bps * a.dma_clocks_per_byte + aud_cycles_per_s elif a.io == "pio": - io_cycles_per_s = io_bps * a.pio_clocks_per_byte + io_cycles_per_s = vid_bps * a.pio_clocks_per_byte + aud_cycles_per_s else: io_cycles_per_s = 0.0 io_pct = 100 * io_cycles_per_s / CPUHZ +aud_pct = 100 * aud_cycles_per_s / CPUHZ FRAME_NET = FRAME * (1 - io_pct / 100) modes = [d.modes(f) for f in range(d.nframes)] @@ -91,9 +117,20 @@ print(f"budget: {a.machine} @ {CLOCKS[a.machine]:.2f} MHz, {FPS:g} fps " f"-> {FRAME:,.0f} cycles/frame") print(f" I/O ({a.io}): {io_bps/1024:.1f} KB/s costs {io_pct:.1f}% of the CPU " f"-> {FRAME_NET:,.0f} cycles/frame left for decoding") +if a.io != "none": + print(f" video {vid_bps/1024:6.1f} KB/s x " + f"{(a.dma_clocks_per_byte if a.io=='dma' else a.pio_clocks_per_byte):g}" + f" clk/B = {io_pct-aud_pct:5.2f}% " + f"(W: still open, ROADMAP B3 / FINDINGS 42.4)\n" + f" audio {RC_AUDIO_BPS/1024:6.2f} KB/s x {a.adpcm_clocks_per_byte:g}" + f" clk/B = {aud_pct:5.2f}% " + f"(SETTLED: read out of the IPL ROM, FINDINGS 52)") if a.io == "dma": - print(f" {a.dma_clocks_per_word:g} clocks/word is an ESTIMATE (FINDINGS 5), " - f"never measured -- see FINDINGS 35.3") + print(f" {a.dma_clocks_per_byte:g} clocks/BYTE, the MC68450 datasheet " + f"floor for an 8-bit port (FINDINGS 43).\n It is not measured on " + f"hardware; what IS settled is that the per-word denominator this\n" + f" used before session 14 was physically impossible -- 2.5 " + f"clocks/byte is below\n the 68000's 4-clock minimum bus cycle.") elif a.io == "none": print(" WARNING: --io none scores the decoder as if the disk were free. " "That is the\n premise FINDINGS 35 overturned; every 'N frames miss' " @@ -114,15 +151,20 @@ TIMED_FRAMES = (("min non-SKIP", 15.4, 31.5), ("median", 48.1, 73.8), ("p90", 82.5, 116.4), ("max non-SKIP", 100.0, 135.8)) if (os.path.abspath(a.container) == os.path.abspath(TIMED) and a.machine == "stock" and a.fps == 12): - print("model vs the frames actually timed on the 68000:") + print("model vs the frames actually timed on the 68000 " + "(the model reads HIGH, and by more\n as the frame gets harder -- " + "so a 'does not fit' from it is the safe direction):") for label, frac, meas in TIMED_FRAMES: i = int(np.argmin(abs(ns - frac))) print(f" {label:<14} non-SKIP {ns[i]:5.1f}% model {pct[i]:6.1f}% " f"measured {meas:5.1f}% error {pct[i]-meas:+.1f} pt") else: - print(f"(no 68000 timings for this container/machine -- the model was " - f"validated to\n within 1 pt on {TIMED} at stock/12fps;\n" - f" run tools/bench/decode.lua to time another container)") + print(f"(no 68000 timings for this container/machine. The model is " + f"validated against four\n frames timed on the 68000, and only on " + f"{TIMED}\n at stock/12fps -- run it on that container to see the " + f"errors, which are a few points\n CONSERVATIVE and grow with the " + f"non-SKIP fraction. Run tools/bench/decode.lua to\n time another " + f"container.)") print(f"\nper-frame cost, % of a {FPS:g}fps frame budget:") print(f" measured-cost model: median {np.median(pct):5.1f} " diff --git a/tools/analysis/12_span_tradeoff.py b/tools/analysis/12_span_tradeoff.py index 63bf0af..ea87794 100644 --- a/tools/analysis/12_span_tradeoff.py +++ b/tools/analysis/12_span_tradeoff.py @@ -1,9 +1,9 @@ #!/usr/bin/env python3 """What does spending the idle bus bandwidth buy back in CPU cycles? - python3 tools/analysis/12_span_tradeoff.py [container.dlx] [--bus 488] + python3 tools/analysis/12_span_tradeoff.py [container.dlx] --bus -FINDINGS 28 leaves the decoder CPU-bound at 110 KB/s on a 488 KB/s pipe. Every +FINDINGS 28 leaves the decoder CPU-bound at 110 KB/s on a much wider pipe. Every codec decision was made when bytes were scarce, so each one trades cycles to save them -- and the cheapest thing a 68000 can be handed is the most expensive thing to store: word-expanded pixels in row-linear runs. @@ -50,8 +50,8 @@ def span_px(npix): # a span is a whole number of units ap = argparse.ArgumentParser() ap.add_argument("container", nargs="?", default="tmp/rc_fr_singe_sasi_rcprofile.dlx") -ap.add_argument("--bus", type=float, default=488.0, - help="sustained KB/s the pipe delivers (FINDINGS 21)") +ap.add_argument("--bus", type=float, required=True, + help="REQUIRED. There is no default: the delivery rate is a property of the medium and this project has never measured it. FINDINGS 42.1 -- the figure this tool used to default to was a user-supplied '4 Mbps' with no provenance, was a tenth of SCSI-1's asynchronous rating, and was never a bus measurement at all. A default let every table in FINDINGS 30-49 be scored against it without anyone restating it. Pass one explicitly.") ap.add_argument("--fps", type=float, default=12.0) a = ap.parse_args() if not os.path.exists(a.container): diff --git a/tools/analysis/14_dmac_chain.py b/tools/analysis/14_dmac_chain.py index bcefe6e..76b6809 100644 --- a/tools/analysis/14_dmac_chain.py +++ b/tools/analysis/14_dmac_chain.py @@ -1,7 +1,7 @@ #!/usr/bin/env python3 """Would letting the HD63450 paint the spans beat letting the 68000 do it? - python3 tools/analysis/14_dmac_chain.py [container.dlx] [--bus 488] + python3 tools/analysis/14_dmac_chain.py [container.dlx] --bus [--dma-px-bus 2] [--disk-bus-byte 1] FINDINGS 29.6 called this the one lever that could move the CPU budget without @@ -11,7 +11,7 @@ prices the two against each other, and the answer turns on a resource neither section costed: the 68000's own LOCAL BUS. FINDINGS 29's "the bus has 4x the headroom the CPU has" is about the SCSI pipe, -110 KB/s of 488. That is a different bus. The 68000's memory bus runs one 4-clock +110 KB/s of the delivery pipe. That is a different bus. The 68000's memory bus runs one 4-clock cycle at a time and carries instruction prefetch as well as data, and tools/analysis/15_bus_occupancy.py measures the decoder using 86.7% of it. @@ -65,17 +65,28 @@ SPAN_BYTES_PX, SPAN_HDR = 2, 6 ap = argparse.ArgumentParser() ap.add_argument("container", nargs="?", default="tmp/rc_fr_singe_scsi_cpufit.dlx") -ap.add_argument("--bus", type=float, default=488.0, help="SCSI pipe, KB/s") +ap.add_argument("--bus", type=float, required=True, + help="REQUIRED. There is no default: the delivery rate is a property of the medium and this project has never measured it. FINDINGS 42.1 -- the figure this tool used to default to was a user-supplied '4 Mbps' with no provenance, was a tenth of SCSI-1's asynchronous rating, and was never a bus measurement at all. A default let every table in FINDINGS 30-49 be scored against it without anyone restating it. Pass one explicitly.") ap.add_argument("--fps", type=float, default=12.0) ap.add_argument("--dma-px-clk", type=float, default=B.DMA_PX_CLK, help="clocks the DMAC spends per pixel, dual-address word " "between two 16-bit ports. 9 is the DATASHEET figure " "(MC68450 Fig 4-25 sheet 4).") -ap.add_argument("--disk-clk-word", type=float, default=8.0, - help="clocks the SCSI DMA steals per word. The datasheet " - "brackets it at 5 (DMAC holds the bus) to 12 (arbitrates " - "per word); FINDINGS 5's estimate of 8 is the midpoint.") +ap.add_argument("--disk-clk-byte", type=float, default=5.0, + help="clocks the SCSI DMA steals per BYTE delivered. The SPC is " + "an 8-bit port, so the DMAC pays per byte, not per word " + "(FINDINGS 43). 5, the default, is the OPTIMISTIC end and " + "what ratectl encodes against: single-address, bus held, no " + "drive wait (Fig 4-25 sheet 2). 9 is dual-address, which is " + "what MAME models and what applies if the board does not " + "drive DACK. Score both.") +ap.add_argument("--disk-clk-word", type=float, default=None, + help="DEPRECATED denominator of FINDINGS 39.7/42, kept so the " + "old tables reproduce: sets --disk-clk-byte to half this") a = ap.parse_args() +if a.disk_clk_word is not None: + a.disk_clk_byte = a.disk_clk_word / 2.0 + if not os.path.exists(a.container): sys.exit(f"missing {a.container}") @@ -154,7 +165,7 @@ def score(design): for k, c in BLK_C.items(): cpu += (mm == k).sum() * c pref, data = B.block_bus(m, spanned) - disk = byt / 2.0 * a.disk_clk_word + disk = byt * a.disk_clk_byte # additive: CPU work, then span painting, then the disk stealing the bus out.append((cpu + span_clk + disk, (pref + data) * B.BUS_CLK, byt, spanned.sum())) @@ -192,7 +203,7 @@ row("frames missing", lambda v: f"{v}/{d.nframes}", row("blocks spanned/frame", lambda v: f"{v:,.0f}", lambda r: r[3].mean()) print(f"\n ADDITIVE: frame = CPU + span painting + disk DMA. The 68000 has no" f"\n cache and a two-word prefetch queue, so it stalls the moment another" - f"\n master takes the bus. Disk debited at {a.disk_clk_word:g} clocks/word.") + f"\n master takes the bus. Disk debited at {a.disk_clk_byte:g} clocks/byte.") # What is left of the case, isolated. v6m = int((res["v6 span"][0] > FRAME_CYC).sum()) diff --git a/tools/analysis/15_bus_occupancy.py b/tools/analysis/15_bus_occupancy.py index f4f15aa..5597968 100644 --- a/tools/analysis/15_bus_occupancy.py +++ b/tools/analysis/15_bus_occupancy.py @@ -140,3 +140,65 @@ print(f"\nprefetch is {100*pref_t.sum()/tot.sum():.0f}% of the decoder's bus tra print(f"A DMAC painting spans at 8 clocks (2 bus cycles) per pixel could use at\n" f"most {free.mean()/2:,.0f} pixels' worth of the mean frame's spare slots " f"-- against {d.nb*16:,} pixels\nin a whole screen.") + +# --------------------------------------------------------------------------- +# THE OTHER TWO MASTERS. Everything above is the 68000's own traffic, and it +# was the whole of this tool until session 20. The frame also has to carry the +# bitstream in off the disk and a byte of ADPCM out to $E92003 every 128 us, +# and neither has ever appeared in a bus figure -- FINDINGS 35's lesson, which +# was about the CLOCK budget, had never been applied to the BUS one. +# +# The DMAC does not overlap with the CPU (buscost.DMA_OVERLAPS = False): the +# 68000 has no cache and a two-word prefetch queue that empties at once, so a +# stolen bus cycle is a stopped CPU. The three demands therefore ADD. +# +# Audio's per-byte figure is SETTLED, not bracketed by taste: +# tools/analysis/21_iplrom_dmac.py reads the IPL ROM's own HD63450 setup and +# finds channel 3 dual-address, 8-bit port, cycle steal without hold, external +# request -- one arbitration per byte, no burst. Video's is NOT settled: it is +# ROADMAP B3 / FINDINGS 42.4-42.6's W, so it is swept rather than picked. +print("\n" + "=" * 72) +print("THE OTHER TWO MASTERS -- what the DMAC takes out of the same frame\n") +FPS = d.fps +CPUHZ = 10e6 # stock X68000, MAME 0.277 x68k.cpp:1133 +FRAME_CLK = CPUHZ / FPS +vid_bpf = sum(n + 4 for (_, n) in d.frames[:NF]) / NF # DLX2 record padding +aud_bpf = B.ADPCM_BYTES_PER_S / FPS +a_lo = aud_bpf * B.ADPCM_CLK_BYTE_BEST +a_hi = aud_bpf * B.ADPCM_CLK_BYTE_WORST +cpu_clk = cyc_t.mean() if cyc_t.any() else float("nan") + +print(f"frame period at {FPS:g} fps on a 10 MHz 68000: {FRAME_CLK:,.0f} clocks") +if cyc_t.any(): + print(f" decoder, MEASURED (C68K) {cpu_clk:>10,.0f} clk " + f"{100*cpu_clk/FRAME_CLK:5.1f}% worst frame " + f"{100*cyc_t.max()/FRAME_CLK:.1f}%") +print(f" audio DMA, {aud_bpf:,.1f} B/frame {a_lo:>10,.0f} clk " + f"{100*a_lo/FRAME_CLK:5.2f}% .. {a_hi:,.0f} clk " + f"({100*a_hi/FRAME_CLK:.2f}%)") +print(f" {B.ADPCM_CLK_BYTE_BEST}..{B.ADPCM_CLK_BYTE_WORST} clk/byte, " + f"from the ROM's own DCR/OCR (21_iplrom_dmac.py). NOT a guess, and\n" + f" not the disk's rate: audio arbitrates for the bus once per byte " + f"and cannot burst.") +print(f"\n video DMA, {vid_bpf:,.0f} B/frame, swept over W -- ROADMAP B3 is " + f"still open:") +print(f" {'W (clk/byte)':<16}{'clk/frame':>12}{'% of frame':>12} " + f"{'CPU+audio+video':>18}") +for W, note in ((5.0, "single address, bus held (11_cpu_budget.py default)"), + (8.0, "FINDINGS 5's long-standing per-word ESTIMATE"), + (12.0, "single address, arbitrated per byte"), + (16.0, "what the ROM programs for SASI (best case)"), + (19.0, "what the ROM programs for SASI (worst case)")): + v = vid_bpf * W + tot_clk = (cpu_clk if cyc_t.any() else 0) + a_lo + v + print(f" {W:<16.0f}{v:>12,.0f}{100*v/FRAME_CLK:>11.1f}% " + f"{100*tot_clk/FRAME_CLK:>17.1f}% {note}") +print(f"\n (the last column adds the MEASURED mean decode and the BEST-CASE " + f"audio, so it is\n the optimistic end of every row. 100% is the frame " + f"deadline at {FPS:g} fps.)") +print(f""" +Audio is {100*a_lo/FRAME_CLK:.2f}%..{100*a_hi/FRAME_CLK:.2f}% of the frame and video is {vid_bpf*5/FRAME_CLK*100:.0f}%..{vid_bpf*19/FRAME_CLK*100:.0f}%. The unpriced audio +stream was never the risk P6 called it -- ON THE BUS. What the same reading of +the ROM found is that the DISK's per-byte cost has a worked example on this +machine, it is 16..19 clocks, and at that price this design does not fit at any +container size. W is the number to attack, and it is a PLAYER decision.""") diff --git a/tools/analysis/16_span_roundtrip.py b/tools/analysis/16_span_roundtrip.py index 8ec536a..4594e9d 100644 --- a/tools/analysis/16_span_roundtrip.py +++ b/tools/analysis/16_span_roundtrip.py @@ -2,7 +2,7 @@ """GATE for the DLX3 span container: does the reference decoder reproduce the encoder's own reconstruction, from the emitted bytes? - python3 tools/analysis/16_span_roundtrip.py [frames_dir] [--kbps 488] + python3 tools/analysis/16_span_roundtrip.py [frames_dir] --kbps Exits non-zero if any frame differs by a single pixel. @@ -23,7 +23,7 @@ generated artefact, not from an assumption. So the thresholds below are asserted, not printed. The `--kbps` default is the BUS rate, not the `scsi` profile's 280: spans are -bought with bytes, and 14_dmac_chain.py scores them against the 488 KB/s pipe. +bought with bytes, and 14_dmac_chain.py scores them against the delivery pipe. At the profile rate the lam search has already spent the allowance and there is nothing left to buy a span with -- which is a real finding about the encoder (FINDINGS 41.2), not a reason for the gate to test nothing. @@ -36,7 +36,8 @@ from dlx import DLX ap = argparse.ArgumentParser() ap.add_argument("frames_dir", nargs="?", default="tmp/fr_singe") -ap.add_argument("--kbps", type=float, default=488.0) +ap.add_argument("--kbps", type=float, required=True, + help="REQUIRED. There is no default: the delivery rate is a property of the medium and this project has never measured it. FINDINGS 42.1 -- the figure this tool used to default to was a user-supplied '4 Mbps' with no provenance, was a tenth of SCSI-1's asynchronous rating, and was never a bus measurement at all. A default let every table in FINDINGS 30-49 be scored against it without anyone restating it. Pass one explicitly.") ap.add_argument("--out", default="tmp/s12_roundtrip") ap.add_argument("--cache", default=None) a = ap.parse_args() diff --git a/tools/analysis/17_span_delivered.py b/tools/analysis/17_span_delivered.py index 38cabbd..fe5bac6 100644 --- a/tools/analysis/17_span_delivered.py +++ b/tools/analysis/17_span_delivered.py @@ -1,7 +1,7 @@ #!/usr/bin/env python3 """What do the spans the ENCODER actually emitted cost, and what do they buy? - python3 tools/analysis/17_span_delivered.py a.dlx [b.dlx ...] [--bus 488] + python3 tools/analysis/17_span_delivered.py a.dlx [b.dlx ...] --bus Every span figure before this one -- FINDINGS 29 through 40, and tools/analysis/12 and 14 -- was scored by SIMULATING span selection over mode @@ -40,12 +40,24 @@ AUDIO_KBPS = 7.8 ap = argparse.ArgumentParser() ap.add_argument("containers", nargs="+") -ap.add_argument("--bus", type=float, default=488.0, help="SCSI pipe, KB/s") +ap.add_argument("--bus", type=float, required=True, + help="REQUIRED. There is no default: the delivery rate is a property of the medium and this project has never measured it. FINDINGS 42.1 -- the figure this tool used to default to was a user-supplied '4 Mbps' with no provenance, was a tenth of SCSI-1's asynchronous rating, and was never a bus measurement at all. A default let every table in FINDINGS 30-49 be scored against it without anyone restating it. Pass one explicitly.") ap.add_argument("--fps", type=float, default=12.0) -ap.add_argument("--disk-clk-word", type=float, default=8.0, - help="clocks the SCSI DMA steals per word (FINDINGS 39.7 " - "brackets it at 5..12; 8 is the midpoint)") +ap.add_argument("--disk-clk-byte", type=float, default=5.0, + help="clocks the SCSI DMA steals per BYTE delivered. The SPC is " + "an 8-bit port, so the DMAC pays per byte, not per word " + "(FINDINGS 43). 5, the default, is the OPTIMISTIC end and " + "what ratectl encodes against: single-address, bus held, no " + "drive wait (Fig 4-25 sheet 2). 9 is dual-address, which is " + "what MAME models and what applies if the board does not " + "drive DACK. Score both.") +ap.add_argument("--disk-clk-word", type=float, default=None, + help="DEPRECATED denominator of FINDINGS 39.7/42, kept so the " + "old tables reproduce: sets --disk-clk-byte to half this") a = ap.parse_args() +if a.disk_clk_word is not None: + a.disk_clk_byte = a.disk_clk_word / 2.0 + def score(path): @@ -57,7 +69,7 @@ def score(path): _, n = d.frames[f] blk = H.cycles(mode) spc = sum(SP.clocks(len(p)) for _, _, p in sp) - disk = n / 2.0 * a.disk_clk_word + disk = n * a.disk_clk_byte rows.append((blk, spc, disk, n, len(sp), sum(len(p) for _, _, p in sp))) return d, np.array(rows).T @@ -81,7 +93,7 @@ for path in a.containers: f"{int((tot > FRAME_CYC).sum()):>6}/{d.nframes:<3}") print(f"\n ADDITIVE: frame = block decode + span painting + disk DMA, the model" - f"\n of 14_dmac_chain.py. Disk debited at {a.disk_clk_word:g} clocks/word " + f"\n of 14_dmac_chain.py. Disk debited at {a.disk_clk_byte:g} clocks/byte " f"over the\n container's own byte count; CPU budget {FRAME_CYC:,.0f} " f"clocks at {a.fps:g} fps.") diff --git a/tools/analysis/18_text_plane_16col.py b/tools/analysis/18_text_plane_16col.py new file mode 100644 index 0000000..20774ec --- /dev/null +++ b/tools/analysis/18_text_plane_16col.py @@ -0,0 +1,88 @@ +#!/usr/bin/env python3 +"""What 256 -> 16 colours actually costs, on real frames. + + python3 tools/analysis/18_text_plane_16col.py [frames_dir] + +FINDINGS 46.3 opened a lead and could not price it: the X68000 text plane is +4bpp planar -- 0.5 bytes/pixel against the graphics planes' 2.0 -- so a LITERAL +uncompressed 16-colour frame is 288.0 KB/s against the shipping compressed +256-colour container's 496.7 KB/s. 42% cheaper on the wire, with no decoder. + +The whole lead turns on one number nobody had computed: the quality cost of 16 +colours. This computes it, and it is deliberately generous to the 16-colour +side on every axis where the hardware allows it: + + * PER-FRAME palettes are legitimate here. The text palette is 16 entries and + reloading it is 16 words a frame -- nothing, against a 833,333-clock budget. + The 256-colour path cannot do this: its palette is shared scene-wide + (vq.scene_palette) because the codec's codebooks are indices INTO it. + * DITHERING is free here, and only here. The tree does not dither (vq.py:32, + "cel art is flat") because dither destroys the inter-frame coherence SKIP + blocks and v7 spans are built on. A literal frame has no codec to wreck, so + Floyd-Steinberg is available to this path at zero runtime cost. + +Both are measured, so the comparison cannot be accused of hobbling the option it +is testing. Reported against the 256-colour scene-palette ceiling (the tree's +existing "palette ceiling" figure) and against the shipping container's PSNR. +""" +import sys, os +sys.path.insert(0, "tools/encoder") +import numpy as np +from PIL import Image +import vq as VQ + +FRAMES = sys.argv[1] if len(sys.argv) > 1 else "tmp/fr_singe" +SHIPPED_PSNR = 29.19 # docs/STATUS.md, --spans all, c=5, 496.7 KB/s + +rgb = VQ.load_frames(FRAMES) +H, W = rgb[0].shape[:2] +n = len(rgb) +print(f"{FRAMES}: {n} frames, {W}x{H}") +print() + +def recon_scene(colors, dither): + """One palette for the whole scene -- what the 256 path is forced to do.""" + d = Image.FLOYDSTEINBERG if dither else Image.NONE + samp = np.concatenate([r.reshape(-1, 3) for r in rgb[::3]]) + ref = Image.fromarray(samp.reshape(-1, 1, 3)).quantize( + colors=colors, method=Image.MEDIANCUT, dither=Image.NONE) + pal = np.array(ref.getpalette()[:colors * 3], np.uint8).reshape(-1, 3) + return [pal[np.asarray(Image.fromarray(r).quantize(palette=ref, dither=d), + np.uint8)] for r in rgb] + +def recon_perframe(colors, dither): + """A fresh palette every frame -- what the text plane can afford.""" + d = Image.FLOYDSTEINBERG if dither else Image.NONE + out = [] + for r in rgb: + q = Image.fromarray(r).quantize(colors=colors, method=Image.MEDIANCUT, + dither=d) + pal = np.array(q.getpalette()[:colors * 3], np.uint8).reshape(-1, 3) + out.append(pal[np.asarray(q, np.uint8)]) + return out + +def report(name, recon): + per = np.array([VQ.psnr(a, b) for a, b in zip(rgb, recon)]) + print(f" {name:<42s} {per.mean():6.2f} dB " + f"(min {per.min():5.2f} max {per.max():5.2f})") + return per.mean() + +print("PSNR vs the 24-bit source, mean over frames:") +c256 = report("256 colours, scene palette [the tree's]", recon_scene(256, False)) +report("256 colours, per-frame palette", recon_perframe(256, False)) +print() +s16 = report("16 colours, scene palette", recon_scene(16, False)) +p16 = report("16 colours, per-frame palette", recon_perframe(16, False)) +p16d = report("16 colours, per-frame + FS dither", recon_perframe(16, True)) +print() +print(f" the 16-colour ceiling is the best of those: {max(s16, p16, p16d):.2f} dB") +print(f" cost of 256 -> 16, at each side's best: " + f"{c256 - max(s16, p16, p16d):.2f} dB") +print() +print(f" for scale, the shipping container delivers {SHIPPED_PSNR:.2f} dB " + f"at 496.7 KB/s") +print(f" a 16-colour literal would deliver " + f"{max(s16, p16, p16d):.2f} dB at 288.0 KB/s") +delta = max(s16, p16, p16d) - SHIPPED_PSNR +print(f" so the text-plane path is {abs(delta):.2f} dB " + f"{'BETTER' if delta > 0 else 'WORSE'} at 58% of the bitrate") diff --git a/tools/analysis/19_ring_stream.py b/tools/analysis/19_ring_stream.py new file mode 100644 index 0000000..f965de4 --- /dev/null +++ b/tools/analysis/19_ring_stream.py @@ -0,0 +1,275 @@ +"""Ring-buffer streaming simulation, against the CONTIGUITY constraint (STATUS 3/4). + +09_buffer_sim.py asked one question -- does cumulative supply ever fall behind +cumulative demand -- and answered it in BYTES. FINDINGS 21 got "zero required +prefill" out of it at 110 and 280 KB/s. That test is necessary and not +sufficient, and the missing half is the whole of STATUS item 3: + + src/player/decode.s reads a frame record with a MONOTONICALLY INCREASING a0 + and no bounds check anywhere. `move.l (a0)+,d0` for the length, `lea + MODEB(a0),a0` for the span section, eleven unrolled `movem.l (a0)+` chains, + `move.b (a0)+` per block index. Nothing in it can survive an address that + wraps mid-record. So the buffer does not merely need ENOUGH BYTES resident + by the deadline -- it needs the WHOLE NEXT RECORD resident and CONTIGUOUS. + +Having enough bytes and having them contiguous are different conditions, and a +byte-counting simulation cannot tell them apart. This one models the ring's +addresses, not just its occupancy. + +THREE WRAP POLICIES, and the point of the tool is that they are not equivalent: + + split the writer wraps mid-record; the reader cannot. Requires a SHADOW of + the ring's first MAXREC bytes mirrored past its end, so any record + start can be read linearly for MAXREC bytes. Every byte landing in + that first MAXREC is written twice. Costs 68000 CLOCKS, forever, at a + rate set by MAXREC/ring -- and those clocks come out of the same + budget the decoder is already spending 77.0% of (FINDINGS 45). + + aligned the writer refuses to start a record it cannot finish before the end + of the ring; it leaves a hole and restarts at 0. Costs RAM (the mean + hole) and nothing else -- no copy, no per-byte work. Needs a frame + INDEX so the fill side knows record boundaries, which a branching + laserdisc game needs anyway to seek to a branch point. + + none the decoder handles the wrap itself. Priced here only to show what it + would cost: a bounds test in the block loop is inside the sequence + FINDINGS 30.4/40 fitted, so it does not cost a branch -- it costs + every span and per-block constant in the tree being re-measured. + Not simulated; see the note printed at the end. + +DEADLINE MODEL, and it is the conservative one: record i must be wholly +resident when frame i's decode BEGINS. The decoder in fact reads a record +progressively over ~77% of a frame time, so a byte arriving mid-frame would in +practice be in time -- but that is a race between the DMAC's fill address and +a0, and this tool refuses to certify a design on a race it cannot see. + +Fill is quantised to 512-byte SCSI blocks: a partial sector is not resident. + + python3 tools/analysis/19_ring_stream.py [container ...] --kbps R [--ring KB] + +`--kbps` is REQUIRED and has no default -- see the argument's help text. +""" +import sys, os, argparse +sys.path.insert(0, "tools/encoder") +import numpy as np +from dlx import DLX +import ratectl as RC + +SECTOR = 512 +# 5 clocks/byte for a 68000 `move.l (a0)+,(a1)+` copy: 20 clocks moves 4 bytes +# on a 16-bit bus (2 read + 2 write bus cycles at 4 clocks, plus the fetch it +# shares with the loop). Deliberately the OPTIMISTIC figure -- a movem-shaped +# copy is what the shadow would really use, and it is the same 5.0. +COPY_CLK_PER_BYTE = 5.0 +CPUHZ = 10_000_000 + + +def records(path): + """Padded record sizes, exactly as the 68000 walks them. + + prep_dlx.py rounds each record START up to 4 (FINDINGS 28.3), so the bytes + the ring must hold per frame are the padded ones, not the payload. + """ + d = DLX(path) + rec = np.array([4 + n + (-(4 + n) % 4) for _, n in d.frames], np.int64) + return d, rec + + +def simulate(rec, fill_per_frame, ring, policy, maxrec): + """Address-level ring simulation. Returns a dict of results. + + The ring is modelled as a write cursor and a read cursor over `ring` bytes. + Supply arrives at `fill_per_frame` bytes per frame time, sector-quantised. + Record i is due at the start of frame i. + """ + n = len(rec) + resident = 0.0 # bytes fully arrived and not yet consumed + carry = 0.0 # sub-sector remainder of the fill + wcur = 0 # write cursor within the ring + holes = [] # bytes wasted per wrap, `aligned` policy + shadow_bytes = 0 # bytes double-written, `split` policy + occ = [] + prefill = 0.0 + late = [] + free = ring + + # Required prefill is solved rather than searched: run once with an infinite + # head start to find the worst deficit, exactly as 09_buffer_sim does, then + # assert the ring can hold it. + deficit = np.maximum.accumulate(np.cumsum(rec - fill_per_frame)) + prefill = float(max(0.0, deficit.max())) + + for i, r in enumerate(rec): + # --- supply for this frame time, sector-quantised + avail = carry + fill_per_frame + sectors = int(avail // SECTOR) + got = sectors * SECTOR + carry = avail - got + + # --- placement: does this frame's arriving data cross the ring end? + if policy == "aligned": + # The writer will not start a record it cannot finish. Charge the + # hole when the NEXT record would not fit in the tail. + if wcur + r > ring: + holes.append(ring - wcur) + wcur = 0 + wcur += r + else: # split + end = wcur + r + if end > ring: + wcur = end - ring + # every byte that landed in the first MAXREC of the ring is + # mirrored into the shadow + shadow_bytes += min(wcur, maxrec) + else: + wcur = end + if wcur <= maxrec: + shadow_bytes += r + elif wcur - r < maxrec: + shadow_bytes += maxrec - (wcur - r) + + resident += got + if resident + 1e-9 < r: + late.append((i, float(r - resident))) + resident -= r + occ.append(resident) + + hole_mean = float(np.mean(holes)) if holes else 0.0 + usable = ring - hole_mean if policy == "aligned" else ring + copy_clk = shadow_bytes * COPY_CLK_PER_BYTE / max(1, n) + return dict(prefill=prefill, late=late, occ=np.array(occ), + holes=holes, hole_mean=hole_mean, usable=usable, + shadow_bytes=shadow_bytes, copy_clk_per_frame=copy_clk, + wraps=len(holes) if policy == "aligned" else None) + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("containers", nargs="*", + default=["tmp/s14_d5_all1500.dlx", + "tmp/rc_fr_singe_scsi_span.dlx"]) + ap.add_argument("--kbps", type=float, required=True, + help="delivered pipe, KB/s. REQUIRED, and deliberately has " + "no default: the delivery rate is a property of the " + "medium and this project has never measured it. The " + "figure that used to sit here was a user-supplied " + "'4 Mbps' with no provenance and was never a bus " + "measurement (FINDINGS 42.1); leaving it as a default " + "let table after table be scored against it without " + "anyone restating what it was.") + ap.add_argument("--ring", type=float, default=256.0, + help="ring size in KB (default 256, FINDINGS 21's sizing)") + a = ap.parse_args() + + FPS = 12 + print(f"ring {a.ring:.0f} KB sector {SECTOR} B " + f"audio {RC.AUDIO_KBPS} KB/s debited from the pipe\n") + + for path in a.containers: + if not os.path.exists(path): + print(f"{path}: MISSING -- skipped\n"); continue + d, rec = records(path) + maxrec = int(rec.max()) + ring = int(a.ring * 1024) + wire = rec.mean() * FPS / 1024 + RC.AUDIO_KBPS + + print(f"=== {path}") + print(f" {d.nframes} frames @ {d.fps}fps, record bytes " + f"min {rec.min():,} median {int(np.median(rec)):,} max {maxrec:,}") + print(f" wire demand {wire:.1f} KB/s " + f"(video {rec.mean()*FPS/1024:.1f} + audio {RC.AUDIO_KBPS}), " + f"including the u32 length and the 4-byte record pad") + + # A required prefill is only a startup cost if the window's MEAN demand + # is under the pipe. If the mean is over, the deficit grows for as long + # as the scene runs and the prefill this window reports is just how far + # it got in 120 frames -- no ring size fixes that, and quoting a KB + # figure for it would be the most flattering possible way to state a + # sustained overrun. FINDINGS 21's "zero prefill" never had to make + # this distinction because it ran far under the pipe it assumed. + if wire > a.kbps: + over = wire - a.kbps + print(f" !! SUSTAINED OVERRUN at the {a.kbps:.0f} KB/s pipe: " + f"demand exceeds supply by {over:.1f} KB/s on the MEAN, not " + f"on a burst.") + print(f" The deficit grows {over*1024/FPS:,.0f} B per frame " + f"for as long as the scene runs -- {over*1024*120/FPS/1024:.0f} " + f"KB over this 120-frame window, {over*60:.0f} KB per minute " + f"of play. Prefill below is where it got in 120 frames, NOT a " + f"startup cost that fixes it.") + + if maxrec > ring: + print(f" !! MAXREC {maxrec:,} > ring {ring:,}: no policy works. " + f"decode.s needs one whole record contiguous.\n") + continue + + # --- the requirement on the medium, which is the useful output, and + # the reason this tool takes no default rate. There is no measured + # pipe figure to score against (42.1), and the intent is to measure + # a BlueSCSI directly -- so the tool reports the THRESHOLD to + # measure against. The sweep is anchored to the container's own + # wire demand rather than to a list of fixed rates, so it stays + # meaningful for any container and privileges no constant. + print(f" {'pipe KB/s':>10} {'vs wire':>8} {'prefill KB':>11} " + f"{'records':>8} {'seek slack':>11}") + for mult in (0.90, 0.95, 1.00, 1.02, 1.05, 1.10, 1.25, 1.50, 2.00): + kbps = wire * mult + fill = (kbps - RC.AUDIO_KBPS) * 1024 / FPS + r = simulate(rec, fill, ring, "aligned", maxrec) + pf = r["prefill"] + # Branch-point seek slack, STATICALLY: with the ring FULL, how many + # frame times can the fill be zero before the next record is not + # resident? It is an upper bound and it assumes the premise that + # FINDINGS 51.3 took apart -- the ring is NOT full at a branch + # point, it is empty, and refilling it takes seconds of play. For + # the measured figure use tools/analysis/20_seek_slack.py, or the + # rig itself (tools/bench/pace_run.sh). Kept here as the ceiling + # this container's record sizes allow, which is what the rest of + # this row is about. + slack = (r["usable"] - maxrec) / rec.mean() + flag = "" + if pf + maxrec > r["usable"]: + flag = " <- does not fit the ring" + print(f" {kbps:>10.1f} {mult:>7.2f}x {pf/1024:>11.1f} " + f"{pf/rec.mean():>8.2f} {slack:>8.1f} fr{flag}") + # smallest pipe needing zero prefill, to 0.1 KB/s + lo, hi = wire, wire + 400 + for _ in range(40): + mid = (lo + hi) / 2 + f = (mid - RC.AUDIO_KBPS) * 1024 / FPS + if simulate(rec, f, ring, "aligned", maxrec)["prefill"] > 0: + lo = mid + else: + hi = mid + print(f" ZERO-PREFILL PIPE: {hi:.1f} KB/s " + f"({hi - wire:+.1f} KB/s over the wire demand, " + f"{100*hi/wire - 100:+.1f}%)") + print(f" ^ this is the number to measure a medium against. It is a " + f"REQUIREMENT, not a verdict.") + + # --- the policy trade, at the default pipe + fill = (a.kbps - RC.AUDIO_KBPS) * 1024 / FPS + print(f" wrap policy, at pipe {a.kbps:.0f} KB/s:") + for policy in ("aligned", "split"): + r = simulate(rec, fill, ring, policy, maxrec) + if policy == "aligned": + print(f" aligned wraps {r['wraps']:3} mean hole " + f"{r['hole_mean']/1024:6.1f} KB usable ring " + f"{r['usable']/1024:6.1f} KB " + f"({100*r['usable']/ring:.1f}%) CPU cost 0") + else: + pct = 100 * r["copy_clk_per_frame"] / (CPUHZ / FPS) + print(f" split shadow {r['shadow_bytes']/1024:8.1f} KB " + f"= {r['copy_clk_per_frame']:8.0f} clk/frame = " + f"{pct:.2f}% of the frame budget, forever RAM cost 0") + print() + + print("The `none` policy -- decoder wraps its own reads -- is not simulated.") + print("It has no RAM or copy cost and it is still the expensive one: the") + print("bounds test lands inside the exact instruction sequences FINDINGS") + print("30.4 and 40 fitted, so it does not cost a branch, it costs every span") + print("and per-block constant in the tree being re-measured. FINDINGS 28.3.") + + +if __name__ == "__main__": + main() diff --git a/tools/analysis/20_seek_slack.py b/tools/analysis/20_seek_slack.py new file mode 100644 index 0000000..bd4ee1a --- /dev/null +++ b/tools/analysis/20_seek_slack.py @@ -0,0 +1,146 @@ +"""Seek slack: how long a branch point can stop delivery (STATUS 4, FINDINGS 51). + +19_ring_stream.py asks whether a container ARRIVES in time, and prints one +"seek slack" column derived statically as (usable ring - maxrec)/mean record. +That is a capacity estimate and it quietly assumes the ring is full when the +seek happens. It is not, and the difference is the whole finding: + + A ring's slack is ACCUMULATED, not owned. It is built out of the surplus + between the pipe and the wire demand, at (pipe - wire) bytes per second, and + a seek spends all of it. How long a branch point can stall is a property of + the ring; how soon the NEXT branch point can be afforded is a property of the + surplus, and a bigger ring makes that one WORSE. + +This is the paced-rig model (tools/bench/stream.lua with DLX_PACE=1) written +independently, and it exists to be compared against it, not to replace it. The +rig drives a real 68000 through a real ring and is the measurement; this is the +cheap sweep that says where to point it. Where they disagree, the rig wins. + + python3 tools/analysis/20_seek_slack.py [container ...] --kbps R [R ...] + [--ring KB [KB ...]] + +`--kbps` is REQUIRED and takes no default, for the reason FINDINGS 50 gives. +""" +import sys, os, argparse +sys.path.insert(0, "tools/encoder") +import numpy as np +from dlx import DLX +import ratectl as RC + +SECTOR = 512 + + +def records(path): + d = DLX(path) + rec = np.array([4 + n + (-(4 + n) % 4) for _, n in d.frames], np.int64) + return d, rec + + +def paced_sim(rec, ring, fill_per_frame, ticks_per_frame=8): + """Paced-decoder ring sim. Returns TWO per-tick lookahead series. + + THE ANSWER IS BRACKETED TO ONE RECORD AND IS NOT SHARPER THAN THAT. At + these rates the pipe delivers almost exactly one record per frame slot, so + "how many records are resident at slot i" depends on whether you look before + or after that slot's delivery -- and the two answers differ by one, every + time. Sampled after, this agreed with the rig's ceiling in 33 of 35 cells; + sampled before, it was exactly one record lower in 33 of 35. Neither is + wrong. Picking the one that matched would have been fitting the model to + the measurement and then reporting the agreement as a cross-check, so both + are returned and the caller prints the range. The rig sits at the top of it. + + The producer is `aligned` (19_ring_stream.py): it will not start a record it + cannot finish before the end of the ring, and it will not place one over + bytes the decoder still owns. The decoder consumes exactly one record per + frame time and releases it whole. + + Sub-stepping matters. Delivery and consumption interleave inside a frame + time on the rig -- the producer runs on MAME's machine-frame notifier, ~5x + per 12fps slot -- and a model that delivers a whole frame's bytes at once + can place a record into space the decoder has not released yet, or refuse + one it has. Eight sub-steps is well past the point the answer stops moving. + """ + n = len(rec) + live = [] # [idx, off, len] still owned by the decoder + wcur, nsent, credit = 0, 0, 0.0 + lo, hi, ring_ref, rate_ref = [], [], 0, 0 + + def overlaps(off, ln): + return any(off < r[1] + r[2] and r[1] < off + ln for r in live) + + for i in range(n): + if nsent < n: + lo.append(sum(1 for r in live if r[0] >= i)) + for _ in range(ticks_per_frame): + credit += fill_per_frame / ticks_per_frame + while nsent < n: + r = int(rec[nsent]) + if credit < r: + rate_ref += 1 + break + w, hole = wcur, 0 + if w + r > ring: + w, hole = 0, ring - wcur + if overlaps(w, r): + ring_ref += 1 + break + # sector quantisation: a partial sector is not resident + credit -= r + live.append([nsent, w, r]) + wcur, nsent = w + r, nsent + 1 + if nsent < n: + hi.append(sum(1 for r in live if r[0] >= i)) + # the decoder consumed record i during the slot and releases it whole + live = [r for r in live if r[0] > i] + return np.array(lo), np.array(hi), ring_ref, rate_ref + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("containers", nargs="*", + default=["tmp/rc_fr_singe_scsi_span.dlx"]) + ap.add_argument("--kbps", type=float, nargs="+", required=True, + help="delivered pipe rates, KB/s. REQUIRED, no default " + "(FINDINGS 50): this project has never measured the " + "delivery pipe and a default is how the last unmeasured " + "one stayed load-bearing for five sessions.") + ap.add_argument("--ring", type=float, nargs="+", + default=[64, 96, 128, 192, 256, 384, 512]) + a = ap.parse_args() + FPS = 12 + + for path in a.containers: + if not os.path.exists(path): + print(f"{path}: MISSING -- skipped\n"); continue + d, rec = records(path) + wire = rec.mean() * FPS / 1024 + RC.AUDIO_KBPS + print(f"=== {path}: {d.nframes} frames @ {d.fps}fps, mean record " + f"{rec.mean()/1024:.1f} KB, wire {wire:.1f} KB/s") + print(f"{'ring KB':>8} {'pipe':>8} {'ceiling':>9} {'build s':>8} " + f"{'mean':>11} bound") + for ring_kb in a.ring: + ring = int(ring_kb * 1024) + if rec.max() > ring: + print(f"{ring_kb:>8.0f} maxrec {rec.max():,} does not fit") + continue + for kbps in a.kbps: + fill = ((kbps - RC.AUDIO_KBPS) * 1024 / FPS) if kbps > 0 else 1e12 + lo, hi, ring_ref, rate_ref = paced_sim(rec, ring, fill) + c_lo, c_hi = int(lo.max()), int(hi.max()) + build = int(np.argmax(hi >= c_hi)) if len(hi) else -1 + print(f"{ring_kb:>8.0f} {kbps:>8.0f} " + f"{f'{c_lo}-{c_hi}':>9} {build/FPS:>8.2f} " + f"{f'{lo.mean():.1f}-{hi.mean():.1f}':>11} " + f"{'ring' if ring_ref else 'rate'}") + # The surplus model, stated so it can be checked against the sweep + # above rather than believed: slack accrues at (pipe - wire) and a + # full ring holds `ceiling` records, so a branch point costs about + # ceiling*mean_record/(pipe - wire) seconds of play to earn back. + print() + print("Slack is accumulated, not owned. A bigger ring raises the ceiling AND") + print("lengthens the climb to it: the surplus (pipe - wire) is what fills it,") + print("and that is set by the encoder and the medium, not by the buffer.") + + +if __name__ == "__main__": + main() diff --git a/tools/analysis/21_iplrom_dmac.py b/tools/analysis/21_iplrom_dmac.py new file mode 100644 index 0000000..17538f3 --- /dev/null +++ b/tools/analysis/21_iplrom_dmac.py @@ -0,0 +1,255 @@ +#!/usr/bin/env python3 +"""What the X68000's own ROM programs into the DMAC -- read out of the bytes. + + python3 tools/analysis/21_iplrom_dmac.py [iplrom.dat] + +FINDINGS 48.4 / ROADMAP B3 left the single-address vs dual-address question +open for the disk, priced it at 242 KB/s and 0.69 dB, and blocked it on +sourcing `scsiexrom.bin` so its DMAC init could be disassembled. The same +question was open for AUDIO and nobody had asked it: ROADMAP P6 budgets ADPCM +at 7.8 KB/s and `11_cpu_budget.py` charges those bytes the DISK's per-byte +rate, which is a guess about a channel whose configuration was never read. + +It does not have to be a guess. **The IPL ROM is on this machine** -- MAME runs +the player rig with `-bios ipl10` -- and it programs all four HD63450 channels +itself. This script reads the configuration straight out of the ROM image and +decodes the MC68450 register fields, so every claim below is a byte at a named +address rather than a recollection about a chip. + +It is a GATE, not a report: each piece of evidence is (address, expected bytes, +what it means), and a mismatch exits non-zero. If a different ROM revision is +pointed at it, it says so instead of quietly decoding something else. + +SOURCED for the field layouts: MC68450 Direct Memory Access Controller, +Motorola, Jul 1989 (bitsavers) -- the same document FINDINGS 39 already cites +for the transfer timings in tools/analysis/buscost.py. + +NOTE THE LAYER: this is the ROM's own choice of configuration, read from the +shipping image. It is not a measurement of a running machine, and it is not +proof that a different configuration is impossible -- our player programs these +registers itself. It is evidence about what Sharp's engineers could get the +board to do, from the vendor, for these exact devices. +""" +import sys, os, argparse, hashlib + +BASE = 0xFE0000 # where the IPL ROM is mapped (and its 0xFF0000 alias) + +# The image this was decoded against. A different revision is a different +# machine's answer, so it is named rather than assumed. +KNOWN = { + "7fd4caabac1d9169e289f0f7bbf71d8e": + "IPL 1.0 (MAME x68000 -bios ipl10), 131,072 B", +} + +# --- MC68450 register map, by offset inside a channel's 0x40 block ---------- +REG = {0x00: "CSR", 0x01: "CER", 0x04: "DCR", 0x05: "OCR", 0x06: "SCR", + 0x07: "CCR", 0x0A: "MTC", 0x0C: "MAR", 0x14: "DAR", 0x1A: "BTC", + 0x1C: "BAR", 0x25: "NIV", 0x27: "EIV", 0x29: "MFC", 0x2D: "CPR", + 0x31: "DFC", 0x39: "BFC"} + +XRM = {0: "burst", + 1: "UNDEFINED", + 2: "cycle steal WITHOUT hold (bus released between operands)", + 3: "cycle steal with hold"} +DTYP = {0: "68000-compatible, EXPLICITLY addressed -> DUAL ADDRESS", + 1: "6800-compatible, EXPLICITLY addressed -> DUAL ADDRESS", + 2: "device with ACK, implicitly addressed -> SINGLE ADDRESS", + 3: "device with ACK and RDY, implicit -> SINGLE ADDRESS"} +DPS = {0: "8-bit port", 1: "16-bit port"} +PCL = {0: "status input", 1: "status input with interrupt", + 2: "start pulse", 3: "abort input"} +SIZE = {0: "byte", 1: "word", 2: "long word", 3: "byte, unpacked"} +CHAIN= {0: "none", 1: "UNDEFINED", 2: "array", 3: "linked array"} +REQG = {0: "auto-request at limited rate", 1: "auto-request at max rate", + 2: "EXTERNAL request (one operand per device request)", + 3: "auto-request first operand, external thereafter"} + + +def dcr(v): + return [f"XRM = {v>>6&3:02b} {XRM[v>>6&3]}", + f"DTYP = {v>>4&3:02b} {DTYP[v>>4&3]}", + f"DPS = {v>>3&1:b} {DPS[v>>3&1]}", + f"PCL = {v&3:02b} {PCL[v&3]}"] + + +def ocr(v): + return [f"DIR = {v>>7&1:b} " + + ("device -> memory (read)" if v & 0x80 else "memory -> device (write)"), + f"SIZE = {v>>4&3:02b} {SIZE[v>>4&3]}", + f"CHAIN= {v>>2&3:02b} {CHAIN[v>>2&3]}", + f"REQG = {v&3:02b} {REQG[v&3]}"] + + +def scr(v): + m = {0: "no count", 1: "increment", 2: "decrement", 3: "UNDEFINED"} + return [f"MAC = {v>>2&3:02b} memory address {m[v>>2&3]}", + f"DAC = {v&3:02b} device address {m[v&3]}"] + + +# --- the evidence ---------------------------------------------------------- +# (address, expected bytes, one-line description). Every register value quoted +# anywhere below comes out of one of these; nothing is typed in twice. +EV = [ + (0xFF0BEA, "49f900e84080197c00080004197c0005", + "boot: lea $E84080,a4 (ch2) ; DCR=$08 ; SCR=$05..."), + (0xFF0C2E, "49f900e840c0197c00800004197c00040006197c00050029197c0001002d" + "197c00050031197c00050039297c00e92003", + "boot: lea $E840C0,a4 (ch3, ADPCM) ; DCR=$80 SCR=$04 MFC=$05 CPR=$01 " + "DFC=$05 BFC=$05 DAR=$E92003"), + (0xFF0D8E, "0480060429052d0031054480460469056d027105", + "boot: the ch0/ch1 init TABLE, ten (offset,value) pairs, written by the " + "loop at $FF0CD8"), + (0xFF0CE4, "217c00e940030014217c00e960010054", + "boot: DAR ch0 = $E94003 (FDC data) ; DAR ch1 = $E96001 (SASI data)"), + (0xFF9A82, "13fc003200e840c5610a13fc000200e920014e75", + "IOCS ADPCM PLAY: OCR(ch3) = $32 ; then command $02 to $E92001"), + (0xFF9A5E, "13fc00b200e840c5612e13fc000400e920014e75", + "IOCS ADPCM RECORD: OCR(ch3) = $B2 ; then command $04 to $E92001"), + (0xFF9A96, "13fc00ff00e840c023c900e840cc33c200e840ca", + "IOCS ADPCM arm: CSR=$FF ; MAR = a1 ; MTC = d2 (DCR/SCR untouched)"), + (0xFF9944, "13fc00ff00e8404013fc00b200e84045601013fc00ff00e8404013fc003200" + "e8404523c900e8404c33c300e8404a13fc008000e840474e75", + "IOCS SASI: OCR(ch1) = $B2 read / $32 write ; MAR ; MTC ; CCR = $80"), +] + +ap = argparse.ArgumentParser() +ap.add_argument("rom", nargs="?", + default=os.path.expanduser("~/mame/roms/iplrom.dat")) +a = ap.parse_args() +if not os.path.exists(a.rom): + sys.exit(f"missing {a.rom} -- point this at the IPL ROM MAME boots the rig " + f"with (-bios ipl10).") +d = open(a.rom, "rb").read() +md5 = hashlib.md5(d).hexdigest() +print(f"{a.rom}: {len(d):,} B, md5 {md5}") +if md5 in KNOWN: + print(f" {KNOWN[md5]}\n") +else: + sys.exit(f"\nUNKNOWN ROM. Every field decoded below was read out of\n" + f" {list(KNOWN.values())[0]}\n" + f"and a different revision is a different machine's answer, not a " + f"detail. Add its\nmd5 to KNOWN only after re-reading the sites -- " + f"the addresses are revision-specific.") + +print("EVIDENCE -- each line is bytes at an address, not a recollection") +bad = 0 +for addr, hx, what in EV: + want = bytes.fromhex(hx) + got = d[addr - BASE: addr - BASE + len(want)] + ok = got == want + bad += not ok + print(f" {'OK ' if ok else 'FAIL'} ${addr:06X} {what}") + if not ok: + print(f" expected {want.hex()}\n got {got.hex()}") +if bad: + sys.exit(f"\nFAIL: {bad} evidence site(s) do not hold. The decode below " + "would be about\nsome other code, so it is not printed.") + +# The ch0/ch1 table, decoded from the bytes rather than restated. +tbl = d[0xFF0D8E - BASE: 0xFF0D8E - BASE + 20] +init = {} +for i in range(0, len(tbl), 2): + off, val = tbl[i], tbl[i + 1] + init[(off >> 6, off & 0x3F)] = val +init[(2, 0x04)] = 0x08 # from the inline moves at $FF0BEA +init[(2, 0x06)] = 0x05 +init[(2, 0x2D)] = 0x03 +init[(3, 0x04)] = 0x80 # ...and at $FF0C2E +init[(3, 0x06)] = 0x04 +init[(3, 0x2D)] = 0x01 + +DEV = {0: ("FDC", "$E94003"), 1: ("SASI", "$E96001"), + 2: ("IOCS _DMAMOVE (general purpose)", "set per call"), + 3: ("ADPCM MSM6258V", "$E92003")} +print("\nWHAT THE ROM PROGRAMS, per channel") +for ch in range(4): + name, dar = DEV[ch] + print(f"\n ch{ch} base $E840{ch*0x40:02X} {name} DAR = {dar}") + v = init[(ch, 0x04)] + print(f" DCR = ${v:02X}") + for line in dcr(v): + print(f" {line}") + v = init[(ch, 0x06)] + print(f" SCR = ${v:02X} " + " ; ".join(scr(v))) + print(f" CPR = ${init[(ch,0x2D)]:02X} channel priority " + f"({init[(ch,0x2D)]}, 0 = highest)") + +print("\nAND THE PER-TRANSFER OCR, written every time a transfer is armed") +for label, ch, v in (("ADPCM playback", 3, 0x32), ("ADPCM record", 3, 0xB2), + ("SASI write", 1, 0x32), ("SASI read", 1, 0xB2)): + print(f"\n {label:<15} ch{ch} OCR = ${v:02X}") + for line in ocr(v): + print(f" {line}") + +print(f""" +WHAT THIS SETTLES + +1. AUDIO IS DUAL ADDRESS, AND IT CANNOT HOLD THE BUS. ch3 DCR = $80: DTYP = + 00, explicitly addressed, so every ADPCM byte is a MEMORY READ FOLLOWED BY A + DEVICE WRITE -- not the single-address 5 clocks the disk debit is written in. + XRM = 10 is cycle steal WITHOUT hold and OCR REQG = 10 is external request, + so the DMAC arbitrates for the bus ONCE PER BYTE and gives it straight back. + There is no burst to amortise the arbitration over. + +2. THE PORT IS 8 BITS AND THE OPERAND IS A BYTE. DCR DPS = 0, OCR SIZE = 11. + One MSM6258V byte is two 4-bit samples, so 15.6 kHz is 7,812.5 BYTES/s and + 7,812.5 DMA REQUESTS/s -- the request count does not halve the way a 16-bit + port's would. That is the FINDINGS 43 unit trap, in the other stream. + +3. THE DISK CHANNEL IS PROGRAMMED IDENTICALLY, AND THAT IS THE BIGGER NEWS. + ch1 (SASI, DAR = $E96001) gets DCR = $80 and OCR = $B2 -- dual address, + 8-bit port, cycle steal WITHOUT hold, external request. Byte by byte, with a + full arbitration each time, exactly like the audio. ch0 (FDC) too. Sharp + programs every explicitly-addressed 8-bit device on this board the same way. + + This is not scsiexrom.bin and it does not close ROADMAP B3 -- a different + ROM drives a different SPC. But it is the same vendor, the same DMAC and the + same class of device, and it lands on the EXPENSIVE side of B3's 242 KB/s. + +4. AND IT IS OUTSIDE THE BRACKET THE PROJECT HAS BEEN COSTING P4 IN. + FINDINGS 42.4-42.6 brackets W, the clocks stolen per delivered byte, at + 5..12, and reports that W <= 6 fits 0/120 frames while W = 8 misses 47/120. + The ROM's own disk configuration costs 16..19. It is still true that the + player programs these registers itself and the choice is ours (42.6) -- but + the only worked example on the machine sits ABOVE the whole bracket, and + nothing in this tree has yet shown that a cheaper configuration is reachable + for an explicitly-addressed port. Treat W <= 12 as a REQUIREMENT ON THE + PLAYER'S DMAC PROGRAMMING, not as a range the hardware hands us. + +5. AUDIO OUTRANKS THE DISK AT THE ARBITER. CPR: FDC 0, ADPCM 1, SASI 2, + _DMAMOVE 3, lower being higher priority. When both channels want the bus in + the same slot, the ROM's arrangement serves ADPCM first. An audio byte is + never the thing that waits; a video byte is. +""") + +# --- what it costs --------------------------------------------------------- +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +import buscost as B + +ADPCM_HZ = 15625.0 # 8 MHz MSM6258V clock / 512 +ADPCM_BPS = ADPCM_HZ / 2 # 4-bit samples, two to a byte +FPS, CPUHZ = 12.0, 10e6 +lo = B.DMA_DUAL_BYTE_CLK + B.DMA_FRONT_CLK + B.DMA_BACK_CLK +hi = B.DMA_DUAL_BYTE_CLK + B.DMA_FRONT_CLK_WORST + B.DMA_BACK_CLK +bpf = ADPCM_BPS / FPS +print(f"WHAT IT COSTS, at the configuration above\n" + f" 15.6 kHz mono = {ADPCM_HZ:,.0f} samples/s = {ADPCM_BPS:,.1f} B/s " + f"= {ADPCM_BPS/1024:.2f} KiB/s\n" + f" (ratectl.AUDIO_KBPS is 7.8, which is this figure in DECIMAL kB; " + f"as KiB it is {ADPCM_BPS/1024:.2f})\n" + f" dual-address byte transfer {B.DMA_DUAL_BYTE_CLK} clk " + f"(read {B.DMA_READ_CLK} + write {B.DMA_WRITE_CLK}, Fig 4-25 sheet 4 note 2)\n" + f" + arbitration, EVERY byte {B.DMA_FRONT_CLK}..{B.DMA_FRONT_CLK_WORST}" + f" front + {B.DMA_BACK_CLK} back (sect 4.5.2.1/4.5.2.2)\n" + f" = {lo}..{hi} clocks per audio byte\n\n" + f" per frame at {FPS:g} fps: {bpf:,.1f} B costs {bpf*lo:,.0f}..{bpf*hi:,.0f} " + f"clocks of {CPUHZ/FPS:,.0f}\n" + f" = {100*bpf*lo/(CPUHZ/FPS):.2f}%..{100*bpf*hi/(CPUHZ/FPS):.2f}% of the " + f"frame, stolen from the 68000\n\n" + f" 11_cpu_budget.py charges audio --dma-clocks-per-byte, default 5, " + f"described\n as 'single-address, bus held, no drive wait'. The ROM says " + f"audio is neither\n single-address nor able to hold the bus, so that " + f"debit is {lo/5:.1f}x..{hi/5:.1f}x too small.\n" + f" In absolute terms it is small -- but it is small IN THE RESOURCE THE " + f"PROJECT IS\n SHORT OF, and it was being taken from the wrong side of " + f"an open question.") diff --git a/tools/analysis/buscost.py b/tools/analysis/buscost.py index 37bd64e..a5ffd15 100644 --- a/tools/analysis/buscost.py +++ b/tools/analysis/buscost.py @@ -63,12 +63,37 @@ DMA_CHAIN_CLK = 36 # Sect 4.5.2.1 front-end overhead 5 clocks best case, 8 worst; 4.5.2.2 # back-end 2 clocks best. Once per period of bus ownership, not per span. DMA_FRONT_CLK, DMA_BACK_CLK = 5, 2 +DMA_FRONT_CLK_WORST = 8 +# Fig 4-25 note 2 again, split out because the ADPCM channel needs the halves +# apart: a DMAC READ is 4 clocks and a WRITE is 5, on either bus width. A +# dual-address BYTE transfer is therefore one 4 and one 5. +DMA_READ_CLK, DMA_WRITE_CLK = 4, 5 +DMA_DUAL_BYTE_CLK = DMA_READ_CLK + DMA_WRITE_CLK # Fig 4-25 sheet 3, SINGLE ADDRESS: W/B READ 4 clocks, W/B WRITE 5 clocks. # A device->memory disk transfer is one memory WRITE = 5 clocks if the DMAC # holds the bus, or 5 + front + back = 12 if it arbitrates per word. # FINDINGS 5's long-standing 8 clk/word ESTIMATE sits inside that range. DMA_DISK_CLK_WORD_HELD, DMA_DISK_CLK_WORD_ARB = 5, 12 +# --- the ADPCM stream, as the IPL ROM actually programs it ----------------- +# READ OUT OF THE ROM, not recalled: tools/analysis/21_iplrom_dmac.py decodes +# the HD63450 registers Sharp's own IPL 1.0 writes, and gates on the bytes still +# being there. Channel 3, DCR = $80, OCR = $32 for playback: +# +# DTYP = 00 explicitly addressed -> DUAL ADDRESS (memory read, device write) +# DPS = 0 8-bit port -> one byte per operand +# XRM = 10 cycle steal WITHOUT hold, and REQG = 10 external request +# -> the DMAC arbitrates ONCE PER BYTE. No burst to amortise over. +# +# So an audio byte costs the dual-address transfer PLUS a full arbitration, +# every time -- unlike a disk record, which can at least be argued to hold the +# bus for a run of bytes. This is the number the audio side of the I/O debit +# should be denominated in; DISK_CLK_BYTE is not it. +ADPCM_SAMPLE_HZ = 15625.0 # MSM6258V, 8 MHz clock / 512 (the 15.6 kHz mode) +ADPCM_BYTES_PER_S = ADPCM_SAMPLE_HZ / 2 # 4-bit samples, two to a byte +ADPCM_CLK_BYTE_BEST = DMA_DUAL_BYTE_CLK + DMA_FRONT_CLK + DMA_BACK_CLK # 16 +ADPCM_CLK_BYTE_WORST = DMA_DUAL_BYTE_CLK + DMA_FRONT_CLK_WORST + DMA_BACK_CLK # 19 + # The 68000 cannot execute while another master owns the bus: no cache, and a # two-word prefetch queue that empties immediately. So DMA time is ADDITIVE to # CPU time, not overlapped -- which is what FINDINGS 35's flat debit assumed diff --git a/tools/bench/check.sh b/tools/bench/check.sh index 7826b27..da0e73c 100755 --- a/tools/bench/check.sh +++ b/tools/bench/check.sh @@ -10,6 +10,25 @@ cd "$(dirname "$0")/../.." python3 tools/encoder/extract.py 00020 tmp/fr_00020 12 crop mkdir -p tmp/snap_verify tmp/snap256 +# --------------------------------------------------------------------------- +# THE ONE PLACE THE RETIRED PIPE FIGURE STILL LIVES. Session 18 removed it as +# a default from every analysis tool and from tools/bench/stream.lua, because it +# was never a bus measurement -- a user-supplied "4 Mbps" with no provenance, +# 10% of SCSI-1's asynchronous rating (FINDINGS 42.1) -- and a default let table +# after table be scored against it without anyone restating what it was. +# +# It survives HERE and only here because the gate container was ENCODED with it, +# and every per-block and span constant in FINDINGS 41/43/45/49 is fitted to that +# container. Changing this number is not an edit, it is a re-encode plus a +# re-measurement of all of them. +# +# It is a CONTAINER RECIPE, not a claim about any medium. Do not read a delivery +# rate out of it, do not copy it into a tool, and do not add a default anywhere +# that would resurrect it. When the pipe is finally measured, this becomes an +# ordinary encoder setting and the comment goes. +GATE_SPAN_KBPS=488 +# --------------------------------------------------------------------------- + run() { # run