diff --git a/README.md b/README.md index a1e7473..1960ddf 100644 --- a/README.md +++ b/README.md @@ -126,6 +126,34 @@ at 488 KB/s a one-deep request queue gives away **6.8% of the pipe and underruns 59 of 120 frames**, a two-deep one gives away 3.4% and underruns none — on a container whose whole surplus over the wire is 8.7% (FINDINGS 55). +**The player runs off a real disc now, and PIO costs 87 clocks a byte.** +`src/player/xfer.i` answers the ring's request mailbox with a real READ(10) to a +real MB89352 instead of a host moving bytes at a modelled rate: 120 records, +4,488,588 B, **pixel-exact out of a 256 KB ring**, with a real mid-stream seek in +a second pass, and the **same 18 wraps** three different transports have now +produced. What it costs is the finding. Subtracting the same 120 frames run +twice gives **87.28 clocks per delivered byte**, and the 68000's own cycle table +for that loop says **87.15** — 0.2% apart, so the cost is the instruction stream +rather than the emulator's device model, and it is the first number this rig has +produced that a real board would also pay. At this container's mean record that +is **391.8% of a 12 fps frame**; the machine's own V-DISP clock agrees from the +other end at **2.57 fps**. Against the W ladder — 22.4% of a frame at 5 clocks a +byte, 85.3% at 19 — **the CPU doing the work itself is 4.6x the worst DMA +configuration this project has found and 17.5x the best.** Getting the DMAC to +hold the bus is no longer worth 9 against 19; it is worth 87 against either, and +it is the only thing left before a player (FINDINGS 58). + +**A record is not a sector, and the cheapest fix is a re-encode.** 117 of 120 +records start part way into a 512 B block, and reading whole blocks into the +ring corrupts the neighbouring records rather than merely wasting bytes — the +block loop reads with no bounds check. PIO absorbs this for free by simply not +storing the bytes outside the window, which is a property that disappears the +moment a DMA channel takes over. Priced three ways: windowed PIO is +1.34% on +the wire and cannot be done by a channel at all; a bounce buffer is +1.34% and +**+5 clocks on every delivered byte**, 22.4% of a frame; sector-aligning records +in the container is **+0.43% and zero clocks**. The last wins on both axes and +joins the re-encode bundle (FINDINGS 58.3). + **The scene graph is in, and the worst gap between two decision points is zero.** `tools/import/scenegraph.py` imports the arcade scene graph — 40 scenes, 516 sequences, 906 input windows — and 5.4% of the game's 612 branch transitions diff --git a/docs/FINDINGS.md b/docs/FINDINGS.md index 2796a4e..ad78a3f 100644 --- a/docs/FINDINGS.md +++ b/docs/FINDINGS.md @@ -5382,3 +5382,160 @@ watching that address. **Does move the premise of every delivery rig in the tree.** Until now the bytes came from a host. They now come off a disc, on the machine's own instructions, byte-exact at two different LBAs. + +--- + +## FINDINGS 58 — the player runs off the disc, and PIO costs 87 clocks a byte (session 25b/26) + +**Emulated.** MAME 0.277, `x68000 -exp1 cz6bs1 -ramsize 2M`, a blank +`scsiexrom.bin` on a private rompath (57.1's substitution, unchanged). No real +hardware ran. `./tools/bench/check.sh` was ALL GREEN before this and ALL GREEN +after, with two new stages. + +**ROADMAP P4b is DONE. P4a is not, and 58.2 is why it is now the item that +decides the project rather than one of two that do.** + +### 58.1 The seam closed: 120 pixel-exact frames, off a real volume + +`src/player/xfer.i` sits behind `src/player/ring.i`'s `XF_*` mailbox in place of +`tools/bench/stream.lua`'s modelled transport. `XF_GO` is answered by a real +READ(10) to a real MB89352 and `XF_ACK` is a word the 68000 bumps when the bytes +have landed — not one a host synthesises from emulated time. + +| | | +|---|---| +| records fetched by the 68000 | **120**, one READ(10) each | +| bytes into the ring | **4,488,588**, and the decode is **pixel-exact** | +| bytes off the disc | **4,548,608** — see 58.3 | +| ring | 256 KB, **18 wraps, 14.7 KB mean hole** | +| a real mid-stream seek | **pass 2 pixel-exact**, ring thrown away and rebuilt | + +**The 18 wraps are the load-bearing assertion, not the byte count.** They are +the same 18 the host producer produced in 49.4 and the same 18 `ring.i` +produced against a modelled transport in 55.4 — a *third* transport, same +tiling. `ring.i` is not supposed to be able to tell which side of the mailbox +answered it, and this is the number that says it could not. + +**The change above the seam is two `bsr`s.** One in `ring_poll`, one in +`ring_seek`'s quiet-wait — and the second is not optional. With the transport +*inside* the machine, the only thing that can retire an outstanding request is +that wait loop itself, so a seek issued with a request in flight spins forever +without it. A host transport retired it on its own time. That is exactly the +kind of difference the seam exists to hide, and it is the one it could not. + +### 58.2 What it costs: 87.28 clocks per delivered byte, and the number is portable + +`tools/bench/xfer_cost.sh` decodes the same 120 frames twice — same ring, same +`stream.s`, same `ring.i` placing every record — and changes only which side of +the mailbox answers: + +| | emulated | per frame | +|---|---|---| +| decode + `ring_poll` alone | 6.7737 s | **67.7%** of a 12 fps frame | +| ...with the real transport | 45.9516 s | **459.5%** | +| **the transport** | **391,779,000 clk** | **391.8%** | + +**= 87.28 clocks per delivered byte**, 86.13 per byte off the FIFO. + +**AND IT IS NOT MAME'S NUMBER, WHICH IS THE HALF THAT MATTERS.** The keep loop +in `src/player/scsi.i`, priced against the 68000's own cycle table — 12 patience +reload, 16 SSTS read, 10 `btst`, 10 branch, 20 DREG read to `(a1)+`, 8 `subq`, +10 branch — is **86 clocks**, and the FIFO also carries the dropped window bytes +of 58.3, which makes it **87.15 per delivered byte**. Measured 87.28. **0.2% +apart.** So the cost is the instruction stream and not a wait on MAME's SPC +model: it is a figure a real board would also pay, and it is the first number +this rig has produced that survives leaving the emulator. The residual, +0.13 +clk/B = 4,989 clocks per record, is the per-*command* cost — select, CDB, +status, message, `xf_service` — and it is the part that does not scale with the +record. + +**Against the ladder, in the same units** (clocks charged to the CPU per +delivered byte, at this container's 37,405 B mean record): + +| | | share of a 12 fps frame | +|---|---|---| +| `W` = 5 | single address, bus HELD | 22.4% | +| `W` = 9 | dual address, held | 40.4% | +| `W` = 12 | single address, arbitrated | 53.9% | +| `W` = 19 | dual address, arbitrated — the IPL ROM's own disk channel (52.5) | 85.3% | +| **PIO 87** | **this rig, measured** | **391.8%** | + +**The PIO transport is 4.6x the worst DMA configuration this project has found +and 17.5x the best.** P4a is not an optimisation of this. It is the difference +between a player and a slideshow, and it is now the *only* thing between the +tree and M2. + +**The player's own clock says the same thing, independently.** Self-paced off +V-DISP, the machine decoded 120 frames in **560 slots** of a 12 fps clock: +**2.57 fps**. That agrees with 12 / 4.595 = 2.61 from the cycle accounting +above, from a completely different instrument. + +**AND "UNDERRUNS: 0/120" IS VACUOUS IN THIS RUN.** A synchronous transport +cannot underrun by construction — a frame cannot start before its record has +landed, because the decoder *is* the transport. The counter that means anything +here is `NO IDLE`: **119 of 120 frames found their slot already open, worst +overrun 441 whole ticks.** `tools/bench/stream.lua` now prints that argument +next to the zero rather than leaving the zero to be quoted. This is the same +class of error as 49.7.2's free-running ring passing at 48 KB: a rig +configuration in which the failure it tests for cannot occur. + +**The resource that binds also flipped, and it is worth naming.** Against a +modelled 488 KB/s pipe this container was rate-bound and the ring never filled +(55.4). Here the ring **fills — 90 refusals for space** — while the decoder +starves. Not of bytes: of *time*. Every previous delivery result in this tree +was measured on a rig where the transport cost the CPU nothing. + +### 58.3 A record is not a sector, and the cheapest fix is a re-encode + +`ring.i` asks for a byte offset and a length, both 4-byte aligned (28.3); +a target answers in 512 B blocks. **On the gate container 117 of 120 records +start part way into a sector.** This is not a rounding nuisance: the bytes on +either side of a record belong to *other* records the decoder may still be +reading, and the block loop walks `a0` with no bounds check (49.2), so a +transport that reads whole sectors straight into the ring **corrupts its +neighbours** — wrong pixels, not a fault. + +`tools/analysis/26_sector_align.py` prices the three ways out: + +| | wire | clocks | +|---|---|---| +| **A. windowed PIO** — read the covering sectors, store only the record | **+1.34%** | 86/B on every byte off the FIFO, and **a DMAC cannot do it at all** | +| **B. bounce buffer** — DMA whole sectors elsewhere, then copy | +1.34% | **+5/B on every delivered byte** = 22.4% of the frame, on top of `W` | +| **C. sector-aligned records** — pad to 512 in the container | **+0.43%** | **zero** | + +A is what shipped in `scsi.i` and what 58.1 ran, and it is free *only* because +the CPU is already touching every byte — the property that disappears the moment +P4a succeeds. B is the cost `aligned` was chosen over `split` to avoid (49.3) +arriving by a different door, and on every byte instead of on a wrap. + +**C wins on both axes**: it is cheaper on the wire than A and B by 0.91 points +of the payload (40,940 B on this scene), and it is the only one of the three a +DMA channel can run without a copy. What it costs is a **container revision** — +a re-encode plus a re-measurement of every constant fitted to the gate +container. That is the class of change ROADMAP already has bundled with P2's +other half (reserve index 0 as black) and 55's two open re-encode questions. +**It should join that bundle**, and P4a should be attempted against a +sector-aligned container rather than against this one. + +It also grows the largest record from 40,984 to 41,472 B, which a 256 KB ring +still holds six times over — so it costs nothing in ring size. + +### 58.4 What this does and does not move + +**Does not move `W`.** Again, and for the same reason 57.6 gives. Nothing here +programmed a DMAC channel; `xf_service` is the CPU standing in for one, through +the DMAC's own door (57.3). + +**Does not measure a delivery rate,** and the rig now refuses to be asked. A +`DLX_XFER=scsi` run rejects a `--kbps` argument outright rather than ignoring +it, and `stream.lua` suppresses `CHANNEL IDLE`, `DEADLINE` and `REQUIRED +PREFILL` instead of printing them as zeros — a zero there reads as "the channel +never stopped", which would be a claim about a medium this tree has never timed. + +**Does close P4b, and does hand P4a a sharper question than it had.** Before +this session, "get the DMAC to hold the bus" was worth 9 clk/B against 19. It is +now worth 87 against either. + +**Does put a real number on the layer below every previous delivery result.** +Everything in FINDINGS 49, 51 and 55 was measured with the bytes arriving free. +They are not free, and 58.2 is the first measurement of what they cost. diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index d081cd4..01cf0bc 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -6,6 +6,8 @@ Amended end of session 22: P3 done (FINDINGS 54). Amended end of session 23: P5 done (FINDINGS 55). Amended end of session 24: G1 done (FINDINGS 56). Amended end of session 25: P4 HALF done (FINDINGS 57). +Amended end of session 26: P4b done, P4a is the last open item before M2 +(FINDINGS 58). **THE COMPLETION TARGET IS M3, THE VERTICAL SLICE** (USER DECISION): one scene tree — a decision point, two outcomes, a death clip — with audio, streaming from @@ -124,7 +126,18 @@ the words read back **out of the palette registers**. 9.70 ms per scene, plus **What is left is the other half of the sentence: reserve index 0 as black with `I = 0` (23.4).** That is ENCODER-side, it changes the container, and it moves every constant fitted to the gate container, so it is a re-encode plus a -re-measurement rather than an edit. Until then the letterbox gets the palette's +re-measurement rather than an edit. + +**THE RE-ENCODE BUNDLE, collected here because it is now four items and they +share one re-measurement.** Nothing below is worth a container revision on its +own; together they are one: + 1. reserve palette index 0 as black, `I = 0` (23.4, this item); + 2. `--spans all` as the default (E2, and it is the loaded lever on the byte + side); + 3. re-derive span selection jointly with `lam` (E3); + 4. **sector-align every record (58.3, new in session 26)** — +0.43% on the + wire, zero clocks, and it is what lets P4a's DMA channel write straight + into the ring with no window and no bounce copy. Until then the letterbox gets the palette's closest thing to black (index 255 on the gate container); `load.i` reports whichever index that is and needs no change when it becomes 0. @@ -160,7 +173,8 @@ period from `htotal - 8`), so the tree's "1/55.46 s granularity" was 1/56.69 s throughout. No 68000 cycle figure moves — the CPU clock is unrelated to the screen — but anything paced by the raster does. 54.5. -**P4. Real transport. HALF DONE, session 25 — FINDINGS 57.** +**P4. Real transport. P4b DONE, session 26 — FINDINGS 58. P4a OPEN, and it is +now the ONLY thing between this tree and M2.** ~~Drive the MB89352 instead of a host file.~~ `src/player/scsi.i` selects a SCSI target and issues READ(10) on the 68000, with no IOCS and no host in the transfer path: **4,096 B from LBA 0 and 2,048 B from LBA 1000, both byte-exact** @@ -184,12 +198,48 @@ cannot then distinguish a CPU-driven byte from a DMAC-driven one at that address**. "The DMAC held the bus" needs evidence that does not come from watching `$EA0015`. -**P4b. `scsi.i` behind `ring.i`'s `XF_*` mailbox**, in place of `stream.lua`'s -modelled transport, gated on the same pixel-exact 120 frames. The seam already -exists: `XF_ACK` stops being a word a host synthesises from emulated time and -becomes one the 68000 bumps when a transfer lands. +~~**P4b. `scsi.i` behind `ring.i`'s `XF_*` mailbox.**~~ **DONE, session 26 — +FINDINGS 58.** `src/player/xfer.i` answers the mailbox with a real READ(10) per +record: **120 records, 4,488,588 B, pixel-exact, out of the same 256 KB ring, +with a real mid-stream seek in a second pass**. The tiling is the SAME 18 wraps +and 14.7 KB mean hole that 49.4's host producer and 55.4's modelled transport +produced — a third transport, same placement, which is the assertion that +`ring.i` could not tell which side of the seam answered it. The change above the +seam is two `bsr`s, and the one in `ring_seek`'s quiet-wait is not optional: +with the transport inside the machine, that loop is the only thing that can +retire an outstanding request. -`W` **did not move by one clock** and is still the largest open number here. +**What it cost is the finding, and it re-prices P4a.** `tools/bench/ +xfer_cost.sh` subtracts the same 120 frames run twice and gets **87.28 clocks +per delivered byte** — against the 68000's own cycle table for the loop, which +says **87.15**. **0.2% apart**, so it is the instruction stream and not MAME's +device model, and it is therefore the first number this rig has produced that +survives leaving the emulator. At this container's 37,405 B mean record that is +**391.8% of a 12 fps frame**, and the machine's own V-DISP clock agrees from the +other end: **2.57 fps**. + + W = 5 single address, bus HELD ............................ 22.4% + W = 9 dual address, held .................................. 40.4% + W = 12 single address, arbitrated .......................... 53.9% + W = 19 dual address, arbitrated -- the IPL ROM's own (52.5) . 85.3% + PIO 87 MEASURED, session 26 ................................ 391.8% + +**So P4a is worth 4.6x the worst DMA configuration in this tree and 17.5x the +best**, where before this session it was worth 9 against 19. `W` itself **did +not move by one clock** and is still the largest open number — but what depends +on it just got much larger. + +**One more thing P4a inherits (58.3).** A record is not a sector: 117 of 120 +start part way into one. PIO absorbs that for free because the CPU is already +touching every byte and simply does not store the ones outside the window — a +property that **disappears the moment the DMAC takes over**, because a channel +writes a contiguous run and cannot drop bytes. The three ways out price as ++1.34% wire and no DMA (windowed PIO), +1.34% wire and **+5 clk/B of copy** +(bounce buffer, which is exactly the cost `aligned` was chosen over `split` to +avoid), or **+0.43% wire and zero clocks** (sector-aligned records in the +container). The last one wins on both axes and is a **re-encode**; see the +bundle under P2. **P4a should be attempted against a sector-aligned container, +not against this one.** *(original item, still the standing description of the `W` question:)* Drive the MB89352 instead of a host file. **Session 23 @@ -328,12 +378,12 @@ Listed for completeness; past M3 these are scope, not risk. **Gated by E4.** - **E4. `H.build` k-means**, 51 s of a 55 s run, once per scene. The thing to attack before C1, and not anything in the per-frame path (27.6). -- **E2. `--spans all` as default.** Still a recommendation, not a measurement +- **E2. `--spans all` as default.** *(re-encode bundle item 2; see P2.)* Still a recommendation, not a measurement (43.6.1), and the only loaded lever on the encoder's byte side (44.3). **It spends every profitable byte, which raises `wire`, which shrinks `pipe - wire`, which lengthens the refill climb after every branch.** That interaction is not priced, and M3 is where it becomes measurable. -- **E3. Re-derive span selection jointly with `lam`** (39.3). +- **E3. Re-derive span selection jointly with `lam`** (39.3). *(bundle item 3.)* - **C2. Framing** — crop vs squash vs wide (FINDINGS 12). Needs an eyeball against arcade reference, not a measurement. Cheap; blocks only final encodes. - **C3. Disk image packaging**, ~1.09 GiB at the candidate rate. @@ -345,12 +395,12 @@ Listed for completeness; past M3 these are scope, not risk. ``` B1 seek+rate ─┐ -B3 DTYP ──────┴─> P4 transport ─┐ - ├─> M2 ─> M3 (COMPLETION TARGET) ─> M4 -P1 P2(half) P3 P5 P7 ───────────┘ ^ - │ -P6 (bus cost DONE, 52) ──────────────────┤ -G1 scene graph (DONE, 56) ───────────────┘ +B3 DTYP ──────┴─> P4a DMA HOLDS THE BUS ──┐ (P4b DONE, 58: the ring is + │ filled off a real volume, and +P1 P2(half) P3 P4b P5 P7 ─────────────────┤ PIO costs 87 clk/B) + ├─> M2 ─> M3 (TARGET) ─> M4 +P6 (bus cost DONE, 52) ───────────────────┤ +G1 scene graph (DONE, 56) ────────────────┘ B2 blanking ─> (page 1; do not pre-build on it) ``` diff --git a/docs/STATUS.md b/docs/STATUS.md index 277d374..f8df448 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -1,3 +1,120 @@ +# Status & next-session handoff — end of session 26 (2026-08-24) + +## Session 26: the player runs off the disc, and PIO costs 87 clocks a byte + +**Green light first and last: `./tools/bench/check.sh` was ALL GREEN before any +of this and ALL GREEN after**, plus two new stages. + +**ROADMAP P4b is DONE. P4a is the only item left before M2. FINDINGS 58.** +**Emulated** — MAME 0.277, `x68000 -exp1 cz6bs1 -ramsize 2M`. No real hardware +ran. + +**1. THE SEAM IS CLOSED.** `src/player/xfer.i` sits behind `src/player/ring.i`'s +`XF_*` mailbox in place of `tools/bench/stream.lua`'s modelled transport. `XF_GO` +is answered by a real READ(10) to a real MB89352; `XF_ACK` is a word the 68000 +bumps when the bytes have landed, not one a host synthesises from emulated time. + +| | | +|---|---| +| 120 records fetched by the 68000 | one READ(10) each | +| 4,488,588 B into a 256 KB ring | **pixel-exact** over all 120 frames | +| a real mid-stream seek, pass 2 | **pixel-exact** after it too | +| 18 wraps, 14.7 KB mean hole | the SAME tiling as 49.4 and 55.4 | + +The 18 wraps are the load-bearing assertion. A host producer made them in +session 18, `ring.i` made them against a modelled transport in session 23, and a +real SPC makes them now — **`ring.i` cannot tell which side of the mailbox +answered it**, which is what makes this a test of the transport rather than of a +new rig. 58.1. + +**The change above the seam is two `bsr`s**, and the one in `ring_seek`'s +quiet-wait is not optional: with the transport *inside* the machine, that loop +is the only thing that can retire an outstanding request, so without it a seek +issued with a request in flight spins forever. A host transport retired it on +its own time. That is exactly the kind of difference the seam exists to hide, +and it is the one it could not. + +**2. WHAT IT COSTS, AND THE NUMBER LEAVES THE EMULATOR.** `tools/bench/ +xfer_cost.sh` runs the same 120 frames twice — same ring, same `stream.s`, same +`ring.i` placing every record — and changes only which side answers: + +| | emulated | per frame | +|---|---|---| +| decode + `ring_poll` alone | 6.7737 s | 67.7% of a 12 fps frame | +| ...with the real transport | 45.9516 s | **459.5%** | +| **the transport alone** | **391,779,000 clk** | **391.8%** | + +**87.28 clocks per delivered byte.** The keep loop in `scsi.i`, priced against +the **68000's own cycle table**, is 86 clocks, and the FIFO also carries the +dropped window bytes of 58.3 — **87.15 predicted, 87.28 measured, 0.2% apart.** +So this is the instruction stream and **not** MAME's SPC model: **the first +number this rig has produced that a real board would also pay.** The machine's +own V-DISP clock agrees from the other end — 120 frames in 560 slots of a 12 fps +clock is **2.57 fps**. 58.2. + +**Against the ladder** (clocks charged to the CPU per delivered byte): +`W`=5 held is 22.4% of a frame, `W`=9 is 40.4%, `W`=19 — the IPL ROM's own disk +channel, 52.5 — is 85.3%, and **PIO is 391.8%**. **P4a is worth 4.6x the worst +DMA configuration in this tree and 17.5x the best**, where before this session +it was worth 9 against 19. `W` itself did not move by one clock. + +**3. "UNDERRUNS: 0/120" IS VACUOUS HERE, and the rig now says so in place.** A +synchronous transport cannot underrun by construction — a frame cannot start +before its record has landed, because the decoder *is* the transport. The +counter that means something is `NO IDLE`: **119 of 120 frames found their slot +already open, worst overrun 441 whole ticks.** Same class of error as 49.7.2's +free-running ring passing at 48 KB: a rig configuration in which the failure +being tested for cannot occur. **The binding resource flipped too** — against a +modelled 488 KB/s pipe this container was rate-bound and the ring never filled; +here the ring **fills, 90 refusals for space**, while the decoder starves for +*time*. + +**4. A RECORD IS NOT A SECTOR, and the cheapest fix is a re-encode.** 117 of 120 +records start part way into a sector. Reading whole sectors into the ring +**corrupts the neighbouring records** — the block loop has no bounds check +(49.2) — so `scsi.i` reads the covering sectors and stores only the window. +That is free in PIO because the CPU is already touching every byte, **and it +stops being free the moment P4a succeeds**: a channel writes a contiguous run +and cannot drop bytes. `tools/analysis/26_sector_align.py` prices the three ways +out: windowed PIO **+1.34% wire, no DMA**; bounce buffer **+1.34% wire and ++5 clk/B of copy** (22.4% of the frame — the cost `aligned` was chosen over +`split` to avoid, arriving by a different door); **sector-aligned records ++0.43% wire and zero clocks.** The last wins on both axes and is a container +change. **ROADMAP now carries a four-item re-encode bundle** (index-0 black, +`--spans all`, joint `lam`, sector alignment) and **P4a should be attempted +against a sector-aligned container, not against this one.** 58.3. + +**5. The rig refuses to be asked for a rate.** `DLX_XFER=scsi` rejects a +non-zero `--kbps` outright rather than ignoring it, and `stream.lua` suppresses +`CHANNEL IDLE`, `DEADLINE` and `REQUIRED PREFILL` rather than printing them as +zeros — a zero there reads as "the channel never stopped", which would be a +claim about a medium this tree has never timed. + +**One rig bug found and fixed in the same sitting:** the host's `nsent` is now +rebased at every seek. `XS_NXFER` is cumulative, so on a second pass it was +already >= `nframes` and the slack sampling never fired — the second pass +reported a ceiling of 0 frames and a build time of **-1 ticks**, which is an +empty series printing as a result. + +**New in the tree:** `src/player/xfer.i`, `tools/bench/mkvol.sh` (one copy of +the volume layout, shared by both SCSI rigs), `tools/bench/xfer_cost.sh`, +`tools/analysis/26_sector_align.py`, `DLX_XFER=scsi` and `DLX_SECONDS` in +`pace_run.sh`, and two `check.sh` stages. `src/player/scsi.i` gained +`sc_in_data` (the three-loop windowed receive) and `scsi_read_win`; `ring.i` +gained the two transport hooks; `stream.s` includes `scsi.i` and `xfer.i`. + +**No decoder code changed.** `decode.bin` is still 1,296 B at the same MD5. + +**Next: P4a, and it is now the whole of what is left before M2.** A DMAC +configuration that HOLDS THE BUS, shown by evidence that does not come from +watching `$EA0015` (57.3: with `exown` asserted, MAME cannot tell a CPU-driven +byte there from a DMAC-driven one). Do it against a sector-aligned container if +the re-encode bundle has been run by then, because 58.3 is otherwise a bounce +copy at +5 clk/B. **P2's remaining half** is unchanged and is now item 1 of +that bundle. + +--- + # Status & next-session handoff — end of session 25 (2026-08-24) ## Session 25: the 68000 reads the disc itself, and P4 was never blocked diff --git a/src/player/ring.i b/src/player/ring.i index 05e5c82..8cbd40f 100644 --- a/src/player/ring.i +++ b/src/player/ring.i @@ -171,6 +171,17 @@ ring_seek: cmp.l XF_GO.l,d1 beq.s .quiet addq.l #1,SK_WAIT.l + bsr xf_service ; the transport hook, and here it is not + ; optional: with the transport INSIDE + ; the machine (src/player/xfer.i) the + ; only thing that can retire the + ; outstanding request is this loop, so + ; without it a seek issued with a + ; request in flight spins forever. A + ; host transport retired it on its own + ; time, which is exactly the kind of + ; difference the seam exists to hide and + ; this one it could not. bra.s .wait .quiet: move.l XF_GO.l,NRETIRE.l ; whatever landed belongs to the scene @@ -195,6 +206,12 @@ ring_seek: ring_poll: movem.l d0-d3/a0-a1,-(sp) addq.l #1,N_POLL.l +; ---- 0. the TRANSPORT, if it lives in this machine. src/player/xfer.i answers +; at most one outstanding request per call and preserves every register; +; with XF_SCSI = 0 it is a tst and a branch, and the host is the transport +; exactly as it was in FINDINGS 55. It goes BEFORE the retire step so that +; a transfer completed here is published in the same poll. + bsr xf_service ; ---- 1. retire. The descriptor is written BEFORE the count that advertises ; it, which is the same order tools/bench/stream.lua used and the reason diff --git a/src/player/scsi.i b/src/player/scsi.i index 30e2d76..12b60d7 100644 --- a/src/player/scsi.i +++ b/src/player/scsi.i @@ -132,6 +132,33 @@ SC_PH = $18208 ; u32 phase we were in when it went wron SC_CDB = $18210 ; 12 B command block, built here SC_MSG = $1821C ; 4 B message byte, either direction +; ---- THE RECORD WINDOW, and why a transport needs one. ROADMAP P4b. +; src/player/ring.i asks for a RECORD: a byte offset into the scene's frame +; stream and a length, both of them 4-byte aligned and neither of them a +; multiple of 512. A SCSI target deals in BLOCKS. On the gate container 117 +; of 120 records start part way into a sector, so a transport that reads only +; whole sectors delivers the record plus up to 511 bytes in front of it and up +; to 511 behind, and those neighbouring bytes belong to records the decoder may +; still be reading -- the block loop walks a0 with no bounds check (49.2), so +; landing them in the ring is a corruption, not a waste. +; +; IN PIO THE FIX IS FREE, and that is the only reason this is affordable here: +; the CPU is already touching every byte, so it simply does not STORE the ones +; outside the window. SC_WSKIP bytes are pulled from the FIFO and dropped, the +; next SC_WKEEP are stored, the rest are pulled and dropped. Three loops rather +; than one steered loop, deliberately: the middle one is then byte-for-byte as +; tight as the un-windowed sc_in_pio, so the per-byte cost this rig reports is +; the transport's and not the window's. +; +; UNDER A DMAC IT IS NOT FREE, and that is P4a's problem arriving early. A +; channel writes a contiguous run to a contiguous address; it cannot be told to +; drop the first 300 bytes. So when the data phase moves to the HD63450 the +; choice is a bounce buffer plus a copy of every byte (the cost `aligned` was +; chosen to avoid, 49.3) or sector-aligned records in the container -- which is +; a re-encode. 57 measured which is cheaper; see FINDINGS 58.3. +SC_WSKIP = $18220 ; u32 bytes to drop before the window +SC_WKEEP = $18224 ; u32 bytes of window to store + ; ---- a TRACE, because a SCSI bring-up cannot be debugged from one error code. ; Four registers at each interesting instant: SSTS, PSNS, INTS, SERR. MAME's ; SCMD_CMD_TRANSFER is a NO-OP unless SSTS_INIT_CONNECTED is set -- it `break`s @@ -395,6 +422,84 @@ sip3: move.b SC_DREG,(a1)+ move.l (sp)+,d0 rts +; ---------------------------------------------------------------- sc_in_data +; Receive d1 bytes in phase d2, storing only the WINDOW: drop SC_WSKIP, store +; SC_WKEEP at (a1), drop whatever is left. This is the DATA IN path; STATUS and +; MESSAGE IN keep sc_in_pio, which is one byte and has no window. +; +; THREE LOOPS, NOT ONE STEERED LOOP. A single loop with a `which third am I in` +; test per byte would cost ~20 clocks on every byte of every record, and the +; number this rig exists to produce is the transport's per-byte cost -- so the +; middle loop is byte-for-byte sc_in_pio's and the window is paid for once at +; each boundary instead of once per byte. +; +; A SPLIT DATA PHASE WOULD RE-SKIP. The counters are re-read from memory on +; every entry, so a target that broke one READ(10) across two DATA IN phases +; would drop the head of the second phase as well. This one does not split -- +; the same limitation sc_in_pio's caller already carries -- and the fix is the +; same one: d5 has to become what each phase actually delivered. +sc_in_data: + movem.l d6-d7,-(sp) + move.b d2,SC_PCTL + move.l d1,d0 + bsr sc_settc + move.b #SCMD_XFER,SC_SCMD + move.l #7,SC_TAG.l ; 7 = TRANSFER issued for an IN phase + bsr sc_snap + move.l SC_WSKIP.l,d6 + move.l SC_WKEEP.l,d7 + sub.l d6,d1 + sub.l d7,d1 ; d1 = trailing bytes to drop + tst.l d6 + beq.s .keep +.drop1: move.l #SC_PATIENCE,d3 +.dw1: move.b SC_SSTS,d0 + btst #0,d0 ; DREG EMPTY -- wait for a byte + beq.s .dg1 + subq.l #1,d3 + bne.s .dw1 + bra .tmo +.dg1: tst.b SC_DREG ; popped and thrown away + subq.l #1,d6 + bne.s .drop1 +.keep: tst.l d7 + beq.s .tail +.keep1: move.l #SC_PATIENCE,d3 +.kw1: move.b SC_SSTS,d0 + btst #0,d0 + beq.s .kg1 + subq.l #1,d3 + bne.s .kw1 + bra .tmo +.kg1: move.b SC_DREG,(a1)+ + subq.l #1,d7 + bne.s .keep1 +.tail: tst.l d1 + beq.s .fin +.tail1: move.l #SC_PATIENCE,d3 +.tw1: move.b SC_SSTS,d0 + btst #0,d0 + beq.s .tg1 + subq.l #1,d3 + bne.s .tw1 + bra.s .tmo +.tg1: tst.b SC_DREG + subq.l #1,d1 + bne.s .tail1 +.fin: movem.l (sp)+,d6-d7 + move.l #8,SC_TAG.l ; 8 = every byte taken from the FIFO + bsr sc_snap + bsr sc_xferend + move.l d0,-(sp) + move.l #9,SC_TAG.l + bsr sc_snap + move.l (sp)+,d0 + rts +.tmo: movem.l (sp)+,d6-d7 + move.l #SCE_TIMEOUT,SC_ERR.l + moveq #-1,d0 + rts + ; ---------------------------------------------------------------- scsi_read ; READ(10) of d4 blocks from LBA d3 into (a1). READ(10) rather than READ(6) ; because a 21-bit LBA and a 256-block ceiling are limits this container will @@ -410,6 +515,21 @@ sip3: move.b SC_DREG,(a1)+ ; finds is both shorter and correct, and it is what the target is entitled to. scsi_read: movem.l d3-d5/a1,-(sp) + move.l d4,d5 + lsl.l #8,d5 + lsl.l #1,d5 ; blocks * 512 + clr.l SC_WSKIP.l ; no window: keep the whole transfer + move.l d5,SC_WKEEP.l + bra.s scr_body +; ---- the same read, delivering only SC_WSKIP..SC_WSKIP+SC_WKEEP of it. The +; caller sets the two words; everything else is identical, which is the point -- +; a windowed read and a whole one must not be able to differ in the protocol. +scsi_read_win: + movem.l d3-d5/a1,-(sp) + move.l d4,d5 + lsl.l #8,d5 + lsl.l #1,d5 +scr_body: ; ---- the command block, built before anything is on the bus lea SC_CDB.l,a0 move.b #$28,(a0)+ ; READ(10) @@ -429,10 +549,6 @@ scsi_read: move.b d0,(a0)+ ; would have emitted bits 31..24/23..16 move.b d4,(a0)+ ; of a count that lives in 15..0. clr.b (a0)+ - ; ---- d5 = bytes of data still expected - move.l d4,d5 - lsl.l #8,d5 - lsl.l #1,d5 ; blocks * 512 clr.l SC_STAT.l bsr sc_select tst.l d0 @@ -482,7 +598,7 @@ scr_din: ; from sp as d3,d4,d5,a1 -- a1 is at 12. move.l d5,d1 moveq #PH_DATAIN,d2 - bsr sc_in_pio + bsr sc_in_data tst.l d0 bmi scr_out bra scr_ph diff --git a/src/player/stream.s b/src/player/stream.s index 0712a49..fce9ca5 100644 --- a/src/player/stream.s +++ b/src/player/stream.s @@ -113,6 +113,11 @@ start: ; interrupt source first. tst.l RINGOWN.l beq.s noring +; ---- the transport, before the producer that will ask it for something. +; ring_init ends in a ring_seek and a seek WAITS for the channel to go quiet, so +; the thing that makes the channel quiet has to exist first. With XF_SCSI = 0 +; this brings up nothing and the host is the transport (FINDINGS 55). + bsr xf_init bsr ring_init tst.l d0 bpl.s noring @@ -335,3 +340,5 @@ desync: move.l #$EE,FLAG.l include "src/player/frame.i" include "src/player/clock.i" include "src/player/ring.i" + include "src/player/scsi.i" + include "src/player/xfer.i" diff --git a/src/player/xfer.i b/src/player/xfer.i new file mode 100644 index 0000000..11b6185 --- /dev/null +++ b/src/player/xfer.i @@ -0,0 +1,148 @@ +; ---------------------------------------------------------------- xfer.i +; THE TRANSPORT BEHIND THE MAILBOX. ROADMAP P4b. +; +; src/player/ring.i has always ended at a seam: it decides which record to +; fetch, where in the ring to put it and when that is safe, writes those four +; words into XF_SLOT and bumps XF_GO, and then polls XF_ACK. On the other side +; of that seam, until now, was tools/bench/stream.lua -- a host moving bytes at +; a MODELLED rate, with XF_ACK synthesised out of emulated time. A player has +; no host. This file is the other side: XF_GO is answered by src/player/scsi.i +; issuing a real READ(10) to a real MB89352, and XF_ACK is a word the 68000 +; bumps when the bytes have landed. +; +; NOTHING ABOVE THE SEAM CHANGED, and that is deliberate for the same reason it +; was in sessions 22 and 23: ring.i cannot tell which transport answered it, so +; a green run here is a test of THIS file rather than of a new producer. The +; two hooks in ring.i (one in ring_poll, one in ring_seek's quiet-wait) are the +; whole of the change on that side, and with XF_SCSI = 0 they are a tst and a +; branch. +; +; IT IS SYNCHRONOUS, AND THAT IS NOT A SHORTCUT -- IT IS THE FINDING. The +; modelled transport overlapped: a request issued at time t landed at t + len/ +; rate while the 68000 got on with decoding, which is what a DMAC channel does. +; Here the CPU moves every byte itself through $EA0015 (57.3: a PIO write to +; that address is discarded, so even "PIO" runs the SPC in DMA mode with the CPU +; standing in for the channel), so `bsr xf_service` does not start a transfer, +; it PERFORMS one. A two-deep request queue therefore buys nothing at all: the +; second slot is filled and drained by the same instruction stream that would +; have been decoding. FINDINGS 55.3's whole result -- that a one-deep queue +; gives away 6.8% of the pipe -- is about a transport that runs in parallel with +; the CPU, and this one does not. +; +; So what this file is FOR is not to be the shipping transport. It is to make +; the shipping transport's cost measurable: the same 120 pixel-exact frames, +; delivered by the machine off a real volume, with the CPU cost of doing it +; charged where a rate model cannot hide it. P4a -- the HD63450 holding the bus +; -- is what makes the transfer overlap again, and until it exists this is the +; honest floor. + +; ---- state. Above src/player/ring.i's instruments (last: SK_WAIT at $1837C) +; and below the disc-offset table at $19400. +XF_SCSI = $18380 ; 1 = the 68000 is the transport (input) +XS_LBA0 = $18384 ; LBA of byte 0 of the scene's frame stream +XS_NXFER = $18388 ; transfers completed +XS_NBYTE = $1838C ; record bytes delivered into the ring +XS_NWIRE = $18390 ; bytes actually read off the disc, sectors and + ; all -- the two differ and 58.3 is why +XS_ERR = $18394 ; SC_ERR of the FIRST failure, 0 = none +XS_ERRAT = $18398 ; ...and the request index it failed on + +; ---------------------------------------------------------------- xf_init +; Clears the instruments and brings the SPC up, if this run has one. Called +; before ring_init, because ring_init ends in a ring_seek and a seek waits on +; the transport. +xf_init: + clr.l XS_NXFER.l + clr.l XS_NBYTE.l + clr.l XS_NWIRE.l + clr.l XS_ERR.l + move.l #-1,XS_ERRAT.l + tst.l XF_SCSI.l + beq.s .out + bsr scsi_init +.out: rts + +; ---------------------------------------------------------------- xf_service +; Answer at most ONE outstanding request, then return. Preserves every +; register: it is called from inside ring_poll, which is itself called from +; inside the decoder's wait loops and must be invisible to them. +; +; ONE PER CALL, not "drain the queue". ring_poll retires exactly one completed +; request per call as well, and a transport that answered both queued requests +; in one visit would hand the retire loop two acks it can only take one poll at +; a time -- which is legal, but it also means the decoder's wait loop would +; disappear for two record times instead of one. One per call keeps the two +; sides stepping at the same rate. +; +; A RECORD IS NOT A SECTOR, and this is where that is dealt with. ring.i asks +; for a byte offset and a length; the target answers in 512 B blocks. So the +; command covers the sectors the record lies in, and SC_WSKIP/SC_WKEEP tell +; src/player/scsi.i's DATA IN loop which of those bytes to store. The ones +; outside the window are pulled from the FIFO and dropped -- they are NOT +; written past the ends of the destination, because the bytes on either side of +; a record in the stream belong to records the decoder may still be reading and +; the block loop has no bounds check (49.2). +; +; XS_NWIRE counts what the disc actually moved and XS_NBYTE what the ring got. +; They are not the same number and the gap is a delivery cost, not an accounting +; detail: it is bytes on the wire that no frame contains. +xf_service: + tst.l XF_SCSI.l + beq.s .idle + move.l XF_ACK.l,d0 + cmp.l XF_GO.l,d0 + bcs.s .work ; XF_ACK < XF_GO: something outstanding +.idle: rts +.work: + movem.l d0-d7/a0-a2,-(sp) + move.l XF_ACK.l,d0 + move.l d0,d1 + and.l #XF_SLM,d1 + lsl.l #4,d1 ; * XF_SLSZ + lea XF_SLOT.l,a0 + adda.l d1,a0 + move.l (a0),d1 ; disc byte offset within the stream + movea.l 4(a0),a1 ; destination in the ring + move.l 8(a0),d2 ; length +; ---- sector arithmetic + move.l d1,d3 + and.l #511,d3 ; bytes of the first sector to drop + move.l d1,d4 + lsr.l #8,d4 + lsr.l #1,d4 ; sector index within the stream + add.l XS_LBA0.l,d4 ; ...and where the stream begins + move.l d3,d5 + add.l d2,d5 + addi.l #511,d5 + lsr.l #8,d5 + lsr.l #1,d5 ; sectors the record lies in + move.l d3,SC_WSKIP.l + move.l d2,SC_WKEEP.l + add.l d2,XS_NBYTE.l + move.l d5,d0 + lsl.l #8,d0 + lsl.l #1,d0 + add.l d0,XS_NWIRE.l + move.l d4,d3 ; d3 = LBA + move.l d5,d4 ; d4 = blocks + bsr scsi_read_win + tst.l d0 + bmi.s .err + addq.l #1,XS_NXFER.l + addq.l #1,XF_ACK.l ; LAST: the bytes are all in the ring + ; before the request is called done + movem.l (sp)+,d0-d7/a0-a2 + rts +; ---- a failed read is NOT acked. The record never becomes resident, the +; decoder spins out in waitrec and reports a stalled producer, and XS_ERR says +; which request failed and why. Acking a failed transfer would publish a +; descriptor for a buffer full of whatever was there before -- and the decoder +; would find a plausible-looking length word in it and desync somewhere else +; entirely. +.err: + tst.l XS_ERR.l + bne.s .err2 + move.l SC_ERR.l,XS_ERR.l + move.l XF_ACK.l,XS_ERRAT.l +.err2: movem.l (sp)+,d0-d7/a0-a2 + rts diff --git a/tools/analysis/26_sector_align.py b/tools/analysis/26_sector_align.py new file mode 100644 index 0000000..a26e315 --- /dev/null +++ b/tools/analysis/26_sector_align.py @@ -0,0 +1,143 @@ +"""A record is not a sector: what the mismatch costs, three ways (P4b, 58.3). + +src/player/ring.i asks the transport for a RECORD -- a byte offset into the +scene's frame stream and a length, both 4-byte aligned because that is what +`move.l (a0)+` needs (28.3) and neither of them a multiple of 512. A SCSI +target answers in 512 B BLOCKS. On the gate container 117 of 120 records start +part way into a sector, so something has to reconcile the two, and the three +ways of doing it are not close. + +WHY IT IS NOT AN IMPLEMENTATION DETAIL. The bytes on either side of a record in +the stream belong to OTHER records -- ones the decoder may still be reading -- +and the block loop walks a0 with no bounds check at all (49.2). So a transport +that reads whole sectors straight into the ring does not waste 500 bytes, it +CORRUPTS the neighbours, and the symptom is wrong pixels rather than a fault. + + A. WINDOWED PIO. Read the sectors the record lies in, store only the record. + src/player/scsi.i does this and it is what FINDINGS 58 measured. It costs + nothing in clocks -- the CPU is touching every byte anyway -- and it costs + the extra sectors on the wire. It CANNOT be done by a DMAC: a channel + writes a contiguous run to a contiguous address and cannot be told to drop + the first 300 bytes. + B. BOUNCE BUFFER. Let the DMAC write whole sectors somewhere else, then copy + the record into the ring. Works under DMA, and costs a copy of every + delivered byte -- which is precisely the cost `aligned` was chosen over + `split` to avoid (49.3, 19_ring_stream.py). + C. SECTOR-ALIGNED RECORDS. Pad each record up to 512 in the container + instead of up to 4. Costs bytes on the disc and in every delivery, and + nothing else at all; the transport becomes a whole-sector read into the + ring with no window and no copy. It is a CONTAINER change -- a re-encode + and a re-measurement of every constant fitted to the gate container, which + is the class of change ROADMAP already has bundled with P2's other half. + + python3 tools/analysis/26_sector_align.py [--ring KB] + +No rate is taken and none is needed: every figure here is a fraction of the +delivered bytes or a count of clocks, and both are rate-free. What a given +delivery rate does with them is 15_bus_occupancy.py's question. +""" +import sys, os, argparse +sys.path.insert(0, "tools/encoder") +from dlx import DLX + +SECTOR = 512 +CPUHZ = 10_000_000 +# 5.0 clocks/byte, and it is 19_ring_stream.py's constant rather than a new one: +# a 68000 `move.l (a0)+,(a1)+` moves 4 bytes in 20 clocks on a 16-bit bus. It +# is the OPTIMISTIC figure there and it is the optimistic figure here. +COPY_CLK_PER_BYTE = 5.0 +# The windowed PIO loop in src/player/scsi.i, from the 68000's cycle table: +# 12 move.l #SC_PATIENCE,d3 patience reload +# 16 move.b SC_SSTS,d0 (xxx).L -> Dn +# 10 btst #0,d0 +# 10 beq.s taken +# 20 move.b SC_DREG,(a1)+ (xxx).L -> (An)+ +# 8 subq.l #1,d7 +# 10 bne.s taken +# FINDINGS 58.2 measured 87.28 clocks per delivered byte against this loop's 86 +# plus 1.15 for the dropped window bytes -- 0.2% apart, which is what says the +# cost is the instruction stream and not MAME's device model. +PIO_CLK_PER_BYTE = 86.0 + +ap = argparse.ArgumentParser() +ap.add_argument("container") +ap.add_argument("--ring", type=int, default=256, help="ring size in KB") +a = ap.parse_args() + +d = DLX(a.container) + +# The disc layout the 68000 walks: [u32 len][body], each record padded up to 4. +# Exactly tools/bench/prep_stream.py's, and it is rebuilt here rather than read +# from tmp/ so this tool works on any container. +off, recs = 0, [] +for (o, n) in d.frames: + ln = 4 + n + ln += (-ln) % 4 + recs.append((off, ln)) + off += ln +payload = sum(ln for _, ln in recs) +nfr = len(recs) +budget = CPUHZ / d.fps + +print(f"{a.container}: {nfr} records, {payload:,} B, {d.fps} fps") +print(f" mean record {payload/nfr:,.0f} B; a {d.fps} fps frame is " + f"{budget:,.0f} clocks") +aligned0 = sum(1 for o, _ in recs if o % SECTOR == 0) +print(f" records that already start on a sector boundary: {aligned0}/{nfr}") +print() + +# ---- A. windowed PIO: the sectors the record lies in, and only the record kept +wire_a = sum(((o % SECTOR) + ln + SECTOR - 1) // SECTOR for o, ln in recs) * SECTOR +drop_a = wire_a - payload +print("A. WINDOWED PIO (src/player/scsi.i, what FINDINGS 58 ran)") +print(f" wire {wire_a:,} B for {payload:,} B of record " + f"= +{100*drop_a/payload:.2f}%") +print(f" clocks {PIO_CLK_PER_BYTE:.0f}/B on EVERY byte off the FIFO, " + f"dropped ones included:") +print(f" {PIO_CLK_PER_BYTE*wire_a/nfr:,.0f} clk/frame " + f"= {100*PIO_CLK_PER_BYTE*wire_a/nfr/budget:.0f}% of the frame") +print( " and it does not survive the move to the DMAC at all: a channel " + "cannot drop bytes.") +print() + +# ---- B. bounce buffer: DMA whole sectors elsewhere, copy the record in +print("B. BOUNCE BUFFER (whole sectors by DMA, then a copy)") +print(f" wire {wire_a:,} B, the same +{100*drop_a/payload:.2f}% -- the " + f"command is identical") +print(f" clocks {COPY_CLK_PER_BYTE:g}/B of copy on every DELIVERED byte, " + f"on top of whatever W the") +print(f" channel steals: {COPY_CLK_PER_BYTE*payload/nfr:,.0f} clk/frame " + f"= {100*COPY_CLK_PER_BYTE*payload/nfr/budget:.1f}% of the frame") +print( " which is the cost `aligned` was chosen over `split` to avoid " + "(49.3), arriving") +print( " by a different door and on every byte instead of on a wrap.") +print() + +# ---- C. sector-aligned records in the container +cur, pad = 0, 0 +for _, ln in recs: + if cur % SECTOR: + pad += SECTOR - (cur % SECTOR) + cur += SECTOR - (cur % SECTOR) + cur += ln +print("C. SECTOR-ALIGNED RECORDS (a container change; a re-encode)") +print(f" wire {cur:,} B for {payload:,} B of record = +{100*pad/payload:.2f}%") +print( " clocks ZERO: the read is a whole-sector read straight into the " + "ring, no window,") +print( " no copy, and the DMAC can do it.") +print() + +ringsz = a.ring * 1024 +print(f" VERDICT, in the currency this project prices delivery in. C is " + f"cheaper on the wire") +print(f" than A and B by {100*(drop_a-pad)/payload:.2f} points of the payload " + f"({drop_a-pad:,} B on this scene),") +print(f" and it is the only one of the three a DMA channel can run without a " + f"copy. What it") +print(f" costs is a container revision and the re-measurement that comes with " + f"one.") +maxrec = max(ln for _, ln in recs) +maxpad = maxrec + (-maxrec) % SECTOR +print(f" It also grows the largest record from {maxrec:,} to {maxpad:,} B, " + f"which a {a.ring} KB") +print(f" ring still holds {ringsz//maxpad} times over.") diff --git a/tools/bench/check.sh b/tools/bench/check.sh index 6bcdd2e..6a987c4 100755 --- a/tools/bench/check.sh +++ b/tools/bench/check.sh @@ -433,6 +433,77 @@ else echo " SKIPPED: no chdman (ships with mame-tools) -- cannot build the volume" fi +echo "--- session 26: the ring is filled off a real SCSI volume (FINDINGS 58) ---" +# ROADMAP P4b. The stage above shows the 68000 can READ the disc. This shows it +# can RUN off it: src/player/xfer.i sits behind src/player/ring.i's XF_* mailbox +# in place of tools/bench/stream.lua's modelled transport, and the same 120 +# frames are decoded out of the same 256 KB ring with NOTHING outside the +# machine in the transfer path -- no host file, no modelled rate, no synthesised +# ack. +# +# WHAT IS GATED, and it is correctness rather than rate on purpose: +# * pixel-exact, which is the only test that can see a wrong record: the +# window in scsi.i decides which of a sector's bytes reach the ring, and a +# window off by one byte desyncs the bitstream rather than faulting (49.2); +# * the SAME 18 wraps and 14.7 KB mean hole -- ring.i's placement policy must +# not be able to tell which transport answered it, and this is the assertion +# that says it could not; +# * every record accounted for: 120 READ(10)s, 4,488,588 B into the ring, and +# 4,548,608 B off the disc. The two byte counts differ by 1.34% because a +# record is not a sector, and that gap is a delivery cost (58.3) -- gating +# both numbers means neither can drift silently into the other; +# * a real mid-stream SEEK with the real transport, in the second pass. This +# is the one path that could not exist before: ring_seek waits for the +# channel to go quiet, and with the transport INSIDE the machine the only +# thing that can retire an outstanding request is that wait loop itself. +# +# NOTHING HERE IS GATED ON RATE and nothing here can be. What the run DOES cost +# is printed by tools/bench/xfer_cost.sh and recorded in FINDINGS 58.2; it is a +# measurement, and a change in it is a re-derivation there rather than a red +# light here. Skipped rather than failed when chdman is absent. +if command -v chdman > /dev/null; then + DLX_PACE=0 DLX_RINGOWN=1 DLX_QDEPTH=2 DLX_XFER=scsi \ + bash tools/bench/pace_run.sh 256 0 > tmp/p4b_check.log 2>&1 || { + echo "FAIL: the 68000 could not run the ring off a real SCSI volume." + tail -12 tmp/p4b_check.log; exit 1; } + grep -aE "REAL TRANSPORT:|SECTOR OVERHEAD|ring: " tmp/p4b_check.log \ + | sed "s/^ *//;s/^/ /" + grep -aq "TRANSPORT FAILED" tmp/p4b_check.log && { + echo "FAIL: a record's READ(10) reported an error."; exit 1; } + grep -aq "REAL TRANSPORT: 120 READ(10)s by the 68000, 4488588 B into the ring" \ + tmp/p4b_check.log || { + echo "FAIL: the 68000 did not fetch all 120 records, or did not fetch" + echo " 4,488,588 B of them. A short record is a desync, not a shortfall." + exit 1; } + grep -aq "SECTOR OVERHEAD: 4548608 B off the disc" tmp/p4b_check.log || { + echo "FAIL: the bytes the DISC moved are no longer 4,548,608. A record is" + echo " not a sector; this is the covering-sector read, and if it moved" + echo " then either the layout or scsi.i's window did. See FINDINGS 58.3." + exit 1; } + grep -aq "ring: 18 wraps" tmp/p4b_check.log || { + echo "FAIL: the placement policy tiled this container differently with a" + echo " real transport behind it than with a modelled one. ring.i is" + echo " not supposed to be able to tell them apart."; exit 1; } + grep -aq "^OK" tmp/p4b_check.log || { + echo "FAIL: the pass off the SCSI volume was not pixel-exact." + tail -4 tmp/p4b_check.log; exit 1; } + DLX_PACE=2 DLX_RINGOWN=1 DLX_QDEPTH=2 DLX_ITER=2 DLX_XFER=scsi \ + DLX_SECONDS=240 bash tools/bench/pace_run.sh 256 0 \ + > tmp/p4b_seek_check.log 2>&1 || { + echo "FAIL: the seek pass off the SCSI volume did not complete." + tail -12 tmp/p4b_seek_check.log; exit 1; } + grep -aE "SEEK PASS|IS VACUOUS" tmp/p4b_seek_check.log | sed "s/^ *//;s/^/ /" + grep -aq "SEEK PASS 2" tmp/p4b_seek_check.log || { + echo "FAIL: no real seek -- the second pass never threw its ring away, so" + echo " ring_seek's quiet-wait was never asked to retire an outstanding" + echo " transfer and this gated nothing."; exit 1; } + grep -aq "^OK" tmp/p4b_seek_check.log || { + echo "FAIL: the decode after a seek off the SCSI volume was not pixel-exact." + tail -4 tmp/p4b_seek_check.log; exit 1; } +else + echo " SKIPPED: no chdman (ships with mame-tools) -- cannot build the volume" +fi + echo "--- session 24: the scene graph, and the gap between branch points (FINDINGS 56) ---" # The arcade scene graph is not in this repo and is not redistributable from # here. tools/import/scenegraph.py is the ONE file in the tree that knows the diff --git a/tools/bench/mkvol.sh b/tools/bench/mkvol.sh new file mode 100755 index 0000000..14c02bc --- /dev/null +++ b/tools/bench/mkvol.sh @@ -0,0 +1,39 @@ +#!/bin/bash +# Build the SCSI VOLUME the P4 rigs read, and the blank card ROM MAME needs to +# instantiate the card. Sourced-by-calling from tools/bench/scsi_run.sh and +# tools/bench/pace_run.sh so there is ONE copy of the layout. +# +# tools/bench/mkvol.sh [container.dlx] +# +# ONE COPY, ON PURPOSE. The volume is tmp/stream_disk.bin -- byte for byte the +# file the host-file ring rig reads -- laid out as 512 B sectors. If two scripts +# each built it, a difference between the SCSI rig and the modelled-transport rig +# could be a difference in what they were reading, and the whole value of running +# both is that it cannot be. This tree has already paid twice for a transform +# with two copies of itself (FINDINGS 49.7.5, and check.sh's dlxload note). +# +# THE BLANK BOOT ROM is the substitution session 25 argued for and it is +# unchanged: MAME refuses to instantiate the CZ-6BS1 without an 8 KB +# `scsiexrom.bin` (CRC 7be488de) that the player never executes, so a zero-filled +# placeholder goes on a SEPARATE rompath and the user's romset is untouched. +# MAME prints WRONG CHECKSUMS, as it should. DO NOT reuse this rompath for +# anything that boots from the card or calls SCSI IOCS -- those DO execute it. +set -e +cd "$(dirname "$0")/../.." +DLX=${1:-tmp/rc_fr_singe_scsi_span.dlx} + +[ -f tmp/stream_disk.bin ] || python3 tools/bench/prep_stream.py "$DLX" > /dev/null +if [ ! -f tmp/dlxdisk.chd ] || [ tmp/stream_disk.bin -nt tmp/dlxdisk.chd ]; then + python3 - <<'PY' +d = open("tmp/stream_disk.bin", "rb").read() +n = (len(d) + 511) // 512 +open("tmp/dlxdisk.img", "wb").write(d + b"\0" * (n * 512 - len(d))) +print(f" disc image: {len(d)} B of records -> {n} sectors") +PY + rm -f tmp/dlxdisk.chd + chdman createhd -i tmp/dlxdisk.img -o tmp/dlxdisk.chd -ss 512 > /dev/null 2>&1 +fi + +mkdir -p tmp/p4roms/x68k_cz6bs1 +[ -f tmp/p4roms/x68k_cz6bs1/scsiexrom.bin ] || \ + head -c 8192 /dev/zero > tmp/p4roms/x68k_cz6bs1/scsiexrom.bin diff --git a/tools/bench/pace_run.sh b/tools/bench/pace_run.sh index 9c03410..7e2d2c2 100755 --- a/tools/bench/pace_run.sh +++ b/tools/bench/pace_run.sh @@ -28,6 +28,13 @@ # needs DLX_PACE=2, because rebasing the frame clock across a pass is the # machine's to do and a host-written tick would carry on counting. # +# DLX_XFER=scsi replaces the MODELLED transport with a real one (ROADMAP P4b, +# src/player/xfer.i): the machine gets a CZ-6BS1 and the same volume the SCSI +# gate reads, this script stops moving bytes altogether, and every record is +# fetched by the 68000 with READ(10). It needs DLX_RINGOWN=1 -- the mailbox it +# answers is ring.i's -- and it FORBIDS a modelled rate, because there is no +# longer anything for one to model. +# # DLX_PACE selects WHO KEEPS THE TIME: 1 (default) is the host writing the tick, # 2 is the 68000 writing it off the CRTC's V-DISP (ROADMAP P3, FINDINGS 54). # Everything else about the run is identical, which is the whole point -- the @@ -41,6 +48,30 @@ DLX=${DLX:-tmp/rc_fr_singe_scsi_span.dlx} PACE=${DLX_PACE:-1} OWN=${DLX_RINGOWN:-0} ITERS=${DLX_ITER:-1} +XFER=${DLX_XFER:-model} +# EMULATED seconds the run is allowed. A pass that is cut short compares a +# half-drawn screen and reads as a wrap bug, so this is raised deliberately +# rather than left to a timeout: a DLX_XFER=scsi pass costs ~46 s of emulated +# time against the modelled transport's ~7, because the CPU moves every byte +# itself (FINDINGS 58.2), and two of them do not fit in 90. +SECS=${DLX_SECONDS:-90} +if [ "$XFER" = scsi ]; then + [ "$OWN" = 1 ] || { echo "DLX_XFER=scsi needs DLX_RINGOWN=1: the transport in" + echo "src/player/xfer.i answers src/player/ring.i's mailbox, and with the" + echo "host owning the ring there is no mailbox to answer."; exit 2; } + # A rate is not merely ignored here, it is REFUSED. The bytes now arrive on + # the emulated machine's own time, and a run labelled "488 KB/s" that did not + # deliver at 488 KB/s is exactly the kind of number this project has twice + # paid for. There is no rate in a DLX_XFER=scsi run, and the log says so. + [ "$KBPS" = 0 ] || { echo "DLX_XFER=scsi takes kbps 0. The transport is real," + echo "so nothing here delivers at a modelled rate -- and MAME's device" + echo "models are functional, not transfer-timing accurate, so the rate it" + echo "DOES deliver at is not a measurement either (docs/BENCHMARK.md)." + exit 2; } + command -v chdman > /dev/null || { echo "DLX_XFER=scsi needs chdman (ships" + echo "with mame-tools) to build the volume."; exit 2; } + bash tools/bench/mkvol.sh "$DLX" +fi if [ "$OWN" = 1 ] && [ "$ITERS" != 1 ] && [ "$PACE" != 2 ]; then echo "DLX_ITER>1 needs DLX_PACE=2: the frame clock is rebased per pass by" echo "src/player/stream.s, and a host-written tick would go on counting" @@ -54,6 +85,11 @@ TAG="r${RING}_k${KBPS}${CUT_AT:+_cut${CUT_AT}x${CUT_FR}}" if [ "$PACE" != 1 ]; then TAG="${TAG}_p$PACE"; fi if [ "$OWN" = 1 ]; then TAG="${TAG}_own"; fi if [ "$ITERS" != 1 ]; then TAG="${TAG}_x$ITERS"; fi +if [ "$XFER" != model ]; then TAG="${TAG}_$XFER"; fi +MAMEX=() +if [ "$XFER" = scsi ]; then + MAMEX=(-exp1 cz6bs1 -rompath "$HOME/mame/roms;./p4roms" -hard dlxdisk.chd) +fi tools/vasm/vasmm68k_mot -Fbin -o tmp/stream.bin src/player/stream.s > /dev/null [ -f tmp/stream_disk.bin ] || python3 tools/bench/prep_stream.py "$DLX" > tmp/prep_stream.log @@ -63,13 +99,15 @@ mkdir -p "tmp/snap_pace_$TAG"; rm -f "tmp/snap_pace_$TAG/x68000"/*.png # command and the run dies with "SDL_VIDEODRIVER=dummy: command not found". CUTENV=(); [ -n "$CUT_AT" ] && CUTENV=(DLX_CUT_AT="$CUT_AT" DLX_CUT_FR="$CUT_FR") ( cd tmp && env DLX_PACE=$PACE DLX_RING_KB=$RING DLX_STREAM_KBPS=$KBPS \ - DLX_RINGOWN=$OWN DLX_ITER=$ITERS \ + DLX_RINGOWN=$OWN DLX_ITER=$ITERS DLX_XFER=$XFER \ ${DLX_PREFILL_FR:+DLX_PREFILL_FR=$DLX_PREFILL_FR} \ "${CUTENV[@]}" DLX_SLACK_CSV="slack_$TAG.csv" \ SDL_VIDEODRIVER=dummy stdbuf -oL timeout -k 5 900 \ - mame x68000 -bios ipl10 -ramsize 2M -video soft -window -sound none \ + mame x68000 -bios ipl10 "${MAMEX[@]}" -ramsize 2M -video soft -window \ + -sound none \ -nothrottle -plugins -autoboot_script ../tools/bench/stream.lua \ - -snapshot_directory "./snap_pace_$TAG" -snapview native -seconds_to_run 90 \ + -snapshot_directory "./snap_pace_$TAG" -snapview native \ + -seconds_to_run $SECS \ > "pace_$TAG.log" 2>&1 ) # The completion marker is not optional: a run killed mid-decode compares a # half-drawn screen and reads as a wrap bug rather than as a truncated run. @@ -77,6 +115,6 @@ grep -q "snapshot taken" "tmp/pace_$TAG.log" || { echo "FAIL($TAG): no snapshot marker -- the pass did not complete." tail -6 "tmp/pace_$TAG.log"; exit 1; } echo "=== $TAG" -grep -aE "decoder (SELF-PACED|PACED|FREE)|FRAME CLOCK|ring: |UNDERRUNS|NO IDLE|SEEK SLACK|RING-BOUND|RATE-BOUND|BUILD TIME|PIPE CUT|DEADLINE|REQUIRED|MACHINE-OWNED|PREFILL:|CHANNEL IDLE|MISPLACED|SEEK PASS" \ +grep -aE "decoder (SELF-PACED|PACED|FREE)|FRAME CLOCK|ring: |UNDERRUNS|NO IDLE|SEEK SLACK|RING-BOUND|RATE-BOUND|BUILD TIME|PIPE CUT|DEADLINE|REQUIRED|MACHINE-OWNED|PREFILL:|CHANNEL IDLE|MISPLACED|SEEK PASS|REAL TRANSPORT|SECTOR OVERHEAD|TRANSPORT FAILED|IS VACUOUS" \ "tmp/pace_$TAG.log" | sed "s/\[STR\] / /" python3 tools/bench/verify_decode.py "$DLX" --snap "tmp/snap_pace_$TAG" | tail -2 diff --git a/tools/bench/scsi_run.sh b/tools/bench/scsi_run.sh index b3625e3..34ee0c5 100755 --- a/tools/bench/scsi_run.sh +++ b/tools/bench/scsi_run.sh @@ -35,25 +35,10 @@ set -e cd "$(dirname "$0")/../.." DLX=${1:-tmp/rc_fr_singe_scsi_span.dlx} -# The container's frame records, laid out as a disc. prep_stream.py already -# writes exactly this file for the ring rig, so the SCSI volume and the host-file -# pipe carry byte-identical bytes and a difference between the two rigs cannot be -# a difference in what they are reading. -[ -f tmp/stream_disk.bin ] || python3 tools/bench/prep_stream.py "$DLX" > /dev/null -if [ ! -f tmp/dlxdisk.chd ] || [ tmp/stream_disk.bin -nt tmp/dlxdisk.chd ]; then - python3 - <<'PY' -d = open("tmp/stream_disk.bin", "rb").read() -n = (len(d) + 511) // 512 -open("tmp/dlxdisk.img", "wb").write(d + b"\0" * (n * 512 - len(d))) -print(f" disc image: {len(d)} B of records -> {n} sectors") -PY - rm -f tmp/dlxdisk.chd - chdman createhd -i tmp/dlxdisk.img -o tmp/dlxdisk.chd -ss 512 > /dev/null 2>&1 -fi - -mkdir -p tmp/p4roms/x68k_cz6bs1 -[ -f tmp/p4roms/x68k_cz6bs1/scsiexrom.bin ] || \ - head -c 8192 /dev/zero > tmp/p4roms/x68k_cz6bs1/scsiexrom.bin +# The container's frame records, laid out as a disc, and the blank card ROM. +# Both are tools/bench/mkvol.sh's, shared with tools/bench/pace_run.sh so the +# SCSI rig and the host-file ring rig cannot be reading different bytes. +bash tools/bench/mkvol.sh "$DLX" tools/vasm/vasmm68k_mot -Fbin -o tmp/scsigate.bin src/player/scsigate.s > /dev/null diff --git a/tools/bench/stream.lua b/tools/bench/stream.lua index 1f98d98..e67eb37 100644 --- a/tools/bench/stream.lua +++ b/tools/bench/stream.lua @@ -125,6 +125,10 @@ local A_NPOLL, A_NISSUE = 0x18360, 0x18364 local A_SLKMIN, A_SLKAT = 0x18368, 0x1836C local A_PFREC, A_PFDONE = 0x18370, 0x18374 local A_NSEEK, A_SKWAIT = 0x18378, 0x1837C +-- src/player/xfer.i, the transport that lives in the machine. +local A_XFSCSI, A_XSLBA0 = 0x18380, 0x18384 +local A_XSNXFER, A_XSNBYTE = 0x18388, 0x1838C +local A_XSNWIRE, A_XSERR, A_XSERRAT = 0x18390, 0x18394, 0x18398 -- The scene header's record index, pushed into RAM before the CPU is launched. -- It sits ABOVE the codebooks rather than in the low RAM around ring.i's own -- tables: setup() runs one raster frame before launch() and the IPL is still @@ -171,6 +175,25 @@ local ITERS = tonumber(os.getenv("DLX_ITER") or "") or 1 -- 2 keeps the next request queued so it never stops. Both are real designs and -- the difference between them is what this rig measures. local QDEPTH = tonumber(os.getenv("DLX_QDEPTH") or "") or 2 +-- WHO MOVES THE BYTES. "model" is this script: a transport that delivers +-- XF_LEN bytes in XF_LEN/rate seconds and acks from emulated time, which is +-- what FINDINGS 49/51/55 were all measured through. "scsi" is ROADMAP P4b -- +-- src/player/xfer.i and src/player/scsi.i on the 68000, a real MB89352, a real +-- volume, and NOTHING of this script in the transfer path. +-- +-- The two are not interchangeable and the log must never let them look it. The +-- modelled transport OVERLAPS with the CPU, which is what a DMAC channel does; +-- the real one here does not, because 57.3 leaves the CPU moving every byte +-- itself. So a "scsi" run measures CORRECTNESS off a real volume and the CPU +-- cost of a PIO transport, and it measures NO rate: every rate-shaped number in +-- this script's report is suppressed rather than printed against a transport +-- that has no model behind it. +local XFER = os.getenv("DLX_XFER") or "model" +local SCSIX = (XFER == "scsi") +-- src/player/scsi.i's SCE_* codes, so a failure names itself. +local SCERRNAME = {[0]="OK", "SELECTION TIMEOUT -- no target answered", + "UNEXPECTED PHASE", "POLL TIMEOUT -- a phase never arrived", + "NON-ZERO SCSI STATUS"} -- One snapshot per frame tick instead of one at the end of the run. This is a -- DOCUMENTATION artefact -- 120 PNGs of a paced player, for a recording -- and -- it is deliberately not on any path tools/bench/check.sh takes. Needs @@ -548,7 +571,17 @@ local function setup() SP:write_u32(A_RNG_SZ, RINGSZ) SP:write_u32(A_PFREC, PREFILL_FR) SP:write_u32(XF_QD, QDEPTH) - install_transport() + if SCSIX then + -- The stream begins at LBA 0 of the volume, because tools/bench/mkvol.sh + -- lays tmp/stream_disk.bin down from sector 0 and nothing else is on it. + -- It is a WORD the machine reads rather than a constant in xfer.i: a + -- shipping volume has a filesystem in front of the stream, and the base + -- LBA is the one number that changes when it does. + SP:write_u32(A_XFSCSI, 1) + SP:write_u32(A_XSLBA0, 0) + else + install_transport() + end end P(string.format("stream.bin=%d B, codebooks %d+%d B, disk %d B, %d frames", #code, META.cb1_len, META.cb4_len, META.disk_len, META.nframes)) @@ -563,9 +596,20 @@ local function setup() RINGOWN and (PREFILL_FR.." records (the machine's own)") or ((PREFILL // 1024).." KB"), META.maxrec)) if RINGOWN then - P(string.format("ring OWNED BY THE 68000 (src/player/ring.i): this script " - .."is a transport with a %d-deep request queue, %d B of " - .."DLX4 index at %06X", QDEPTH, #idxblob, IDXRAM)) + if SCSIX then + P(string.format("REAL TRANSPORT (ROADMAP P4b, src/player/xfer.i): the " + .."68000 fetches every record itself with READ(10) off a " + .."CZ-6BS1. This script moves NO bytes and models NO " + .."rate. %d B of DLX4 index at %06X.", #idxblob, IDXRAM)) + P(" the queue depth cannot deepen anything here: the transport is " + .."SYNCHRONOUS (xfer.i), so a queued request is drained by the same " + .."instruction stream that would be decoding. FINDINGS 55.3's 1-vs-2 " + .."result is about a transport that OVERLAPS; this one does not.") + else + P(string.format("ring OWNED BY THE 68000 (src/player/ring.i): this script " + .."is a transport with a %d-deep request queue, %d B of " + .."DLX4 index at %06X", QDEPTH, #idxblob, IDXRAM)) + end end if META.maxrec > RINGSZ then P("RING TOO SMALL: one record does not fit. stream.s needs a whole record " @@ -588,13 +632,48 @@ local st, t0, t_rel = "boot", nil, 0 -- would flatter every record by that much. local tick_t = {} local pace, min_ahead, min_at = -1, math.huge, -1 +-- DLX_XFER=scsi: the machine's seek counter, and the transfer count it stood at +-- when that seek happened. -1 means "not sampled yet", so the first sample +-- rebases without announcing a pass that did not happen. +local xs_nseek, xs_base = -1, 0 local sum_ahead, n_ahead = 0, 0 local slack_series = {} SUB = emu.add_machine_frame_notifier(function() local ok, err = pcall(function() local t = T() - if RINGOWN then transport_service(t) end + if RINGOWN and not SCSIX then transport_service(t) end + if SCSIX and st ~= "boot" then + -- The machine's own count of completed transfers, which is what `nsent` + -- means in a run where this script sends nothing. Sampled rather than + -- derived, so the slack window still closes when the producer runs out of + -- records to place instead of running on into the drain tail. + -- + -- REBASED AT EVERY SEEK, and it has to be: XS_NXFER is cumulative over + -- the whole run, so on a second pass it is already >= nframes and the + -- slack sampling below -- gated on `nsent < META.nframes` -- would never + -- fire again. It did not, and the second pass reported a ceiling of 0 + -- frames and a build time of -1 ticks, which is an empty series printing + -- as a result. ring.i's own seek counter is the exact place to rebase: + -- a seek discards every outstanding request by definition. + local ns = SP:read_u32(A_NSEEK) + if ns ~= xs_nseek then + -- ANNOUNCE ONLY A SEEK WITH PLAY BEHIND IT. A pass opens with two + -- seeks that are not branch points -- ring_init's, and stream.s's at + -- the top of `outer` -- and both happen before a single record has been + -- fetched. Announcing those numbered the first pass "SEEK PASS 2" and + -- the real one "3". XS_NXFER separates them exactly: a seek taken with + -- transfers already behind it is the one that threw a ring away. + if xs_nseek >= 0 and SP:read_u32(A_XSNXFER) > 0 then + npass = npass + 1 + P(string.format("SEEK PASS %d: the machine seeked at %.3f s and is " + .."refilling from empty", npass, t)) + end + xs_nseek, xs_base = ns, SP:read_u32(A_XSNXFER) + end + nsent = SP:read_u32(A_XSNXFER) - xs_base + if nsent < 0 then nsent = 0 end + end if st == "boot" then if t < 3.0 then return end setup(); last_t, pf_t0 = t, t; st = "prefill"; return @@ -794,6 +873,41 @@ SUB = emu.add_machine_frame_notifier(function() -- PLAYER's loop, not of the medium, and no host-filled run could see -- it -- the host producer placed records whenever it liked. local span = (arrival[nissue] or t) - (t0 or t) + if SCSIX then + -- WHAT A REAL TRANSPORT CAN AND CANNOT BE ASKED HERE. The modelled + -- transport knew when every byte landed because it decided; this one + -- does not report, and MAME's device models are functional rather + -- than transfer-timing accurate, so even if it did the number would + -- not be a rate (docs/BENCHMARK.md, 42.5). So CHANNEL IDLE, + -- DEADLINE and REQUIRED PREFILL are not printed at all rather than + -- printed as zeros -- a zero here reads as "the channel never + -- stopped", which would be a claim about a medium this rig has + -- never timed. + local nb, nw = SP:read_u32(A_XSNBYTE), SP:read_u32(A_XSNWIRE) + local nx = SP:read_u32(A_XSNXFER) + P(string.format("REAL TRANSPORT: %d READ(10)s by the 68000, %d B " + .."into the ring", nx, nb)) + -- THE COST THE CONTAINER PAYS FOR NOT BEING SECTOR-ALIGNED. A + -- record starts wherever the previous one ended, rounded up to 4; + -- a target answers in 512 B blocks. So the command covers the + -- sectors the record lies in and src/player/scsi.i's window drops + -- the rest. In PIO those bytes cost nothing but wire; they are + -- still bytes the disc moved that no frame contains. + P(string.format("SECTOR OVERHEAD: %d B off the disc for %d B of " + .."record = %.2f%% the medium moved and no frame " + .."contains", nw, nb, + nb > 0 and 100*(nw - nb)/nb or 0)) + local e, eat = SP:read_u32(A_XSERR), SP:read_u32(A_XSERRAT) + if e ~= 0 then + P(string.format("TRANSPORT FAILED: SC_ERR=%d on request %d (%s)", + e, eat, SCERRNAME[e] or "see src/player/scsi.i")) + end + if skw > 0 then + P(string.format(" the last seek waited %d polls for the channel " + .."to go quiet before it could start", skw)) + end + goto ringdone + end P(string.format("CHANNEL IDLE: %.1f ms over %d gaps (worst %.1f ms) = " .."%.1f%% of the %.2f s the transport was needed for; " .."busy %.1f ms", @@ -819,6 +933,7 @@ SUB = emu.add_machine_frame_notifier(function() P(string.format(" the last seek waited %d polls for the channel " .."to go quiet before it could start", skw)) end + ::ringdone:: end -- The decoder's own spin counter, kept for what it is: evidence that -- stream.s outran the pipe, NOT evidence of an underrun. See the note @@ -829,6 +944,24 @@ SUB = emu.add_machine_frame_notifier(function() -- opposite thing, so the two are never printed in the same words. P(string.format("UNDERRUNS: %d/%d frames waited past their %d fps slot " .."(%d polls)", stalls, META.nframes, META.fps, spins)) + if SCSIX then + -- AND IT IS VACUOUS HERE, WHICH IS WORTH MORE THAN THE ZERO IS. + -- A synchronous transport cannot underrun by construction: the + -- decoder does not go on until the record is in its hand, because + -- it fetched the record itself. What it does instead is MISS THE + -- SLOT, and that is the NO IDLE line above -- 442 whole ticks of + -- overrun is not a late present, it is a player running at a + -- fraction of its frame rate. Quoting "0 underruns" from a + -- DLX_XFER=scsi run without this line would be the most flattering + -- possible reading of the least flattering result in the tree. + local ticks = pace + 1 + P(string.format(" 0 IS VACUOUS: the transport is synchronous, so a " + .."frame cannot start before its record has landed " + .."-- the decoder IS the transport. The honest rate " + .."is %d frames in %d slots of %d fps = %.2f fps.", + META.nframes, ticks, META.fps, + ticks > 0 and META.fps*META.nframes/ticks or 0)) + end -- The other way a paced frame can go wrong, and it is not the same -- failure. An UNDERRUN is the pipe: the record was not there. This -- is the CPU: the record was there, the slot was already open, and @@ -906,6 +1039,18 @@ SUB = emu.add_machine_frame_notifier(function() end -- The delivery result. Deadline for record i is release + i/fps. + -- Only this script can compute it, because only this script knows when + -- a byte landed -- and in a DLX_XFER=scsi run it does not. Printing it + -- from an empty arrival table would report "0/120 records late", which + -- is the most flattering possible reading of no data at all. + if SCSIX then + P("DEADLINE: not measured. The transport is in the machine and does " + .."not report arrival times, and MAME's SCSI is not transfer-timing " + .."accurate, so there is no honest deadline to grade against here. " + .."The sharp instrument that DOES survive is the decoder's own " + .."UNDERRUNS count above -- the machine takes it, and it is exact.") + st = "settle"; return + end local misses, worst, prefill_s = 0, 0.0, 0.0 for i = 0, META.nframes-1 do local a = arrival[i+1] diff --git a/tools/bench/xfer_cost.sh b/tools/bench/xfer_cost.sh new file mode 100755 index 0000000..1a01497 --- /dev/null +++ b/tools/bench/xfer_cost.sh @@ -0,0 +1,112 @@ +#!/bin/bash +# What the PIO transport costs the 68000, by subtraction (ROADMAP P4b, +# FINDINGS 58.2). +# +# tools/bench/xfer_cost.sh [ring_kb] +# +# THE MEASUREMENT IS AN A/B AND HAS TO BE. Nothing in the machine can time +# itself finely enough: src/player/clock.i counts V-DISP at 56.69 Hz and the +# thing being priced is a per-BYTE cost. So the same 120 frames are decoded +# twice, from the same ring, by the same src/player/stream.s, with the same +# src/player/ring.i placing every record -- and the ONLY difference is which +# side of the XF_* mailbox answers: +# +# model tools/bench/stream.lua, an unlimited modelled pipe. Bytes appear +# in the ring for free; the emulated time is decode plus ring_poll. +# scsi src/player/xfer.i and src/player/scsi.i, a real MB89352 and a real +# volume. The CPU moves every byte itself. +# +# Both runs are FREE-RUNNING (DLX_PACE=0). A paced run would measure the pace, +# not the work, and the difference this script exists for would vanish into the +# wait loop. +# +# WHAT THE SUBTRACTION IS HONEST ABOUT, and it is worth stating twice: this is a +# count of 68000 CLOCKS, not a delivery rate. It says what the player pays to +# move a byte with its own instruction stream. It says nothing about how fast +# the medium can supply one -- MAME's device models are functional, not +# transfer-timing accurate (docs/BENCHMARK.md, 42.5), and `W`, the clocks a DMAC +# steals per delivered byte, is untouched by every line here. +# +# The two run boundaries are sampled on the machine-frame notifier, so each end +# is up to 17.64 ms coarse (54.5). Against a difference of tens of seconds that +# is under a tenth of a percent, and the analytic cross-check below is what +# actually establishes the figure -- the subtraction only has to agree with it. +set -e +cd "$(dirname "$0")/../.." +RING=${1:-256} +command -v chdman > /dev/null || { echo "needs chdman (mame-tools)"; exit 2; } + +echo "=== modelled transport, unlimited pipe" +DLX_PACE=0 DLX_RINGOWN=1 DLX_QDEPTH=2 bash tools/bench/pace_run.sh "$RING" 0 \ + > tmp/xfer_cost_model.log 2>&1 +echo "=== real transport, a CZ-6BS1 and a real volume" +DLX_PACE=0 DLX_RINGOWN=1 DLX_QDEPTH=2 DLX_XFER=scsi \ + bash tools/bench/pace_run.sh "$RING" 0 > tmp/xfer_cost_scsi.log 2>&1 + +python3 - <<'PY' +import re, sys + +def one(tag, log): + t = open(log, "rb").read().decode("latin1") + m = re.search(r"decoded (\d+) frames in ([0-9.]+) s emulated", t) + if not m: sys.exit(f"{log}: no completion line -- the pass did not finish") + return int(m.group(1)), float(m.group(2)), t + +nfr, t_model, _ = one("model", "tmp/pace_r256_k0_p0_own.log") +nfr2, t_scsi, ts = one("scsi", "tmp/pace_r256_k0_p0_own_scsi.log") +assert nfr == nfr2, "the two runs decoded different frame counts" +nb = int(re.search(r"REAL TRANSPORT: \d+ READ\(10\)s by the 68000, (\d+) B", ts).group(1)) +nw = int(re.search(r"SECTOR OVERHEAD: (\d+) B off the disc", ts).group(1)) + +CPUHZ = 10_000_000 +FPS = 12 +budget = CPUHZ / FPS +d = (t_scsi - t_model) * CPUHZ + +print(f" {nfr} frames, {nb:,} B of record, {nw:,} B off the disc") +print(f" decode + ring_poll alone {t_model:8.4f} s emulated " + f"= {100*t_model*CPUHZ/nfr/budget:5.1f}% of a {FPS} fps frame") +print(f" ...with the real transport {t_scsi:8.4f} s emulated " + f"= {100*t_scsi*CPUHZ/nfr/budget:5.1f}%") +print(f" TRANSPORT COST: {d:,.0f} clocks = {d/nb:.2f} per delivered byte, " + f"{d/nw:.2f} per byte off the FIFO") +print(f" {d/nfr:,.0f} clk/frame = " + f"{100*d/nfr/budget:.1f}% of a {FPS} fps frame") + +# ---- THE CROSS-CHECK, and it is the half that makes the number portable. +# If the measured cost is the 68000's own instruction stream, it must equal the +# 68000's cycle table for the loop in src/player/scsi.i. If it does NOT, the +# difference is time spent waiting on MAME's SPC model -- which is emulator +# behaviour and would not survive contact with a board. +LOOP = [("move.l #SC_PATIENCE,d3", 12), ("move.b SC_SSTS,d0 (xxx).L->Dn", 16), + ("btst #0,d0", 10), ("beq.s taken", 10), + ("move.b SC_DREG,(a1)+ (xxx).L->(An)+", 20), + ("subq.l #1,d7", 8), ("bne.s taken", 10)] +loop = sum(c for _, c in LOOP) +print(f"\n the keep loop in src/player/scsi.i, from the 68000's cycle table:") +for n, c in LOOP: print(f" {c:3d} {n}") +pred = loop * nw / nb +print(f" --- {loop} clk per byte off the FIFO, and the FIFO carries the " + f"dropped\n window bytes too: {pred:.2f} per DELIVERED byte") +err = 100 * abs(pred - d/nb) / (d/nb) +print(f" MEASURED {d/nb:.2f} vs PREDICTED {pred:.2f} -- {err:.1f}% apart, so " + f"the cost is the\n instruction stream and not MAME's SPC: it is a " + f"figure a real board would also pay.") +resid = d/nb - pred +print(f" residual {resid:+.2f} clk/B = {resid*nb/nfr:,.0f} clk/record of " + f"select, CDB, status,\n message and xf_service -- the per-COMMAND cost, " + f"which is the part that does not\n scale with the record.") + +print(f"\n AGAINST THE LADDER, in the same units (clocks charged to the CPU " + f"per delivered\n byte). W is the DMAC's steal, 52.5's bracket:") +for W, name in [(5, "single address, bus HELD"), (9, "dual address, held"), + (12, "single address, arbitrated"), + (19, "dual address, arbitrated -- the IPL ROM's own disk ch")]: + print(f" W = {W:2d} {name:52s} {100*W*nb/nfr/budget:5.1f}% of the frame") +print(f" PIO {d/nb:.0f} this rig, measured " + f" {100*d/nfr/budget:15.1f}%") +print(f"\n So the PIO transport is {d/nb/19:.1f}x the WORST DMA configuration " + f"this project has\n found and {d/nb/5:.1f}x the best. P4a is not an " + f"optimisation of this; it is the\n difference between a player and a " + f"slideshow.") +PY