diff --git a/README.md b/README.md index c6c0727..6f21159 100644 --- a/README.md +++ b/README.md @@ -625,6 +625,13 @@ tools/bench/ MAME Lua injection harness and 68000 benchmark sources. a new config cannot weaken the gate. crtc_mode.lua is the single source of truth for CRTC R00-R08 and R20. Do not write CRTC values anywhere else. + verify_packed_audio.py is P6c's gate and it reads the + SPEAKER: it accounts for every byte of a DLXP2's audio against + MAME's own -wavwrite capture, ONE DELIVERED BYTE AT A TIME, + because the chip plays a byte as two nibbles only when the DMA + write lands inside the right sound-stream slice. A player's own + counters cannot gate this -- all of them stayed right through a + bug that overwrote the buffer the channel was reading. prep_dlx.py/decode.lua/verify_decode.py load, time and verify decode.s. prep_stream.py/stream.lua do the same for stream.s, but lay the container out as a DISK in a host file and feed it @@ -704,7 +711,19 @@ tools/encoder/ hybrid VQ encoder and DLX3 container writer. 24 models the ring with the 68000 owning it: the request queue, the poll-only-when-not-decoding rule and 54.4's frame cadence, and reports the pipe the player's own loop gives away. -src/player/ decode.s is the 68000 DLX3 decoder with a preloaded-stream +src/player/ packed.s is the DECODER-FREE player: it brings up its own + display, builds the DMA chain, keeps its own clock, fetches + every record off a real volume and -- since FINDINGS 68 -- feeds + the container's own audio lumps to the MSM6258 on channel 3 + while it does. Its audio half is three things: a lump ring + (pg_afill/pg_afetch), the remainder accumulator that stops the + padding becoming drift (pg_apay, 67.2), and a service routine + that runs from inside dma.i's transfer wait rather than once a + frame (pg_aserv through DM_HOOK). The last of those is the + whole difference between 0.51 ms of seam and 236 ms. + adpcm.i is the transport, every register byte of it decoded out + of the IPL ROM by tools/analysis/21_iplrom_dmac.py. + decode.s is the 68000 DLX3 decoder with a preloaded-stream front-end. stream.s is the same decoder behind a bounded ring. load.i is the LOAD-time half: codebook expansion and palette packing, out of the raw container header, with loadgate.s as diff --git a/docs/FINDINGS.md b/docs/FINDINGS.md index 7ff6ba5..874636c 100644 --- a/docs/FINDINGS.md +++ b/docs/FINDINGS.md @@ -7219,3 +7219,152 @@ whole sectors and the wire pays for the zero ones. disc is still unmeasured. * **The lump buffer is not allocated anywhere.** 65.3 charges 14,336 B for double-buffering the cadence and no player holds it. + +## 68. The container is played — and holding the bus costs the audio 463 times the seam (session 36) + +**ROADMAP P6c. Session 36.** `src/player/packed.s` (the audio path), +`src/player/dma.i` (`DM_HOOK`), `src/player/scsi.i`, `tools/encoder/adpcm.py` +(`decode_state`), `tools/bench/verify_packed_audio.py`, +`tools/bench/prep_packed.py`, `tools/bench/packed.lua`, +`tools/bench/packed_run.sh`, `tools/bench/check.sh`. + +**NAME THE LAYER.** 68000 code fetched an interleaved DLXP2 off a real MB89352 +volume, ran the video path against it, and fed the audio lumps to the machine's +**MSM6258 as MAME models it** through the IPL ROM's own HD63450 channel 3. The +capture is MAME's `-wavwrite`. **No board ran**, and every number below is the +emulated machine's. What is new is that a byte of this container has now been +**heard** on some layer, which 67.6 recorded that none had. + +### 68.1 It plays, and the gate is the speaker rather than the counters + +All **78,125 B** of the container's audio reached the chip, in order, and every +sample the chip produced from them is exact against `adpcm.py` run with the four +axes **out of the container's own header** — **156,257 samples**, `signal = +sample/8` recovering the chip's stream with a worst residual of 1 count. + +`packed.s` also reports 11 of 11 lumps armed, 11 fetched, 78,125 B of payload, +**0 starves and 0 re-arms with bytes still outstanding**. Those numbers are not +the gate and must not be: see 68.4. + +### 68.2 THE HEADLINE. Held, the bus costs the audio 236 ms of replayed byte + +The MSM6258 has no FIFO and **no starvation state**: when its DMA channel counts +out it goes on decoding nibbles out of whatever byte its data register still +holds (MAME `okim6258::sound_stream_update` reads `m_data_in` unconditionally +while PLAYING). So the interval between a channel counting out and the CPU +arming the next lump is not a gap — it is a **held nibble pair driving the +predictor**, and it is measurable in the capture as bytes that were played for +more than two nibbles. + +Same container, same player, same 120 frames, one bit of DCR different: + +| | stealing | held | +|---|---:|---:| +| audio service calls | **250,240** | **369** | +| ...per frame shown | 2,085 | 3.1 | +| lump boundaries with a seam | 4 of 10 | **10 of 10** | +| worst seam | 2 samples, **0.13 ms** | 1,138 samples, **72.83 ms** | +| total replayed | 8 samples, 0.51 ms | 3,694 samples, **236.42 ms** | +| ...as a share of the audio | 0.0051% | **2.31%** | +| bytes wrong | 0 | 0 | + +**The bytes are identical and the sound is not.** Held is 673x fewer looks at +the channel and 463x the seam. A 72.8 ms seam is most of a 12 fps frame of one +byte pair repeating: not a dropout, a buzz, once every 0.92 s. + +This is FINDINGS 64.3 arriving on the audio side. 64.3 measured a held channel +costing the frame CLOCK 47% of its ticks *without the clock being able to tell*; +the same halt costs the audio its service, and the audio can tell — in the only +place the interval exists, which is the sound. + +**And it is why the seam is small in the stealing configuration at all.** The +one place a 68000 driving this video path has time is **inside the transfer +wait**. Of the 250,240 service calls, **240 are the frame loop's** two-a-frame +and the other **250,000 came from `DM_HOOK`**, dma.i's per-trip hook. Held, the +same player gets 369: the same 240, plus the 129 single trips round the wait +that 131 burst transfers (120 records and 11 lumps) still allow. Servicing the +audio only from the frame loop is a 90 ms seam by construction, in either +configuration — the hook is the whole of the difference. + +### 68.3 The drift is measured now, not derived + +67.2 computed that feeding the chip the whole 7,168 B lump instead of the +payload runs 0.09% fast, and priced it at 1.25 s of lip-sync over the game. The +accumulator (`acc += F*hz; n = acc/(2*fps); acc %= 2*fps`, three lines in +`pg_apay`) is now in a player and the capture prices what it avoided, over the +10 lumps that carry a whole group: + + payload 71,614 B against 71,680 B of lump space + excess 66 B = 0.092% = 8.45 ms over 9.17 s + -> 1.26 s over the game's 22.8 min + +**67.2 predicted 1.25 and the machine played 1.26.** The accumulator ends the +scene at 1/24, which is where the arithmetic says it should. + +The gate does not merely *check* the payload lengths, it **cannot pass without +them**: the walk accounts for every byte of `DLXP.audio()`, and a whole-lump +player's stream contains 6.54 B of zero at the end of each group that are not in +that stream at all. + +### 68.4 Two bugs, and neither is visible from inside the player + +**(a) Clearing `DM_BARV` does not unchain a channel.** `dma.i` reads `DM_BARV` +to decide whether to write MAR and MTC, but the register the *channel* obeys is +**OCR bits 3-2**, which `pg_dmacfg` sets to %10 for the whole scene. So the +lump fetch wrote MAR and MTC and the channel ignored both and loaded them from +whatever BAR held (`hd63450.cpp` `dma_transfer_start`). It does not fail as a +bad address: the SPC's phase never completes and the read returns **POLL +TIMEOUT — on the lump and then on every record after it**. A wedged transport +that reads exactly like a disc problem. + +**(b) The refill overwrote the lump that was playing.** The resident set is +`AKF - AK + 1` lumps, not `AKF - AK`: `AK` is the *next* lump to arm, so the one +channel 3 is reading is `AK-1` and its slot is not free. Off by one, the fetcher +ran three lumps ahead of a three-slot ring and lump 3 landed on top of lump 0 +**while the channel was reading it**. + +**Every counter in the player stayed right**: 11 of 11 armed, 11 fetched, 78,125 +B of payload, no starve, no late re-arm — and the sound was wrong from **0.2 s** +in. This is 67.4's failure mode arriving on the audio side. Nothing parses a +packed container, so a wrong byte is not an error, it is a sound; the only +instrument that can see it is the stream the chip produced. + +### 68.5 The rig fact that shapes the gate: a byte is not always two nibbles + +MAME's `okim6258::data_w` sets `m_data_in` **and resets `m_nibble_shift` to 0**. +The DMA channel writes at 7,812.5 B/s and the stream decodes at 15,625 Hz, and +those are two independent clocks — so where a write lands inside a sound-stream +slice decides whether the byte before it was played as two nibbles, as one (the +high nibble dropped), or as three or more. + +Measured over the stealing run: **78,120 of 78,125 bytes were played as exactly +two nibbles (99.994%)**, one as one, four as four. So a whole-stream +`decode(data) == capture` cannot be the check, and a tolerance would not be one +either. The gate's model is one line — *byte `b` was played as `c` nibbles from +the cycle `(b&15, b>>4)`, `c >= 1`* — and it finds `c` for every byte, requiring +all `c` samples and 32 samples of continuation to be exact. **Every `c > 2` in +both runs was the last byte of a lump**, asserted, which is what makes the same +walk both the correctness check and the seam measurement. + +This is a property of the emulator's audio scheduling, not of the machine and +not of the format. On silicon the chip latches a byte and consumes both nibbles. + +### 68.6 What was NOT established + +* **Two slots was not tried.** Three lump slots and a prefill of two is what + ran; the minimum buffering depth is an experiment nobody has done, so 68's + 49,152 B of slot (21,504 B of lump) is an upper bound and not a requirement. + The slot stride is 16,384 B for a 7,168 B lump to make the address a shift; + a shipping player should charge `A*512` and pay the `mulu`. +* **The seam figures are this apparatus's.** They are set by how often the CPU + looks at the channel, which is set by how long the emulated transport takes to + deliver a record (90.7 ms against a 12 fps slot of 83.3). The *ratio* between + held and stealing is the finding; the absolute milliseconds are not `W`. +* **Nothing has been listened to.** The capture is an instrument. 66's risk is + unchanged and so is 66.3's: **the level is still open downward and the loudest + passage on the disc is still unmeasured.** +* **A second pass does not replay the audio.** `PG_ITER > 1` repeats the video + and leaves `PG_AK` at `PG_NLUMP`; the run that matters is one pass. +* **The silicon is untouched.** All four decoder axes are still MAME's, and + session 34's fifth hardware item — play a known nibble stream on a real + MSM6258V and record the line out — is still the cheap way to settle them. diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index 081b0fe..5ba6698 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -129,9 +129,26 @@ minutes**. The payload is a remainder, `acc += 11*15625; n = acc//24; acc %= 24` which is `clock.i`'s shape for `clock.i`'s reason and the **third** time this tree has met the pattern. The four ADPCM axes ride in the header and the gate flips each one to prove they earn it (**order −31.99 dB, formula −24.86**). The -wire is **589.6 KB/s**, 65.3's figure to the tenth. **What is left of P6 is the +wire is **589.6 KB/s**, 65.3's figure to the tenth. ~~**What is left of P6 is the last quarter: no audio has been played out of this container on any layer, and -two DMA channels have never run at once.** +two DMA channels have never run at once.**~~ + +Amended end of session 36: **P6c IS DONE, AND HOLDING THE BUS COSTS THE AUDIO +463x THE SEAM (FINDINGS 68).** All 78,125 B of the container's audio reached the +chip, in order, sample-exact against the four axes in its own header, out of a +player that was fetching records off the same disc at the same time. **Two DMA +channels have now run at once and the interaction is not in the bytes, it is in +the timing**: the MSM6258 has no starvation state, so the interval between a +channel counting out and the CPU arming the next lump is a held nibble pair +driving the predictor. Stealing, the seam is **0.51 ms over ten seconds**; +**held, it is 236 ms — 2.31% of the audio, every one of the ten lump boundaries, +worst 72.8 ms**, because a burst channel halts the 68000 and the only place a +68000 driving this video path has time is inside the transfer wait (250,000 of +250,240 service calls). That is FINDINGS 64.3 reaching the audio: **the held +configuration cannot feed a second consumer**, and B1's answer now decides a +sound as well as a picture. 67.2's drift is measured rather than derived — +predicted 1.25 s, played **1.26 s**. **What is left of P6 is the level (66.3) +and the refill climb with a second consumer through a real branch point.** **THE COMPLETION TARGET IS M3, THE VERTICAL SLICE** (USER DECISION): one scene tree — a decision point, two outcomes, a death clip — with audio, streaming from @@ -774,12 +791,18 @@ unchanged (F=11, A=14, 589.6 KB/s) and the **payload** did not: a lump's audio i 7,161 or 7,162 B of a 7,168 B sector run, and feeding the chip the whole lump drifts 1.25 s over the game. -**P6c. AUDIO OUT OF THE CONTAINER, ON THE MACHINE (new, session 35).** Every +**P6c. AUDIO OUT OF THE CONTAINER, ON THE MACHINE. DONE, session 36 — FINDINGS +68.** The lump buffer is `PG_ABUF` (three slots, prefill two, and the minimum +depth is still unmeasured), the remainder accumulator is `pg_apay`, and the +service runs from inside `dma.i`'s transfer wait through `DM_HOOK` because once +a frame is a 90 ms seam by construction. **Two channels have run at once**: the +bytes are identical in both DMAC configurations and the sound is not — held +costs 236 ms of replayed byte against stealing's 0.51 ms. ~~Every piece exists and none is joined up. The container carries the bytes (67); the transport is `src/player/adpcm.i`'s channel-3 configuration, which is the IPL ROM's own and worked first time (66.1). What is missing is **the lump buffer** — 14,336 B, double-buffered, allocated by nobody — and **the remainder -accumulator**, which is three instructions and is not optional. +accumulator**, which is three instructions and is not optional.~~ **And the interaction neither half has met: TWO CHANNELS AT ONCE.** The video channel holds the bus and halts the 68000, which already costs the frame clock @@ -915,11 +938,13 @@ B2 blanking ─┬─ NOT blanked ─> K3's DMAC-DIRECT player is the one: 54.9% B1 BURST rate (NEW, 64.2) ──> which of the two K3/K4 wins, if B2 blanks P6 audio: encoder gated (65), P6a the chip's own decoder measured (66), - P6b DLXP2 written and gated, 589.6 KB/s on the wire (67) - └─> P6c AUDIO OUT OF THE CONTAINER ─┬─ the bytes exist, the transport - needs no board: the transport │ exists, the lump buffer does not - is the IPL ROM's own (66.1) └─> TWO CHANNELS AT ONCE, which is - the one thing nothing has run + P6b DLXP2 written and gated, 589.6 KB/s on the wire (67), + P6c PLAYED -- 78,125 B to the chip beside the video channel (68) + └─> and the second consumer is now ANOTHER INPUT TO B1/B2: + stealing -> 0.51 ms of seam over 10 s \ same bytes, + held -> 236 ms, every lump boundary / different sound + └─> what is LEFT of P6: the LEVEL (66.3, oldest open item) and the + refill climb with the second consumer through a branch point ``` **Read that top-left branch as the project's live question.** Everything else diff --git a/docs/STATUS.md b/docs/STATUS.md index 4b31080..34bc5af 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -1,3 +1,128 @@ +# Status & next-session handoff — end of session 36 (2026-08-25) + +## Session 36: the container is played, and holding the bus costs the audio + +**Green light first and last: `./tools/bench/check.sh` was ALL GREEN before any +of this (`tmp/check_s36_start.log`) and ALL GREEN after +(`tmp/check_s36_end.log`)** — the same stages, plus one new one. + +**FINDINGS 68. ROADMAP P6c is DONE.** `src/player/packed.s` (the audio path), +`src/player/dma.i` (`DM_HOOK`), `src/player/scsi.i`, `tools/encoder/adpcm.py` +(`decode_state`), `tools/bench/verify_packed_audio.py`, +`tools/bench/prep_packed.py`, `tools/bench/packed.lua`, +`tools/bench/packed_run.sh`, `tools/bench/check.sh`. + +**NAME THE LAYER.** 68000 code fetched an interleaved DLXP2 off a real MB89352 +volume, ran the video path against it, and fed the lumps to the **MSM6258 as +MAME models it** through the IPL ROM's own channel 3. The instrument is MAME's +`-wavwrite`. **No board ran.** What is new is that a byte of this container has +now been *heard* on some layer — 67.6 recorded that none had. + +**1. IT PLAYS.** All **78,125 B** of the container's audio reached the chip, in +order, and every sample is exact against `adpcm.py` run with the four axes out +of the container's own header — **156,257 samples**, worst residual 1 count on +`signal = sample/8`. 11 of 11 lumps armed and fetched, 0 starves. + +**2. THE HEADLINE, AND IT IS 64.3 ARRIVING ON THE AUDIO SIDE.** The chip has no +FIFO and **no starvation state**: when its channel counts out it goes on +decoding the byte its data register still holds. So the gap between a count-out +and the next arm is a **held nibble pair driving the predictor**, and it is in +the capture. Same container, same player, one bit of DCR: + +| | stealing | held | +|---|---:|---:| +| service calls | 250,240 | 369 | +| boundaries with a seam | 4 of 10 | **10 of 10** | +| worst seam | 0.13 ms | **72.83 ms** | +| total replayed | 0.51 ms (0.005%) | **236.42 ms (2.31%)** | +| bytes wrong | 0 | 0 | + +**The bytes are identical and the sound is not.** 673x fewer looks at the +channel, 463x the seam. And the reason stealing wins is `DM_HOOK`: **250,000 of +the 250,240 service calls came from inside dma.i's transfer wait**, which is the +only time a 68000 driving this video path has. Once a frame is a 90 ms seam by +construction, in either configuration. + +**3. THE DRIFT IS MEASURED NOW.** `pg_apay` is 67.2's accumulator in a player, +and the capture prices what it avoided over the 10 lumps that carry a whole +group: 71,614 B against 71,680 B of lump space, **0.092%, 8.45 ms over 9.17 s -> +1.26 s over the game's 22.8 min. 67.2 predicted 1.25.** The accumulator ends at +1/24, where the arithmetic says. + +**4. TWO BUGS, AND NEITHER IS VISIBLE FROM INSIDE THE PLAYER.** Clearing +`DM_BARV` does not unchain a channel — the register the channel obeys is OCR +bits 3-2 — and the symptom is POLL TIMEOUT on the lump *and every record after +it*, which reads like a disc problem. And the refill ran one lump too far ahead +and **overwrote the buffer channel 3 was reading**: 11 of 11 armed, 11 fetched, +78,125 B, no starve, and the sound wrong from 0.2 s in. That is 67.4's failure +mode on the audio side and it is why the gate is a WAV. + +**5. THE RIG FACT THAT SHAPES THE GATE.** MAME's `okim6258::data_w` resets the +nibble select, and the DMA (7,812.5 B/s) and the stream (15,625 Hz) are +independent clocks — so a delivered byte is *usually* two nibbles and sometimes +one or three. Measured: **78,120 of 78,125 (99.994%)** were exactly two. So the +gate walks the stream **one delivered byte at a time**, and every stretched byte +in both runs was the last byte of a lump. + +**RISKS IN THIS SESSION'S RESULT:** + +- **Two slots was not tried.** Three lump slots and a prefill of two is what + ran; the minimum depth is unmeasured, so the RAM figure is an upper bound. +- **The seam milliseconds are the apparatus's**, set by how long the emulated + transport takes a record. The held/stealing RATIO is the finding. +- **Nothing has been listened to.** The capture is an instrument, not a listen. +- **The level is still open downward** (66.3) and the loudest passage on the + disc is still unmeasured. Unchanged since session 34. +- **The silicon is untouched.** All four axes are still MAME's. + +## HANDOFF — start here + +**THE TREE IS ALL GREEN**, session 36's stage included. + +### The work, in the order it should be done + +**1. THE AUDIO LEVEL, which 66.3 reopened and two sessions have now deferred.** +The 10-bit clamp is 12.1 dB below where the encoder was clamping and it is +INSIDE the recursion, so a scene that clips does not merely distort, it drives +the predictor. **Measure the loudest passage on the disc before choosing a +level.** This is the oldest open item in P6 and it needs no new machinery. + +**2. THE REFILL CLIMB WITH A SECOND CONSUMER, through a real branch point** +(51.3, 55.4). The last quarter of P6 that is not P6c. The slack table is in +`32_audio_wire.py`; nothing has been run. Session 36 gives it the thing it was +missing — a player that actually holds two streams at once — so this is now a +matter of pointing it at a scene change rather than of building anything. + +**3. THE BUFFERING DEPTH, which is cheap now.** `PG_ANBUF` is 3 and `PG_APRE` is +a mailbox. Two slots is one constant and one run, and it is 7,168 B of a +machine that has 99,328 B of record buffer to find elsewhere. + +### What is still BLOCKED, so it is not picked up by mistake + +**K4 — the packed player that is on screen — is conditional on B2**, a board +question. **E7, E4 and C1** are parked (61.8), and **P4a's wiring** is parked +with the ring K3 deleted. + +**The hardware list is unchanged and is the user's**: B1 (sustained AND the +data-phase BURST rate, 64.2 — acceptance is **589.6 KB/s**), B2 (blanking), B3 +(`#EXREQ`), B4 (a byte write to a palette register), and session 34's fifth: +play a known nibble stream on a real MSM6258V and record the line out. **Session +36 adds nothing to that list and strengthens the case for the fifth**: every +decoder axis the player now depends on is still MAME's. + +### Reproducing this session + + ./tools/bench/check.sh # ALL GREEN + bash tools/bench/packed_run.sh tmp/packed_singe.dlxp + # run 5 is the audio gate, run 6 the held contrast + python3 tools/bench/verify_packed_audio.py tmp/packed_singe.dlxp \ + tmp/packed_aud.wav tmp/packed_aud.json + +**WHAT IS NEXT.** The level, and then the refill climb with the second consumer +that now exists. + +--- + # Status & next-session handoff — end of session 35 (2026-08-25) ## Session 35: the packed container gets sound, and the padding is drift diff --git a/src/player/dma.i b/src/player/dma.i index cf11618..7deb387 100644 --- a/src/player/dma.i +++ b/src/player/dma.i @@ -126,6 +126,36 @@ DM_LEN = $18524 ; u32 bytes the channel was asked for ; DM_USE landed on ring.i's slot and turned a stage red (above). DM_BARV = $18528 ; u32 array base address, 0 = no chain DM_BTCV = $1852C ; u32 array entry count (BTC) +; ---- THE SERVICE HOOK, and it is the whole of what a SECOND consumer needs +; from this file. ROADMAP P6c. A player that feeds ADPCM has to look at the +; audio channel more often than once a frame: the MSM6258 has no FIFO, so the +; instant its channel counts out the chip goes on replaying whatever byte its +; data register still holds -- and at 12 fps a once-a-frame re-arm makes that +; replay 90 ms long, which is not a gap, it is a buzz. +; +; THE ONLY PLACE A 68000 HAS TO SPARE IS INSIDE THIS WAIT. In the STEALING +; configuration the CPU goes round the loop below thousands of times per record +; (measured: 1,100,520 trips over 120 frames) and every one of them is time the +; disc is delivering and the CPU is not. So the hook is called from there, and +; the second consumer costs the video path nothing it was using. +; +; AND IN THE HELD CONFIGURATION THE HOOK CANNOT RUN AT ALL, which is not a bug +; in it: a burst channel HALTS the 68000 (dma_transfer_start asserts +; INPUT_LINE_HALT) and the CPU does not execute the loop, or anything else, +; until the record has landed. That asymmetry is the measurement -- FINDINGS +; 64.3 showed a held channel costs the frame CLOCK half its ticks, and this is +; the same fact reaching the audio. +; +; ZERO BY DEFAULT and every other front-end in this tree leaves it zero, so the +; cost to them is a `move.l` and a `beq` per trip. A mailbox that defaulted to +; a new behaviour is how DM_USE landed on ring.i's slot (above). +DM_HOOK = $18530 ; u32 0 = none, else a routine to call + ; on every trip round the transfer wait. + ; d0 is dead here and a0 is saved round + ; the call, so the hook may trash both; + ; it must preserve EVERYTHING else, + ; because sc_in_dma's own d3/d4/d5 and + ; scsi_read's a1 are live across it. DM_PATIENCE = 4000000 ; the wait is bounded like every other ; ---------------------------------------------------------------- sc_in_dma @@ -191,6 +221,13 @@ sc_in_dma: clr.l DM_SPIN.l move.l #DM_PATIENCE,d3 .wait: addq.l #1,DM_SPIN.l + move.l DM_HOOK.l,d0 ; the second consumer's slot -- see above + beq.s .nohook + move.l a0,-(sp) + movea.l d0,a0 + jsr (a0) + movea.l (sp)+,a0 +.nohook: move.b DM_CSR,d4 btst #4,d4 ; ERR bne.s .err diff --git a/src/player/packed.s b/src/player/packed.s index 229a610..a6f9bc7 100644 --- a/src/player/packed.s +++ b/src/player/packed.s @@ -94,6 +94,28 @@ PG_CADF = $18924 ; audio cadence: frames between one lump and the ; screen, which is a picture, and a gate that ; only looked for errors would pass it. PG_CADA = $18928 ; ...and the sectors in a lump +PG_AUDON = $1892C ; 1 = PLAY the container's audio (ROADMAP P6c). + ; Separate from PG_CADF because the two are + ; different facts: the cadence has to be obeyed + ; by the LBA arithmetic whether or not a chip is + ; being fed, and a run that skips the lumps + ; without playing them is the control this one + ; is measured against. +PG_AFPS = $18960 ; the CONTAINER's fps, which is NOT PG_FPS. The + ; pace is a rig knob (DLX_PK_FPS) and the audio + ; was CUT at the container's rate; computing the + ; payload from the pace would make a 6 fps gate + ; run hand the chip half as many bytes a lump + ; and still look self-consistent. +PG_AHZ = $18964 ; the chip's sample rate out of the header +PG_ALBA0 = $18968 ; LBA of lump 0 (off_aud/512 -- sector 1) +PG_NLUMP = $1896C ; lumps in the scene +PG_ABYTES = $18970 ; the stream's total payload, so the LAST lump + ; is short rather than padded +PG_APRE = $18974 ; lumps fetched before frame 0. A NUMBER and + ; not a constant: what the audio needs in hand + ; is set by how far the transport runs behind + ; real time, which is a property of the medium. ; ---- outputs PG_SHOWN = $18930 ; frames displayed. Bumped AFTER bit 11 is @@ -117,6 +139,69 @@ PG_LOSTV = $18958 ; V-DISP edges the machine did not see, summed PG_ARRN = $1895C ; entries the array was built with (an assertion ; the rig reads back rather than a comment) +; ---- the audio path's own account. ROADMAP P6c. +PG_AARM = $18980 ; lumps ARMED on channel 3 +PG_AFET = $18984 ; lumps FETCHED off the disc +PG_ABYT = $18988 ; payload bytes handed to the chip, padding NOT + ; included -- the number FINDINGS 67.2 is about +PG_ADRY = $1898C ; times the service found the channel counted out + ; and NO lump fetched to follow it: a STARVE, and + ; the chip replays its last byte until it is fed +PG_ASEAM = $18990 ; service calls that found the channel counted + ; out. Held, one of these is a whole frame of + ; replayed byte; stealing, it is one trip round + ; the transfer wait. +PG_ASRV = $18994 ; times pg_aserv was ENTERED at all -- the + ; denominator PG_ASEAM is meaningless without, + ; and the direct measure of what holding the bus + ; costs a second consumer +PG_ACSR = $18998 ; channel 3's CSR at the end of the scene +PG_ACER = $1899C ; ...and CER with it +PG_ALATE = $189A0 ; re-arms issued with MTC still non-zero. By + ; construction this is 0 -- the service only + ; arms on COC -- and a non-zero reading means + ; bytes were dropped rather than played. +PG_AMTC0 = $189A4 ; MTC one instruction after the FIRST start, the + ; same witness dma.i uses: proof a byte moved +PG_AFERR = $189A8 ; SC_ERR of the first FAILED lump fetch. Its own + ; slot and not PG_ERR's: a lump fetch that fails + ; leaves the picture perfect and the sound gone, + ; and a shared error word would report it as a + ; video failure on whatever frame came next. +PG_AFERA = $189AC ; ...and the lump it failed on +; ---- and the state it keeps. In RAM and not in registers, because pg_aserv +; is called from inside dma.i's transfer wait as well as from the frame loop. +PG_AK = $189B0 ; next lump to ARM +PG_AKF = $189B4 ; next lump to FETCH +PG_AACC = $189B8 ; the remainder accumulator (FINDINGS 67.2) +PG_ABUSY = $189BC ; 1 = a lump fetch is in flight; the hook must + ; not re-enter the transport it is running under +PG_ASLOTA = $189C0 ; slot the next ARM reads from +PG_ASLOTF = $189C4 ; slot the next FETCH writes to + +; ---- the lump buffers. 65.3 charged 14,336 B for double-buffering the cadence +; and FINDINGS 67.6 recorded that no player held it; this is where it is held. +; +; THREE SLOTS, AND THE MINIMUM IS NOT KNOWN. Three is what was run and what +; FINDINGS 68 measures; two was not tried. The reason to suspect the answer is +; not one is that the two clocks are independent -- the chip consumes a lump in +; exactly 11/12 s and the video path decides when the next one is fetched, and +; under this apparatus the video path is 8.9% slow (a record lands in 90.7 ms +; against a 12 fps slot of 83.3, packed_free_steal). Whether that costs a third +; slot or only a deeper prefill is an experiment nobody has run, so PG_APRE is a +; mailbox rather than a constant and this number is an upper bound rather than a +; requirement. +; +; THE SLOT IS 16,384 B FOR A*512 = 7,168 B OF LUMP, which is 9,216 B a slot of +; address space nothing uses. It buys a shift instead of a multiply on the slot +; address and it is only address space -- nothing is written to it -- but it is +; not free on a machine with a filesystem in it and a shipping player should +; charge A*512 and pay the mulu. +PG_ABUF = $20000 +PG_ASTRIDE = $4000 ; 16,384 B a slot -- A*512 = 7,168 with room, and + ; a power of two so the slot address is a shift +PG_ANBUF = 3 + ; ---- the array chain. 193 x 6 B = 1,158 B, built once at scene setup. ; $1B000 and not $19000: dmagate.s's two arrays live at $19000/$19100 and ; ring.i's disc-offset table at $19400, and a front-end that shares an address @@ -183,6 +268,24 @@ start: clr.l PG_TSPIN.l clr.l PG_GSPIN.l clr.l PG_LOSTV.l + clr.l PG_AARM.l + clr.l PG_AFET.l + clr.l PG_ABYT.l + clr.l PG_ADRY.l + clr.l PG_ASEAM.l + clr.l PG_ASRV.l + clr.l PG_ACSR.l + clr.l PG_ACER.l + clr.l PG_ALATE.l + clr.l PG_AMTC0.l + clr.l PG_AFERR.l + move.l #-1,PG_AFERA.l + clr.l PG_AK.l + clr.l PG_AKF.l + clr.l PG_AACC.l + clr.l PG_ABUSY.l + clr.l PG_ASLOTA.l + clr.l PG_ASLOTF.l ; ---- 1. the display. THE PLAYER OWNS IT, and that is a change from every ; other rig in this tree: tools/bench/stream.lua and decode.lua call @@ -224,7 +327,14 @@ start: .noclk: move.l CLK_VDISP.l,PG_VD0.l -; ---- 6. the scene. +; ---- 6. THE CHIP, and it is last for the same reason the clock was: ad_setup +; touches the YM2151 and the PPI, and the ~100 ms it then has to settle for +; (FINDINGS 66.4) is 100 ms this player would otherwise spend with a transport +; half brought up. It also has to be after scsi_init, because the prefill is a +; READ(10) like any other. + bsr pg_ainit + +; ---- 7. the scene. move.l PG_ITER.l,d6 tst.l d6 bne.s .it @@ -234,7 +344,20 @@ pg_pass: moveq #0,d7 ; d7 = frame index within the pass pg_loop: bsr pg_gate ; wait for tick d7 +; THE AUDIO IS SERVICED THREE TIMES A FRAME AND THAT IS NOT BELT AND BRACES. +; Once here, once after the record has landed, and -- in the STEALING +; configuration only -- on every trip round dma.i's transfer wait, through +; DM_HOOK. The chip has no FIFO: the instant channel 3 counts out it goes on +; replaying whatever byte its data register still holds, so the interval between +; a count-out and the next arm is not silence, it is a held nibble pair driving +; the predictor. Held, the CPU is halted for the whole transfer and the two +; calls below are the ONLY two it gets; that difference is what PG_ASRV counts. + bsr pg_aserv bsr pg_frame ; and paint it + move.l d0,-(sp) + bsr pg_aserv + bsr pg_afill ; ...and fetch at most ONE lump, so the + move.l (sp)+,d0 ; audio's share of the frame is bounded tst.l d0 bmi.s pg_failed addq.l #1,d7 @@ -251,11 +374,13 @@ pg_loop: move.l CLK_VDISP.l,PG_VDISP.l bsr clk_stop + bsr pg_areport move.l #$FF,PG_FLAG.l bra.s pg_hold pg_failed: move.l CLK_VDISP.l,PG_VDISP.l bsr clk_stop + bsr pg_areport move.l #$E2,PG_FLAG.l pg_hold: bra.s pg_hold @@ -528,7 +653,343 @@ pg_static: movem.l (sp)+,d0-d3/a0-a1 rts + +; =========================================================================== +; THE AUDIO PATH. ROADMAP P6c -- the container's own bytes, out of a channel, +; beside the video channel that holds the bus. +; +; WHAT IS NEW HERE AND WHAT IS NOT. The transport is not new: it is adpcm.i's +; channel-3 configuration, every byte of which tools/analysis/21_iplrom_dmac.py +; decoded out of the IPL ROM, and it worked first time (FINDINGS 66.1). The +; container is not new either: DLXP2 has carried the lumps since FINDINGS 67. +; What no layer of this project has ever done is JOIN THEM -- and the two things +; that had to be built to do it are the two 67.6 named as missing: +; +; THE LUMP BUFFER, which is below and is 3 x 16,384 B rather than the 14,336 B +; 65.3 charged. Two slots is right when the transport keeps up with real +; time. This apparatus does not: a record lands in 90.7 ms against a 12 fps +; slot of 83.3 ms, so the video path runs 8.9% slow and the chip does not +; run slow with it -- 15,625 Hz is a crystal, not a pace. The third slot is +; the prefill that covers the difference and PG_APRE is how much of it is +; filled before frame 0. +; +; THE REMAINDER ACCUMULATOR, which is pg_apay and is the whole of FINDINGS +; 67.2. A lump is A*512 = 7,168 B of SPACE; eleven frames of audio is +; 11*15625/24 = 7,161.4583... B. Handing the chip the lump instead of the +; payload is 0.09% too much audio, and 0.09% is not waste, it is DRIFT: +; 0.84 ms a group, 1.25 s of lip-sync over the game's 22.8 minutes. It is +; the same arithmetic clock.i carries one dimension over, and it is the +; third time this tree has met it (54, 65.3, 67.2). +; +; AND THE FAILURE MODE IS THE FORMAT'S OWN (67.4): nothing here parses anything. +; A lump read one sector out is not an error, it is 7,168 B of noise; a payload +; length one byte out is not an error either, it is a rate. Both are why +; tools/bench/verify_packed_audio.py reads the CAPTURE rather than the counters. + +; ---------------------------------------------------------------- pg_ainit +; Bring the chip up, prefill PG_APRE lumps, arm the first and start playing. +; ORDER IS THE GATE'S, unchanged from adpcmgate.s and for its reason: the chip +; resets its accumulator, its step index AND its nibble select when it is told +; to PLAY and begins consuming immediately out of whatever its data register +; holds -- so the CHANNEL goes first and the CPU waits for MTC to move, which is +; proof a byte was actually taken, before the PLAY. +pg_ainit: + tst.l PG_AUDON.l + beq .out + movem.l d0-d4/a1,-(sp) + bsr ad_setup +; ---- SETTLE, and FINDINGS 66.4 is why. The 8 MHz ADPCM clock is CT1 in the +; YM2151's port register $1B -- a different device from the divider -- and this +; machine delivers that write on the sound system's own schedule rather than at +; the store. A transfer started in the same breath plays its first ~17 ms at +; the PREVIOUS clock, which reads exactly like a broken probe. A player sets +; its clock once at boot and can afford 100 ms; this is that once. + move.l #60000,d0 +.settle:subq.l #1,d0 + bne.s .settle + +; ---- the prefill. At least one lump, or there is nothing to arm. + move.l PG_APRE.l,d4 + bne.s .pre + moveq #1,d4 +.pre: + cmp.l PG_NLUMP.l,d4 + bls.s .pl + move.l PG_NLUMP.l,d4 +.pl: tst.l d4 + beq.s .done +.pf: bsr pg_afetch + subq.l #1,d4 + bne.s .pf + +; ---- arm lump 0, then wait for the channel to actually take byte 0 before +; PLAY. Not a delay loop: the condition is MTC having moved, which is the +; channel's own account of a byte having left RAM. + bsr pg_aarm + move.l #4000000,d3 + move.w A3_MTC,d0 + andi.l #$FFFF,d0 + move.l d0,d2 ; d2 = MTC as armed +.first: move.w A3_MTC,d0 + andi.l #$FFFF,d0 + cmp.l d2,d0 + bne.s .go + subq.l #1,d3 + bne.s .first +.go: move.l d0,PG_AMTC0.l + bsr ad_play +; ---- and NOW the hook, which is the second consumer's whole claim on the video +; path (dma.i). After the play, so that a trip round a transfer wait cannot +; find a channel that has not been started yet. + move.l #pg_aserv,DM_HOOK.l +.done: + movem.l (sp)+,d0-d4/a1 +.out: rts + +; ---------------------------------------------------------------- pg_aserv +; ONE LOOK AT THE AUDIO CHANNEL. Called from the frame loop twice a frame and, +; in the stealing configuration, from inside dma.i's transfer wait -- so it must +; preserve every register and must be cheap on the path where nothing is due. +; +; IT ARMS ONLY ON COC. The alternative -- re-arm at the group boundary whatever +; the channel is doing -- drops MTC bytes on the floor, and dropped bytes are a +; rate error in the same direction 67.2's padding was. PG_ALATE is the +; assertion that it never happens and is 0 on a correct run. +pg_aserv: + movem.l d0-d2/a1,-(sp) + addq.l #1,PG_ASRV.l + tst.l PG_AUDON.l + beq.s .out + tst.l PG_ABUSY.l ; a lump fetch is in flight and IT is + bne.s .out ; the transfer this hook is running + ; under -- see pg_afetch + move.l PG_AK.l,d0 + cmp.l PG_NLUMP.l,d0 + bcc.s .out ; the scene's audio is all armed + moveq #0,d1 + move.b A3_CSR,d1 + btst #7,d1 ; COC: the channel counted out and the + beq.s .out ; chip is now replaying its last byte + addq.l #1,PG_ASEAM.l + move.l PG_AKF.l,d1 + cmp.l d1,d0 + bcs.s .arm + addq.l #1,PG_ADRY.l ; STARVE: nothing fetched to follow it. + bra.s .out ; Counted, not absorbed -- the chip is + ; audibly stuck on one byte pair. +.arm: bsr pg_aarm +.out: movem.l (sp)+,d0-d2/a1 + rts + +; ---------------------------------------------------------------- pg_aarm +; Arm channel 3 on the lump in slot PG_ASLOTA for its PAYLOAD length, and +; advance. Trashes d0-d2/a1, which pg_aserv and pg_ainit have saved. +pg_aarm: + tst.l PG_AARM.l + beq.s .clean ; the FIRST arm has no previous lump and + ; MTC out of reset is not ours to read + move.w A3_MTC,d0 + andi.l #$FFFF,d0 + beq.s .clean + addq.l #1,PG_ALATE.l ; bytes the previous lump never played +.clean: + bsr pg_apay ; d0 = this lump's payload + tst.l d0 + beq.s .skip + move.l d0,d1 + add.l d0,PG_ABYT.l + move.l PG_ASLOTA.l,d0 + lsl.l #8,d0 + lsl.l #6,d0 ; slot * $4000 + add.l #PG_ABUF,d0 + movea.l d0,a1 + bsr ad_arm ; d1 = MTC, a1 = MAR + addq.l #1,PG_AARM.l +.skip: + addq.l #1,PG_AK.l + move.l PG_ASLOTA.l,d0 + addq.l #1,d0 + cmp.l #PG_ANBUF,d0 + bcs.s .st + moveq #0,d0 +.st: move.l d0,PG_ASLOTA.l + rts + +; ---------------------------------------------------------------- pg_apay +; d0 = the PAYLOAD of the next lump, in bytes. FINDINGS 67.2, and it is three +; lines of arithmetic because the alternative is 1.25 s of lip-sync: +; +; acc += F*hz ; n = acc / (2*fps) ; acc %= (2*fps) +; +; THE fps IS THE CONTAINER'S (PG_AFPS) AND NOT THE PACE. The stream was cut at +; the rate it was cut at; a run paced slower is a run whose PICTURES are slow, +; and computing the payload from the pace would hand the chip half a lump and +; still balance. +; +; divu is a WORD divide and the quotient has to fit 16 bits: at F=11, 15,625 Hz +; and 12 fps it is 7,161 or 7,162, and the accumulator it is taken from never +; exceeds F*hz + (2*fps-1) = 171,898. +pg_apay: + movem.l d1-d2,-(sp) + move.l PG_CADF.l,d0 + move.l PG_AHZ.l,d1 + mulu d1,d0 ; F*hz = 171,875 + add.l PG_AACC.l,d0 + move.l PG_AFPS.l,d1 + add.l d1,d1 ; 2*fps = 24, two samples to a byte + divu d1,d0 + move.l d0,d2 + clr.w d2 + swap d2 ; divu leaves the REMAINDER in the high + move.l d2,PG_AACC.l ; half, and it is 11 at F=11 -- which is + andi.l #$FFFF,d0 ; the whole finding +; ---- and the LAST lump is short rather than padded: the stream ends where it +; ends and the geometry would otherwise hand the chip whatever followed it. + move.l PG_ABYTES.l,d1 + sub.l PG_ABYT.l,d1 + bcc.s .lim + moveq #0,d1 +.lim: cmp.l d1,d0 + bls.s .out + move.l d1,d0 +.out: movem.l (sp)+,d1-d2 + rts + +; ---------------------------------------------------------------- pg_afill +; At most ONE lump a frame, and only if a slot is free. One, because a lump is +; A*512 = 7,168 B off the same disc the record came off and a frame that fetched +; two would spend 26 ms of its slot on audio. +; +; AND "FREE" COUNTS THE LUMP THAT IS STILL PLAYING, which is the bug this +; session shipped and the capture caught. PG_AK is the NEXT lump to arm, so the +; one channel 3 is reading right now is PG_AK-1 and its slot is not free -- the +; resident set is PG_AKF-PG_AK+1 lumps, not PG_AKF-PG_AK. Off by one, the +; fetcher runs three lumps ahead instead of two and lump 3 lands on top of lump +; 0 WHILE THE CHANNEL IS READING IT. +; +; NOTHING IN THE PLAYER CAN SEE THAT HAPPEN. Every counter stays right: 11 of +; 11 lumps armed, 11 fetched, 78,125 B of payload, no starve, no late re-arm -- +; and the sound is wrong from 0.2 s in, because the channel goes on delivering +; bytes at exactly the right rate out of a buffer somebody else is writing. +; That is FINDINGS 67.4's failure mode arriving on the audio side: nothing +; parses a packed container, so a wrong byte is not an error, it is a sound. +; It is why the gate reads the CAPTURE (tools/bench/verify_packed_audio.py). +pg_afill: + movem.l d0-d1,-(sp) + tst.l PG_AUDON.l + beq.s .out + move.l PG_AKF.l,d0 + cmp.l PG_NLUMP.l,d0 + bcc.s .out ; the whole stream is fetched + sub.l PG_AK.l,d0 + addq.l #1,d0 ; ...+1 for the lump being PLAYED out of + ; its slot right now (see above) + cmp.l #PG_ANBUF,d0 + bcc.s .out ; every slot is spoken for + bsr pg_afetch +.out: movem.l (sp)+,d0-d1 + rts + +; ---------------------------------------------------------------- pg_afetch +; READ(10) lump PG_AKF into slot PG_ASLOTF. Trashes d0-d4/a1. +; +; lump k is at LBA PG_ALBA0 + k*(F*PG_RECS + PG_CADA) +; +; ARITHMETIC, like the record's, and for the same reason: a group is F records +; and one lump of A sectors, both fixed, so the format still has no index and +; still needs none (FINDINGS 67.1). That was the packed branch's whole claim +; and a second stream at an unrelated rate was the thing that could have ended +; it; it did not. +; +; THREE THINGS ARE TURNED OFF ROUND THE READ AND ALL THREE MATTER. +; +; DM_BARV, because the video path leaves the channel pointed at an array of +; 193 GVRAM destinations and a lump fetched through it would be painted on +; the screen instead of played. +; +; THE OCR's CHAIN FIELD, AND CLEARING DM_BARV IS NOT ENOUGH -- this cost the +; session its first wrong answer. dma.i reads DM_BARV to decide whether to +; write MAR and MTC, but the register the CHANNEL obeys is OCR bits 3-2, and +; pg_dmacfg has set them to %10 for the whole scene. So the "unchained" +; read wrote MAR and MTC and the channel then ignored both and loaded them +; from whatever BAR happened to hold (hd63450.cpp dma_transfer_start). It +; does not fail as a bad address: the SPC's phase never completes and the +; read comes back POLL TIMEOUT, on the LUMP and then on every record after +; it -- a wedged transport that reads like a disc problem. +; +; DM_HOOK's reentry, through PG_ABUSY: pg_aserv is called from inside the very +; transfer wait this read is running under, and an arm issued from there +; would point channel 3 at a buffer being written into. +pg_afetch: + movem.l d0-d5/a1,-(sp) + move.l #1,PG_ABUSY.l + move.l DM_BARV.l,-(sp) + move.l DM_OCRV.l,-(sp) + clr.l DM_BARV.l + move.l DM_OCRV.l,d0 + andi.l #$FFFFFFF3,d0 ; CHAIN = %00, no chaining + move.l d0,DM_OCRV.l + + move.l PG_CADF.l,d3 + move.l PG_RECS.l,d0 + mulu d0,d3 ; F records... + move.l PG_CADA.l,d0 + add.l d0,d3 ; ...and the lump in front of them + move.l PG_AKF.l,d0 + mulu d3,d0 ; k * the group's sectors + add.l PG_ALBA0.l,d0 + move.l d0,d3 + move.l PG_CADA.l,d4 + move.l PG_ASLOTF.l,d0 + lsl.l #8,d0 + lsl.l #6,d0 + add.l #PG_ABUF,d0 + movea.l d0,a1 + bsr scsi_read + tst.l d0 + bmi.s .err + addq.l #1,PG_AFET.l + addq.l #1,PG_AKF.l + move.l PG_ASLOTF.l,d0 + addq.l #1,d0 + cmp.l #PG_ANBUF,d0 + bcs.s .st + moveq #0,d0 +.st: move.l d0,PG_ASLOTF.l + bra.s .done +.err: tst.l PG_AFERR.l + bne.s .done + move.l SC_ERR.l,PG_AFERR.l + move.l PG_AKF.l,PG_AFERA.l +.done: + move.l (sp)+,DM_OCRV.l + move.l (sp)+,DM_BARV.l + clr.l PG_ABUSY.l + movem.l (sp)+,d0-d5/a1 + rts + +; ---------------------------------------------------------------- pg_areport +; The channel's own account, and the chip stopped. ad_halt matters to the +; CAPTURE and not to the machine: left playing the chip goes on replaying its +; last byte forever, and tools/bench/verify_packed_audio.py would then have to +; find the end of the stream in a tail that never ends. +pg_areport: + tst.l PG_AUDON.l + beq.s .out + movem.l d0,-(sp) + clr.l DM_HOOK.l + moveq #0,d0 + move.b A3_CSR,d0 + move.l d0,PG_ACSR.l + moveq #0,d0 + move.b A3_CER,d0 + move.l d0,PG_ACER.l + bsr ad_halt + movem.l (sp)+,d0 +.out: rts + include "src/player/clock.i" + include "src/player/adpcm.i" ; scsi.i includes src/player/dma.i itself, at its foot -- so it is NOT included ; here. The DMAC constants and sc_in_dma come in with the transport that uses ; them, which is also why dmagate.s includes only scsi.i. diff --git a/src/player/scsi.i b/src/player/scsi.i index 8be8013..912a779 100644 --- a/src/player/scsi.i +++ b/src/player/scsi.i @@ -241,6 +241,12 @@ sci1: nop clr.l SC_ERR.l clr.l DM_USE.l ; PIO unless a caller asks otherwise, ; AFTER this call (src/player/dma.i) + clr.l DM_HOOK.l ; ...and no second consumer is being + ; serviced until one installs itself, + ; for the same reason and in the same + ; direction: a mailbox that defaults to + ; a behaviour is how DM_USE landed on + ; ring.i's slot. clr.l SC_TRN.l move.l #0,SC_TAG.l bsr sc_snap diff --git a/tools/bench/check.sh b/tools/bench/check.sh index e8fd051..4da1d98 100755 --- a/tools/bench/check.sh +++ b/tools/bench/check.sh @@ -741,12 +741,41 @@ if [ -f "$HOME/mame/roms/x68000.zip" ] || [ -d "$HOME/mame/roms/x68000" ]; then DLX_PK_GATE_ONLY=1 DLX_PK_NFR="$NF" bash tools/bench/packed_run.sh \ tmp/packed_singe.dlxp > tmp/packed_gate_stage.log 2>&1 \ || { cat tmp/packed_gate_stage.log; exit 1; } - grep -aE "^ (FLAG|array|chain|frame clock|late frames|WRITE WINDOW)|^OK " \ - tmp/packed_gate_stage.log + # SPLIT AT RUN 5, because the audio run below is in the same log and its + # verifier emits OK lines too -- one grep over the whole file would print + # session 36's result under session 31's heading. + sed -n '1,/--- 5. THE AUDIO/p' tmp/packed_gate_stage.log | \ + grep -aE "^ (FLAG|array|chain|frame clock|late frames|WRITE WINDOW)|^OK " else echo " SKIPPED: no x68000 romset -- the player was not run" fi +echo "--- session 36: THE CONTAINER'S OWN AUDIO, OUT OF THE CHIP (FINDINGS 68) ---" +# ROADMAP P6c. Every piece of this existed before this stage did and none of it +# was joined up: the container has carried the lumps since 67, the transport is +# the IPL ROM's channel-3 configuration from 66, and what was missing was the +# lump buffer and the remainder accumulator in a player. This is the run where +# a byte of the container reaches the speaker. +# +# WHY THE GATE IS A WAV AND NOT A COUNTER. src/player/packed.s reports lumps +# armed, lumps fetched, payload bytes and starves, and EVERY ONE OF THOSE CAN BE +# RIGHT WHILE THE SOUND IS WRONG -- which is not hypothetical, it is the bug +# this session shipped: the refill ran one lump too far ahead and overwrote the +# buffer channel 3 was reading out of, and the player's account of it was +# 11 of 11 armed, 11 fetched, 78,125 B, no starve. Nothing parses a packed +# container (FINDINGS 67.4), so a wrong byte is not an error, it is a sound. +# tools/bench/verify_packed_audio.py accounts for all 78,125 bytes against +# MAME's own capture, one delivered byte at a time. +# +# The run is part of packed_run.sh's gate half, so DLX_PK_GATE_ONLY takes it. +if [ -f tmp/packed_gate_stage.log ] && \ + grep -aq -- "--- 5. THE AUDIO" tmp/packed_gate_stage.log; then + sed -n '/--- 5. THE AUDIO/,$p' tmp/packed_gate_stage.log | \ + grep -aE "^(OK|FAIL) |^ AUDIO:|^ {5}(nibbles per|worst|payload|-> |the player)" +else + echo " SKIPPED: no x68000 romset, or the container is silent" +fi + echo "--- session 33: AUDIO -- the encoder, and what it does to the wire (FINDINGS 65) ---" # ROADMAP P6, everything in it except the bus half session 20 closed. The audio # is the SAME WINDOW as the frames -- 00223 from 539.4 s for 10 s -- because an diff --git a/tools/bench/packed.lua b/tools/bench/packed.lua index 2153c4a..2e3f68d 100644 --- a/tools/bench/packed.lua +++ b/tools/bench/packed.lua @@ -68,11 +68,19 @@ local META = loadfile("packed_meta.lua")() local PG_FLAG, PG_NFR, PG_FPS, PG_LBA0 = 0x18900, 0x18904, 0x18908, 0x1890C local PG_RECS, PG_PALL, PG_HELD = 0x18910, 0x18914, 0x18918 local PG_PACEON, PG_ITER = 0x1891C, 0x18920 -local PG_CADF, PG_CADA = 0x18924, 0x18928 +local PG_CADF, PG_CADA, PG_AUDON = 0x18924, 0x18928, 0x1892C local PG_SHOWN, PG_ERR, PG_ERRAT = 0x18930, 0x18934, 0x18938 local PG_LATE, PG_LATE1, PG_LATEM = 0x1893C, 0x18940, 0x18944 local PG_VDISP, PG_VD0, PG_TSPIN = 0x18948, 0x1894C, 0x18950 local PG_GSPIN, PG_LOSTV, PG_ARRN = 0x18954, 0x18958, 0x1895C +-- ROADMAP P6c: the audio path's inputs and its own account of what it did. +local PG_AFPS, PG_AHZ, PG_ALBA0 = 0x18960, 0x18964, 0x18968 +local PG_NLUMP, PG_ABYTES, PG_APRE = 0x1896C, 0x18970, 0x18974 +local PG_AARM, PG_AFET, PG_ABYT = 0x18980, 0x18984, 0x18988 +local PG_ADRY, PG_ASEAM, PG_ASRV = 0x1898C, 0x18990, 0x18994 +local PG_ACSR, PG_ACER, PG_ALATE = 0x18998, 0x1899C, 0x189A0 +local PG_AMTC0, PG_AK, PG_AKF, PG_AACC = 0x189A4, 0x189B0, 0x189B4, 0x189B8 +local PG_AFERR, PG_AFERA = 0x189A8, 0x189AC local PG_ARR = 0x1B000 -- src/player/clock.i and src/player/scsi.i, read for diagnosis only. local CLK_PACE, CLK_VDISP, CLK_ERR = 0x18034, 0x18064, 0x1806C @@ -85,6 +93,13 @@ local ITERS = tonumber(os.getenv("DLX_PK_ITER") or "") or 1 local NFR = tonumber(os.getenv("DLX_PK_NFR") or "") or META.nframes local FPS = tonumber(os.getenv("DLX_PK_FPS") or "") or META.fps local CSV = os.getenv("DLX_PK_CSV") +-- ROADMAP P6c. OFF by default and it is not a convenience: a run with the chip +-- silent is the CONTROL this one is read against, and every gate that existed +-- before session 36 is that control. DLX_PK_APRE is the lumps fetched before +-- frame 0 -- see src/player/packed.s on why the answer is not 1. +local AUDIO = (os.getenv("DLX_PK_AUD") or "0") == "1" +local APRE = tonumber(os.getenv("DLX_PK_APRE") or "") or 2 +local AJSON = os.getenv("DLX_PK_AJSON") local SNAP_DELAY = 2 local SCERRNAME = {[0]="OK", "SELECTION TIMEOUT -- no target answered", @@ -114,6 +129,16 @@ local function setup() SP:write_u32(PG_CADF, META.cad_f or 0) SP:write_u32(PG_CADA, META.cad_a or 0) SP:write_u32(PG_SHOWN, 0) + -- P6c. AUDON is separate from the cadence on purpose: the LBA arithmetic has + -- to skip the lumps whether or not a chip is being fed, and a run that skips + -- them without playing them is the control this one is measured against. + SP:write_u32(PG_AUDON, (AUDIO and (META.has_audio or 0) == 1) and 1 or 0) + SP:write_u32(PG_AFPS, META.fps) -- the CONTAINER's, NOT the pace + SP:write_u32(PG_AHZ, META.aud_hz or 0) + SP:write_u32(PG_ALBA0, META.lba_aud or 0) + SP:write_u32(PG_NLUMP, META.n_lumps or 0) + SP:write_u32(PG_ABYTES, META.aud_bytes or 0) + SP:write_u32(PG_APRE, APRE) P(string.format("packed.bin=%d B, %dx%d %d fps, %d of %d frames, %d passes", #code, META.W, META.H, META.fps, NFR, META.nframes, ITERS)) local cad = "" @@ -130,6 +155,19 @@ local function setup() .."whole cost of it on the video path", META.aud_bytes, META.aud_hz, META.n_lumps, META.cad_a, META.cad_f)) end + if AUDIO and (META.has_audio or 0) == 1 then + P(string.format("AUDIO ON: lump k at LBA %d + k*%d, payload 11*%d/24 B a " + .."group -- the PAYLOAD and not the %d B lump (FINDINGS " + .."67.2). Decoder from the header: %s/%s, %d-bit clamp, " + .."accumulator %d at PLAY. Prefill %d lumps of %d.", + META.lba_aud, META.cad_f*META.rec_sectors + META.cad_a, + META.aud_hz, META.cad_a*512, META.aud_variant, + META.aud_order, META.aud_bits, META.aud_init, + APRE, META.n_lumps)) + elseif (META.has_audio or 0) == 1 then + P("audio present in the container and NOT played -- this is the silent " + .."control (DLX_PK_AUD=1 plays it)") + end P(string.format("channel: %s, %s", HELD and "BUS HELD (burst, max rate)" or "CYCLE STEALING", PACED and ("SELF-PACED at "..FPS.." fps off V-DISP" @@ -268,6 +306,74 @@ SUB = emu.add_machine_frame_notifier(function() P(string.format("CPU: %d trips round the TRANSFER wait in total, %d round " .."the PACE gate", SP:read_u32(PG_TSPIN), SP:read_u32(PG_GSPIN))) + if AUDIO and (META.has_audio or 0) == 1 then + local armed, fet = SP:read_u32(PG_AARM), SP:read_u32(PG_AFET) + local byt, dry = SP:read_u32(PG_ABYT), SP:read_u32(PG_ADRY) + local seam, srv = SP:read_u32(PG_ASEAM), SP:read_u32(PG_ASRV) + local late, acc = SP:read_u32(PG_ALATE), SP:read_u32(PG_AACC) + P(string.format("AUDIO: %d of %d lumps armed, %d fetched, %d B of " + .."payload handed to the chip (the stream is %d B)", + armed, META.n_lumps, fet, byt, META.aud_bytes)) + -- THE PADDING, CHARGED. A player that fed the chip the whole lump + -- would have handed it n_lumps*A*512 B; the difference is the drift + -- FINDINGS 67.2 priced at 1.25 s over the game, and printing both + -- numbers is the only way the accumulator is visible from outside. + -- FULL groups only. The last lump is short when the scene's frame + -- count is not a multiple of F, and averaging that in reports the + -- scene's TAIL as though it were the cadence -- which is a different + -- number from the drift and looks like a worse one. The percentage + -- lives in verify_packed_audio.py, which knows each lump's payload. + local nfull = META.n_lumps + if NFR % META.cad_f ~= 0 then nfull = nfull - 1 end + local lumpb = nfull * META.cad_a * 512 + P(string.format(" %d whole groups: %d B of lump space for the " + .."payload the accumulator asked for. The whole-lump " + .."player feeds the chip that space, and the excess is " + .."DRIFT and not waste (67.2). Accumulator left at " + .."%d/%d.", nfull, lumpb, acc, 2*META.fps)) + P(string.format(" service: %d calls, %d found the channel " + .."counted out, %d of those had NO lump ready (a " + .."STARVE -- the chip replays its last byte)", + srv, seam, dry)) + P(string.format(" re-arms with MTC still non-zero: %d (bytes " + .."fetched and never played; 0 is the correct value)", + late)) + local ferr = SP:read_u32(PG_AFERR) + if ferr ~= 0 then + P(string.format(" A LUMP FETCH FAILED on lump %d: %s -- the " + .."picture is unaffected and the sound is gone, which " + .."is why this has its own error word", + SP:read_u32(PG_AFERA), SCERRNAME[ferr] or ("code "..ferr))) + end + P(string.format(" channel 3 at the end: CSR=$%02X CER=$%02X, " + .."MTC one instruction after the first START = %d", + SP:read_u32(PG_ACSR), SP:read_u32(PG_ACER), + SP:read_u32(PG_AMTC0))) + -- WHAT HOLDING THE BUS COSTS A SECOND CONSUMER, and it is this line. + -- Stealing, pg_aserv runs from inside dma.i's transfer wait as well as + -- twice a frame; held, the 68000 is HALTED for the whole transfer and + -- the two frame-loop calls are all it gets. The ratio is the audio's + -- half of FINDINGS 64.3. + P(string.format(" -> %.1f service calls per frame shown. %s", + shown > 0 and srv/shown or 0, + HELD and ("BUS HELD: the 68000 is halted for the whole " + .."transfer, so DM_HOOK never runs and this is the " + .."two frame-loop calls and nothing else.") + or ("CYCLE STEALING: DM_HOOK ran from inside the " + .."transfer wait, which is where a 68000 driving this " + .."video path has any time at all."))) + if AJSON then + local f = io.open(AJSON, "w") + f:write(string.format('{"armed":%d,"fetched":%d,"bytes":%d,' + ..'"starve":%d,"seam":%d,"serv":%d,"late":%d,"acc":%d,' + ..'"csr":%d,"cer":%d,"held":%s,"fps":%d,"shown":%d}\n', + armed, fet, byt, dry, seam, srv, late, acc, + SP:read_u32(PG_ACSR), SP:read_u32(PG_ACER), + HELD and "true" or "false", FPS, shown)) + f:close() + P("audio counters -> "..AJSON) + end + end local open = 0 for _,v in ipairs(r20seen) do open = open + v end P(string.format("WRITE WINDOW OPEN on %d of %d host frames (%.1f%%) -- " diff --git a/tools/bench/packed_run.sh b/tools/bench/packed_run.sh index c47a0aa..977bb15 100755 --- a/tools/bench/packed_run.sh +++ b/tools/bench/packed_run.sh @@ -70,7 +70,7 @@ run() { ( cd tmp && env SDL_VIDEODRIVER=dummy "$@" stdbuf -oL timeout -k 5 900 \ mame x68000 -bios ipl10 -exp1 cz6bs1 \ -rompath "$HOME/mame/roms;./p4roms" -hard dlxpdisk.chd \ - -ramsize 2M -video soft -window -sound none -nothrottle -plugins \ + -ramsize 2M -video soft -window $SOUNDARGS -nothrottle -plugins \ -autoboot_script ../tools/bench/packed.lua \ -snapshot_directory "./snap_packed_$tag" -snapview native \ -seconds_to_run "$SECS" > "packed_$tag.log" 2>&1 ) @@ -80,6 +80,11 @@ run() { grep -a "^\[PK\]" "tmp/packed_$tag.log" | sed 's/^\[PK\] / /' } fail() { echo "FAIL: $1"; exit 1; } +# -sound none for every run that is not about sound, which is all of them until +# run 5. 15,625 is not a preference there: it is the chip's own stream rate +# (8 MHz / 512), and equal rates are what keep MAME's resampler from filtering +# the thing being measured (FINDINGS 66, adpcm_run.sh). +SOUNDARGS="-sound none" echo "--- 1. THE GATE: $NFR records, paced at $GATE_FPS fps, channel stealing ---" SECS=$(( NFR / GATE_FPS + 25 )) @@ -120,6 +125,39 @@ grep -aq "0 frames NOT SAMPLED" tmp/packed_gate.log || \ python3 tools/bench/verify_packed.py "$DLXP" --snap tmp/snap_packed_gate \ --map tmp/packed_snaps_gate.csv --min-frames "$NFR" +# --------------------------------------------------------------------------- +# 5. THE AUDIO. ROADMAP P6c: the container's own bytes, out of channel 3, +# beside the video channel. Paced at the CONTAINER's rate rather than the +# gate's half rate, because the audio was cut at 12 fps and a 6 fps run would +# starve the chip for half of every group -- the picture can be slowed down and +# a crystal cannot. Nothing is snapshotted; the instrument is the WAV, and +# tools/bench/verify_packed_audio.py accounts for every byte of the stream in +# it. The tag is `aud` and not `audio` because tmp/packed_audio.log is +# tools/analysis/34_packed_audio.py's, in check.sh. +# +# CYCLE STEALING, and that is a result rather than a setting -- run 6 below is +# the same run with the bus held and it is a CONTRAST, not a gate. +AUDIO_ON=$(sed -n 's/.*has_audio = \([01]\),.*/\1/p' tmp/packed_meta.lua) +CFPS=$(sed -n 's/^ fps = \([0-9]*\),.*/\1/p' tmp/packed_meta.lua) +if [ "$AUDIO_ON" = "1" ]; then + echo + echo "--- 5. THE AUDIO: the container's own lumps, out of channel 3, while" + echo " the video channel is on the same bus (ROADMAP P6c) ---" + SECS=$(( NFR / 8 + 30 )) + SOUNDARGS="-samplerate 15625 -wavwrite packed_aud.wav" + run aud DLX_PK_HELD=0 DLX_PK_PACE=1 DLX_PK_FPS="$CFPS" DLX_PK_NFR=$NFR \ + DLX_PK_AUD=1 DLX_PK_AJSON=packed_aud.json + SOUNDARGS="-sound none" + grep -aq "^\[PK\] FLAG=\$FF" tmp/packed_aud.log || \ + fail "the audio run did not reach the end of the scene." + python3 tools/bench/verify_packed_audio.py "$DLXP" tmp/packed_aud.wav \ + tmp/packed_aud.json || \ + fail "the chip did not play the container. The counters above can all be + right while this fails -- nothing parses a packed container, so a lump + fetched into a buffer that is still being read is not an error, it is a + sound (FINDINGS 67.4, and it is the bug session 36 shipped and caught)." +fi + if [ "${DLX_PK_GATE_ONLY:-0}" = "1" ]; then exit 0; fi echo @@ -150,6 +188,41 @@ LOST=$(sed -n 's/.*-> \([0-9]*\) edges LOST.*/\1/p' tmp/packed_held_paced.log | rasters -- and the comparison with the stealing run below is then a comparison of two configurations that do the same thing." LOSTS=$(sed -n 's/.*-> \([0-9]*\) edges LOST.*/\1/p' tmp/packed_gate.log | head -1) + +# --------------------------------------------------------------------------- +# 6. THE SAME AUDIO WITH THE BUS HELD, and this is the interaction ROADMAP P6c +# said neither half had met. A burst channel HALTS the 68000 for the whole +# 88 ms record, so the audio service cannot run during it -- src/player/dma.i's +# DM_HOOK is never reached and the chip is looked at twice a frame instead of +# two thousand times. The bytes are the same bytes either way; what changes is +# WHEN the next lump is armed, and the chip has no starvation state: what it +# does in between is replay the byte pair it is holding. +# +# NOT A GATE. Both configurations play the container byte for byte and the +# verifier passes on both; the difference is entirely in the seams, and a seam +# is a design cost rather than a correctness one. +if [ "$AUDIO_ON" = "1" ]; then + echo + echo "--- 6. THE AUDIO AGAIN, WITH THE BUS HELD (the contrast, not a gate) ---" + SECS=$(( NFR / 6 + 30 )) + SOUNDARGS="-samplerate 15625 -wavwrite packed_aud_held.wav" + run aud_held DLX_PK_HELD=1 DLX_PK_PACE=1 DLX_PK_FPS="$CFPS" DLX_PK_NFR=$NFR \ + DLX_PK_AUD=1 DLX_PK_AJSON=packed_aud_held.json + SOUNDARGS="-sound none" + python3 tools/bench/verify_packed_audio.py "$DLXP" tmp/packed_aud_held.wav \ + tmp/packed_aud_held.json || fail "the held run did not play the + container. The bytes are not what holding the bus was expected to cost." + echo + echo " THE SEAM, STEALING AGAINST HELD -- audio does not merely cost clocks:" + for t in aud aud_held; do + printf ' %-9s ' "$t" + python3 tools/bench/verify_packed_audio.py "$DLXP" "tmp/packed_$t.wav" \ + | sed -n 's/^ worst \(.*\)$/\1/p' | head -1 + done + echo " Stealing, the 68000 sees the channel from inside dma.i's transfer" + echo " wait. Held, it is halted for the whole record and cannot look at all." +fi + echo echo " V-DISP edges lost: $LOST held at 12 fps, $LOSTS stealing at $GATE_FPS fps." echo " A player keeps a clock, reads a stick and feeds ADPCM. Which of the two" diff --git a/tools/bench/prep_packed.py b/tools/bench/prep_packed.py index 64f54f6..48bcea3 100644 --- a/tools/bench/prep_packed.py +++ b/tools/bench/prep_packed.py @@ -49,6 +49,18 @@ with open(out, "w") as fh: ("has_audio", int(d.has_audio)), ("aud_bytes", d.aud_bytes), ("aud_hz", d.aud_hz), ("n_lumps", d.n_lumps), + # ROADMAP P6c. lba_aud is off_aud/512 and is a HEADER field + # rather than the constant 1 it happens to equal: a shipping + # volume has a filesystem in front of the stream and the + # player adds PG_LBA0's base to neither of them by accident. + ("lba_aud", d.off_aud // SECTOR if d.has_audio else 0), + # the four axes, so the rig can print what the container says + # it was encoded for and the verifier can decode with it + ("aud_variant", '"%s"' % d.decoder()["variant"] + if d.has_audio else '""'), + ("aud_order", '"%s"' % d.decoder()["order"] + if d.has_audio else '""'), + ("aud_bits", d.aud_bits), ("aud_init", d.aud_init), ("entries", entries)]: fh.write(f" {k} = {v},\n") fh.write("}\n") diff --git a/tools/bench/verify_packed_audio.py b/tools/bench/verify_packed_audio.py new file mode 100644 index 0000000..2478ba1 --- /dev/null +++ b/tools/bench/verify_packed_audio.py @@ -0,0 +1,334 @@ +#!/usr/bin/env python3 +"""The container's own audio, READ BACK OFF THE SPEAKER. ROADMAP P6c. + + python3 tools/bench/verify_packed_audio.py [counters.json] + +WHY THIS READS THE CAPTURE AND NOT THE PLAYER'S COUNTERS. src/player/packed.s +reports how many lumps it armed and how many payload bytes it handed the chip, +and every one of those numbers can be right while the sound is wrong. Nothing +in this format parses anything (FINDINGS 67.4): a lump fetched one sector out is +not an error, it is 7,168 B of noise played at the right length; a payload one +byte long is not an error either, it is a rate. The only instrument that can +tell those apart from a correct run is the stream the chip actually produced. + +WHAT IS CHECKED, and it is the whole scene rather than a sample of it: + + 1. every lump's payload, decoded with the FOUR AXES OUT OF THE CONTAINER'S OWN + HEADER (FINDINGS 66/67.3), appears in the capture SAMPLE-EXACT and in + order. Not "close": the recursion is exact arithmetic and MAME's okim6258 + puts `signal << 4` into a stream the machine routes to the speaker at gain + 0.50, so a chip sample is `signal * 8` and recovering it is a division and + not a rounding. The residual is asserted. + + 2. the PAYLOAD lengths are the accumulator's and not the lump's. A player + that fed the chip the whole A*512 B lump runs 0.09% fast -- 1.25 s of + lip-sync over the game (67.2) -- and the difference between the two is 6.54 + B a group, which is 13 samples. So this is checked by LENGTH: lump k's run + of matched samples must be exactly 2*lump_bytes(k), and that alternates + 14,322 / 14,324 rather than being 14,336 every time. + + 3. THE SEAMS, measured rather than assumed. Channel 3 counts out at the end + of a lump and the chip has no FIFO and no starvation state -- it goes on + decoding whatever byte its data register still holds, alternating that + byte's low and high nibbles, until the CPU arms the next lump. Those + samples are NOT silence, they are the recursion running on a repeated byte, + and the state they leave behind is what lump k+1 decodes from. So the + search below carries the state across the seam and reports its LENGTH, + which is the audible cost of every design decision on the video path. + +THE NEGATIVE CONTROL IS BUILT IN. A seam is found by searching for the repeat +count that makes the next lump match; if the player had fed the wrong bytes, no +repeat count would make it match and the run fails rather than sliding. +""" +import json, os, struct, sys, wave + +sys.path.insert(0, os.path.join(os.path.dirname(os.path.abspath(__file__)), + "..", "encoder")) +import adpcm +from dlxp import DLXP, lump_bytes as _lump_bytes + + +def dlxp_lump_bytes(d, k): + """What the CADENCE gives lump k, before the stream's end + truncates it -- so a lump that is short because the scene ran + out can be told from one that is short because the remainder + arithmetic said so.""" + return _lump_bytes(k, d.cad_f, d.fps, d.aud_hz) + +SCALE = 8 # okim6258's `signal << 4` into a 32768 stream, times the + # machine's 0.50 speaker route. verify_adpcm_chip.py's. +RESID = 2 # counts of slack on that recovery, as 66 measured it +LOOK = 32 # samples of continuation a candidate run has to survive +MAXRUN = 60000 # nibbles one delivered byte may be stretched over: 3.8 s + # at 15,625 Hz, far past any seam a working player makes. + # A bound is what makes a failure say "not found" rather + # than run until the host is bored. +LEAD = 400000 # samples of silence before PLAY + + +def stepper(dec): + """One sample of the recursion, as a closure over the container's own four + axes. adpcm.decode_state is the same arithmetic and is what the whole-lump + paths use; this exists because the walk below needs it ONE NIBBLE AT A TIME + and a function call per sample over 78,125 bytes is the difference between + a gate that runs in seconds and one that does not.""" + lo, hi = adpcm.clamp_bounds(dec["bits"]) + variant, step, adj = dec["variant"], adpcm.STEP, adpcm.INDEX_ADJUST + + def one(st, n): + sig, idx = st + sig += adpcm.delta(n, step[idx], variant) + sig = lo if sig < lo else (hi if sig > hi else sig) + idx += adj[n & 7] + idx = 0 if idx < 0 else (48 if idx > 48 else idx) + return sig, (sig, idx) + return one + + +def walk(rec, pos, st, data, dec): + """Account for EVERY byte the player handed the chip, in order. + + WHY A WALK AND NOT A COMPARISON. A whole-stream `decode(data) == capture` + is the check this wanted to be and it does not survive contact with the + machine. The MSM6258 has no FIFO and no handshake at all: the DMA channel + writes a byte into the data register whenever #DRQ3 asks, and the chip + decodes a nibble out of whatever is in there on every sample tick. Those + are two clocks -- 7,812.5 B/s and 15,625 Hz -- and MAME's okim6258 data_w + RESETS the nibble select on every write. So a byte is normally played as + two nibbles, and near a boundary it can be played as one (the high nibble + dropped) or as three or more (the low/high pair repeated) purely from where + the write lands inside a sound-stream slice. + + THE MODEL IS THEREFORE ONE LINE: byte b was played as `c` nibbles taken from + the cycle (b&15, b>>4), c >= 1. This walk finds `c` for every byte, and the + HISTOGRAM of c is the result -- c=2 everywhere is a chip being fed exactly + at its own rate, and every c>2 is the chip replaying a byte while the 68000 + was somewhere else, which is what a SEAM is. + + It is not a loose check. Every one of the `c` samples has to be exactly + right, the run has to be followed by LOOK samples that are exactly right, + and a byte the player never sent leaves no c at all. Returns + (fail_index, pos, runs); fail_index is None on success. + """ + one = stepper(dec) + runs = [] + n = len(data) + pos_ = pos + for i in range(n): + b = data[i] + pair = (b & 15, b >> 4) + nxt = data[i + 1] & 15 if i + 1 < n else None + # DEEP FIRST, THEN SHALLOW. LOOK samples of continuation is what tells + # a real seam from a coincidence -- a repeated pair can agree with the + # next lump's first nibble for one sample and does not for thirty-two. + # But the window is 16 bytes wide and a SECOND stretched byte inside it + # rejects the true answer as well as the false ones, so a byte that no + # candidate survives is retried with a shallow window rather than + # failing -- and the retry window is TWO samples, the next byte's own + # pair, because a four-sample one reaches into the stretched byte + # itself. The cost of resolving it wrong is a failed walk, not a pass. + got = _pick(rec, pos_, st, pair, nxt, data, i, one, LOOK) + if got is None: + got = _pick(rec, pos_, st, pair, nxt, data, i, one, 2) + if got is None: + return i, pos_, runs + c, st = got + pos_ += c + runs.append(c) + return None, pos_, runs + + +def _pick(rec, pos, st, pair, nxt, data, i, one, look): + """The run length for one byte: every candidate `c` whose samples are exact + and whose continuation survives `look`, resolved to c=2 where c=2 is one of + them. Two nibbles a byte is what the two clocks agree on; anything else is + an event and an event needs the evidence, which is what `look` is.""" + cands = [] + s2, c = st, 0 + while c < MAXRUN and pos + c < len(rec): + smp, s3 = one(s2, pair[c & 1]) + if rec[pos + c] != smp: + break + c += 1 + s2 = s3 + if nxt is None: + cands.append((c, s2)) + break + smp2, _ = one(s2, nxt) + if pos + c < len(rec) and rec[pos + c] == smp2 \ + and _look(rec, pos + c, s2, data, i + 1, one, look): + cands.append((c, s2)) + if c > 2 and len(cands) >= 2: + break + if not cands: + return None + for cc in cands: + if cc[0] == 2: + return cc + return cands[0] + + +def _look(rec, pos, st, data, i, one, look=LOOK): + """`look` samples of continuation, assuming two nibbles a byte from here. + + This is what tells a real seam from a coincidence. At a lump boundary the + repeated pair can happen to agree with the next lump's first nibble for one + sample; it does not go on agreeing for thirty-two. + """ + n, k = len(data), 0 + while k < look and i < n: + b = data[i] + for nib in (b & 15, b >> 4): + if pos >= len(rec): + return True + smp, st = one(st, nib) + if rec[pos] != smp: + return False + pos += 1 + k += 1 + i += 1 + return True + + +def main(): + if len(sys.argv) < 3: + sys.exit(__doc__) + d = DLXP(sys.argv[1]) + counters = json.load(open(sys.argv[3])) if len(sys.argv) > 3 else None + if not d.has_audio: + sys.exit("this container is silent -- there is nothing to have heard") + + dec = d.decoder() + fails = [] + def ck(ok, msg): + print(("OK " if ok else "FAIL ") + msg) + if not ok: + fails.append(msg) + + w = wave.open(sys.argv[2]) + n, ch, rate = w.getnframes(), w.getnchannels(), w.getframerate() + s = struct.unpack("<%dh" % (n * ch), w.readframes(n)) + left, right = s[0::ch], s[1::ch] + ck(rate == d.aud_hz, + f"the capture is at {rate:,} Hz and the chip's stream is {d.aud_hz:,} -- " + f"equal rates are what keep MAME's resampler out of the measurement") + ck(list(left) == list(right), "both speakers carry the same samples (pan 00)") + ck(any(left), "the capture contains a signal at all") + if not any(left) or rate != d.aud_hz: + return 1 + rec = [round(v / SCALE) for v in left] + start = next(i for i, v in enumerate(rec) if v) + ck(start < LEAD, f"the chip starts playing {start/rate:.2f} s in") + + print(f"--- {sys.argv[1]}: {d.n_lumps} lumps, decoder {dec['variant']}/" + f"{dec['order']}, {dec['bits']}-bit clamp, accumulator {dec['init']} " + f"at PLAY -- ALL FOUR out of the header (67.3)") + + # ---- THE STREAM, ACCOUNTED FOR BYTE BY BYTE. d.audio() is the container's + # own lumps reassembled BY PAYLOAD -- 67.2's accumulator, not the padded + # sector runs -- so a player that fed the chip whole lumps does not merely + # score worse here, it fails to walk: the 6.54 B of zero at the end of a + # lump are nibbles that are not in this stream. + data = d.audio() + bad, pos, runs = walk(rec, start, (dec["init"], 0), data, dec) + ck(bad is None, + f"all {len(data):,} bytes of the container's audio reached the chip, in " + f"order, and every sample the chip produced from them is exact" + + ("" if bad is None else f" -- byte {bad:,} of {len(data):,} does not " + f"account for capture sample {pos:,}")) + if bad is not None: + k = bad * 2 * d.fps // (d.cad_f * d.aud_hz) + print(f" that is inside lump {k}, {bad - sum(len(d.lump(j)) for j in range(k)):,} B in") + return 1 + + matched = pos - start + worst = max(abs(v - SCALE * round(v / SCALE)) + for v in left[start:start + matched]) + ck(worst <= RESID, + f"every one of {matched:,} matched samples is within {worst} of a " + f"multiple of {SCALE} -- so `signal = sample/{SCALE}` recovers the chip's " + f"own stream rather than rounding to the nearest story") + + # ---- 1. THE FEED. c=2 is a chip being fed at exactly its own rate. + hist = {} + for c in runs: + hist[c] = hist.get(c, 0) + 1 + two = hist.get(2, 0) + ck(two * 1000 >= len(runs) * 999, + f"{two:,} of {len(runs):,} bytes ({two*100/len(runs):.3f}%) were played " + f"as exactly two nibbles -- the chip was paced by its own #DRQ3 and not " + f"by the CPU") + print(f" nibbles per delivered byte: " + + ", ".join(f"{c}x{v:,}" for c, v in sorted(hist.items()))) + + # ---- 2. THE SEAMS, which is what every c > 2 is. A lump's channel counts + # out and the chip goes on replaying the last byte until the CPU arms the + # next one; the excess nibbles ARE that interval, measured in the only place + # it exists, which is the sound. + seams = [(i, c - 2) for i, c in enumerate(runs) if c > 2] + print(f"--- THE SEAMS: {len(seams)} byte(s) were stretched, out of " + f"{d.n_lumps - 1} lump boundaries") + if seams: + ex = sum(c for _, c in seams) + print(f" worst {max(c for _, c in seams)} samples = " + f"{max(c for _, c in seams)/rate*1000:.2f} ms; total {ex} samples " + f"= {ex/rate*1000:.2f} ms of replayed byte over " + f"{matched/rate:.2f} s of audio ({ex*100/matched:.4f}%)") + # and every stretched byte must BE a lump boundary -- a stretch anywhere + # else is the CPU losing the chip in the middle of a buffer. + ends = set() + off = 0 + for k in range(d.n_lumps): + off += len(d.lump(k)) + ends.add(off - 1) + stray = [i for i, _ in seams if i not in ends] + ck(not stray, + f"every stretched byte is the LAST byte of a lump ({len(stray)} were not)" + + ("" if not stray else f" -- first at byte {stray[0]:,}, which is the " + f"chip running dry in the middle of a buffer")) + + # ---- 3. THE PAYLOAD IS THE ACCUMULATOR'S (FINDINGS 67.2). The walk + # already proves it -- a whole-lump player's stream contains the padding and + # would not walk -- so what is left is to price what was avoided. The LAST + # lump is left out: 120 frames is not a multiple of F=11, so it carries ten + # frames of audio and is short for an arithmetic reason and not a rate one. + full = [k for k in range(d.n_lumps) + if len(d.lump(k)) == dlxp_lump_bytes(d, k)] + ck(len(full) >= d.n_lumps - 1, + f"{len(full)} of {d.n_lumps} lumps carry a whole group of audio") + if len(full) > 1: + pad = d.cad_a * 512 + got = sum(len(d.lump(k)) for k in full) + over = pad * len(full) - got + secs = got * 2 / rate + print(f"--- THE PADDING IS DRIFT (FINDINGS 67.2), over the {len(full)} " + f"lumps that carry a whole group") + print(f" payload {got:,} B against {pad*len(full):,} B of lump " + f"space: {over:,} B more, {over*100/got:.3f}%, " + f"{over*2/rate*1000:.2f} ms over {secs:.2f} s of audio") + print(f" -> {over*2/rate/secs*22.8*60:.2f} s of lip-sync over the " + f"game's 22.8 min, and the accumulator in pg_apay is the three " + f"lines that do not spend it") + + if counters: + ck(counters["late"] == 0, + f"the player never re-armed a channel that still had bytes to send " + f"({counters['late']} did)") + ck(counters["starve"] == 0, + f"the player never found the channel counted out with no lump ready " + f"({counters['starve']} times it did)") + ck(counters["bytes"] == len(data), + f"the player's own byte count ({counters['bytes']:,}) is the " + f"container's whole payload ({len(data):,})") + print(f" the player: {counters['armed']} lumps armed, " + f"{counters['fetched']} fetched, {counters['serv']:,} service " + f"calls over {counters['shown']} frames " + f"({'BUS HELD' if counters['held'] else 'CYCLE STEALING'})") + + print("PACKED AUDIO GATE " + ("GREEN" if not fails + else f"RED: {len(fails)} failed")) + return 1 if fails else 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/tools/encoder/adpcm.py b/tools/encoder/adpcm.py index 7fe4f45..174a6f0 100644 --- a/tools/encoder/adpcm.py +++ b/tools/encoder/adpcm.py @@ -100,18 +100,35 @@ def delta(nibble, step, variant): return -d if nibble & 8 else d -def decode(nibbles, variant="shift", init=0, bits=12): - """Nibbles -> signed samples. State is (signal, step index); the step index - is 0 at the start of a stream and `init` is where the accumulator starts.""" +def decode_state(nibbles, variant="shift", state=None, init=0, bits=12): + """The recursion, with its STATE in and out. (samples, (signal, idx)). + + `decode` is this with the state thrown away, and it is written this way + round rather than duplicated because a stream that STOPS and RESUMES is not + a hypothetical here: an MSM6258 fed by a DMA channel goes on decoding + whatever byte its data register still holds when the channel counts out + (MAME okim6258 sound_stream_update reads m_data_in unconditionally while + PLAYING), so the samples between one lump and the next are the recursion + continuing over a repeated byte. tools/bench/verify_packed_audio.py has to + carry that state across the seam to check the lump on the far side of it, + and a second copy of the loop is a second place the clamp can drift. + """ lo, hi = clamp_bounds(bits) - signal, idx, out = init, 0, [] + signal, idx = state if state is not None else (init, 0) + out = [] for n in nibbles: signal += delta(n, STEP[idx], variant) signal = lo if signal < lo else (hi if signal > hi else signal) idx += INDEX_ADJUST[n & 7] idx = 0 if idx < 0 else (48 if idx > 48 else idx) out.append(signal) - return out + return out, (signal, idx) + + +def decode(nibbles, variant="shift", init=0, bits=12): + """Nibbles -> signed samples. State is (signal, step index); the step index + is 0 at the start of a stream and `init` is where the accumulator starts.""" + return decode_state(nibbles, variant, None, init, bits)[0] def encode(samples, variant="shift", init=0, bits=12):