diff --git a/README.md b/README.md index 4b79875..de8abcc 100644 --- a/README.md +++ b/README.md @@ -357,6 +357,34 @@ ADPCM is recursive and a rounding difference does not stay where it happens. So which one the chip runs is not a footnote; it is a precondition on shipping any audio at all, and MAME's x68000 has the chip to ask (FINDINGS 65). +**So the chip was asked, and the encoder was wrong on four things rather than +one.** 68000 code programs the machine's own ADPCM DMA channel exactly as the +IPL ROM programs it — the register bytes are decoded out of the ROM image, not +recalled — and feeds the chip a designed 1,678-nibble stream at the chip's own +pace, 7,811.4 bytes a second against the format's 7,812.5. Sixteen candidate +decoder models are then fitted to what MAME captured, and **exactly one +reproduces it sample-exact over all 1,678 samples**, with a negative control on +every axis: flip one and the match dies. The chip runs the datasheet's +truncation, takes the **low** nibble of a byte first, clamps its accumulator at +**10 bits**, and starts it at **−2**. `adpcm.py` defaulted to the opposite of +all four. + +And the expensive one is not the one the paragraph above worried about. Getting +the delta formula wrong costs −2.88 dB; getting the **nibble order** wrong costs +**−25.74 dB**. The earlier "high nibble first, measured" was a real measurement +of *ffmpeg*, i.e. of the Dialogic VOX **file** convention — a different question +from what a chip does with a byte written to its data register, with a different +answer. The 10-bit clamp costs nothing on this window and only because the +window peaks at **435 of 511**: it is a −13.4 dBFS passage with 1.4 dB of +headroom, where the encoder had been clamping 12.1 dB higher. So the audio +**level** is an open choice again, downward, and the loudest passage on the disc +is unmeasured (FINDINGS 66). + +**Name the layer**: that is MAME's device model, measured end to end through the +machine's real transport. It settles the rig — an emulated audio test encoded +against the wrong model is 25 dB of nothing — and it does not settle the +silicon. + **And audio is what the packed container's best property finally costs something for.** A packed record is 97 sectors and its address is arithmetic — no index, and none can be needed. Audio is 651.0417 bytes a frame slot, a rate diff --git a/docs/FINDINGS.md b/docs/FINDINGS.md index b27bbb0..c9c0d70 100644 --- a/docs/FINDINGS.md +++ b/docs/FINDINGS.md @@ -6949,3 +6949,129 @@ correct because Sharp wrote it. DLXP2 would be built from, and no byte of one has been written. - **What audio does to a scene change.** The slack table is here, but 51.3's refill climb with a second consumer through a real branch point is not. + +--- + +## 66. The chip is asked, and it disagrees with the encoder on all four axes — the largest of them is not the one we were worried about (session 34) + +**ROADMAP P6a.** `src/player/adpcm.i`, `src/player/adpcmgate.s`, +`tools/bench/prep_adpcm.py`, `tools/bench/adpcm.lua`, +`tools/bench/adpcm_run.sh`, `tools/bench/verify_adpcm_chip.py`, +`tools/analysis/33_adpcm_model.py`. + +**Name the layer.** 68000 code drives the transport; the thing measured is +**MAME 0.277's `okim6258` device model**, end to end through the machine's real +DMA path. It settles the **rig** — an emulated audio test encoded against the +wrong model is 25 dB of nothing — and it does **not** settle the silicon. What a +real MSM6258V does is still a hardware/datasheet item. + +### 66.0 A MAME source tree IS reachable from this machine, and 64.4 is struck + +FINDINGS 64.4 and `tools/encoder/adpcm.py`'s header both record "no MAME source +tree is on this machine". There is no tree on disk, but the machine has network +and `raw.githubusercontent.com/mamedev/mame/mame0277/...` fetches. That is how +this session designed its experiment rather than swept blindly, and it is worth +recording because two sessions reasoned about MAME's device model as an +unopenable box when it was one `curl` away. + +**It does not replace the measurement and did not become one.** The installed +binary is Ubuntu's 0.277 and the tag is upstream's; whether the two are the same +bytes is not something a fetch can say. Everything below is read out of a +capture from the binary that is actually here. + +### 66.1 The transport is the IPL ROM's own, and it works first time + +`src/player/adpcm.i` programs HD63450 **channel 3** with the bytes +`tools/analysis/21_iplrom_dmac.py` decodes out of the IPL ROM at `$FF0C2E` and +`$FF9A82`: `DCR = $80` (dual address, 8-bit port, cycle steal without hold), +`SCR = $04`, `MFC = DFC = $05`, `CPR = $01`, `DAR = $E92003`, `OCR = $32` +(memory→device, byte, **external request**), `CCR = $80`, then command `$02` to +`$E92001`. 839 bytes moved in **0.1074 s = 7,811.4 B/s** against the format's own +7,812.5, `CSR = $E0`, `CER = $00`, `MTC = 0`. **This is P6b's transport, not +scaffolding.** + +**Two things session 33 could not have guessed and did not have to.** 65.5 fed +the chip from Lua, got silence, swept control 0..3 × port C 0..15 and stopped. +Both reasons are ordinary: + +* **The PPI's port C is an INPUT until it is told otherwise.** ADPCM pan and the + sample-rate divider are port C bits; an i8255 out of reset has every port an + input, so writes to `$E9A005` move a latch nothing is reading. Control word + `$92` first, and only then does `$08` mean *pan both, ÷512*. +* **`$01` is COMMAND_STOP.** `$02` is PLAY. The sweep that "covered" 0..3 wrote + `$01` in the probe that swept port C, so the chip was never playing in it. + +### 66.2 THE HEADLINE: four axes were wrong, and the expensive one is the NIBBLE ORDER + +The probe is 1,678 nibbles — 16 zero nibbles of prologue, a trigger, an encoded +sine, then loud bursts — and `verify_adpcm_chip.py` searches **sixteen candidate +decoder models** (nibble feed × delta formula × clamp × initial accumulator) +crossed with three capture decimations and a prologue length. **Exactly one +reproduces the capture, over 1,678 consecutive samples, sample-exact**, and each +axis carries a negative control: flip it alone and the closest surviving +alternative disagrees on 826, 1,504, 156 and 1,522 samples respectively. + +| axis | `adpcm.py` default | **the chip** | cost of getting it wrong ALONE | +|---|---|---|---:| +| nibble order | high first | **LOW first** | **−25.74 dB** | +| delta formula | `shift` | **`terms`** | −2.88 dB | +| clamp | 12-bit | **10-bit** | 0.00 dB *(on this window)* | +| accumulator at PLAY | 0 | **−2** | −0.45 dB | +| **all four at once** | | | **−10.38 dB** against 21.97 | + +**65.2 named the wrong axis as the risk.** It priced the delta formula at 25 dB +and left nibble order recorded as "HIGH FIRST, **measured**". That measurement +was real and it was **against ffmpeg**, i.e. about the Dialogic VOX *file* +convention — not about what a chip does with a byte written to its data +register. The two are different questions with different answers, and the one +this port needs is the second. `verify_adpcm.py` keeps ffmpeg's parameters +deliberately: a reference check whose reference has been adjusted to agree is +not a check. `adpcm.CHIP` carries the measured set, and anything that encodes +**for the machine** passes it explicitly. + +### 66.3 The clamp is 10-bit, it costs nothing here, and that is the finding that will bite + +The MSM6258's D/A is 10-bit and the model clamps the **accumulator** there, so it +is inside the recursion rather than an output scaling. On the Singe window it is +free — encode for 12 bits or for 10 and the answer is **21.99 dB either way, +with zero samples on the clamp** — for one reason only: that window peaks at +**435 of 511**, i.e. **1.4 dB of headroom**, and it is a −13.4 dBFS passage. + +A 10-bit accumulator is **12.1 dB smaller** than the 12-bit word the encoder was +clamping to. **65.1's "the level is not a lever" survives downward and is now +wrong upward**: normalising still buys nothing, and a passage a few dB louder +than this one does not fit. **Nothing in this project has measured the loudest +passage on the disc**, so the audio level is an open choice, not a settled one. + +### 66.4 A rig fact that cost this session most of its time, and is worth the space + +The 8 MHz ADPCM master clock is **CT1 in the YM2151's port register `$1B`**, in a +different device from the divider. MAME delivers that write to the ADPCM chip on +the **sound system's own schedule**, not at the instant of the store — so a +transfer started in the same breath as the setup plays its first ~17 ms at the +previous clock. The symptom is specific and misleading: the capture's first +~130 samples arrive in **exact identical pairs**, the rest do not, and **no model +fits a stream that changed rate part way through** — which reads exactly like a +broken probe. `adpcmgate.s` spins ~100 ms after `ad_setup` and says why. A +player sets its clock once at boot and never meets this. + +**The general lesson is the one this tree keeps relearning**: a run that fails to +match is not evidence about the thing being measured until the apparatus has +been shown to be steady. Three quarters of the diagnosis here was spent +disproving hypotheses about the *chip* for a symptom that was about the *clock +write*. + +### 66.5 What is still open, stated so it is not read as closed + +* **The silicon.** Every value in 66.2 is MAME's. A real MSM6258V may differ on + any of the four, and the two published references already disagree on one. + This is a datasheet or a board, and it is cheap on a board: play a known + nibble stream and record the line out. +* **Nothing has played as audio.** The capture is a measurement instrument, not + a listening test, and no ADPCM has reached a speaker on real hardware. +* **The encoder is still greedy** (65 risk list, unchanged): exhaustive + per-sample search, no lookahead. 21.99 dB is this format's floor here. +* **P6b is now unblocked and its bytes are decided**: DLXP2 must be encoded with + `adpcm.CHIP`, and 65.3's cadence arithmetic (F=11, A=14, wire 582.0 → 589.6 + KB/s) is untouched by any of this — it is a byte count, and none of the four + axes changes how many bytes a second the format needs. diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index 9d04010..d77c61c 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -96,6 +96,25 @@ The codec container, which already has an index and variable records, pays branch's simplification, and 64's risk list predicted there would be one without knowing what. +Amended end of session 34: **P6a IS DONE, AND THE ENCODER WAS WRONG ON FOUR AXES +RATHER THAN ONE (FINDINGS 66).** 68000 code programs HD63450 channel 3 with the +IPL ROM's own ADPCM bytes — dual address, 8-bit port, cycle steal, EXTERNAL +request — feeds the MSM6258 at the chip's own pace (**7,811.4 B/s against +7,812.5**), and one of **sixteen** candidate decoder models reproduces MAME's +capture **sample-exact over 1,678 consecutive samples**, with a negative control +on every axis. The chip runs **`terms`, LOW nibble first, a 10-bit accumulator, +starting at −2**, and `tools/encoder/adpcm.py` defaulted to the opposite of all +four. **65.2 named the wrong axis as the risk**: the delta formula is worth +−2.88 dB and the NIBBLE ORDER is worth **−25.74 dB**, and 65.1's "high first, +measured" was a measurement of ffmpeg's VOX file convention rather than of a +chip's data register. Two things came with it. **(1) The 10-bit clamp is free on +the Singe window and only because that window peaks at 435 of 511** — 1.4 dB of +headroom on a −13.4 dBFS passage, where the encoder had been clamping 12.1 dB +higher — so **the audio LEVEL is an open choice again**, downward, and the +loudest passage on the disc is unmeasured. **(2) The transport is P6b's, not +scaffolding**, and it worked first time. **P6b is next and its bytes are +decided: DLXP2 encodes with `adpcm.CHIP`.** + **THE COMPLETION TARGET IS M3, THE VERTICAL SLICE** (USER DECISION): one scene tree — a decision point, two outcomes, a death clip — with audio, streaming from a real SCSI volume on a stock 2 MB machine, playable. That is the point at which @@ -657,7 +676,13 @@ Buildable, and empty until P4: there is nothing to boot from yet. **Exit criterion: one decision point, two outcomes, a death clip, with audio, playing from disc on stock hardware.** -**P6. Audio — and it is the largest unpriced risk left in the project.** +**P6. Audio. P6a DONE, session 34 — FINDINGS 66.** ~~and it is the largest +unpriced risk left in the project.~~ Three quarters of P6 closed in session 33 +and the fourth precondition — which decoder the chip runs — closed in 34, on the +machine, through the real DMA channel. **What is left of P6 is P6b (DLXP2, a +container with sound in it) and the refill climb with a second consumer.** + +*(original framing, kept because every figure below is still the live one:)* MSM6258 ADPCM, 15.6 kHz mono, **7.8 KB/s**. That figure is in `ratectl.py`'s budget and nowhere else: not extracted, not encoded, not interleaved into the container, and **never priced on the bus**. Two reasons to treat it as a risk diff --git a/docs/STATUS.md b/docs/STATUS.md index b9eb735..ba2eef8 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -1,4 +1,122 @@ -# Status & next-session handoff — end of session 33 (2026-08-25) +# Status & next-session handoff — end of session 34 (2026-08-25) + +## Session 34: the chip is asked, and the encoder was wrong on four axes + +**Green light first and last: `./tools/bench/check.sh` was ALL GREEN before any +of this (`tmp/check_s34_start.log`) and ALL GREEN after +(`tmp/check_s34_end.log`)** — the same stages, plus one new one. + +**FINDINGS 66. ROADMAP P6a is DONE.** `src/player/adpcm.i`, +`src/player/adpcmgate.s`, `tools/bench/prep_adpcm.py`, `tools/bench/adpcm.lua`, +`tools/bench/adpcm_run.sh`, `tools/bench/verify_adpcm_chip.py`, +`tools/analysis/33_adpcm_model.py`. + +**NAME THE LAYER.** 68000 code drove the transport; the thing measured is +**MAME 0.277's `okim6258` device model**, end to end through the machine's own +DMA path. No board ran. It settles the RIG and not the silicon. + +**0. A MAME SOURCE TREE IS REACHABLE AND 64.4 IS STRUCK.** There is no tree on +disk, but this machine has network and the upstream tag fetches. Two sessions +reasoned about MAME's device model as an unopenable box when it was one `curl` +away. It designed the experiment; it did not become the result. + +**1. THE TRANSPORT IS THE IPL ROM'S OWN AND IT WORKED FIRST TIME.** HD63450 +channel 3, `DCR=$80 SCR=$04 MFC=DFC=$05 CPR=$01 DAR=$E92003 OCR=$32 CCR=$80`, +then `$02` to `$E92001` — every byte decoded out of the ROM by +`21_iplrom_dmac.py`. **839 B in 0.1074 s = 7,811.4 B/s** against the format's +7,812.5, `CSR=$E0 CER=$00 MTC=0`. **This is P6b's transport arriving early.** + +Session 33's silence (65.5) had two ordinary causes and neither was a register +semantic worth guessing at: **the PPI's port C is an INPUT until control word +`$92` says otherwise**, so the pan/divider writes went to a latch nothing read; +and **`$01` is COMMAND_STOP**, which is what the port-C sweep wrote every time. + +**2. THE HEADLINE, AND IT IS NOT THE AXIS 65 WAS WORRIED ABOUT.** Sixteen +candidate models, three capture decimations, a searched prologue: **exactly one +reproduces the capture over 1,678 consecutive samples, sample-exact**, and every +axis has a negative control. + +| axis | `adpcm.py` default | the chip | wrong ALONE | +|---|---|---|---:| +| nibble order | high first | **LOW first** | **−25.74 dB** | +| delta formula | `shift` | **`terms`** | −2.88 dB | +| clamp | 12-bit | **10-bit** | 0.00 dB (this window) | +| accumulator at PLAY | 0 | **−2** | −0.45 dB | +| all four | | | **−10.38 dB** against 21.97 | + +**65.1's "nibble order: HIGH FIRST, measured" was true and was about ffmpeg** — +the Dialogic VOX *file* convention — not about what a chip does with a byte in +its data register. `verify_adpcm.py` keeps ffmpeg's parameters on purpose; the +measured set is `adpcm.CHIP` and callers that encode FOR the machine pass it. + +**3. THE 10-BIT CLAMP IS FREE HERE AND ONLY BY LUCK.** Encode for 12 bits or for +10 and the Singe window is **21.99 dB either way, zero samples on the clamp** — +because it peaks at **435 of 511**, **1.4 dB of headroom**, on a −13.4 dBFS +passage. The clamp is **12.1 dB below** where the encoder was clamping and it is +INSIDE the recursion. **65.1's "the level is not a lever" survives downward and +is now wrong upward**, and **the loudest passage on the disc is unmeasured**. + +**4. THE RIG FACT THAT COST THIS SESSION MOST OF ITS TIME.** The 8 MHz ADPCM +clock is CT1 in the YM2151's `$1B`, in a different device from the divider, and +MAME delivers it on the sound system's schedule rather than at the store. A +transfer started in the same breath as the setup plays its first ~17 ms at the +old clock; the capture's first ~130 samples come out in **exact identical +pairs** and no model fits a stream that changed rate part way through — which +reads exactly like a broken probe. `adpcmgate.s` settles ~100 ms and says why. + +**RISKS IN THIS SESSION'S RESULT:** + +- **The silicon is untouched.** All four values are MAME's. A real MSM6258V may + differ on any of them, and the two published references already disagree on + one. Cheap on a board: play a known nibble stream, record the line out. +- **Nothing has played as audio.** The capture is an instrument, not a listen. +- **The encoder is still greedy** — no lookahead, unchanged from 65. +- **The 10-bit clamp is a ceiling nobody has measured the disc against.** + +## HANDOFF — start here + +**THE TREE IS ALL GREEN**, session 34's stage included. + +### The work, in the order it should be done + +**1. P6b — DLXP2, a container with sound in it.** Unblocked, and its bytes are +now decided: encode with **`adpcm.CHIP`** (`terms`, low nibble first, 10-bit, +init −2), not with the defaults. 65.3 is all the arithmetic and none of it +moved — cadence **F=11, A=14**, `LBA(i) = LBA0 + i*97 + floor(i/11)*14`, +14,336 B held, wire **582.0 → 589.6 KB/s** — because a byte count does not care +which decoder reads the bytes. + +**2. THE AUDIO LEVEL, which 66.3 reopened.** The encoder now has 12.1 dB less +room than it thought. Measure the loudest passage on the disc before choosing a +level; a scene that clips inside the recursion does not merely distort, it +drives the predictor. + +**3. WHAT IS LEFT OF P6 AFTER THAT** is the fourth quarter: 51.3's refill climb +with a second consumer through a real branch point. The slack table is in +`32_audio_wire.py`; nothing has been run. + +### What is still BLOCKED, so it is not picked up by mistake + +**K4 — the packed player that is on screen — is conditional on B2**, a board +question. **E7, E4 and C1** are parked (61.8), and **P4a's wiring** is parked +with the ring K3 deleted. + +**The hardware list is unchanged and is the user's**: B1 (sustained AND the +data-phase BURST rate, 64.2), B2 (blanking — the five-minute half), B3 +(`#EXREQ`), B4 (a byte write to a palette register). **Session 34 adds a fifth +that is cheaper than any of them**: play a known nibble stream on a real +MSM6258V and record the line out, which settles 66.2's four axes on silicon. + +### Reproducing this session + + ./tools/bench/check.sh # ALL GREEN + ./tools/bench/adpcm_run.sh # the chip gate on its own + python3 tools/analysis/33_adpcm_model.py tmp/au_singe.raw + +**WHAT IS NEXT.** P6b: a container with sound in it, encoded for the chip that +was just measured. + +--- ## Session 33: audio gets an encoder, and the packed container's best property gets a bill diff --git a/src/player/adpcm.i b/src/player/adpcm.i new file mode 100644 index 0000000..5239e6d --- /dev/null +++ b/src/player/adpcm.i @@ -0,0 +1,108 @@ +; --------------------------------------------------------------------------- +; adpcm.i -- the MSM6258V, driven the way the machine's own ROM drives it. +; ROADMAP P6a, and it is P6b's transport arriving early rather than scaffolding. +; +; NOTHING HERE IS INVENTED. Every register value below is one that +; tools/analysis/21_iplrom_dmac.py decodes OUT OF THE IPL ROM's own bytes, at +; the addresses it prints: channel 3's DCR/SCR/MFC/CPR/DFC/DAR at $FF0C2E and +; the per-transfer OCR = $32 plus command $02 at $FF9A82. That is the one +; ADPCM path on this board that is known-correct because Sharp wrote it. +; +; WHY THIS FILE EXISTS AT ALL. Session 33 fed the chip from Lua and got +; silence, swept control 0..3 against port C 0..15, and stopped rather than +; guess (65.5). Two of the reasons are visible from here and neither is a +; register semantic anybody had to guess: +; +; * THE PPI'S PORT C IS NOT AN OUTPUT UNTIL IT IS TOLD TO BE. The ADPCM pan +; and the sample-rate divider are port C bits, and an i8255 in its reset +; state has every port an INPUT -- so a write to $E9A005 changes a latch +; nobody is reading and the pan never leaves wherever it was. Control word +; $92 (mode 0, A and B input, both halves of C OUTPUT) is what makes the +; other write mean anything. +; * AND FEEDING IT SLOWLY IS NOT FEEDING IT. The chip has no FIFO and no +; starvation state: it consumes a nibble every sample period out of whatever +; its data register last held, forever. A byte per host frame is not a +; quiet chip, it is the same two nibbles 130 times, which saturates in six +; samples. The feed has to be paced by the chip, which is what channel 3 +; and its request line are FOR. +; +; THE CLOCK IS TWO WRITES AND THEY ARE IN DIFFERENT DEVICES. 15,625 Hz is +; 8 MHz / 512: the 8 MHz comes from CT1 in the YM2151's port register $1B, and +; the /512 from port C bits 3,2 = 10. Neither is readable, so the rate is +; verified from the OTHER end -- the capture's own sample count. + +AD_CTRLR = $E92001 ; W: command R: status (bit7 = NOT playing) +AD_DATAR = $E92003 ; W: the byte the chip takes two nibbles from +AD_PLAY = $02 ; COMMAND_PLAY -- session 33's probes wrote $01, +AD_STOP = $01 ; which is COMMAND_STOP +PPI_PC = $E9A005 +PPI_CTL = $E9A007 +PPI_COUT = $92 ; mode 0, A/B input, BOTH halves of C output +PPI_RATE = $08 ; pan 00 = both, rate 10 = /512 = 15,625 Hz +YM_ADDR = $E90001 +YM_DATA = $E90003 +YM_CT = $1B ; CT1 in bit 1: 0 = ADPCM master clock 8 MHz + +AD_DMAC = $E840C0 ; HD63450 channel 3 -- the ADPCM channel, and +A3_CSR = AD_DMAC+$00 ; the one the ROM points at $E92003 +A3_CER = AD_DMAC+$01 +A3_DCR = AD_DMAC+$04 +A3_OCR = AD_DMAC+$05 +A3_SCR = AD_DMAC+$06 +A3_CCR = AD_DMAC+$07 +A3_MTC = AD_DMAC+$0A +A3_MAR = AD_DMAC+$0C +A3_DAR = AD_DMAC+$14 +A3_MFC = AD_DMAC+$29 +A3_CPR = AD_DMAC+$2D +A3_DFC = AD_DMAC+$31 + +A3_DCRV = $80 ; XRM 10 cycle steal w/o hold, DTYP 00 dual + ; address, DPS 0 8-bit port (IPL $FF0C2E) +A3_OCRV = $32 ; DIR memory->device, SIZE 11 byte unpacked, + ; CHAIN 00, REQG 10 EXTERNAL REQUEST (IPL + ; $FF9A82). External request is what makes the + ; chip the pacemaker: one byte per #DRQ3, and + ; #DRQ3 ticks at half the sample rate. +A3_SCRV = $04 ; MAC 01 memory increment, DAC 00 -- the device + ; address is a REGISTER and must not walk +A3_CCRST = $80 + +; --------------------------------------------------------------- ad_setup +; The clock and the pan. No arguments, no result; trashes d0. +ad_setup: + move.b #YM_CT,YM_ADDR + moveq #60,d0 ; the YM2151 wants settling between the +.ymw: subq.l #1,d0 ; address write and the data write + bne.s .ymw + move.b #$00,YM_DATA ; CT1 = 0 -> ADPCM master clock 8 MHz + move.b #PPI_COUT,PPI_CTL ; ...and NOW port C drives something + move.b #PPI_RATE,PPI_PC ; pan both, /512 + rts + +; ----------------------------------------------------------------- ad_arm +; Arm channel 3 to feed (a1) for d1 bytes and start it. Trashes d0. +; The channel is started BEFORE the chip is told to play (see ad_play), so that +; byte 0 is already in the data register when the accumulator is reset. +ad_arm: + move.b #$FF,A3_CSR ; CSR is write-one-to-clear: a stale COC + ; would pass the wait loop instantly + move.b #A3_DCRV,A3_DCR + move.b #A3_SCRV,A3_SCR + move.b #$05,A3_MFC + move.b #$05,A3_DFC + move.b #$01,A3_CPR ; the ROM's own priority: ADPCM outranks + ; the disk at the arbiter (52.5 item 5) + move.l #AD_DATAR,A3_DAR + move.b #A3_OCRV,A3_OCR + move.l a1,A3_MAR + move.w d1,A3_MTC + move.b #A3_CCRST,A3_CCR + rts + +ad_play: + move.b #AD_PLAY,AD_CTRLR + rts +ad_halt: + move.b #AD_STOP,AD_CTRLR + rts diff --git a/src/player/adpcmgate.s b/src/player/adpcmgate.s new file mode 100644 index 0000000..d259b0a --- /dev/null +++ b/src/player/adpcmgate.s @@ -0,0 +1,112 @@ +; Front-end for the ADPCM transport (ROADMAP P6a), for the rig. +; +; It plays ONE buffer of nibbles the host pushed into RAM and reports what the +; channel did. What is being measured is not this code -- it is the CHIP: which +; delta formula, which nibble of a byte first, where the accumulator clamps, and +; what it starts at. tools/bench/verify_adpcm_chip.py reads all four out of +; MAME's own -wavwrite capture. +; +; THE ONE THING THIS FILE HAS TO GET RIGHT is the order of the two starts. The +; chip resets its accumulator, its step index AND its nibble select when it is +; told to PLAY, and it begins consuming immediately out of whatever its data +; register holds. So the channel goes first and the CPU waits for MTC to move +; -- proof that a byte has actually been taken -- before the PLAY. The stream +; still has a prologue of unknown length, because the gap between PLAY and the +; NEXT #DRQ3 is not ours to set; the prologue is 16 zero nibbles for exactly +; that reason and the verifier reads its length off the capture. + + include "src/player/geom.i" + +AD_FLAG = $18600 ; u32 0 idle, 1 armed, 2 playing, $FF done, $EE error +AD_BUF = $18604 ; u32 where the nibble bytes are +AD_LEN = $18608 ; u32 how many bytes +AD_MTC0 = $1860C ; u32 MTC at the instant PLAY was written +AD_CSRF = $18610 ; u32 CSR at completion +AD_CERF = $18614 ; u32 CER with it +AD_MTCF = $18618 ; u32 MTC with it +AD_MARF = $1861C ; u32 MAR with it -- where the channel stopped +AD_SPIN = $18620 ; u32 trips round the wait loop +AD_STAT = $18624 ; u32 the chip's own status byte while playing +AD_PATIENCE = 60000000 +AD_SETTLE = 60000 ; ~100 ms at 10 MHz, 18 clocks a trip ; the wait is bounded like every other + + org $10000 +start: + bsr ad_setup + + ; ---- SETTLE, and it is not superstition. The 8 MHz ADPCM clock is + ; CT1 in the YM2151's port register, and this machine delivers that + ; write to the ADPCM chip on the SOUND system's own schedule rather + ; than at the instant of the store -- so a transfer started in the same + ; breath as ad_setup plays its first ~17 ms at the PREVIOUS clock. The + ; symptom is exact: the capture's first 130-odd samples come out in + ; identical PAIRS, because the chip is clocking half as fast as the + ; capture, and every model then fails to fit a stream that changed rate + ; part way through. ~100 ms of nothing costs the gate nothing and a + ; player sets its clock once at boot. + move.l #AD_SETTLE,d0 +.settle:subq.l #1,d0 + bne.s .settle + + movea.l AD_BUF.l,a1 + move.l AD_LEN.l,d1 + bsr ad_arm + move.l #1,AD_FLAG.l + + ; ---- wait for the channel to actually take byte 0. Not a delay loop: + ; the condition is MTC having moved, which is the channel's own account. + move.l #AD_PATIENCE,d3 +.first: move.w A3_MTC,d0 + andi.l #$FFFF,d0 + cmp.l AD_LEN.l,d0 + bne.s .go + subq.l #1,d3 + bne.s .first + bra bad + +.go: move.l d0,AD_MTC0.l + bsr ad_play + move.l #2,AD_FLAG.l + moveq #0,d0 + move.b AD_CTRLR,d0 ; bit 7 clear = the chip says it is playing + move.l d0,AD_STAT.l + + clr.l AD_SPIN.l + move.l #AD_PATIENCE,d3 +.wait: addq.l #1,AD_SPIN.l + move.b A3_CSR,d4 + btst #4,d4 ; ERR -- CER says which + bne.s bad + btst #7,d4 ; COC + bne.s .fin + subq.l #1,d3 + bne.s .wait + bra.s bad + +.fin: bsr report + ; The chip is left PLAYING deliberately: it goes on replaying the last + ; byte it was given, which the verifier ignores. Stopping here would + ; put a silence in the capture at a point the host would then have to + ; find, and the capture already has a length it knows. + move.l #$FF,AD_FLAG.l +hold: bra.s hold + +bad: bsr report + move.l #$EE,AD_FLAG.l + bra.s hold + +report: + moveq #0,d0 + move.b A3_CSR,d0 + move.l d0,AD_CSRF.l + moveq #0,d0 + move.b A3_CER,d0 + move.l d0,AD_CERF.l + move.w A3_MTC,d0 + andi.l #$FFFF,d0 + move.l d0,AD_MTCF.l + move.l A3_MAR,d0 + move.l d0,AD_MARF.l + rts + + include "src/player/adpcm.i" diff --git a/tools/analysis/33_adpcm_model.py b/tools/analysis/33_adpcm_model.py new file mode 100644 index 0000000..17be4ed --- /dev/null +++ b/tools/analysis/33_adpcm_model.py @@ -0,0 +1,123 @@ +#!/usr/bin/env python3 +"""What the chip's own decoder model costs the encoder. ROADMAP P6a, after it. + +tools/bench/adpcm_run.sh MEASURED four things about the MSM6258 as this machine +models it, and tools/encoder/adpcm.py had a different value for every one: + + axis encoder default the chip how it was measured + feed both-high-first both-LOW-first + formula shift terms 1,678 samples, sample-exact + clamp 12-bit 10-bit one model of sixteen matched + init 0 -2 + +This file prices them, on the same ten seconds of the same stream every audio +figure in this project is quoted against (tmp/au_singe.raw, FINDINGS 65). It +takes an explicit source file rather than defaulting to one, for the same reason +every rate in this tree is an explicit argument (FINDINGS 50). + +THE ONE THAT IS NOT A UNIT SLIP is the CLAMP. The other three are conventions: +get one wrong and the decode is wrong, get it right and nothing is lost. A +10-bit accumulator is a smaller container, and it is INSIDE the recursion -- the +predictor cannot represent what will not fit -- so it costs SNR even when the +encoder knows about it and encodes for it. That is a ceiling on this format on +this machine and it is not recoverable by encoding harder. +""" +import math, os, sys +sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "encoder")) +import adpcm + +RAW = sys.argv[1] if len(sys.argv) > 1 else "tmp/au_singe.raw" +CHIP = dict(variant="terms", order="low", bits=10, init=-2) +ENC = dict(variant="shift", order="high", bits=12, init=0) + + +def snr_db(ref, got): + num = sum(float(s) * s for s in ref) + den = sum((float(a) - b) ** 2 for a, b in zip(ref, got)) + if den == 0: return float("inf") + return 10.0 * math.log10(num / den) if num else float("-inf") + + +def main(): + import struct + if not os.path.exists(RAW): + print(f"no {RAW} -- run tools/encoder/extract_audio.py first") + return 2 + pcm = struct.unpack("<%dh" % (os.path.getsize(RAW) // 2), open(RAW, "rb").read()) + src12 = [max(-2048, min(2047, x >> 4)) for x in pcm] + print(f"{RAW}: {len(src12):,} samples, peak {max(abs(v) for v in src12)} " + f"in 12-bit units") + + print() + print("1. THE COST OF ENCODING FOR THE WRONG CHIP, all four axes at once") + print(" Encode under the encoder's defaults; play it on the chip. The") + print(" nibble ORDER is not a decode parameter -- it decides which nibble") + print(" of each byte the chip takes -- so it is applied by re-reading the") + print(" encoder's own packed bytes the way the chip reads them.") + nib = adpcm.encode(src12, ENC["variant"], init=ENC["init"], bits=ENC["bits"]) + same = adpcm.decode(nib, ENC["variant"], init=ENC["init"], bits=ENC["bits"]) + data = adpcm.pack(nib, ENC["order"]) + asread = list(adpcm.unpack(data, len(nib), CHIP["order"])) + cross = adpcm.decode(asread, CHIP["variant"], init=CHIP["init"], bits=CHIP["bits"]) + print(f" encoded and decoded on the encoder's model : {snr_db(src12, same):7.2f} dB") + print(f" encoded on the encoder's, played on the chip: {snr_db(src12, cross):7.2f} dB") + + print() + print("2. ONE AXIS AT A TIME, so the bill is itemised rather than lumped") + for name, key, val in (("nibble order", "order", CHIP["order"]), + ("delta formula", "variant", CHIP["variant"]), + ("clamp", "bits", CHIP["bits"]), + ("initial accumulator", "init", CHIP["init"])): + m = dict(ENC); m[key] = val + d = adpcm.pack(nib, ENC["order"]) + rd = list(adpcm.unpack(d, len(nib), m["order"])) + got = adpcm.decode(rd, m["variant"], init=m["init"], bits=m["bits"]) + print(f" {name:22s} wrong only here: {snr_db(src12, got):7.2f} dB") + + print() + print("3. AND THE ONE THAT IS NOT A CONVENTION. Encode FOR the chip -- the") + print(" encoder knows the model and searches against it -- and compare a") + print(" 10-bit accumulator with a 12-bit one on the same seconds.") + for bits in (12, 10): + n = adpcm.encode(src12, CHIP["variant"], init=CHIP["init"], bits=bits) + r = adpcm.decode(n, CHIP["variant"], init=CHIP["init"], bits=bits) + clip = sum(1 for v in r if v in adpcm.clamp_bounds(bits)) + print(f" encoded and decoded at {bits}-bit: {snr_db(src12, r):7.2f} dB" + f" ({clip:,} of {len(r):,} samples sit ON the clamp)") + + print() + print("4. WHAT THE LEVEL DOES NOW, and it did nothing before (65.1).") + print(" At 12 bits the disc's -13.4 dBFS peak had headroom to spare and") + print(" normalising bought 0.00 dB. A 10-bit accumulator is 4x smaller,") + print(" so the same signal is no longer comfortably inside it.") + peak = max(abs(v) for v in src12) + for name, g in (("as recorded", 1.0), + ("scaled to fit 10 bits", 500.0 / peak), + ("half of that", 250.0 / peak)): + sc = [max(-512, min(511, int(round(v * g)))) for v in src12] + n = adpcm.encode(sc, CHIP["variant"], init=CHIP["init"], bits=CHIP["bits"]) + r = adpcm.decode(n, CHIP["variant"], init=CHIP["init"], bits=CHIP["bits"]) + print(f" {name:24s} x{g:5.2f} peak {max(abs(v) for v in sc):4d} " + f"{snr_db(sc, r):7.2f} dB") + print() + print("5. THE HEADROOM, which is the part of this that will bite later.") + hd = 20 * math.log10(511.0 / peak) + print(f" This window peaks at {peak} of the 10-bit accumulator's 511, so it") + print(f" has {hd:.1f} dB of headroom left -- and it is a QUIET passage: the") + print(" disc peaks at -13.4 dBFS here (65.1). A 10-bit accumulator is") + print(f" {20*math.log10(2047.0/511.0):.1f} dB smaller than the 12-bit word the encoder was") + print(" clamping to, so a passage only a few dB louder than this one does") + print(" not fit and the predictor CLIPS inside the recursion. Nothing in") + print(" this project has measured the loudest passage on the disc; until") + print(" something does, the audio level is an OPEN choice and not a") + print(" settled one, and 65.1's `the level is not a lever` is now wrong") + print(" in one direction: it is not a lever UPWARD.") + print() + print(" The rows in 4 are NOT comparable as absolute quality") + print(" -- each is scored against its OWN scaled reference, so what they") + print(" compare is how well the format tracks a signal of that size.") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/tools/bench/adpcm.lua b/tools/bench/adpcm.lua new file mode 100644 index 0000000..fb9ad8c --- /dev/null +++ b/tools/bench/adpcm.lua @@ -0,0 +1,88 @@ +-- Play one buffer of ADPCM nibbles on the emulated MSM6258V, from 68000 code +-- (ROADMAP P6a). The Lua here does what Lua is allowed to do in this tree: +-- push bytes in, start the CPU, read the mailbox out. It is NOT in the feed +-- path -- session 33's probe was, and a host that writes the data register at +-- host-frame rate is not feeding a chip that consumes at 15,625 Hz (65.5). +-- +-- WHAT THE MEASUREMENT IS. MAME's -wavwrite capture, at a sample rate chosen +-- to EQUAL the chip's stream rate so nothing resamples it, is the chip's own +-- output. tools/bench/verify_adpcm_chip.py reads the four model axes out of it. +M = manager.machine +SP = M.devices[":maincpu"].spaces["program"] + +local META = loadfile("adpcm_meta.lua")() +local AD_FLAG, AD_BUF, AD_LEN = 0x18600, 0x18604, 0x18608 +local AD_MTC0, AD_CSRF, AD_CERF = 0x1860C, 0x18610, 0x18614 +local AD_MTCF, AD_MARF, AD_SPIN, AD_STAT = 0x18618, 0x1861C, 0x18620, 0x18624 + +local code do local f=io.open("adpcmgate.bin","rb"); code=f:read("a"); f:close() end +local data do local f=io.open("adpcm_data.bin","rb"); data=f:read("a"); f:close() end + +local function P(s) print("[ADP] "..s) end +local function T() local t=M.time; return t.seconds + t.attoseconds/1e18 end + +local st, t0, tplay = "boot", nil, nil +SUB = emu.add_machine_frame_notifier(function() + local ok, err = pcall(function() + local t = T() + if st == "boot" then + if t < 3.0 then return end + for i = 1, #data do SP:write_u8(META.buf + i - 1, string.byte(data, i)) end + for i = 1, #code do SP:write_u8(0x10000 + i - 1, string.byte(code, i)) end + SP:write_u32(AD_FLAG, 0) + SP:write_u32(AD_BUF, META.buf) + SP:write_u32(AD_LEN, META.nbytes) + local cpu = M.devices[":maincpu"] + cpu.state["SR"].value = 0x2700 -- supervisor, ALL interrupts masked + cpu.state["SP"].value = 0x8000 + cpu.state["PC"].value = 0x10000 + P(string.format("pushed %d B of code and %d B of nibbles at 0x%X", + #code, META.nbytes, META.buf)) + -- THE CAPTURE'S OWN CLOCK. The wav starts at t=0 of the run, so the host + -- has to know when PLAY happened to find the stream in it -- but it is + -- NOT used as the alignment: the verifier searches a small window around + -- it, because a host frame is 17.6 ms and a sample is 64 us. + st, t0 = "running", t + return + end + if st == "running" then + local fl = SP:read_u32(AD_FLAG) + if fl == 2 and not tplay then + tplay = t + P(string.format("PLAY at t=%.4f s, chip status $%02X (bit7 clear = playing), " + .."MTC then = %d of %d", t, SP:read_u32(AD_STAT), + SP:read_u32(AD_MTC0), META.nbytes)) + end + if fl == 0xFF or fl == 0xEE then + P(string.format("channel finished: CSR=$%02X CER=$%02X MTC=%d MAR=$%06X " + .."spin=%d", SP:read_u32(AD_CSRF), SP:read_u32(AD_CERF), + SP:read_u32(AD_MTCF), SP:read_u32(AD_MARF), + SP:read_u32(AD_SPIN))) + local dt = t - (tplay or t) + P(string.format("%d bytes took %.4f s = %.1f B/s " + .."(15,625 nibbles/s wants 7,812.5)", + META.nbytes, dt, META.nbytes/dt)) + if fl == 0xEE then P("ERROR: the gate flagged a channel error or a timeout") end + local f = io.open("adpcm_run.lua", "w") + f:write(string.format("return { tplay = %.9f, ok = %s, nbytes = %d,\n" + .." csr = %d, cer = %d, mtc = %d, spin = %d }\n", + tplay or -1, tostring(fl == 0xFF), META.nbytes, + SP:read_u32(AD_CSRF), SP:read_u32(AD_CERF), + SP:read_u32(AD_MTCF), SP:read_u32(AD_SPIN))) + f:close() + st = "drain"; t0 = t + return + end + if t - t0 > 60 then P("TIMEOUT flag="..string.format("%08X", fl)); M:exit() end + return + end + if st == "drain" then + -- let the capture run past the end of the stream, so a truncated wav is + -- never mistaken for a short stream + if t - t0 < 0.3 then return end + P("done") + M:exit() + end + end) + if not ok then print("[ADP] LUA ERROR: "..tostring(err)); M:exit() end +end) diff --git a/tools/bench/adpcm_run.sh b/tools/bench/adpcm_run.sh new file mode 100755 index 0000000..eb4e3f7 --- /dev/null +++ b/tools/bench/adpcm_run.sh @@ -0,0 +1,56 @@ +#!/bin/bash +# ROADMAP P6a: ask the machine's own MSM6258V which decoder it is. +# +# tools/bench/adpcm_run.sh +# +# WHAT A GREEN RUN MEANS: 68000 code programmed HD63450 channel 3 exactly as the +# IPL ROM programs it -- dual address, 8-bit port, cycle steal, EXTERNAL request +# -- fed the chip a designed nibble stream at the chip's own pace, and exactly +# ONE of sixteen candidate decoder models reproduces MAME's capture of the +# result SAMPLE-EXACT, with every one of the four axes shown to matter. +# +# WHAT IT DOES NOT MEAN: anything about an MSM6258. This is MAME's device model +# measured end to end through the machine's real transport. It settles the RIG. +# The silicon stays on the hardware list. +set -e +cd "$(dirname "$0")/../.." + +tools/vasm/vasmm68k_mot -Fbin -o tmp/adpcmgate.bin src/player/adpcmgate.s > /dev/null +python3 tools/bench/prep_adpcm.py + +# -samplerate 15625 is not a preference: it is the chip's own stream rate +# (8 MHz / 512), and equal rates are what keep MAME's resampler from filtering +# the thing being measured. The first cut of this ran at the default 48000 and +# every reconstructed sample arrived as an interpolated pair. +( cd tmp && SDL_VIDEODRIVER=dummy stdbuf -oL timeout -k 5 300 \ + mame x68000 -bios ipl10 -ramsize 2M -video soft -window \ + -samplerate 15625 -wavwrite adpcm.wav -nothrottle -plugins \ + -autoboot_script ../tools/bench/adpcm.lua \ + -seconds_to_run 12 > adpcm_run.log 2>&1 ) +grep -aq "^\[ADP\] done" tmp/adpcm_run.log || { + echo "FAIL: the ADPCM gate did not finish -- no completion marker." + tail -8 tmp/adpcm_run.log; exit 1; } +grep -a "^\[ADP\]" tmp/adpcm_run.log | sed 's/^\[ADP\] / /' + +fail() { echo "FAIL: $1"; exit 1; } +if grep -aq "^\[ADP\] ERROR" tmp/adpcm_run.log; then + fail "the channel reported an error or the gate timed out -- see CSR/CER above." +fi +grep -aq "bit7 clear = playing" tmp/adpcm_run.log || \ + fail "the chip never reported itself playing." +# THE FEED RATE IS A GATE, not a note. The chip is the pacemaker: one byte per +# #DRQ3 and #DRQ3 at half the sample rate. If the bytes went out at some other +# rate then the channel was NOT being paced by the device, and every sample +# below is of a stream that arrived faster or slower than it was consumed -- +# which is precisely the failure session 33 hit from Lua. +RATE=$(sed -n 's/.*= \([0-9.]*\) B\/s .*/\1/p' tmp/adpcm_run.log | head -1) +python3 - "$RATE" <<'PY' || fail "the feed was not paced by the chip (see above)." +import sys +r = float(sys.argv[1]) +want = 7812.5 +print(f" feed rate {r:,.1f} B/s against the chip's own {want:,.1f} B/s " + f"({100*(r-want)/want:+.2f}%)") +sys.exit(0 if abs(r - want) / want < 0.02 else 1) +PY + +python3 tools/bench/verify_adpcm_chip.py tmp/adpcm.wav tmp/adpcm_seq.json diff --git a/tools/bench/check.sh b/tools/bench/check.sh index b9326d0..e4cf3c4 100755 --- a/tools/bench/check.sh +++ b/tools/bench/check.sh @@ -753,4 +753,21 @@ python3 tools/analysis/32_audio_wire.py tmp/packed_singe.dlxp \ > tmp/audio_wire.log 2>&1 || { cat tmp/audio_wire.log; exit 1; } grep -aE "^ ( 1| 11| 81) |THE FLOOR|F=1 |F=11|is ZERO|SOUND IS WHAT" tmp/audio_wire.log +echo "--- session 34: THE CHIP'S OWN DECODER, off the machine (FINDINGS 66) ---" +# ROADMAP P6a. 68000 code programs HD63450 channel 3 exactly as the IPL ROM +# programs it and feeds the MSM6258 a designed nibble stream at the chip's own +# pace; ONE of sixteen candidate decoder models reproduces MAME's capture +# sample-exact, and every axis has a negative control. The encoder disagreed +# with the chip on ALL FOUR axes, and the largest of them is not the delta +# formula 65 named -- it is the NIBBLE ORDER, at -25.7 dB. +bash tools/bench/adpcm_run.sh > tmp/adpcm_gate.log 2>&1 || { + cat tmp/adpcm_gate.log; exit 1; } +grep -aE "^(OK|FAIL) |^ (feed|nibbles|delta|clamp|accumulator)" tmp/adpcm_gate.log \ + | sed 's/^/ /' +# And the bill, on the same ten seconds every other audio figure is quoted on. +python3 tools/analysis/33_adpcm_model.py tmp/au_singe.raw > tmp/adpcm_model.log 2>&1 \ + || { cat tmp/adpcm_model.log; exit 1; } +grep -aE "wrong only here|played on the chip|decoded on the encoder|headroom left" \ + tmp/adpcm_model.log + echo "ALL GREEN" diff --git a/tools/bench/prep_adpcm.py b/tools/bench/prep_adpcm.py new file mode 100644 index 0000000..1f31404 --- /dev/null +++ b/tools/bench/prep_adpcm.py @@ -0,0 +1,141 @@ +#!/usr/bin/env python3 +"""Build the nibble stream that asks the MSM6258 which decoder it is. +ROADMAP P6a. + +WHAT HAS TO BE DISCRIMINATED, and it is four things rather than the one +FINDINGS 65 named: + + 1. DELTA FORMULA -- 'shift' (ffmpeg's adpcm_ima_oki) against 'terms' (the + datasheet's per-term truncation). Worth 25 dB (65.2). + 2. NIBBLE ORDER -- which half of a byte handed to the data register is played + FIRST. 65.1 measured 'high' AGAINST FFMPEG, which is a fact about the VOX + file convention and not about a chip's data register. + 3. THE CLAMP -- the accumulator saturates somewhere, and where is inside + the recursion, so it is not an output scaling that can be undone. + 4. THE INITIAL ACCUMULATOR at the instant of PLAY. + +The stream is in three parts and each part exists for a reason: + + PROLOGUE, 16 zero nibbles. Nibble 0 moves the step index DOWN, so it stays + pinned at 0 and the delta is a constant +2 under every candidate. That makes + the prologue a RAMP that both formulas agree on, which is what absorbs the one + thing this rig cannot control: how many times the chip consumes byte 0 before + the channel delivers byte 1. The verifier reads that count off the capture + instead of assuming it. + + SEGMENT A, a quiet sine, encoded by tools/encoder/adpcm.py itself. Amplitude + 300 keeps it clear of even the 10-bit clamp, so A discriminates the FORMULA + and the ORDER without the clamp confounding either. Using the shipping + encoder rather than a hand-written pattern is deliberate: the nibbles the chip + is asked about are the kind of nibbles it will be sent. + + SEGMENT B, loud bursts. It exists ONLY to cross the 10-bit clamp, which + segment A is built never to reach, and it is last because a clamp is + irreversible state and everything after it would be measuring segment B. +""" +import json, math, os, sys +sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "encoder")) +import adpcm + +PRO_NIB = 16 # prologue nibbles (byte 0 = $00, so a repeat costs nothing) +A_SAMPLES = 1500 # segment A, one nibble each +A_AMP = 300 # clear of the 10-bit clamp at 511 with room for the ramp +A_HZ = 61.0 # ~256 samples a cycle at 15,625 Hz: many step indices +RATE = 15625.0 +BUF = 0x30000 # where the harness pushes the bytes +OUT_BIN = "tmp/adpcm_data.bin" +OUT_META = "tmp/adpcm_meta.lua" +OUT_SEQ = "tmp/adpcm_seq.json" + + +# THE TRIGGER, and it is here because the first cut of this file did not have +# one and measured ONE differing sample in 1,676. The two formulas are +# IDENTICAL whenever the step value is a multiple of 8: +# +# terms - shift = b2*floor(r/2) + b3*floor(r/4) - floor((4*b2+2*b3+1)*r/8) +# +# with r = step mod 8 and (b1,b2,b3) the nibble's low three bits. It is zero +# for r = 0, and the step table STARTS at 16. A quiet signal never moves the +# step index off its floor, so a probe made of quiet nibbles asks the chip a +# question that has the same answer either way. +# +# nibble 4 at step 16: delta 18 under both, and it moves the index to 2 +# nibble 3 at step 19: shift 16, terms 15 <- the two states part company +# +# After that they never rejoin, because the delta is added to a running +# predictor -- so ONE two-nibble trigger converts the rest of the stream into +# discriminating evidence. That is the same recursion 65.2 priced at 25 dB, +# used deliberately instead of suffered. +TRIGGER = [4, 3] + + +def segment_a(): + """The trigger, then a sine encoded by the shipping encoder. The model the + sine is encoded under does not matter for discrimination -- once the trigger + has parted the two states, any nibble stream keeps them apart -- so the + defaults are used and the choice is recorded rather than tuned. Using the + shipping encoder rather than a hand-written pattern is the point: the + nibbles the chip is asked about are the kind of nibbles it will be sent.""" + sig = [int(round(A_AMP * math.sin(2 * math.pi * A_HZ * i / RATE))) + for i in range(A_SAMPLES)] + return TRIGGER + list(adpcm.encode(sig, "shift")) + + +def segment_b(): + """Loud, and alternating in sign so the step index does not simply pin: 40 + up, 40 down, twice. Under a 10-bit accumulator this saturates; under a + 12-bit one it does not, and that difference is the whole point of it.""" + return ([7] * 40 + [15] * 40) * 2 + + +def main(): + core = segment_a() + segment_b() + nibs = [0] * PRO_NIB + core + data = adpcm.pack(nibs, "high") # HIGH first: the encoder's convention, + # which is one of the things on trial + os.makedirs("tmp", exist_ok=True) + open(OUT_BIN, "wb").write(data) + + # HOW MUCH DISCRIMINATING POWER IS IN IT, counted rather than asserted. A + # probe that cannot separate two candidates reports a match against both and + # a gate that did not count this would call that a result. + ref = adpcm.decode(nibs, "shift", init=-2, bits=10) + axes = {} + for name, kw in (("formula", dict(variant="terms")), + ("order", dict(order="low")), + ("clamp", dict(bits=12)), + ("init", dict(init=0))): + order = kw.pop("order", "high") + n2 = ([0] * PRO_NIB + + list(adpcm.unpack(data, len(nibs), order))[PRO_NIB:]) \ + if order != "high" else nibs + n2 = list(adpcm.unpack(data, len(nibs), order)) + alt = adpcm.decode(n2, kw.get("variant", "shift"), + init=kw.get("init", -2), bits=kw.get("bits", 10)) + d = sum(1 for a, b in zip(ref, alt) if a != b) + axes[name] = d + + seq = {"nibbles": nibs, "core": core, "pro": PRO_NIB, + "bytes": len(data), "buf": BUF, "axes": axes, + "a_samples": A_SAMPLES, "a_amp": A_AMP, "a_hz": A_HZ} + json.dump(seq, open(OUT_SEQ, "w")) + + with open(OUT_META, "w") as f: + f.write("return {\n") + f.write(f" buf = 0x{BUF:X},\n") + f.write(f" nbytes = {len(data)},\n") + f.write(f" nnibs = {len(nibs)},\n") + f.write("}\n") + + print(f" probe stream: {len(nibs)} nibbles = {len(data)} B " + f"= {len(nibs)/RATE*1000:.1f} ms at 15,625 Hz") + print(f" prologue {PRO_NIB} zero nibbles, segment A {len(segment_a())} " + f"(sine {A_AMP} @ {A_HZ} Hz), segment B {len(segment_b())} (loud)") + print(" DISCRIMINATING POWER -- samples that change when ONE axis is " + "flipped away from MAME's own model:") + for k, v in axes.items(): + print(f" {k:8s} {v:5d} of {len(ref)}") + + +if __name__ == "__main__": + main() diff --git a/tools/bench/probe_adpcm4.lua b/tools/bench/probe_adpcm4.lua new file mode 100644 index 0000000..6766534 --- /dev/null +++ b/tools/bench/probe_adpcm4.lua @@ -0,0 +1,33 @@ +-- SPIKE (session 34). Not a gate: it exists to find out whether the chip can be +-- driven at all, what scale its samples arrive at in a -wavwrite capture, and +-- where it clamps. It sets EVERY thing session 33's probes left to the IPL: +-- * YM2151 reg $1B bit1 = 0 -> CT1 = 0 -> ADPCM master clock 8 MHz +-- * PPI control $92 -> port C is an OUTPUT (without this the i8255's +-- out_pc_callback never fires and the pan and +-- divider writes go nowhere) +-- * PPI port C $08 -> pan 00 = BOTH, rate 10 = /512 -> 15,625 Hz +-- * ctrl $02 = COMMAND_PLAY -- session 33's probes wrote $01, COMMAND_STOP. +M = manager.machine +local sp = M.devices[":maincpu"].spaces["program"] +local YMA, YMD = 0xE90001, 0xE90003 +local PPIC, PPICTL = 0xE9A005, 0xE9A007 +local CTRL, DATA = 0xE92001, 0xE92003 +local BYTE = tonumber(os.getenv("AD_BYTE") or "0x77") +local n = 0 +SUB = emu.add_machine_frame_notifier(function() + n = n + 1 + if n == 40 then + sp:write_u8(YMA, 0x1B); sp:write_u8(YMD, 0x00) + elseif n == 60 then + sp:write_u8(YMA, 0x1B); sp:write_u8(YMD, 0x00) + sp:write_u8(PPICTL, 0x92) + sp:write_u8(PPIC, 0x08) + print(string.format("[AD4] portC readback = $%02X", sp:read_u8(PPIC))) + sp:write_u8(CTRL, 0x02) + sp:write_u8(DATA, BYTE) + print(string.format("[AD4] PLAY, data $%02X, status = $%02X", + BYTE, sp:read_u8(CTRL))) + elseif n == 90 then + print("[AD4] done"); M:exit() + end +end) diff --git a/tools/bench/verify_adpcm.py b/tools/bench/verify_adpcm.py index 0ce2e87..a42d346 100644 --- a/tools/bench/verify_adpcm.py +++ b/tools/bench/verify_adpcm.py @@ -2,6 +2,14 @@ """Gate tools/encoder/adpcm.py against the only independent decoder on this machine: ffmpeg's `adpcm_ima_oki`. +THIS FILE IS NOT ABOUT THE X68000's CHIP and after session 34 that distinction +is load-bearing. It checks this implementation against an independent one, so +its parameters stay ffmpeg's -- variant 'shift', high nibble first, a 12-bit +clamp, accumulator from 0. What the MACHINE's MSM6258 does is measured by +tools/bench/adpcm_run.sh and it is a different set of four values on all four +axes (adpcm.CHIP, FINDINGS 66). Do not "fix" the defaults here to match it: a +reference check whose reference has been adjusted to agree is not a check. + There is no ffmpeg ENCODER for this format -- `adpcm_ima_oki` is decode-only -- so the encoder here cannot be checked against a reference implementation. What CAN be checked, and is, is that the decoder our encoder runs in its own loop is @@ -70,14 +78,22 @@ ck(ff == ours, f"variant 'shift' is SAMPLE-EXACT vs ffmpeg over {len(nibs)} nibb lowfirst = [adpcm.unpack(data, len(nibs))[i ^ 1] for i in range(len(nibs))] bad = [v * 16 for v in adpcm.decode(lowfirst, "shift")] ndiff = sum(1 for a, b in zip(ff, bad) if a != b) -ck(ndiff > 0, f"low-nibble-first DISAGREES on {ndiff}/{len(ff)} -- so the order is measured, not assumed") +ck(ndiff > 0, f"low-nibble-first DISAGREES on {ndiff}/{len(ff)} -- so what ffmpeg reads is measured, not assumed") +# AND IT IS A FACT ABOUT A FILE FORMAT, NOT ABOUT A CHIP. Session 33 recorded +# this line as "nibble order: HIGH FIRST, measured", which it is -- of the VOX +# convention ffmpeg implements. Session 34 asked the machine's own MSM6258 the +# same question through HD63450 channel 3 and got the OTHER answer: the chip +# takes the LOW nibble of a delivered byte first (FINDINGS 66), and encoding +# for the wrong one of the two costs -25.7 dB on the Singe window. The two +# claims do not conflict; they are about different things, and only one of them +# is about the machine this is being ported to. print("--- and the second variant is not the same decoder ---") terms = [v * 16 for v in adpcm.decode(adpcm.unpack(data, len(nibs)), "terms")] d = [abs(a - b) // 16 for a, b in zip(ff, terms)] nd = sum(1 for x in d if x) ck(nd > 0, f"variant 'terms' differs on {nd}/{len(d)} samples, max {max(d)} in 12-bit units" - " -- OPEN: which one the MSM6258 runs is unmeasured") + " -- and 'terms' is the one the machine runs (adpcm_run.sh, FINDINGS 66)") print("--- and getting the variant wrong is NOT a rounding error ---") # THE MEASUREMENT THAT CHANGED THIS FROM A FOOTNOTE INTO AN OPEN ITEM. The two diff --git a/tools/bench/verify_adpcm_chip.py b/tools/bench/verify_adpcm_chip.py new file mode 100644 index 0000000..a4973fa --- /dev/null +++ b/tools/bench/verify_adpcm_chip.py @@ -0,0 +1,176 @@ +#!/usr/bin/env python3 +"""Read the MSM6258's own decoder out of a MAME capture. ROADMAP P6a. + +FIVE THINGS ARE UNKNOWN, not the one FINDINGS 65 named: + + feed which nibble(s) of a delivered BYTE the chip actually plays + variant 'shift' | 'terms' the delta formula (65.2, worth 25 dB) + bits 12 | 10 where the accumulator clamps + init 0 | -2 the accumulator at PLAY + (and the capture's own decimation, below) + +`feed` is on the list because the run MEASURED it and it is not what anybody +assumed. Fed by HD63450 channel 3 in the IPL ROM's own configuration, this +machine plays ONE nibble per delivered byte -- the prologue is 16 zero nibbles +in 8 bytes and the accumulator climbs by 8 steps, not 16. A probe that had +assumed two would have found no model that fit and reported a broken rig. So +the hypothesis is enumerated with the others and the capture picks. + +THE CAPTURE'S DECIMATION is enumerated for the same reason. MAME resamples the +chip's stream to the wav's rate, and a filtered 2x upsample is not recognisable +from a single sample -- on a slow ramp it looks like an exact repeat and on a +step it does not. So (factor, phase) is searched over {1x, 2x phase 0, 2x phase +1}, and the SCALE RESIDUAL is then checked on the decimated stream: MAME's +okim6258 puts `signal << 4` into a stream scaled to 32768 and the machine routes +it to the speaker at gain 0.50, so a chip sample is `signal * 8`. If the +winning decimation does not land within a couple of counts of a multiple of 8 on +every sample, it is not the chip's own stream and the run says so instead of +rounding to the nearest story. + +WHAT THIS DOES NOT SETTLE. It measures MAME's device model driven through the +machine's real transport. It settles the RIG -- an emulated audio test encoded +against the wrong model is 25 dB of nothing -- and it leaves the silicon where +it was: needing a board or a datasheet. +""" +import json, os, struct, sys, wave + +sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "encoder")) +import adpcm + +WAV = sys.argv[1] if len(sys.argv) > 1 else "tmp/adpcm.wav" +SEQ = sys.argv[2] if len(sys.argv) > 2 else "tmp/adpcm_seq.json" +SCALE = 8 # okim6258's <<4, times the machine's 0.50 speaker route +PRO_MAX = 40 # zero nibbles the prologue is allowed to have grown by +SKEW = 8 # samples of slack on where PLAY lands in the capture +DECIM = ((1, 0), (2, 0), (2, 1)) +FEEDS = ("both-high-first", "both-low-first", "low-only", "high-only") + +fails = [] +def ck(ok, msg): + print(("OK " if ok else "FAIL ") + msg) + if not ok: fails.append(msg) + + +def nibbles_for(data, feed): + """The nibble sequence the chip is hypothesised to have PLAYED, given the + bytes the channel delivered.""" + if feed == "both-high-first": + return list(adpcm.unpack(data, None, "high")) + if feed == "both-low-first": + return list(adpcm.unpack(data, None, "low")) + if feed == "low-only": + return [b & 15 for b in data] + return [b >> 4 for b in data] + + +def main(): + seq = json.load(open(SEQ)) + pro, core = seq["pro"], seq["core"] + data = adpcm.pack([0] * pro + core, "high") + pro_bytes = pro // 2 # bytes of prologue, all $00 + + w = wave.open(WAV) + rate = w.getframerate() + n, ch = w.getnframes(), w.getnchannels() + s = struct.unpack("<%dh" % (n * ch), w.readframes(n)) + left, right = list(s[0::ch]), list(s[1::ch]) + ck(left == right, "both speakers carry the same samples (pan 00 = BOTH)") + ck(any(left), "the capture contains a signal at all") + if not any(left): + return 1 + + # ---- search: (decimation) x (feed) x (variant, bits, init) x (prologue) + results = {} + for fac, ph in DECIM: + rec = [round(v / SCALE) for v in left[ph::fac]] + nz = next((i for i, v in enumerate(rec) if v), None) + if nz is None: + continue + lo, hi = max(0, nz - SKEW), nz + 1 + for feed in FEEDS: + base = nibbles_for(data, feed) + # a repeat of byte 0 costs whole nibbles under 'both' and one nibble + # under 'low-only'/'high-only'; either way it is zeros + for extra in range(PRO_MAX): + for variant in ("shift", "terms"): + for bits in (12, 10): + for init in (0, -2): + want = adpcm.decode([0] * extra + base, variant, + init=init, bits=bits) + for off in range(lo, hi): + if rec[off:off + len(want)] == want: + results.setdefault( + (feed, variant, bits, init), + (fac, ph, extra, off, len(want))) + print("--- candidates that reproduce the capture SAMPLE-EXACT ---") + for k, v in results.items(): + print(f" feed={k[0]:<15s} variant={k[1]:<5s} bits={k[2]} init={k[3]:<2d}" + f" decimation {v[0]}x phase {v[1]}, prologue +{v[2]}, " + f"{v[4]:,} samples") + ck(len(results) == 1, + f"exactly one model reproduces the capture ({len(results)} did)") + if len(results) != 1: + return 1 + + model, (fac, ph, extra, off, ln) = next(iter(results.items())) + feed, variant, bits, init = model + + # ---- the scale residual, on the stream the winner actually matched + seg = left[ph::fac][off:off + ln] + worst = max(abs(v - SCALE * round(v / SCALE)) for v in seg) + ck(worst <= 2, + f"every matched sample is within {worst} of a multiple of {SCALE} -- so " + f"`signal = round(sample/{SCALE})` is a recovery and not a rounding") + + print("--- THE CHIP, AS THIS MACHINE MODELS IT ---") + print(f" nibbles played {feed}") + print(f" delta formula {variant}") + print(f" clamp {bits}-bit accumulator " + f"{adpcm.clamp_bounds(bits)}") + print(f" accumulator at PLAY {init}") + print(f" matched {ln:,} consecutive samples, " + f"capture decimated {fac}x at phase {ph}") + print(f" chip stream rate = {rate}/{fac} = {rate/fac:,.1f} Hz, and the " + f"channel delivered {len(data):,} B") + + # ---- THE NEGATIVE CONTROLS. Flip one axis alone; the match must die. + # Without these an axis the probe is BLIND to reads exactly like an axis it + # has settled, which is 58.3's vacuous-counter trap in a new place. + print("--- and every axis was actually asked (flip one, the match dies) ---") + rec = [round(v / SCALE) for v in left[ph::fac]] + flips = {"feed": [f for f in FEEDS if f != feed], + "formula": ["terms" if variant == "shift" else "shift"], + "clamp": [12 if bits == 10 else 10], + "init": [0 if init == -2 else -2]} + for name, alts in flips.items(): + worst_axis = None + for a in alts: + m = dict(feed=feed, variant=variant, bits=bits, init=init) + m[{"feed": "feed", "formula": "variant", + "clamp": "bits", "init": "init"}[name]] = a + want = adpcm.decode([0] * extra + nibbles_for(data, m["feed"]), + m["variant"], init=m["init"], bits=m["bits"]) + got = rec[off:off + len(want)] + d = sum(1 for x, y in zip(got, want) if x != y) + worst_axis = d if worst_axis is None else min(worst_axis, d) + ck(worst_axis > 0, + f"{name:8s} flipped: the closest alternative still disagrees on " + f"{worst_axis:,} of {ln:,} samples") + + print("--- against what tools/encoder/adpcm.py DEFAULTS to ---") + cur = {"feed": "both-high-first", "formula": "shift", "clamp": 12, "init": 0} + got = {"feed": feed, "formula": variant, "clamp": bits, "init": init} + for k in cur: + print(f" {k:8s} encoder {str(cur[k]):<15s} chip {str(got[k]):<15s}" + f" {'agree' if cur[k] == got[k] else 'DISAGREE'}") + + json.dump({"feed": feed, "variant": variant, "bits": bits, "init": init, + "decimation": fac, "phase": ph, "extra": extra, + "matched": ln, "chip_rate": rate / fac}, + open("tmp/adpcm_model.json", "w")) + print("ADPCM CHIP GATE " + ("GREEN" if not fails else f"RED: {len(fails)} failed")) + return 1 if fails else 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/tools/encoder/adpcm.py b/tools/encoder/adpcm.py index 742d954..7fe4f45 100644 --- a/tools/encoder/adpcm.py +++ b/tools/encoder/adpcm.py @@ -43,6 +43,48 @@ SAMPLE_MIN, SAMPLE_MAX = -2048, 2047 # the 12-bit DAC word VARIANTS = ("shift", "terms") +# --------------------------------------------------------------------------- +# THE THREE AXES THAT WERE FIXED CONSTANTS UNTIL SESSION 34, and every one of +# them turned out to be a real choice that a decoder can get wrong. FINDINGS 65 +# priced the `variant` axis at 25 dB and left the other three unnamed; MAME's +# okim6258 disagrees with this file on ALL THREE, so they are parameters now and +# tools/bench/adpcm_run.sh measures which values the emulated chip runs. +# +# ORDER which nibble of a byte is played FIRST. 'high' is the Dialogic VOX +# file convention and is what ffmpeg's adpcm_ima_oki reads, which is +# what 65.1 measured. That is a fact about a FILE FORMAT. What the +# chip does with a byte handed to its data register is a different +# question and MAME answers it 'low'. +# INIT the accumulator at the instant the chip is told to PLAY. This file +# started it at 0; MAME's okim6258 resets it to -2. +# BITS where the accumulator CLAMPS. This file clamped at the 12-bit ADPCM +# word; the MSM6258's own D/A is 10-bit and MAME clamps there, INSIDE +# the recursion, so it is not a post-hoc output scaling. +# +# Defaults are unchanged, so tools/bench/verify_adpcm.py still measures exactly +# what it measured in session 33: ffmpeg's decoder, high nibble first. +ORDERS = ("high", "low") + +# WHAT THE MACHINE'S OWN CHIP DOES, MEASURED -- tools/bench/adpcm_run.sh, one +# model of sixteen reproducing 1,678 consecutive samples of a MAME capture +# sample-exact, with a negative control on every axis (FINDINGS 66). It is a +# measurement of MAME's device model driven through the real transport, not of +# an MSM6258; the silicon is still a hardware item. +# +# THE DEFAULTS ABOVE ARE DELIBERATELY *NOT* THESE. The defaults are ffmpeg's +# adpcm_ima_oki, because tools/bench/verify_adpcm.py's whole value is that it +# checks this file against an independent implementation, and a default that +# had drifted to match the thing under test would end that. Anything that +# ENCODES FOR THE MACHINE passes CHIP explicitly. +CHIP = dict(variant="terms", order="low", bits=10, init=-2) + + +def clamp_bounds(bits): + """The accumulator's clamp, as MAME's okim6258 computes it: max = 2^(b-1)-1, + min = -2^(b-1). Note it is NOT symmetric, and the asymmetry is load-bearing + on a signal that saturates.""" + return -(1 << (bits - 1)), (1 << (bits - 1)) - 1 + def delta(nibble, step, variant): """The reconstruction step for one nibble, in 12-bit units.""" @@ -58,21 +100,21 @@ def delta(nibble, step, variant): return -d if nibble & 8 else d -def decode(nibbles, variant="shift"): - """Nibbles -> 12-bit signed samples. State is (signal, step index), both - zero at the start of a stream, which is what the chip resets to.""" - signal, idx, out = 0, 0, [] +def decode(nibbles, variant="shift", init=0, bits=12): + """Nibbles -> signed samples. State is (signal, step index); the step index + is 0 at the start of a stream and `init` is where the accumulator starts.""" + lo, hi = clamp_bounds(bits) + signal, idx, out = init, 0, [] for n in nibbles: signal += delta(n, STEP[idx], variant) - signal = SAMPLE_MIN if signal < SAMPLE_MIN else ( - SAMPLE_MAX if signal > SAMPLE_MAX else signal) + signal = lo if signal < lo else (hi if signal > hi else signal) idx += INDEX_ADJUST[n & 7] idx = 0 if idx < 0 else (48 if idx > 48 else idx) out.append(signal) return out -def encode(samples, variant="shift"): +def encode(samples, variant="shift", init=0, bits=12): """12-bit signed samples -> nibbles. The nibble is chosen by EXHAUSTIVE SEARCH over all sixteen, minimising the @@ -82,37 +124,46 @@ def encode(samples, variant="shift"): nothing offline. The decoder is run INSIDE the loop, so the encoder can never drift away from what the decoder will reconstruct. """ - signal, idx, out = 0, 0, bytearray() + lo, hi = clamp_bounds(bits) + signal, idx, out = init, 0, bytearray() for s in samples: step = STEP[idx] best, best_err = 0, None for n in range(16): v = signal + delta(n, step, variant) - v = SAMPLE_MIN if v < SAMPLE_MIN else (SAMPLE_MAX if v > SAMPLE_MAX else v) + v = lo if v < lo else (hi if v > hi else v) err = (v - s) ** 2 if best_err is None or err < best_err: best, best_err = n, err signal += delta(best, step, variant) - signal = SAMPLE_MIN if signal < SAMPLE_MIN else ( - SAMPLE_MAX if signal > SAMPLE_MAX else signal) + signal = lo if signal < lo else (hi if signal > hi else signal) idx += INDEX_ADJUST[best & 7] idx = 0 if idx < 0 else (48 if idx > 48 else idx) out.append(best) return bytes(out) -def pack(nibbles): - """Nibbles -> bytes, HIGH NIBBLE FIRST. An odd count pads with a 0 nibble, - which is the quietest one the format has (delta = step/8).""" +def pack(nibbles, order="high"): + """Nibbles -> bytes. `order` names which nibble of a byte is played FIRST; + 'high' is the VOX file convention. An odd count pads with a 0 nibble, which + is the quietest one the format has (delta = step/8).""" + if order not in ORDERS: + raise ValueError(f"unknown nibble order {order!r}") n = bytes(nibbles) if len(n) & 1: n += b"\0" - return bytes((n[i] << 4) | n[i + 1] for i in range(0, len(n), 2)) + if order == "high": + return bytes((n[i] << 4) | n[i + 1] for i in range(0, len(n), 2)) + return bytes((n[i + 1] << 4) | n[i] for i in range(0, len(n), 2)) -def unpack(data, count=None): +def unpack(data, count=None, order="high"): + if order not in ORDERS: + raise ValueError(f"unknown nibble order {order!r}") out = bytearray() for b in data: - out.append(b >> 4) - out.append(b & 15) + if order == "high": + out.append(b >> 4); out.append(b & 15) + else: + out.append(b & 15); out.append(b >> 4) return bytes(out[:count] if count is not None else out)