diff --git a/README.md b/README.md index 5160553..4d33504 100644 --- a/README.md +++ b/README.md @@ -510,6 +510,35 @@ group to enter off-boundary and gives back **12,288 B of RAM for +4.36 KB/s of wire**. The deciding number is what one more SCSI command costs, and that is hardware's (FINDINGS 70). +**The player seeks with sound on it now — and the chip's predictor does not seek +with it.** `pg_aseek` rebuilds the four numbers a branch needs — the lump index, +the stream position, the remainder accumulator and the byte offset into the +group — and issues the second read: **132,162 B of spliced stream accounted for +byte by byte in MAME's own capture**, across a branch at frame 37 that is four +frames into lump 3 and so deliberately *not* on a group boundary. 70.3's silence +is gone, and what is left is smaller and is not a player problem. **The +MSM6258's accumulator is a pure integrator with no leakage term**, so a branch +that hands the chip bytes chosen for a state it is not in produces **a DC offset +that does not decay**: playing through, **−355 of 511 with AC 0.00** — the right +shape from the wrong ground — and still **−108 four seconds later**. Stop and +re-PLAY the chip and all 62,500 post-seek samples are *exactly* a decode from the +container's own start state, with the whole error a **single permanent constant, +−65**. A re-PLAY is **5.5x better and neither is zero**. + +**The only fix that reaches zero is the encoder's, and it costs 0.33 dB.** A +player cannot set the chip's accumulator, only reset it — so a branch is free +only if the stream was *encoded* with the predictor reset where the branch +lands. Resetting every frame makes all 119 of the container's branch points +exact for **21.99 → 21.66 dB**, because the OKI step table's floor is a constant +16 and the recursion re-converges in a few samples. Across the container's frame +boundaries the offset a re-PLAY costs averages **55.4 of 511 (10.8%)** and +reaches **313**, and the host's arithmetic for the machine's own branch point +agrees with the capture **to the digit**. The seek also turned up a race that +had passed this gate three times: `pg_ainit` waited on a *read-back* MTC before +telling the chip to PLAY, which can start the scene **one byte in, forever**, +with every counter in the player exact — found by locating the capture's opening +samples in the container image (FINDINGS 71). + **Nothing outside-derived is committed here.** The scene graph is not redistributable from this tree; it is regenerated from a reader's own clones into gitignored `tmp/`, and `tools/import/scenegraph.py` is the single file in @@ -698,6 +727,13 @@ tools/analysis/ measurement scripts, numbered in the order they were written. costs. It reads only DLXSCENE1 and both containers' own headers, and its one design assumption -- one container per SCENE -- is printed in its own output. + 37 prices what a BRANCH costs the chip, which is a codec + property rather than a player one: the accumulator has no leak, + so a re-PLAY at a branch is an exact permanent DC offset and + playing through is a larger one that decays only by clamping. + It gives the census over every frame boundary, checks its own + arithmetic for the machine's branch point against what the + capture measured, and prices the encoder-side fix. buscost.py is the shared bus-cycle table. The per-block constants live in tools/encoder/vq_hybrid.py and are imported, never copied. diff --git a/docs/FINDINGS.md b/docs/FINDINGS.md index 8bd94bd..2ad948d 100644 --- a/docs/FINDINGS.md +++ b/docs/FINDINGS.md @@ -7709,3 +7709,206 @@ uniform would be a quarter out at F=3. At the shipped F=11 the ratio is 6. **The 409/203 split is the graph's, not a container manifest's.** No scene has been encoded but one; the mapping from a scene to a DLXP2 is a design statement this tree has made and not yet built. + +# Findings — session 39 (2026-08-25) + +## 71. The player seeks with sound on it — and the chip's predictor does not seek with it (session 39) + +**ROADMAP P6d**, and it is the item session 38's handoff put first. 70.3 named +exactly what was missing — "`src/player/packed.s` starts `PG_AK`/`PG_AKF` at +lump 0 and has no audio seek path at all" — and priced its absence at a **mean +416.5 ms of silence** over the arcade's 409 within-container seek targets, worst +833.3, with only 36 of the 409 free. This is that path, built and run. + + bash tools/bench/packed_run.sh # runs 1, 5, 6, 7 -- 7 is this + python3 tools/analysis/37_audio_seek.py # the host half + ./tools/bench/check.sh # ALL GREEN + +`src/player/packed.s` (`pg_aseek`, `pg_udiv32`, `PG_SEEKF`, `PG_ARST`, +`PG_APOS`, `PG_ASKIP`, `PG_AMTCA`), `src/player/adpcm.i` (`ad_abort`), +`tools/bench/packed_run.sh` run 7, `tools/bench/verify_packed_audio.py` +(`--seek`, the spliced stream, the split walk), `tools/analysis/37_audio_seek.py`. +The player is **3,954 → 4,340 bytes**. + +### 71.1 The seek path: four numbers and a second read + +The video half is unchanged and is ROADMAP K3's whole claim arriving intact: a +packed record is found by arithmetic from the frame index, so a video seek is +the assignment `d7 = f`. The audio half is not arithmetic, because audio rides a +**cadence** rather than a frame. A DLXP2 group is `lump k, then F records`, so +lump k lies at a *lower* address than every record of its group but the first, +and reading forward from record f picks up lump k+1. + +What a seek to frame f has to rebuild is four things, and three of them are +invisible if they are wrong: + +| | | +|---|---| +| `PG_AK`/`PG_AKF` | the lump index, `k = f/F`, and a **second READ(10)** at `PG_ALBA0 + k*(F*PG_RECS + PG_CADA)` | +| `PG_APOS` | the stream POSITION, `floor(k*F*hz/(2*fps))` — what the tail limiter measures the last lump against | +| `PG_AACC` | the remainder accumulator **at lump k**, which is that same division's remainder (67.2). Wrong, this is not an error, it is a *rate* | +| `PG_ASKIP` | the byte offset of frame f inside lump k, `floor((acc + (f mod F)*hz)/(2*fps))` | + +`PG_APOS` is a **new cell and not a rename**. `PG_ABYT` is what the chip was +handed; `PG_APOS` is where the container has got to, and a seek moves one and +not the other. One cell for both facts gives a last lump that is long by the +skip, and the player's own byte count agrees with it. + +`pg_udiv32` exists for one call. `floor(f*hz/(2*fps))` passes 65,535 after **101 +seconds of scene** at 12 fps and 15,625 Hz, and the 68000's `divu` does not fail +on a quotient that does not fit — it sets V, leaves the destination alone and +carries on. Every other division in this player is a bare `divu` and is right to +be; this one is not. + +**Measured, on the emulated machine, off a real MB89352 volume**: two passes, +the second starting at **frame 37** — four frames into lump 3, chosen *not* to +be a multiple of F, because a target on a group boundary needs no offset and 36 +of the arcade's 409 do land on one. 203 frames, 19 lumps armed and fetched, and +the spliced stream of **132,162 B accounted for byte by byte in MAME's own +capture**, in both chip configurations. The skip the player computed is **2,604 B** +and the cadence says 2,604. + +### 71.2 Two bugs, both silent, both caught by the capture and not by a counter + +**The first was mine and took four minutes.** `pg_udiv32` uses `d4` as a work +register and `pg_aseek` was still holding `hz` in it, so the offset came out +**1 byte where the cadence says 2,604**. That is not an error and it is not +silence: it is **166 ms of the wrong part of the scene, played at exactly the +right rate**, with every counter in the player agreeing with it. + +**The second was already in the tree and had passed this gate three times.** +`pg_ainit` arms channel 3, then reads MTC back, then waits for that read-back +value to change before telling the chip to PLAY — the point being that a byte +must be in the data register first. That is the same test as "a byte has left +RAM" **only if no byte leaves between the channel start and the read**. One +does, whenever `#DRQ3`'s phase falls that way: the read-back already says 7,160, +the loop waits for 7,159, and the PLAY lands after the channel has fetched byte +1 **on top of byte 0**. The chip then plays the scene from one byte in, forever. + +Nothing in the player can see it. Every lump armed, every byte delivered, every +counter exact, and the sound is the whole stream shifted by one byte — 67.4's +failure mode landing on the one byte of a scene that a player cannot check. It +was found by taking the capture's opening samples and **locating them in the +container image**: sector 1 + 1. The fix is one cell: the witness is now the +count that was *written* (`PG_AMTCA`), not one read back. + +**It is a race, so it did not reproduce on demand.** Sessions 36–38 passed with +it, because the arm happened to land in the other half of a 128 µs window; 386 +bytes of new code moved the phase. What makes it a bug rather than bad luck is +that the witness was a read. + +### 71.3 THE FINDING: the predictor does not seek, and the error is DC + +Every byte can arrive, in order, exactly — and the samples still be wrong. The +MSM6258's accumulator is a **pure integrator of deltas with no leakage term**, +so a branch that hands the chip bytes chosen for a state it is not in does not +produce a transient with a time constant. It produces **a DC offset that does +not decay**. + +`verify_packed_audio.py` measures it against the state the *encoder* assumed — +`adpcm.encode` runs its decoder inside its own search loop, so the state it +chose byte B(f)'s nibbles for is the state a continuous play reaches at B(f). +Both designs, same branch, same bytes: + +**A. The chip PLAYS THROUGH the branch** (accumulator −318, step index 47 — +where 1,054 ms of replayed byte had driven it): + +| window | DC | AC | +|---|---:|---:| +| 0–6 ms | −269.3 | 158.49 | +| 6–64 ms | −355.9 | 10.53 | +| **64–320 ms** | **−355.0** | **0.00** | +| 0.3–1.0 s | −274.3 | 45.21 | +| 1–2 s | −212.1 | 39.21 | +| 2–4 s | **−108.1** | 20.21 | + +**AC 0.00**: the chip is decoding the right *shape* from the wrong *ground*. And +four seconds later the ground is still 21% of full scale out. What decay there +is comes from the signal's own clamping, not from the recursion forgetting. + +**B. The chip is STOPPED and re-PLAYED** (`PG_ARST=1`; PLAY sets the accumulator +to the container's `init` and the step index to 0). All **62,500 post-seek +samples are EXACTLY a decode from `init` and step index 0** — not close, every +sample — and the whole error against the encoder's intent is **the single +constant −65**, which is `init − acc(target)` and nothing else. 12.7% of the +10-bit clamp's headroom, permanent, inaudible as a tone, and one click at the +branch. + +**So a re-PLAY is 5.5x better and neither is zero.** −355 against −65. This is +now a design decision with a number under it rather than an implementation +detail, and `PG_ARST` is the mailbox that carries it. + +**The host agrees with the machine to the digit.** `37_audio_seek.py` computes +`init − acc(24,088)` from the container's own bytes and gets **−65**; MAME's +capture measured the chip at −65, constant over 62,500 samples. One number, two +layers, and the gate asserts the equality rather than printing both. + +**The census, over all 119 frame boundaries of the container** — every point a +branch in this design can land on, because 56.3's targets are frame indices: + +| | | +|---|---:| +| mean \|DC\| a re-PLAY costs | **55.4 of 511 (10.8%)** | +| median / p90 / worst | 41 / 126 / **313 (61.3%)** | +| step index the encoder assumed there | 0..26 of 48, median 12 | + +A re-PLAY sets the step index to 0 as well, so **a branch into a loud passage +gets the offset *and* a step index that has to climb back**. The offset is the +signal's own value at the cut, so it scales with the programme — and the disc +peaks at 946 of 2048 (69.2). + +### 71.4 The only fix that makes a branch free is the ENCODER'S, and it is cheap + +A player cannot set the chip's accumulator; it can only reset it. So the way to +make a branch cost nothing is to encode the stream with the predictor **reset at +every point a branch can land on**, and then re-PLAY there — the two states are +then the same state and the error is zero, not small. + +Priced on the same ten seconds every audio figure in this tree is quoted on: + +| reset every | SNR | vs shipped | branch points made free | +|---|---:|---:|---:| +| never (shipped) | 21.99 dB | — | 0 of 119 | +| 11 frames (the cadence) | 21.98 dB | −0.01 | 10 of 119 | +| **1 frame** | **21.66 dB** | **−0.33** | **119 of 119** | + +**Making every frame in the container an exact branch target costs 0.33 dB.** +The reason it is not twelve times anything is that the OKI step table's floor is +a constant 16 and the recursion re-converges in a few samples — the same +property FINDINGS 69 found working the other way, when attenuating walked the +programme down toward a quantiser that stays put. + +That is a container change (a DLXP3) and **it is not in `tools/encoder`**. It +costs bytes nothing and SNR 0.33 dB, and what decides it is whether a branch is +allowed to land anywhere or only on frames the encoder was told about — which +the scene graph already knows (56.3). + +### 71.5 What this does NOT establish + +1. **The seek's own time cost is not separable under this apparatus.** The seam + at the branch measures **1,054 ms**, and that is the *transport running 8.9% + slow* (68): 120 frames at the measured 10.85 fps is 11.06 s of picture + against 10.00 s of audio, so the chip had run out before the branch arrived. + 70.3's arithmetic — 7,168 B is 11.7 ms at 600 KB/s against a mean 416 ms of + silence — stands and is still arithmetic. **The extra SCSI command's overhead + is B1's and is still unmeasured.** +2. **The chip's branch behaviour is MAME's `okim6258`**: PLAY sets the + accumulator to −2, the step index to 0 and the nibble select to 0, *only if + it was not already playing*. That is the model FINDINGS 66 fitted to the + machine, not a measurement of an MSM6258V, and it joins session 34's fifth + hardware item. **`ad_abort` is in the player for the silicon and this rig + cannot tell**: MAME's `hd63450` restarts a channel whatever it was doing, + and the MC68450 calls STR-while-active an operation timing error. +3. **`#DRQ3` ticking while the chip is STOPPED is a rig fact.** On this machine + it is the driver's own divider (`x68k.cpp`'s `adpcm_drq_tick`, 29.6), so the + channel keeps fetching and the re-PLAY's MTC wait terminates. A real + MSM6258V that stops requesting when stopped would leave that wait spinning; + the bounded count is what turns it into a report rather than a hang. +4. **One container, one branch point on the machine.** The census is host + arithmetic over ten seconds of one −13.4 dBFS passage. The 409/203 split is + still the graph's, and one container per *sequence* rather than per scene + makes every seek land on frame 0 and empties 70.3, 70.4 and this. +5. **The cadence question 70.4 reopened is now answerable and is not answered + here.** With a working seek path the silence F=11 costs is zero and the trade + is back to padding against RAM, where 32_audio_wire.py left it — plus one + SCSI command per branch, which is B1's. diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index 1748510..af0cd6a 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -194,6 +194,25 @@ alternative is F=1, which gives back 12,288 B of RAM for +4.36 KB/s of wire. one extra SCSI command costs. **What is left of P6 is nothing** — 69.4's two unbuilt cases and 70.3's seek path are player work, not open questions. +Amended end of session 39: **P6d IS DONE — THE PLAYER SEEKS WITH SOUND ON IT — +AND WHAT IT UNCOVERED IS AN ENCODER ITEM (FINDINGS 71).** `pg_aseek` rebuilds +the lump index, the stream position, the remainder accumulator and the byte +offset into the group, issues the second READ(10) 70.3 asked for, and re-arms +channel 3 part way into the buffer: **132,162 B of spliced stream accounted for +byte by byte in MAME's own capture**, across a branch at frame 37 that is +deliberately *not* on a group boundary, in both chip configurations. 70.3's +416.5 ms of silence is gone. **What replaces it is smaller and is not a player +problem: the MSM6258's accumulator is an integrator with no leak, so a branch is +a DC offset that does not decay** — −355 of 511 playing through (still −108 four +seconds later, AC 0.00), an exact permanent −65 if the chip is STOPped and +re-PLAYed. **A re-PLAY is 5.5x better and neither is zero.** The only fix that +reaches zero is the ENCODER's: reset the predictor where a branch can land, and +**resetting every frame makes all 119 of the container's branch points exact for +0.33 dB**. That is a DLXP3 and it is now the cheapest open item in the tree. The +seek also found a race that had passed this gate three times — `pg_ainit` waited +on a *read-back* MTC and could start the scene one byte in, silently — which is +71.2 and is fixed. + **THE COMPLETION TARGET IS M3, THE VERTICAL SLICE** (USER DECISION): one scene tree — a decision point, two outcomes, a death clip — with audio, streaming from a real SCSI volume on a stock 2 MB machine, playable. That is the point at which diff --git a/docs/STATUS.md b/docs/STATUS.md index 0ee32ec..594cda4 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -1,3 +1,125 @@ +# Status & next-session handoff — end of session 39 (2026-08-25) + +## Session 39: the player seeks with sound on it, and the chip's predictor does not seek with it + +**FINDINGS 71.** ROADMAP P6d, and it is the item session 38's handoff put first. +70.3 named exactly what was missing — "`src/player/packed.s` starts +`PG_AK`/`PG_AKF` at lump 0 and has no audio seek path at all" — and priced its +absence at a **mean 416.5 ms of silence** over the arcade's 409 +within-container seek targets. `pg_aseek` is that path: it rebuilds the lump +index, the stream position, the remainder accumulator and the byte offset into +the group, issues the **second READ(10)** at the lump's own LBA, and re-arms +channel 3 part way into the buffer. + +**Measured on the machine, off a real volume**: two passes over the container, +the second starting at **frame 37** — four frames into lump 3, chosen *not* to +be a multiple of F, because a target on a group boundary needs no offset and 36 +of the arcade's 409 land on one. **132,162 B of spliced stream accounted for +byte by byte in MAME's own capture**, in both chip configurations, and the skip +the 68000 computed is **2,604 B** against the cadence's 2,604. + +**THE HEADLINE, and it is a property of the codec rather than of the player: the +predictor does not seek.** The MSM6258's accumulator is a pure integrator with +**no leakage term**, so a branch that hands the chip bytes chosen for a state it +is not in produces **a DC offset that does not decay** — not a transient with a +time constant. Playing *through* the branch: DC **−355** with **AC 0.00** (the +right shape from the wrong ground), still **−108** four seconds later. Stopping +and re-PLAYing the chip: all **62,500 post-seek samples are exactly a decode +from the container's own `init`**, and the whole error is **the single constant +−65**, 12.7% of the 10-bit clamp's headroom, permanent. **A re-PLAY is 5.5x +better and neither is zero**, so `PG_ARST` is a mailbox with a number under it +rather than a preference. + +**And the host agrees with the machine to the digit.** +`tools/analysis/37_audio_seek.py` computes `init − acc(24,088)` out of the +container's own bytes and gets **−65**; the capture measured −65. The gate +asserts the equality. Its census over all 119 frame boundaries: mean **55.4 of +511 (10.8%)**, median 41, p90 126, worst **313 (61.3%)**. + +**THE FIX IS THE ENCODER'S AND IT IS CHEAP.** A player cannot set the chip's +accumulator, only reset it — so the way to make a branch free is to encode with +the predictor reset where a branch can land, and re-PLAY there. **Resetting +every frame makes all 119 branch points exact for 0.33 dB** (21.99 → 21.66), +because the step table's floor is a constant 16 and the recursion re-converges +in a few samples. It is a container change (a DLXP3) and it is deliberately +**not** in `tools/encoder`. + +**TWO SILENT BUGS, BOTH CAUGHT BY THE CAPTURE AND NOT BY A COUNTER.** Mine: +`pg_udiv32` trashes `d4` and `pg_aseek` was holding `hz` in it, so the offset +came out **1 byte instead of 2,604** — 166 ms of the wrong part of the scene at +exactly the right rate, every counter agreeing. **And one that was already in +the tree and had passed this gate three times**: `pg_ainit` waited for a +*read-back* MTC to change before PLAY, which is the same test as "a byte has +left RAM" only if no byte leaves between the channel start and the read. One +does whenever `#DRQ3`'s phase falls that way, and the chip then plays the scene +**one byte in, forever**. Found by locating the capture's opening samples in the +container image: sector 1 + 1. The witness is now the count that was *written*. +It is a race — 386 bytes of new code moved the phase — and what makes it a bug +rather than bad luck is that the witness was a read. + +**What changed in the tree.** `src/player/packed.s` 3,954 → 4,340 B (`pg_aseek`, +`pg_udiv32`, `PG_SEEKF`, `PG_ARST`, `PG_APOS`, `PG_ASKIP`, `PG_AMTCA`, and the +pace gate now counts TICKS rather than frame indices, because after a seek they +are different numbers); `ad_abort` in `adpcm.i`; run 7 of `packed_run.sh` (both +chip configurations, ~20 s); `verify_packed_audio.py` grew `--seek`, the spliced +expected stream, and a walk that SPLITS at a chip reset; one new analysis file +and two new check.sh stages. + +## HANDOFF — start here + +**THE TREE IS ALL GREEN**, session 39's two stages included. + +### The work, in the order it should be done + +**1. THE PREDICTOR-RESET CONTAINER (71.4), and it is now the cheapest thing that +removes a whole class of error.** Reset the encoder's predictor at every frame +(0.33 dB) or at the frames the scene graph names as targets (free at those and +unchanged elsewhere), carry the choice in the DLXP header the way the four ADPCM +axes are carried (67.3), and have `pg_aseek` re-PLAY. The player half already +exists and is `PG_ARST`. This is the only path to a branch that costs the audio +nothing, and it is entirely inside this project. + +**2. THE BUFFERING DEPTH, still cheap and now three sessions old.** `PG_ANBUF` +is 3 and `PG_APRE` is a mailbox. Two slots is one constant and one run. 70.2's +reason to care is unchanged: the audio buffer is the packed branch's *only* +buffer. + +**3. THE SILENT-CLIP AND SHORT-AUDIO CASES (69.4).** Unchanged: a scene whose +audio is shorter than its frames (`00199`), and a scene with no audio track at +all (`00176`). Both are decided by what the lump count is computed from. + +**4. THE CADENCE PICK (70.4) is answerable now and was not before.** With a +working seek path the silence F=11 costs is zero, so the trade goes back to +padding against RAM — plus one SCSI command per branch, which is B1's. + +### What is still BLOCKED, so it is not picked up by mistake + +**K4 — the packed player that is on screen — is conditional on B2**, a board +question. **E7, E4 and C1** are parked (61.8), and **P4a's wiring** is parked +with the ring K3 deleted. + +**The hardware list is the user's** and session 39 adds nothing to it but does +sharpen the fifth. B1 (sustained AND data-phase BURST rate, plus session 38's +sixth: what one extra SCSI command costs), B2 (blanking), B3 (`#EXREQ`), B4 (a +byte write to a palette register), and session 34's fifth — play a known nibble +stream on a real MSM6258V. **Session 39 gives that fifth two more questions and +both are small**: does a real MSM6258V reset its accumulator, step index and +nibble select on PLAY *only when it was not already playing*, the way MAME's +model does; and does it go on asserting `#DRQ` while STOPped. The second decides +whether `pg_aseek`'s re-PLAY wait terminates on silicon; the bounded count is +what turns a `no` into a report rather than a hang. + +### Reproducing this session + + bash tools/bench/packed_run.sh # run 7 is the seek, ~20 s + python3 tools/analysis/37_audio_seek.py # the host half, ~30 s + ./tools/bench/check.sh # ALL GREEN + +**WHAT IS NEXT.** The predictor-reset container — the first thing in a while +that makes a measured cost go to zero rather than down. + +--- + # Status & next-session handoff — end of session 38 (2026-08-25) ## Session 38: the climb meets a branch point with sound on it, and the cadence has a third column diff --git a/src/player/adpcm.i b/src/player/adpcm.i index 5239e6d..dc00f45 100644 --- a/src/player/adpcm.i +++ b/src/player/adpcm.i @@ -106,3 +106,26 @@ ad_play: ad_halt: move.b #AD_STOP,AD_CTRLR rts + +; --------------------------------------------------------------- ad_abort +; STOP channel 3 where it stands, and clear what stopping it posts. +; +; A SEEK IS THE ONLY THING IN THIS PLAYER THAT NEEDS THIS. Every other re-arm +; happens at COC, where the channel has already counted itself out and there is +; nothing to stop; a seek arrives MID-LUMP, because a branch is a frame index +; and a frame does not know about the cadence (FINDINGS 70.3). +; +; SAB is CCR bit 4. CSR is then written $FF -- write-one-to-clear -- to take +; down COC and ERR together, because the abort posts a channel error (CER $11) +; and pg_aserv's whole test is "did the channel count out": a stale COC would +; make the very next service call arm a lump that is already playing. +; +; NAME THE LAYER. MAME's hd63450 does not NEED the abort -- its +; dma_transfer_start reloads MAR/MTC and restarts the timer whatever the channel +; was doing. The MC68450 does: STR written to an active channel is an operation +; timing error, and the transfer that is running is not the one that was asked +; for. The abort is here for the silicon, and the run below cannot tell. +ad_abort: + move.b #$10,A3_CCR ; SAB -- software abort + move.b #$FF,A3_CSR ; ...and the COC/ERR it posts + rts diff --git a/src/player/packed.s b/src/player/packed.s index a6f9bc7..f9eefe3 100644 --- a/src/player/packed.s +++ b/src/player/packed.s @@ -116,6 +116,26 @@ PG_APRE = $18974 ; lumps fetched before frame 0. A NUMBER and ; not a constant: what the audio needs in hand ; is set by how far the transport runs behind ; real time, which is a property of the medium. +PG_ARST = $1897C ; 1 = STOP and re-PLAY the chip at a branch, + ; 0 = leave it playing and re-arm underneath it. + ; A DESIGN CHOICE AND NOT A TIDINESS ONE: the + ; MSM6258's accumulator is a pure integrator + ; with no leak, so whatever state mismatch a + ; branch creates is a DC offset that DOES NOT + ; DECAY. PLAY resets the accumulator to the + ; container's own `init` and the step index to + ; 0, which is the state the encoder's first byte + ; was chosen for -- but not the state it chose + ; byte B(f) for. Neither is zero and the two + ; are 5.5x apart; FINDINGS 71.3 is the run. +PG_SEEKF = $18978 ; THE BRANCH POINT. Passes after the first + ; start HERE rather than at frame 0, which is + ; what makes PG_ITER a SEEK rather than a + ; replay. On the video path that is the whole + ; of it -- a packed record is found by + ; arithmetic from the frame index, so seeking is + ; an assignment (ROADMAP K3). On the audio path + ; it is pg_aseek, and FINDINGS 70.3 is why. ; ---- outputs PG_SHOWN = $18930 ; frames displayed. Bumped AFTER bit 11 is @@ -178,6 +198,40 @@ PG_ABUSY = $189BC ; 1 = a lump fetch is in flight; the hook must ; not re-enter the transport it is running under PG_ASLOTA = $189C0 ; slot the next ARM reads from PG_ASLOTF = $189C4 ; slot the next FETCH writes to +; ---- and the four cells the SEEK adds (FINDINGS 70.3, 71). +PG_APOS = $189C8 ; the STREAM POSITION in bytes -- what pg_apay + ; measures the last lump against. It is NOT + ; PG_ABYT and the difference is the seek: a + ; seek moves the position without handing the + ; chip anything, and a seek into the middle of a + ; lump advances the position by the whole lump + ; while the chip is fed only the tail of it. + ; One cell for two facts is how a counter that + ; is right becomes a stream that is wrong. +PG_ASKIP = $189CC ; bytes to drop from the head of the NEXT arm, + ; because a branch lands `f mod F` frames into + ; its group. Consumed by the arm that honours + ; it -- a skip left set is a second lump + ; entered part way through for no reason. +PG_ASKN = $189D0 ; audio seeks performed +PG_ASKB = $189D4 ; ...and the bytes they skipped, summed. The + ; only witness in the player that the offset + ; arithmetic did anything: a seek path that + ; always landed on a group boundary would be + ; indistinguishable from one that had no offset + ; term at all, and 36 of the arcade's 409 + ; targets DO land on one (70.3). +PG_ASKA = $189D8 ; 1 = the next arm is a SEEK arm, so the MTC it + ; finds left over is not a dropped byte and must + ; not be charged to PG_ALATE +PG_AMTCA = $189E0 ; the MTC the channel was last ARMED with. A + ; WITNESS THAT WAS WRITTEN RATHER THAN READ -- + ; see pg_ainit, and it is worth a cell because + ; the read-back version of it silently ate byte + ; 0 of the scene. +PG_TSEQ = $189DC ; frames played in the RUN, which after a seek + ; is not the frame index. The pace gate counts + ; ticks and a tick is a slot, not a picture. ; ---- the lump buffers. 65.3 charged 14,336 B for double-buffering the cadence ; and FINDINGS 67.6 recorded that no player held it; this is where it is held. @@ -286,6 +340,13 @@ start: clr.l PG_ABUSY.l clr.l PG_ASLOTA.l clr.l PG_ASLOTF.l + clr.l PG_APOS.l + clr.l PG_ASKIP.l + clr.l PG_ASKN.l + clr.l PG_ASKB.l + clr.l PG_ASKA.l + clr.l PG_TSEQ.l + clr.l PG_AMTCA.l ; ---- 1. the display. THE PLAYER OWNS IT, and that is a change from every ; other rig in this tree: tools/bench/stream.lua and decode.lua call @@ -340,8 +401,8 @@ start: bne.s .it moveq #1,d6 .it: + moveq #0,d7 ; the FIRST pass starts at frame 0 pg_pass: - moveq #0,d7 ; d7 = frame index within the pass pg_loop: bsr pg_gate ; wait for tick d7 ; THE AUDIO IS SERVICED THREE TIMES A FRAME AND THAT IS NOT BELT AND BRACES. @@ -359,18 +420,31 @@ pg_loop: bsr pg_afill ; ...and fetch at most ONE lump, so the move.l (sp)+,d0 ; audio's share of the frame is bounded tst.l d0 - bmi.s pg_failed + bmi pg_failed addq.l #1,d7 + addq.l #1,PG_TSEQ.l ; ...and a TICK, which after a seek is + ; not the same number as the frame move.l PG_NFR.l,d0 cmp.l d0,d7 - bcs.s pg_loop -; ---- a pass boundary. For this container a SEEK IS ARITHMETIC: there is no -; ring to discard, no index to walk and no prefill to climb, so the next pass's -; first record is simply LBA0 again. That is the whole of what K3 deletes, and -; it is worth one line of code and four of comment because it is the largest -; simplification in the project (ROADMAP K3) and it does not look like one. + bcs pg_loop +; ---- A BRANCH POINT. On the video path a seek IS ARITHMETIC and that is the +; largest simplification in the project (ROADMAP K3): there is no ring to +; discard, no index to walk and no prefill to climb, so the next pass's first +; record is LBA0 + f*97 + (f//F)*A and the seek is the assignment below. +; +; THE AUDIO IS NOT ARITHMETIC AND THAT IS FINDINGS 70.3. A DLXP2 group is +; `lump k, then F records`, so lump k lies at a LOWER address than every record +; of its group but the first: read forward from record f and the next lump to +; arrive is k+1, which carries frame (k+1)*F, and the frames from f to +; (k+1)*F-1 have picture and no sound. On the arcade's own 409 within-container +; seek targets that is a mean of 416.5 ms and a worst of 833.3. pg_aseek is the +; second read that removes it. subq.l #1,d6 - bne.s pg_pass + beq.s .fin + move.l PG_SEEKF.l,d7 ; the video seek, entire + bsr pg_aseek ; ...and the audio, which is not + bra pg_pass +.fin: move.l CLK_VDISP.l,PG_VDISP.l bsr clk_stop @@ -392,27 +466,36 @@ pg_hold: ; tick ALREADY past did not idle for a single poll, which means the previous ; frame used its whole slot and then some -- that is the underrun this player ; can have, and it is counted rather than absorbed. +; AND IT COUNTS TICKS, NOT FRAMES. Until session 39 the comparison was against +; d7 directly, which is the same number only while the run plays frame 0 first +; and never goes back. A seek makes them different: pass 2 restarts at frame f +; while the clock is at tick n, and a gate that compared f against n would call +; every frame after a backwards branch LATE and every frame after a forwards one +; free. PG_TSEQ is the slot count and a slot is a slot whatever picture is in +; it. pg_gate: movem.l d0-d1,-(sp) tst.l PG_PACEON.l beq.s .out + move.l PG_TSEQ.l,d1 move.l CLK_PACE.l,d0 - cmp.l d7,d0 - bcs.s .wait ; PACE < d7: early, the common case - tst.l d7 - beq.s .out ; frame 0 starts AT tick 0 by definition + cmp.l d1,d0 + bcs.s .wait ; PACE < tick: early, the common case + tst.l d1 + beq.s .out ; tick 0 starts AT tick 0 by definition tst.l PG_LATE.l bne.s .nf - move.l d7,PG_LATE1.l + move.l d7,PG_LATE1.l ; the FRAME, because that is what a + ; reader can look at in the container .nf: addq.l #1,PG_LATE.l - sub.l d7,d0 ; whole ticks overrun + sub.l d1,d0 ; whole ticks overrun cmp.l PG_LATEM.l,d0 bls.s .out move.l d0,PG_LATEM.l bra.s .out .wait: addq.l #1,PG_GSPIN.l move.l CLK_PACE.l,d0 - cmp.l d7,d0 + cmp.l d1,d0 bcs.s .wait .out: movem.l (sp)+,d0-d1 rts @@ -725,11 +808,26 @@ pg_ainit: ; ---- arm lump 0, then wait for the channel to actually take byte 0 before ; PLAY. Not a delay loop: the condition is MTC having moved, which is the ; channel's own account of a byte having left RAM. +; +; AND THE VALUE IT IS COMPARED AGAINST IS THE ONE THAT WAS WRITTEN, NOT ONE READ +; BACK. Until session 39 this read MTC after the arm and waited for it to +; differ from THAT -- which is the same test only if no byte leaves RAM between +; the channel start and the read. One does, whenever #DRQ3's phase falls that +; way: the read-back then already says 7,160, the loop waits for 7,159, and the +; PLAY lands after the channel has fetched byte 1 ON TOP of byte 0. The chip +; starts the scene one byte in. +; +; NOTHING IN THE PLAYER CAN SEE THAT. Every lump is armed, every byte is +; delivered, the counters are exact and the sound is the whole stream shifted by +; one byte -- 67.4's failure mode, on the one byte in the scene that a player +; cannot check. It was found by locating the capture's opening samples in the +; container image: sector 1 + 1. And it is a RACE, so it does not reproduce on +; demand -- the tree passed this gate for three sessions with the bug in it, +; because the arm happened to land in the other half of a 128 us window. What +; makes it a bug rather than bad luck is that the witness was a read. bsr pg_aarm move.l #4000000,d3 - move.w A3_MTC,d0 - andi.l #$FFFF,d0 - move.l d0,d2 ; d2 = MTC as armed + move.l PG_AMTCA.l,d2 ; d2 = MTC as the channel was ARMED .first: move.w A3_MTC,d0 andi.l #$FFFF,d0 cmp.l d2,d0 @@ -785,6 +883,12 @@ pg_aserv: ; Arm channel 3 on the lump in slot PG_ASLOTA for its PAYLOAD length, and ; advance. Trashes d0-d2/a1, which pg_aserv and pg_ainit have saved. pg_aarm: + tst.l PG_ASKA.l ; a SEEK arm. The previous lump was + beq.s .prev ; abandoned on purpose and whatever MTC + clr.l PG_ASKA.l ; it has left is not a dropped byte -- + bra.s .clean ; charging it to PG_ALATE would turn the +.prev: ; assertion that this player never drops + ; a byte into a count of its own seeks tst.l PG_AARM.l beq.s .clean ; the FIRST arm has no previous lump and ; MTC out of reset is not ours to read @@ -796,13 +900,37 @@ pg_aarm: bsr pg_apay ; d0 = this lump's payload tst.l d0 beq.s .skip - move.l d0,d1 +; ---- THE STREAM ADVANCES BY THE WHOLE LUMP AND THE CHIP MAY NOT. PG_APOS is +; where the container's stream has got to, which is what pg_apay's tail limiter +; measures against; PG_ABYT is what the chip was handed, which is what the +; capture has to account for. They are equal until a seek lands part way into a +; group, and keeping one cell for both is exactly the kind of bookkeeping this +; format punishes silently: the last lump would come out long by the skip and +; the player's own byte count would agree with it. + add.l d0,PG_APOS.l + move.l PG_ASKIP.l,d1 + beq.s .nosk + clr.l PG_ASKIP.l ; consumed HERE and nowhere else + cmp.l d0,d1 + bcc.s .skip ; the offset is inside the lump by + ; construction (f mod F < F). Refusing + ; rather than trusting is what makes + ; that a statement: MTC 0 is a 65,536 + ; byte transfer on this part, not a + ; no-op, so an offset past the end would + ; play 64 KB of whatever follows. + sub.l d1,d0 +.nosk: add.l d0,PG_ABYT.l + move.l d0,d2 ; d2 = the MTC, while d0 builds the MAR move.l PG_ASLOTA.l,d0 lsl.l #8,d0 lsl.l #6,d0 ; slot * $4000 add.l #PG_ABUF,d0 + add.l d1,d0 ; ...+ the seek's offset into the lump movea.l d0,a1 + move.l d2,d1 + move.l d1,PG_AMTCA.l ; ...and what we asked for, recorded bsr ad_arm ; d1 = MTC, a1 = MAR addq.l #1,PG_AARM.l .skip: @@ -846,8 +974,8 @@ pg_apay: ; ---- and the LAST lump is short rather than padded: the stream ends where it ; ends and the geometry would otherwise hand the chip whatever followed it. move.l PG_ABYTES.l,d1 - sub.l PG_ABYT.l,d1 - bcc.s .lim + sub.l PG_APOS.l,d1 ; POSITION, not bytes fed -- a seek + bcc.s .lim ; moves one and not the other moveq #0,d1 .lim: cmp.l d1,d0 bls.s .out @@ -855,6 +983,190 @@ pg_apay: .out: movem.l (sp)+,d1-d2 rts +; ---------------------------------------------------------------- pg_aseek +; TAKE THE AUDIO TO FRAME d7. FINDINGS 70.3, and this is the whole of what that +; finding asked for. +; +; WHY THE VIDEO PATH NEEDS NOTHING AND THE AUDIO PATH NEEDS ALL OF IT. A packed +; record is found by arithmetic from the frame index, so a video seek is the +; assignment `d7 = f` -- ROADMAP K3's large simplification, and it survives +; sound. Audio does not ride the frame; it rides a CADENCE. Lump k sits at a +; LOWER address than every record of its group but the first, so a player that +; carries on reading forward from record f picks up lump k+1, which carries +; frame (k+1)*F: the frames from f to (k+1)*F-1 have picture and no sound. +; Measured on the arcade's own 409 within-container seek targets at the shipped +; F=11 that is a mean of 416.5 ms, a worst of 833.3, and only 36 of the 409 land +; on a boundary and cost nothing. The fix is a SECOND READ -- lump k is up to +; 503,808 B behind record f, so it cannot be picked up by reading early. +; +; THE STATE A SEEK HAS TO REBUILD IS FOUR NUMBERS AND NOT ONE: +; +; PG_AK / PG_AKF the lump index k = f/F, and the fetch that follows it +; PG_APOS the stream POSITION, floor(k*F*hz/(2*fps)), which is what +; pg_apay's tail limiter measures the last lump against +; PG_AACC the remainder accumulator AT lump k -- the same division's +; remainder. Seeding it wrong does not fail: it is a RATE, +; and 67.2 is the finding about what a rate error costs +; PG_ASKIP the byte offset of frame f inside lump k, because f is not +; a multiple of F +; +; AND ONE IT CANNOT REBUILD: THE CHIP'S PREDICTOR. An MSM6258 has no seek. Its +; accumulator and step index are the product of every nibble it has decoded +; since PLAY, and a seek hands it bytes the encoder chose for a state it is not +; in. That is not silence and it is not a wrong byte, so NOTHING IN THIS PLAYER +; CAN SEE IT -- every counter below stays right. It is measured in the only +; place it exists, which is the capture (tools/bench/verify_packed_audio.py +; --seek), and priced in FINDINGS 71. +; +; ORDER: FETCH, THEN ABORT, THEN ARM. The other order is one line shorter and +; it is 90 ms of held byte: the new lump is a READ(10) like any other and the +; chip has no starvation state, so aborting first means the data register holds +; one byte for the whole of the read. Fetching first leaves the OLD lump +; playing across the branch -- the wrong content for a fraction of a slot, which +; is a cut and not a stall. For the same reason the fetch goes into the slot +; the ARM will read from and not into PG_ASLOTF: that slot is by construction +; not the one channel 3 is reading out of, and writing into the one that is, is +; the bug session 36 shipped (pg_afill). +pg_aseek: + movem.l d0-d5/a1,-(sp) + tst.l PG_AUDON.l + beq .out + move.l PG_CADF.l,d1 + beq .out ; a silent container has no cadence to + ; enter off-boundary, which is F=1's + ; whole argument one limit further on + ; (70.4) + addq.l #1,PG_ASKN.l + +; ---- k = f/F, and r = f mod F, the frames into the group + move.l d7,d0 + divu d1,d0 + move.l d0,d2 + clr.w d2 + swap d2 ; d2 = r (divu leaves the remainder in + andi.l #$FFFF,d0 ; the high half -- the same shape + move.l d0,d5 ; pg_frame's third term reads) + +; ---- the stream position at the START of lump k, and the accumulator with it. +; A 32-BIT QUOTIENT, and pg_udiv32 exists for this one call: at 12 fps and +; 15,625 Hz floor(f*hz/(2*fps)) passes 65,535 after 101 seconds of scene, and +; the 68000's `divu` does not FAIL on a quotient that does not fit -- it sets V, +; leaves the destination alone and carries on. A seek past 101 s would have +; returned the position of whatever the previous divide left in d0. + mulu d1,d0 ; k*F -- the frames before the group + move.l PG_AHZ.l,d4 + mulu d4,d0 ; ...times hz. 16x16 -> 32, and k*F is + ; a frame index: this design is one + ; container per SCENE and 65,535 frames + ; is 91 minutes at 12 fps, so the bound + ; on the multiply is the game + move.l PG_AFPS.l,d1 + add.l d1,d1 ; 2*fps -- two samples to a byte + bsr pg_udiv32 ; d0 = position, d3 = remainder + move.l d0,PG_APOS.l + move.l d3,PG_AACC.l + +; ---- and the byte OFFSET of frame f inside lump k. Small on purpose, so the +; 32-bit divide is needed once and not twice: acc < 2*fps and r < F, so +; (acc + r*hz) is at most F*hz + 2*fps - 1 = the same 171,898 pg_apay bounds. + move.l PG_AHZ.l,d4 ; RELOADED, because pg_udiv32 uses d4 as + ; its own work register and leaves the + ; remainder sitting in it. Carrying the + ; stale value gave skip = 1 where the + ; cadence says 2,604 -- which is not an + ; error and is not silence: it is 166 ms + ; of the wrong part of the scene, played + ; at exactly the right rate, with every + ; counter in this player agreeing with + ; it. The CAPTURE is what caught it, + ; which is 67.4's failure mode arriving + ; one more time. + move.l d2,d0 + mulu d4,d0 ; r*hz + add.l d3,d0 ; + the accumulator AT k, which is what + divu d1,d0 ; makes this the exact byte and not a + andi.l #$FFFF,d0 ; rounded one + move.l d0,PG_ASKIP.l + add.l d0,PG_ASKB.l + +; ---- the second read. The old lump plays through it. + move.l d5,PG_AK.l + move.l d5,PG_AKF.l + move.l PG_ASLOTA.l,PG_ASLOTF.l + bsr pg_afetch + tst.l PG_AFERR.l + bne.s .out ; a failed lump fetch leaves the picture + ; perfect and the sound gone. Left + ; unarmed, pg_aserv finds the channel + ; counted out with nothing behind it and + ; counts a STARVE, which is the honest + ; report and is why the fetch has an + ; error word of its own + +; ---- and only now stop the channel, so the seam is the arm and not the read. + tst.l PG_ARST.l + beq.s .noreset + bsr ad_halt ; ...and the CHIP with it, if this run +.noreset: ; is the one that resets the predictor + bsr ad_abort + move.l #1,PG_ASKA.l + bsr pg_aarm + +; ---- THE RE-PLAY, and its order is pg_ainit's for pg_ainit's reason. PLAY +; resets the accumulator, the step index AND the nibble select, and the chip +; begins decoding immediately out of whatever its data register holds -- so the +; channel goes first and the CPU waits for the count it WROTE to move before the +; PLAY. Get that wrong and the branch starts one byte in, which is the bug +; session 39 found in pg_ainit and it does not announce itself. +; +; THE CHANNEL DOES NOT STALL WHILE THE CHIP IS STOPPED: on this machine #DRQ3 +; is the driver's own divider (x68k.cpp's adpcm_drq_tick, 29.6) and not a line +; the chip drives out of its play state. That is a RIG fact and it is the one +; thing here a real board could contradict -- an MSM6258V that stops requesting +; when it is stopped would leave this wait spinning, and the bounded count below +; is what turns that into a report rather than a hang. + tst.l PG_ARST.l + beq.s .out + move.l #4000000,d3 + move.l PG_AMTCA.l,d2 +.rp: move.w A3_MTC,d0 + andi.l #$FFFF,d0 + cmp.l d2,d0 + bne.s .rgo + subq.l #1,d3 + bne.s .rp +.rgo: bsr ad_play +.out: movem.l (sp)+,d0-d5/a1 + rts + +; ---------------------------------------------------------------- pg_udiv32 +; d0 = d0 / d1.w with a 32-BIT QUOTIENT. Out: d0 = quotient, d3 = remainder. +; TRASHES d4, and pg_aseek reloads hz out of RAM because of it. Divide the high half, carry its remainder into the low half -- +; the second quotient fits 16 bits because the remainder is less than the +; divisor, so neither `divu` can overflow. +; +; EVERY OTHER DIVISION IN THIS PLAYER IS A BARE `divu` AND IS RIGHT TO BE. +; pg_frame's f/F, pg_apay's acc/(2*fps) and pg_aseek's own f/F all have small +; quotients by construction. This one does not, and the difference matters +; because a `divu` overflow is silent: V set, destination untouched, execution +; continuing with a stale answer. +pg_udiv32: + move.l d0,d4 + clr.w d4 + swap d4 ; d4 = the dividend's high word + divu d1,d4 ; d4 = [remainder : quotient hi] + moveq #0,d3 + move.w d4,d3 ; d3 = quotient hi + move.w d0,d4 ; d4 = [remainder : dividend low] + divu d1,d4 ; d4 = [remainder : quotient lo] + swap d3 + move.w d4,d3 ; d3 = the 32-bit quotient + clr.w d4 + swap d4 ; d4 = the remainder + move.l d3,d0 + move.l d4,d3 + rts + ; ---------------------------------------------------------------- pg_afill ; At most ONE lump a frame, and only if a slot is free. One, because a lump is ; A*512 = 7,168 B off the same disc the record came off and a frame that fetched diff --git a/tools/analysis/37_audio_seek.py b/tools/analysis/37_audio_seek.py new file mode 100644 index 0000000..c8902d7 --- /dev/null +++ b/tools/analysis/37_audio_seek.py @@ -0,0 +1,238 @@ +#!/usr/bin/env python3 +"""What a BRANCH costs the chip, and what an encoder could do about it. + + python3 tools/analysis/37_audio_seek.py [container.dlxp] [--raw au.raw] + python3 tools/analysis/37_audio_seek.py --gate # the check.sh stage + +FINDINGS 71. Session 39 put an audio seek path in src/player/packed.s and ran +it: 132,162 B across a real branch, every byte accounted for in MAME's own +capture. That settled the BYTES. This settles what is left, which is a +property of the codec rather than of the player and which no counter in the +player can reach. + +THE MSM6258'S ACCUMULATOR HAS NO LEAKAGE TERM. It is a pure integrator of +deltas, clamped, and nothing pulls it back toward zero. So a branch that hands +the chip bytes chosen for a state it is not in does not produce a transient with +a time constant -- it produces a DC OFFSET THAT NEVER DECAYS. The machine run +measures both designs at one branch point; this measures the CENSUS, over every +frame boundary of the container, and prices the only fix that is worth anything, +which is in the encoder and not in the player. + + * PLAY THROUGH the branch: the chip keeps whatever accumulator and step index + the previous scene's audio left it in. Unbounded, and its decay is the + signal's own clamping rather than the recursion forgetting. + + * STOP and re-PLAY: the accumulator goes to the container's `init` and the + step index to 0 -- a state this script knows exactly, so the error is + EXACTLY `init - acc(target)`, constant, forever. + + * ...and the third option is the ENCODER'S: encode the stream with the + predictor RESET at every point a branch can land on. Then a re-PLAYing + player is not close, it is exact. What that costs is a codec question and + is measured below. + +NAME THE LAYER. Everything here is host arithmetic over one container and its +source PCM. The chip's four axes are the ones FINDINGS 66 measured on the +machine and 67.3 put in the header; the branch behaviour is the one session 39 +ran. No emulator is involved and no rate is claimed. +""" +import argparse, math, os, statistics, sys + +HERE = os.path.dirname(os.path.abspath(__file__)) +sys.path.insert(0, os.path.join(HERE, "..", "encoder")) +import adpcm +from dlxp import DLXP + + +def acc_trajectory(nibbles, dec): + """The decoder's accumulator after every sample. This IS the encoder's + assumed state, because adpcm.encode runs its decoder inside its own search + loop -- the encoder cannot hold a state the decoder will not reach.""" + lo, hi = adpcm.clamp_bounds(dec["bits"]) + sig, idx = dec["init"], 0 + out = [] + for n in nibbles: + sig += adpcm.delta(n, adpcm.STEP[idx], dec["variant"]) + sig = lo if sig < lo else (hi if sig > hi else sig) + idx += adpcm.INDEX_ADJUST[n & 7] + idx = 0 if idx < 0 else (48 if idx > 48 else idx) + out.append((sig, idx)) + return out + + +def encode_reset(src12, dec, period): + """adpcm.encode with the predictor forced back to (init, 0) every `period` + samples. period=None is the ordinary encode. + + THIS CHANGES THE BYTES, so it is a container property and not a flag a + player can set. It is written here rather than in tools/encoder/adpcm.py + because nothing has decided to ship it: 71.5 is the trade and the deciding + number is a hardware one.""" + if period is None: + return adpcm.encode(src12, variant=dec["variant"], init=dec["init"], + bits=dec["bits"]) + out = bytearray() + for i in range(0, len(src12), period): + out += adpcm.encode(src12[i:i + period], variant=dec["variant"], + init=dec["init"], bits=dec["bits"]) + return bytes(out) + + +def snr(ref, got): + n = min(len(ref), len(got)) + sig = sum(x * x for x in ref[:n]) + err = sum((ref[i] - got[i]) ** 2 for i in range(n)) + if err == 0: + return float("inf") + return 10 * math.log10(sig / err) if sig else float("-inf") + + +def decode_reset(nib, dec, period): + if period is None: + return list(adpcm.decode_state(nib, variant=dec["variant"], + init=dec["init"], bits=dec["bits"])[0]) + out = [] + for i in range(0, len(nib), period): + out += list(adpcm.decode_state(nib[i:i + period], variant=dec["variant"], + init=dec["init"], bits=dec["bits"])[0]) + return out + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("container", nargs="?", default="tmp/packed_singe.dlxp") + ap.add_argument("--raw", default="tmp/au_singe.raw") + ap.add_argument("--gate", action="store_true") + a = ap.parse_args() + + d = DLXP(a.container) + if not d.has_audio: + sys.exit("this container is silent -- there is no branch to price") + dec = d.decoder() + lo, hi = adpcm.clamp_bounds(dec["bits"]) + data = d.audio() + nib = adpcm.unpack(data, len(data) * 2, order=dec["order"]) + traj = acc_trajectory(nib, dec) + + fails = [] + def ck(ok, msg): + print(("OK " if ok else "FAIL ") + msg) + if not ok: + fails.append(msg) + + # ---- 1. THE CENSUS. What a re-PLAY costs at every frame boundary the + # container has, which is every point a branch in this design can land on: + # 56.3's targets are frame indices and this player seeks to a frame. + den = 2 * d.fps + # THE INDEX IS A NIBBLE INDEX AND THE POSITION IS A BYTE ONE, which is the + # one conversion in this file and it is worth the line: getting it wrong + # reads the trajectory at HALF the target and produces a census that is + # entirely plausible -- a distribution of the right shape over the wrong + # instants. The cross-check against the machine's own branch point below is + # what caught it. + pos = lambda f: 2 * (f * d.aud_hz // den) - 1 + frames = [f for f in range(1, d.nframes) if pos(f) < len(traj)] + dcs = [abs(dec["init"] - traj[pos(f)][0]) for f in frames] + idxs = [traj[pos(f)][1] for f in frames] + dcs_s = sorted(dcs) + p = lambda q: dcs_s[min(len(dcs_s) - 1, int(q * len(dcs_s)))] + print(f"--- 1. A RE-PLAYED BRANCH COSTS `init - acc(target)`, EXACTLY AND " + f"FOREVER. {len(frames)} frame boundaries of {a.container}:") + print(f" |DC| against the {hi}-unit clamp: mean {statistics.mean(dcs):.1f} " + f"({statistics.mean(dcs)*100/hi:.1f}%), median {statistics.median(dcs):.0f}, " + f"p90 {p(0.90)}, worst {max(dcs)} ({max(dcs)*100/hi:.1f}%)") + print(f" ...and the step index the encoder assumed at those points runs " + f"{min(idxs)}..{max(idxs)} of 48, median {statistics.median(idxs):.0f} " + f"-- a re-PLAY sets it to 0, so a branch into a LOUD passage gets the " + f"offset AND a step index that has to climb back") + # The machine run's own branch, so the two layers are checked against each + # other rather than merely agreeing in prose. + F37 = 37 + dc37 = dec["init"] - traj[pos(F37)][0] + print(f" frame {F37}, the branch tools/bench/packed_run.sh runs on the " + f"machine: DC {dc37} -- and MAME's capture measured the chip at " + f"exactly that, constant over 62,500 samples (FINDINGS 71.3)") + ck(abs(dc37) == 65, + f"the host's arithmetic for the machine's own branch point is {abs(dc37)} " + f"and the capture said 65 -- one number, two layers") + + # ---- 2. THE DECAY THAT ISN'T. A re-PLAY's error is constant BY + # CONSTRUCTION -- same step index, same nibbles, one offset -- and playing + # through is not, because the step indices differ too. The point of + # measuring it here is that the constancy is a PROPERTY OF THE PREDICTOR + # and not of the ten seconds this container happens to hold. + print(f"--- 2. AND IT DOES NOT DECAY. The accumulator is an integrator with " + f"no leak: a re-PLAY changes the STARTING value and nothing else, so " + f"the same nibbles produce the same deltas and the offset is carried " + f"to the end of the stream. The machine agrees -- AC 0.00 over four " + f"seconds (FINDINGS 71.3). Playing THROUGH the branch is worse and is " + f"not constant, because the step index differs as well: -355 falling " + f"to -108 over four seconds, which is clamping and not forgetting.") + + # ---- 3. THE ENCODER'S FIX, PRICED. Reset the predictor where a branch can + # land and a re-PLAYing player is EXACT rather than close. + if not os.path.exists(a.raw): + print(f" (no {a.raw}: the encoder trade below needs the source PCM)") + return 1 if fails else 0 + import struct + pcm = struct.unpack("<%dh" % (os.path.getsize(a.raw) // 2), + open(a.raw, "rb").read()) + src12 = [max(-2048, min(2047, x >> 4)) for x in pcm][:len(nib)] + per_frame = d.aud_hz // den * 2 # samples in one frame slot + print(f"--- 3. THE ONLY FIX THAT MAKES A BRANCH FREE IS THE ENCODER'S, and " + f"here is its bill. Reset the predictor every N frames when encoding; " + f"a player that re-PLAYs at a branch landing on one of those points is " + f"then EXACT, not close:") + print(f" {"reset every":>24} {'SNR dB':>8} {'vs shipped':>10} " + f"{'branch points made free':>24}") + base = None + rows = [] + for label, period in [("never (shipped)", None), + (f"{d.cad_f} frames (the cadence)", d.cad_f * per_frame), + ("1 frame", per_frame)]: + nb = encode_reset(src12, dec, period) + got = decode_reset(nb, dec, period) + v = snr(src12, got) + if base is None: + base = v + free = (0 if period is None + else (len(frames) // d.cad_f if period != per_frame + else len(frames))) + rows.append((label, v, v - base, free)) + print(f" {label:>24} {v:8.2f} {v-base:+10.2f} " + f"{free:>15} of {len(frames)}") + # THE ASSERTION IS THE ORDER AND THE SIGN, not the decibel: the source PCM + # is a property of the disc and the encoder is greedy, so the exact figures + # move with the window. What must not move is that resetting COSTS SNR and + # that resetting more often costs more -- if it ever came out free, the + # predictor would not be doing anything and the codec would be pointless. + ck(rows[1][1] <= rows[0][1] + 1e-9 and rows[2][1] <= rows[1][1] + 1e-9, + f"resetting the predictor costs SNR, and resetting it more often costs " + f"more: {rows[0][1]:.2f} -> {rows[1][1]:.2f} -> {rows[2][1]:.2f} dB") + ck(rows[2][1] > rows[0][1] - 3.0, + f"...and a reset EVERY FRAME is {rows[0][1]-rows[2][1]:.2f} dB, which is " + f"the price of making all {len(frames)} of this container's branch points " + f"exact. The step table's floor is a constant 16 and the recursion " + f"re-converges in a few samples, which is why twelve resets a second is " + f"not twelve times anything") + + print(f"--- 4. WHAT THIS DOES NOT SETTLE.") + print(f" * Nothing here is a rate and nothing here ran on silicon. The " + f"branch behaviour is MAME's okim6258 -- PLAY sets the accumulator to " + f"-2, the step index to 0 and the nibble select to 0 -- which is the " + f"model FINDINGS 66 fitted to the machine and NOT a measurement of an " + f"MSM6258V. It joins session 34's fifth hardware item.") + print(f" * The census is ONE container, ten seconds, one passage at " + f"-13.4 dBFS (FINDINGS 69). The offset a re-PLAY costs is the signal's " + f"own value at the cut, so a louder passage costs more, up to the " + f"clamp -- and the disc peaks at 946 of 2048 (69.2).") + print(f" * The reset-every-frame encode is NOT in tools/encoder. It is " + f"a container change (a DLXP3), it costs bytes nothing and SNR " + f"something, and what decides it is whether a branch is allowed to " + f"land anywhere or only on frames the encoder was told about.") + print("AUDIO SEEK GATE " + ("GREEN" if not fails else f"RED: {len(fails)}")) + return 1 if fails else 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/tools/bench/check.sh b/tools/bench/check.sh index 4854ac8..fee9358 100755 --- a/tools/bench/check.sh +++ b/tools/bench/check.sh @@ -770,7 +770,7 @@ echo "--- session 36: THE CONTAINER'S OWN AUDIO, OUT OF THE CHIP (FINDINGS 68) - # The run is part of packed_run.sh's gate half, so DLX_PK_GATE_ONLY takes it. if [ -f tmp/packed_gate_stage.log ] && \ grep -aq -- "--- 5. THE AUDIO" tmp/packed_gate_stage.log; then - sed -n '/--- 5. THE AUDIO/,$p' tmp/packed_gate_stage.log | \ + sed -n '/--- 5. THE AUDIO/,/--- 7. THE SEEK/p' tmp/packed_gate_stage.log | \ grep -aE "^(OK|FAIL) |^ AUDIO:|^ {5}(nibbles per|worst|payload|-> |the player)" else echo " SKIPPED: no x68000 romset, or the container is silent" @@ -846,6 +846,41 @@ python3 tools/analysis/35_audio_level.py --gate > tmp/audio_level.log 2>&1 \ grep -aE "DISC PEAK|LOUDEST PASSAGE|THE CLAMP|LEVEL GATE|NO AUDIO TRACK|^ 1\.0000|^ 0\.5" \ tmp/audio_level.log +echo "--- session 39: THE SEEK, on the machine, with sound across it (FINDINGS 71) ---" +# Run 7 of packed_run.sh, split out of the same log for the same reason session +# 36's stage is split out of it: three verifiers in one file all emit OK lines, +# and one grep over the whole thing prints this session's result under session +# 31's heading. +if [ -f tmp/packed_gate_stage.log ] && \ + grep -aq -- "--- 7. THE SEEK" tmp/packed_gate_stage.log; then + sed -n '/--- 7. THE SEEK/,$p' tmp/packed_gate_stage.log | \ + grep -aE "^(OK|FAIL) |^ {4}-- the chip|^ {5}(the chip was|error against|-> )|DC +-?[0-9.]+ +AC" +else + echo " SKIPPED: no x68000 romset, or the container is silent" +fi + +echo "--- session 39: the audio seek path, run across a real branch (FINDINGS 71) ---" +# ROADMAP P6d, and it is the item session 38's handoff put first: 70.3 named +# what was missing -- "src/player/packed.s starts PG_AK/PG_AKF at lump 0 and has +# no audio seek path at all" -- and priced its absence at a mean 416.5 ms of +# silence over the arcade's 409 within-container seek targets. +# +# The MACHINE half of this is run 7 of tools/bench/packed_run.sh, above: two +# passes over the container with the second starting at frame 37, which is four +# frames into lump 3 and therefore NOT on a group boundary, so the byte offset +# inside the lump is load-bearing. It runs in both chip configurations. +# +# THIS STAGE IS THE HOST HALF, and what it adds is the census the one branch +# point on the machine cannot give. The chip's accumulator is an integrator with +# NO LEAKAGE TERM, so the state error a branch creates is a DC offset that never +# decays -- and the only fix that makes a branch free is in the ENCODER, not the +# player. Both are priced here, and section 1 checks the host's arithmetic for +# the machine's own branch point against what MAME's capture measured. +python3 tools/analysis/37_audio_seek.py > tmp/audio_seek.log 2>&1 \ + || { cat tmp/audio_seek.log; exit 1; } +grep -aE "^(OK|FAIL) |RE-PLAYED BRANCH|\|DC\| against|step index the encoder|frame 37, the branch|reset every|never \(shipped\)|frames \(the cadence\)|1 frame |ONLY FIX" \ + tmp/audio_seek.log + echo "--- session 38: the refill climb with a second consumer, through a real branch point (FINDINGS 70) ---" # ROADMAP P6, the oldest item in it: 65.6 and 67.6 both recorded that the slack # table existed and that 51.3's climb had never met a branch point with audio on diff --git a/tools/bench/packed.lua b/tools/bench/packed.lua index 2e3f68d..0cac9b4 100644 --- a/tools/bench/packed.lua +++ b/tools/bench/packed.lua @@ -36,8 +36,16 @@ -- DLX_PK_PACE 1 = the machine holds itself to the container's fps off -- V-DISP (default). 0 free-runs, which tests the CHAIN with -- the clock out of the way. --- DLX_PK_ITER passes over the scene (default 1). >1 exercises the seek, --- which for this container is arithmetic and nothing else. +-- DLX_PK_ITER passes over the scene (default 1). >1 exercises the SEEK. +-- DLX_PK_SEEK the frame passes after the first start at (default 0, i.e. +-- a replay). On the video path the seek IS arithmetic and +-- nothing else; on the audio path it is a second read at a +-- separate LBA, because a DLXP2 group puts lump k in FRONT of +-- its records and a branch lands `f mod F` frames into it +-- (FINDINGS 70.3). Setting this to a frame that is NOT a +-- multiple of the cadence is the point: 36 of the arcade's +-- 409 within-container seek targets land on a boundary and +-- 373 do not. -- DLX_PK_NFR play only the first N frames (default: all of them) -- DLX_PK_FPS pace at this rate instead of the container's. NOT a -- cosmetic knob and not a way to make a number look better: @@ -76,6 +84,9 @@ local PG_GSPIN, PG_LOSTV, PG_ARRN = 0x18954, 0x18958, 0x1895C -- ROADMAP P6c: the audio path's inputs and its own account of what it did. local PG_AFPS, PG_AHZ, PG_ALBA0 = 0x18960, 0x18964, 0x18968 local PG_NLUMP, PG_ABYTES, PG_APRE = 0x1896C, 0x18970, 0x18974 +local PG_SEEKF, PG_ARST = 0x18978, 0x1897C +local PG_APOS, PG_ASKIP = 0x189C8, 0x189CC +local PG_ASKN, PG_ASKB, PG_TSEQ = 0x189D0, 0x189D4, 0x189DC local PG_AARM, PG_AFET, PG_ABYT = 0x18980, 0x18984, 0x18988 local PG_ADRY, PG_ASEAM, PG_ASRV = 0x1898C, 0x18990, 0x18994 local PG_ACSR, PG_ACER, PG_ALATE = 0x18998, 0x1899C, 0x189A0 @@ -90,6 +101,11 @@ local CRTC_R20 = 0xE80028 local HELD = (os.getenv("DLX_PK_HELD") or "1") == "1" local PACED = (os.getenv("DLX_PK_PACE") or "1") == "1" local ITERS = tonumber(os.getenv("DLX_PK_ITER") or "") or 1 +local SEEKF = tonumber(os.getenv("DLX_PK_SEEK") or "") or 0 +-- ROADMAP P6d. 1 = STOP and re-PLAY the chip at a branch. Not a tidiness +-- knob: the chip's accumulator has no leak, so the two settings are a large +-- decaying error against a small permanent one (FINDINGS 71.3). +local ARST = (os.getenv("DLX_PK_ARST") or "0") == "1" local NFR = tonumber(os.getenv("DLX_PK_NFR") or "") or META.nframes local FPS = tonumber(os.getenv("DLX_PK_FPS") or "") or META.fps local CSV = os.getenv("DLX_PK_CSV") @@ -123,6 +139,8 @@ local function setup() SP:write_u32(PG_HELD, HELD and 1 or 0) SP:write_u32(PG_PACEON, PACED and 1 or 0) SP:write_u32(PG_ITER, ITERS) + SP:write_u32(PG_SEEKF, SEEKF) + SP:write_u32(PG_ARST, ARST and 1 or 0) -- DLXP2's cadence. Zero for a silent container, and the 68000 branches on the -- zero: a player told the wrong cadence does not fail, it reads an audio lump -- as a record and paints it. @@ -139,6 +157,23 @@ local function setup() SP:write_u32(PG_NLUMP, META.n_lumps or 0) SP:write_u32(PG_ABYTES, META.aud_bytes or 0) SP:write_u32(PG_APRE, APRE) + if ITERS > 1 then + local grp = (META.cad_f or 0) > 0 and (SEEKF % META.cad_f) or 0 + P(string.format("SEEK: %d passes, and passes 2..%d start at FRAME %d%s", + ITERS, ITERS, SEEKF, + (META.cad_f or 0) == 0 and " (silent container)" + or string.format(" -- lump %d, %d frame(s) into its group " + .."of %d, so the audio needs a second read and a byte " + .."offset (FINDINGS 70.3)%s", + SEEKF // META.cad_f, grp, META.cad_f, + grp == 0 and " -- ON a group boundary, the free case" + or ""))) + P(ARST and " the chip is STOPPED and re-PLAYED at the branch: its " + .."accumulator goes to the container's own init and its step " + .."index to 0" + or " the chip PLAYS THROUGH the branch: it keeps the predictor " + .."state the previous scene's audio left it in") + end P(string.format("packed.bin=%d B, %dx%d %d fps, %d of %d frames, %d passes", #code, META.W, META.H, META.fps, NFR, META.nframes, ITERS)) local cad = "" @@ -338,6 +373,20 @@ SUB = emu.add_machine_frame_notifier(function() P(string.format(" re-arms with MTC still non-zero: %d (bytes " .."fetched and never played; 0 is the correct value)", late)) + -- THE SEEK'S OWN ACCOUNT. PG_ASKN is the second reads and PG_ASKB the + -- bytes they skipped at the head of a lump: a seek path with no offset + -- term would report the first and zero for the second, and would be + -- indistinguishable from a correct one on any target that happened to + -- land on a group boundary. + local skn, skb = SP:read_u32(PG_ASKN), SP:read_u32(PG_ASKB) + if skn > 0 then + P(string.format(" SEEK: %d audio seek(s), %d B skipped into " + .."the head of a lump. Stream position ended at %d B " + .."and the chip was handed %d -- they differ BY the " + .."skip, which is the whole reason they are two cells " + .."(FINDINGS 71).", skn, skb, SP:read_u32(PG_APOS), + byt)) + end local ferr = SP:read_u32(PG_AFERR) if ferr ~= 0 then P(string.format(" A LUMP FETCH FAILED on lump %d: %s -- the " @@ -366,10 +415,15 @@ SUB = emu.add_machine_frame_notifier(function() local f = io.open(AJSON, "w") f:write(string.format('{"armed":%d,"fetched":%d,"bytes":%d,' ..'"starve":%d,"seam":%d,"serv":%d,"late":%d,"acc":%d,' - ..'"csr":%d,"cer":%d,"held":%s,"fps":%d,"shown":%d}\n', + ..'"csr":%d,"cer":%d,"held":%s,"fps":%d,"shown":%d,' + ..'"seekn":%d,"seekb":%d,"pos":%d,"seekf":%d,"iters":%d,' + ..'"arst":%s}\n', armed, fet, byt, dry, seam, srv, late, acc, SP:read_u32(PG_ACSR), SP:read_u32(PG_ACER), - HELD and "true" or "false", FPS, shown)) + HELD and "true" or "false", FPS, shown, + SP:read_u32(PG_ASKN), SP:read_u32(PG_ASKB), + SP:read_u32(PG_APOS), SEEKF, ITERS, + ARST and "true" or "false")) f:close() P("audio counters -> "..AJSON) end diff --git a/tools/bench/packed_run.sh b/tools/bench/packed_run.sh index 977bb15..0d188c0 100755 --- a/tools/bench/packed_run.sh +++ b/tools/bench/packed_run.sh @@ -158,6 +158,65 @@ if [ "$AUDIO_ON" = "1" ]; then sound (FINDINGS 67.4, and it is the bug session 36 shipped and caught)." fi +# --------------------------------------------------------------------------- +# 7. THE SEEK, WITH SOUND ON IT. FINDINGS 70.3 asked for this and named what +# was missing: "src/player/packed.s starts PG_AK/PG_AKF at lump 0 and has no +# audio seek path at all". This is that path, run. +# +# THE FRAME IS CHOSEN NOT TO BE A MULTIPLE OF THE CADENCE, and that is the whole +# design of the run. A DLXP2 group is `lump k, then F records`, so a branch +# that lands on a group boundary needs no offset and no second read -- and 36 of +# the arcade's 409 within-container seek targets do land on one. A run that +# picked one of those would exercise the arithmetic that was already there and +# report success. DLX_PK_SEEK defaults below to a frame `f mod F != 0`, so the +# byte offset inside the lump is load-bearing: get it wrong and the chip is fed +# a stream that starts up to F frames early, which is not an error, it is a +# rate, and only verify_packed_audio.py's spliced walk can see it. +# +# AND THE SECOND RESULT IS ONE NO COUNTER CAN REACH: the chip's predictor does +# not seek. Every byte can arrive, in order, exactly -- and the samples still +# be wrong, because the encoder chose them for a state a continuous play would +# have been in. The verifier measures that against its own control. +if [ "$AUDIO_ON" = "1" ] && [ "${DLX_PK_NOSEEK:-0}" != "1" ]; then + SEEKF=${DLX_PK_SEEK:-37} + CADF=$(sed -n 's/^ cad_f = \([0-9]*\),.*/\1/p' tmp/packed_meta.lua) + [ $((SEEKF % CADF)) -ne 0 ] || \ + fail "the seek frame $SEEKF is a multiple of the cadence $CADF, so it lands + ON a group boundary -- the one case that needs no byte offset and would + pass with the offset arithmetic deleted (FINDINGS 70.3)." + echo + echo "--- 7. THE SEEK: two passes, the second starting at frame $SEEKF --" + echo " lump $((SEEKF / CADF)), $((SEEKF % CADF)) frame(s) into its group" + echo " of $CADF (FINDINGS 70.3/71) ---" + TOTFR=$(( NFR + NFR - SEEKF )) + SECS=$(( TOTFR / 8 + 35 )) + # BOTH CONFIGURATIONS, and they are not two speeds of one thing. The chip's + # accumulator is an integrator with no leak, so what a branch costs is set by + # the state it lands in: play THROUGH and the chip keeps whatever the previous + # scene left it in; STOP and re-PLAY and it goes to the container's own `init` + # with the step index at 0. Neither is zero and they are 5.5x apart, so the + # run measures both and FINDINGS 71.3 chooses. + for M in 0 1; do + TAG=seek; [ "$M" = 1 ] && TAG=seek_rst + echo " -- the chip $([ "$M" = 1 ] && echo 'STOPPED and re-PLAYED' \ + || echo 'PLAYING THROUGH') the branch" + SOUNDARGS="-samplerate 15625 -wavwrite packed_$TAG.wav" + run $TAG DLX_PK_HELD=0 DLX_PK_PACE=1 DLX_PK_FPS="$CFPS" DLX_PK_NFR=$NFR \ + DLX_PK_AUD=1 DLX_PK_AJSON=packed_$TAG.json \ + DLX_PK_ITER=2 DLX_PK_SEEK=$SEEKF DLX_PK_ARST=$M + SOUNDARGS="-sound none" + grep -aq "^\[PK\] FLAG=\$FF" "tmp/packed_$TAG.log" || \ + fail "the $TAG run did not reach the end of the second pass." + python3 tools/bench/verify_packed_audio.py "$DLXP" "tmp/packed_$TAG.wav" \ + "tmp/packed_$TAG.json" --seek "$SEEKF" --iters 2 || \ + fail "the player did not play the container ACROSS A BRANCH ($TAG). + Nothing here parses anything (FINDINGS 67.4): a seek that fetched the + wrong lump plays 7,168 B of the wrong part of the scene, and one that + dropped the byte offset plays the right lump from up to $((CADF-1)) + frames too early -- neither is an error and both are a sound." + done +fi + if [ "${DLX_PK_GATE_ONLY:-0}" = "1" ]; then exit 0; fi echo diff --git a/tools/bench/verify_packed_audio.py b/tools/bench/verify_packed_audio.py index 2478ba1..2e6b31e 100644 --- a/tools/bench/verify_packed_audio.py +++ b/tools/bench/verify_packed_audio.py @@ -1,7 +1,8 @@ #!/usr/bin/env python3 """The container's own audio, READ BACK OFF THE SPEAKER. ROADMAP P6c. - python3 tools/bench/verify_packed_audio.py [counters.json] + python3 tools/bench/verify_packed_audio.py \ + [counters.json] [--seek FRAME --iters N] WHY THIS READS THE CAPTURE AND NOT THE PLAYER'S COUNTERS. src/player/packed.s reports how many lumps it armed and how many payload bytes it handed the chip, @@ -36,6 +37,25 @@ WHAT IS CHECKED, and it is the whole scene rather than a sample of it: search below carries the state across the seam and reports its LENGTH, which is the audible cost of every design decision on the video path. + 4. THE SEEK, when the run made one (--seek). A branch is a frame index and + a DLXP2 group puts lump k in FRONT of its records, so the player has to + issue a second read and enter the lump `f mod F` frames in (FINDINGS 70.3). + The stream the chip should then have been fed is the container's own bytes + SPLICED -- everything, then everything from byte floor(f*hz/(2*fps)) on -- + and the walk above accounts for it as one continuous run, which is the + check: a player that dropped the byte offset feeds a stream that starts up + to F frames early and NOTHING ABOUT IT IS AN ERROR. It is a rate. + + AND THE THING THE PLAYER CANNOT SEE. An MSM6258 has no seek: its + accumulator and step index are the product of every nibble since PLAY, so a + seek hands it bytes the encoder chose for a state it is not in. The + samples are then wrong while the recursion re-converges, and they are wrong + WITHOUT ANY BYTE BEING WRONG -- every counter in the player stays right and + the walk above still passes, because the walk tracks the chip rather than + the intent. This measures it: the chip's own samples across the splice + against the samples the ENCODER meant, which are the same bytes decoded + from the state a continuous play would have been in. + THE NEGATIVE CONTROL IS BUILT IN. A seam is found by searching for the repeat count that makes the next lump match; if the player had fed the wrong bytes, no repeat count would make it match and the run fails rather than sliding. @@ -190,9 +210,88 @@ def _look(rec, pos, st, data, i, one, look=LOOK): return True +def stream_pos(d, frame): + """The byte of the ADPCM stream frame `frame` starts at. Exact, and the + exactness is the same one FINDINGS 67.2 is about: 15,625 samples a second + over 24 half-frames does not divide, and a seek that rounded it would be a + rate error rather than a byte error.""" + return frame * d.aud_hz // (2 * d.fps) + + +def spliced(d, seek, iters): + """The stream the player should have fed, as (label, bytes) chunks. + + One pass is the container's lumps in order. A pass after a SEEK starts at + stream byte stream_pos(seek): the tail of lump k = seek//F, then every lump + after it. The chunks are kept separate because their BOUNDARIES are what + the stretched-byte assertion is measured against -- the chip replays its + last byte at a lump end and at a seek, and nowhere else. + """ + out = [(f"lump {k}", d.lump(k)) for k in range(d.n_lumps)] + if iters <= 1 or seek is None: + return out + b0 = stream_pos(d, seek) + for it in range(1, iters): + off = 0 + for k in range(d.n_lumps): + lb = d.lump(k) + if off + len(lb) <= b0: + off += len(lb) + continue + cut = max(0, b0 - off) + out.append((f"pass {it+1} lump {k}" + (f" +{cut} B" if cut else ""), + lb[cut:])) + off += len(lb) + return out + + +def walk_reset(rec, start, data, dec, sp): + """The walk, SPLIT AT A CHIP RESET. + + A STOP/PLAY at the branch is a discontinuity the continuous model cannot + cross: the accumulator, the step index and the nibble select all go back to + the container's own start state, so no repeat count of the last byte of + pass 1 can be followed by pass 2's first sample. The walk would report + "byte 78,124 does not account for capture sample N", which is true and is + the wrong complaint. + + So pass 1 is walked without its final byte, the RESTART is searched for -- + the first capture position at which pass 2 walks cleanly from `init` -- and + pass 2 is walked from there. The final byte's run length is then the gap, + which is exactly what it is: the chip replaying it while the 68000 fetched + a lump and stopped the chip. + + The search is cheap because the first post-reset sample is DETERMINED: one + nibble of a known byte from a known state. Only positions carrying that + value are tried at all. + """ + one = stepper(dec) + bad, pos, runs = walk(rec, start, (dec["init"], 0), data[:sp - 1], dec) + if bad is not None: + return bad, pos, runs, None + first, _ = one((dec["init"], 0), data[sp] & 15) + K = 64 # bytes of pass 2 a candidate must + for p in range(pos, min(pos + MAXRUN, len(rec))): # survive before the + if rec[p] != first: # full walk is run + continue + b2, _, _ = walk(rec, p, (dec["init"], 0), data[sp:sp + K], dec) + if b2 is None: + b3, pos3, runs3 = walk(rec, p, (dec["init"], 0), data[sp:], dec) + if b3 is None: + return None, pos3, runs + [p - pos] + runs3, p + return sp - 1, pos, runs, None + + def main(): - if len(sys.argv) < 3: + argv = [a for a in sys.argv[1:]] + seek, iters = None, 1 + while "--seek" in argv: + i = argv.index("--seek"); seek = int(argv[i+1]); del argv[i:i+2] + while "--iters" in argv: + i = argv.index("--iters"); iters = int(argv[i+1]); del argv[i:i+2] + if len(argv) < 2: sys.exit(__doc__) + sys.argv = [sys.argv[0]] + argv d = DLXP(sys.argv[1]) counters = json.load(open(sys.argv[3])) if len(sys.argv) > 3 else None if not d.has_audio: @@ -229,8 +328,26 @@ def main(): # sector runs -- so a player that fed the chip whole lumps does not merely # score worse here, it fails to walk: the 6.54 B of zero at the end of a # lump are nibbles that are not in this stream. - data = d.audio() - bad, pos, runs = walk(rec, start, (dec["init"], 0), data, dec) + chunks = spliced(d, seek, iters) + data = b"".join(c for _, c in chunks) + if seek is not None and iters > 1: + print(f"--- THE SEEK: {iters} passes, passes 2..{iters} start at frame " + f"{seek} = stream byte {stream_pos(d, seek):,}, which is " + f"{seek % d.cad_f} frame(s) into lump {seek // d.cad_f} " + f"(FINDINGS 70.3). The stream below is SPLICED and is " + f"{len(data):,} B against the container's {d.aud_bytes:,}.") + replay = bool(counters and counters.get("arst")) + if replay and seek is not None and iters > 1: + sp0 = sum(len(c) for _, c in chunks[:d.n_lumps]) + bad, pos, runs, rp = walk_reset(rec, start, data, dec, sp0) + if bad is None: + print(f" the chip was STOPPED and re-PLAYED at the branch: the " + f"walk is SPLIT there, and pass 2 restarts at capture sample " + f"{rp:,}, {runs[sp0-1]:,} samples after the last byte of pass " + f"1 was handed over ({runs[sp0-1]/rate*1000:.0f} ms of " + f"replayed byte and stopped chip)") + else: + bad, pos, runs = walk(rec, start, (dec["init"], 0), data, dec) ck(bad is None, f"all {len(data):,} bytes of the container's audio reached the chip, in " f"order, and every sample the chip produced from them is exact" @@ -278,8 +395,8 @@ def main(): # else is the CPU losing the chip in the middle of a buffer. ends = set() off = 0 - for k in range(d.n_lumps): - off += len(d.lump(k)) + for _, c in chunks: + off += len(c) ends.add(off - 1) stray = [i for i, _ in seams if i not in ends] ck(not stray, @@ -310,6 +427,129 @@ def main(): f"game's 22.8 min, and the accumulator in pg_apay is the three " f"lines that do not spend it") + # ---- 4. WHAT A SEEK COSTS THE CHIP, and it is the thing no counter in the + # player can reach. The walk above proves every BYTE arrived; this asks + # whether the SAMPLES the chip made out of them are the ones the encoder + # meant. They are not, and they cannot be: the MSM6258's accumulator is a + # pure integrator with NO LEAKAGE TERM, so a state mismatch at a branch is a + # DC offset that does not decay -- it is not a transient with a time + # constant, and calling it one would be the flattering reading. + # + # THE REFERENCE IS THE ENCODER'S OWN STATE, not a fresh one. adpcm.py + # encoded the stream in one pass, so the state it chose byte B(f)'s nibbles + # for is the state a CONTINUOUS play reaches at B(f). The FRESH control is + # the other design -- STOP the chip and PLAY it again at the branch, which + # is what DLX_PK_ARST does -- and the two numbers are what choose between + # them. Under --replay the fresh series is not a control at all: it is the + # prediction, and it has to be sample-exact. + if seek is not None and iters > 1 and bad is None: + one = stepper(dec) + sp = sum(len(c) for _, c in chunks[:d.n_lumps]) + b0 = stream_pos(d, seek) + whole = d.audio() + + def run_from(st, i0, nmax): + """The samples the chip WOULD have made from byte i0 on, had it been + in state `st` -- with the run lengths it actually used, so the two + series are sample-aligned across a seam as well as a byte.""" + out, i = [], i0 + while i < len(data) and len(out) < nmax: + b, c = data[i], runs[i] + for j in range(c): + smp, st = one(st, (b & 15) if j % 2 == 0 else (b >> 4)) + out.append(smp) + i += 1 + return out + + if replay: + st = (dec["init"], 0) # by construction: PLAY sets + else: # both, and the run above + st = (dec["init"], 0) # proved it sample-exact + for i in range(sp): # ...otherwise the chip is + b, c = data[i], runs[i] # wherever the previous + for j in range(c): # scene's audio left it, + _, st = one(st, (b & 15) if j % 2 == 0 else (b >> 4)) + pos0 = start + sum(runs[:sp]) + N = min(4 * rate, len(rec) - pos0) + + ref = (dec["init"], 0) # ...and the ENCODER's state + for by in whole[:b0]: # at the SAME stream byte, + for nib in (by & 15, by >> 4): # reached continuously + _, ref = one(ref, nib) + + got = rec[pos0:pos0 + N] + want = run_from(ref, sp, N) + fresh = run_from((dec["init"], 0), sp, N) + n = min(len(got), len(want), len(fresh)) + e = [got[i] - want[i] for i in range(n)] + + def band(lo, hi): + seg = e[lo:min(hi, n)] + if not seg: + return None + m = sum(seg) / len(seg) + ac = (sum((x - m) ** 2 for x in seg) / len(seg)) ** 0.5 + return m, ac, max(abs(x) for x in seg) + + lo_c, hi_c = adpcm.clamp_bounds(dec["bits"]) + print(f"--- THE PREDICTOR DOES NOT SEEK (FINDINGS 71). The chip's state " + f"at the branch is accumulator {st[0]}, step index {st[1]}; the " + f"encoder chose byte {b0:,}'s nibbles for accumulator {ref[0]}, " + f"step index {ref[1]}.") + print(f" error against the encoder's intent, DC and AC separately " + f"(full scale is {hi_c}):") + for lo, hi, lab in [(0, 100, "0-6 ms"), (100, 1000, "6-64 ms"), + (1000, 5000, "64-320 ms"), (5000, rate, "0.3-1.0 s"), + (rate, 2*rate, "1-2 s"), (2*rate, 4*rate, "2-4 s")]: + r = band(lo, hi) + if r: + print(f" {lab:>10} DC {r[0]:8.1f} AC {r[1]:7.2f} " + f"|max| {r[2]}") + if replay: + # THE STRONGEST FORM THIS CAN TAKE. A STOP/PLAY puts the chip in a + # state this script knows exactly, so the prediction is not "close", + # it is every sample. A run that claimed to reset and did not fails + # here and passes everything else on the page. + diff = [i for i in range(n) if got[i] != fresh[i]] + ck(not diff, + f"the chip was STOPPED and re-PLAYED at the branch, so all " + f"{n:,} post-seek samples are EXACTLY a decode from the " + f"container's own accumulator ({dec['init']}) and step index 0" + + ("" if not diff else f" -- {len(diff):,} differ, first at " + f"sample {diff[0]}")) + dc = [x for x in e] + const = len(set(dc)) == 1 + ck(const, + f"...and the whole error against the encoder's intent is the " + f"SINGLE CONSTANT {dc[0]}" + + ("" if const else f" -- it takes {len(set(dc))} values, so the " + f"step indices differ too and this is distortion, not offset") + ) + if const: + print(f" -> a re-PLAYED branch costs a PERMANENT DC offset " + f"of {dc[0]} = {abs(dc[0])*100/hi_c:.1f}% of full scale " + f"and {abs(dc[0])*100/511:.1f}% of the 10-bit clamp's " + f"headroom. It is inaudible as a tone and it is not free: " + f"it is headroom, and it clicks once at the branch.") + else: + r0, r4 = band(0, 100), band(2*rate, 4*rate) + ck(band(1000, 5000)[1] < abs(band(1000, 5000)[0]), + f"the error at the branch is an OFFSET and not distortion: over " + f"64-320 ms its DC is {band(1000,5000)[0]:.1f} and its AC is " + f"{band(1000,5000)[1]:.2f}, so the chip is decoding the right " + f"shape from the wrong ground") + if r4: + print(f" -> playing THROUGH the branch, the offset is still " + f"{r4[0]:.0f} four seconds later ({abs(r4[0])*100/hi_c:.0f}%" + f" of full scale). There is no leakage term in this " + f"predictor; what decay there is comes from the signal's " + f"own clamping, not from the recursion forgetting.") + # AND THE CONTROL THAT MAKES EITHER READING MEAN ANYTHING. + ck(any(want[i] != fresh[i] for i in range(n)), + f"the two references are distinguishable over these {n:,} samples, " + f"so 'carry the predictor' and 'reset it' are different runs and this " + f"measurement has a subject") + if counters: ck(counters["late"] == 0, f"the player never re-armed a channel that still had bytes to send " @@ -318,8 +558,32 @@ def main(): f"the player never found the channel counted out with no lump ready " f"({counters['starve']} times it did)") ck(counters["bytes"] == len(data), - f"the player's own byte count ({counters['bytes']:,}) is the " - f"container's whole payload ({len(data):,})") + f"the player's own byte count ({counters['bytes']:,}) is the whole " + f"stream it should have fed ({len(data):,})") + if seek is not None and iters > 1: + # THE TWO CELLS, ASSERTED APART. A player that kept one counter for + # "where the stream is" and "what the chip got" reports a number + # that is right for neither, and the symptom is a LAST LUMP that is + # long by the skip -- which is not an error, it is a rate. + want_skip = (iters - 1) * (stream_pos(d, seek) + - stream_pos(d, seek - seek % d.cad_f)) + ck(counters.get("seekn") == iters - 1, + f"the player made {counters.get('seekn')} audio seek(s) for " + f"{iters-1} branch point(s) -- the second read FINDINGS 70.3 " + f"asks for, at a separate LBA") + ck(counters.get("seekb") == want_skip, + f"it skipped {counters.get('seekb')} B into the head of a lump " + f"and the cadence says {want_skip} -- the byte offset is what " + f"makes a branch land on its own frame instead of up to " + f"{d.cad_f-1} frames early") + ck(counters["pos"] == d.aud_bytes, + f"the stream POSITION ended at {counters['pos']:,} B, the " + f"container's whole stream ({d.aud_bytes:,}) -- while the chip " + f"was handed {counters['bytes']:,}. TWO CELLS FOR TWO FACTS: the " + f"position is where the container has got to and is what the " + f"last lump's length is measured against; the byte count is what " + f"the capture has to account for, and a seek moves one and not " + f"the other") print(f" the player: {counters['armed']} lumps armed, " f"{counters['fetched']} fetched, {counters['serv']:,} service " f"calls over {counters['shown']} frames "