drwily/Dragon-s-Lair-X68k
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Put sound on the wire, and find three LSBs are worth 25 dB

Browse Source 
ROADMAP P6, everything in the item except the bus half session 20 closed.
tools/encoder/adpcm.py is an MSM6258 codec, tools/encoder/extract_audio.py
takes the same seconds of the same stream the frames come from,
tools/bench/verify_adpcm.py is the gate, tools/analysis/32_audio_wire.py the
container arithmetic.

There is no reference encoder -- ffmpeg has a decoder for this format and none
the other way -- so what is gated is the decoder the encoder runs INSIDE its
own nibble search, sample-exact against ffmpeg's over 4,268 nibbles. An
encoder that agrees with its own wrong decoder is what that catches. The Singe
window: 156,250 samples -> 78,125 B at 21.97 dB, which is 7,812.5 B/s to the
byte. Normalising the disc's -13.4 dBFS level moves the SNR 21.97 -> 21.97, so
the level is not a lever.

And the two published delta formulas are not the same codec. They differ by at
most 3 in 12-bit units; encode for one and decode on the other and the SNR
goes 21.97 -> -2.88 dB, the noise louder than the signal, because ADPCM is
recursive the way the video codec is temporally recursive. Which one the chip
runs is now P6a and it is a precondition on shipping any audio.

And audio is the first thing the packed branch's simplification has cost
anything for. A record has no index BY DESIGN, so audio cannot be per-record
without making records variable; it rides a fixed cadence (F, A), the obvious
F=1 wastes 57.3% of every audio sector, and the pick is F=11 A=14 -- 0.09%
padding, 14,336 B held, wire 582.0 -> 589.6 KB/s. The codec container, which
kept its index, pays zero.

The MAME experiment did not work and 65.5 says so: :okim6258 is there at
$E92001/$E92003, read out of the machine's own program map, and feeding it
from Lua recorded silence across control 0..3 x port C 0..15. The register
semantics were not guessed at further.

FINDINGS 65. check.sh ALL GREEN before and after, with a new stage.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
This commit is contained in:
prosolis
2026-08-25 09:11:03 -07:00
parent 6f698ca226
commit f925a1dd9a
12 changed files with 1069 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+30 -1
View File
@@ -339,6 +339,34 @@ from taking. It believed it was at 12 fps; the screen was at **6.37**. Only the
host's raster count contradicts it, and the gate asserts on the difference
(FINDINGS 64.3).
**And the sound has an encoder, whose most useful output so far is a warning.**
The X68000's audio is an OKI MSM6258 — 4 bits a sample, 15,625 of them a second,
7,812.5 bytes a second exactly. `tools/encoder/adpcm.py` encodes the same ten
seconds the frames come from: **78,125 B at 21.97 dB**. There is no reference
encoder to check it against — ffmpeg has a decoder for this format and no encoder
— so what is gated is the decoder the encoder runs *inside its own nibble
search*, sample-exact against ffmpeg's over 4,268 nibbles. An encoder that agrees
with its own wrong decoder is what that catches.
**And the two published versions of this codec are not the same codec.** ffmpeg
computes a nibble's contribution as `((2*(n&7)+1) * step) >> 3`; the OKI
datasheet truncates per term. They differ by **at most 3 in 12-bit units**.
Encode for one and decode on the other and the signal-to-noise ratio goes from
**21.97 dB to −2.88 dB — the noise comes out louder than the signal**, because
ADPCM is recursive and a rounding difference does not stay where it happens. So
which one the chip runs is not a footnote; it is a precondition on shipping any
audio at all, and MAME's x68000 has the chip to ask (FINDINGS 65).
**And audio is what the packed container's best property finally costs
something for.** A packed record is 97 sectors and its address is arithmetic —
no index, and none can be needed. Audio is 651.0417 bytes a frame slot, a rate
with no arithmetic relationship to 12 fps, so it cannot ride the record without
making records variable length and bringing an index back. It rides a fixed
cadence instead — every 11 frames, 14 sectors — which wastes **0.09%**, where
the obvious one-lump-per-record cadence wastes **57.3%** of every audio sector.
The wire goes **582.0 → 589.6 KB/s**. The codec container, which already has the
index the packed one deleted, pays **zero**.
**The scene graph is in, and the worst gap between two decision points is
zero.** `tools/import/scenegraph.py` imports the arcade scene graph — 40 scenes,
516 sequences, 906 input windows — and 5.4% of the game's 612 branch transitions
@@ -371,7 +399,8 @@ cores, the ring and paced-ring passes, the DMAC configuration gate and the
load-time transforms on both cores, then imports and gates the scene graph
when a DirkSimple checkout is present, then builds the packed container and
renders it through px68k's own GVRAM model, then **runs the packed player for
120 frames off a real volume and compares every one of them**, then prints
120 frames off a real volume and compares every one of them**, then encodes the
same window's audio and gates it against ffmpeg's decoder, then prints
`ALL GREEN`.
## Reproducing this
docs/FINDINGS.md
+181
View File
@@ -6768,3 +6768,184 @@ throughout) — and no MAME source tree was available on this machine to name it
third** that decides whether a DMAC-direct packed player shows a picture.
- **Whether B is buildable as described.** It is priced off a measured blit and
a measured ladder, and no line of it has been written.
---
## 65. Audio has an encoder, and the packed container's best property is what makes it cost (session 33)
**ROADMAP P6**, everything in the item except the bus half session 20 closed.
`tools/encoder/adpcm.py` is the codec, `tools/encoder/extract_audio.py` the
extraction, `tools/bench/verify_adpcm.py` the gate, `tools/analysis/32_audio_wire.py`
the container arithmetic. All of it is **host arithmetic and one emulator
introspection**; no board ran, and the one MAME experiment that was attempted
did not work — 65.5 says so rather than leaving it out.
### 65.1 There is no reference encoder, so the gate had to be built sideways
The X68000's ADPCM is an OKI MSM6258V: 4 bits a sample, two to a byte, 12-bit
signal word, and three rates that are 8 MHz over 512, 768 and 1024. **ffmpeg has
a decoder for the format (`adpcm_ima_oki`) and no encoder**, so there is nothing
to diff an encoder against.
What `verify_adpcm.py` gates instead is the thing that can actually be wrong:
the encoder runs a decoder **inside its own loop** to choose each nibble, and
that decoder is checked **sample-exact against ffmpeg's** over 4,268 nibbles.
An encoder that agrees with its own wrong decoder is exactly the failure mode a
round-trip test cannot see.
Two facts fell out of building it, and both were **measured rather than assumed**:
| | |
|---|---|
| nibble order | **HIGH NIBBLE FIRST** — reading low-first disagrees with ffmpeg on 3,285 of 4,268 samples, and that mismatch is carried as the gate's negative control |
| the step table | 49 entries, **BUILT** as `floor(16 * 1.1**k)` and checked against the published list, so a transcription slip is not one of the things that can be wrong |
On the window this project gates everything on (00223 @539.4 s, 10.000 s, the
same seconds as `tmp/fr_singe`): **156,250 samples → 78,125 B, SNR 21.97 dB**,
and 78,125 B / 10.000 s is **7,812.5 B/s to the byte**, which is 52's figure
arriving from the other direction.
**A negative worth having: the level is not a lever.** The disc's window peaks
at −13.4 dBFS, using 435 of the 12-bit word's 2,048. Normalising it — gain ×4.7,
one sample clipped — moves the SNR from **21.97 dB to 21.97 dB**. The step
table's adaptation covers the range, so there is no headroom win to collect and
no reason to touch the disc's level.
### 65.2 The two available references DISAGREE, and it costs 25 dB
The delta a nibble contributes has two forms in circulation:
'shift' delta = ((2*(n&7) + 1) * step) >> 3
'terms' delta = step/8 + (n&4)*step + (n&2)*step/2 + (n&1)*step/4,
each term truncated independently
`'shift'` is what ffmpeg computes — **verified sample-exact here, so that is a
measurement of the decoder that ships, not a reading of its source.** `'terms'`
is the OKI datasheet's own form, the one a chip builds out of shifts and adds,
and it is what MAME's `okim6258` is understood to compute. **That last clause is
NOT verified**: no MAME source tree is on this machine (64.4).
They differ **on 1,000 of 4,268 sampled nibbles, by at most 3 in 12-bit units**,
which reads like something nobody could hear. **It is not.**
| encoded | decoded | SNR |
|---|---|---:|
| `shift` | `shift` | **21.97 dB** |
| `shift` | `terms` | **−2.88 dB** |
| `terms` | `terms` | 21.99 dB |
| `terms` | `shift` | −3.38 dB |
**The noise is louder than the signal.** Per-sample disagreement over the real
window: **max 257, mean 78.2**, against a source whose RMS is 74.4. A 3-LSB
formula difference becomes a 25 dB loss because **ADPCM is RECURSIVE** — the
delta is added to a running predictor and the nibble also moves the step index,
so a disagreement does not stay where it happens. It is the same shape as the
codec's temporal recursion, one dimension down: 64.1 used that recursion to make
one frame audit 120, and here the same property turns a rounding difference into
a broken stream.
**So "which formula does the MSM6258 run" is not a footnote. It is a
precondition on shipping any audio at all**, and it has to be answered before an
encoder's output is committed to a container.
### 65.3 The interleave, and why the obvious cadence is the wrong one
**A packed record is 49,664 B = 97 sectors and its address is `LBA0 + i*97`.
There is no index and none can be needed** — that is the format's whole claim
(63, 64.1). Audio is a stream at a rate with no arithmetic relationship to the
frame rate: at 15,625 Hz a 12 fps slot is **651.0417 B**, and the `.0417` is the
same remainder the frame clock carries (54), because 8 MHz / 512 / 2 / 12 has a
3 in the denominator that no power of two clears.
Give record *i* the audio belonging to slot *i* and the records become variable
length — and the moment records are variable length the format needs an index
and stops being the format. So audio rides a **fixed cadence**: every `F`
frames, `A` whole sectors, placed between records, leaving
LBA(i) = LBA0 + i*97 + floor(i/F)*A
which is still arithmetic. Choosing `(F, A)` is a rational approximation to
`15625/12288 = 1.271565755` from above, and **the obvious cadence is the worst
point in the space**:
| F | A | lump | needs | padding | wire adds | held (2 lumps) |
|---:|---:|---:|---:|---:|---:|---:|
| **1** — one lump a record | 2 | 1,024 B | 651.0 B | **57.29%** | **12.00 KB/s** | 2,048 B |
| 3 | 4 | 2,048 B | 1,953.1 B | 4.86% | 8.00 KB/s | 4,096 B |
| 7 | 9 | 4,608 B | 4,557.3 B | 1.11% | 7.71 KB/s | 9,216 B |
| **11** | **14** | **7,168 B** | 7,161.5 B | **0.09%** | **7.64 KB/s** | **14,336 B** |
| 81 | 103 | 52,736 B | 52,734.4 B | 0.003% | 7.63 KB/s | 105,472 B |
**F=11 is the pick.** It buys 57.2 points of padding for 12,288 B of RAM over
the naive cadence; the floor of the sweep buys the last 0.09 of a point for
91,136 B more, and on a machine where K4 already wants 99,328 B for two record
buffers that second trade is not one.
**The wire, then:** the packed container's sustained requirement was **582.0 KB/s
silent** and is **589.6 KB/s with sound** (+1.31%). A literal frame's bitrate is
geometry and cannot be talked down; the audio on top of it is 7.63 KB/s of
payload and can only be talked down by choosing a worse chip rate.
### 65.4 And this is the first price anyone has found for the packed branch's own simplification
The codec container **pays none of it**. `rc_fr_singe_scsi_span.dlx` already
carries an index and already has variable records (4,096..41,472 B, sector-aligned
since DLX5), so it can put exactly 651.0417 B of audio in record *i* and pad only
to the sector it was going to pad to anyway: **zero audio padding**, wire
440.4 → 448.1 KB/s.
"A record's length is geometry, so there is no index and none can be needed" is
what makes the packed player a page of arithmetic instead of a parser — and it
is **exactly** the property that makes a second stream at an unrelated rate cost
a cadence, a padding fraction and a 14,336 B buffer. It is a small cost and it
is not zero, and **nothing in FINDINGS 61-64 predicted it**. 64's risk list said
"a simplification that large usually hides something"; this is the first thing
it hid.
The bus half reproduces 52 exactly, which is why the tool prints it: 651.0 B a
slot at the IPL ROM's own channel-3 cost of 16..19 clk/B is **10,417..12,370
clocks = 1.25%..1.48% of a slot**. **The interaction 52 could not have had** is
with 64.2's write window: a DMAC-direct packed player holds the GVRAM window
open for the whole data phase, so an audio channel stealing the bus during that
phase makes the phase longer — audio costs **darkness**, not just clocks. It is
negligible against a dark fraction that is already 1.0, and it is not negligible
against K4's 27.3% paint. That is the third time this session the two players
have ranked differently on a column that is not clocks.
### 65.5 The experiment that did not work, stated rather than omitted
65.2's open question has an obvious apparatus: **MAME's x68000 has the chip**,
so it can be asked. Introspection got as far as fact and no further, and the
facts are worth keeping because the next attempt starts from them rather than
from folklore:
| | |
|---|---|
| device | **`:okim6258`**, shortname `okim6258` — it is there |
| registers | **`$E92001`** and **`$E92003`**, each one byte, read out of the maincpu program map by `tools/bench/probe_adpcm.lua` |
| also on the map | `$E9A000-$E9BFFF`, the PPI that carries ADPCM pan and the clock divider |
**Feeding the chip from Lua produced no audio.** `tools/bench/probe_adpcm2.lua`
writes a control byte to `$E92001` and nibble pairs to `$E92003`;
`probe_adpcm3.lua` sweeps PPI port C at `$E9A005` over all sixteen low-nibble
values with a loud burst under each. **Control 0..3 × port C 0..15: MAME's
`-wavwrite` capture is silent throughout, 0 of 567,360 samples non-zero.**
The register *semantics* are the gap — which control value starts playback,
whether port C needs the PPI's mode word set first, and whether the chip has a
clock at all until the divider is written. **None of that was guessed at further,
because guessing at it is how a rig produces a confident wrong answer.** The way
to do this is from 68000 code with the IPL ROM's own channel-3 DMAC
configuration, which `tools/analysis/21_iplrom_dmac.py` already reads out of the
ROM — the real design, and the one path in the machine that is known to be
correct because Sharp wrote it.
### 65.6 What this does not settle
- **Which delta formula the chip runs.** 65.2, and it is worth 25 dB.
- **Anything on a board.** No hardware ran. The `-wavwrite` silence is a
statement about an apparatus, not about a chip.
- **The container.** DLXP1 has no audio section; 65.3 is the arithmetic a
DLXP2 would be built from, and no byte of one has been written.
- **What audio does to a scene change.** The slack table is here, but 51.3's
refill climb with a second consumer through a real branch point is not.
docs/ROADMAP.md
+67 -4
View File
@@ -74,6 +74,28 @@ the CPU opens the window only for the measured 27.3% blit, so it is on screen
the clock cannot tell** — 46.9% of V-DISP edges lost, zero late frames reported,
the player believing 12 fps while the screen ran at 6.37. **The open item is now
K4.**
Amended end of session 33: **P6 HAS AN ENCODER AND A PRICE (FINDINGS 65).**
`tools/encoder/adpcm.py` is an MSM6258 codec whose in-loop decoder is gated
SAMPLE-EXACT against ffmpeg's `adpcm_ima_oki` — there is no reference encoder
for this format, so that is the only check available and it is the one that
catches an encoder agreeing with its own wrong decoder. The Singe window encodes
to **78,125 B at 21.97 dB**, which is 7,812.5 B/s to the byte and 52's figure
arriving from the other direction, and **normalising the disc's −13.4 dBFS level
buys 0.00 dB**, so the level is not a lever. Two things came with it. **(1) The
two published delta formulas disagree by at most 3 in 12-bit units and that is
worth 25 dB** — encode with one and decode with the other and the SNR goes from
21.97 dB to **−2.88 dB**, because ADPCM is recursive the way the codec is
temporally recursive (64.1). Which one the chip runs is now **P6a** and it is a
precondition on shipping any audio. **(2) The packed container's own best
property is what makes audio cost.** A record has no index BY DESIGN, so audio
cannot be per-record without making records variable; it rides a fixed cadence
`(F, A)`, the obvious cadence F=1 wastes **57.3%** of every audio sector, and the
pick is **F=11, A=14 — 0.09% padding, 14,336 B held, wire 582.0 → 589.6 KB/s**.
The codec container, which already has an index and variable records, pays
**zero** padding. That is the first cost anyone has found for the packed
branch's simplification, and 64's risk list predicted there would be one without
knowing what.
**THE COMPLETION TARGET IS M3, THE VERTICAL SLICE** (USER DECISION): one scene
tree — a decision point, two outcomes, a death clip — with audio, streaming from
a real SCSI volume on a stock 2 MB machine, playable. That is the point at which
@@ -664,10 +686,46 @@ charging audio the disk's 5. Both worries above resolve:
7.8 was decimal kB being multiplied by 1024; 2.4% high, now derived from the
sample rate in `buscost.ADPCM_BYTES_PER_S`.
**What is still open in P6 is everything except the bus:** extraction, encode,
container interleave, and what a second stream does to `wire` — and therefore to
`pipe - wire`, and therefore to 51.3's refill climb. That last one is the
interaction to price next, and it is E2's question with a second consumer in it.
~~**What is still open in P6 is everything except the bus:** extraction, encode,
container interleave, and what a second stream does to `wire`.~~ **THREE OF THE
FOUR ARE DONE, session 33 — FINDINGS 65.** `tools/encoder/extract_audio.py`
takes the same seconds of the same stream the frames come from;
`tools/encoder/adpcm.py` encodes them (**78,125 B, 21.97 dB**, gated against
ffmpeg's decoder sample-exact by `tools/bench/verify_adpcm.py`);
`tools/analysis/32_audio_wire.py` is the interleave and the wire. **The packed
container's cadence is `F=11, A=14`** — 0.09% padding, 14,336 B held, 582.0 →
**589.6 KB/s** — and the naive one-lump-per-record cadence would have wasted
57.3% of every audio sector. **The codec container pays zero padding**, because
it already has the index the packed one deleted (65.4).
**What is left in P6 is the fourth: the refill climb with a second consumer
through a real branch point** (51.3, 55.4). The slack table is in
`32_audio_wire.py` — at 582.0 KB/s exactly, silent breaks even and sounded
starves — but a branch point has not been run with audio on the wire.
**P6a. WHICH DELTA FORMULA DOES THE MSM6258 RUN? (new, session 33, FINDINGS
65.2 — and it is a precondition, not a refinement.)** ffmpeg computes
`((2*(n&7)+1) * step) >> 3`; the OKI datasheet's form truncates per term. They
differ by **at most 3 in 12-bit units per sample** and, because ADPCM is
recursive, **encoding for one and decoding on the other costs 25 dB — the noise
comes out louder than the signal.** No encoder output can be committed to a
container before this is answered.
**It does not need a board.** MAME's x68000 HAS the chip: `:okim6258`, with the
68000 reaching it at **`$E92001` and `$E92003`** — both read out of the machine's
own program map by `tools/bench/probe_adpcm.lua`, not from folklore. What did
NOT work is feeding it from Lua: `probe_adpcm2.lua` / `probe_adpcm3.lua` swept
control 0..3 against PPI port C 0..15 and `-wavwrite` recorded silence
throughout (65.5). The gap is the register semantics, and the way to close it is
**from 68000 code with the IPL ROM's own channel-3 DMAC configuration**, which
`21_iplrom_dmac.py` already reads out of the ROM — the one ADPCM path in this
machine that is known-correct because Sharp wrote it. That is also the real
design, so it is not scaffolding.
**P6b. A CONTAINER WITH SOUND IN IT.** DLXP1 has no audio section. 65.3 is the
arithmetic a DLXP2 is built from and no byte of one is written. It waits on P6a,
because a container full of audio encoded against the wrong formula is 25 dB of
work to redo.
**E6. Container v2** — audio interleave, per-record index, scene table. Depends
on P6's answer and on P5's index.
@@ -794,6 +852,11 @@ B2 blanking ─┬─ NOT blanked ─> K3's DMAC-DIRECT player is the one: 54.9%
(64.2). Neither answer kills the branch and each
picks a different player.
B1 BURST rate (NEW, 64.2) ──> which of the two K3/K4 wins, if B2 blanks
P6 audio DONE bar one (65): encoder gated, cadence F=11/A=14, 589.6 KB/s
└─> P6a WHICH DELTA FORMULA? ─┬─ needs no board: MAME has :okim6258
worth 25 dB, so it is a │ at $E92001/$E92003 (65.5)
PRECONDITION not a tweak └─> then P6b, a container with sound in it
```
**Read that top-left branch as the project's live question.** Everything else
docs/STATUS.md
+171 -1
View File
@@ -1,4 +1,174 @@
# Status & next-session handoff — end of session 32 (2026-08-25)
# Status & next-session handoff — end of session 33 (2026-08-25)
## Session 33: audio gets an encoder, and the packed container's best property gets a bill
**Green light first and last: `./tools/bench/check.sh` was ALL GREEN before any
of this (`tmp/check_s33_start.log`) and ALL GREEN after** — the same stages, plus
one new one.
**FINDINGS 65. ROADMAP P6 is three-quarters done and has grown two sub-items.**
`tools/encoder/adpcm.py`, `tools/encoder/extract_audio.py`,
`tools/bench/verify_adpcm.py`, `tools/analysis/32_audio_wire.py`, and three
throwaway MAME probes kept because their failure is a finding
(`tools/bench/probe_adpcm*.lua`).
**Everything below is HOST ARITHMETIC plus one MAME introspection.** No board
ran, and the one MAME *experiment* attempted did not work — 65.5 says so.
**0. SESSION 32'S WORK WAS UNCOMMITTED AND ITS STATUS BLOCK HAD NO HANDOFF.**
Both fixed before anything new was written: the tree was re-gated ALL GREEN,
the handoff written, and the whole of session 32 committed as one change.
**1. THE ENCODER, AND THE GATE IT NEEDED INSTEAD.** ffmpeg has a DECODER for
this format (`adpcm_ima_oki`) and **no encoder**, so there is nothing to diff
against. What is gated instead is the decoder the encoder runs **inside its own
nibble search** — sample-exact against ffmpeg's over 4,268 nibbles. An encoder
that agrees with its own wrong decoder is what that catches.
| | |
|---|---|
| the window | 00223 @539.4 s, 10.000 s — **the same seconds as `tmp/fr_singe`** |
| encoded | 156,250 samples → **78,125 B, SNR 21.97 dB** |
| the rate | 78,125 B / 10 s = **7,812.5 B/s to the byte** — 52's figure, from the other end |
| nibble order | **HIGH FIRST, measured** — low-first mismatches ffmpeg on 3,285 of 4,268, and that is the gate's negative control |
| the level | disc peaks at **−13.4 dBFS**; normalising ×4.7 moves the SNR **21.97 → 21.97**. Not a lever |
**2. THE HEADLINE, AND IT IS A THREE-LSB DIFFERENCE THAT COSTS 25 dB.** The two
published delta formulas — ffmpeg's `((2*(n&7)+1)*step)>>3` and the OKI
datasheet's per-term truncation — differ **by at most 3 in 12-bit units**.
Encode for one, decode on the other:
| encoded | decoded | SNR |
|---|---|---:|
| `shift` | `shift` | **21.97 dB** |
| `shift` | `terms` | **−2.88 dB** |
**The noise comes out louder than the signal** (mean disagreement 78.2 against a
source RMS of 74.4). **ADPCM is recursive**, so a rounding difference does not
stay where it happens — the same property, one dimension down, that let 64.1's
gate audit 120 frames by comparing one. **Which formula the chip runs is
therefore a PRECONDITION on shipping any audio**, and it is ROADMAP P6a.
**3. AND THE PACKED CONTAINER'S BEST PROPERTY IS WHAT MAKES AUDIO COST.** A
packed record is 97 sectors and its address is `LBA0 + i*97` because a literal
frame's length is geometry — **no index, and none can be needed** (63, 64.1).
Audio at 15,625 Hz is **651.0417 B a slot**, and the `.0417` is the same
remainder the frame clock carries (54). Per-record audio makes records variable
length, which needs an index, which ends the format. So audio rides a **fixed
cadence** — every `F` frames, `A` whole sectors — and `(F, A)` is a rational
approximation to 15625/12288 from above:
| F | A | lump | padding | wire adds | held |
|---:|---:|---:|---:|---:|---:|
| **1 — the obvious one** | 2 | 1,024 B | **57.29%** | 12.00 KB/s | 2,048 B |
| **11 — the pick** | **14** | **7,168 B** | **0.09%** | **7.64 KB/s** | **14,336 B** |
| 81 — the floor | 103 | 52,736 B | 0.003% | 7.63 KB/s | 105,472 B |
**Wire: 582.0 KB/s silent → 589.6 KB/s with sound (+1.31%).** F=11 buys 57.2
points of padding for 12,288 B of RAM; the floor buys the last 0.09 of a point
for 91,136 B more, and K4 already wants 99,328 B.
**4. THE CODEC CONTAINER PAYS NONE OF IT.** It already has an index and already
has variable records, so it puts exactly 651.0417 B in record *i* and pads to
the sector it was padding to anyway: **zero audio padding**, 440.4 → 448.1 KB/s.
**That is the first price anyone has found for the packed branch's own
simplification.** 64's risk list said a deletion that large usually hides
something; this is the first thing it hid, and it is small — a cadence, a
padding fraction and 14,336 B.
**5. THE EXPERIMENT THAT DID NOT WORK, and the facts it did leave.** MAME's
x68000 **has the chip** — `:okim6258`, reached at **`$E92001` and `$E92003`**,
both read out of the machine's own program map rather than from folklore, with
the PPI at `$E9A000-$E9BFFF`. **Feeding it from Lua produced silence**: control
0..3 against port C 0..15, `-wavwrite` capture **0 of 567,360 samples non-zero**.
The gap is register semantics and it was **not guessed at further**. The way to
close it is from 68000 code with the IPL ROM's own channel-3 configuration,
which `21_iplrom_dmac.py` already reads out of the ROM.
**RISKS IN THIS SESSION'S RESULT, stated rather than left to be found:**
- **The encoder is greedy, not optimal.** Each nibble is chosen by exhaustive
search over all sixteen minimising THIS sample's error; a nibble also moves
the step index, so a locally worse choice can pay later. 21.97 dB is a floor
for this format, not its ceiling, and no lookahead was tried.
- **21.97 dB is against the 12-bit word**, not against the disc's 16-bit PCM,
and the source was already resampled and downmixed to mono by ffmpeg. The
downmix matrix is ffmpeg's default and was not chosen.
- **P6a is open and it is worth 25 dB**, so every byte `adpcm.py` has produced
is provisional.
- **Nothing has played.** No audio has left an emulated machine, let alone a
board.
## HANDOFF — start here
**THE TREE IS ALL GREEN**, session 33's stage included.
### The work, in the order it should be done
**1. P6a — WHICH DELTA FORMULA, and it needs no board.** It is worth **25 dB**
(65.2), so every byte the encoder has produced is provisional until it is
answered, and it is the cheapest open item in the project by a distance. MAME's
x68000 has the chip and this session found where it lives — `:okim6258` at
**`$E92001`/`$E92003`**, PPI at `$E9A000-$E9BFFF`, read out of the machine's own
program map (65.5).
**Do NOT retry the Lua feed.** It was swept — control 0..3 × port C 0..15 —
and recorded silence, and the gap is register semantics that nobody here should
be guessing at. **Do it from 68000 code with the IPL ROM's own channel-3 DMAC
configuration**, which `tools/analysis/21_iplrom_dmac.py` already reads out of
the ROM: dual address, 8-bit port, cycle steal without hold, external request.
That is the one ADPCM path in this machine that is known-correct because Sharp
wrote it, and it is also the real design — so it is not scaffolding, it is P6b's
transport arriving early.
The discriminating stream is already worked out: **twelve loud nibbles to climb
the step index, then every nibble in turn**, whose two reconstructions differ
first at sample 3 (523 against 522). Capture with `-wavwrite` and compare against
`adpcm.decode(nibs, 'shift')` and `adpcm.decode(nibs, 'terms')`. **Watch the
clipping** — the sequence above saturates at 2,047 within six samples, which
destroys discrimination; build one that alternates sign to hold the signal
mid-range while the step index climbs.
**2. THEN P6b — DLXP2, a container with sound in it.** 65.3 is all the
arithmetic: cadence **F=11, A=14**, `LBA(i) = LBA0 + i*97 + floor(i/11)*14`,
14,336 B held, wire 589.6 KB/s. It waits on P6a, because a container full of
audio encoded against the wrong formula is 25 dB of work to redo.
**3. WHAT IS LEFT OF P6 AFTER THAT** is the fourth quarter: 51.3's refill climb
with a second consumer through a real branch point. The slack table is in
`32_audio_wire.py`; nothing has been run.
### What is still BLOCKED, so it is not picked up by mistake
**K4 — the packed player that is on screen — is conditional on B2**, a board
question. If buffer mode does NOT blank, K3's player is already on screen the
whole slot and K4's paint is 27.3% of a frame spent on nothing. **E7, E4 and C1**
are parked (61.8), and **P4a's wiring** is parked with the ring K3 deleted.
**The hardware list is unchanged and is the user's**: B1 (sustained AND the
data-phase BURST rate, 64.2), B2 (blanking — the five-minute half), B3 (`#EXREQ`),
B4 (a byte write to a palette register).
### Risks that are OURS, not hardware
1. **P6a is open and it is worth 25 dB.** Everything in `adpcm.py`'s output is
provisional.
2. **The encoder is greedy.** Exhaustive per-sample search, no lookahead;
21.97 dB is this format's floor here, not its ceiling.
3. **Nothing has played.** No audio has left an emulated machine.
4. **The packed branch's simplification has now cost something once** (65.4).
It was predicted in the abstract and it was small. It may not be the only one.
### Reproducing this session
./tools/bench/check.sh # ALL GREEN
python3 tools/encoder/extract_audio.py 00223 tmp/au_singe.raw 15625 539.4 10.0
python3 tools/bench/verify_adpcm.py tmp/au_singe.raw
python3 tools/analysis/32_audio_wire.py tmp/packed_singe.dlxp
**WHAT IS NEXT.** P6a: ask the chip which formula it runs, from 68000 code.
---
## Session 32: the packed player runs, and the write window turns out to be the frame
tools/analysis/32_audio_wire.py
+227
View File
@@ -0,0 +1,227 @@
#!/usr/bin/env python3
"""WHAT DOES AUDIO DO TO THE CONTAINER? ROADMAP P6, the half that is not the bus.
python3 tools/analysis/32_audio_wire.py [packed.dlxp] [--audio tmp/au_singe.raw]
[--rate KB/s ...]
Session 20 (FINDINGS 52) closed the bus half of P6: a second DMA consumer at
7,812.5 B/s is 1.25%..1.48% of a frame, about 4% of what the decoder leaves, and
the 7.8 kB/s figure survived with a unit correction. ROADMAP P6 then says, in
as many words, that EVERYTHING ELSE in the item is open: extraction, an encoder,
the container interleave, and what a second stream does to `wire` and therefore
to `pipe - wire` and therefore to 51.3's refill climb.
This file is the container interleave and the wire. It is arithmetic over the
real container's real geometry -- no MAME run, no board.
THE THING THAT MAKES IT INTERESTING, and it is a property of DLXP1 rather than
of audio: **a packed container has no index and cannot have one.** A record's
address is `LBA0 + i*97` because a literal frame's length is geometry (FINDINGS
63, 64.1). Audio is a stream at a rate that has nothing to do with the frame
rate, so the naive interleave -- give record i the audio bytes belonging to slot
i -- makes records VARIABLE LENGTH, and the moment records are variable length
the format needs an index and stops being the format.
So the interleave has to be a FIXED CADENCE: every F frames, A whole sectors of
audio, placed between records. Then
LBA(i) = LBA0 + i*RECSEC + floor(i/F)*A
which is still two multiplies and a divide -- arithmetic, no index, nothing
walked -- and the only cost is that A*512 must be at least F frames' worth of
audio, so the padding is whatever A*512 exceeds it by. Choosing (F, A) is a
rational-approximation problem and the answer is NOT the obvious cadence.
"""
import argparse, os, sys
from fractions import Fraction
sys.path.insert(0, "tools/encoder")
sys.path.insert(0, "tools/analysis")
import buscost as B
from dlxp import DLXP, SECTOR
ap = argparse.ArgumentParser()
ap.add_argument("container", nargs="?", default="tmp/packed_singe.dlxp")
ap.add_argument("--audio", default="tmp/au_singe.raw",
help="raw s16le mono at the chip rate, from extract_audio.py")
ap.add_argument("--codec", default="tmp/rc_fr_singe_scsi_span.dlx",
help="the codec container, for the same arithmetic on the other branch")
ap.add_argument("--rate", type=float, nargs="*",
default=[453.6, 500.0, 582.0, 600.0, 650.0, 700.0],
help="explicit sustained delivery rates, KB/s")
a = ap.parse_args()
d = DLXP(a.container)
SLOT_S = 1.0 / d.fps
FRAME_CLK = B.CPU_HZ * SLOT_S if hasattr(B, "CPU_HZ") else 10_000_000 * SLOT_S
RECSEC = d.rec_bytes // SECTOR
print(f"""
=== THE STREAM =========================================================
The chip is an MSM6258V on an 8 MHz clock and it has three rates and no
others. Every budget in this tree is written against the first one.""")
RATES = {512: 15625.0, 768: 8_000_000/768, 1024: 7812.5}
print(f"\n {'divisor':>8} {'samples/s':>11} {'bytes/s':>10} {'B per 1/%d s slot' % d.fps:>19} exact?")
for div, hz in RATES.items():
bps = hz / 2
per = bps / d.fps
print(f" 8MHz/{div:<4} {hz:11,.1f} {bps:10,.1f} {per:19,.4f} "
f"{'yes' if per == int(per) else 'NO -- a remainder, like the frame clock (54)'}")
HZ = 15625.0
AU_BPS = HZ / 2 # 4 bits a sample, two samples to a byte
AU_FRAME = AU_BPS / d.fps # 651.0416... B, and the point is the dots
print(f"""
The shipping rate's per-slot figure is {AU_FRAME:,.4f} B and it is NOT an
integer -- 8 MHz / 512 / 2 / {d.fps} has a 12 in the denominator that 2**k
cannot clear. That is the same shape as FINDINGS 54's frame clock: what a
player carries is a remainder, not a count, and a container that rounds it
either drifts or underruns.""")
if os.path.exists(a.audio):
n16 = os.path.getsize(a.audio) // 2
secs = n16 / HZ
print(f"""
MEASURED, on the window this project gates everything on (00223 @539.4s,
{secs:.3f} s, tools/encoder/extract_audio.py):
{n16:,} samples -> {n16//2:,} B of ADPCM = {n16/2/secs:,.1f} B/s
which is {AU_BPS:,.1f} to the byte, so the rate is the rate.""")
print(f"""
=== THE INTERLEAVE, AND WHY THE OBVIOUS CADENCE IS THE WRONG ONE =======
A packed record is {d.rec_bytes:,} B = {RECSEC} sectors EXACTLY and its address is
arithmetic. Audio rides between records at a fixed cadence -- every F frames,
A whole sectors -- so that LBA(i) stays arithmetic. A must satisfy
A * {SECTOR} >= F * {AU_FRAME:,.4f} i.e. A/F >= {Fraction(int(AU_BPS*2), int(2*SECTOR*d.fps))} = {AU_FRAME/SECTOR:.9f}
and everything above that ratio is PADDING that the wire pays for and nothing
plays. Here is the whole small-F space, best A for each F:""")
target = Fraction(int(round(AU_BPS * 2)), 2 * SECTOR * d.fps) # sectors per frame, exact
rows, floor = [], None
for F in range(1, 241):
A = -(-(target.numerator * F) // target.denominator) # ceil(F * target)
have, need = A * SECTOR, F * AU_FRAME
waste = (have - need) / need
add = have / F * d.fps / 1024 # what the cadence puts on the wire, KB/s
rows.append((waste, F, A, have, need, add))
print(f"\n {'F':>4} {'A':>4} {'A*512 B':>10} {'needs':>12} {'padding':>9} {'waste':>7}"
f" {'wire adds':>10} {'player RAM':>11}")
seen = None
for waste, F, A, have, need, add in rows:
show = F <= 4 or seen is None or waste < seen - 1e-12
if seen is None or waste < seen: seen = waste
if show:
print(f" {F:4d} {A:4d} {have:10,} {need:12,.1f} {have-need:9,.1f} "
f"{100*waste:6.2f}% {add:9.2f} KB/s {have:9,} B")
best = sorted(rows)
w, F, A, have, need, add = best[0]
f1 = next(x for x in rows if x[1] == 1)
print(f""" THE FLOOR OF THAT SWEEP is F={F}, A={A}: {100*w:.3f}% padding, {add:.2f} KB/s of
wire for {AU_BPS/1024:.2f} KB/s of audio.
THE OBVIOUS CADENCE IS THE WORST ONE. F=1 -- one audio lump per record, which
is what "interleave the audio into the frame" means if nobody does the
arithmetic -- needs A={f1[2]} and costs {100*f1[0]:.1f}% padding: {AU_FRAME:,.1f} B rounded up to
{f1[3]:,}, so {f1[3]-AU_FRAME:,.1f} B of every record is nothing at all, and the wire pays
{f1[5]:.2f} KB/s for {AU_BPS/1024:.2f} KB/s of audio. That is {f1[5]-add:.2f} KB/s thrown away for
no reason but the cadence.
=== WHAT IT DOES TO THE WIRE ===========================================""")
vid_kbs = d.rec_bytes * d.fps / 1024
for label, cad in (("F=1 (one lump a record)", f1), (f"F={F} (the floor)", best[0])):
tot = vid_kbs + cad[5]
print(f" {label:26s} video {vid_kbs:7.1f} + audio {cad[5]:5.2f} = {tot:7.1f} KB/s "
f"({100*(tot/vid_kbs-1):+.2f}%)")
print(f"""
And this is what B1's acceptance test becomes. The packed container's
sustained requirement was {vid_kbs:.1f} KB/s SILENT (FINDINGS 61.5, 63) and it is
{vid_kbs + add:.1f} KB/s with sound. A literal frame's bitrate is geometry and cannot
be talked down; the audio on top of it is {add:.2f} KB/s and can only be talked down
by choosing a worse chip rate.""")
f11 = next(x for x in rows if x[1] == 11)
print(f"""
AND THE CADENCE HAS A SECOND PRICE, WHICH IS RAM. A cadence of F frames means
the player is holding F frames of audio, and holding it TWICE -- the channel
fills lump n+1 while the chip drains lump n, the same reason K4 needs two
record buffers (64.2). So the floor of the sweep is not the answer:
F={f1[1]:<3} {f1[3]:>7,} B a lump, {2*f1[3]:>7,} B held {100*f1[0]:6.2f}% padding {f1[5]:5.2f} KB/s
F={f11[1]:<3} {f11[3]:>7,} B a lump, {2*f11[3]:>7,} B held {100*f11[0]:6.2f}% padding {f11[5]:5.2f} KB/s <- the pick
F={F:<3} {have:>7,} B a lump, {2*have:>7,} B held {100*w:6.2f}% padding {add:5.2f} KB/s
F={f11[1]} buys {100*(f1[0]-f11[0]):.1f} points of padding for {2*f11[3]-2*f1[3]:,} B of RAM, and F={F} buys the
last {100*(f11[0]-w):.2f} of a point for {2*have-2*f11[3]:,} B more. On a machine where K4 already
wants 99,328 B for two record buffers, the second trade is not one.
=== THE ASYMMETRY: THE CODEC CONTAINER PAYS NONE OF THIS ===============""")
if os.path.exists(a.codec):
sys.path.insert(0, "tools/encoder")
from dlx import DLX
c = DLX(a.codec)
lens = c.record_lengths() if callable(getattr(c, "record_lengths", None)) else c.record_lengths
cwire = sum(lens) / len(lens) * c.fps / 1024
print(f""" {os.path.basename(a.codec)}: {c.nframes} records, index {'PRESENT' if c.has_index else 'absent'},
records already VARIABLE ({min(lens):,}..{max(lens):,} B, mean {sum(lens)/len(lens):,.0f}) and
sector-aligned since DLX5 (60.1). A container that already carries an index
and already has variable records can put EXACTLY {AU_FRAME:,.1f} B of audio in record i
and pad only to the sector it was going to pad to anyway -- so its audio
padding is not 57.3% and not 1.11%, it is ZERO, and its wire goes
{cwire:.1f} -> {cwire + AU_BPS/1024:.1f} KB/s ({100*(AU_BPS/1024)/cwire:+.2f}%).
THAT IS THE FIRST COST THIS PROJECT HAS FOUND FOR THE PACKED BRANCH'S OWN
SIMPLIFICATION. "A record's length is geometry, so there is no index and none
can be needed" (63, 64.1) is what makes the packed player a page of arithmetic
instead of a parser -- and it is exactly the property that makes a second
stream at an unrelated rate cost padding, a cadence, and a buffer. It is a
small cost ({f11[5]-AU_BPS/1024:.2f} KB/s at the pick, {2*f11[3]:,} B of RAM) and it is not zero, and
nothing in FINDINGS 61-64 predicted it.""")
else:
print(f" SKIPPED: no codec container at {a.codec}")
print(f"""
=== WHAT IT DOES TO SLACK (51.3) =======================================
Slack is ACCUMULATED out of pipe - wire, so a second consumer does not cost a
fixed amount -- it costs the accumulation rate, and what a branch point costs is
set by that (51.3, 55.4). Silent vs sounded, at explicit rates:
{'pipe':>8} {'silent':>14} {'sounded':>14} what a second of play banks""")
for kbps in a.rate:
s_sl, a_sl = kbps - vid_kbs, kbps - (vid_kbs + add)
def fmt(x): return f"{x:+8.1f} KB/s" if x >= 0 else f"{x:+8.1f} KB/s"
print(f" {kbps:8.1f} {fmt(s_sl):>14} {fmt(a_sl):>14} "
+ ("both starve" if a_sl < 0 and s_sl < 0
else "SOUND IS WHAT BREAKS IT" if s_sl >= 0 > a_sl
else f"{a_sl/s_sl*100:.0f}% of the silent rate" if s_sl > 0 else ""))
AUCLK_LO = AU_FRAME * B.ADPCM_CLK_BYTE_BEST
AUCLK_HI = AU_FRAME * B.ADPCM_CLK_BYTE_WORST
print(f"""
=== AND WHAT IT DOES TO THE FRAME (the half session 20 already closed) ==
{AU_FRAME:,.1f} B a slot at {B.ADPCM_CLK_BYTE_BEST}..{B.ADPCM_CLK_BYTE_WORST} clocks a byte (the IPL ROM's OWN channel-3
configuration, read out of the ROM by 21_iplrom_dmac.py, not chosen here) is
{AUCLK_LO:,.0f}..{AUCLK_HI:,.0f} clocks = {100*AUCLK_LO/FRAME_CLK:.2f}%..{100*AUCLK_HI/FRAME_CLK:.2f}% of a {SLOT_S*1000:.2f} ms slot.
That reproduces FINDINGS 52 exactly, which is the point of printing it.
THE INTERACTION 52 COULD NOT HAVE HAD is with 64.2's write window. A
DMAC-direct packed player holds the GVRAM window open for the whole data
phase, and an audio channel stealing the bus during that phase makes the phase
LONGER -- so audio does not merely cost clocks, it costs DARKNESS:
extra dark per slot = {100*AUCLK_LO/FRAME_CLK:.2f}%..{100*AUCLK_HI/FRAME_CLK:.2f}% of the slot, on top of
record/(burst x slot), which is already 1.0 at the wire
It is small against a dark fraction that is already 1.0, and it is not small
against K4's {100*227553/FRAME_CLK:.1f}% paint. For the CPU-painted player the audio steals
from the paint and not from the picture, which is the third time this session
the two players have ranked differently on a column that is not clocks.
""")
tools/bench/check.sh
+18
View File
@@ -735,4 +735,22 @@ else
echo " SKIPPED: no x68000 romset -- the player was not run"
fi
echo "--- session 33: AUDIO -- the encoder, and what it does to the wire (FINDINGS 65) ---"
# ROADMAP P6, everything in it except the bus half session 20 closed. The audio
# is the SAME WINDOW as the frames -- 00223 from 539.4 s for 10 s -- because an
# audio stream that is not the same seconds as the picture is not this project's
# audio, and a gate that lets the two drift apart would never say so.
[ -f tmp/au_singe.raw ] || python3 tools/encoder/extract_audio.py 00223 tmp/au_singe.raw 15625 539.4 10.0
# There is NO ffmpeg encoder for this format -- adpcm_ima_oki is decode-only --
# so the encoder cannot be checked against a reference. What is checked is that
# the decoder our encoder runs in its own loop IS ffmpeg's, sample for sample.
# An encoder that agrees with its own wrong decoder is the failure this catches.
python3 tools/bench/verify_adpcm.py tmp/au_singe.raw || exit 1
# And the container arithmetic. The interesting line is the padding: a packed
# record has no index BY DESIGN, so audio has to ride a fixed cadence, and the
# obvious cadence throws away a third of every audio sector.
python3 tools/analysis/32_audio_wire.py tmp/packed_singe.dlxp \
> tmp/audio_wire.log 2>&1 || { cat tmp/audio_wire.log; exit 1; }
grep -aE "^ ( 1| 11| 81) |THE FLOOR|F=1 |F=11|is ZERO|SOUND IS WHAT" tmp/audio_wire.log
echo "ALL GREEN"
tools/bench/probe_adpcm.lua
+28
View File
@@ -0,0 +1,28 @@
-- Ask MAME what the x68000's ADPCM device is, and where the 68000 reaches it.
-- FINDINGS 64.4 recorded that no MAME source tree is on this machine; this is
-- the way to ask the same question without one.
M = manager.machine
local done = false
SUB = emu.add_machine_frame_notifier(function()
if done then return end
done = true
print("[AD] === devices whose tag looks like an ADPCM chip ===")
for tag, dev in pairs(M.devices) do
local t = tag:lower()
if t:find("adpcm") or t:find("oki") or t:find("msm") or t:find("6258") then
print(string.format("[AD] DEV %-24s shortname=%s", tag, tostring(dev.shortname)))
end
end
local sp = M.devices[":maincpu"].spaces["program"]
print("[AD] === program map, $E90000..$EA0000 ===")
local ok, err = pcall(function()
for _, e in ipairs(sp.map.entries) do
if e.address_start >= 0xE90000 and e.address_start < 0xEA0000 then
print(string.format("[AD] MAP %08X-%08X", e.address_start, e.address_end))
end
end
end)
if not ok then print("[AD] MAP unavailable: " .. tostring(err)) end
print("[AD] done")
M:exit()
end)
tools/bench/probe_adpcm2.lua
+36
View File
@@ -0,0 +1,36 @@
-- Feed the x68000's OWN okim6258 a known nibble stream and let MAME record what
-- comes out, so that "which delta formula does the chip use" is a MEASUREMENT
-- and not a reading of source code that is not on this machine (64.4).
--
-- The feed is deliberately SLOW -- a byte every host frame, where real time
-- wants ~138 -- because the question is not the rate. A starved chip holds its
-- last sample, so the wave is a STAIRCASE of the reconstructed values, which is
-- exactly the sequence the two candidate formulas disagree about.
M = manager.machine
local sp = M.devices[":maincpu"].spaces["program"]
local CTRL, DATA = 0xE92001, 0xE92003
local CMD = tonumber(os.getenv("AD_CMD") or "1")
-- 12 loud nibbles to climb the step index, then every nibble in turn: the pairs
-- where the two formulas differ are all at step indices above the floor.
local nibs = {}
for i = 1, 12 do nibs[#nibs+1] = 7 end
for i = 0, 15 do nibs[#nibs+1] = i end
for i = 0, 15 do nibs[#nibs+1] = i end
local bytes = {}
for i = 1, #nibs, 2 do bytes[#bytes+1] = nibs[i] * 16 + nibs[i+1] end
local n, started = 0, false
SUB = emu.add_machine_frame_notifier(function()
n = n + 1
if n == 30 then
sp:write_u8(CTRL, CMD)
started = true
print(string.format("[AD] ctrl $%02X written to $%06X", CMD, CTRL))
elseif started and n > 30 and (n - 30) <= #bytes then
sp:write_u8(DATA, bytes[n - 30])
elseif started and (n - 30) == #bytes + 20 then
print("[AD] fed " .. #bytes .. " bytes = " .. #nibs .. " nibbles")
print("[AD] done")
M:exit()
end
end)
tools/bench/probe_adpcm3.lua
+24
View File
@@ -0,0 +1,24 @@
-- Sweep the PPI's port C -- which is where the X68000 puts ADPCM pan and the
-- chip's clock divider -- and feed a loud burst under each value, so that the
-- WAV says which value un-mutes the chip. Nothing here is assumed: the segment
-- boundaries are printed and the analysis reads the wave against them.
M = manager.machine
local sp = M.devices[":maincpu"].spaces["program"]
local CTRL, DATA, PPIC = 0xE92001, 0xE92003, 0xE9A005
local SEG = 40 -- host frames per segment
local n = 0
SUB = emu.add_machine_frame_notifier(function()
n = n + 1
if n <= 20 then return end
local k = n - 20
local seg = math.floor((k - 1) / SEG)
local off = (k - 1) % SEG
if seg > 15 then print("[AD] done"); M:exit(); return end
if off == 0 then
sp:write_u8(PPIC, seg)
sp:write_u8(CTRL, 1)
print(string.format("[AD] seg %2d portC=$%02X starts at host frame %d", seg, seg, n))
elseif off <= 30 then
sp:write_u8(DATA, 0x77) -- two loud positive nibbles
end
end)
tools/bench/verify_adpcm.py
+124
View File
@@ -0,0 +1,124 @@
#!/usr/bin/env python3
"""Gate tools/encoder/adpcm.py against the only independent decoder on this
machine: ffmpeg's `adpcm_ima_oki`.
There is no ffmpeg ENCODER for this format -- `adpcm_ima_oki` is decode-only --
so the encoder here cannot be checked against a reference implementation. What
CAN be checked, and is, is that the decoder our encoder runs in its own loop is
byte-for-byte the decoder that ships in ffmpeg. An encoder that agrees with its
own wrong decoder is exactly the failure this catches.
Usage: verify_adpcm.py [wav_or_raw12 ...]
"""
import os, struct, subprocess, sys, random, math
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "encoder"))
import adpcm
TMP = "tmp/adpcm_gate"
# The step table as it is printed in the OKI datasheet and in every
# implementation of this format. adpcm.py BUILDS its table from 16*1.1**k; if
# the two ever disagree, one of them is a typo and this says which.
CANON = [16,17,19,21,23,25,28,31,34,37,41,45,50,55,60,66,73,80,88,97,107,118,
130,143,157,173,190,209,230,253,279,307,337,371,408,449,494,544,598,
658,724,796,876,963,1060,1166,1282,1411,1552]
fails = []
def ck(ok, msg):
print(("OK " if ok else "FAIL ") + msg)
if not ok: fails.append(msg)
def ffmpeg_decode(data, rate=15625):
"""Decode packed OKI ADPCM through ffmpeg, by wrapping it in a WAV whose
format tag is 0x0010 (WAVE_FORMAT_OKI_ADPCM). Returns 16-bit samples."""
os.makedirs(TMP, exist_ok=True)
fmt = struct.pack("<HHIIHHH", 0x0010, 1, rate, rate, 1, 4, 0)
body = (b"WAVE" + b"fmt " + struct.pack("<I", len(fmt)) + fmt
+ b"data" + struct.pack("<I", len(data)) + data)
w = f"{TMP}/probe.wav"
open(w, "wb").write(b"RIFF" + struct.pack("<I", len(body)) + body)
raw = subprocess.check_output(
["ffmpeg", "-v", "error", "-i", w, "-f", "s16le", "-acodec", "pcm_s16le", "-"])
return list(struct.unpack("<%dh" % (len(raw) // 2), raw))
def snr_db(ref, got):
"""Signal-to-noise over the 12-bit sample word."""
num = sum(float(s) * s for s in ref)
den = sum((float(a) - b) ** 2 for a, b in zip(ref, got))
if den == 0: return float("inf")
return 10.0 * math.log10(num / den) if num else float("-inf")
print("--- the step table ---")
ck(adpcm.STEP == CANON, f"49 entries, built = published (16*1.1**k), {adpcm.STEP[0]}..{adpcm.STEP[-1]}")
print("--- our decoder vs ffmpeg's adpcm_ima_oki ---")
random.seed(1234)
nibs = ([7]*12 + [i % 16 for i in range(256)]
+ [random.randrange(16) for _ in range(4000)])
data = adpcm.pack(nibs)
ff = ffmpeg_decode(data)
ours = [v * 16 for v in adpcm.decode(adpcm.unpack(data, len(nibs)), "shift")]
ck(len(ff) == len(ours), f"sample count {len(ff)} = {len(ours)}")
ck(ff == ours, f"variant 'shift' is SAMPLE-EXACT vs ffmpeg over {len(nibs)} nibbles")
# NEGATIVE CONTROL. A gate that passes whatever it is handed proves nothing;
# reading the nibbles the other way round has to go red, or "high nibble first"
# is an assertion rather than a measurement.
lowfirst = [adpcm.unpack(data, len(nibs))[i ^ 1] for i in range(len(nibs))]
bad = [v * 16 for v in adpcm.decode(lowfirst, "shift")]
ndiff = sum(1 for a, b in zip(ff, bad) if a != b)
ck(ndiff > 0, f"low-nibble-first DISAGREES on {ndiff}/{len(ff)} -- so the order is measured, not assumed")
print("--- and the second variant is not the same decoder ---")
terms = [v * 16 for v in adpcm.decode(adpcm.unpack(data, len(nibs)), "terms")]
d = [abs(a - b) // 16 for a, b in zip(ff, terms)]
nd = sum(1 for x in d if x)
ck(nd > 0, f"variant 'terms' differs on {nd}/{len(d)} samples, max {max(d)} in 12-bit units"
" -- OPEN: which one the MSM6258 runs is unmeasured")
print("--- and getting the variant wrong is NOT a rounding error ---")
# THE MEASUREMENT THAT CHANGED THIS FROM A FOOTNOTE INTO AN OPEN ITEM. The two
# variants differ by at most 3 in 12-bit units PER SAMPLE, which reads like
# something nobody could hear. ADPCM is RECURSIVE -- the delta is added to a
# running predictor and the nibble also moves the step index -- so the
# disagreement does not stay where it happens. It is the same shape as the
# codec's temporal recursion (64.1), one dimension down.
if os.path.exists("tmp/au_singe.raw"):
raw = open("tmp/au_singe.raw", "rb").read()
pcm = struct.unpack("<%dh" % (len(raw) // 2), raw)
ref = [max(-2048, min(2047, x >> 4)) for x in pcm]
nib = adpcm.encode(ref, "shift")
same, cross = adpcm.decode(nib, "shift"), adpcm.decode(nib, "terms")
err = [abs(x - y) for x, y in zip(same, cross)]
print(f" encoded 'shift', decoded 'shift': SNR {snr_db(ref, same):6.2f} dB")
print(f" encoded 'shift', decoded 'terms': SNR {snr_db(ref, cross):6.2f} dB"
f" <- the noise is LOUDER THAN THE SIGNAL")
print(f" per-sample disagreement over {len(err):,} samples: max {max(err)}, "
f"mean {sum(err)/len(err):.1f} in 12-bit units")
ck(snr_db(ref, cross) < 0,
"a 3-LSB formula disagreement costs ~25 dB, because ADPCM is RECURSIVE")
else:
print(" SKIPPED: no tmp/au_singe.raw (tools/encoder/extract_audio.py)")
print("--- the encoder, through the gated decoder ---")
for path in (sys.argv[1:] or []):
raw = open(path, "rb").read()
if raw[:4] == b"RIFF":
raw = subprocess.check_output(["ffmpeg", "-v", "error", "-i", path,
"-f", "s16le", "-ac", "1", "-ar", "15625", "-"])
pcm16 = struct.unpack("<%dh" % (len(raw) // 2), raw)
src = [max(-2048, min(2047, s >> 4)) for s in pcm16]
nib = adpcm.encode(src, "shift")
packed = adpcm.pack(nib)
rec_ff = [v // 16 for v in ffmpeg_decode(packed)][:len(src)]
rec_us = adpcm.decode(nib, "shift")
ck(rec_ff == rec_us,
f"{os.path.basename(path)}: encoder's own reconstruction = ffmpeg's, {len(src)} samples")
print(f" {len(src)} samples, {len(packed)} B, SNR {snr_db(src, rec_us):.2f} dB "
f"(12-bit word; the source is already quantised to it)")
print("ADPCM GATE " + ("GREEN" if not fails else f"RED: {len(fails)} failed"))
sys.exit(1 if fails else 0)
tools/encoder/adpcm.py
+118
View File
@@ -0,0 +1,118 @@
#!/usr/bin/env python3
"""MSM6258 (OKI/Dialogic) 4-bit ADPCM -- encoder, decoder, and the fact that
there are TWO decoders and they are not the same one.
The X68000's ADPCM is an OKI MSM6258V clocked at 8 MHz, dividing to 15,625 /
10,417 / 7,812.5 samples a second, 4 bits each, two samples to a byte
(FINDINGS 52, buscost.ADPCM_SAMPLE_HZ). The sample word is 12 bits signed.
WHY THIS FILE HAS TWO DECODERS. Nothing in this repo can be trusted to say what
the chip does, and the two references available on this machine DISAGREE:
VARIANT 'shift' delta = ((2*(n&7) + 1) * step) >> 3
This is ffmpeg's `adpcm_ima_oki`, and `gate_vs_ffmpeg()`
reproduces it SAMPLE-EXACT, so it is not a reading of source
code -- it is a measurement of the decoder that ships.
VARIANT 'terms' delta = step/8 + (n&4 ? step : 0) + (n&2 ? step/2 : 0)
+ (n&1 ? step/4 : 0), each term truncated
This is the OKI datasheet's own form, the one an ADPCM chip
can actually build out of shifts and adds, and it is what
MAME's okim6258 is understood to compute. NOT VERIFIED HERE:
no MAME source tree is on this machine (FINDINGS 64.4).
They differ on 445 of 2,268 sampled nibbles, by up to 4 in 12-bit units --
small, and small is not zero. Which one the machine runs is an open question
with an experiment attached: MAME's x68000 HAS an okim6258, so it can be asked
rather than argued about.
Nibble order is HIGH NIBBLE FIRST within a byte -- measured, not assumed, by the
same gate: reading low-first mismatches ffmpeg on 1,728 of 2,268 samples.
"""
# The 49-entry OKI step table. floor(16 * 1.1**k) for k in 0..48 -- built rather
# than pasted, so a transcription slip is not one of the things that can be
# wrong here.
STEP = [int(16 * 1.1**k) for k in range(49)]
# The nibble magnitude's effect on the step index. Four quiet nibbles walk it
# down one, four loud ones walk it up by more.
INDEX_ADJUST = (-1, -1, -1, -1, 2, 4, 6, 8)
SAMPLE_MIN, SAMPLE_MAX = -2048, 2047 # the 12-bit DAC word
VARIANTS = ("shift", "terms")
def delta(nibble, step, variant):
"""The reconstruction step for one nibble, in 12-bit units."""
if variant == "shift":
d = ((2 * (nibble & 7) + 1) * step) >> 3
elif variant == "terms":
d = step // 8
if nibble & 4: d += step
if nibble & 2: d += step // 2
if nibble & 1: d += step // 4
else:
raise ValueError(f"unknown variant {variant!r}")
return -d if nibble & 8 else d
def decode(nibbles, variant="shift"):
"""Nibbles -> 12-bit signed samples. State is (signal, step index), both
zero at the start of a stream, which is what the chip resets to."""
signal, idx, out = 0, 0, []
for n in nibbles:
signal += delta(n, STEP[idx], variant)
signal = SAMPLE_MIN if signal < SAMPLE_MIN else (
SAMPLE_MAX if signal > SAMPLE_MAX else signal)
idx += INDEX_ADJUST[n & 7]
idx = 0 if idx < 0 else (48 if idx > 48 else idx)
out.append(signal)
return out
def encode(samples, variant="shift"):
"""12-bit signed samples -> nibbles.
The nibble is chosen by EXHAUSTIVE SEARCH over all sixteen, minimising the
reconstruction error of this sample. That is greedy rather than optimal --
a nibble also moves the step index, so a locally worse choice can pay later
-- but it is what a chip-matched encoder is expected to do and it costs
nothing offline. The decoder is run INSIDE the loop, so the encoder can
never drift away from what the decoder will reconstruct.
"""
signal, idx, out = 0, 0, bytearray()
for s in samples:
step = STEP[idx]
best, best_err = 0, None
for n in range(16):
v = signal + delta(n, step, variant)
v = SAMPLE_MIN if v < SAMPLE_MIN else (SAMPLE_MAX if v > SAMPLE_MAX else v)
err = (v - s) ** 2
if best_err is None or err < best_err:
best, best_err = n, err
signal += delta(best, step, variant)
signal = SAMPLE_MIN if signal < SAMPLE_MIN else (
SAMPLE_MAX if signal > SAMPLE_MAX else signal)
idx += INDEX_ADJUST[best & 7]
idx = 0 if idx < 0 else (48 if idx > 48 else idx)
out.append(best)
return bytes(out)
def pack(nibbles):
"""Nibbles -> bytes, HIGH NIBBLE FIRST. An odd count pads with a 0 nibble,
which is the quietest one the format has (delta = step/8)."""
n = bytes(nibbles)
if len(n) & 1:
n += b"\0"
return bytes((n[i] << 4) | n[i + 1] for i in range(0, len(n), 2))
def unpack(data, count=None):
out = bytearray()
for b in data:
out.append(b >> 4)
out.append(b & 15)
return bytes(out[:count] if count is not None else out)
tools/encoder/extract_audio.py
+45
View File
@@ -0,0 +1,45 @@
#!/usr/bin/env python3
"""Extract the audio of a Blu-ray window as mono PCM at an MSM6258 sample rate.
The video side of this window is tools/encoder/extract.py; the arguments mean
the same things and are meant to be given the same values, because an audio
stream that is not the same seconds as the frames is not this project's audio.
The disc is AC-3 5.1 at 48 kHz. The arcade original is MONO, so this downmixes
-- ffmpeg's default matrix, dialogue from the centre channel included -- and
resamples to the chip's rate. Nothing here shapes, gates or normalises the
level: what the ADPCM encoder is handed is what the disc has, so that the SNR
it reports is the codec's and not a gain stage's.
"""
import getpass, os, subprocess, sys
BDROM = os.environ.get("DLX_BDROM") or f"/media/{getpass.getuser()}/BDROM"
STREAM_DIR = f"{BDROM}/BDMV/STREAM"
# 8 MHz / {512, 768, 1024}. The chip has no other rates and 15,625 is the one
# every budget in this project is written against (FINDINGS 52).
RATES = {15625: 512, 10417: 768, 7813: 1024}
def extract(stream, out, rate=15625, start=None, dur=None):
if rate not in RATES:
raise SystemExit(f"{rate} is not an MSM6258 rate: {sorted(RATES)}")
src = f"{STREAM_DIR}/{stream}.m2ts"
cmd = ["ffmpeg", "-v", "error"]
if start is not None: cmd += ["-ss", str(start)]
if dur is not None: cmd += ["-t", str(dur)]
cmd += ["-i", src, "-vn", "-ac", "1", "-ar", str(rate),
"-f", "s16le", "-acodec", "pcm_s16le", out, "-y"]
subprocess.check_call(cmd)
n = os.path.getsize(out) // 2
print(f"{stream}: {n} samples @ {rate} Hz mono = {n/rate:.3f} s -> {out}")
return n
if __name__ == "__main__":
# extract_audio.py <stream> <out.raw> [rate] [start_s] [dur_s]
stream, out = sys.argv[1], sys.argv[2]
rate = int(sys.argv[3]) if len(sys.argv) > 3 else 15625
start = float(sys.argv[4]) if len(sys.argv) > 4 else None
dur = float(sys.argv[5]) if len(sys.argv) > 5 else None
extract(stream, out, rate, start, dur)