Files
Dragon-s-Lair-X68k/docs/STATUS.md
T
prosolis 8b5f51704c Take the climb to a real branch point, and find the cadence has a third column
FINDINGS 70, and it closes ROADMAP P6.  65.6 recorded that "the slack table is
here, but 51.3's refill climb with a second consumer through a real branch point
is not", and 67.6 repeated it verbatim two sessions later.
tools/analysis/36_branch_audio.py is that run: 51.3's surplus model, 56.3's 612
real branch points, 32's cadence arithmetic, both containers' headers and the
buffer constants of the player 68 measured, in one place.  No machine -- all
five were already numbers.

AUDIO IS 1.7% OF THE CODEC WIRE AND UP TO 3.30x OF THE CLIMB.  Slack accrues out
of pipe - wire, which is a small difference of two large numbers, so a second
consumer is not priced by its share of the wire -- which is how every budget in
this tree since FINDINGS 8 has quoted it.  At 451.4 KB/s (the container's own
zero-prefill pipe, 49.5) in a 256 KB ring the climb goes 6.70 s -> 22.11 s and
the branch points arriving under it go 482/612 (79%) -> 603/612 (99%).  At 488
it is 1.19x, at 600 it is 1.05x.  Correction to 56.4: it charged audio at
ratectl.AUDIO_KBPS = 7.8, which is 7,812.5 B/s in decimal kB; in binary KB the
figure is 7.6294, so the placeholder was 2.24% high and every column moved in
the flattering direction by less than one part in six hundred of the wire.

AND ON THE PACKED BRANCH THERE IS NO CLIMB AT ALL.  A record goes straight into
GVRAM, so the video lookahead is zero records and 56.4's alarming column is
vacuous when every frame arrives with less lookahead than the one before it.
Acceptance there is a per-frame deadline, not an average, which is why B1 has to
name a burst rate.  The only consumer on that branch with any lookahead is the
audio one: 1.833 s of sound held against 0.000 s of picture.  And at 589.6 KB/s
-- the acceptance figure this project quotes -- the sounded container's surplus
is exactly zero: it plays and never banks a byte, while the silent one climbs
its record in 6.38 s.

THE COST NOBODY HAD COUNTED.  A DLXP2 group is `lump k, then F records`, so lump
k is at a LOWER address than every record of its group but the first, and a seek
to record i finds its audio behind it.  Measured on the 409 within-container
seek targets of the arcade's own graph at the shipped F=11: mean 416.5 ms of
silence entering a branch, median 416.7, p90 750.0, worst 833.3, and only 36 of
409 land on a group boundary.  The other 203 -- the scene changes -- are free by
construction, because lump 0 sits between the header and record 0.  The
container's own start is the one branch point the cadence costs nothing at, and
it was the only one anybody had looked at.

SO THE CADENCE PICK IS REOPENED, on a third column.  32_audio_wire chose F=11 on
padding and RAM; F=1, the cadence it called the worst one, has no group to enter
off-boundary at all, and gives back 12,288 B of RAM for +4.36 KB/s of wire
(0.74% of the acceptance figure).  The alternative is a second read: lump k is
up to 503,808 B behind, so it is a separate command of 7,168 B -- 11.7 ms
against 416, 36x cheaper in time -- and src/player/packed.s starts PG_AK/PG_AKF
at lump 0 and has no audio seek path at all.  Nothing here decides it; the
deciding number is what one more SCSI command costs, and that is B1's.  The
content is not uniform mod F either: the measured mean runs 0.67x..1.23x of a
uniform assumption across the sweep, and the 1.00x at F=11 is a coincidence.

New check.sh stage, ~10 s, and it gates the STRUCTURE and deliberately not the
milliseconds: audio never shortens a climb, the shipped cadence strands most
within-container branches, F=1 strands none, and the lump read is an order of
magnitude cheaper than the silence it removes.

The one thing 70.3 and 70.4 rest on is a DESIGN ASSUMPTION and is printed in the
tool's own output: one container per SCENE (53, 55.1, 56.3).  One container per
sequence makes every seek land on frame 0 and empties both sections.

No encoder, player or container byte moved.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
2026-08-25 14:21:32 -07:00

4034 lines
221 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Status & next-session handoff — end of session 38 (2026-08-25)
## Session 38: the climb meets a branch point with sound on it, and the cadence has a third column
**FINDINGS 70.** ROADMAP P6's last and oldest item, named by 65.6 and repeated
verbatim by 67.6: 51.3's refill climb with a second consumer, through a real
branch point. `tools/analysis/36_branch_audio.py` puts five things that had
never been in the same room together — 51.3's surplus model, 56.3's 612 real
branch points, 32's cadence arithmetic, both containers, and the buffer
constants of the player 68 measured. It needs no machine: all five were already
numbers.
**THE HEADLINE. Audio is 1.7% of the codec wire and up to 3.30x of the climb.**
Slack accrues out of `pipe wire`, which is a small difference of two large
numbers, so a second consumer is *not* priced by its share of the wire — which
is how every budget in this tree since FINDINGS 8 has quoted it. At 451.4 KB/s
(the container's own zero-prefill pipe, 49.5) in a 256 KB ring the climb goes
**6.70 s → 22.11 s** and the branch points arriving under it go **482/612 (79%)
→ 603/612 (99%)**. At 488 it is 1.19x, at 600 it is 1.05x.
**AND ON THE PACKED BRANCH THERE IS NO CLIMB AT ALL.** The packed player puts a
record straight into GVRAM, so its **video lookahead is zero records** and
51.3's question has no answer there: 56.4's alarming column is vacuous when
every frame arrives with less lookahead than the one before it. Its acceptance
is a per-frame deadline, not an average, which is why B1 has to name a burst
rate. **The only consumer on that branch with any lookahead is the audio one**
1.833 s of sound held against 0.000 s of picture. And on the CPU-painted variant
the acceptance figure itself reads differently: **at 589.6 KB/s the sounded
container's surplus is exactly zero**, so it plays and never accumulates a byte,
while the silent one climbs its record in 6.38 s.
**THE COST NOBODY HAD COUNTED.** A DLXP2 group is `lump k, then F records`, so
lump *k* is at a LOWER address than every record of its group but the first. A
seek to record *i* finds its audio behind it and the next lump ahead carries
frame *(k+1)F*. Measured on the **409 within-container seek targets of the
arcade's own graph** at the shipped F=11: **mean 416.5 ms of silence entering a
branch, worst 833.3 ms, and only 36 of 409 land on a group boundary.** The other
203 — the scene changes — are free by construction, because lump 0 sits between
the header and record 0. **The container's own start is the one branch point the
cadence costs nothing at, and it is the only one anybody had looked at.**
**SO THE CADENCE PICK IS REOPENED**, on a third column. F=1 — the cadence
32_audio_wire called the worst one — has no group to enter off-boundary, and it
gives back **12,288 B of RAM** for **+4.36 KB/s of wire (0.74%)**. The
alternative is a second read: lump *k* is up to 503,808 B behind, so it is a
separate command of 7,168 B — **11.7 ms against 416 ms, 36x cheaper in time**
and `src/player/packed.s` has no audio seek path at all. **Nothing here decides
it; the deciding number is the extra command's overhead, and that is B1's.**
**What changed in the tree.** One new analysis file and one new ~10 s check.sh
stage that gates the STRUCTURE and deliberately not the milliseconds: audio
never shortens a climb, the shipped cadence strands most within-container
branches, F=1 strands none, and the lump read is an order of magnitude cheaper
than the silence it removes. No encoder, player or container byte moved.
## HANDOFF — start here
**THE TREE IS ALL GREEN**, session 38's stage included.
### The work, in the order it should be done
**1. THE AUDIO SEEK PATH (70.3), and it is now the cheapest thing that moves
something.** `PG_AK`/`PG_AKF` start at lump 0 and there is no way to enter a
group off-boundary. One extra read at `PG_ALBA0 + k*(F*PG_RECS + PG_CADA)` plus
a byte offset inside the lump is the whole of it, and until it exists the player
cannot branch with sound. It also turns 70.4's reopened cadence question back
into a non-question.
**2. THE BUFFERING DEPTH, which is still cheap.** Unchanged from session 37's
handoff: `PG_ANBUF` is 3 and `PG_APRE` is a mailbox. Two slots is one constant
and one run. 70.2 adds a reason to care — the audio buffer is the packed
branch's *only* buffer.
**3. THE SILENT-CLIP AND SHORT-AUDIO CASES (69.4).** Unchanged: a scene whose
audio is shorter than its frames (`00199`), and a scene with no audio track at
all (`00176`). Both are decided by what the lump count is computed from.
### What is still BLOCKED, so it is not picked up by mistake
**K4 — the packed player that is on screen — is conditional on B2**, a board
question. **E7, E4 and C1** are parked (61.8), and **P4a's wiring** is parked
with the ring K3 deleted.
**The hardware list is unchanged and is the user's**: B1 (sustained AND the
data-phase BURST rate, 64.2 — acceptance is **589.6 KB/s**, and 70.2 is the
sharpest statement yet of why the two rates are not interchangeable), B2
(blanking), B3 (`#EXREQ`), B4 (a byte write to a palette register), and session
34's fifth, play a known nibble stream on a real MSM6258V and record the line
out. **Session 38 adds a sixth question to B1 and it is small**: what one extra
SCSI command costs, because that is the number 70.4's cadence trade turns on.
### Reproducing this session
python3 tools/analysis/36_branch_audio.py --kbps 451.4 488.0 513.2 600.0 \
--ring 256 512 # ~10 s, the essay
python3 tools/analysis/36_branch_audio.py --gate # the check.sh stage
./tools/bench/check.sh # ALL GREEN
**WHAT IS NEXT.** The audio seek path, which is the first thing in this project
that the content has asked for rather than the hardware.
---
# Status & next-session handoff — end of session 37 (2026-08-25)
## Session 37: the audio level is measured off the whole disc, and the headroom is not worth buying
**FINDINGS 69.** ROADMAP P6, the item 66.3 reopened in session 34 and sessions
35 and 36 both deferred. It needed no new machinery on the machine and it did
need the whole disc: `tools/analysis/35_audio_level.py` reads every stream of
the unique scene footage (`00000`-`00201`, FINDINGS 32.1) through
`extract_audio.py`'s own chain and encodes windows of it with `adpcm.CHIP`.
**What was open.** The chip clamps its accumulator at 10 bits INSIDE the
recursion, so anything the encoder is handed above 511 (12-bit units) is
unreachable. The ten seconds this tree gates everything on peak at 435 of 511 —
it fits, and it fits by accident, because that window is 13.4 dBFS. Nothing
knew what the loudest passage of the game was.
**What it measures.**
| | |
|---|---:|
| audio on the game's own footage | 1,291.6 s = 21.5 min, 201 of 202 streams |
| **disc peak** | **946 of 2048 = 6.71 dBFS** (`00200` @ 2.11 s) |
| the clamp | 511 = 12.06 dBFS — the disc is **5.35 dB over** |
| samples above it | 687 of 20,182,000 = **0.0034%** |
| ...in | 402 events, **44.0 ms**, longest 0.90 ms, median run 1 |
**THE HEADLINE, and it is a negative: the level does not change.** Forty 2 s
windows drawn over the game's timeline, encoded at six gains: the disc's own
level (`s16 >> 4`, gain 1.0) has the best mean SNR (**22.03 dB**) and the best
median, and loses the worst-window column to 3 dB by **0.04 dB**. The gain that
guarantees zero clamping disc-wide (0.5402 = 511/946) costs **0.85 dB of mean
SNR across the whole game** to buy back **1.90 dB on the 2.11 s that clamp**.
The OKI step table's floor is a constant 16 and does not scale with the signal,
so attenuating walks the programme down toward a quantiser that stays put.
**And 66.3's mechanism does not survive a control.** The worry was that a
clamped accumulator is a wrong state the next nibble is applied to. Error after
a clamp run IS elevated ~5x — and so is the same window at a gain that never
clamps, read at the same indices, because those samples are simply LOUD. Worst
ratio against the control over 64 offsets: **1.28**, and the clamped encode's
whole-window mean |error| is **lower** (4.71 vs 5.05). The reason is that
`adpcm.encode` runs the chip's clamp inside its own sixteen-way search, so it
always knows the state the chip is in, saturated included. **The worry was
correct about the mechanism and aimed one layer too late**: an encoder clamping
at 12 bits while the chip clamps at 10 — what this tree shipped until session
34 — is exactly the divergence described, and 66 already closed it.
**What changed in the tree.** `--audio-gain` on `pack.py` (default 1.0), which
makes the level a named parameter with a measurement behind it instead of a
shift buried in a list comprehension; `pack.py` now prints the encoded window's
peak against the clamp and the count above it (gate window: **435 of 511, 0
clamped**). `tmp/packed_singe.dlxp` was rebuilt and is **byte-identical** to the
one session 36 ran off a volume, all 6,039,040 B. A new check.sh stage (~18 s)
gates the disc peak and the clamp census.
**Three rig facts recorded in 69.4**, because a shipping encoder meets all
three: `00176` has **no audio track at all** (ffmpeg exits non-zero — "silent
clip" has to be distinguishable from "extract failed"); `00199` is 61.31 s of
video with 1.25 s of audio, which is 60 s of the 75 s gap between the disc's
1,366.6 s of footage and its 1,291.6 s of audio, and a cadence-driven container
computes its lump count from the FRAME count; and 18 stream pairs share
duration, peak and RMS, of which 7 are byte-identical audio (25.0 s, all of them
3.5 s clips — the reused death animations).
## HANDOFF — start here
**THE TREE IS ALL GREEN**, session 37's stage included.
### The work, in the order it should be done
**1. THE REFILL CLIMB WITH A SECOND CONSUMER, through a real branch point**
(51.3, 55.4). Now the oldest open item in P6 and unchanged by this session: the
slack table is in `32_audio_wire.py`, nothing has been run, and session 36 gave
it the player that holds two streams at once. Pointing it at a scene change is
the work; nothing has to be built.
**2. THE BUFFERING DEPTH, which is cheap.** `PG_ANBUF` is 3 and `PG_APRE` is a
mailbox. Two slots is one constant and one run, and it is 7,168 B of a machine
that has 99,328 B of record buffer to find elsewhere (68.6).
**3. THE SILENT-CLIP AND SHORT-AUDIO CASES (69.4).** Neither is hard and
neither has an answer in the tree: a scene whose audio is shorter than its
frames, and a scene with no audio track. Both are decided by what the lump
count is computed from, and both are cheap to get wrong quietly, which is this
format's whole failure mode (67.4).
### What is still BLOCKED, so it is not picked up by mistake
**K4 — the packed player that is on screen — is conditional on B2**, a board
question. **E7, E4 and C1** are parked (61.8), and **P4a's wiring** is parked
with the ring K3 deleted.
**The hardware list is unchanged and is the user's**: B1 (sustained AND the
data-phase BURST rate, 64.2 — acceptance is **589.6 KB/s**), B2 (blanking), B3
(`#EXREQ`), B4 (a byte write to a palette register), and session 34's fifth,
play a known nibble stream on a real MSM6258V and record the line out. **Session
37 sharpens the fifth**: the 10-bit clamp this whole session is about is a
DRIVER SETTING — `x68k.cpp:1089` calls `set_outbits(OUTPUT_10BITS)` — not a chip
constant, so it is MAME's reading of the board. If real silicon runs 12-bit the
disc clears the clamp by 6.7 dB and the answer here is unchanged and cheaper.
### Reproducing this session
python3 tools/analysis/35_audio_level.py --gate # ~18 s
python3 tools/analysis/35_audio_level.py --survey 40 --ladder --recover
# ~7 min: the ladder, the disc-wide survey and the clamp-recovery control
./tools/bench/check.sh # ALL GREEN
**WHAT IS NEXT.** The refill climb with the second consumer, through a real
branch point.
---
# Status & next-session handoff — end of session 36 (2026-08-25)
## Session 36: the container is played, and holding the bus costs the audio
**Green light first and last: `./tools/bench/check.sh` was ALL GREEN before any
of this (`tmp/check_s36_start.log`) and ALL GREEN after
(`tmp/check_s36_end.log`)** — the same stages, plus one new one.
**FINDINGS 68. ROADMAP P6c is DONE.** `src/player/packed.s` (the audio path),
`src/player/dma.i` (`DM_HOOK`), `src/player/scsi.i`, `tools/encoder/adpcm.py`
(`decode_state`), `tools/bench/verify_packed_audio.py`,
`tools/bench/prep_packed.py`, `tools/bench/packed.lua`,
`tools/bench/packed_run.sh`, `tools/bench/check.sh`.
**NAME THE LAYER.** 68000 code fetched an interleaved DLXP2 off a real MB89352
volume, ran the video path against it, and fed the lumps to the **MSM6258 as
MAME models it** through the IPL ROM's own channel 3. The instrument is MAME's
`-wavwrite`. **No board ran.** What is new is that a byte of this container has
now been *heard* on some layer — 67.6 recorded that none had.
**1. IT PLAYS.** All **78,125 B** of the container's audio reached the chip, in
order, and every sample is exact against `adpcm.py` run with the four axes out
of the container's own header — **156,257 samples**, worst residual 1 count on
`signal = sample/8`. 11 of 11 lumps armed and fetched, 0 starves.
**2. THE HEADLINE, AND IT IS 64.3 ARRIVING ON THE AUDIO SIDE.** The chip has no
FIFO and **no starvation state**: when its channel counts out it goes on
decoding the byte its data register still holds. So the gap between a count-out
and the next arm is a **held nibble pair driving the predictor**, and it is in
the capture. Same container, same player, one bit of DCR:
| | stealing | held |
|---|---:|---:|
| service calls | 250,240 | 369 |
| boundaries with a seam | 4 of 10 | **10 of 10** |
| worst seam | 0.13 ms | **72.83 ms** |
| total replayed | 0.51 ms (0.005%) | **236.42 ms (2.31%)** |
| bytes wrong | 0 | 0 |
**The bytes are identical and the sound is not.** 673x fewer looks at the
channel, 463x the seam. And the reason stealing wins is `DM_HOOK`: **250,000 of
the 250,240 service calls came from inside dma.i's transfer wait**, which is the
only time a 68000 driving this video path has. Once a frame is a 90 ms seam by
construction, in either configuration.
**3. THE DRIFT IS MEASURED NOW.** `pg_apay` is 67.2's accumulator in a player,
and the capture prices what it avoided over the 10 lumps that carry a whole
group: 71,614 B against 71,680 B of lump space, **0.092%, 8.45 ms over 9.17 s ->
1.26 s over the game's 22.8 min. 67.2 predicted 1.25.** The accumulator ends at
1/24, where the arithmetic says.
**4. TWO BUGS, AND NEITHER IS VISIBLE FROM INSIDE THE PLAYER.** Clearing
`DM_BARV` does not unchain a channel — the register the channel obeys is OCR
bits 3-2 — and the symptom is POLL TIMEOUT on the lump *and every record after
it*, which reads like a disc problem. And the refill ran one lump too far ahead
and **overwrote the buffer channel 3 was reading**: 11 of 11 armed, 11 fetched,
78,125 B, no starve, and the sound wrong from 0.2 s in. That is 67.4's failure
mode on the audio side and it is why the gate is a WAV.
**5. THE RIG FACT THAT SHAPES THE GATE.** MAME's `okim6258::data_w` resets the
nibble select, and the DMA (7,812.5 B/s) and the stream (15,625 Hz) are
independent clocks — so a delivered byte is *usually* two nibbles and sometimes
one or three. Measured: **78,120 of 78,125 (99.994%)** were exactly two. So the
gate walks the stream **one delivered byte at a time**, and every stretched byte
in both runs was the last byte of a lump.
**RISKS IN THIS SESSION'S RESULT:**
- **Two slots was not tried.** Three lump slots and a prefill of two is what
ran; the minimum depth is unmeasured, so the RAM figure is an upper bound.
- **The seam milliseconds are the apparatus's**, set by how long the emulated
transport takes a record. The held/stealing RATIO is the finding.
- **Nothing has been listened to.** The capture is an instrument, not a listen.
- **The level is still open downward** (66.3) and the loudest passage on the
disc is still unmeasured. Unchanged since session 34.
- **The silicon is untouched.** All four axes are still MAME's.
## HANDOFF — start here
**THE TREE IS ALL GREEN**, session 36's stage included.
### The work, in the order it should be done
**1. THE AUDIO LEVEL, which 66.3 reopened and two sessions have now deferred.**
The 10-bit clamp is 12.1 dB below where the encoder was clamping and it is
INSIDE the recursion, so a scene that clips does not merely distort, it drives
the predictor. **Measure the loudest passage on the disc before choosing a
level.** This is the oldest open item in P6 and it needs no new machinery.
**2. THE REFILL CLIMB WITH A SECOND CONSUMER, through a real branch point**
(51.3, 55.4). The last quarter of P6 that is not P6c. The slack table is in
`32_audio_wire.py`; nothing has been run. Session 36 gives it the thing it was
missing — a player that actually holds two streams at once — so this is now a
matter of pointing it at a scene change rather than of building anything.
**3. THE BUFFERING DEPTH, which is cheap now.** `PG_ANBUF` is 3 and `PG_APRE` is
a mailbox. Two slots is one constant and one run, and it is 7,168 B of a
machine that has 99,328 B of record buffer to find elsewhere.
### What is still BLOCKED, so it is not picked up by mistake
**K4 — the packed player that is on screen — is conditional on B2**, a board
question. **E7, E4 and C1** are parked (61.8), and **P4a's wiring** is parked
with the ring K3 deleted.
**The hardware list is unchanged and is the user's**: B1 (sustained AND the
data-phase BURST rate, 64.2 — acceptance is **589.6 KB/s**), B2 (blanking), B3
(`#EXREQ`), B4 (a byte write to a palette register), and session 34's fifth:
play a known nibble stream on a real MSM6258V and record the line out. **Session
36 adds nothing to that list and strengthens the case for the fifth**: every
decoder axis the player now depends on is still MAME's.
### Reproducing this session
./tools/bench/check.sh # ALL GREEN
bash tools/bench/packed_run.sh tmp/packed_singe.dlxp
# run 5 is the audio gate, run 6 the held contrast
python3 tools/bench/verify_packed_audio.py tmp/packed_singe.dlxp \
tmp/packed_aud.wav tmp/packed_aud.json
**WHAT IS NEXT.** The level, and then the refill climb with the second consumer
that now exists.
---
# Status & next-session handoff — end of session 35 (2026-08-25)
## Session 35: the packed container gets sound, and the padding is drift
**Green light first and last: `./tools/bench/check.sh` was ALL GREEN before any
of this (`tmp/check_s35_start.log`) and ALL GREEN after
(`tmp/check_s35_end.log`)** — the same stages, plus one new one.
**FINDINGS 67. ROADMAP P6b is DONE.** `tools/encoder/dlxp.py` (DLXP2),
`tools/encoder/pack.py --audio`, `tools/analysis/34_packed_audio.py`,
`src/player/packed.s`, `tools/bench/prep_packed.py`, `tools/bench/packed.lua`,
`tools/analysis/30_packed_container.py`, `tools/bench/check.sh`.
**NAME THE LAYER.** The container is **host arithmetic**. What ran on the
emulated 68000 is its *video* consequence: `packed.s` now carries the third LBA
term and fetches 120 records out of an interleaved container off a real MB89352
volume, all 120 pixel-exact. **No audio byte has been played out of this
container on any layer.**
**1. THE FORMAT.** 64-byte header; from sector 1, groups of one `A`-sector audio
lump then `F` records. `record i = off_frm + i*rec + (i//F)*A*512`,
`lump k = off_aud + k*(F*rec + A*512)`. **Still no index and still none needed**,
which was the packed branch's whole claim and is the thing a second stream could
have ended. The lump goes **before** the group it feeds; 65.3's formula put it
after, and a stream is read forwards. On the 68000 it costs **six instructions
once a frame** and zero parsing. The four ADPCM axes ride in the header as
fields, not a version number.
**2. THE HEADLINE, AND 65.3 CALLED IT THE WRONG THING.** A lump is 7,168 B of
*space*; eleven frames of audio is **7,161.4583… B**, so the payload alternates
**7,161 / 7,162** and the rest is zero. A player that fed the chip the whole
lump — which is what "14 sectors every 11 frames" invites — runs **0.09% fast**,
and that is not waste, it is **drift**: 0.84 ms a group, **1.25 s of lip-sync
over the game's 22.8 minutes**. What a player carries is one accumulator,
`acc += 11*15625; n = acc//24; acc %= 24` — the same shape as `clock.i` and for
the same reason. **Third time this tree has met it** (54, 65.3, now).
**3. THE GATE, AND ITS THREE NEGATIVE CONTROLS.** Nothing parses a packed
container, so a lump one sector out does not fail, it paints. `34_packed_audio.py`
therefore (a) **partitions the whole file** — 131 spans, no overlap, no gap,
ending on the last byte — because a per-record read cannot see an off-by-one
that shifts everything after it; (b) asserts a **cadence-blind player** gets
exactly records 11..119 wrong, **109 of 120, and frames 0..10 identical either
way**, which is how such a bug survives a rig that checks frame 0; (c) **flips
each of the four ADPCM axes one at a time** — nibble order **31.99 dB**, delta
formula **24.86 dB**, clamp 0.00, accumulator 0.49 — so the header fields are
load-bearing rather than documentation. And the picture is gated against a
**silent control** built from the same frames: all 120 records byte-identical.
**4. THE WIRE IS THE PREDICTION.** 582.0 + 7.64 = **589.6 KB/s**, which is
65.3's figure to the tenth, because none of 66's four axes changes a byte count.
**RISKS IN THIS SESSION'S RESULT:**
- **Nothing has been heard.** The container has audio in it; no transport has
fed it to a chip, on any layer.
- **The lump buffer is not allocated anywhere.** 65.3 charges 14,336 B for
double-buffering the cadence and no player holds it.
- **Two channels have never run at once.** The audio DMA is 1.25%..1.48% of a
frame *in isolation* (52); the video channel holds the bus (64.2/64.3).
- **The level is still open downward** (66.3) and the loudest passage on the
disc is still unmeasured.
## HANDOFF — start here
**THE TREE IS ALL GREEN**, session 35's stage included.
### The work, in the order it should be done
**1. THE LAST QUARTER OF P6 — audio out of the container, on the machine.**
Everything it needs now exists and none of it has been joined up: the container
carries the bytes (67), the transport is `adpcm.i`'s channel-3 configuration
which worked first time (66.1), and the missing piece is **the lump buffer and
the remainder accumulator in a player**. Do the accumulator as 67.2 states it or
the run drifts 1.25 s over a game.
**2. AND THEN THE THING NEITHER HALF HAS MET: TWO CHANNELS AT ONCE.** The video
channel holds the bus and halts the 68000 (64.3 — it costs the frame clock half
its ticks *without the clock being able to tell*). An audio channel that has to
be serviced during that hold has never been run. This is the interaction 52
could not have had, and `32_audio_wire.py` names it: audio does not merely cost
clocks, it costs **darkness**.
**3. THE REFILL CLIMB WITH A SECOND CONSUMER** through a real branch point
(51.3, 55.4). The slack table is in `32_audio_wire.py`; nothing has been run.
**4. THE AUDIO LEVEL, which 66.3 reopened.** Measure the loudest passage on the
disc before choosing a level.
### What is still BLOCKED, so it is not picked up by mistake
**K4 — the packed player that is on screen — is conditional on B2**, a board
question. **E7, E4 and C1** are parked (61.8), and **P4a's wiring** is parked
with the ring K3 deleted.
**The hardware list is unchanged and is the user's**: B1 (sustained AND the
data-phase BURST rate, 64.2 — and the acceptance figure is now **589.6 KB/s**,
not 582.0), B2 (blanking), B3 (`#EXREQ`), B4 (a byte write to a palette
register), and session 34's fifth: play a known nibble stream on a real
MSM6258V and record the line out.
### Reproducing this session
./tools/bench/check.sh # ALL GREEN
python3 tools/encoder/pack.py tmp/fr_singe tmp/packed_singe.dlxp \
--nframes 120 --audio tmp/au_singe.raw
python3 tools/analysis/34_packed_audio.py # the DLXP2 gate on its own
**WHAT IS NEXT.** P6's last quarter: the container's own audio, out of a
channel, beside the video channel that holds the bus.
---
## Session 34: the chip is asked, and the encoder was wrong on four axes
**Green light first and last: `./tools/bench/check.sh` was ALL GREEN before any
of this (`tmp/check_s34_start.log`) and ALL GREEN after
(`tmp/check_s34_end.log`)** — the same stages, plus one new one.
**FINDINGS 66. ROADMAP P6a is DONE.** `src/player/adpcm.i`,
`src/player/adpcmgate.s`, `tools/bench/prep_adpcm.py`, `tools/bench/adpcm.lua`,
`tools/bench/adpcm_run.sh`, `tools/bench/verify_adpcm_chip.py`,
`tools/analysis/33_adpcm_model.py`.
**NAME THE LAYER.** 68000 code drove the transport; the thing measured is
**MAME 0.277's `okim6258` device model**, end to end through the machine's own
DMA path. No board ran. It settles the RIG and not the silicon.
**0. A MAME SOURCE TREE IS REACHABLE AND 64.4 IS STRUCK.** There is no tree on
disk, but this machine has network and the upstream tag fetches. Two sessions
reasoned about MAME's device model as an unopenable box when it was one `curl`
away. It designed the experiment; it did not become the result.
**1. THE TRANSPORT IS THE IPL ROM'S OWN AND IT WORKED FIRST TIME.** HD63450
channel 3, `DCR=$80 SCR=$04 MFC=DFC=$05 CPR=$01 DAR=$E92003 OCR=$32 CCR=$80`,
then `$02` to `$E92001` — every byte decoded out of the ROM by
`21_iplrom_dmac.py`. **839 B in 0.1074 s = 7,811.4 B/s** against the format's
7,812.5, `CSR=$E0 CER=$00 MTC=0`. **This is P6b's transport arriving early.**
Session 33's silence (65.5) had two ordinary causes and neither was a register
semantic worth guessing at: **the PPI's port C is an INPUT until control word
`$92` says otherwise**, so the pan/divider writes went to a latch nothing read;
and **`$01` is COMMAND_STOP**, which is what the port-C sweep wrote every time.
**2. THE HEADLINE, AND IT IS NOT THE AXIS 65 WAS WORRIED ABOUT.** Sixteen
candidate models, three capture decimations, a searched prologue: **exactly one
reproduces the capture over 1,678 consecutive samples, sample-exact**, and every
axis has a negative control.
| axis | `adpcm.py` default | the chip | wrong ALONE |
|---|---|---|---:|
| nibble order | high first | **LOW first** | **25.74 dB** |
| delta formula | `shift` | **`terms`** | 2.88 dB |
| clamp | 12-bit | **10-bit** | 0.00 dB (this window) |
| accumulator at PLAY | 0 | **2** | 0.45 dB |
| all four | | | **10.38 dB** against 21.97 |
**65.1's "nibble order: HIGH FIRST, measured" was true and was about ffmpeg**
the Dialogic VOX *file* convention — not about what a chip does with a byte in
its data register. `verify_adpcm.py` keeps ffmpeg's parameters on purpose; the
measured set is `adpcm.CHIP` and callers that encode FOR the machine pass it.
**3. THE 10-BIT CLAMP IS FREE HERE AND ONLY BY LUCK.** Encode for 12 bits or for
10 and the Singe window is **21.99 dB either way, zero samples on the clamp**
because it peaks at **435 of 511**, **1.4 dB of headroom**, on a 13.4 dBFS
passage. The clamp is **12.1 dB below** where the encoder was clamping and it is
INSIDE the recursion. **65.1's "the level is not a lever" survives downward and
is now wrong upward**, and **the loudest passage on the disc is unmeasured**.
**4. THE RIG FACT THAT COST THIS SESSION MOST OF ITS TIME.** The 8 MHz ADPCM
clock is CT1 in the YM2151's `$1B`, in a different device from the divider, and
MAME delivers it on the sound system's schedule rather than at the store. A
transfer started in the same breath as the setup plays its first ~17 ms at the
old clock; the capture's first ~130 samples come out in **exact identical
pairs** and no model fits a stream that changed rate part way through — which
reads exactly like a broken probe. `adpcmgate.s` settles ~100 ms and says why.
**RISKS IN THIS SESSION'S RESULT:**
- **The silicon is untouched.** All four values are MAME's. A real MSM6258V may
differ on any of them, and the two published references already disagree on
one. Cheap on a board: play a known nibble stream, record the line out.
- **Nothing has played as audio.** The capture is an instrument, not a listen.
- **The encoder is still greedy** — no lookahead, unchanged from 65.
- **The 10-bit clamp is a ceiling nobody has measured the disc against.**
## HANDOFF — start here
**THE TREE IS ALL GREEN**, session 34's stage included.
### The work, in the order it should be done
**1. P6b — DLXP2, a container with sound in it.** Unblocked, and its bytes are
now decided: encode with **`adpcm.CHIP`** (`terms`, low nibble first, 10-bit,
init 2), not with the defaults. 65.3 is all the arithmetic and none of it
moved — cadence **F=11, A=14**, `LBA(i) = LBA0 + i*97 + floor(i/11)*14`,
14,336 B held, wire **582.0 → 589.6 KB/s** — because a byte count does not care
which decoder reads the bytes.
**2. THE AUDIO LEVEL, which 66.3 reopened.** The encoder now has 12.1 dB less
room than it thought. Measure the loudest passage on the disc before choosing a
level; a scene that clips inside the recursion does not merely distort, it
drives the predictor.
**3. WHAT IS LEFT OF P6 AFTER THAT** is the fourth quarter: 51.3's refill climb
with a second consumer through a real branch point. The slack table is in
`32_audio_wire.py`; nothing has been run.
### What is still BLOCKED, so it is not picked up by mistake
**K4 — the packed player that is on screen — is conditional on B2**, a board
question. **E7, E4 and C1** are parked (61.8), and **P4a's wiring** is parked
with the ring K3 deleted.
**The hardware list is unchanged and is the user's**: B1 (sustained AND the
data-phase BURST rate, 64.2), B2 (blanking — the five-minute half), B3
(`#EXREQ`), B4 (a byte write to a palette register). **Session 34 adds a fifth
that is cheaper than any of them**: play a known nibble stream on a real
MSM6258V and record the line out, which settles 66.2's four axes on silicon.
### Reproducing this session
./tools/bench/check.sh # ALL GREEN
./tools/bench/adpcm_run.sh # the chip gate on its own
python3 tools/analysis/33_adpcm_model.py tmp/au_singe.raw
**WHAT IS NEXT.** P6b: a container with sound in it, encoded for the chip that
was just measured.
---
## Session 33: audio gets an encoder, and the packed container's best property gets a bill
**Green light first and last: `./tools/bench/check.sh` was ALL GREEN before any
of this (`tmp/check_s33_start.log`) and ALL GREEN after** — the same stages, plus
one new one.
**FINDINGS 65. ROADMAP P6 is three-quarters done and has grown two sub-items.**
`tools/encoder/adpcm.py`, `tools/encoder/extract_audio.py`,
`tools/bench/verify_adpcm.py`, `tools/analysis/32_audio_wire.py`, and three
throwaway MAME probes kept because their failure is a finding
(`tools/bench/probe_adpcm*.lua`).
**Everything below is HOST ARITHMETIC plus one MAME introspection.** No board
ran, and the one MAME *experiment* attempted did not work — 65.5 says so.
**0. SESSION 32'S WORK WAS UNCOMMITTED AND ITS STATUS BLOCK HAD NO HANDOFF.**
Both fixed before anything new was written: the tree was re-gated ALL GREEN,
the handoff written, and the whole of session 32 committed as one change.
**1. THE ENCODER, AND THE GATE IT NEEDED INSTEAD.** ffmpeg has a DECODER for
this format (`adpcm_ima_oki`) and **no encoder**, so there is nothing to diff
against. What is gated instead is the decoder the encoder runs **inside its own
nibble search** — sample-exact against ffmpeg's over 4,268 nibbles. An encoder
that agrees with its own wrong decoder is what that catches.
| | |
|---|---|
| the window | 00223 @539.4 s, 10.000 s — **the same seconds as `tmp/fr_singe`** |
| encoded | 156,250 samples → **78,125 B, SNR 21.97 dB** |
| the rate | 78,125 B / 10 s = **7,812.5 B/s to the byte** — 52's figure, from the other end |
| nibble order | **HIGH FIRST, measured** — low-first mismatches ffmpeg on 3,285 of 4,268, and that is the gate's negative control |
| the level | disc peaks at **13.4 dBFS**; normalising ×4.7 moves the SNR **21.97 → 21.97**. Not a lever |
**2. THE HEADLINE, AND IT IS A THREE-LSB DIFFERENCE THAT COSTS 25 dB.** The two
published delta formulas — ffmpeg's `((2*(n&7)+1)*step)>>3` and the OKI
datasheet's per-term truncation — differ **by at most 3 in 12-bit units**.
Encode for one, decode on the other:
| encoded | decoded | SNR |
|---|---|---:|
| `shift` | `shift` | **21.97 dB** |
| `shift` | `terms` | **2.88 dB** |
**The noise comes out louder than the signal** (mean disagreement 78.2 against a
source RMS of 74.4). **ADPCM is recursive**, so a rounding difference does not
stay where it happens — the same property, one dimension down, that let 64.1's
gate audit 120 frames by comparing one. **Which formula the chip runs is
therefore a PRECONDITION on shipping any audio**, and it is ROADMAP P6a.
**3. AND THE PACKED CONTAINER'S BEST PROPERTY IS WHAT MAKES AUDIO COST.** A
packed record is 97 sectors and its address is `LBA0 + i*97` because a literal
frame's length is geometry — **no index, and none can be needed** (63, 64.1).
Audio at 15,625 Hz is **651.0417 B a slot**, and the `.0417` is the same
remainder the frame clock carries (54). Per-record audio makes records variable
length, which needs an index, which ends the format. So audio rides a **fixed
cadence** — every `F` frames, `A` whole sectors — and `(F, A)` is a rational
approximation to 15625/12288 from above:
| F | A | lump | padding | wire adds | held |
|---:|---:|---:|---:|---:|---:|
| **1 — the obvious one** | 2 | 1,024 B | **57.29%** | 12.00 KB/s | 2,048 B |
| **11 — the pick** | **14** | **7,168 B** | **0.09%** | **7.64 KB/s** | **14,336 B** |
| 81 — the floor | 103 | 52,736 B | 0.003% | 7.63 KB/s | 105,472 B |
**Wire: 582.0 KB/s silent → 589.6 KB/s with sound (+1.31%).** F=11 buys 57.2
points of padding for 12,288 B of RAM; the floor buys the last 0.09 of a point
for 91,136 B more, and K4 already wants 99,328 B.
**4. THE CODEC CONTAINER PAYS NONE OF IT.** It already has an index and already
has variable records, so it puts exactly 651.0417 B in record *i* and pads to
the sector it was padding to anyway: **zero audio padding**, 440.4 → 448.1 KB/s.
**That is the first price anyone has found for the packed branch's own
simplification.** 64's risk list said a deletion that large usually hides
something; this is the first thing it hid, and it is small — a cadence, a
padding fraction and 14,336 B.
**5. THE EXPERIMENT THAT DID NOT WORK, and the facts it did leave.** MAME's
x68000 **has the chip**`:okim6258`, reached at **`$E92001` and `$E92003`**,
both read out of the machine's own program map rather than from folklore, with
the PPI at `$E9A000-$E9BFFF`. **Feeding it from Lua produced silence**: control
0..3 against port C 0..15, `-wavwrite` capture **0 of 567,360 samples non-zero**.
The gap is register semantics and it was **not guessed at further**. The way to
close it is from 68000 code with the IPL ROM's own channel-3 configuration,
which `21_iplrom_dmac.py` already reads out of the ROM.
**RISKS IN THIS SESSION'S RESULT, stated rather than left to be found:**
- **The encoder is greedy, not optimal.** Each nibble is chosen by exhaustive
search over all sixteen minimising THIS sample's error; a nibble also moves
the step index, so a locally worse choice can pay later. 21.97 dB is a floor
for this format, not its ceiling, and no lookahead was tried.
- **21.97 dB is against the 12-bit word**, not against the disc's 16-bit PCM,
and the source was already resampled and downmixed to mono by ffmpeg. The
downmix matrix is ffmpeg's default and was not chosen.
- **P6a is open and it is worth 25 dB**, so every byte `adpcm.py` has produced
is provisional.
- **Nothing has played.** No audio has left an emulated machine, let alone a
board.
## HANDOFF — start here
**THE TREE IS ALL GREEN**, session 33's stage included.
### The work, in the order it should be done
**1. P6a — WHICH DELTA FORMULA, and it needs no board.** It is worth **25 dB**
(65.2), so every byte the encoder has produced is provisional until it is
answered, and it is the cheapest open item in the project by a distance. MAME's
x68000 has the chip and this session found where it lives — `:okim6258` at
**`$E92001`/`$E92003`**, PPI at `$E9A000-$E9BFFF`, read out of the machine's own
program map (65.5).
**Do NOT retry the Lua feed.** It was swept — control 0..3 × port C 0..15 —
and recorded silence, and the gap is register semantics that nobody here should
be guessing at. **Do it from 68000 code with the IPL ROM's own channel-3 DMAC
configuration**, which `tools/analysis/21_iplrom_dmac.py` already reads out of
the ROM: dual address, 8-bit port, cycle steal without hold, external request.
That is the one ADPCM path in this machine that is known-correct because Sharp
wrote it, and it is also the real design — so it is not scaffolding, it is P6b's
transport arriving early.
The discriminating stream is already worked out: **twelve loud nibbles to climb
the step index, then every nibble in turn**, whose two reconstructions differ
first at sample 3 (523 against 522). Capture with `-wavwrite` and compare against
`adpcm.decode(nibs, 'shift')` and `adpcm.decode(nibs, 'terms')`. **Watch the
clipping** — the sequence above saturates at 2,047 within six samples, which
destroys discrimination; build one that alternates sign to hold the signal
mid-range while the step index climbs.
**2. THEN P6b — DLXP2, a container with sound in it.** 65.3 is all the
arithmetic: cadence **F=11, A=14**, `LBA(i) = LBA0 + i*97 + floor(i/11)*14`,
14,336 B held, wire 589.6 KB/s. It waits on P6a, because a container full of
audio encoded against the wrong formula is 25 dB of work to redo.
**3. WHAT IS LEFT OF P6 AFTER THAT** is the fourth quarter: 51.3's refill climb
with a second consumer through a real branch point. The slack table is in
`32_audio_wire.py`; nothing has been run.
### What is still BLOCKED, so it is not picked up by mistake
**K4 — the packed player that is on screen — is conditional on B2**, a board
question. If buffer mode does NOT blank, K3's player is already on screen the
whole slot and K4's paint is 27.3% of a frame spent on nothing. **E7, E4 and C1**
are parked (61.8), and **P4a's wiring** is parked with the ring K3 deleted.
**The hardware list is unchanged and is the user's**: B1 (sustained AND the
data-phase BURST rate, 64.2), B2 (blanking — the five-minute half), B3 (`#EXREQ`),
B4 (a byte write to a palette register).
### Risks that are OURS, not hardware
1. **P6a is open and it is worth 25 dB.** Everything in `adpcm.py`'s output is
provisional.
2. **The encoder is greedy.** Exhaustive per-sample search, no lookahead;
21.97 dB is this format's floor here, not its ceiling.
3. **Nothing has played.** No audio has left an emulated machine.
4. **The packed branch's simplification has now cost something once** (65.4).
It was predicted in the abstract and it was small. It may not be the only one.
### Reproducing this session
./tools/bench/check.sh # ALL GREEN
python3 tools/encoder/extract_audio.py 00223 tmp/au_singe.raw 15625 539.4 10.0
python3 tools/bench/verify_adpcm.py tmp/au_singe.raw
python3 tools/analysis/32_audio_wire.py tmp/packed_singe.dlxp
**WHAT IS NEXT.** P6a: ask the chip which formula it runs, from 68000 code.
---
## Session 32: the packed player runs, and the write window turns out to be the frame
**Green light first and last: `./tools/bench/check.sh` was ALL GREEN before any
of this (`tmp/check_s32_start.log`) and ALL GREEN after** — the same stages, plus
one new one.
**FINDINGS 64. ROADMAP K3 is done.** `src/player/packed.s` (2,898 B),
`tools/bench/packed.lua`, `tools/bench/prep_packed.py`,
`tools/bench/packed_run.sh`, `tools/bench/verify_packed.py`,
`tools/analysis/31_display_duty.py`. `tools/bench/mkvol.sh` grew a DLXP arm —
for a packed container the file **is** the volume, which is the format's own
claim arriving at the disc.
**Everything below is EMULATED**: MAME 0.277 `x68000`, `-bios ipl10`, stock
10 MHz / 2 MB, `-exp1 cz6bs1`, a real CHD volume. No real hardware ran.
**1. THE PLAYER WORKS, AND IT IS THE STRONGEST END-TO-END RESULT IN THE TREE.**
The 68000 brings up its own display (CRTC mode, both scroll register pairs,
priority `vc1 = $0002`, R20 bit 11), builds its own **193-entry** DMA chain,
keeps its own frame clock off V-DISP, and fetches every record itself with
`READ(10)`. The rig writes no picture byte, no palette entry and no CRTC
register.
| | |
|---|---|
| frames | **120 of 120 pixel-exact**, EVERY ONE COMPARED |
| both palette orders | palette FIRST and `--palette-last`, **120/120 each** |
| chain | `$E82000`/512 B then 192 rows of 256 B at the 1,024 B stride — one start |
| record `i` | LBA `1 + i*97`. Arithmetic. No index, no ring, no decoder |
The gate had to change shape: `verify_decode.py` checks the LAST frame and that
audits 120 because the codec is temporally recursive. **A packed frame is a
literal**, so `verify_packed.py` compares all of them, letterbox included.
**2. THE HEADLINE, AND IT IS NOT THE GOOD HALF.** The free-running run — which
is what a 12 fps player becomes once the transfer is longer than the slot —
reported a number no budget in this tree has a column for: **the write window
was open on 99.5% of the host frames.** Every frame was pixel-exact and almost none of them was
visible. It is arithmetic: a packed write needs R20 bit 11, buffer mode blanks
the layer, and a DMAC-direct player holds the window open for the whole data
phase — there is no second page to hide behind, because the packed layout spends
both.
dark fraction of a slot = record bytes / (DATA-PHASE rate x slot)
**The rate in that expression is the BURST rate, not the sustained one**, and
that is a third hardware number **B1 has no test for**. At the container's own
582.0 KB/s the dark fraction is **1.0**: every frame delivered, on time,
pixel-exact, and **none of them displayed**.
**3. AND IT REVERSES 61.5's RANKING.** A packed player that DMAs into RAM with
the window SHUT and paints with the measured `movem` blit (27.3% of a slot) is
on screen **72.7% of every slot at any rate**. The two are equal only at a
data-phase rate of **2,131 KB/s = 3.7x the wire**. Below that — every rate
anyone has proposed — **the player with the CPU in the loop is on screen
longer.** That is ROADMAP **K4**, and it costs 82.2% of a frame at the 9 clk/B
floor against 54.9%, plus 99,328 B of RAM for two record buffers.
**4. A HELD CHANNEL COSTS THE FRAME CLOCK HALF ITS TICKS, AND THE CLOCK CANNOT
TELL.** `clock.i` counts V-DISP interrupts; a held channel halts the 68000; the
MFP's pending bit is one bit. Held at 12 fps: **487 of 1,038 edges lost
(46.9%)**, and the player reported **zero late frames** — because `PACE` is
advanced by the ISR the channel stops from running, so the loss cancels out of
the comparison. The player believed 12 fps; the screen was at **6.37**. Only the
host's raster count contradicts it, and `packed_run.sh` gates on the difference
being non-zero. Stealing at half rate loses 0.6%. Held, the CPU went round its
transfer wait **120 times in 120 frames** — once each; it never executed during a
transfer.
**5. THE CHANNEL CONFIGURATION DOES NOT SET THE TRANSPORT'S TIME.** Free-running,
held and stealing delivered the same record within **0.5%** (90.27 vs 90.72 ms).
That figure is the APPARATUS — MAME's device models carry no transfer timing —
but the negative is real and no arithmetic could have given it: **a channel
configuration buys who owns the CPU, not when the picture appears.**
**RISKS IN THIS SESSION'S RESULT, stated rather than left to be found:**
- **Everything in 2 and 3 is conditional on B2**, and B2 is still unmeasured on
a board. What changed is the stakes: it now picks WHICH packed player, and
each answer makes the other one's design work wasted.
- **MAME's ~535 KB/s transport ceiling is not diagnosed.** It is not the DMAC
(both configurations agree) and not the CPU (held, it is halted throughout),
and no MAME source tree was on this machine to name it. It bounds nothing in
the arithmetic, which is geometry, but it is the reason the gate is paced at
half rate and that should not be mistaken for a rig limitation being worked
around.
- **The cadence sampler is quantised to one host frame** (17.6 ms). The MEAN
over 120 frames is not (±0.30 ms); the min/median/max are the sampler's.
## HANDOFF — start here
**THE TREE IS ALL GREEN**, session 32's stage included (`tmp/check_s33_start.log`,
ALL GREEN, ~6 min).
### The work, in the order it should be done
**1. NOT K4 — NOT YET.** ROADMAP K4 is written and priced (82.2% of a frame at
the 9 clk/B floor, 99,328 B of RAM, on screen 72.7% of every slot at any rate),
and it is **conditional on B2**, which is a board question nobody here can
answer. If buffer mode does NOT blank, K3's player is already on screen the
whole slot and K4's paint is 27.3% of a frame spent on nothing. 61.7 wrote that
rule for the codec; it applies to the branch that replaced it. **Do not open
K4 until B2 comes back.**
**2. WHAT IS NOT BLOCKED IS P6 — AUDIO.** It is the largest unpriced item left
in the project and it is on M3's critical path, which is the completion target.
The bus half closed in session 20 (52: 1.25%..1.48% of a frame, and the unit
error was caught) — **everything else in it is open**: extraction, an encoder,
the container interleave, and what a second stream does to `wire` and therefore
to 51.3's refill climb. It is independent of B2 in the part that matters: both
packed containers and the codec carry the same audio.
Three things make it a session rather than a chore:
- **No ffmpeg ENCODER exists for this format.** `adpcm_ima_oki` is a decoder
only, so the encoder is ours to write and ours to gate.
- **The two available references DISAGREE, and it was MEASURED not recalled**
(session 33, `tmp/adpcm_probe/`). ffmpeg's `adpcm_ima_oki` is
`((2*delta+1) * step) >> 3`, 12-bit signal clipped to [-2048, 2047], output
x16, **high nibble first** — reproduced sample-exact over 2,268 nibbles. The
OKI datasheet form truncates per term (`step/8 + step + step/2 + step/4`,
selected by nibble bits) and differs on **445 of those 2,268 samples, by up to
4 in 12-bit units**. Which one the machine runs is not a matter of taste, and
**MAME has an `okim6258` in the x68000 — so it can be asked.**
- **DLXP1 has no room for it.** A packed record is 97 sectors EXACTLY because
49,664 B is what a literal frame costs; audio does not fit in the slack,
because there is none. What audio does to the packed record's geometry is a
container question, not an encoder one.
### What is PARKED, so it is not re-derived
**E7, E4 and C1** (61.8), and **P4a's wiring** for the video path (K3 deletes the
ring from it). Do not delete any of it — B2 is unanswered.
### Risks that are OURS, not hardware
1. **A chained transfer has never run back to back at 12 fps and now it has**,
120 times, pixel-exact — but only at HALF rate, because at full rate there is
nothing to snapshot (64.2). The gate's pacing is a consequence of the finding.
2. **The packed player deletes `ring.i`, `xfer.i` and most of `stream.s`.**
Unchanged: a simplification that large usually hides something, and it has
now run clean for 120 frames without finding out what.
3. **MAME's ~535 KB/s transport ceiling is undiagnosed** (64.4). It bounds
nothing in the arithmetic, which is geometry.
4. **B2 sits over the branch and now picks WHICH PLAYER**, not how much headroom
one has.
### Reproducing this session
./tools/bench/check.sh # ALL GREEN
bash tools/bench/packed_run.sh tmp/packed_singe.dlxp # all four runs
python3 tools/analysis/31_display_duty.py
**WHAT IS NEXT.** P6: audio, from the disc to a byte count that the container
has to carry.
---
## Session 31: the packed container, and the palette that buys 2.31 dB has a price
**Green light first and last: `./tools/bench/check.sh` was ALL GREEN before any
of this (`tmp/check_s31_start.log`) and ALL GREEN after** — the same stages, plus
one new one.
**FINDINGS 63.** Mostly **host arithmetic** over the Blu-ray's own frames; the
one rendering check is **px68k's real `x68k/gvram.c`**, linked headless the way
`tools/bench/c68k` links its CPU core. No MAME run was needed and no real
hardware ran. 47.4 — does buffer mode blank the layer — is still the board
question, and this session made it **bigger**.
**THE HEADLINE. ROADMAP K2 is done.** `tools/encoder/dlxp.py` is the format,
`pack.py` the encoder, `tools/analysis/30_packed_container.py` the gate and the
re-derivation, `tools/bench/gvpack/verify_dlxp.py` the second-emulator render.
| | |
|---|---|
| record | **49,664 B = 97 sectors EXACTLY**, 512 B palette + 49,152 B picture |
| index | **none, and none can be needed** — a packed record's length is geometry |
| wire | **582.0 KB/s**, exactly what 61.9 predicted |
| encode | **3.3 s** for 120 frames, against ~55 s for the codec |
| render | **index-exact on px68k, with the harness computing no interleave** |
**1. THE RE-DERIVATION SESSION 30 ASKED FOR, and it had TWO debts not one**
(63.3). 61.9's 34.08 dB was PIL's free 256-colour MEDIANCUT; the unnamed second
debt was that **every PSNR this project has quoted — 29.19, 31.33, 34.08 — is
RGB888, upstream of the `GGGGGRRRRRBBBBBI` word (23.3)**, and a packed record
carries that word and nothing else.
| | RGB888 | **GRB555** |
|---|---:|---:|
| CODEC, the gate container (440.4 KB/s) | 29.07 | **28.72** |
| 256c scene palette — the codec's CEILING | 31.32 | **30.79** |
| PACKED, 254c SCENE palette *(control)* | 31.32 | **30.79** |
| **PACKED CONTAINER, 254c PER-FRAME** | **34.05** | **33.10** |
61.9 survives the real builder: **34.05 against 34.08**. The layout's two
reserved entries cost **+0.0003 dB**. The GRB555 word costs 0.53 dB and costs
every row, so it moves no comparison.
**2. THE CONTROL IS THE FINDING UNDER THE HEADLINE.** A packed container with a
SCENE palette lands **exactly on the codec's ceiling**, as it must. So the whole
+2.31 dB the packed branch has over that ceiling is **the per-frame palette and
nothing else** — not the packing, not literal frames, not 254 colours.
**3. 62.5 IS PRICED, AND THE ORDER IS A WASH** (63.4). Palette first 20.32 dB,
palette last 20.33 — 0.01 apart, so the choice can be made on other grounds, and
`dlxp.py` makes it a container flag rather than an assumption.
**4. AND THE MISMATCH IS NOT A WASH.** **231 of 256 palette entries change every
frame (90%)**, and a picture under the neighbouring frame's palette is **12.8 dB
worse**. It is a wipe, not a flash, and the transfer is 55.2% of a frame slot —
so at 12 fps a colour-scrambled region is on screen for roughly half of every
slot, forever. **B2 stopped being a headroom question and became a picture
question again**: if buffer mode does not blank, it decides not just which
player exists (61.6) but **which packed container** does. The codec never had
this exposure — its palette is scene-constant, so its tear is old picture
against new picture, never old colours against new ones.
**5. THE FALLBACK IS ALREADY IN THE ENCODER.** `pack.py --scene-palette
--no-palette`: **30.79 dB, zero churn, nothing to mismatch, +2.07 dB on the
shipping codec as the display renders both, and 576.0 KB/s instead of 582.0.**
The per-frame palette is now a **priced** +2.31 dB rather than a free one.
## HANDOFF — start here
**THE TREE IS ALL GREEN.** Session 31's work is a commit of its own.
### The work, in the order it should be done
**1. K3 — END TO END, OFF THE DISC.** Scene setup — palette, page-1 X-scroll
384, priority `vc1 = 0x0002`, R20 bit 11, the 384 zeroed words a row and the
letterbox rows — then **one chained DMA a frame** out of a DLXP1 record, with
the palette as the 193rd or 1st array entry (63.4 says either). Gate it
pixel-exact over the same 120-frame window the decoder is gated on. The
container is `tmp/packed_singe.dlxp` and `check.sh` rebuilds it every run.
**2. RUN BOTH PALETTE ORDERS while K3 is being built.** It is a flag
(`--palette-last`), the containers already exist, and 63.4 bounds the cost but
cannot decide it — dB over a whole frame is not what an eye sees in a wipe.
### What is PARKED, so it is not re-derived
**E7, E4 and C1** (61.8), and **P4a's wiring** for the video path. Do not delete
any of it — B2 is unanswered and 48.1's prior leans against packing.
### Risks that are OURS, not hardware
1. **A chained transfer has never run back to back at 12 fps.** Unchanged since
session 30: run 9 was seven entries, once. A frame is 193 and a second is
twelve frames.
2. ~~34.08 dB is PIL's MEDIANCUT~~ **CLOSED, 63.3: 34.05 dB against this
project's own builder, and the hardware word is charged on top.**
3. **The packed player deletes `ring.i`, `xfer.i` and most of `stream.s` from
the video path.** A simplification that large usually hides something.
4. **B2 sits over all of it, and 63.4 raised what it decides.** B4 still sits
over the palette entry specifically, with a small blast radius.
### Reproducing this session
./tools/bench/check.sh # ALL GREEN
python3 tools/encoder/pack.py tmp/fr_singe tmp/packed_singe.dlxp --nframes 120
python3 tools/analysis/30_packed_container.py tmp/packed_singe.dlxp
python3 tools/bench/gvpack/verify_dlxp.py tmp/packed_singe.dlxp 0 --controls
**WHAT IS NEXT.** K3: end to end, off the disc.
---
# Status & next-session handoff — end of session 30 (2026-08-25)
## Session 30: one channel start paints a whole frame, and the palette has no handler to be wrong about
**Green light first and last: `./tools/bench/check.sh` was ALL GREEN before any
of this (`tmp/check_s30_start.log`) and ALL GREEN after** — the same stages, with
three new runs inside the DMA gate.
**FINDINGS 62.** **Emulated** — MAME 0.277, `x68000 -bios ipl10`, `-exp1 cz6bs1`.
No real hardware ran. 47.4 — does buffer mode blank the layer — is still the
board question and is still open, and **this session opened a second one.**
**FIRST, THE UNRECORDED WORK WAS COMMITTED.** Sessions 28 and 29 were sitting in
a dirty tree with the last commit being session 27's. They are one commit now
(`1be428c`), because their edits share files and could not be split honestly
after the fact. Session 30's own work is a commit of its own.
**THE HEADLINE. ROADMAP K1 is done and the answer is the good one: a DMA channel
writes the palette registers at `$E82000`, and one array-chained start crosses
from device registers into GVRAM.** So a packed frame is **one channel start**:
a 193-entry array — the palette, then 192 picture rows at the 1,024 B line
stride — walked by the channel with the 68000 halted from the first byte to the
last. `dmagate.s` runs 7, 8 and 9; `dma_run.sh` gates all three; `check.sh` runs
it.
| run | result |
|---|---|
| **7. DMA → `$E82000`, bus held** | **512 B byte-exact in 256 register words**, read back out of the registers by the 68000 |
| **8. the SAME transfer aimed at RAM** | byte-exact at `$2C000`, and **256 of 256 palette words still poison** |
| **9. ONE array-chained start, palette + 6 rows** | **2,048 B byte-exact across BOTH kinds of destination**, `MAR` at `+5,376` |
**1. THE DESTINATION IS POISONED NOW, and it should have been all along** (62.1).
Runs 46 wrote into RAM that was zero and GVRAM that was stale, against a record
that is mostly pad — "it matches the disc" was a weaker claim than it read as.
`dg_poison` puts `$A500|i` in palette word *i* first, and the host **counts**
whether that actually discriminates rather than assuming it: 511 of 512
positions differ from the disc's bytes, and the gate refuses a run under 500.
**2. THE CONTROL IS AN ATTRIBUTION CONTROL, and it is a different shape from run
5's** (62.2). There is no mode bit here to point at. The same transfer aimed
20 KB away makes two claims at once: the bytes land at `$2C000`, and the palette
still reads poison — so what reached `$E82000` in run 7 was decided by the
channel's `MAR`. The second half is also the positive control on the readback
path, because the poison it finds is what the CPU wrote.
**3. THE ARRAY IS SCENE-CONSTANT** (62.3). The row bases never change: the
packed layout spends both 256-colour pages — page 0 the low byte of a word,
page 1 the high — so there is no page to flip into. The 1,158 B array is built
once at scene setup and started once a frame. **What is still on the CPU per
frame is the channel start and the READ(10)**, and neither is priced here: say
"no per-frame PAINT work", not "no per-frame CPU work".
**4. AND THE NEW BOARD QUESTION, B4** (62.4). **MAME cannot be asked this
one.** `x68k.cpp:817` maps the palette to `palette_device::read16/write16`,
`emupal.cpp:417` forwards to `memory_array::write16`, and `memarray.h:75` is a
plain `COMBINE_DATA` — RAM that honours `mem_mask`, with **no handler that could
refuse a byte write**. That is a different bound from 61.2's: GVRAM has a real
256-colour arm, which is why run 5 could find a mechanism to fail on. What a
real X68000 palette register does with a byte write is **UNMEASURED — not
folklore, an absence.** B4 is the cheapest hardware item in the project: write
`$A5` to `$E82000`, `$5A` to `$E82001`, read the word back. A negative costs
0.28% of a frame and nothing else, which is why it sits below B2.
**5. ONE DESIGN CHOICE IS FILED, NOT ANSWERED** (62.5). Palette FIRST or 193rd
is visible on screen — old rows under the new palette, or new rows under the old
one — for one paint either way. It is moot if buffer mode blanks (B2). It is
named so the final player's order is a decision and not an accident of which run
happened to be written first.
## HANDOFF — start here
**THE TREE IS ALL GREEN AND COMMITTED.** Two commits: `1be428c` (sessions 28+29,
unrecorded until now) and session 30's own.
### The work, in the order it should be done
**1. K2 — A PACKED CONTAINER.** Simpler than a coded one: quantise to **254**
colours (index 0 is the transparency key, black at 255 — 47.2), interleave
column *i* with *i+128*, emit a per-frame palette, sector-align the record. No
VQ, no rate control, no mode map, no `lam`. `encode.py` already sector-aligns
(DLX5) and `prep_frame.py --pack-transparent` already emits the exact pixel
layout. **Re-derive 61.9's 34.08 dB against `vq.scene_palette` / `H.build`
here** — the measured figure is PIL's MEDIANCUT and is a direction, not the
player's number (risk 2 below).
**2. K3 — END TO END, OFF THE DISC.** Scene setup — palette, page-1 X-scroll
384, priority `vc1 = 0x0002`, R20 bit 11 — then **one chained DMA a frame**,
now including the palette entry K1 just established. Gate it pixel-exact over
the same 120-frame window the decoder is gated on.
### What is PARKED, so it is not re-derived
**E7, E4 and C1** (61.8), and **P4a's wiring** for the video path: a DMAC-direct
packed player has no ring at all. Do not delete any of it — B2 is unanswered and
48.1's prior leans against packing.
### Risks that are OURS, not hardware
1. **A chained transfer has never run back to back at 12 fps.** Unchanged by
this session: run 9 was seven entries, once. A frame is 193 and a second is
twelve frames.
2. **34.08 dB is PIL's MEDIANCUT**, not `vq.scene_palette` / `H.build`.
3. **The packed player deletes `ring.i`, `xfer.i` and most of `stream.s` from
the video path.** A simplification that large usually hides something.
4. **B2 sits over all of it**, and now B4 sits over the palette entry
specifically — with a known, small blast radius.
### Reproducing this session
./tools/bench/check.sh # ALL GREEN, ~5 min
bash tools/bench/dma_run.sh # runs 1-10, with controls
**WHAT IS NEXT.** K2: the packed container.
---
# Status & next-session handoff — end of session 29 (2026-08-25)
## Session 29: the decoder-free packed player fits the budget the codec misses
**Green light first and last: `./tools/bench/check.sh` was ALL GREEN before any
of this and ALL GREEN after**, plus one new stage.
**FINDINGS 61.** **Emulated** — MAME 0.277, `x68000 -bios ipl10`, and
`-exp1 cz6bs1` for the transport runs. No real hardware ran. 47.4 — does buffer
mode blank the layer — is still the board question and is still open.
**THE HEADLINE.** At the 9 clk/B dual-address floor, which 59.2 established is
the only configuration this machine can be shown to run:
| architecture | B/frame | % of a 12 fps frame at W=9 | KB/s |
|---|---:|---:|---:|
| **CODEC, the shipping design** | 37,585 | **110.4% — MISSES** | 440.4 |
| **decoder-free PACKED, DMAC device→GVRAM** | 49,152 | **55.2% — FITS** | 576.0 |
| **decoder-free PACKED, CPU-painted** | 49,152 | **81.6% — FITS** | 576.0 |
**Decoding 37,585 bytes costs 109% of a frame; NOT decoding 49,152 costs 54%.**
44.7 said this in advance on a cost model that has since been thrown away — "the
codec is not there to save CPU, it is there to save the wire" — and the measured
model agrees and goes further. **The question is now entirely a medium question.**
**1. THE PACKED PAINT IS MEASURED, and 47.6.1's assumption is retired.**
`blit.s` V8 is V1 with 128 words a row instead of 256. **227,553 clocks, 27.3%
of a frame** — 51.0% of V1, and **100.8% of V3, the unpacked WRITE-ONLY floor**.
The packed blit costs what the unpacked path pays to write alone, source read
thrown in free. V1 came out at 53.6% again, unmoved since session 9, which is
what licenses quoting V8 next to it.
**2. A CHANNEL FILLS GVRAM OFF THE DISC, AND WALKS THE LINE STRIDE ITSELF.**
Three new runs in `dmagate.s`, all gated: bit 11 SET → 2,048 B byte-exact, half
into page 1 and half into page 0, CPU never executed; bit 11 CLEAR → **457 bytes
lost, every one at an EVEN offset and not one ODD byte harmed**; and **array
chained, 8 rows at the 1,024 B line stride from ONE start.** 47.6.2 is answered.
**3. THE FIRST CUT OF THAT TEST COULD NOT HAVE FAILED.** It OR-ed bit 11 onto
whatever the IPL left, and the IPL leaves `$0B16` — bit 11 already set, COL
`%11`, which writes whole words either way. It passed and proved nothing. Both
runs now write R20 outright and differ in exactly one bit.
**4. AND THE CONTROL'S SIGNATURE IS A PLACE, NOT A COUNT.** "Half the bytes must
differ" turned the gate RED on a correct run: `gvram_w` ignores `mem_mask` in the
masked arm, so the high half keeps stale GVRAM, and where the record is pad the
stale half matches by coincidence — 567 of 1,024 did. The claim the mechanism
actually makes is that odd bytes survive and only even ones are lost.
**5. THE CODEC CANNOT HAVE THE PACKING — 47.6.4 CLOSED, both ways.** A 4×4 block
owns four bytes at stride 2. Writing them one at a time (V9) is **97.9% of a
frame against V4's 76.6% — 28% DEARER, for no saving on the wire at all.**
Pairing the blocks 128 columns apart (V10) gets the `movem` back at 38.5%, and
pays in the mode map: **SKIP falls from 66.3% of blocks to 46.1% of pairs**, so
1.60× as many blocks are painted — about **+60% on the bytes**, against an E7
target that needs them **down 35%**.
**6. WHAT B1 AND B2 NOW DECIDE.** 576 KB/s sustained, fixed, with no lever —
against 453.6 KB/s (gate container, zero prefill), 327 KB/s (E7's target) and
folklore that says 0.71.7 MB/s with no benchmark behind it. And under MAME's
blanking reading the **cheap architecture is the dark one**: a CPU-painted packed
player is black for 27.3% of a frame, the cheaper DMAC-direct one for 30113%,
because the black interval is the paint and the CPU's transport lands in RAM
where bit 11 is irrelevant.
**7. AND THE PICTURE IS BETTER, WHICH NOBODY HAD ASKED (61.9).** The codec is
**capped at 31.33 dB** — every codeword is an index into the scene palette — and
delivers 29.19. A literal frame has no codebooks, so per-frame palettes are legal
and it simply **IS 34.08 dB: +4.89 dB on the shipping container, +2.75 dB past a
ceiling the codec cannot cross**, for +1.0% on the wire and 0.28% of a frame.
**The packed player is not a quality compromise bought with clocks. It is better
on both, and its whole cost is on the wire.**
**ENCODER WORK IS PARKED (USER DECISION).** The handoff first said this was no
reason to stop codec work. It does not survive its own arithmetic: that rested on
576 KB/s against **E7's 327, which does not exist**, when the codec that exists is
440 KB/s and 110.4% of a frame — a 1.31x gap, not 1.76x. Packed needs two facts;
the codec needs E7 to succeed (two of three levers already measured inert, 60.4
and 60.5) AND the medium AND a ship at ~100% of the frame with no margin. **E7 and
E4 are both parked** — E4 too, because `H.build`'s k-means builds VQ codebooks and
a literal player has no VQ — and C1 follows E4. **The codec stays on disk and
nothing is built on it**, because 48.1's prior leans against packing and if
buffer mode blanks it is the only path left (48.3). That is inventory, not work.
---
## HANDOFF — start here
**THE TREE IS ALL GREEN** (`tmp/check_s29_handoff.log`, and four times before it this session), and **DIRTY WITH TWO
SESSIONS IN IT.** Session 28's work was never committed either, so `git status`
shows 26 modified files spanning both sessions plus one new file. The last
commit is session 27's (`8800d8f`, "Price the transport against the frame").
**Nothing here is half-finished — it is unrecorded, which is a different
problem.** Commit before starting anything, or the next green light cannot be
attributed to a session.
**THE DIRECTION CHANGED (USER DECISION, session 29): build the DECODER-FREE
PACKED player, keep the codec on disk and build nothing on it.** The reasoning is
61.8; the numbers that forced it are 61.4 (55.2% of a frame against the codec's
110.4%) and 61.9 (+4.89 dB, and the codec is *capped* 2.75 dB below it).
### The work, in the order it should be done
**1. CAN A DMA CHANNEL WRITE THE PALETTE REGISTERS AT `$E82000`? (61.9, open)**
The cheapest item and the one with the largest structural payoff. If the palette
takes a byte-wide DMA the way GVRAM does in buffer mode, it becomes a **193rd
array-chain entry** and ONE channel start paints a whole frame — palette
included, CPU halted throughout, no per-frame CPU work in the video path at all.
It is the same shape as the three probes that worked this session: another run in
`src/player/dmagate.s`, host-side byte comparison in `tools/bench/dma.lua`, an
assertion in `tools/bench/dma_run.sh`, **and a negative control** — see the traps
below, this is exactly where session 29 nearly shipped a vacuous pass.
**2. A PACKED CONTAINER.** Simpler than a coded one, not harder: quantise to 254
colours, interleave column *i* with column *i+128*, emit a per-frame palette,
sector-align the record. No VQ, no rate control, no mode map, no `lam`.
`encode.py` already sector-aligns (DLX5) and
`tools/bench/prep_frame.py --pack-transparent` already emits the exact pixel
layout 47.2 verified pixel-exactly on both emulators.
**3. END TO END, OFF THE DISC.** Scene setup — palette, page-1 X-scroll **384**,
priority `vc1 = 0x0002`, R20 bit 11 — then one chained DMA a frame. Gate it the
way the existing decoder is gated: pixel-exact over the same 120-frame window
against `dlx.py`'s reconstruction.
### What is PARKED, so it is not re-derived
**E7, E4 and C1** (61.8). E4 is included on purpose: `H.build`'s k-means builds
**VQ codebooks**, and a literal player has no VQ. **P4a's wiring** — the channel
behind `ring.i`'s `XF_*` mailbox — is parked *for the video path* too: a
DMAC-direct packed player has **no ring at all**. Do not delete any of it; B2 is
unanswered and 48.1's prior leans against packing.
### Risks that are OURS, not hardware
1. **A chained transfer has never run back to back at 12 fps.** 61.2's run 6 was
eight rows, once. A frame is 192 entries and a second is twelve frames.
2. **34.08 dB is PIL's MEDIANCUT**, not `vq.scene_palette` / `H.build`. The
direction is measured; re-derive the number against the builder that ships it
before quoting it as the player's.
3. **The packed player deletes `ring.i`, `xfer.i` and most of `stream.s` from the
video path.** A simplification that large usually hides something.
4. **B2 sits over all of it.** Items 1 and 2 survive a bad answer; item 3 does
not.
### Traps this session stepped in, both caught by the tree's own rules
**A test that could not fail.** The first GVRAM run OR-ed R20 bit 11 onto
whatever the IPL left — and the IPL leaves `$0B16`, bit 11 **already set**, COL
`%11`, which writes whole words either way. It passed and proved nothing. Both
runs now write R20 outright and differ in exactly one bit.
**An assertion that was right about the mechanism and wrong about the number.**
"The masked control must lose half the bytes" turned the gate RED on a correct
run: `gvram_w` ignores `mem_mask` in the masked arm, so the high half keeps stale
GVRAM, and where the record is pad it matches the disc by coincidence — 567 of
1,024 did. **The mechanism's signature was a PLACE, not a count**: odd bytes
survive, only even ones can be lost. That is what the gate asserts now.
### Reproducing this session
./tools/bench/check.sh # ALL GREEN, ~5 min
python3 tools/analysis/29_packed_player.py tmp/rc_fr_singe_scsi_span.dlx
bash tools/bench/dma_run.sh # runs 4-6, with controls
python3 tools/analysis/18_text_plane_16col.py # the PSNR rows of 61.9
`check.sh` gained one stage that runs `blit.lua` and gates the ORDER of the paint
costs, not their values — V8 < V1, V9 > V4, V10 < V4 — because a tree where any
of those flipped has a different answer to 44.7 and should say so out loud.
**WHAT IS NEXT.** Item 1 above: the palette-register DMA probe.
---
# Status & next-session handoff — end of session 28 (2026-08-25)
## Session 28: the container agrees with the medium, and two encoder levers are not levers
**Green light first and last: `./tools/bench/check.sh` was ALL GREEN before any
of this and ALL GREEN after** — and after a full re-encode of the gate
container, which is the first time this tree has replaced it.
**ROADMAP's RE-ENCODE BUNDLE (all four items under P2) is DONE. FINDINGS 60.**
**Emulated** — MAME 0.277 and px68k's C68K core. No real hardware ran.
**1. DLX5 — RECORDS ARE SECTORS.** `encode.py` pads every record to 512 B and
the scene header with it, so the frame stream starts on a sector boundary and
every record inside it does. `dlx.py` reads `DLX5`, and `record_lengths()` is
now the ONE place the alignment rule lives — it replaced four hand-copied
`4 + n + (-(4+n) % 4)` expressions in the analysis tools.
| gate container | DLX4 | **DLX5** |
|---|---:|---:|
| records starting on a sector boundary | 3/120 | **120/120** |
| off the disc, for 4,488,577 B of record | 4,548,608 (+1.34%) | **4,510,208 (+0.48%)** |
| into the ring | 4,488,588 | **4,510,208** |
| window or bounce copy | 0 PIO / +5 clk/B DMA | **0, and a channel can run it** |
The two byte counts are now the SAME NUMBER and check.sh gates on that
identity, both read out of the container rather than written into the script.
**2. THE CONSUMER HAD TO BE TOLD, and the ring's own audit caught it.**
`stream.s` released up to the last byte it READ, rounded to 4; under DLX5 that
strands up to 511 B of pad per record. `RD_PTR MISMATCH after frame 0` on the
first run. The release rounds to `RECALN` (geom.i) now. Every frame still
decoded pixel-exact while the free-space arithmetic was wrong, which is the
point: alignment is a contract with two sides.
**3. RESERVED BLACK (23.4, open since session 5).** The scene palette is 255
picture colours with index 0 held at (0,0,0), so the letterbox is true black and
`I = 0` falls out of `pack_palette`'s existing rule with no special case.
**0.04 dB** on the Singe window; the palette ceiling does not move.
**4. `--spans all` AS THE DEFAULT: MEASURED, AND REFUSED.** 267.9 KB/s / 28.88
dB / 2 frames late at `need`, against 448.2 KB/s / 29.07 dB / 1 late at `all`.
**+67% of the wire for +0.19 dB and one frame of 120**, against a container the
frame budget already says is 35% too big. `need` stays the default; the GATE
container keeps `all`, because it is a fixture and not a recipe.
**5. JOINT SPAN/LAM (E3): implemented, and it is a NO-OP.** `--joint-spans`
hands the span pass's freed bytes back to the lam search and re-spans. All four
cells of `{need,all} x {greedy,joint}` are BYTE-IDENTICAL, and so is
`--rc-floor open`. The reason is structural: **`lam` never leaves its floor on
any of 120 frames at either floor the encoder offers** (44.3), so there is
nothing to spend the freed bytes on. Kept, defaulted off.
**6. THE APPARATUS LIED SILENTLY.** MAME 0.277 served the compressed CHD's OWN
FILE BYTES as sector data — the buffer after READ(10) at LBA 0 was byte-for-byte
`dlxdisk.chd`'s first 4,096 bytes, starting `MComprHD` — while every READ
reported success and `chdman verify` passed. Not size, not geometry, not the
codec: the DLX5 image fails compressed at the working volume's exact length and
with a conventional geometry, and passes `-c none`. `mkvol.sh` builds
uncompressed now. **Only the per-byte comparison in `scsi.lua` could see it**; a
gate checking status and length would have been green all session.
**7. THE BUDGET BARELY MOVED, which is what a precondition should do.**
Decode 68.5% → **68.6%**, headroom 6.74 → **6.69 clk/B**, `W`=5 total 92.2% →
**92.4%**, E7's target 328 → **327 KB/s**, B1's zero-prefill requirement 451.4 →
**453.6 KB/s**. Every conclusion in 59.7 survives. `15_bus_occupancy.py` now
charges the **padded** record, because the disc carries the pad whether or not a
frame contains it.
**8. TWO CACHES WOULD HAVE TESTED THE WRONG ARTEFACT.** `16_span_roundtrip.py`
and `13_cpu_ratectl.py` cache `H.build` in a pickle keyed on the frames
directory alone; `H.build` gained an option this session. Both now store the
build parameters with the model and rebuild on a mismatch.
**WHAT IS NEXT.** P4a's wiring: the DMA channel behind `ring.i`'s `XF_*` mailbox
in place of `xfer.i`'s PIO loop, which 59.4 blocked on exactly this
precondition. `xfer.i`'s sector arithmetic already degenerates correctly —
`SC_WSKIP` is 0 on every record now — so what changes is which loop moves the
bytes. `dma_run.sh`'s windowed-read refusal becomes a negative control.
---
# Status at end of session 27 (2026-08-24)
## Session 27: the DMAC drives the data phase, and auto-request is charged by time
**Green light first and last: `./tools/bench/check.sh` was ALL GREEN before any
of this and ALL GREEN after**, plus one new stage.
**ROADMAP P4a is DONE at the transport level. FINDINGS 59.** **Emulated**
MAME 0.277, `x68000 -exp1 cz6bs1 -ramsize 2M`. No real hardware ran.
**1. THE CHANNEL DRIVES THE DATA PHASE.** `src/player/dma.i` programs HD63450
channel 1 and `src/player/dmagate.s` reads the same 2,048 B at LBA 1000 three
ways, with the host comparing all three against its own copy of the image:
| | bytes | MTC one instruction after START | CPU trips round its wait loop |
|---|---|---|---|
| PIO, the path 58.2 measured | byte-exact | — | — |
| **DMA, bus HELD** (DCR `$00`, OCR `$81`) | **byte-exact** | **0 of 2048** | **1** |
| **DMA, cycle STEALING** (DCR `$80`, OCR `$80`) | **byte-exact** | 2048 of 2048 | 426 |
Both channels reported `CSR = $E0`, `CER = $00`, `MTC = 0`, `MAR` exactly
`+2048`.
**2. THE EVIDENCE NEVER READS `$EA0015`, which is the whole design.** 57.3 said
it could not: with the DMAC's OWN asserted, MAME cannot tell a CPU-driven byte
at that address from a DMAC-driven one. So the discriminator is **the CPU's own
progress** — MTC sampled by the instruction *after* the one that starts the
channel. Held, the whole transfer happened between two instructions. **The gate
was checked against its own negative**: with the stealing register pair in the
held slot, every byte still arrives and `dma_run.sh` goes RED. A counter that
cannot come out different is 58.3's vacuous "UNDERRUNS: 0/120" again.
**3. THREE THINGS THIS MODEL CANNOT BE ASKED**, read out of MAME's source: the
card has **no request line to the DMAC** (its flow control is DTACK, so external
request cannot be run); **single address** cannot be run either (only channel 0
has device callbacks); and **only burst is modelled as held** (`(dcr & 0xc0) ==
0`, so cycle-steal-with-hold shares the not-held path). Of the four rows of the
`W` ladder, exactly one — dual address, held, 9 clk/B — has a code path here,
and it is the one demonstrated. The slot pinout has `#EXREQ` at B36, so a real
card plausibly drives it; that is ROADMAP B3, sharpened.
**4. AUTO-REQUEST IS CHARGED BY TIME, NOT BY BYTE — and this outlives the
emulator.** Every `W` in this project presumes the device asks for each byte. An
auto-requested channel spends its share of the bus whether or not a byte is
there, so a record's cost scales with **how long it takes to arrive**: halve the
rate and the CPU cost of the same record doubles. `tools/analysis/
28_autorequest_cost.py` prices it from MC68450 §3.8/§5.2.3.3.2, gating its
formulas against Table 5-3's sixteen rows first. At 37,405 B and **460 KB/s, an
explicit rate**: max rate costs the whole **95.3% of a frame**; of the GCR's
four bus shares only **BR = 00, 50%, carries the rate**, at 10.61 clk/B and
47.6% of a frame. **The GCR is a design lever nothing in this tree had named.**
**5. THE WINDOW IS REFUSED, so the re-encode is now a precondition.** A channel
writes a contiguous run and cannot drop the 300 B in front of a record, so
`sc_in_data` refuses a windowed read when the data phase is the DMAC's
(`SCE_WINDOW`) rather than writing the neighbours into the ring (49.2). 58.3's
"sector-aligned records should join the bundle" has become "the transport will
not run without them".
**6. WHAT IT ALL COSTS, and this is the part that re-ranks the roadmap.**
`15_bus_occupancy.py` now answers "what can the frame afford" as well as "what
does each `W` cost". On the gate container at 12 fps, with the decode term
MEASURED: decode 68.5%, audio 1.25%, **headroom 30.2% = 6.74 clocks per byte**
at a 37,403 B record.
| transport | clk/B | video | total |
|---|---|---|---|
| PIO, measured (58.2) | 87.28 | 391.7% | **461.5%** |
| dual address, held — **the floor** | 9 | 40.4% | **110.2%** |
| single address, held | 5 | 22.4% | **92.2%** |
**P4a cut the transport from 391.7% to 40..95%, four to ten times — and it still
does not fit.** A dual-address byte is 4 clocks of device read plus 5 of memory
write, so **9 is a floor and the frame affords 6.74**; no GCR share and no
delivery rate goes under it. At 460 KB/s max-rate totals 165.1% and LRAR at 50%
totals 117.4%. So the fit turns on **B3, which has stopped being a constant to
look up: does a real CZ-6BS1 drive `#EXREQ`?** If yes, single address fits at
92.2% with 7.8% spare. If no, the container must reach **328 KB/s of payload**
(new ROADMAP E7), against 438 now.
**The scope matters and is stated in 59.7:** this is the GATE container, the
heaviest thing the encoder emits. The lighter `cpufit` family was NOT priced —
the tool refuses it, correctly, because the C68K measurement belongs to the gate
container — so E7 begins with a harness re-run, and until then "34% too big" is
about the fixture, not about the project.
**ONE COLLISION, AND THE BEFORE-HALF OF `check.sh` CAUGHT IT.** `DM_USE` was
first placed at `$18300`, which is `ring.i`'s `XF_SLOT` mailbox — `scsi.i`'s
trace ends at `$182FF` and the next 160 bytes are the ring's. The ring rig's
first record request therefore wrote a non-zero word into what the transport
read as "use the DMAC", and **the P4b stage, which this session did not touch,
went red**. The map is now `$18500`. Green-before is not a formality: it is what
made a red in an untouched stage unambiguous. 59.6.
**New in the tree:** `src/player/dma.i`, `src/player/dmagate.s`,
`tools/bench/dma.lua`, `tools/bench/dma_run.sh`,
`tools/analysis/27_dmac_config.py` (decodes what the player programs, out of the
player's own source), `tools/analysis/28_autorequest_cost.py`,
`tools/analysis/mc68450.py` (the MC68450 field tables, now ONE copy, shared with
`21_iplrom_dmac.py`), and one `check.sh` stage. `src/player/scsi.i` gained the
`DM_USE` dispatch and `SCE_WINDOW`.
**No decoder code changed.** `decode.bin` is still 1,296 B at the same MD5.
**Next: THE RE-ENCODE BUNDLE, and it is now the only thing between this tree and
M2.** Four items sharing one re-measurement (ROADMAP P2): index-0 black,
`--spans all` as default, joint `lam`, and **sector-aligned records**, which
59.4 promoted from preference to precondition. After it, P4a's channel goes
behind `ring.i`'s mailbox and the 120-frame pixel-exact pass runs with the CPU
free — which is the shape of M2.
---
# Status & next-session handoff — end of session 26 (2026-08-24)
## Session 26: the player runs off the disc, and PIO costs 87 clocks a byte
**Green light first and last: `./tools/bench/check.sh` was ALL GREEN before any
of this and ALL GREEN after**, plus two new stages.
**ROADMAP P4b is DONE. P4a is the only item left before M2. FINDINGS 58.**
**Emulated** — MAME 0.277, `x68000 -exp1 cz6bs1 -ramsize 2M`. No real hardware
ran.
**1. THE SEAM IS CLOSED.** `src/player/xfer.i` sits behind `src/player/ring.i`'s
`XF_*` mailbox in place of `tools/bench/stream.lua`'s modelled transport. `XF_GO`
is answered by a real READ(10) to a real MB89352; `XF_ACK` is a word the 68000
bumps when the bytes have landed, not one a host synthesises from emulated time.
| | |
|---|---|
| 120 records fetched by the 68000 | one READ(10) each |
| 4,488,588 B into a 256 KB ring | **pixel-exact** over all 120 frames |
| a real mid-stream seek, pass 2 | **pixel-exact** after it too |
| 18 wraps, 14.7 KB mean hole | the SAME tiling as 49.4 and 55.4 |
The 18 wraps are the load-bearing assertion. A host producer made them in
session 18, `ring.i` made them against a modelled transport in session 23, and a
real SPC makes them now — **`ring.i` cannot tell which side of the mailbox
answered it**, which is what makes this a test of the transport rather than of a
new rig. 58.1.
**The change above the seam is two `bsr`s**, and the one in `ring_seek`'s
quiet-wait is not optional: with the transport *inside* the machine, that loop
is the only thing that can retire an outstanding request, so without it a seek
issued with a request in flight spins forever. A host transport retired it on
its own time. That is exactly the kind of difference the seam exists to hide,
and it is the one it could not.
**2. WHAT IT COSTS, AND THE NUMBER LEAVES THE EMULATOR.** `tools/bench/
xfer_cost.sh` runs the same 120 frames twice — same ring, same `stream.s`, same
`ring.i` placing every record — and changes only which side answers:
| | emulated | per frame |
|---|---|---|
| decode + `ring_poll` alone | 6.7737 s | 67.7% of a 12 fps frame |
| ...with the real transport | 45.9516 s | **459.5%** |
| **the transport alone** | **391,779,000 clk** | **391.8%** |
**87.28 clocks per delivered byte.** The keep loop in `scsi.i`, priced against
the **68000's own cycle table**, is 86 clocks, and the FIFO also carries the
dropped window bytes of 58.3 — **87.15 predicted, 87.28 measured, 0.2% apart.**
So this is the instruction stream and **not** MAME's SPC model: **the first
number this rig has produced that a real board would also pay.** The machine's
own V-DISP clock agrees from the other end — 120 frames in 560 slots of a 12 fps
clock is **2.57 fps**. 58.2.
**Against the ladder** (clocks charged to the CPU per delivered byte):
`W`=5 held is 22.4% of a frame, `W`=9 is 40.4%, `W`=19 — the IPL ROM's own disk
channel, 52.5 — is 85.3%, and **PIO is 391.8%**. **P4a is worth 4.6x the worst
DMA configuration in this tree and 17.5x the best**, where before this session
it was worth 9 against 19. `W` itself did not move by one clock.
**3. "UNDERRUNS: 0/120" IS VACUOUS HERE, and the rig now says so in place.** A
synchronous transport cannot underrun by construction — a frame cannot start
before its record has landed, because the decoder *is* the transport. The
counter that means something is `NO IDLE`: **119 of 120 frames found their slot
already open, worst overrun 441 whole ticks.** Same class of error as 49.7.2's
free-running ring passing at 48 KB: a rig configuration in which the failure
being tested for cannot occur. **The binding resource flipped too** — against a
modelled 488 KB/s pipe this container was rate-bound and the ring never filled;
here the ring **fills, 90 refusals for space**, while the decoder starves for
*time*.
**4. A RECORD IS NOT A SECTOR, and the cheapest fix is a re-encode.** 117 of 120
records start part way into a sector. Reading whole sectors into the ring
**corrupts the neighbouring records** — the block loop has no bounds check
(49.2) — so `scsi.i` reads the covering sectors and stores only the window.
That is free in PIO because the CPU is already touching every byte, **and it
stops being free the moment P4a succeeds**: a channel writes a contiguous run
and cannot drop bytes. `tools/analysis/26_sector_align.py` prices the three ways
out: windowed PIO **+1.34% wire, no DMA**; bounce buffer **+1.34% wire and
+5 clk/B of copy** (22.4% of the frame — the cost `aligned` was chosen over
`split` to avoid, arriving by a different door); **sector-aligned records
+0.43% wire and zero clocks.** The last wins on both axes and is a container
change. **ROADMAP now carries a four-item re-encode bundle** (index-0 black,
`--spans all`, joint `lam`, sector alignment) and **P4a should be attempted
against a sector-aligned container, not against this one.** 58.3.
**5. The rig refuses to be asked for a rate.** `DLX_XFER=scsi` rejects a
non-zero `--kbps` outright rather than ignoring it, and `stream.lua` suppresses
`CHANNEL IDLE`, `DEADLINE` and `REQUIRED PREFILL` rather than printing them as
zeros — a zero there reads as "the channel never stopped", which would be a
claim about a medium this tree has never timed.
**One rig bug found and fixed in the same sitting:** the host's `nsent` is now
rebased at every seek. `XS_NXFER` is cumulative, so on a second pass it was
already >= `nframes` and the slack sampling never fired — the second pass
reported a ceiling of 0 frames and a build time of **-1 ticks**, which is an
empty series printing as a result.
**New in the tree:** `src/player/xfer.i`, `tools/bench/mkvol.sh` (one copy of
the volume layout, shared by both SCSI rigs), `tools/bench/xfer_cost.sh`,
`tools/analysis/26_sector_align.py`, `DLX_XFER=scsi` and `DLX_SECONDS` in
`pace_run.sh`, and two `check.sh` stages. `src/player/scsi.i` gained
`sc_in_data` (the three-loop windowed receive) and `scsi_read_win`; `ring.i`
gained the two transport hooks; `stream.s` includes `scsi.i` and `xfer.i`.
**No decoder code changed.** `decode.bin` is still 1,296 B at the same MD5.
**Next: P4a, and it is now the whole of what is left before M2.** A DMAC
configuration that HOLDS THE BUS, shown by evidence that does not come from
watching `$EA0015` (57.3: with `exown` asserted, MAME cannot tell a CPU-driven
byte there from a DMAC-driven one). Do it against a sector-aligned container if
the re-encode bundle has been run by then, because 58.3 is otherwise a bounce
copy at +5 clk/B. **P2's remaining half** is unchanged and is now item 1 of
that bundle.
---
# Status & next-session handoff — end of session 25 (2026-08-24)
## Session 25: the 68000 reads the disc itself, and P4 was never blocked
**Green light first and last: `./tools/bench/check.sh` was ALL GREEN before any
of this and ALL GREEN after**, plus a new SCSI stage.
**ROADMAP P4 is HALF DONE. FINDINGS 57.** **Emulated** — MAME 0.277,
`x68000 -exp1 cz6bs1`. No real hardware ran.
**1. THE BLOCKER WAS A MISSING FILE, NOT A MISSING MODEL — and this tree already
knew.** Session 21's handoff recorded, re-checked rather than assumed, that
"MAME's `x68000` has no MB89352 path". It has one: `-exp1 cz6bs1` instantiates a
*Fujitsu MB89352 @ 5.00 MHz* next to the HD63450, and **FINDINGS 32.4 had read
that card's DMA glue in session 9**. What is actually absent is the 8 KB
`scsiexrom.bin`, which MAME requires to instantiate the card and which **the
player never executes** — driving the SPC registers directly has been the plan
since `docs/BENCHMARK.md` item 4 in session 2. A zero-filled placeholder on a
separate rompath starts the machine; the user's romset is untouched. **B3 is
untouched too**: it wants that ROM's bytes disassembled, and a blank one has
none. 57.1.
**2. THE RESULT.** `src/player/scsi.i` on the 68000, no IOCS and no host in the
transfer path, selects the target and issues READ(10):
| | |
|---|---|
| 4,096 B from LBA 0 | byte-for-byte against the host's image |
| 2,048 B from LBA 1000 | byte-for-byte |
The non-zero LBA is the half that matters: **a driver that emits a malformed LBA
field still passes LBA 0**, because zero is what a malformed field usually is.
The volume is `tmp/stream_disk.bin`, the same file the ring rig reads, so the two
rigs cannot differ in what they are reading. 57.4.
**3. The register map is measured, and it corrects MAME's own documentation.**
The probe walks `$EA0000..$EA003F` one address at a time and survives each bus
error. **60 of 64 answer**; the two holes are `$EA0007` and `$EA001F` — exactly
the TMOD and EXBF the **MB89352 omits and the MB87030 has**. MAME leaves HOLES
and does *not* "shift subsequent indices", which its own device summary claims,
and that is what keeps DREG at index 10 and at `$EA0015`. TEMP took `$A5` and
gave it back. 57.2.
**4. THE DATA REGISTER IS DMA-ONLY, AND A PIO WRITE VANISHES.** `x68k_scsiext.cpp`
glues `$EA0015` and nothing else: with `exown()` asserted and DRQ low a write is
**discarded silently** — no error bit, no status change, no interrupt. Quieting
all four DMAC channels does not change it. So every transfer runs the SPC in DMA
mode and the CPU moves the bytes through `$EA0015` itself, via `dma_w`/`dma_r`:
**the CPU standing in for the DMAC, through the DMAC's own door.**
**The cost to the argument, stated because it is easy to overclaim:** with
`exown` asserted at idle, **MAME cannot distinguish a CPU-driven byte at
`$EA0015` from a DMAC-driven one.** This rig shows the DATA PATH and cannot, by
itself, show that the HD63450 is driving it — which is exactly what ROADMAP
calls P4's first job. Whether a real CZ-6BS1 also refuses PIO here is **not
settled**; it is a property of MAME's model. 57.3.
**5. `W` DID NOT MOVE. Not by one clock.** MAME's device models are functional,
not transfer-timing accurate, and 42.5 reads its DMAC configured in wall-clock
attotimes rather than per-operand cycles. `W` is still the project's largest
open number and still wants a board.
**6. Five bugs, four of them silent** (57.5), recorded because the pattern is the
finding: a chained-`rol` transfer counter that loaded ZERO; a FIFO byte mistaken
for a bus byte; a fixed phase sequence where the bus decides the order; the
discarded PIO write; and an initiator that must drop ACK *and then* release the
bus. The last **only appeared once there were two reads** — one read passed
byte-exact and the second could not select. A player issues one command per
record, so that failure would have been universal in the ring and invisible in a
one-read demonstration.
**New in the tree:** `src/player/scsi.i` (the driver), `src/player/scsigate.s`
(the probe and gate front-end), `tools/bench/scsi.lua`, `tools/bench/scsi_run.sh`,
and a `check.sh` stage that builds the volume with `chdman`, gates the register
window at 60 of 64 and both reads byte-exact, and is skipped when `chdman` is
absent.
**No decoder code changed.** `decode.bin` is still 1,296 B at the same MD5.
**Next:** **P4's other half**, in two pieces. (a) The HD63450 driving the data
phase in a configuration that HOLDS THE BUS — and 57.3 is a warning about how it
will have to be shown, because this apparatus cannot tell a DMAC-driven byte
from a CPU-driven one at `$EA0015`. (b) `scsi.i` behind `ring.i`'s `XF_*`
mailbox in place of `stream.lua`'s modelled transport, gated on the same
pixel-exact 120 frames — the seam is already there, and `XF_ACK` becomes
something the 68000 bumps when a transfer lands rather than something a host
synthesises. **P2's remaining half** (reserve index 0 as black) is unchanged and
still bundled with the two other re-encode-class questions from 55.
---
# Status & next-session handoff — end of session 24 (2026-08-24)
## Session 24: the scene graph is in, and the worst gap between two decision points is zero
**Green light first and last: `./tools/bench/check.sh` was ALL GREEN before any
of this and ALL GREEN after**, plus a new import stage.
**ROADMAP G1 is DONE. FINDINGS 56.** G1 was scheduled early because it is a
*measurement input*, and it paid for that immediately.
**1. THE ANSWER: the worst gap is zero.** Over 612 distinct transitions into a
seek, taking the earliest instant each input window opens and chaining play
across sequences the disc plays through without seeking:
| | worst | p10 | median | p75 | p90 | best |
|---|---:|---:|---:|---:|---:|---:|
| seconds of play between two seeks | **0.000** | 0.950 | 3.473 | 5.800 | 9.548 | 82.497 |
**33 of the 612 (5.4%) are zero** — an input window that opens at t=0 of a clip
the disc seeked to, so two seeks can fall back to back with no play at all
(`flaming_ropes.enter_room -> fall_to_death`: press right on the first frame and
you die). **51.2's slack rule can be answered NO by the content, not by the
buffer**, and no amount of ring is a defence. 56.3.
**2. Most branch points arrive before the ring has refilled, at every rate this
tree has considered.** 51.3's climb against the game's own gaps, gate container:
| ring KB | pipe KB/s | ceiling | climb s | branch points under the climb |
|---:|---:|---:|---:|---:|
| 256 | 451.4 | 3 | 20.83 | 601/612 (98%) |
| 256 | 488.0 | 7 | 6.11 | **468/612 (76%)** |
| 256 | 513.2 | 7 | 3.81 | 370/612 (60%) |
| 256 | 600.0 | 7 | 1.66 | 129/612 (21%) |
| 512 | 488.0 | 11 | 9.60 | **551/612 (90%)** |
**A bigger ring makes this worse and now content says so too**: same rate, 76%
at 256 KB and 90% at 512 KB, because doubling the ceiling does not touch
`pipe - wire`. **The ring is not the lever; the surplus is.** 56.4.
**3. It does not break — it removes margin.** A branch on an empty ring costs
the prefill, not the climb: **149.7 ms (1.80 frame slots) at 488 KB/s**, 162.0 ms
if it is a scene change carrying the 6,164-byte header. So the finding is not
"this fails", it is **"this design runs permanently at minimum lookahead, and
the arcade content is what puts it there"**. The mechanical seek is still
unmodelled (B1) and is charged on top. 56.5.
**4. CORRECTION to FINDINGS 16: there is only one transcription.** 16 cleared
two permissively licensed sources and planned to diff them "to catch
transcription errors". The SNES project's own `data/events/README.md` says its
chapters are "derived from DirkSimple game data" — a second *copy*, not a second
transcription. The diff runs anyway and catches conversion errors: durations
agree 388/505 within one frame, branch structure 470/505, and of the 35
differences **16 are renames and 18 of the other 19 are the SNES conversion
dropping the arcade's diagonals**. Zero transcription discrepancies were found
and none could have been. 56.2, 56.6.
**5. Two constraints on the input layer, free with the import.** The arcade uses
**eight directions plus action and start** (diagonals are 21 of 906 windows —
droppable by a port that must, not by one aiming at the arcade), and the
**shortest input window is 98 ms** against a frame slot of 72.13 or 90.16 ms
(54.4). **Input cannot be polled on the frame tick**; `clock.i`'s V-DISP
interrupt already runs 8.6x faster at 0.1% of the budget. 56.7.
**6. The coupling to outside source is contained to one file (USER DECISION).**
`tools/import/scenegraph.py` is the only file in this tree that knows those
projects exist — their paths, table names, timing formulas, constants — and it
writes **`DLXSCENE1`**, our own schema, into gitignored `tmp/`.
`tools/analysis/25_scene_graph.py` reads only that. Nothing is vendored, nothing
outside-derived is committed, and the generated table carries its own `sources`
attribution block. The split was made after the measurement and changed no
number in it.
**New in the tree:** `tools/import/` (new directory, one file),
`tools/analysis/25_scene_graph.py`, and a `check.sh` stage that imports, gates
on 516 sequences / 906 input windows, and runs the analysis — skipped when there
is no checkout, like the px68k and IPL ROM stages.
**No 68000 code ran and none changed.** `decode.bin` is still 1,296 B at the
same MD5.
**Next:** **P4** (drive the MB89352, settle `W`) still decides the project and
still needs hardware or a MAME that models the SPC. What 56 changes about it:
the transport now has a *content* requirement as well as a rate one — it has to
survive a branch with an empty ring at zero notice, 5.4% of the time. **P2's
remaining half** (reserve index 0 as black) is unchanged and still bundled with
the two other re-encode-class questions from 55: the delivered-rate rate point
and 54.4's short slot. All three are still one re-encode plus one
re-measurement, and still want deciding together.
---
# Status & next-session handoff — end of session 23 (2026-08-24)
## Session 23: the 68000 fills its own ring, and the player's request loop turns out to cost more than the medium does
**Green light first and last: `./tools/bench/check.sh` was ALL GREEN before any
of this and ALL GREEN after**, 120/120 on both cores, no `TRUNCATED`, plus two
new ring stages.
**ROADMAP P5 is DONE. FINDINGS 55.** P5 was the last M2 item buildable in this
tree, and it is the third and last policy to move off the host: the loader went
in session 21, the frame clock in 22, and the ring producer now.
**1. The container had to change, and it is the first format change since
session 12.** `aligned` asks whether the next record fits before the end of the
ring — a question about a record's length asked **before it is fetched** — and
every reader in this tree answered it by WALKING the frame stream, which is
exactly what a player streaming off a disc cannot do. **DLX4** carries
`nframes` u16 record lengths in the scene header. The frame payloads are
**byte-identical** to the DLX3 encode, asserted record for record, so no
constant fitted to the gate container moves; the scene header goes 5,920 →
**6,164 B**. `dlx.py` refuses a container whose index disagrees with its own
walk. 55.1.
**2. `src/player/ring.i` reproduces the host producer exactly.** Same 18 wraps,
same 14.7 KB mean hole, pixel-exact — a third independent implementation of
`aligned` landing on the same tiling (55.2). The host now **audits** every
placement instead of making it.
**3. THE ONE THAT MOVES SOMETHING: the disc stands still whenever the player is
not asking.** A channel only moves bytes while it has a request and only the CPU
can issue one, so there is a gap between every pair of records that is a
property of the **player's loop, not the medium** — and no host-filled run could
see it, so no rate table in this tree contains it. At 488 KB/s in a 256 KB ring,
changing nothing but how many requests the player may have outstanding:
| queue | channel idle | underruns | slack ceiling | bound by |
|---:|---:|---:|---:|---|
| 1 request | **6.8%** | **59/120** | 2 | rate |
| 2 requests | **3.4%** | **0/120** | 5 | ring |
This container's surplus over the wire at 488 KB/s is 8.7% of the pipe, and a
one-deep request loop spends 6.8% of it on nothing — **most of the surplus
51.3's lookahead is accumulated out of**. A second slot costs no clocks and the
hardware has four channels (52.1). 55.3.
**4. Prefill is the weaker lever, and now it has a number.** Six records of
prefill — half a second of black at every scene start — still leaves 24
underruns at a one-deep queue; a second queue slot leaves none. **A prefill buys
a one-off cushion a rate-bound pipe spends immediately; a queued request buys
the rate back every frame.** The shipped policy is 2 records, and it is not 1
because 51.2 measured that *n* resident records buy *n-1* frame times. 55.4.
**5. The slack rule and a seek are in the player.** `ring_may_seek` is 51.2's
rule as arithmetic the player runs; `ring_seek` quiets the channel, takes record
*j*'s disc address out of the index and empties the ring. Rehearsed as a second
pass: 240 records, seek at 12.87 s, refill from empty, **0 underruns after it
and the last frame of the second pass pixel-exact**. The seek shows up as the
run's worst channel gap, **397.5 ms** — disc idle, not mechanical seek, which is
still unmodelled. 55.5.
**6. Three silent bugs and one instrument correction**, all written up in 55.7,
because every one of them produced wrong pixels or a desync rather than a fault:
the read cursor's wrap rule not matching the writer's; the free-space test
deciding a wrap before it knew whether the LIVE span was the wrapping one; the
request queue gated on completion instead of retirement (only reachable above
depth 1). The fourth is the rig's: MAME renders a screen line by line, so
snapshotting the frame in which the decoder finished captures a **tear** — it
looked exactly like a decoder bug and was not. The rig now settles one frame
before capturing. No previously reported result is affected.
**New in the tree:** `src/player/ring.i` (the producer, the prefill, the slack
rule and the seek); `tools/analysis/24_ring_owner.py` (an independent model of
all of it, sharing no code with the rig). `encode.py`/`dlx.py` gain DLX4 and its
index cross-check; `prep_stream.py` emits the index and checks it against the
disk image; `load.i` accepts DLX3 or DLX4. `stream.s` gains the ring hooks and a
polling wait loop; `stream.lua` becomes a **transport** built out of memory taps
(exact issue and completion times — see 55.7 for why nothing in a tap may touch
the memory space); `pace_run.sh` gains `DLX_RINGOWN`, `DLX_QDEPTH`,
`DLX_PREFILL_FR`, `DLX_ITER`. `check.sh` gains two stages: the machine-owned
ring, and a seek with the decode after it.
**`decode.s` and `frame.i` are unchanged**, `decode.bin` still 1,296 B at the
same MD5, and a host-filled run executes none of the new code — so every
FINDINGS 49/51 figure stands.
**Still open in P2:** unchanged — the encoder does not reserve a black entry
(23.4).
**Next:** M2 has no item left that this tree can build. **P4** (drive the
MB89352, settle `W`) still decides the project and still needs hardware or a
MAME that models the SPC; 55.3 sharpens what to ask of it, because the ladder
now has a second axis — the clocks stolen per byte AND the fraction of the pipe
the player's own loop gives away. **G1** (import the scene graph) is what would
let this tree ask the question 55.5 rehearsed but could not pose: what is the
worst gap between two real decision points, and does the refill climb survive
it. **P7** (boot from the volume) is buildable but empty until P4.
**A question 55.3 raises and does not answer:** the encoder is fitted to a pipe
that delivers continuously. It does not, and by 3-7% depending on the player's
queue. Whether the rate point should be set against the *delivered* rate rather
than the nominal one is the same class of change as the reserved black entry and
the short-slot question from 54.4 — a re-encode plus a re-measurement, and all
three should be decided together.
---
# Status & next-session handoff — end of session 22 (2026-08-24)
## Session 22: the frame clock moves onto the 68000, and the 12 fps frame turns out not to exist
**Green light first and last: `./tools/bench/check.sh` was ALL GREEN before any
of this and ALL GREEN after**, 120/120 on both cores, no `TRUNCATED`, plus two
new frame-clock stages.
**ROADMAP P3 is DONE. FINDINGS 54.** P3 was one of the two items session 21's
handoff called buildable here, and it is the one that touches every other number
in the project — because the tick is what the word "frame" in "% of a frame"
means.
**1. Neither of the two sources P3 named can do it, and the enumeration is the
finding.** The MFP's timer clock is 16 MHz/4 = 4 MHz, its prescalers are
`{4,10,16,50,64,100,200}` and its data register is 8 bits, so the **slowest tick
any single timer can make is 78.125 Hz** — 6.5x faster than a frame — and
**4e6/12 = 333,333.33 is not an integer**, so no setting reaches 12 Hz at all.
The raster has no whole divide near 12 either: 4 refreshes is 13.86 fps and 5 is
11.09. `tools/analysis/23_frame_clock.py` walks all 7x256 timer settings rather
than asserting it. 54.1, 54.2.
**2. The clock is the raster with a remainder, and it is exact by construction.**
`src/player/clock.i` takes the V-DISP falling edge on MFP GPIP4 — the start of
vertical blanking, which is when a player would present — and adds `fps*VTOTAL`
per edge to a 16-bit accumulator, emitting a tick at 31,500 and keeping the
remainder. Long-run rate is `fps*VTOTAL/VTOTAL` = **12.000000 fps exactly**.
Both constants are **read out of the CRTC at init**, so the clock is derived from
the registers that generate the raster it counts. Measured: **3,000 interrupts,
649 ticks where 649.1429 were due**. The gate is stated in ticks, not ppm — a
remainder is off by at most one tick over any window, so ppm would let a longer
window advertise a tighter clock for free. 54.2.
**3. It costs 181.35 clocks per V-DISP; 838 per frame; 0.1006% of the budget.**
The host cannot time this — its granularity is 17.64 ms and the interrupt is
microseconds — so **the 68000 times it itself**: a one-instruction loop over a
3,000-refresh window, run with the clock off and on, with the loop's own cost
calibrated rather than looked up. The calibration landed on **38.000002 clocks**
per iteration, which is both the check that licenses the subtraction and an
independent confirmation of `buscost.py`'s model. The 181.35 then decomposes
exactly, leaving **43.99 clocks for the interrupt exception** — the textbook 44,
measured rather than recalled. A timer-based clock would have cost 3.6x this at
an arbitrary phase against the scan. 54.3.
**4. THE ONE THAT MOVES SOMETHING: there is no 83.33 ms frame, and there never
was.** 12 fps on a 55.4577 Hz raster is 4.6215 refreshes, so a frame is shown for
**4 refreshes (72.13 ms) or 5 (90.16 ms)**, 37.9% of them short. The
833,333-clock budget every figure in this project is priced against is the
**mean** slot; the short one is **13.4% under it**, and 10 of the gate
container's 120 frames do not fit it.
**And the cadence was already in the tree, unnamed.** `stream.lua`'s tick is
`floor((t - t_rel) * fps)`, which looks uniform and is not: Lua only sees the
machine at frame boundaries, so its ticks land on refreshes and its gaps were
always 4 or 5. **Every host-paced result in FINDINGS 49 and 51 already carried
this cadence.** P3 did not introduce it; it moved who produces it onto the
machine and made it visible.
**It is not a dropped frame.** The pace gate says only "not before tick i", so an
overrun eats the next frame's idle and the clock recovers itself; the cost is one
frame presented a refresh late. On the gate container that is **4 frames of 120
with no idle left, against 1 for the nominal model** — and the expensive one is
**frame 0 at 111% of budget**, because the first frame of a scene has nothing to
SKIP against. So the cost lands **at a scene change**, next to 53.2's 18.96 ms of
loader and the seek. `stream.s` counts this itself now, and the rig's count
matches an offline model of the divider **exactly**: 4/120, first at frame 1, on
both tick sources. 54.4.
**5. An instrument correction the whole tree was reading.** `x68k_crtc.cpp
refresh_mode()` builds the frame period from `scr.max_x * scr.max_y` with
`scr.max_x = m_htotal - 8` — one character cell short, an inclusive bound used as
a count. **MAME's raster is fast by 368/360 = 2.2222%**: 56.6901 Hz measured
against the registers' 55.4577, agreeing to six digits with the arithmetic. So
every "1/55.46 s granularity" note in this tree was wrong and is **1/56.69 s**;
corrected in six files with the derivation put once in `crtc_mode.lua`. **No
conclusion changes and no 68000 cycle figure moves** — the CPU clock is unrelated
to the screen — but anything *paced* by the raster runs 2.22% fast under MAME, so
`clock.lua` reports both rates and de-skews, and the interrupt is priced against
the hardware refresh count. 54.5.
**New in the tree:** `src/player/clock.i` (the clock) and
`src/player/clockgate.s` (its measurement front-end); `tools/bench/clock.lua`,
`clock_cost.py`, `clock_run.sh` (the rig); `tools/analysis/23_frame_clock.py`
(the enumeration and the cadence pricing). `stream.s` gains `CLKON` and a
late-frame counter ahead of the wait loop; `stream.lua` gains `DLX_PACE=2` and
takes its deadlines from the ticks the machine actually emitted rather than from
a host model; `pace_run.sh` gains `DLX_PACE` selection, with the default tag left
alone so `pace_sweep.sh` still finds its logs. `check.sh` gains two stages: the
clock's own measurement, and 120 frames decoded pixel-exact with nothing outside
the machine deciding when a frame may start.
**`decode.s` and `frame.i` are unchanged.** `decode.bin` is still 1,296 B at the
same MD5. The pace gate's wait loop is byte-for-byte the one FINDINGS 51
measured, and the free-running path executes none of the new code, so every
FINDINGS 49 figure stands.
**Still open in P2:** unchanged — the encoder does not reserve a black entry
(23.4), so the letterbox still gets the palette's closest thing to black.
**Next:** P5 (per-record index, prefill policy, the accumulated-slack rule in the
player rather than the rig) is buildable here and is now the last M2 item that
is. G1 (import the scene graph) still needs fetching, and is still the one that
would let this tree ask what the worst gap between consecutive decision points
is. P4 still decides the project and still cannot be measured here.
**A question 54.4 raises and does not answer:** every rate-control and budget
figure in this project is fitted to an 833,333-clock frame, and 37.9% of frames
get 721,270. Whether the encoder should be fitted to the SHORT slot instead of
the mean is a re-encode plus a re-measurement — the same class of change as the
reserved black entry — and it should be decided with P5's numbers in hand, not
before.
---
# Status & next-session handoff — end of session 21 (2026-08-24)
## Session 21: the loader moves onto the 68000, and a scene change gets a price
**Green light first and last: `./tools/bench/check.sh` was ALL GREEN before any
of this and ALL GREEN after**, 120/120 on both cores, no `TRUNCATED`, plus a new
load-time stage.
**ROADMAP P1 is DONE and P2 is half done. FINDINGS 53.** Session 20's handoff
named P4 as the item that decides the project, and P4 is **blocked in this
tree** — re-checked, not assumed: there is still no `scsiexrom.bin` anywhere on
this machine (`~/mame/roms/x68000.zip` holds six files, four IPLs, a cgrom and
an sram), MAME's `x68000` has no MB89352 path, and `hd63450.cpp` decodes no
DTYP. **Nothing here can measure W.** P1+P2 was the M2 item that could be built
here, and it is the one that touches an already-measured number: it lands at a
scene change, where FINDINGS 51.3's refill climb is.
**1. The transforms are on the 68000 and they are byte-exact.** `src/player/
load.i` expands both codebooks to word-per-pixel form and packs the palette to
`GGGGGRRRRRBBBBBI` with the shared LSB chosen per entry, out of the RAW
container header. Gated **byte-for-byte against `tools/bench/dlxload.py`**,
which stays the reference — what changed is where the transforms run, not what
they produce. The palette half is read back **out of the palette registers at
`$E82000`**, so "the words reached the hardware" is part of what passes. Both
CPU cores emit the same 10,752 B. 53.1.
**2. A scene change costs 18.96 ms of 68000 time, 22.8% of one 12 fps frame.**
Boot costs 24.70 ms. Split: codebooks 92,609 clocks, palette entries 97,019,
scratch tables 52,919. Cross-checked on px68k's C68K, which agrees to 1.4% on
the scene-change figure. 53.2.
**3. The scratch tables describe the machine, not the scene.** They are the
CRTC's 6-to-8-bit rendering, its square, and the per-channel error difference —
so `pal_tables` is a separate entry point, built once at boot. **5.29 ms off
every scene change**, 22% of what a naive port of `dlxload.py` would have cost
per scene. 53.3.
**4. THE ONE THAT MOVES SOMETHING: the scene header is 5,920 bytes nothing has
ever counted.** Palette + CB1 + CB4, and it must arrive before frame 0 can be
decoded. It is not part of any frame record, so no rate table in this tree
includes it. `tools/analysis/22_scene_load.py` prices it across explicit rates.
The whole fixed cost of a scene change is about **a third of one frame slot**
but its cost in FINDINGS 51.3's currency, accumulated slack, is divided by the
surplus `pipe - wire` and so is hypersensitive: **138 ms of extra climb at 488
KB/s, and 1.099 s at 451.4 KB/s**, for the same 5,920 bytes. 53.5.
**5. An instrument disagreement worth carrying forward.** The two CPU cores
agree to <3% on every stage except the table build, where they differ by 16.4%
— and that is the only code in this tree that multiplies. **px68k's C68K charges
a flat 50 clocks for `MULU`/`MULS`** regardless of operand (`c68kmacro.h:1869`),
where the 68000 charges 38+2n. That explains 4,608 of the 8,703 clock gap;
**4,095 clocks are not explained and are recorded as open.** Nothing in
`src/player/` outside these three instructions multiplies (checked), so no
figure in FINDINGS 24-52 is affected — but the second opinion this tree leans on
over-charges multiplies, and a future measurement containing one must not be
taken from it uncorrected. 53.4.
**6. Shipping pre-expanded codebooks was considered and refused.** It would
trade 9.26 ms of 68000 time for 5,120 more bytes in every scene header — 10.5 ms
of pipe at 488 KB/s, and bytes that lengthen the climb. Close to a wash in
milliseconds, not a wash in kind: the CPU is idle during a seek and the pipe is
what this project is short of. **Derived, not measured.** 53.6.
**New in the tree:** `src/player/load.i` (the transforms) and
`src/player/loadgate.s` (its front-end, 488 B); `tools/bench/prep_load.py`,
`load.lua`, `verify_load.py`, `load_run.sh` (the rig, both cores);
`tools/analysis/22_scene_load.py` (the pricing). `tools/bench/c68k/harness.c`
gains a `--loadraw` mode, which also makes its flag-watch address a variable
rather than a constant. `check.sh` gains a stage that gates byte-exactness on
both cores, and deliberately does **not** gate the cycle counts — MAME's clock
is 1/56.69 s (1/55.46 when that was written; 54.5) and a wall timing would make
the green light host-sensitive, the
same reason `blit.s` and `span.sh` are not in it.
**`decode.s` and `stream.s` are unchanged.** Nothing in the per-frame path was
touched; `decode.bin` is still 1,296 B at the same MD5.
**Still open in P2:** the encoder does not reserve a black entry (23.4), so the
letterbox still gets the palette's closest thing to black (index 255 here). That
half is encoder-side, it changes the container, and it moves every constant
fitted to the gate container — a re-encode plus a re-measurement, not an edit.
**Next:** P3 (a real frame clock from the MFP or VBL) and P5 (per-record index,
prefill policy, the accumulated-slack rule in the player rather than the rig)
are both buildable here. G1 (import the scene graph) is the one that would let
this tree ask a question it currently cannot: what is the worst gap between
consecutive decision points, and does the refill climb survive it. P4 still
decides the project and still cannot be measured here.
---
# Status & next-session handoff — end of session 20 (2026-08-24)
## Session 20: the DMAC configuration was in the IPL ROM the whole time
**Green light first and last: `./tools/bench/check.sh` was ALL GREEN before any
of this and ALL GREEN after**, 120/120 on both cores, no `TRUNCATED`, plus a new
ROM-evidence stage.
**ROADMAP's "do this first" item is DONE. FINDINGS 52.** It asked for the cheap
thing — put the ADPCM DMA stream on the bus and see what it does to the 86.7%
— and doing it needed a clocks-per-byte figure for the audio channel that the
tree did not have. `11_cpu_budget.py` was charging audio bytes the *disk's*
rate, 5 clk/B, described in its own help text as "single-address, bus held".
**Audio was being charged the favourable end of ROADMAP B3, an open question
worth 242 KB/s.**
**1. It never had to be a guess.** The X68000 IPL ROM programs all four HD63450
channels itself, and **the ROM is on this machine** — MAME boots the rig with
`-bios ipl10`. `tools/analysis/21_iplrom_dmac.py` reads the configuration out of
the image and decodes the MC68450 fields. Eight (address, expected bytes,
meaning) sites; a mismatch or an unknown ROM revision exits non-zero. Needs no
emulator, runs in milliseconds, and is now in `check.sh`. **Layer: a static read
of a shipping ROM image, not a running machine and not real hardware.**
**2. Audio is dual-address and cannot hold the bus: 16..19 clk/B.** ch3
`DCR = $80`, `OCR = $32`: explicitly addressed (9 clocks, read 4 + write 5),
cycle steal **without hold**, `REQG = 10` external request — the DMAC
arbitrates once per byte with no burst to amortise the 5..8 + 2 arbitration
over. The old debit was 3.2x..3.8x too small. 52.2.
**3. And on the bus it is still nothing.** 651.0 B/frame x 16..19 =
**1.25%..1.48% of the frame**, about 4% of what the decoder leaves. **P6's bus
risk does not materialise.** The unit worry was worth checking and nearly
right: 15.6 kHz = 8 MHz / 512 = 15,625 samples/s, two 4-bit samples to a byte =
**7,812.5 B/s exactly**; `AUDIO_KBPS = 7.8` is that in decimal kB and was being
multiplied by 1024. 2.4% high, now derived from the sample rate. 52.3/52.4.
**4. THE ONE THAT MOVES SOMETHING: the disk channel is programmed identically.**
ch1 (SASI) is `DCR = $80` too, and so is ch0 (FDC) — Sharp programs every
explicitly-addressed 8-bit device on this board the same way. By the same
arithmetic that is **16..19 clocks per delivered byte**, where FINDINGS 42.4
brackets W at **5..12** and 42.5 reports `W = 8` already missing 47/120 frames.
**The only worked example of a disk DMA configuration on this machine sits above
the entire bracket**, and at that price the design does not fit at any container
size. It is not `scsiexrom.bin`, so **B3 stays open** — what changed is that a
cheap configuration is now the thing that has to be *shown*. `W <= 12` is a
requirement on the player's DMAC programming, not a range the hardware hands us.
**This is now the largest open number in the project, ahead of the rate.** 52.5.
**5. An unforced cross-check fell out.** `15_bus_occupancy.py` now sweeps W over
the whole frame period. Its `W = 8` row lands at 105.7% of the frame — agreeing
with 42.5's "misses 47/120", from mode histograms and bus clocks respectively,
two models that share no code. 52.5.
**6. Audio outranks the disk at the arbiter.** CPR: FDC 0, ADPCM 1, SASI 2,
`_DMAMOVE` 3. When both want the bus, ADPCM is served first — an audio byte
never waits, a video byte does. Relevant to 51's smooth-rate delivery model.
52.6.
**New in the tree:** `tools/analysis/21_iplrom_dmac.py` (the ROM decoder/gate);
`15_bus_occupancy.py` gains a "THE OTHER TWO MASTERS" section pricing both DMA
consumers against the frame period; `buscost.py` gains the ADPCM constants and
the split read/write DMAC timings; `11_cpu_budget.py` charges audio and video
separately, with `--adpcm-clocks-per-byte`. Its stale "validated to within 1 pt"
line is corrected — the model reads 2..10 pt HIGH and by more as the frame gets
harder, which was already true at HEAD.
**Also: the README now shows the thing working.** `tools/bench/stream.lua`
gains `DLX_SNAP_EVERY=1` (needs `DLX_PACE`; off by default and on no path
`check.sh` takes), which snapshots every frame tick instead of once at the end,
and `tools/media/make_readme_media.py` turns those PNGs into `docs/img/`. The
stills and both clips are **MAME's own screen pixels**, not a re-render.
Building it turned up something worth recording. 116 of 119 captured frames are
pixel-exact against `tools/encoder/dlx.py`; **three are torn** — frame *n* on
top, frame *n-1* below the tear line — because MAME captured the screen while
the block loop was partway down it. `decode.s` writes straight to the displayed
page (one display path, 28.1), so **a real player tears the same way**; this is
the first time that consequence has been visible rather than argued. The script
ASSERTS the tear (every differing pixel must come from the previous frame) and
refuses to build otherwise, rather than trimming the frames and quietly
reporting "every frame I kept is exact".
Second correction the capture forced: the snapshot fires at the tick, *before*
frame *n* is decoded, so the obvious reading is that snapshot *n* holds frame
*n-1*. It does not — MAME renders the screen at the end of the machine frame, by
which time the 68000 has finished frame *n*. Checked, not reasoned about.
**`decode.s` and `stream.s` are unchanged.** Nothing in `src/player/` was
touched; `decode.bin` is still 1,296 B at the same MD5.
**Next:** P4 is the item that decides the project now, and 52.5 says so with a
number. Everything else in M2 (P1, P2, P3, P5, P7) is still buildable here.
---
# Status & next-session handoff — end of session 19 (2026-08-24)
## Session 19: the ring rig gets a frame clock, and a branch point costs 4.83 s of play
**Green light first and last: `./tools/bench/check.sh` was ALL GREEN before any
of this and ALL GREEN after**, 120/120 on both cores, no `TRUNCATED`, plus a new
paced-ring stage.
**Item 4 is DONE. FINDINGS 51.** Items 1, 2 and 3 were all still blocked on
hardware this tree does not have — a real board for 1 and 2, `scsiexrom.bin`
for 3 — and none of them moved. Item 4 was the one that could be built here.
`src/player/stream.s` had no frame clock, so it asked for record *i* the instant
it finished *i-1*, outran any finite pipe, and never let the ring back up. That
is why the 49.1 ring sweep passed at 48 KB. It now has `PACE`/`PACEON`
(`$18034`/`$18038`) and the producer supplies a 12 fps tick, so
`FR_HEAD-FR_TAIL` finally means what it is read to mean: **whole frames the
decoder could still draw with delivery stopped dead.** `PACEON=0` free-runs and
is what the wrap gate still uses, so every figure in FINDINGS 49 is unmoved.
**1. Small rings carry almost nothing.** Paced, on the gate container: 64 KB
holds **2 frames**, 96 KB holds 3, 128 KB holds 45, 256 KB holds 78, 512 KB
holds 1415. All pixel-exact, which is exactly why the unpaced sweep passed
them. 51.1.
**2. Tolerance is `ceiling - 1`, measured by cutting the pipe.** With 7 records
resident, a 6-frame-time cut underruns nothing and a 7-frame-time cut underruns.
The last record pays for the pipe's restart (~0.9 frame times to place 36.5 KB).
**256 KB buys 500 ms of dead pipe, not 583.** 51.2.
**3. THE ONE THAT MOVES SOMETHING: slack is accumulated, not owned.** It is
built out of `pipe - wire` and a seek spends all of it. At 488 KB/s a 256 KB ring
takes **4.83 s of play** to reach its 7-frame ceiling from empty; 512 KB takes
8.42 s to reach 14. **A bigger ring raises the ceiling AND lengthens the climb.**
So a branch point does not ask "is the buffer big enough", it asks "has there
been enough play since the last one" — and Dragon's Lair's decision points are
seconds apart. First statement in this tree about back-to-back branches. 51.3.
**4. The rig now says which resource is binding.** Rate refusals and ring
refusals are counted separately: at 460 KB/s every ring from 192 KB to 512 KB is
**RATE-BOUND**, ceiling 4, and never fills in 120 frames — larger rings are dead
RAM in that scene. Clearing the arrival deadline (451.4 KB/s, 49.5) and being
able to absorb a seek are **different requirements**, and the gap is large. 51.4.
**5. Independent agreement, honestly bracketed.** `tools/analysis/20_seek_slack.py`
is the model rewritten in Python from record sizes, sharing no code with the Lua
producer. **35/35 of the rig's ceilings fall inside its bracket, 33/35 at the
top.** The bracket is one record wide because the pipe delivers ~one record per
slot, so the answer depends on sampling before or after that slot's delivery.
Both are reported rather than picking the one that matched — that would have
been fitting the model and calling it a cross-check. 51.5.
**6. Pacing exposed two producer defects, both invisible free-running.** The
`RD_PTR` cross-check was really testing how often `reap()` ran (it asserted per
retired record; `RD_PTR` names only `tail-1`), and `reap()` was skipped for the
whole duration of a cut, so the ring looked full through a seek. 51.6.
**New in the tree:** `tools/bench/pace_run.sh` (one paced run, pixel-verified),
`tools/bench/pace_sweep.sh` (ring x pipe grid), `tools/analysis/20_seek_slack.py`
(the independent sim). `check.sh` gains a paced stage that gates pixel-exactness,
zero underruns, and the 256 KB ceiling — the last one because a change in it is
a change in what a branch point can afford, and that should not slip past as a
log line.
**`decode.s` is unchanged and still asserted:** 1,296 bytes, same MD5. The pace
gate is in `stream.s` only (1,396 -> 1,418 B) and outside `src/player/frame.i`,
so no per-block or span constant moves.
---
# Session 18 and earlier
## Session 18: the streaming path exists, and the shipping rate does not fit the pipe
> **THE DELIVERY RATE HAS NO WORKING FIGURE — retired session 18 (USER
> DECISION).** Sections below written before session 18 name a "4 Mbps" pipe
> constant and score tables against it. **Read every one of those as history.**
> It was never a bus measurement: user-supplied, no provenance, 10% of SCSI-1's
> asynchronous rating (FINDINGS 42.1), and FINDINGS 49.5 caught the shipping
> candidate exceeding it while nothing in the tree was comparing the two.
>
> It is now gone as a default from every analysis tool and from
> `tools/bench/stream.lua` — `--bus` / `--kbps` / `DLX_STREAM_KBPS` are
> **required arguments** with no fallback, so no table can be scored against a
> rate its own output does not state. The one survivor is `GATE_SPAN_KBPS` in
> `tools/bench/check.sh`, which is a **container recipe**, not a delivery claim:
> the gate container was encoded with it and every per-block and span constant
> in FINDINGS 41/43/45/49 is fitted to that container, so changing it is a
> re-encode plus a re-measurement, not an edit.
>
> **What to use instead:** `tools/analysis/19_ring_stream.py` reports the
> **zero-prefill pipe** — the rate a medium must clear for a container to need no
> prefill. That is a requirement to measure a BlueSCSI against, not a constant to
> design on. For the session-14 candidate it is **513.2 KB/s**.
**Green light re-run first: `./tools/bench/check.sh` was ALL GREEN** before any
of this, 120/120 on both cores, no `TRUNCATED`.
**Items 1 and 2 were both blocked in this tree, and checked rather than
assumed.** Item 1 needs a real board. Item 2 needs the CZ-6BS1's
`scsiexrom.bin`: `~/mame/roms/x68000.zip` holds six files — `cgrom`, four IPLs,
`sram` — and there is no SCSI ext ROM anywhere on this machine. MAME's
`hd63450.cpp` decodes no DTYP, so the emulator cannot stand in (48.4). **USER
DECISION: build item 3.**
**Item 3 is DONE, and item 4 folded into it. FINDINGS 49.**
`src/player/stream.s` + `tools/bench/prep_stream.py` + `tools/bench/stream.lua`
decode the gate container **out of a bounded ring**, container in a host file:
| ring | machine | result |
|---|---|---|
| **256 KB** | **stock 2 MB** | **120/120, final frame pixel-exact** |
| 128 / 96 / 80 / 64 / 48 KB | stock 2 MB | 120/120, pixel-exact |
**The rig's RAM ceiling is gone.** 45 raised `RIG_RAM` to 6 MB because 5,261,814 B
of stream did not fit 2 MB. The streaming rig holds ~256 KB and reads the rest
from the host, so the machine it runs on is now **the machine the player
targets**.
**1. The constraint is CONTIGUITY, not byte count, and 09_buffer_sim could not
see it.** The block loop and span chain read with a monotonically increasing
`a0` and no bounds check anywhere, so the ring needs the whole next record
**resident AND contiguous**. 49.2.
**2. `aligned` beats `split` and it is not close.** Producer refuses to start a
record it cannot finish, leaves a hole: **9.1% of a 256 KB ring, zero clocks**.
Letting records wrap and shadowing the ring head costs **5.57% of the frame
budget, forever** — and the decoder is already at 91.1% at p90. (Both are
`s14_d5_all1500`'s; the gate container makes it 5.7% of the ring against 3.64%
of the budget. The costs are per container, not universal.) `aligned` also
needs a per-record index, which a branching laserdisc game needs anyway. 49.3.
**3. Two independent implementations agree exactly.** The Python sim (from record
sizes) and the Lua producer (driving a real 68000) both give **18 wraps, 14.7 KB
mean hole, 94.3% usable**. They share no code. 49.4.
**4. THE ONE THAT MOVES SOMETHING: the shipping candidate does not fit the pipe.**
`s14_d5_all1500` is **496.7 KB/s**; the pipe this tree has simulated against
since session 2 is **488 KB/s**. Those two numbers had never been put side by
side. It is **8.7 KB/s over on the MEAN — not a burst a ring absorbs.** The
deficit grows **523 KB per minute of play**; no ring size fixes it.
It was never caught because 42.1 retired the pipe as the binding resource and
built the rate controller to bind on clocks, with **no pipe term at all** — a
defensible decision. What was not decided is that FINDINGS 21's buffer sizing
and its "zero required prefill" would keep standing on a constant the design had
stopped enforcing. **That is what item 4 has been open since session 7 for.**
**The output is a requirement on the medium, not a verdict**, because 488 is
unmeasured folklore:
| container | wire | **zero-prefill pipe** |
|---|---:|---:|
| `s14_d5_all1500` (the candidate) | 496.7 KB/s | **513.2 KB/s** |
| `rc_fr_singe_scsi_span` (the gate) | 446.1 KB/s | **451.4 KB/s** |
**513.2 KB/s is now a hardware acceptance test** — 33% of SCSI-1 async, 10% of
sync. Very likely met; never shown to be met. 49.5.
**5. The rig's first version measured the wrong thing, and it is worth knowing
why.** `stream.s` has no frame clock, so it outruns any finite pipe and reported
**91 of 120 frames "stalled" at a pipe the same run shows is fast enough**. It
now records when each record becomes RESIDENT and checks that against a 12 fps
deadline: **1/120 late by 4.9 ms at 488, 0/120 at 520**. 49.6.
**`decode.s` is unchanged, and provably.** The block loop and span chain moved to
`src/player/frame.i` and the constants to `geom.i` so both front-ends assemble
from literally the same bytes — every cycle constant in FINDINGS 24/30/40/41 is
fitted to those bytes. `decode.s` still assembles to **1,296 bytes**, same MD5,
and `prep_dlx.py` still emits a byte-identical blob after the loader maths moved
to `tools/bench/dlxload.py`. **Both are now asserted in `check.sh`**, along with
the ring pass itself.
---
# Session 17 and earlier
## Session 17: the blanking question is not a tie, and it leans the wrong way
**Green light re-run first: `./tools/bench/check.sh` is ALL GREEN**, 120/120 on
both cores, no `TRUNCATED`. Nothing in session 16's uncommitted tree has drifted.
**FINDINGS 48. Session 16 filed 47.4 as "two emulators disagree, both readings
plausible." That framing was too generous to the outcome we want.**
1. **px68k is silent, not dissenting.** Read with `grep -a``gvram.c` is
EUC-JP, so a plain `grep` silently reports *nothing* — R20's high byte
(`CRTC_Regs[0x28]&8`) appears in **one file and six places, all address
decode**: three in `GVRAM_Read`, three in `GVRAM_Write`. **No px68k display
code reads the bit anywhere.** It does not model buffer mode as
non-blanking; it does not model the display side of buffer mode at all.
MAME's blanking is a deliberate commented claim made twice, and it models
bit 12 the same way for the text layer. **An assertion and a silence are not
a tie.**
2. **Sharp's own register table names the bit MAME's way.** R20 bit 11 G-MEM:
`%0 表示用 / %1 バッファ用` — "for display" / "for buffer" — and
`bit 10〜8 は無効`, i.e. the colour-mode field goes *invalid* in buffer mode.
COL is what the display side decodes a plane structure from. That is a
mechanism for blanking, not just a name for it. The counter-reading survives
only on the parenthetical (`G-VRAM が 65536 色表示時と同じ構造になる`), which
describes the structure and does not say the screen goes dark.
3. **The MAME branch has no survivable partial-blank.** The blank interval is
the paint, not the frame — but the measured blit is 53.6% of budget, packed
halves the words, so the picture is dark for ~2754% of every frame at 12fps.
That is a 12 Hz strobe over the whole image, and the packed layout has **no
page left to flip to**, because both 256-colour pages carry picture. There is
no version where the packing is merely expensive.
**This is a shifted prior, not a result. It still needs the board.** But it
should be read before anyone spends a session building on 1.0 B/pixel.
**Item 2 lost its cheap method, and the loss is informative.** px68k **does not
emulate the MB89352 at all** — `x68k/scsi.c` is 81 lines that synthesise a fake
64-byte CZ-6BS1 boot ROM and trap IOCS `$F5` on the host (`SPCはエミュレートしない`,
its own header). So the "second emulator agrees" method that carried 46/47 was
never available for single- vs dual-address. What item 2 actually wants is the
**CZ-6BS1's `scsiexrom.bin`** (8 KB, CRC `7be488de`, not present here)
disassembled for the DCR it writes. **The MC68450 datasheet pins the field:
DTYP `00`/`01` are "Explicitly Addressed" (dual), `10`/`11` are "Implicitly
Addressed" (single), and 3.6.1.2 names the two protocols "dual address" and
"single address" in so many words.** 43.2's 5.0-vs-9.0 clocks/byte is two bits
in one byte the boot ROM writes at init. Not another emulator — and MAME's
`hd63450.cpp` decodes no DTYP at all, so it could never have answered.
FINDINGS 48.4.
**One thing was confirmed rather than doubted.** px68k's `kaiseki.txt` — the
author's own 2014 analysis notes, predating all of this — states the 256-colour
word interleave (`Page0の(0,0), Page1の(0,0), Page0の(1,0)...`) independently.
The packed layout's *premise* is solid from a third source; only its visibility
is in question. FINDINGS 48.5.
---
## Where session 16 left it
**The strongest test in the tree now covers the whole window, on both cores.**
FINDINGS 45. The pixel-exact gate was auditing 37 of 120 frames; it is now
**120/120 on MAME's 68000 and on px68k's C68K**, and `./tools/bench/check.sh`
is **ALL GREEN** with no `TRUNCATED` line.
Session 15 handed this over as "it needs the chunk-streaming rig, not a longer
pass." **That was wrong, and cheaply so — it needed neither.** The constraint
was the *rig's* memory, not the player's: `prep_dlx.py` preloads the whole
container into emulated RAM and `check.sh` ran the machine at `-ramsize 2M`, so
a 5,261,814 B stream was truncated to the prefix that fit. The gate runs under
`DLX_VERIFY_ONLY=1`, which drops the cost anchors entirely and asserts only
pixel-exactness, so the 2 MB was never load-bearing *here* — and preloading a
whole container is unlike the shipping player at any size, because the player
streams into a ring buffer. `RIG_RAM=6` in `check.sh` covers all 120 frames.
**The raise is licensed by measurement, not by convenience.** Run the full
timing pass at 2M and at 6M and the five synthetic anchors come out
**bit-identical** — 40,729 / 921,187 / 1,376,881 / 1,229,883 / 506,533 cycles —
despite sitting at different addresses in the two layouts. MAME's cycle model
does not depend on `-ramsize` over this range, so every per-block constant in
FINDINGS 24/30/41 is unmoved.
**And the 37-frame prefix was a biased sample.** It overstated the mean cost of
the window by 8.2%, and it never saw the quiet end:
| anchor | 37-frame prefix | full 120 |
|---|---:|---:|
| min non-SKIP | 25.1% of blocks, 61.9% of budget | **15.2%, 53.6%** |
| median | 45.4%, 79.7% | **41.1%, 81.1%** |
| p90 | 52.3%, 93.1% | **48.5%, 91.1%** |
| max non-SKIP | 62.5%, 91.8% | 62.5%, 91.8% (same frame) |
| C68K sequential-pass mean | 693,886 cyc, **83.3%** | **641,444 cyc, 77.0%** |
The direction is lucky rather than designed: the prefix was **pessimistic**, so
nothing downstream was flattered and no headroom claim was resting on the
missing frames.
**Item 2 is answered, and the answer is no.** FINDINGS 46. There is no packed
256-colour write path — no graphics mode on this machine puts two horizontally
adjacent pixels in one 16-bit word. The sub-word fields are **pages, not
pixels**: page 0 and page 1 are the two bytes of one word *at the same screen
coordinate* (px68k derives `line` identically for both, and the `$C80000` alias
lands on the adjacent byte), exactly as two independent documents describe. The
near-miss — scroll page 1 by one pixel and interleave — fails on byte count, not
on addressing, and 46.2 records why so it is not re-derived. **FINDINGS 44.7
stands unchanged**: the decoder-free player still needs 1,152 KB/s and 1.61 GB
and is still killed by the medium.
**Chasing it opened one more surface, and the same session closed it.** The 2:1
tax is a property of the *graphics* planes; the **text plane is 4bpp planar**
four planes at `0x20000` stride, 1024x1024 bits each — so 0.5 bytes/pixel against
2.0, and an **uncompressed** 16-colour frame is 288.0 KB/s against the shipping
compressed 256-colour stream's 496.7. 42% cheaper on the wire, no decoder.
**It costs 5.84 dB and that kills it.** `tools/analysis/18_text_plane_16col.py`,
120 frames, generous to the 16-colour side (per-frame palettes, which the
256-colour path cannot use because its codebooks index a scene-wide palette):
| | mean PSNR |
|---|---:|
| 256 colours, scene palette (the tree's) | 31.33 |
| 256 colours, per-frame palette | 34.08 |
| 16 colours, scene palette | 23.17 |
| **16 colours, per-frame palette** | **25.49** |
Against the shipping container's **29.19 dB at 496.7 KB/s**, a 16-colour literal
is **25.49 dB at 288.0 KB/s — 3.70 dB worse for 58% of the bitrate.** The wire
saving does not pay for the colours. **The user's call was to drop the 16-colour
direction outright and the number agrees**, so it is closed, not parked.
FINDINGS 7's 256-colour claim now rests on a measurement. (One caveat recorded
in 46.3: a Floyd-Steinberg row came out bit-identical to the undithered one —
PIL ignored `dither=` under `MEDIANCUT` — so that row is void and excluded.)
**Then the user said to keep chasing it, and the "no" turned out to be wrong.**
FINDINGS 46.5. The masking is defeatable: **CRTC R20 bit 11** ("G-VRAM set to
buffer") bypasses the depth switch and writes the **full 16 bits** unmasked.
MAME's `gvram_w` and px68k's `GVRAM_Write` both implement it — px68k's comment
names the shipping game that used it, `65536モードのVRAMアクセス(Nemesis用)`
so it is a mechanism, not an emulator quirk. **The 2:1 tax is a property of the
default write path, not of the memory.**
**And the two 256-colour pages have independent scroll**, which 46.1 also missed:
px68k's `Grp_DrawLine8(int page, int opaq)` indexes `GrphScrollX[page*8]`,
selects the byte within the word by page, and takes an opacity flag. 46.2
dismissed interleaving assuming a 1-pixel scroll; **scrolling by 128 makes the
used words contiguous**:
- write words 0..127 of each row, unmasked, full 16 bits
- page 0 unscrolled → screen columns 0..127
- page 1 X-scrolled +128, opaque, above → screen columns 128..255
- page 1's storage past 128 lands off the edge of the real 256x256 mode already
in use (FINDINGS 23)
**128 contiguous words carry 256 pixels: 1.0 B/pixel against 2.0** — 576 KB/s and
0.81 GB, exactly the halving 44.7 said would reopen the whole design. `movem`-shaped,
no stride, no transparency mask to maintain.
**This is a derivation, not a result** — see 46.6 for the three things untested.
But unlike items that need a board, **the tree can answer this one**: it is a
register setup and a snapshot, and `verify_frame256.py` already compares
pixel-exactly.
**The packed layout was then built and run on both emulators. It works.**
FINDINGS 47. 46.6 was a derivation; it is now a result:
| | result | palette ceiling |
|---|---|---:|
| MAME, `verify_frame256.py` | **256x192 pixel-exact, letterbox true black** | 40.83 dB |
| px68k, `verify_gvpack.py` | **256x192 index-exact** | 40.83 dB |
**Per-frame payload: 128 words/row x 192 rows = 49,152 bytes for 49,152 pixels —
1.0 B/pixel against 2.0.** The write path was measured directly: masked, writing
`AB5C` leaves page0=`5C` page1=`00` (the high byte destroyed); with R20 bit 11
set it leaves page0=`5C` page1=`AB`. That is the 2:1 tax and its off switch in
one table.
Four negative controls behave — bit 11 off, scroll removed, and the unpacked
control. `tools/bench/gvpack` links px68k's real `x68k/gvram.c`, the way
`tools/bench/c68k` links its CPU core.
**But the two emulators disagree twice, and the second one decides everything.**
1. *Priority register `0xE82500`.* At `0x0000` MAME hides page 1 (right half
black) while px68k puts page 0 on top transparently and renders correctly.
They agree at `0x0002`, which is what the layout uses — so the result stands,
but on a register they model differently.
2. **Does buffer mode BLANK the display?** `probe_bit11_blank.lua` is the
known-good 256-colour test with *one line added*. **MAME: the screen goes
fully black.** **px68k: it does not blank**`Grp_DrawLine8` never reads that
bit. If MAME is right, the graphics layer is blanked for the whole time the
CPU or DMAC is painting, and a 12fps player shows black for whatever fraction
of each frame the paint takes. If px68k is right, the packing is free.
**That is now the cheapest high-value hardware fact outstanding** — cheaper than
single-vs-dual-address, because one real board plus a two-line probe settles it
and it moves more numbers.
**What it is worth if it goes px68k's way (DERIVED, not measured):** wire 1,152 →
**576 KB/s**, game 1.61 → **0.81 GB**, DMAC device→GVRAM 59.0% → **29.5%** of the
clock budget. And one reversal: 44.7 concluded "a CPU-painted full-frame literal
does not fit at any physically reachable price." Packed, one `movem` word carries
two pixels, so a pixel costs `(9.143 + 2c)/2 = 4.571 + c` — **9.57 clocks at c=5,
against a 16.95 budget. It fits, at c=5 and at c=9.** Withdrawn conditionally on
the blanking question.
Reproduce:
```
python3 tools/bench/prep_frame.py tmp/fr_00020 tmp/frame256p.bin 0 --pack-transparent
( cd tmp && SDL_VIDEODRIVER=dummy mame x68000 -bios ipl10 -ramsize 2M -video soft \
-window -sound none -nothrottle -plugins -seconds_to_run 8 \
-autoboot_script ../tools/bench/show_frame256_packed.lua \
-snapshot_directory ./snap_pack -snapview native )
python3 tools/bench/verify_frame256.py tmp/snap_pack/x68000/0000.png tmp/frame256p.bin
make -s -C tools/bench/gvpack && tools/bench/gvpack/gvpack tmp/frame256p.bin tmp/gvpack_px68k.raw
python3 tools/bench/gvpack/verify_gvpack.py tmp/gvpack_px68k.raw tmp/frame256p.bin
```
Not wired into `check.sh` yet: it is a capability probe, not something the
shipping player depends on. It should join the green light the moment anything
in the tree starts relying on the packed layout.
Green light: `./tools/bench/check.sh` **ALL GREEN**, before and after.
## NEXT SESSION, in order
**AS OF SESSION 32 the top of the list is this, and the rest of this section is
the session-19 list kept for its detail.** `docs/ROADMAP.md` is the current
shape; where the two disagree, ROADMAP is the one that was edited last.
1. **B2, and it is now the fork rather than a headroom question.**
`tools/bench/probe_bit11_blank.lua` is written and settles it in minutes on a
real board. FINDINGS 64.2 is why it moved to the top: **if buffer mode
blanks, the K3 player that now exists shows nothing at the container's own
wire and K4 is the player; if it does not blank, K3 is the player and K4's
27.3% paint is spent on nothing.** Each answer makes the other branch's work
wasted, so building either before the answer is the one thing to avoid.
2. **B1 gained a THIRD number: the DATA-PHASE BURST RATE** (64.2). Sustained
throughput decides whether record *i* arrives before slot *i*; the burst rate
decides how much of the slot the picture is on screen. A drive with a
read-ahead cache can pass one and fail the other. The acceptance test is
`record / (burst x slot)` = the dark fraction, and it is **1.0 at 582.0 KB/s**
and 0.273 — the same as K4's paint — only at **2,131 KB/s**.
3. **K4, and NOT before item 1.** ROADMAP has the shape. The one thing in it
that has never been run is the **overlap**: a channel filling buffer *i+1*
while the CPU paints buffer *i*, which is the first time the DMAC and the
68000 have had to want the bus at the same time for a whole scene.
4. **Diagnose MAME's transport ceiling, or stop quoting it.** ~535 KB/s a record
in both channel configurations, mechanism unknown, no MAME source on this
machine. It bounds nothing — the arithmetic in 64.2 is geometry — but it is
why `check.sh`'s packed stage is paced at half rate, and an undiagnosed
apparatus constant is exactly the shape of thing FINDINGS 42.1 caught.
---
**Item 4 is CLOSED (session 19, FINDINGS 51).** The list below is otherwise
unchanged, and items 1-3 are all still blocked on hardware this machine does not
have. What session 19 adds to item 1 is a SECOND number to measure the medium
against, and it is the harder one:
- **451.4 KB/s** is the zero-prefill pipe for the gate container (49.5) — enough
to arrive on time in a straight line.
- **Absorbing a seek is a different requirement.** At 460 KB/s every ring from
192 KB to 512 KB is rate-bound and never fills. The rate that makes a branch
point affordable is set by `pipe - wire` and how long the scene runs between
branches, not by the buffer — so measuring the BlueSCSI's *seek time* matters
as much as its throughput, and neither is known. 51.3/51.4.
Item 5 (`--spans all`) now has a second reason to be careful: it spends every
profitable byte, which raises `wire`, which shrinks `pipe - wire`, which
lengthens the climb back to a full ring after every branch. That interaction is
not priced.
## The list
0. **Green light first.** `./tools/bench/check.sh`. It is slower again: it now
also runs the 120-frame **ring-buffer** pass on a 2 MB machine, re-derives
`prep_dlx.py`'s blob, and asserts `decode.bin`'s MD5.
**Do not run two MAME jobs at once** — session 18 did, and two `decode.lua`
runs sharing one log file produced a 0-byte log and 15 wasted minutes. The
warning is already in this document; it is there because it keeps happening.
1. **Measure the BlueSCSI. It is now the only thing that unblocks the I/O side.**
FINDINGS 50 retired the pipe constant outright (USER DECISION): it is gone as
a default from every analysis tool and from `stream.lua`, which now all
REQUIRE an explicit rate. **There is no working delivery figure in this repo
any more, and that is deliberate.**
The consequence is that anything needing a delivery rate to mean something is
now waiting on a measurement — which is the honest state, and it is why this
is item 1. `19_ring_stream.py` gives the threshold to measure against:
**513.2 KB/s** for the session-14 candidate, **451.4 KB/s** for the gate
container. Zero-prefill pipes, per container, from real record sizes.
**Do not substitute a guess.** If the measurement is not available this
session, run the tools at several explicit rates and report the sensitivity,
rather than picking one and letting it become the next constant. That is
precisely how the last one survived five sessions after 42.1 called it
folklore.
If it clears 513.2, the 8.7 KB/s overrun of 49.5 is not one, and the rate
controller can go on binding purely on clocks *with that fact written down*.
If it does not, the two levers are giving the rate controller a pipe term
(it has none — it binds on `decode + c*bytes`) or re-encoding lower; the
first is more informative and neither should be priced before the
measurement exists.
2. **Settle whether buffer mode blanks the display.** Unchanged, and still
blocked on a real board. **Read FINDINGS 48 first — session 17 shifted the
prior toward MAME and toward "unusable."** `tools/bench/probe_bit11_blank.lua`
is written and settles it in minutes. Second, smaller, same sitting: the
priority register `0xE82500` at `0x0000` (47.3). **Do not pre-build on
1.0 B/pixel.**
3. **Settle single-address vs dual-address.** Unchanged, still the largest open
number: 242 KB/s and 0.69 dB. **Session 18 confirmed the artefact is not on
this machine**: `~/mame/roms/x68000.zip` has `cgrom`, four IPLs and `sram`,
and no SCSI ext ROM anywhere on the box. So this needs `scsiexrom.bin`
(8 KB, CRC `7be488de`) sourced, then its DMAC init disassembled for **DCR's
DTYP** field: `10`/`11` = single (5.0 clk/B), `00`/`01` = dual (9.0).
FINDINGS 48.4. A board or schematic is the fallback, not the first move.
4. **DONE, session 19 — FINDINGS 51.** ~~Pace the decoder, then measure
branch-point stall tolerance.~~ The rig has a frame clock, the ceilings are
measured at seven ring sizes and five pipe rates, the `ceiling - 1` tolerance
was falsified by cutting the pipe, and the result that matters is that slack
is accumulated: 4.83 s of play to refill 256 KB at 488 KB/s. The original
statement of the gap, kept because it is still the right description of what
was wrong: FINDINGS 49.7.2 is the sharpest gap the new rig leaves: because `stream.s` free-runs,
the ring never backs up, so the ring-size sweep tests **wrap correctness** at
each size and **not buffering**. 48 KB passes and is single-buffered — do not
read it as a viable player buffer. A paced decoder (MFP timer, or Lua gating)
turns the rig into the seek-tolerance test Dragon's Lair actually needs, and
it is the last piece of the delivery story.
5. **Make `--spans all` the default.** Still a recommendation rather than a
measurement (43.6.1), and 44.3 sharpens it: it is the only lever on the
encoder's byte side that changes anything. **Note it interacts with item 1**
`all` spends every profitable byte, which is what put the candidate over the
pipe in the first place.
6. **Re-derive span selection jointly with `lam`** (39.3). The selector ranks on
net clocks; `lam` still prices bytes against a KB/s target rather than
against 5 clocks each — and if item 1 goes route (b), `lam` gets a real KB/s
ceiling back and this question changes shape.
## What session 15 settled
**Session 14's item 2 was aimed at a lever that is not loaded.** FINDINGS 44.
The bucket diagnosis of 43.5 is correct as a mechanism and worth one frame of
120 in practice, because at `--spans all` the byte side of the rate controller
does not bind at all:
| `--spans all`, c=5, the 120-frame `singe` window | KB/s | PSNR | over budget |
|---|---:|---:|---:|
| **shipped, `--bucket-frames 8`** | **496.7** | **29.19** | **1/120** |
| `--bucket-frames 32` | 496.7 | 29.19 | 1/120 |
| `--bucket-frames 1` — no banking at all | 498.0 | 29.19 | 1/120 |
| `--rc-floor open` — lam floor 1.0 | 503.7 | 29.21 | 1/120 |
A 32-frame bucket emits the **same container byte for byte** as an 8-frame one,
and `lam` never leaves its floor of 10.0 on any of 120 frames. The rate this
project reports is set by the span pass and by `mu`; `--kbps` and the bucket
are not the levers. Measure whether the lever is loaded before pulling it.
**Two real unit inconsistencies were found, fixed, and defaulted OFF on
measurement** — `--joint-decide` and `--joint-bucket` turn them on:
| c=5, `--spans all` | KB/s | PSNR | mean frame clocks | over |
|---|---:|---:|---:|---:|
| **shipped** | **496.7** | **29.19** | **740,049** | 1/120 |
| `--joint-decide` — the lagrangian sees the disk | 482.5 | 29.17 | 745,438 | 1/120 |
| `--joint-bucket` — the bucket may not lend clocks | 506.4 | 29.18 | 754,429 | 1/120 |
The first is the more interesting one even though it does nothing: priced per
delivered byte, a RAW block costs `400.4 + 16c` and a V4 block `448.2 + 4c`,
which **cross at c = 3.98 — just under 43.1's hard floor of 4.** So `mu`'s
V4 -> RAW escape hatch, FINDINGS 28.8 and session 8's `0c`, never existed on
real hardware: it spends 12 clocks of bus to save 47.8 of CPU. Correcting it
moves 0.3% of blocks and 0.02 dB.
Defaults are unchanged in effect: `s14_d5_all1500` re-encodes to the same MD5.
**An encode is 95% k-means, and it is now 2.7x faster, exactly.** 60.6 s -> 29.4 s
for a 120-frame window, from three fixes to `VQ.assign` (a materialised `C.T`, a
cache-sized chunk, and a thread pool over the chunk loop). Bit-identical labels,
so every container still hashes the same. FINDINGS 44.5.
**A decoder-free player fits the clocks and dies on the medium.** Streaming raw
preprocessed frames straight into video memory — DMAC device -> GVRAM, no
decoder in the loop — fits at c=5 with 41% of the frame to spare, but needs
1,152 KB/s sustained (~79% of SCSI-1 async, nothing left for audio or seeks) and
1.61 GB for the game, against the 0.70 GB the shipping container needs. The
cause is that 256-colour GVRAM is one pixel per word with the high byte
discarded, so half of every byte pulled off the disk is thrown away by the
hardware on arrival. FINDINGS 44.7. **Session 16 confirmed that layout is not
escapable on the graphics planes (FINDINGS 46) — so this stands.**
## What session 14 settled
**Session 13's binding unknown `W` was in the wrong unit, and correcting it
costs the project 41% of its rate and 1.85 dB.** FINDINGS 43.
`W` was charged per WORD of delivered stream. The MB89352 is an **8-bit** SPC,
so the DMAC pays per BYTE. Every I/O debit in the project since FINDINGS 5 has
been charged at half rate, and the favourable end of 39.7's 5..12 bracket was
never physically reachable: **a 68000 bus cycle is four clocks and the SPC
hands over one byte per cycle, so nothing can cost less than 4 clocks/byte.**
5 clocks/word is 2.5 clocks/byte — 62% of one bus cycle, and a 4 MB/s DMA on a
link that runs at 1.5 MB/s asynchronous.
The datasheet, per byte, device-to-memory, bus held (MC68450 Fig 4-25):
| how the DMAC is programmed | clocks/byte |
|---|---:|
| single address, D->M (sheet 2) | **5.0** |
| dual address, byte, no packing (sheet 4) | **9.0** |
| dual address, byte packed (sheet 3) | 16.5 |
**Nothing session 13 emitted fits at any of these.** `s13_280p1500`, the
0/120 candidate, is 44/120 at 5 clocks/byte and 120/120 at 9.
### What survives, and it had to be re-encoded rather than re-scored
The encoder was making its decisions in the same wrong units: `ratectl`
bisected `mu` against 833,333 cycles of *decode* with no disk term, and
`spans.select()` admitted a run only if it beat the blocks **on cycles alone**,
explicitly ignoring the bytes it added. Both now work in one currency — clocks —
and `--disk-clk-byte 0` re-emits session 13's container byte for byte, so the
change is the price and not the codec.
| | KB/s | PSNR | over budget | span px |
|---|---:|---:|---:|---:|
| s13's claim (2.5 clk/B) | 837.4 | 31.04 dB | 0/120 | 62.3% |
| **`s14_d5_all1500` — single address** | **496.7** | **29.19 dB** | **1/120** | 30.7% |
| `s14_d9_all1500` — dual address | 255.0 | 28.50 dB | **1/120** | 3.9% |
**The one frame over is frame 0 in both** — the intra frame, which 28.5/31
established is emitted late on purpose. Every other frame lands at or under
100.0%. `17_span_delivered.py` shares no code with the encoder's accounting and
reproduces both rows to the digit.
**The remaining lever is not ours to pull.** A span pixel costs `9.143 + 2c`
clocks and a V1 block pixel `18.74 + 0.25c`, so **a span beats a block only for
`c < 5.48`** — which falls between the two rows above. Single-address needs the
CZ-6BS1 to drive the SPC's DACK from the bus's `#EXACK` (pin B37, which the bus
does have). That is the *board's* wiring, not our code, and MAME models only the
dual-address row.
Green light: `./tools/bench/check.sh` **ALL GREEN**, re-run after the encoder
change — 0/120 rate-control drift under the joint cost, DLX3 round-trip exact.
### session 14's list
Items 0, 2 and 4 are answered by FINDINGS 44; the rest carry forward,
reordered, in the session-15 list at the top of this file.
0. **Green light first.** `./tools/bench/check.sh` (~6 min, Blu-ray mounted).
1. **Settle single-address vs dual-address.** It is worth 242 KB/s and 0.69 dB
and it is the largest open number in the project. It is a **hardware** fact:
the CZ-6BS1 schematic, or a real board, or a period service manual. MAME
cannot answer it (`x68k_scsiext.cpp` has no DACK path and no DRQ line from
the slot to the DMAC at all, though the real bus has `#EXREQ`/`#EXACK`), so
do not spend a session in the emulator on it. FINDINGS 43.2/43.3.
2. **Fix the rate controller's bucket.** Diagnosed, not fixed (43.5): it banks
bytes across 8 frames, and bytes are now clocks, which cannot be banked
because there is no double buffer. `corr(bytes, frame clocks) = 0.989`. The
per-frame ceiling should be joint and hard; the bucket should smooth only
what is left after the disk is paid.
3. **Get a 68000 to decode one of the new containers.** Unchanged from session
13's item 3 and still the strongest claim resting on the weakest coverage —
but *narrower* now: `s14_d5_all1500` is 30.7% span coverage against the
gate's 26.6%, where session 13's candidate was 62.3%. This is the cheapest
it will ever be to close. A 496.7 KB/s stream is ~10 frames in a 2 MB
machine, so it still wants a chunk-streaming rig, not a longer pass.
4. **Make `--spans all` the default** (my recommendation, not a measurement):
with a joint cost, `need` stops early and leaves profitable clock savings
unbought — 275.8 KB/s / 28.92 dB / 2 frames over, against `all`'s 496.7 /
29.19 / 1. FINDINGS 43.6.
5. **Re-run the ring-buffer simulation at ~497 KB/s** (29.5/30.7, open since
session 7). Cheaper than the 837 KB/s it was going to have to survive.
6. **Re-derive span selection jointly with `lam`** (39.3). Half-done: the
selector now ranks on net clocks, but `lam` still prices bytes against a
KB/s target rather than against 5 clocks each.
**Do not start by hand-optimising `decode.s`.** Five sessions running, and more
true than ever: at `c=5` the disk is 26.6% of the median frame and the decoder
is not what misses.
**Do not treat 1/120 as settled either.** FINDINGS 43.8 lists five things it
does not establish, and 43.9 is about how the last favourable table survived a
section written specifically to distrust it.
---
## What session 13 settled
Session 12 ended by asking the user to choose a rate point. The user instead
challenged the constant the question rested on — and it does not hold. The
488 KB/s figure was never a SCSI bus figure: **SCSI-1 is 1.5 MB/s asynchronous
and 5 MB/s synchronous** (ANSI X3.131-1986), so the design has been running
against 10% of the bus. Once bytes are that cheap the span pass simply
saturates, and it does so on its own at ~837 KB/s.
| container | KB/s | span px | PSNR | over @5 | @8 | @12 |
|---|---:|---:|---:|---:|---:|---:|
| the session-12 gate | 487.7 | 26.6% | 29.63 dB | 11/120 | 34/120 | 78/120 |
| **`s13_280p1500` — the candidate** | **837.4** | **62.3%** | **31.04 dB** | **0/120** | 21/120 | 63/120 |
| `s13_lossless` — a probe, not shippable | 934.6 | 70.4% | 31.19 dB | **0/120** | 47/120 | — |
**0/120 is the first time anything here has fitted 12fps on every frame**, and
it is simultaneously the best picture yet — 0.29 dB off a 31.33 dB palette
ceiling, i.e. off exact for this display. `mu` is never spent at these rates, so
FINDINGS 31's 0.62 dB CPU-fit cost is refunded. **Quality and deadline have
stopped competing**, which was true of no earlier session.
The mechanism: a v7 span pixel costs 2 wire bytes and **9.143 clocks**, a RAW
block pixel costs 1 byte and **25.03**. Spans win for any `W < 31.7`. The budget
is 16.95 clocks/pixel, so **a full-frame literal fits iff `W <= ~6.5`.**
> **Withdrawn, session 15.** That threshold is per WORD: 6.5 clocks/word is
> 3.25 clocks/byte, under the 4-clock floor, so it was never reachable. In byte
> units the breakeven is **c = 3.906** against a floor of 4.0 — a CPU-painted
> full-frame literal misses at every real price, by 1.1% at the floor itself.
> FINDINGS 44.7.
### Which makes `W` the whole result
| `W` clocks/word | 934.6 KB/s stream | over budget |
|---:|---|---:|
| 5 | median 83.0%, worst 91.0% | **0/120** |
| 6 | median 87.6%, worst 97.0% | **0/120** |
| 7 | median 92.2%, worst 103.0% | 21/120 |
| 8 | median 96.8%, worst 109.0% | 47/120 |
Moving `W` across FINDINGS 39.7's datasheet bracket costs more frames than
moving the rate from 280 to 935 KB/s wins. It has displaced the 4 Mbps figure as
the most load-bearing unmeasured number in the project.
**Two things are known about `W` and they point the good way.** MAME's CZ-6BS1
glue is cycle-steal **with the bus held**`x68k_scsiext.cpp:110-136` gates
`#DTACK` on `DRQ` while the DMAC's OWN is asserted — which is the 5 clk/word end
of the bracket, not the ~12 arbitrated end. And `W` decomposes into `5 clocks of
MC68450 + however long the drive makes the DMAC wait`; the deployment target has
been SD-backed SCSI since session 2 (FINDINGS 21.2), which collapses the wait
term. FINDINGS 42.5, 42.6.
**Also corrected: "never `x68ksupr`" was never a hardware claim.** Internal and
external are the same MB89352 and neither needs a driver (IOCS is in ROM); the
difference is that MAME has not implemented the internal DMA glue
(`x68k.cpp:1176`, `// TODO`). The external board **is** the way to test the DMA
path in MAME, and that is what item 1 below does.
Green light: `./tools/bench/check.sh` **ALL GREEN** at the start of session 13,
unchanged since. Nothing in session 13 touched the encoder, the decoder or the
container — this session moved numbers and documents only.
**Session 13's list is superseded** — its items 1 and 2 are done and its
answer did not survive them (FINDINGS 43); the rest are carried forward,
reordered, in the session-14 list at the top of this file.
**And its headline is withdrawn.** The 0/120 above, the 31.04 dB, the
saturation at 837 KB/s and "quality and deadline have stopped competing"
were all scored with the disk debited per WORD to a byte-wide port. The
reasoning in this section about the SCSI bus rating (42.1) stands; every
number downstream of it was charged at half rate. Read FINDINGS 43.
---
## What session 12 settled
Session 12 built v7 into the player. **`src/player/decode.s` paints v7 literal
spans, and it is pixel-exact under both CPU cores** over a container where every
frame carries 128-216 spans covering up to 38% of the picture. FINDINGS 41.
The container is **DLX3**: a span section between the mode header and the block
payload, `{u32 GVRAM address, u16 coarse disp}` per span with the fine
displacement mid-stream. `tools/analysis/16_span_roundtrip.py` gates it and is
in `check.sh`.
**The measured cost transfers.** Two synthetic all-SPAN anchors price v7 inside
`decode.s` at **151.2 and 225.6 clocks per 4x4 block**, against FINDINGS 40's
table of 151 and 226 — 0.2% on both emulators.
### The two things that were not on the list
**1. There are TWO byte budgets, and FINDINGS 40's 18/120 was scored at the
wrong one.** The `scsi` profile is 280 KB/s; `14_dmac_chain.py` scores spans
against the 488 KB/s PIPE, which is 40,977 B/frame against 23,228. At the
profile rate the lam search has already spent the allowance and spans fire on 5
frames of 120. The profile is a chosen quality rate point; the pipe is hardware.
`--kbps` and `--span-kbps` are now separate, and spans run before `mu` because a
span pays in bytes and `mu` pays in picture. FINDINGS 41.2.
| 120-frame `scsi` window | KB/s | over budget | PSNR |
|---|---:|---:|---:|
| no spans | 278.3 | 86/120 | 29.27 dB |
| spans, profile budget only | 280.0 | 77/120 | 29.23 dB |
| **spans on the 488 KB/s pipe** | 487.7 | **34/120** | **29.63 dB** |
**2. `C_SKIP_MIXED` was never measured, and it was 18% low — 45.0, now 55.0.**
It is the one constant in the cost table that came from a derivation, because
the synthetic frame that would measure it cannot exist (a byte needs a coded
block for its SKIP to be mixed). Four new bracketing anchors measure it on both
emulators, and with it corrected the model predicts a real spanned decode to
**-0.06% mean / 0.09% worst**, against -2.99% / 4.30% before. It matters here
because **a span marks its run SKIP**, so mixed SKIPs are the dominant
population in exactly the frames spans are judged on. FINDINGS 41.5.
**And the metric everything has been quoted in is unstable.** 34/120 delivered
against 14's simulated 18/120 is a 1.4% difference in mean frame cost. 55 of 120
frames sit within 5% of the deadline because the rate controller aims there, so
a 1% cost shift moves 22 frames. Quote the distribution, not the count.
FINDINGS 41.6.
Green light: `./tools/bench/check.sh` **ALL GREEN**, now gating on a span-heavy
DLX3 container.
---
## What session 11 settled
Session 11 measured v7 in `blit.s` and left it there; session 12 built it into
the player. Items 0 and 1 of session 11's list are done (FINDINGS 40, 41) and
the rest are carried forward in the list above.
## What session 10 settled
Session 10 cross-checked the whole cycle model against a second emulator, then
found that the model was denominated in the wrong currency.
**FINDINGS 38 is the headline: the project is BUS-bound, not CPU-bound.** Nothing
since FINDINGS 24 had counted the 68000's local memory bus — one 4-clock cycle at
a time, carrying instruction prefetch as well as data. Measured, the decoder
occupies **86.7%** of it, and **prefetch is 62% of that traffic**. Scoring the
`scsi` window with the bus as the shared resource, **52 of 53 missed frames are
bus-limited and one is CPU-limited**. Every optimisation since 24 has been aimed
at the budget that is not binding.
The measurement is two sources checking each other: `c68k_bench` counts every bus
callback exactly, and a static walk of `decode.lst` adds the prefetch no emulator
here can report. The walk reproduces the measured data half to **0.04%**, which
is what licenses its prefetch half.
**The DMAC array-chain LOSES, on the datasheet.** My first pass derived its cost
from bus arithmetic — a read cycle plus a write cycle, 8 clocks a pixel — and
scored it at 1/120 frames over budget against v6's 10/120. Then I read the
MC68450 manual (Motorola Jul 1989, bitsavers). **Fig 4-25 sheet 4: a
dual-address word between two 16-bit ports is 9 clocks, because note 2 gives the
DMAC 4-clock reads and 5-clock WRITES.** The 68000 writes in 4.
| per pixel | clocks | source |
|---|---:|---|
| DMAC dual-address word | **9.000** | MC68450 Fig 4-25 sheet 4 |
| v6 `movem` chain | **9.152** | MEASURED, FINDINGS 30 |
Scored additively (see below), against the same mode maps:
| | frames over budget |
|---|---:|
| today | 84/120 |
| v6 span as built | 55/120 |
| **v6 with a finer chain tail — software only** | **18/120** |
| DMAC chain | 12/120 |
**86% of the DMAC's advantage is v6's 24-pixel padding quantum**, which is a
property of v6's unrolled `movem` chain and fixable in software: add a second
chain of 2-register units for the tail and the padding drops from 23 pixels to
3, at no per-span cost. **Recommendation (mine): fix the quantum, drop the
DMAC.** Six frames of 120 does not buy a reserved channel, a two-region
container layout, and a timing dependency neither emulator here can verify.
FINDINGS 39.
The container work is not wasted either way: v6's record and an HD63450 chaining
entry are both 6 bytes, so **the chain array is the span table** and nothing has
to change if a hardware measurement later moves the DMAC's number. FINDINGS 39.1.
**I also got FINDINGS 35 wrong before the datasheet corrected me.** I argued its
flat CPU debit was too pessimistic and rescored the window at 53/120 using
`max(CPU, bus)`. A 68000 has no cache and a two-word prefetch queue, so it
stalls the moment another master takes the bus, and the MC68450 hands the bus
over in *slabs* under limited-rate auto-request rather than interleaving per
operand. **DMA is additive. 84/120 stands.** What 86.7% bus occupancy really
says is that there is almost no room to overlap anything on this machine.
FINDINGS 38.3.
**And the datasheet settled STATUS's own "most load-bearing unmeasured number".**
FINDINGS 5's 8 clocks/word for the SCSI DMA: Fig 4-25 sheet 3 gives
single-address write 5 clocks, so it is **5 clk/word with the bus held and ~12
if the DMAC arbitrates per word**. 8 is the midpoint of a bracket the datasheet
supports. Which end applies is a player design decision — how the MB89352 drives
REQ, and whether cycle-steal-with-hold is used — worth 7 clocks a word on a
480 KB/s stream. FINDINGS 39.7.
**The decoder is now pixel-exact under two independent CPU cores.**
`tools/bench/c68k/` links px68k's C68K core into a headless harness and decodes
the same container to the same pixels. Cycle-table error against MAME is bounded
at **3.3%**, and it runs against us. Two incidental results worth keeping: MAME
0.277's `M68000` is the **microcode core**, not Musashi (`m68000.lst` +
`m68000gen.py`), so this is two structurally different timing models agreeing
rather than two tables; and FINDINGS 28.8's "V4 costs more than RAW" reproduces
independently. FINDINGS 37.
---
## What session 10 put on the list
Kept for the reasoning, not the priorities: items 0 and 1 are done (FINDINGS 40)
and the rest are carried forward, reordered, in the session-11 list at the top.
Everything below this point is from session 9 or earlier and still stands unless
a later section above says otherwise.
### session 10's list
0. **Green light first.** `./tools/bench/check.sh` (~5 min, Blu-ray mounted).
Verified green at end of session 9. The gate container is now
`tmp/rc_fr_singe_scsi_cpufit.dlx`, and it decodes **80 of 120 frames**, not
120: the rig loads the whole stream into a 2 MB machine and a `scsi` window
is 2.84 MB, which overran the top of RAM by 940 KB and produced a failure
that looked exactly like a decoder bug. `prep_dlx.py` now truncates to what
fits and says so; `verify_decode.py --nframes` replays the same prefix; and
the stage fails loudly if the sequential pass did not complete. FINDINGS 36.
At 278.6 KB/s a 2 MB machine holds ~6.7 s of stream, so **the strongest test
in the tree can only ever audit a prefix of a window.** Getting the coverage
back means gating on more than one window, not one longer pass.
1. **Measure the finer chain tail with `span.sh`.** It is the largest single win
on the table — 55/120 to 18/120 — it is software only, and the tool that
measures it already exists: `blit.s` gains a v7 with a 2-register tail chain,
`prep_spans.py` generates the streams, `span.sh` times all of it in ~25 s and
asserts every config drew a pixel-exact frame. The 18/120 figure is DERIVED
from a conservative 56-clocks-per-4-pixel-unit model and should not be
believed until it is measured. FINDINGS 39.4.
2. **Make sure the player actually gets DMA.** DMA-vs-PIO is a property of our
code, not the board: the CZ-6BS1's DMA path is real and modelled
(FINDINGS 32.4), but if the player reads through IOCS and IOCS does PIO we
get the 120/120 row. `docs/BENCHMARK.md` item 4 (drive the MB89352 directly)
is no longer an optimisation. Benchmark `x68000 -exp1 cz6bs1`, **never
`x68ksupr`** — MAME's internal SCSI has no DMA glue at all (`// TODO:
duplicate DMA glue from CZ-6BS1`) and would measure a PIO fallback the real
machine does not have.
3. **Re-decide the framerate, now that it is the lever that pays for the disk.**
10 fps absorbs the DMA steal on current estimates. This was item 5 and a
quality question; it is now arithmetic. It is still the user's call.
4. **Then spans, on the CPU.** The format is fully specified by measurement
(FINDINGS 30.2, costs 30.5, scene-cut arithmetic 30.6) and `decode.s` does
not implement them yet. Build the v7 tail from item 1 into it. The DMAC
variant is costed and loses (39.5); the container is identical either way
(39.1), so nothing is foreclosed.
5. **Re-run the ring-buffer simulation at the surviving rate** and confirm the
488 KB/s figure's provenance (FINDINGS 29.5/30.7, still open).
**Do not start by hand-optimising `decode.s`.** The hand-derived timings agree
with the measurements to 0.5% on V1 and 1% on RAW (FINDINGS 28.4), and FINDINGS
34 confirms the model on a second container. The inner loop is close to what the
instruction set allows; the cycles to be won are in the budget, not the loop.
**Three of this session's near-misses had the same shape: an unobservable run
almost produced a false finding.** The 23-minute "hang" (FINDINGS 34.1), the
RAM overrun that looked like a 4x-slow decoder (FINDINGS 36), and the truncated
gate that reported 49,005 differing pixels. In each case the instrument was
broken, not the thing being measured. **Always `stdbuf -oL` a MAME job that
prints progress.** A file is
block-buffered too, so a long run is unobservable until it exits — and an
unobservable run that is merely finishing looks exactly like one that is wedged.
Session 8 lost this measurement to that; session 9 lost 23 minutes to it before
spending 25 seconds getting the answer with `stdbuf`. FINDINGS 34.1.
---
## What session 8 settled
0. **The mode decision can see cycles, it is on by default, and it costs
0.26 dB.** `decide(ctx, lam, mu)` minimises `D + lam*bytes + mu*cycles`;
`ratectl` bisects mu per frame against a HARD 833,333-cycle ceiling (bytes
bank in the ring buffer, cycles cannot — there is no double buffer to decode
ahead into). `sasi` 37/120 misses -> 1, `scsi` 51 -> 1. Bitrate does not
move: mu changes which modes are bought, not how many bytes. FINDINGS 31,
`tools/analysis/13_cpu_ratectl.py`.
0b. **28.7's "11 frames are impossible" was too pessimistic — it is 1.** That
floor held the SKIP set fixed; the real decision can also move a block to
SKIP, which above ~90% non-SKIP is the only lever left. FINDINGS 31.3.
0c. **V4 collapses when cycles are priced**, as 28.8 predicted: 25.2 -> 20.3%
of blocks at `sasi` and **15.0 -> 5.3%** at `scsi`, where RAW takes it. RAW
is dearer in bytes and cheaper in cycles, so the byte lagrangian's preference
inverts and only the byte-rich profile can take the escape.
0d. **SKIP's price depends on its neighbours, and the way out is two cost
functions**: a ranking constant inside the per-block lagrangian, the exact
clustered rule (`vq_hybrid.cycles`, validated to 1 point against the 68000)
for the frame-level bisection. That function is now defined once and imported
by `11_cpu_budget.py`. FINDINGS 31.4.
0e. **Both controllers are gated against decoder drift.**
`09_ratectl_drift.py` runs bytes-only AND bytes+cycles; both 0/120.
1. **The span is measured: 43.7 cycles/span + 9.152/pixel, fitted to 0.3% over
eleven span lengths.** `tools/bench/blit.s` v5/v6, `prep_spans.py`,
`span.lua`, driven by `tools/bench/span.sh` (~25 s, not in `check.sh`
because it is a wall timing). FINDINGS 30.
2. **Only in an encoder-assisted format.** `{u32 absolute GVRAM address, u16
jump displacement}` into an unrolled chain, versus `(x, npix)` and a decoder
that works it out: 43.7 + 9.152 against 97.9 + 10.46. All the arithmetic a
span decoder would do per frame is known at encode time. FINDINGS 30.2.
3. **The per-pixel cost is a function of register pressure**, which FINDINGS 24
could not have shown: 9.08 was a fixed blit with 12 registers free, v5 can
spare 8 and pays 10.46, v6 gets 12 back by making the encoder hold the state.
4. **Short spans die in the remainder path, and the fix is padding.** A 12-pixel
span costs more than a 16-pixel one in v5. v6 has no remainder path: lengths
are multiples of 24 pixels, padding is free of everything but bytes, and an
overrun past the visible 256 lands in the invisible half of the 1024-byte
line stride. FINDINGS 30.3.
5. **Odd-`x` alignment is free** (259.0 vs 261.8 cycles/span) — expected on a
16-bit bus, now measured rather than assumed.
6. **The trade is smaller than 29 derived but the conclusion holds**, including
29.4's reopening of the scene cut. All 23 timing configs also drew a
pixel-exact frame, so nothing here was timed against a decoder that skipped
work. FINDINGS 30.5/30.6.
---
## What session 7 settled
1. **68000 code parses a bitstream and draws frames, pixel-exact.**
`src/player/decode.s` + `tools/bench/decode.lua`. 120 frames of the Singe
window decoded in sequence, all four block modes, verified against the new
reference decoder `tools/encoder/dlx.py`. Because SKIP blocks are claims
about the previous frame, the last frame is only right if all 120 were.
In `check.sh` now. **FINDINGS 28.**
2. **It does not fit.** Mean 81.7% of a 12fps frame, p90 116.4%, worst 135.8%;
31% of frames miss at `sasi`, 42% at `scsi`. Zero-wait-state floor, as ever.
3. **The dual-display-path plan (FINDINGS 24.5/25.6) is withdrawn as incoherent
— the sixth false premise this project has caught.** The compose path needs a
RAM copy of the previous reconstruction; the direct path's whole selling
point is that it keeps none. Mixing them displays stale pixels on **70 of 120
frames**, worst frame 18.8% of the screen. Every coherent repair is worse
than not mixing. `tools/analysis/10_pathmix_drift.py`, kept runnable as a
counterexample and gated in `check.sh`. FINDINGS 28.1.
4. **24.5 also compared a copy against a copy.** Its 53.6% and 76.6% both come
from `blit.s` and neither includes decoding. Compose = decode-into-RAM *plus*
the 53.6% blit, so it is strictly dearer than decoding into GVRAM. There was
never a crossover. The player has **one path and no reference frame**, which
also gives back 96 KB.
5. **The four block modes cost 300 / 448 / 400 cycles, not one number.** V4 is
1.49x a V1 block while the mode decision charges it 4x the bytes. The 24.5
model is 2.03x optimistic at the median. `tools/analysis/11_cpu_budget.py`
reproduces all four frames timed on the 68000 to within 1 point. FINDINGS 28.2.
6. **The container is big-endian but not aligned, and on a 68000 that is an
address error, not a slow read.** Frame records are variable-length and laid
end to end, so their boundaries land on odd addresses. Frame 0 decoded
perfectly, then the length read for frame 1 vectored into the IPL and sat
there for 59 emulated seconds looking like an infinite loop. Found by dumping
PC and the address registers — the code was right, the data layout was not.
FINDINGS 28.3. **Encoder gap: `encode.py` should pad records to 4 bytes.**
Measured cost 1.66 B/frame = 20 B/s against 110 KB/s.
7. **A full frame does not fit at 12fps in any mode.** All-V1 is 110.5%, all-V4
165.2%, all-RAW 147.6%. At most ~88% of the screen can change in one frame
however cheaply it is coded, and scene cuts change 100%. FINDINGS 28.5.
---
## What session 6 settled
1. **Rate control works, is wired in, and is ON by default.** `encode.py`
bisects lam per frame under a leaky bucket; `--fixed-lam` restores session 5
behaviour. FINDINGS 27.
2. **Both overshoots are closed for under 1 dB.** On the Singe window, totals
including audio: `sasi` 137.4 -> **109.5 KB/s** (target 110) for -0.60 dB,
`scsi` 381.6 -> **280.0 KB/s** (target 280) for -0.91 dB. Zero frames hit the
lam=800 cliff at either profile. FINDINGS 27.2.
3. **The FINDINGS 26 desync is gone by construction, not by tuning.** The
encoder is frame-drivable (`vq_hybrid.frame_ctx` / `decide` / `paint`) and
rate control feeds back the frame it actually emitted. The regression test
`tools/analysis/09_ratectl_drift.py` goes 111/120 drifting frames -> **0**,
and it is now part of `./tools/bench/check.sh`. FINDINGS 27.1.
4. **Rate control makes the display path cheaper.** Raising lam moves blocks to
SKIP and V1, so there is less to write: `scsi`'s median display cost drops
53.6% -> 47.1%. The decoder conclusion of 25.6 is unaffected. FINDINGS 27.3.
5. **FINDINGS 26.5 was wrong in both halves, and this is the fifth false premise
this project has caught.** `_paint` was not the bottleneck (14% of a frame)
and the ladder was never "minutes" (~18 s; the minutes were k-means in
`build`). Vectorising it was still right — 17.1x — but what actually makes
per-frame rate control affordable is that `VQ.assign` depends on neither
`lam` nor `prev`, so it is cached: a 12-step search over 120 frames costs
**0.31 s** against 49.1 s. FINDINGS 27.6.
6. **`--prefill` is a trap and defaults to 0.** It buys a permission to overshoot
of exactly bucket/nframes; at prefill=1.0 the Singe window goes to 116.3 KB/s
against a 110 ceiling, and on a 14-frame clip it disables the controller
outright. FINDINGS 27.4.
7. **Fixed-lam `sasi` was already 5% over target on 00020**, the clip everyone
called easy — nothing noticed because the profile table quotes PSNR, not
bitrate. FINDINGS 27.5.
8. **1.2-second clips cannot be used to judge rate control.** The bucket's
startup transient is bucket/nframes: 6% on a 10 s window, 20% on 00020. Same
lesson as FINDINGS 25.3, different costume.
---
## Start here: is the tree still green?
```
./tools/bench/check.sh
```
~4 min, needs the Blu-ray mounted. From source media it re-runs both display
regression tests, the rate-control drift test (session 6, now covering BOTH
controllers -- bytes, and bytes+cycles), the display-path coherency
counterexample and a **120-frame 68000 decode** (session 7), then prints
`ALL GREEN`. Verified green at end of session 8.
Do not run two of these at once, and do not run one alongside a MAME timing
job: they share `tmp/` snapshot directories and log files, and the second run
silently truncates the first one's output.
If it fails, fix that before doing anything else — everything downstream assumes
the display path is pixel-exact.
The two session-7 stages are worth knowing the shape of before they fail on you:
- `10_pathmix_drift.py` is expected to exit **non-zero**; `check.sh` fails if it
ever starts passing, because that would mean the counterexample behind the
one-path decoder had stopped reproducing.
- the decode stage needs `tmp/rc_fr_singe_sasi_rcprofile.dlx` and will spend
~55 s encoding it if it is missing, nearly all of that k-means in `H.build`.
## Decisions locked
| decision | value | why |
|---|---|---|
| Target CPU | 68000 @ 10MHz (stock) | hardest honest constraint |
| Display mode | 256 colors, 256x192 in 256x256 CRTC mode | every mode is 1 word-access/pixel, so 256c is free vs 16c |
| Double buffer | **none** — page 1 sacrificed | enables `movem.l` 24px bursts; delta coding needs a RAM reference frame anyway |
| **Codec** | **hybrid VQ: SKIP / V1 4x4 / V4 four-2x2 / RAW, per-block rate-distortion** | flat 4x4 VQ was measured and rejected — see FINDINGS 9-10 |
| **Quality modes** | **one: `scsi`** (USER DECISION, session 9 — `sasi` dropped) | a SASI volume is 40 MB and the game is 146 MiB at the LOWEST rate this codec makes. FINDINGS 32 |
| Delivery medium | **SD-backed SCSI** (BlueSCSI / SCSI2SD), as locked in session 2 | capacity does not bind on SD at any rate this codec makes; it is what killed SASI, and it is what rules CD-ROM OUT (**1.09 GiB** at the session-13 candidate rate, against a CD's ~620 MiB and ~150 KB/s at 1x). FINDINGS 32.3, 42.7 |
| **SCSI DMA handshake** | **UNDECIDED — and it decides the project** | `W`, clocks stolen per word, bracketed 5..12 by MC68450 Fig 4-25. `W<=6` fits 0/120 frames; `W=8` misses 47/120. It is a property of how the player drives the MB89352, so it is ours to choose, not to receive. FINDINGS 42.4-42.6 |
| Profile axis | **I/O bandwidth only** | the profiles say nothing about CPU; **both target the same stock 10MHz 68000**, and the Super has SCSI at 10MHz. FINDINGS 28.7 |
| Framerate | 12 fps, **explicit decimation** | source has zero duplicate frames; no free "twos" win |
| Emulator | MAME 0.277 x68000 | accurate enough that measured cycles mean something |
| SNES project reuse | **MIT — cleared** | `data/events/` scene graph is reusable with attribution |
### The SASI/SCSI question is RESOLVED — and in session 9 SASI was DROPPED
Session 1 left "which machine do we target" open. Session 2's answer was **ship
both**, as two quality profiles. **Session 9 retired `sasi`** (USER DECISION) on
CAPACITY, not bandwidth: a SASI volume is limited to 40 MB, and the 22.8 minutes
of unique scene footage on the source Blu-ray is **146 MiB even at 110 KB/s** —
more than the machine's whole 4-unit SASI address space. FINDINGS 32.
Everything below this line about *two* profiles is the session-2..8 record, kept
because every measurement in FINDINGS 27-31 was taken against it. One profile
ships:
| profile | target | lam | quality (00020 / 00146) | machine |
|---|---|---|---|---|
| ~~`sasi`~~ | ~~110 KB/s~~ | ~~60~~ | ~~36.9 / 29.6 dB~~ | **RETIRED session 9** |
| `scsi` | 280 KB/s | 10 (floor) | 39.4 / 32.3 dB | Super/XVI, or CZ-6BS1 board |
~~The 110 KB/s **rate point** is not necessarily gone with the interface: a 1x
CD-ROM sustains ~150 KB/s and CD-ROM is the only period medium with room for the
whole game (374 MiB at `scsi`, 600 MiB with spans). The user's call was to ship
one profile now and settle the medium when the pipe is measured.~~ FINDINGS 32.3.
**SUPERSEDED by FINDINGS 42.** There is no rate point left to choose: the span
pass saturates at ~837 KB/s and the candidate container is **`s13_280p1500`,
837.4 KB/s = 1.09 GiB for the whole game**. That is unremarkable on SD and
impossible anywhere period, so **CD-ROM is closed, not parked**. `--kbps` is now
a quality knob whose value barely matters above ~800 KB/s of span budget; the
number that matters is `W`.
| what | KB/s | whole game (1366.6 s) |
|---|---:|---:|
| `scsi` profile as set | 280.0 | 0.36 GiB |
| session-12 gate | 487.7 | 0.64 GiB |
| **candidate `s13_280p1500`** | **837.4** | **1.09 GiB** |
| probe `s13_lossless` | 934.6 | 1.22 GiB |
**That "machine" column is about the BUS, not the CPU.** The profiles are an
I/O-bandwidth axis and say nothing about clock speed: the X68000 Super has
built-in SCSI at 10 MHz (`x68k.cpp:1194`, `40_MHz_XTAL/4`, same as the base
machine), and only the XVI is faster. **Both profiles target the same stock
10 MHz 68000**, so both must fit the same 833,333-cycle frame budget — and as of
session 7 neither does. FINDINGS 28.7.
**As of session 6 `lam` is a floor, not a setting.** The target is a ceiling and
the encoder bisects lam per frame to stay under it; the profile's lam is the
best quality it is allowed to spend on a quiet frame. On the worst sustained
window that takes `sasi` from 137.4 to 109.5 KB/s and `scsi` from 381.6 to
280.0 KB/s, for -0.60 and -0.91 dB. FINDINGS 27.2.
Sized against the user's working figure of **4 Mbps = 488 KB/s sustained**, on
SD-backed SCSI (BlueSCSI / SCSI2SD) — so that rate is a bus-limited **constant**,
not an average over seek latency.
**Both profiles fit with room.** Ring-buffer simulation on the real per-frame
sizes gives **zero required prefill** for every scene at both profiles: the fill
delivers 40.69 KB per frame time and only one measured frame (42.10 KB) exceeds
that, recovered by the next. A 256 KB buffer carries ~1 s of stall tolerance,
far more than an SD-backed seek needs. FINDINGS 21.
An earlier warning here said `scsi` did not fit because a frame peaked at 96.4%
of the pipe. That compared instantaneous demand to a sustained rate as if they
had to match frame-by-frame; with a buffer the test is cumulative, and it passes.
`scsi` is now within **0.5 dB of the palette ceiling** on 00020. These were
initially set at 45 / 75 KB/s, which was 12% / 7% bus utilisation — read off the
RD curve rather than derived from the hardware. See FINDINGS 17.
Codebooks are **k=256 with 1-byte indices** in both profiles. k=1024 was measured
and rejected — see FINDINGS 14, it was a false-good result from a rate model
that undercharged the index. Do not ship past `lam~800`; FINDINGS 15 has the cliff.
Because of the RAW escape mode, `lam=0` is **pixel-exact** against the palettised
frame (measured 0.00 dB loss). The profiles are two points on one continuous
rate-distortion curve, not two codecs.
---
## What session 3 settled
1. **The display path works and is verified end to end.** First real frame on an
emulated X68000 screen: `docs/images/x68k_first_frame_compare.png`. Full
write-up in **FINDINGS 22**. Everything before this session was Python-side
or a headless `-video none` run, which cannot snapshot at all.
2. **The render is pixel-exact, not merely close.** With monitor contrast at 15,
all 256 palette entries render exactly as `GGGGGRRRRRBBBBBI` + `pal6bit`
predicts. That exactness is the regression test — see
`tools/bench/verify_frame.py`, which exits non-zero if it ever drifts.
3. **Three hardware facts that were previously assumed are now confirmed from
MAME 0.277 source**, not folklore: the palette word format, the 1024-byte
GVRAM line stride, and the 256-colour page aliasing in `HARDWARE.md`. All
three were already written down correctly; they are now cited.
4. ~~**A new quality ceiling was measured** — the 15-bit+I palette alone costs
38.88 dB.~~ **Superseded by session 4:** that figure assumed the shared LSB
`I` is always 1. Chosen per entry, the ceiling is **40.81 dB**. FINDINGS 23.3.
5. **Two shell traps that wedged session 2's background jobs** are documented in
the working-setup section below. They cost ~1.5 h of wall clock and a wedged
CPU core, and one of them was hit again this session.
## What session 5 settled
1. **68000 code drew a frame, and the blit was measured.** `tools/bench/blit.s`
+ `blit.lua`. The snapshot passes `verify_frame256.py` unchanged — pixel-exact
in the real 256x256 mode. **FINDINGS 23.5 is closed**: no longer "proven from
Lua only".
2. **The 38% full-frame blit estimate is dead. It is 53.6%.** And that is a
zero-wait-state floor — MAME models no GVRAM wait states, so real hardware is
worse. FINDINGS 24. Every variant was hand-derived from the MC68000 timing
tables before being measured and the two agree to 0.006-0.43%, so this is not
another MAME artefact.
3. **Reading the source frame is exactly half the blit cost** (V1 53.6% vs a
write-only floor V3 of 27.1%). That is what makes the architecture question
below live.
4. **That number is now measured, and the answer is "implement both paths".**
On the worst sustained window found on the disc, 30% of frames (`sasi`) to
53% (`scsi`) sit above the 70% crossover and want the flat blit; the rest
want direct-to-GVRAM. A player that picks per frame — the mode headers are
parsed before any pixel is written, so the count is free — pays a **median
37.0%** and is **capped at 53.6%**. FINDINGS 25.6.
5. **The sustained action sequence exists, was found by measurement, and breaks
both profiles.** `tools/analysis/07_motion_survey.py` scans a whole stream
for the hottest sliding window; on 00223 it is t=539.4s, the Singe endgame,
at 2.01x the stream mean. There, fixed-lam `sasi` overshoots 110 -> 129.6
KB/s (+18%) and `scsi` 280 -> 373.8 KB/s (+34%). **Rate control is no longer
insurance — it is required.** FINDINGS 25.3.
6. **The two largest streams on the disc are bonus material, not game footage.**
00216 is the feature with a burned-in commentary PiP; 00215 is the commentary
itself. **00223 (9.4 min) is the clean one.** A size-ranked survey would have
encoded live action. FINDINGS 25.1.
7. **Rate control is unsound as written, caught before wiring it up.** The
lam-ladder in `ratectl.py` picks frames from independent temporal chains,
so SKIP blocks reference reconstructions the decoder never saw: 111 of 120
frames drift, worst frame 43.4%, reported PSNR overstated 0.36 dB. Regression
test `tools/analysis/09_ratectl_drift.py`. FINDINGS 26.
8. **On hard content the scene palette, not the display, is the binding
ceiling** — 31.33 dB on the Singe window against 39.90 dB on 00020 and 40.81
dB for the X68000 display. `scsi` is already within 0.51 dB of it.
FINDINGS 25.4.
### Superseded within session 5
4a. **The decoder architecture hinged on one unmeasured number.** Writing
codewords straight into GVRAM costs 76.6% of the frame budget for a *full*
frame (V4 — the 1024-byte stride kills the `movem.l` burst), but scales with
the non-SKIP block fraction and needs **no RAM reference frame at all**,
because the previous frame is already in GVRAM. Compose-then-blit is a flat
53.6%. **They cross at 70% of blocks changed.** FINDINGS 24.5.
---
## What session 4 settled
1. **A real 256x256 CRTC mode exists and is verified.** `crtc_mode.lua`, derived
from `x68k_crtc.cpp`'s divisor ladder rather than recalled — the derivation is
self-checking (368 = 1104/3 exactly, so the horizontal registers divide by
three with no remainder). Snapshot is native 256x512, active area pixel-exact,
letterbox true black. FINDINGS 23. The x=512 wrap of FINDINGS 22.5 is gone.
2. **The palette ceiling was wrong by 2 dB, in our favour.** The shared LSB `I`
must be chosen **per palette entry**, not hardcoded to 1. Doing so lifts the
display ceiling from 38.85 to **40.81 dB** and is the only way to get true
black at all (`pal6bit(1) = 4`). 102 of 256 entries want `I = 0`. This
supersedes FINDINGS 22.4 and gives `scsi` ~2 dB more headroom than believed.
**The encoder does not do this yet** — see the encoder-gaps list.
3. **Letterboxing costs one palette entry.** 255 colours + a reserved black at
index 0, with `I = 0` on it. `prep_frame.py --reserve-black`. FINDINGS 23.4.
4. **MAME's graphics double-scan is phase-shifted one raster line** — pairs are
(1,2),(3,4),..., not (0,1), because `get_gfx_pixel` halves the *absolute*
scanline and `vbegin = 41` is odd. Cost a false failure. FINDINGS 23.2.
---
## What session 2 settled
1. **The critical-path question is answered.** "Does VQ soften Bluth's linework
unacceptably?" — **flat 4x4 VQ: yes, badly. The hybrid (SKIP/V1/V4/RAW): no.**
Verified by eye, not just PSNR. See `docs/FINDINGS.md` 9-11 and the two
images in `docs/images/`. Both profiles use **k=256**; see item 2b.
2. **Session 1's 12fps bitrate was wrong** (183 KB/s claimed, 340 KB/s measured).
Halving the framerate does not halve the bitrate. FINDINGS 8.
2b. **A fourth false-good result was produced and caught this session** — k=1024
codebooks looked like a +2.4 dB free win because the rate model charged 1 byte
for a 10-bit index. FINDINGS 14. The k=256 configuration ships.
3. **The 256-colour palettised frame is the real quality ceiling** and it looks
excellent. Judge the codec against that, not against 1080p.
4. Encoder exists and produces a real bitstream: `tools/encoder/`.
---
## Encoder — working
```
python3 tools/encoder/extract.py 00020 /tmp/fr_00020 12 crop
python3 tools/encoder/encode.py /tmp/fr_00020 out.dlx --profile sasi --preview p.png
```
| file | role |
|---|---|
| `extract.py` | .m2ts -> 256x192 PNGs, 12fps, spatial-only denoise |
| `vq.py` | palette, blockify, hand-rolled k-means (no sklearn on this box), PSNR |
| `vq_hybrid.py` | the codec: 4 block modes + lagrangian mode decision |
| `ratectl.py` | SASI/SCSI profiles, leaky-bucket rate control |
| `encode.py` | CLI + `DLX1` container writer |
`DLX1` container layout is documented in the `encode.py` docstring. All
multi-byte fields are **big-endian** so the 68000 reads them with a plain `move`.
### Known encoder gaps
- ~~Rate control is written but not yet wired into `encode.py`.~~ **DONE,
session 6.** It is on by default; `--fixed-lam` restores the old behaviour.
Gated by `tools/analysis/09_ratectl_drift.py`, which is now in `check.sh`.
- **Payload is deliberately NOT entropy-coded** — deflate decode does not fit in
the 68000's frame budget (FINDINGS 17.2). Do not "optimise" this later.
- ~~**Frame records are not aligned.**~~ **DONE.** The container is `DLX2` and
`write_container` (`encode.py:139-156`) pads both the frame table and every
record to a 4-byte boundary, inside the rate controller's accounting rather
than after it, and `encode.py` reports the cost in B/frame and B/s. Unaligned
is an ADDRESS ERROR on a 68000, not a slow read (FINDINGS 28.3); `DLX1` is
still read, so the old unaligned containers have not been invalidated.
- ~~**The mode decision is blind to CPU cost.**~~ **DONE**, sessions 12-19.
`vq_hybrid.decide()` (`vq_hybrid.py:218`) minimises
`distortion + lam*bytes + mu*(decode cycles + byte_clk*bytes)` against the
MEASURED per-mode cycle costs, and `cycles()` scores a whole frame with the
exact clustered SKIP rule rather than a per-block constant. This entry sat in
the gap list for several sessions after it was closed; **the list is not
self-maintaining, and stale entries here become phantom work in the roadmap.**
- **Palette packing is not implemented in the encoder.** It still emits 24-bit
palettes; the X68000 word packing happens Lua-side. Whatever writes real
palette words must pick `I` per entry by minimum squared error (FINDINGS 23.3,
worth 1.96 dB) and reserve index 0 as black with `I = 0` (FINDINGS 23.4).
- Codebooks are per-scene and rebuilt from scratch; no inter-scene reuse.
- ~~`_paint` is a Python per-block loop.~~ **DONE, session 6** — vectorised,
17.1x. It was never the bottleneck, though: `VQ.assign` is 78% of a frame and
`H.build`'s k-means is 51 s of a 55 s run. **That k-means is now the thing to
attack before the full-disc survey**, not anything in the per-frame path.
FINDINGS 27.6.
---
## Working setup (unchanged from session 1, re-verified)
**MAME ROMs** — `~/mame/roms/x68000.zip`. Must pass **`-bios ipl10`**.
```
mame x68000 -bios ipl10 -video none -sound none -nothrottle -seconds_to_run 3
```
**Assembler** — `tools/vasm/vasmm68k_mot -Fbin -o out.bin in.s`
**Blu-ray** — `udisksctl loop-setup -r -f DRAGONS_LAIR.iso` -> `/media/reala-misaki/BDROM`
(still mounted as of end of session 2).
**MAME Lua harness** — `tools/bench/*.lua`, working. Three gotchas (retain the
notifier subscription in a global; the stack register is `SP` not `A7`;
`autoboot_script` fires at PC=0 before boot) are documented in FINDINGS.
**Two shell traps, both hit again this session:**
- piping MAME (or any long job) through `grep` block-buffers — write to a file.
- `pkill -f <pattern>` matches your own shell and kills it (exit 144).
Use `pkill -x` or kill by PID.
- **`pgrep -f <name> | xargs kill` kills your own shell too — exit 144.** Same
root cause as the `pkill -f` trap above: the shell's own command line contains
the pattern. **Hit again in session 5**, which makes it four times across three
sessions. Kill by PID captured at launch (`$!`), or use `pkill -x`.
- **`until ! pgrep -f foo.py; do sleep; done` watcher loops never exit.** The
watching shell's own command line contains the string `foo.py`, so `pgrep -f`
matches the watcher itself and the loop spins forever. Session 2 left 11 of
these wedged for over an hour. Wait on the PID (`while kill -0 $PID`) or on a
sentinel file the job touches when it finishes -- never on a `-f` name match.
- **`timeout N mame ...` does not kill MAME.** MAME catches SIGTERM and, with an
autoboot script blocked waiting on a flag that never arrives, never reaches
its shutdown path. `timeout` without `-k` then waits forever while MAME burns
a full core at `-nothrottle`. Always `timeout -k 5 N`.
---
## Disk throughput benchmark — still blocked, no longer gating
`IOCS _B_READ` returns -1 uniformly. Full diagnosis and the four untested
hypotheses are in session 1's notes (git history of this file, commit 65112b9);
the ordered plan for retrying is in **`docs/BENCHMARK.md`**.
**Status changed twice this session — read this rather than the git history.**
It was briefly promoted to critical-path while the working bandwidth figure was
misread as 4 MB/s. With the correct figure (**4 Mbps = 488 KB/s**) and the
ring-buffer simulation showing **zero required prefill** for both profiles
(FINDINGS 21), the design no longer hangs on it. Pixel-exact on SCSI is **not**
available at 4 Mbps — it needs 92-97% of the pipe — so there is no longer a
"measure it and maybe ship transparent" decision waiting.
What the benchmark is still worth doing for:
- **Confirming the 4 Mbps figure.** It is user-supplied and its provenance is
not recorded. Every profile hangs off it.
- **Confirming DMA is actually used.** If transfers fall back to PIO the CPU
cost rises far above the ~12-15% cycle-steal estimate and CPU becomes the
binding constraint. This is the worst plausible outcome and the cheapest to
check — do it first.
**Do not try to get the bandwidth number out of MAME.** Its SCSI/SASI devices are
functional models, not timing-accurate; a KB/s figure from MAME measures the
emulator's scheduler. `docs/BENCHMARK.md` covers the three-tier approach
(MAME validates the path, derivation bounds it, real hardware settles it).
## Display path — VERIFIED (session 3), in a real mode (session 4), by 68000 code (session 5), by a 68000 DECODER (session 7).
The first real frame is on screen: `docs/images/x68k_first_frame_compare.png`.
**Session 7 went from copying a frame to parsing one.** `src/player/decode.s`
reads DLX1, dispatches all four block modes and writes straight into GVRAM;
120 frames decoded in sequence are pixel-exact against `tools/encoder/dlx.py`
(`tools/bench/verify_decode.py`, in `check.sh`). The blit numbers below are
still correct for what they measured — a *copy* — but they are no longer the
display-path budget: the decoder costs 300/448/400 cycles per V1/V4/RAW block
and misses the 12fps budget on 31% of frames. FINDINGS 28.
**Session 5 closed the gap this paragraph used to describe.** GVRAM is now
filled by 68000 instructions and the result is still pixel-exact, and the blit
cost is measured rather than estimated: **53.6% of a 12fps frame**, not 38%
(FINDINGS 24). The remaining caveat is different and narrower: MAME models
**no GVRAM wait states**, so 53.6% is a floor and real hardware is worse.
Full write-up in **FINDINGS 22**. Harness: `tools/bench/show_frame.lua` +
`tools/bench/prep_frame.py`.
Three facts the player MUST honour, none of which were guessable:
| what | where | value |
|---|---|---|
| **Un-hide the graphics layer** | CRTC R20 `$E80028` | clear bit 11 ("G-VRAM set to buffer"); IPL leaves `0x0B16` |
| Colour setup (256c) | CRTC R20 bits 9-8 | `0x0100` |
| **Monitor contrast** | `$E8E001` bits 3-0 | IPL leaves **14**; write **15** or everything renders 7% dark |
The `R20 = 0x0116` value quoted here in session 3 is the **768-wide IPL timing**
with the gate cleared. The shipping value is **`R20 = 0x0110`** — see the mode
table in `tools/bench/crtc_mode.lua`, which is now the single source of truth
for all of R00-R08 and R20.
Bit 11 is the one that cost the most time: GVRAM writes land and read back
correctly while the layer is invisible, so the video controller looks guilty and
is not. Contrast `0` blanks the screen — free fade-to-black for transitions.
Palette format is now **confirmed from MAME source**, not assumed:
`GGGGGRRRRRBBBBBI` (G 15:11, R 10:6, B 5:1, shared LSB I), expanded as
`pal6bit((field<<1)|I)`. With contrast at 15 the render is **pixel-exact**.
Ceiling: the 15-bit+I palette costs **40.81 dB** against the 24-bit palettised
source, once `I` is chosen per entry (FINDINGS 23.3 — session 3's 38.88 dB
assumed `I = 1`). Still the same order as the `scsi` profile's own codec error
(39.4 dB), so `scsi` remains near display-transparent, with ~2 dB more headroom
than session 3 thought.
Snapshot recipe that works (`-video none` CANNOT snapshot):
```
SDL_VIDEODRIVER=dummy mame x68000 -bios ipl10 -video soft -window \
-sound none -nothrottle -plugins -autoboot_script <script>.lua \
-snapshot_directory ./snap -snapview native -seconds_to_run 6
```
`-snapview native` drops MAME's LED artwork and gives a clean 768x512 screen.
## Next steps, in priority order
1. ~~**Measure the non-SKIP block fraction.**~~ **DONE, session 5**, and its
answer **WITHDRAWN in session 7** — FINDINGS 28.1/28.2. It concluded
"implement both display paths and pick per frame, median 37.0%, capped at
53.6%". Mixing the paths is incoherent (the compose path needs a RAM
reference the direct path never writes) and the two costs it compared were
both copies with no decode in either. **The shipping decoder has one path.**
The non-SKIP fraction is still reported by `encode.py` and is still the right
thing to look at — it is just no longer a switch. Original framing kept
below, because its instruction to report the distribution rather than the
mean is the part that held up:
FINDINGS 24.5: compose-in-RAM-then-blit costs a flat 53.6% of the frame
budget; decode-direct-to-GVRAM costs 76.6% x (fraction of blocks that are not
SKIP) and needs no RAM reference frame. **They cross at 70%.** Which side of
70% the content sits on decides which decoder inner loop to write, so this
must come before writing one.
**It needs no new machinery** — the mode decision in `vq_hybrid.py` already
computes it per frame and simply never reports it. Add the histogram
(SKIP / V1 / V4 / RAW counts per frame) to `encode.py` output and run it over
the clips already extracted. Report the *distribution*, not the mean: a
scene-cut frame is ~100% non-SKIP and a held frame near 0%, and the mean of
those two is a number describing no actual frame.
1b. ~~**Wire rate control into `encode.py`.**~~ **DONE, session 6.** FINDINGS 27.
Both overshoots closed for under 1 dB, drift test at zero, `check.sh` gates
it. The remaining rate-control question is not a defect: whether `--rc-floor
open` is worth taking on quiet content. It measured as worth **0.00 dB** on
the Singe window (no frame there is quiet enough to saturate the bucket), so
it needs a genuinely quiet scene to decide, and it is a quality-per-byte
judgement rather than a correctness one.
2. ~~**68000 decoder skeleton.**~~ **DONE, session 7.** `src/player/decode.s`,
pixel-exact over 120 frames, gated in `check.sh`. It answered the question it
was written to answer, and the answer is no: **it does not fit** — mean 81.7%
of a 12fps frame, 31% of frames over 100%. FINDINGS 28. The follow-on is
priority 0 at the top of this file.
2a. **Re-budget everything against the MEASURED per-mode costs**, not 53.6% and
not 38%. Session 7 replaced the model twice over (FINDINGS 28.2): the display
path is not one number times a block fraction, and the median frame is 74.4%
rather than 36.6%. The original note is kept below because its warning about
downstream figures derived from a dead estimate is exactly what happened
again.
~~Re-budget everything against 53.6%, not 38%.~~ Several downstream figures
were derived from the old estimate. The blit alone now eats over half the
frame at 12fps in the compose-then-blit design, before any decode, and MAME
models no GVRAM wait states so that is a floor. This may reopen questions
that were closed against the 38% number — check FINDINGS 17.2's entropy-coding
rejection, which was argued as "54% LZ4 with no room beside a 38% blit". The
conclusion gets *stronger*, not weaker, but the arithmetic should be restated.
2b. **Pad frame records to 4 bytes in `encode.py`.** Not optional: unaligned
records are an address error on a 68000 (FINDINGS 28.3), and `prep_dlx.py`
currently repairs it at load time, which the shipping player streaming from
disc cannot do. The padding is real bytes on disc, so it has to be inside
the rate controller's accounting, not added after it. 20 B/s at 12fps.
3. **Full-disc survey.** Now scoped by session 5 rather than open-ended: the
worst *sustained* window is measured (FINDINGS 25), so what remains is the
distribution over content, not the worst case.
- Classify **content / menu / bonus** — not just menu vs content. FINDINGS
25.1: the two largest streams are bonus material and look like content by
size, duration and bitrate alike.
- Run `tools/analysis/07_motion_survey.py` per stream first; it is cheap
(96x72 greyscale) and gives a hot-window shortlist so the expensive encode
only runs where it matters.
- ~~Vectorise `_paint` before this run.~~ Done. The cost to attack now is
`H.build`'s k-means: 51 s of a 55 s run, and it runs once per scene.
- ~~Do it after rate control (1b), or it measures an encoder nobody ships.~~
Rate control is in, so the survey now measures the shipping encoder.
5. **Confirm DMA vs PIO in MAME** (see the benchmark section above) — cheap, and
the only thing that could still move CPU into the binding position.
6. **Resolve the framing question** (FINDINGS 12: crop vs squash vs wide).
Needs an eyeball against arcade reference, not a measurement.
7. **Import the scene graph.** SNES project `data/events/` (MIT, cleared),
cross-checked against DirkSimple (zlib) which transcribed the same data
independently — diff them to catch transcription errors before committing
any of it to 68000 tables.
8. **ADPCM audio.** MSM6258, 15.6kHz mono, 7.8 KB/s — already budgeted in
`ratectl.py`, not yet extracted or encoded.
### Explicitly abandoned — do not re-propose
- ~~Entropy-code the payload.~~ Deflate decode is ~216% of the frame budget on a
68000; LZ4 is ~54% with no room beside a 38% blit (FINDINGS 17.2). All bitrates
are raw payload. This also demotes the "247 KB/s lossless" figure in FINDINGS 8
to a compression upper bound, not a shippable design.
- ~~k=1024 codebooks.~~ False-good result from a rate model that charged 1 byte
for a 10-bit index (FINDINGS 14). k=256 wins at every matched bitrate.
- ~~Flat 4x4 VQ.~~ Rejected by eye (FINDINGS 9).
## Not yet started
- **A player, as opposed to a decoder.** `src/player/decode.s` parses DLX1,
dispatches all four block modes and draws pixel-exact frames, but it decodes
from RAM that Lua pre-loaded. There is no disc streaming, no ring buffer, no
audio, no timing against the VBL, and no scene branching.
- **Codebook expansion on the 68000.** `prep_dlx.py` does it host-side because
it is a load-time cost and including it would flatter or damn the inner loop.
The player must do it: 8 KB + 2 KB per scene.
- ADPCM audio extraction/encoding
- Disk image packaging
- Game logic (scene branching, input windows, death clips)
## Reproducing the 256x256 mode result (session 4)
```
python3 tools/encoder/extract.py 00020 tmp/fr_00020 12 crop
python3 tools/bench/prep_frame.py tmp/fr_00020 tmp/frame256.bin 0 --reserve-black
mkdir -p tmp/snap256 && cd tmp && SDL_VIDEODRIVER=dummy timeout -k 5 90 mame x68000 \
-bios ipl10 -video soft -window -sound none -nothrottle -plugins \
-autoboot_script ../tools/bench/show_frame256.lua \
-snapshot_directory ./snap256 -snapview native -seconds_to_run 6
cd .. && python3 tools/bench/verify_frame256.py
```
Exits non-zero on any drift. Expected: `256x512 native, double-scan exact,
active 256x192 pixel-exact, letterbox true black`, ceiling 40.81 dB.
## Reproducing the display result
```
python3 tools/encoder/extract.py 00020 tmp/fr_00020 12 crop
python3 tools/bench/prep_frame.py tmp/fr_00020 tmp/frame.bin 0
mkdir -p tmp/snap_verify && cd tmp && SDL_VIDEODRIVER=dummy mame x68000 -bios ipl10 \
-video soft -window -sound none -nothrottle -plugins \
-autoboot_script ../tools/bench/show_frame.lua \
-snapshot_directory ./snap_verify -snapview native -seconds_to_run 6
cd .. && python3 tools/bench/verify_frame.py
```
Verified cold from the Blu-ray at end of session 3: exact match, 38.88 dB.
(That 38.88 is correct *for this test*: `show_frame.lua` still packs `I = 1`.
The 40.81 dB ceiling comes from `show_frame256.lua`, which picks `I` per entry.)
`tmp/` is gitignored scratch. The frames are NOT in the repo — regenerate them
with `extract.py`; the earlier ones lived in `/tmp` and do not survive a reboot.
## Reference material on this box (not in the repo)
- **MC68450 DMAC manual: `~/src/mc68450.pdf`** (Motorola, Jul 1989, from
bitsavers; `curl` it with a browser User-Agent or you get a 403). This is the
primary source for FINDINGS 39 and it overturned two derived answers in one
session. Fig 4-25 is the timing table — sheet 1 chaining, sheet 3
single-address, sheet 4 dual-address; note 2 (4-clock reads, 5-clock writes)
is the one that mattered. Sects 4.5.2.1-3 are the arbitration overheads,
5.2.3.2 the limited-rate auto-request slabs. `pdftotext` handles it.
- **px68k source: `~/src/px68k`** — only `m68000/c68k.c` is used, by
`tools/bench/c68k/`.
- **MAME 0.277 source: `~/src/mame-mame0277/`** (tarball `~/src/mame0277.tar.gz`).
Downloaded this session to settle the graphics-layer question. The files that
matter are `src/mame/sharp/x68k_v.cpp`, `x68k_crtc.cpp`, `x68k_crtc.h`,
`x68k.cpp`. **Read these before theorising about X68000 video behaviour** —
six register-poking attempts failed against a gate that one grep found.
- Blu-ray mounted at `/media/reala-misaki/BDROM` via
`udisksctl loop-setup -r -f DRAGONS_LAIR.iso`.
---
## Parked ideas (not scheduled, not abandoned)
- **Cliff Hanger, retitled as Lupin III** (user, session 4). Stern's 1983
laserdisc game was cut from *Castle of Cagliostro* and *Mystery of Mamo* with
the Lupin branding stripped; a port would restore it. Technically **cheaper
than this project**: same content class (cel animation, flat colour, hard
cuts), ~13 min of footage vs Dragon's Lair's ~22, and flatter linework than
Bluth's, so fewer blocks should escape to V4/RAW. The codec, the display path,
and `crtc_mode.lua` would all drop straight in.
**The real cost is media prep, not code:** there is no clean master cut to
Stern's scene boundaries the way `DRAGONS_LAIR.iso` is, so the footage would
have to be sourced and cut to match. Not to be started until the CPU path is
proven — it changes nothing about whether this design works.
## Reproducing the blit measurement (session 5)
```
python3 tools/encoder/extract.py 00020 tmp/fr_00020 12 crop
python3 tools/bench/prep_frame.py tmp/fr_00020 tmp/frame256.bin 0 --reserve-black
tools/vasm/vasmm68k_mot -Fbin -o tmp/blit.bin tools/bench/blit.s
mkdir -p tmp/snap_blit && cd tmp && SDL_VIDEODRIVER=dummy timeout -k 5 900 mame x68000 \
-bios ipl10 -video soft -window -sound none -nothrottle -plugins \
-autoboot_script ../tools/bench/blit.lua \
-snapshot_directory ./snap_blit -snapview native -seconds_to_run 120
```
~25 s wall. Prints cycles/frame and % of a 12fps budget for V1-V4, and snapshots
V1's output. To check that snapshot is still pixel-exact:
`sed 's|snap256|snap_blit|' tools/bench/verify_frame256.py | python3 -`
Not added to `check.sh`: `check.sh` asserts pixel-exactness, and asserting wall
timings there would make the green-light check sensitive to host load.
## Reproducing the streaming result (session 18)
The ring-buffer pass, on a **stock 2 MB machine**, container in a host file:
```
DLX=tmp/rc_fr_singe_scsi_span.dlx
tools/vasm/vasmm68k_mot -Fbin -o tmp/stream.bin src/player/stream.s
python3 tools/bench/prep_stream.py "$DLX"
mkdir -p tmp/snap_stream && rm -f tmp/snap_stream/x68000/*.png
( cd tmp && DLX_STREAM_KBPS=0 SDL_VIDEODRIVER=dummy stdbuf -oL \
timeout -k 5 600 mame x68000 -bios ipl10 -ramsize 2M -video soft -window \
-sound none -nothrottle -plugins -autoboot_script ../tools/bench/stream.lua \
-snapshot_directory ./snap_stream -snapview native -seconds_to_run 90 \
> stream_check.log 2>&1 )
python3 tools/bench/verify_decode.py "$DLX" --snap tmp/snap_stream
```
Env knobs: `DLX_RING_KB` (default 256), `DLX_STREAM_KBPS` (**REQUIRED, no
default**; `0` = unlimited, which isolates the WRAP question from the DELIVERY
one), `DLX_PREFILL_KB` (default 0).
`19_ring_stream.py`'s `--kbps` is required too, as are `--bus`/`--kbps` on
12/14/16/17. None of them has a default any more — see the retirement note at
the top of this document.
`DLX_STREAM_KBPS=0` is the right setting for a **correctness** gate: the wrap
policy is what is being tested and an unlimited pipe removes delivery as a
variable. Use a finite rate to get the `DEADLINE`/`REQUIRED PREFILL` lines.
The ring simulation, which needs no emulator and takes a second:
```
python3 tools/analysis/19_ring_stream.py # both containers
python3 tools/analysis/19_ring_stream.py <c.dlx> --kbps 512 --ring 128
```
It reports the **zero-prefill pipe** — the threshold a medium has to clear for
the container to need no prefill at all — which is the number to measure a
BlueSCSI against. It also warns explicitly when demand exceeds supply on the
MEAN, because a "required prefill" figure for a sustained overrun is the most
flattering possible way to state one.
**Do not run this alongside another MAME job.** They share `tmp/` logs and
snapshot directories and the second run silently truncates the first.
## Reproducing the packed player (session 32)
```
python3 tools/encoder/pack.py tmp/fr_singe tmp/packed_singe.dlxp --nframes 120
./tools/bench/packed_run.sh tmp/packed_singe.dlxp
```
~4 min, four MAME runs. `DLX_PK_GATE_ONLY=1` runs only the first, which is what
`check.sh` takes. Expected: 120 of 120 pixel-exact; 193 chain entries; 0 late
frames; 0 frames unsampled; a free-running transport of ~90 ms a record; and
**487 of 1,038 V-DISP edges lost in the held configuration**, which is the run's
whole point and is asserted rather than admired.
The palette-LAST container is a flag and not a re-encode:
```
python3 tools/encoder/pack.py tmp/fr_singe tmp/packed_singe_last.dlxp \
--nframes 120 --palette-last
./tools/bench/packed_run.sh tmp/packed_singe_last.dlxp
```
The chain assertion is read out of the container's own flags, so a run that
built the array the other way round fails instead of painting 192 rows of
picture into the palette registers.
And the arithmetic underneath the result, which needs no emulator:
```
python3 tools/analysis/31_display_duty.py tmp/packed_singe.dlxp \
--rate 582 700 1164 2131 3000
```
`--rate` is REQUIRED and has no default, for FINDINGS 50's reason.
**Do not run this alongside another MAME job.** It shares `tmp/` logs with every
other rig here.
## Reproducing the decoder result (session 7)
```
python3 tools/encoder/encode.py tmp/fr_singe tmp/rc_fr_singe_sasi_rcprofile.dlx --profile sasi
python3 tools/bench/prep_dlx.py tmp/rc_fr_singe_sasi_rcprofile.dlx
tools/vasm/vasmm68k_mot -Fbin -o tmp/decode.bin src/player/decode.s
mkdir -p tmp/snap_decode && cd tmp && SDL_VIDEODRIVER=dummy timeout -k 5 900 mame x68000 \
-bios ipl10 -ramsize 2M -video soft -window -sound none -nothrottle -plugins \
-autoboot_script ../tools/bench/decode.lua \
-snapshot_directory ./snap_decode -snapview native -seconds_to_run 150
cd .. && python3 tools/bench/verify_decode.py tmp/rc_fr_singe_sasi_rcprofile.dlx
```
~90 s wall. Prints cycles/frame and % of a 12fps budget for four real frames
spanning the non-SKIP distribution, four synthetic single-mode frames, and one
full 120-frame pass; then verifies the last frame is pixel-exact. Expected:
median 73.8%, p90 116.4%, max 135.8%, mean 81.7%; V1 299.9 / V4 448.2 / RAW
400.4 cycles per block.
`-ramsize 2M` here matters for a different reason than it used to. MAME defaults
to 4M and the locked *target* is a stock 2 MB machine, so a timing reproduction
quoting the target's memory should use it. But the rig preloads the whole
container, so at 2M this run is truncated to the prefix that fits — 37 of 120
frames on the current span-heavy container — and its real-frame anchors are
therefore a biased sample (session 16 measured the bias at 8.2% on the mean;
FINDINGS 45.3). The **synthetic** anchors are unaffected: they come out
bit-identical at 2M and 6M, which is what licenses `check.sh` running the
verify-only gate at `RIG_RAM=6`. For anchors over the whole window, run this
with `--ram 0x600000` and `-ramsize 6M`.
`DLX_VERIFY_ONLY=1` drops the timing anchors, which is how `check.sh` runs it.
Score a container against the measured costs without touching MAME:
```
python3 tools/analysis/11_cpu_budget.py tmp/rc_fr_singe_scsi_rcprofile.dlx
```
And re-demonstrate why there is only one display path (exits non-zero **by
design** — it is the counterexample):
```
python3 tools/analysis/10_pathmix_drift.py # 70/120 frames corrupt
python3 tools/analysis/10_pathmix_drift.py --fix direct # clean, and cheapest
```
## Reproducing the rate-control result (session 6)
```
python3 tools/encoder/extract.py 00223 tmp/fr_singe 12 crop 539.4 10.0
for prof in sasi scsi; do
python3 tools/encoder/encode.py tmp/fr_singe tmp/rc_$prof.dlx --profile $prof --fixed-lam
python3 tools/encoder/encode.py tmp/fr_singe tmp/rc_$prof.dlx --profile $prof
done
python3 tools/analysis/09_ratectl_drift.py # must exit 0, zero drifting frames
```
Expected, totals including the 7.8 KB/s audio allowance: `sasi` 137.4 -> 109.5
KB/s and 27.82 -> 27.22 dB; `scsi` 381.6 -> 280.0 KB/s and 30.81 -> 29.90 dB;
zero frames at the lam=800 cliff in either. ~55 s per encode, nearly all of it
k-means in `H.build`.
The block-mode map now renders the rate-controlled encoder by default:
```
python3 tools/analysis/08_mode_map.py tmp/fr_singe tmp/singe_modes_rc.webm \
--profile sasi --scale 2 # add --fixed-lam to compare
```
**Do not judge rate control on `tmp/fr_00020`.** It is 14 frames; the leaky
bucket's startup transient is bucket/nframes, so it lands 18% under target there
for reasons that have nothing to do with the content. FINDINGS 27.5.
## Reproducing the sustained-action result (session 5)
```
python3 tools/analysis/07_motion_survey.py 00223 10 # -> hottest window t=539.4s
python3 tools/encoder/extract.py 00223 tmp/fr_singe 12 crop 539.4 10.0
python3 tools/encoder/encode.py tmp/fr_singe tmp/singe_sasi.dlx --profile sasi
python3 tools/encoder/encode.py tmp/fr_singe tmp/singe_scsi.dlx --profile scsi
python3 tools/analysis/08_mode_map.py tmp/fr_singe tmp/singe_modes.webm \
--profile sasi --scale 2
```
`extract.py` now takes optional `[start_s] [dur_s]` — needed because 00223 is
9.4 min and the windows that stress the codec are seconds long.
`08_mode_map.py` renders palettised source | decoded | block-mode map at 12fps.
Output format follows the extension; **prefer `.webm`** — GIF re-quantises to
256 colours, which is a poor fit for output whose subject is colour fidelity,
and runs larger. It uses `yuv444p` because the mode map is flat saturated colour
on a 4-pixel grid and chroma subsampling smears exactly those edges.