ROADMAP P6, everything in the item except the bus half session 20 closed. tools/encoder/adpcm.py is an MSM6258 codec, tools/encoder/extract_audio.py takes the same seconds of the same stream the frames come from, tools/bench/verify_adpcm.py is the gate, tools/analysis/32_audio_wire.py the container arithmetic. There is no reference encoder -- ffmpeg has a decoder for this format and none the other way -- so what is gated is the decoder the encoder runs INSIDE its own nibble search, sample-exact against ffmpeg's over 4,268 nibbles. An encoder that agrees with its own wrong decoder is what that catches. The Singe window: 156,250 samples -> 78,125 B at 21.97 dB, which is 7,812.5 B/s to the byte. Normalising the disc's -13.4 dBFS level moves the SNR 21.97 -> 21.97, so the level is not a lever. And the two published delta formulas are not the same codec. They differ by at most 3 in 12-bit units; encode for one and decode on the other and the SNR goes 21.97 -> -2.88 dB, the noise louder than the signal, because ADPCM is recursive the way the video codec is temporally recursive. Which one the chip runs is now P6a and it is a precondition on shipping any audio. And audio is the first thing the packed branch's simplification has cost anything for. A record has no index BY DESIGN, so audio cannot be per-record without making records variable; it rides a fixed cadence (F, A), the obvious F=1 wastes 57.3% of every audio sector, and the pick is F=11 A=14 -- 0.09% padding, 14,336 B held, wire 582.0 -> 589.6 KB/s. The codec container, which kept its index, pays zero. The MAME experiment did not work and 65.5 says so: :okim6258 is there at $E92001/$E92003, read out of the machine's own program map, and feeding it from Lua recorded silence across control 0..3 x port C 0..15. The register semantics were not guessed at further. FINDINGS 65. check.sh ALL GREEN before and after, with a new stage. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
3464 lines
191 KiB
Markdown
3464 lines
191 KiB
Markdown
# Status & next-session handoff — end of session 33 (2026-08-25)
|
||
|
||
## Session 33: audio gets an encoder, and the packed container's best property gets a bill
|
||
|
||
**Green light first and last: `./tools/bench/check.sh` was ALL GREEN before any
|
||
of this (`tmp/check_s33_start.log`) and ALL GREEN after** — the same stages, plus
|
||
one new one.
|
||
|
||
**FINDINGS 65. ROADMAP P6 is three-quarters done and has grown two sub-items.**
|
||
`tools/encoder/adpcm.py`, `tools/encoder/extract_audio.py`,
|
||
`tools/bench/verify_adpcm.py`, `tools/analysis/32_audio_wire.py`, and three
|
||
throwaway MAME probes kept because their failure is a finding
|
||
(`tools/bench/probe_adpcm*.lua`).
|
||
|
||
**Everything below is HOST ARITHMETIC plus one MAME introspection.** No board
|
||
ran, and the one MAME *experiment* attempted did not work — 65.5 says so.
|
||
|
||
**0. SESSION 32'S WORK WAS UNCOMMITTED AND ITS STATUS BLOCK HAD NO HANDOFF.**
|
||
Both fixed before anything new was written: the tree was re-gated ALL GREEN,
|
||
the handoff written, and the whole of session 32 committed as one change.
|
||
|
||
**1. THE ENCODER, AND THE GATE IT NEEDED INSTEAD.** ffmpeg has a DECODER for
|
||
this format (`adpcm_ima_oki`) and **no encoder**, so there is nothing to diff
|
||
against. What is gated instead is the decoder the encoder runs **inside its own
|
||
nibble search** — sample-exact against ffmpeg's over 4,268 nibbles. An encoder
|
||
that agrees with its own wrong decoder is what that catches.
|
||
|
||
| | |
|
||
|---|---|
|
||
| the window | 00223 @539.4 s, 10.000 s — **the same seconds as `tmp/fr_singe`** |
|
||
| encoded | 156,250 samples → **78,125 B, SNR 21.97 dB** |
|
||
| the rate | 78,125 B / 10 s = **7,812.5 B/s to the byte** — 52's figure, from the other end |
|
||
| nibble order | **HIGH FIRST, measured** — low-first mismatches ffmpeg on 3,285 of 4,268, and that is the gate's negative control |
|
||
| the level | disc peaks at **−13.4 dBFS**; normalising ×4.7 moves the SNR **21.97 → 21.97**. Not a lever |
|
||
|
||
**2. THE HEADLINE, AND IT IS A THREE-LSB DIFFERENCE THAT COSTS 25 dB.** The two
|
||
published delta formulas — ffmpeg's `((2*(n&7)+1)*step)>>3` and the OKI
|
||
datasheet's per-term truncation — differ **by at most 3 in 12-bit units**.
|
||
Encode for one, decode on the other:
|
||
|
||
| encoded | decoded | SNR |
|
||
|---|---|---:|
|
||
| `shift` | `shift` | **21.97 dB** |
|
||
| `shift` | `terms` | **−2.88 dB** |
|
||
|
||
**The noise comes out louder than the signal** (mean disagreement 78.2 against a
|
||
source RMS of 74.4). **ADPCM is recursive**, so a rounding difference does not
|
||
stay where it happens — the same property, one dimension down, that let 64.1's
|
||
gate audit 120 frames by comparing one. **Which formula the chip runs is
|
||
therefore a PRECONDITION on shipping any audio**, and it is ROADMAP P6a.
|
||
|
||
**3. AND THE PACKED CONTAINER'S BEST PROPERTY IS WHAT MAKES AUDIO COST.** A
|
||
packed record is 97 sectors and its address is `LBA0 + i*97` because a literal
|
||
frame's length is geometry — **no index, and none can be needed** (63, 64.1).
|
||
Audio at 15,625 Hz is **651.0417 B a slot**, and the `.0417` is the same
|
||
remainder the frame clock carries (54). Per-record audio makes records variable
|
||
length, which needs an index, which ends the format. So audio rides a **fixed
|
||
cadence** — every `F` frames, `A` whole sectors — and `(F, A)` is a rational
|
||
approximation to 15625/12288 from above:
|
||
|
||
| F | A | lump | padding | wire adds | held |
|
||
|---:|---:|---:|---:|---:|---:|
|
||
| **1 — the obvious one** | 2 | 1,024 B | **57.29%** | 12.00 KB/s | 2,048 B |
|
||
| **11 — the pick** | **14** | **7,168 B** | **0.09%** | **7.64 KB/s** | **14,336 B** |
|
||
| 81 — the floor | 103 | 52,736 B | 0.003% | 7.63 KB/s | 105,472 B |
|
||
|
||
**Wire: 582.0 KB/s silent → 589.6 KB/s with sound (+1.31%).** F=11 buys 57.2
|
||
points of padding for 12,288 B of RAM; the floor buys the last 0.09 of a point
|
||
for 91,136 B more, and K4 already wants 99,328 B.
|
||
|
||
**4. THE CODEC CONTAINER PAYS NONE OF IT.** It already has an index and already
|
||
has variable records, so it puts exactly 651.0417 B in record *i* and pads to
|
||
the sector it was padding to anyway: **zero audio padding**, 440.4 → 448.1 KB/s.
|
||
**That is the first price anyone has found for the packed branch's own
|
||
simplification.** 64's risk list said a deletion that large usually hides
|
||
something; this is the first thing it hid, and it is small — a cadence, a
|
||
padding fraction and 14,336 B.
|
||
|
||
**5. THE EXPERIMENT THAT DID NOT WORK, and the facts it did leave.** MAME's
|
||
x68000 **has the chip** — `:okim6258`, reached at **`$E92001` and `$E92003`**,
|
||
both read out of the machine's own program map rather than from folklore, with
|
||
the PPI at `$E9A000-$E9BFFF`. **Feeding it from Lua produced silence**: control
|
||
0..3 against port C 0..15, `-wavwrite` capture **0 of 567,360 samples non-zero**.
|
||
The gap is register semantics and it was **not guessed at further**. The way to
|
||
close it is from 68000 code with the IPL ROM's own channel-3 configuration,
|
||
which `21_iplrom_dmac.py` already reads out of the ROM.
|
||
|
||
**RISKS IN THIS SESSION'S RESULT, stated rather than left to be found:**
|
||
|
||
- **The encoder is greedy, not optimal.** Each nibble is chosen by exhaustive
|
||
search over all sixteen minimising THIS sample's error; a nibble also moves
|
||
the step index, so a locally worse choice can pay later. 21.97 dB is a floor
|
||
for this format, not its ceiling, and no lookahead was tried.
|
||
- **21.97 dB is against the 12-bit word**, not against the disc's 16-bit PCM,
|
||
and the source was already resampled and downmixed to mono by ffmpeg. The
|
||
downmix matrix is ffmpeg's default and was not chosen.
|
||
- **P6a is open and it is worth 25 dB**, so every byte `adpcm.py` has produced
|
||
is provisional.
|
||
- **Nothing has played.** No audio has left an emulated machine, let alone a
|
||
board.
|
||
|
||
## HANDOFF — start here
|
||
|
||
**THE TREE IS ALL GREEN**, session 33's stage included.
|
||
|
||
### The work, in the order it should be done
|
||
|
||
**1. P6a — WHICH DELTA FORMULA, and it needs no board.** It is worth **25 dB**
|
||
(65.2), so every byte the encoder has produced is provisional until it is
|
||
answered, and it is the cheapest open item in the project by a distance. MAME's
|
||
x68000 has the chip and this session found where it lives — `:okim6258` at
|
||
**`$E92001`/`$E92003`**, PPI at `$E9A000-$E9BFFF`, read out of the machine's own
|
||
program map (65.5).
|
||
|
||
**Do NOT retry the Lua feed.** It was swept — control 0..3 × port C 0..15 —
|
||
and recorded silence, and the gap is register semantics that nobody here should
|
||
be guessing at. **Do it from 68000 code with the IPL ROM's own channel-3 DMAC
|
||
configuration**, which `tools/analysis/21_iplrom_dmac.py` already reads out of
|
||
the ROM: dual address, 8-bit port, cycle steal without hold, external request.
|
||
That is the one ADPCM path in this machine that is known-correct because Sharp
|
||
wrote it, and it is also the real design — so it is not scaffolding, it is P6b's
|
||
transport arriving early.
|
||
|
||
The discriminating stream is already worked out: **twelve loud nibbles to climb
|
||
the step index, then every nibble in turn**, whose two reconstructions differ
|
||
first at sample 3 (523 against 522). Capture with `-wavwrite` and compare against
|
||
`adpcm.decode(nibs, 'shift')` and `adpcm.decode(nibs, 'terms')`. **Watch the
|
||
clipping** — the sequence above saturates at 2,047 within six samples, which
|
||
destroys discrimination; build one that alternates sign to hold the signal
|
||
mid-range while the step index climbs.
|
||
|
||
**2. THEN P6b — DLXP2, a container with sound in it.** 65.3 is all the
|
||
arithmetic: cadence **F=11, A=14**, `LBA(i) = LBA0 + i*97 + floor(i/11)*14`,
|
||
14,336 B held, wire 589.6 KB/s. It waits on P6a, because a container full of
|
||
audio encoded against the wrong formula is 25 dB of work to redo.
|
||
|
||
**3. WHAT IS LEFT OF P6 AFTER THAT** is the fourth quarter: 51.3's refill climb
|
||
with a second consumer through a real branch point. The slack table is in
|
||
`32_audio_wire.py`; nothing has been run.
|
||
|
||
### What is still BLOCKED, so it is not picked up by mistake
|
||
|
||
**K4 — the packed player that is on screen — is conditional on B2**, a board
|
||
question. If buffer mode does NOT blank, K3's player is already on screen the
|
||
whole slot and K4's paint is 27.3% of a frame spent on nothing. **E7, E4 and C1**
|
||
are parked (61.8), and **P4a's wiring** is parked with the ring K3 deleted.
|
||
|
||
**The hardware list is unchanged and is the user's**: B1 (sustained AND the
|
||
data-phase BURST rate, 64.2), B2 (blanking — the five-minute half), B3 (`#EXREQ`),
|
||
B4 (a byte write to a palette register).
|
||
|
||
### Risks that are OURS, not hardware
|
||
|
||
1. **P6a is open and it is worth 25 dB.** Everything in `adpcm.py`'s output is
|
||
provisional.
|
||
2. **The encoder is greedy.** Exhaustive per-sample search, no lookahead;
|
||
21.97 dB is this format's floor here, not its ceiling.
|
||
3. **Nothing has played.** No audio has left an emulated machine.
|
||
4. **The packed branch's simplification has now cost something once** (65.4).
|
||
It was predicted in the abstract and it was small. It may not be the only one.
|
||
|
||
### Reproducing this session
|
||
|
||
./tools/bench/check.sh # ALL GREEN
|
||
python3 tools/encoder/extract_audio.py 00223 tmp/au_singe.raw 15625 539.4 10.0
|
||
python3 tools/bench/verify_adpcm.py tmp/au_singe.raw
|
||
python3 tools/analysis/32_audio_wire.py tmp/packed_singe.dlxp
|
||
|
||
**WHAT IS NEXT.** P6a: ask the chip which formula it runs, from 68000 code.
|
||
|
||
---
|
||
|
||
## Session 32: the packed player runs, and the write window turns out to be the frame
|
||
|
||
**Green light first and last: `./tools/bench/check.sh` was ALL GREEN before any
|
||
of this (`tmp/check_s32_start.log`) and ALL GREEN after** — the same stages, plus
|
||
one new one.
|
||
|
||
**FINDINGS 64. ROADMAP K3 is done.** `src/player/packed.s` (2,898 B),
|
||
`tools/bench/packed.lua`, `tools/bench/prep_packed.py`,
|
||
`tools/bench/packed_run.sh`, `tools/bench/verify_packed.py`,
|
||
`tools/analysis/31_display_duty.py`. `tools/bench/mkvol.sh` grew a DLXP arm —
|
||
for a packed container the file **is** the volume, which is the format's own
|
||
claim arriving at the disc.
|
||
|
||
**Everything below is EMULATED**: MAME 0.277 `x68000`, `-bios ipl10`, stock
|
||
10 MHz / 2 MB, `-exp1 cz6bs1`, a real CHD volume. No real hardware ran.
|
||
|
||
**1. THE PLAYER WORKS, AND IT IS THE STRONGEST END-TO-END RESULT IN THE TREE.**
|
||
The 68000 brings up its own display (CRTC mode, both scroll register pairs,
|
||
priority `vc1 = $0002`, R20 bit 11), builds its own **193-entry** DMA chain,
|
||
keeps its own frame clock off V-DISP, and fetches every record itself with
|
||
`READ(10)`. The rig writes no picture byte, no palette entry and no CRTC
|
||
register.
|
||
|
||
| | |
|
||
|---|---|
|
||
| frames | **120 of 120 pixel-exact**, EVERY ONE COMPARED |
|
||
| both palette orders | palette FIRST and `--palette-last`, **120/120 each** |
|
||
| chain | `$E82000`/512 B then 192 rows of 256 B at the 1,024 B stride — one start |
|
||
| record `i` | LBA `1 + i*97`. Arithmetic. No index, no ring, no decoder |
|
||
|
||
The gate had to change shape: `verify_decode.py` checks the LAST frame and that
|
||
audits 120 because the codec is temporally recursive. **A packed frame is a
|
||
literal**, so `verify_packed.py` compares all of them, letterbox included.
|
||
|
||
**2. THE HEADLINE, AND IT IS NOT THE GOOD HALF.** The free-running run — which
|
||
is what a 12 fps player becomes once the transfer is longer than the slot —
|
||
reported a number no budget in this tree has a column for: **the write window
|
||
was open on 99.5% of the host frames.** Every frame was pixel-exact and almost none of them was
|
||
visible. It is arithmetic: a packed write needs R20 bit 11, buffer mode blanks
|
||
the layer, and a DMAC-direct player holds the window open for the whole data
|
||
phase — there is no second page to hide behind, because the packed layout spends
|
||
both.
|
||
|
||
dark fraction of a slot = record bytes / (DATA-PHASE rate x slot)
|
||
|
||
**The rate in that expression is the BURST rate, not the sustained one**, and
|
||
that is a third hardware number **B1 has no test for**. At the container's own
|
||
582.0 KB/s the dark fraction is **1.0**: every frame delivered, on time,
|
||
pixel-exact, and **none of them displayed**.
|
||
|
||
**3. AND IT REVERSES 61.5's RANKING.** A packed player that DMAs into RAM with
|
||
the window SHUT and paints with the measured `movem` blit (27.3% of a slot) is
|
||
on screen **72.7% of every slot at any rate**. The two are equal only at a
|
||
data-phase rate of **2,131 KB/s = 3.7x the wire**. Below that — every rate
|
||
anyone has proposed — **the player with the CPU in the loop is on screen
|
||
longer.** That is ROADMAP **K4**, and it costs 82.2% of a frame at the 9 clk/B
|
||
floor against 54.9%, plus 99,328 B of RAM for two record buffers.
|
||
|
||
**4. A HELD CHANNEL COSTS THE FRAME CLOCK HALF ITS TICKS, AND THE CLOCK CANNOT
|
||
TELL.** `clock.i` counts V-DISP interrupts; a held channel halts the 68000; the
|
||
MFP's pending bit is one bit. Held at 12 fps: **487 of 1,038 edges lost
|
||
(46.9%)**, and the player reported **zero late frames** — because `PACE` is
|
||
advanced by the ISR the channel stops from running, so the loss cancels out of
|
||
the comparison. The player believed 12 fps; the screen was at **6.37**. Only the
|
||
host's raster count contradicts it, and `packed_run.sh` gates on the difference
|
||
being non-zero. Stealing at half rate loses 0.6%. Held, the CPU went round its
|
||
transfer wait **120 times in 120 frames** — once each; it never executed during a
|
||
transfer.
|
||
|
||
**5. THE CHANNEL CONFIGURATION DOES NOT SET THE TRANSPORT'S TIME.** Free-running,
|
||
held and stealing delivered the same record within **0.5%** (90.27 vs 90.72 ms).
|
||
That figure is the APPARATUS — MAME's device models carry no transfer timing —
|
||
but the negative is real and no arithmetic could have given it: **a channel
|
||
configuration buys who owns the CPU, not when the picture appears.**
|
||
|
||
**RISKS IN THIS SESSION'S RESULT, stated rather than left to be found:**
|
||
|
||
- **Everything in 2 and 3 is conditional on B2**, and B2 is still unmeasured on
|
||
a board. What changed is the stakes: it now picks WHICH packed player, and
|
||
each answer makes the other one's design work wasted.
|
||
- **MAME's ~535 KB/s transport ceiling is not diagnosed.** It is not the DMAC
|
||
(both configurations agree) and not the CPU (held, it is halted throughout),
|
||
and no MAME source tree was on this machine to name it. It bounds nothing in
|
||
the arithmetic, which is geometry, but it is the reason the gate is paced at
|
||
half rate and that should not be mistaken for a rig limitation being worked
|
||
around.
|
||
- **The cadence sampler is quantised to one host frame** (17.6 ms). The MEAN
|
||
over 120 frames is not (±0.30 ms); the min/median/max are the sampler's.
|
||
|
||
## HANDOFF — start here
|
||
|
||
**THE TREE IS ALL GREEN**, session 32's stage included (`tmp/check_s33_start.log`,
|
||
ALL GREEN, ~6 min).
|
||
|
||
### The work, in the order it should be done
|
||
|
||
**1. NOT K4 — NOT YET.** ROADMAP K4 is written and priced (82.2% of a frame at
|
||
the 9 clk/B floor, 99,328 B of RAM, on screen 72.7% of every slot at any rate),
|
||
and it is **conditional on B2**, which is a board question nobody here can
|
||
answer. If buffer mode does NOT blank, K3's player is already on screen the
|
||
whole slot and K4's paint is 27.3% of a frame spent on nothing. 61.7 wrote that
|
||
rule for the codec; it applies to the branch that replaced it. **Do not open
|
||
K4 until B2 comes back.**
|
||
|
||
**2. WHAT IS NOT BLOCKED IS P6 — AUDIO.** It is the largest unpriced item left
|
||
in the project and it is on M3's critical path, which is the completion target.
|
||
The bus half closed in session 20 (52: 1.25%..1.48% of a frame, and the unit
|
||
error was caught) — **everything else in it is open**: extraction, an encoder,
|
||
the container interleave, and what a second stream does to `wire` and therefore
|
||
to 51.3's refill climb. It is independent of B2 in the part that matters: both
|
||
packed containers and the codec carry the same audio.
|
||
|
||
Three things make it a session rather than a chore:
|
||
- **No ffmpeg ENCODER exists for this format.** `adpcm_ima_oki` is a decoder
|
||
only, so the encoder is ours to write and ours to gate.
|
||
- **The two available references DISAGREE, and it was MEASURED not recalled**
|
||
(session 33, `tmp/adpcm_probe/`). ffmpeg's `adpcm_ima_oki` is
|
||
`((2*delta+1) * step) >> 3`, 12-bit signal clipped to [-2048, 2047], output
|
||
x16, **high nibble first** — reproduced sample-exact over 2,268 nibbles. The
|
||
OKI datasheet form truncates per term (`step/8 + step + step/2 + step/4`,
|
||
selected by nibble bits) and differs on **445 of those 2,268 samples, by up to
|
||
4 in 12-bit units**. Which one the machine runs is not a matter of taste, and
|
||
**MAME has an `okim6258` in the x68000 — so it can be asked.**
|
||
- **DLXP1 has no room for it.** A packed record is 97 sectors EXACTLY because
|
||
49,664 B is what a literal frame costs; audio does not fit in the slack,
|
||
because there is none. What audio does to the packed record's geometry is a
|
||
container question, not an encoder one.
|
||
|
||
### What is PARKED, so it is not re-derived
|
||
|
||
**E7, E4 and C1** (61.8), and **P4a's wiring** for the video path (K3 deletes the
|
||
ring from it). Do not delete any of it — B2 is unanswered.
|
||
|
||
### Risks that are OURS, not hardware
|
||
|
||
1. **A chained transfer has never run back to back at 12 fps and now it has**,
|
||
120 times, pixel-exact — but only at HALF rate, because at full rate there is
|
||
nothing to snapshot (64.2). The gate's pacing is a consequence of the finding.
|
||
2. **The packed player deletes `ring.i`, `xfer.i` and most of `stream.s`.**
|
||
Unchanged: a simplification that large usually hides something, and it has
|
||
now run clean for 120 frames without finding out what.
|
||
3. **MAME's ~535 KB/s transport ceiling is undiagnosed** (64.4). It bounds
|
||
nothing in the arithmetic, which is geometry.
|
||
4. **B2 sits over the branch and now picks WHICH PLAYER**, not how much headroom
|
||
one has.
|
||
|
||
### Reproducing this session
|
||
|
||
./tools/bench/check.sh # ALL GREEN
|
||
bash tools/bench/packed_run.sh tmp/packed_singe.dlxp # all four runs
|
||
python3 tools/analysis/31_display_duty.py
|
||
|
||
**WHAT IS NEXT.** P6: audio, from the disc to a byte count that the container
|
||
has to carry.
|
||
|
||
---
|
||
|
||
## Session 31: the packed container, and the palette that buys 2.31 dB has a price
|
||
|
||
**Green light first and last: `./tools/bench/check.sh` was ALL GREEN before any
|
||
of this (`tmp/check_s31_start.log`) and ALL GREEN after** — the same stages, plus
|
||
one new one.
|
||
|
||
**FINDINGS 63.** Mostly **host arithmetic** over the Blu-ray's own frames; the
|
||
one rendering check is **px68k's real `x68k/gvram.c`**, linked headless the way
|
||
`tools/bench/c68k` links its CPU core. No MAME run was needed and no real
|
||
hardware ran. 47.4 — does buffer mode blank the layer — is still the board
|
||
question, and this session made it **bigger**.
|
||
|
||
**THE HEADLINE. ROADMAP K2 is done.** `tools/encoder/dlxp.py` is the format,
|
||
`pack.py` the encoder, `tools/analysis/30_packed_container.py` the gate and the
|
||
re-derivation, `tools/bench/gvpack/verify_dlxp.py` the second-emulator render.
|
||
|
||
| | |
|
||
|---|---|
|
||
| record | **49,664 B = 97 sectors EXACTLY**, 512 B palette + 49,152 B picture |
|
||
| index | **none, and none can be needed** — a packed record's length is geometry |
|
||
| wire | **582.0 KB/s**, exactly what 61.9 predicted |
|
||
| encode | **3.3 s** for 120 frames, against ~55 s for the codec |
|
||
| render | **index-exact on px68k, with the harness computing no interleave** |
|
||
|
||
**1. THE RE-DERIVATION SESSION 30 ASKED FOR, and it had TWO debts not one**
|
||
(63.3). 61.9's 34.08 dB was PIL's free 256-colour MEDIANCUT; the unnamed second
|
||
debt was that **every PSNR this project has quoted — 29.19, 31.33, 34.08 — is
|
||
RGB888, upstream of the `GGGGGRRRRRBBBBBI` word (23.3)**, and a packed record
|
||
carries that word and nothing else.
|
||
|
||
| | RGB888 | **GRB555** |
|
||
|---|---:|---:|
|
||
| CODEC, the gate container (440.4 KB/s) | 29.07 | **28.72** |
|
||
| 256c scene palette — the codec's CEILING | 31.32 | **30.79** |
|
||
| PACKED, 254c SCENE palette *(control)* | 31.32 | **30.79** |
|
||
| **PACKED CONTAINER, 254c PER-FRAME** | **34.05** | **33.10** |
|
||
|
||
61.9 survives the real builder: **34.05 against 34.08**. The layout's two
|
||
reserved entries cost **+0.0003 dB**. The GRB555 word costs 0.53 dB and costs
|
||
every row, so it moves no comparison.
|
||
|
||
**2. THE CONTROL IS THE FINDING UNDER THE HEADLINE.** A packed container with a
|
||
SCENE palette lands **exactly on the codec's ceiling**, as it must. So the whole
|
||
+2.31 dB the packed branch has over that ceiling is **the per-frame palette and
|
||
nothing else** — not the packing, not literal frames, not 254 colours.
|
||
|
||
**3. 62.5 IS PRICED, AND THE ORDER IS A WASH** (63.4). Palette first 20.32 dB,
|
||
palette last 20.33 — 0.01 apart, so the choice can be made on other grounds, and
|
||
`dlxp.py` makes it a container flag rather than an assumption.
|
||
|
||
**4. AND THE MISMATCH IS NOT A WASH.** **231 of 256 palette entries change every
|
||
frame (90%)**, and a picture under the neighbouring frame's palette is **12.8 dB
|
||
worse**. It is a wipe, not a flash, and the transfer is 55.2% of a frame slot —
|
||
so at 12 fps a colour-scrambled region is on screen for roughly half of every
|
||
slot, forever. **B2 stopped being a headroom question and became a picture
|
||
question again**: if buffer mode does not blank, it decides not just which
|
||
player exists (61.6) but **which packed container** does. The codec never had
|
||
this exposure — its palette is scene-constant, so its tear is old picture
|
||
against new picture, never old colours against new ones.
|
||
|
||
**5. THE FALLBACK IS ALREADY IN THE ENCODER.** `pack.py --scene-palette
|
||
--no-palette`: **30.79 dB, zero churn, nothing to mismatch, +2.07 dB on the
|
||
shipping codec as the display renders both, and 576.0 KB/s instead of 582.0.**
|
||
The per-frame palette is now a **priced** +2.31 dB rather than a free one.
|
||
|
||
## HANDOFF — start here
|
||
|
||
**THE TREE IS ALL GREEN.** Session 31's work is a commit of its own.
|
||
|
||
### The work, in the order it should be done
|
||
|
||
**1. K3 — END TO END, OFF THE DISC.** Scene setup — palette, page-1 X-scroll
|
||
384, priority `vc1 = 0x0002`, R20 bit 11, the 384 zeroed words a row and the
|
||
letterbox rows — then **one chained DMA a frame** out of a DLXP1 record, with
|
||
the palette as the 193rd or 1st array entry (63.4 says either). Gate it
|
||
pixel-exact over the same 120-frame window the decoder is gated on. The
|
||
container is `tmp/packed_singe.dlxp` and `check.sh` rebuilds it every run.
|
||
|
||
**2. RUN BOTH PALETTE ORDERS while K3 is being built.** It is a flag
|
||
(`--palette-last`), the containers already exist, and 63.4 bounds the cost but
|
||
cannot decide it — dB over a whole frame is not what an eye sees in a wipe.
|
||
|
||
### What is PARKED, so it is not re-derived
|
||
|
||
**E7, E4 and C1** (61.8), and **P4a's wiring** for the video path. Do not delete
|
||
any of it — B2 is unanswered and 48.1's prior leans against packing.
|
||
|
||
### Risks that are OURS, not hardware
|
||
|
||
1. **A chained transfer has never run back to back at 12 fps.** Unchanged since
|
||
session 30: run 9 was seven entries, once. A frame is 193 and a second is
|
||
twelve frames.
|
||
2. ~~34.08 dB is PIL's MEDIANCUT~~ **CLOSED, 63.3: 34.05 dB against this
|
||
project's own builder, and the hardware word is charged on top.**
|
||
3. **The packed player deletes `ring.i`, `xfer.i` and most of `stream.s` from
|
||
the video path.** A simplification that large usually hides something.
|
||
4. **B2 sits over all of it, and 63.4 raised what it decides.** B4 still sits
|
||
over the palette entry specifically, with a small blast radius.
|
||
|
||
### Reproducing this session
|
||
|
||
./tools/bench/check.sh # ALL GREEN
|
||
python3 tools/encoder/pack.py tmp/fr_singe tmp/packed_singe.dlxp --nframes 120
|
||
python3 tools/analysis/30_packed_container.py tmp/packed_singe.dlxp
|
||
python3 tools/bench/gvpack/verify_dlxp.py tmp/packed_singe.dlxp 0 --controls
|
||
|
||
**WHAT IS NEXT.** K3: end to end, off the disc.
|
||
|
||
---
|
||
|
||
# Status & next-session handoff — end of session 30 (2026-08-25)
|
||
|
||
## Session 30: one channel start paints a whole frame, and the palette has no handler to be wrong about
|
||
|
||
**Green light first and last: `./tools/bench/check.sh` was ALL GREEN before any
|
||
of this (`tmp/check_s30_start.log`) and ALL GREEN after** — the same stages, with
|
||
three new runs inside the DMA gate.
|
||
|
||
**FINDINGS 62.** **Emulated** — MAME 0.277, `x68000 -bios ipl10`, `-exp1 cz6bs1`.
|
||
No real hardware ran. 47.4 — does buffer mode blank the layer — is still the
|
||
board question and is still open, and **this session opened a second one.**
|
||
|
||
**FIRST, THE UNRECORDED WORK WAS COMMITTED.** Sessions 28 and 29 were sitting in
|
||
a dirty tree with the last commit being session 27's. They are one commit now
|
||
(`1be428c`), because their edits share files and could not be split honestly
|
||
after the fact. Session 30's own work is a commit of its own.
|
||
|
||
**THE HEADLINE. ROADMAP K1 is done and the answer is the good one: a DMA channel
|
||
writes the palette registers at `$E82000`, and one array-chained start crosses
|
||
from device registers into GVRAM.** So a packed frame is **one channel start**:
|
||
a 193-entry array — the palette, then 192 picture rows at the 1,024 B line
|
||
stride — walked by the channel with the 68000 halted from the first byte to the
|
||
last. `dmagate.s` runs 7, 8 and 9; `dma_run.sh` gates all three; `check.sh` runs
|
||
it.
|
||
|
||
| run | result |
|
||
|---|---|
|
||
| **7. DMA → `$E82000`, bus held** | **512 B byte-exact in 256 register words**, read back out of the registers by the 68000 |
|
||
| **8. the SAME transfer aimed at RAM** | byte-exact at `$2C000`, and **256 of 256 palette words still poison** |
|
||
| **9. ONE array-chained start, palette + 6 rows** | **2,048 B byte-exact across BOTH kinds of destination**, `MAR` at `+5,376` |
|
||
|
||
**1. THE DESTINATION IS POISONED NOW, and it should have been all along** (62.1).
|
||
Runs 4–6 wrote into RAM that was zero and GVRAM that was stale, against a record
|
||
that is mostly pad — "it matches the disc" was a weaker claim than it read as.
|
||
`dg_poison` puts `$A500|i` in palette word *i* first, and the host **counts**
|
||
whether that actually discriminates rather than assuming it: 511 of 512
|
||
positions differ from the disc's bytes, and the gate refuses a run under 500.
|
||
|
||
**2. THE CONTROL IS AN ATTRIBUTION CONTROL, and it is a different shape from run
|
||
5's** (62.2). There is no mode bit here to point at. The same transfer aimed
|
||
20 KB away makes two claims at once: the bytes land at `$2C000`, and the palette
|
||
still reads poison — so what reached `$E82000` in run 7 was decided by the
|
||
channel's `MAR`. The second half is also the positive control on the readback
|
||
path, because the poison it finds is what the CPU wrote.
|
||
|
||
**3. THE ARRAY IS SCENE-CONSTANT** (62.3). The row bases never change: the
|
||
packed layout spends both 256-colour pages — page 0 the low byte of a word,
|
||
page 1 the high — so there is no page to flip into. The 1,158 B array is built
|
||
once at scene setup and started once a frame. **What is still on the CPU per
|
||
frame is the channel start and the READ(10)**, and neither is priced here: say
|
||
"no per-frame PAINT work", not "no per-frame CPU work".
|
||
|
||
**4. AND THE NEW BOARD QUESTION, B4** (62.4). **MAME cannot be asked this
|
||
one.** `x68k.cpp:817` maps the palette to `palette_device::read16/write16`,
|
||
`emupal.cpp:417` forwards to `memory_array::write16`, and `memarray.h:75` is a
|
||
plain `COMBINE_DATA` — RAM that honours `mem_mask`, with **no handler that could
|
||
refuse a byte write**. That is a different bound from 61.2's: GVRAM has a real
|
||
256-colour arm, which is why run 5 could find a mechanism to fail on. What a
|
||
real X68000 palette register does with a byte write is **UNMEASURED — not
|
||
folklore, an absence.** B4 is the cheapest hardware item in the project: write
|
||
`$A5` to `$E82000`, `$5A` to `$E82001`, read the word back. A negative costs
|
||
0.28% of a frame and nothing else, which is why it sits below B2.
|
||
|
||
**5. ONE DESIGN CHOICE IS FILED, NOT ANSWERED** (62.5). Palette FIRST or 193rd
|
||
is visible on screen — old rows under the new palette, or new rows under the old
|
||
one — for one paint either way. It is moot if buffer mode blanks (B2). It is
|
||
named so the final player's order is a decision and not an accident of which run
|
||
happened to be written first.
|
||
|
||
## HANDOFF — start here
|
||
|
||
**THE TREE IS ALL GREEN AND COMMITTED.** Two commits: `1be428c` (sessions 28+29,
|
||
unrecorded until now) and session 30's own.
|
||
|
||
### The work, in the order it should be done
|
||
|
||
**1. K2 — A PACKED CONTAINER.** Simpler than a coded one: quantise to **254**
|
||
colours (index 0 is the transparency key, black at 255 — 47.2), interleave
|
||
column *i* with *i+128*, emit a per-frame palette, sector-align the record. No
|
||
VQ, no rate control, no mode map, no `lam`. `encode.py` already sector-aligns
|
||
(DLX5) and `prep_frame.py --pack-transparent` already emits the exact pixel
|
||
layout. **Re-derive 61.9's 34.08 dB against `vq.scene_palette` / `H.build`
|
||
here** — the measured figure is PIL's MEDIANCUT and is a direction, not the
|
||
player's number (risk 2 below).
|
||
|
||
**2. K3 — END TO END, OFF THE DISC.** Scene setup — palette, page-1 X-scroll
|
||
384, priority `vc1 = 0x0002`, R20 bit 11 — then **one chained DMA a frame**,
|
||
now including the palette entry K1 just established. Gate it pixel-exact over
|
||
the same 120-frame window the decoder is gated on.
|
||
|
||
### What is PARKED, so it is not re-derived
|
||
|
||
**E7, E4 and C1** (61.8), and **P4a's wiring** for the video path: a DMAC-direct
|
||
packed player has no ring at all. Do not delete any of it — B2 is unanswered and
|
||
48.1's prior leans against packing.
|
||
|
||
### Risks that are OURS, not hardware
|
||
|
||
1. **A chained transfer has never run back to back at 12 fps.** Unchanged by
|
||
this session: run 9 was seven entries, once. A frame is 193 and a second is
|
||
twelve frames.
|
||
2. **34.08 dB is PIL's MEDIANCUT**, not `vq.scene_palette` / `H.build`.
|
||
3. **The packed player deletes `ring.i`, `xfer.i` and most of `stream.s` from
|
||
the video path.** A simplification that large usually hides something.
|
||
4. **B2 sits over all of it**, and now B4 sits over the palette entry
|
||
specifically — with a known, small blast radius.
|
||
|
||
### Reproducing this session
|
||
|
||
./tools/bench/check.sh # ALL GREEN, ~5 min
|
||
bash tools/bench/dma_run.sh # runs 1-10, with controls
|
||
|
||
**WHAT IS NEXT.** K2: the packed container.
|
||
|
||
---
|
||
|
||
# Status & next-session handoff — end of session 29 (2026-08-25)
|
||
|
||
## Session 29: the decoder-free packed player fits the budget the codec misses
|
||
|
||
**Green light first and last: `./tools/bench/check.sh` was ALL GREEN before any
|
||
of this and ALL GREEN after**, plus one new stage.
|
||
|
||
**FINDINGS 61.** **Emulated** — MAME 0.277, `x68000 -bios ipl10`, and
|
||
`-exp1 cz6bs1` for the transport runs. No real hardware ran. 47.4 — does buffer
|
||
mode blank the layer — is still the board question and is still open.
|
||
|
||
**THE HEADLINE.** At the 9 clk/B dual-address floor, which 59.2 established is
|
||
the only configuration this machine can be shown to run:
|
||
|
||
| architecture | B/frame | % of a 12 fps frame at W=9 | KB/s |
|
||
|---|---:|---:|---:|
|
||
| **CODEC, the shipping design** | 37,585 | **110.4% — MISSES** | 440.4 |
|
||
| **decoder-free PACKED, DMAC device→GVRAM** | 49,152 | **55.2% — FITS** | 576.0 |
|
||
| **decoder-free PACKED, CPU-painted** | 49,152 | **81.6% — FITS** | 576.0 |
|
||
|
||
**Decoding 37,585 bytes costs 109% of a frame; NOT decoding 49,152 costs 54%.**
|
||
44.7 said this in advance on a cost model that has since been thrown away — "the
|
||
codec is not there to save CPU, it is there to save the wire" — and the measured
|
||
model agrees and goes further. **The question is now entirely a medium question.**
|
||
|
||
**1. THE PACKED PAINT IS MEASURED, and 47.6.1's assumption is retired.**
|
||
`blit.s` V8 is V1 with 128 words a row instead of 256. **227,553 clocks, 27.3%
|
||
of a frame** — 51.0% of V1, and **100.8% of V3, the unpacked WRITE-ONLY floor**.
|
||
The packed blit costs what the unpacked path pays to write alone, source read
|
||
thrown in free. V1 came out at 53.6% again, unmoved since session 9, which is
|
||
what licenses quoting V8 next to it.
|
||
|
||
**2. A CHANNEL FILLS GVRAM OFF THE DISC, AND WALKS THE LINE STRIDE ITSELF.**
|
||
Three new runs in `dmagate.s`, all gated: bit 11 SET → 2,048 B byte-exact, half
|
||
into page 1 and half into page 0, CPU never executed; bit 11 CLEAR → **457 bytes
|
||
lost, every one at an EVEN offset and not one ODD byte harmed**; and **array
|
||
chained, 8 rows at the 1,024 B line stride from ONE start.** 47.6.2 is answered.
|
||
|
||
**3. THE FIRST CUT OF THAT TEST COULD NOT HAVE FAILED.** It OR-ed bit 11 onto
|
||
whatever the IPL left, and the IPL leaves `$0B16` — bit 11 already set, COL
|
||
`%11`, which writes whole words either way. It passed and proved nothing. Both
|
||
runs now write R20 outright and differ in exactly one bit.
|
||
|
||
**4. AND THE CONTROL'S SIGNATURE IS A PLACE, NOT A COUNT.** "Half the bytes must
|
||
differ" turned the gate RED on a correct run: `gvram_w` ignores `mem_mask` in the
|
||
masked arm, so the high half keeps stale GVRAM, and where the record is pad the
|
||
stale half matches by coincidence — 567 of 1,024 did. The claim the mechanism
|
||
actually makes is that odd bytes survive and only even ones are lost.
|
||
|
||
**5. THE CODEC CANNOT HAVE THE PACKING — 47.6.4 CLOSED, both ways.** A 4×4 block
|
||
owns four bytes at stride 2. Writing them one at a time (V9) is **97.9% of a
|
||
frame against V4's 76.6% — 28% DEARER, for no saving on the wire at all.**
|
||
Pairing the blocks 128 columns apart (V10) gets the `movem` back at 38.5%, and
|
||
pays in the mode map: **SKIP falls from 66.3% of blocks to 46.1% of pairs**, so
|
||
1.60× as many blocks are painted — about **+60% on the bytes**, against an E7
|
||
target that needs them **down 35%**.
|
||
|
||
**6. WHAT B1 AND B2 NOW DECIDE.** 576 KB/s sustained, fixed, with no lever —
|
||
against 453.6 KB/s (gate container, zero prefill), 327 KB/s (E7's target) and
|
||
folklore that says 0.7–1.7 MB/s with no benchmark behind it. And under MAME's
|
||
blanking reading the **cheap architecture is the dark one**: a CPU-painted packed
|
||
player is black for 27.3% of a frame, the cheaper DMAC-direct one for 30–113%,
|
||
because the black interval is the paint and the CPU's transport lands in RAM
|
||
where bit 11 is irrelevant.
|
||
|
||
**7. AND THE PICTURE IS BETTER, WHICH NOBODY HAD ASKED (61.9).** The codec is
|
||
**capped at 31.33 dB** — every codeword is an index into the scene palette — and
|
||
delivers 29.19. A literal frame has no codebooks, so per-frame palettes are legal
|
||
and it simply **IS 34.08 dB: +4.89 dB on the shipping container, +2.75 dB past a
|
||
ceiling the codec cannot cross**, for +1.0% on the wire and 0.28% of a frame.
|
||
**The packed player is not a quality compromise bought with clocks. It is better
|
||
on both, and its whole cost is on the wire.**
|
||
|
||
**ENCODER WORK IS PARKED (USER DECISION).** The handoff first said this was no
|
||
reason to stop codec work. It does not survive its own arithmetic: that rested on
|
||
576 KB/s against **E7's 327, which does not exist**, when the codec that exists is
|
||
440 KB/s and 110.4% of a frame — a 1.31x gap, not 1.76x. Packed needs two facts;
|
||
the codec needs E7 to succeed (two of three levers already measured inert, 60.4
|
||
and 60.5) AND the medium AND a ship at ~100% of the frame with no margin. **E7 and
|
||
E4 are both parked** — E4 too, because `H.build`'s k-means builds VQ codebooks and
|
||
a literal player has no VQ — and C1 follows E4. **The codec stays on disk and
|
||
nothing is built on it**, because 48.1's prior leans against packing and if
|
||
buffer mode blanks it is the only path left (48.3). That is inventory, not work.
|
||
|
||
---
|
||
|
||
## HANDOFF — start here
|
||
|
||
**THE TREE IS ALL GREEN** (`tmp/check_s29_handoff.log`, and four times before it this session), and **DIRTY WITH TWO
|
||
SESSIONS IN IT.** Session 28's work was never committed either, so `git status`
|
||
shows 26 modified files spanning both sessions plus one new file. The last
|
||
commit is session 27's (`8800d8f`, "Price the transport against the frame").
|
||
**Nothing here is half-finished — it is unrecorded, which is a different
|
||
problem.** Commit before starting anything, or the next green light cannot be
|
||
attributed to a session.
|
||
|
||
**THE DIRECTION CHANGED (USER DECISION, session 29): build the DECODER-FREE
|
||
PACKED player, keep the codec on disk and build nothing on it.** The reasoning is
|
||
61.8; the numbers that forced it are 61.4 (55.2% of a frame against the codec's
|
||
110.4%) and 61.9 (+4.89 dB, and the codec is *capped* 2.75 dB below it).
|
||
|
||
### The work, in the order it should be done
|
||
|
||
**1. CAN A DMA CHANNEL WRITE THE PALETTE REGISTERS AT `$E82000`? (61.9, open)**
|
||
The cheapest item and the one with the largest structural payoff. If the palette
|
||
takes a byte-wide DMA the way GVRAM does in buffer mode, it becomes a **193rd
|
||
array-chain entry** and ONE channel start paints a whole frame — palette
|
||
included, CPU halted throughout, no per-frame CPU work in the video path at all.
|
||
It is the same shape as the three probes that worked this session: another run in
|
||
`src/player/dmagate.s`, host-side byte comparison in `tools/bench/dma.lua`, an
|
||
assertion in `tools/bench/dma_run.sh`, **and a negative control** — see the traps
|
||
below, this is exactly where session 29 nearly shipped a vacuous pass.
|
||
|
||
**2. A PACKED CONTAINER.** Simpler than a coded one, not harder: quantise to 254
|
||
colours, interleave column *i* with column *i+128*, emit a per-frame palette,
|
||
sector-align the record. No VQ, no rate control, no mode map, no `lam`.
|
||
`encode.py` already sector-aligns (DLX5) and
|
||
`tools/bench/prep_frame.py --pack-transparent` already emits the exact pixel
|
||
layout 47.2 verified pixel-exactly on both emulators.
|
||
|
||
**3. END TO END, OFF THE DISC.** Scene setup — palette, page-1 X-scroll **384**,
|
||
priority `vc1 = 0x0002`, R20 bit 11 — then one chained DMA a frame. Gate it the
|
||
way the existing decoder is gated: pixel-exact over the same 120-frame window
|
||
against `dlx.py`'s reconstruction.
|
||
|
||
### What is PARKED, so it is not re-derived
|
||
|
||
**E7, E4 and C1** (61.8). E4 is included on purpose: `H.build`'s k-means builds
|
||
**VQ codebooks**, and a literal player has no VQ. **P4a's wiring** — the channel
|
||
behind `ring.i`'s `XF_*` mailbox — is parked *for the video path* too: a
|
||
DMAC-direct packed player has **no ring at all**. Do not delete any of it; B2 is
|
||
unanswered and 48.1's prior leans against packing.
|
||
|
||
### Risks that are OURS, not hardware
|
||
|
||
1. **A chained transfer has never run back to back at 12 fps.** 61.2's run 6 was
|
||
eight rows, once. A frame is 192 entries and a second is twelve frames.
|
||
2. **34.08 dB is PIL's MEDIANCUT**, not `vq.scene_palette` / `H.build`. The
|
||
direction is measured; re-derive the number against the builder that ships it
|
||
before quoting it as the player's.
|
||
3. **The packed player deletes `ring.i`, `xfer.i` and most of `stream.s` from the
|
||
video path.** A simplification that large usually hides something.
|
||
4. **B2 sits over all of it.** Items 1 and 2 survive a bad answer; item 3 does
|
||
not.
|
||
|
||
### Traps this session stepped in, both caught by the tree's own rules
|
||
|
||
**A test that could not fail.** The first GVRAM run OR-ed R20 bit 11 onto
|
||
whatever the IPL left — and the IPL leaves `$0B16`, bit 11 **already set**, COL
|
||
`%11`, which writes whole words either way. It passed and proved nothing. Both
|
||
runs now write R20 outright and differ in exactly one bit.
|
||
|
||
**An assertion that was right about the mechanism and wrong about the number.**
|
||
"The masked control must lose half the bytes" turned the gate RED on a correct
|
||
run: `gvram_w` ignores `mem_mask` in the masked arm, so the high half keeps stale
|
||
GVRAM, and where the record is pad it matches the disc by coincidence — 567 of
|
||
1,024 did. **The mechanism's signature was a PLACE, not a count**: odd bytes
|
||
survive, only even ones can be lost. That is what the gate asserts now.
|
||
|
||
### Reproducing this session
|
||
|
||
./tools/bench/check.sh # ALL GREEN, ~5 min
|
||
python3 tools/analysis/29_packed_player.py tmp/rc_fr_singe_scsi_span.dlx
|
||
bash tools/bench/dma_run.sh # runs 4-6, with controls
|
||
python3 tools/analysis/18_text_plane_16col.py # the PSNR rows of 61.9
|
||
|
||
`check.sh` gained one stage that runs `blit.lua` and gates the ORDER of the paint
|
||
costs, not their values — V8 < V1, V9 > V4, V10 < V4 — because a tree where any
|
||
of those flipped has a different answer to 44.7 and should say so out loud.
|
||
|
||
**WHAT IS NEXT.** Item 1 above: the palette-register DMA probe.
|
||
|
||
|
||
|
||
---
|
||
|
||
# Status & next-session handoff — end of session 28 (2026-08-25)
|
||
|
||
## Session 28: the container agrees with the medium, and two encoder levers are not levers
|
||
|
||
**Green light first and last: `./tools/bench/check.sh` was ALL GREEN before any
|
||
of this and ALL GREEN after** — and after a full re-encode of the gate
|
||
container, which is the first time this tree has replaced it.
|
||
|
||
**ROADMAP's RE-ENCODE BUNDLE (all four items under P2) is DONE. FINDINGS 60.**
|
||
**Emulated** — MAME 0.277 and px68k's C68K core. No real hardware ran.
|
||
|
||
**1. DLX5 — RECORDS ARE SECTORS.** `encode.py` pads every record to 512 B and
|
||
the scene header with it, so the frame stream starts on a sector boundary and
|
||
every record inside it does. `dlx.py` reads `DLX5`, and `record_lengths()` is
|
||
now the ONE place the alignment rule lives — it replaced four hand-copied
|
||
`4 + n + (-(4+n) % 4)` expressions in the analysis tools.
|
||
|
||
| gate container | DLX4 | **DLX5** |
|
||
|---|---:|---:|
|
||
| records starting on a sector boundary | 3/120 | **120/120** |
|
||
| off the disc, for 4,488,577 B of record | 4,548,608 (+1.34%) | **4,510,208 (+0.48%)** |
|
||
| into the ring | 4,488,588 | **4,510,208** |
|
||
| window or bounce copy | 0 PIO / +5 clk/B DMA | **0, and a channel can run it** |
|
||
|
||
The two byte counts are now the SAME NUMBER and check.sh gates on that
|
||
identity, both read out of the container rather than written into the script.
|
||
|
||
**2. THE CONSUMER HAD TO BE TOLD, and the ring's own audit caught it.**
|
||
`stream.s` released up to the last byte it READ, rounded to 4; under DLX5 that
|
||
strands up to 511 B of pad per record. `RD_PTR MISMATCH after frame 0` on the
|
||
first run. The release rounds to `RECALN` (geom.i) now. Every frame still
|
||
decoded pixel-exact while the free-space arithmetic was wrong, which is the
|
||
point: alignment is a contract with two sides.
|
||
|
||
**3. RESERVED BLACK (23.4, open since session 5).** The scene palette is 255
|
||
picture colours with index 0 held at (0,0,0), so the letterbox is true black and
|
||
`I = 0` falls out of `pack_palette`'s existing rule with no special case.
|
||
**0.04 dB** on the Singe window; the palette ceiling does not move.
|
||
|
||
**4. `--spans all` AS THE DEFAULT: MEASURED, AND REFUSED.** 267.9 KB/s / 28.88
|
||
dB / 2 frames late at `need`, against 448.2 KB/s / 29.07 dB / 1 late at `all`.
|
||
**+67% of the wire for +0.19 dB and one frame of 120**, against a container the
|
||
frame budget already says is 35% too big. `need` stays the default; the GATE
|
||
container keeps `all`, because it is a fixture and not a recipe.
|
||
|
||
**5. JOINT SPAN/LAM (E3): implemented, and it is a NO-OP.** `--joint-spans`
|
||
hands the span pass's freed bytes back to the lam search and re-spans. All four
|
||
cells of `{need,all} x {greedy,joint}` are BYTE-IDENTICAL, and so is
|
||
`--rc-floor open`. The reason is structural: **`lam` never leaves its floor on
|
||
any of 120 frames at either floor the encoder offers** (44.3), so there is
|
||
nothing to spend the freed bytes on. Kept, defaulted off.
|
||
|
||
**6. THE APPARATUS LIED SILENTLY.** MAME 0.277 served the compressed CHD's OWN
|
||
FILE BYTES as sector data — the buffer after READ(10) at LBA 0 was byte-for-byte
|
||
`dlxdisk.chd`'s first 4,096 bytes, starting `MComprHD` — while every READ
|
||
reported success and `chdman verify` passed. Not size, not geometry, not the
|
||
codec: the DLX5 image fails compressed at the working volume's exact length and
|
||
with a conventional geometry, and passes `-c none`. `mkvol.sh` builds
|
||
uncompressed now. **Only the per-byte comparison in `scsi.lua` could see it**; a
|
||
gate checking status and length would have been green all session.
|
||
|
||
**7. THE BUDGET BARELY MOVED, which is what a precondition should do.**
|
||
Decode 68.5% → **68.6%**, headroom 6.74 → **6.69 clk/B**, `W`=5 total 92.2% →
|
||
**92.4%**, E7's target 328 → **327 KB/s**, B1's zero-prefill requirement 451.4 →
|
||
**453.6 KB/s**. Every conclusion in 59.7 survives. `15_bus_occupancy.py` now
|
||
charges the **padded** record, because the disc carries the pad whether or not a
|
||
frame contains it.
|
||
|
||
**8. TWO CACHES WOULD HAVE TESTED THE WRONG ARTEFACT.** `16_span_roundtrip.py`
|
||
and `13_cpu_ratectl.py` cache `H.build` in a pickle keyed on the frames
|
||
directory alone; `H.build` gained an option this session. Both now store the
|
||
build parameters with the model and rebuild on a mismatch.
|
||
|
||
**WHAT IS NEXT.** P4a's wiring: the DMA channel behind `ring.i`'s `XF_*` mailbox
|
||
in place of `xfer.i`'s PIO loop, which 59.4 blocked on exactly this
|
||
precondition. `xfer.i`'s sector arithmetic already degenerates correctly —
|
||
`SC_WSKIP` is 0 on every record now — so what changes is which loop moves the
|
||
bytes. `dma_run.sh`'s windowed-read refusal becomes a negative control.
|
||
|
||
---
|
||
|
||
# Status at end of session 27 (2026-08-24)
|
||
|
||
## Session 27: the DMAC drives the data phase, and auto-request is charged by time
|
||
|
||
**Green light first and last: `./tools/bench/check.sh` was ALL GREEN before any
|
||
of this and ALL GREEN after**, plus one new stage.
|
||
|
||
**ROADMAP P4a is DONE at the transport level. FINDINGS 59.** **Emulated** —
|
||
MAME 0.277, `x68000 -exp1 cz6bs1 -ramsize 2M`. No real hardware ran.
|
||
|
||
**1. THE CHANNEL DRIVES THE DATA PHASE.** `src/player/dma.i` programs HD63450
|
||
channel 1 and `src/player/dmagate.s` reads the same 2,048 B at LBA 1000 three
|
||
ways, with the host comparing all three against its own copy of the image:
|
||
|
||
| | bytes | MTC one instruction after START | CPU trips round its wait loop |
|
||
|---|---|---|---|
|
||
| PIO, the path 58.2 measured | byte-exact | — | — |
|
||
| **DMA, bus HELD** (DCR `$00`, OCR `$81`) | **byte-exact** | **0 of 2048** | **1** |
|
||
| **DMA, cycle STEALING** (DCR `$80`, OCR `$80`) | **byte-exact** | 2048 of 2048 | 426 |
|
||
|
||
Both channels reported `CSR = $E0`, `CER = $00`, `MTC = 0`, `MAR` exactly
|
||
`+2048`.
|
||
|
||
**2. THE EVIDENCE NEVER READS `$EA0015`, which is the whole design.** 57.3 said
|
||
it could not: with the DMAC's OWN asserted, MAME cannot tell a CPU-driven byte
|
||
at that address from a DMAC-driven one. So the discriminator is **the CPU's own
|
||
progress** — MTC sampled by the instruction *after* the one that starts the
|
||
channel. Held, the whole transfer happened between two instructions. **The gate
|
||
was checked against its own negative**: with the stealing register pair in the
|
||
held slot, every byte still arrives and `dma_run.sh` goes RED. A counter that
|
||
cannot come out different is 58.3's vacuous "UNDERRUNS: 0/120" again.
|
||
|
||
**3. THREE THINGS THIS MODEL CANNOT BE ASKED**, read out of MAME's source: the
|
||
card has **no request line to the DMAC** (its flow control is DTACK, so external
|
||
request cannot be run); **single address** cannot be run either (only channel 0
|
||
has device callbacks); and **only burst is modelled as held** (`(dcr & 0xc0) ==
|
||
0`, so cycle-steal-with-hold shares the not-held path). Of the four rows of the
|
||
`W` ladder, exactly one — dual address, held, 9 clk/B — has a code path here,
|
||
and it is the one demonstrated. The slot pinout has `#EXREQ` at B36, so a real
|
||
card plausibly drives it; that is ROADMAP B3, sharpened.
|
||
|
||
**4. AUTO-REQUEST IS CHARGED BY TIME, NOT BY BYTE — and this outlives the
|
||
emulator.** Every `W` in this project presumes the device asks for each byte. An
|
||
auto-requested channel spends its share of the bus whether or not a byte is
|
||
there, so a record's cost scales with **how long it takes to arrive**: halve the
|
||
rate and the CPU cost of the same record doubles. `tools/analysis/
|
||
28_autorequest_cost.py` prices it from MC68450 §3.8/§5.2.3.3.2, gating its
|
||
formulas against Table 5-3's sixteen rows first. At 37,405 B and **460 KB/s, an
|
||
explicit rate**: max rate costs the whole **95.3% of a frame**; of the GCR's
|
||
four bus shares only **BR = 00, 50%, carries the rate**, at 10.61 clk/B and
|
||
47.6% of a frame. **The GCR is a design lever nothing in this tree had named.**
|
||
|
||
**5. THE WINDOW IS REFUSED, so the re-encode is now a precondition.** A channel
|
||
writes a contiguous run and cannot drop the 300 B in front of a record, so
|
||
`sc_in_data` refuses a windowed read when the data phase is the DMAC's
|
||
(`SCE_WINDOW`) rather than writing the neighbours into the ring (49.2). 58.3's
|
||
"sector-aligned records should join the bundle" has become "the transport will
|
||
not run without them".
|
||
|
||
**6. WHAT IT ALL COSTS, and this is the part that re-ranks the roadmap.**
|
||
`15_bus_occupancy.py` now answers "what can the frame afford" as well as "what
|
||
does each `W` cost". On the gate container at 12 fps, with the decode term
|
||
MEASURED: decode 68.5%, audio 1.25%, **headroom 30.2% = 6.74 clocks per byte**
|
||
at a 37,403 B record.
|
||
|
||
| transport | clk/B | video | total |
|
||
|---|---|---|---|
|
||
| PIO, measured (58.2) | 87.28 | 391.7% | **461.5%** |
|
||
| dual address, held — **the floor** | 9 | 40.4% | **110.2%** |
|
||
| single address, held | 5 | 22.4% | **92.2%** |
|
||
|
||
**P4a cut the transport from 391.7% to 40..95%, four to ten times — and it still
|
||
does not fit.** A dual-address byte is 4 clocks of device read plus 5 of memory
|
||
write, so **9 is a floor and the frame affords 6.74**; no GCR share and no
|
||
delivery rate goes under it. At 460 KB/s max-rate totals 165.1% and LRAR at 50%
|
||
totals 117.4%. So the fit turns on **B3, which has stopped being a constant to
|
||
look up: does a real CZ-6BS1 drive `#EXREQ`?** If yes, single address fits at
|
||
92.2% with 7.8% spare. If no, the container must reach **328 KB/s of payload**
|
||
(new ROADMAP E7), against 438 now.
|
||
|
||
**The scope matters and is stated in 59.7:** this is the GATE container, the
|
||
heaviest thing the encoder emits. The lighter `cpufit` family was NOT priced —
|
||
the tool refuses it, correctly, because the C68K measurement belongs to the gate
|
||
container — so E7 begins with a harness re-run, and until then "34% too big" is
|
||
about the fixture, not about the project.
|
||
|
||
**ONE COLLISION, AND THE BEFORE-HALF OF `check.sh` CAUGHT IT.** `DM_USE` was
|
||
first placed at `$18300`, which is `ring.i`'s `XF_SLOT` mailbox — `scsi.i`'s
|
||
trace ends at `$182FF` and the next 160 bytes are the ring's. The ring rig's
|
||
first record request therefore wrote a non-zero word into what the transport
|
||
read as "use the DMAC", and **the P4b stage, which this session did not touch,
|
||
went red**. The map is now `$18500`. Green-before is not a formality: it is what
|
||
made a red in an untouched stage unambiguous. 59.6.
|
||
|
||
**New in the tree:** `src/player/dma.i`, `src/player/dmagate.s`,
|
||
`tools/bench/dma.lua`, `tools/bench/dma_run.sh`,
|
||
`tools/analysis/27_dmac_config.py` (decodes what the player programs, out of the
|
||
player's own source), `tools/analysis/28_autorequest_cost.py`,
|
||
`tools/analysis/mc68450.py` (the MC68450 field tables, now ONE copy, shared with
|
||
`21_iplrom_dmac.py`), and one `check.sh` stage. `src/player/scsi.i` gained the
|
||
`DM_USE` dispatch and `SCE_WINDOW`.
|
||
|
||
**No decoder code changed.** `decode.bin` is still 1,296 B at the same MD5.
|
||
|
||
**Next: THE RE-ENCODE BUNDLE, and it is now the only thing between this tree and
|
||
M2.** Four items sharing one re-measurement (ROADMAP P2): index-0 black,
|
||
`--spans all` as default, joint `lam`, and **sector-aligned records**, which
|
||
59.4 promoted from preference to precondition. After it, P4a's channel goes
|
||
behind `ring.i`'s mailbox and the 120-frame pixel-exact pass runs with the CPU
|
||
free — which is the shape of M2.
|
||
|
||
---
|
||
|
||
# Status & next-session handoff — end of session 26 (2026-08-24)
|
||
|
||
## Session 26: the player runs off the disc, and PIO costs 87 clocks a byte
|
||
|
||
**Green light first and last: `./tools/bench/check.sh` was ALL GREEN before any
|
||
of this and ALL GREEN after**, plus two new stages.
|
||
|
||
**ROADMAP P4b is DONE. P4a is the only item left before M2. FINDINGS 58.**
|
||
**Emulated** — MAME 0.277, `x68000 -exp1 cz6bs1 -ramsize 2M`. No real hardware
|
||
ran.
|
||
|
||
**1. THE SEAM IS CLOSED.** `src/player/xfer.i` sits behind `src/player/ring.i`'s
|
||
`XF_*` mailbox in place of `tools/bench/stream.lua`'s modelled transport. `XF_GO`
|
||
is answered by a real READ(10) to a real MB89352; `XF_ACK` is a word the 68000
|
||
bumps when the bytes have landed, not one a host synthesises from emulated time.
|
||
|
||
| | |
|
||
|---|---|
|
||
| 120 records fetched by the 68000 | one READ(10) each |
|
||
| 4,488,588 B into a 256 KB ring | **pixel-exact** over all 120 frames |
|
||
| a real mid-stream seek, pass 2 | **pixel-exact** after it too |
|
||
| 18 wraps, 14.7 KB mean hole | the SAME tiling as 49.4 and 55.4 |
|
||
|
||
The 18 wraps are the load-bearing assertion. A host producer made them in
|
||
session 18, `ring.i` made them against a modelled transport in session 23, and a
|
||
real SPC makes them now — **`ring.i` cannot tell which side of the mailbox
|
||
answered it**, which is what makes this a test of the transport rather than of a
|
||
new rig. 58.1.
|
||
|
||
**The change above the seam is two `bsr`s**, and the one in `ring_seek`'s
|
||
quiet-wait is not optional: with the transport *inside* the machine, that loop
|
||
is the only thing that can retire an outstanding request, so without it a seek
|
||
issued with a request in flight spins forever. A host transport retired it on
|
||
its own time. That is exactly the kind of difference the seam exists to hide,
|
||
and it is the one it could not.
|
||
|
||
**2. WHAT IT COSTS, AND THE NUMBER LEAVES THE EMULATOR.** `tools/bench/
|
||
xfer_cost.sh` runs the same 120 frames twice — same ring, same `stream.s`, same
|
||
`ring.i` placing every record — and changes only which side answers:
|
||
|
||
| | emulated | per frame |
|
||
|---|---|---|
|
||
| decode + `ring_poll` alone | 6.7737 s | 67.7% of a 12 fps frame |
|
||
| ...with the real transport | 45.9516 s | **459.5%** |
|
||
| **the transport alone** | **391,779,000 clk** | **391.8%** |
|
||
|
||
**87.28 clocks per delivered byte.** The keep loop in `scsi.i`, priced against
|
||
the **68000's own cycle table**, is 86 clocks, and the FIFO also carries the
|
||
dropped window bytes of 58.3 — **87.15 predicted, 87.28 measured, 0.2% apart.**
|
||
So this is the instruction stream and **not** MAME's SPC model: **the first
|
||
number this rig has produced that a real board would also pay.** The machine's
|
||
own V-DISP clock agrees from the other end — 120 frames in 560 slots of a 12 fps
|
||
clock is **2.57 fps**. 58.2.
|
||
|
||
**Against the ladder** (clocks charged to the CPU per delivered byte):
|
||
`W`=5 held is 22.4% of a frame, `W`=9 is 40.4%, `W`=19 — the IPL ROM's own disk
|
||
channel, 52.5 — is 85.3%, and **PIO is 391.8%**. **P4a is worth 4.6x the worst
|
||
DMA configuration in this tree and 17.5x the best**, where before this session
|
||
it was worth 9 against 19. `W` itself did not move by one clock.
|
||
|
||
**3. "UNDERRUNS: 0/120" IS VACUOUS HERE, and the rig now says so in place.** A
|
||
synchronous transport cannot underrun by construction — a frame cannot start
|
||
before its record has landed, because the decoder *is* the transport. The
|
||
counter that means something is `NO IDLE`: **119 of 120 frames found their slot
|
||
already open, worst overrun 441 whole ticks.** Same class of error as 49.7.2's
|
||
free-running ring passing at 48 KB: a rig configuration in which the failure
|
||
being tested for cannot occur. **The binding resource flipped too** — against a
|
||
modelled 488 KB/s pipe this container was rate-bound and the ring never filled;
|
||
here the ring **fills, 90 refusals for space**, while the decoder starves for
|
||
*time*.
|
||
|
||
**4. A RECORD IS NOT A SECTOR, and the cheapest fix is a re-encode.** 117 of 120
|
||
records start part way into a sector. Reading whole sectors into the ring
|
||
**corrupts the neighbouring records** — the block loop has no bounds check
|
||
(49.2) — so `scsi.i` reads the covering sectors and stores only the window.
|
||
That is free in PIO because the CPU is already touching every byte, **and it
|
||
stops being free the moment P4a succeeds**: a channel writes a contiguous run
|
||
and cannot drop bytes. `tools/analysis/26_sector_align.py` prices the three ways
|
||
out: windowed PIO **+1.34% wire, no DMA**; bounce buffer **+1.34% wire and
|
||
+5 clk/B of copy** (22.4% of the frame — the cost `aligned` was chosen over
|
||
`split` to avoid, arriving by a different door); **sector-aligned records
|
||
+0.43% wire and zero clocks.** The last wins on both axes and is a container
|
||
change. **ROADMAP now carries a four-item re-encode bundle** (index-0 black,
|
||
`--spans all`, joint `lam`, sector alignment) and **P4a should be attempted
|
||
against a sector-aligned container, not against this one.** 58.3.
|
||
|
||
**5. The rig refuses to be asked for a rate.** `DLX_XFER=scsi` rejects a
|
||
non-zero `--kbps` outright rather than ignoring it, and `stream.lua` suppresses
|
||
`CHANNEL IDLE`, `DEADLINE` and `REQUIRED PREFILL` rather than printing them as
|
||
zeros — a zero there reads as "the channel never stopped", which would be a
|
||
claim about a medium this tree has never timed.
|
||
|
||
**One rig bug found and fixed in the same sitting:** the host's `nsent` is now
|
||
rebased at every seek. `XS_NXFER` is cumulative, so on a second pass it was
|
||
already >= `nframes` and the slack sampling never fired — the second pass
|
||
reported a ceiling of 0 frames and a build time of **-1 ticks**, which is an
|
||
empty series printing as a result.
|
||
|
||
**New in the tree:** `src/player/xfer.i`, `tools/bench/mkvol.sh` (one copy of
|
||
the volume layout, shared by both SCSI rigs), `tools/bench/xfer_cost.sh`,
|
||
`tools/analysis/26_sector_align.py`, `DLX_XFER=scsi` and `DLX_SECONDS` in
|
||
`pace_run.sh`, and two `check.sh` stages. `src/player/scsi.i` gained
|
||
`sc_in_data` (the three-loop windowed receive) and `scsi_read_win`; `ring.i`
|
||
gained the two transport hooks; `stream.s` includes `scsi.i` and `xfer.i`.
|
||
|
||
**No decoder code changed.** `decode.bin` is still 1,296 B at the same MD5.
|
||
|
||
**Next: P4a, and it is now the whole of what is left before M2.** A DMAC
|
||
configuration that HOLDS THE BUS, shown by evidence that does not come from
|
||
watching `$EA0015` (57.3: with `exown` asserted, MAME cannot tell a CPU-driven
|
||
byte there from a DMAC-driven one). Do it against a sector-aligned container if
|
||
the re-encode bundle has been run by then, because 58.3 is otherwise a bounce
|
||
copy at +5 clk/B. **P2's remaining half** is unchanged and is now item 1 of
|
||
that bundle.
|
||
|
||
---
|
||
|
||
# Status & next-session handoff — end of session 25 (2026-08-24)
|
||
|
||
## Session 25: the 68000 reads the disc itself, and P4 was never blocked
|
||
|
||
**Green light first and last: `./tools/bench/check.sh` was ALL GREEN before any
|
||
of this and ALL GREEN after**, plus a new SCSI stage.
|
||
|
||
**ROADMAP P4 is HALF DONE. FINDINGS 57.** **Emulated** — MAME 0.277,
|
||
`x68000 -exp1 cz6bs1`. No real hardware ran.
|
||
|
||
**1. THE BLOCKER WAS A MISSING FILE, NOT A MISSING MODEL — and this tree already
|
||
knew.** Session 21's handoff recorded, re-checked rather than assumed, that
|
||
"MAME's `x68000` has no MB89352 path". It has one: `-exp1 cz6bs1` instantiates a
|
||
*Fujitsu MB89352 @ 5.00 MHz* next to the HD63450, and **FINDINGS 32.4 had read
|
||
that card's DMA glue in session 9**. What is actually absent is the 8 KB
|
||
`scsiexrom.bin`, which MAME requires to instantiate the card and which **the
|
||
player never executes** — driving the SPC registers directly has been the plan
|
||
since `docs/BENCHMARK.md` item 4 in session 2. A zero-filled placeholder on a
|
||
separate rompath starts the machine; the user's romset is untouched. **B3 is
|
||
untouched too**: it wants that ROM's bytes disassembled, and a blank one has
|
||
none. 57.1.
|
||
|
||
**2. THE RESULT.** `src/player/scsi.i` on the 68000, no IOCS and no host in the
|
||
transfer path, selects the target and issues READ(10):
|
||
|
||
| | |
|
||
|---|---|
|
||
| 4,096 B from LBA 0 | byte-for-byte against the host's image |
|
||
| 2,048 B from LBA 1000 | byte-for-byte |
|
||
|
||
The non-zero LBA is the half that matters: **a driver that emits a malformed LBA
|
||
field still passes LBA 0**, because zero is what a malformed field usually is.
|
||
The volume is `tmp/stream_disk.bin`, the same file the ring rig reads, so the two
|
||
rigs cannot differ in what they are reading. 57.4.
|
||
|
||
**3. The register map is measured, and it corrects MAME's own documentation.**
|
||
The probe walks `$EA0000..$EA003F` one address at a time and survives each bus
|
||
error. **60 of 64 answer**; the two holes are `$EA0007` and `$EA001F` — exactly
|
||
the TMOD and EXBF the **MB89352 omits and the MB87030 has**. MAME leaves HOLES
|
||
and does *not* "shift subsequent indices", which its own device summary claims,
|
||
and that is what keeps DREG at index 10 and at `$EA0015`. TEMP took `$A5` and
|
||
gave it back. 57.2.
|
||
|
||
**4. THE DATA REGISTER IS DMA-ONLY, AND A PIO WRITE VANISHES.** `x68k_scsiext.cpp`
|
||
glues `$EA0015` and nothing else: with `exown()` asserted and DRQ low a write is
|
||
**discarded silently** — no error bit, no status change, no interrupt. Quieting
|
||
all four DMAC channels does not change it. So every transfer runs the SPC in DMA
|
||
mode and the CPU moves the bytes through `$EA0015` itself, via `dma_w`/`dma_r`:
|
||
**the CPU standing in for the DMAC, through the DMAC's own door.**
|
||
|
||
**The cost to the argument, stated because it is easy to overclaim:** with
|
||
`exown` asserted at idle, **MAME cannot distinguish a CPU-driven byte at
|
||
`$EA0015` from a DMAC-driven one.** This rig shows the DATA PATH and cannot, by
|
||
itself, show that the HD63450 is driving it — which is exactly what ROADMAP
|
||
calls P4's first job. Whether a real CZ-6BS1 also refuses PIO here is **not
|
||
settled**; it is a property of MAME's model. 57.3.
|
||
|
||
**5. `W` DID NOT MOVE. Not by one clock.** MAME's device models are functional,
|
||
not transfer-timing accurate, and 42.5 reads its DMAC configured in wall-clock
|
||
attotimes rather than per-operand cycles. `W` is still the project's largest
|
||
open number and still wants a board.
|
||
|
||
**6. Five bugs, four of them silent** (57.5), recorded because the pattern is the
|
||
finding: a chained-`rol` transfer counter that loaded ZERO; a FIFO byte mistaken
|
||
for a bus byte; a fixed phase sequence where the bus decides the order; the
|
||
discarded PIO write; and an initiator that must drop ACK *and then* release the
|
||
bus. The last **only appeared once there were two reads** — one read passed
|
||
byte-exact and the second could not select. A player issues one command per
|
||
record, so that failure would have been universal in the ring and invisible in a
|
||
one-read demonstration.
|
||
|
||
**New in the tree:** `src/player/scsi.i` (the driver), `src/player/scsigate.s`
|
||
(the probe and gate front-end), `tools/bench/scsi.lua`, `tools/bench/scsi_run.sh`,
|
||
and a `check.sh` stage that builds the volume with `chdman`, gates the register
|
||
window at 60 of 64 and both reads byte-exact, and is skipped when `chdman` is
|
||
absent.
|
||
|
||
**No decoder code changed.** `decode.bin` is still 1,296 B at the same MD5.
|
||
|
||
**Next:** **P4's other half**, in two pieces. (a) The HD63450 driving the data
|
||
phase in a configuration that HOLDS THE BUS — and 57.3 is a warning about how it
|
||
will have to be shown, because this apparatus cannot tell a DMAC-driven byte
|
||
from a CPU-driven one at `$EA0015`. (b) `scsi.i` behind `ring.i`'s `XF_*`
|
||
mailbox in place of `stream.lua`'s modelled transport, gated on the same
|
||
pixel-exact 120 frames — the seam is already there, and `XF_ACK` becomes
|
||
something the 68000 bumps when a transfer lands rather than something a host
|
||
synthesises. **P2's remaining half** (reserve index 0 as black) is unchanged and
|
||
still bundled with the two other re-encode-class questions from 55.
|
||
|
||
---
|
||
|
||
# Status & next-session handoff — end of session 24 (2026-08-24)
|
||
|
||
## Session 24: the scene graph is in, and the worst gap between two decision points is zero
|
||
|
||
**Green light first and last: `./tools/bench/check.sh` was ALL GREEN before any
|
||
of this and ALL GREEN after**, plus a new import stage.
|
||
|
||
**ROADMAP G1 is DONE. FINDINGS 56.** G1 was scheduled early because it is a
|
||
*measurement input*, and it paid for that immediately.
|
||
|
||
**1. THE ANSWER: the worst gap is zero.** Over 612 distinct transitions into a
|
||
seek, taking the earliest instant each input window opens and chaining play
|
||
across sequences the disc plays through without seeking:
|
||
|
||
| | worst | p10 | median | p75 | p90 | best |
|
||
|---|---:|---:|---:|---:|---:|---:|
|
||
| seconds of play between two seeks | **0.000** | 0.950 | 3.473 | 5.800 | 9.548 | 82.497 |
|
||
|
||
**33 of the 612 (5.4%) are zero** — an input window that opens at t=0 of a clip
|
||
the disc seeked to, so two seeks can fall back to back with no play at all
|
||
(`flaming_ropes.enter_room -> fall_to_death`: press right on the first frame and
|
||
you die). **51.2's slack rule can be answered NO by the content, not by the
|
||
buffer**, and no amount of ring is a defence. 56.3.
|
||
|
||
**2. Most branch points arrive before the ring has refilled, at every rate this
|
||
tree has considered.** 51.3's climb against the game's own gaps, gate container:
|
||
|
||
| ring KB | pipe KB/s | ceiling | climb s | branch points under the climb |
|
||
|---:|---:|---:|---:|---:|
|
||
| 256 | 451.4 | 3 | 20.83 | 601/612 (98%) |
|
||
| 256 | 488.0 | 7 | 6.11 | **468/612 (76%)** |
|
||
| 256 | 513.2 | 7 | 3.81 | 370/612 (60%) |
|
||
| 256 | 600.0 | 7 | 1.66 | 129/612 (21%) |
|
||
| 512 | 488.0 | 11 | 9.60 | **551/612 (90%)** |
|
||
|
||
**A bigger ring makes this worse and now content says so too**: same rate, 76%
|
||
at 256 KB and 90% at 512 KB, because doubling the ceiling does not touch
|
||
`pipe - wire`. **The ring is not the lever; the surplus is.** 56.4.
|
||
|
||
**3. It does not break — it removes margin.** A branch on an empty ring costs
|
||
the prefill, not the climb: **149.7 ms (1.80 frame slots) at 488 KB/s**, 162.0 ms
|
||
if it is a scene change carrying the 6,164-byte header. So the finding is not
|
||
"this fails", it is **"this design runs permanently at minimum lookahead, and
|
||
the arcade content is what puts it there"**. The mechanical seek is still
|
||
unmodelled (B1) and is charged on top. 56.5.
|
||
|
||
**4. CORRECTION to FINDINGS 16: there is only one transcription.** 16 cleared
|
||
two permissively licensed sources and planned to diff them "to catch
|
||
transcription errors". The SNES project's own `data/events/README.md` says its
|
||
chapters are "derived from DirkSimple game data" — a second *copy*, not a second
|
||
transcription. The diff runs anyway and catches conversion errors: durations
|
||
agree 388/505 within one frame, branch structure 470/505, and of the 35
|
||
differences **16 are renames and 18 of the other 19 are the SNES conversion
|
||
dropping the arcade's diagonals**. Zero transcription discrepancies were found
|
||
and none could have been. 56.2, 56.6.
|
||
|
||
**5. Two constraints on the input layer, free with the import.** The arcade uses
|
||
**eight directions plus action and start** (diagonals are 21 of 906 windows —
|
||
droppable by a port that must, not by one aiming at the arcade), and the
|
||
**shortest input window is 98 ms** against a frame slot of 72.13 or 90.16 ms
|
||
(54.4). **Input cannot be polled on the frame tick**; `clock.i`'s V-DISP
|
||
interrupt already runs 8.6x faster at 0.1% of the budget. 56.7.
|
||
|
||
**6. The coupling to outside source is contained to one file (USER DECISION).**
|
||
`tools/import/scenegraph.py` is the only file in this tree that knows those
|
||
projects exist — their paths, table names, timing formulas, constants — and it
|
||
writes **`DLXSCENE1`**, our own schema, into gitignored `tmp/`.
|
||
`tools/analysis/25_scene_graph.py` reads only that. Nothing is vendored, nothing
|
||
outside-derived is committed, and the generated table carries its own `sources`
|
||
attribution block. The split was made after the measurement and changed no
|
||
number in it.
|
||
|
||
**New in the tree:** `tools/import/` (new directory, one file),
|
||
`tools/analysis/25_scene_graph.py`, and a `check.sh` stage that imports, gates
|
||
on 516 sequences / 906 input windows, and runs the analysis — skipped when there
|
||
is no checkout, like the px68k and IPL ROM stages.
|
||
|
||
**No 68000 code ran and none changed.** `decode.bin` is still 1,296 B at the
|
||
same MD5.
|
||
|
||
**Next:** **P4** (drive the MB89352, settle `W`) still decides the project and
|
||
still needs hardware or a MAME that models the SPC. What 56 changes about it:
|
||
the transport now has a *content* requirement as well as a rate one — it has to
|
||
survive a branch with an empty ring at zero notice, 5.4% of the time. **P2's
|
||
remaining half** (reserve index 0 as black) is unchanged and still bundled with
|
||
the two other re-encode-class questions from 55: the delivered-rate rate point
|
||
and 54.4's short slot. All three are still one re-encode plus one
|
||
re-measurement, and still want deciding together.
|
||
|
||
---
|
||
|
||
# Status & next-session handoff — end of session 23 (2026-08-24)
|
||
|
||
## Session 23: the 68000 fills its own ring, and the player's request loop turns out to cost more than the medium does
|
||
|
||
**Green light first and last: `./tools/bench/check.sh` was ALL GREEN before any
|
||
of this and ALL GREEN after**, 120/120 on both cores, no `TRUNCATED`, plus two
|
||
new ring stages.
|
||
|
||
**ROADMAP P5 is DONE. FINDINGS 55.** P5 was the last M2 item buildable in this
|
||
tree, and it is the third and last policy to move off the host: the loader went
|
||
in session 21, the frame clock in 22, and the ring producer now.
|
||
|
||
**1. The container had to change, and it is the first format change since
|
||
session 12.** `aligned` asks whether the next record fits before the end of the
|
||
ring — a question about a record's length asked **before it is fetched** — and
|
||
every reader in this tree answered it by WALKING the frame stream, which is
|
||
exactly what a player streaming off a disc cannot do. **DLX4** carries
|
||
`nframes` u16 record lengths in the scene header. The frame payloads are
|
||
**byte-identical** to the DLX3 encode, asserted record for record, so no
|
||
constant fitted to the gate container moves; the scene header goes 5,920 →
|
||
**6,164 B**. `dlx.py` refuses a container whose index disagrees with its own
|
||
walk. 55.1.
|
||
|
||
**2. `src/player/ring.i` reproduces the host producer exactly.** Same 18 wraps,
|
||
same 14.7 KB mean hole, pixel-exact — a third independent implementation of
|
||
`aligned` landing on the same tiling (55.2). The host now **audits** every
|
||
placement instead of making it.
|
||
|
||
**3. THE ONE THAT MOVES SOMETHING: the disc stands still whenever the player is
|
||
not asking.** A channel only moves bytes while it has a request and only the CPU
|
||
can issue one, so there is a gap between every pair of records that is a
|
||
property of the **player's loop, not the medium** — and no host-filled run could
|
||
see it, so no rate table in this tree contains it. At 488 KB/s in a 256 KB ring,
|
||
changing nothing but how many requests the player may have outstanding:
|
||
|
||
| queue | channel idle | underruns | slack ceiling | bound by |
|
||
|---:|---:|---:|---:|---|
|
||
| 1 request | **6.8%** | **59/120** | 2 | rate |
|
||
| 2 requests | **3.4%** | **0/120** | 5 | ring |
|
||
|
||
This container's surplus over the wire at 488 KB/s is 8.7% of the pipe, and a
|
||
one-deep request loop spends 6.8% of it on nothing — **most of the surplus
|
||
51.3's lookahead is accumulated out of**. A second slot costs no clocks and the
|
||
hardware has four channels (52.1). 55.3.
|
||
|
||
**4. Prefill is the weaker lever, and now it has a number.** Six records of
|
||
prefill — half a second of black at every scene start — still leaves 24
|
||
underruns at a one-deep queue; a second queue slot leaves none. **A prefill buys
|
||
a one-off cushion a rate-bound pipe spends immediately; a queued request buys
|
||
the rate back every frame.** The shipped policy is 2 records, and it is not 1
|
||
because 51.2 measured that *n* resident records buy *n-1* frame times. 55.4.
|
||
|
||
**5. The slack rule and a seek are in the player.** `ring_may_seek` is 51.2's
|
||
rule as arithmetic the player runs; `ring_seek` quiets the channel, takes record
|
||
*j*'s disc address out of the index and empties the ring. Rehearsed as a second
|
||
pass: 240 records, seek at 12.87 s, refill from empty, **0 underruns after it
|
||
and the last frame of the second pass pixel-exact**. The seek shows up as the
|
||
run's worst channel gap, **397.5 ms** — disc idle, not mechanical seek, which is
|
||
still unmodelled. 55.5.
|
||
|
||
**6. Three silent bugs and one instrument correction**, all written up in 55.7,
|
||
because every one of them produced wrong pixels or a desync rather than a fault:
|
||
the read cursor's wrap rule not matching the writer's; the free-space test
|
||
deciding a wrap before it knew whether the LIVE span was the wrapping one; the
|
||
request queue gated on completion instead of retirement (only reachable above
|
||
depth 1). The fourth is the rig's: MAME renders a screen line by line, so
|
||
snapshotting the frame in which the decoder finished captures a **tear** — it
|
||
looked exactly like a decoder bug and was not. The rig now settles one frame
|
||
before capturing. No previously reported result is affected.
|
||
|
||
**New in the tree:** `src/player/ring.i` (the producer, the prefill, the slack
|
||
rule and the seek); `tools/analysis/24_ring_owner.py` (an independent model of
|
||
all of it, sharing no code with the rig). `encode.py`/`dlx.py` gain DLX4 and its
|
||
index cross-check; `prep_stream.py` emits the index and checks it against the
|
||
disk image; `load.i` accepts DLX3 or DLX4. `stream.s` gains the ring hooks and a
|
||
polling wait loop; `stream.lua` becomes a **transport** built out of memory taps
|
||
(exact issue and completion times — see 55.7 for why nothing in a tap may touch
|
||
the memory space); `pace_run.sh` gains `DLX_RINGOWN`, `DLX_QDEPTH`,
|
||
`DLX_PREFILL_FR`, `DLX_ITER`. `check.sh` gains two stages: the machine-owned
|
||
ring, and a seek with the decode after it.
|
||
|
||
**`decode.s` and `frame.i` are unchanged**, `decode.bin` still 1,296 B at the
|
||
same MD5, and a host-filled run executes none of the new code — so every
|
||
FINDINGS 49/51 figure stands.
|
||
|
||
**Still open in P2:** unchanged — the encoder does not reserve a black entry
|
||
(23.4).
|
||
|
||
**Next:** M2 has no item left that this tree can build. **P4** (drive the
|
||
MB89352, settle `W`) still decides the project and still needs hardware or a
|
||
MAME that models the SPC; 55.3 sharpens what to ask of it, because the ladder
|
||
now has a second axis — the clocks stolen per byte AND the fraction of the pipe
|
||
the player's own loop gives away. **G1** (import the scene graph) is what would
|
||
let this tree ask the question 55.5 rehearsed but could not pose: what is the
|
||
worst gap between two real decision points, and does the refill climb survive
|
||
it. **P7** (boot from the volume) is buildable but empty until P4.
|
||
|
||
**A question 55.3 raises and does not answer:** the encoder is fitted to a pipe
|
||
that delivers continuously. It does not, and by 3-7% depending on the player's
|
||
queue. Whether the rate point should be set against the *delivered* rate rather
|
||
than the nominal one is the same class of change as the reserved black entry and
|
||
the short-slot question from 54.4 — a re-encode plus a re-measurement, and all
|
||
three should be decided together.
|
||
|
||
---
|
||
|
||
# Status & next-session handoff — end of session 22 (2026-08-24)
|
||
|
||
## Session 22: the frame clock moves onto the 68000, and the 12 fps frame turns out not to exist
|
||
|
||
**Green light first and last: `./tools/bench/check.sh` was ALL GREEN before any
|
||
of this and ALL GREEN after**, 120/120 on both cores, no `TRUNCATED`, plus two
|
||
new frame-clock stages.
|
||
|
||
**ROADMAP P3 is DONE. FINDINGS 54.** P3 was one of the two items session 21's
|
||
handoff called buildable here, and it is the one that touches every other number
|
||
in the project — because the tick is what the word "frame" in "% of a frame"
|
||
means.
|
||
|
||
**1. Neither of the two sources P3 named can do it, and the enumeration is the
|
||
finding.** The MFP's timer clock is 16 MHz/4 = 4 MHz, its prescalers are
|
||
`{4,10,16,50,64,100,200}` and its data register is 8 bits, so the **slowest tick
|
||
any single timer can make is 78.125 Hz** — 6.5x faster than a frame — and
|
||
**4e6/12 = 333,333.33 is not an integer**, so no setting reaches 12 Hz at all.
|
||
The raster has no whole divide near 12 either: 4 refreshes is 13.86 fps and 5 is
|
||
11.09. `tools/analysis/23_frame_clock.py` walks all 7x256 timer settings rather
|
||
than asserting it. 54.1, 54.2.
|
||
|
||
**2. The clock is the raster with a remainder, and it is exact by construction.**
|
||
`src/player/clock.i` takes the V-DISP falling edge on MFP GPIP4 — the start of
|
||
vertical blanking, which is when a player would present — and adds `fps*VTOTAL`
|
||
per edge to a 16-bit accumulator, emitting a tick at 31,500 and keeping the
|
||
remainder. Long-run rate is `fps*VTOTAL/VTOTAL` = **12.000000 fps exactly**.
|
||
Both constants are **read out of the CRTC at init**, so the clock is derived from
|
||
the registers that generate the raster it counts. Measured: **3,000 interrupts,
|
||
649 ticks where 649.1429 were due**. The gate is stated in ticks, not ppm — a
|
||
remainder is off by at most one tick over any window, so ppm would let a longer
|
||
window advertise a tighter clock for free. 54.2.
|
||
|
||
**3. It costs 181.35 clocks per V-DISP; 838 per frame; 0.1006% of the budget.**
|
||
The host cannot time this — its granularity is 17.64 ms and the interrupt is
|
||
microseconds — so **the 68000 times it itself**: a one-instruction loop over a
|
||
3,000-refresh window, run with the clock off and on, with the loop's own cost
|
||
calibrated rather than looked up. The calibration landed on **38.000002 clocks**
|
||
per iteration, which is both the check that licenses the subtraction and an
|
||
independent confirmation of `buscost.py`'s model. The 181.35 then decomposes
|
||
exactly, leaving **43.99 clocks for the interrupt exception** — the textbook 44,
|
||
measured rather than recalled. A timer-based clock would have cost 3.6x this at
|
||
an arbitrary phase against the scan. 54.3.
|
||
|
||
**4. THE ONE THAT MOVES SOMETHING: there is no 83.33 ms frame, and there never
|
||
was.** 12 fps on a 55.4577 Hz raster is 4.6215 refreshes, so a frame is shown for
|
||
**4 refreshes (72.13 ms) or 5 (90.16 ms)**, 37.9% of them short. The
|
||
833,333-clock budget every figure in this project is priced against is the
|
||
**mean** slot; the short one is **13.4% under it**, and 10 of the gate
|
||
container's 120 frames do not fit it.
|
||
|
||
**And the cadence was already in the tree, unnamed.** `stream.lua`'s tick is
|
||
`floor((t - t_rel) * fps)`, which looks uniform and is not: Lua only sees the
|
||
machine at frame boundaries, so its ticks land on refreshes and its gaps were
|
||
always 4 or 5. **Every host-paced result in FINDINGS 49 and 51 already carried
|
||
this cadence.** P3 did not introduce it; it moved who produces it onto the
|
||
machine and made it visible.
|
||
|
||
**It is not a dropped frame.** The pace gate says only "not before tick i", so an
|
||
overrun eats the next frame's idle and the clock recovers itself; the cost is one
|
||
frame presented a refresh late. On the gate container that is **4 frames of 120
|
||
with no idle left, against 1 for the nominal model** — and the expensive one is
|
||
**frame 0 at 111% of budget**, because the first frame of a scene has nothing to
|
||
SKIP against. So the cost lands **at a scene change**, next to 53.2's 18.96 ms of
|
||
loader and the seek. `stream.s` counts this itself now, and the rig's count
|
||
matches an offline model of the divider **exactly**: 4/120, first at frame 1, on
|
||
both tick sources. 54.4.
|
||
|
||
**5. An instrument correction the whole tree was reading.** `x68k_crtc.cpp
|
||
refresh_mode()` builds the frame period from `scr.max_x * scr.max_y` with
|
||
`scr.max_x = m_htotal - 8` — one character cell short, an inclusive bound used as
|
||
a count. **MAME's raster is fast by 368/360 = 2.2222%**: 56.6901 Hz measured
|
||
against the registers' 55.4577, agreeing to six digits with the arithmetic. So
|
||
every "1/55.46 s granularity" note in this tree was wrong and is **1/56.69 s**;
|
||
corrected in six files with the derivation put once in `crtc_mode.lua`. **No
|
||
conclusion changes and no 68000 cycle figure moves** — the CPU clock is unrelated
|
||
to the screen — but anything *paced* by the raster runs 2.22% fast under MAME, so
|
||
`clock.lua` reports both rates and de-skews, and the interrupt is priced against
|
||
the hardware refresh count. 54.5.
|
||
|
||
**New in the tree:** `src/player/clock.i` (the clock) and
|
||
`src/player/clockgate.s` (its measurement front-end); `tools/bench/clock.lua`,
|
||
`clock_cost.py`, `clock_run.sh` (the rig); `tools/analysis/23_frame_clock.py`
|
||
(the enumeration and the cadence pricing). `stream.s` gains `CLKON` and a
|
||
late-frame counter ahead of the wait loop; `stream.lua` gains `DLX_PACE=2` and
|
||
takes its deadlines from the ticks the machine actually emitted rather than from
|
||
a host model; `pace_run.sh` gains `DLX_PACE` selection, with the default tag left
|
||
alone so `pace_sweep.sh` still finds its logs. `check.sh` gains two stages: the
|
||
clock's own measurement, and 120 frames decoded pixel-exact with nothing outside
|
||
the machine deciding when a frame may start.
|
||
|
||
**`decode.s` and `frame.i` are unchanged.** `decode.bin` is still 1,296 B at the
|
||
same MD5. The pace gate's wait loop is byte-for-byte the one FINDINGS 51
|
||
measured, and the free-running path executes none of the new code, so every
|
||
FINDINGS 49 figure stands.
|
||
|
||
**Still open in P2:** unchanged — the encoder does not reserve a black entry
|
||
(23.4), so the letterbox still gets the palette's closest thing to black.
|
||
|
||
**Next:** P5 (per-record index, prefill policy, the accumulated-slack rule in the
|
||
player rather than the rig) is buildable here and is now the last M2 item that
|
||
is. G1 (import the scene graph) still needs fetching, and is still the one that
|
||
would let this tree ask what the worst gap between consecutive decision points
|
||
is. P4 still decides the project and still cannot be measured here.
|
||
|
||
**A question 54.4 raises and does not answer:** every rate-control and budget
|
||
figure in this project is fitted to an 833,333-clock frame, and 37.9% of frames
|
||
get 721,270. Whether the encoder should be fitted to the SHORT slot instead of
|
||
the mean is a re-encode plus a re-measurement — the same class of change as the
|
||
reserved black entry — and it should be decided with P5's numbers in hand, not
|
||
before.
|
||
|
||
---
|
||
|
||
# Status & next-session handoff — end of session 21 (2026-08-24)
|
||
|
||
## Session 21: the loader moves onto the 68000, and a scene change gets a price
|
||
|
||
**Green light first and last: `./tools/bench/check.sh` was ALL GREEN before any
|
||
of this and ALL GREEN after**, 120/120 on both cores, no `TRUNCATED`, plus a new
|
||
load-time stage.
|
||
|
||
**ROADMAP P1 is DONE and P2 is half done. FINDINGS 53.** Session 20's handoff
|
||
named P4 as the item that decides the project, and P4 is **blocked in this
|
||
tree** — re-checked, not assumed: there is still no `scsiexrom.bin` anywhere on
|
||
this machine (`~/mame/roms/x68000.zip` holds six files, four IPLs, a cgrom and
|
||
an sram), MAME's `x68000` has no MB89352 path, and `hd63450.cpp` decodes no
|
||
DTYP. **Nothing here can measure W.** P1+P2 was the M2 item that could be built
|
||
here, and it is the one that touches an already-measured number: it lands at a
|
||
scene change, where FINDINGS 51.3's refill climb is.
|
||
|
||
**1. The transforms are on the 68000 and they are byte-exact.** `src/player/
|
||
load.i` expands both codebooks to word-per-pixel form and packs the palette to
|
||
`GGGGGRRRRRBBBBBI` with the shared LSB chosen per entry, out of the RAW
|
||
container header. Gated **byte-for-byte against `tools/bench/dlxload.py`**,
|
||
which stays the reference — what changed is where the transforms run, not what
|
||
they produce. The palette half is read back **out of the palette registers at
|
||
`$E82000`**, so "the words reached the hardware" is part of what passes. Both
|
||
CPU cores emit the same 10,752 B. 53.1.
|
||
|
||
**2. A scene change costs 18.96 ms of 68000 time, 22.8% of one 12 fps frame.**
|
||
Boot costs 24.70 ms. Split: codebooks 92,609 clocks, palette entries 97,019,
|
||
scratch tables 52,919. Cross-checked on px68k's C68K, which agrees to 1.4% on
|
||
the scene-change figure. 53.2.
|
||
|
||
**3. The scratch tables describe the machine, not the scene.** They are the
|
||
CRTC's 6-to-8-bit rendering, its square, and the per-channel error difference —
|
||
so `pal_tables` is a separate entry point, built once at boot. **5.29 ms off
|
||
every scene change**, 22% of what a naive port of `dlxload.py` would have cost
|
||
per scene. 53.3.
|
||
|
||
**4. THE ONE THAT MOVES SOMETHING: the scene header is 5,920 bytes nothing has
|
||
ever counted.** Palette + CB1 + CB4, and it must arrive before frame 0 can be
|
||
decoded. It is not part of any frame record, so no rate table in this tree
|
||
includes it. `tools/analysis/22_scene_load.py` prices it across explicit rates.
|
||
The whole fixed cost of a scene change is about **a third of one frame slot** —
|
||
but its cost in FINDINGS 51.3's currency, accumulated slack, is divided by the
|
||
surplus `pipe - wire` and so is hypersensitive: **138 ms of extra climb at 488
|
||
KB/s, and 1.099 s at 451.4 KB/s**, for the same 5,920 bytes. 53.5.
|
||
|
||
**5. An instrument disagreement worth carrying forward.** The two CPU cores
|
||
agree to <3% on every stage except the table build, where they differ by 16.4%
|
||
— and that is the only code in this tree that multiplies. **px68k's C68K charges
|
||
a flat 50 clocks for `MULU`/`MULS`** regardless of operand (`c68kmacro.h:1869`),
|
||
where the 68000 charges 38+2n. That explains 4,608 of the 8,703 clock gap;
|
||
**4,095 clocks are not explained and are recorded as open.** Nothing in
|
||
`src/player/` outside these three instructions multiplies (checked), so no
|
||
figure in FINDINGS 24-52 is affected — but the second opinion this tree leans on
|
||
over-charges multiplies, and a future measurement containing one must not be
|
||
taken from it uncorrected. 53.4.
|
||
|
||
**6. Shipping pre-expanded codebooks was considered and refused.** It would
|
||
trade 9.26 ms of 68000 time for 5,120 more bytes in every scene header — 10.5 ms
|
||
of pipe at 488 KB/s, and bytes that lengthen the climb. Close to a wash in
|
||
milliseconds, not a wash in kind: the CPU is idle during a seek and the pipe is
|
||
what this project is short of. **Derived, not measured.** 53.6.
|
||
|
||
**New in the tree:** `src/player/load.i` (the transforms) and
|
||
`src/player/loadgate.s` (its front-end, 488 B); `tools/bench/prep_load.py`,
|
||
`load.lua`, `verify_load.py`, `load_run.sh` (the rig, both cores);
|
||
`tools/analysis/22_scene_load.py` (the pricing). `tools/bench/c68k/harness.c`
|
||
gains a `--loadraw` mode, which also makes its flag-watch address a variable
|
||
rather than a constant. `check.sh` gains a stage that gates byte-exactness on
|
||
both cores, and deliberately does **not** gate the cycle counts — MAME's clock
|
||
is 1/56.69 s (1/55.46 when that was written; 54.5) and a wall timing would make
|
||
the green light host-sensitive, the
|
||
same reason `blit.s` and `span.sh` are not in it.
|
||
|
||
**`decode.s` and `stream.s` are unchanged.** Nothing in the per-frame path was
|
||
touched; `decode.bin` is still 1,296 B at the same MD5.
|
||
|
||
**Still open in P2:** the encoder does not reserve a black entry (23.4), so the
|
||
letterbox still gets the palette's closest thing to black (index 255 here). That
|
||
half is encoder-side, it changes the container, and it moves every constant
|
||
fitted to the gate container — a re-encode plus a re-measurement, not an edit.
|
||
|
||
**Next:** P3 (a real frame clock from the MFP or VBL) and P5 (per-record index,
|
||
prefill policy, the accumulated-slack rule in the player rather than the rig)
|
||
are both buildable here. G1 (import the scene graph) is the one that would let
|
||
this tree ask a question it currently cannot: what is the worst gap between
|
||
consecutive decision points, and does the refill climb survive it. P4 still
|
||
decides the project and still cannot be measured here.
|
||
|
||
---
|
||
|
||
# Status & next-session handoff — end of session 20 (2026-08-24)
|
||
|
||
## Session 20: the DMAC configuration was in the IPL ROM the whole time
|
||
|
||
**Green light first and last: `./tools/bench/check.sh` was ALL GREEN before any
|
||
of this and ALL GREEN after**, 120/120 on both cores, no `TRUNCATED`, plus a new
|
||
ROM-evidence stage.
|
||
|
||
**ROADMAP's "do this first" item is DONE. FINDINGS 52.** It asked for the cheap
|
||
thing — put the ADPCM DMA stream on the bus and see what it does to the 86.7%
|
||
— and doing it needed a clocks-per-byte figure for the audio channel that the
|
||
tree did not have. `11_cpu_budget.py` was charging audio bytes the *disk's*
|
||
rate, 5 clk/B, described in its own help text as "single-address, bus held".
|
||
**Audio was being charged the favourable end of ROADMAP B3, an open question
|
||
worth 242 KB/s.**
|
||
|
||
**1. It never had to be a guess.** The X68000 IPL ROM programs all four HD63450
|
||
channels itself, and **the ROM is on this machine** — MAME boots the rig with
|
||
`-bios ipl10`. `tools/analysis/21_iplrom_dmac.py` reads the configuration out of
|
||
the image and decodes the MC68450 fields. Eight (address, expected bytes,
|
||
meaning) sites; a mismatch or an unknown ROM revision exits non-zero. Needs no
|
||
emulator, runs in milliseconds, and is now in `check.sh`. **Layer: a static read
|
||
of a shipping ROM image, not a running machine and not real hardware.**
|
||
|
||
**2. Audio is dual-address and cannot hold the bus: 16..19 clk/B.** ch3
|
||
`DCR = $80`, `OCR = $32`: explicitly addressed (9 clocks, read 4 + write 5),
|
||
cycle steal **without hold**, `REQG = 10` external request — the DMAC
|
||
arbitrates once per byte with no burst to amortise the 5..8 + 2 arbitration
|
||
over. The old debit was 3.2x..3.8x too small. 52.2.
|
||
|
||
**3. And on the bus it is still nothing.** 651.0 B/frame x 16..19 =
|
||
**1.25%..1.48% of the frame**, about 4% of what the decoder leaves. **P6's bus
|
||
risk does not materialise.** The unit worry was worth checking and nearly
|
||
right: 15.6 kHz = 8 MHz / 512 = 15,625 samples/s, two 4-bit samples to a byte =
|
||
**7,812.5 B/s exactly**; `AUDIO_KBPS = 7.8` is that in decimal kB and was being
|
||
multiplied by 1024. 2.4% high, now derived from the sample rate. 52.3/52.4.
|
||
|
||
**4. THE ONE THAT MOVES SOMETHING: the disk channel is programmed identically.**
|
||
ch1 (SASI) is `DCR = $80` too, and so is ch0 (FDC) — Sharp programs every
|
||
explicitly-addressed 8-bit device on this board the same way. By the same
|
||
arithmetic that is **16..19 clocks per delivered byte**, where FINDINGS 42.4
|
||
brackets W at **5..12** and 42.5 reports `W = 8` already missing 47/120 frames.
|
||
**The only worked example of a disk DMA configuration on this machine sits above
|
||
the entire bracket**, and at that price the design does not fit at any container
|
||
size. It is not `scsiexrom.bin`, so **B3 stays open** — what changed is that a
|
||
cheap configuration is now the thing that has to be *shown*. `W <= 12` is a
|
||
requirement on the player's DMAC programming, not a range the hardware hands us.
|
||
**This is now the largest open number in the project, ahead of the rate.** 52.5.
|
||
|
||
**5. An unforced cross-check fell out.** `15_bus_occupancy.py` now sweeps W over
|
||
the whole frame period. Its `W = 8` row lands at 105.7% of the frame — agreeing
|
||
with 42.5's "misses 47/120", from mode histograms and bus clocks respectively,
|
||
two models that share no code. 52.5.
|
||
|
||
**6. Audio outranks the disk at the arbiter.** CPR: FDC 0, ADPCM 1, SASI 2,
|
||
`_DMAMOVE` 3. When both want the bus, ADPCM is served first — an audio byte
|
||
never waits, a video byte does. Relevant to 51's smooth-rate delivery model.
|
||
52.6.
|
||
|
||
**New in the tree:** `tools/analysis/21_iplrom_dmac.py` (the ROM decoder/gate);
|
||
`15_bus_occupancy.py` gains a "THE OTHER TWO MASTERS" section pricing both DMA
|
||
consumers against the frame period; `buscost.py` gains the ADPCM constants and
|
||
the split read/write DMAC timings; `11_cpu_budget.py` charges audio and video
|
||
separately, with `--adpcm-clocks-per-byte`. Its stale "validated to within 1 pt"
|
||
line is corrected — the model reads 2..10 pt HIGH and by more as the frame gets
|
||
harder, which was already true at HEAD.
|
||
|
||
**Also: the README now shows the thing working.** `tools/bench/stream.lua`
|
||
gains `DLX_SNAP_EVERY=1` (needs `DLX_PACE`; off by default and on no path
|
||
`check.sh` takes), which snapshots every frame tick instead of once at the end,
|
||
and `tools/media/make_readme_media.py` turns those PNGs into `docs/img/`. The
|
||
stills and both clips are **MAME's own screen pixels**, not a re-render.
|
||
|
||
Building it turned up something worth recording. 116 of 119 captured frames are
|
||
pixel-exact against `tools/encoder/dlx.py`; **three are torn** — frame *n* on
|
||
top, frame *n-1* below the tear line — because MAME captured the screen while
|
||
the block loop was partway down it. `decode.s` writes straight to the displayed
|
||
page (one display path, 28.1), so **a real player tears the same way**; this is
|
||
the first time that consequence has been visible rather than argued. The script
|
||
ASSERTS the tear (every differing pixel must come from the previous frame) and
|
||
refuses to build otherwise, rather than trimming the frames and quietly
|
||
reporting "every frame I kept is exact".
|
||
|
||
Second correction the capture forced: the snapshot fires at the tick, *before*
|
||
frame *n* is decoded, so the obvious reading is that snapshot *n* holds frame
|
||
*n-1*. It does not — MAME renders the screen at the end of the machine frame, by
|
||
which time the 68000 has finished frame *n*. Checked, not reasoned about.
|
||
|
||
**`decode.s` and `stream.s` are unchanged.** Nothing in `src/player/` was
|
||
touched; `decode.bin` is still 1,296 B at the same MD5.
|
||
|
||
**Next:** P4 is the item that decides the project now, and 52.5 says so with a
|
||
number. Everything else in M2 (P1, P2, P3, P5, P7) is still buildable here.
|
||
|
||
---
|
||
|
||
# Status & next-session handoff — end of session 19 (2026-08-24)
|
||
|
||
## Session 19: the ring rig gets a frame clock, and a branch point costs 4.83 s of play
|
||
|
||
**Green light first and last: `./tools/bench/check.sh` was ALL GREEN before any
|
||
of this and ALL GREEN after**, 120/120 on both cores, no `TRUNCATED`, plus a new
|
||
paced-ring stage.
|
||
|
||
**Item 4 is DONE. FINDINGS 51.** Items 1, 2 and 3 were all still blocked on
|
||
hardware this tree does not have — a real board for 1 and 2, `scsiexrom.bin`
|
||
for 3 — and none of them moved. Item 4 was the one that could be built here.
|
||
|
||
`src/player/stream.s` had no frame clock, so it asked for record *i* the instant
|
||
it finished *i-1*, outran any finite pipe, and never let the ring back up. That
|
||
is why the 49.1 ring sweep passed at 48 KB. It now has `PACE`/`PACEON`
|
||
(`$18034`/`$18038`) and the producer supplies a 12 fps tick, so
|
||
`FR_HEAD-FR_TAIL` finally means what it is read to mean: **whole frames the
|
||
decoder could still draw with delivery stopped dead.** `PACEON=0` free-runs and
|
||
is what the wrap gate still uses, so every figure in FINDINGS 49 is unmoved.
|
||
|
||
**1. Small rings carry almost nothing.** Paced, on the gate container: 64 KB
|
||
holds **2 frames**, 96 KB holds 3, 128 KB holds 4–5, 256 KB holds 7–8, 512 KB
|
||
holds 14–15. All pixel-exact, which is exactly why the unpaced sweep passed
|
||
them. 51.1.
|
||
|
||
**2. Tolerance is `ceiling - 1`, measured by cutting the pipe.** With 7 records
|
||
resident, a 6-frame-time cut underruns nothing and a 7-frame-time cut underruns.
|
||
The last record pays for the pipe's restart (~0.9 frame times to place 36.5 KB).
|
||
**256 KB buys 500 ms of dead pipe, not 583.** 51.2.
|
||
|
||
**3. THE ONE THAT MOVES SOMETHING: slack is accumulated, not owned.** It is
|
||
built out of `pipe - wire` and a seek spends all of it. At 488 KB/s a 256 KB ring
|
||
takes **4.83 s of play** to reach its 7-frame ceiling from empty; 512 KB takes
|
||
8.42 s to reach 14. **A bigger ring raises the ceiling AND lengthens the climb.**
|
||
So a branch point does not ask "is the buffer big enough", it asks "has there
|
||
been enough play since the last one" — and Dragon's Lair's decision points are
|
||
seconds apart. First statement in this tree about back-to-back branches. 51.3.
|
||
|
||
**4. The rig now says which resource is binding.** Rate refusals and ring
|
||
refusals are counted separately: at 460 KB/s every ring from 192 KB to 512 KB is
|
||
**RATE-BOUND**, ceiling 4, and never fills in 120 frames — larger rings are dead
|
||
RAM in that scene. Clearing the arrival deadline (451.4 KB/s, 49.5) and being
|
||
able to absorb a seek are **different requirements**, and the gap is large. 51.4.
|
||
|
||
**5. Independent agreement, honestly bracketed.** `tools/analysis/20_seek_slack.py`
|
||
is the model rewritten in Python from record sizes, sharing no code with the Lua
|
||
producer. **35/35 of the rig's ceilings fall inside its bracket, 33/35 at the
|
||
top.** The bracket is one record wide because the pipe delivers ~one record per
|
||
slot, so the answer depends on sampling before or after that slot's delivery.
|
||
Both are reported rather than picking the one that matched — that would have
|
||
been fitting the model and calling it a cross-check. 51.5.
|
||
|
||
**6. Pacing exposed two producer defects, both invisible free-running.** The
|
||
`RD_PTR` cross-check was really testing how often `reap()` ran (it asserted per
|
||
retired record; `RD_PTR` names only `tail-1`), and `reap()` was skipped for the
|
||
whole duration of a cut, so the ring looked full through a seek. 51.6.
|
||
|
||
**New in the tree:** `tools/bench/pace_run.sh` (one paced run, pixel-verified),
|
||
`tools/bench/pace_sweep.sh` (ring x pipe grid), `tools/analysis/20_seek_slack.py`
|
||
(the independent sim). `check.sh` gains a paced stage that gates pixel-exactness,
|
||
zero underruns, and the 256 KB ceiling — the last one because a change in it is
|
||
a change in what a branch point can afford, and that should not slip past as a
|
||
log line.
|
||
|
||
**`decode.s` is unchanged and still asserted:** 1,296 bytes, same MD5. The pace
|
||
gate is in `stream.s` only (1,396 -> 1,418 B) and outside `src/player/frame.i`,
|
||
so no per-block or span constant moves.
|
||
|
||
---
|
||
|
||
# Session 18 and earlier
|
||
|
||
## Session 18: the streaming path exists, and the shipping rate does not fit the pipe
|
||
|
||
> **THE DELIVERY RATE HAS NO WORKING FIGURE — retired session 18 (USER
|
||
> DECISION).** Sections below written before session 18 name a "4 Mbps" pipe
|
||
> constant and score tables against it. **Read every one of those as history.**
|
||
> It was never a bus measurement: user-supplied, no provenance, 10% of SCSI-1's
|
||
> asynchronous rating (FINDINGS 42.1), and FINDINGS 49.5 caught the shipping
|
||
> candidate exceeding it while nothing in the tree was comparing the two.
|
||
>
|
||
> It is now gone as a default from every analysis tool and from
|
||
> `tools/bench/stream.lua` — `--bus` / `--kbps` / `DLX_STREAM_KBPS` are
|
||
> **required arguments** with no fallback, so no table can be scored against a
|
||
> rate its own output does not state. The one survivor is `GATE_SPAN_KBPS` in
|
||
> `tools/bench/check.sh`, which is a **container recipe**, not a delivery claim:
|
||
> the gate container was encoded with it and every per-block and span constant
|
||
> in FINDINGS 41/43/45/49 is fitted to that container, so changing it is a
|
||
> re-encode plus a re-measurement, not an edit.
|
||
>
|
||
> **What to use instead:** `tools/analysis/19_ring_stream.py` reports the
|
||
> **zero-prefill pipe** — the rate a medium must clear for a container to need no
|
||
> prefill. That is a requirement to measure a BlueSCSI against, not a constant to
|
||
> design on. For the session-14 candidate it is **513.2 KB/s**.
|
||
|
||
|
||
**Green light re-run first: `./tools/bench/check.sh` was ALL GREEN** before any
|
||
of this, 120/120 on both cores, no `TRUNCATED`.
|
||
|
||
**Items 1 and 2 were both blocked in this tree, and checked rather than
|
||
assumed.** Item 1 needs a real board. Item 2 needs the CZ-6BS1's
|
||
`scsiexrom.bin`: `~/mame/roms/x68000.zip` holds six files — `cgrom`, four IPLs,
|
||
`sram` — and there is no SCSI ext ROM anywhere on this machine. MAME's
|
||
`hd63450.cpp` decodes no DTYP, so the emulator cannot stand in (48.4). **USER
|
||
DECISION: build item 3.**
|
||
|
||
**Item 3 is DONE, and item 4 folded into it. FINDINGS 49.**
|
||
`src/player/stream.s` + `tools/bench/prep_stream.py` + `tools/bench/stream.lua`
|
||
decode the gate container **out of a bounded ring**, container in a host file:
|
||
|
||
| ring | machine | result |
|
||
|---|---|---|
|
||
| **256 KB** | **stock 2 MB** | **120/120, final frame pixel-exact** |
|
||
| 128 / 96 / 80 / 64 / 48 KB | stock 2 MB | 120/120, pixel-exact |
|
||
|
||
**The rig's RAM ceiling is gone.** 45 raised `RIG_RAM` to 6 MB because 5,261,814 B
|
||
of stream did not fit 2 MB. The streaming rig holds ~256 KB and reads the rest
|
||
from the host, so the machine it runs on is now **the machine the player
|
||
targets**.
|
||
|
||
**1. The constraint is CONTIGUITY, not byte count, and 09_buffer_sim could not
|
||
see it.** The block loop and span chain read with a monotonically increasing
|
||
`a0` and no bounds check anywhere, so the ring needs the whole next record
|
||
**resident AND contiguous**. 49.2.
|
||
|
||
**2. `aligned` beats `split` and it is not close.** Producer refuses to start a
|
||
record it cannot finish, leaves a hole: **9.1% of a 256 KB ring, zero clocks**.
|
||
Letting records wrap and shadowing the ring head costs **5.57% of the frame
|
||
budget, forever** — and the decoder is already at 91.1% at p90. (Both are
|
||
`s14_d5_all1500`'s; the gate container makes it 5.7% of the ring against 3.64%
|
||
of the budget. The costs are per container, not universal.) `aligned` also
|
||
needs a per-record index, which a branching laserdisc game needs anyway. 49.3.
|
||
|
||
**3. Two independent implementations agree exactly.** The Python sim (from record
|
||
sizes) and the Lua producer (driving a real 68000) both give **18 wraps, 14.7 KB
|
||
mean hole, 94.3% usable**. They share no code. 49.4.
|
||
|
||
**4. THE ONE THAT MOVES SOMETHING: the shipping candidate does not fit the pipe.**
|
||
`s14_d5_all1500` is **496.7 KB/s**; the pipe this tree has simulated against
|
||
since session 2 is **488 KB/s**. Those two numbers had never been put side by
|
||
side. It is **8.7 KB/s over on the MEAN — not a burst a ring absorbs.** The
|
||
deficit grows **523 KB per minute of play**; no ring size fixes it.
|
||
|
||
It was never caught because 42.1 retired the pipe as the binding resource and
|
||
built the rate controller to bind on clocks, with **no pipe term at all** — a
|
||
defensible decision. What was not decided is that FINDINGS 21's buffer sizing
|
||
and its "zero required prefill" would keep standing on a constant the design had
|
||
stopped enforcing. **That is what item 4 has been open since session 7 for.**
|
||
|
||
**The output is a requirement on the medium, not a verdict**, because 488 is
|
||
unmeasured folklore:
|
||
|
||
| container | wire | **zero-prefill pipe** |
|
||
|---|---:|---:|
|
||
| `s14_d5_all1500` (the candidate) | 496.7 KB/s | **513.2 KB/s** |
|
||
| `rc_fr_singe_scsi_span` (the gate) | 446.1 KB/s | **451.4 KB/s** |
|
||
|
||
**513.2 KB/s is now a hardware acceptance test** — 33% of SCSI-1 async, 10% of
|
||
sync. Very likely met; never shown to be met. 49.5.
|
||
|
||
**5. The rig's first version measured the wrong thing, and it is worth knowing
|
||
why.** `stream.s` has no frame clock, so it outruns any finite pipe and reported
|
||
**91 of 120 frames "stalled" at a pipe the same run shows is fast enough**. It
|
||
now records when each record becomes RESIDENT and checks that against a 12 fps
|
||
deadline: **1/120 late by 4.9 ms at 488, 0/120 at 520**. 49.6.
|
||
|
||
**`decode.s` is unchanged, and provably.** The block loop and span chain moved to
|
||
`src/player/frame.i` and the constants to `geom.i` so both front-ends assemble
|
||
from literally the same bytes — every cycle constant in FINDINGS 24/30/40/41 is
|
||
fitted to those bytes. `decode.s` still assembles to **1,296 bytes**, same MD5,
|
||
and `prep_dlx.py` still emits a byte-identical blob after the loader maths moved
|
||
to `tools/bench/dlxload.py`. **Both are now asserted in `check.sh`**, along with
|
||
the ring pass itself.
|
||
|
||
---
|
||
|
||
# Session 17 and earlier
|
||
|
||
## Session 17: the blanking question is not a tie, and it leans the wrong way
|
||
|
||
**Green light re-run first: `./tools/bench/check.sh` is ALL GREEN**, 120/120 on
|
||
both cores, no `TRUNCATED`. Nothing in session 16's uncommitted tree has drifted.
|
||
|
||
**FINDINGS 48. Session 16 filed 47.4 as "two emulators disagree, both readings
|
||
plausible." That framing was too generous to the outcome we want.**
|
||
|
||
1. **px68k is silent, not dissenting.** Read with `grep -a` — `gvram.c` is
|
||
EUC-JP, so a plain `grep` silently reports *nothing* — R20's high byte
|
||
(`CRTC_Regs[0x28]&8`) appears in **one file and six places, all address
|
||
decode**: three in `GVRAM_Read`, three in `GVRAM_Write`. **No px68k display
|
||
code reads the bit anywhere.** It does not model buffer mode as
|
||
non-blanking; it does not model the display side of buffer mode at all.
|
||
MAME's blanking is a deliberate commented claim made twice, and it models
|
||
bit 12 the same way for the text layer. **An assertion and a silence are not
|
||
a tie.**
|
||
|
||
2. **Sharp's own register table names the bit MAME's way.** R20 bit 11 G-MEM:
|
||
`%0 表示用 / %1 バッファ用` — "for display" / "for buffer" — and
|
||
`bit 10〜8 は無効`, i.e. the colour-mode field goes *invalid* in buffer mode.
|
||
COL is what the display side decodes a plane structure from. That is a
|
||
mechanism for blanking, not just a name for it. The counter-reading survives
|
||
only on the parenthetical (`G-VRAM が 65536 色表示時と同じ構造になる`), which
|
||
describes the structure and does not say the screen goes dark.
|
||
|
||
3. **The MAME branch has no survivable partial-blank.** The blank interval is
|
||
the paint, not the frame — but the measured blit is 53.6% of budget, packed
|
||
halves the words, so the picture is dark for ~27–54% of every frame at 12fps.
|
||
That is a 12 Hz strobe over the whole image, and the packed layout has **no
|
||
page left to flip to**, because both 256-colour pages carry picture. There is
|
||
no version where the packing is merely expensive.
|
||
|
||
**This is a shifted prior, not a result. It still needs the board.** But it
|
||
should be read before anyone spends a session building on 1.0 B/pixel.
|
||
|
||
**Item 2 lost its cheap method, and the loss is informative.** px68k **does not
|
||
emulate the MB89352 at all** — `x68k/scsi.c` is 81 lines that synthesise a fake
|
||
64-byte CZ-6BS1 boot ROM and trap IOCS `$F5` on the host (`SPCはエミュレートしない`,
|
||
its own header). So the "second emulator agrees" method that carried 46/47 was
|
||
never available for single- vs dual-address. What item 2 actually wants is the
|
||
**CZ-6BS1's `scsiexrom.bin`** (8 KB, CRC `7be488de`, not present here)
|
||
disassembled for the DCR it writes. **The MC68450 datasheet pins the field:
|
||
DTYP `00`/`01` are "Explicitly Addressed" (dual), `10`/`11` are "Implicitly
|
||
Addressed" (single), and 3.6.1.2 names the two protocols "dual address" and
|
||
"single address" in so many words.** 43.2's 5.0-vs-9.0 clocks/byte is two bits
|
||
in one byte the boot ROM writes at init. Not another emulator — and MAME's
|
||
`hd63450.cpp` decodes no DTYP at all, so it could never have answered.
|
||
FINDINGS 48.4.
|
||
|
||
**One thing was confirmed rather than doubted.** px68k's `kaiseki.txt` — the
|
||
author's own 2014 analysis notes, predating all of this — states the 256-colour
|
||
word interleave (`Page0の(0,0), Page1の(0,0), Page0の(1,0)...`) independently.
|
||
The packed layout's *premise* is solid from a third source; only its visibility
|
||
is in question. FINDINGS 48.5.
|
||
|
||
---
|
||
|
||
## Where session 16 left it
|
||
|
||
**The strongest test in the tree now covers the whole window, on both cores.**
|
||
FINDINGS 45. The pixel-exact gate was auditing 37 of 120 frames; it is now
|
||
**120/120 on MAME's 68000 and on px68k's C68K**, and `./tools/bench/check.sh`
|
||
is **ALL GREEN** with no `TRUNCATED` line.
|
||
|
||
Session 15 handed this over as "it needs the chunk-streaming rig, not a longer
|
||
pass." **That was wrong, and cheaply so — it needed neither.** The constraint
|
||
was the *rig's* memory, not the player's: `prep_dlx.py` preloads the whole
|
||
container into emulated RAM and `check.sh` ran the machine at `-ramsize 2M`, so
|
||
a 5,261,814 B stream was truncated to the prefix that fit. The gate runs under
|
||
`DLX_VERIFY_ONLY=1`, which drops the cost anchors entirely and asserts only
|
||
pixel-exactness, so the 2 MB was never load-bearing *here* — and preloading a
|
||
whole container is unlike the shipping player at any size, because the player
|
||
streams into a ring buffer. `RIG_RAM=6` in `check.sh` covers all 120 frames.
|
||
|
||
**The raise is licensed by measurement, not by convenience.** Run the full
|
||
timing pass at 2M and at 6M and the five synthetic anchors come out
|
||
**bit-identical** — 40,729 / 921,187 / 1,376,881 / 1,229,883 / 506,533 cycles —
|
||
despite sitting at different addresses in the two layouts. MAME's cycle model
|
||
does not depend on `-ramsize` over this range, so every per-block constant in
|
||
FINDINGS 24/30/41 is unmoved.
|
||
|
||
**And the 37-frame prefix was a biased sample.** It overstated the mean cost of
|
||
the window by 8.2%, and it never saw the quiet end:
|
||
|
||
| anchor | 37-frame prefix | full 120 |
|
||
|---|---:|---:|
|
||
| min non-SKIP | 25.1% of blocks, 61.9% of budget | **15.2%, 53.6%** |
|
||
| median | 45.4%, 79.7% | **41.1%, 81.1%** |
|
||
| p90 | 52.3%, 93.1% | **48.5%, 91.1%** |
|
||
| max non-SKIP | 62.5%, 91.8% | 62.5%, 91.8% (same frame) |
|
||
| C68K sequential-pass mean | 693,886 cyc, **83.3%** | **641,444 cyc, 77.0%** |
|
||
|
||
The direction is lucky rather than designed: the prefix was **pessimistic**, so
|
||
nothing downstream was flattered and no headroom claim was resting on the
|
||
missing frames.
|
||
|
||
**Item 2 is answered, and the answer is no.** FINDINGS 46. There is no packed
|
||
256-colour write path — no graphics mode on this machine puts two horizontally
|
||
adjacent pixels in one 16-bit word. The sub-word fields are **pages, not
|
||
pixels**: page 0 and page 1 are the two bytes of one word *at the same screen
|
||
coordinate* (px68k derives `line` identically for both, and the `$C80000` alias
|
||
lands on the adjacent byte), exactly as two independent documents describe. The
|
||
near-miss — scroll page 1 by one pixel and interleave — fails on byte count, not
|
||
on addressing, and 46.2 records why so it is not re-derived. **FINDINGS 44.7
|
||
stands unchanged**: the decoder-free player still needs 1,152 KB/s and 1.61 GB
|
||
and is still killed by the medium.
|
||
|
||
**Chasing it opened one more surface, and the same session closed it.** The 2:1
|
||
tax is a property of the *graphics* planes; the **text plane is 4bpp planar** —
|
||
four planes at `0x20000` stride, 1024x1024 bits each — so 0.5 bytes/pixel against
|
||
2.0, and an **uncompressed** 16-colour frame is 288.0 KB/s against the shipping
|
||
compressed 256-colour stream's 496.7. 42% cheaper on the wire, no decoder.
|
||
|
||
**It costs 5.84 dB and that kills it.** `tools/analysis/18_text_plane_16col.py`,
|
||
120 frames, generous to the 16-colour side (per-frame palettes, which the
|
||
256-colour path cannot use because its codebooks index a scene-wide palette):
|
||
|
||
| | mean PSNR |
|
||
|---|---:|
|
||
| 256 colours, scene palette (the tree's) | 31.33 |
|
||
| 256 colours, per-frame palette | 34.08 |
|
||
| 16 colours, scene palette | 23.17 |
|
||
| **16 colours, per-frame palette** | **25.49** |
|
||
|
||
Against the shipping container's **29.19 dB at 496.7 KB/s**, a 16-colour literal
|
||
is **25.49 dB at 288.0 KB/s — 3.70 dB worse for 58% of the bitrate.** The wire
|
||
saving does not pay for the colours. **The user's call was to drop the 16-colour
|
||
direction outright and the number agrees**, so it is closed, not parked.
|
||
FINDINGS 7's 256-colour claim now rests on a measurement. (One caveat recorded
|
||
in 46.3: a Floyd-Steinberg row came out bit-identical to the undithered one —
|
||
PIL ignored `dither=` under `MEDIANCUT` — so that row is void and excluded.)
|
||
|
||
**Then the user said to keep chasing it, and the "no" turned out to be wrong.**
|
||
FINDINGS 46.5. The masking is defeatable: **CRTC R20 bit 11** ("G-VRAM set to
|
||
buffer") bypasses the depth switch and writes the **full 16 bits** unmasked.
|
||
MAME's `gvram_w` and px68k's `GVRAM_Write` both implement it — px68k's comment
|
||
names the shipping game that used it, `65536モードのVRAMアクセス(Nemesis用)` —
|
||
so it is a mechanism, not an emulator quirk. **The 2:1 tax is a property of the
|
||
default write path, not of the memory.**
|
||
|
||
**And the two 256-colour pages have independent scroll**, which 46.1 also missed:
|
||
px68k's `Grp_DrawLine8(int page, int opaq)` indexes `GrphScrollX[page*8]`,
|
||
selects the byte within the word by page, and takes an opacity flag. 46.2
|
||
dismissed interleaving assuming a 1-pixel scroll; **scrolling by 128 makes the
|
||
used words contiguous**:
|
||
|
||
- write words 0..127 of each row, unmasked, full 16 bits
|
||
- page 0 unscrolled → screen columns 0..127
|
||
- page 1 X-scrolled +128, opaque, above → screen columns 128..255
|
||
- page 1's storage past 128 lands off the edge of the real 256x256 mode already
|
||
in use (FINDINGS 23)
|
||
|
||
**128 contiguous words carry 256 pixels: 1.0 B/pixel against 2.0** — 576 KB/s and
|
||
0.81 GB, exactly the halving 44.7 said would reopen the whole design. `movem`-shaped,
|
||
no stride, no transparency mask to maintain.
|
||
|
||
**This is a derivation, not a result** — see 46.6 for the three things untested.
|
||
But unlike items that need a board, **the tree can answer this one**: it is a
|
||
register setup and a snapshot, and `verify_frame256.py` already compares
|
||
pixel-exactly.
|
||
|
||
**The packed layout was then built and run on both emulators. It works.**
|
||
FINDINGS 47. 46.6 was a derivation; it is now a result:
|
||
|
||
| | result | palette ceiling |
|
||
|---|---|---:|
|
||
| MAME, `verify_frame256.py` | **256x192 pixel-exact, letterbox true black** | 40.83 dB |
|
||
| px68k, `verify_gvpack.py` | **256x192 index-exact** | 40.83 dB |
|
||
|
||
**Per-frame payload: 128 words/row x 192 rows = 49,152 bytes for 49,152 pixels —
|
||
1.0 B/pixel against 2.0.** The write path was measured directly: masked, writing
|
||
`AB5C` leaves page0=`5C` page1=`00` (the high byte destroyed); with R20 bit 11
|
||
set it leaves page0=`5C` page1=`AB`. That is the 2:1 tax and its off switch in
|
||
one table.
|
||
|
||
Four negative controls behave — bit 11 off, scroll removed, and the unpacked
|
||
control. `tools/bench/gvpack` links px68k's real `x68k/gvram.c`, the way
|
||
`tools/bench/c68k` links its CPU core.
|
||
|
||
**But the two emulators disagree twice, and the second one decides everything.**
|
||
|
||
1. *Priority register `0xE82500`.* At `0x0000` MAME hides page 1 (right half
|
||
black) while px68k puts page 0 on top transparently and renders correctly.
|
||
They agree at `0x0002`, which is what the layout uses — so the result stands,
|
||
but on a register they model differently.
|
||
2. **Does buffer mode BLANK the display?** `probe_bit11_blank.lua` is the
|
||
known-good 256-colour test with *one line added*. **MAME: the screen goes
|
||
fully black.** **px68k: it does not blank** — `Grp_DrawLine8` never reads that
|
||
bit. If MAME is right, the graphics layer is blanked for the whole time the
|
||
CPU or DMAC is painting, and a 12fps player shows black for whatever fraction
|
||
of each frame the paint takes. If px68k is right, the packing is free.
|
||
|
||
**That is now the cheapest high-value hardware fact outstanding** — cheaper than
|
||
single-vs-dual-address, because one real board plus a two-line probe settles it
|
||
and it moves more numbers.
|
||
|
||
**What it is worth if it goes px68k's way (DERIVED, not measured):** wire 1,152 →
|
||
**576 KB/s**, game 1.61 → **0.81 GB**, DMAC device→GVRAM 59.0% → **29.5%** of the
|
||
clock budget. And one reversal: 44.7 concluded "a CPU-painted full-frame literal
|
||
does not fit at any physically reachable price." Packed, one `movem` word carries
|
||
two pixels, so a pixel costs `(9.143 + 2c)/2 = 4.571 + c` — **9.57 clocks at c=5,
|
||
against a 16.95 budget. It fits, at c=5 and at c=9.** Withdrawn conditionally on
|
||
the blanking question.
|
||
|
||
Reproduce:
|
||
```
|
||
python3 tools/bench/prep_frame.py tmp/fr_00020 tmp/frame256p.bin 0 --pack-transparent
|
||
( cd tmp && SDL_VIDEODRIVER=dummy mame x68000 -bios ipl10 -ramsize 2M -video soft \
|
||
-window -sound none -nothrottle -plugins -seconds_to_run 8 \
|
||
-autoboot_script ../tools/bench/show_frame256_packed.lua \
|
||
-snapshot_directory ./snap_pack -snapview native )
|
||
python3 tools/bench/verify_frame256.py tmp/snap_pack/x68000/0000.png tmp/frame256p.bin
|
||
make -s -C tools/bench/gvpack && tools/bench/gvpack/gvpack tmp/frame256p.bin tmp/gvpack_px68k.raw
|
||
python3 tools/bench/gvpack/verify_gvpack.py tmp/gvpack_px68k.raw tmp/frame256p.bin
|
||
```
|
||
Not wired into `check.sh` yet: it is a capability probe, not something the
|
||
shipping player depends on. It should join the green light the moment anything
|
||
in the tree starts relying on the packed layout.
|
||
|
||
Green light: `./tools/bench/check.sh` **ALL GREEN**, before and after.
|
||
|
||
## NEXT SESSION, in order
|
||
|
||
**AS OF SESSION 32 the top of the list is this, and the rest of this section is
|
||
the session-19 list kept for its detail.** `docs/ROADMAP.md` is the current
|
||
shape; where the two disagree, ROADMAP is the one that was edited last.
|
||
|
||
1. **B2, and it is now the fork rather than a headroom question.**
|
||
`tools/bench/probe_bit11_blank.lua` is written and settles it in minutes on a
|
||
real board. FINDINGS 64.2 is why it moved to the top: **if buffer mode
|
||
blanks, the K3 player that now exists shows nothing at the container's own
|
||
wire and K4 is the player; if it does not blank, K3 is the player and K4's
|
||
27.3% paint is spent on nothing.** Each answer makes the other branch's work
|
||
wasted, so building either before the answer is the one thing to avoid.
|
||
|
||
2. **B1 gained a THIRD number: the DATA-PHASE BURST RATE** (64.2). Sustained
|
||
throughput decides whether record *i* arrives before slot *i*; the burst rate
|
||
decides how much of the slot the picture is on screen. A drive with a
|
||
read-ahead cache can pass one and fail the other. The acceptance test is
|
||
`record / (burst x slot)` = the dark fraction, and it is **1.0 at 582.0 KB/s**
|
||
and 0.273 — the same as K4's paint — only at **2,131 KB/s**.
|
||
|
||
3. **K4, and NOT before item 1.** ROADMAP has the shape. The one thing in it
|
||
that has never been run is the **overlap**: a channel filling buffer *i+1*
|
||
while the CPU paints buffer *i*, which is the first time the DMAC and the
|
||
68000 have had to want the bus at the same time for a whole scene.
|
||
|
||
4. **Diagnose MAME's transport ceiling, or stop quoting it.** ~535 KB/s a record
|
||
in both channel configurations, mechanism unknown, no MAME source on this
|
||
machine. It bounds nothing — the arithmetic in 64.2 is geometry — but it is
|
||
why `check.sh`'s packed stage is paced at half rate, and an undiagnosed
|
||
apparatus constant is exactly the shape of thing FINDINGS 42.1 caught.
|
||
|
||
---
|
||
|
||
|
||
**Item 4 is CLOSED (session 19, FINDINGS 51).** The list below is otherwise
|
||
unchanged, and items 1-3 are all still blocked on hardware this machine does not
|
||
have. What session 19 adds to item 1 is a SECOND number to measure the medium
|
||
against, and it is the harder one:
|
||
|
||
- **451.4 KB/s** is the zero-prefill pipe for the gate container (49.5) — enough
|
||
to arrive on time in a straight line.
|
||
- **Absorbing a seek is a different requirement.** At 460 KB/s every ring from
|
||
192 KB to 512 KB is rate-bound and never fills. The rate that makes a branch
|
||
point affordable is set by `pipe - wire` and how long the scene runs between
|
||
branches, not by the buffer — so measuring the BlueSCSI's *seek time* matters
|
||
as much as its throughput, and neither is known. 51.3/51.4.
|
||
|
||
Item 5 (`--spans all`) now has a second reason to be careful: it spends every
|
||
profitable byte, which raises `wire`, which shrinks `pipe - wire`, which
|
||
lengthens the climb back to a full ring after every branch. That interaction is
|
||
not priced.
|
||
|
||
## The list
|
||
|
||
|
||
0. **Green light first.** `./tools/bench/check.sh`. It is slower again: it now
|
||
also runs the 120-frame **ring-buffer** pass on a 2 MB machine, re-derives
|
||
`prep_dlx.py`'s blob, and asserts `decode.bin`'s MD5.
|
||
**Do not run two MAME jobs at once** — session 18 did, and two `decode.lua`
|
||
runs sharing one log file produced a 0-byte log and 15 wasted minutes. The
|
||
warning is already in this document; it is there because it keeps happening.
|
||
|
||
1. **Measure the BlueSCSI. It is now the only thing that unblocks the I/O side.**
|
||
FINDINGS 50 retired the pipe constant outright (USER DECISION): it is gone as
|
||
a default from every analysis tool and from `stream.lua`, which now all
|
||
REQUIRE an explicit rate. **There is no working delivery figure in this repo
|
||
any more, and that is deliberate.**
|
||
|
||
The consequence is that anything needing a delivery rate to mean something is
|
||
now waiting on a measurement — which is the honest state, and it is why this
|
||
is item 1. `19_ring_stream.py` gives the threshold to measure against:
|
||
**513.2 KB/s** for the session-14 candidate, **451.4 KB/s** for the gate
|
||
container. Zero-prefill pipes, per container, from real record sizes.
|
||
|
||
**Do not substitute a guess.** If the measurement is not available this
|
||
session, run the tools at several explicit rates and report the sensitivity,
|
||
rather than picking one and letting it become the next constant. That is
|
||
precisely how the last one survived five sessions after 42.1 called it
|
||
folklore.
|
||
|
||
If it clears 513.2, the 8.7 KB/s overrun of 49.5 is not one, and the rate
|
||
controller can go on binding purely on clocks *with that fact written down*.
|
||
If it does not, the two levers are giving the rate controller a pipe term
|
||
(it has none — it binds on `decode + c*bytes`) or re-encoding lower; the
|
||
first is more informative and neither should be priced before the
|
||
measurement exists.
|
||
|
||
2. **Settle whether buffer mode blanks the display.** Unchanged, and still
|
||
blocked on a real board. **Read FINDINGS 48 first — session 17 shifted the
|
||
prior toward MAME and toward "unusable."** `tools/bench/probe_bit11_blank.lua`
|
||
is written and settles it in minutes. Second, smaller, same sitting: the
|
||
priority register `0xE82500` at `0x0000` (47.3). **Do not pre-build on
|
||
1.0 B/pixel.**
|
||
|
||
3. **Settle single-address vs dual-address.** Unchanged, still the largest open
|
||
number: 242 KB/s and 0.69 dB. **Session 18 confirmed the artefact is not on
|
||
this machine**: `~/mame/roms/x68000.zip` has `cgrom`, four IPLs and `sram`,
|
||
and no SCSI ext ROM anywhere on the box. So this needs `scsiexrom.bin`
|
||
(8 KB, CRC `7be488de`) sourced, then its DMAC init disassembled for **DCR's
|
||
DTYP** field: `10`/`11` = single (5.0 clk/B), `00`/`01` = dual (9.0).
|
||
FINDINGS 48.4. A board or schematic is the fallback, not the first move.
|
||
|
||
4. **DONE, session 19 — FINDINGS 51.** ~~Pace the decoder, then measure
|
||
branch-point stall tolerance.~~ The rig has a frame clock, the ceilings are
|
||
measured at seven ring sizes and five pipe rates, the `ceiling - 1` tolerance
|
||
was falsified by cutting the pipe, and the result that matters is that slack
|
||
is accumulated: 4.83 s of play to refill 256 KB at 488 KB/s. The original
|
||
statement of the gap, kept because it is still the right description of what
|
||
was wrong: FINDINGS 49.7.2 is the sharpest gap the new rig leaves: because `stream.s` free-runs,
|
||
the ring never backs up, so the ring-size sweep tests **wrap correctness** at
|
||
each size and **not buffering**. 48 KB passes and is single-buffered — do not
|
||
read it as a viable player buffer. A paced decoder (MFP timer, or Lua gating)
|
||
turns the rig into the seek-tolerance test Dragon's Lair actually needs, and
|
||
it is the last piece of the delivery story.
|
||
|
||
5. **Make `--spans all` the default.** Still a recommendation rather than a
|
||
measurement (43.6.1), and 44.3 sharpens it: it is the only lever on the
|
||
encoder's byte side that changes anything. **Note it interacts with item 1** —
|
||
`all` spends every profitable byte, which is what put the candidate over the
|
||
pipe in the first place.
|
||
|
||
6. **Re-derive span selection jointly with `lam`** (39.3). The selector ranks on
|
||
net clocks; `lam` still prices bytes against a KB/s target rather than
|
||
against 5 clocks each — and if item 1 goes route (b), `lam` gets a real KB/s
|
||
ceiling back and this question changes shape.
|
||
|
||
## What session 15 settled
|
||
|
||
**Session 14's item 2 was aimed at a lever that is not loaded.** FINDINGS 44.
|
||
The bucket diagnosis of 43.5 is correct as a mechanism and worth one frame of
|
||
120 in practice, because at `--spans all` the byte side of the rate controller
|
||
does not bind at all:
|
||
|
||
| `--spans all`, c=5, the 120-frame `singe` window | KB/s | PSNR | over budget |
|
||
|---|---:|---:|---:|
|
||
| **shipped, `--bucket-frames 8`** | **496.7** | **29.19** | **1/120** |
|
||
| `--bucket-frames 32` | 496.7 | 29.19 | 1/120 |
|
||
| `--bucket-frames 1` — no banking at all | 498.0 | 29.19 | 1/120 |
|
||
| `--rc-floor open` — lam floor 1.0 | 503.7 | 29.21 | 1/120 |
|
||
|
||
A 32-frame bucket emits the **same container byte for byte** as an 8-frame one,
|
||
and `lam` never leaves its floor of 10.0 on any of 120 frames. The rate this
|
||
project reports is set by the span pass and by `mu`; `--kbps` and the bucket
|
||
are not the levers. Measure whether the lever is loaded before pulling it.
|
||
|
||
**Two real unit inconsistencies were found, fixed, and defaulted OFF on
|
||
measurement** — `--joint-decide` and `--joint-bucket` turn them on:
|
||
|
||
| c=5, `--spans all` | KB/s | PSNR | mean frame clocks | over |
|
||
|---|---:|---:|---:|---:|
|
||
| **shipped** | **496.7** | **29.19** | **740,049** | 1/120 |
|
||
| `--joint-decide` — the lagrangian sees the disk | 482.5 | 29.17 | 745,438 | 1/120 |
|
||
| `--joint-bucket` — the bucket may not lend clocks | 506.4 | 29.18 | 754,429 | 1/120 |
|
||
|
||
The first is the more interesting one even though it does nothing: priced per
|
||
delivered byte, a RAW block costs `400.4 + 16c` and a V4 block `448.2 + 4c`,
|
||
which **cross at c = 3.98 — just under 43.1's hard floor of 4.** So `mu`'s
|
||
V4 -> RAW escape hatch, FINDINGS 28.8 and session 8's `0c`, never existed on
|
||
real hardware: it spends 12 clocks of bus to save 47.8 of CPU. Correcting it
|
||
moves 0.3% of blocks and 0.02 dB.
|
||
|
||
Defaults are unchanged in effect: `s14_d5_all1500` re-encodes to the same MD5.
|
||
|
||
**An encode is 95% k-means, and it is now 2.7x faster, exactly.** 60.6 s -> 29.4 s
|
||
for a 120-frame window, from three fixes to `VQ.assign` (a materialised `C.T`, a
|
||
cache-sized chunk, and a thread pool over the chunk loop). Bit-identical labels,
|
||
so every container still hashes the same. FINDINGS 44.5.
|
||
|
||
**A decoder-free player fits the clocks and dies on the medium.** Streaming raw
|
||
preprocessed frames straight into video memory — DMAC device -> GVRAM, no
|
||
decoder in the loop — fits at c=5 with 41% of the frame to spare, but needs
|
||
1,152 KB/s sustained (~79% of SCSI-1 async, nothing left for audio or seeks) and
|
||
1.61 GB for the game, against the 0.70 GB the shipping container needs. The
|
||
cause is that 256-colour GVRAM is one pixel per word with the high byte
|
||
discarded, so half of every byte pulled off the disk is thrown away by the
|
||
hardware on arrival. FINDINGS 44.7. **Session 16 confirmed that layout is not
|
||
escapable on the graphics planes (FINDINGS 46) — so this stands.**
|
||
|
||
## What session 14 settled
|
||
|
||
**Session 13's binding unknown `W` was in the wrong unit, and correcting it
|
||
costs the project 41% of its rate and 1.85 dB.** FINDINGS 43.
|
||
|
||
`W` was charged per WORD of delivered stream. The MB89352 is an **8-bit** SPC,
|
||
so the DMAC pays per BYTE. Every I/O debit in the project since FINDINGS 5 has
|
||
been charged at half rate, and the favourable end of 39.7's 5..12 bracket was
|
||
never physically reachable: **a 68000 bus cycle is four clocks and the SPC
|
||
hands over one byte per cycle, so nothing can cost less than 4 clocks/byte.**
|
||
5 clocks/word is 2.5 clocks/byte — 62% of one bus cycle, and a 4 MB/s DMA on a
|
||
link that runs at 1.5 MB/s asynchronous.
|
||
|
||
The datasheet, per byte, device-to-memory, bus held (MC68450 Fig 4-25):
|
||
|
||
| how the DMAC is programmed | clocks/byte |
|
||
|---|---:|
|
||
| single address, D->M (sheet 2) | **5.0** |
|
||
| dual address, byte, no packing (sheet 4) | **9.0** |
|
||
| dual address, byte packed (sheet 3) | 16.5 |
|
||
|
||
**Nothing session 13 emitted fits at any of these.** `s13_280p1500`, the
|
||
0/120 candidate, is 44/120 at 5 clocks/byte and 120/120 at 9.
|
||
|
||
### What survives, and it had to be re-encoded rather than re-scored
|
||
|
||
The encoder was making its decisions in the same wrong units: `ratectl`
|
||
bisected `mu` against 833,333 cycles of *decode* with no disk term, and
|
||
`spans.select()` admitted a run only if it beat the blocks **on cycles alone**,
|
||
explicitly ignoring the bytes it added. Both now work in one currency — clocks —
|
||
and `--disk-clk-byte 0` re-emits session 13's container byte for byte, so the
|
||
change is the price and not the codec.
|
||
|
||
| | KB/s | PSNR | over budget | span px |
|
||
|---|---:|---:|---:|---:|
|
||
| s13's claim (2.5 clk/B) | 837.4 | 31.04 dB | 0/120 | 62.3% |
|
||
| **`s14_d5_all1500` — single address** | **496.7** | **29.19 dB** | **1/120** | 30.7% |
|
||
| `s14_d9_all1500` — dual address | 255.0 | 28.50 dB | **1/120** | 3.9% |
|
||
|
||
**The one frame over is frame 0 in both** — the intra frame, which 28.5/31
|
||
established is emitted late on purpose. Every other frame lands at or under
|
||
100.0%. `17_span_delivered.py` shares no code with the encoder's accounting and
|
||
reproduces both rows to the digit.
|
||
|
||
**The remaining lever is not ours to pull.** A span pixel costs `9.143 + 2c`
|
||
clocks and a V1 block pixel `18.74 + 0.25c`, so **a span beats a block only for
|
||
`c < 5.48`** — which falls between the two rows above. Single-address needs the
|
||
CZ-6BS1 to drive the SPC's DACK from the bus's `#EXACK` (pin B37, which the bus
|
||
does have). That is the *board's* wiring, not our code, and MAME models only the
|
||
dual-address row.
|
||
|
||
Green light: `./tools/bench/check.sh` **ALL GREEN**, re-run after the encoder
|
||
change — 0/120 rate-control drift under the joint cost, DLX3 round-trip exact.
|
||
|
||
### session 14's list
|
||
|
||
Items 0, 2 and 4 are answered by FINDINGS 44; the rest carry forward,
|
||
reordered, in the session-15 list at the top of this file.
|
||
|
||
|
||
0. **Green light first.** `./tools/bench/check.sh` (~6 min, Blu-ray mounted).
|
||
|
||
1. **Settle single-address vs dual-address.** It is worth 242 KB/s and 0.69 dB
|
||
and it is the largest open number in the project. It is a **hardware** fact:
|
||
the CZ-6BS1 schematic, or a real board, or a period service manual. MAME
|
||
cannot answer it (`x68k_scsiext.cpp` has no DACK path and no DRQ line from
|
||
the slot to the DMAC at all, though the real bus has `#EXREQ`/`#EXACK`), so
|
||
do not spend a session in the emulator on it. FINDINGS 43.2/43.3.
|
||
|
||
2. **Fix the rate controller's bucket.** Diagnosed, not fixed (43.5): it banks
|
||
bytes across 8 frames, and bytes are now clocks, which cannot be banked
|
||
because there is no double buffer. `corr(bytes, frame clocks) = 0.989`. The
|
||
per-frame ceiling should be joint and hard; the bucket should smooth only
|
||
what is left after the disk is paid.
|
||
|
||
3. **Get a 68000 to decode one of the new containers.** Unchanged from session
|
||
13's item 3 and still the strongest claim resting on the weakest coverage —
|
||
but *narrower* now: `s14_d5_all1500` is 30.7% span coverage against the
|
||
gate's 26.6%, where session 13's candidate was 62.3%. This is the cheapest
|
||
it will ever be to close. A 496.7 KB/s stream is ~10 frames in a 2 MB
|
||
machine, so it still wants a chunk-streaming rig, not a longer pass.
|
||
|
||
4. **Make `--spans all` the default** (my recommendation, not a measurement):
|
||
with a joint cost, `need` stops early and leaves profitable clock savings
|
||
unbought — 275.8 KB/s / 28.92 dB / 2 frames over, against `all`'s 496.7 /
|
||
29.19 / 1. FINDINGS 43.6.
|
||
|
||
5. **Re-run the ring-buffer simulation at ~497 KB/s** (29.5/30.7, open since
|
||
session 7). Cheaper than the 837 KB/s it was going to have to survive.
|
||
|
||
6. **Re-derive span selection jointly with `lam`** (39.3). Half-done: the
|
||
selector now ranks on net clocks, but `lam` still prices bytes against a
|
||
KB/s target rather than against 5 clocks each.
|
||
|
||
**Do not start by hand-optimising `decode.s`.** Five sessions running, and more
|
||
true than ever: at `c=5` the disk is 26.6% of the median frame and the decoder
|
||
is not what misses.
|
||
|
||
**Do not treat 1/120 as settled either.** FINDINGS 43.8 lists five things it
|
||
does not establish, and 43.9 is about how the last favourable table survived a
|
||
section written specifically to distrust it.
|
||
|
||
---
|
||
|
||
## What session 13 settled
|
||
|
||
Session 12 ended by asking the user to choose a rate point. The user instead
|
||
challenged the constant the question rested on — and it does not hold. The
|
||
488 KB/s figure was never a SCSI bus figure: **SCSI-1 is 1.5 MB/s asynchronous
|
||
and 5 MB/s synchronous** (ANSI X3.131-1986), so the design has been running
|
||
against 10% of the bus. Once bytes are that cheap the span pass simply
|
||
saturates, and it does so on its own at ~837 KB/s.
|
||
|
||
| container | KB/s | span px | PSNR | over @5 | @8 | @12 |
|
||
|---|---:|---:|---:|---:|---:|---:|
|
||
| the session-12 gate | 487.7 | 26.6% | 29.63 dB | 11/120 | 34/120 | 78/120 |
|
||
| **`s13_280p1500` — the candidate** | **837.4** | **62.3%** | **31.04 dB** | **0/120** | 21/120 | 63/120 |
|
||
| `s13_lossless` — a probe, not shippable | 934.6 | 70.4% | 31.19 dB | **0/120** | 47/120 | — |
|
||
|
||
**0/120 is the first time anything here has fitted 12fps on every frame**, and
|
||
it is simultaneously the best picture yet — 0.29 dB off a 31.33 dB palette
|
||
ceiling, i.e. off exact for this display. `mu` is never spent at these rates, so
|
||
FINDINGS 31's 0.62 dB CPU-fit cost is refunded. **Quality and deadline have
|
||
stopped competing**, which was true of no earlier session.
|
||
|
||
The mechanism: a v7 span pixel costs 2 wire bytes and **9.143 clocks**, a RAW
|
||
block pixel costs 1 byte and **25.03**. Spans win for any `W < 31.7`. The budget
|
||
is 16.95 clocks/pixel, so **a full-frame literal fits iff `W <= ~6.5`.**
|
||
|
||
> **Withdrawn, session 15.** That threshold is per WORD: 6.5 clocks/word is
|
||
> 3.25 clocks/byte, under the 4-clock floor, so it was never reachable. In byte
|
||
> units the breakeven is **c = 3.906** against a floor of 4.0 — a CPU-painted
|
||
> full-frame literal misses at every real price, by 1.1% at the floor itself.
|
||
> FINDINGS 44.7.
|
||
|
||
### Which makes `W` the whole result
|
||
|
||
| `W` clocks/word | 934.6 KB/s stream | over budget |
|
||
|---:|---|---:|
|
||
| 5 | median 83.0%, worst 91.0% | **0/120** |
|
||
| 6 | median 87.6%, worst 97.0% | **0/120** |
|
||
| 7 | median 92.2%, worst 103.0% | 21/120 |
|
||
| 8 | median 96.8%, worst 109.0% | 47/120 |
|
||
|
||
Moving `W` across FINDINGS 39.7's datasheet bracket costs more frames than
|
||
moving the rate from 280 to 935 KB/s wins. It has displaced the 4 Mbps figure as
|
||
the most load-bearing unmeasured number in the project.
|
||
|
||
**Two things are known about `W` and they point the good way.** MAME's CZ-6BS1
|
||
glue is cycle-steal **with the bus held** — `x68k_scsiext.cpp:110-136` gates
|
||
`#DTACK` on `DRQ` while the DMAC's OWN is asserted — which is the 5 clk/word end
|
||
of the bracket, not the ~12 arbitrated end. And `W` decomposes into `5 clocks of
|
||
MC68450 + however long the drive makes the DMAC wait`; the deployment target has
|
||
been SD-backed SCSI since session 2 (FINDINGS 21.2), which collapses the wait
|
||
term. FINDINGS 42.5, 42.6.
|
||
|
||
**Also corrected: "never `x68ksupr`" was never a hardware claim.** Internal and
|
||
external are the same MB89352 and neither needs a driver (IOCS is in ROM); the
|
||
difference is that MAME has not implemented the internal DMA glue
|
||
(`x68k.cpp:1176`, `// TODO`). The external board **is** the way to test the DMA
|
||
path in MAME, and that is what item 1 below does.
|
||
|
||
Green light: `./tools/bench/check.sh` **ALL GREEN** at the start of session 13,
|
||
unchanged since. Nothing in session 13 touched the encoder, the decoder or the
|
||
container — this session moved numbers and documents only.
|
||
|
||
**Session 13's list is superseded** — its items 1 and 2 are done and its
|
||
answer did not survive them (FINDINGS 43); the rest are carried forward,
|
||
reordered, in the session-14 list at the top of this file.
|
||
|
||
**And its headline is withdrawn.** The 0/120 above, the 31.04 dB, the
|
||
saturation at 837 KB/s and "quality and deadline have stopped competing"
|
||
were all scored with the disk debited per WORD to a byte-wide port. The
|
||
reasoning in this section about the SCSI bus rating (42.1) stands; every
|
||
number downstream of it was charged at half rate. Read FINDINGS 43.
|
||
|
||
|
||
---
|
||
|
||
## What session 12 settled
|
||
|
||
Session 12 built v7 into the player. **`src/player/decode.s` paints v7 literal
|
||
spans, and it is pixel-exact under both CPU cores** over a container where every
|
||
frame carries 128-216 spans covering up to 38% of the picture. FINDINGS 41.
|
||
|
||
The container is **DLX3**: a span section between the mode header and the block
|
||
payload, `{u32 GVRAM address, u16 coarse disp}` per span with the fine
|
||
displacement mid-stream. `tools/analysis/16_span_roundtrip.py` gates it and is
|
||
in `check.sh`.
|
||
|
||
**The measured cost transfers.** Two synthetic all-SPAN anchors price v7 inside
|
||
`decode.s` at **151.2 and 225.6 clocks per 4x4 block**, against FINDINGS 40's
|
||
table of 151 and 226 — 0.2% on both emulators.
|
||
|
||
### The two things that were not on the list
|
||
|
||
**1. There are TWO byte budgets, and FINDINGS 40's 18/120 was scored at the
|
||
wrong one.** The `scsi` profile is 280 KB/s; `14_dmac_chain.py` scores spans
|
||
against the 488 KB/s PIPE, which is 40,977 B/frame against 23,228. At the
|
||
profile rate the lam search has already spent the allowance and spans fire on 5
|
||
frames of 120. The profile is a chosen quality rate point; the pipe is hardware.
|
||
`--kbps` and `--span-kbps` are now separate, and spans run before `mu` because a
|
||
span pays in bytes and `mu` pays in picture. FINDINGS 41.2.
|
||
|
||
| 120-frame `scsi` window | KB/s | over budget | PSNR |
|
||
|---|---:|---:|---:|
|
||
| no spans | 278.3 | 86/120 | 29.27 dB |
|
||
| spans, profile budget only | 280.0 | 77/120 | 29.23 dB |
|
||
| **spans on the 488 KB/s pipe** | 487.7 | **34/120** | **29.63 dB** |
|
||
|
||
**2. `C_SKIP_MIXED` was never measured, and it was 18% low — 45.0, now 55.0.**
|
||
It is the one constant in the cost table that came from a derivation, because
|
||
the synthetic frame that would measure it cannot exist (a byte needs a coded
|
||
block for its SKIP to be mixed). Four new bracketing anchors measure it on both
|
||
emulators, and with it corrected the model predicts a real spanned decode to
|
||
**-0.06% mean / 0.09% worst**, against -2.99% / 4.30% before. It matters here
|
||
because **a span marks its run SKIP**, so mixed SKIPs are the dominant
|
||
population in exactly the frames spans are judged on. FINDINGS 41.5.
|
||
|
||
**And the metric everything has been quoted in is unstable.** 34/120 delivered
|
||
against 14's simulated 18/120 is a 1.4% difference in mean frame cost. 55 of 120
|
||
frames sit within 5% of the deadline because the rate controller aims there, so
|
||
a 1% cost shift moves 22 frames. Quote the distribution, not the count.
|
||
FINDINGS 41.6.
|
||
|
||
Green light: `./tools/bench/check.sh` **ALL GREEN**, now gating on a span-heavy
|
||
DLX3 container.
|
||
|
||
---
|
||
|
||
## What session 11 settled
|
||
|
||
Session 11 measured v7 in `blit.s` and left it there; session 12 built it into
|
||
the player. Items 0 and 1 of session 11's list are done (FINDINGS 40, 41) and
|
||
the rest are carried forward in the list above.
|
||
|
||
## What session 10 settled
|
||
|
||
Session 10 cross-checked the whole cycle model against a second emulator, then
|
||
found that the model was denominated in the wrong currency.
|
||
|
||
**FINDINGS 38 is the headline: the project is BUS-bound, not CPU-bound.** Nothing
|
||
since FINDINGS 24 had counted the 68000's local memory bus — one 4-clock cycle at
|
||
a time, carrying instruction prefetch as well as data. Measured, the decoder
|
||
occupies **86.7%** of it, and **prefetch is 62% of that traffic**. Scoring the
|
||
`scsi` window with the bus as the shared resource, **52 of 53 missed frames are
|
||
bus-limited and one is CPU-limited**. Every optimisation since 24 has been aimed
|
||
at the budget that is not binding.
|
||
|
||
The measurement is two sources checking each other: `c68k_bench` counts every bus
|
||
callback exactly, and a static walk of `decode.lst` adds the prefetch no emulator
|
||
here can report. The walk reproduces the measured data half to **0.04%**, which
|
||
is what licenses its prefetch half.
|
||
|
||
**The DMAC array-chain LOSES, on the datasheet.** My first pass derived its cost
|
||
from bus arithmetic — a read cycle plus a write cycle, 8 clocks a pixel — and
|
||
scored it at 1/120 frames over budget against v6's 10/120. Then I read the
|
||
MC68450 manual (Motorola Jul 1989, bitsavers). **Fig 4-25 sheet 4: a
|
||
dual-address word between two 16-bit ports is 9 clocks, because note 2 gives the
|
||
DMAC 4-clock reads and 5-clock WRITES.** The 68000 writes in 4.
|
||
|
||
| per pixel | clocks | source |
|
||
|---|---:|---|
|
||
| DMAC dual-address word | **9.000** | MC68450 Fig 4-25 sheet 4 |
|
||
| v6 `movem` chain | **9.152** | MEASURED, FINDINGS 30 |
|
||
|
||
Scored additively (see below), against the same mode maps:
|
||
|
||
| | frames over budget |
|
||
|---|---:|
|
||
| today | 84/120 |
|
||
| v6 span as built | 55/120 |
|
||
| **v6 with a finer chain tail — software only** | **18/120** |
|
||
| DMAC chain | 12/120 |
|
||
|
||
**86% of the DMAC's advantage is v6's 24-pixel padding quantum**, which is a
|
||
property of v6's unrolled `movem` chain and fixable in software: add a second
|
||
chain of 2-register units for the tail and the padding drops from 23 pixels to
|
||
3, at no per-span cost. **Recommendation (mine): fix the quantum, drop the
|
||
DMAC.** Six frames of 120 does not buy a reserved channel, a two-region
|
||
container layout, and a timing dependency neither emulator here can verify.
|
||
FINDINGS 39.
|
||
|
||
The container work is not wasted either way: v6's record and an HD63450 chaining
|
||
entry are both 6 bytes, so **the chain array is the span table** and nothing has
|
||
to change if a hardware measurement later moves the DMAC's number. FINDINGS 39.1.
|
||
|
||
**I also got FINDINGS 35 wrong before the datasheet corrected me.** I argued its
|
||
flat CPU debit was too pessimistic and rescored the window at 53/120 using
|
||
`max(CPU, bus)`. A 68000 has no cache and a two-word prefetch queue, so it
|
||
stalls the moment another master takes the bus, and the MC68450 hands the bus
|
||
over in *slabs* under limited-rate auto-request rather than interleaving per
|
||
operand. **DMA is additive. 84/120 stands.** What 86.7% bus occupancy really
|
||
says is that there is almost no room to overlap anything on this machine.
|
||
FINDINGS 38.3.
|
||
|
||
**And the datasheet settled STATUS's own "most load-bearing unmeasured number".**
|
||
FINDINGS 5's 8 clocks/word for the SCSI DMA: Fig 4-25 sheet 3 gives
|
||
single-address write 5 clocks, so it is **5 clk/word with the bus held and ~12
|
||
if the DMAC arbitrates per word**. 8 is the midpoint of a bracket the datasheet
|
||
supports. Which end applies is a player design decision — how the MB89352 drives
|
||
REQ, and whether cycle-steal-with-hold is used — worth 7 clocks a word on a
|
||
480 KB/s stream. FINDINGS 39.7.
|
||
|
||
**The decoder is now pixel-exact under two independent CPU cores.**
|
||
`tools/bench/c68k/` links px68k's C68K core into a headless harness and decodes
|
||
the same container to the same pixels. Cycle-table error against MAME is bounded
|
||
at **3.3%**, and it runs against us. Two incidental results worth keeping: MAME
|
||
0.277's `M68000` is the **microcode core**, not Musashi (`m68000.lst` +
|
||
`m68000gen.py`), so this is two structurally different timing models agreeing
|
||
rather than two tables; and FINDINGS 28.8's "V4 costs more than RAW" reproduces
|
||
independently. FINDINGS 37.
|
||
|
||
---
|
||
|
||
## What session 10 put on the list
|
||
|
||
Kept for the reasoning, not the priorities: items 0 and 1 are done (FINDINGS 40)
|
||
and the rest are carried forward, reordered, in the session-11 list at the top.
|
||
Everything below this point is from session 9 or earlier and still stands unless
|
||
a later section above says otherwise.
|
||
|
||
### session 10's list
|
||
|
||
0. **Green light first.** `./tools/bench/check.sh` (~5 min, Blu-ray mounted).
|
||
Verified green at end of session 9. The gate container is now
|
||
`tmp/rc_fr_singe_scsi_cpufit.dlx`, and it decodes **80 of 120 frames**, not
|
||
120: the rig loads the whole stream into a 2 MB machine and a `scsi` window
|
||
is 2.84 MB, which overran the top of RAM by 940 KB and produced a failure
|
||
that looked exactly like a decoder bug. `prep_dlx.py` now truncates to what
|
||
fits and says so; `verify_decode.py --nframes` replays the same prefix; and
|
||
the stage fails loudly if the sequential pass did not complete. FINDINGS 36.
|
||
|
||
At 278.6 KB/s a 2 MB machine holds ~6.7 s of stream, so **the strongest test
|
||
in the tree can only ever audit a prefix of a window.** Getting the coverage
|
||
back means gating on more than one window, not one longer pass.
|
||
|
||
1. **Measure the finer chain tail with `span.sh`.** It is the largest single win
|
||
on the table — 55/120 to 18/120 — it is software only, and the tool that
|
||
measures it already exists: `blit.s` gains a v7 with a 2-register tail chain,
|
||
`prep_spans.py` generates the streams, `span.sh` times all of it in ~25 s and
|
||
asserts every config drew a pixel-exact frame. The 18/120 figure is DERIVED
|
||
from a conservative 56-clocks-per-4-pixel-unit model and should not be
|
||
believed until it is measured. FINDINGS 39.4.
|
||
|
||
2. **Make sure the player actually gets DMA.** DMA-vs-PIO is a property of our
|
||
code, not the board: the CZ-6BS1's DMA path is real and modelled
|
||
(FINDINGS 32.4), but if the player reads through IOCS and IOCS does PIO we
|
||
get the 120/120 row. `docs/BENCHMARK.md` item 4 (drive the MB89352 directly)
|
||
is no longer an optimisation. Benchmark `x68000 -exp1 cz6bs1`, **never
|
||
`x68ksupr`** — MAME's internal SCSI has no DMA glue at all (`// TODO:
|
||
duplicate DMA glue from CZ-6BS1`) and would measure a PIO fallback the real
|
||
machine does not have.
|
||
|
||
3. **Re-decide the framerate, now that it is the lever that pays for the disk.**
|
||
10 fps absorbs the DMA steal on current estimates. This was item 5 and a
|
||
quality question; it is now arithmetic. It is still the user's call.
|
||
|
||
4. **Then spans, on the CPU.** The format is fully specified by measurement
|
||
(FINDINGS 30.2, costs 30.5, scene-cut arithmetic 30.6) and `decode.s` does
|
||
not implement them yet. Build the v7 tail from item 1 into it. The DMAC
|
||
variant is costed and loses (39.5); the container is identical either way
|
||
(39.1), so nothing is foreclosed.
|
||
|
||
5. **Re-run the ring-buffer simulation at the surviving rate** and confirm the
|
||
488 KB/s figure's provenance (FINDINGS 29.5/30.7, still open).
|
||
|
||
**Do not start by hand-optimising `decode.s`.** The hand-derived timings agree
|
||
with the measurements to 0.5% on V1 and 1% on RAW (FINDINGS 28.4), and FINDINGS
|
||
34 confirms the model on a second container. The inner loop is close to what the
|
||
instruction set allows; the cycles to be won are in the budget, not the loop.
|
||
|
||
**Three of this session's near-misses had the same shape: an unobservable run
|
||
almost produced a false finding.** The 23-minute "hang" (FINDINGS 34.1), the
|
||
RAM overrun that looked like a 4x-slow decoder (FINDINGS 36), and the truncated
|
||
gate that reported 49,005 differing pixels. In each case the instrument was
|
||
broken, not the thing being measured. **Always `stdbuf -oL` a MAME job that
|
||
prints progress.** A file is
|
||
block-buffered too, so a long run is unobservable until it exits — and an
|
||
unobservable run that is merely finishing looks exactly like one that is wedged.
|
||
Session 8 lost this measurement to that; session 9 lost 23 minutes to it before
|
||
spending 25 seconds getting the answer with `stdbuf`. FINDINGS 34.1.
|
||
|
||
---
|
||
|
||
## What session 8 settled
|
||
|
||
0. **The mode decision can see cycles, it is on by default, and it costs
|
||
0.26 dB.** `decide(ctx, lam, mu)` minimises `D + lam*bytes + mu*cycles`;
|
||
`ratectl` bisects mu per frame against a HARD 833,333-cycle ceiling (bytes
|
||
bank in the ring buffer, cycles cannot — there is no double buffer to decode
|
||
ahead into). `sasi` 37/120 misses -> 1, `scsi` 51 -> 1. Bitrate does not
|
||
move: mu changes which modes are bought, not how many bytes. FINDINGS 31,
|
||
`tools/analysis/13_cpu_ratectl.py`.
|
||
0b. **28.7's "11 frames are impossible" was too pessimistic — it is 1.** That
|
||
floor held the SKIP set fixed; the real decision can also move a block to
|
||
SKIP, which above ~90% non-SKIP is the only lever left. FINDINGS 31.3.
|
||
0c. **V4 collapses when cycles are priced**, as 28.8 predicted: 25.2 -> 20.3%
|
||
of blocks at `sasi` and **15.0 -> 5.3%** at `scsi`, where RAW takes it. RAW
|
||
is dearer in bytes and cheaper in cycles, so the byte lagrangian's preference
|
||
inverts and only the byte-rich profile can take the escape.
|
||
0d. **SKIP's price depends on its neighbours, and the way out is two cost
|
||
functions**: a ranking constant inside the per-block lagrangian, the exact
|
||
clustered rule (`vq_hybrid.cycles`, validated to 1 point against the 68000)
|
||
for the frame-level bisection. That function is now defined once and imported
|
||
by `11_cpu_budget.py`. FINDINGS 31.4.
|
||
0e. **Both controllers are gated against decoder drift.**
|
||
`09_ratectl_drift.py` runs bytes-only AND bytes+cycles; both 0/120.
|
||
1. **The span is measured: 43.7 cycles/span + 9.152/pixel, fitted to 0.3% over
|
||
eleven span lengths.** `tools/bench/blit.s` v5/v6, `prep_spans.py`,
|
||
`span.lua`, driven by `tools/bench/span.sh` (~25 s, not in `check.sh`
|
||
because it is a wall timing). FINDINGS 30.
|
||
2. **Only in an encoder-assisted format.** `{u32 absolute GVRAM address, u16
|
||
jump displacement}` into an unrolled chain, versus `(x, npix)` and a decoder
|
||
that works it out: 43.7 + 9.152 against 97.9 + 10.46. All the arithmetic a
|
||
span decoder would do per frame is known at encode time. FINDINGS 30.2.
|
||
3. **The per-pixel cost is a function of register pressure**, which FINDINGS 24
|
||
could not have shown: 9.08 was a fixed blit with 12 registers free, v5 can
|
||
spare 8 and pays 10.46, v6 gets 12 back by making the encoder hold the state.
|
||
4. **Short spans die in the remainder path, and the fix is padding.** A 12-pixel
|
||
span costs more than a 16-pixel one in v5. v6 has no remainder path: lengths
|
||
are multiples of 24 pixels, padding is free of everything but bytes, and an
|
||
overrun past the visible 256 lands in the invisible half of the 1024-byte
|
||
line stride. FINDINGS 30.3.
|
||
5. **Odd-`x` alignment is free** (259.0 vs 261.8 cycles/span) — expected on a
|
||
16-bit bus, now measured rather than assumed.
|
||
6. **The trade is smaller than 29 derived but the conclusion holds**, including
|
||
29.4's reopening of the scene cut. All 23 timing configs also drew a
|
||
pixel-exact frame, so nothing here was timed against a decoder that skipped
|
||
work. FINDINGS 30.5/30.6.
|
||
|
||
---
|
||
|
||
## What session 7 settled
|
||
|
||
1. **68000 code parses a bitstream and draws frames, pixel-exact.**
|
||
`src/player/decode.s` + `tools/bench/decode.lua`. 120 frames of the Singe
|
||
window decoded in sequence, all four block modes, verified against the new
|
||
reference decoder `tools/encoder/dlx.py`. Because SKIP blocks are claims
|
||
about the previous frame, the last frame is only right if all 120 were.
|
||
In `check.sh` now. **FINDINGS 28.**
|
||
2. **It does not fit.** Mean 81.7% of a 12fps frame, p90 116.4%, worst 135.8%;
|
||
31% of frames miss at `sasi`, 42% at `scsi`. Zero-wait-state floor, as ever.
|
||
3. **The dual-display-path plan (FINDINGS 24.5/25.6) is withdrawn as incoherent
|
||
— the sixth false premise this project has caught.** The compose path needs a
|
||
RAM copy of the previous reconstruction; the direct path's whole selling
|
||
point is that it keeps none. Mixing them displays stale pixels on **70 of 120
|
||
frames**, worst frame 18.8% of the screen. Every coherent repair is worse
|
||
than not mixing. `tools/analysis/10_pathmix_drift.py`, kept runnable as a
|
||
counterexample and gated in `check.sh`. FINDINGS 28.1.
|
||
4. **24.5 also compared a copy against a copy.** Its 53.6% and 76.6% both come
|
||
from `blit.s` and neither includes decoding. Compose = decode-into-RAM *plus*
|
||
the 53.6% blit, so it is strictly dearer than decoding into GVRAM. There was
|
||
never a crossover. The player has **one path and no reference frame**, which
|
||
also gives back 96 KB.
|
||
5. **The four block modes cost 300 / 448 / 400 cycles, not one number.** V4 is
|
||
1.49x a V1 block while the mode decision charges it 4x the bytes. The 24.5
|
||
model is 2.03x optimistic at the median. `tools/analysis/11_cpu_budget.py`
|
||
reproduces all four frames timed on the 68000 to within 1 point. FINDINGS 28.2.
|
||
6. **The container is big-endian but not aligned, and on a 68000 that is an
|
||
address error, not a slow read.** Frame records are variable-length and laid
|
||
end to end, so their boundaries land on odd addresses. Frame 0 decoded
|
||
perfectly, then the length read for frame 1 vectored into the IPL and sat
|
||
there for 59 emulated seconds looking like an infinite loop. Found by dumping
|
||
PC and the address registers — the code was right, the data layout was not.
|
||
FINDINGS 28.3. **Encoder gap: `encode.py` should pad records to 4 bytes.**
|
||
Measured cost 1.66 B/frame = 20 B/s against 110 KB/s.
|
||
7. **A full frame does not fit at 12fps in any mode.** All-V1 is 110.5%, all-V4
|
||
165.2%, all-RAW 147.6%. At most ~88% of the screen can change in one frame
|
||
however cheaply it is coded, and scene cuts change 100%. FINDINGS 28.5.
|
||
|
||
---
|
||
|
||
## What session 6 settled
|
||
|
||
1. **Rate control works, is wired in, and is ON by default.** `encode.py`
|
||
bisects lam per frame under a leaky bucket; `--fixed-lam` restores session 5
|
||
behaviour. FINDINGS 27.
|
||
2. **Both overshoots are closed for under 1 dB.** On the Singe window, totals
|
||
including audio: `sasi` 137.4 -> **109.5 KB/s** (target 110) for -0.60 dB,
|
||
`scsi` 381.6 -> **280.0 KB/s** (target 280) for -0.91 dB. Zero frames hit the
|
||
lam=800 cliff at either profile. FINDINGS 27.2.
|
||
3. **The FINDINGS 26 desync is gone by construction, not by tuning.** The
|
||
encoder is frame-drivable (`vq_hybrid.frame_ctx` / `decide` / `paint`) and
|
||
rate control feeds back the frame it actually emitted. The regression test
|
||
`tools/analysis/09_ratectl_drift.py` goes 111/120 drifting frames -> **0**,
|
||
and it is now part of `./tools/bench/check.sh`. FINDINGS 27.1.
|
||
4. **Rate control makes the display path cheaper.** Raising lam moves blocks to
|
||
SKIP and V1, so there is less to write: `scsi`'s median display cost drops
|
||
53.6% -> 47.1%. The decoder conclusion of 25.6 is unaffected. FINDINGS 27.3.
|
||
5. **FINDINGS 26.5 was wrong in both halves, and this is the fifth false premise
|
||
this project has caught.** `_paint` was not the bottleneck (14% of a frame)
|
||
and the ladder was never "minutes" (~18 s; the minutes were k-means in
|
||
`build`). Vectorising it was still right — 17.1x — but what actually makes
|
||
per-frame rate control affordable is that `VQ.assign` depends on neither
|
||
`lam` nor `prev`, so it is cached: a 12-step search over 120 frames costs
|
||
**0.31 s** against 49.1 s. FINDINGS 27.6.
|
||
6. **`--prefill` is a trap and defaults to 0.** It buys a permission to overshoot
|
||
of exactly bucket/nframes; at prefill=1.0 the Singe window goes to 116.3 KB/s
|
||
against a 110 ceiling, and on a 14-frame clip it disables the controller
|
||
outright. FINDINGS 27.4.
|
||
7. **Fixed-lam `sasi` was already 5% over target on 00020**, the clip everyone
|
||
called easy — nothing noticed because the profile table quotes PSNR, not
|
||
bitrate. FINDINGS 27.5.
|
||
8. **1.2-second clips cannot be used to judge rate control.** The bucket's
|
||
startup transient is bucket/nframes: 6% on a 10 s window, 20% on 00020. Same
|
||
lesson as FINDINGS 25.3, different costume.
|
||
|
||
---
|
||
|
||
## Start here: is the tree still green?
|
||
|
||
```
|
||
./tools/bench/check.sh
|
||
```
|
||
~4 min, needs the Blu-ray mounted. From source media it re-runs both display
|
||
regression tests, the rate-control drift test (session 6, now covering BOTH
|
||
controllers -- bytes, and bytes+cycles), the display-path coherency
|
||
counterexample and a **120-frame 68000 decode** (session 7), then prints
|
||
`ALL GREEN`. Verified green at end of session 8.
|
||
|
||
Do not run two of these at once, and do not run one alongside a MAME timing
|
||
job: they share `tmp/` snapshot directories and log files, and the second run
|
||
silently truncates the first one's output.
|
||
If it fails, fix that before doing anything else — everything downstream assumes
|
||
the display path is pixel-exact.
|
||
|
||
The two session-7 stages are worth knowing the shape of before they fail on you:
|
||
- `10_pathmix_drift.py` is expected to exit **non-zero**; `check.sh` fails if it
|
||
ever starts passing, because that would mean the counterexample behind the
|
||
one-path decoder had stopped reproducing.
|
||
- the decode stage needs `tmp/rc_fr_singe_sasi_rcprofile.dlx` and will spend
|
||
~55 s encoding it if it is missing, nearly all of that k-means in `H.build`.
|
||
|
||
## Decisions locked
|
||
|
||
| decision | value | why |
|
||
|---|---|---|
|
||
| Target CPU | 68000 @ 10MHz (stock) | hardest honest constraint |
|
||
| Display mode | 256 colors, 256x192 in 256x256 CRTC mode | every mode is 1 word-access/pixel, so 256c is free vs 16c |
|
||
| Double buffer | **none** — page 1 sacrificed | enables `movem.l` 24px bursts; delta coding needs a RAM reference frame anyway |
|
||
| **Codec** | **hybrid VQ: SKIP / V1 4x4 / V4 four-2x2 / RAW, per-block rate-distortion** | flat 4x4 VQ was measured and rejected — see FINDINGS 9-10 |
|
||
| **Quality modes** | **one: `scsi`** (USER DECISION, session 9 — `sasi` dropped) | a SASI volume is 40 MB and the game is 146 MiB at the LOWEST rate this codec makes. FINDINGS 32 |
|
||
| Delivery medium | **SD-backed SCSI** (BlueSCSI / SCSI2SD), as locked in session 2 | capacity does not bind on SD at any rate this codec makes; it is what killed SASI, and it is what rules CD-ROM OUT (**1.09 GiB** at the session-13 candidate rate, against a CD's ~620 MiB and ~150 KB/s at 1x). FINDINGS 32.3, 42.7 |
|
||
| **SCSI DMA handshake** | **UNDECIDED — and it decides the project** | `W`, clocks stolen per word, bracketed 5..12 by MC68450 Fig 4-25. `W<=6` fits 0/120 frames; `W=8` misses 47/120. It is a property of how the player drives the MB89352, so it is ours to choose, not to receive. FINDINGS 42.4-42.6 |
|
||
| Profile axis | **I/O bandwidth only** | the profiles say nothing about CPU; **both target the same stock 10MHz 68000**, and the Super has SCSI at 10MHz. FINDINGS 28.7 |
|
||
| Framerate | 12 fps, **explicit decimation** | source has zero duplicate frames; no free "twos" win |
|
||
| Emulator | MAME 0.277 x68000 | accurate enough that measured cycles mean something |
|
||
| SNES project reuse | **MIT — cleared** | `data/events/` scene graph is reusable with attribution |
|
||
|
||
### The SASI/SCSI question is RESOLVED — and in session 9 SASI was DROPPED
|
||
Session 1 left "which machine do we target" open. Session 2's answer was **ship
|
||
both**, as two quality profiles. **Session 9 retired `sasi`** (USER DECISION) on
|
||
CAPACITY, not bandwidth: a SASI volume is limited to 40 MB, and the 22.8 minutes
|
||
of unique scene footage on the source Blu-ray is **146 MiB even at 110 KB/s** —
|
||
more than the machine's whole 4-unit SASI address space. FINDINGS 32.
|
||
|
||
Everything below this line about *two* profiles is the session-2..8 record, kept
|
||
because every measurement in FINDINGS 27-31 was taken against it. One profile
|
||
ships:
|
||
|
||
| profile | target | lam | quality (00020 / 00146) | machine |
|
||
|---|---|---|---|---|
|
||
| ~~`sasi`~~ | ~~110 KB/s~~ | ~~60~~ | ~~36.9 / 29.6 dB~~ | **RETIRED session 9** |
|
||
| `scsi` | 280 KB/s | 10 (floor) | 39.4 / 32.3 dB | Super/XVI, or CZ-6BS1 board |
|
||
|
||
~~The 110 KB/s **rate point** is not necessarily gone with the interface: a 1x
|
||
CD-ROM sustains ~150 KB/s and CD-ROM is the only period medium with room for the
|
||
whole game (374 MiB at `scsi`, 600 MiB with spans). The user's call was to ship
|
||
one profile now and settle the medium when the pipe is measured.~~ FINDINGS 32.3.
|
||
|
||
**SUPERSEDED by FINDINGS 42.** There is no rate point left to choose: the span
|
||
pass saturates at ~837 KB/s and the candidate container is **`s13_280p1500`,
|
||
837.4 KB/s = 1.09 GiB for the whole game**. That is unremarkable on SD and
|
||
impossible anywhere period, so **CD-ROM is closed, not parked**. `--kbps` is now
|
||
a quality knob whose value barely matters above ~800 KB/s of span budget; the
|
||
number that matters is `W`.
|
||
|
||
| what | KB/s | whole game (1366.6 s) |
|
||
|---|---:|---:|
|
||
| `scsi` profile as set | 280.0 | 0.36 GiB |
|
||
| session-12 gate | 487.7 | 0.64 GiB |
|
||
| **candidate `s13_280p1500`** | **837.4** | **1.09 GiB** |
|
||
| probe `s13_lossless` | 934.6 | 1.22 GiB |
|
||
|
||
**That "machine" column is about the BUS, not the CPU.** The profiles are an
|
||
I/O-bandwidth axis and say nothing about clock speed: the X68000 Super has
|
||
built-in SCSI at 10 MHz (`x68k.cpp:1194`, `40_MHz_XTAL/4`, same as the base
|
||
machine), and only the XVI is faster. **Both profiles target the same stock
|
||
10 MHz 68000**, so both must fit the same 833,333-cycle frame budget — and as of
|
||
session 7 neither does. FINDINGS 28.7.
|
||
|
||
**As of session 6 `lam` is a floor, not a setting.** The target is a ceiling and
|
||
the encoder bisects lam per frame to stay under it; the profile's lam is the
|
||
best quality it is allowed to spend on a quiet frame. On the worst sustained
|
||
window that takes `sasi` from 137.4 to 109.5 KB/s and `scsi` from 381.6 to
|
||
280.0 KB/s, for -0.60 and -0.91 dB. FINDINGS 27.2.
|
||
|
||
Sized against the user's working figure of **4 Mbps = 488 KB/s sustained**, on
|
||
SD-backed SCSI (BlueSCSI / SCSI2SD) — so that rate is a bus-limited **constant**,
|
||
not an average over seek latency.
|
||
|
||
**Both profiles fit with room.** Ring-buffer simulation on the real per-frame
|
||
sizes gives **zero required prefill** for every scene at both profiles: the fill
|
||
delivers 40.69 KB per frame time and only one measured frame (42.10 KB) exceeds
|
||
that, recovered by the next. A 256 KB buffer carries ~1 s of stall tolerance,
|
||
far more than an SD-backed seek needs. FINDINGS 21.
|
||
|
||
An earlier warning here said `scsi` did not fit because a frame peaked at 96.4%
|
||
of the pipe. That compared instantaneous demand to a sustained rate as if they
|
||
had to match frame-by-frame; with a buffer the test is cumulative, and it passes.
|
||
|
||
`scsi` is now within **0.5 dB of the palette ceiling** on 00020. These were
|
||
initially set at 45 / 75 KB/s, which was 12% / 7% bus utilisation — read off the
|
||
RD curve rather than derived from the hardware. See FINDINGS 17.
|
||
|
||
Codebooks are **k=256 with 1-byte indices** in both profiles. k=1024 was measured
|
||
and rejected — see FINDINGS 14, it was a false-good result from a rate model
|
||
that undercharged the index. Do not ship past `lam~800`; FINDINGS 15 has the cliff.
|
||
|
||
Because of the RAW escape mode, `lam=0` is **pixel-exact** against the palettised
|
||
frame (measured 0.00 dB loss). The profiles are two points on one continuous
|
||
rate-distortion curve, not two codecs.
|
||
|
||
---
|
||
|
||
## What session 3 settled
|
||
|
||
1. **The display path works and is verified end to end.** First real frame on an
|
||
emulated X68000 screen: `docs/images/x68k_first_frame_compare.png`. Full
|
||
write-up in **FINDINGS 22**. Everything before this session was Python-side
|
||
or a headless `-video none` run, which cannot snapshot at all.
|
||
2. **The render is pixel-exact, not merely close.** With monitor contrast at 15,
|
||
all 256 palette entries render exactly as `GGGGGRRRRRBBBBBI` + `pal6bit`
|
||
predicts. That exactness is the regression test — see
|
||
`tools/bench/verify_frame.py`, which exits non-zero if it ever drifts.
|
||
3. **Three hardware facts that were previously assumed are now confirmed from
|
||
MAME 0.277 source**, not folklore: the palette word format, the 1024-byte
|
||
GVRAM line stride, and the 256-colour page aliasing in `HARDWARE.md`. All
|
||
three were already written down correctly; they are now cited.
|
||
4. ~~**A new quality ceiling was measured** — the 15-bit+I palette alone costs
|
||
38.88 dB.~~ **Superseded by session 4:** that figure assumed the shared LSB
|
||
`I` is always 1. Chosen per entry, the ceiling is **40.81 dB**. FINDINGS 23.3.
|
||
5. **Two shell traps that wedged session 2's background jobs** are documented in
|
||
the working-setup section below. They cost ~1.5 h of wall clock and a wedged
|
||
CPU core, and one of them was hit again this session.
|
||
|
||
## What session 5 settled
|
||
|
||
1. **68000 code drew a frame, and the blit was measured.** `tools/bench/blit.s`
|
||
+ `blit.lua`. The snapshot passes `verify_frame256.py` unchanged — pixel-exact
|
||
in the real 256x256 mode. **FINDINGS 23.5 is closed**: no longer "proven from
|
||
Lua only".
|
||
2. **The 38% full-frame blit estimate is dead. It is 53.6%.** And that is a
|
||
zero-wait-state floor — MAME models no GVRAM wait states, so real hardware is
|
||
worse. FINDINGS 24. Every variant was hand-derived from the MC68000 timing
|
||
tables before being measured and the two agree to 0.006-0.43%, so this is not
|
||
another MAME artefact.
|
||
3. **Reading the source frame is exactly half the blit cost** (V1 53.6% vs a
|
||
write-only floor V3 of 27.1%). That is what makes the architecture question
|
||
below live.
|
||
4. **That number is now measured, and the answer is "implement both paths".**
|
||
On the worst sustained window found on the disc, 30% of frames (`sasi`) to
|
||
53% (`scsi`) sit above the 70% crossover and want the flat blit; the rest
|
||
want direct-to-GVRAM. A player that picks per frame — the mode headers are
|
||
parsed before any pixel is written, so the count is free — pays a **median
|
||
37.0%** and is **capped at 53.6%**. FINDINGS 25.6.
|
||
5. **The sustained action sequence exists, was found by measurement, and breaks
|
||
both profiles.** `tools/analysis/07_motion_survey.py` scans a whole stream
|
||
for the hottest sliding window; on 00223 it is t=539.4s, the Singe endgame,
|
||
at 2.01x the stream mean. There, fixed-lam `sasi` overshoots 110 -> 129.6
|
||
KB/s (+18%) and `scsi` 280 -> 373.8 KB/s (+34%). **Rate control is no longer
|
||
insurance — it is required.** FINDINGS 25.3.
|
||
6. **The two largest streams on the disc are bonus material, not game footage.**
|
||
00216 is the feature with a burned-in commentary PiP; 00215 is the commentary
|
||
itself. **00223 (9.4 min) is the clean one.** A size-ranked survey would have
|
||
encoded live action. FINDINGS 25.1.
|
||
7. **Rate control is unsound as written, caught before wiring it up.** The
|
||
lam-ladder in `ratectl.py` picks frames from independent temporal chains,
|
||
so SKIP blocks reference reconstructions the decoder never saw: 111 of 120
|
||
frames drift, worst frame 43.4%, reported PSNR overstated 0.36 dB. Regression
|
||
test `tools/analysis/09_ratectl_drift.py`. FINDINGS 26.
|
||
8. **On hard content the scene palette, not the display, is the binding
|
||
ceiling** — 31.33 dB on the Singe window against 39.90 dB on 00020 and 40.81
|
||
dB for the X68000 display. `scsi` is already within 0.51 dB of it.
|
||
FINDINGS 25.4.
|
||
|
||
### Superseded within session 5
|
||
4a. **The decoder architecture hinged on one unmeasured number.** Writing
|
||
codewords straight into GVRAM costs 76.6% of the frame budget for a *full*
|
||
frame (V4 — the 1024-byte stride kills the `movem.l` burst), but scales with
|
||
the non-SKIP block fraction and needs **no RAM reference frame at all**,
|
||
because the previous frame is already in GVRAM. Compose-then-blit is a flat
|
||
53.6%. **They cross at 70% of blocks changed.** FINDINGS 24.5.
|
||
|
||
---
|
||
|
||
## What session 4 settled
|
||
|
||
1. **A real 256x256 CRTC mode exists and is verified.** `crtc_mode.lua`, derived
|
||
from `x68k_crtc.cpp`'s divisor ladder rather than recalled — the derivation is
|
||
self-checking (368 = 1104/3 exactly, so the horizontal registers divide by
|
||
three with no remainder). Snapshot is native 256x512, active area pixel-exact,
|
||
letterbox true black. FINDINGS 23. The x=512 wrap of FINDINGS 22.5 is gone.
|
||
2. **The palette ceiling was wrong by 2 dB, in our favour.** The shared LSB `I`
|
||
must be chosen **per palette entry**, not hardcoded to 1. Doing so lifts the
|
||
display ceiling from 38.85 to **40.81 dB** and is the only way to get true
|
||
black at all (`pal6bit(1) = 4`). 102 of 256 entries want `I = 0`. This
|
||
supersedes FINDINGS 22.4 and gives `scsi` ~2 dB more headroom than believed.
|
||
**The encoder does not do this yet** — see the encoder-gaps list.
|
||
3. **Letterboxing costs one palette entry.** 255 colours + a reserved black at
|
||
index 0, with `I = 0` on it. `prep_frame.py --reserve-black`. FINDINGS 23.4.
|
||
4. **MAME's graphics double-scan is phase-shifted one raster line** — pairs are
|
||
(1,2),(3,4),..., not (0,1), because `get_gfx_pixel` halves the *absolute*
|
||
scanline and `vbegin = 41` is odd. Cost a false failure. FINDINGS 23.2.
|
||
|
||
---
|
||
|
||
## What session 2 settled
|
||
|
||
1. **The critical-path question is answered.** "Does VQ soften Bluth's linework
|
||
unacceptably?" — **flat 4x4 VQ: yes, badly. The hybrid (SKIP/V1/V4/RAW): no.**
|
||
Verified by eye, not just PSNR. See `docs/FINDINGS.md` 9-11 and the two
|
||
images in `docs/images/`. Both profiles use **k=256**; see item 2b.
|
||
2. **Session 1's 12fps bitrate was wrong** (183 KB/s claimed, 340 KB/s measured).
|
||
Halving the framerate does not halve the bitrate. FINDINGS 8.
|
||
2b. **A fourth false-good result was produced and caught this session** — k=1024
|
||
codebooks looked like a +2.4 dB free win because the rate model charged 1 byte
|
||
for a 10-bit index. FINDINGS 14. The k=256 configuration ships.
|
||
3. **The 256-colour palettised frame is the real quality ceiling** and it looks
|
||
excellent. Judge the codec against that, not against 1080p.
|
||
4. Encoder exists and produces a real bitstream: `tools/encoder/`.
|
||
|
||
---
|
||
|
||
## Encoder — working
|
||
|
||
```
|
||
python3 tools/encoder/extract.py 00020 /tmp/fr_00020 12 crop
|
||
python3 tools/encoder/encode.py /tmp/fr_00020 out.dlx --profile sasi --preview p.png
|
||
```
|
||
|
||
| file | role |
|
||
|---|---|
|
||
| `extract.py` | .m2ts -> 256x192 PNGs, 12fps, spatial-only denoise |
|
||
| `vq.py` | palette, blockify, hand-rolled k-means (no sklearn on this box), PSNR |
|
||
| `vq_hybrid.py` | the codec: 4 block modes + lagrangian mode decision |
|
||
| `ratectl.py` | SASI/SCSI profiles, leaky-bucket rate control |
|
||
| `encode.py` | CLI + `DLX1` container writer |
|
||
|
||
`DLX1` container layout is documented in the `encode.py` docstring. All
|
||
multi-byte fields are **big-endian** so the 68000 reads them with a plain `move`.
|
||
|
||
### Known encoder gaps
|
||
- ~~Rate control is written but not yet wired into `encode.py`.~~ **DONE,
|
||
session 6.** It is on by default; `--fixed-lam` restores the old behaviour.
|
||
Gated by `tools/analysis/09_ratectl_drift.py`, which is now in `check.sh`.
|
||
- **Payload is deliberately NOT entropy-coded** — deflate decode does not fit in
|
||
the 68000's frame budget (FINDINGS 17.2). Do not "optimise" this later.
|
||
- ~~**Frame records are not aligned.**~~ **DONE.** The container is `DLX2` and
|
||
`write_container` (`encode.py:139-156`) pads both the frame table and every
|
||
record to a 4-byte boundary, inside the rate controller's accounting rather
|
||
than after it, and `encode.py` reports the cost in B/frame and B/s. Unaligned
|
||
is an ADDRESS ERROR on a 68000, not a slow read (FINDINGS 28.3); `DLX1` is
|
||
still read, so the old unaligned containers have not been invalidated.
|
||
- ~~**The mode decision is blind to CPU cost.**~~ **DONE**, sessions 12-19.
|
||
`vq_hybrid.decide()` (`vq_hybrid.py:218`) minimises
|
||
`distortion + lam*bytes + mu*(decode cycles + byte_clk*bytes)` against the
|
||
MEASURED per-mode cycle costs, and `cycles()` scores a whole frame with the
|
||
exact clustered SKIP rule rather than a per-block constant. This entry sat in
|
||
the gap list for several sessions after it was closed; **the list is not
|
||
self-maintaining, and stale entries here become phantom work in the roadmap.**
|
||
- **Palette packing is not implemented in the encoder.** It still emits 24-bit
|
||
palettes; the X68000 word packing happens Lua-side. Whatever writes real
|
||
palette words must pick `I` per entry by minimum squared error (FINDINGS 23.3,
|
||
worth 1.96 dB) and reserve index 0 as black with `I = 0` (FINDINGS 23.4).
|
||
- Codebooks are per-scene and rebuilt from scratch; no inter-scene reuse.
|
||
- ~~`_paint` is a Python per-block loop.~~ **DONE, session 6** — vectorised,
|
||
17.1x. It was never the bottleneck, though: `VQ.assign` is 78% of a frame and
|
||
`H.build`'s k-means is 51 s of a 55 s run. **That k-means is now the thing to
|
||
attack before the full-disc survey**, not anything in the per-frame path.
|
||
FINDINGS 27.6.
|
||
|
||
---
|
||
|
||
## Working setup (unchanged from session 1, re-verified)
|
||
|
||
**MAME ROMs** — `~/mame/roms/x68000.zip`. Must pass **`-bios ipl10`**.
|
||
```
|
||
mame x68000 -bios ipl10 -video none -sound none -nothrottle -seconds_to_run 3
|
||
```
|
||
**Assembler** — `tools/vasm/vasmm68k_mot -Fbin -o out.bin in.s`
|
||
|
||
**Blu-ray** — `udisksctl loop-setup -r -f DRAGONS_LAIR.iso` -> `/media/reala-misaki/BDROM`
|
||
(still mounted as of end of session 2).
|
||
|
||
**MAME Lua harness** — `tools/bench/*.lua`, working. Three gotchas (retain the
|
||
notifier subscription in a global; the stack register is `SP` not `A7`;
|
||
`autoboot_script` fires at PC=0 before boot) are documented in FINDINGS.
|
||
|
||
**Two shell traps, both hit again this session:**
|
||
- piping MAME (or any long job) through `grep` block-buffers — write to a file.
|
||
- `pkill -f <pattern>` matches your own shell and kills it (exit 144).
|
||
Use `pkill -x` or kill by PID.
|
||
- **`pgrep -f <name> | xargs kill` kills your own shell too — exit 144.** Same
|
||
root cause as the `pkill -f` trap above: the shell's own command line contains
|
||
the pattern. **Hit again in session 5**, which makes it four times across three
|
||
sessions. Kill by PID captured at launch (`$!`), or use `pkill -x`.
|
||
- **`until ! pgrep -f foo.py; do sleep; done` watcher loops never exit.** The
|
||
watching shell's own command line contains the string `foo.py`, so `pgrep -f`
|
||
matches the watcher itself and the loop spins forever. Session 2 left 11 of
|
||
these wedged for over an hour. Wait on the PID (`while kill -0 $PID`) or on a
|
||
sentinel file the job touches when it finishes -- never on a `-f` name match.
|
||
- **`timeout N mame ...` does not kill MAME.** MAME catches SIGTERM and, with an
|
||
autoboot script blocked waiting on a flag that never arrives, never reaches
|
||
its shutdown path. `timeout` without `-k` then waits forever while MAME burns
|
||
a full core at `-nothrottle`. Always `timeout -k 5 N`.
|
||
|
||
---
|
||
|
||
## Disk throughput benchmark — still blocked, no longer gating
|
||
|
||
`IOCS _B_READ` returns -1 uniformly. Full diagnosis and the four untested
|
||
hypotheses are in session 1's notes (git history of this file, commit 65112b9);
|
||
the ordered plan for retrying is in **`docs/BENCHMARK.md`**.
|
||
|
||
**Status changed twice this session — read this rather than the git history.**
|
||
It was briefly promoted to critical-path while the working bandwidth figure was
|
||
misread as 4 MB/s. With the correct figure (**4 Mbps = 488 KB/s**) and the
|
||
ring-buffer simulation showing **zero required prefill** for both profiles
|
||
(FINDINGS 21), the design no longer hangs on it. Pixel-exact on SCSI is **not**
|
||
available at 4 Mbps — it needs 92-97% of the pipe — so there is no longer a
|
||
"measure it and maybe ship transparent" decision waiting.
|
||
|
||
What the benchmark is still worth doing for:
|
||
- **Confirming the 4 Mbps figure.** It is user-supplied and its provenance is
|
||
not recorded. Every profile hangs off it.
|
||
- **Confirming DMA is actually used.** If transfers fall back to PIO the CPU
|
||
cost rises far above the ~12-15% cycle-steal estimate and CPU becomes the
|
||
binding constraint. This is the worst plausible outcome and the cheapest to
|
||
check — do it first.
|
||
|
||
**Do not try to get the bandwidth number out of MAME.** Its SCSI/SASI devices are
|
||
functional models, not timing-accurate; a KB/s figure from MAME measures the
|
||
emulator's scheduler. `docs/BENCHMARK.md` covers the three-tier approach
|
||
(MAME validates the path, derivation bounds it, real hardware settles it).
|
||
|
||
## Display path — VERIFIED (session 3), in a real mode (session 4), by 68000 code (session 5), by a 68000 DECODER (session 7).
|
||
|
||
The first real frame is on screen: `docs/images/x68k_first_frame_compare.png`.
|
||
|
||
**Session 7 went from copying a frame to parsing one.** `src/player/decode.s`
|
||
reads DLX1, dispatches all four block modes and writes straight into GVRAM;
|
||
120 frames decoded in sequence are pixel-exact against `tools/encoder/dlx.py`
|
||
(`tools/bench/verify_decode.py`, in `check.sh`). The blit numbers below are
|
||
still correct for what they measured — a *copy* — but they are no longer the
|
||
display-path budget: the decoder costs 300/448/400 cycles per V1/V4/RAW block
|
||
and misses the 12fps budget on 31% of frames. FINDINGS 28.
|
||
|
||
**Session 5 closed the gap this paragraph used to describe.** GVRAM is now
|
||
filled by 68000 instructions and the result is still pixel-exact, and the blit
|
||
cost is measured rather than estimated: **53.6% of a 12fps frame**, not 38%
|
||
(FINDINGS 24). The remaining caveat is different and narrower: MAME models
|
||
**no GVRAM wait states**, so 53.6% is a floor and real hardware is worse.
|
||
Full write-up in **FINDINGS 22**. Harness: `tools/bench/show_frame.lua` +
|
||
`tools/bench/prep_frame.py`.
|
||
|
||
Three facts the player MUST honour, none of which were guessable:
|
||
|
||
| what | where | value |
|
||
|---|---|---|
|
||
| **Un-hide the graphics layer** | CRTC R20 `$E80028` | clear bit 11 ("G-VRAM set to buffer"); IPL leaves `0x0B16` |
|
||
| Colour setup (256c) | CRTC R20 bits 9-8 | `0x0100` |
|
||
| **Monitor contrast** | `$E8E001` bits 3-0 | IPL leaves **14**; write **15** or everything renders 7% dark |
|
||
|
||
The `R20 = 0x0116` value quoted here in session 3 is the **768-wide IPL timing**
|
||
with the gate cleared. The shipping value is **`R20 = 0x0110`** — see the mode
|
||
table in `tools/bench/crtc_mode.lua`, which is now the single source of truth
|
||
for all of R00-R08 and R20.
|
||
|
||
Bit 11 is the one that cost the most time: GVRAM writes land and read back
|
||
correctly while the layer is invisible, so the video controller looks guilty and
|
||
is not. Contrast `0` blanks the screen — free fade-to-black for transitions.
|
||
|
||
Palette format is now **confirmed from MAME source**, not assumed:
|
||
`GGGGGRRRRRBBBBBI` (G 15:11, R 10:6, B 5:1, shared LSB I), expanded as
|
||
`pal6bit((field<<1)|I)`. With contrast at 15 the render is **pixel-exact**.
|
||
|
||
Ceiling: the 15-bit+I palette costs **40.81 dB** against the 24-bit palettised
|
||
source, once `I` is chosen per entry (FINDINGS 23.3 — session 3's 38.88 dB
|
||
assumed `I = 1`). Still the same order as the `scsi` profile's own codec error
|
||
(39.4 dB), so `scsi` remains near display-transparent, with ~2 dB more headroom
|
||
than session 3 thought.
|
||
|
||
Snapshot recipe that works (`-video none` CANNOT snapshot):
|
||
```
|
||
SDL_VIDEODRIVER=dummy mame x68000 -bios ipl10 -video soft -window \
|
||
-sound none -nothrottle -plugins -autoboot_script <script>.lua \
|
||
-snapshot_directory ./snap -snapview native -seconds_to_run 6
|
||
```
|
||
`-snapview native` drops MAME's LED artwork and gives a clean 768x512 screen.
|
||
|
||
## Next steps, in priority order
|
||
|
||
1. ~~**Measure the non-SKIP block fraction.**~~ **DONE, session 5**, and its
|
||
answer **WITHDRAWN in session 7** — FINDINGS 28.1/28.2. It concluded
|
||
"implement both display paths and pick per frame, median 37.0%, capped at
|
||
53.6%". Mixing the paths is incoherent (the compose path needs a RAM
|
||
reference the direct path never writes) and the two costs it compared were
|
||
both copies with no decode in either. **The shipping decoder has one path.**
|
||
The non-SKIP fraction is still reported by `encode.py` and is still the right
|
||
thing to look at — it is just no longer a switch. Original framing kept
|
||
below, because its instruction to report the distribution rather than the
|
||
mean is the part that held up:
|
||
FINDINGS 24.5: compose-in-RAM-then-blit costs a flat 53.6% of the frame
|
||
budget; decode-direct-to-GVRAM costs 76.6% x (fraction of blocks that are not
|
||
SKIP) and needs no RAM reference frame. **They cross at 70%.** Which side of
|
||
70% the content sits on decides which decoder inner loop to write, so this
|
||
must come before writing one.
|
||
**It needs no new machinery** — the mode decision in `vq_hybrid.py` already
|
||
computes it per frame and simply never reports it. Add the histogram
|
||
(SKIP / V1 / V4 / RAW counts per frame) to `encode.py` output and run it over
|
||
the clips already extracted. Report the *distribution*, not the mean: a
|
||
scene-cut frame is ~100% non-SKIP and a held frame near 0%, and the mean of
|
||
those two is a number describing no actual frame.
|
||
|
||
1b. ~~**Wire rate control into `encode.py`.**~~ **DONE, session 6.** FINDINGS 27.
|
||
Both overshoots closed for under 1 dB, drift test at zero, `check.sh` gates
|
||
it. The remaining rate-control question is not a defect: whether `--rc-floor
|
||
open` is worth taking on quiet content. It measured as worth **0.00 dB** on
|
||
the Singe window (no frame there is quiet enough to saturate the bucket), so
|
||
it needs a genuinely quiet scene to decide, and it is a quality-per-byte
|
||
judgement rather than a correctness one.
|
||
|
||
2. ~~**68000 decoder skeleton.**~~ **DONE, session 7.** `src/player/decode.s`,
|
||
pixel-exact over 120 frames, gated in `check.sh`. It answered the question it
|
||
was written to answer, and the answer is no: **it does not fit** — mean 81.7%
|
||
of a 12fps frame, 31% of frames over 100%. FINDINGS 28. The follow-on is
|
||
priority 0 at the top of this file.
|
||
|
||
2a. **Re-budget everything against the MEASURED per-mode costs**, not 53.6% and
|
||
not 38%. Session 7 replaced the model twice over (FINDINGS 28.2): the display
|
||
path is not one number times a block fraction, and the median frame is 74.4%
|
||
rather than 36.6%. The original note is kept below because its warning about
|
||
downstream figures derived from a dead estimate is exactly what happened
|
||
again.
|
||
~~Re-budget everything against 53.6%, not 38%.~~ Several downstream figures
|
||
were derived from the old estimate. The blit alone now eats over half the
|
||
frame at 12fps in the compose-then-blit design, before any decode, and MAME
|
||
models no GVRAM wait states so that is a floor. This may reopen questions
|
||
that were closed against the 38% number — check FINDINGS 17.2's entropy-coding
|
||
rejection, which was argued as "54% LZ4 with no room beside a 38% blit". The
|
||
conclusion gets *stronger*, not weaker, but the arithmetic should be restated.
|
||
|
||
2b. **Pad frame records to 4 bytes in `encode.py`.** Not optional: unaligned
|
||
records are an address error on a 68000 (FINDINGS 28.3), and `prep_dlx.py`
|
||
currently repairs it at load time, which the shipping player streaming from
|
||
disc cannot do. The padding is real bytes on disc, so it has to be inside
|
||
the rate controller's accounting, not added after it. 20 B/s at 12fps.
|
||
|
||
3. **Full-disc survey.** Now scoped by session 5 rather than open-ended: the
|
||
worst *sustained* window is measured (FINDINGS 25), so what remains is the
|
||
distribution over content, not the worst case.
|
||
- Classify **content / menu / bonus** — not just menu vs content. FINDINGS
|
||
25.1: the two largest streams are bonus material and look like content by
|
||
size, duration and bitrate alike.
|
||
- Run `tools/analysis/07_motion_survey.py` per stream first; it is cheap
|
||
(96x72 greyscale) and gives a hot-window shortlist so the expensive encode
|
||
only runs where it matters.
|
||
- ~~Vectorise `_paint` before this run.~~ Done. The cost to attack now is
|
||
`H.build`'s k-means: 51 s of a 55 s run, and it runs once per scene.
|
||
- ~~Do it after rate control (1b), or it measures an encoder nobody ships.~~
|
||
Rate control is in, so the survey now measures the shipping encoder.
|
||
|
||
5. **Confirm DMA vs PIO in MAME** (see the benchmark section above) — cheap, and
|
||
the only thing that could still move CPU into the binding position.
|
||
6. **Resolve the framing question** (FINDINGS 12: crop vs squash vs wide).
|
||
Needs an eyeball against arcade reference, not a measurement.
|
||
7. **Import the scene graph.** SNES project `data/events/` (MIT, cleared),
|
||
cross-checked against DirkSimple (zlib) which transcribed the same data
|
||
independently — diff them to catch transcription errors before committing
|
||
any of it to 68000 tables.
|
||
8. **ADPCM audio.** MSM6258, 15.6kHz mono, 7.8 KB/s — already budgeted in
|
||
`ratectl.py`, not yet extracted or encoded.
|
||
|
||
### Explicitly abandoned — do not re-propose
|
||
- ~~Entropy-code the payload.~~ Deflate decode is ~216% of the frame budget on a
|
||
68000; LZ4 is ~54% with no room beside a 38% blit (FINDINGS 17.2). All bitrates
|
||
are raw payload. This also demotes the "247 KB/s lossless" figure in FINDINGS 8
|
||
to a compression upper bound, not a shippable design.
|
||
- ~~k=1024 codebooks.~~ False-good result from a rate model that charged 1 byte
|
||
for a 10-bit index (FINDINGS 14). k=256 wins at every matched bitrate.
|
||
- ~~Flat 4x4 VQ.~~ Rejected by eye (FINDINGS 9).
|
||
|
||
## Not yet started
|
||
- **A player, as opposed to a decoder.** `src/player/decode.s` parses DLX1,
|
||
dispatches all four block modes and draws pixel-exact frames, but it decodes
|
||
from RAM that Lua pre-loaded. There is no disc streaming, no ring buffer, no
|
||
audio, no timing against the VBL, and no scene branching.
|
||
- **Codebook expansion on the 68000.** `prep_dlx.py` does it host-side because
|
||
it is a load-time cost and including it would flatter or damn the inner loop.
|
||
The player must do it: 8 KB + 2 KB per scene.
|
||
- ADPCM audio extraction/encoding
|
||
- Disk image packaging
|
||
- Game logic (scene branching, input windows, death clips)
|
||
|
||
## Reproducing the 256x256 mode result (session 4)
|
||
|
||
```
|
||
python3 tools/encoder/extract.py 00020 tmp/fr_00020 12 crop
|
||
python3 tools/bench/prep_frame.py tmp/fr_00020 tmp/frame256.bin 0 --reserve-black
|
||
mkdir -p tmp/snap256 && cd tmp && SDL_VIDEODRIVER=dummy timeout -k 5 90 mame x68000 \
|
||
-bios ipl10 -video soft -window -sound none -nothrottle -plugins \
|
||
-autoboot_script ../tools/bench/show_frame256.lua \
|
||
-snapshot_directory ./snap256 -snapview native -seconds_to_run 6
|
||
cd .. && python3 tools/bench/verify_frame256.py
|
||
```
|
||
Exits non-zero on any drift. Expected: `256x512 native, double-scan exact,
|
||
active 256x192 pixel-exact, letterbox true black`, ceiling 40.81 dB.
|
||
|
||
## Reproducing the display result
|
||
|
||
```
|
||
python3 tools/encoder/extract.py 00020 tmp/fr_00020 12 crop
|
||
python3 tools/bench/prep_frame.py tmp/fr_00020 tmp/frame.bin 0
|
||
mkdir -p tmp/snap_verify && cd tmp && SDL_VIDEODRIVER=dummy mame x68000 -bios ipl10 \
|
||
-video soft -window -sound none -nothrottle -plugins \
|
||
-autoboot_script ../tools/bench/show_frame.lua \
|
||
-snapshot_directory ./snap_verify -snapview native -seconds_to_run 6
|
||
cd .. && python3 tools/bench/verify_frame.py
|
||
```
|
||
Verified cold from the Blu-ray at end of session 3: exact match, 38.88 dB.
|
||
(That 38.88 is correct *for this test*: `show_frame.lua` still packs `I = 1`.
|
||
The 40.81 dB ceiling comes from `show_frame256.lua`, which picks `I` per entry.)
|
||
|
||
`tmp/` is gitignored scratch. The frames are NOT in the repo — regenerate them
|
||
with `extract.py`; the earlier ones lived in `/tmp` and do not survive a reboot.
|
||
|
||
## Reference material on this box (not in the repo)
|
||
|
||
- **MC68450 DMAC manual: `~/src/mc68450.pdf`** (Motorola, Jul 1989, from
|
||
bitsavers; `curl` it with a browser User-Agent or you get a 403). This is the
|
||
primary source for FINDINGS 39 and it overturned two derived answers in one
|
||
session. Fig 4-25 is the timing table — sheet 1 chaining, sheet 3
|
||
single-address, sheet 4 dual-address; note 2 (4-clock reads, 5-clock writes)
|
||
is the one that mattered. Sects 4.5.2.1-3 are the arbitration overheads,
|
||
5.2.3.2 the limited-rate auto-request slabs. `pdftotext` handles it.
|
||
- **px68k source: `~/src/px68k`** — only `m68000/c68k.c` is used, by
|
||
`tools/bench/c68k/`.
|
||
- **MAME 0.277 source: `~/src/mame-mame0277/`** (tarball `~/src/mame0277.tar.gz`).
|
||
Downloaded this session to settle the graphics-layer question. The files that
|
||
matter are `src/mame/sharp/x68k_v.cpp`, `x68k_crtc.cpp`, `x68k_crtc.h`,
|
||
`x68k.cpp`. **Read these before theorising about X68000 video behaviour** —
|
||
six register-poking attempts failed against a gate that one grep found.
|
||
- Blu-ray mounted at `/media/reala-misaki/BDROM` via
|
||
`udisksctl loop-setup -r -f DRAGONS_LAIR.iso`.
|
||
|
||
---
|
||
|
||
## Parked ideas (not scheduled, not abandoned)
|
||
|
||
- **Cliff Hanger, retitled as Lupin III** (user, session 4). Stern's 1983
|
||
laserdisc game was cut from *Castle of Cagliostro* and *Mystery of Mamo* with
|
||
the Lupin branding stripped; a port would restore it. Technically **cheaper
|
||
than this project**: same content class (cel animation, flat colour, hard
|
||
cuts), ~13 min of footage vs Dragon's Lair's ~22, and flatter linework than
|
||
Bluth's, so fewer blocks should escape to V4/RAW. The codec, the display path,
|
||
and `crtc_mode.lua` would all drop straight in.
|
||
**The real cost is media prep, not code:** there is no clean master cut to
|
||
Stern's scene boundaries the way `DRAGONS_LAIR.iso` is, so the footage would
|
||
have to be sourced and cut to match. Not to be started until the CPU path is
|
||
proven — it changes nothing about whether this design works.
|
||
|
||
## Reproducing the blit measurement (session 5)
|
||
|
||
```
|
||
python3 tools/encoder/extract.py 00020 tmp/fr_00020 12 crop
|
||
python3 tools/bench/prep_frame.py tmp/fr_00020 tmp/frame256.bin 0 --reserve-black
|
||
tools/vasm/vasmm68k_mot -Fbin -o tmp/blit.bin tools/bench/blit.s
|
||
mkdir -p tmp/snap_blit && cd tmp && SDL_VIDEODRIVER=dummy timeout -k 5 900 mame x68000 \
|
||
-bios ipl10 -video soft -window -sound none -nothrottle -plugins \
|
||
-autoboot_script ../tools/bench/blit.lua \
|
||
-snapshot_directory ./snap_blit -snapview native -seconds_to_run 120
|
||
```
|
||
~25 s wall. Prints cycles/frame and % of a 12fps budget for V1-V4, and snapshots
|
||
V1's output. To check that snapshot is still pixel-exact:
|
||
`sed 's|snap256|snap_blit|' tools/bench/verify_frame256.py | python3 -`
|
||
|
||
Not added to `check.sh`: `check.sh` asserts pixel-exactness, and asserting wall
|
||
timings there would make the green-light check sensitive to host load.
|
||
|
||
## Reproducing the streaming result (session 18)
|
||
|
||
The ring-buffer pass, on a **stock 2 MB machine**, container in a host file:
|
||
|
||
```
|
||
DLX=tmp/rc_fr_singe_scsi_span.dlx
|
||
tools/vasm/vasmm68k_mot -Fbin -o tmp/stream.bin src/player/stream.s
|
||
python3 tools/bench/prep_stream.py "$DLX"
|
||
mkdir -p tmp/snap_stream && rm -f tmp/snap_stream/x68000/*.png
|
||
( cd tmp && DLX_STREAM_KBPS=0 SDL_VIDEODRIVER=dummy stdbuf -oL \
|
||
timeout -k 5 600 mame x68000 -bios ipl10 -ramsize 2M -video soft -window \
|
||
-sound none -nothrottle -plugins -autoboot_script ../tools/bench/stream.lua \
|
||
-snapshot_directory ./snap_stream -snapview native -seconds_to_run 90 \
|
||
> stream_check.log 2>&1 )
|
||
python3 tools/bench/verify_decode.py "$DLX" --snap tmp/snap_stream
|
||
```
|
||
|
||
Env knobs: `DLX_RING_KB` (default 256), `DLX_STREAM_KBPS` (**REQUIRED, no
|
||
default**; `0` = unlimited, which isolates the WRAP question from the DELIVERY
|
||
one), `DLX_PREFILL_KB` (default 0).
|
||
|
||
`19_ring_stream.py`'s `--kbps` is required too, as are `--bus`/`--kbps` on
|
||
12/14/16/17. None of them has a default any more — see the retirement note at
|
||
the top of this document.
|
||
|
||
`DLX_STREAM_KBPS=0` is the right setting for a **correctness** gate: the wrap
|
||
policy is what is being tested and an unlimited pipe removes delivery as a
|
||
variable. Use a finite rate to get the `DEADLINE`/`REQUIRED PREFILL` lines.
|
||
|
||
The ring simulation, which needs no emulator and takes a second:
|
||
|
||
```
|
||
python3 tools/analysis/19_ring_stream.py # both containers
|
||
python3 tools/analysis/19_ring_stream.py <c.dlx> --kbps 512 --ring 128
|
||
```
|
||
|
||
It reports the **zero-prefill pipe** — the threshold a medium has to clear for
|
||
the container to need no prefill at all — which is the number to measure a
|
||
BlueSCSI against. It also warns explicitly when demand exceeds supply on the
|
||
MEAN, because a "required prefill" figure for a sustained overrun is the most
|
||
flattering possible way to state one.
|
||
|
||
**Do not run this alongside another MAME job.** They share `tmp/` logs and
|
||
snapshot directories and the second run silently truncates the first.
|
||
|
||
## Reproducing the packed player (session 32)
|
||
|
||
```
|
||
python3 tools/encoder/pack.py tmp/fr_singe tmp/packed_singe.dlxp --nframes 120
|
||
./tools/bench/packed_run.sh tmp/packed_singe.dlxp
|
||
```
|
||
~4 min, four MAME runs. `DLX_PK_GATE_ONLY=1` runs only the first, which is what
|
||
`check.sh` takes. Expected: 120 of 120 pixel-exact; 193 chain entries; 0 late
|
||
frames; 0 frames unsampled; a free-running transport of ~90 ms a record; and
|
||
**487 of 1,038 V-DISP edges lost in the held configuration**, which is the run's
|
||
whole point and is asserted rather than admired.
|
||
|
||
The palette-LAST container is a flag and not a re-encode:
|
||
```
|
||
python3 tools/encoder/pack.py tmp/fr_singe tmp/packed_singe_last.dlxp \
|
||
--nframes 120 --palette-last
|
||
./tools/bench/packed_run.sh tmp/packed_singe_last.dlxp
|
||
```
|
||
The chain assertion is read out of the container's own flags, so a run that
|
||
built the array the other way round fails instead of painting 192 rows of
|
||
picture into the palette registers.
|
||
|
||
And the arithmetic underneath the result, which needs no emulator:
|
||
```
|
||
python3 tools/analysis/31_display_duty.py tmp/packed_singe.dlxp \
|
||
--rate 582 700 1164 2131 3000
|
||
```
|
||
`--rate` is REQUIRED and has no default, for FINDINGS 50's reason.
|
||
|
||
**Do not run this alongside another MAME job.** It shares `tmp/` logs with every
|
||
other rig here.
|
||
|
||
## Reproducing the decoder result (session 7)
|
||
|
||
```
|
||
python3 tools/encoder/encode.py tmp/fr_singe tmp/rc_fr_singe_sasi_rcprofile.dlx --profile sasi
|
||
python3 tools/bench/prep_dlx.py tmp/rc_fr_singe_sasi_rcprofile.dlx
|
||
tools/vasm/vasmm68k_mot -Fbin -o tmp/decode.bin src/player/decode.s
|
||
mkdir -p tmp/snap_decode && cd tmp && SDL_VIDEODRIVER=dummy timeout -k 5 900 mame x68000 \
|
||
-bios ipl10 -ramsize 2M -video soft -window -sound none -nothrottle -plugins \
|
||
-autoboot_script ../tools/bench/decode.lua \
|
||
-snapshot_directory ./snap_decode -snapview native -seconds_to_run 150
|
||
cd .. && python3 tools/bench/verify_decode.py tmp/rc_fr_singe_sasi_rcprofile.dlx
|
||
```
|
||
~90 s wall. Prints cycles/frame and % of a 12fps budget for four real frames
|
||
spanning the non-SKIP distribution, four synthetic single-mode frames, and one
|
||
full 120-frame pass; then verifies the last frame is pixel-exact. Expected:
|
||
median 73.8%, p90 116.4%, max 135.8%, mean 81.7%; V1 299.9 / V4 448.2 / RAW
|
||
400.4 cycles per block.
|
||
|
||
`-ramsize 2M` here matters for a different reason than it used to. MAME defaults
|
||
to 4M and the locked *target* is a stock 2 MB machine, so a timing reproduction
|
||
quoting the target's memory should use it. But the rig preloads the whole
|
||
container, so at 2M this run is truncated to the prefix that fits — 37 of 120
|
||
frames on the current span-heavy container — and its real-frame anchors are
|
||
therefore a biased sample (session 16 measured the bias at 8.2% on the mean;
|
||
FINDINGS 45.3). The **synthetic** anchors are unaffected: they come out
|
||
bit-identical at 2M and 6M, which is what licenses `check.sh` running the
|
||
verify-only gate at `RIG_RAM=6`. For anchors over the whole window, run this
|
||
with `--ram 0x600000` and `-ramsize 6M`.
|
||
|
||
`DLX_VERIFY_ONLY=1` drops the timing anchors, which is how `check.sh` runs it.
|
||
|
||
Score a container against the measured costs without touching MAME:
|
||
```
|
||
python3 tools/analysis/11_cpu_budget.py tmp/rc_fr_singe_scsi_rcprofile.dlx
|
||
```
|
||
And re-demonstrate why there is only one display path (exits non-zero **by
|
||
design** — it is the counterexample):
|
||
```
|
||
python3 tools/analysis/10_pathmix_drift.py # 70/120 frames corrupt
|
||
python3 tools/analysis/10_pathmix_drift.py --fix direct # clean, and cheapest
|
||
```
|
||
|
||
## Reproducing the rate-control result (session 6)
|
||
|
||
```
|
||
python3 tools/encoder/extract.py 00223 tmp/fr_singe 12 crop 539.4 10.0
|
||
for prof in sasi scsi; do
|
||
python3 tools/encoder/encode.py tmp/fr_singe tmp/rc_$prof.dlx --profile $prof --fixed-lam
|
||
python3 tools/encoder/encode.py tmp/fr_singe tmp/rc_$prof.dlx --profile $prof
|
||
done
|
||
python3 tools/analysis/09_ratectl_drift.py # must exit 0, zero drifting frames
|
||
```
|
||
Expected, totals including the 7.8 KB/s audio allowance: `sasi` 137.4 -> 109.5
|
||
KB/s and 27.82 -> 27.22 dB; `scsi` 381.6 -> 280.0 KB/s and 30.81 -> 29.90 dB;
|
||
zero frames at the lam=800 cliff in either. ~55 s per encode, nearly all of it
|
||
k-means in `H.build`.
|
||
|
||
The block-mode map now renders the rate-controlled encoder by default:
|
||
```
|
||
python3 tools/analysis/08_mode_map.py tmp/fr_singe tmp/singe_modes_rc.webm \
|
||
--profile sasi --scale 2 # add --fixed-lam to compare
|
||
```
|
||
|
||
**Do not judge rate control on `tmp/fr_00020`.** It is 14 frames; the leaky
|
||
bucket's startup transient is bucket/nframes, so it lands 18% under target there
|
||
for reasons that have nothing to do with the content. FINDINGS 27.5.
|
||
|
||
## Reproducing the sustained-action result (session 5)
|
||
|
||
```
|
||
python3 tools/analysis/07_motion_survey.py 00223 10 # -> hottest window t=539.4s
|
||
python3 tools/encoder/extract.py 00223 tmp/fr_singe 12 crop 539.4 10.0
|
||
python3 tools/encoder/encode.py tmp/fr_singe tmp/singe_sasi.dlx --profile sasi
|
||
python3 tools/encoder/encode.py tmp/fr_singe tmp/singe_scsi.dlx --profile scsi
|
||
python3 tools/analysis/08_mode_map.py tmp/fr_singe tmp/singe_modes.webm \
|
||
--profile sasi --scale 2
|
||
```
|
||
`extract.py` now takes optional `[start_s] [dur_s]` — needed because 00223 is
|
||
9.4 min and the windows that stress the codec are seconds long.
|
||
|
||
`08_mode_map.py` renders palettised source | decoded | block-mode map at 12fps.
|
||
Output format follows the extension; **prefer `.webm`** — GIF re-quantises to
|
||
256 colours, which is a poor fit for output whose subject is colour fidelity,
|
||
and runs larger. It uses `yuv444p` because the mode map is flat saturated colour
|
||
on a 4-pixel grid and chroma subsampling smears exactly those edges.
|