drwily/Dragon-s-Lair-X68k
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Align the container to the disc, and find the decoder-free packed player fits

Browse Source 
Two sessions, unrecorded until now, committed together because their edits
share files and cannot be split cleanly after the fact.

Session 28 (FINDINGS 60): the container is DLX5 -- every record sector-aligned,
120/120 starting on a boundary where 3/120 did, +0.48% on the wire and zero
clocks -- and the ring's release rounds to RECALN so no pad is stranded.  Two
encoder levers measured and refused: `--spans all` buys +0.19 dB for +67% of
the wire, and joint span/lam selection emits byte-identical containers because
`lam` never leaves its floor on any of 120 frames.

Session 29 (FINDINGS 61): the packed full-frame blit is 27.3% of a 12 fps
frame, a channel fills GVRAM in buffer mode off the disc with the CPU halted,
and it walks the 1,024 B line stride itself through array chaining.  At the
9 clk/B dual-address floor the codec is 110.4% of a frame and a decoder-free
packed literal player is 55.2%, at +4.89 dB -- 2.75 dB past a ceiling the
codec's scene-wide palette cannot cross.  Encoder work is parked; the codec is
kept and not built on.

check.sh is ALL GREEN before and after, plus one new stage that gates the ORDER
of the measured paint costs rather than their values.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
This commit is contained in:
prosolis
2026-08-25 06:54:27 -07:00
parent 8800d8f8c0
commit 1be428c270
28 changed files with 2203 additions and 144 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/FINDINGS.md
+507
View File
@@ -5803,3 +5803,510 @@ cross-check at the top of the tool is what licenses every number under it.
**Re-deriving 59.7 against a lighter container needs that harness re-run first**,
and until it is, "34% too big" is a statement about the heaviest container and
not about the project.
---
## 60. The re-encode bundle: the container agrees with the medium, and two encoder levers turn out not to be levers (session 28)
**Emulated** — MAME 0.277, `x68000 -exp1 cz6bs1 -ramsize 2M/6M`, plus px68k's
C68K core in `tools/bench/c68k`. **No real hardware ran.** `./tools/bench/
check.sh` was ALL GREEN before this session and is ALL GREEN after it, including
a full re-encode of the gate container.
ROADMAP's re-encode bundle was four items collected under P2 because they shared
one re-measurement. All four are now closed, and **two of them close as
negatives** — which is the more useful half of the session.
### 60.1 DLX5: the container is laid out in sectors, and the disc now moves exactly the records
`tools/encoder/encode.py` pads every record up to **512 bytes** instead of 4,
and pads the scene header so the frame stream itself starts on a sector
boundary. `tools/encoder/dlx.py` reads it as `DLX5`; `rec_align` is a property
of the container version and `record_lengths()` is the one place the rule is
applied, replacing four hand-copied `4 + n + (-(4+n) % 4)` expressions in the
analysis tools.
**Why 4 was not enough** is 58.3 option C and 59.4: a SCSI target answers in
512 B blocks, 117 of 120 DLX4 records started part way into one, and
`sc_in_data` **refuses** a windowed read when the data phase belongs to the
DMAC, because a channel writes a contiguous run and cannot drop the 300 bytes in
front of a record. Windowed PIO absorbed that for free and does not survive the
move to the channel.
| on the gate container | DLX4 | **DLX5** |
|---|---:|---:|
| records starting on a sector boundary | 3/120 | **120/120** |
| bytes off the disc, for 4,488,577 B of record | 4,548,608 (+1.34%) | **4,510,208 (+0.48%)** |
| bytes into the ring | 4,488,588 | **4,510,208** |
| clocks of window or bounce copy | 0 (PIO only) / +5 clk/B (DMA) | **0, and a channel can run it** |
| largest record | 40,984 B | 41,472 B — a 256 KB ring still holds it 6 times |
**The disc figure and the ring figure are now the same number, and check.sh
gates on that identity** rather than on two constants. Both are read out of the
container at check time: the old gate had 4,488,588 and 4,548,608 written into
it as literals, and the re-encode went red on both — correctly, because the
container had changed and the expectation had not. **A gate whose expected value
is a literal tests the literal.**
### 60.2 The consumer had to be told, and the failure was a byte-exact wrong answer
`stream.s` released the ring up to the last byte it *read*, rounded to 4. Under
DLX5 that strands up to 511 bytes of pad per record, and the ring's own audit
caught it on frame 0: `RD_PTR MISMATCH: decoder released 00040F08, record ends
00041000`. The release now rounds to `RECALN` (geom.i), which is the record
alignment the container guarantees, and is correct only because the ring base is
`RECALN`-aligned too — stated where the constant is defined rather than assumed.
**This is the shape of the whole item.** Sector alignment is not a tidier
version of 4-byte alignment; it is a contract with *two* sides, and the producer
side alone would have drifted the free-space arithmetic by up to 511 B a frame
with every frame still decoding pixel-exact.
### 60.3 Reserved black at index 0 — the letterbox is true black, and it costs 0.04 dB
23.4, open since session 5, was the other half of P2. `VQ.scene_palette` now
quantises the picture into **255 entries and reserves index 0 as (0,0,0)**;
`dlxload.pack_palette` gives it `I = 0` by its own minimum-squared-error rule
with no special case, so 23.3's "the bars sit at RGB (4,4,4)" goes away for
free. GVRAM cleared to zero displays entry 0, and a free mediancut palette put a
real image colour there — on 00020 f0001, `(206,192,176)`, used by 210 image
pixels.
Black is **reserved, not withheld**: the mapper may still spend index 0 on
genuinely black pixels. What the reservation buys is that index 0 is black
whatever the scene contains, which is what the letterbox needs and what a free
palette cannot promise. Measured cost on the Singe window: **28.92 dB → 28.88 dB
at `--spans need`**, and the palette ceiling is unmoved at 31.32 dB.
### 60.4 `--spans all` as the default: MEASURED, and the recommendation is REFUSED
E2 has been "a recommendation, not a measurement" since 43.6.1. It is now a
measurement, on the DLX5 container, 120 frames, the gate recipe
(`--kbps 280 --span-kbps 488`):
| | KB/s incl. audio | PSNR | frames that miss the 12 fps deadline |
|---|---:|---:|---:|
| `--spans need` (shipped default) | **267.9** | 28.88 | 2/120 |
| `--spans all` | 448.2 | 29.07 | 1/120 |
**+67% of the wire, for +0.19 dB and one frame of 120.** Session 27 established
that the frame affords 6.69 clocks a byte and that a dual-address byte costs 9,
so the container's problem is that it is too big; spending 180 KB/s to buy a
fifth of a decibel is the wrong direction, and `need` stays the default.
**The item is closed, and the answer is no.** The gate container keeps
`--spans all` — it is a fixture chosen to exercise the newest path in the
decoder, not a shipping recipe, and check.sh says so.
### 60.5 Joint span/lam selection: implemented, measured, and it is a NO-OP for a reason
E3 (39.3 item 5) asked for span selection to be re-derived jointly with `lam`
instead of greedily after it. The argument is sound on paper: the span pass
*removes* the block payload of every block it covers, so the frame lands under
its byte allowance and the blocks that were **not** spanned were priced at a
`lam` chosen as if those bytes were still needed.
`ratectl._refit_joint` (`--joint-spans`) hands the freed bytes back to the lam
search and re-spans, to a fixed point or two rounds. A round is kept only if the
frame still fits both ceilings it was already fitting, so `lam` can only fall
and the un-spanned blocks can only improve.
**It emits byte-identical containers.** All four cells of `{need, all} x {greedy,
joint}` produce two md5s, not four — and so does `--rc-floor open`:
| | lam, 120 frames | container |
|---|---|---|
| `need` greedy / joint | 10.0 min, median, p90, max | **identical** |
| `all` greedy / joint | 10.0 min, median, p90, max | **identical** |
| `all --rc-floor open` greedy / joint | 1.0 min, median, p90, max | **identical** |
**The reason is 44.3, and it is structural rather than incidental: `lam` never
leaves its floor on any frame, at either floor the encoder offers.** The block
coder at the profile floor already lands under the per-frame byte budget, so
there is nothing for a joint re-search to spend the freed bytes on. E3 is not a
lever, and it is not a lever for the same reason `--kbps` and the bucket are
not: **the byte side of this encoder is governed by the span pass and by `mu`.**
The code stays, defaulted off, because the measurement is the finding and a
future container that moved `lam` off its floor would make the question live
again. That is 44's pattern kept deliberately: ask whether the lever is loaded
before pulling it.
### 60.6 The apparatus lied, silently, and only a byte comparison caught it
**MAME 0.277 served a compressed CHD's own file bytes as sector data.** On the
DLX5 volume, every `READ(10)` reported success and returned the wrong bytes: the
destination buffer after a 4,096 B read at LBA 0 was **byte-for-byte the first
4,096 bytes of `dlxdisk.chd`, starting `MComprHD`** — the CHD file header — while
`chdman verify` reported both SHA1s correct.
Isolated by experiment, and the trigger is the image's **content**:
| volume | compression | result |
|---|---|---|
| DLX4 gate, 8,768 sectors | default (lzma/zlib/huff/flac) | **byte-exact** |
| DLX5 gate, 8,809 sectors | default | CHD header served as LBA 0 |
| DLX5 gate, padded to 9,072 sectors, 16x63 geometry | default | same failure |
| DLX5 gate, **truncated to 8,768 sectors** — the working length | default | same failure |
| DLX5 gate | `-c zlib` alone | same failure |
| DLX5 gate | **`-c none`** | **byte-exact** |
So it is not size, not geometry and not the codec; compression decides it and
uncompressed is sound. **The MAME-side cause is not diagnosed** and is not
claimed here. `tools/bench/mkvol.sh` now builds the volume `-c none`, at 4.5 MB
in `tmp/` against 1.6 MB, with the measurement written next to the flag.
**What is worth keeping is not the workaround.** The failure was *silent at the
transport layer* — the SPC handshake completed, the phase sequence was correct,
the byte count was right, and `scsi_read` returned 0. The only thing in the tree
that could see it was `tools/bench/scsi.lua` comparing every delivered byte
against the host's copy of the same image. A transport gate that checked status
and length would have been green all session, and FINDINGS 58's "byte-exact
against the host's copy" would have been the load-bearing phrase nobody noticed.
### 60.7 What the bundle did to the budget: almost nothing, which is the result
The re-measurement the four items shared, on the rebuilt gate container, with
`tmp/c68k_frames.csv` regenerated by the same `check.sh` run:
| | before (DLX4, session 27) | **after (DLX5, session 28)** |
|---|---:|---:|
| decoder, MEASURED on C68K | 68.5% of the frame | **68.6%** |
| delivered bytes a frame | 37,403 | **37,585** (the pad is delivered, so it is charged) |
| headroom after decode + best-case audio | 6.74 clk/B | **6.69 clk/B** |
| single address held, `W`=5, total | 92.2% | **92.4%** |
| dual address held, `W`=9, total | 110.2% | **110.4%** |
| E7's byte target at the 9 clk/B floor | 328 KB/s, 34% too big | **327 KB/s, 35% too big** |
| B1's zero-prefill delivery requirement | 451.4 KB/s | **453.6 KB/s** |
| scene header (P1/53.5, 22_scene_load) | 6,164 B | **6,656 B** |
**Every conclusion in 59.7 survives unchanged**, which is what a precondition is
supposed to do: it removes an obstacle without moving the arithmetic. The two
numbers that did move are both the alignment pad being paid honestly —
`15_bus_occupancy.py` now charges the **padded** record rather than the payload,
because the disc carries the pad whether or not a frame contains it.
### 60.8 Two caches that would have tested the wrong artefact
`H.build` acquired an option this session (`reserve_black`), and two tools cache
its output in a pickle **keyed on the frames directory alone** —
`16_span_roundtrip.py`, which is the span container's round-trip gate, and
`13_cpu_ratectl.py`. A stale pickle would have let the gate round-trip a
container built from the old palette while the shipping encoder emitted the new
one: green, and testing an artefact that no longer exists. Both now store the
build parameters with the model and rebuild on a mismatch.
### 60.9 What is now true, and what P4a still owes
**M2's re-encode bundle is done and the container is what the DMA path needs.**
`dma_run.sh` still gates on the *refusal* — `WINDOWED DMA READ REFUSED, as it
must be` — which is now a negative control rather than a description of the
container, because the container no longer asks for a windowed read.
What is left of P4a is the wiring: putting the channel behind `ring.i`'s `XF_*`
mailbox in place of the PIO loop in `xfer.i`, which 59.4 blocked on exactly this
precondition. `xfer.i`'s sector arithmetic already degenerates correctly —
`SC_WSKIP` is 0 and `SC_WKEEP` is the whole record on every record — so what
changes is which loop moves the bytes, not what is asked for.
## 61. The decoder-free packed player fits the clocks the codec misses, and the codec cannot have the packing (session 29)
**Emulated.** MAME 0.277, `x68000 -bios ipl10`, and for the transport runs
`-exp1 cz6bs1` with a zero-filled `scsiexrom.bin`. No real hardware ran, and
47.4 — does buffer mode blank the layer — still wants a board.
FINDINGS 44.7 asked what a player with no codec at all would cost and answered
"it fits the clocks and dies on the medium". 46.5/47.1 found the off switch for
the 2:1 GVRAM tax, 47.2 built the packed layout and rendered it pixel-exactly on
both emulators, and **47.5 re-derived the budget on a cost model that has since
been replaced in full** — the transport was an unmeasured `c` when it was
written, and sessions 25b–28 measured it (58.2), bounded it (59.2) and priced
what a frame can afford against it (59.7/60.7). 47.6.1 filed the packed paint's
`movem` shape as an *assumption*; 47.6.2 said the DMAC "has not been near this";
47.6.4 left the codec's own survival under the layout untouched.
All four are answered here. `tools/analysis/29_packed_player.py` is the
arithmetic, `tools/bench/blit.s` V8/V9/V10 are the paint measurements, and
`src/player/dmagate.s` runs 4–6 are the transport ones.
### 61.1 The packed paint, MEASURED: it costs what the unpacked path pays to WRITE
`blit.s` V8 is V1 with one thing different — a row is 128 words instead of 256,
because R20 bit 11 lets a word carry two picture bytes — and `blit.lua` times it
in the same run as V1, V2 and V3, so it is quoted against numbers that have not
moved since session 9.
| variant | clk/frame | % of a 12 fps frame | what it moves |
|---|---:|---:|---|
| V1 unpacked `movem` blit | 446,286 | 53.6% | 96 KB read + 96 KB write |
| V2 byte-source expansion | 1,284,174 | 154.1% | 48 KB read + 96 KB write |
| V3 unpacked WRITE-ONLY floor | 225,789 | 27.1% | no source read at all |
| **V8 PACKED `movem` blit** | **227,553** | **27.3%** | **48 KB read + 48 KB write** |
All four put the same 49,152 pixels on screen. **V8 is 51.0% of V1 and 100.8% of
V3: the packed blit costs what the unpacked one pays to write alone, with its
source read thrown in free.** It is not exactly half of V1 because the 192-row
loop does not halve with the words — per word V1 is 9.080 clocks and V8 is
9.259, and the difference is the row loop amortised over half as many bursts.
V2 is worth keeping in view: it is the "send 1 byte a pixel and let the CPU
expand it" trade, and at 154.1% it is not a trade, which is why the unpacked
path has no cheap wire.
V8 is a **timing** variant and does not set bit 11. MAME's `gvram_w` carries no
timing in either arm, so the bit cannot move a cycle; what it moves is the
picture, and the picture is what `show_frame256_packed.lua` and `gvpack` already
verify pixel-exactly (47.2). The source is pre-interleaved host-side, which is
the honest half of the claim — the packing is an encoder-side transform, the
same argument 46.3 made for the text plane.
### 61.2 The channel writes GVRAM in buffer mode, and it walks the line stride itself
47.6.2 was right that nothing in this tree had ever pointed a channel at
`$C00000`. `dmagate.s` now does, three times, and `dma_run.sh` gates all three:
| run | R20 | result |
|---|---|---|
| **4. DMA → GVRAM, bus held, bit 11 SET** | `$0916` | **2,048 B byte-exact against the disc**; 1,024 landed in page 1 (high half), 1,024 in page 0 |
| **5. the SAME, bit 11 CLEAR** | `$0116` | **457 bytes lost, every one of them at an EVEN offset; not one ODD byte harmed** |
| **6. ARRAY CHAINED, 8 rows at the 1024 B stride** | `$0916` | **2,048 B byte-exact at eight separate row bases from ONE start**; `MAR` ended at `+7,424` |
In all three the discriminator reads the same as 59.1's: **`MTC` sampled by the
instruction after `START` is 0 of 2,048 and the CPU went round its wait loop
once** — the whole transfer happened between two instructions.
**Run 5 exists because the first cut of run 4 was a test that could not fail.**
It OR-ed bit 11 on top of whatever the IPL left, and the IPL leaves `$0B16`
(22.1) — bit 11 **already set**, and `COL = %11`, the 65,536-colour setup, which
writes whole words with or without it. It passed and proved nothing. Both runs
now write R20 outright and differ in exactly that bit.
**And the control's signature is a PLACE, not a count.** The obvious assertion —
"half the bytes must differ" — is wrong, and asserting it turned the gate red on
a correct run. In masked 256-colour mode `gvram_w` takes `data & 0x00ff` and
ignores `mem_mask`, so a byte written to an even address is never stored and the
high half keeps what it held; where the record is pad, the stale half matches the
disc by coincidence, and 567 of 1,024 did. What the mechanism says is that
**every odd byte survives and only even ones can be lost**, and that is what the
gate asserts.
Run 6 is the one that changes an architecture. A picture row is 256 B of a
1,024 B line stride, so a frame is **192 destinations, not one**, and a channel
writes a contiguous run — the same limitation that makes it refuse a windowed
read (58.3). 46.6 said "no stride for a DMAC to skip" about the bytes *within* a
row and never examined the rows. The MC68450 answers with sequential array
chaining, `src/player/dma.i` now programs it behind a `DM_BARV` mailbox that is
**zero unless a caller asks**, and the channel walked an 8-entry array on its
own. **The CPU does not restart the channel per row.**
### 61.3 The codec cannot have the packing — 47.6.4, closed
A 4×4 block owns four bytes at **stride 2** under the packed layout, because the
high bytes of its four words belong to the block 128 columns away. There are
exactly two ways a block decoder could live with that, and `blit.s` V9 and V10
are them:
| variant | clk/frame | % of frame | vs V4 |
|---|---:|---:|---:|
| V4 block order, UNPACKED (the shipping shape) | 637,971 | 76.6% | — |
| **V9 block order, PACKED, 16 `move.b` at stride 2** | **815,839** | **97.9%** | **+28%** |
| **V10 block order, PACKED, blocks PAIRED** | **320,456** | **38.5%** | **−50%** |
**V9 is 28% DEARER than V4, and buys nothing on the wire** — a codeword is
already one byte a pixel, so the packed layout has no bytes left to save a block
decoder. That route costs 177,868 clocks a frame for nothing.
**V10 halves the paint and pays for it in the mode map.** Pairing the block at
x with the block at x+128 gives V4's `movem` shape back, and a pair skips only
if *both* its blocks skip. On the gate container:
| | now | paired |
|---|---:|---:|
| SKIP | **66.3%** of blocks | **46.1%** of pairs |
| painted | **33.7%** | **53.9%** |
So pairing paints **1.60×** as many blocks for **0.50×** the paint per block:
**−20% on the clock, and about +60% on the BYTES**, because a coded block is
bytes in the container whether its half of the pair changed or not. E7 needs the
bytes **down 35%**.
**The packed layout is not an upgrade the codec can take. It is the thing you
get instead of the codec.**
### 61.4 The budget, re-derived on the measured model
Every cell below is CPU work plus transport plus best-case audio (10,417 clk,
1.25%, from the IPL ROM's own channel-3 setup — 21_iplrom_dmac.py). None of them
overlap: `buscost.DMA_OVERLAPS = False`.
| architecture | B/frame | W=5 | **W=9** | W=12 | W=16 | W=19 |
|---|---:|---:|---:|---:|---:|---:|
| CODEC, CPU-decoded (the shipping design) | 37,585 | 92.4% | **110.4%** | 124.0% | 142.0% | 155.5% |
| free / DMAC device→GVRAM / unpacked | 98,304 | 61.1% | 108.2% | 143.6% | 190.8% | 226.2% |
| **free / DMAC device→GVRAM / PACKED** | **49,152** | 31.6% | **55.2%** | 72.9% | 96.5% | 114.1% |
| free / CPU-painted / unpacked, 2 B/px wire | 98,304 | 113.8% | 161.0% | 196.4% | 243.5% | 278.9% |
| free / CPU-painted / unpacked, 1 B/px wire | 49,152 | 184.8% | 208.4% | 226.1% | 249.7% | 267.4% |
| **free / CPU-painted / PACKED** | **49,152** | 58.0% | **81.6%** | 99.3% | 122.9% | 140.6% |
**W=9 is the column that matters**, because 59.2 found the only configurations
this machine can be shown to run are dual-address, and a dual-address byte is a
4-clock read of the device plus a 5-clock write to memory. Everything left of it
is a hardware fact nobody here has (ROADMAP B3).
**At the floor the CODEC misses by 10.4% and the DECODER-FREE PACKED PLAYER FITS
WITH 45% TO SPARE.** Decoding 37,585 bytes costs 109% of a frame; not decoding
49,152 costs 54%.
That is not a small correction to 47.5. **44.7 said it in advance, on a cost
model that has since been thrown away: "the codec is not there to save CPU — it
is there to save the wire."** The measured model agrees and goes further: the
CPU side is not merely affordable without the codec, it is *strictly cheaper*.
### 61.5 So it is entirely a medium question, and the medium is the unmeasured one
| architecture | B/frame | KB/s | GB for 22.8 min |
|---|---:|---:|---:|
| CODEC, gate container | 37,585 | 440.4 | 0.62 |
| CODEC, `need` default | — | 267.9 | ~0.38 |
| CODEC, E7's target at the 9 clk/B floor | 27,924 | 327 | 0.46 |
| **decoder-free PACKED, either architecture** | **49,152** | **576.0** | **0.81** |
| decoder-free unpacked | 98,304 | 1,152.0 | 1.61 |
**576 KB/s, sustained, with no lever to pull.** A codec's bitrate is a lever; a
literal frame's is geometry, and no scene in the picture costs less than another.
ROADMAP B1 is unmeasured, and the 0.7–1.7 MB/s usually quoted for BlueSCSI on an
X68000 is **folklore with no published benchmark behind it**. 576 KB/s sits
inside that range, which is exactly the reason the range has to be measured
rather than cited: **a codec at 327 KB/s survives a slower answer and a literal
frame does not degrade, it drops.**
**B1 has therefore changed character.** It used to set how much headroom the
player has. It now decides which player exists.
### 61.6 And under MAME's blanking reading, the cheap architecture is the dark one
47.4's question is untouched by any of this and it still needs a board. What the
measured paint does is turn 48.3's range into numbers, and it exposes an
asymmetry nobody had stated: **R20 bit 11 only has to be set across the GVRAM
writes, and where those writes come from decides how long that is.**
| architecture | black interval | bit 11 set for |
|---|---:|---|
| free / DMAC device→GVRAM / unpacked | 60%..225% of a frame | the whole DMA |
| free / DMAC device→GVRAM / PACKED | 30%..113% of a frame | the whole DMA |
| free / CPU-painted / unpacked | 53.6% | the blit only |
| **free / CPU-painted / PACKED** | **27.3%** | **the blit only** |
The CPU-painted packed path has the smallest black window of any of them,
because its transport lands in RAM where bit 11 is irrelevant and only the
227,553-clock blit needs the bit. **The DMAC-direct path, which is cheaper in
clocks at every rung of the ladder, is the one that must hold the bit across its
whole transfer.** If MAME is right, the cheap architecture is the dark one and
the expensive one is merely dim.
Both are a strobe at the frame rate over the whole picture, and the packed layout
has no page to flip to: both 256-colour pages carry picture, which is the entire
point of it (48.3). If px68k is right, every number in 61.4 and 61.5 stands as
written. 48.1's prior — an assertion against a silence — has not moved.
### 61.7 What this does NOT establish
1. **No clock here is a transport clock.** MAME's DMAC runs on wall-clock
attotimes and models a held bus by halting the CPU (42.5), so runs 4–6 settle
which configurations *work* and not what one costs. Every `W` in 61.4 is
datasheet arithmetic except PIO, and the array-chain entry at 36 clocks is
Fig 4-25 sheet 1, not a measurement.
2. **One frame's worth of transport, not a stream.** Run 6 chained eight rows,
not 192, and nothing here ran a chained transfer back to back at 12 fps or
through `ring.i`. A decoder-free player has no ring at all in the DMAC-direct
form, which is a simplification this tree has not tested.
3. **The paint is a lower bound.** Every `blit.s` number is instruction cycles
against zero-wait-state memory; real GVRAM stalls the CPU and neither
emulator models it (`blit.lua`'s header, and 45's note on C68K).
4. **61.3's pairing cost is one container's.** 66.3% / 46.1% is the gate
container, deliberately the heaviest thing the encoder emits. A lighter
container has a different SKIP map and would pair differently — though not,
at that margin, differently enough to change the sign.
5. **No audio, no branching, no seek.** 61.4 charges the audio DMA and nothing
else. A decoder-free player still has to make 56's branch decisions and 51's
seeks, and at 576 KB/s it has less slack to make them in, not more.
### 61.8 Encoder work is parked (session 29, USER DECISION)
61.4 first closed with "nothing here is a reason to stop work on the codec — it
is the only branch that survives a slow answer to B1." **That does not survive
its own arithmetic and is withdrawn.**
It rested on 576 KB/s against **E7's 327 KB/s target, which does not exist.** The
codec that exists is **440 KB/s and 110.4% of a frame**, so the gap is **1.31x,
not 1.76x** — and reaching 327 needs a 35% byte reduction after 60.4 and 60.5
measured two of the encoder's three byte levers and found neither is a lever. The
reward on success is a design at ~100% of the frame with no margin, which is where
55.2% already is.
The two branches are not symmetric, and counting what each NEEDS is the argument:
| | facts it needs |
|---|---|
| **decoder-free packed** | buffer mode does not blank (B2); medium clears 576 KB/s (B1) |
| **codec** | E7 succeeds (unproven, two levers dead); **and** medium clears 327; **and** it ships at ~100% of a frame |
**So E7 and E4 are parked, and C1 with them.** E4 is included deliberately: it is
`H.build`'s k-means, it builds **VQ codebooks**, and a literal player has no VQ.
**What is NOT parked is the codec itself.** It stays on disk, gated by check.sh,
and nothing is built on it. 48.1's prior leans against packing — MAME asserts the
blanking semantic twice and deliberately, px68k's display path never reads the
bit at all, and an assertion against a silence is not a tie — and 48.3 stands:
if buffer mode blanks there is no version of the packed player that is merely
expensive. In that branch the codec is the only path left. **Keeping a working
decoder is inventory; building on it is work, and the work waits on B2.**
### 61.9 And the picture is BETTER — the codec is capped below the thing replacing it
Asked while scoping what a packed player would take, and it is the finding that
makes the branch worth building rather than merely affordable. The numbers are
`18_text_plane_16col.py`'s, over the same 120-frame window everything else is
measured on; 46.3 computed them to give the *text plane* something to be scored
against and never turned them on the 256-colour path itself.
| | PSNR vs the 24-bit source |
|---|---:|
| shipping container (the codec, as it ships) | **29.19 dB** |
| 256 colours, SCENE palette — **the codec's CEILING** | **31.33 dB** |
| 256 colours, **PER-FRAME** palette | **34.08 dB** |
**The middle row is a ceiling, not a rival.** Every codeword the codec emits is
an index *into* `vq.scene_palette`, so no bitrate takes it past 31.33 dB; it
spends 440 KB/s getting within 2.14 dB of it.
**A literal frame has no codebooks, so nothing forces the scene palette on it,
and per-frame palettes become legal.** 46.3 stated the constraint in as many
words while arguing the *other* side — "the 256-colour path cannot do this: its
palette is shared scene-wide because the codec's codebooks are indices INTO it."
Remove the codec and the constraint goes with it.
**So the decoder-free packed player is +4.89 dB on the shipping container and
+2.75 dB past a ceiling the codec cannot cross** — while costing 55.2% of a frame
against 110.4%. It is not a quality compromise bought with clocks. It is better
on both, and the whole of its cost is on the wire.
What the per-frame palette costs:
| | |
|---|---|
| wire | 512 B/frame → 49,664 B, **582.0 KB/s (+1.0%)** |
| clocks | ~2,370, **0.28% of a frame** if the CPU writes it — DERIVED from V8's measured 9.259 clk/word in the same `movem` shape |
| colours | **254, not 256** — the packed layout spends index 0 on the transparency key and puts black at 255 (47.2, `prep_frame.py --pack-transparent`), where `--reserve-black` spends one. A reserved entry measured 0.04 dB in 60.3, so this is noise against +4.89 |
**Two things this does NOT settle.**
1. **Whether a channel can write `$E82000`.** If the palette registers take a
byte-wide DMA the way GVRAM does in buffer mode, the palette is a 193rd array
entry and costs the CPU nothing at all — one channel start still paints a
whole frame. 61.2 only ever pointed a channel at GVRAM. **This is the next
probe, and it is the same shape as the ones that worked.**
2. **The quantiser is PIL's MEDIANCUT, not this project's.** `vq.scene_palette`
and `H.build` are what would actually ship the palette. The direction is
measured and the magnitude is about right; re-derive the per-frame figure
against the real builder before quoting it as the player's number.
docs/ROADMAP.md
+208 -52
View File
@@ -14,6 +14,24 @@ made sector-aligned records a precondition the transport enforces rather than a
preference (FINDINGS 59). **And 59.7 re-ranks what is left: the frame affords
6.74 clocks a byte, a dual-address byte costs 9, so B3 stopped being a constant
to look up and became the question of whether the design fits at all.**
Amended end of session 28: **THE RE-ENCODE BUNDLE IS DONE — all four items, one
re-measurement (FINDINGS 60).** The container is DLX5, sector-aligned, and the
disc now moves exactly the records. Two of the four closed as NEGATIVES: E2's
`--spans all` default is refused on measurement, and E3's joint span/lam
selection emits byte-identical containers because `lam` never leaves its floor.
**What is left of M2 is P4a's wiring** — the DMA channel behind `ring.i`'s
mailbox — and the budget did not move: headroom 6.74 -> 6.69 clk/B, so every
conclusion in 59.7 stands.
Amended end of session 29: **THE DECODER-FREE PACKED PLAYER IS BACK, MEASURED,
AND IT FITS THE CLOCK BUDGET THE CODEC MISSES (FINDINGS 61).** The packed
full-frame blit is **27.3%** of a 12 fps frame — measured, not assumed — a
channel fills GVRAM in buffer mode off the disc with the CPU halted, and it
walks the 1,024 B line stride itself through array chaining. At the 9 clk/B
floor the codec is 110.4% and a decoder-free packed player is **55.2%**. It asks
**576 KB/s, fixed**, against E7's 327 KB/s target. **So B1 stopped setting how
much headroom the player has and started deciding WHICH PLAYER EXISTS**, and B2
stopped being a nice-to-have. The codec cannot be packed: 47.6.4 is closed and
the answer is no, both ways (61.3).
**THE COMPLETION TARGET IS M3, THE VERTICAL SLICE** (USER DECISION): one scene
tree — a decision point, two outcomes, a death clip — with audio, streaming from
@@ -41,7 +59,7 @@ these units:
| **Delivery rate** | **no working figure, deliberately** (FINDINGS 50, USER DECISION). Every tool REQUIRES an explicit rate. |
| **Seek time** | **no figure at all, and never had one.** 51.3/51.4 made it matter. |
| **W, clocks stolen per delivered byte** | 5 single-address held, 9 dual held, 12 single arbitrated; the IPL ROM's own disk channel is **16..19** (52.5). **The largest open number in the project.** Session 27 added the row underneath it: with **no external request line** on the card (59.2) the channel is auto-requested and is charged **by time rather than by byte**, so at 460 KB/s a 50% bus share costs **10.61 clk/B** and a smaller share cannot carry the rate at all (59.3). |
| **The frame's headroom for a transport** | **6.74 clk/B** — 30.2% of a 12 fps frame, after the MEASURED decode (68.5%) and best-case audio (1.25%), at the gate container's 37,403 B record. **New in session 27, 59.7, and it is the number every row above is now read against.** A dual-address byte's floor is 9. |
| **The frame's headroom for a transport** | **6.69 clk/B** — 30.2% of a 12 fps frame, after the MEASURED decode (68.6%) and best-case audio (1.25%), at the DLX5 gate container's **37,585 B delivered record** (session 28: the sector pad is delivered, so `15_bus_occupancy.py` charges it). **It is the number every row above is read against**, and a dual-address byte's floor is 9. It was 6.74 against the DLX4 container; the bundle moved it by 0.05 and moved no conclusion. |
---
@@ -67,10 +85,18 @@ None of these block M2 or M3 software work, because session 18 forced every rate
to be an explicit argument. They set constants, and two of them decide how much
headroom the finished player has.
**B1. Measure the BlueSCSI — throughput AND seek time.**
**B1. Measure the BlueSCSI — throughput AND seek time.** **Session 29 gave this
a second acceptance test that is not a codec figure at all: 576 KB/s SUSTAINED,
which is what a decoder-free packed literal frame costs and cannot be talked down
from (FINDINGS 61.5).** A codec's bitrate is a lever; a literal frame's is
geometry. So the measurement now has three thresholds to be read against —
453.6 KB/s (the gate container needs no prefill), 327 KB/s (E7's target at the
dual-address floor) and 576 KB/s (no decoder at all) — and which of them the
medium clears decides which player gets built.
Throughput has an acceptance test already derived from real record sizes:
**513.2 KB/s** for the session-14 candidate, **451.4 KB/s** for the gate
container (`19_ring_stream.py`, FINDINGS 49.5). Seek time has nothing.
**513.2 KB/s** for the session-14 candidate, **453.6 KB/s** for the DLX5 gate
container (`19_ring_stream.py`, FINDINGS 49.5; the figure was 451.4 before
session 28's re-encode and the sector pad raised it). Seek time has nothing.
51.3/51.4 is why the second half matters: slack is *accumulated* out of
`pipe - wire`, so what a branch point costs is set by the rate and the time since
the last branch, not by the ring size. At 460 KB/s every ring from 192 KB to
@@ -83,6 +109,16 @@ and settles it in minutes on a real board. FINDINGS 48 shifted the prior toward
MAME and toward "unusable" — **do not pre-build on 1.0 B/pixel**. Same sitting:
the priority register `0xE82500` at `0x0000` (47.3).
> **Session 29 raised what this is worth, and gave it a number to be worth
> (FINDINGS 61).** It used to gate a derived halving. It now gates a player that
> has been measured to fit a budget the shipping design misses — 55.2% of a frame
> against 110.4% at the 9 clk/B floor. And 61.6 found an asymmetry worth carrying
> to the board: the black interval is the PAINT, not the frame, so a CPU-painted
> packed player is dark for **27.3%** of a frame while the cheaper DMAC-direct one
> is dark for **30..113%**. Under MAME's reading the cheap architecture is the
> dark one. **B2 and B1 are now the same decision from two sides, and B2 is the
> five-minute half.**
**B3. Single-address vs dual-address DMA — and now, first, DOES THE CARD DRIVE
`#EXREQ`?** 242 KB/s and 0.69 dB was the old framing. **Session 27 promoted this
to the item that decides whether the design fits at all (59.7).** The frame
@@ -105,9 +141,38 @@ is still the cheapest paper route to the same answer, because whatever Sharp's
own driver programs into DCR's DTYP is a statement about what the card supports.
FINDINGS 48.4. Not on this machine (checked, session 18).
**Ranking, stated because B1 used to outrank this:** B1 (throughput and seek)
sets how much headroom the finished player has. **B3 decides whether there is
any.** If the card drives `#EXREQ`, the ladder applies and the design fits with
**Ranking, amended session 29.** It was: B1 sets how much headroom the player
has, B3 decides whether there is any. FINDINGS 61 adds a third reading — **B1
and B2 together decide which player exists.** If the medium clears 576 KB/s
sustained and buffer mode does not blank, the decoder-free packed literal fits at
the dual-address floor B3 cannot get under, and B3 stops mattering for video at
all. If it does not, B3 is still the question. The three hardware facts are no longer
independent, and B2 is by far the cheapest of them.
> **ENCODER WORK IS PARKED — USER DECISION, session 29.** The first draft of this
> amendment said "nothing here is a reason to stop work on the codec". That does
> not survive its own arithmetic. It rested on comparing the packed player's
> 576 KB/s against **E7's 327 KB/s target, which does not exist**: the codec that
> exists is 440 KB/s and 110.4% of a frame, so the real gap is 1.31x, not 1.76x.
> And the branches are not symmetric. **Packed needs two facts** — buffer mode
> does not blank, medium clears 576 KB/s. **The codec needs E7 to succeed** —
> unproven, and 60.4/60.5 measured two of its three levers inert — **AND** the
> medium to clear 327, **AND** it ships at ~100% of the frame with no margin,
> which is where 55.2% is now.
>
> **E7 and E4 are both parked**, E4 included: `H.build`'s k-means builds VQ
> codebooks and a literal player has no VQ. C1 is gated by E4 and follows.
>
> **The codec is KEPT AND NOT BUILT ON.** That is inventory, not work. B2 is
> unanswered and 48.1's prior leans against packing — an assertion against a
> silence — and if buffer mode blanks there is no version of the packed player
> that is merely expensive (48.3), at which point the codec is the only path
> left. Keeping a working decoder on disk costs nothing; building on it costs
> sessions.
**The older ranking, which still holds inside the codec branch:** B1 (throughput
and seek) sets how much headroom the finished player has. **B3 decides whether
there is any.** If the card drives `#EXREQ`, the ladder applies and the design fits with
room. If it does not, the fallback is limited-rate auto-request at a share the
player picks (P4c), and the container has to come down to **328 KB/s of payload**
to fit at the 9 clk/B floor — 34% below where the gate container sits (59.7).
@@ -124,6 +189,43 @@ to fit at the 9 clk/B floor — 34% below where the gate container sits (59.7).
---
## The packed branch — what building it means (session 29, USER DECISION)
**This is where the work goes now.** FINDINGS 61: a decoder-free packed literal
player is **55.2% of a frame at the 9 clk/B dual-address floor** against the
codec's 110.4%, and **+4.89 dB** on the shipping container because a literal
frame is not tied to a scene palette the codec's codewords index into. It costs
**582 KB/s, fixed, with no lever.** Three items, in order.
- **K1. Can a DMA channel write the palette registers at `$E82000`?** (61.9.)
The cheapest item in the project and the one with the largest structural
payoff: if the palette takes a byte-wide DMA the way GVRAM does in buffer mode,
it becomes a **193rd array-chain entry** and one channel start paints a whole
frame — palette included, CPU halted throughout, **no per-frame CPU work in the
video path at all**. Same shape as 61.2's three runs: a run in `dmagate.s`, a
host comparison in `dma.lua`, an assertion in `dma_run.sh`, **and a negative
control.**
- **K2. A packed container.** Quantise to **254** colours (index 0 is the
transparency key, black at 255 — 47.2), interleave column *i* with *i+128*,
emit a per-frame palette, sector-align the record. Simpler than a coded one:
no VQ, no rate control, no mode map, no `lam`. `encode.py` already aligns
(DLX5) and `prep_frame.py --pack-transparent` already emits the layout.
**Re-derive 61.9's 34.08 dB against `vq.scene_palette` here** — the measured
figure is PIL's MEDIANCUT and is a direction, not the player's number.
- **K3. End to end, off the disc.** Palette, page-1 X-scroll 384, priority
`vc1 = 0x0002`, R20 bit 11; one chained DMA a frame. Gated pixel-exact over the
same 120-frame window the decoder is gated on.
**What K3 deletes, and why that is a risk and not a win to be banked:** a
DMAC-direct packed player has **no ring** — `ring.i`, `xfer.i` and most of
`stream.s` leave the video path, and **P4a's wiring is parked with them.** A
simplification that large usually hides something, and 61.7.2 names the specific
untested thing: a chained transfer has never run back to back at 12 fps.
**K1 and K2 survive a bad answer to B2. K3 does not.** Do K1 first.
---
## M2 — a player, as opposed to a decoder
`decode.s` draws pixel-exact frames from RAM Lua pre-loaded; `stream.s` decodes
@@ -145,32 +247,57 @@ Shipping the codebooks pre-expanded was considered and refused: it trades
9.26 ms of CPU for 5,120 more header bytes, which is a wash in milliseconds and
not a wash in kind (53.6).
**P2. Palette packing on the 68000. HALF DONE, session 21 — FINDINGS 53.**
~~**P2. Palette packing on the 68000.**~~ **DONE, session 28 — FINDINGS 60.**
The half that was open was the encoder's, and it closed with the whole bundle
below. Session 21 — FINDINGS 53 — did the packing itself:
~~The encoder still emits RGB888; the X68000 word packing is Lua-side.~~ The
packing is on the 68000: `pal_pack` writes 256 words straight into `$E82000`
with `I` chosen per entry by minimum squared error (**1.96 dB**, 23.3), gated on
the words read back **out of the palette registers**. 9.70 ms per scene, plus
5.29 ms of scene-independent table build hoisted to boot (53.3).
**What is left is the other half of the sentence: reserve index 0 as black with
`I = 0` (23.4).** That is ENCODER-side, it changes the container, and it moves
every constant fitted to the gate container, so it is a re-encode plus a
re-measurement rather than an edit.
~~**What is left is the other half of the sentence: reserve index 0 as black with
`I = 0` (23.4).**~~ **DONE, session 28 — 60.3.** `VQ.scene_palette` quantises
the picture into 255 entries and holds index 0 at (0,0,0); `pack_palette` gives
it `I = 0` by its own minimum-squared-error rule, so 23.3's "the bars sit at
RGB (4,4,4)" needed no special case. **0.04 dB** on the Singe window, palette
ceiling unmoved. Black is reserved, not withheld — the mapper may still spend
index 0 on genuinely black pixels; what it buys is that index 0 is black
whatever the scene contains.
**THE RE-ENCODE BUNDLE, collected here because it is now four items and they
share one re-measurement.** Nothing below is worth a container revision on its
own; together they are one:
1. reserve palette index 0 as black, `I = 0` (23.4, this item);
2. `--spans all` as the default (E2, and it is the loaded lever on the byte
side);
3. re-derive span selection jointly with `lam` (E3);
4. **sector-align every record (58.3; PROMOTED TO A PRECONDITION in session
27, 59.4 — `sc_in_data` refuses a windowed read under the DMAC, so this is
what the channel is waiting for)** — +0.43% on the
wire, zero clocks, and it is what lets P4a's DMA channel write straight
into the ring with no window and no bounce copy. Until then the letterbox gets the palette's
closest thing to black (index 255 on the gate container); `load.i` reports
whichever index that is and needs no change when it becomes 0.
~~**THE RE-ENCODE BUNDLE, collected here because it is now four items and they
share one re-measurement.**~~ **DONE, session 28 — FINDINGS 60. Two of the four
closed as NEGATIVES, which is the more useful half.** The four were:
1. ~~reserve palette index 0 as black, `I = 0`~~ **DONE** (23.4, 60.3);
2. ~~`--spans all` as the default~~ **MEASURED AND REFUSED** (E2, 60.4).
267.9 KB/s / 28.88 dB / 2 frames late at `need`, against 448.2 KB/s /
29.07 dB / 1 late at `all`: **+67% of the wire for +0.19 dB and one frame
of 120**, against a container the budget already says is 35% too big.
`need` stays the default. **It was called "the loaded lever on the byte
side" and it is — it is just loaded the wrong way**, and E7 is why. The
GATE container keeps `all`: it is a fixture, not a recipe;
3. ~~re-derive span selection jointly with `lam`~~ **IMPLEMENTED, MEASURED,
NO-OP** (E3, 60.5). `--joint-spans` hands the span pass's freed bytes back
to the lam search and re-spans; all four cells of `{need,all} x
{greedy,joint}` are BYTE-IDENTICAL, and so is `--rc-floor open`. **`lam`
never leaves its floor on any of 120 frames at either floor the encoder
offers** (44.3), so there is nothing to spend the freed bytes on. The code
stays, defaulted off, because a container that moved `lam` off its floor
would make the question live again;
4. ~~**sector-align every record**~~ **DONE — the container is DLX5** (58.3
option C, promoted to a precondition by 59.4; 60.1). Realised cost
**+0.48%** on the wire against the +0.43% predicted, zero clocks, and
**120/120 records start on a sector boundary** where 3/120 did. The disc
and the ring now move the SAME 4,510,208 B and check.sh gates on that
identity, both figures read out of the container instead of written into
the script — the old literals went red on the re-encode, correctly.
**The consumer had to be told too** (60.2): `stream.s` released the ring
to the last byte it READ, which strands up to 511 B of pad a record, and
the ring's own audit caught it on frame 0 while every frame still decoded
pixel-exact. The release rounds to `RECALN` now.
The letterbox no longer gets the palette's closest thing to black — item 1 put
true black at index 0, and `load.i` needed no change, as it said it would not.
~~**P3. A real frame clock.**~~ **DONE, session 22 — FINDINGS 54.**
`src/player/clock.i` derives the tick from the CRTC's own V-DISP through the
@@ -244,11 +371,15 @@ the ladder exactly one, dual address held, has a code path here, and it is the
one demonstrated. The slot pinout has `#EXREQ` at B36, so a real card plausibly
drives it — **that is now B3's sharpest form**.
**What is left of P4a is downstream of the container, not of the DMAC** (59.4):
`sc_in_data` **refuses** a windowed read when the data phase is the channel's,
because a channel writes a contiguous run and cannot drop the 300 B in front of
a record. So putting the channel behind `ring.i`'s mailbox waits on the
re-encode bundle.
~~**What is left of P4a is downstream of the container, not of the DMAC**
(59.4)~~ — **and that block is GONE as of session 28.** `sc_in_data` refused a
windowed read because a channel cannot drop the 300 B in front of a record; the
container is DLX5 now and no record asks for a window. `xfer.i`'s sector
arithmetic already degenerates correctly — `SC_WSKIP` is 0 and `SC_WKEEP` is the
whole record on every one of the 120 — so **what is left of P4a is the wiring:
which loop moves the bytes.** `dma_run.sh`'s windowed-read refusal stays as a
negative control rather than as a description of the container. **This is the
next item, and it is the last one before M2** (60.9).
**P4c (new, and it is a DESIGN CHOICE the tree had not named).** Auto-request is
charged **by time, not by byte** — the channel spends its share of the bus
@@ -305,7 +436,9 @@ container). The last one wins on both axes and is a **re-encode**; see the
bundle under P2. **P4a should be attempted against a sector-aligned container,
not against this one.** *(Session 27: it was, in the only sense that mattered —
the transport now REFUSES the windowed case rather than being trusted not to
reach it, so the bundle is a precondition rather than a plan. 59.4.)*
reach it, so the bundle is a precondition rather than a plan. 59.4. Session 28:
the container IS one — the realised wire cost is +0.48% against the +0.43%
predicted here, and 120/120 records start on a sector boundary. 60.1.)*
*(original item, still the standing description of the `W` question:)*
Drive the MB89352 instead of a host file. **Session 23
@@ -408,11 +541,14 @@ interaction to price next, and it is E2's question with a second consumer in it.
on P6's answer and on P5's index.
**E7. A BYTE TARGET, AND IT COMES OUT OF THE BUS BUDGET RATHER THAN OUT OF
TASTE (new, session 27, FINDINGS 59.7).** The frame affords **6.74 clocks a
byte** after the measured decode and the audio, and a dual-address byte costs
**9**. So *if* B3 comes back saying the card cannot drive `#EXREQ`, the container
has to reach **27,995 B a frame — 328 KB/s of payload** to fit at 12 fps, where
the gate container sits at 37,403 B and 438 KB/s: **34% too big.**
TASTE (new, session 27, FINDINGS 59.7; re-measured session 28, 60.7).** The
frame affords **6.69 clocks a byte** after the measured decode and the audio,
and a dual-address byte costs **9**. So *if* B3 comes back saying the card
cannot drive `#EXREQ`, the container has to reach **27,924 B a frame — 327 KB/s
of payload** to fit at 12 fps, where the DLX5 gate container delivers 37,585 B
and 440 KB/s: **35% too big.** (It was 6.74 / 328 / 34% against the DLX4
container. **The bundle moved the target by one KB/s and moved no conclusion**,
which is what a precondition is supposed to do.)
Three things make this less alarming than the number looks, and one makes it
worse:
@@ -465,15 +601,24 @@ Listed for completeness; past M3 these are scope, not risk.
not menu vs content: the two largest streams are bonus material and look like
content by size, duration and bitrate alike (25.1). Run
`07_motion_survey.py` per stream first for a hot-window shortlist.
**Gated by E4.**
- **E4. `H.build` k-means**, 51 s of a 55 s run, once per scene. The thing to
attack before C1, and not anything in the per-frame path (27.6).
- **E2. `--spans all` as default.** *(re-encode bundle item 2; see P2.)* Still a recommendation, not a measurement
(43.6.1), and the only loaded lever on the encoder's byte side (44.3). **It
spends every profitable byte, which raises `wire`, which shrinks `pipe - wire`,
which lengthens the refill climb after every branch.** That interaction is not
priced, and M3 is where it becomes measurable.
- **E3. Re-derive span selection jointly with `lam`** (39.3). *(bundle item 3.)*
**Gated by E4, and parked with it (session 29).**
- ~~**E4. `H.build` k-means**, 51 s of a 55 s run, once per scene.~~ **PARKED,
session 29 (USER DECISION).** It was the thing to attack before C1. It builds
**VQ codebooks**, and a decoder-free packed player has no VQ — so this is
encoder work on the branch that is no longer being built on. It comes back if
and only if B2 goes MAME's way. C1 is gated by it and is parked with it.
- ~~**E2. `--spans all` as default.**~~ **MEASURED AND REFUSED, session 28 —
60.4.** It was "a recommendation, not a measurement" since 43.6.1 and it is a
measurement now: **+67% of the wire for +0.19 dB and one frame of 120.** It
IS the loaded lever on the byte side (44.3) — it is loaded the wrong way, and
E7 is why. What the entry predicted is exactly what it does: it raises `wire`,
which shrinks `pipe - wire`, which lengthens the refill climb. `need` stays
the default; the GATE container keeps `all` because it is a fixture.
- ~~**E3. Re-derive span selection jointly with `lam`** (39.3).~~
**IMPLEMENTED, MEASURED, NO-OP, session 28 — 60.5.** `--joint-spans` emits
byte-identical containers in all four `{need,all} x {greedy,joint}` cells and
at both lam floors, because **`lam` never leaves its floor on any of 120
frames**. Kept and defaulted off.
- **C2. Framing** — crop vs squash vs wide (FINDINGS 12). Needs an eyeball
against arcade reference, not a measurement. Cheap; blocks only final encodes.
- **C3. Disk image packaging**, ~1.09 GiB at the candidate rate.
@@ -487,15 +632,26 @@ Listed for completeness; past M3 these are scope, not risk.
P4a DONE (59): the channel drives the data phase and
holds the bus -- 391.7% of a frame becomes 40..95%
│
B3 #EXREQ? ──┬─ YES ─> single address, 5 clk/B, 92.2% ── FITS ──┐
B3 #EXREQ? ──┬─ YES ─> single address, 5 clk/B, 92.4% ── FITS ──┐
│ │
└─ NO ──> auto-request, 9 clk/B FLOOR, 110.2% ──> E7 byte target
(the frame affords 6.74; 59.7) 328 KB/s
└─ NO ──> auto-request, 9 clk/B FLOOR, 110.4% ──> E7 byte target
(the frame affords 6.69; 59.7, 60.7) 327 KB/s
│
P2 re-encode bundle (sector alignment is now a PRECONDITION, 59.4) ┤
P1 P3 P4b P5 P7, P6 bus cost (52), G1 scene graph (56) ────────────┼─> M2 ─>
P2 re-encode bundle DONE (60): DLX5, records ARE sectors ─────────┤
E2 refused on measurement, E3 a no-op -- 60.4, 60.5 │
E7/E4/C1 PARKED session 29: encoder work waits on B2 ───────────┤
│
K1 palette-register DMA? ─> K2 packed container ─> K3 end to end ──┤
(61.9; K1/K2 survive a bad B2, K3 does not) │
P4a WIRING (the channel behind ring.i's mailbox) <- THE LAST ITEM ─┤
P1 P2 P3 P4b P5 P7, P6 bus cost (52), G1 scene graph (56) ────────┼─> M2 ─>
B1 seek+rate (sets HEADROOM, not fit) ─────────────────────────────┘ M3 ─> M4
B2 blanking ─> (page 1; do not pre-build on it)
B2 blanking ─┬─ NOT blanked ─> the DECODER-FREE PACKED player is live (61):
│ 55.2% of a frame at the 9 clk/B FLOOR, against
│ the codec's 110.4% -- and then B1 decides it,
│ because it wants 576 KB/s FIXED (61.4, 61.5)
└─ blanked ──────> a 27..113% duty strobe; there is no version of
it that is merely expensive (48.3, 61.6)
```
**Read that top-left branch as the project's live question.** Everything else
docs/STATUS.md
+258 -1
View File
@@ -1,4 +1,261 @@
# Status & next-session handoff — end of session 27 (2026-08-24)
# Status & next-session handoff — end of session 29 (2026-08-25)
## Session 29: the decoder-free packed player fits the budget the codec misses
**Green light first and last: `./tools/bench/check.sh` was ALL GREEN before any
of this and ALL GREEN after**, plus one new stage.
**FINDINGS 61.** **Emulated** — MAME 0.277, `x68000 -bios ipl10`, and
`-exp1 cz6bs1` for the transport runs. No real hardware ran. 47.4 — does buffer
mode blank the layer — is still the board question and is still open.
**THE HEADLINE.** At the 9 clk/B dual-address floor, which 59.2 established is
the only configuration this machine can be shown to run:
| architecture | B/frame | % of a 12 fps frame at W=9 | KB/s |
|---|---:|---:|---:|
| **CODEC, the shipping design** | 37,585 | **110.4% — MISSES** | 440.4 |
| **decoder-free PACKED, DMAC device→GVRAM** | 49,152 | **55.2% — FITS** | 576.0 |
| **decoder-free PACKED, CPU-painted** | 49,152 | **81.6% — FITS** | 576.0 |
**Decoding 37,585 bytes costs 109% of a frame; NOT decoding 49,152 costs 54%.**
44.7 said this in advance on a cost model that has since been thrown away — "the
codec is not there to save CPU, it is there to save the wire" — and the measured
model agrees and goes further. **The question is now entirely a medium question.**
**1. THE PACKED PAINT IS MEASURED, and 47.6.1's assumption is retired.**
`blit.s` V8 is V1 with 128 words a row instead of 256. **227,553 clocks, 27.3%
of a frame** — 51.0% of V1, and **100.8% of V3, the unpacked WRITE-ONLY floor**.
The packed blit costs what the unpacked path pays to write alone, source read
thrown in free. V1 came out at 53.6% again, unmoved since session 9, which is
what licenses quoting V8 next to it.
**2. A CHANNEL FILLS GVRAM OFF THE DISC, AND WALKS THE LINE STRIDE ITSELF.**
Three new runs in `dmagate.s`, all gated: bit 11 SET → 2,048 B byte-exact, half
into page 1 and half into page 0, CPU never executed; bit 11 CLEAR → **457 bytes
lost, every one at an EVEN offset and not one ODD byte harmed**; and **array
chained, 8 rows at the 1,024 B line stride from ONE start.** 47.6.2 is answered.
**3. THE FIRST CUT OF THAT TEST COULD NOT HAVE FAILED.** It OR-ed bit 11 onto
whatever the IPL left, and the IPL leaves `$0B16` — bit 11 already set, COL
`%11`, which writes whole words either way. It passed and proved nothing. Both
runs now write R20 outright and differ in exactly one bit.
**4. AND THE CONTROL'S SIGNATURE IS A PLACE, NOT A COUNT.** "Half the bytes must
differ" turned the gate RED on a correct run: `gvram_w` ignores `mem_mask` in the
masked arm, so the high half keeps stale GVRAM, and where the record is pad the
stale half matches by coincidence — 567 of 1,024 did. The claim the mechanism
actually makes is that odd bytes survive and only even ones are lost.
**5. THE CODEC CANNOT HAVE THE PACKING — 47.6.4 CLOSED, both ways.** A 4×4 block
owns four bytes at stride 2. Writing them one at a time (V9) is **97.9% of a
frame against V4's 76.6% — 28% DEARER, for no saving on the wire at all.**
Pairing the blocks 128 columns apart (V10) gets the `movem` back at 38.5%, and
pays in the mode map: **SKIP falls from 66.3% of blocks to 46.1% of pairs**, so
1.60× as many blocks are painted — about **+60% on the bytes**, against an E7
target that needs them **down 35%**.
**6. WHAT B1 AND B2 NOW DECIDE.** 576 KB/s sustained, fixed, with no lever —
against 453.6 KB/s (gate container, zero prefill), 327 KB/s (E7's target) and
folklore that says 0.7–1.7 MB/s with no benchmark behind it. And under MAME's
blanking reading the **cheap architecture is the dark one**: a CPU-painted packed
player is black for 27.3% of a frame, the cheaper DMAC-direct one for 30–113%,
because the black interval is the paint and the CPU's transport lands in RAM
where bit 11 is irrelevant.
**7. AND THE PICTURE IS BETTER, WHICH NOBODY HAD ASKED (61.9).** The codec is
**capped at 31.33 dB** — every codeword is an index into the scene palette — and
delivers 29.19. A literal frame has no codebooks, so per-frame palettes are legal
and it simply **IS 34.08 dB: +4.89 dB on the shipping container, +2.75 dB past a
ceiling the codec cannot cross**, for +1.0% on the wire and 0.28% of a frame.
**The packed player is not a quality compromise bought with clocks. It is better
on both, and its whole cost is on the wire.**
**ENCODER WORK IS PARKED (USER DECISION).** The handoff first said this was no
reason to stop codec work. It does not survive its own arithmetic: that rested on
576 KB/s against **E7's 327, which does not exist**, when the codec that exists is
440 KB/s and 110.4% of a frame — a 1.31x gap, not 1.76x. Packed needs two facts;
the codec needs E7 to succeed (two of three levers already measured inert, 60.4
and 60.5) AND the medium AND a ship at ~100% of the frame with no margin. **E7 and
E4 are both parked** — E4 too, because `H.build`'s k-means builds VQ codebooks and
a literal player has no VQ — and C1 follows E4. **The codec stays on disk and
nothing is built on it**, because 48.1's prior leans against packing and if
buffer mode blanks it is the only path left (48.3). That is inventory, not work.
---
## HANDOFF — start here
**THE TREE IS ALL GREEN** (`tmp/check_s29_handoff.log`, and four times before it this session), and **DIRTY WITH TWO
SESSIONS IN IT.** Session 28's work was never committed either, so `git status`
shows 26 modified files spanning both sessions plus one new file. The last
commit is session 27's (`8800d8f`, "Price the transport against the frame").
**Nothing here is half-finished — it is unrecorded, which is a different
problem.** Commit before starting anything, or the next green light cannot be
attributed to a session.
**THE DIRECTION CHANGED (USER DECISION, session 29): build the DECODER-FREE
PACKED player, keep the codec on disk and build nothing on it.** The reasoning is
61.8; the numbers that forced it are 61.4 (55.2% of a frame against the codec's
110.4%) and 61.9 (+4.89 dB, and the codec is *capped* 2.75 dB below it).
### The work, in the order it should be done
**1. CAN A DMA CHANNEL WRITE THE PALETTE REGISTERS AT `$E82000`? (61.9, open)**
The cheapest item and the one with the largest structural payoff. If the palette
takes a byte-wide DMA the way GVRAM does in buffer mode, it becomes a **193rd
array-chain entry** and ONE channel start paints a whole frame — palette
included, CPU halted throughout, no per-frame CPU work in the video path at all.
It is the same shape as the three probes that worked this session: another run in
`src/player/dmagate.s`, host-side byte comparison in `tools/bench/dma.lua`, an
assertion in `tools/bench/dma_run.sh`, **and a negative control** — see the traps
below, this is exactly where session 29 nearly shipped a vacuous pass.
**2. A PACKED CONTAINER.** Simpler than a coded one, not harder: quantise to 254
colours, interleave column *i* with column *i+128*, emit a per-frame palette,
sector-align the record. No VQ, no rate control, no mode map, no `lam`.
`encode.py` already sector-aligns (DLX5) and
`tools/bench/prep_frame.py --pack-transparent` already emits the exact pixel
layout 47.2 verified pixel-exactly on both emulators.
**3. END TO END, OFF THE DISC.** Scene setup — palette, page-1 X-scroll **384**,
priority `vc1 = 0x0002`, R20 bit 11 — then one chained DMA a frame. Gate it the
way the existing decoder is gated: pixel-exact over the same 120-frame window
against `dlx.py`'s reconstruction.
### What is PARKED, so it is not re-derived
**E7, E4 and C1** (61.8). E4 is included on purpose: `H.build`'s k-means builds
**VQ codebooks**, and a literal player has no VQ. **P4a's wiring** — the channel
behind `ring.i`'s `XF_*` mailbox — is parked *for the video path* too: a
DMAC-direct packed player has **no ring at all**. Do not delete any of it; B2 is
unanswered and 48.1's prior leans against packing.
### Risks that are OURS, not hardware
1. **A chained transfer has never run back to back at 12 fps.** 61.2's run 6 was
eight rows, once. A frame is 192 entries and a second is twelve frames.
2. **34.08 dB is PIL's MEDIANCUT**, not `vq.scene_palette` / `H.build`. The
direction is measured; re-derive the number against the builder that ships it
before quoting it as the player's.
3. **The packed player deletes `ring.i`, `xfer.i` and most of `stream.s` from the
video path.** A simplification that large usually hides something.
4. **B2 sits over all of it.** Items 1 and 2 survive a bad answer; item 3 does
not.
### Traps this session stepped in, both caught by the tree's own rules
**A test that could not fail.** The first GVRAM run OR-ed R20 bit 11 onto
whatever the IPL left — and the IPL leaves `$0B16`, bit 11 **already set**, COL
`%11`, which writes whole words either way. It passed and proved nothing. Both
runs now write R20 outright and differ in exactly one bit.
**An assertion that was right about the mechanism and wrong about the number.**
"The masked control must lose half the bytes" turned the gate RED on a correct
run: `gvram_w` ignores `mem_mask` in the masked arm, so the high half keeps stale
GVRAM, and where the record is pad it matches the disc by coincidence — 567 of
1,024 did. **The mechanism's signature was a PLACE, not a count**: odd bytes
survive, only even ones can be lost. That is what the gate asserts now.
### Reproducing this session
./tools/bench/check.sh # ALL GREEN, ~5 min
python3 tools/analysis/29_packed_player.py tmp/rc_fr_singe_scsi_span.dlx
bash tools/bench/dma_run.sh # runs 4-6, with controls
python3 tools/analysis/18_text_plane_16col.py # the PSNR rows of 61.9
`check.sh` gained one stage that runs `blit.lua` and gates the ORDER of the paint
costs, not their values — V8 < V1, V9 > V4, V10 < V4 — because a tree where any
of those flipped has a different answer to 44.7 and should say so out loud.
**WHAT IS NEXT.** Item 1 above: the palette-register DMA probe.
---
# Status & next-session handoff — end of session 28 (2026-08-25)
## Session 28: the container agrees with the medium, and two encoder levers are not levers
**Green light first and last: `./tools/bench/check.sh` was ALL GREEN before any
of this and ALL GREEN after** — and after a full re-encode of the gate
container, which is the first time this tree has replaced it.
**ROADMAP's RE-ENCODE BUNDLE (all four items under P2) is DONE. FINDINGS 60.**
**Emulated** — MAME 0.277 and px68k's C68K core. No real hardware ran.
**1. DLX5 — RECORDS ARE SECTORS.** `encode.py` pads every record to 512 B and
the scene header with it, so the frame stream starts on a sector boundary and
every record inside it does. `dlx.py` reads `DLX5`, and `record_lengths()` is
now the ONE place the alignment rule lives — it replaced four hand-copied
`4 + n + (-(4+n) % 4)` expressions in the analysis tools.
| gate container | DLX4 | **DLX5** |
|---|---:|---:|
| records starting on a sector boundary | 3/120 | **120/120** |
| off the disc, for 4,488,577 B of record | 4,548,608 (+1.34%) | **4,510,208 (+0.48%)** |
| into the ring | 4,488,588 | **4,510,208** |
| window or bounce copy | 0 PIO / +5 clk/B DMA | **0, and a channel can run it** |
The two byte counts are now the SAME NUMBER and check.sh gates on that
identity, both read out of the container rather than written into the script.
**2. THE CONSUMER HAD TO BE TOLD, and the ring's own audit caught it.**
`stream.s` released up to the last byte it READ, rounded to 4; under DLX5 that
strands up to 511 B of pad per record. `RD_PTR MISMATCH after frame 0` on the
first run. The release rounds to `RECALN` (geom.i) now. Every frame still
decoded pixel-exact while the free-space arithmetic was wrong, which is the
point: alignment is a contract with two sides.
**3. RESERVED BLACK (23.4, open since session 5).** The scene palette is 255
picture colours with index 0 held at (0,0,0), so the letterbox is true black and
`I = 0` falls out of `pack_palette`'s existing rule with no special case.
**0.04 dB** on the Singe window; the palette ceiling does not move.
**4. `--spans all` AS THE DEFAULT: MEASURED, AND REFUSED.** 267.9 KB/s / 28.88
dB / 2 frames late at `need`, against 448.2 KB/s / 29.07 dB / 1 late at `all`.
**+67% of the wire for +0.19 dB and one frame of 120**, against a container the
frame budget already says is 35% too big. `need` stays the default; the GATE
container keeps `all`, because it is a fixture and not a recipe.
**5. JOINT SPAN/LAM (E3): implemented, and it is a NO-OP.** `--joint-spans`
hands the span pass's freed bytes back to the lam search and re-spans. All four
cells of `{need,all} x {greedy,joint}` are BYTE-IDENTICAL, and so is
`--rc-floor open`. The reason is structural: **`lam` never leaves its floor on
any of 120 frames at either floor the encoder offers** (44.3), so there is
nothing to spend the freed bytes on. Kept, defaulted off.
**6. THE APPARATUS LIED SILENTLY.** MAME 0.277 served the compressed CHD's OWN
FILE BYTES as sector data — the buffer after READ(10) at LBA 0 was byte-for-byte
`dlxdisk.chd`'s first 4,096 bytes, starting `MComprHD` — while every READ
reported success and `chdman verify` passed. Not size, not geometry, not the
codec: the DLX5 image fails compressed at the working volume's exact length and
with a conventional geometry, and passes `-c none`. `mkvol.sh` builds
uncompressed now. **Only the per-byte comparison in `scsi.lua` could see it**; a
gate checking status and length would have been green all session.
**7. THE BUDGET BARELY MOVED, which is what a precondition should do.**
Decode 68.5% → **68.6%**, headroom 6.74 → **6.69 clk/B**, `W`=5 total 92.2% →
**92.4%**, E7's target 328 → **327 KB/s**, B1's zero-prefill requirement 451.4 →
**453.6 KB/s**. Every conclusion in 59.7 survives. `15_bus_occupancy.py` now
charges the **padded** record, because the disc carries the pad whether or not a
frame contains it.
**8. TWO CACHES WOULD HAVE TESTED THE WRONG ARTEFACT.** `16_span_roundtrip.py`
and `13_cpu_ratectl.py` cache `H.build` in a pickle keyed on the frames
directory alone; `H.build` gained an option this session. Both now store the
build parameters with the model and rebuild on a mismatch.
**WHAT IS NEXT.** P4a's wiring: the DMA channel behind `ring.i`'s `XF_*` mailbox
in place of `xfer.i`'s PIO loop, which 59.4 blocked on exactly this
precondition. `xfer.i`'s sector arithmetic already degenerates correctly —
`SC_WSKIP` is 0 on every record now — so what changes is which loop moves the
bytes. `dma_run.sh`'s windowed-read refusal becomes a negative control.
---
# Status at end of session 27 (2026-08-24)
## Session 27: the DMAC drives the data phase, and auto-request is charged by time