Align the container to the disc, and find the decoder-free packed player fits
Two sessions, unrecorded until now, committed together because their edits share files and cannot be split cleanly after the fact. Session 28 (FINDINGS 60): the container is DLX5 -- every record sector-aligned, 120/120 starting on a boundary where 3/120 did, +0.48% on the wire and zero clocks -- and the ring's release rounds to RECALN so no pad is stranded. Two encoder levers measured and refused: `--spans all` buys +0.19 dB for +67% of the wire, and joint span/lam selection emits byte-identical containers because `lam` never leaves its floor on any of 120 frames. Session 29 (FINDINGS 61): the packed full-frame blit is 27.3% of a 12 fps frame, a channel fills GVRAM in buffer mode off the disc with the CPU halted, and it walks the 1,024 B line stride itself through array chaining. At the 9 clk/B dual-address floor the codec is 110.4% of a frame and a decoder-free packed literal player is 55.2%, at +4.89 dB -- 2.75 dB past a ceiling the codec's scene-wide palette cannot cross. Encoder work is parked; the codec is kept and not built on. check.sh is ALL GREEN before and after, plus one new stage that gates the ORDER of the measured paint costs rather than their values. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
This commit is contained in:
prosolis
+258
-1
@@ -1,4 +1,261 @@
|
||||
# Status & next-session handoff — end of session 27 (2026-08-24)
|
||||
# Status & next-session handoff — end of session 29 (2026-08-25)
|
||||
|
||||
## Session 29: the decoder-free packed player fits the budget the codec misses
|
||||
|
||||
**Green light first and last: `./tools/bench/check.sh` was ALL GREEN before any
|
||||
of this and ALL GREEN after**, plus one new stage.
|
||||
|
||||
**FINDINGS 61.** **Emulated** — MAME 0.277, `x68000 -bios ipl10`, and
|
||||
`-exp1 cz6bs1` for the transport runs. No real hardware ran. 47.4 — does buffer
|
||||
mode blank the layer — is still the board question and is still open.
|
||||
|
||||
**THE HEADLINE.** At the 9 clk/B dual-address floor, which 59.2 established is
|
||||
the only configuration this machine can be shown to run:
|
||||
|
||||
| architecture | B/frame | % of a 12 fps frame at W=9 | KB/s |
|
||||
|---|---:|---:|---:|
|
||||
| **CODEC, the shipping design** | 37,585 | **110.4% — MISSES** | 440.4 |
|
||||
| **decoder-free PACKED, DMAC device→GVRAM** | 49,152 | **55.2% — FITS** | 576.0 |
|
||||
| **decoder-free PACKED, CPU-painted** | 49,152 | **81.6% — FITS** | 576.0 |
|
||||
|
||||
**Decoding 37,585 bytes costs 109% of a frame; NOT decoding 49,152 costs 54%.**
|
||||
44.7 said this in advance on a cost model that has since been thrown away — "the
|
||||
codec is not there to save CPU, it is there to save the wire" — and the measured
|
||||
model agrees and goes further. **The question is now entirely a medium question.**
|
||||
|
||||
**1. THE PACKED PAINT IS MEASURED, and 47.6.1's assumption is retired.**
|
||||
`blit.s` V8 is V1 with 128 words a row instead of 256. **227,553 clocks, 27.3%
|
||||
of a frame** — 51.0% of V1, and **100.8% of V3, the unpacked WRITE-ONLY floor**.
|
||||
The packed blit costs what the unpacked path pays to write alone, source read
|
||||
thrown in free. V1 came out at 53.6% again, unmoved since session 9, which is
|
||||
what licenses quoting V8 next to it.
|
||||
|
||||
**2. A CHANNEL FILLS GVRAM OFF THE DISC, AND WALKS THE LINE STRIDE ITSELF.**
|
||||
Three new runs in `dmagate.s`, all gated: bit 11 SET → 2,048 B byte-exact, half
|
||||
into page 1 and half into page 0, CPU never executed; bit 11 CLEAR → **457 bytes
|
||||
lost, every one at an EVEN offset and not one ODD byte harmed**; and **array
|
||||
chained, 8 rows at the 1,024 B line stride from ONE start.** 47.6.2 is answered.
|
||||
|
||||
**3. THE FIRST CUT OF THAT TEST COULD NOT HAVE FAILED.** It OR-ed bit 11 onto
|
||||
whatever the IPL left, and the IPL leaves `$0B16` — bit 11 already set, COL
|
||||
`%11`, which writes whole words either way. It passed and proved nothing. Both
|
||||
runs now write R20 outright and differ in exactly one bit.
|
||||
|
||||
**4. AND THE CONTROL'S SIGNATURE IS A PLACE, NOT A COUNT.** "Half the bytes must
|
||||
differ" turned the gate RED on a correct run: `gvram_w` ignores `mem_mask` in the
|
||||
masked arm, so the high half keeps stale GVRAM, and where the record is pad the
|
||||
stale half matches by coincidence — 567 of 1,024 did. The claim the mechanism
|
||||
actually makes is that odd bytes survive and only even ones are lost.
|
||||
|
||||
**5. THE CODEC CANNOT HAVE THE PACKING — 47.6.4 CLOSED, both ways.** A 4×4 block
|
||||
owns four bytes at stride 2. Writing them one at a time (V9) is **97.9% of a
|
||||
frame against V4's 76.6% — 28% DEARER, for no saving on the wire at all.**
|
||||
Pairing the blocks 128 columns apart (V10) gets the `movem` back at 38.5%, and
|
||||
pays in the mode map: **SKIP falls from 66.3% of blocks to 46.1% of pairs**, so
|
||||
1.60× as many blocks are painted — about **+60% on the bytes**, against an E7
|
||||
target that needs them **down 35%**.
|
||||
|
||||
**6. WHAT B1 AND B2 NOW DECIDE.** 576 KB/s sustained, fixed, with no lever —
|
||||
against 453.6 KB/s (gate container, zero prefill), 327 KB/s (E7's target) and
|
||||
folklore that says 0.7–1.7 MB/s with no benchmark behind it. And under MAME's
|
||||
blanking reading the **cheap architecture is the dark one**: a CPU-painted packed
|
||||
player is black for 27.3% of a frame, the cheaper DMAC-direct one for 30–113%,
|
||||
because the black interval is the paint and the CPU's transport lands in RAM
|
||||
where bit 11 is irrelevant.
|
||||
|
||||
**7. AND THE PICTURE IS BETTER, WHICH NOBODY HAD ASKED (61.9).** The codec is
|
||||
**capped at 31.33 dB** — every codeword is an index into the scene palette — and
|
||||
delivers 29.19. A literal frame has no codebooks, so per-frame palettes are legal
|
||||
and it simply **IS 34.08 dB: +4.89 dB on the shipping container, +2.75 dB past a
|
||||
ceiling the codec cannot cross**, for +1.0% on the wire and 0.28% of a frame.
|
||||
**The packed player is not a quality compromise bought with clocks. It is better
|
||||
on both, and its whole cost is on the wire.**
|
||||
|
||||
**ENCODER WORK IS PARKED (USER DECISION).** The handoff first said this was no
|
||||
reason to stop codec work. It does not survive its own arithmetic: that rested on
|
||||
576 KB/s against **E7's 327, which does not exist**, when the codec that exists is
|
||||
440 KB/s and 110.4% of a frame — a 1.31x gap, not 1.76x. Packed needs two facts;
|
||||
the codec needs E7 to succeed (two of three levers already measured inert, 60.4
|
||||
and 60.5) AND the medium AND a ship at ~100% of the frame with no margin. **E7 and
|
||||
E4 are both parked** — E4 too, because `H.build`'s k-means builds VQ codebooks and
|
||||
a literal player has no VQ — and C1 follows E4. **The codec stays on disk and
|
||||
nothing is built on it**, because 48.1's prior leans against packing and if
|
||||
buffer mode blanks it is the only path left (48.3). That is inventory, not work.
|
||||
|
||||
---
|
||||
|
||||
## HANDOFF — start here
|
||||
|
||||
**THE TREE IS ALL GREEN** (`tmp/check_s29_handoff.log`, and four times before it this session), and **DIRTY WITH TWO
|
||||
SESSIONS IN IT.** Session 28's work was never committed either, so `git status`
|
||||
shows 26 modified files spanning both sessions plus one new file. The last
|
||||
commit is session 27's (`8800d8f`, "Price the transport against the frame").
|
||||
**Nothing here is half-finished — it is unrecorded, which is a different
|
||||
problem.** Commit before starting anything, or the next green light cannot be
|
||||
attributed to a session.
|
||||
|
||||
**THE DIRECTION CHANGED (USER DECISION, session 29): build the DECODER-FREE
|
||||
PACKED player, keep the codec on disk and build nothing on it.** The reasoning is
|
||||
61.8; the numbers that forced it are 61.4 (55.2% of a frame against the codec's
|
||||
110.4%) and 61.9 (+4.89 dB, and the codec is *capped* 2.75 dB below it).
|
||||
|
||||
### The work, in the order it should be done
|
||||
|
||||
**1. CAN A DMA CHANNEL WRITE THE PALETTE REGISTERS AT `$E82000`? (61.9, open)**
|
||||
The cheapest item and the one with the largest structural payoff. If the palette
|
||||
takes a byte-wide DMA the way GVRAM does in buffer mode, it becomes a **193rd
|
||||
array-chain entry** and ONE channel start paints a whole frame — palette
|
||||
included, CPU halted throughout, no per-frame CPU work in the video path at all.
|
||||
It is the same shape as the three probes that worked this session: another run in
|
||||
`src/player/dmagate.s`, host-side byte comparison in `tools/bench/dma.lua`, an
|
||||
assertion in `tools/bench/dma_run.sh`, **and a negative control** — see the traps
|
||||
below, this is exactly where session 29 nearly shipped a vacuous pass.
|
||||
|
||||
**2. A PACKED CONTAINER.** Simpler than a coded one, not harder: quantise to 254
|
||||
colours, interleave column *i* with column *i+128*, emit a per-frame palette,
|
||||
sector-align the record. No VQ, no rate control, no mode map, no `lam`.
|
||||
`encode.py` already sector-aligns (DLX5) and
|
||||
`tools/bench/prep_frame.py --pack-transparent` already emits the exact pixel
|
||||
layout 47.2 verified pixel-exactly on both emulators.
|
||||
|
||||
**3. END TO END, OFF THE DISC.** Scene setup — palette, page-1 X-scroll **384**,
|
||||
priority `vc1 = 0x0002`, R20 bit 11 — then one chained DMA a frame. Gate it the
|
||||
way the existing decoder is gated: pixel-exact over the same 120-frame window
|
||||
against `dlx.py`'s reconstruction.
|
||||
|
||||
### What is PARKED, so it is not re-derived
|
||||
|
||||
**E7, E4 and C1** (61.8). E4 is included on purpose: `H.build`'s k-means builds
|
||||
**VQ codebooks**, and a literal player has no VQ. **P4a's wiring** — the channel
|
||||
behind `ring.i`'s `XF_*` mailbox — is parked *for the video path* too: a
|
||||
DMAC-direct packed player has **no ring at all**. Do not delete any of it; B2 is
|
||||
unanswered and 48.1's prior leans against packing.
|
||||
|
||||
### Risks that are OURS, not hardware
|
||||
|
||||
1. **A chained transfer has never run back to back at 12 fps.** 61.2's run 6 was
|
||||
eight rows, once. A frame is 192 entries and a second is twelve frames.
|
||||
2. **34.08 dB is PIL's MEDIANCUT**, not `vq.scene_palette` / `H.build`. The
|
||||
direction is measured; re-derive the number against the builder that ships it
|
||||
before quoting it as the player's.
|
||||
3. **The packed player deletes `ring.i`, `xfer.i` and most of `stream.s` from the
|
||||
video path.** A simplification that large usually hides something.
|
||||
4. **B2 sits over all of it.** Items 1 and 2 survive a bad answer; item 3 does
|
||||
not.
|
||||
|
||||
### Traps this session stepped in, both caught by the tree's own rules
|
||||
|
||||
**A test that could not fail.** The first GVRAM run OR-ed R20 bit 11 onto
|
||||
whatever the IPL left — and the IPL leaves `$0B16`, bit 11 **already set**, COL
|
||||
`%11`, which writes whole words either way. It passed and proved nothing. Both
|
||||
runs now write R20 outright and differ in exactly one bit.
|
||||
|
||||
**An assertion that was right about the mechanism and wrong about the number.**
|
||||
"The masked control must lose half the bytes" turned the gate RED on a correct
|
||||
run: `gvram_w` ignores `mem_mask` in the masked arm, so the high half keeps stale
|
||||
GVRAM, and where the record is pad it matches the disc by coincidence — 567 of
|
||||
1,024 did. **The mechanism's signature was a PLACE, not a count**: odd bytes
|
||||
survive, only even ones can be lost. That is what the gate asserts now.
|
||||
|
||||
### Reproducing this session
|
||||
|
||||
./tools/bench/check.sh # ALL GREEN, ~5 min
|
||||
python3 tools/analysis/29_packed_player.py tmp/rc_fr_singe_scsi_span.dlx
|
||||
bash tools/bench/dma_run.sh # runs 4-6, with controls
|
||||
python3 tools/analysis/18_text_plane_16col.py # the PSNR rows of 61.9
|
||||
|
||||
`check.sh` gained one stage that runs `blit.lua` and gates the ORDER of the paint
|
||||
costs, not their values — V8 < V1, V9 > V4, V10 < V4 — because a tree where any
|
||||
of those flipped has a different answer to 44.7 and should say so out loud.
|
||||
|
||||
**WHAT IS NEXT.** Item 1 above: the palette-register DMA probe.
|
||||
|
||||
|
||||
|
||||
---
|
||||
|
||||
# Status & next-session handoff — end of session 28 (2026-08-25)
|
||||
|
||||
## Session 28: the container agrees with the medium, and two encoder levers are not levers
|
||||
|
||||
**Green light first and last: `./tools/bench/check.sh` was ALL GREEN before any
|
||||
of this and ALL GREEN after** — and after a full re-encode of the gate
|
||||
container, which is the first time this tree has replaced it.
|
||||
|
||||
**ROADMAP's RE-ENCODE BUNDLE (all four items under P2) is DONE. FINDINGS 60.**
|
||||
**Emulated** — MAME 0.277 and px68k's C68K core. No real hardware ran.
|
||||
|
||||
**1. DLX5 — RECORDS ARE SECTORS.** `encode.py` pads every record to 512 B and
|
||||
the scene header with it, so the frame stream starts on a sector boundary and
|
||||
every record inside it does. `dlx.py` reads `DLX5`, and `record_lengths()` is
|
||||
now the ONE place the alignment rule lives — it replaced four hand-copied
|
||||
`4 + n + (-(4+n) % 4)` expressions in the analysis tools.
|
||||
|
||||
| gate container | DLX4 | **DLX5** |
|
||||
|---|---:|---:|
|
||||
| records starting on a sector boundary | 3/120 | **120/120** |
|
||||
| off the disc, for 4,488,577 B of record | 4,548,608 (+1.34%) | **4,510,208 (+0.48%)** |
|
||||
| into the ring | 4,488,588 | **4,510,208** |
|
||||
| window or bounce copy | 0 PIO / +5 clk/B DMA | **0, and a channel can run it** |
|
||||
|
||||
The two byte counts are now the SAME NUMBER and check.sh gates on that
|
||||
identity, both read out of the container rather than written into the script.
|
||||
|
||||
**2. THE CONSUMER HAD TO BE TOLD, and the ring's own audit caught it.**
|
||||
`stream.s` released up to the last byte it READ, rounded to 4; under DLX5 that
|
||||
strands up to 511 B of pad per record. `RD_PTR MISMATCH after frame 0` on the
|
||||
first run. The release rounds to `RECALN` (geom.i) now. Every frame still
|
||||
decoded pixel-exact while the free-space arithmetic was wrong, which is the
|
||||
point: alignment is a contract with two sides.
|
||||
|
||||
**3. RESERVED BLACK (23.4, open since session 5).** The scene palette is 255
|
||||
picture colours with index 0 held at (0,0,0), so the letterbox is true black and
|
||||
`I = 0` falls out of `pack_palette`'s existing rule with no special case.
|
||||
**0.04 dB** on the Singe window; the palette ceiling does not move.
|
||||
|
||||
**4. `--spans all` AS THE DEFAULT: MEASURED, AND REFUSED.** 267.9 KB/s / 28.88
|
||||
dB / 2 frames late at `need`, against 448.2 KB/s / 29.07 dB / 1 late at `all`.
|
||||
**+67% of the wire for +0.19 dB and one frame of 120**, against a container the
|
||||
frame budget already says is 35% too big. `need` stays the default; the GATE
|
||||
container keeps `all`, because it is a fixture and not a recipe.
|
||||
|
||||
**5. JOINT SPAN/LAM (E3): implemented, and it is a NO-OP.** `--joint-spans`
|
||||
hands the span pass's freed bytes back to the lam search and re-spans. All four
|
||||
cells of `{need,all} x {greedy,joint}` are BYTE-IDENTICAL, and so is
|
||||
`--rc-floor open`. The reason is structural: **`lam` never leaves its floor on
|
||||
any of 120 frames at either floor the encoder offers** (44.3), so there is
|
||||
nothing to spend the freed bytes on. Kept, defaulted off.
|
||||
|
||||
**6. THE APPARATUS LIED SILENTLY.** MAME 0.277 served the compressed CHD's OWN
|
||||
FILE BYTES as sector data — the buffer after READ(10) at LBA 0 was byte-for-byte
|
||||
`dlxdisk.chd`'s first 4,096 bytes, starting `MComprHD` — while every READ
|
||||
reported success and `chdman verify` passed. Not size, not geometry, not the
|
||||
codec: the DLX5 image fails compressed at the working volume's exact length and
|
||||
with a conventional geometry, and passes `-c none`. `mkvol.sh` builds
|
||||
uncompressed now. **Only the per-byte comparison in `scsi.lua` could see it**; a
|
||||
gate checking status and length would have been green all session.
|
||||
|
||||
**7. THE BUDGET BARELY MOVED, which is what a precondition should do.**
|
||||
Decode 68.5% → **68.6%**, headroom 6.74 → **6.69 clk/B**, `W`=5 total 92.2% →
|
||||
**92.4%**, E7's target 328 → **327 KB/s**, B1's zero-prefill requirement 451.4 →
|
||||
**453.6 KB/s**. Every conclusion in 59.7 survives. `15_bus_occupancy.py` now
|
||||
charges the **padded** record, because the disc carries the pad whether or not a
|
||||
frame contains it.
|
||||
|
||||
**8. TWO CACHES WOULD HAVE TESTED THE WRONG ARTEFACT.** `16_span_roundtrip.py`
|
||||
and `13_cpu_ratectl.py` cache `H.build` in a pickle keyed on the frames
|
||||
directory alone; `H.build` gained an option this session. Both now store the
|
||||
build parameters with the model and rebuild on a mismatch.
|
||||
|
||||
**WHAT IS NEXT.** P4a's wiring: the DMA channel behind `ring.i`'s `XF_*` mailbox
|
||||
in place of `xfer.i`'s PIO loop, which 59.4 blocked on exactly this
|
||||
precondition. `xfer.i`'s sector arithmetic already degenerates correctly —
|
||||
`SC_WSKIP` is 0 on every record now — so what changes is which loop moves the
|
||||
bytes. `dma_run.sh`'s windowed-read refusal becomes a negative control.
|
||||
|
||||
---
|
||||
|
||||
# Status at end of session 27 (2026-08-24)
|
||||
|
||||
## Session 27: the DMAC drives the data phase, and auto-request is charged by time
|
||||
|
||||
|
||||
Reference in New Issue
Block a user