ROADMAP P6c, FINDINGS 68. 78,125 B of a DLXP2's audio out of channel 3, sample-exact, while the video channel fetches records off the same disc. The two pieces 67.6 said were missing: the lump buffer (pg_afill/pg_afetch, three slots and the minimum is unmeasured) and 67.2's remainder accumulator (pg_apay). The capture prices what the accumulator avoided at 1.26 s of lip-sync over the game, against 67.2's predicted 1.25. The finding is the third piece, which nothing had named: the MSM6258 has no starvation state, so the gap between a channel counting out and the next arm is a held nibble pair driving the predictor. Stealing, the seam is 0.51 ms over ten seconds because dma.i's new DM_HOOK services the chip from inside the transfer wait -- 250,000 of 250,240 looks. Held, the 68000 is halted and gets 369: every one of the ten lump boundaries has a seam, worst 72.8 ms, 2.31% of the audio. Identical bytes, different sound. 64.3 reaching the audio. Two bugs, and no counter in the player could see either. Clearing DM_BARV does not unchain a channel -- OCR bits 3-2 are what it obeys -- and the symptom is POLL TIMEOUT on the lump and every record after it. And the refill ran one lump ahead of its ring and overwrote the buffer the channel was reading: 11 of 11 armed, 11 fetched, no starve, and the sound wrong from 0.2 s in. Which is why the gate is a WAV: verify_packed_audio.py walks the stream one delivered byte at a time, because MAME's okim6258 resets the nibble select on every write and a byte is two nibbles only 99.994% of the time. check.sh ALL GREEN before (tmp/check_s36_start.log) and after (tmp/check_s36_end.log), with the new stage. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
7371 lines
395 KiB
Markdown
7371 lines
395 KiB
Markdown
# Findings — session 1 (2026-08-23)
|
||
|
||
All numbers here are MEASURED unless marked ESTIMATE or FOLKLORE.
|
||
|
||
---
|
||
|
||
|
||
> **THE DELIVERY RATE HAS NO WORKING FIGURE — retired session 18 (USER
|
||
> DECISION).** Sections below written before session 18 name a "4 Mbps" pipe
|
||
> constant and score tables against it. **Read every one of those as history.**
|
||
> It was never a bus measurement: user-supplied, no provenance, 10% of SCSI-1's
|
||
> asynchronous rating (FINDINGS 42.1), and FINDINGS 49.5 caught the shipping
|
||
> candidate exceeding it while nothing in the tree was comparing the two.
|
||
>
|
||
> It is now gone as a default from every analysis tool and from
|
||
> `tools/bench/stream.lua` — `--bus` / `--kbps` / `DLX_STREAM_KBPS` are
|
||
> **required arguments** with no fallback, so no table can be scored against a
|
||
> rate its own output does not state. The one survivor is `GATE_SPAN_KBPS` in
|
||
> `tools/bench/check.sh`, which is a **container recipe**, not a delivery claim:
|
||
> the gate container was encoded with it and every per-block and span constant
|
||
> in FINDINGS 41/43/45/49 is fitted to that container, so changing it is a
|
||
> re-encode plus a re-measurement, not an edit.
|
||
>
|
||
> **What to use instead:** `tools/analysis/19_ring_stream.py` reports the
|
||
> **zero-prefill pipe** — the rate a medium must clear for a container to need no
|
||
> prefill. That is a requirement to measure a BlueSCSI against, not a constant to
|
||
> design on. For the session-14 candidate it is **513.2 KB/s**.
|
||
|
||
## 1. Source material
|
||
|
||
`DRAGONS_LAIR.iso` — 16 GB, UDF 2.x, **decrypted** (no AACS dir).
|
||
Loop-mounted read-only at `/media/reala-misaki/BDROM` via `udisksctl loop-setup -r -f`.
|
||
(7-Zip cannot read UDF 2.x; use the loop mount.)
|
||
|
||
- **224 `.m2ts` streams**, 1920x1080, **MPEG-2, progressive, 23.976 fps**
|
||
- Size histogram: 47 <5MB, 138 5-50MB, 22 50-150MB, 14 150-400MB, 3 >400MB
|
||
- The 185 sub-50MB streams are the **arcade branching scenes already split into
|
||
individual clips** — we get scene boundaries for free.
|
||
- Big streams are full-feature playthroughs: 00215 (1376s), 00216 (1151s), 00223 (566s)
|
||
- Typical scene clip ~60s (00203/00205/00199), some ~100s (00164/00212)
|
||
|
||
**Gotcha:** clip durations vary wildly. Always read `format=duration` and seek
|
||
relative to it. Seeking to a fixed offset silently yields 0 frames on short clips.
|
||
|
||
---
|
||
|
||
## 2. GVRAM layout [verified — see HARDWARE.md for source]
|
||
|
||
**One 16-bit word per pixel position in EVERY color mode.** Bit depth does not
|
||
change VRAM bandwidth; it only subdivides the word.
|
||
|
||
`addr = page_base + y*1024 + x*2` — adjacent pixels are 2 bytes apart in all modes.
|
||
|
||
Consequence: low bit depth buys **no speed**. 16-color mode is strictly worse than
|
||
256-color (same bus traffic, 1/16 the palette). Page-alias writes are hardware
|
||
auto-masked, so 16-color needs no software read-modify-write — but it's still
|
||
one word-access per pixel.
|
||
|
||
**Chosen: 256 colors, 256x192 active area.**
|
||
In 256-color mode P0=low byte, P1=high byte of each word. Sacrificing page 1 as a
|
||
double-buffer lets a `move.l` cover two pixel positions, enabling `movem.l` bursts
|
||
(12 regs = 48 bytes = 24 pixels). Identical blit cost to 65536-color mode but
|
||
**half the on-disk data**.
|
||
|
||
---
|
||
|
||
## 3. Content measurements (8 scenes sampled, 5s each at 40% into each clip)
|
||
|
||
| metric | mean | p90 |
|
||
|---|---|---|
|
||
| pixels changed / frame | 20.1% | 30.2% |
|
||
| **blit cost** | **~64k cycles** | **~97k cycles** |
|
||
| naive delta+RLE frame size | 15.5 KB | 19.6 KB |
|
||
|
||
Budget is **833,333 cycles/frame** @ 12fps on a 10MHz 68000.
|
||
|
||
### => THE CPU IS NOT THE BOTTLENECK. I/O IS.
|
||
Blit uses **under 8%** of budget. The naive row-span+RLE codec achieves only
|
||
**3.2:1**, giving **365 KB/s / 470 MB** at 24fps (~183 KB/s / 235 MB at 12fps).
|
||
|
||
Per-scene variance is extreme: static dialogue ~30 KB/s, action ~700 KB/s.
|
||
Any codec needs a hard bitrate ceiling, not just a good average.
|
||
|
||
### "Shot on twos" — ASSUMPTION FAILED
|
||
Dedupe found **zero** duplicate frames across all 8 scenes (`uniq=120/120`,
|
||
24.0 fps effective). This Blu-ray is a restoration where every frame is unique.
|
||
We do NOT get halved data for free. **Decimation to 12fps must be explicit.**
|
||
|
||
A weak alternation signature does exist (even-index pairs 40.7% vs odd 27.5%,
|
||
ratio 1.5x, with occasional true-duplicate pairs at 0.03-0.19%), but it is
|
||
irregular — Bluth mixed ones and twos; action is animated on ones.
|
||
|
||
---
|
||
|
||
## 4. MEASUREMENT TRAPS — read before trusting any pipeline number
|
||
|
||
Three separate false results were produced and caught this session. All three
|
||
looked plausible. Guard against them:
|
||
|
||
1. **Per-frame Floyd-Steinberg dithering destroys temporal coherence.**
|
||
Error diffusion is chaotic: a +/-1 input change cascades across the row and
|
||
produces a completely different index pattern. First run reported 31.5% pixels
|
||
changed with near-zero variance (median 31.6, p90 32.3, max 32.7) while source
|
||
mean-abs-diff was 0.09 — i.e. visually identical frames. That flat variance is
|
||
the tell: **real animation has scene-dependent variance; noise does not.**
|
||
Use no dithering (cel art is flat) or ordered/Bayer (spatially fixed, temporally stable).
|
||
|
||
2. **Temporal denoise smears motion.** `hqdn3d=4:3:6:4` — the `6:4` are temporal
|
||
params. It flattened real motion, which then measured as "no motion" and
|
||
produced an absurd 0.8 fps / 4 MB result. **Use spatial-only: `hqdn3d=4:3:0:0`.**
|
||
|
||
3. **Exact-match dedupe fails on a noisy source.** MPEG-2 grain means near-duplicate
|
||
frames differ by +/-1 and are never bit-exact. Use a threshold on
|
||
"% pixels differing by more than N levels", and pick the threshold from the
|
||
observed distribution, not a guess. A 2% threshold ate genuine animation when
|
||
mean consecutive change was only 0.9%.
|
||
|
||
**Sanity rule: if a result has suspiciously low variance, or is suspiciously
|
||
good, it is probably an artifact of the measurement, not a property of the content.**
|
||
|
||
Scripts kept in `tools/analysis/` — 01 and 02 are marked BROKEN deliberately as
|
||
regression references; 03 and 04 are the correct ones.
|
||
|
||
---
|
||
|
||
## 5. Storage interface — the SASI/SCSI split
|
||
|
||
> **SUPERSEDED IN PART.** The claim below that DMA means streaming "costs
|
||
> essentially no CPU" is **wrong** — see 19. The bandwidth figures here are
|
||
> folklore; the working figure was **4 Mbps = 488 KB/s** (21), and **42.1
|
||
> retires that too**: it was never a bus figure. The 50-pin SCSI-1 5 MB/s below
|
||
> is the correct bus rating. What binds is not the pipe but `W`, the clocks the
|
||
> DMA steals per word — and this section's 8 clk/word is a bracket midpoint,
|
||
> not a measurement (39.7, 42.6).
|
||
|
||
[Yasuma, X68030 internal SCSI controller]
|
||
|
||
- Interface: **SCSI-1**, 50-pin, 5 MB/s bus spec
|
||
- Controller: **Fujitsu MB89352** SPC
|
||
- Transfer mode: **DMA** (via **HD63450** DMAC)
|
||
- Bus: X68000 original bus, **16-bit @ 10MHz**
|
||
|
||
**Even on the X68030, SCSI runs at 10MHz 16-bit DMA.** Storage bandwidth does
|
||
NOT scale with CPU — the controller sits on the original bus. HD63450's 12.5MHz
|
||
official ceiling is why the X68030 runs at 25MHz. An "HSCSI" TSR forces PIO/FIFO
|
||
transfer instead of DMA but was marginal even at 25MHz.
|
||
|
||
Because it's DMA, **streaming costs essentially no CPU** — this stacks with the
|
||
8% blit utilisation. The 68000 really is nearly idle.
|
||
|
||
### Model split — IMPORTANT
|
||
**The 10MHz models (original X68000, ACE, PRO, EXPERT) use SASI, not SCSI.**
|
||
Built-in SCSI starts at the X68000 **Super** (1990) and continues through XVI,
|
||
Compact, X68030. SCSI on earlier machines needs the **Sharp CZ-6BS1** board
|
||
in an I/O slot (MAME models this: `-exp1 cz6bs1`).
|
||
|
||
| target | bandwidth | naive codec (365 KB/s) | VQ codec (~30 KB/s) |
|
||
|---|---|---|---|
|
||
| SASI (stock ACE/EXPERT) | ~300-500 KB/s FOLKLORE | infeasible | comfortable |
|
||
| SCSI (Super+, or CZ-6BS1) | ~1 MB/s FOLKLORE | tight but viable | trivial |
|
||
|
||
Derived bounds (ESTIMATE): 16-bit @10MHz with 4-clock bus cycle = 5 MB/s absolute
|
||
ceiling; HD63450 single-address DMA ~8 clocks/word => ~2.5 MB/s practical ceiling,
|
||
before SCSI-1 async handshake and drive latency.
|
||
|
||
**No measured benchmark was obtained — see STATUS.md.** The ~300-500 KB/s and
|
||
~1 MB/s figures are folklore-grade; I could not find a primary measurement.
|
||
|
||
---
|
||
|
||
## 6. Codec decision: vector quantization (Cinepak-style)
|
||
|
||
Given ~8x CPU headroom and an I/O ceiling, spend CPU to buy bandwidth.
|
||
|
||
- Split frame into 4x4 blocks, encode each as a 1-byte index into a per-scene codebook
|
||
- Decode = 16-byte copy from a lookup table: nearly free
|
||
- A **full** frame = 256*192/16 = **3,072 bytes** — a hard 16:1 floor before delta
|
||
- Add block-level delta on top; action scenes ~2-3 KB/frame
|
||
- => roughly **30 KB/s, ~40 MB total**, with a *deterministic* bitrate ceiling
|
||
|
||
Divergence from the SNES project (below): use a **per-scene codebook with delta
|
||
updates**, not a per-frame rebuild. We trade adaptivity for bandwidth because we
|
||
have 2MB RAM to keep a codebook resident and CPU to spare.
|
||
|
||
**Risk not yet evaluated:** 4x4 VQ with a 256-entry codebook will visibly soften
|
||
detail. Bluth's fine ink linework is what suffers. Prototype and eyeball before committing.
|
||
|
||
---
|
||
|
||
## 7. Comparison: astrobleem/SNES-SuperDragonsLairArcade
|
||
|
||
Reached the **same core architecture independently** — "512 tiles per frame" is
|
||
vector quantization (8x8 codebook + tilemap). Good validation.
|
||
|
||
But: the SNES PPU has **no bitmap mode**, so tiles are forced on them by display
|
||
hardware. The X68000 has a real linear framebuffer, so VQ is a *compression
|
||
choice* we can tune or drop per-scene.
|
||
|
||
**MSU-1 is a bandwidth cheat we don't have.** It's a modern flash-cart coprocessor
|
||
giving memory-mapped streaming the real SNES never had. Their budget: 512 tiles x
|
||
32 bytes (4bpp 8x8) + tilemap ~= 18 KB/frame => **~430 KB/s** at 23.976fps.
|
||
That's *higher* than the 365 KB/s we'd reject on SASI. (ESTIMATE: my arithmetic on
|
||
their stated tile budget, not a measured figure.)
|
||
|
||
Where we're ahead: 256 simultaneous colors from a 65536 palette vs their 4bpp
|
||
sub-palettes needing a tile-aware palette optimizer plus a spatial smoothing pass
|
||
to hide 8x8 palette seams. That problem doesn't exist for us. Plus 68000@10MHz
|
||
vs 65816@3.58MHz, and 2MB vs 128KB.
|
||
|
||
**Most valuable thing in that repo is NOT the codec — it's `data/events/`:**
|
||
516 chapter definitions across 29 scenes as XML, plus
|
||
`data/chapter_event_inventory.md`. That's the arcade scene graph and input-timing
|
||
structure, entirely hardware-independent — the whole game-logic layer we'd
|
||
otherwise reverse-engineer from the arcade ROM.
|
||
|
||
**TODO: check their license before planning to reuse it.**
|
||
Their 516 chapters are finer-grained than our 224 Blu-ray streams, so mapping
|
||
their event table onto our footage means subdividing streams by timecode.
|
||
|
||
Caveat: all of the above is from README/repo-tree summaries, not their source.
|
||
|
||
---
|
||
---
|
||
|
||
# Findings — session 2 (2026-08-23)
|
||
|
||
## 8. CORRECTION to session 1: halving the framerate does NOT halve the bitrate
|
||
|
||
> **PARTLY SUPERSEDED.** The framerate correction stands. The
|
||
> "changed-spans + deflate = 247 KB/s" figure is a **compression upper bound,
|
||
> not a shippable design** — deflate decode does not fit the 68000's frame
|
||
> budget. See 17.2.
|
||
|
||
Session 1 measured 365 KB/s for naive delta+RLE at 24 fps and wrote
|
||
"(~183 KB/s at 12fps)". **That extrapolation is wrong.** Decimating to 12 fps
|
||
roughly doubles the per-frame delta, so the *rate* stays nearly flat.
|
||
|
||
Re-measured directly on 12 fps decimated frames (4 scenes, 66 frames):
|
||
|
||
| codec (all LOSSLESS w.r.t. the 256-colour frame) | B/frame | KB/s @12 | 22 min | ratio |
|
||
|---|---|---|---|---|
|
||
| raw 8bpp 256x192 | 49152 | 576 | 743 MB | 1.0:1 |
|
||
| session 1 row-span + RLE | 29055 | 340 | 439 MB | 1.7:1 |
|
||
| XOR vs prev + deflate | 30196 | 354 | 456 MB | 1.6:1 |
|
||
| **changed-spans + deflate** | **21110** | **247** | **319 MB** | **2.3:1** |
|
||
| changed-spans + LZMA | 18759 | 220 | 283 MB | 2.6:1 |
|
||
|
||
Session 1's own RLE re-measured at 12 fps gives **340 KB/s, not 183**.
|
||
Any plan that assumed 183 KB/s was based on a bad number.
|
||
|
||
Deflate-class entropy coding on top of the span payload is worth **1.4x** over
|
||
hand-rolled RLE, and LZ decode is cheap on a 68000 (byte copies), so the
|
||
lossless floor is ~247 KB/s / 319 MB. That is **infeasible on SASI** and
|
||
**tight but real on SCSI**.
|
||
|
||
## 9. Flat 4x4 VQ at k=256 is NOT acceptable — confirmed by eye
|
||
|
||
The risk flagged in 6 is real. At k=256, 4x4:
|
||
|
||
| scene | palette-only PSNR | after VQ | VQ loss |
|
||
|---|---|---|---|
|
||
| 00010 | 38.35 | 29.68 | 8.67 dB |
|
||
| 00020 | 39.90 | 32.67 | 7.22 dB |
|
||
| 00146 | 35.25 | 29.35 | 5.89 dB |
|
||
| 00181 | 41.92 | 32.87 | 9.05 dB |
|
||
|
||
Visually: Dirk's face disintegrates, teeth and eyes turn to mush, ink outlines
|
||
break into 4-pixel stair-steps, colour bleeds across block boundaries.
|
||
|
||

|
||
*Left: 1080p source. Middle: 256-colour palettised 256x192 — the quality ceiling,
|
||
and it is excellent. Right: flat 4x4 VQ at k=256. This is the result that killed
|
||
the flat-VQ architecture.*
|
||
|
||
**Crucially, the 256-colour palettised frame itself looks excellent.** Flat cel
|
||
art with a per-scene median-cut palette and no dithering is near-transparent
|
||
(35-42 dB). So the palette is not the problem and 256 colours is not the
|
||
problem — **block VQ is**. The quality ceiling we should hold ourselves to is
|
||
the palettised frame, not the 1080p source.
|
||
|
||
## 10. Hybrid VQ (Cinepak V1/V4 + SKIP) — this is the codec
|
||
|
||
Per 4x4 block, choose by rate-distortion: SKIP (reuse previous frame),
|
||
V1 (one 4x4 codeword, 1 byte), or V4 (four 2x2 codewords, 4 bytes),
|
||
with a 2-bit-per-block mode header. `lam` is the lagrangian rate knob.
|
||
|
||
Measured, k1=k4=256, 4 scenes (mean of the per-scene table in the session log):
|
||
|
||
| lam | PSNR | loss vs palette | SKIP% | V1% | V4% | B/frame | KB/s @12 |
|
||
|---|---|---|---|---|---|---|---|
|
||
| 0 (max quality) | 33.9 | 4.9 | 30.8 | 18.5 | 50.8 | 7574 | 88.8 |
|
||
| 200 | 31.9 | 5.9 | 44.0 | 37.6 | 18.4 | 4183 | 49.0 |
|
||
| 1000 | 31.6 | 7.3 | 47.4 | 47.7 | 4.9 | 2841 | 33.3 |
|
||
| 5000 | 25.5 | 13.3 | 55.6 | 44.4 | 0.0 | 2134 | 25.0 |
|
||
|
||
At a **matched ~30 KB/s** the hybrid beats flat 4x4 VQ by ~1 dB, and unlike flat
|
||
VQ it keeps scaling: at 89 KB/s it reaches within **4.9 dB of the palette
|
||
ceiling**, which flat VQ cannot reach at any bitrate.
|
||
|
||
Note V4% collapses to 0 at lam=5000 — that is the knob doing exactly what it
|
||
should: under a hard ceiling, detail blocks are the first thing sacrificed.
|
||
|
||
## 11. Codebook size sweep (flat 4x4, for reference)
|
||
|
||
> **SUPERSEDED.** The k=1024 result below is an artifact of a rate model that
|
||
> charged 1 byte for a 10-bit index. k=256 ships. See 14.
|
||
|
||
| block | k | PSNR | loss | key B | changed% | KB/s @12 | codebook RAM |
|
||
|---|---|---|---|---|---|---|---|
|
||
| 4x4 | 256 | 30.46 | 8.39 | 3072 | 52.7 | 28.5 | 8K |
|
||
| 4x4 | 1024 | 32.89 | 5.96 | 3840 | 56.6 | 35.6 | 32K |
|
||
|
||
+2.4 dB for 24K more RAM and 7 KB/s. With 2 MB of RAM, a 1024-entry codebook is
|
||
cheap and clearly worth it. (RAM figure is the word-expanded form the blitter
|
||
wants: k * 16 px * 2 bytes.)
|
||
|
||
## 12. Source framing — OPEN
|
||
|
||
The Blu-ray is **full-frame 1920x1080 16:9 with no pillarboxing**. The arcade
|
||
original is 4:3. The extractor currently centre-crops 1440x1080, which is the
|
||
arcade-faithful choice but discards image the 2006 remaster added. Options are
|
||
`crop` (default), `squash`, `wide` in `tools/encoder/extract.py`.
|
||
**Not yet decided; needs an eyeball comparison against arcade reference.**
|
||
|
||
## 13. Stream inventory correction
|
||
|
||
Session 1 said "typical scene clip ~60s". Sampled directly: the ~3-5 MB streams
|
||
are **1.2-1.7 s** clips — these are the individual arcade death/action moments,
|
||
which is exactly the granularity the game logic needs. Some 60 s streams
|
||
(e.g. 00203) are **menu screens, not content**. Any survey must classify
|
||
menu vs content before averaging, or the bitrate numbers are diluted by static
|
||
menus.
|
||
|
||
## 14. A FOURTH false-good result — and the correction
|
||
|
||
Add this to the 4 list. The mechanism was new but the shape was identical.
|
||
|
||
**The false result:** flat and hybrid VQ both showed **+2.4 dB for k=1024 over
|
||
k=256** at an apparently similar bitrate, which made a 1024-entry codebook look
|
||
like an obvious win. The k=1024 quality ladder rendered from that run looked
|
||
great at "45 KB/s".
|
||
|
||
**The bug:** the rate-distortion model in `vq_hybrid.encode()` charged **1 byte**
|
||
per codebook index unconditionally. A 1024-entry codebook needs a **10-bit index,
|
||
stored as 2 bytes**. So every k=1024 measurement understated the V1 and V4
|
||
payload by exactly 2x, *and* the lagrangian mode decision was choosing V4 on the
|
||
belief that four codewords cost 4 bytes when they cost 8.
|
||
|
||
**After charging the true index cost** (`idx_bytes` is now explicit and defaults
|
||
from the codebook size), matched-bitrate comparison on scene 00020:
|
||
|
||
| KB/s | k=256 (1-byte idx) | k=1024 (2-byte idx) |
|
||
|---|---|---|
|
||
| ~32-42 | **33.87 dB** @ 32.5 | 28.91 dB @ 42.3 |
|
||
| ~44-52 | **34.80 dB** @ 44.1 | 35.13 dB @ 52.5 |
|
||
| ~72-86 | **35.87 dB** @ 72.2 | 36.51 dB @ 86.0 |
|
||
|
||
k=1024 buys +0.3 to +0.6 dB for +19% bitrate — a wash at best — and at the low
|
||
end where the SASI profile lives it is **5 dB worse**, because the 2-byte index
|
||
floor dominates once V4 is priced out.
|
||
|
||
**k=256 with 1-byte indices is the shipping choice.** It is also the better
|
||
decoder: a plain `move.b` index with no alignment case, and an 8 KB codebook
|
||
instead of 32 KB.
|
||
|
||
**The general lesson, again:** the comparison was not wrong about VQ, it was
|
||
wrong about *cost*. When a knob looks like a free win, check that the rate model
|
||
is charging for it. Same failure family as 4.1-4.3: a plausible number produced
|
||
by a pipeline that was not measuring what it claimed to measure.
|
||
|
||
## 15. Rate-distortion curve of the shipping codec (k=256, corrected)
|
||
|
||
Scene 00020 (Dirk screaming, close-up face — the hardest case for linework),
|
||
and 00146. Includes the 2-bit-per-block mode header. No entropy coding yet.
|
||
|
||
| lam | 00020 PSNR | 00020 KB/s | 00146 PSNR | 00146 KB/s | SKIP | V1 | V4 | RAW |
|
||
|---|---|---|---|---|---|---|---|---|
|
||
| 25 | 38.68 | 182.2 | 31.04 | 193.5 | ~37% | ~24% | ~13% | ~26% |
|
||
| 100 | 35.87 | 72.2 | 29.04 | 72.5 | ~41% | ~34% | ~21% | ~4% |
|
||
| 300 | 34.80 | 44.1 | 28.28 | 44.4 | ~44% | ~42% | ~14% | 0% |
|
||
| 800 | 33.87 | 32.5 | 27.77 | 36.1 | ~46% | ~48% | ~5% | 0% |
|
||
| 2000 | 27.57 | 25.5 | 24.88 | 30.2 | ~50% | ~49% | ~1% | 0% |
|
||
|
||
Palette ceilings: 00020 = 39.90 dB, 00146 = 35.25 dB.
|
||
|
||

|
||
*The shipping codec across the rate knob. Top: source, palette ceiling, lam=25.
|
||
Bottom: lam=100 (`scsi` profile), lam=300 (`sasi` profile), lam=800.
|
||
Both shipping profiles hold Bluth's linework; the failure only starts past lam=800.*
|
||
|
||
Two things to read off this table:
|
||
- **The cliff is between lam=800 and lam=2000.** That is where V4 is priced out
|
||
entirely and detail blocks have nowhere to go. Do not ship past lam~800.
|
||
- **RAW is doing real work at high bitrate** (26% of blocks at lam=25) and
|
||
vanishes by lam=300. It is what makes the top of the curve reach the palette
|
||
ceiling, and it costs the decoder nothing — RAW is the cheapest mode to blit.
|
||
|
||
## 16. Licences cleared for the game-logic layer
|
||
|
||
Both checked this session:
|
||
|
||
- **astrobleem/SNES-SuperDragonsLairArcade — MIT**, "Copyright (c) 2026 Chad
|
||
Doebelin". `data/events/` holds 516 XML chapter definitions with timing and
|
||
event data. Reusable with attribution.
|
||
- **icculus/DirkSimple — zlib.** Independent from-scratch reimplementation of
|
||
the game logic in Lua, scene/timing tables in `game.lua`. Also permissive.
|
||
|
||
Having **two independent permissively-licensed transcriptions** of the arcade
|
||
scene graph is better than one: they can be diffed against each other to catch
|
||
transcription errors before any of it is committed to 68000 tables.
|
||
|
||
## 17. The profiles were set far too low — and entropy coding is a CPU trap
|
||
|
||
> **PARTLY SUPERSEDED.** 17.1's diagnosis (the profiles were not derived from
|
||
> hardware) and 17.2's CPU analysis both stand. But 17 reasoned against a
|
||
> misread bandwidth of 4 MB/s; the correct figure was taken as **4 Mbps =
|
||
> 488 KB/s**, so the "ship pixel-exact if SCSI sustains >=800 KB/s" conclusion
|
||
> in 17.5 was withdrawn. See 18 and 21.
|
||
>
|
||
> **17.5 IS REINSTATED BY 42.3.** The 488 KB/s figure that withdrew it was
|
||
> itself unsourced, and the delivered stream is 837.4 KB/s at 0.29 dB off the
|
||
> palette ceiling — 17.5's threshold and 17.5's conclusion, arrived at from the
|
||
> other end five sessions later. Its reasoning was sound; only its bandwidth
|
||
> number was wrong, and it was wrong in the direction that made it look wrong.
|
||
|
||
Prompted by the user asking why the SCSI profile was only 75 KB/s. It should not
|
||
have been. Two separate errors, one of them serious.
|
||
|
||
### 17.1 The profile bitrates were not derived from the hardware at all
|
||
|
||
They were read off the knee of the rate-distortion curve and then presented as
|
||
though bandwidth-driven. Against the (folklore) bus figures from 5:
|
||
|
||
| profile | was | bus figure | utilisation |
|
||
|---|---|---|---|
|
||
| `sasi` | 45 KB/s | ~300-500 KB/s | **12%** |
|
||
| `scsi` | 75 KB/s | ~1 MB/s | **7%** |
|
||
|
||
Nothing justified leaving 90% of the pipe unused. Raised to `sasi` 110 KB/s
|
||
(lam=60) and `scsi` 280 KB/s (lam=10), which is 35% and 28% utilisation —
|
||
still conservative, because the bus figures are folklore.
|
||
|
||
### 17.2 CPU is NOT the reason to stay low — but entropy coding would be
|
||
|
||
Budget is 833,333 cycles/frame at 12 fps. At session 1's measured ~6.5 cycles
|
||
per GVRAM pixel write:
|
||
|
||
| work | cycles | % of budget |
|
||
|---|---|---|
|
||
| blit 20.1% of pixels (session 1's 24fps figure) | 64k | 7.7% |
|
||
| blit 40% of pixels (the same content at 12fps) | 128k | 15.3% |
|
||
| **blit the FULL frame, every frame** | **319k** | **38.3%** |
|
||
| deflate decode, ~30 KB/frame output | 1,800k | **216%** |
|
||
| LZ4/LZSS decode, ~30 KB/frame output | 450k | **54%** |
|
||
|
||
Two conclusions, and the second one corrects 8:
|
||
|
||
- **Raising the VQ bitrate is nearly free on CPU.** Even a full-frame pixel-exact
|
||
blit fits in 38% of budget, and VQ decode is table copies — RAW, the mode that
|
||
dominates at high bitrate, is the *cheapest* mode to blit, not the dearest.
|
||
- **The 247 KB/s "lossless changed-spans + deflate" figure in 8 is a compression
|
||
upper bound, NOT a shippable design.** Deflate's Huffman decode is bitwise and
|
||
costs about 2.2x the entire frame budget on a 68000. Even byte-oriented LZ4 at
|
||
54% leaves too little beside a 38% blit. **Do not plan on entropy coding.**
|
||
All profile bitrates are raw payload.
|
||
|
||
This inverts session 1's "the CPU is idle, I/O is the ceiling" for the *decode*
|
||
path specifically: the blit is cheap, but any bit-oriented decompressor is not.
|
||
VQ is the right architecture precisely because its decode is a table copy.
|
||
|
||
### 17.3 The hybrid at lam=0 IS the lossless codec
|
||
|
||
Measured, un-entropy-coded raw payload, and deflated for reference only:
|
||
|
||
| scene | lam=0 raw | lam=0 deflated | lossless changed-spans+deflate | PSNR |
|
||
|---|---|---|---|---|
|
||
| 00020 | 442.1 KB/s | 274.5 KB/s | 267.3 KB/s | 39.90 = ceiling |
|
||
| 00146 | 467.6 KB/s | 223.2 KB/s | 219.1 KB/s | 35.25 = ceiling |
|
||
|
||
The hybrid at `lam=0` converges to within 3% of the purpose-built lossless coder.
|
||
That confirms the architecture unifies: there is no separate lossless path to
|
||
maintain, just the same bitstream with the knob open.
|
||
|
||
### 17.4 Full curve in raw (shippable) bytes
|
||
|
||
| lam | 00020 PSNR | 00020 KB/s | 00146 PSNR | 00146 KB/s | RAW% |
|
||
|---|---|---|---|---|---|
|
||
| 0 | **39.90** (exact) | 442.1 | **35.25** (exact) | 467.6 | ~76% |
|
||
| 10 | 39.38 | 248.1 | 32.27 | 305.2 | ~44% |
|
||
| 25 | 38.68 | 182.2 | 31.04 | 193.5 | ~26% |
|
||
| 60 | 36.94 | 108.0 | 29.61 | 103.1 | ~10% |
|
||
| 150 | 35.31 | 55.6 | 28.63 | 56.1 | ~1% |
|
||
| 300 | 34.80 | 44.1 | 28.28 | 44.4 | 0% |
|
||
|
||
### 17.5 This makes the blocked disk benchmark critical-path
|
||
|
||
Session 1 judged it "NOT on the critical path" because VQ at 30 KB/s was correct
|
||
whether SASI did 300 or 600 KB/s. That reasoning no longer holds. The profiles
|
||
now sit at 110 and 280 KB/s, close enough to the folklore ceilings that the
|
||
error bars matter, and **if SCSI sustains >=800 KB/s the correct `scsi` profile
|
||
is lam=0 — pixel-exact video.** Whether this port ships transparent or lossy on
|
||
SCSI is now waiting on one measurement.
|
||
|
||
## 18. Peak-to-mean burstiness — the mean was hiding the problem
|
||
|
||
> **SUPERSEDED — DO NOT ACT ON THIS SECTION.** The peak-vs-sustained comparison
|
||
> below is the **wrong test**. With a ring buffer the correct test is cumulative
|
||
> demand vs cumulative supply, and both profiles pass it with **zero required
|
||
> prefill**. `scsi` at lam=10 ships. See 21. The per-frame peak numbers
|
||
> themselves are still valid data; only the conclusion drawn from them is wrong.
|
||
|
||
Prompted by the user clarifying that the bandwidth figure is **4 Mbps = 488 KB/s**,
|
||
not 4 MB/s. That is ~8x tighter than what 17 was reasoning against, and it
|
||
changes the answer.
|
||
|
||
Per-frame instantaneous rate (video + 7.8 KB/s audio), 12 fps:
|
||
|
||
| scene | lam | mean | p90 | **max** | peak/mean | max as % of 488 KB/s |
|
||
|---|---|---|---|---|---|---|
|
||
| 00010 | 60 | 95.0 | 127.3 | 138.8 | 1.46 | 28.4% |
|
||
| 00010 | 10 | 198.9 | 266.1 | 284.0 | 1.43 | 58.2% |
|
||
| 00020 | 60 | 115.8 | 155.4 | 222.3 | 1.92 | 45.5% |
|
||
| 00020 | 10 | 255.9 | 391.2 | **470.8** | 1.84 | **96.4%** |
|
||
|
||
**The `scsi` profile as committed in f0f2f80 does not fit 4 Mbps.** Its mean is a
|
||
comfortable 52% of the pipe, but it peaks at 96.4% — and a frame that arrives
|
||
late is a *dropped frame*, not a slow one. Sizing a real-time stream on the mean
|
||
is the mistake; peak/mean is 1.4-1.9x on 1.2-1.7s clips and will be worse across
|
||
a full scene.
|
||
|
||
Two ways out, and only one is good:
|
||
- Size for the peak: `lam=25`, mean 194 KB/s. Costs a full step of quality.
|
||
- **Rate-control to the mean and carry a leaky bucket:** `lam=10` fits, and buys
|
||
back +0.7 dB (00020) / +1.2 dB (00146).
|
||
|
||
`ratectl.py` was written in session 2 but **never wired into `encode.py`**. This
|
||
demotes that from a loose end to the highest-value unfinished work in the repo.
|
||
|
||
## 19. Cycle-stealing DMA is not free DMA — 5 was wrong
|
||
|
||
FINDINGS 5 concluded "because it's DMA, streaming costs essentially no CPU —
|
||
this stacks with the 8% blit utilisation. The 68000 really is nearly idle."
|
||
|
||
The HD63450 steals bus cycles from the 68000 at roughly 8 clocks per 16-bit word:
|
||
|
||
| stream | words/s | clocks/s | CPU stolen | + full-frame blit |
|
||
|---|---|---|---|---|
|
||
| 110 KB/s | 56,320 | 450,560 | 4.5% | 42.8% |
|
||
| 250 KB/s | 128,000 | 1,024,000 | 10.2% | 48.5% |
|
||
| 450 KB/s | 230,400 | 1,843,200 | 18.4% | 56.7% |
|
||
| 488 KB/s | 249,856 | 1,998,848 | 20.0% | 58.3% |
|
||
|
||
At the rates the profiles now use, streaming costs **10-20% of the machine**.
|
||
Still affordable — nothing here breaks — but **bandwidth and CPU are one budget,
|
||
not two**, and any future headroom argument has to spend from both. The
|
||
"nearly idle" framing should not be reused.
|
||
|
||
(The 8 clocks/word figure is session 1's ESTIMATE from HD63450 timing, not a
|
||
measurement. It is the weakest link in this table.)
|
||
|
||
## 20. Where the profiles should come from
|
||
|
||
`tools/encoder/profile_gen.py` now derives lam from a bandwidth figure rather
|
||
than from the shape of the RD curve, accounting for audio, peak/mean, and
|
||
reporting DMA steal. Full benchmarking methodology — and why MAME cannot answer
|
||
the bandwidth question — is in `docs/BENCHMARK.md`.
|
||
|
||
The 4 Mbps figure itself is **user-supplied and its provenance is not recorded**.
|
||
Every profile now hangs off it, so it is worth pinning down.
|
||
|
||
## 21. Correction to 18 — the peak test was the wrong test
|
||
|
||
18 flagged that `scsi` "does not fit 4 Mbps" because a frame peaked at 96.4% of
|
||
the sustained rate. **That was the wrong comparison**, and the user was right to
|
||
push back. It measured instantaneous frame demand against a sustained rate as if
|
||
they had to match frame-by-frame. They do not: the disk keeps filling *during*
|
||
the frame, and any shortfall is absorbed by a ring buffer.
|
||
|
||
The correct test is whether **cumulative** demand ever outruns cumulative supply.
|
||
Simulated at a constant 488 KB/s fill, 12 fps, using the real per-frame sizes:
|
||
|
||
| scene | lam | mean KB/s | worst frame | **required prefill** | stall tolerance @256KB |
|
||
|---|---|---|---|---|---|
|
||
| 00010 | 10 | 198.9 | 23.67 KB | **0.0 KB** | 15.4 frames |
|
||
| 00020 | 10 | 255.9 | 39.23 KB | **0.0 KB** | 12.0 frames |
|
||
| 00146 | 10 | **313.0** | 42.10 KB | **0.0 KB** | 9.8 frames |
|
||
| 00181 | 10 | 211.1 | 25.25 KB | **0.0 KB** | 14.6 frames |
|
||
| (all) | 60 | 95-116 | 11-19 KB | **0.0 KB** | 26-32 frames |
|
||
|
||
Fill delivers **40.69 KB per frame time**. Only one measured frame exceeds that
|
||
(00146, 42.10 KB) and it is recovered by the following frame. **No scene needs
|
||
any prefill at all**, and a 256 KB buffer — 12.5% of RAM — carries ~1 second of
|
||
stall tolerance at `lam=10`, which is orders of magnitude more than an SD-backed
|
||
seek requires.
|
||
|
||
`scsi` at `lam=10` stands. The hardest scene sampled (00146) runs 313 KB/s mean,
|
||
64% of the pipe, with zero underrun risk.
|
||
|
||
### Why SD-backed changes the sizing rule
|
||
The deployment target is BlueSCSI / SCSI2SD, not a period spinning drive. That
|
||
was noted as a caveat in 5 but its consequence was not carried through:
|
||
|
||
- The sustained rate is a **bus-limited constant**, not an average over variable
|
||
seek latency. There is no long tail to leave margin for.
|
||
- Seek is ~microseconds, so branch-point stalls are a non-issue against a buffer
|
||
measured in whole seconds.
|
||
- Therefore we can size much closer to the ceiling than spinning-disk practice
|
||
would suggest. Conservative margins here are cargo-culted from a constraint
|
||
this deployment does not have.
|
||
|
||
**The SASI/SCSI split is about BUS PROTOCOL, not media.** SD emulation removes
|
||
seek latency from both, but a SASI bus is still slower than a SCSI one. Two
|
||
profiles remain the right design; both are now predictable constants rather than
|
||
distributions.
|
||
|
||
### What rate control is actually for now
|
||
Its value drops from load-bearing to **insurance**. Intra-scene peaks are a
|
||
non-problem. But we have measured **4 clips of 1.2-1.7s** out of 224 streams, and
|
||
00146 already runs 23% hotter than 00020. A sustained action sequence could
|
||
plausibly exceed the pipe where a 1.7s clip does not. Rate control gives a
|
||
*deterministic ceiling* across content we have not measured yet — which was the
|
||
original reason for choosing VQ over a lossless delta in the first place.
|
||
|
||
Still worth wiring in. No longer a blocker for shipping `scsi` at `lam=10`.
|
||
|
||
## 22. The display path, measured — first real frame on the X68000
|
||
|
||
Everything before this section was Python-side or a headless `-video none` run.
|
||
This is the first time pixels reached an emulated X68000 screen, and it produced
|
||
four hardware facts and one blocker that no amount of reasoning would have found.
|
||
|
||
**Scope — read this before quoting the result.** The X68000's *video* hardware
|
||
did the rendering: CRTC, GVRAM page decoding and the 15-bit+I palette lookup are
|
||
all genuinely emulated, which is why the output is bit-exact against the
|
||
hardware's colour math. But the pixels were written into GVRAM by a MAME Lua
|
||
script calling `SP:write_u16()` — the host poking emulated memory. **No 68000
|
||
instruction was executed to draw this frame.**
|
||
|
||
The equivalent is proving a framebuffer works by writing to it from a debugger.
|
||
It says the display path is correct; it says *nothing* about whether the 68000
|
||
can fill that framebuffer in time. Lua writes cost zero 68000 cycles, so the 38%
|
||
full-frame blit estimate that the entire CPU budget rests on remains completely
|
||
unvalidated. That is next step (2), the decoder skeleton, and it is untouched.
|
||
|
||
Reproduce:
|
||
```
|
||
python3 tools/bench/prep_frame.py <framedir> tmp/frame.bin 0
|
||
cd tmp && SDL_VIDEODRIVER=dummy mame x68000 -bios ipl10 -video soft -window \
|
||
-sound none -nothrottle -plugins -autoboot_script ../tools/bench/show_frame.lua \
|
||
-snapshot_directory ./snap -snapview native -seconds_to_run 6
|
||
```
|
||
|
||
### 22.1 The blocker: CRTC R20 bit 11 hides the graphics layer
|
||
The IPL leaves **CRTC R20 (`$E80028`) = `0x0B16`**. Bit 11 is *"G-VRAM set to
|
||
buffer"*, and MAME's `x68k_v.cpp` bails out of `draw_gfx()` on it outright:
|
||
|
||
```c
|
||
if (m_crtc->gfx_layer_buffer()) // if graphic layers are set to buffer, they aren't visible
|
||
return false;
|
||
// x68k_crtc.h: bool gfx_layer_buffer() const { return BIT(m_reg[20], 11); }
|
||
```
|
||
|
||
While that bit is set, GVRAM writes still land and read back correctly — which
|
||
is exactly what makes it so misleading. Six separate attempts at the video
|
||
controller (`$E82400/$E82500/$E82600`) rendered black with every register
|
||
reading back the intended value. **The video controller was never the problem.**
|
||
|
||
`R20` bits 9-8 select the colour setup, and this determines how `$C00000` is
|
||
decoded: `0x0300` = 65536c (16 bits/word), `0x0100` = 256c (low byte),
|
||
`0x0000` = 16c (4 bits). Set `R20 = 0x0116` for our mode.
|
||
|
||
### 22.2 Monitor contrast: the IPL leaves it at 14, not 15
|
||
`$E8E001` bits 3-0 are monitor contrast; MAME does
|
||
`m_screen->set_brightness(contrast * 0x11)`. The IPL leaves it at **14**, which
|
||
scales all output to 14/15 = 93.3%. Every rendered colour came out ~7% dark
|
||
until this was set to 15. **The player must write `$E8E001 = 15` at startup.**
|
||
|
||
Contrast `0` blanks the screen entirely (`x68k_v.cpp:661`) — that is the cheap
|
||
fade-to-black for scene transitions, no palette animation required.
|
||
|
||
### 22.3 Palette format CONFIRMED (was previously an assumption)
|
||
`PALETTE(config, m_gfxpalette).set_format(2, &x68k_state::GGGGGRRRRRBBBBBI, 256)`
|
||
|
||
```
|
||
bit 15..11 10..6 5..1 0
|
||
GGGGG RRRRR BBBBB I <- I is a shared LSB for all three channels
|
||
```
|
||
Expansion is `pal6bit((field << 1) | I)`, i.e. `(v << 2) | (v >> 4)`.
|
||
With contrast at 15, **all 256 entries render exactly as this predicts** — the
|
||
frame is pixel-identical, not merely close. GVRAM line stride is confirmed as
|
||
512 words = 1024 bytes, matching `HARDWARE.md`.
|
||
|
||
### 22.4 A new quality ceiling: the 15-bit palette costs 38.88 dB
|
||
> **Superseded by 23.3.** The 38.88 dB figure assumed the shared LSB `I` is
|
||
> always 1. Choosing `I` per palette entry by minimum error lifts the ceiling to
|
||
> **40.81 dB** on the same frame. The conclusion below ("`scsi` is close to
|
||
> display-transparent") is therefore weaker than stated — there is ~2 dB more
|
||
> headroom than this section claims.
|
||
Section 3 called the 256-colour palettised frame "the real quality ceiling".
|
||
That was measured in 24-bit RGB. The hardware palette only stores 5 bits per
|
||
channel plus a shared LSB, so there is a **second** quantisation below it:
|
||
|
||
| stage | PSNR |
|
||
|---|---|
|
||
| 24-bit palettised source -> X68000 15-bit+I display | **38.88 dB** |
|
||
| `scsi` profile codec error (00020, FINDINGS 15) | 39.4 dB |
|
||
|
||
The codec's error at `scsi` is **the same order as the display's own error**.
|
||
On real hardware `scsi` is therefore close to display-transparent, and pushing
|
||
`lam` below 10 buys quality the monitor cannot show. This bounds how much the
|
||
`scsi` profile is worth raising — it does not change the profiles themselves.
|
||
|
||
Caveat: measured on one frame (00020 f0001). It is a property of the palette,
|
||
not the content, so it should generalise, but it has not been checked across
|
||
scenes.
|
||
|
||
### 22.5 Why the first frame appears twice
|
||
GVRAM is a 512-pixel-wide page while the IPL's CRTC is still in its 768-wide
|
||
text timing, so the layer repeats at exactly x=512. This is correct hardware
|
||
behaviour, not a bug. The player sets its own CRTC mode and the wrap disappears.
|
||
No CRTC timing table has been written yet — the harness deliberately keeps the
|
||
IPL's timing so that no invented CRTC values are in play.
|
||
|
||
|
||
## 23. A real CRTC mode: 256x192 inside 256x256 (session 4)
|
||
|
||
Session 3's harness borrowed the IPL's 768x512 text timing and invented no CRTC
|
||
values, which is why the frame repeated at x=512 (22.5). This session derived a
|
||
real 256x256 mode table from MAME 0.277 source and verified it by snapshot.
|
||
Table: `tools/bench/crtc_mode.lua`. Regression test: `tools/bench/verify_frame256.py`.
|
||
|
||

|
||
|
||
*Left: palettised source. Right: the emulated X68000's native 256x512 raster —
|
||
256 dots wide, 512 scanlines carrying 256 double-scanned graphics rows, with the
|
||
192-row picture letterboxed in true black.*
|
||
|
||
### 23.1 The table, and why it needed no guessing
|
||
`refresh_mode()` in `x68k_crtc.cpp` selects the dot clock as
|
||
`(reg20 bit4 ? 69.55199MHz : 38.86363MHz) / div`, with `div` from a ladder keyed
|
||
on `reg20 & 0x1f`. Three entries matter:
|
||
|
||
| `reg20 & 0x1f` | div | dot clock | mode |
|
||
|---|---|---|---|
|
||
| `0x16` | 2 | 34.776 MHz | IPL's 768 wide, 31.5kHz |
|
||
| `0x11` | 3 | 23.184 MHz | 512 wide, 31.5kHz |
|
||
| `0x10` | 6 | 11.592 MHz | **256 wide, 31.5kHz, graphics double-scanned** |
|
||
|
||
The IPL's `R00 = 137` gives `m_htotal = (137+1)*8 = 1104` dots, and
|
||
`34.776e6 / 1104 = 31500.0 Hz` **exactly**. Holding the same line rate at div 6
|
||
needs `11.592e6 / 31500 = 368` dots `= 46` chars, so `R00 = 45`.
|
||
|
||
`368 = 1104/3` exactly, so every horizontal register is the 768-mode value
|
||
divided by three, and the active window divides without remainder:
|
||
`(124-28)/3 = 32` chars `= 256` dots. **No horizontal value was recalled or
|
||
estimated.** Only the blanking split rounds: the 768 mode is sync/back/front =
|
||
14/14/14 chars, `/3 = 4.67` each, and the closest integer triple summing to
|
||
`46-32 = 14` is 5/5/4.
|
||
|
||
| reg | value | meaning |
|
||
|---|---|---|
|
||
| R00 | 45 | H total, 46 chars = 368 dots -> 31500.0 Hz |
|
||
| R01 | 5 | H sync end (3.45 us) |
|
||
| R02 | 10 | H display begin -> `hbegin = 81` |
|
||
| R03 | 42 | H display end -> `hend = 336`; inclusive width `336-81+1 = 256` |
|
||
| R04 | 567 | V total, 568 scanlines -> 55.46 Hz |
|
||
| R05 | 5 | V sync end |
|
||
| R06 | 40 | V display begin -> `vbegin = 41` |
|
||
| R07 | 552 | V display end -> 512 scanlines = 256 double-scanned rows |
|
||
| R08 | 27 | H sync adjust (MAME stores it and never reads it) |
|
||
| R20 | `0x0110` | display (not buffer), 256-colour, 31.5kHz, 256x256 |
|
||
|
||
**The vertical registers are NOT halved**, which is the one thing that looks
|
||
wrong and is not. The CRTC still generates a 568-line raster; "256 lines" is a
|
||
graphics-layer double-scan applied in `draw_gfx()` (`x68k_v.cpp:401`), not a
|
||
change to the raster. Halving R04 would ask the monitor for 110 Hz. MAME emits
|
||
a `visarea larger then reg[20]` logerror for this; it is cosmetic.
|
||
|
||
Total blanking time is identical to the 768 mode (112 dots at 11.592 MHz =
|
||
336 dots at 34.776 MHz = 9.66 us), which is the property a real monitor cares
|
||
about — so this table should be safe on hardware, though that is untested.
|
||
|
||
### 23.2 MAME's double-scan is phase-shifted by one raster line
|
||
`get_gfx_pixel()` indexes `m_gfxbitmap.pix(scanline / divisor, pixel)` using the
|
||
**absolute** scanline, and `vbegin = 41` is odd. So in the native 256x512
|
||
snapshot the identical row pairs are `(1,2), (3,4), ...` and row 0 is a lone
|
||
half-line. Even rows are graphics rows 0..255. This cost a false failure before
|
||
it was understood; the regression test now asserts the shifted pairing
|
||
explicitly so a change in MAME's behaviour is visible rather than confusing.
|
||
|
||
### 23.3 The shared LSB `I` must be chosen per palette entry — worth 1.96 dB
|
||
Session 3's `pack()` hardcoded `I = 1`. That is not free: `I` is shared by all
|
||
three channels and each renders as `pal6bit((field << 1) | I)`, so with `I = 1`
|
||
the darkest reachable value is `pal6bit(1) = 4`, and **true black does not
|
||
exist**. Choosing `I` per entry to minimise summed squared error over R,G,B:
|
||
|
||
| rule | ceiling vs 24-bit palettised (00020 f0001) | entries with I=0 |
|
||
|---|---|---|
|
||
| `I = 1` fixed (session 3) | 38.85 dB | 0 |
|
||
| `I` per entry, min squared error | **40.81 dB** | 102 / 256 |
|
||
|
||
Nearly **2 dB for free**, and 102 of 256 entries want `I = 0` — this is not a
|
||
corner case. It supersedes the ceiling in 22.4 and means `scsi` has about 2 dB
|
||
more headroom before it hits the display than that section claimed.
|
||
|
||
The encoder does not yet do this. `tools/encoder/` still emits 24-bit palettes
|
||
and the packing happens Lua-side; whatever eventually writes X68000 palette
|
||
words must use the per-entry rule.
|
||
|
||
### 23.4 Letterboxing requires a reserved black palette entry
|
||
GVRAM cleared to zero displays **palette entry 0**, and a free mediancut palette
|
||
puts a real image colour there — on 00020 f0001 it was `(206,192,176)`, used by
|
||
210 image pixels, so it cannot simply be repurposed. A 256x192 picture in a
|
||
256x256 mode has 64 blank rows, so the palette must be built with **255 colours
|
||
plus a reserved black at index 0** (`prep_frame.py --reserve-black`). Combined
|
||
with 23.3, entry 0 also needs `I = 0` or the bars sit at RGB (4,4,4).
|
||
|
||
Cost: one of 256 entries. Measured quality effect: none visible — the ceiling
|
||
figure in 23.3 is already measured on the 255-colour palette.
|
||
|
||
### 23.5 What is still not proven
|
||
GVRAM was again filled from Lua. **No 68000 instruction has drawn a pixel yet**,
|
||
and the 38% full-frame blit estimate underpinning the CPU budget remains
|
||
unvalidated. What this section adds is that the *target mode* is now real, so
|
||
68000 code has a defined geometry to write into: 256 words per visible row, a
|
||
1024-byte line stride, and rows 32..223 of a 256-row page.
|
||
|
||
---
|
||
|
||
## 24. The blit, measured on the 68000 — the 38% estimate was wrong (session 5)
|
||
|
||
**The first 68000 instructions in this project to draw a pixel.** Everything in
|
||
22 and 23 was GVRAM filled from Lua, which costs zero 68000 cycles. This section
|
||
replaces the estimate that the whole CPU budget rested on with a measurement.
|
||
|
||
Harness: `tools/bench/blit.s` + `tools/bench/blit.lua`. Four variants of a
|
||
full-frame 256x192 paint, each looped to run ~4 emulated seconds, timed from
|
||
`machine.time` between two flag writes by the 68000 itself.
|
||
|
||
| variant | what it does | cycles/frame | % of a 12fps frame |
|
||
|---|---|---:|---:|
|
||
| **V1** | `movem.l` blit from a word-expanded RAM frame (96KB read + 96KB write) | **446,286** | **53.6%** |
|
||
| V2 | naive `move.b`/`move.w` per pixel from a byte source | 1,284,174 | 154.1% |
|
||
| **V3** | write-only floor — registers preloaded, no source read at all | **225,789** | **27.1%** |
|
||
| **V4** | the same 96KB of writes issued in **4x4 block order** | **637,971** | **76.6%** |
|
||
|
||
The 12fps budget is 833,333 cycles (10.0 MHz confirmed from `x68k.cpp:1133`,
|
||
`40_MHz_XTAL / 4`).
|
||
|
||
### 24.1 The numbers are cross-checked against hand-derived cycle counts
|
||
Every variant was predicted from the MC68000 timing tables *before* the run
|
||
(`MOVEM.L` M->R `(An)+` = 12+8n, `(d16,An)` = 16+8n; R->M `(An)` = 8+8n,
|
||
`(d16,An)` = 12+8n) and then measured:
|
||
|
||
| | predicted | measured | error |
|
||
|---|---:|---:|---:|
|
||
| V1 | 447,744 | 446,286 | 0.33% |
|
||
| V2 | 1,284,096 | 1,284,174 | 0.006% |
|
||
| V3 | 225,792 | 225,789 | 0.001% |
|
||
| V4 | 640,704 | 637,971 | 0.43% |
|
||
|
||
This agreement is the point. A MAME timing number on its own would be worth
|
||
little given how many false-good results this project has produced (FINDINGS 4);
|
||
two independent derivations landing within half a percent is worth something.
|
||
The residual error is the frame-granularity of the measurement — Lua gets no
|
||
cycle counter (`luaengine.cpp` exposes `machine.time` and nothing from
|
||
`device_execute_interface`), so timing resolution is one video frame, 18.03 ms.
|
||
|
||
### 24.2 SCOPE: these are instruction cycles, and therefore a LOWER BOUND
|
||
MAME's `gvram_w`/`gvram_r` (`x68k_crtc.cpp:501,595`) contain **no timing at
|
||
all** — no wait states, no `adjust_icount`. GVRAM in MAME is as fast as main
|
||
RAM. Real X68000 GVRAM stalls the CPU on access, so every figure above is a
|
||
floor, not a prediction. **Do not quote these as hardware numbers.** Interrupts
|
||
were masked (`SR = $2700`) so the IPL's timer and VBL handlers could not steal
|
||
cycles into the measurement; a real player will take interrupts on top.
|
||
|
||
### 24.3 The 38% estimate is dead — a full-frame blit is 53.6%
|
||
The realistic "decode into a RAM frame, then blit it" design costs **53.6% of
|
||
the frame budget before decoding a single block**, and that is the zero-wait-
|
||
state floor. The estimate the CPU budget has been carrying since session 1 was
|
||
38%. It was optimistic by 41%.
|
||
|
||
The cause is visible in the V1/V3 gap: **reading the source frame is exactly
|
||
half the total cost** (221,952 of 446,286 cycles). The 68000 pays 8 cycles per
|
||
longword read and 8 per longword written, and in 256-colour mode a pixel
|
||
occupies a whole word of address space, so a frame is 96KB of traffic in each
|
||
direction rather than 48KB.
|
||
|
||
### 24.4 The high byte of every GVRAM write is discarded — confirmed from source
|
||
`gvram_w` case `0x0100` writes `data & 0x00ff` with `mem_mask 0x00ff`. So in
|
||
256-colour mode the CPU cannot pack two pixels into one word, and the odd bytes
|
||
of a word-expanded source frame never need clearing — V1 exploits this by
|
||
leaving them uninitialised. This is why 96KB, not 48KB, is the irreducible
|
||
write traffic.
|
||
|
||
### 24.5 The architecture question, and where it turns over
|
||
> **Superseded by FINDINGS 28.1/28.2 (session 7).** The two-path plan below is
|
||
> incoherent — the compose path needs a RAM reference the direct path never
|
||
> writes — and its two costs are both *copies*, so they were never comparable to
|
||
> a decode. The "76.6% x non-SKIP fraction" model is also 2.03x optimistic:
|
||
> the four block modes cost 300/448/400 cycles, not one figure. One path ships.
|
||
|
||
V4 prices the access pattern a decoder that writes codewords **straight into
|
||
GVRAM** actually has: 4 rows of 8 bytes at a 1024-byte stride per 4x4 block. The
|
||
same 96KB of writes costs **76.6%** in block order versus 53.6% row-linear — the
|
||
stride destroys the `movem.l` burst, 208 cycles per block against a theoretical
|
||
best of ~150.
|
||
|
||
But a decoder never writes every block: SKIP blocks cost **nothing at all**, and
|
||
the previous frame is already sitting in GVRAM, so **no RAM reference frame is
|
||
needed for SKIP to work**. So the two designs scale differently:
|
||
|
||
- **compose-in-RAM then blit** — flat 53.6%, independent of how much changed
|
||
- **decode-direct-to-GVRAM** — 76.6% x (fraction of non-SKIP blocks)
|
||
|
||
**They cross at 70% of blocks changed.** Below that, writing straight into GVRAM
|
||
wins, and it also drops the 96KB RAM reference frame entirely. Above it, the
|
||
flat blit wins.
|
||
|
||
**This makes the non-SKIP block fraction the single most important unmeasured
|
||
number in the project.** It is already computable from the encoder — it is a
|
||
by-product of the mode decision in `vq_hybrid.py` — and it has never been
|
||
reported. Measure it before writing any decoder inner loop, because it selects
|
||
which inner loop to write.
|
||
|
||
### 24.6 The frame the 68000 drew is pixel-exact
|
||
V1's output was snapshotted and passes `verify_frame256.py` unchanged: `256x512
|
||
native, double-scan exact, active 256x192 pixel-exact, letterbox true black`,
|
||
40.81 dB. So 68000 code drives the mode of FINDINGS 23 correctly, and 23.5 is
|
||
now closed.
|
||
|
||
---
|
||
|
||
## 25. The sustained action sequence, found and measured (session 5)
|
||
|
||
STATUS has carried "a *sustained* action sequence is the one thing that could
|
||
still break the bitrate" as the open risk since session 2. Every clip measured
|
||
before this was 1.2-1.7 s. This section closes it: **it does break the profiles,
|
||
though not the bus.**
|
||
|
||
### 25.1 The two largest streams on the disc are not game footage
|
||
A survey that sorts 224 streams by size and encodes the biggest would have
|
||
measured **live action**:
|
||
|
||
| stream | size | what it actually is |
|
||
|---|---:|---|
|
||
| 00216 | 3777 MB | the feature with a **burned-in picture-in-picture commentary** |
|
||
| 00215 | 3475 MB | the commentary itself, full-screen live action |
|
||
| **00223** | **1802 MB** | **clean animation, 9.4 min — the one to use** |
|
||
|
||
The PiP in 00216 is burned into video stream 0, not a selectable secondary
|
||
stream, so there is no ffmpeg flag that recovers a clean frame from it. This
|
||
extends FINDINGS 13's menu-vs-content warning: the classification needed is
|
||
**content / menu / bonus**, and bonus material is the one that looks most like
|
||
content by every cheap metric (size, duration, bitrate).
|
||
|
||
### 25.2 Picking the worst window by measurement, not by eye
|
||
`tools/analysis/07_motion_survey.py` scans a whole stream at 96x72 and reports
|
||
the highest-mean sliding window of inter-frame absolute difference. On 00223:
|
||
|
||
```
|
||
6793 frames @12fps = 566.1s
|
||
motion energy mean 9.40 median 5.60 p90 21.70 max 112.39
|
||
hottest sustained 10s window: t = 539.4s (2.01x stream mean)
|
||
quietest 10s window: t = 144.2s (0.19x stream mean)
|
||
```
|
||
|
||
The 10.6x spread between the quietest and hottest sustained windows is the whole
|
||
argument for not sampling clips by hand. `t = 539.4s` is the Singe endgame.
|
||
|
||
### 25.3 Both profiles overshoot on that window — rate control is now required
|
||
Encoding those 120 frames at the shipping profiles, with the fixed `lam` the CLI
|
||
currently uses:
|
||
|
||
| profile | target | measured | overshoot | PSNR | palette ceiling |
|
||
|---|---:|---:|---:|---:|---:|
|
||
| `sasi` | 110 KB/s | **129.6 KB/s** | **+18%** | 27.82 dB | 31.33 dB |
|
||
| `scsi` | 280 KB/s | **373.8 KB/s** | **+34%** | 30.81 dB | 31.33 dB |
|
||
| *(00020 baseline, `sasi`)* | 110 KB/s | 108.0 KB/s | -2% | 36.94 dB | 39.90 dB |
|
||
|
||
**This reclassifies rate control from insurance to a requirement.** STATUS has
|
||
had "wire rate control into `encode.py`" at priority 3-4 since session 2 with the
|
||
note "no longer a blocker (FINDINGS 21)". That was true of the clips measured
|
||
then. It is not true of this one. `ratectl.encode_rate_controlled()` already
|
||
exists and builds a per-frame lam ladder; it has simply never been hooked up.
|
||
|
||
Note what did **not** break: 373.8 + 7.8 = 381.6 KB/s is still under the 488 KB/s
|
||
working figure, so FINDINGS 21's ring-buffer conclusion survives — but at 78% of
|
||
the pipe sustained over ten seconds rather than the comfortable margin implied by
|
||
1.7 s clips.
|
||
|
||
### 25.4 The palette ceiling is content-dependent, and on hard content it binds
|
||
The 256-colour scene palette costs **31.33 dB** on this window against **39.90 dB**
|
||
on 00020 — 8.6 dB worse. Fire, lava and smoke gradients are exactly what a
|
||
256-entry mediancut palette handles worst.
|
||
|
||
This inverts an assumption the project has been carrying. FINDINGS 23.3 put the
|
||
X68000 display ceiling at 40.81 dB and treated it as comfortably clear of the
|
||
codec's own error. On this content the **scene palette (31.33 dB), not the
|
||
display hardware (40.81 dB), is the binding constraint** — and `scsi` is already
|
||
within 0.51 dB of it. Spending bits to close that last half-dB is spending them
|
||
against a ceiling that is not the display's.
|
||
|
||
### 25.5 `scsi` collapses to RAW under stress
|
||
Mode distribution on this window is qualitatively different from anything
|
||
measured before:
|
||
|
||
| profile | SKIP | V1 | V4 | RAW |
|
||
|---|---:|---:|---:|---:|
|
||
| `sasi` (lam=60) | 45.6% | 16.3% | 24.2% | 13.9% |
|
||
| `scsi` (lam=10) | 26.2% | 5.5% | 7.1% | **61.2%** |
|
||
| *00020, `sasi`* | 46.9% | 24.1% | 17.8% | 11.2% |
|
||
|
||
At `lam=10` the rate-distortion decision finds literal pixels cheaper than any
|
||
codeword for 61% of blocks — the codebooks are simply not describing this
|
||
content. That is the mechanism behind the +34% overshoot in 25.3, and it is a
|
||
rate-control problem, not a codec-structure problem: the RD decision is behaving
|
||
correctly for the lam it was given.
|
||
|
||
### 25.6 The decoder needs BOTH display paths, chosen per frame
|
||
> **Superseded by FINDINGS 28.1 (session 7).** Mixing the paths displays stale
|
||
> pixels on 70 of these 120 frames. The "median 37.0%, capped at 53.6%" below is
|
||
> the cost of an incorrect player; every coherent version is dearer, and plain
|
||
> direct-to-GVRAM is the cheapest of them.
|
||
|
||
Applying FINDINGS 24.5's crossover to the real per-frame distribution:
|
||
|
||
| | median non-SKIP | p90 | frames over the 70% crossover |
|
||
|---|---:|---:|---:|
|
||
| `sasi`, Singe window | 48.4% | 82.8% | 36 / 120 (30%) |
|
||
| `scsi`, Singe window | 70.8% | 92.4% | 64 / 120 (53%) |
|
||
| `sasi`, 00020 | 54.0% | 88.8% | 3 / 14 (21%) |
|
||
|
||
Neither path wins outright: **30-53% of frames want the flat blit and the rest
|
||
want direct-to-GVRAM.** A player that implements both and picks per frame — the
|
||
mode headers are parsed before any pixel is written, so the count is free — pays
|
||
a median of **37.0%** of the frame budget and is capped at **53.6%**. A player
|
||
that implements only direct-to-GVRAM pays up to 76.6% and would miss frames on
|
||
the scene cuts.
|
||
|
||
So the answer to 24.5 is "both", and the selection is a one-line comparison
|
||
against a block count the decoder already has in hand.
|
||
|
||
### 25.7 What this does not measure
|
||
One 10 s window of one stream, at fixed lam, with `_paint` still a Python loop.
|
||
The full-disc survey is still not done, and the numbers above are the *worst*
|
||
window rather than a distribution over content. What has changed is that the
|
||
worst case is now a measurement rather than a worry.
|
||
|
||
---
|
||
|
||
## 26. Rate control is unsound as written — found before wiring it up (session 5)
|
||
|
||
FINDINGS 25.3 promoted rate control from insurance to a requirement. Reading
|
||
`ratectl.py` before wiring it into `encode.py` turned up a correctness bug that
|
||
would have produced exactly the kind of plausible-looking wrong result this
|
||
project keeps catching (FINDINGS 4, 9, 14, 18).
|
||
|
||
### 26.1 The lam ladder desynchronises the encoder from the decoder
|
||
`H.encode()` is **temporally recursive**: SKIP blocks are copied from the
|
||
previous *reconstruction*, and `prev = out` closes the loop
|
||
(`vq_hybrid.py:84-109`). A frame's output therefore depends on every frame
|
||
before it in that same run.
|
||
|
||
`encode_rate_controlled()` runs `H.encode()` once per lam over the **whole
|
||
sequence**, building a ladder of independent temporal chains, then picks each
|
||
frame from whichever rung fits the budget. When frame *f* comes from rung *i*
|
||
and frame *f-1* was emitted from rung *j != i*, the SKIP blocks in *f* reference
|
||
a reconstruction **the decoder never saw**.
|
||
|
||
Measured on the Singe window (`tools/analysis/09_ratectl_drift.py`, 120 frames,
|
||
5 rungs, target 110 KB/s):
|
||
|
||
| | |
|
||
|---|---|
|
||
| rung switches | **67** over 120 frames |
|
||
| frames whose emitted output differs from what the encoder recorded | **111 / 120** |
|
||
| worst frame | **21,339 px = 43.4% of the frame** |
|
||
| encoder-vs-decoder agreement, worst frame | 27.1 dB |
|
||
| reported PSNR overstatement | **0.36 dB** |
|
||
|
||
The 0.36 dB is the least interesting number here. The encoder is reporting
|
||
quality for a reconstruction that will never exist, and 43% of a frame differing
|
||
is a visible artefact whatever the mean says.
|
||
|
||
**The fix is structural, not a tuning change:** `H.encode()` must become
|
||
frame-drivable — take `prev` and one lam, return one frame — so rate control can
|
||
feed back the frame it actually emitted. The current whole-sequence signature is
|
||
what makes the ladder tempting in the first place.
|
||
|
||
### 26.2 The ladder spans 250x past the shippable range
|
||
`lam_hi=2e5`, but FINDINGS 15 puts the quality cliff between lam=800 and
|
||
lam=2000 and says do not ship past lam~800. Every rung above ~800 is
|
||
unshippable, so a frame that only fits at lam=9457 has not been rate-controlled,
|
||
it has been destroyed. Cap `lam_hi` at 800 and let a frame that cannot fit
|
||
overrun the bucket — a visible overrun is a better failure than silent garbage.
|
||
|
||
### 26.3 The ladder is far too coarse where it matters
|
||
With `steps=5` the geomspace lands on 1 / 21 / 447 / 9457 / 200000, and **only
|
||
two rungs were ever chosen**. The budget is 8,721 B/frame; the two straddling
|
||
rungs deliver 23,183 B (lam=21) and 3,071 B (lam=447) — a **7.5x** gap across
|
||
the operating point. Rate control cannot land near a target it has to jump over.
|
||
|
||
The module docstring already describes the right approach — *"per frame we
|
||
binary-search lam to land inside a byte budget"* — but the implementation is a
|
||
fixed precomputed ladder. Doc and code disagree; the doc is correct.
|
||
|
||
### 26.4 What does work
|
||
The leaky bucket lands the mean where it should: **109.1 KB/s against a 110
|
||
target**, with 32% of frames over the per-frame budget and banked by the bucket.
|
||
That mechanism is sound and worth keeping. It is the per-frame lam *selection*
|
||
underneath it that needs rebuilding, not the bucket.
|
||
|
||
### 26.5 Cost note before starting
|
||
Each rung is a full-sequence encode and `_paint` is still a Python per-block
|
||
loop, so a 5-rung run over 120 frames takes minutes. **Vectorise `_paint`
|
||
first** — it is already on the list for the full-disc survey and it makes the
|
||
rate-control work practical rather than merely faster.
|
||
|
||
## 27. Rate control, rebuilt and wired in (session 6)
|
||
|
||
FINDINGS 26 stopped the session-5 rate controller before it shipped: it picked
|
||
frames out of independently-encoded whole-sequence runs, so 111 of 120 frames
|
||
referenced reconstructions the decoder would never see. The fix was structural,
|
||
as 26.1 said it had to be. It is now wired into `encode.py` and **on by
|
||
default** for a profile.
|
||
|
||
### 27.1 The encoder is frame-drivable, and the drift is zero by construction
|
||
`vq_hybrid` now exposes one frame at a time — `frame_ctx(m, f, prev)` /
|
||
`decide(ctx, lam)` / `paint(m, ctx, mode)` — and `encode()` is a thin loop over
|
||
that API. Rate control drives the same three calls and feeds back **the frame it
|
||
actually emitted** as the next frame's `prev`. There is no ladder to pick from,
|
||
so the desync has no way to occur.
|
||
|
||
`tools/analysis/09_ratectl_drift.py`, unchanged in what it asserts:
|
||
|
||
| | session 5 | session 6 |
|
||
|---|---|---|
|
||
| frames whose emitted output differs from what the encoder recorded | 111 / 120 | **0 / 120** |
|
||
| worst frame | 21,339 px (43.4%) | **0 px** |
|
||
| reported PSNR overstatement | 0.36 dB | **0.00 dB** |
|
||
|
||
This is the harder case for that test on purpose: it runs with `lam_lo=1.0`, so
|
||
lam moves on 117 of 119 frame boundaries. Under the old ladder, 67 rung switches
|
||
were enough to corrupt 111 frames.
|
||
|
||
### 27.2 Both overshoots are closed, and they cost under 1 dB
|
||
The Singe window (FINDINGS 25.3), which is the worst sustained window on the
|
||
disc. Totals include the 7.8 KB/s ADPCM allowance:
|
||
|
||
| profile | target | fixed lam (session 5) | rate-controlled | quality cost |
|
||
|---|---|---|---|---|
|
||
| `sasi` | 110 KB/s | 137.4 KB/s (**+25%**) | **109.5 KB/s** | 27.82 → 27.22 dB (−0.60) |
|
||
| `scsi` | 280 KB/s | 381.6 KB/s (**+36%**) | **280.0 KB/s** | 30.81 → 29.90 dB (−0.91) |
|
||
|
||
Zero frames hit the lam=800 cliff at either profile, so nothing was destroyed to
|
||
get there (26.2's failure mode did not trigger). `sasi` needed lam to reach 183
|
||
at worst against a floor of 60; `scsi` reached 58.7 against 10. The controller
|
||
is working an order of magnitude below the cliff, which is where the search
|
||
range being capped at 800 rather than 2e5 stops mattering at all — and that is
|
||
the point: a range that never needs its top is a range you can trust.
|
||
|
||
`scsi` still sits **1.43 dB** from the scene palette ceiling of 31.33 dB
|
||
(FINDINGS 25.4), against 0.51 dB before. The ceiling, not the codec, is still
|
||
what bounds this content.
|
||
|
||
The percentages differ from 25.3's +18%/+34% because those compared video
|
||
payload against the total target; the table above compares like with like
|
||
(total against total). The payload figures are unchanged: 129.6 and 373.8 KB/s.
|
||
|
||
### 27.3 Rate control makes the display path cheaper, not dearer
|
||
The decoder-architecture numbers of FINDINGS 25.6 were measured on the
|
||
fixed-lam encoder. Re-measured under rate control, on the same window, with the
|
||
player picking the cheaper of compose-then-blit and direct-to-GVRAM per frame:
|
||
|
||
| profile | median display cost | frames above the 70% crossover |
|
||
|---|---|---|
|
||
| `sasi` fixed → RC | 37.0% → **36.6%** | 30.0% → 26.7% |
|
||
| `scsi` fixed → RC | 53.6% → **47.1%** | 53.3% → 35.8% |
|
||
|
||
Raising lam moves blocks to SKIP and V1, which is fewer blocks to write. The
|
||
"implement both paths, pick per frame" conclusion is unaffected and the cap is
|
||
still 53.6%.
|
||
|
||
### 27.4 The quality floor barely matters; the prefill matters, wrongly
|
||
Two knobs were measured rather than guessed.
|
||
|
||
**`--rc-floor`** decides whether a quiet frame may spend more than the fixed-lam
|
||
profile would. On the Singe window it is worth nothing — 109.5 vs 110.0 KB/s and
|
||
**0.00 dB** — because no frame on that window is quiet enough for the bucket to
|
||
saturate. The default is `profile` (never spend more than session 5 would), so
|
||
rate control cannot regress content that already fits.
|
||
|
||
**`--prefill`** models how full the player's buffer is at scene start. It is
|
||
tempting and it is a trap, so it defaults to 0:
|
||
|
||
| clip | prefill 0.0 | 0.5 | 1.0 | target |
|
||
|---|---|---|---|---|
|
||
| Singe, 120 fr, `sasi` | 109.5 | 112.9 | **116.3** | 110 |
|
||
| Singe, 120 fr, `scsi` | 280.0 | 289.1 | **298.2** | 280 |
|
||
| 00020, 14 fr, `sasi` | 92.0 | **115.8** | **115.8** | 110 |
|
||
| 00020, 14 fr, `scsi` | 224.8 | **255.9** | **255.9** | 280 |
|
||
|
||
(`scsi` on 00020 is the one cell where prefill looks harmless: the clip fits
|
||
under 280 either way. That is the content being easy, not the knob being safe.)
|
||
|
||
Prefill buys a permission to overshoot of exactly `bucket / nframes`. At 8
|
||
frames of bucket over 120 frames that is 6.2% — measured — and on a 14-frame
|
||
clip the bucket is larger than the clip, so rate control switches itself off and
|
||
reproduces fixed-lam exactly (lam never leaves its floor: min = median = max =
|
||
60). **A prefill that makes a target look met has disabled the controller.**
|
||
|
||
### 27.5 The 00020 undershoot is a clip-length artefact, not a bug
|
||
At prefill 0 the 14-frame 00020 clip lands at 92.0 KB/s against a 110 ceiling —
|
||
0.66 dB given away for nothing. That is the leaky bucket's startup transient:
|
||
the first `bucket_frames` frames cannot draw on a bank they have not accumulated.
|
||
It is bounded by `bucket / nframes`, so it is 6% on a 10-second window and 20%
|
||
on a 1.2-second one.
|
||
|
||
The lesson is the one FINDINGS 25.3 already taught in a different costume: **a
|
||
1.2-second clip cannot be used to judge rate control.** Real scenes are tens of
|
||
seconds. Do not tune the bucket against 00020.
|
||
|
||
Worth recording separately: fixed-lam `sasi` on 00020 delivers 115.8 KB/s — the
|
||
supposedly easy clip was **already 5% over its target**, which nothing had
|
||
noticed because the profile table quotes its PSNR and not its bitrate.
|
||
|
||
### 27.6 FINDINGS 26.5's cost premise was wrong in both halves
|
||
26.5 said a rate-control experiment was minutes because `_paint` is a Python
|
||
per-block loop, and told the next session to vectorise it first. Vectorising it
|
||
was correct and it is **17.1x faster**, but it was never the bottleneck, and the
|
||
ladder was never minutes. Measured per frame, 256x192:
|
||
|
||
| | ms |
|
||
|---|---|
|
||
| `VQ.assign` x2 — codeword search | **22.83** |
|
||
| SKIP error against `prev` | 1.40 |
|
||
| `decide` — argmin at one lam | 0.06 |
|
||
| `paint`, vectorised | 0.29 |
|
||
| `paint`, old per-block loop | 4.93 |
|
||
|
||
`_paint` was 14% of a frame. A 5-rung ladder over 120 frames was ~18 s of
|
||
encoding, not minutes — the "few minutes" in the drift test's docstring was
|
||
`H.build`'s k-means (51 s), which no amount of vectorising `_paint` would have
|
||
touched.
|
||
|
||
What actually makes per-frame rate control affordable is that `VQ.assign`'s
|
||
output depends on **neither `lam` nor `prev`**, so it is computed once per frame
|
||
and a lam search only re-runs the 0.06 ms argmin:
|
||
|
||
| | |
|
||
|---|---|
|
||
| 12-step per-frame lam search, 120 frames, symbols cached | **0.31 s** |
|
||
| the same search by re-running whole-sequence encodes | 49.10 s |
|
||
|
||
That is a 158x difference, and it is the reason the controller can afford a real
|
||
bisection instead of a 5-rung ladder — which was the actual defect in 26.3.
|
||
|
||
The cache holds **one frame**. At ~133 KB of intermediates per frame, caching
|
||
the sequence would cost 900 MB on a 9.4-minute stream to save nothing: every
|
||
caller works a frame at a time.
|
||
|
||
---
|
||
|
||
## 28. The 68000 decoder exists, is pixel-exact, and does not fit (session 7)
|
||
|
||
`src/player/decode.s` parses DLX1 and draws frames on the emulated X68000. It is
|
||
**pixel-exact across a 120-frame sequential run** of the worst sustained window
|
||
on the disc (`tools/bench/verify_decode.py`), exercising all four block modes
|
||
and the full temporal recursion — the last frame is only right if every frame
|
||
before it was.
|
||
|
||
It is also **too slow**. On that window, at the shipping `sasi` profile:
|
||
|
||
| | non-SKIP blocks | measured cost |
|
||
|---|---:|---:|
|
||
| cheapest frame | 15.4% | 31.5% of a 12fps frame |
|
||
| median frame | 47.8% | 73.8% |
|
||
| p90 frame | 82.5% | **116.4%** |
|
||
| worst frame | 100.0% | **135.8%** |
|
||
| mean over the window | 47.8% | **81.7%** |
|
||
|
||
**31% of frames miss the 833,333-cycle budget**, and like every figure since
|
||
FINDINGS 24 these are instruction cycles against zero-wait-state memory, so
|
||
they are a floor. This is the first time CPU, not disk, is the binding
|
||
constraint — FINDINGS 21 retired the bandwidth worry, and this replaces it.
|
||
|
||
### 28.1 The dual-path plan of 24.5/25.6 was incoherent, and is withdrawn
|
||
FINDINGS 24.5 specified two display paths chosen per frame on the non-SKIP
|
||
count, and 25.6 costed the mix at "median 37.0%, capped at 53.6%". Two of its
|
||
premises cannot both hold:
|
||
|
||
- compose-in-RAM-then-blit exists to make the blit **row-linear**, so it must
|
||
assemble a **full** frame in RAM. The pixels it does not decode this frame —
|
||
the SKIP blocks — can only come from a RAM copy of the previous
|
||
reconstruction.
|
||
- decode-direct-to-GVRAM's stated advantage is that **"no RAM reference frame
|
||
is needed"**, because the previous frame is already in GVRAM.
|
||
|
||
So every direct frame silently invalidates the reference the next compose frame
|
||
reads. Simulated on the Singe window at the crossover the plan specifies
|
||
(`tools/analysis/10_pathmix_drift.py`): **70 of 120 frames display pixels no
|
||
correct player would display**, first at frame 2, worst frame 18.8% of the
|
||
screen. This is FINDINGS 26 in different clothing — two code paths disagreeing
|
||
about what "the previous frame" means — and it is the **sixth** false premise
|
||
this project has caught before it shipped.
|
||
|
||
Every coherent repair is worse than not mixing at all:
|
||
|
||
| strategy | median | p90 | max | correct |
|
||
|---|---:|---:|---:|---|
|
||
| mix per frame, as specified | 36.6% | 53.6% | 53.6% | **no** |
|
||
| mix, direct also writes the RAM reference | 53.6% | 68.4% | 81.4% | yes |
|
||
| mix, re-read GVRAM into RAM on each switch | 36.6% | 107.2% | 107.2% | yes, 13 frames miss |
|
||
| compose only | 53.6% | 53.6% | 53.6% | yes |
|
||
| **direct only** | **36.6%** | 62.5% | 76.6% | yes |
|
||
|
||
(Costs in that table are 24.5's own model, for like-for-like comparison; 28.2
|
||
replaces the model itself.)
|
||
|
||
**24.5 also compared the wrong two things.** Its 53.6% and 76.6% are both
|
||
*copies* measured in `blit.s` — neither includes decoding. A real compose path
|
||
costs decode-into-RAM **plus** the 53.6% blit, so it is strictly dearer than
|
||
decoding straight into GVRAM, whatever the block mix. There was never a
|
||
crossover to find.
|
||
|
||
**The decoder therefore implements one path, direct-to-GVRAM**, and drops the
|
||
96 KB RAM reference frame entirely.
|
||
|
||
### 28.2 The four block modes do not cost the same, and V4 is the expensive one
|
||
24.5's model — "76.6% of a frame x the non-SKIP fraction" — prices every
|
||
non-SKIP block as one `movem.l` burst. Measured separately, with synthetic
|
||
single-mode frames (`tools/bench/prep_dlx.py`):
|
||
|
||
| mode | cycles/block | vs the 24.5 model (207.8) |
|
||
|---|---:|---:|
|
||
| SKIP, in an all-SKIP header byte | 13.3 | model says 0 |
|
||
| SKIP, inside a mixed byte | ~45 | model says 0 |
|
||
| V1 (one 4x4 codeword) | **299.9** | 1.44x |
|
||
| V4 (four 2x2 codewords) | **448.2** | 2.16x |
|
||
| RAW (16 literal indices) | **400.4** | 1.93x |
|
||
|
||
Applied to the real per-frame histograms (`tools/analysis/11_cpu_budget.py`),
|
||
the model reproduces all four frames timed on the 68000 to within **1
|
||
percentage point**, and shows 24.5 to be **2.03x optimistic at the median**.
|
||
|
||
Where the cycles actually go over the window:
|
||
|
||
| mode | % of blocks | % of cycles |
|
||
|---|---:|---:|
|
||
| SKIP | 46.4% | 9.2% |
|
||
| V1 | 19.8% | 26.1% |
|
||
| V4 | **25.2%** | **49.7%** |
|
||
| RAW | 8.5% | 15.0% |
|
||
|
||
**V4 is a quarter of the blocks and half the cycles.** It costs 1.49x a V1 block
|
||
while the mode decision in `vq_hybrid.py` charges it only its 4x payload bytes.
|
||
The lagrangian trades distortion against *bytes*; on this machine it now has to
|
||
trade distortion against *cycles* as well.
|
||
|
||
### 28.3 The container is big-endian but not aligned, and that is an address error
|
||
The DLX1 header docstring says every multi-byte field is big-endian "so the
|
||
68000 reads them with a plain `move`". Alignment is the other half of that
|
||
sentence and the container does not have it: frame records are
|
||
`[u32 length][768-byte mode header][payload]` laid end to end with arbitrary
|
||
payload lengths, so record boundaries land on odd addresses.
|
||
|
||
`move.l (a0)+,d0` at an odd address is an **address error** on a 68000 — not a
|
||
slow read. The first run decoded frame 0 perfectly, consumed exactly its 8,715
|
||
payload bytes, then read frame 1's length at `$03220F` and vectored into the IPL
|
||
at `$FF059A`, where it sat for 59 emulated seconds looking like an infinite
|
||
loop. It was found by dumping PC and the address registers, not by reading the
|
||
source: the code was correct, the data layout was not.
|
||
|
||
The decoder now rounds each record start up to 4. **The container should carry
|
||
the padding itself** so a streaming player can DMA records into place: measured
|
||
cost on this window is **199 bytes over 120 frames — 1.66 B/frame, 20 B/s**
|
||
against a 110 KB/s budget. Until `encode.py` does it, `prep_dlx.py` realigns at
|
||
load time.
|
||
|
||
### 28.4 The measurements agree with hand-derived MC68000 timings
|
||
As in FINDINGS 24, each figure was derived from the instruction timing tables
|
||
before being believed. A V1 block, summing dispatch, index decode, the indexed
|
||
`movem.l` load and four `movem.l` stores, plus its quarter share of the header
|
||
loop: **298.5 cycles derived against 299.9 measured — 0.5%.** RAW derives to
|
||
396 against 400.4 measured (1%). V4 derives to 415 against 448 (7%, the gap
|
||
being in the indexed two-register `movem.l`, the mode this decoder uses most
|
||
heavily). So these are 68000 cycles, not a MAME artefact.
|
||
|
||
### 28.5 A full frame does not fit at 12fps in ANY mode
|
||
An all-V1 frame — the cheapest possible way to redraw all 3,072 blocks — costs
|
||
**921,187 cycles, 110.5% of the budget**. All-V4 is 165.2% and all-RAW 147.6%.
|
||
|
||
So the ceiling is structural, not a tuning problem: **at 12fps on a 10MHz 68000
|
||
no more than ~88% of the screen can change in one frame**, however cheaply it is
|
||
coded. Scene cuts change 100% of it. Either a cut gets one late frame (the
|
||
outgoing content is unrelated, so this may be free to the eye), or cuts have to
|
||
be spread across two frame times, or the framerate has to come down — at 10fps
|
||
the budget is 1,000,000 cycles and an all-V1 frame fits.
|
||
|
||
### 28.6 What this does not measure
|
||
One 10 s window of one stream at one profile, and MAME still models no GVRAM
|
||
wait states. The `scsi` profile will be worse: FINDINGS 25.5 has it collapsing
|
||
to RAW under stress, and RAW is 1.93x the old model's block. Nothing here has
|
||
been run on `00020` or on quiet content, where the median frame is far cheaper.
|
||
|
||
### 28.7 The profiles are an I/O axis; the CPU limit is the clock
|
||
`sasi` and `scsi` are two points on one rate-distortion curve, chosen against
|
||
**disk bandwidth**. They say nothing about CPU, and the locked target CPU is a
|
||
stock 10MHz 68000 for both. So both have to fit the same 833,333 cycles:
|
||
|
||
| | sasi | scsi |
|
||
|---|---:|---:|
|
||
| stock / Super, 10 MHz | median 74.4%, **31% of frames miss** | median 94.9%, **42% miss** |
|
||
| XVI, 16.67 MHz | median 44.6%, 0% miss | median 56.9%, 0% miss |
|
||
|
||
Clocks confirmed from MAME 0.277 `x68k.cpp:1133/1194/1200`: `x68000` and
|
||
`x68ksupr` are **both** `40_MHz_XTAL/4` = 10 MHz, and only `x68kxvi` is faster
|
||
at `33.33_MHz_XTAL/2`. The Super has SCSI at 10 MHz, so a faster bus does not
|
||
imply a faster CPU — the XVI column above is headroom, not a target.
|
||
|
||
`sasi` is the cheaper profile, but choosing it is not a fix: it still misses
|
||
31% of frames. The cycle ceiling has to be enforced in the encoder either way.
|
||
|
||
**How much of the miss is the encoder's to fix.** Re-coding every non-SKIP block
|
||
as V1 — the cheapest mode, quality ignored — is the floor any mode assignment
|
||
can reach:
|
||
|
||
| | frames that miss | recoverable by re-coding | impossible at 12fps |
|
||
|---|---:|---:|---:|
|
||
| `sasi` | 37/120 | 26 | **11** (from 89.8% non-SKIP up) |
|
||
| `scsi` | 51/120 | 39 | **12** (from 91.9% non-SKIP up) |
|
||
|
||
So a cost-aware mode decision can reach about **three quarters** of the misses.
|
||
The remaining ~10% of frames are 28.5's ceiling in practice: past ~90% non-SKIP
|
||
no mode assignment fits, because the blocks have to be drawn at all. Those
|
||
frames need a structural answer — a late frame at a cut, a cut spread over two
|
||
frame times, or a lower framerate — not a better encoder.
|
||
|
||
### 28.8 V4 costs more cycles than RAW, so it is CPU-dominated by it
|
||
448.2 against 400.4 cycles. RAW is also pixel-exact where V4 is lossy, so V4's
|
||
only advantage is that it costs 4 payload bytes instead of 16. **On the CPU
|
||
axis V4 is strictly dominated**, which inverts the mode preference the byte
|
||
lagrangian has: an encoder short of cycles but not of bytes should buy RAW
|
||
wherever it would have bought V4, and gain quality doing it.
|
||
|
||
That escape is only open to the byte-rich profile. `scsi` already spends 41.3%
|
||
of its blocks on RAW (FINDINGS 25.5 saw it "collapse to RAW under stress" and
|
||
read that as a failure; on the CPU axis it is the cheap direction). `sasi` at
|
||
110 KB/s cannot afford it, so its only lever is V4 -> V1 -> SKIP, every step of
|
||
which costs quality. **The CPU constraint therefore bites harder on `sasi` in
|
||
quality terms even though it bites less in cycles.**
|
||
|
||
Caveat: this ordering is a property of *this* decoder, not of the codec. V4's
|
||
cost is four indexed `movem.l` lookups; pairing sub-block rows into
|
||
`movem.l d0/d2,(a4)` would save ~16 of 448 cycles, which narrows the gap to RAW
|
||
without closing it.
|
||
|
||
---
|
||
|
||
## 29. Trading bytes for cycles: the bus has 4x the headroom the CPU has (session 7)
|
||
|
||
> **ALSO SUPERSEDED IN PART BY 38.** "The bus has 4x the headroom the CPU has"
|
||
> is about the SCSI pipe. The 68000's LOCAL bus is a different resource and the
|
||
> decoder occupies 86.7% of it, so trading cycles for bytes is not free in the
|
||
> currency that turned out to bind. 29.6's DMAC idea is costed in 39.
|
||
>
|
||
> **SUPERSEDED IN PART BY 30, which measured it.** The mode survives and the
|
||
> conclusion holds, but every number in this section moved: a span costs 43.7
|
||
> cycles + 9.152/pixel *only* in an encoder-assisted format (the obvious
|
||
> decoder is 97.9 + 10.46), spans beat V1 from runs of 4 blocks and not 2, and
|
||
> the re-priced trade-off is 52.0% median / 10 misses, not 43.0% / 8. Read 30's
|
||
> tables over 29.3's. 29.5's other three items are still open, and 29.6 stands.
|
||
>
|
||
> **STATUS AT THE TIME: DERIVED, NOT MEASURED.** No 68000 had executed a span
|
||
> decoder. The per-pixel figure it rests on *is* measured (FINDINGS 24 V1) but
|
||
> at full row width; the per-span overhead was hand-derived. FINDINGS 4 is why
|
||
> it was labelled and then tested rather than believed.
|
||
|
||
FINDINGS 28 leaves the project CPU-bound while the **bus sits 4x idle**: `sasi`
|
||
spends 110 KB/s of a 488 KB/s pipe. That asymmetry is exploitable, because the
|
||
codec was designed when bytes were the scarce thing and every one of its
|
||
decisions trades cycles to save them.
|
||
|
||
### 29.1 The decoder pays per changed PIXEL; the disk pays per BYTE
|
||
Per-pixel costs, all measured:
|
||
|
||
| what | cycles/pixel | source |
|
||
|---|---:|---|
|
||
| write-only floor (no source read) | 4.59 | FINDINGS 24 V3 |
|
||
| **row-linear copy from word-expanded RAM** | **9.08** | FINDINGS 24 V1 |
|
||
| block-order copy, same bytes | 12.98 | FINDINGS 24 V4 |
|
||
| V1 codebook block | 18.74 | FINDINGS 28.2 |
|
||
| RAW, byte literals unpacked to words | 25.03 | FINDINGS 28.2 |
|
||
| naive per-pixel byte expansion | 26.13 | FINDINGS 24 V2 |
|
||
|
||
Two structural facts fall out. **The 1024-byte stride costs 43%** — the same
|
||
bytes cost 12.98 cycles/px in 4x4 block order against 9.08 row-linear, because
|
||
the stride breaks the `movem.l` burst. And **unpacking bytes to words costs more
|
||
than the write itself**: 25.03 against 9.08.
|
||
|
||
So the two cheapest things a decoder can be handed are *word-expanded* pixels
|
||
in *row-linear runs* — and both cost bytes on disc, which is what we have.
|
||
|
||
### 29.2 Codebooks are a byte optimisation that now costs cycles
|
||
A word-expanded literal 4x4 block, `movem.l (a0)+,d0-d7` straight from the
|
||
stream buffer into GVRAM, derives to **~240 cycles** — cheaper than V1's
|
||
measured 299.9, and pixel-exact. V1 is dearer *because* it is compressed: it
|
||
pays an index decode and an indexed `movem.l` that a literal does not, and then
|
||
does exactly the same four writes. It buys 31 bytes and spends 60 cycles.
|
||
|
||
**Every codebook mode is CPU-dominated by a literal.** V4 was already dominated
|
||
by RAW (28.8); with word-expanded literals available, so is V1. The VQ codebook
|
||
earns its place only while bytes are scarce.
|
||
|
||
### 29.3 Row-linear literal spans, priced against the real mode maps
|
||
Replace the per-block escape with a per-row **span**: `(x, count, word-expanded
|
||
pixels)`, decoded with `movem.l` bursts. A run of L horizontally adjacent dirty
|
||
blocks becomes 4 spans of 4L pixels, deriving to `4 * (50 + 4L * 9.08)` cycles
|
||
against `300L` for V1 — **cheaper for any run of 2 blocks or more**, at 32 bytes
|
||
per block instead of 1.
|
||
|
||
Applied greedily (buy the best cycles-saved-per-byte until the bus budget is
|
||
gone) to the *unchanged* mode maps of the `sasi` Singe window:
|
||
|
||
| | today | + literal spans |
|
||
|---|---:|---:|
|
||
| median frame | 74.4% | **43.0%** |
|
||
| p90 frame | 115.1% | **83.6%** |
|
||
| worst frame | 136.2% | **106.2%** |
|
||
| frames missing the budget | **37/120** | **8/120** |
|
||
| bitrate | 101.7 KB/s | 453.2 KB/s (bus 488) |
|
||
|
||
And the fit is structural rather than lucky: **spans get cheaper exactly where
|
||
blocks get expensive.** A span amortises its overhead over a long run, and long
|
||
runs are what a high-change frame is made of. The frames that miss today are the
|
||
frames spans help most.
|
||
|
||
### 29.4 This reopens 28.5, which said a scene cut cannot fit
|
||
28.5 concluded that no mode assignment fits a 100%-changed frame at 12fps,
|
||
because the cheapest full redraw available — all-V1 — is 110.5%. That was true
|
||
of *the mode set the codec has*. Adding a byte-expensive, cycle-cheap mode
|
||
changes the arithmetic: mixing a fraction `x` of the frame as spans against V1
|
||
for the rest,
|
||
|
||
- CPU needs `x >= 0.19`
|
||
- the 40,977 B/frame bus budget allows `x <= 0.39`
|
||
|
||
**The interval is not empty.** A scene cut fits at 12fps if roughly a quarter to
|
||
a third of it arrives as word-expanded row-linear literals. 28.5's "structural
|
||
ceiling" was a ceiling of the bitstream, not of the machine.
|
||
|
||
### 29.5 What has to be measured before any of this is believed
|
||
1. **Span cost on the 68000.** The 50-cycle per-span overhead is derived, and
|
||
the 9.08 cycles/px is measured at *full row width* with 12-register bursts —
|
||
a short or oddly-aligned span cannot burst as well, so short spans are
|
||
flattered here. Extend `tools/bench/blit.s` with a span variant and measure
|
||
it against run length. **This is the load-bearing number.**
|
||
2. **Re-run the ring-buffer simulation at ~450 KB/s.** FINDINGS 21's zero
|
||
required prefill was established at 110 and 280 KB/s against a 488 KB/s pipe.
|
||
At 453 the margin is a tenth of what it was, and 21's own caveat was that the
|
||
test is cumulative — it needs redoing, not extrapolating.
|
||
3. **Confirm the 4 Mbps figure**, which is user-supplied with no recorded
|
||
provenance and which this design would run at 93% of. It has been a "would be
|
||
nice" since session 1; a design that leans on it makes it load-bearing.
|
||
4. **Confirm DMA, not PIO** (STATUS priority 5). At 453 KB/s a PIO fallback puts
|
||
the transfer cost on the CPU we are trying to relieve. Cheapest check
|
||
available and now the most consequential.
|
||
|
||
### 29.6 The other lever, not yet costed: let the DMAC do the copy
|
||
The X68000 has an HD63450 DMAC (4 channels, `x68k.cpp:1046`). Channel 3 is
|
||
ADPCM — confirmed, `adpcm_drq_tick` asserts `drq3_w` — but memory-to-memory
|
||
transfer on a free channel would take the GVRAM copy off the CPU entirely,
|
||
leaving it only the parsing. This is the one idea here that could move the
|
||
budget without spending a single extra byte.
|
||
|
||
It cannot be settled in MAME: like the SCSI/SASI devices (BENCHMARK.md), the
|
||
HD63450 is a functional model, so a timing number out of it would measure the
|
||
emulator's scheduler. It needs hand-derivation against the datasheet plus real
|
||
hardware — the same three-tier approach the disk benchmark already documents.
|
||
|
||
## 30. The span, measured: the mode survives, and it is an encoder format (session 8)
|
||
|
||
FINDINGS 29 priced a new decoder mode at `4 * (50 + 4L*9.08)` cycles and marked
|
||
the whole section DERIVED. This is the measurement. `tools/bench/blit.s` gained
|
||
two span variants, `tools/bench/prep_spans.py` generates one stream per span
|
||
length, `tools/bench/span.lua` times them, and `tools/bench/span.sh` runs the
|
||
lot, and the whole thing takes about 25 seconds.
|
||
|
||
Same scope as every 68000 figure since FINDINGS 24: instruction cycles against
|
||
MAME's zero-wait-state GVRAM, interrupts masked. A **lower bound**, not a
|
||
prediction.
|
||
|
||
### 30.1 What was measured
|
||
Twelve `v5` configs and eleven `v6` configs, each cutting the **same** 256x192
|
||
frame into spans of a different length, so the work differs only in how finely
|
||
it is cut. Regressing `cycles = A*spans + B*pixels` over a set reads the
|
||
per-span overhead and the per-pixel cost straight off.
|
||
|
||
Every config draws the whole picture, the picture is cleared before each run and
|
||
snapshotted after, and all 23 snapshots are checked pixel-exact by
|
||
`tools/bench/verify_frame256.py`. A config cannot time fast by writing nothing.
|
||
|
||
| | per span | per pixel | fit error |
|
||
|---|---:|---:|---:|
|
||
| **v5** — decoder handed `(x, npix)`, works the copy out | **97.9** | **10.459** | ±1.4%, and only on spans that are a whole number of bursts |
|
||
| **v6** — encoder hands it an address and a jump | **43.7** | **9.152** | **±0.3% over all 11 lengths** |
|
||
| *29's assumption* | *50.0* | *9.080* | — |
|
||
|
||
**29's arithmetic was right about a format nobody had written yet.** v6 hits it
|
||
almost exactly; v5 — the obvious decoder, and the one 29 was describing — is
|
||
2.24x dearer per span and 14% dearer per pixel.
|
||
|
||
### 30.2 Why the difference is a format difference, not an optimisation
|
||
v5's record is `(x, npix)`, so the decoder computes the destination, divides
|
||
`npix` into 16-pixel bursts, and handles the 0..15 remainder: about 122 cycles
|
||
of arithmetic and branching per span before a single pixel moves. All of it is
|
||
known at encode time.
|
||
|
||
v6's record is `{u32 absolute GVRAM address, u16 jump displacement}` and nothing
|
||
else. The displacement jumps into an unrolled chain of eleven 24-pixel copy
|
||
units, so a span of any supported length is straight-line code with no loop, no
|
||
remainder, and no address arithmetic — `move.l (a0)+,a2` / `move.w (a0)+,d0` /
|
||
`jmp v6ch(pc,d0.w)`, then `movem.l` pairs. GVRAM is at $C00000 on every X68000,
|
||
so absolute destinations are a legitimate thing to bake into a stream.
|
||
|
||
Two consequences of that format, both cheap:
|
||
- **Span lengths are multiples of 24 pixels** and a run pads up to it. The
|
||
padding costs bytes and its own pixels, nothing else, and it is *correct on
|
||
screen*: a literal span carries true pixels of the current frame, so painting
|
||
a clean neighbour is a no-op visually.
|
||
- **A span may overrun the visible 256 pixels of its row by up to 23.** Free:
|
||
the line stride is 1024 bytes and only the first 512 are displayed, so the
|
||
overrun lands in the invisible half of the line.
|
||
|
||
### 30.3 The remainder path is where a short span actually dies
|
||
v5's cost per span, measured, against its length:
|
||
|
||
| span | 4 px | 8 px | 12 px | 16 px | 20 px | 24 px | 32 px |
|
||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||
| cycles/span | 180.3 | 240.9 | 296.3 | 261.8 | 347.7 | 401.9 | 430.7 |
|
||
| cycles/pixel | 45.08 | 30.11 | 25.46 | **16.36** | 17.65 | 17.27 | **13.46** |
|
||
|
||
A 12-pixel span costs *more* than a 16-pixel one. Everything below the 16-pixel
|
||
burst width goes through `move.l`/`move.w` at roughly 10 cycles a pixel plus the
|
||
per-span overhead, and 29's warning that "short spans are flattered" was
|
||
correct — but the fix is to pad them up to a burst, not to avoid them. v6 has no
|
||
remainder path at all, which is most of why its fit is linear to 0.3%.
|
||
|
||
### 30.4 Registers are the reason the per-pixel cost moved
|
||
FINDINGS 24's 9.08 cycles/pixel came from a fixed blit with 12 registers free
|
||
for `movem.l` and no live state. A span decoder keeps a stream pointer, a
|
||
destination and counters live, so v5 can spare only 8 registers per burst — 32
|
||
bytes instead of 48 — and pays 10.46 cycles/pixel for it. v6 gets back to 12
|
||
registers precisely because the encoder holds the state instead, and lands at
|
||
9.152. **The per-pixel figure is a function of how much the decoder has to
|
||
remember**, which is not something the FINDINGS 24 measurement could have shown.
|
||
|
||
Two smaller results, both cheap and both worth having on the record:
|
||
- **Odd-`x` alignment is free.** Spans starting at an odd pixel run their bursts
|
||
at `addr mod 4 == 2` and cost 259.0 cycles/span against 261.8 aligned — inside
|
||
the timing granularity. The 68000's 16-bit bus does not care, as expected;
|
||
now it is measured rather than assumed.
|
||
- **A full-row span is 154 cycles per 4x4 block**, the floor this mode can
|
||
reach, against V1's measured 299.9.
|
||
|
||
### 30.5 Re-pricing: the trade holds, and it is smaller
|
||
`tools/analysis/12_span_tradeoff.py` now runs on measured constants. Same greedy
|
||
(buy the best cycles-saved-per-byte until the bus budget is gone), same
|
||
unmodified mode maps, same Singe window:
|
||
|
||
| | today | 29 (derived) | **30 (measured)** |
|
||
|---|---:|---:|---:|
|
||
| `sasi` median frame | 74.4% | 43.0% | **52.0%** |
|
||
| `sasi` worst frame | 136.2% | 106.2% | **108.7%** |
|
||
| `sasi` frames missing | 37/120 | 8/120 | **10/120** |
|
||
| `sasi` bitrate | 101.7 KB/s | 453.2 | **448.0 KB/s** |
|
||
| `scsi` median frame | 94.9% | 69.4% | **74.6%** |
|
||
| `scsi` frames missing | 51/120 | 18/120 | **25/120** |
|
||
|
||
And the break-even moved. Cycles per 4x4 block in a run of L blocks, v6, with
|
||
each of the run's 4 spans padded to a whole 24-pixel unit:
|
||
|
||
| L | 1 | 2 | 4 | 8 | 16 | 64 |
|
||
|---|---:|---:|---:|---:|---:|---:|
|
||
| cycles/block | 1053 | 527 | **263** | 242 | 176 | 154 |
|
||
|
||
So a run beats all-V1 (299.9) **from L=4 up**, not from L=2 as 29.3 claimed, and
|
||
runs of 1-3 blocks all cost the same 1053 cycles because they pad to the same
|
||
single unit. A cost-aware mode decision should not offer a span below 4 blocks
|
||
at all.
|
||
|
||
### 30.6 29.4 survives: a scene cut still fits at 12fps
|
||
Mixing a fraction `x` of a 100%-changed frame as full-row spans against V1 for
|
||
the rest, on measured costs (154 cycles and 33.4 bytes per block):
|
||
|
||
- CPU needs `x >= 0.196`
|
||
- the 40,977 B/frame bus budget allows `x <= 0.373`
|
||
|
||
The interval is not empty — narrower than 29.4's 0.19..0.39, same conclusion.
|
||
FINDINGS 28.5's "a scene cut cannot fit" was a ceiling of the bitstream, not of
|
||
the machine, and that now rests on a measurement. `12_span_tradeoff.py` prints
|
||
this arithmetic and will say so if it ever stops being true.
|
||
|
||
### 30.7 What this does NOT settle
|
||
The three remaining items of 29.5 are unchanged and are now **more** load-bearing,
|
||
because the measured design runs at 448 KB/s of a 488 KB/s pipe rather than 453:
|
||
re-run the ring-buffer simulation at that rate, confirm the 4 Mbps figure's
|
||
provenance, and confirm DMA rather than PIO. A PIO fallback would put a 448 KB/s
|
||
transfer back on the CPU this mode exists to relieve.
|
||
|
||
Also unmeasured: **the parse cost of a span-heavy stream**. Every figure here
|
||
times the copy. The 68000 also has to read the mode map and dispatch: the
|
||
re-priced `sasi` stream buys 8773 spans across 120 frames, a mean of 73 a frame,
|
||
and each one's three-instruction dispatch is inside the fitted 43.7 — but the
|
||
mode-map walk that decides a span exists is not. `decode.s` does not implement
|
||
spans yet.
|
||
|
||
## 31. The mode decision can see cycles now, and it costs 0.26 dB (session 8)
|
||
|
||
FINDINGS 28 left the decoder missing 31% of frames at `sasi` and 42% at `scsi`
|
||
while the mode decision minimised `D + lam*R` — distortion against BYTES — on a
|
||
machine whose binding budget is CYCLES. This is the second controller.
|
||
|
||
`vq_hybrid.decide(ctx, lam, mu)` now minimises `D + lam*bytes + mu*cycles`, and
|
||
`ratectl.encode_rate_controlled(cycle_budget=...)` bisects `mu` per frame
|
||
against 833,333 cycles with the `lam` bisection nested inside it.
|
||
`tools/analysis/13_cpu_ratectl.py` measures what it costs.
|
||
|
||
### 31.1 The result
|
||
Worst sustained window, 120 frames, same targets, same quality floors:
|
||
|
||
| | PSNR | KB/s | CPU median | CPU max | frames missing |
|
||
|---|---:|---:|---:|---:|---:|
|
||
| `sasi` bytes only | 27.22 dB | 109.5 | 74.4% | 136.2% | **37/120** |
|
||
| `sasi` + cycle ceiling | **26.95 dB** | 109.4 | 81.5% | 110.6% | **1/120** |
|
||
| `scsi` bytes only | 29.90 dB | 280.0 | 94.9% | 146.6% | **51/120** |
|
||
| `scsi` + cycle ceiling | **29.27 dB** | 278.6 | 99.6% | 110.6% | **1/120** |
|
||
|
||
**36 of 37 misses at `sasi` for 0.26 dB, 50 of 51 at `scsi` for 0.62 dB.** The
|
||
bitrate does not move: the byte controller still binds, and mu changes *which*
|
||
modes are bought rather than how many bytes.
|
||
|
||
`sasi` pays less quality than `scsi` because it had less to give up: it was
|
||
already short of bytes, so the cycle-cheap directions it takes (V4 -> V1, and
|
||
blocks it can afford to hold) were near where the byte lagrangian already sat.
|
||
28.8 predicted the shape of this and got the sign right.
|
||
|
||
Mode mix, `sasi`, bytes-only -> with the ceiling: SKIP 46.4 -> 47.1%,
|
||
V1 19.8 -> 23.0%, **V4 25.2 -> 20.3%**, RAW 8.5 -> 9.6%. At `scsi` the V4
|
||
collapse is dramatic — **15.0 -> 5.3%**, with RAW taking it at 41.3 -> 43.2%,
|
||
which is 28.8's inversion happening in practice: RAW is dearer in bytes and
|
||
cheaper in cycles, so a byte-rich profile buys its way out of V4.
|
||
|
||
Only **46 of 120 frames need any mu at all** at `sasi`; the median frame is
|
||
decided at mu=0 and is unchanged from session 6.
|
||
|
||
### 31.2 The one frame that cannot fit is the intra frame, not a hard case
|
||
Both profiles miss exactly one frame, both at 110.6% — the all-V1 floor of
|
||
FINDINGS 28.5 — and in both it is **frame 0**. It has no previous
|
||
reconstruction, so every block must be coded, which is the definition of a
|
||
100%-changed frame. A scene cut mid-stream is the same thing.
|
||
|
||
That is the correct behaviour rather than a failure, and it is worth being
|
||
explicit about why: at `MU_CLIFF` a block only becomes SKIP if holding the
|
||
previous reconstruction costs less than ~28,665 units of distortion. A frame
|
||
with nothing on screen worth holding stays fully coded and is emitted **late on
|
||
purpose**, exactly as a frame that will not fit at `LAM_CLIFF` is emitted over
|
||
budget. Freezing a cut to make a deadline is the worse failure.
|
||
|
||
### 31.3 28.7 was too pessimistic, and the reason is instructive
|
||
28.7 estimated that only ~three quarters of the misses were the encoder's to
|
||
fix — 26 of 37 at `sasi` — because re-coding every non-SKIP block as V1 still
|
||
missed 11 frames. Measured, the controller fixes **36 of 37**.
|
||
|
||
The gap is that 28.7's floor held the SKIP set fixed and asked "how cheap can
|
||
the blocks we already decided to draw be?". The real decision can also **move a
|
||
block to SKIP**, paying distortion for it, and above ~90% non-SKIP that is the
|
||
only lever left. So 28.7's floor was a floor for a fixed SKIP set, not for the
|
||
mode decision. Two conclusions of 28.7 stand: the profiles are an I/O axis and
|
||
both must fit the same 10 MHz budget.
|
||
|
||
### 31.4 SKIP is not a constant, and the way out is two cost functions
|
||
A SKIP block costs 13.25 cycles when all four blocks sharing its header byte are
|
||
SKIP (one `tst.b` clears the group) and ~45 in a mixed byte, so its price
|
||
depends on its neighbours — which a per-block lagrangian cannot see. Picking one
|
||
number is a real trade: 45 overcharges clustered SKIPs and pushes the encoder
|
||
away from the mode that saves the most cycles, 13.25 undercharges isolated ones
|
||
and lets frames overrun.
|
||
|
||
The resolution is that **the budget check does not have to use the same cost
|
||
function as the mode decision**. `decide()` uses 13.25 purely to *rank* modes
|
||
within a block, where the choice only scales the incentive (the V1-SKIP gap
|
||
moves 12% between the two candidates). The controller scores whole frames with
|
||
`vq_hybrid.cycles()`, the exact clustered rule, validated to 1 point against the
|
||
68000 — so the bisection converges on what the machine will really do, whatever
|
||
the ranking constant was. That function is now defined once and imported by
|
||
`11_cpu_budget.py`, rather than living in two places that can drift apart.
|
||
|
||
### 31.5 Both controllers are gated against decoder drift
|
||
The mu controller varies the mode map frame to frame exactly as the lam
|
||
controller does, so it is exposed to the FINDINGS 26.1 failure — an encoder
|
||
reporting a reconstruction the decoder will never produce. `09_ratectl_drift.py`
|
||
now runs **both** configurations and both report 0/120 drifting frames, 0.00 dB
|
||
overstatement. The CPU ceiling is on by default in `encode.py`
|
||
(`--no-cpu-fit` restores session 7 behaviour).
|
||
|
||
### 31.6 With spans on top, the window fits completely
|
||
Re-running the span pricing of FINDINGS 30 against a cost-aware container —
|
||
lever B first, then lever A on what it leaves:
|
||
|
||
| `sasi` | bytes only | + cycle ceiling | + ceiling + spans |
|
||
|---|---:|---:|---:|
|
||
| median frame | 74.4% | 81.5% | **56.8%** |
|
||
| worst frame | 136.2% | 110.6% | **91.5%** |
|
||
| frames missing | 37/120 | 1/120 | **0/120** |
|
||
| bitrate | 101.7 KB/s | 101.6 | 449.3 KB/s |
|
||
|
||
The intra frame lands at 91.5% — spans are what make a full redraw fit, which is
|
||
30.6's arithmetic arriving in a real container. That row is still a **model** of
|
||
a bitstream nothing implements; the two levers have never run on the 68000
|
||
together, and the ring-buffer question of 30.7 gets sharper at 449 KB/s.
|
||
|
||
## 32. SASI is dropped, and the reason is capacity, not bandwidth (session 9)
|
||
|
||
**USER DECISION**: drop the `sasi` profile. A SASI volume on this machine is
|
||
limited to 40 MB, and the game does not fit in one.
|
||
|
||
That ends the two-quality-mode decision of session 2. `scsi` is now the only
|
||
profile, and `encode.py --profile` has one choice. The retired 110 KB/s rate
|
||
point is not deleted from the record, for the reason in 32.3.
|
||
|
||
### 32.1 How much video there actually is
|
||
Measured off the source Blu-ray rather than recalled: the unique scene footage
|
||
is streams `00000`-`00201`, **1366.6 s = 22.8 min**. The longer streams
|
||
(`00215` 1376 s, `00216` 1152 s, `00223` 566 s) are compilations of the same
|
||
material and are not additional content — 00223 is the window every codec
|
||
measurement in this project has been taken on. Total across all 224 streams is
|
||
88.3 min, which is the figure to *not* quote.
|
||
|
||
22.8 min agrees with the ~22 min of laserdisc footage the arcade original is
|
||
usually credited with, which is the cross-check that the compilations really
|
||
are duplicates.
|
||
|
||
At the rates this codec has actually produced, including the 7.8 KB/s audio
|
||
allowance:
|
||
|
||
| stream | rate | whole game |
|
||
|---|---:|---:|
|
||
| retired 110 KB/s profile | 109.4 KB/s | **146.0 MiB** |
|
||
| `scsi`, measured (FINDINGS 31) | 278.6 KB/s | **371.8 MiB** |
|
||
| `scsi` + spans (MODEL, 31.6) | 449.3 KB/s | **599.6 MiB** |
|
||
|
||
### 32.2 Where the 40 MB actually comes from
|
||
It is not a bus-addressing limit. MAME 0.277's `src/mame/sharp/x68k_hdc.cpp`
|
||
builds the SASI LBA from a 6-byte Group-0 CDB as
|
||
`(cmd[1] & 0x1f) << 16 | cmd[2] << 8 | cmd[3]` — **21 bits of 256-byte blocks,
|
||
so 512 MiB is addressable per unit**. `call_create` makes a 20 MB image
|
||
(`0x13c98` blocks) because that is what a period drive was.
|
||
|
||
So the 40 MB ceiling is a **Human68k / IPL volume-format and period-drive
|
||
limit**, not something the SASI command set imposes. That distinction does not
|
||
rescue the profile: four units at 40 MB is 160 MiB, and 146.0 MiB of video
|
||
would consume essentially the entire SASI address space of the machine at the
|
||
*lowest* rate this codec has ever produced, leaving nothing for Human68k, the
|
||
player, or the game's own data.
|
||
|
||
*Scope: the 21-bit CDB and the 256-byte block are read out of MAME's
|
||
implementation. The 40 MB volume figure is the user's, and is consistent with
|
||
Human68k's SASI partitioning; it has not been measured here.*
|
||
|
||
### 32.3 The rate point may come back, under a different name
|
||
Dropping SASI removes an interface, not a bitrate, and the two are on different
|
||
axes — the profile axis has been I/O bandwidth only since FINDINGS 28.7. The
|
||
awkward part is that **capacity and bandwidth now pull in opposite directions**:
|
||
|
||
- the only period medium with room for 371.8 MiB (let alone 599.6) is
|
||
**CD-ROM** at 540-650 MB, and
|
||
- a **1x CD-ROM sustains ~150 KB/s**, which is *below* the surviving 280 KB/s
|
||
profile and much nearer the rate that was just retired.
|
||
|
||
A SCSI hard disk has the bandwidth but has to be large for the era at 372 MiB.
|
||
**The user's call was to ship `scsi` as the only profile now and settle the
|
||
medium when the pipe is measured** — the blocked disk benchmark
|
||
(`docs/BENCHMARK.md`) and the DMA-vs-PIO check of FINDINGS 29.5.
|
||
|
||
**Correction to the framing above, found after that call was made.** The
|
||
medium is less open than this section first presented it. FINDINGS 21.2 already
|
||
committed the deployment target to **SD-backed SCSI (BlueSCSI / SCSI2SD)**, in
|
||
session 2, and that is the premise the whole 488 KB/s constant rests on. On SD
|
||
there is no capacity problem at any rate this codec produces — an SD card is
|
||
gigabytes — and no seek tail either. So:
|
||
|
||
- **Capacity does not choose between the survivors.** It killed SASI, whose
|
||
40 MB ceiling is a Human68k volume-format limit that SD emulation does not
|
||
lift, and it does not bind on SD-backed SCSI at all.
|
||
- **CD-ROM is the one that capacity rules out**, not in. With spans the stream
|
||
is 487.1 KB/s = **650.1 MiB** for the whole game, past a CD's ~620 MiB
|
||
usable — and 487 KB/s is more than 3x a 1x CD-ROM's ~150 KB/s. A CD-ROM
|
||
delivery would mean giving up the span lever *and* re-deriving a profile
|
||
around 150 KB/s.
|
||
|
||
So the open question is not "which medium" but the one FINDINGS 29.5/30.7
|
||
already had: **confirm the 488 KB/s figure's provenance, and confirm DMA**.
|
||
`profile_gen.py` exists precisely to re-derive a profile from a measured
|
||
bandwidth once there is one.
|
||
|
||
### 32.4 What MAME says about the SCSI path that survives
|
||
Read out of MAME 0.277 while settling 32.2, and directly relevant because the
|
||
medium decision is now the thing gating the profile:
|
||
|
||
**The CZ-6BS1's DMA is real and fully modelled.** `x68k_scsiext.cpp` wires the
|
||
MB89352's DREQ to the expansion slot and replaces the data register at
|
||
`$EA0015` with DMA-aware glue: on a DMA cycle (`m_slot->exown()`, driven by
|
||
`m_hd63450->own()`) a read goes to `spc->dma_r()` and #DTACK is negated until
|
||
DRQ asserts. `x68k.cpp:1114` closes the loop the other way
|
||
(`out_dtack_callback -> hd63450_device::dtack_w`). That is a genuine
|
||
DMAC-driven transfer with hardware flow control, on the **stock `x68000`
|
||
driver** — the one MAME marks working. This is the configuration FINDINGS 29.5
|
||
asked about, and the answer for this board is **DMA, not PIO**.
|
||
|
||
**The internal SCSI of the Super/XVI/030 is NOT modelled that way**, and it is
|
||
a trap. `x68k.cpp:1176` reads, verbatim, `// TODO: duplicate DMA glue from
|
||
CZ-6BS1`. So MAME's internal SCSI is PIO-only. A benchmark run on `x68ksupr`
|
||
would measure a PIO fallback the real machine does not have — on top of those
|
||
drivers already being MACHINE_NOT_WORKING (FINDINGS 28.7). **Benchmark
|
||
`x68000 -exp1 cz6bs1`, not `x68ksupr`.**
|
||
|
||
**CD-ROM is a first-class SCSI device on the internal bus** — `x68k.cpp:1168`
|
||
puts an `NSCSI_CDROM` at ID 6 by default — but the CZ-6BS1 card's own device
|
||
list offers `harddisk` only. So the CD-ROM delivery route of 32.3 is
|
||
emulatable, but not on the board whose DMA is modelled, without a source
|
||
change.
|
||
|
||
None of this is a transfer RATE. `docs/BENCHMARK.md`'s split still holds and is
|
||
worth restating because 32.3 defers a decision to a measurement: MAME can
|
||
settle whether the path works and whether it is DMA, and **cannot** settle
|
||
KB/s, because its device models are functional rather than transfer-timing
|
||
accurate. The rate half of the medium question needs derivation or real
|
||
hardware, not a longer MAME run.
|
||
|
||
## 33. The container carries its own alignment: DLX1 -> DLX2 (session 9)
|
||
|
||
The encoder gap left open since session 7 (FINDINGS 28.3, STATUS item 4) is
|
||
closed. `encode.py` now emits **DLX2**, which pads every frame record up to a
|
||
4-byte boundary — the first one included, by padding the codebook tables so
|
||
`off_frm` is aligned. `dlx.py` reads both versions; DLX1 containers stay
|
||
readable because every measurement in FINDINGS 28-31 was taken on one.
|
||
|
||
Measured on the same 120-frame window:
|
||
|
||
| | record starts not 4-aligned | padding cost |
|
||
|---|---:|---:|
|
||
| DLX1 (through session 8) | **94/120** | 0 (the loader added 180 B) |
|
||
| DLX2 (now) | **0/120** | 160 B = 1.33 B/frame = **16 B/s** |
|
||
|
||
16 B/s against 278.6 KB/s is 0.006% of the stream. The thing it buys is not
|
||
speed: an odd `move.l (a0)+` on a 68000 is an **address error**, which vectors
|
||
into the IPL and presents as an infinite loop, not as a slow read. That is the
|
||
bug that cost session 7 an afternoon.
|
||
|
||
`tools/bench/prep_dlx.py` still realigns at load time and now says whether it
|
||
had to — `0/120 record starts unaligned -- the container carries its own
|
||
padding` on a DLX2 input. It is kept rather than deleted because it is what
|
||
makes the session 7-8 containers decodable, and those are the containers the
|
||
published timings belong to.
|
||
|
||
**Cross-check that this changed nothing else:** re-encoding the `scsi` window
|
||
with the DLX2 writer reproduces FINDINGS 31.1 exactly — 29.27 dB, 278.6 KB/s,
|
||
median 99.6% / max 110.6% of a 12fps frame, 1/120 frames missing. The padding
|
||
is additive; it does not touch the mode decision.
|
||
|
||
## 34. The cost model, checked against the machine on a cost-aware container (session 9)
|
||
|
||
STATUS item 1. Everything in FINDINGS 31 was the validated cost MODEL
|
||
(`vq_hybrid.cycles`) applied to a container it had never been checked against —
|
||
the 1-point validation of 28.2 belongs to the *session 7* stream. This is the
|
||
cost-aware container timed on the emulated 68000, same harness, same scope
|
||
(instruction cycles, zero-wait-state GVRAM, interrupts masked; a LOWER BOUND).
|
||
|
||
| anchor | non-SKIP | model | measured | error |
|
||
|---|---:|---:|---:|---:|
|
||
| min non-SKIP | 15.4% | 254,683 cyc / 30.6% | 262,751 / 31.5% | **-3.07%** |
|
||
| median | 53.2% | 681,199 / 81.7% | 690,251 / 82.8% | **-1.31%** |
|
||
| p90 | 75.7% | 832,116 / 99.9% | 834,213 / 100.1% | **-0.25%** |
|
||
| max non-SKIP | 100.0% | 921,293 / 110.6% | 921,187 / 110.5% | **+0.01%** |
|
||
| whole 120-frame mean | — | 649,089 / 77.9% | 657,081 / 78.8% | **-1.22%** |
|
||
|
||
The model holds, and its error is **signed**: it under-predicts by 1-3% on light
|
||
frames and converges to exact on heavy ones. That is the right direction to be
|
||
wrong in for a ceiling controller — the bisection is tightest where the model is
|
||
most accurate — but it means the median frame is ~1 point dearer than FINDINGS
|
||
31 reports, not cheaper.
|
||
|
||
The four synthetic single-mode frames reproduce session 7 exactly: all-V1
|
||
110.5%, all-V4 165.2%, all-RAW 147.6%, all-SKIP 4.9%. Those are properties of
|
||
`decode.s`, not of the container, so agreeing across two different streams is
|
||
the cross-check that the harness is measuring what it claims.
|
||
|
||
### 34.1 The 23-minute "hang" was the buffering trap again
|
||
The session-8 note said this run "was still going at 12 minutes of CPU". It was
|
||
re-run here and sat at 99.9% CPU for **23 minutes** with a 0-byte log, then was
|
||
killed. Re-launched under **`stdbuf -oL`** with `-seconds_to_run 60`, the
|
||
identical plan completed in about **25 seconds of wall time** and printed every
|
||
line as it went — MAME reports `Average speed: 528.72% (52 seconds)`, so the
|
||
whole plan needs ~52 emulated seconds and the machine runs it at 5x realtime.
|
||
|
||
The lesson is the one already in STATUS, one level deeper: it is not enough to
|
||
write MAME's output to a file instead of a pipe. **A file is block-buffered too**,
|
||
so a long MAME run is unobservable until it exits, and an unobservable run that
|
||
is merely finishing looks exactly like one that is wedged. Session 8 lost the
|
||
measurement to that, and session 9 lost 23 minutes to it before spending 25
|
||
seconds getting the answer. **`stdbuf -oL` on every MAME job that prints
|
||
progress.**
|
||
|
||
## 35. The CPU budget has never had the disk in it (session 9)
|
||
|
||
> **TESTED BY 38 AND IT STANDS.** Session 10 first argued that the flat
|
||
> subtraction here is too pessimistic -- that the disk DMA could hide in bus
|
||
> cycles the CPU was not using -- and scored the same window at 53/120 instead
|
||
> of 84/120. **That was wrong.** A 68000 has no cache and a two-word prefetch
|
||
> queue, so it stalls as soon as another master takes the bus; DMA time is
|
||
> additive, which is exactly what this section assumed. The 84/120 stands and
|
||
> 38.3 now reproduces it.
|
||
|
||
**Raised by the user: "PIO is such a CPU killer. DMA is not. I'm concerned about
|
||
us drawing the wrong conclusions."** The concern is correct, and it is larger
|
||
than the labelling question of 32.4. This is the seventh false premise this
|
||
project has caught, and the most expensive one.
|
||
|
||
Every CPU figure in FINDINGS 24 through 34 is measured against **833,333 cycles
|
||
per frame**, the full 10 MHz clock divided by 12 fps. Nothing has ever been
|
||
subtracted from it for moving the bitstream off the disk. The decoder has been
|
||
scored as though the data arrives for free.
|
||
|
||
### 35.1 What the transfer actually costs
|
||
`profile_gen.py` has carried `DMA_CLOCKS_PER_WORD = 8` since session 2 (FINDINGS
|
||
5, an **ESTIMATE** from HD63450 timing, never measured) and prints a "DMA steal"
|
||
line — but that line was only ever compared against the 38.3% *blit* figure of
|
||
FINDINGS 17, which FINDINGS 24 superseded and which was never the decoder cost.
|
||
It was never debited from the decoder budget.
|
||
|
||
At the rates that matter, on a 10 MHz 68000:
|
||
|
||
| stream | DMA @ 8 clk/word | PIO, unrolled (~12 clk/B) | PIO, byte loop (~20 clk/B) |
|
||
|---|---:|---:|---:|
|
||
| `scsi`, 278.6 KB/s | **11.4%** | 34.2% | 57.1% |
|
||
| `scsi` + spans, 487.1 KB/s | **20.0%** | 59.9% | **99.8%** |
|
||
|
||
The PIO columns are hand-derived floors, not measurements: a byte from an I/O
|
||
register plus a store is 16 cycles on a 68000 before any loop overhead. They are
|
||
here to size the risk, and the size of the risk is that **PIO at the span rate
|
||
consumes the entire machine**.
|
||
|
||
### 35.2 What that does to the conclusions of FINDINGS 31
|
||
Debiting the DMA steal — the *cheap* case, the one we are hoping for:
|
||
|
||
| | KB/s | steal | budget left | median | p90 | worst | fits? |
|
||
|---|---:|---:|---:|---:|---:|---:|:--|
|
||
| `scsi` today | 278.6 | 11.4% | 738,238 | **112.4%** | 112.9% | 124.8% | **no** |
|
||
| `scsi` + spans | 487.1 | 20.0% | 667,070 | **98.3%** | 102.8% | 114.3% | **no** |
|
||
|
||
FINDINGS 31's headline — "1 frame of 120 misses" — is measured against a budget
|
||
with no I/O in it. With DMA debited the surviving profile does not fit at all:
|
||
the *median* frame is over. And 31.6's "with spans the window fits completely"
|
||
becomes a worst frame of 114.3%, because **the span lever buys cycles by
|
||
spending bandwidth, and the bandwidth comes back out of the CPU as steal.**
|
||
Spans still help — 112.4% -> 98.3% at the median, 14 points — but they no longer
|
||
close the gap on their own.
|
||
|
||
### 35.3 Why this is not settled by the DMA finding of 32.4
|
||
32.4 established that the CZ-6BS1's DMA *path* exists and is modelled. Three
|
||
things it does not establish, and all three are load-bearing:
|
||
|
||
1. **DMA vs PIO is a property of OUR player, not of the board.** The hardware
|
||
supports DMA; if the player reads through IOCS and IOCS does PIO, we get PIO
|
||
and the table above. `docs/BENCHMARK.md` item 4 already proposed driving the
|
||
MB89352 registers directly for exactly this reason — that is now not an
|
||
optimisation but the difference between fitting and not.
|
||
2. **8 clocks per word has never been measured.** It is now the single most
|
||
load-bearing unmeasured number in the project: at 8 the port is marginal, at
|
||
12 it is dead, at 4 it is comfortable. It comes from a datasheet reading in
|
||
session 2 and nothing has checked it since.
|
||
3. **MAME cannot settle it.** Its device models are functional, not
|
||
transfer-timing accurate (BENCHMARK.md), and it models no GVRAM wait states
|
||
either — so a MAME run can confirm the transfer is a DMA cycle and cannot
|
||
price it. This needs derivation from the HD63450 and MB89352 datasheets, or
|
||
real hardware.
|
||
|
||
### 35.4 What this does and does not overturn
|
||
It does **not** overturn the decoder measurements: 300/448/400 cycles per block
|
||
and the model validation of FINDINGS 34 are properties of `decode.s` and stand
|
||
unchanged. What it overturns is every statement of the form "N frames of 120
|
||
miss the budget", because the budget was wrong. Those all need re-running
|
||
against `833,333 * (1 - steal)` once `steal` is a measurement rather than a
|
||
datasheet estimate.
|
||
|
||
It also sharpens the framerate question of STATUS item 5 considerably. At 10 fps
|
||
the budget is 1,000,000 cycles and the same DMA steal is proportionally smaller
|
||
per frame, which is now a much stronger argument for 10 fps than "one late frame
|
||
per cut" ever was.
|
||
|
||
### 35.5 `11_cpu_budget.py` now debits it, and 10 fps absorbs it
|
||
The tool takes `--io dma|pio|none` (default **dma**) and prints the budget it is
|
||
actually scoring against. On `tmp/rc_fr_singe_scsi_cpufit.dlx`:
|
||
|
||
| `--io` | budget left | median | worst | frames missing |
|
||
|---|---:|---:|---:|---:|
|
||
| `none` — the pre-session-9 premise | 833,333 | 99.6% | 110.6% | **1/120** |
|
||
| `dma` (8 clk/word, estimated) | 738,234 | 112.4% | 124.8% | **84/120** |
|
||
| `pio` (12 clk/B, floor) | 548,036 | 151.4% | 168.1% | **120/120** |
|
||
|
||
`--io none` prints a warning naming FINDINGS 35, so the old number cannot be
|
||
produced by accident.
|
||
|
||
**At 10 fps and DMA the same container goes back to 1/120** — median 93.7%,
|
||
worst 104.0%. That is conservative, because it holds the 12 fps byte rate: a
|
||
real 10 fps encode carries ~17% fewer bytes per second, so the steal falls too.
|
||
|
||
This changes what the framerate decision (STATUS item 5) is *for*. It was a
|
||
quality question about one late frame per scene cut. It is now the lever that
|
||
pays for the disk, and on current estimates it is the difference between a
|
||
stream that fits and one that misses 70% of its frames.
|
||
|
||
## 36. A `scsi` window does not fit in the machine the test rig emulates (session 9)
|
||
|
||
Swapping the decoder gate onto the surviving profile's container made it fail —
|
||
`frame 119 not pixel-exact: 49,005 px differ`. That is not a decoder bug and not
|
||
the DLX2 change. **The container does not fit in RAM.**
|
||
|
||
`tools/bench/decode.lua` loads the entire stream into emulated memory at
|
||
`STREAM = 0x30000`, and the locked target is a stock **2 MB** machine:
|
||
|
||
| container | stream | ends at | verdict |
|
||
|---|---:|---:|---|
|
||
| session 7-8 `sasi` | 1,108,888 B | 0x13EB98 = 1.25 MB | fits |
|
||
| `scsi` cost-aware | 2,840,860 B | 0x2E591C = 2.90 MB | **overruns 0x200000 by 940 KB** |
|
||
|
||
The loader wrote 940 KB past the top of memory, the decoder then parsed whatever
|
||
that reads back as, and the run neither completed its sequential pass nor drew
|
||
the right picture. Every 68000 decode verification before session 9 was done on
|
||
a container small enough to fit by accident — the `sasi` profile was a third the
|
||
bitrate, so nobody met this.
|
||
|
||
**This is a property of the test rig, not of the player.** The shipping player
|
||
streams from disk into a ring buffer and holds seconds of video, not minutes.
|
||
But it does bound what the rig can prove: at 278.6 KB/s, a 2 MB machine holds
|
||
about **6.7 seconds** of stream, so the strongest test in the tree can only ever
|
||
audit a prefix of a window.
|
||
|
||
The fix keeps the test honest rather than making it pass:
|
||
- `prep_dlx.py` truncates the frame list to what fits, **prints that it did**,
|
||
and takes `--ram` / `--all-frames`. On the `scsi` window it keeps **80 of
|
||
120** frames.
|
||
- `verify_decode.py` takes `--nframes` so the reference decoder replays exactly
|
||
the prefix the 68000 decoded, instead of running 40 frames ahead of it.
|
||
- `check.sh` reads the count back out of `decode_meta.lua` and passes it through,
|
||
and now **fails loudly if the sequential pass did not complete** — the missing
|
||
`snapshot taken` marker — instead of reporting a pixel diff against a
|
||
half-drawn screen. That guard is what turned this from a mystery into a
|
||
five-minute diagnosis.
|
||
|
||
Verifying a prefix is still a real test: SKIP blocks make every frame a claim
|
||
about the one before it, so frame 79 is only correct if all 80 were. What is
|
||
lost is coverage of the last 40 frames, and the honest way to get it back is to
|
||
gate on more than one window rather than to pretend one pass covers everything.
|
||
|
||
**The timing confirms the diagnosis.** Truncated to 80 frames the pass completes
|
||
in **8 emulated seconds** and the frame is pixel-exact; the model predicts
|
||
~6.6 s for 80 frames at this container's cost, so that is the expected number.
|
||
The 120-frame run that overran RAM could not finish the same work in **44**.
|
||
A decoder reading garbage does not run slowly for an interesting reason — it was
|
||
parsing lengths out of unmapped memory and walking wherever they pointed. Any
|
||
"the decoder is 4x slower than the model on RAW-heavy streams" conclusion drawn
|
||
from that run would have been entirely false, which is the third time in this
|
||
session that an unobservable run nearly produced a wrong finding.
|
||
|
||
## 37. A second emulator, and MAME is not running the core we thought (session 10)
|
||
|
||
Every 68000 cycle figure in FINDINGS 24-35 came from one instrument. This is a
|
||
second one, run against byte-for-byte the same `decode.bin` and the same
|
||
container.
|
||
|
||
`tools/bench/c68k/` links **px68k's C68K core** into a headless harness: a
|
||
hand-built X68000 memory map, no SDL, no ROMs, no emulated machine. The decoder
|
||
touches nothing but RAM, the control block and GVRAM, so the machine around it
|
||
was never part of the measurement.
|
||
|
||
### 37.1 What the two instruments actually are
|
||
**MAME 0.277's `M68000` is not Musashi.** `src/devices/cpu/m68000/m68000.lst`
|
||
plus `m68000gen.py`: it is the microcode core, where timing emerges from the
|
||
modelled micro-sequence and 4-clock bus cycles. C68K is a static per-instruction
|
||
cycle table (`ORI_CLOCKS_*` / `EA_CLOCKS_*` in `c68kmacro.h`), hand-transcribed
|
||
from the Motorola manual by a different author.
|
||
|
||
Those are two different ways of arriving at a number, which is what makes the
|
||
agreement worth something. It would be worth much less if both were tables.
|
||
|
||
### 37.2 The harness is self-validating
|
||
It decodes all 80 frames and dumps the screen; `verify_c68k.py` checks it
|
||
against `tools/encoder/dlx.py` **pixel for pixel, on palette indices**. That is
|
||
the licence for the cycle numbers: the harness rebuilds px68k's memory model
|
||
from scratch -- byte-swapped RAM (`mem_wrap.c:420`), GVRAM word writes that
|
||
discard the high byte -- and any of it being subtly wrong would still print
|
||
plausible cycles. It could not print a pixel-exact 80-frame temporal recursion.
|
||
|
||
It does. **`decode.s` is now pixel-exact under two independent CPU cores.**
|
||
|
||
### 37.3 The numbers
|
||
```
|
||
anchor MAME C68K delta MAME C68K of a 12fps frame
|
||
min non-SKIP 42.8% 600982 620760 +3.29% 72.1% 74.5%
|
||
median 65.2% 841038 869036 +3.33% 100.9% 104.3%
|
||
p90 72.9% 836124 856872 +2.48% 100.3% 102.8%
|
||
max non-SKIP 100.0% 921187 923090 +0.21% 110.5% 110.8%
|
||
synthetic all-SKIP 40729 40946 +0.53% 4.9% 4.9%
|
||
synthetic all-V1 921187 923090 +0.21% 110.5% 110.8%
|
||
synthetic all-V4 1376881 1420754 +3.19% 165.2% 170.5%
|
||
synthetic all-RAW -- 1273298 -- 152.8%
|
||
```
|
||
|
||
The `all-RAW` cell is empty because MAME's timed pass did not reach it. That is
|
||
an operational fact worth recording: with `-video soft -nothrottle` this box runs
|
||
`x68000` at about **0.033x realtime**, so `decode.lua`'s eight anchors plus two
|
||
full passes — ~48 emulated seconds — cost ~25 minutes of wall clock, and two runs
|
||
were killed by their own `timeout`. The C68K harness does the same work in
|
||
seconds because it emulates a CPU and not a machine. **Anchor MAME runs by wall
|
||
clock, not by `-seconds_to_run`.**
|
||
|
||
**Cycle-table error is bounded at 3.3%, and it runs against us** -- C68K reads
|
||
high on every anchor. Nothing here rescues FINDINGS 35. The disagreement is
|
||
mode-dependent (all-V1 +0.21%, all-V4 +3.19%), so it localises to the V4 path's
|
||
indexed two-register `movem.l`, not to a systematic clock difference.
|
||
|
||
**FINDINGS 28.8 is confirmed independently**: under C68K, V4 (170.5%) still
|
||
costs more than RAW (152.8%). That conclusion inverts the encoder's mode
|
||
preference, so having it from a second core matters more than most.
|
||
|
||
### 37.4 What it does not settle
|
||
px68k has no bus-timing model anywhere in `x68k/*.c` -- grep it. Neither
|
||
instrument charges GVRAM wait states, so this is **the same lower bound,
|
||
measured twice**. It bounds cycle-table error. It says nothing about the
|
||
distance to a real X68000; that is still BENCHMARK.md Tier 3.
|
||
|
||
### 37.5 One trap, recorded because it will catch the next person
|
||
C68K is 64-bit-unsafe by construction: its `MOVEM` macros do
|
||
`src = (UINT32)(&D0)` -- they truncate the host address of the register file and
|
||
dereference it -- and `C68k_Set_Fetch` keeps the opcode-fetch base in a `UINT32`.
|
||
Under the default PIE the binary loads near `0x555555550000` and the first
|
||
`movem` segfaults. The Makefile builds `-no-pie` and the harness mmaps its arena
|
||
`MAP_32BIT`. Both are load-bearing, not tidiness.
|
||
|
||
## 38. The bus, measured: the project is bus-bound, not CPU-bound (session 10)
|
||
|
||
> **This supersedes part of 29 and part of 35.** FINDINGS 29's "the bus has 4x
|
||
> the headroom the CPU has" is true of the SCSI pipe and false of the 68000's
|
||
> local bus, and they are different resources. FINDINGS 35's flat CPU debit for
|
||
> the disk charges the CPU for bus cycles it was not going to use.
|
||
|
||
Everything since FINDINGS 24 has been costed in CPU clocks. The 68000 has
|
||
another budget nobody had counted: its **memory bus**, one 4-clock cycle at a
|
||
time, carrying instruction prefetch as well as data.
|
||
|
||
### 38.1 Two sources that check each other
|
||
`tools/bench/c68k/c68k_bench` counts every Read/Write callback the C68K core
|
||
makes -- exact, because C68K splits a long access into two word calls, which is
|
||
what the 16-bit bus does. It cannot count **instruction prefetch**: C68K reads
|
||
opcodes straight through a host pointer with no callback, and MAME exposes no
|
||
fetch count either.
|
||
|
||
So `tools/analysis/15_bus_occupancy.py` derives prefetch by walking
|
||
`decode.s`'s straight-line paths in `tools/bench/decode.lst` and multiplying by
|
||
each frame's mode histogram. The same walk also predicts the data half -- and
|
||
that half is measurable:
|
||
|
||
```
|
||
measured mean 66,700 data bus cycles/frame
|
||
derived mean 66,672 error -0.04% mean, 0.06% worst
|
||
```
|
||
|
||
The walk reproduces the measurement, so its prefetch figure stands on the same
|
||
footing. `15_bus_occupancy.py` exits non-zero if that check ever stops holding.
|
||
|
||
### 38.2 The result
|
||
```
|
||
mean median worst frame
|
||
bus slots in a frame 201,497 211,013 230,772
|
||
data accesses 66,672 68,044 105,216
|
||
instruction prefetch 108,002 110,982 122,910
|
||
total bus cycles 174,674 181,998 193,248
|
||
bus OCCUPANCY 86.7% 86.8% 88.3%
|
||
slots left for a DMAC 26,823 26,618 21,115
|
||
```
|
||
|
||
**The decoder occupies 86.7% of its own bus, and prefetch is 62% of that.** A
|
||
data-only count understates occupancy by about 2x, which is exactly the mistake
|
||
an instrumented emulator would lead you into.
|
||
|
||
Per mode, bus clocks against measured clocks: V1 204/299.9 (68%), V4 308/448.2
|
||
(69%), RAW 316/400.4 (79%), and the v6 span **9.0/9.152 (98%)**.
|
||
|
||
### 38.3 What that does to the frame budget -- and one wrong turn
|
||
The first thing done with 86.7% was to argue that FINDINGS 35's flat CPU debit
|
||
for the disk is too pessimistic: the decoder leaves ~26,800 bus slots a frame
|
||
idle against the disk's ~23,000, so score it as contention,
|
||
`frame = max(CPU clocks, 4 x bus cycles)`, and the window misses 53/120 rather
|
||
than 84/120.
|
||
|
||
**That is wrong, and the MC68450 manual is what says so.** A 68000 relinquishes
|
||
the bus on BGACK and cannot execute without it -- no cache, a two-word prefetch
|
||
queue that empties immediately. Worse, the DMAC does not interleave at operand
|
||
granularity by default: limited-rate auto-request hands it the bus in *bursts*
|
||
of `2(BT+4)` clocks out of a sample period of `2(BT+BR+5)`, taking
|
||
`2^-(BR+1)` of the bandwidth in slabs (MC68450 sect 5.2.3.2, Fig 5-2). During a
|
||
slab the CPU is stopped.
|
||
|
||
So **DMA time is additive to CPU time**, which is what FINDINGS 35 assumed all
|
||
along. `14_dmac_chain.py` reproduces its 84/120 exactly in the `today` column.
|
||
|
||
What 86.7% *does* say is worse than the thing it appeared to rescue: **there is
|
||
almost no room to overlap anything.** The 13.3% of bus slots the decoder leaves
|
||
idle are single gaps inside a `movem`-heavy loop, not windows a bus master can
|
||
be handed. Any design whose case rests on DMA hiding under CPU work on this
|
||
machine should be assumed dead until measured on hardware.
|
||
|
||
The measurement still earns its place: it is what prices the span painter
|
||
against a DMAC in 39, and it is the reason the answer there came out the way it
|
||
did.
|
||
|
||
### 38.4 What is not counted
|
||
Bus arbitration. The 68000's BR/BG/BGACK handover costs cycles a cycle-steal
|
||
DMA cannot avoid, and the disk debit here embeds it only insofar as FINDINGS 5's
|
||
8 clocks/word already does. Also: no GVRAM wait states, as everywhere since 24.
|
||
Both make the real occupancy **higher** than 86.7%, not lower.
|
||
|
||
## 39. The DMAC chain against the span: the datasheet says no (session 10)
|
||
|
||
FINDINGS 29.6 named "let the DMAC do the copy" the one lever that could move the
|
||
budget without spending a byte, and left it uncosted. This costs it, and the
|
||
answer is **no** -- but only after the constants came from the MC68450 manual
|
||
rather than from bus arithmetic, which is the whole lesson of the section.
|
||
|
||
### 39.1 They are the same container
|
||
v6's record is `{u32 absolute GVRAM address, u16 jump displacement}` = 6 bytes.
|
||
An MC68450/HD63450 **array-chaining entry** is `{u32 memory address, u16
|
||
transfer count}` = 6 bytes. Set the channel dual-address, direction
|
||
device->memory, Sequence Control counting both addresses up: MAR reloads per
|
||
entry (the GVRAM destination), DAR walks the stream buffer, MTC is the span's
|
||
word count. **The chain array IS the span table.** Every byte figure in
|
||
FINDINGS 30 carries over, and this is not a fork in the format -- the encoder
|
||
emits the same thing either way, only the executor changes. That much is real
|
||
and survives everything below.
|
||
|
||
### 39.2 The first answer was wrong by a clock
|
||
Session 10 first derived the DMAC's cost from bus arithmetic: moving a pixel is
|
||
a read cycle plus a write cycle, 2 bus cycles, 8 clocks, against v6's measured
|
||
9.152 -- a 12.6% edge. On that basis the design scored 1/120 frames over budget
|
||
against v6's 10/120 and looked decisive.
|
||
|
||
The datasheet does not agree. **MC68450 Fig 4-25 sheet 4**, dual address /
|
||
operand size WORD / device size 16 bits, D->M or M->D:
|
||
`{WORD READ, WORD WRITE}` = **9 CLOCKS**. Confirmed by the long-operand row, two
|
||
of each for 18. And **Fig 4-25 note 2** says why: the DMAC's reads take four
|
||
clocks and its **writes take five**. The 68000 writes in four.
|
||
|
||
| per pixel | clocks | source |
|
||
|---|---:|---|
|
||
| DMAC, dual-address word, two 16-bit ports | **9.000** | MC68450 Fig 4-25 sheet 4 |
|
||
| v6 `movem` chain | **9.152** | MEASURED, FINDINGS 30 |
|
||
|
||
**1.7%.** One clock on every DMAC write is the entire difference between a
|
||
12.6% win and a rounding error. Per span, sequential array chaining costs
|
||
**36 clocks** (Fig 4-25 sheet 1: three word reads for the 6-byte entry, plus
|
||
reload) against v6's measured 43.7 -- the DMAC's one genuine edge, and it is
|
||
7.7 clocks.
|
||
|
||
### 39.3 Scored additively, as 38.3 requires
|
||
```
|
||
today v6 span v6 fine tail DMAC chain
|
||
bitrate KB/s 270.8 479.2 479.9 479.9
|
||
frame, median 108.1% 99.3% 96.5% 95.0%
|
||
frame, worst 114.7% 112.0% 111.4% 110.3%
|
||
frames missing 84/120 55/120 18/120 12/120
|
||
blocks spanned/frame 0 727 839 845
|
||
```
|
||
`today` reproduces FINDINGS 35's 84/120 exactly, which is the check that the
|
||
scenario lines up.
|
||
|
||
### 39.4 What the DMAC actually buys, and who else can sell it
|
||
`v6 fine tail` is the decomposition. v6 pads every span up to 24 pixels because
|
||
its copy is an unrolled chain of 12-register `movem` units; adding a second,
|
||
finer chain of 2-register units caps the padding at 3 pixels instead of 23, for
|
||
the price of some more unrolled code and **nothing per span**. Priced
|
||
conservatively (a 4-pixel unit costs 56 clocks against a full unit's 220 for 24,
|
||
so it is dearer per pixel and paid at most once a span):
|
||
|
||
| | frames over |
|
||
|---|---:|
|
||
| v6 as built | 55/120 |
|
||
| **v6 with a finer chain tail -- software only** | **18/120** |
|
||
| DMAC chain | 12/120 |
|
||
|
||
**86% of the DMAC's advantage over v6 is the 24-pixel padding quantum**, and
|
||
that is a property of v6's unrolled chain, not of the CPU. The residual is 1.7%
|
||
a pixel and 7.7 clocks a span, worth 6 frames of 120.
|
||
|
||
Break-even against all-V1 moves the same way: v6 as built needs a run of 4
|
||
blocks, v6 with the finer tail needs 3, the DMAC needs 1.
|
||
|
||
### 39.5 The verdict
|
||
**Fix the quantum in software.** Six frames of 120 does not buy a reserved DMAC
|
||
channel, a two-region container layout, and a dependency on transfer timing that
|
||
cannot be verified in either emulator on this box. The `v6 fine tail` figure is
|
||
itself DERIVED and should be measured with `span.sh` before it is believed --
|
||
that is a day's work in a tool that already exists, against a hardware
|
||
dependency that needs an actual X68000.
|
||
|
||
Keep 39.1 on the record. If a later measurement moves the DMAC's per-pixel cost
|
||
below 8 clocks -- for instance if GVRAM tolerates a four-clock DMAC write in a
|
||
way the datasheet's typical-system assumption does not model -- the container
|
||
does not have to change to take advantage of it.
|
||
|
||
### 39.6 What else would have to be true, if it is ever revisited
|
||
- **A free channel.** Four exist; channel 3 is ADPCM (`adpcm_drq_tick` asserts
|
||
`drq3_w`) and the SCSI stream needs one.
|
||
- **Two regions per frame.** Chaining fetches entries from an array while DAR
|
||
walks the pixel data, so the span table and the literal words cannot be
|
||
interleaved as v6 interleaves them.
|
||
- **The mode-map walk stays on the CPU.** 39.3 charges it; FINDINGS 30.7 flagged
|
||
that 12_span_tradeoff.py did not.
|
||
|
||
### 39.7 A number the datasheet settled on the way past
|
||
FINDINGS 5's **8 clocks/word** for the SCSI DMA has been an unsourced estimate
|
||
since session 1 and STATUS has called it the most load-bearing unmeasured number
|
||
in the project. Fig 4-25 sheet 3 gives single-address `W/B READ` 4 clocks and
|
||
`W/B WRITE` 5; a device->memory disk transfer is one memory write. So it is
|
||
**5 clocks/word if the DMAC holds the bus** and about **12 if it arbitrates per
|
||
word** (front-end 5 best case / 8 worst, sect 4.5.2.1; back-end 2, sect 4.5.2.2).
|
||
The feature list's "up to 5 Megabytes per Second at 10 MHz, no wait states"
|
||
is the held-bus case: 2 bytes per 4-clock cycle.
|
||
|
||
**8 is the midpoint of a bracket the datasheet supports, not a guess.** Which
|
||
end applies depends on how the MB89352 drives REQ and whether cycle-steal-with-
|
||
hold is used, which is a design decision the player has not made yet -- and it
|
||
is worth 7 clocks a word on a 480 KB/s stream, so it is worth making
|
||
deliberately.
|
||
|
||
---
|
||
|
||
## 40. The finer chain tail, measured: v7 (session 11)
|
||
|
||
FINDINGS 39.4 attributed **86% of the DMAC array-chain's advantage over v6** to
|
||
one thing that is not a property of the DMAC at all -- v6's 24-pixel padding
|
||
quantum -- and derived that fixing it in software would take the `scsi` window
|
||
from 55/120 frames over budget to 18/120. It labelled that figure DERIVED and
|
||
said it should not be believed until `span.sh` measured it. This measures it.
|
||
|
||
### 40.1 The result
|
||
`tools/bench/blit.s` gains **v7**: v6's 24-pixel coarse chain with a second,
|
||
finer chain appended. Measured over thirteen span lengths by
|
||
`tools/bench/span.sh`, every one of which drew a **pixel-exact** frame:
|
||
|
||
```
|
||
cycles = 66.0 per span + 9.143 per COARSE pixel + 9.978 per FINE pixel
|
||
```
|
||
|
||
fitting all 13 lengths to within **0.2%** -- and the fit is not flattered by its
|
||
own configs, because the three-term model was identified on span lengths chosen
|
||
so that every fine remainder a real span can have (0, 4, 8, 12, 16, 20) appears.
|
||
v5 and v6 re-measure to 97.9 + 10.459 and 43.7 + 9.152, reproducing FINDINGS 30
|
||
exactly, so the harness has not drifted underneath the new variant.
|
||
|
||
| clocks per 4x4 block, run of L blocks | L=1 | L=2 | L=3 | L=4 | L=8 | L=64 |
|
||
|---|---:|---:|---:|---:|---:|---:|
|
||
| v6 as built | 1053 | 527 | 351 | 263 | 241 | 154 |
|
||
| **v7** | **424** | **292** | **248** | **226** | **183** | **151** |
|
||
| DMAC chain (datasheet) | 288 | 216 | 192 | 180 | 162 | 146 |
|
||
|
||
**Break-even against all-V1 (299.9) moves from L=4 to L=2 blocks.** 39.4
|
||
predicted L=3; the measurement is better than the derivation.
|
||
|
||
### 40.2 The scoring, and a derivation that was right for the wrong reasons
|
||
Rescoring the same `scsi` window with `14_dmac_chain.py`, the same additive
|
||
model, the same mode maps:
|
||
|
||
| | frames over budget |
|
||
|---|---:|
|
||
| today | 84/120 |
|
||
| v6 span as built | 55/120 |
|
||
| **v7, MEASURED** | **18/120** |
|
||
| DMAC chain (datasheet) | 12/120 |
|
||
|
||
**18/120, exactly the derived figure.** That agreement is a coincidence of two
|
||
cancelling errors, and it is worth writing down because a match this clean would
|
||
otherwise be read as confirmation:
|
||
|
||
- 39.4 assumed a **2-register `movem` tail**, derived at 56 clocks per 4 pixels
|
||
= 14.0 clocks/pixel. The real tail costs **9.978** -- 29% cheaper.
|
||
- 39.4 assumed the second entry point costs **nothing per span**. It costs
|
||
**22.3 clocks** (66.0 against v6's 43.7), because it is a second
|
||
`move.w (a0)+,d0` and a second `jmp`.
|
||
|
||
The per-pixel win and the per-span loss are within a frame of each other over
|
||
this window. **The conclusion survives; the reasoning behind it did not.**
|
||
|
||
### 40.3 The instruction the derivation should have picked
|
||
A 2-register `movem` is the obvious "smaller unit of the same thing", and it is
|
||
the wrong instruction. Per 4 pixels:
|
||
|
||
| tail unit | bus cycles | clocks | per pixel |
|
||
|---|---:|---:|---:|
|
||
| `movem.l (a0)+,d0-d1` + `movem.l d0-d1,(a2)` + `lea` | 14 | 56 | 14.0 |
|
||
| **2 x `move.l (a0)+,(a2)+`** | **10** | **40** | **~10.0** |
|
||
|
||
`movem` pays two instruction words and a `lea` to move what two of the plainest
|
||
instructions on the machine move with post-increment on both sides. v7's fine
|
||
unit is therefore **one `move.l (a0)+,(a2)+` = 2 pixels**, which also makes the
|
||
padding quantum **2** rather than 4 -- and a span is a run of 4x4 blocks, so its
|
||
length is always a multiple of 4 and **the padding is exactly zero**. 39.4's
|
||
"caps the padding at 3 pixels" was pessimistic by three pixels.
|
||
|
||
The derived bus model predicts the measurement well once the right instruction
|
||
is in it: 5 bus cycles = 20 clocks per 2 pixels = 10.0/pixel against 9.978
|
||
measured, and 54 bus cycles = 216 clocks per 24 against 9.143*24 = 219.4.
|
||
|
||
### 40.4 Where the fine displacement lives, and why it is not in the record
|
||
Two chains need two entry points, and the second one has to survive the coarse
|
||
copy. Holding it in a register would cost a payload register -- v6's whole
|
||
reason for a 24-pixel unit is that it has exactly 12 free (`d0-d6/a1/a3-a6`).
|
||
|
||
So it is not in the span record. **It is in the stream**, after the coarse
|
||
pixels and before the fine ones. The coarse chain falls out into
|
||
`move.w (a0)+,d0 / jmp v7fh(pc,d0.w)`, and at that instant `d0` is dead payload
|
||
and `a0` is pointing exactly at it. The decoder holds nothing extra across the
|
||
copy and keeps all 12 registers.
|
||
|
||
The record is still `{u32 absolute GVRAM address, u16 coarse displacement}`;
|
||
the container costs **2 more bytes per span**, which `14_dmac_chain.py` charges.
|
||
|
||
### 40.5 The verdict, now measured rather than argued
|
||
**FINDINGS 39.5 stands: fix the quantum in software, drop the DMAC.** v7 takes
|
||
back **37 of the 43 frames** the DMAC chain would, using an instruction sequence
|
||
that needs no reserved channel, no two-region container, and no transfer timing
|
||
that neither emulator on this box can verify. 39.1 still holds if that ever
|
||
changes: the chain array and the span table are the same six bytes.
|
||
|
||
### 40.6 The 13-minute run that measured nothing
|
||
`span.sh` ran for 13 minutes producing an empty log and zero snapshots, and the
|
||
same MAME command with a shorter `-seconds_to_run` completed the identical work
|
||
in 30 seconds. The cause is still not identified. What matters is that **the
|
||
run was unobservable in both directions**: MAME's stdout did not reach the log
|
||
until exit even under `stdbuf -oL`, and the snapshots -- the one artefact that
|
||
would have shown progress -- may themselves only land at exit.
|
||
|
||
So the bisection that resolved it did not chase the hang. It **shrank the
|
||
stimulus** instead: `tmp/spans_meta.lua` carries byte offsets into a blob that
|
||
`prep_spans.py` writes once, so deleting config lines from the metadata runs any
|
||
subset in seconds against the same unmodified stream file. v5 alone, v7 alone
|
||
and the full set at a shorter run all completed; the wedge never reproduced.
|
||
|
||
This is the fourth instance of the pattern FINDINGS 34.1 named, and it is the
|
||
first where **the instrument was unobservable but the thing being measured was
|
||
fine**. `span.sh` now runs at `-seconds_to_run 200`, measured at 30 s wall for
|
||
all 36 configs, and asserts the snapshot count against the number of configs in
|
||
the generated metadata rather than a literal 23 -- so adding a config can no
|
||
longer silently weaken the pixel-exactness gate.
|
||
|
||
## 41. v7 is in the player, and the model it is scored by was 18% wrong (session 12)
|
||
|
||
> **41.2's FRAMING IS SUPERSEDED BY 42.** This section ends by naming the rate
|
||
> point as a fork the user must choose. There is no fork: the span pass
|
||
> saturates at ~837 KB/s on its own, and the 488 KB/s ceiling it is scored
|
||
> against here was never a bus figure (42.1). The measurements below stand; the
|
||
> "two byte budgets" mechanism of 41.2 is what made 42 findable.
|
||
|
||
FINDINGS 40 measured v7 in `tools/bench/blit.s` and left it there. This builds it
|
||
into `src/player/decode.s`, defines the container that carries it, and scores
|
||
what the encoder actually delivers rather than what a selection model predicts.
|
||
Three things came out of it that were not on the list.
|
||
|
||
### 41.1 The decoder, and the format
|
||
`src/player/decode.s` gains `paint_spans`, which is `blit.s` v7 verbatim -- the
|
||
same instruction sequence, deliberately, because the 66.0/9.143/9.978 fit was
|
||
measured on that sequence and a tidier rewrite would silently invalidate it.
|
||
|
||
The container is **DLX3**: the span section sits between the 768-byte mode
|
||
header and the block payload, because that is the only place the 68000 can
|
||
reach without first parsing something of variable length.
|
||
|
||
```
|
||
u32 payload length
|
||
768 B mode header spanned blocks read SKIP
|
||
u16 nspans
|
||
nspans * { u32 GVRAM address, u16 coarse disp, c*48 B,
|
||
u16 fine disp, f*4 B }
|
||
block payload V1 -> 1 B, V4 -> 4 B, RAW -> 16 B
|
||
```
|
||
|
||
Every span record is a multiple of 4 bytes (4+2+48c+2+4f), so the section needs
|
||
no internal padding and the block payload starts aligned. `a1`, the mode-header
|
||
cursor, is one of v7's twelve payload registers, so it goes on the stack across
|
||
the pass: two long accesses a frame, against the 24 pixels a register buys per
|
||
chain unit.
|
||
|
||
**Pixel-exact under both CPU cores on the first run**, over a container where
|
||
every frame carries 128-216 spans painting up to 38% of the picture, with full
|
||
temporal recursion. `tools/analysis/16_span_roundtrip.py` is the new gate and it
|
||
is in `check.sh`: encode, write the container, read it back with the reference
|
||
decoder, compare to what the encoder recorded. It asserts it emitted enough
|
||
spans to have tested anything -- a round-trip over a span-less container is
|
||
green by vacuity, which is FINDINGS 40.6's lesson about the snapshot count.
|
||
|
||
### 41.2 There are TWO byte budgets, and conflating them hides the whole win
|
||
The first measured span encode looked like a regression: at the `scsi` profile
|
||
spans fired on 5 of 120 frames and bought almost nothing. The cause is not the
|
||
codec. **The lam search had already spent the byte allowance**, so the span pass
|
||
inherited a few hundred bytes of room.
|
||
|
||
FINDINGS 40's 18/120 was never scored at 280 KB/s. `14_dmac_chain.py` defaults
|
||
to `--bus 488` -- the PIPE -- and gives each frame 40,977 bytes. The profile's
|
||
is 23,228. **Those are two different budgets and only one of them is hardware.**
|
||
The profile is a chosen quality rate point; the pipe is a ceiling. Bytes between
|
||
the two buy a better picture if spent on `lam`, the 68000's deadline if spent on
|
||
spans, and nothing at all if left unspent.
|
||
|
||
So the encoder now takes both: `--kbps` sets the quality target and
|
||
`--span-kbps` the ceiling the span pass may draw on, flat per frame and not
|
||
banked, because a pipe cannot be saved up. The quality bucket is credited with
|
||
the BLOCK payload only -- charging it the span bytes drives it to its floor on
|
||
the first spanned frame and starves every later frame of quality for a budget
|
||
the spans were never drawing on.
|
||
|
||
Spans also run BEFORE `mu`, and that ordering is the point. Both controllers
|
||
make a frame decode in time; `mu` pays in quality and a span pays in bytes, and
|
||
a span carries literal source pixels so it *removes* that run's quantisation
|
||
error. Spending bytes we already have beats spending picture.
|
||
|
||
| 120-frame `scsi` window | KB/s | over budget | PSNR |
|
||
|---|---:|---:|---:|
|
||
| no spans | 278.3 | 86/120 | 29.27 dB |
|
||
| spans, profile budget only | 280.0 | 77/120 | 29.23 dB |
|
||
| **spans on the 488 KB/s pipe** | 487.7 | **34/120** | **29.63 dB** |
|
||
|
||
Scored with `17_span_delivered.py`, which reads the emitted span section and
|
||
prices exactly those spans -- no selection model at all -- in 14's additive
|
||
model: block decode + span painting + disk DMA.
|
||
|
||
### 41.3 The blit.s fit transfers into the player, to 0.2%
|
||
`prep_dlx.py` gained two synthetic all-SPAN frames (full-row runs, and 4-block
|
||
runs at the break-even). They price v7 inside `decode.s` against the constants
|
||
`span.sh` fitted in `blit.s`:
|
||
|
||
| | predicted | MAME | C68K | error |
|
||
|---|---:|---:|---:|---:|
|
||
| all-SPAN-64 (192 spans x 256 px) | 505,636 | 506,533 | 506,824 | +0.18% |
|
||
| all-SPAN-4 (3072 spans x 16 px) | 734,193 | 735,133 | 735,304 | +0.13% |
|
||
|
||
Per 4x4 block that is **151.2 and 225.6 clocks, against FINDINGS 40's table of
|
||
151 and 226**. The mode costs what it was said to cost, in the real decoder, on
|
||
two emulators.
|
||
|
||
### 41.4 The rig had been writing past the top of RAM
|
||
`prep_dlx.py` truncated the real frames to a RAM budget and then appended its
|
||
synthetic timing frames ON TOP, 26 KB past the 0x200000 top of a 2 MB machine.
|
||
Survivable while it lasted, because the modes it overran are data-independent:
|
||
reading junk payload costs a V1 or a RAW block exactly what reading pixels
|
||
costs, so the anchors timed correctly by luck.
|
||
|
||
**A span is not data-independent.** Its two jump displacements come out of the
|
||
stream, so an out-of-RAM span record jumps into open bus. The synthetic frames
|
||
are now built first and their size comes out of the budget, with an assertion
|
||
that the stream ends below the top of RAM.
|
||
|
||
### 41.5 C_SKIP_MIXED was never measured, and it was 18% low
|
||
Chasing a 3% gap between the model and the measured decode turned up the one
|
||
constant in `vq_hybrid`'s cost table that came from a derivation rather than a
|
||
measurement: **the cost of a SKIP block sharing its header byte with a coded
|
||
block.** It was 45.0 from session 7 to session 12. It is **55.0**.
|
||
|
||
Every other constant comes from a synthetic frame of a single mode, and there
|
||
was no such frame for a mixed SKIP, *because one cannot exist* -- the byte has
|
||
to hold a coded block for the SKIP to be mixed at all. So `prep_dlx.py` now
|
||
emits four frames that bracket it, (3 SKIP + 1 V1), (1 SKIP + 3 V1), and the
|
||
same pair with RAW, each pair solving for the SKIP cost and its partner's
|
||
together:
|
||
|
||
| | MAME | C68K |
|
||
|---|---:|---:|
|
||
| mixed SKIP, from the V1 pair | 55.03 | 56.50 |
|
||
| mixed SKIP, from the RAW pair | 55.83 | 56.50 |
|
||
| V1, solved back out | 300.66 | 300.50 |
|
||
|
||
The partner solves back to its own anchored value to 0.2%, which is what says
|
||
the pair is measuring the SKIP rather than absorbing it. 55.0 is taken because
|
||
every other constant in the table is MAME's.
|
||
|
||
**The header bytes ROTATE through all four positions, and that is load-bearing.**
|
||
`decode.s` reaches a block's mode bits with `lsr.b #6/#4/#2` and no shift at all
|
||
for the last one, so a block costs 52/48/44/34 clocks of dispatch depending on
|
||
where in its byte it sits. A fixed pattern like 0x01 pins every SKIP to the
|
||
three expensive slots and every V1 to the free one, and solving two such
|
||
equations returns a number that describes no real frame. The first attempt did
|
||
exactly that, and a single-parameter fit against real frames then "confirmed"
|
||
87.7 -- collinear with the span term, and wrong.
|
||
|
||
With the constant corrected the model predicts the measured decode of a real
|
||
spanned container to **-0.06% on the mean and 0.09% worst frame**, against
|
||
-2.99% and 4.30% as it stood.
|
||
|
||
It matters more than 10 clocks a block sounds, because **a span marks its run
|
||
SKIP**: a spanned container is made largely of mixed SKIPs, so this is the
|
||
dominant population in exactly the frames spans are judged on. It is imported
|
||
now, not copied, in `spans.py` and `14_dmac_chain.py`.
|
||
|
||
*Incidental, and it resolved a false lead:* RAW and V4 read 3.2-3.5% higher on
|
||
C68K than on MAME, on mixed and pure frames alike. That is FINDINGS 37's known
|
||
table spread, not a property of mixed bytes -- but comparing a C68K-derived
|
||
solve against a MAME-derived anchor made it look like one for an hour.
|
||
|
||
### 41.6 Frames-over-budget is not a safe headline any more
|
||
The delivered 34/120 against 14's simulated 18/120 is a **1.4% difference in
|
||
mean frame cost** (786,381 clocks against 774,356). The metric is that
|
||
sensitive because the rate controller *aims* at the deadline: 55 of 120 frames
|
||
land within 5% of it, and shifting every frame by 1% moves the count from 25 to
|
||
47.
|
||
|
||
| every frame shifted by | -3% | -2% | -1% | 0 | +1% | +2% | +3% |
|
||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||
| frames over budget | 18 | 21 | 25 | **31** | 47 | 61 | 63 |
|
||
|
||
This was a fair metric when nothing controlled to the budget. It is now a
|
||
measurement of where the controller aims, and any cost-model error is amplified
|
||
into a large count change -- which is how 41.5's 18% error stayed invisible.
|
||
**Report the cost distribution; quote the count only with its sensitivity.**
|
||
Add this to the §4 measurement traps.
|
||
|
||
## 42. The rate point dissolves: bytes are nearly free, and one constant decides everything (session 13)
|
||
|
||
Session 12 ended by naming the rate point as "the first real fork since the
|
||
profile was set" and asking the user to choose one. **The question turned out to
|
||
be mis-posed, and the user is what posed it correctly**: challenged on where the
|
||
488 KB/s constant came from, the answer is that it never came from the SCSI bus
|
||
at all, and the bus has roughly eight times that in it. Once bytes are that
|
||
cheap the rate point is not a choice, it is a saturation point the encoder finds
|
||
on its own -- and what actually decides whether the game runs is **the number of
|
||
clocks the SCSI DMA steals per word**, which is still unmeasured and now worth
|
||
more frames than every optimisation since FINDINGS 24 combined.
|
||
|
||
### 42.1 The 488 KB/s constant was never a bus figure
|
||
FINDINGS 18/21 recorded it as a user-supplied "4 Mbps" with no provenance, and
|
||
29.5 item 3 has carried "confirm the 4 Mbps figure" as an open item ever since.
|
||
Checked against the standard: **SCSI-1 (ANSI X3.131-1986) is an 8-bit bus at
|
||
~1.5 MB/s asynchronous and 5 MB/s synchronous** -- MB/s, not Mbps. The working
|
||
constant is 10% of the asynchronous rating and 3% of the deployment target's
|
||
(FINDINGS 21.2 committed to SD-backed SCSI in session 2).
|
||
|
||
So the pipe was never the binding resource on the I/O side. What binds is that
|
||
**a delivered byte is charged to the 68000's frame budget**, because the DMAC
|
||
stalls the CPU rather than overlapping with it (38.3). The pipe's units are
|
||
KB/s; the real currency is clocks.
|
||
|
||
### 42.2 The cheapest way to put a pixel on this screen is to not code it
|
||
Per pixel, in frame-budget clocks, with `W` the SCSI DMA's clocks per word:
|
||
|
||
| | wire bytes | paint | disk debit | total |
|
||
|---|---:|---:|---:|---|
|
||
| v7 literal span | 2 | 9.143 | `W` | `9.143 + W` |
|
||
| RAW block | 1 | 25.03 | `W/2` | `25.03 + 0.5W` |
|
||
| V1 block | 0.25 | 18.74 | `W/8` | `18.74 + 0.125W` |
|
||
|
||
A span pixel costs **two** wire bytes, not one: X68000 GVRAM in 256-colour mode
|
||
is one pixel per WORD with the high byte discarded (`spans.py:44`, and it is the
|
||
memory model both emulators are pixel-exact against), so the span path is a
|
||
straight `movem` copy that cannot pack. That 2x on bytes is exactly what buys
|
||
9.143 clocks/pixel.
|
||
|
||
**A span beats a RAW block for any `W < 31.7`**, i.e. everywhere in the 5..12
|
||
bracket of 39.7, and by a factor of nearly 2. The codec's expensive modes exist
|
||
to save bytes, and bytes have stopped being the scarce thing.
|
||
|
||
The budget is 833,333 clocks for 49,152 pixels = **16.95 clocks/pixel**. A
|
||
full-frame literal therefore needs `9.143 + W` to fit under ~16.1 after span and
|
||
all-SKIP overhead: **lossless fits iff `W` is about 6.5 or less.**
|
||
|
||
### 42.3 Measured: the span pass saturates at ~935 KB/s, 0.13 dB off the display
|
||
Encoded with `--kbps 280` held fixed and `--span-kbps` swept, so every column
|
||
below has the same quality target and differs only in what the span pass was
|
||
allowed to draw on. Scored with `17_span_delivered.py` over the emitted span
|
||
sections:
|
||
|
||
| container | KB/s | span px | PSNR | over @5 | @8 | @12 |
|
||
|---|---:|---:|---:|---:|---:|---:|
|
||
| `s12_280_off` | 278.6 | 0% | 29.27 dB | 76/120 | 86/120 | 109/120 |
|
||
| `rc_fr_singe_scsi_span` (the gate) | 487.7 | 26.6% | 29.63 dB | 11/120 | 34/120 | 78/120 |
|
||
| `s13_280p700` | 693.8 | 48.3% | 30.28 dB | 16/120 | 28/120 | 63/120 |
|
||
| `s13_280p1000` | 815.4 | 60.2% | 30.93 dB | 7/120 | 20/120 | 64/120 |
|
||
| `s13_280p1500` | 837.4 | 62.3% | 31.04 dB | **0/120** | 21/120 | 63/120 |
|
||
| `s13_lossless` (`--kbps 2000`) | 934.6 | 70.4% | **31.19 dB** | **0/120** | 47/120 | -- |
|
||
|
||
**`s13_280p1500` and `s13_280p2200` are byte-identical files.** The span pass
|
||
stops finding spans worth taking at ~837 KB/s; asking for more bytes returns the
|
||
same container. The rate point is not a choice between quality and deadline any
|
||
more, it is a saturation the encoder reaches by itself.
|
||
|
||
Three results worth separating out:
|
||
|
||
1. **0/120 is the first time anything in this project has fitted 12fps on every
|
||
frame.** It is also the best picture yet: 31.19 dB against a **31.33 dB
|
||
palette ceiling**, 0.13 dB off exact for this display.
|
||
2. **`mu` is never spent.** At span budgets >= 1000 KB/s, 0/120 frames need the
|
||
CPU-fit lagrangian at all. The 0.62 dB FINDINGS 31 paid to make frames decode
|
||
in time is refunded in full -- spans buy the deadline with bytes, and a span
|
||
is pixel-exact, so the trade is quality-positive in both directions.
|
||
3. **Quality and deadline stopped competing.** Every earlier section on this
|
||
project trades one against the other. Above ~800 KB/s more bytes improve
|
||
both at once, until the span pass runs out of runs worth spanning.
|
||
|
||
### 42.4 So the whole result now hangs on `W`, and only on `W`
|
||
Read the columns of 42.3 sideways rather than down. At 934.6 KB/s:
|
||
|
||
| `W`, clocks/word | median frame | worst | over budget |
|
||
|---:|---:|---:|---:|
|
||
| 5 | 83.0% | 91.0% | **0/120** |
|
||
| 6 | 87.6% | 97.0% | **0/120** |
|
||
| 6.5 | -- | -- | crossover |
|
||
| 7 | 92.2% | 103.0% | 21/120 |
|
||
| 8 | 96.8% | 109.0% | 47/120 |
|
||
|
||
Moving `W` across its datasheet bracket costs more frames than moving the rate
|
||
from 280 to 935 KB/s wins. **`W` is now the most load-bearing unmeasured number
|
||
in the project**, displacing the 4 Mbps figure it just retired.
|
||
|
||
### 42.5 MAME settles which end of the bracket applies, and cannot settle the number
|
||
STATUS has carried "benchmark `x68000 -exp1 cz6bs1`, never `x68ksupr`" since
|
||
session 10, and the note reads as a hardware claim. **It is not one, and the
|
||
user was right to challenge it.** Both machines use the same MB89352 SPC --
|
||
`x68k.cpp:834` maps it at `$E96020`, `x68k_scsiext.cpp:83` at `$EA0000` -- and
|
||
neither needs a driver because SCSI IOCS is in ROM. The difference is an
|
||
emulation gap: `x68k.cpp:1176` is literally `// TODO: duplicate DMA glue from
|
||
CZ-6BS1`.
|
||
|
||
The external board's glue is modelled, and reading it answers the mode question:
|
||
|
||
- `x68k_scsiext.cpp:110-136`: a transfer is a DMA cycle when `m_slot->exown()`
|
||
-- the HD63450's **OWN** -- is asserted, so the DMAC holds the bus and the
|
||
68000 is off it. It single-address reads the data register at `$EA0015` and
|
||
**negates `#DTACK` whenever `DRQ` is not ready**, stalling rather than
|
||
arbitrating away.
|
||
- `x68k.cpp:1114-1115, 1122-1123`: the slot's DTACK feeds `hd63450::dtack_w`
|
||
and the DMAC's `own()` feeds back to the slot.
|
||
|
||
So the modelled CZ-6BS1 path is **cycle-steal with the bus held** -- the 5
|
||
clocks/word end of 39.7's bracket, not the ~12 arbitrated end. That is the end
|
||
where everything above fits.
|
||
|
||
**But MAME cannot give the constant, and asking it for one would be reading back
|
||
a hand-set table.** `x68k.cpp:1047-1048` configures the DMAC with
|
||
`set_clocks(attotime::from_usec(2), from_nsec(450), from_usec(4), ...)` and
|
||
`set_burst_clocks(...)` -- wall-clock attotimes, not the MC68450's per-operand
|
||
cycle counts. Same lesson as FINDINGS 39: the datasheet supplies constants, the
|
||
emulator supplies structure. Use MAME to confirm which handshake the player's
|
||
code actually provokes; use Fig 4-25 for what it costs.
|
||
|
||
### 42.6 Where the wait actually comes from, and why flash is the right premise
|
||
With DTACK gating the cost decomposes:
|
||
|
||
W = 5 clocks (MC68450 single-address write, Fig 4-25 sheet 3)
|
||
+ however long the drive makes the DMAC wait for DRQ
|
||
|
||
The 5 is silicon and fixed. **The rest is the device**, and it is where 8 and 12
|
||
came from -- 8 was taken as a bracket midpoint in FINDINGS 5, not measured. A
|
||
period spinning SCSI-1 drive supplies a real wait; SD-backed SCSI with a modern
|
||
controller collapses it toward zero and leaves the floor. The deployment target
|
||
has been SD since session 2 (21.2), so **the favourable end of the bracket is
|
||
the one the actual hardware is on** -- which is what makes 42.3's 0/120 worth
|
||
taking seriously rather than filing as a best case.
|
||
|
||
This is a better-founded version of the user's argument than the bus rating was.
|
||
The bus rating is true and irrelevant; the wait term is what their premise
|
||
actually buys.
|
||
|
||
### 42.7 What is NOT established
|
||
Stated plainly, because 42.3 is the most favourable table this project has ever
|
||
produced and that is exactly when it should be distrusted:
|
||
|
||
1. **No 68000 has decoded a 70%-span container.** Every figure in 42.3 is the
|
||
additive cost model. That model is validated to -0.06% mean / 0.09% worst on
|
||
spanned containers (41.5), but the heaviest one ever *run* is the gate's
|
||
26.6%. The rig loads the stream into a 2 MB machine, so a 934.6 KB/s stream
|
||
is about **5 frames** -- getting coverage back needs a different rig
|
||
(stream-in-chunks, or a larger machine as a deliberately-labelled non-target),
|
||
not a longer pass. This is FINDINGS 36's lesson pointing at a new wall.
|
||
2. **`W` is unmeasured**, and 42.4 is the whole result's sensitivity to it.
|
||
3. **The ring buffer has never been simulated near this rate.** 98 KB/frame of
|
||
span payload against 2 MB with no double buffer; `09_buffer_sim.py` last ran
|
||
at 110 and 280 KB/s (29.5/30.7).
|
||
4. **Capacity is fine but should be stated:** 934.6 KB/s x 1366.6 s = **1.22 GB**
|
||
for the whole game. Irrelevant on SD, fatal on anything period. This closes
|
||
CD-ROM permanently rather than parking it (32.3).
|
||
5. **Span selection is still greedy after `lam`** (STATUS item 2, 39.3). At
|
||
saturation that probably leaves something on the table rather than costing
|
||
anything, but it has not been checked at this rate.
|
||
6. **The `--kbps 2000` container is not a shippable configuration**, it is a
|
||
probe: it lets `lam` fall to its floor and asks what the span pass does with
|
||
an unbounded budget. `s13_280p1500` at 837.4 KB/s is the honest candidate.
|
||
|
||
### 42.8 FINDINGS 17.5 was right, and was withdrawn for the wrong reason
|
||
Session 4 concluded: **"ship pixel-exact if SCSI sustains >=800 KB/s."** Session
|
||
5 withdrew it, because 17 had reasoned against a misread 4 MB/s and the
|
||
correction to 4 Mbps = 488 KB/s put 800 out of reach.
|
||
|
||
The correction was to the wrong direction of the error. 4 MB/s was indeed a
|
||
misread of the bus, but 488 KB/s was not a measurement either -- and the bus
|
||
really does have ~1.5-5 MB/s in it (42.1). **The delivered stream is 837.4 KB/s
|
||
at 0.29 dB off the palette ceiling**: 17.5's threshold, and 17.5's conclusion,
|
||
reached from the opposite end five sessions later by a route that never cited
|
||
it.
|
||
|
||
Worth recording as a methodology result rather than a curiosity. The project's
|
||
habit of appending corrections rather than editing history is what made this
|
||
recoverable at all -- but 17.5 spent five sessions marked "not available"
|
||
because a number nobody had sourced was allowed to retire a conclusion that had
|
||
been reasoned properly. **A correction is only as good as the constant behind
|
||
it**, and this is the second time that constant is the one at fault (compare
|
||
39: bus arithmetic retired the DMAC's first costing, and the datasheet retired
|
||
the retirement).
|
||
|
||
## 43. The disk debit was denominated per word, and the SPC is a byte-wide port (session 14)
|
||
|
||
Session 13 made `W` -- the clocks the SCSI DMA steals -- the one binding unknown
|
||
in the project, and 42.4 showed the whole result swinging on where in a 5..12
|
||
bracket it landed. **The bracket was in the wrong unit.** `W` was charged per
|
||
WORD of delivered stream, and the MB89352 is an 8-bit port: the DMAC pays per
|
||
BYTE. The debit is 2x what every table since FINDINGS 5 has charged, the
|
||
favourable end of the bracket was never physically reachable, and 42.3's 0/120
|
||
does not survive.
|
||
|
||
The result is not lost, but it is re-anchored 41% lower in rate and 1.85 dB
|
||
lower in picture, and it had to be re-encoded rather than re-scored -- because
|
||
the encoder was making its decisions in the wrong units too (43.6).
|
||
|
||
### 43.1 The floor argument needs no datasheet
|
||
A 68000 bus cycle is four clocks minimum. The SPC delivers one byte per bus
|
||
cycle. Therefore **no DMA of this device can cost less than 4 clocks/byte = 8
|
||
clocks/word**, before a single clock of DMAC overhead, the write cycle that
|
||
puts the byte in RAM, or any wait the drive imposes.
|
||
|
||
The retired bracket's own midpoint, 8 clocks/word, is that floor exactly; its
|
||
favourable end, 5 clocks/word, is **2.5 clocks/byte -- 62% of a single bus
|
||
cycle**. It implied a 4 MB/s DMA on a 10 MHz bus, on a SCSI-1 link 42.1 had
|
||
just established runs at 1.5 MB/s asynchronous. The number that retired the
|
||
4 Mbps figure should have retired this one in the same paragraph.
|
||
|
||
Independent cross-check, of the kind this project files as folklore rather than
|
||
measurement: BlueSCSI-class throughput on an X68000 is discussed at 0.7-1.7
|
||
MB/s. 5 clocks/byte is a 2 MB/s ceiling and 9 is 1.11 MB/s; 2.5 clocks/byte is
|
||
4 MB/s, which no one has ever reported on this machine.
|
||
|
||
### 43.2 The datasheet, per byte, for the transfer this actually is
|
||
Device-to-memory, 8-bit device, MC68450 Fig 4-25, all while the DMAC owns the
|
||
bus (note 2: reads 4 clocks, writes 5):
|
||
|
||
| how the DMAC is programmed | clocks per BYTE | source |
|
||
|---|---:|---|
|
||
| single address, D->M | **5.0** | sheet 2 |
|
||
| dual address, byte, no packing | **9.0** | sheet 4, the note on the 9-clock word case |
|
||
| dual address, byte packed to word writes | **16.5** | sheet 3: 14 clocks then 19, per two bytes |
|
||
|
||
Byte packing is *dearer*, not cheaper: the FIFO path carries 10 clocks of
|
||
inter-cycle overhead per operand where the unpacked path carries none.
|
||
|
||
To this, per period of bus ownership, add front-end overhead of **5 clocks best
|
||
case (Fig 4-23) or 8 worst (Fig 4-24)** and back-end of **2 (4.5.2.2)**.
|
||
Amortised over a sector that is noise; taken per operand it would add 7-10
|
||
clocks to every byte. **That is what actually rules out arbitrating per byte**
|
||
-- not the OWN pin (43.3).
|
||
|
||
**Which row applies is a wiring question, and it is worth 4 clocks on every
|
||
byte of the game.** Single-address needs the SPC's DACK driven from the bus's
|
||
`#EXACK` (pin B37, and the bus really does have it, `x68kexp.h`). MAME models
|
||
the dual-address row: an `exown`-gated byte read of `$EA0015` written to memory
|
||
with `space.write_byte` (`x68k_scsiext.cpp:110-121`, `hd63450.cpp:383-385`), and
|
||
it has no DACK path at all, so **MAME cannot settle this one either way.**
|
||
|
||
### 43.3 42.5's reading of the OWN pin was over-read
|
||
42.5 concluded from `x68k_scsiext.cpp`'s `m_slot->exown()` gate that the modelled
|
||
path is "cycle steal with the bus held", i.e. the 5-clock end. It does not say
|
||
that. `hd63450.cpp:366,447` asserts `m_own(0)` before **every** `single_transfer`
|
||
and negates it after, in every request-generation mode, and `x68kexp.h:131`
|
||
inverts it. So `exown()` distinguishes **a DMAC access from a CPU access** --
|
||
DMA versus PIO -- and says nothing about hold versus arbitrate. The check was
|
||
answering a different question than the one asked of it.
|
||
|
||
Two things about the board do survive, and they are the useful half:
|
||
|
||
1. **The glue's flow control is a stalled bus cycle, not a released bus.** When
|
||
`DRQ` is low the board negates `#DTACK` and the DMAC waits mid-cycle
|
||
(`hd63450.cpp:449`, `if (!m_dtack) return;` -- the operand does not advance).
|
||
There is no path by which it hands the bus back inside a word. So whatever
|
||
the drive makes it wait, it waits **holding the bus**, which is why the wait
|
||
term of 42.6 is charged to the frame budget in full.
|
||
2. **MAME has no DRQ line from the expansion slot to the DMAC.** `x68kexp.cpp`
|
||
has no such callback; DMAC channel 0 is wired to the FDC and channel 3 to the
|
||
ADPCM (`x68k.cpp:1052-1053,159`). The real bus has `#EXREQ`/`#EXACK`
|
||
(B36/B37), so external-request modes exist on hardware and are simply absent
|
||
from the model. **Any request-generation experiment run in MAME would be
|
||
measuring the gap, not the board.**
|
||
|
||
### 43.4 What it costs, on the containers session 13 already had
|
||
`17_span_delivered.py` and `14_dmac_chain.py` now take `--disk-clk-byte`;
|
||
`--disk-clk-word` is kept and halves it, so session 13's tables reproduce
|
||
exactly. Frames over the 12fps budget, 120-frame `singe` window:
|
||
|
||
| container | KB/s | 2.5 c/B (s13) | 5 c/B | 9 c/B | 16.5 c/B |
|
||
|---|---:|---:|---:|---:|---:|
|
||
| `s12_280_off` | 278.6 | 76/120 | 100/120 | 118/120 | 120/120 |
|
||
| `rc_fr_singe_scsi_span` (the gate) | 487.7 | 11/120 | 71/120 | 119/120 | 120/120 |
|
||
| `s13_280p1000` | 815.4 | 7/120 | 45/120 | 120/120 | 120/120 |
|
||
| **`s13_280p1500`** (s13's candidate) | 837.4 | **0/120** | 44/120 | 120/120 | 120/120 |
|
||
| `s13_lossless` | 934.6 | **0/120** | 87/120 | 120/120 | 120/120 |
|
||
|
||
**Nothing session 13 emitted fits at any point in the real bracket.** The
|
||
mechanism is 42.2's own table read at the right price. Per pixel, with `c` the
|
||
clocks per delivered byte:
|
||
|
||
v7 literal span 2 bytes 9.143 + 2c
|
||
RAW block 1 byte 25.03 + c
|
||
V1 block 0.25 B 18.74 + 0.25c
|
||
|
||
A span beats RAW for `c < 15.9` -- still true everywhere real. But **a span
|
||
beats a V1 block only for `c < 5.48`**, and that crossover sits between the
|
||
single-address row and the dual-address row of 43.2. The span pass is not
|
||
robust to the wiring question; it is decided by it. The encodes in 43.6
|
||
confirm the prediction: spans paint 30.7% of the picture at `c=5` and 3.9% at
|
||
`c=9`.
|
||
|
||
### 43.5 A frame's cost is now, to three figures, its byte count
|
||
On `s13_280p1500` at 5 clocks/byte, `corr(bytes, total frame clocks) = 0.989`,
|
||
and the disk term's p10..p90 spread (281k..464k clocks) is most of the frame
|
||
total's (713k..966k). Frames that miss carry 87,203 bytes on average against
|
||
61,295 for frames that fit.
|
||
|
||
**Which breaks the rate controller's bucket.** It banks bytes across 8 frames
|
||
because the player's ring buffer can hold them -- true, and irrelevant now:
|
||
those bytes are also clocks, and FINDINGS 28 established there is no double
|
||
buffer to decode ahead into, so **a frame that borrows bytes from the bucket
|
||
borrows clocks it cannot bank.** Byte smoothing was free when bytes were free.
|
||
It is now a direct source of deadline misses.
|
||
|
||
### 43.6 The encoder was never told a byte costs anything either
|
||
This is why the session could not just rescore. `ratectl.py` bisected `mu`
|
||
against 833,333 cycles of DECODE, with no disk term, and `spans.select()`
|
||
admitted a run "only if the span beats the blocks it replaces on cycles ALONE"
|
||
-- explicitly ignoring the bytes it adds. Both now work in one currency:
|
||
|
||
- `ratectl.DISK_CLK_BYTE` (default **5.0**, `encode.py --disk-clk-byte`) is
|
||
charged inside the CPU ceiling, so every fit test is `decode + c*bytes`.
|
||
- `spans.select()` admits and ranks on **net** clocks, `(clocks won) - c*(bytes
|
||
added)`. At `c=0` both reduce exactly to the old rules, and
|
||
`--disk-clk-byte 0` re-emits `s13_280p1500` byte for byte (8,501,948 B,
|
||
31.04 dB) -- so the change is the price, not the codec.
|
||
|
||
Re-encoded honestly, 120-frame `singe` window, 12 fps:
|
||
|
||
| | KB/s | PSNR | loss vs ceiling | over budget | span px |
|
||
|---|---:|---:|---:|---:|---:|
|
||
| s13's claim (2.5 c/B) | 837.4 | 31.04 dB | 0.29 dB | 0/120 | 62.3% |
|
||
| **`s14_d5_all1500`, single address** | **496.7** | **29.19 dB** | 2.14 dB | **1/120** | 30.7% |
|
||
| `s14_d9_all1500`, dual address | 255.0 | 28.50 dB | 2.83 dB | **1/120** | 3.9% |
|
||
|
||
**The one frame over is frame 0** in both -- the intra frame, which FINDINGS
|
||
28.5/31 already established is emitted late on purpose because there is nothing
|
||
on screen to hold. **Every other frame lands at or under 100.0%**: the
|
||
controller now binds exactly on the joint budget. `17_span_delivered.py`, which
|
||
shares no code with the encoder's own accounting, reproduces both rows to the
|
||
digit.
|
||
|
||
Two smaller results fall out:
|
||
|
||
1. **`--spans all` is now the better rule, and `--spans need` the worse one**
|
||
(275.8 KB/s, 28.92 dB, 2/120). `need` stops as soon as the frame fits and
|
||
leaves profitable clock savings unbought; once profitability is measured in
|
||
one currency, spending every profitable byte *is* the optimum. My
|
||
recommendation is to make `all` the default; that is my inference, not a
|
||
measurement.
|
||
2. **`--span-kbps` has stopped binding.** 700 and 1500 produce identical files.
|
||
Session 13's saturation was the byte ceiling running out; this one is the
|
||
span pass running out of runs that pay, which is a property of the codec
|
||
rather than of a chosen number.
|
||
|
||
### 43.7 What this withdraws
|
||
- **42.3's 0/120 and "quality and deadline stopped competing".** They compete
|
||
again, and harder than before: a byte now buys picture and spends deadline.
|
||
- **42.3's "`mu` is never spent"** and the refund of FINDINGS 31's 0.62 dB.
|
||
`mu` is spent on 103 of 120 frames at `c=5` and 118 at `c=9`.
|
||
- **42.8's revival of 17.5** ("ship pixel-exact if SCSI sustains >=800 KB/s").
|
||
The stream that fits is 496.7 KB/s and 2.14 dB off the palette ceiling, so
|
||
17.5's threshold is not met and its conclusion does not return. The
|
||
methodology point in 42.8 stands and now applies to itself.
|
||
- **`docs/BENCHMARK.md`'s "~8 clocks/word => ~2.5 MB/s practical ceiling".**
|
||
The ceiling is 2 MB/s at best (5 clocks/byte) and 1.11 MB/s dual-address.
|
||
- **FINDINGS 5's 8 clocks/word**, retroactively, wherever it was used: every
|
||
I/O debit in the project before this section was charged at half rate.
|
||
|
||
### 43.8 What is NOT established
|
||
1. **Single-address versus dual-address is a hardware fact this tree cannot
|
||
check.** It is worth 242 KB/s and 0.69 dB, and MAME models only the dual
|
||
row (43.2). It needs the CZ-6BS1 schematic or a real board -- and note it is
|
||
the *board's* wiring, not our code, so unlike every previous item on this
|
||
list it is not a design decision we get to make.
|
||
2. **The 5 and the 9 are datasheet floors with the drive wait set to zero.**
|
||
42.6's argument that SD-backed SCSI collapses that term is unchanged and
|
||
still unmeasured.
|
||
3. **Front-end and back-end overhead are excluded**, which assumes at least a
|
||
sector per period of bus ownership. If the player ends up taking the bus per
|
||
operand, add 7-10 clocks to every byte and nothing fits at all.
|
||
4. **Still no 68000 has decoded any of these containers** (42.7 item 1). The
|
||
new ones are lighter in spans than the gate, so that gap is narrower than it
|
||
was, but it is the same gap.
|
||
5. **The bucket has not been fixed**, only diagnosed (43.5).
|
||
|
||
### 43.9 The unit was never written down
|
||
The error is one substitution -- bytes/2 for bytes -- and it survived from
|
||
FINDINGS 5 through nine sessions, two cost-model rewrites, a datasheet reading
|
||
that corrected the *value* of the same constant (39.7), and a section devoted
|
||
to distrusting the table it produced (42.7, which lists six things 42.3 did not
|
||
establish and does not list its own denominator).
|
||
|
||
What let it hide: `W` was carried as "clocks per word" in three tools and a
|
||
half-dozen tables, and **the device's port width was never in the same sentence
|
||
as it.** The 8-bit-ness of the MB89352 was known -- it is visible in
|
||
`install_device(..., 0x00ff00ff)` and in every register map in the project --
|
||
but it lived in the I/O notes while `W` lived in the budget arithmetic.
|
||
|
||
The rule this project already had (FINDINGS 33: a design that counts only CPU
|
||
shows a win the I/O it created takes away) needed one more clause: **check what
|
||
the unit is denominated in, on the device that supplies it.** A per-word debit
|
||
for a byte-wide port is a factor of two, and a factor of two is the difference
|
||
between this game running and not.
|
||
|
||
---
|
||
|
||
## 44. The byte-side rate controller is inert, and both inconsistencies in it are worth under 2% (session 15)
|
||
|
||
STATUS's item 2 asked for the leaky bucket to be fixed: 43.5 diagnosed it as
|
||
banking bytes that are now clocks, across a player with no double buffer to
|
||
bank clocks in, and called it "a direct source of deadline misses". The
|
||
diagnosis is correct as a mechanism. **It is not a source of anything at the
|
||
operating point this project actually recommends**, and neither is the second,
|
||
larger-looking inconsistency found next to it. Both are now implemented, both
|
||
measure as a wash or a regression, and **both ship off by default** --
|
||
`--joint-decide` and `--joint-bucket` turn them on.
|
||
|
||
Everything below is the 120-frame `singe` window, `--profile scsi --kbps 280
|
||
--span-kbps 1500`, `c = 5` clocks/byte, 12 fps, scored both by the encoder and
|
||
by `17_span_delivered.py`, which shares no code with it.
|
||
|
||
### 44.1 The mode decision was the last place a byte was free
|
||
43.6 charged the disk inside the rate controller's fit test and inside
|
||
`spans.select()`, but `vq_hybrid.decide()` still minimised `D + lam*bytes +
|
||
mu*cycles` with `cycles` meaning DECODE cycles only. So while `mu` was
|
||
enforcing a joint budget from above, the per-block lagrangian underneath it
|
||
still believed delivery was free.
|
||
|
||
**That inverts FINDINGS 28.8.** RAW is 400.4 cycles against V4's 448.2, so with
|
||
a free byte, raising `mu` buys cycles by moving V4 -> RAW -- which is exactly
|
||
what 28.8 observed and what session 8's `0c` recorded as V4 collapsing. Priced
|
||
per delivered byte, a RAW block costs `400.4 + 16c` and a V4 block `448.2 + 4c`:
|
||
|
||
| c, clocks/byte | V1 | V4 | RAW |
|
||
|---:|---:|---:|---:|
|
||
| 0 | 299.9 | 448.2 | **400.4** |
|
||
| 2.5 (the retired 5 clk/word) | 302.4 | 458.2 | **440.4** |
|
||
| **3.98** | 303.9 | **464.3** | **464.3** |
|
||
| 5 (single address) | 304.9 | **468.2** | 480.4 |
|
||
| 9 (dual address) | 308.9 | **484.2** | 544.4 |
|
||
|
||
**The crossover is c = 3.98 and 43.1's floor argument is c >= 4.** A 68000 bus
|
||
cycle is four clocks and the SPC hands over one byte per cycle, so RAW's cycle
|
||
advantage does not exist on any real machine: it was spending 12 clocks of bus
|
||
to save 47.8 of CPU. The escape hatch was an artefact of the same free byte
|
||
that FINDINGS 43 found everywhere else.
|
||
|
||
### 44.2 And correcting it changes almost nothing
|
||
`decide()` now takes `byte_clk` and prices a payload byte at `lam + mu*byte_clk`.
|
||
At `byte_clk = 0` it is the old decision exactly, and the containers of session
|
||
14 re-encode to the same MD5.
|
||
|
||
| `--spans all`, c=5 | KB/s | PSNR | mean frame clocks | over budget | worst |
|
||
|---|---:|---:|---:|---:|---:|
|
||
| the shipped decision | 496.7 | **29.19** | **740,049** | 1/120 | 112.9% |
|
||
| `--joint-decide` | 482.5 | 29.17 | 745,438 | 1/120 | 112.9% |
|
||
|
||
It **buys 6,058 clocks of disk with 17,207 clocks of block decode** -- a net
|
||
5,389 clocks a frame in the wrong direction. Scored at c=4 and c=9 it is the
|
||
marginally worse container at every price, so this is not a bet on which row of
|
||
43.2 wins. The reason is scale, not sign: at the `mu` the controller actually
|
||
settles on (median 0.74-0.91) the added byte price is `mu*c` ~ 4 against a
|
||
`lam` floor of 10, and the RAW/V4 decision is dominated by V4's distortion term
|
||
rather than by either. RAW moves 16.0% -> 15.7% of blocks.
|
||
|
||
**The inconsistency was real, the correction is right, and the effect is 0.02 dB.**
|
||
|
||
### 44.3 The bucket does not bind, and at `--spans all` nothing on the byte side does
|
||
Before fixing the bucket, measure whether it is loaded. It is not:
|
||
|
||
| `--spans all`, c=5 | KB/s | PSNR | over budget | container |
|
||
|---|---:|---:|---:|---|
|
||
| `--bucket-frames 8` (shipped) | 496.7 | 29.19 | 1/120 | baseline |
|
||
| `--bucket-frames 32` | 496.7 | 29.19 | 1/120 | **byte-identical** |
|
||
| `--bucket-frames 1` (no banking at all) | 498.0 | 29.19 | 1/120 | +1.3 KB/s |
|
||
| `--rc-floor open` (lam floor 1.0) | 503.7 | 29.21 | 1/120 | +0.02 dB |
|
||
|
||
**`lam` never leaves its floor of 10.0 on any of 120 frames**, in any of these,
|
||
and quadrupling the bucket emits the same bytes. With `--spans off` at 259.3
|
||
KB/s under a 280 KB/s target, a 1-frame bucket and an 8-frame bucket are again
|
||
byte-identical. The block coder at the profile floor simply lands under the
|
||
per-frame byte budget, so there is nothing for the bucket to lend and nothing
|
||
for the lam bisection to do. **The rate this project reports is set by the span
|
||
pass and by `mu`; `--kbps` and the bucket are not the levers.**
|
||
|
||
The one place the bucket does cost something is the mode 43.6.1 recommends
|
||
abandoning: at `--spans need` it is worth exactly one frame of 120.
|
||
|
||
### 44.4 The cap, and why capping only half a frame is worse than not capping
|
||
`--joint-bucket` caps what the bucket may lend at what the frame's clock budget
|
||
can still absorb after its own block decode -- `(cycle_budget - cycles(mode0))
|
||
/ c`, priced at the mode map the un-banked budget buys, and never below that
|
||
budget. The bucket then smooths only what is left after the disk is paid, which
|
||
is what STATUS item 2 asked for.
|
||
|
||
| c=5 | KB/s | PSNR | median frame | over budget |
|
||
|---|---:|---:|---:|---:|
|
||
| `need`, shipped | 275.8 | 28.92 | 99.8% | 2/120 |
|
||
| **`need --joint-bucket`** | 302.1 | 28.91 | 99.9% | **1/120** |
|
||
| `all`, shipped | **496.7** | **29.19** | **87.9%** | 1/120 |
|
||
| `all --joint-bucket` | 506.4 | 29.18 | 89.6% | 1/120 |
|
||
| `all --joint-bucket`, cap extended to the span section | 273.7 | 28.88 | 99.4% | 1/120 |
|
||
|
||
**It RAISES the bitrate at `--spans all`**, which is the tell: capping the block
|
||
payload does not remove those bytes, it moves them into the span section, which
|
||
draws on its own flat pipe (41.2) and is not under the cap at all. So the third
|
||
row is the honest reading of "the per-frame ceiling should be joint and hard" --
|
||
and it is the worst container here, losing 0.30 dB and 233 KB/s for no change in
|
||
the frames-over count, because the cap starves the pass that was buying the
|
||
deadline in the first place.
|
||
|
||
There is no per-frame ceiling on the whole frame record anywhere in this
|
||
encoder, and **44.3 is why adding one has not been urgent**: the joint budget is
|
||
enforced after the span pass by `mu`, which is a controller that pays in
|
||
picture, and the byte-side ceiling it would replace is not binding.
|
||
|
||
### 44.5 An encode is 95% k-means, and that is now 2.7x faster, exactly
|
||
Prompted by the user observing that this should not take a minute a scene.
|
||
Profiled, a 120-frame encode was 60.6 s of which 60.8 s was two k-means runs
|
||
(the rest of the encoder, rate control and the span pass included, is about a
|
||
second). `VQ.assign` was all of it, and three things were on the floor, none of
|
||
which changes a label:
|
||
|
||
| | 1,474,560 2x2 blocks, k=256 |
|
||
|---|---:|
|
||
| as written | 1.83 s |
|
||
| `C.T` materialised once (a view makes BLAS re-copy it per chunk) | 1.02 s |
|
||
| chunk 8192 -> 2048 (the `(chunk,k)` temporary, cache not memory; 32768 is 2.80 s) | 0.81 s |
|
||
| 8 threads over the chunk loop (numpy releases the GIL in both matmul and argmin) | **0.31 s** |
|
||
|
||
Partitioning rows cannot change an `argmin`, so the labels are bit-identical to
|
||
the serial ones and every container this encoder emits still hashes the same --
|
||
which is the assertion, not the hope: `s14_d5_all1500` re-encodes to
|
||
`d13d142b...` on both sides of the change. Whole encode 60.6 s -> 29.4 s,
|
||
`H.build` 60.9 -> 22.2 s. What is left is `np.add.at` in the centroid update
|
||
(~5 s of 22), and taking it costs the bit-exactness, so it stays.
|
||
|
||
At 0.18 s/frame the 22.8 minutes of unique scene footage is ~50 minutes of
|
||
codebook training, single machine, single pass.
|
||
|
||
### 44.6 What this does NOT establish
|
||
1. **It does not re-open the c=5 vs c=9 question** (43.8.1), which is still the
|
||
largest open number and still a hardware fact this tree cannot check.
|
||
2. **The 1/120 is the same 1/120** -- frame 0, the intra frame, emitted late on
|
||
purpose. Nothing here moved it, and 43.8's five caveats all stand.
|
||
3. **`--spans all` remains a recommendation, not a measurement** (43.6.1), and
|
||
44.3 sharpens why it matters: it is the only lever on this side of the
|
||
encoder that does anything.
|
||
4. **The pixel-exact gate now covers 37 of 120 frames**, down from 80 in session
|
||
10 (FINDINGS 36) -- the span-heavy container outgrew the 2 MB machine. The
|
||
strongest test in the tree audits under a third of the window it names. That
|
||
makes STATUS item 3's chunk-streaming rig load-bearing rather than optional.
|
||
5. **Two fixes, both correct, both defaulted off.** The pattern worth keeping is
|
||
that the second one was found by asking whether the lever was loaded before
|
||
pulling it, and 44.3 took four encodes to establish -- against a session that
|
||
could have been spent making a bucket cap work.
|
||
|
||
### 44.7 No decoder at all: what a literal frame costs, and where it dies
|
||
Asked by the user: does streaming raw preprocessed frames straight into video
|
||
memory save the CPU? On clocks, yes -- completely. It dies on the medium, and
|
||
the arithmetic is worth writing down because it also retires the last line of
|
||
42.2 still standing in the wrong units.
|
||
|
||
**Two versions, and the split is the answer.** A frame is 256x192 = 49,152
|
||
pixels; 256-colour GVRAM is one pixel per WORD with the high byte discarded
|
||
(42.2, `spans.py:44`, and the memory model both emulators are pixel-exact
|
||
against), so a literal frame is **98,304 bytes of GVRAM writes** against a
|
||
budget of 833,333 clocks = **16.95 clocks/pixel**.
|
||
|
||
*1. The CPU paints it.* This is what a v7 span already is -- literal GRB555
|
||
words, preprocessed offline, copied by `movem` -- at `9.143 + 2c` clocks/pixel:
|
||
|
||
| | clocks/pixel | vs 16.95 |
|
||
|---|---:|---|
|
||
| c = 3.906 | 16.95 | breakeven |
|
||
| **c = 4.0**, the 43.1 floor | **17.14** | **misses by 1.1%** |
|
||
| c = 5.0, single address | 19.14 | misses by 13% |
|
||
| c = 9.0, dual address | 27.14 | misses by 60% |
|
||
|
||
**A CPU-painted full-frame literal does not fit at any physically reachable
|
||
price**, and it misses at the floor itself. This supersedes 42.2's "lossless
|
||
fits iff `W` is about 6.5 or less", which was denominated per WORD: 6.5
|
||
clocks/word is 3.25 clocks/byte, below the 4-clock floor, so that threshold was
|
||
never reachable either. FINDINGS 43 withdrew the tables downstream of the unit
|
||
error but not this line; it is withdrawn here.
|
||
|
||
*2. The DMAC writes device -> GVRAM and the CPU is not in the loop.* The 9.143
|
||
disappears and a pixel costs only its two wire bytes:
|
||
|
||
| c | clocks/frame | % of budget | wire | 22.8 min of game |
|
||
|---:|---:|---:|---:|---:|
|
||
| 4.0 | 393,216 | 47.2% | 1,152 KB/s | 1.61 GB |
|
||
| **5.0** single address | **491,520** | **59.0%** | 1,152 KB/s | 1.61 GB |
|
||
| 9.0 dual address (the row MAME models) | 884,736 | **106.2%** | 1,152 KB/s | 1.61 GB |
|
||
| 16.5 packed | 1,622,016 | 194.6% | 1,152 KB/s | 1.61 GB |
|
||
|
||
**At c=5 it fits, with 41% of the frame budget left and no decoder at all.** It
|
||
is defeated by delivery, not by the 68000: 1,152 KB/s is ~79% of SCSI-1
|
||
asynchronous (42.1) with nothing left for audio or seeks, and 1.61 GB is 2.3x
|
||
the 0.70 GB the shipping 496.7 KB/s container already needs -- which is itself
|
||
already past a CD-ROM. At c=9 it does not fit the clocks either.
|
||
|
||
**The waste is specific and it is the same one the span path pays.** Because the
|
||
high byte is discarded, half of every byte pulled off the disk is thrown away by
|
||
the hardware on arrival: two bytes of disk per byte of picture. That 2:1 is why
|
||
a span pixel costs `2c` and why 43.4's span-vs-V1 crossover sits at c=5.48. The
|
||
codec is not there to save CPU -- 44.7 shows the CPU can be removed outright --
|
||
it is there to save the wire.
|
||
|
||
**The one thing that would change this answer** is a packed write path into
|
||
256-colour GVRAM, two pixels per word. That halves the wire to 576 KB/s and
|
||
806 MB and puts a decoder-free player back in play. **It is NOT established
|
||
either way.** The tree's basis for one-pixel-per-word is that MAME and px68k are
|
||
both pixel-exact against that model, which is evidence about two emulators, not
|
||
about the CRTC and the palette hardware. It is the same class of question as
|
||
43.8.1 -- a service manual or a real board settles it, an emulator cannot, and
|
||
both emulators here would model a packed path identically wrong if it exists.
|
||
|
||
Note this is a different transfer from the one FINDINGS 39 costed and rejected.
|
||
39 priced **memory -> GVRAM** (the DMAC replacing v6's `movem` chain out of a
|
||
RAM stream buffer) and it lost by a clock. This is **device -> GVRAM**, with no
|
||
RAM staging and no CPU, and it loses to the disk instead.
|
||
|
||
## 45. The strongest test in the tree was short by 83 frames, and the fix was the rig's memory (session 16)
|
||
|
||
STATUS item 1 said the pixel-exact gate "needs the chunk-streaming rig, not a
|
||
longer pass." It needed neither. It needed a bigger emulated machine, and the
|
||
reason that was not obvious is worth more than the fix.
|
||
|
||
### 45.1 The constraint was the rig's, and the gate does not measure timing
|
||
`prep_dlx.py` preloads the whole container into emulated RAM at `STREAM=0x30000`
|
||
and `check.sh` ran the machine at `-ramsize 2M`, so the span-heavy gate container
|
||
-- 5,261,814 B of stream, ending at 0x534BF6 -- was truncated to the prefix that
|
||
fit. That was **37 of 120 frames**, down from 80 in session 10 (FINDINGS 36) as
|
||
the container grew.
|
||
|
||
The 2 MB was carried over from the timing rig, where it is correct: the locked
|
||
target is a stock 2 MB machine. But `check.sh` runs this gate under
|
||
`DLX_VERIFY_ONLY=1`, which **drops the cost anchors entirely** and asserts only
|
||
pixel-exactness. A verify-only pass makes no claim about the target's memory, so
|
||
it was never the 2 MB that was load-bearing here -- and preloading a whole
|
||
container is unlike the shipping player at *any* size, because the player streams
|
||
from disk into a ring buffer and never holds a window at once.
|
||
|
||
`RIG_RAM=6` in `check.sh` (MAME's x68000 accepts 1M-12M) covers all 120 frames.
|
||
|
||
### 45.2 The raise is licensed by measurement, not by convenience
|
||
Raising the emulated RAM to make a test pass is exactly the move that should be
|
||
distrusted, so it was checked rather than asserted. The full timing pass -- not
|
||
verify-only -- was run at `-ramsize 2M` and `-ramsize 6M`, and the **five
|
||
synthetic anchors come out bit-identical**:
|
||
|
||
| synthetic frame | 2M | 6M |
|
||
|---|---:|---:|
|
||
| all-SKIP | 40,729 | 40,729 |
|
||
| all-V1 | 921,187 | 921,187 |
|
||
| all-V4 | 1,376,881 | 1,376,881 |
|
||
| all-RAW | 1,229,883 | 1,229,883 |
|
||
| all-SPAN-64 | 506,533 | 506,533 |
|
||
|
||
They sit at **different addresses** in the two layouts -- the synthetic block is
|
||
placed after a 37-frame stream in one and a 120-frame stream in the other -- and
|
||
still cost the same, so MAME's cycle model does not depend on `-ramsize` over
|
||
this range. Every per-block constant in FINDINGS 24/30/41 is measured from these
|
||
frames and is therefore unmoved by the change.
|
||
|
||
### 45.3 The 37-frame prefix was a biased sample, and the quiet end was missing
|
||
The gate is now 120/120 pixel-exact on **both** cores -- MAME's 68000 and px68k's
|
||
C68K. (The C68K harness never had a RAM ceiling at all: its arena is 16 MB and
|
||
`RAMTOP` is defined but unused, so it was short only because it reads the blob
|
||
`prep_dlx.py` truncated.) What the extra 83 frames show is that the prefix was
|
||
not representative:
|
||
|
||
| anchor | 37-frame prefix | full 120 |
|
||
|---|---:|---:|
|
||
| min non-SKIP | 25.1% of blocks, 61.9% of budget | **15.2%, 53.6%** |
|
||
| median | 45.4%, 79.7% | **41.1%, 81.1%** |
|
||
| p90 | 52.3%, 93.1% | **48.5%, 91.1%** |
|
||
| max non-SKIP | 62.5%, 91.8% | 62.5%, 91.8% (same frame) |
|
||
| C68K sequential-pass mean | 693,886 cyc, **83.3%** | **641,444 cyc, 77.0%** |
|
||
|
||
The prefix **overstated the mean cost of the window by 8.2%**. It caught the
|
||
worst frame -- the max is the same frame in both -- but it never saw the quiet
|
||
end: the true minimum is 15.2% non-SKIP against the prefix's 25.1%, and 53.6% of
|
||
budget against 61.9%. The distribution the anchors exist to sample was cut off at
|
||
one end, which is the failure mode FINDINGS 25.6 warned about in a different
|
||
guise: a prefix is not a sample.
|
||
|
||
Note the direction. The prefix was **pessimistic**, so nothing downstream of it
|
||
was flattered, and no headroom claim in this tree was resting on the missing
|
||
frames. That is luck, not design.
|
||
|
||
### 45.4 What this does NOT establish
|
||
1. **The streaming path is still untested, at either RAM size.** The rig
|
||
preloads; the player streams into a ring buffer. This gate proves the decoder
|
||
is pixel-exact over a whole window, and says nothing about the ring buffer,
|
||
the chunk boundaries, or the disk. The chunk-streaming rig STATUS item 1
|
||
called for is still unbuilt -- it was just never what the 37/120 needed.
|
||
2. **The 2 MB target is unchanged.** `RIG_RAM` is the emulated machine's memory
|
||
for a verify-only pass. `span.sh` and the session-7 timing reproduction still
|
||
run at 2M, and the shipping player's memory budget is untouched.
|
||
3. **The truncation guard stays.** `prep_dlx.py` still truncates and announces
|
||
it, and `check.sh` still reads the count back and greps for `TRUNCATED`. A
|
||
heavier container, or a lowered `RIG_RAM`, brings it straight back.
|
||
4. **The moved anchors are a re-measurement, not a regression.** No constant
|
||
changed; the frames the anchors point at did.
|
||
|
||
## 46. The 256-colour mask is defeatable, and a packed path is back in play (session 16)
|
||
|
||
STATUS item 2 asked whether 256-colour GVRAM has a PACKED write path -- two
|
||
pixels per word rather than one pixel per word with the high byte discarded. It
|
||
**The first answer was no and it was wrong** -- or rather, it was right about the
|
||
default write path and missed the register that turns the masking off. 46.1-46.3
|
||
record what was established and what was measured, and are kept as written.
|
||
**46.5 is the correction and it is the important part of this section.**
|
||
|
||
### 46.1 The answer is no, and the sub-word fields are PAGES, not pixels
|
||
Three independent lines, none of them a service manual (see 46.4 on evidence
|
||
class):
|
||
|
||
1. **A community hardware guide** (x68000-dev-guide, `docs/graphics.md`): "each
|
||
pixel occupies exactly one word (2 bytes), regardless of the color mode." In
|
||
256-colour mode the word packs **2 pages** -- page 0 is mask `$00FF` at the
|
||
`$C00000` alias, page 1 is mask `$FF00` at `$C80000`.
|
||
2. **A Japanese retro-computing writeup** (wizforest), independently: 1 word = 1
|
||
dot in every display mode, and in 256-colour mode only the lower 8 bits of a
|
||
1 MB region are valid. It states the masking as the well-known nuisance it
|
||
was, at a *different* depth: "16色モードでは 16bit を書き込んでもハード的に
|
||
マスクされてしまって 4bit しか書きこまれない" -- in 16-colour mode a 16-bit
|
||
write is hardware-masked down to 4 bits.
|
||
3. **px68k's write path**, which is the mechanism the other two describe:
|
||
|
||
```c
|
||
case 1: /* 256 colors */
|
||
if ( adr<0x100000 )
|
||
if ( !(adr&1) ) { /* the other byte of the word: discarded */
|
||
...
|
||
if (adr&0x80000) adr+=1; /* the $C80000 alias IS the other byte */
|
||
adr &= 0x7ffff;
|
||
GVRAM[adr] = data;
|
||
```
|
||
|
||
**The decisive detail is that both pages derive `line` identically**, from
|
||
`(adr&0x7ffff)>>10` -- 1024 bytes per scanline for 512 pixels. Page 0 and page 1
|
||
are the two bytes of one word **at the same screen coordinate**. The same holds
|
||
in 16-colour mode, where `page = (adr>>17)&0x0c` selects one of four nibbles of
|
||
the word: four pages, one coordinate. There is **no graphics mode in which one
|
||
16-bit word holds two horizontally adjacent pixels.**
|
||
|
||
### 46.2 The near-miss is worth writing down so it is not re-derived
|
||
Page 1 has its own scroll register, so scrolling it one pixel relative to page 0
|
||
would put its byte at screen x+1 while page 0's sits at x -- two adjacent screen
|
||
pixels from one word. It does not work, and the arithmetic is why: page 1 stores
|
||
a byte for **every** coordinate, not every other one, so to let page 0 show
|
||
through on alternate columns you must write page 1's transparent index there.
|
||
That is the same byte count. 1024 bytes per row still buys ~512 screen pixels.
|
||
The 2:1 is structural, not an addressing accident.
|
||
|
||
**So FINDINGS 44.7 stands unchanged.** The decoder-free player still needs
|
||
1,152 KB/s and 1.61 GB, and is still killed by the medium. The `2c` span pixel,
|
||
the c=5.48 span-vs-V1 crossover, and every figure resting on two disk bytes per
|
||
picture byte are unmoved.
|
||
|
||
### 46.3 But the text plane is 4bpp PLANAR, and that is 4x denser on the wire
|
||
The tax is a property of the *graphics* planes. The X68000's text plane is not
|
||
laid out that way at all -- px68k's `TVRAM_Write` addresses **four planes at
|
||
`0x20000` stride**, each 0x20000 = 131,072 B = 1024x1024 **bits**. That is 4bpp
|
||
planar: **0.5 bytes per pixel**, against 2.0 for 256-colour graphics.
|
||
|
||
The wire arithmetic, and it is only arithmetic:
|
||
|
||
| surface | B/pixel | frame (256x192) | at 12fps | 22.8 min of game |
|
||
|---|---:|---:|---:|---:|
|
||
| 256-colour graphics, literal (44.7) | 2.0 | 98,304 | 1,152 KB/s | 1.61 GB |
|
||
| **shipping DLX3 container, compressed** | -- | -- | **496.7 KB/s** | **0.70 GB** |
|
||
| **4bpp planar text plane, literal** | **0.5** | **24,576** | **288.0 KB/s** | **0.40 GB** |
|
||
|
||
**An UNCOMPRESSED 16-colour frame is 42% cheaper on the wire than this project's
|
||
compressed 256-colour stream**, and it needs no decoder at all -- the planar
|
||
conversion is an encoder-side transform, so the disk delivers plane words that
|
||
go straight out. On clocks it is not close either: 24,576 bytes/frame is
|
||
`24,576c`, which at c=5 is 122,880 clocks = **14.7% of a 12fps budget**.
|
||
|
||
**It was measured the same session, and it is dead.** `tools/analysis/18_text_plane_16col.py`
|
||
over the 120-frame `singe` window, PSNR against the 24-bit source, generous to
|
||
the 16-colour side on every axis the hardware allows -- per-frame palettes (the
|
||
text palette is 16 entries; reloading it is 16 words a frame, nothing against
|
||
833,333 clocks) which the 256-colour path *cannot* use, because its codebooks
|
||
are indices into a scene-wide palette:
|
||
|
||
| | mean PSNR | min | max |
|
||
|---|---:|---:|---:|
|
||
| 256 colours, scene palette (the tree's) | 31.33 | 27.08 | 34.08 |
|
||
| 256 colours, per-frame palette | 34.08 | 32.19 | 38.32 |
|
||
| 16 colours, scene palette | 23.17 | 17.36 | 25.87 |
|
||
| **16 colours, per-frame palette** | **25.49** | 22.99 | 28.48 |
|
||
|
||
**256 -> 16 costs 5.84 dB at each side's best.** Against the shipping container's
|
||
**29.19 dB at 496.7 KB/s**, a 16-colour literal delivers **25.49 dB at 288.0
|
||
KB/s** -- **3.70 dB worse for 58% of the bitrate.** A codec that buys 3.70 dB for
|
||
1.72x the bytes is doing its job; the wire saving does not pay for the colours.
|
||
|
||
**Closed.** The user's call was to drop the 16-colour direction outright, and the
|
||
number agrees with it, so the text plane is not pursued. FINDINGS 7's 256-colour
|
||
claim stands, and it now stands on a measurement rather than on preference.
|
||
|
||
Two honest notes on that measurement:
|
||
- **The dither row is void.** A Floyd-Steinberg run was included and came out
|
||
*bit-identical* to the undithered one, which means PIL ignored `dither=` under
|
||
`MEDIANCUT` rather than that dither is free. It is left out of the table. It
|
||
does not change the conclusion -- dither lowers PSNR by construction, and the
|
||
lead was already 3.70 dB short.
|
||
- **Everything else about the path stayed unestablished and now stays that way**:
|
||
TVRAM wait states (every `c` in this tree is a GVRAM figure), the text plane's
|
||
geometry and priority against the graphics planes, and the fact that a planar
|
||
word spans 16 pixels of one bitplane, so VQ blocks and v7 spans -- both chunky
|
||
-- would not survive the change unaltered. None of it was worth measuring once
|
||
the colour cost came in.
|
||
|
||
### 46.4 Evidence class, stated plainly
|
||
46.1 is **secondary documentation plus an emulator's mechanism**, not primary. No
|
||
service manual, CRTC databook, or real board was consulted. What changed since
|
||
44.7 is the *kind* of evidence: 44.7 rested on MAME and px68k both being
|
||
pixel-exact against one-pixel-per-word, which is evidence about two emulators
|
||
that could be identically wrong. Now two independent documents describe the same
|
||
mechanism -- sub-word fields are pages sharing a coordinate -- and px68k's code
|
||
implements exactly that mechanism, including the `$C80000` alias landing on the
|
||
adjacent byte. Agreement on a mechanism is much harder to get wrong by accident
|
||
than agreement on an output.
|
||
|
||
It is still not a board. STATUS item 3 (single- vs dual-address, 43.2/43.3) is
|
||
unaffected by any of this and remains the largest open hardware fact.
|
||
|
||
### 46.5 CORRECTION: CRTC R20 bit 11 turns the masking off
|
||
46.1 concluded there is no packed write path. That conclusion was drawn from
|
||
px68k's `GVRAM_Write` and two documents describing the default behaviour, and it
|
||
**missed a register bit that both emulators implement**. MAME's
|
||
`x68k_crtc_device::gvram_w` shows it first:
|
||
|
||
```c
|
||
if (m_reg[20] & 0x0800) /* "G-VRAM set to buffer" */
|
||
{
|
||
if (offset < 0x40000)
|
||
m_gvram_write_callback(offset, data, mem_mask); /* FULL WORD, unmasked */
|
||
}
|
||
else switch (m_reg[20] & 0x0300)
|
||
{
|
||
case 0x0100: /* 256 colour */
|
||
if (offset < 0x40000)
|
||
m_gvram_write_callback(offset, data & 0x00ff, 0x00ff);
|
||
else if (offset < 0x80000)
|
||
m_gvram_write_callback(offset - 0x40000, (data & 0x00ff) << 8, 0xff00);
|
||
```
|
||
|
||
The `case 0x0100` arm confirms 46.1 exactly -- `offset - 0x40000` is the **same
|
||
word, other byte**, and `data & 0x00ff` throws the CPU's high byte away on both
|
||
aliases. But the `m_reg[20] & 0x0800` arm bypasses the depth switch entirely and
|
||
writes the full 16 bits.
|
||
|
||
**px68k has the identical bit**, which is what makes this a mechanism and not a
|
||
MAME quirk: `if (CRTC_Regs[0x28]&8)`, commented `65536モードのVRAMアクセス
|
||
(Nemesis用)` -- "65536-mode VRAM access, for Nemesis". px68k's `CRTC_Regs` is
|
||
byte-indexed, so `[0x28]` is the **high byte of R20** and bit 3 of it is bit 11
|
||
of the register. Same bit, same effect, two independent implementations, and a
|
||
named shipping game that used it.
|
||
|
||
**So the 2:1 tax is a property of the default write path, not of the memory.**
|
||
|
||
### 46.6 A contiguous packed layout, derived and NOT yet tested
|
||
The second thing 46.1 missed is that the two 256-colour pages have **independent
|
||
scroll registers**. px68k's `Grp_DrawLine8(int page, int opaq)` indexes
|
||
`GrphScrollX[page*8]` and `GrphScrollY[page*8]`, selects the byte within the word
|
||
with `add esi, ecx` (page 0 -> low, page 1 -> high), and takes an `opaq` flag --
|
||
so the pages composite with transparency and can be offset from each other.
|
||
|
||
46.2 dismissed interleaving on byte count, and that dismissal assumed a 1-pixel
|
||
scroll. **Scrolling by 128 instead makes the used words contiguous**, which is
|
||
the whole difference:
|
||
|
||
- GVRAM row stride is fixed at 1024 bytes = 512 words (`shl esi, 10`).
|
||
- Write words 0..127 of each row, **unmasked, full 16 bits** (R20 bit 11).
|
||
- Page 0, unscrolled: `page0[i]` displays at screen column `i` -> columns 0..127.
|
||
- Page 1, X-scrolled by **+128**: `page1[i]` displays at column `i+128` ->
|
||
columns 128..255.
|
||
- Page 1 opaque and above page 0, so page 0's stale storage at 128..511 is
|
||
covered; page 1's storage at 128..511 displays at 256..383, which is off the
|
||
edge of the **real 256x256 mode this project already uses** (FINDINGS 23).
|
||
|
||
**128 contiguous words carry 256 pixels: 1.0 byte per pixel, against 2.0.** No
|
||
transparency mask to maintain, no stride for a DMAC to skip, and the writes are
|
||
`movem`-shaped. If it holds, it is exactly the halving 44.7 named:
|
||
|
||
| | B/pixel | frame | at 12fps | 22.8 min |
|
||
|---|---:|---:|---:|---:|
|
||
| 256-colour, default masked path | 2.0 | 98,304 | 1,152 KB/s | 1.61 GB |
|
||
| **packed via R20 bit 11 + page scroll** | **1.0** | **49,152** | **576 KB/s** | **0.81 GB** |
|
||
|
||
**This is a derivation, not a result.** What is CONFIRMED is the register bit
|
||
(both emulators, plus a named game) and the per-page scroll and opacity (px68k's
|
||
draw path). What is DERIVED and untested is the layout above: whether R20 bit 11
|
||
coexists with 256-colour *display* rather than forcing the 65536-colour
|
||
interpretation, whether the video controller's priority and transparency
|
||
registers can put page 1 over page 0 the way this needs, and what the 256-wide
|
||
screen does to page 1's off-edge storage.
|
||
|
||
**Unlike 43.2 and 43.8.1, this one the tree CAN answer.** It is a display-model
|
||
question, both emulators implement the mechanism, and the rig already screenshots
|
||
and compares pixel-exactly (`tools/bench/verify_frame256.py`). It is a register
|
||
setup and a snapshot, not a service manual -- and if it holds it halves the wire
|
||
for every path in this project, the shipping codec included.
|
||
|
||
## 47. The packed layout works on both emulators, and they disagree about what it costs (session 16)
|
||
|
||
46.6 derived a 1.0 byte/pixel layout and did not test it. It has now been built
|
||
and run on **both** emulators. **It renders correctly on both.** The two
|
||
disagree on two register semantics, and one of those disagreements decides
|
||
whether the thing is usable.
|
||
|
||
### 47.1 The write path, measured directly
|
||
`tools/bench/probe_packed.lua` writes one word and reads the two page aliases
|
||
back, under MAME:
|
||
|
||
| R20 | wrote | raw word | page 0 (low byte) | page 1 (high byte) |
|
||
|---|---|---:|---:|---:|
|
||
| 0x0110, bit 11 = 0 | `AB5C` | `005C` | `5C` | `00` |
|
||
| **0x0910, bit 11 = 1** | `AB5C` | `AB5C` | `5C` | **`AB`** |
|
||
|
||
**That is the 2:1 tax, and its off switch, in one table.** Masked, the CPU's high
|
||
byte is destroyed. In buffer mode one word write lands two picture bytes.
|
||
46.5's reading of `gvram_w` is confirmed by experiment, not just by code.
|
||
|
||
### 47.2 The packed layout renders correctly, on both
|
||
`tools/bench/show_frame256_packed.lua` (MAME) and `tools/bench/gvpack` (px68k):
|
||
|
||
- page 0, opaque bottom, unscrolled -> screen columns 0..127
|
||
- page 1, transparent top, X-scroll **384** -> columns 128..255, because column
|
||
`c` fetches `page1[(c+384) & 511]`
|
||
- words 0..127 of each row carry both halves: `(right << 8) | left`
|
||
- words 128..511 zeroed **once** -- page 1's storage at 384..511 sits under
|
||
columns 0..127 and must read 0 so the opaque page 0 shows through. Static
|
||
setup, not per-frame payload.
|
||
- the blob is built `--pack-transparent`: index 0 is the transparency key, so it
|
||
never appears in the picture and black lives at 255.
|
||
|
||
| | result | palette ceiling |
|
||
|---|---|---:|
|
||
| MAME, `verify_frame256.py` | **256x192 pixel-exact, letterbox true black** | 40.83 dB |
|
||
| px68k, `verify_gvpack.py` | **256x192 index-exact, letterbox on 255** | 40.83 dB |
|
||
|
||
**Per-frame payload: 128 words/row x 192 rows = 24,576 words = 49,152 bytes for
|
||
49,152 pixels. 1.0 B/pixel, against 2.0.**
|
||
|
||
`tools/bench/gvpack` links px68k's real `x68k/gvram.c` -- the address decode, the
|
||
bit-11 write path, the page-byte selection, the scroll wrap and the index-0
|
||
transparency test are px68k's own code, the way `tools/bench/c68k` links its CPU
|
||
core. The only mirrored part is `windraw.c`'s twelve-line page-ordering
|
||
dispatch, which is SDL-bound; it is quoted verbatim in `pick_order()`.
|
||
|
||
**Four negative controls, because a test that cannot fail proves nothing:**
|
||
|
||
| control | expected | got |
|
||
|---|---|---|
|
||
| ordinary unpacked 2.0 B/px path | pass | pass |
|
||
| packed, **bit 11 OFF** | fail | fail: right half is 24,576 px of index 0 |
|
||
| packed, **page-1 scroll removed** | fail | fail: 49,073 px differ |
|
||
| packed, priority `vc1=0x00` | fail (per MAME) | **PASS on px68k** -- see 47.3 |
|
||
|
||
### 47.3 Disagreement 1: the priority register, when the fields are equal
|
||
Video controller R1 (`0xE82500`) decides which page composites on top.
|
||
|
||
| `vc1` | MAME | px68k |
|
||
|---|---|---|
|
||
| `0x0000` | page 1 **not shown** -- right half black, 24,576 px differ | page 0 on top **transparently** -- renders correctly |
|
||
| `0x0002` | page 1 on top, **correct** | page 1 on top, **correct** |
|
||
|
||
They agree at `0x0002` and that is what the layout uses, so the result stands on
|
||
a setting both model identically. But the packed layout's correctness rests on a
|
||
register the two emulators do not model the same way, and neither is authority.
|
||
|
||
### 47.4 Disagreement 2: does buffer mode BLANK the display? -- and this one decides it
|
||
`tools/bench/probe_bit11_blank.lua` is the known-good 256-colour test with **one
|
||
line added**, setting bit 11:
|
||
|
||
- **MAME: the screen goes fully black.** Max channel 0, zero non-black pixels.
|
||
Buffer mode is a write *window*, not a display mode -- which is why the working
|
||
test clears bit 11 after painting.
|
||
- **px68k: it does not blank.** `Grp_DrawLine8` never reads `CRTC_Regs[0x28]`,
|
||
and `gvpack --keepbuffer` renders the frame correctly with the bit still set.
|
||
|
||
**This is the question the packed path lives or dies on.** If MAME is right, the
|
||
graphics layer is blanked for the whole time the CPU or DMAC is painting, and a
|
||
12fps full-frame player would show black for whatever fraction of each frame the
|
||
paint takes. If px68k is right, the packing is free. Both are plausible readings
|
||
of "G-VRAM set to buffer", and px68k's own comment -- `65536モードのVRAMアクセス
|
||
(Nemesis用)` -- says the bit exists for a game that blasted graphics through it,
|
||
which is at least consistent with the write-window reading.
|
||
|
||
**It is a hardware fact, and it is now the cheapest high-value one outstanding**
|
||
-- cheaper than 43.2, because a single real board plus the two-line probe above
|
||
settles it, and because the answer moves more numbers.
|
||
|
||
### 47.5 What it would be worth, DERIVED
|
||
Arithmetic on measured constants, not measurements. **Every line below is void if
|
||
47.4 goes MAME's way**, and the mode-decision cost model (44.3, 43.1) would need
|
||
re-deriving from scratch either way.
|
||
|
||
| | default masked | **packed** |
|
||
|---|---:|---:|
|
||
| bytes/frame into GVRAM | 98,304 | **49,152** |
|
||
| word writes/frame | 49,152 | **24,576** |
|
||
| wire at 12fps | 1,152 KB/s | **576 KB/s** |
|
||
| 22.8 min of game | 1.61 GB | **0.81 GB** |
|
||
| DMAC device->GVRAM, c=5 | 59.0% of budget | **29.5%** |
|
||
| CPU-painted literal, c=5 | 19.14 clocks/px (**misses** by 13%) | **9.57 (fits, 44% spare)** |
|
||
| CPU-painted literal, c=9 | 27.14 (**misses** by 60%) | **13.57 (fits)** |
|
||
|
||
The CPU-painted row is the one that overturns something. 44.7 concluded "a
|
||
CPU-painted full-frame literal does not fit at any physically reachable price,
|
||
and it misses at the floor itself." Packed, one `movem` word carries two pixels,
|
||
so the per-pixel cost becomes `(9.143 + 2c)/2 = 4.571 + c` -- and it fits at c=5
|
||
**and** at c=9. That conclusion is withdrawn, conditionally on 47.4.
|
||
|
||
### 47.6 What this does NOT establish
|
||
1. **Correctness was tested, cost was not.** Both harnesses write GVRAM
|
||
directly -- MAME through Lua's address space, `gvpack` by calling
|
||
`GVRAM_Write`. Neither runs 68000 instructions, so no clock in 47.5 is
|
||
measured. The `movem` shape of the packed writes is an assumption.
|
||
2. **The DMAC has not been near this.** 44.7's device->GVRAM transfer in buffer
|
||
mode is untested, and 43.2's single- vs dual-address question sits underneath
|
||
every `c` in 47.5.
|
||
3. **One frame, not a stream.** A static frame was painted and snapshotted.
|
||
Nothing here exercises per-frame toggling of bit 11, and if MAME is right
|
||
about blanking, that toggling is the whole problem.
|
||
4. **The codec was not considered.** VQ blocks and v7 spans address chunky
|
||
pixels; under the packed layout a word spans two columns 128 apart. Whether
|
||
the existing codec survives that is untouched -- 47 is about a literal frame.
|
||
5. **Two of 46.6's three assumptions held, one was wrong.** R20 bit 11 does
|
||
coexist with 256-colour display (after clearing it), and the off-edge storage
|
||
behaves. The priority guess was wrong: 46.6 assumed page 0 on top; it is
|
||
page 1.
|
||
|
||
## 48. The blanking disagreement is not symmetric, and the paper trail favours MAME (session 17)
|
||
|
||
47.4 filed the blanking question as two emulators disagreeing, and called both
|
||
readings of "G-VRAM set to buffer" equally plausible. **They are not equally
|
||
plausible, and the two implementations are not making the same kind of
|
||
statement.** Nothing here is a board, so 47.4 is not closed -- but the prior
|
||
moves, and it moves against the packed layout.
|
||
|
||
### 48.1 px68k is silent, not dissenting
|
||
`grep -a` matters here: `x68k/gvram.c` carries EUC-JP comments, so a plain
|
||
`grep` treats it as binary and reports **no matches at all** for any pattern.
|
||
Read with `-a`, R20's high byte appears in exactly one file:
|
||
|
||
$ for f in x68k/*.c x11/*.c; do n=$(grep -ac "CRTC_Regs\[0x28\]" $f); ...
|
||
x68k/gvram.c: 6
|
||
|
||
and all six are the address decode -- lines 150/153/158 inside `GVRAM_Read`,
|
||
211/221/225 inside `GVRAM_Write`, where `CRTC_Regs[0x28]&8` is R20 bit 11 and
|
||
line 211 carries the comment `65536モードのVRAMアクセス(Nemesis用)`.
|
||
|
||
**No px68k display code reads the bit anywhere.** `Grp_DrawLine8` and
|
||
`x11/windraw.c` never consult it. So px68k does not model buffer mode as
|
||
non-blanking; it does not model the display side of buffer mode **at all**.
|
||
Its "no" is an omission.
|
||
|
||
MAME's "yes" is the opposite -- a deliberate, commented claim, made twice:
|
||
|
||
x68k_v.cpp:407 if(m_crtc->gfx_layer_buffer()) // if graphic layers are set
|
||
return false; // to buffer, they aren't visible
|
||
x68k_v.cpp:766 if((m_video.gfx_pri == priority) && !m_crtc->gfx_layer_buffer() && ...
|
||
|
||
and the parallel bit 12 is modelled the same way for the text layer
|
||
(`x68k_v.cpp:756`, `!m_crtc->text_layer_buffer()`). Someone implemented a
|
||
semantic on purpose. **An assertion and a silence are not a tie.**
|
||
|
||
### 48.2 The register table names the bit "for display / for buffer"
|
||
Sharp's own R20 bit map, transcribed on Data Crystal's X68k IOMAP:
|
||
|
||
| bit | name | %0 | %1 |
|
||
|---|---|---|---|
|
||
| 12 | T-MEM | `表示用` | `バッファ用` |
|
||
| **11** | **G-MEM** | **`表示用`** | **`バッファ用`(bit 10〜8 は無効)**<br>`(G-VRAM が 65536 色表示時と同じ構造になる)` |
|
||
| 10 | SIZE | 512x512 | 1024x1024 |
|
||
| 9-8 | COL | %00 16 / %01 256 / %11 65536 | |
|
||
|
||
Two things in that one row:
|
||
|
||
1. **The bit is named "for display" against "for buffer"** -- not "16-bit write"
|
||
or "unmasked access". The naming is MAME's reading.
|
||
2. **`bit 10〜8 は無効` -- the colour-mode field goes invalid.** COL is what the
|
||
display side decodes a plane structure from. A layer whose colour mode is
|
||
undefined has nothing to render with, which is a *mechanism* for blanking
|
||
rather than a restatement of it.
|
||
|
||
The counter-reading survives, and it is the parenthetical: the doc says the
|
||
structure becomes the 65536-colour one, and does **not** say the screen goes
|
||
dark. That is why this is a shifted prior and not a result.
|
||
|
||
### 48.3 The cost of the MAME branch, DERIVED, and it is not a partial blank
|
||
47.4 left "measure what fraction of a frame the paint takes" as the fallback.
|
||
Bit 11 only has to be set across the GVRAM writes, so the blank interval is the
|
||
paint, not the frame. The measured blit is **53.6% of the frame budget**
|
||
(session 9) unpacked; packed halves the word count, so the floor is ~27% and the
|
||
ceiling ~54% depending on how much of the blit is stores.
|
||
|
||
**Either end is fatal for this content.** The graphics layer would be visible
|
||
between roughly half and three-quarters of each frame at 12fps, with the black
|
||
interval locked to frame rate -- a 12 Hz strobe over the whole picture, not a
|
||
tear or a partial band. And the packed layout has **no page left to flip to**:
|
||
both 256-colour pages carry picture, which is the entire point of it. There is
|
||
no version of the MAME branch where the packing is merely expensive.
|
||
|
||
### 48.4 px68k cannot testify about SCSI at all (bears on 43.2)
|
||
Checked while looking for a second opinion on single- vs dual-address. **px68k
|
||
does not emulate the MB89352.** `x68k/scsi.c` is 81 lines: it synthesises a
|
||
64-byte fake CZ-6BS1 boot ROM at `$EA0020` (the `Human68k` signature, the IOCS
|
||
`$F5` vector, `move.b d1,$e9f800`) and traps the IOCS call on the host. Its own
|
||
header says so -- `SCSI IOCS を特殊処理で対応。SPCはエミュレートしない`.
|
||
|
||
So the "second emulator" method that settled 46/47 **is not available for
|
||
43.2**, and never was. MAME models the dual-address row and has no DACK path;
|
||
px68k models no SPC. Item 2 needs the CZ-6BS1's `scsiexrom.bin` (8 KB, CRC
|
||
`7be488de`, absent here) disassembled for its DMAC DCR programming -- or a
|
||
schematic. It does not need another emulator.
|
||
|
||
**And the field to read is now pinned to the primary source.** MC68450
|
||
datasheet (Motorola ADI1216, Jul-89 printing), section 3.6.1 -- the DCR field
|
||
order is `XRM | DTYP | DPS | PCL`, MSB to LSB, and DTYP is the answer in one
|
||
two-bit field:
|
||
|
||
| DTYP | datasheet text, verbatim | addressing |
|
||
|---|---|---|
|
||
| `00` | `M68000 Compatible, Explicitly Addressed` | **dual** |
|
||
| `01` | `M6800 Compatible, Explicitly Addressed` | **dual** |
|
||
| `10` | `Device with ACK, Implicitly Addressed` | **single** |
|
||
| `11` | `Device with ACK and RDY, Implicitly Addressed` | **single** |
|
||
|
||
and 3.6.1.2 states the equivalence outright rather than leaving it to be
|
||
inferred:
|
||
|
||
> For M68000 type devices, the DMAC will use a **dual address transfer
|
||
> protocol** by running M68000 type bus cycles to transfer data to or from the
|
||
> device registers and a second bus cycle to complete the operand transfer from
|
||
> or to memory. [...] In the remaining two device protocols, the DMAC asserts
|
||
> the acknowledge signal to **implicitly address the device during a single
|
||
> address transfer** while it is explicitly addressing a memory location.
|
||
|
||
So 43.2's 5.0-vs-9.0 clocks/byte is decided by two bits in one byte the
|
||
CZ-6BS1's boot ROM writes at init. (Bit *positions* within DCR are the
|
||
conventional 7-6/5-4/3/2-0 split; the datasheet's own bit-number row did not
|
||
survive OCR, so treat the positions as unconfirmed and the field order and
|
||
encodings as quoted.)
|
||
|
||
**MAME confirms the negative half of this from the other side:**
|
||
`hd63450.cpp` contains **no DTYP handling at all** -- the only device-shape
|
||
field it decodes is `ocr & 0x30` (operand size). It cannot express an
|
||
implicitly-addressed device, which is why 43.2 was right that MAME settles
|
||
nothing here.
|
||
|
||
### 48.5 One thing the hunt did confirm independently
|
||
px68k's `kaiseki.txt` (the author's own analysis notes, 2014/2/14), on GVRAM
|
||
structure, unprompted and predating any of this:
|
||
|
||
256色の場合は、Page0の(0,0), Page1の(0,0), Page0の(1,0), Page1の(1,0)... と交互に並ぶ。
|
||
|
||
Page 0 and page 1 bytes alternate within the word at the *same* coordinate.
|
||
That is 46.1's page-not-pixel finding from a third source, and it is the
|
||
premise the packed layout is built on. **The layout's premise is solid; only
|
||
its visibility is in doubt.**
|
||
|
||
## 49. The streaming path, built and run: contiguity is the constraint, and the shipping rate does not fit the pipe (session 18)
|
||
|
||
STATUS item 3 has been open since session 7, and 45.4.1 stated the gap in as
|
||
many words: **the gate proves the decoder is pixel-exact over a whole window and
|
||
says nothing about how the bytes get to it.** `tools/bench/decode.lua` preloads
|
||
5,261,814 B of container into emulated RAM and lets `a0` walk through all of it.
|
||
The shipping player never holds a window at once.
|
||
|
||
That rig is now built. `src/player/stream.s`, `tools/bench/prep_stream.py` and
|
||
`tools/bench/stream.lua` decode the gate container **out of a bounded ring**,
|
||
with a modelled SCSI pipe as the producer, and the container living in a HOST
|
||
file rather than in emulated RAM.
|
||
|
||
### 49.1 The result: pixel-exact from a ring one twentieth the size of the stream
|
||
|
||
| ring | machine | wraps | mean hole | result |
|
||
|---|---|---:|---:|---|
|
||
| 256 KB | **stock 2 MB** | 18 | 14.7 KB (5.7%) | **120/120, final frame pixel-exact** |
|
||
| 128 KB | stock 2 MB | 37 | 10.6 KB (8.3%) | 120/120, pixel-exact |
|
||
| 96 KB | stock 2 MB | 54 | 15.6 KB (16.2%) | 120/120, pixel-exact |
|
||
| 80 KB | stock 2 MB | 60 | 8.9 KB (11.2%) | 120/120, pixel-exact |
|
||
| 64 KB | stock 2 MB | 107 | 23.4 KB (36.6%) | 120/120, pixel-exact |
|
||
| 48 KB | stock 2 MB | 111 | 8.9 KB (18.5%) | 120/120, pixel-exact |
|
||
|
||
Verified by `tools/bench/verify_decode.py`, the same comparison that gates
|
||
`decode.s`: the last frame against `tools/encoder/dlx.py`'s reconstruction, and
|
||
because a SKIP block is a claim about the previous frame still being in GVRAM,
|
||
the last frame is only right if all 120 were.
|
||
|
||
**A side effect worth naming: the rig's RAM ceiling is gone.** FINDINGS 44.6.4
|
||
audited 37/120 frames because the container did not fit 2 MB, and 45 raised
|
||
`RIG_RAM` to 6 MB to fix it. The streaming rig holds ~256 KB of stream and
|
||
reads the rest from the host, so **a stock 2 MB machine runs the whole window** —
|
||
and the machine it runs on is now the machine the player targets, rather than a
|
||
rig-shaped one.
|
||
|
||
### 49.2 The constraint is CONTIGUITY, and a byte-counting simulation cannot see it
|
||
`09_buffer_sim.py` asked whether cumulative supply ever falls behind cumulative
|
||
demand, in bytes, and FINDINGS 21 answered "zero required prefill". That test is
|
||
necessary and not sufficient. The block loop and the span chain read the stream
|
||
with a monotonically increasing `a0` and **no bounds check anywhere** —
|
||
`move.l (a0)+,d0`, `lea MODEB(a0),a0`, eleven unrolled `movem.l (a0)+`, a
|
||
`move.b (a0)+` per block index. None of it survives an address that wraps
|
||
mid-record.
|
||
|
||
So the ring needs the **whole next record resident AND contiguous**, not merely
|
||
enough bytes by the deadline. Those are different conditions and only the
|
||
second one is a byte count. `tools/analysis/19_ring_stream.py` models the ring's
|
||
addresses rather than its occupancy.
|
||
|
||
### 49.3 `aligned` beats `split`, and it is not close
|
||
Two policies can give the reader a contiguous record. Both columns below are for
|
||
**`s14_d5_all1500`**, the shipping candidate, in a 256 KB ring — the costs scale
|
||
with the container's record sizes, so they must be quoted per container:
|
||
|
||
| policy | mechanism | RAM cost | CPU cost |
|
||
|---|---|---:|---:|
|
||
| **`aligned`** | producer refuses to start a record it cannot finish; leaves a hole, restarts at 0 | 23.4 KB mean hole = **9.1%** of the ring | **0** |
|
||
| `split` | records wrap; ring's first MAXREC bytes mirrored into a shadow past its end | 0 | **46,394 clk/frame = 5.57% of the frame budget, forever** |
|
||
|
||
For the lighter gate container `rc_fr_singe_scsi_span` the same trade is
|
||
**5.7% of the ring against 3.64% of the frame budget** — same direction, same
|
||
verdict, smaller numbers.
|
||
|
||
The decoder already spends **77.0% of the budget on the mean frame and 91.1% at
|
||
p90** (FINDINGS 45). `split` puts p90 at **96.7%**. RAM is the thing this machine
|
||
has 2 MB of; clocks are the thing it has none of.
|
||
|
||
`aligned` also needs a per-record **index** on the fill side — and a branching
|
||
laserdisc game needs one anyway to seek to a branch point. **The policy that
|
||
costs no clocks reuses a structure the player cannot avoid.**
|
||
|
||
The third option — teach the block loop to wrap its own reads — is the expensive
|
||
one, and not because of the branch. A bounds test lands *inside* the instruction
|
||
sequences FINDINGS 30.4 and 40 fitted their constants to, so it does not cost a
|
||
compare: it costs **every span and per-block figure in the tree being
|
||
re-measured**.
|
||
|
||
### 49.4 Two independent implementations agree exactly
|
||
The Python simulation predicts the ring's behaviour from record sizes alone; the
|
||
Lua producer drives a real 68000 through MAME. They agree to the digit, and at
|
||
two ring sizes rather than one -- so it is not a coincidence of a single tiling:
|
||
|
||
| ring | | wraps | mean hole | usable ring |
|
||
|---|---|---:|---:|---:|
|
||
| 256 KB | `19_ring_stream.py` (from record sizes) | 18 | 14.7 KB | 94.3% |
|
||
| 256 KB | `stream.lua` (driving the 68000) | **18** | **14.7 KB** | **94.3%** |
|
||
| 128 KB | `19_ring_stream.py` | 37 | 10.6 KB | 91.7% |
|
||
| 128 KB | `stream.lua` | **37** | **10.6 KB** | **91.7%** |
|
||
|
||
They share no code. This is the same class of check as FINDINGS 46's — agreement
|
||
on a *mechanism* rather than on an output.
|
||
|
||
### 49.5 The shipping candidate does not fit the 488 KB/s pipe, and nothing was checking
|
||
`s14_d5_all1500` is the session-14 candidate: **29.19 dB at 496.7 KB/s** (43.6).
|
||
The pipe constant this tree has simulated against since session 2 is
|
||
**488 KB/s**. Those two numbers have never been put side by side.
|
||
|
||
wire demand 496.7 KB/s - pipe 488.0 KB/s = 8.7 KB/s OVER, on the MEAN
|
||
|
||
This is **not a burst a ring absorbs**. The deficit grows 744 B per frame for as
|
||
long as the scene runs — 87 KB over the 120-frame window, **523 KB per minute of
|
||
play**. No ring size fixes a sustained overrun, and quoting the window's 118.4 KB
|
||
"required prefill" for it would be the most flattering possible way to state it.
|
||
|
||
**Why it was never caught.** 42.1 established that 488 was never a bus figure and
|
||
that the binding resource is *clocks*, not KB/s — so the rate controller was
|
||
built to bind on `decode + c*bytes` and has **no pipe term at all**. That was a
|
||
defensible decision. What was not decided is that FINDINGS 21's buffer sizing,
|
||
and its "zero required prefill", would keep standing on a constant the design had
|
||
stopped enforcing. Item 4 has been open since session 7 for exactly this reason.
|
||
|
||
**The useful output is a requirement on the medium, not a verdict.** Since 488 is
|
||
unmeasured folklore and the intent is to measure a BlueSCSI directly, the tool
|
||
reports the threshold to measure *against*:
|
||
|
||
| container | wire | **zero-prefill pipe** | vs 488 |
|
||
|---|---:|---:|---:|
|
||
| `s14_d5_all1500` (the candidate) | 496.7 KB/s | **513.2 KB/s** | +5% |
|
||
| `rc_fr_singe_scsi_span` (the gate) | 446.1 KB/s | **451.4 KB/s** | -8% |
|
||
|
||
**513.2 KB/s is now a hardware acceptance test**, and it is 33% of SCSI-1's
|
||
asynchronous rating and 10% of its synchronous one (42.1). It is very likely met;
|
||
it has never been shown to be met.
|
||
|
||
### 49.6 The rig measures ARRIVAL, and the first version of it measured the wrong thing
|
||
`stream.s` has no frame clock — it asks for the next record the instant it
|
||
finishes the last. So it outruns any finite pipe, and its spin counter reports
|
||
**91 of 120 frames "stalled" at a pipe the same run shows is fast enough**. A
|
||
shipping player waits for vblank and spends that same time idle. Reporting that
|
||
count as an underrun would have been a false finding of exactly the shape this
|
||
project keeps filing.
|
||
|
||
The rig now records the emulated time at which each record becomes **resident**
|
||
and checks it against a 12 fps deadline, which is a question about arrival alone
|
||
and does not need the decoder paced:
|
||
|
||
| pipe | decoder waited | **records late** | worst | **required prefill** |
|
||
|---|---:|---:|---:|---:|
|
||
| unlimited | 0/120 | **0/120** | — | 0 |
|
||
| 520 KB/s | 85/120 | **0/120** | — | 0 |
|
||
| 488 KB/s | 91/120 | **1/120** | 4.9 ms (0.06 fr) | 2.3 KB |
|
||
| 460 KB/s | 97/120 | **2/120** | 9.7 ms (0.12 fr) | 4.4 KB |
|
||
|
||
The 2.3 KB at 488 against the simulation's 0.0 KB is a modelling difference, not
|
||
a disagreement: the Lua producer delivers in **whole records**, the simulation in
|
||
**512-byte sectors**, and whole-record granularity is the more conservative of
|
||
the two by up to one record's worth of latency.
|
||
|
||
### 49.7 What this does NOT establish
|
||
1. **The DMAC's clock debit is not modelled.** The pipe here is a constant byte
|
||
rate on the emulated clock. It is honest about *arrival order and residency*,
|
||
which is what a ring manages, and says nothing about the clocks the DMAC
|
||
steals from the 68000 while it delivers (43.2, and `W` is still undecided).
|
||
**A zero-late result from this rig means "the bytes were in time", NOT "the
|
||
frame fits".**
|
||
2. **Buffer stall-tolerance at a branch point is still untested.** Because the
|
||
decoder free-runs, the ring never backs up, so the ring-size sweep in 49.1
|
||
tests *wrap correctness* at each size and **not** the buffering that a seek
|
||
needs. At 64 KB the mean hole is 36.6% and effective capacity is one record —
|
||
it is single-buffered, and it passes anyway. Do not read 48 KB as a viable
|
||
player buffer.
|
||
3. **The producer is not an MB89352.** No sector-level command overhead, no
|
||
arbitration, no seek. `19_ring_stream.py` quantises to 512 B; the rig does not.
|
||
4. **The hole is a tiling effect and is not monotonic in ring size** — 96 KB
|
||
wastes more than 80 KB does. It depends on how record sizes tile the ring, so
|
||
a ring should be sized against the *measured* hole for the container it will
|
||
carry, not against a fraction.
|
||
5. **An unexplained cross-emulator gap, left open rather than explained away.**
|
||
The ring pass decodes the gate container at **561,532 cycles/frame under
|
||
MAME**. FINDINGS 45's figure for the same container is **641,444 cycles/frame,
|
||
and it is px68k's C68K core**, not MAME's -- so the two are not comparable and
|
||
the 12% between them is not evidence of anything yet. Getting MAME's own
|
||
full-pass number for this container needs a `decode.lua` timing run that did
|
||
not complete inside 25 minutes on this host; it was killed rather than left to
|
||
race another MAME job, which is how session 18 lost its first attempt at it.
|
||
**No figure in this section rests on the comparison.** Whoever picks it up:
|
||
run `decode.lua` on `rc_fr_singe_scsi_span.dlx` with nothing else touching
|
||
`tmp/`, and compare its "full 120-frame pass" against 561,532. If they agree,
|
||
the C68K/MAME delta is the thing to explain; if they do not, `stream.s`'s
|
||
frame loop is.
|
||
6. **`decode.s` is unchanged, and provably.** The block loop and span chain were
|
||
moved to `src/player/frame.i` and the constants to `geom.i` so both
|
||
front-ends assemble from literally the same bytes; `decode.s` still assembles
|
||
to the same **1,296 bytes** it did before the split, and `prep_dlx.py` still
|
||
emits a byte-identical blob after the loader maths moved to
|
||
`tools/bench/dlxload.py`. Both are asserted in `check.sh`.
|
||
|
||
## 50. The pipe constant is retired (session 18, USER DECISION)
|
||
|
||
**USER DECISION, after 49.5: remove the delivery-rate constant from the repo as
|
||
a live number.** 42.1 established in session 13 that it was never a bus figure —
|
||
a user-supplied "4 Mbps" with no recorded provenance, 10% of SCSI-1's
|
||
asynchronous rating — and 29.5 item 3 had carried "confirm the 4 Mbps figure" as
|
||
an open item since session 7. It was never confirmed. It was also never removed,
|
||
and 49.5 is what that cost: the shipping candidate ran 8.7 KB/s over it for five
|
||
sessions with nothing in the tree comparing the two.
|
||
|
||
### 50.1 What was actually wrong was the DEFAULT, not the number
|
||
The number being unmeasured was known and written down. What kept it load-bearing
|
||
was that **six tools defaulted to it** — `12_span_tradeoff.py`,
|
||
`14_dmac_chain.py`, `16_span_roundtrip.py`, `17_span_delivered.py`,
|
||
`19_ring_stream.py` and `tools/bench/stream.lua`. A default is how a figure gets
|
||
into a table without appearing in the sentence that reports the table. Every
|
||
span figure in FINDINGS 30–41 was scored against it; none of them had to say so.
|
||
|
||
**All six now take a REQUIRED argument with no fallback.** A tool that cannot run
|
||
without being told the rate cannot quietly assume one, and a result that had to
|
||
name its rate to exist is a result whose provenance travels with it.
|
||
|
||
A small confirmation of the same point fell out of doing it: writing the new
|
||
help text broke `--help` on four of the tools -- the text said "10%" and argparse
|
||
read it as a format spec -- and **nobody would have noticed**, because none of
|
||
these tools had ever been run with `--help` in this tree. The removal was the
|
||
first thing that made them state their own arguments out loud.
|
||
|
||
That is the transferable part: **an unmeasured constant is not made safe by
|
||
documenting that it is unmeasured.** 42.1 documented it perfectly and it went on
|
||
silently underwriting tables for five more sessions. It is made safe by deleting
|
||
the default.
|
||
|
||
### 50.2 What survives, and why it is not the same thing
|
||
`GATE_SPAN_KBPS` in `tools/bench/check.sh`. The gate container was **encoded**
|
||
with it, and every per-block and span constant in FINDINGS 41/43/45/49 is fitted
|
||
to that container. Changing it is a re-encode plus a re-measurement of all of
|
||
them, not an edit.
|
||
|
||
It is a **container recipe, not a delivery claim**, and check.sh says so at the
|
||
point of use. The distinction is the whole reason it can stay: nothing reads a
|
||
medium's throughput out of it.
|
||
|
||
### 50.3 What replaces it: a requirement, not a constant
|
||
`19_ring_stream.py` reports the **zero-prefill pipe** — the rate a medium must
|
||
clear for a given container to need no prefill at all:
|
||
|
||
| container | wire demand | **zero-prefill pipe** |
|
||
|---|---:|---:|
|
||
| `s14_d5_all1500` (the candidate) | 496.7 KB/s | **513.2 KB/s** |
|
||
| `rc_fr_singe_scsi_span` (the gate) | 446.1 KB/s | **451.4 KB/s** |
|
||
|
||
Its rate sweep is now anchored to each container's own wire demand (0.90x to
|
||
2.00x) rather than to a fixed list of absolute rates, so it privileges no
|
||
constant and stays meaningful for any container.
|
||
|
||
**This is a number to MEASURE A MEDIUM AGAINST, not one to design on**, which is
|
||
the difference that mattered. The BlueSCSI has never been benchmarked on this
|
||
machine and the intent has always been to measure it directly; it now has a
|
||
threshold to be measured against.
|
||
|
||
### 50.4 What this does NOT do
|
||
1. **It does not measure anything.** The delivery rate remains unknown. The
|
||
tree is now honest about that rather than carrying a placeholder — which is
|
||
a smaller claim than it sounds, and the right one.
|
||
2. **It does not re-derive the tables that were scored against the old figure.**
|
||
FINDINGS 30–41's span figures stand as measured *at that rate*; what changed
|
||
is that nothing new can be scored there without saying so. Anything that
|
||
needs a delivery rate to mean something is now waiting on a measurement, and
|
||
should be.
|
||
3. **It does not touch the gate container**, so the green light and every
|
||
constant fitted to it are unmoved. `./tools/bench/check.sh` is **ALL GREEN**
|
||
after the removal.
|
||
|
||
## 51. Pacing the decoder: seek slack is accumulated, not owned (session 19)
|
||
|
||
STATUS item 4, open since session 7 in one form or another and sharpened by
|
||
49.7.2. `tools/bench/stream.lua` decodes 120 frames out of a bounded ring and
|
||
the pass is pixel-exact at every ring size down to 48 KB — and that result could
|
||
not be read as a statement about buffering, because `src/player/stream.s` has no
|
||
frame clock. It asks for record *i* the instant it finishes record *i-1*, so it
|
||
outruns any finite pipe, the ring never backs up, and the producer's overlap
|
||
test is never the thing that refuses a placement. **A ring-size sweep under
|
||
those conditions tests wrap correctness at each size and nothing else.**
|
||
|
||
The rig now has a frame clock. `PACE`/`PACEON` (`$18034`/`$18038`) are written
|
||
by the producer — vblank or an MFP timer in the player — and frame *i* may not
|
||
start before tick *i*. `PACEON=0` leaves the loop free-running and is what the
|
||
green light's wrap gate still uses, so 49's figures are unmoved.
|
||
|
||
### 51.1 The ceiling: what a ring is worth once it is full
|
||
Every cell below is a full 120-frame decode on MAME's 68000, pixel-verified
|
||
against `tools/encoder/dlx.py`, on the gate container (`rc_fr_singe_scsi_span`,
|
||
446.1 KB/s wire, 36.5 KB mean record). "Ceiling" is the largest number of whole
|
||
records resident and unconsumed, i.e. **the frames the decoder could still draw
|
||
with delivery stopped dead**. Pipe `0` is unlimited, which isolates the ring's
|
||
own capacity from the rate.
|
||
|
||
| ring | unlimited | 460 KB/s | 488 KB/s | 520 KB/s | 600 KB/s |
|
||
|---:|---:|---:|---:|---:|---:|
|
||
| 64 KB | 2 | 2 | 2 | 2 | 2 |
|
||
| 96 KB | 3 | 3 | 3 | 3 | 3 |
|
||
| 128 KB | 5 | 4 | 4 | 4 | 4 |
|
||
| 192 KB | 6 | 4 | 5 | 5 | 5 |
|
||
| 256 KB | 8 | 4 | 7 | 7 | 7 |
|
||
| 384 KB | 11 | 4 | 10 | 11 | 11 |
|
||
| 512 KB | 15 | 4 | 11 | 14 | 14 |
|
||
|
||
**64 KB carries two frames and 96 KB carries three.** 49.7.2 warned that 48 KB
|
||
was single-buffered and passing anyway; this is the number, and it says the
|
||
small end of that sweep was measuring nothing about delivery.
|
||
|
||
### 51.2 Tolerance is `ceiling - 1`, and it was falsified rather than asserted
|
||
`DLX_CUT_AT`/`DLX_CUT_FR` stop the pipe dead at a chosen tick, as a seek does.
|
||
At 256 KB and 520 KB/s, with **7 records resident**:
|
||
|
||
| cut | underruns |
|
||
|---:|---:|
|
||
| 2 frame times | 0 |
|
||
| 6 | **0** |
|
||
| 7 | **1** |
|
||
| 8 | 1 |
|
||
| 10 | 1 |
|
||
|
||
**Seven resident records buy six frame times, not seven** — 500 ms, not 583.
|
||
The last one is spent covering the pipe's restart: a 36.5 KB record takes ~0.9
|
||
frame times to place at 512 KB/s of video, so the record due immediately after
|
||
the cut is still arriving when its slot opens. A design that reads the resident
|
||
count as its stall budget is over by one record, every time.
|
||
|
||
Every cut run stayed **pixel-exact**, including the ones that underran. That is
|
||
the expected shape and worth stating: under-delivery makes a frame LATE, not
|
||
wrong — `waitrec` spins and the decode is byte-identical when it resumes.
|
||
A rig that reported corruption here would be reporting its own bug.
|
||
|
||
### 51.3 The result that bears on a branching game
|
||
Slack is not a property the buffer has. It is **accumulated out of the surplus
|
||
between the pipe and the wire demand**, and a seek spends all of it.
|
||
|
||
| ring | pipe | ceiling | play needed to reach it, from empty |
|
||
|---:|---:|---:|---:|
|
||
| 256 KB | 488 KB/s | 7 fr (583 ms) | **4.83 s** |
|
||
| 256 KB | 520 KB/s | 7 fr | 2.83 s |
|
||
| 256 KB | 600 KB/s | 7 fr | 1.67 s |
|
||
| 512 KB | 520 KB/s | 14 fr (1.17 s) | **8.42 s** |
|
||
| 512 KB | 600 KB/s | 14 fr | 8.42 s |
|
||
|
||
**A bigger ring raises the ceiling and lengthens the climb to it.** The fill
|
||
rate is `pipe - wire`, which is set by the encoder and the medium; the ring only
|
||
sets where the climb stops. So the question a branch point asks is not "is the
|
||
buffer big enough" but "has there been enough play since the last branch point
|
||
to refill it" — and at 488 KB/s with a 256 KB ring the answer is **4.83 seconds
|
||
of play**. Two branch decisions closer together than that and the second one has
|
||
no buffer to spend, at any ring size. Dragon's Lair's decision points are
|
||
seconds apart.
|
||
|
||
This is the first statement in this tree about back-to-back branches, and it is
|
||
a consequence of the delivery model, not of the decoder.
|
||
|
||
### 51.4 Which resource is binding, said out loud
|
||
The producer now counts its refusals separately. A **rate** refusal (no credit
|
||
for a whole record) means a bigger ring buys nothing; a **ring** refusal (the
|
||
decoder still owns those bytes) means a faster pipe buys nothing. From the
|
||
decoder's side the two are identical — "no new record" — and they have opposite
|
||
fixes.
|
||
|
||
**At 460 KB/s every ring from 192 KB to 512 KB reports RATE-BOUND with a ceiling
|
||
of 4 and never fills inside 120 frames.** 460 is 13.9 KB/s over the container's
|
||
446.1 wire demand; that surplus fills 146 KB in the whole 10-second window. Any
|
||
ring larger than that is dead RAM in this scene. The zero-prefill pipe for this
|
||
container is 451.4 (49.5) — **clearing the arrival deadline and being able to
|
||
absorb a seek are different requirements, and the gap between them is large.**
|
||
|
||
### 51.5 An independent implementation agrees, within one record
|
||
`tools/analysis/20_seek_slack.py` is the same model written from record sizes in
|
||
Python, sharing no code with the Lua producer — the 49.4 pattern. Over all 35
|
||
cells of the grid:
|
||
|
||
- **35/35 of the rig's ceilings fall inside the sim's bracket**, and 33/35 sit
|
||
at the top of it.
|
||
|
||
The bracket is one record wide and is reported as a range rather than a number,
|
||
deliberately. At these rates the pipe delivers almost exactly one record per
|
||
frame slot, so "records resident at slot *i*" differs by one depending on
|
||
whether the sample is taken before or after that slot's delivery. Sampled after,
|
||
the sim matched the rig's ceiling in 33/35 cells; sampled before, it was exactly
|
||
one low in 33/35. Picking the sampling point that matched, and then reporting
|
||
the match as a cross-check, would have been fitting the model to the
|
||
measurement. Both are returned and the caller prints the range.
|
||
|
||
### 51.6 Two rig defects the pacing exposed
|
||
Both were in the producer, both were invisible while the decoder free-ran, and
|
||
both would have made the first paced result wrong in a plausible direction.
|
||
|
||
1. **The `RD_PTR` cross-check was really a test of how often `reap()` ran.** It
|
||
asserted, for every record it retired, that the decoder's released-to pointer
|
||
equalled that record's end. `RD_PTR` is a single pointer and names the end of
|
||
record `tail-1`; retiring several records in one pass — normal the moment the
|
||
decoder is paced and the pipe stops — made it fire a false MISMATCH on the
|
||
earlier ones. It now asserts on `tail-1` only, which is the invariant that
|
||
actually holds.
|
||
2. **`reap()` was skipped for the duration of a cut.** The cut returned early
|
||
from `produce()`, so the ring looked full for the whole seek and the producer
|
||
could not restart against space the decoder had long since released. A seek
|
||
stops delivery; it does not stop the decoder. Only credit and placement stop
|
||
now.
|
||
|
||
### 51.7 What this does NOT establish
|
||
1. **The pipe is still a model and the medium is still unmeasured.** Every rate
|
||
in 51.1 is a chosen input, not a measurement — FINDINGS 50 stands, and
|
||
`pace_run.sh` requires the rate for the same reason. The columns are a
|
||
sensitivity table; none of them is a claim about a BlueSCSI.
|
||
2. **The DMAC's clock debit is still not modelled** (49.7.1). A paced decoder
|
||
makes the ring's behaviour honest and does nothing about the clocks the DMAC
|
||
steals while it fills. "Survives a 500 ms cut" means the bytes were there,
|
||
not that the frames fit.
|
||
3. **One container, one scene.** The ceilings are in whole records, so they move
|
||
with record size; a scene with heavier frames has a lower ceiling in the same
|
||
ring. Nothing here is a universal per-KB figure and the tools take the
|
||
container as an argument for that reason.
|
||
4. **The pace gate costs the rig a `tst.l`/`beq.s` per frame** — `stream.s` is
|
||
1,418 bytes against 1,396. It is outside `src/player/frame.i`, so every
|
||
per-block and span constant in FINDINGS 24/30/40/41 is untouched, and
|
||
`decode.bin` is still 1,296 bytes at the same MD5 (asserted in `check.sh`).
|
||
5. **Seek TIME itself is not modelled.** The cut is a chosen duration. What a
|
||
real seek costs on the target medium is part of the same unmeasured
|
||
question as the rate, and it is the other half of what item 1 buys.
|
||
|
||
---
|
||
|
||
## 52. The DMAC configuration was never a mystery: it is in the IPL ROM (session 20)
|
||
|
||
ROADMAP called the ADPCM stream "the largest unpriced risk left in the project"
|
||
and asked for one cheap thing first — **put the audio DMA on the bus and see
|
||
what it does to the 86.7%.** Doing that needs a clocks-per-byte figure for the
|
||
audio channel, and the tree did not have one: `11_cpu_budget.py` charged audio
|
||
bytes `--dma-clocks-per-byte`, the *disk's* rate, defaulting to 5 and described
|
||
in its own help text as "single-address, bus held, no drive wait". That is a
|
||
description of the SCSI channel, and it is the favourable end of ROADMAP B3, an
|
||
open question worth 242 KB/s. **Audio was being charged the disk's guess.**
|
||
|
||
It did not have to be a guess for either of them. The X68000's IPL ROM programs
|
||
all four HD63450 channels itself, and **the ROM is on this machine** — MAME
|
||
boots the player rig with `-bios ipl10`. `tools/analysis/21_iplrom_dmac.py`
|
||
reads the configuration out of the image and decodes the MC68450 register
|
||
fields. It is a gate, not a report: every value is (address, expected bytes,
|
||
meaning), eight sites, and a mismatch or an unrecognised ROM revision exits
|
||
non-zero rather than decoding some other code. It is in `check.sh`, needs no
|
||
emulator, and runs in milliseconds.
|
||
|
||
**NAME THE LAYER.** This is not a measurement of a running machine and not real
|
||
hardware. It is the shipping ROM image (IPL 1.0, md5 `7fd4caab…`, 131,072 B),
|
||
read statically — evidence about what Sharp's engineers configured this board
|
||
to do, from the vendor, for these exact devices. Field layouts are SOURCED from
|
||
MC68450, Motorola, Jul 1989, the document `buscost.py` already cites.
|
||
|
||
### 52.1 What the ROM programs
|
||
|
||
Boot, `$FF0CCA`–`$FF0C58`: a ten-pair table at `$FF0D8E` initialises channels 0
|
||
and 1, then two inline runs do channels 2 and 3.
|
||
|
||
| ch | device | DAR | DCR | decoded |
|
||
|---|---|---|---|---|
|
||
| 0 | FDC | `$E94003` | `$80` | dual address, 8-bit port, cycle steal **without hold** |
|
||
| 1 | **SASI** | `$E96001` | `$80` | dual address, 8-bit port, cycle steal without hold |
|
||
| 2 | IOCS `_DMAMOVE` | per call | `$08` | dual address, **16-bit** port, burst |
|
||
| 3 | **ADPCM** | `$E92003` | `$80` | dual address, 8-bit port, cycle steal without hold |
|
||
|
||
Per transfer, `$FF9A82` (ADPCM play) and `$FF9944` (SASI): **OCR = `$32`** for
|
||
memory→device, `$B2` for device→memory. Both decode to `SIZE = 11` (byte),
|
||
`CHAIN = 00` (none) and — the load-bearing field — **`REQG = 10`, external
|
||
request: one operand per device request.**
|
||
|
||
### 52.2 Audio is dual-address and cannot hold the bus. 16..19 clocks a byte.
|
||
`DTYP = 00` is *explicitly addressed*, so every ADPCM byte is a memory read
|
||
followed by a device write: 4 + 5 = **9 clocks** (Fig 4-25 sheet 4, note 2 —
|
||
already in `buscost.py` for the span work). `XRM = 10` is cycle steal without
|
||
hold and `REQG = 10` is external request, so **the DMAC arbitrates for the bus
|
||
once per byte and hands it straight back.** There is no burst to amortise the
|
||
front-end (5..8 clocks, §4.5.2.1) and back-end (2, §4.5.2.2) over. The audio
|
||
byte costs **16 clocks best case, 19 worst** — not 5.
|
||
|
||
### 52.3 The byte rate, derived rather than restated
|
||
`ratectl.AUDIO_KBPS = 7.8` had no derivation next to it, and ROADMAP flagged it
|
||
as exactly the kind of figure that cost the project a 2x error in FINDINGS 43.
|
||
It survives, with a correction of units: 15.6 kHz is the MSM6258V's 8 MHz clock
|
||
÷512 = **15,625 samples/s**, 4 bits each, two to a byte = **7,812.5 B/s**. The
|
||
7.8 is that in *decimal* kB; `11_cpu_budget.py` was multiplying it by 1024, so
|
||
it read 2.4% high. Harmless, and now derived from the sample rate in
|
||
`buscost.ADPCM_BYTES_PER_S` instead of typed in.
|
||
|
||
**The request count does not halve.** 7,812.5 B/s is 7,812.5 DMA requests/s,
|
||
because the port is 8 bits and the operand is a byte. That is the FINDINGS 43
|
||
trap in the other stream, and it does not spring: nobody had denominated audio
|
||
per word.
|
||
|
||
### 52.4 So: what the audio does to the bus. Almost nothing.
|
||
651.0 B/frame at 12 fps × 16..19 clocks = **10,417..12,370 clocks of 833,333 —
|
||
1.25% to 1.48% of the frame.** The decoder's measured mean is 68.5% of the frame
|
||
period on the gate container, so audio takes about **4% of what the decoder
|
||
leaves**, and about 1.7% of it on the worst frame (which is already at 110.8%
|
||
and misses with or without audio).
|
||
|
||
**P6's bus risk does not materialise.** The concern was sound and the answer is
|
||
that a second DMA consumer at 7.8 kB/s is not what a bus at 88% occupancy is
|
||
short of. The debit was 3.2x..3.8x understated, and it is still small.
|
||
|
||
### 52.5 THE ONE THAT MOVES SOMETHING: the disk channel is programmed the same
|
||
ch1, SASI, `DCR = $80`, `OCR = $B2` — **dual address, 8-bit port, cycle steal
|
||
without hold, external request.** Byte by byte, full arbitration each time. ch0
|
||
(FDC) too. Sharp programs every explicitly-addressed 8-bit device on this board
|
||
identically, and by 52.2's arithmetic that is **16..19 clocks per delivered
|
||
byte**.
|
||
|
||
**A CORRECTION TO THIS SECTION AS FIRST WRITTEN, made in the same session.**
|
||
It cited 42.4's sensitivity table — `W <= 6` fits 0/120 frames, `W = 8` misses
|
||
47/120 — as though those figures were in clocks per BYTE. **They are per WORD,
|
||
and FINDINGS 43 voided them**: 43 is the section that caught `W` being charged
|
||
per word to a byte-wide port, and it says in terms that 42.3's 0/120 was never
|
||
physically reachable. Quoting them here would have re-imported the exact 2x unit
|
||
error 43 exists to have corrected, one section after using the same trap as a
|
||
warning. They are struck, and nothing below depends on them.
|
||
|
||
In the corrected unit the ladder is a per-byte cost of the DMAC's own
|
||
configuration, and it is the ladder `buscost.py` already carries:
|
||
|
||
| configuration | clk/byte |
|
||
|---|---:|
|
||
| single address, bus held | 5 |
|
||
| dual address, bus held | 9 |
|
||
| single address, arbitrated per byte | 12 |
|
||
| **dual address, arbitrated per byte — what the ROM programs** | **16..19** |
|
||
|
||
`15_bus_occupancy.py` sweeps it on the gate container (37,403 B/frame, measured
|
||
mean decode 570,958 clocks):
|
||
|
||
| W clk/B | video clk/frame | % of frame | CPU + audio + video |
|
||
|---:|---:|---:|---:|
|
||
| 5 | 187,017 | 22.4% | 92.2% |
|
||
| 8 | 299,228 | 35.9% | 105.7% |
|
||
| 12 | 448,842 | 53.9% | 123.6% |
|
||
| **16** | 598,455 | 71.8% | **141.6%** |
|
||
| **19** | 710,666 | 85.3% | **155.0%** |
|
||
|
||
**This table is the statement, and it is not corroborated by 42.4.** The
|
||
resemblance between the `W = 8` row here and 42.4's 47/120 is a coincidence of
|
||
two different units on two different containers at two different rates, and
|
||
calling it a cross-check — as this section did when first written — was
|
||
manufacturing agreement out of a unit error.
|
||
|
||
**This does not close B3.** `scsiexrom.bin` drives an MB89352, not the SASI
|
||
port, and a different ROM may configure it differently. What changed is the
|
||
prior and the framing: 42.6 says the handshake "is ours to choose, not to
|
||
receive", and that is still true — but **nothing in this tree has shown a
|
||
cheaper configuration is reachable for an explicitly-addressed 8-bit port, and
|
||
the vendor's own answer is the expensive one.** Holding the bus, which is what
|
||
separates 9 from 16..19, is a *requirement on the player's DMAC programming*
|
||
rather than a range the hardware hands us. It is now the largest open number in
|
||
the project, ahead of the rate.
|
||
|
||
### 52.6 Audio outranks the disk at the arbiter
|
||
CPR: FDC 0, **ADPCM 1**, SASI 2, `_DMAMOVE` 3 — lower is higher priority. With
|
||
the ROM's arrangement, when both channels want the bus in the same slot ADPCM is
|
||
served first. An audio byte is never the thing that waits; **a video byte is.**
|
||
Relevant to 51's underrun analysis, which models delivery as a smooth rate.
|
||
|
||
### 52.7 What this does NOT establish
|
||
1. **Static read of a ROM image, not a running machine.** No emulator executed
|
||
this code for the purpose; the claim is about bytes in the shipping image.
|
||
Real hardware would confirm the registers, not the timings.
|
||
2. **The timings are datasheet, not measured.** 9 clocks for the transfer and
|
||
5..8 / 2 for the arbitration come from MC68450 Fig 4-25 and §4.5.2. They are
|
||
the same source `buscost.py` already rests on for the span work, and they
|
||
have never been checked against a board.
|
||
3. **IPL 1.0 is a pre-SCSI machine.** ch1 is SASI. B3 stands.
|
||
4. **Our player is not obliged to copy the ROM.** It programs these registers
|
||
itself. 52.5 is a prior and a warning, not a measured ceiling — and the
|
||
experiment that would settle it is P4, not another reading.
|
||
5. **Nothing here is an audio implementation.** P6's other risks — extraction,
|
||
encode, container interleave, the second stream's effect on `wire` and hence
|
||
on 51.3's refill climb — are untouched. Only the bus question is answered.
|
||
6. **The frame-period accounting assumes the DMAC does not overlap the CPU**
|
||
(`buscost.DMA_OVERLAPS = False`), which is FINDINGS 35's premise: no cache,
|
||
a two-word prefetch queue. A stolen bus cycle is a stopped 68000.
|
||
|
||
---
|
||
|
||
## 53. The loader moves onto the 68000, and a scene change finally has a price (session 21)
|
||
|
||
ROADMAP P1 and P2. Since session 1 the two load-time transforms have been done
|
||
**host-side**, in `tools/bench/dlxload.py`, with the rigs pushing the *result*
|
||
into emulated RAM: the codebooks expanded to word-per-pixel form (CB1 to 32 B an
|
||
entry, CB4 to 8 B) and the 24-bit palette packed to `GGGGGRRRRRBBBBBI` with the
|
||
shared LSB chosen per entry. That was the right call while the inner loop was
|
||
what was being measured — charging a once-per-scene cost to the per-frame path
|
||
would have flattered or damned it for no reason — but **a player has no host.**
|
||
|
||
`src/player/load.i` does both on the 68000, out of the raw container header as
|
||
it comes off the disc. `tools/bench/loadgate.s` is its front-end, the way
|
||
`decode.s` is `frame.i`'s.
|
||
|
||
**NAME THE LAYER.** Everything here is **emulated**: MAME 0.277 `x68000`,
|
||
`-bios ipl10`, stock 10 MHz / 2 MB, cross-checked on px68k's C68K core. Nothing
|
||
has run on real hardware.
|
||
|
||
### 53.1 It reproduces dlxload.py exactly, on both cores
|
||
|
||
`dlxload.py` stays the reference — what changed is **where the transforms run,
|
||
not what they produce** — so the gate is byte-for-byte, not "close enough":
|
||
|
||
- **CB1 8,192 B, CB4 2,048 B, palette 512 B: identical.** A wrong codebook byte
|
||
is a wrong colour in every block that uses that codeword, in every frame of
|
||
the scene, and a wrong shared LSB is a *slightly* wrong colour, which is
|
||
exactly the sort of defect that gets attributed to the codec.
|
||
- **The palette half is read back out of the palette registers at `$E82000`**,
|
||
not out of a RAM shadow, so "the words reached the hardware" is part of what
|
||
passes.
|
||
- **The darkest-entry index agrees too** (255 on the gate container). It comes
|
||
out of an `argmin` whose tie-break has to match numpy's — first index at the
|
||
minimum wins — and it is what the letterbox is filled with.
|
||
- **Both CPU cores produced the same 10,752 bytes**, and the same as the host.
|
||
|
||
`tools/bench/load_run.sh` runs it and `check.sh` gates it.
|
||
|
||
### 53.2 What it costs, measured on two cores
|
||
|
||
| stage | MAME clocks | C68K clocks | Δ | data bus (C68K) |
|
||
|---|---|---|---|---|
|
||
| scratch tables (boot only) | 52,919 | 61,622 | +16.4% | 6.5% |
|
||
| **P1** codebook expansion | 92,609 | 95,304 | +2.9% | **43.2%** |
|
||
| **P2** palette entries | 97,019 | 97,348 | +0.4% | 14.3% |
|
||
| BOOT: all three | 246,957 | 253,614 | +2.7% | 23.2% |
|
||
| **SCENE CHANGE: P1 + P2** | **189,627** | 192,322 | +1.4% | 28.6% |
|
||
|
||
**A scene change costs 18.96 ms of 68000 time — 22.8% of one 12 fps frame.**
|
||
Boot costs 24.70 ms. Bus occupancy is data accesses only (C68K does not see
|
||
prefetch), so it is a lower bound; the expansion is the bus-heaviest thing here
|
||
because it is a pure copy, and it still runs alone.
|
||
|
||
**The stages are exactly additive on the exact core.** `P1 + P2 - SCENE = 330`
|
||
clocks, and `tables + P1 + P2 - 2x330 = 253,614 = BOOT`, to the clock — 330 is
|
||
the front-end's own per-pass overhead. On MAME the same identity closes to 1.8%,
|
||
which is one tick of its host clock over the 0.99 s run. Two instruments,
|
||
two granularities, one arithmetic. (That tick was written here as 1/55.46 s and
|
||
is **1/56.69 s** — MAME's raster, not the hardware's, 54.5. 17.64 ms over 990 ms
|
||
is 1.78%, so the sentence was right and the label was wrong.)
|
||
|
||
### 53.3 The scratch tables are scene-independent, so they are not in the scene path
|
||
|
||
Packing a palette entry needs the squared error of both choices of the shared
|
||
LSB, per channel. That is three table reads and a sign test here, out of three
|
||
tables — the 6-bit-to-8-bit rendering the CRTC performs, its square, and the
|
||
per-channel error difference — and **not one of them describes the scene.** They
|
||
describe the machine. `pal_tables` is therefore a separate entry point from
|
||
`pal_pack`, built once at boot: **5.29 ms saved on every scene change**, 22% of
|
||
what a naive port of `dlxload.py` would have charged per scene.
|
||
|
||
### 53.4 The two cores disagree only where the multiplies are, and C68K is wrong in kind
|
||
|
||
The table build is the only code in this tree that multiplies, and it is the
|
||
only stage where the two cores disagree by more than 3%. **px68k's C68K charges
|
||
a flat 50 clocks for `MULU` and `MULS`** regardless of the operand
|
||
(`c68kmacro.h:1869/1883`, `RET(50 + EA_CLOCKS_...)`); the 68000 charges
|
||
**38 + 2n**, n counting bits in the source. For the 576 multiplies this code
|
||
executes, the real total is 24,192 clocks against C68K's 28,800: **the flat rate
|
||
explains 4,608 of the 8,703 clock gap, and 4,095 clocks — 7.7% of the stage —
|
||
are NOT explained.** Recorded as open rather than rounded away; the residual is
|
||
somewhere else in the two cycle tables and this stage is not worth the hunt.
|
||
|
||
**The consequence is general and belongs in the reader's head:** where a future
|
||
measurement contains multiplies, C68K over-charges them, and it is the second
|
||
opinion this tree leans on for every cycle figure. Nothing else in
|
||
`src/player/` multiplies — index scaling is `lsl.w #5`/`#3` by construction —
|
||
so no figure in FINDINGS 24-52 is affected.
|
||
|
||
### 53.5 Where the cost actually lands: the scene change, priced
|
||
|
||
`tools/analysis/22_scene_load.py`, cycle counts parsed out of the rig's own log
|
||
rather than pasted in as constants. Three costs in three units, and **the third
|
||
is the one that compounds**:
|
||
|
||
- **BYTES.** The header region is **5,920 B** (palette 768 + CB1 4,096 + CB4
|
||
1,024 + 32) and it must arrive before frame 0 can be decoded. It is not part
|
||
of any frame record, so **no rate table in this tree has ever counted it.**
|
||
- **CLOCKS.** 189,627, from 53.2.
|
||
- **ACCUMULATED SLACK.** Those bytes are bytes the pipe did not spend filling
|
||
the ring, so they cost play-time at the surplus rate `pipe - wire` — the
|
||
currency FINDINGS 51.3 established a branch point spends.
|
||
|
||
| pipe KB/s | header ms | + load ms | total | frame slots | surplus KB/s | slack cost |
|
||
|---|---|---|---|---|---|---|
|
||
| 451.4 | 12.81 | 18.96 | 31.77 | 0.38 | 5.3 | **1.099 s** |
|
||
| 488 | 11.85 | 18.96 | 30.81 | 0.37 | 41.9 | 0.138 s |
|
||
| 513.2 | 11.27 | 18.96 | 30.23 | 0.36 | 67.1 | 0.086 s |
|
||
| 600 | 9.64 | 18.96 | 28.60 | 0.34 | 153.9 | 0.038 s |
|
||
|
||
(Rates are explicit arguments with no default, FINDINGS 50. `wire` is 446.1
|
||
KB/s on the gate container, audio included.)
|
||
|
||
**Two readings, and the second is the finding.** First: the whole fixed cost of
|
||
a scene change is **about a third of one frame slot** — it is not what makes a
|
||
branch point expensive, the seek and the refill climb are. Second: **the slack
|
||
cost is hypersensitive to the rate**, because it is divided by a surplus that
|
||
goes to zero. At 488 KB/s the header lengthens the climb by 138 ms; at 451.4
|
||
KB/s — the *arrival-deadline* rate for this same container, 49.5 — the same
|
||
5,920 bytes cost **1.1 seconds of play**. The header is cheap only where the
|
||
pipe already has room, which is the same place everything else in this project
|
||
is cheap.
|
||
|
||
### 53.6 The alternative that was not taken
|
||
|
||
The encoder could ship the codebooks pre-expanded and P1 would not exist. That
|
||
trades **9.26 ms of 68000 time for 5,120 more bytes in every scene header** —
|
||
10.5 ms of pipe at 488 KB/s, and 5,120 bytes that lengthen the climb again by
|
||
the arithmetic above. **Derived, not measured**, from the two figures either
|
||
side of it. It is close to a wash in milliseconds and it is not a wash in
|
||
*kind*: the CPU is idle during a seek and the pipe is the resource this project
|
||
is short of. The transform stays on the 68000.
|
||
|
||
### 53.7 What is still open in P2
|
||
|
||
**The encoder still does not reserve a black entry** (23.4), so the letterbox
|
||
gets the palette's closest thing to black — index 255 here — rather than a true
|
||
black with `I = 0`. That half of P2 is encoder-side, it changes the container,
|
||
and it moves every constant fitted to the gate container, so it is a re-encode
|
||
plus a re-measurement rather than an edit. `load.i` is ready for it: it reads
|
||
whatever the palette section holds and reports the darkest index either way.
|
||
|
||
---
|
||
|
||
## 54. The frame clock moves onto the 68000, and the 12 fps frame turns out never to have existed (session 22)
|
||
|
||
ROADMAP P3, and the item was phrased "needs MFP timer or VBL" — which quietly
|
||
assumes one of those can do it. Neither can, and finding out why produced a
|
||
better clock than either and a correction to an instrument the whole tree reads.
|
||
|
||
`src/player/clock.i` is the clock; `src/player/clockgate.s` and
|
||
`tools/bench/clock.lua` measure it; `tools/analysis/23_frame_clock.py`
|
||
enumerates the space it was chosen from and prices its cadence. Two new stages
|
||
in `tools/bench/check.sh` gate it.
|
||
|
||
**Layer: MAME 0.277's emulated X68000, not real hardware.** The MFP, the CRTC
|
||
and the interrupt sequence are all the emulator's. Where the emulator and the
|
||
registers disagree — and they do, 54.5 — the code is built on the registers.
|
||
|
||
### 54.1 No MFP timer can tick at 12 Hz, and none can tick as slowly as a frame
|
||
|
||
The MC68901's timer clock on this board is 16 MHz / 4 = 4 MHz
|
||
(`sharp/x68k.cpp:1027-1028`), its prescaler ladder is `{4, 10, 16, 50, 64, 100,
|
||
200}` (`machine/mc68901.cpp:173`) and its data register is 8 bits. So:
|
||
|
||
* the **slowest** tick a single timer can produce is 4e6/(200·256) =
|
||
**78.125 Hz**, which is 6.5× faster than a 12 fps frame — a software divider
|
||
is required whatever the source;
|
||
* **4e6/12 = 333,333.33 is not an integer**, so no prescale/data pair divides to
|
||
12 Hz at all. `23_frame_clock.py` walks all 7 × 256 of them and finds zero.
|
||
|
||
A timer clock is therefore not "the simple option". It is a divider *plus* an
|
||
interrupt rate 3.6× higher than the raster's, at an arbitrary phase against the
|
||
scan.
|
||
|
||
### 54.2 The raster cannot do it by whole division either — and the fix is exact
|
||
|
||
V-DISP is on MFP GPIP4 (`x68k.cpp:1139`), and the same pin is Timer A's event
|
||
input (`mc68901.cpp:167`, `GPIO_TIMER = {GPIP_4, GPIP_3}`); its interrupt is
|
||
channel 6, `IR_GPIP_4 = $40` (`mc68901.cpp:76`). The raster is 31,500/568 =
|
||
**55.4577 Hz** exactly. Every whole divide misses:
|
||
|
||
| Timer A event count | fps | error |
|
||
|---:|---:|---:|
|
||
| 4 | 13.8644 | +15.54% |
|
||
| 5 | 11.0915 | −7.57% |
|
||
|
||
12 fps needs **4.6215 refreshes per frame**. So the divider keeps a remainder:
|
||
|
||
```
|
||
each V-DISP: acc += fps*VTOTAL ; 12*568 = 6816
|
||
if acc >= 31500: acc -= 31500 ; PACE += 1
|
||
```
|
||
|
||
Long-run rate is `fps·VTOTAL/VTOTAL` = **12.000000 fps exactly**, with a
|
||
remainder that never accumulates. Both constants are **read out of the CRTC at
|
||
init** — `R04+1` for VTOTAL, `R20` bit 4 checked for the 31.5 kHz mode — so the
|
||
clock is derived from the registers that generate the raster it counts, and the
|
||
two cannot drift apart. The accumulator peaks at 38,316, so it is 16-bit
|
||
arithmetic on a 68000; `clk_init` refuses rather than overflow (the ceiling is
|
||
fps < 59.9 at this VTOTAL).
|
||
|
||
**Measured over 3,000 refreshes: 3,000 interrupts, 649 ticks, where 649.1429
|
||
were exactly due — an error of −0.14 ticks, i.e. the remainder still held.** The
|
||
gate is stated in ticks and not in ppm on purpose: a remainder-keeping divider
|
||
is off by at most one tick over *any* window, so quoting ppm would let a longer
|
||
window advertise a tighter clock for nothing.
|
||
|
||
### 54.3 It costs 181.35 clocks per V-DISP — 838 per frame, 0.10% of the budget
|
||
|
||
The host cannot time this: MAME's Lua sees the machine once per screen frame,
|
||
17.64 ms, and the interrupt costs microseconds. So the **68000 times it
|
||
itself**. `clockgate.s` runs a one-instruction loop for a window of 3,000
|
||
refreshes with the clock off and again with it armed:
|
||
|
||
```
|
||
clock off: iters0·L = clocks in the window -> L
|
||
clock on: iters1·L + ints·H = clocks in the window -> H
|
||
```
|
||
|
||
`L` is **calibrated, not looked up** — the point is to price the clock on the
|
||
machine rather than against `buscost.py`, which is the table being checked.
|
||
|
||
| | |
|
||
|---|---:|
|
||
| loop iteration, calibrated over 13,926,121 of them | **38.000002 clocks** |
|
||
| per V-DISP interrupt, over 3,000 | **181.35 clocks** |
|
||
| per 12 fps frame (4.6215 interrupts) | **838 clocks = 0.1006%** |
|
||
|
||
`L` landing on a whole number to seven digits is the check that licenses the
|
||
subtraction, and it is also an independent confirmation of `buscost.py`'s model:
|
||
`addq.l #1,(xxx).L` is 3 instruction words + 4 data accesses = 7 bus cycles = 28
|
||
clocks, plus 10 for the `bra.s`.
|
||
|
||
**The 181.35 decomposes exactly.** By the same model the handler body is 130
|
||
clocks on a V-DISP that emits no tick and 164 on one that does; over the measured
|
||
649/3,000 mix that is 137.355, leaving **43.99 clocks for the interrupt exception
|
||
sequence** — the textbook 44, measured here rather than recalled.
|
||
|
||
For comparison, the cheapest exact MFP-timer clock would interrupt 16.7 times a
|
||
frame instead of 4.6: **3.6× the cost, for a tick with no fixed relationship to
|
||
the scan.**
|
||
|
||
### 54.4 THE ONE THAT MOVES SOMETHING: there is no 83.33 ms frame, and there never was
|
||
|
||
12 fps on a 55.4577 Hz raster is 4.6215 refreshes, so a frame is shown for **4
|
||
refreshes (72.13 ms) or 5 (90.16 ms)** — 37.9% of them short. The 833,333-clock
|
||
budget every figure in this project is priced against is the **mean** slot, and
|
||
the short slot is **13.4% under it**.
|
||
|
||
With the per-frame decode costs (`tmp/c68k_frames.csv`, 120 frames of the gate
|
||
container) run through the actual divider and the actual pace gate:
|
||
|
||
| tick source | short slot | frames over it | no idle left |
|
||
|---|---:|---:|---:|
|
||
| nominal 1/fps model (no raster has it) | 833,333 | 1/120 | **1/120** |
|
||
| the host tick, as `stream.lua` really emits it | 705,590 | 10/120 | **4/120** |
|
||
| the 68000's clock, hardware raster | 721,270 | 10/120 | **4/120** |
|
||
| the 68000's clock, MAME's raster | 705,590 | 10/120 | **4/120** |
|
||
|
||
**The cadence was already there and nothing had named it.** `stream.lua`'s tick
|
||
is `floor((t - t_rel) * fps)` — which *looks* uniform and is not, because Lua
|
||
only sees the machine at frame boundaries, so its ticks land on refreshes and its
|
||
gaps are the same two whole numbers. Every host-paced result in FINDINGS 49 and
|
||
51 already carried a 4/5 cadence. **P3 did not introduce it. It moved who
|
||
produces it onto the machine and made it visible.**
|
||
|
||
**A short slot is not a dropped frame.** The pace gate says only "not before
|
||
tick i", so a frame that overruns spends the next frame's idle and the clock
|
||
recovers itself; the cost is one frame presented a refresh late. What the table
|
||
counts is frames with no idle left, and the difference between the nominal row
|
||
and the raster rows — **1 against 4** — is the entire price of the cadence on
|
||
this container.
|
||
|
||
**The expensive frame is frame 0, at 923,146 clocks = 111% of the nominal
|
||
budget**: the first frame of a scene has nothing to SKIP against, so it is the
|
||
whole picture in one slot. Most of what follows it in those counts is that
|
||
transient draining. It also means the cost lands **at a scene change**, next to
|
||
FINDINGS 53.2's 18.96 ms of loader and the seek — not spread over the window.
|
||
|
||
`src/player/stream.s` now counts this itself (`LATEFR`/`LATEMAX`/`LATE1ST`), and
|
||
the rig's count matches the offline model **exactly**: 4/120, first at frame 1,
|
||
on both tick sources. The counter sits ahead of the wait loop and the
|
||
free-running path executes none of it, so FINDINGS 49's figures are untouched.
|
||
|
||
### 54.5 MAME's raster runs 2.22% fast, and the whole tree has been sampling it
|
||
|
||
`x68k_crtc.cpp refresh_mode()` computes the frame period as
|
||
`(scr.max_x * scr.max_y)` dots over the dot clock, with
|
||
`scr.max_x = m_htotal - 8` — one character cell short, and an **inclusive
|
||
rectangle bound used as a count**. In the 256-wide mode that is 360 where the
|
||
registers say 368, so MAME's refresh is fast by **368/360 = 1.02222**:
|
||
|
||
* registers: 31,500/568 = **55.4577 Hz**
|
||
* MAME, measured by `clock.lua` over 3,000 frames: **56.6901 Hz**
|
||
|
||
The two agree to six digits with `clock_69m()/6 / (360·568)`, so this is the
|
||
mechanism and not a coincidence. Consequences, and the third one is why it is
|
||
worth this much space:
|
||
|
||
1. **Every "1/55.46 s granularity" note in this tree was wrong** — it is
|
||
1/56.69 s, 17.64 ms. Corrected in `decode.lua`, `load.lua`, `span.lua`,
|
||
`blit.lua`, `loadgate.s` and `check.sh`, with the derivation put once in
|
||
`crtc_mode.lua`. No conclusion changes: 53.2's "one tick over the 0.99 s run"
|
||
is 1.78% at the corrected figure and was quoted as 1.8%.
|
||
2. **68000 cycle figures are untouched.** The CPU clock is 40 MHz/4 and has
|
||
nothing to do with the screen. Nothing in FINDINGS 24–53 moves.
|
||
3. **A raster-paced player runs 2.22% fast under MAME**, so the rig measures
|
||
12.267 fps where the hardware would give 12.000. `clock.lua` reports both and
|
||
de-skews, and `clock_run.sh` prices the interrupt against the **hardware**
|
||
refresh count — charging the player the emulator's extra interrupts would
|
||
overstate the cost by that same 2.2%.
|
||
|
||
**Do not "fix" 55.4577 to match the measurement.** It is the hardware's, derived
|
||
from the dot clocks, and it is what the divider is built on.
|
||
|
||
### 54.6 What had to be turned off, and why it is in the file
|
||
|
||
The rigs launch the 68000 at `SR=$2700` into a machine the IPL ROM has already
|
||
booted, so the MFP arrives with whatever IOCS enabled on it and vectors pointing
|
||
into IOCS. Lowering the mask without disarming it would vector into code we did
|
||
not put there. `clk_init` writes `IERA = IERB = 0` first — which on the MC68901
|
||
clears the matching pending bits with them (`mc68901.cpp REGISTER_IERA/B`,
|
||
`m_ipr &= m_ier`) — then arms GPIP4 alone, takes the falling edge (AER bit 4
|
||
clear: the **start of vertical blanking**, which is when a player would present),
|
||
and drops to `SR=$2500`. Levels 1–5 stay masked, so the DMAC (IRQ3) and the SCC
|
||
(IRQ5) cannot get in. `VR` is written with **S clear**, so an acknowledge clears
|
||
the pending bit by itself and the handler needs no end-of-interrupt write.
|
||
|
||
The handler saves only the **low word** of `d0`, because every operation in it is
|
||
a word operation — which is legal precisely because `clk_init` proved the
|
||
accumulator fits 16 bits. `decode.s` and `frame.i` were checked for stack tricks
|
||
before the mask was lowered: the only `a7` use in either is one `move.l a1,-(sp)`
|
||
pair, so an interrupt cannot corrupt decoder state. The 120-frame self-paced
|
||
decode being pixel-exact is the test of that, and it is gated.
|
||
|
||
---
|
||
|
||
## 55. The 68000 fills its own ring, and the player's request loop costs more than the medium does (session 23)
|
||
|
||
ROADMAP P5, the last M2 item this tree could build. FINDINGS 49 and 51 measured
|
||
a ring that a HOST filled: `tools/bench/stream.lua` held the record index, chose
|
||
where every record went, wrote the descriptor and advertised it. The 68000 only
|
||
consumed. That is the same shape session 21 found in the loader and session 22
|
||
in the frame clock — a policy living outside the machine that has to run inside
|
||
it — and it was the last one in the delivery path.
|
||
|
||
`src/player/ring.i` is that policy on the 68000: `aligned` placement, the
|
||
descriptor ring, a prefill, an accumulated-slack rule and a seek. The rig keeps
|
||
only what is genuinely not the CPU's — a transport that answers one request at a
|
||
time at a modelled rate, which is what an SPC and one DMAC channel are.
|
||
|
||
### 55.1 The container had to change: DLX4 carries a record index
|
||
|
||
`aligned` asks whether the NEXT record fits before the end of the ring, which is
|
||
a question about a record's length asked **before it is fetched**. Every reader
|
||
in this tree learned record boundaries by **walking** the frame stream — reading
|
||
each record's length word to find the next — and that is exactly what a player
|
||
streaming off a disc cannot do: the length word of record *i+1* is one of the
|
||
bytes it has not fetched. A branch point needs the same table a second time, to
|
||
seek to record *j* without reading what lies between.
|
||
|
||
DLX4 adds `nframes` u16 longword-counts to the scene header, ahead of the frame
|
||
stream. Costs, measured on the gate container:
|
||
|
||
| | DLX3 | DLX4 |
|
||
|---|---:|---:|
|
||
| scene header | 5,920 B | **6,164 B** (+240 index, +4 header) |
|
||
| frame payloads | — | **byte-identical**, all 120 |
|
||
|
||
The payloads being byte-identical is asserted rather than assumed: the same
|
||
encode was written both ways and compared record for record, so **no constant
|
||
fitted to the gate container moves**. `dlx.py` cross-checks the index against
|
||
its own walk of the stream and refuses a container where they disagree, and
|
||
`prep_stream.py` checks it again against the disk image it lays out. Lengths
|
||
rather than offsets: 2 bytes a frame instead of 4, and the disc offsets are a
|
||
running sum the player builds once at scene load (`ROFF`, 4 B/record of RAM).
|
||
|
||
In 53.5's currency the 244 bytes are small — 0.5 ms of pipe at 488 KB/s — but
|
||
they are on the same side of the ledger as the 5,920 that section priced, and
|
||
the scene header is now **6,164 B that must arrive before frame 0**.
|
||
|
||
### 55.2 It reproduces the host producer's tiling exactly
|
||
|
||
Third independent implementation of `aligned`, on the gate container in a 256 KB
|
||
ring:
|
||
|
||
| producer | wraps | mean hole | pixel-exact |
|
||
|---|---:|---:|:--:|
|
||
| `19_ring_stream.py` (Python, from record sizes) | 18 | 14.7 KB | — |
|
||
| `stream.lua` (host, driving the 68000) | 18 | 14.7 KB | yes |
|
||
| **`ring.i` (the 68000 itself)** | **18** | **14.7 KB** | **yes** |
|
||
|
||
The host now **audits** rather than produces: every placement the machine makes
|
||
is checked against the host's own index and its own list of records the decoder
|
||
has not consumed, and the run is refused on the first disagreement. That is what
|
||
makes the pixel-exact result a statement about `ring.i` and not about a new rig.
|
||
|
||
### 55.3 THE ONE THAT MOVES SOMETHING: the channel is idle whenever the player is not asking
|
||
|
||
A channel only moves bytes while it has a request, and only the CPU can give it
|
||
one. Between the completion of record *i* and the issue of record *i+1* the disc
|
||
**stands still**, and the length of that gap is a property of the player's loop,
|
||
not of the medium. No host-filled run could see it — the host producer placed
|
||
records whenever it liked — so **no rate table in this tree contains it**.
|
||
|
||
Measured on the machine, same container, same 256 KB ring, same 488 KB/s, the
|
||
only difference being how many requests the player may have outstanding:
|
||
|
||
| queue | channel idle | gaps | underruns | slack ceiling | mean slack | bound by |
|
||
|---:|---:|---:|---:|---:|---:|---|
|
||
| **1 request** | **669.0 ms, 6.8%** | 119 | **59/120** | 2 | 1.0 | rate |
|
||
| **2 requests** | **317.5 ms, 3.4%** | 9 | **0/120** | 5 | 3.5 | ring |
|
||
|
||
The surplus this container has over the wire at 488 KB/s is 8.7% of the pipe,
|
||
and a one-deep request loop spends 6.8% of it on nothing. **That is most of the
|
||
surplus 51.3's lookahead is accumulated out of**, which is why the same ring at
|
||
the same rate goes from rate-bound with a ceiling of 2 to ring-bound with a
|
||
ceiling of 5 on a change with no bytes in it at all.
|
||
|
||
A second queued slot costs the 68000 nothing per frame and is available on the
|
||
hardware: the HD63450 has four channels and the IPL programs all of them (52.1).
|
||
|
||
### 55.4 Prefill is the weaker lever, and now there is a number for it
|
||
|
||
Prefill in whole records, at 488 KB/s in a 256 KB ring, every cell pixel-exact:
|
||
|
||
| prefill | 1 | 2 | 3 | 4 | 6 |
|
||
|---|---:|---:|---:|---:|---:|
|
||
| underruns, 1-deep queue | 66 | 59 | 49 | 47 | 24 |
|
||
| underruns, 2-deep queue | 1 | **0** | **0** | **0** | **0** |
|
||
|
||
**A prefill buys a one-off cushion that a rate-bound pipe spends immediately; a
|
||
queued request buys the rate back every frame.** Six records of prefill is half
|
||
a second of black screen at the start of every scene and still leaves 24
|
||
underruns; a second slot leaves none for nothing. The policy `ring_prefill`
|
||
implements is therefore small — 2 records — and the reason it is not 1 is 51.2:
|
||
*n* resident records buy *n-1* frame times, so releasing at 1 starts a scene
|
||
with a stall budget of zero.
|
||
|
||
### 55.5 The slack rule is in the player now, and so is a seek
|
||
|
||
`ring_may_seek` answers 51.2's rule as arithmetic the player can run — "resident
|
||
minus one, against the frames this branch will cost" — instead of a line in a
|
||
rig's log. `ring_seek` takes a record number, waits the channel quiet (an
|
||
outstanding transfer is bytes already on their way to an address about to be
|
||
declared free), takes the disc address out of the index, and empties the ring.
|
||
|
||
Rehearsed as a second pass over the same scene: **240 records placed, the seek
|
||
at 12.87 s, the ring refilled from empty, 0 underruns after it, and the final
|
||
frame of the second pass pixel-exact**. The seek's cost shows up exactly where
|
||
55.3 says it would — as the worst channel gap of the run, **397.5 ms** — and
|
||
that is the disc idle, not a mechanical seek, which is still unmodelled (51.7.5).
|
||
|
||
### 55.6 An independent model, and where it does and does not agree
|
||
|
||
`tools/analysis/24_ring_owner.py` is the same producer written from record
|
||
sizes and per-frame decode costs, sharing no code with the rig — the 49.4/51.5
|
||
arrangement. It reads the DLX4 index the machine reads, and it reproduces 54.4's
|
||
4-or-5-refresh cadence rather than averaging it away.
|
||
|
||
| 488 KB/s | rig Q=1 | model Q=1 | rig Q=2 | model Q=2 |
|
||
|---|---:|---:|---:|---:|
|
||
| channel idle | 6.8% | 8.8% | 3.4% | 5.3% |
|
||
| slack ceiling | 2 | 3 | 5 | 6 |
|
||
| mean slack | 1.0 | 1.1 | 3.5 | 4.2 |
|
||
| underruns | 59/120 | 11/120 | 0/120 | 0/120 |
|
||
|
||
The model runs **one record ahead** of the rig, which is the same one-record
|
||
bracket 51.5 recorded and reported rather than tuned away. The underrun count at
|
||
Q=1 is the one number that disagrees badly, and it is a threshold statistic on a
|
||
quantity sitting at 1: with a mean slack of one record, whether each individual
|
||
frame's record lands before or after its tick is decided by details neither model
|
||
has. **The agreement that matters is the resource statement** — a one-deep queue
|
||
loses 7-9% of the pipe and two-thirds of the lookahead — and on that they agree.
|
||
|
||
The model's mean decode cost, 561,126 clk/frame, lands within 0.07% of the
|
||
561,532 the ring pass measured under MAME (49.7.5), from the encoder's own
|
||
constants.
|
||
|
||
### 55.7 Three bugs and one instrument correction, recorded because they were all silent
|
||
|
||
1. **The reader's wrap rule was not the writer's.** Stepping the read cursor
|
||
past record *i* lands on the end of record *i*, which is where record *i+1*
|
||
went only if it FITTED there. Using the wrong record's length left the cursor
|
||
inside the hole, and one more retirement pushed it past the end of the ring
|
||
and wrapped it to an address unrelated to any record. The live span computed
|
||
from that is *shorter* than the truth, so the producer places over a record
|
||
the decoder has not read. Symptom: a bitstream desync, not a fault.
|
||
2. **The free-space test decided the wrap before it knew the shape.** When the
|
||
LIVE span is the one that wraps, the ring base is not free and `aligned` may
|
||
not restart there. Deciding from `WCUR + len > SZ` alone overwrote live
|
||
records. Same symptom.
|
||
3. **The queue was gated on completion instead of retirement.** A slot stays in
|
||
use until the descriptor has been read out of it, which happens one poll after
|
||
the ack at the earliest. Gating on the ack let the CPU overwrite a slot whose
|
||
descriptor had not been published; `DESC` for that frame stayed zero and the
|
||
decoder decoded address zero. This one only exists at a queue depth above 1,
|
||
and it is why the two-deep result took three attempts to obtain.
|
||
4. **The rig was capturing a torn screen.** MAME renders a screen line by line
|
||
and the machine-frame notifier fires at the END of that frame, so a bitmap for
|
||
a frame in which GVRAM changed holds lines from before and after the change.
|
||
Snapshotting it captures a tear, which reads as a pixel-exactness failure in
|
||
the bottom blocks plus a broken double-scan pairing. It only bites when the
|
||
decoder finishes its last frame late in a screen frame, so it appeared for the
|
||
first time in a run with underruns. The rig now waits one whole frame before
|
||
the capture. **No previously reported result is affected** — every one of them
|
||
finished its last frame with idle to spare — but the check.sh gates would have
|
||
been flaky under any future run that did not.
|
||
|
||
An instrument note, not a bug: under the self-clock the host's "records late"
|
||
report grades arrivals against the tick times it OBSERVED, which are up to
|
||
17.64 ms late (54.5), so it understates lateness — 6 records late where the
|
||
68000 itself counted 59 frames that had to wait. The decoder's own stall counter
|
||
is the sharp instrument. Host-paced runs take their deadlines from a host model
|
||
and are exact, so 49.6's table is unaffected.
|
||
|
||
### 55.8 What this does NOT establish
|
||
|
||
1. **The transport is still a model.** It delivers at a chosen byte rate with an
|
||
exact clock; it is not an MB89352. No command overhead, no arbitration, no
|
||
sector granularity, no mechanical seek. `W` — the clocks the DMAC steals per
|
||
delivered byte — is still undecided and still unmeasurable here (P4, 52.5).
|
||
**A zero-underrun result means the bytes were in time, not that the frames
|
||
fit.**
|
||
2. **The rates are chosen inputs.** FINDINGS 50 stands: every column is a
|
||
sensitivity, not a claim about a BlueSCSI.
|
||
3. **One container, one scene, one ring size.** The ceilings are in whole
|
||
records and move with record size (51.7.3).
|
||
4. **The seek is a rewind, not a branch.** It exercises the machinery — quiet
|
||
the channel, empty the ring, address record *j* out of the index, refill —
|
||
against a container that has one scene in it. What the worst gap between two
|
||
real decision points is still needs the scene graph (ROADMAP G1).
|
||
5. **`decode.s` and `frame.i` are unchanged** and `decode.bin` is still 1,296 B
|
||
at the same MD5. `stream.s` grew to 2,814 B: the ring producer, plus a
|
||
two-instruction test at the top of the pace wait that routes a self-filled
|
||
run into a polling wait loop. The legacy wait loops are byte for byte the ones
|
||
FINDINGS 51 measured and a host-filled run executes none of the new code.
|
||
|
||
---
|
||
|
||
# Findings — session 24 (2026-08-24)
|
||
|
||
## 56. The scene graph is in, and the worst gap between two decision points is zero (session 24)
|
||
|
||
ROADMAP G1, and it was scheduled early *because it is a measurement input*.
|
||
FINDINGS 51.3 established that a ring's lookahead is accumulated out of
|
||
`pipe - wire` and that a seek spends all of it, so what a branch point costs is
|
||
set by the rate and by **the time since the last branch**. 55.5 rehearsed a seek
|
||
on the machine and then said out loud that it could not ask the question that
|
||
matters, because nothing in this tree knew where the branch points *are*.
|
||
|
||
Now it does, in two files with a hard line between them (USER DECISION,
|
||
session 24):
|
||
|
||
- **`tools/import/scenegraph.py` is the only file in this tree that knows
|
||
anything about somebody else's source.** Their file layout, table names,
|
||
timing formulas and magic constants are wired into it and nowhere else. It
|
||
writes **`DLXSCENE1`**, this project's own schema — a clip's length, its exits
|
||
and the earliest instant each of them can fire — into gitignored `tmp/`.
|
||
- **`tools/analysis/25_scene_graph.py` reads only DLXSCENE1** and could not name
|
||
an outside project if it wanted to.
|
||
|
||
Nothing is vendored and nothing outside-derived is committed. That line is worth
|
||
drawing before the game-logic layer exists rather than after: when one of those
|
||
projects moves, exactly one file in this tree breaks, and when a scene table is
|
||
eventually committed rather than regenerated, there is one place the attribution
|
||
already lives. Restructuring changed no number in this section — the split was
|
||
made after the measurement and the whole output was re-run byte for byte.
|
||
|
||
### 56.1 What was imported, and what gates it
|
||
|
||
**40 scenes, 516 sequences, 906 input windows**, plus `scene_manager.rows` — the
|
||
13x3 scene order the arcade walks. **282 of the 516 sequences (54.7%) are
|
||
entered by a seek**; the rest play on from wherever the disc already is, which
|
||
is the distinction the whole finding rests on.
|
||
|
||
The source table is Lua, and there is no Lua interpreter on this box, so the
|
||
importer contains a **deliberately small Lua-subset parser**: table constructors,
|
||
literals, bare identifiers, the file's four timing helpers and `+`/`-` between
|
||
them. Anything else is a parse error rather than a silent skip. Three gates
|
||
stand behind that choice:
|
||
|
||
1. the four timing helpers are **matched against the text that defines them** —
|
||
`frame / 23.976`, the `- 6297.0` ROM offset, `noseek` returning `-1`,
|
||
`(seconds * 1000) + ms`. If upstream changes a formula, the tool stops
|
||
rather than keeps evaluating the old one.
|
||
2. the parse must reach **516 sequences and 906 input windows** exactly. A
|
||
parser that quietly dropped a branch would produce a *smaller* graph and a
|
||
*longer* worst gap, i.e. it would fail in the flattering direction.
|
||
3. `check.sh` runs the import and then the analysis, and **skips rather than
|
||
fails** when there is no checkout, like every other outside-this-repo input
|
||
in the tree.
|
||
|
||
### 56.2 CORRECTION to FINDINGS 16: there is only one transcription
|
||
|
||
> 16 cleared two permissively licensed transcriptions and planned to **diff them
|
||
> against each other to catch transcription errors**. That plan does not work.
|
||
|
||
The SNES project's own `data/events/README.md` states its 516 chapter XMLs are
|
||
"**derived from DirkSimple game data**". They are a *conversion* of the same
|
||
transcription, not a second one. The diff below is still worth running — it
|
||
catches conversion errors — but it **cannot** catch a transcription error,
|
||
because there is nothing independent to compare against. DirkSimple is the
|
||
single source, and its own provenance is the arcade ROM's data table.
|
||
|
||
That correction was cheap to make and it was one sentence of a README away from
|
||
never being made at all. It belongs with FINDINGS 42.1: a plausible source with
|
||
no provenance is folklore, and so is a plausible cross-check.
|
||
|
||
### 56.3 THE MEASUREMENT: the worst gap is zero, and 5.4% of branches are
|
||
|
||
Chaining play across non-seeking sequences and taking the **earliest** moment an
|
||
input window opens (the least play a clip can deliver before the branch it leads
|
||
to), over 612 distinct transitions into a seek, excluding attract mode:
|
||
|
||
| | seconds of play |
|
||
|---|---:|
|
||
| worst | **0.000** |
|
||
| p10 | 0.950 |
|
||
| p25 | 1.966 |
|
||
| median | 3.473 |
|
||
| p75 | 5.800 |
|
||
| p90 | 9.548 |
|
||
| best | 82.497 |
|
||
|
||
**33 of the 612 (5.4%) are zero**: an input window that opens at t=0 of a clip
|
||
the disc *seeked to*, so two seeks can fall back to back with no play between
|
||
them at all. `flaming_ropes.enter_room -> fall_to_death` is one — press right on
|
||
the frame the clip starts and the player dies immediately.
|
||
|
||
This is not an edge case to be designed around; it is the game. **A rule of the
|
||
form "has there been enough play since the last branch" — 51.2's slack rule,
|
||
`ring_may_seek` in `src/player/ring.i` — can be answered NO by the content,
|
||
not by the buffer**, and no amount of ring is a defence.
|
||
|
||
**203 of the 612 end the scene**, which in this design is also a container
|
||
change and needs 6,164 header bytes before frame 0 (53, 55.1). The worst
|
||
scene-change gap is **0.541 s** and the median is 2.501 s.
|
||
|
||
### 56.4 The median branch point arrives before the ring has refilled
|
||
|
||
51.3's climb, against the game's own gap distribution. Gate container
|
||
(`rc_fr_singe_scsi_span.dlx`, mean record 36.5 KB, wire 446.1 KB/s); rates are
|
||
explicit arguments and every one of them is a sensitivity, not a claim:
|
||
|
||
| ring KB | pipe KB/s | ceiling | climb s | branch points under the climb |
|
||
|---:|---:|---:|---:|---:|
|
||
| 256 | 451.4 | 3 | 20.83 | 601/612 (98%) |
|
||
| 256 | 488.0 | 7 | 6.11 | 468/612 (76%) |
|
||
| 256 | 513.2 | 7 | 3.81 | 370/612 (60%) |
|
||
| 256 | 600.0 | 7 | 1.66 | 129/612 (21%) |
|
||
| 512 | 488.0 | 11 | 9.60 | 551/612 (90%) |
|
||
| 512 | 513.2 | 14 | 7.63 | 512/612 (84%) |
|
||
| 512 | 600.0 | 14 | 3.32 | 301/612 (49%) |
|
||
|
||
Two things fall out, and the second is the one that costs something.
|
||
|
||
1. **At every rate this tree has considered, most branch points arrive with less
|
||
lookahead than the one before them.** At 488 KB/s in a 256 KB ring — the
|
||
configuration `check.sh` gates — that is 76%.
|
||
2. **A bigger ring makes this metric worse, and now content says so too.** 51.3
|
||
derived it from the surplus alone; here the same rate goes from 76% under the
|
||
climb at 256 KB to 90% at 512 KB, because doubling the ring doubles the
|
||
ceiling without touching `pipe - wire`. **The ring is not the lever. The
|
||
surplus is.**
|
||
|
||
### 56.5 What a branch actually costs when the gap bought nothing
|
||
|
||
The climb is what a player needs to *tolerate* the next branch. What it *pays*
|
||
at one is the prefill, because the ring is empty after a seek and 55.4's shipped
|
||
policy releases the decoder at 2 records:
|
||
|
||
| pipe KB/s | 2-record prefill | as a scene change (+6,164 B) |
|
||
|---:|---:|---:|
|
||
| 451.4 | 161.8 ms (1.94 frame slots) | 175.2 ms (2.10) |
|
||
| 488.0 | 149.7 ms (1.80) | 162.0 ms (1.94) |
|
||
|
||
So a zero-play branch is **not** a failure: it costs about two frame slots of
|
||
black, every time. What it removes is margin. A player that branches at p10
|
||
(0.950 s) has spent its whole lookahead and rebuilt almost none of it, and the
|
||
next slow record has nothing behind it. **The finding is not "this breaks", it
|
||
is "this design runs permanently at minimum lookahead, and the arcade content is
|
||
what puts it there."** 22_scene_load.py prices the scene-change case properly,
|
||
clocks included; the mechanical seek is still unmodelled (B1) and is charged on
|
||
top of all of it.
|
||
|
||
### 56.6 What the cross-check IS worth, now that it is not a cross-check
|
||
|
||
Run anyway, against 518 SNES chapter XMLs, with the 40 scene abbreviations
|
||
matched to DirkSimple scenes **from the data** — the abbreviation's letters must
|
||
be a subsequence of the scene name, ranked by sequence-name overlap, with a
|
||
reversed-scene rule for the thirteen mirrored scenes. (Overlap alone mapped
|
||
`snkr` to `black_knight`, because two scenes full of `seqN` names look alike.)
|
||
|
||
- **Start times do not compare at all.** The SNES XMLs are on their own
|
||
extraction's timeline; DirkSimple is on arcade ROM frames minus 6,297 ms. The
|
||
difference is not a constant and **does not even keep its sign** (spread
|
||
-8.0 s .. +6.4 s). Reported as a spread rather than as a check, because a
|
||
check it is not.
|
||
- **Durations do compare** — they are offset-invariant. 388/505 agree within one
|
||
laserdisc frame (76.8%), median difference 0 ms.
|
||
- **Branch structure compares**: 470/505 chapters (93.1%) carry the identical
|
||
set of (input -> target) edges. Of the 35 that differ, **16 are renames**
|
||
(`captured_by_ghouls` -> `ghoul_capture`) and **18 of the remaining 19 are the
|
||
SNES conversion dropping the arcade's diagonals**. The 19th is
|
||
`intr_castle_exterior`, where the SNES *added* a skip.
|
||
|
||
**Zero transcription discrepancies were found, and none could have been.** What
|
||
the diff produced is one useful fact about our own input layer, below.
|
||
|
||
### 56.7 Two constraints on the input layer, from the same import
|
||
|
||
1. **The arcade uses eight directions plus action and start.** Counted over the
|
||
906 windows: left 233, right 217, up 209, down 152, action 72, **upleft 13,
|
||
upright 4, downleft 3, downright 1**, start 2. The diagonals are 21 windows
|
||
out of 906 — rare enough to be dropped by a port that had to (the SNES one
|
||
did) and **not** droppable by one aiming at the arcade.
|
||
2. **The shortest input window is 98 ms.** Median 950 ms (11.4 frame slots), p10
|
||
393 ms, but the floor is 98 ms — and by 54.4 a 12 fps frame slot is 72.13 ms
|
||
or 90.16 ms, never 83.33. **A 98 ms window is one or two frames wide.**
|
||
Polling input on the frame tick is therefore marginal by construction: the
|
||
input layer has to run off something faster than the frame clock, and
|
||
`src/player/clock.i` already owns an MFP interrupt at 181.35 clocks a
|
||
V-DISP (54.2) that is 8.6x faster and costs 0.1% of the budget.
|
||
|
||
### 56.8 What this does NOT establish
|
||
|
||
1. **No rate here is measured.** Every column is a sensitivity across explicit
|
||
rates (FINDINGS 50). B1 is still open and the mechanical seek still has no
|
||
figure at all.
|
||
2. **The gap model is a lower bound by construction.** It takes the earliest
|
||
instant an input window opens, so it is what an *expert* player can force,
|
||
not what a typical one produces. That is the right bound for a buffer
|
||
design and the wrong one for describing play.
|
||
3. **It is the arcade's graph, not this port's.** The 612 transitions assume
|
||
the port reproduces every branch. Nothing has been mapped onto our 224
|
||
Blu-ray streams yet — the SNES project's 516 chapters are finer-grained than
|
||
our streams, and that mapping is still C1's problem.
|
||
4. **Nothing ran on the 68000 this session.** This is host-side analysis of an
|
||
imported table. `decode.s`, `stream.s`, `ring.i`, `clock.i` and `load.i` are
|
||
untouched, `decode.bin` is still 1,296 B at the same MD5, and the green light
|
||
was ALL GREEN before and after.
|
||
5. **The scene graph is not vendored, and the coupling is contained.** Neither
|
||
repo ships here and neither is redistributable from this tree; both are
|
||
permissive (DirkSimple zlib, Ryan C. Gordon; SNES project MIT, Chad
|
||
Doebelin) and both are cloned by the reader. The `sources` block of every
|
||
generated table carries the attribution. **`tmp/scenegraph.json` is
|
||
generated, gitignored and derived data**: committing it, or any table built
|
||
from it, is redistribution and the attribution has to travel with it.
|
||
|
||
---
|
||
|
||
## FINDINGS 57 — the 68000 reads the disc itself, and P4 was never blocked
|
||
|
||
Session 25. ROADMAP P4, first half. Green light ALL GREEN before and after.
|
||
**Emulated** — MAME 0.277, `x68000 -exp1 cz6bs1`. No real hardware.
|
||
|
||
### 57.1 The blocker was a missing FILE, not a missing MODEL, and the tree already knew
|
||
Session 21 recorded P4 as "blocked in this tree", re-checked rather than
|
||
assumed, on three grounds. One of them is wrong:
|
||
|
||
> there is still no `scsiexrom.bin` anywhere on this machine ... **MAME's
|
||
> `x68000` has no MB89352 path**, and `hd63450.cpp` decodes no DTYP.
|
||
|
||
MAME 0.277's `x68000` **does** have one. `-listslots` offers `cz6bs1` on `exp1`
|
||
and `exp2`; `-listdevices x68000 -exp1 cz6bs1` shows a *Fujitsu MB89352 SCSI
|
||
controller @ 5.00 MHz* on a SCSI bus with a hard disk at ID 0, alongside the
|
||
HD63450. **FINDINGS 32.4 had already established this in session 9** — the
|
||
CZ-6BS1's DMA glue, `$EA0000`, the data register at `$EA0015` — and 42.5 built
|
||
on it in session 14. The session-21 note is a regression in the record, not a
|
||
discovery.
|
||
|
||
What is genuinely missing is only the 8 KB `scsiexrom.bin` (CRC `7be488de`),
|
||
which MAME **requires** to instantiate the card: without it the machine refuses
|
||
to start at all. **That is the entire blocker, and it is not one**, because the
|
||
player drives the SPC registers directly and never executes that ROM — which was
|
||
already the plan in `docs/BENCHMARK.md` item 4, written in session 2, long
|
||
before the file turned out to be absent. `tools/bench/scsi_run.sh` supplies a
|
||
zero-filled placeholder on its own rompath, leaves the user's romset untouched,
|
||
and lets MAME print WRONG CHECKSUMS as it should.
|
||
|
||
**The substitution is honest here and would not be everywhere.** Anything that
|
||
boots from the card, or calls SCSI IOCS, *does* execute that ROM. Do not reuse
|
||
the rompath for those. **B3 is untouched**: it wants the ROM's bytes
|
||
disassembled for the DCR it writes, and a blank one has none.
|
||
|
||
### 57.2 The register map, measured rather than inferred
|
||
32.4 quoted one address. `src/player/scsigate.s` probes `$EA0000..$EA003F` one
|
||
address at a time, with a bus-error handler that records the fault, steps the
|
||
index and re-enters the loop — so a hole costs an entry in the map rather than
|
||
the rest of the run. **60 of 64 addresses answer.**
|
||
|
||
| | |
|
||
|---|---|
|
||
| registers | odd bytes, `$EA0001 + 2n`, `n = 0..14` |
|
||
| `$EA0007` (n=3, TMOD) | **BUS ERROR** |
|
||
| `$EA001F` (n=15, EXBF) | **BUS ERROR** |
|
||
| `$EA0017` (n=11, TEMP) | wrote `$A5`, read back `$A5` |
|
||
| `$EA0015` (n=10, DREG) | as 32.4 said |
|
||
|
||
The two holes are exactly the two registers the **MB89352 omits and the MB87030
|
||
has**, which independently confirms which part MAME is modelling. It also
|
||
**corrects the device's own documentation**: the summary of `mb87030.cpp` says
|
||
the MB89351/352 "skip TMOD and EXBF, shifting subsequent indices accordingly",
|
||
and the machine says MAME leaves *holes* and shifts nothing — which is what
|
||
keeps DREG at index 10 and at `$EA0015`. The bytes win. The first version of
|
||
this probe walked upward with `move.b (a0)+`, took a bus error at `$EA0006`, and
|
||
knew one address was dead and nothing about the other 57; **a sequential dump
|
||
reports the first hole as the answer**.
|
||
|
||
### 57.3 The data register is DMA-only on this card, and a PIO write vanishes
|
||
`x68k_scsiext.cpp` installs its own handler on `$EA0015` **and on no other
|
||
address**:
|
||
|
||
write: if (exown()) { if (!drq) dtack_w(1); else dma_w(data); }
|
||
else dreg_w(data);
|
||
|
||
On this machine `exown()` — the HD63450's OWN, fed back to the slot by
|
||
`x68k.cpp` — is asserted where a PIO write needs it not to be. The `else` arm is
|
||
unreachable, and **a byte written to `$EA0015` with the SPC in PROGRAM transfer
|
||
mode is discarded silently**: no error bit, no status change, no interrupt.
|
||
Quieting all four DMAC channels (`CCR = 0`, `CSR = $FF`) does not change it.
|
||
|
||
It was **measured, not reasoned about** — the gate writes `$5A` to `$EA0015` and
|
||
reads it straight back, and gets `$00` with the FIFO still empty — because ten
|
||
command bytes vanishing without trace looks exactly like a target refusing a
|
||
command, and that is how it first presented.
|
||
|
||
So every transfer issues `SCMD` **without** the PROGRAM bit, which puts the SPC
|
||
in DMA mode and makes it raise DRQ, and the CPU then moves the bytes through
|
||
`$EA0015` itself, in via `dma_w` and out via `dma_r`. **The CPU stands in for
|
||
the DMAC, through the DMAC's own door.**
|
||
|
||
**What that costs the argument, stated because it is easy to overclaim.** With
|
||
`exown` asserted at idle, MAME **cannot distinguish a CPU-driven byte at
|
||
`$EA0015` from a DMAC-driven one**. This rig therefore demonstrates the DATA
|
||
PATH and cannot, on its own, demonstrate that the HD63450 is the thing driving
|
||
it — which is precisely what ROADMAP calls P4's first job. Whether a real
|
||
CZ-6BS1 also refuses PIO here is **not settled**: it is a property of MAME's
|
||
model and it wants a board.
|
||
|
||
### 57.4 The result
|
||
`src/player/scsi.i`, 68000 code, no IOCS and no host in the transfer path:
|
||
selects the target, and issues **READ(10)** twice.
|
||
|
||
READ(10) OK: 4096 B from LBA 0 match the host's image byte for byte
|
||
READ(10) OK: 2048 B from LBA 1000 match too
|
||
|
||
The second one is the half that matters: **a driver that emits a malformed LBA
|
||
field still passes LBA 0**, because zero is what a malformed field usually is.
|
||
|
||
The volume is `tmp/stream_disk.bin` — the *same* file `prep_stream.py` already
|
||
writes for the ring rig — so the SCSI volume and the host-file pipe carry
|
||
byte-identical bytes, and a difference between the two rigs cannot be a
|
||
difference in what they are reading.
|
||
|
||
### 57.5 Five bugs, and four of them were silent
|
||
Recorded because the pattern is the finding: **nothing in a SCSI bring-up tells
|
||
you what you did wrong.** Every one of these presented as a phase that never
|
||
arrived.
|
||
|
||
1. **`sc_settc` wrote the wrong three bytes.** Three chained `rol.l #8` put the
|
||
original bits 31..24, 23..16 and 15..8 into TCH/TCM/TCL, so a count of 10
|
||
loaded a transfer counter of **zero**. MAME completed the TRANSFER instantly
|
||
and silently, and the bus sat in command phase. Now written low-byte-first
|
||
with `lsr`. The same trap was live in the CDB's block-count field.
|
||
2. **A byte handed to a FIFO is not a byte on the bus.** The driver returned
|
||
from the command phase with the last byte still in the SPC, asked what phase
|
||
the bus was in, and got COMMAND — which reads as a target refusing the
|
||
command. Fixed by waiting for `XFER IN PROGRESS` to clear.
|
||
3. **A fixed phase sequence is wrong.** The first version ran select → command →
|
||
data → status → message; the target came up in MESSAGE OUT with ATN asserted
|
||
and the driver called it an unexpected phase. **The bus decides the order.**
|
||
`scsi_read` is now a phase loop, which is both shorter and correct.
|
||
4. **The discarded PIO write** of 57.3.
|
||
5. **The initiator must let go of the bus, in two steps.** After the final
|
||
message byte the SPC still holds ACK — `PSNS` reads `$4F`, REQ low and ACK
|
||
high — and a target cannot drop BSY into that. It needs `SCMD` reset-ACK/REQ
|
||
**and then** bus release. **This one only appeared once there were two
|
||
reads**: one read passed byte-exact and every conclusion drawn from it was
|
||
sound, and the second could not select. A player issues one command per
|
||
record, so the failure would have been universal in the ring and invisible in
|
||
the demonstration.
|
||
|
||
### 57.6 What this does and does not move
|
||
**Does not move `W`.** Not by one clock. MAME's device models are functional,
|
||
not transfer-timing accurate (`docs/BENCHMARK.md`), and 42.5 reads its DMAC
|
||
configured in wall-clock attotimes rather than per-operand cycles. `W` remains
|
||
the project's largest open number and still wants a board.
|
||
|
||
**Does not finish P4.** What is done is the correctness half — BENCHMARK's Tier
|
||
1, "does our read path work at all". What is left is the half ROADMAP calls
|
||
P4's first job: **a DMAC configuration that holds the bus**, and then the driver
|
||
behind `ring.i`'s `XF_*` mailbox in place of `stream.lua`'s modelled transport,
|
||
gated on the same pixel-exact 120 frames. 57.3 is a warning about the first of
|
||
those: this apparatus cannot tell a DMAC-driven byte from a CPU-driven one at
|
||
`$EA0015`, so "the DMAC held the bus" will need evidence that does not come from
|
||
watching that address.
|
||
|
||
**Does move the premise of every delivery rig in the tree.** Until now the
|
||
bytes came from a host. They now come off a disc, on the machine's own
|
||
instructions, byte-exact at two different LBAs.
|
||
|
||
---
|
||
|
||
## FINDINGS 58 — the player runs off the disc, and PIO costs 87 clocks a byte (session 25b/26)
|
||
|
||
**Emulated.** MAME 0.277, `x68000 -exp1 cz6bs1 -ramsize 2M`, a blank
|
||
`scsiexrom.bin` on a private rompath (57.1's substitution, unchanged). No real
|
||
hardware ran. `./tools/bench/check.sh` was ALL GREEN before this and ALL GREEN
|
||
after, with two new stages.
|
||
|
||
**ROADMAP P4b is DONE. P4a is not, and 58.2 is why it is now the item that
|
||
decides the project rather than one of two that do.**
|
||
|
||
### 58.1 The seam closed: 120 pixel-exact frames, off a real volume
|
||
|
||
`src/player/xfer.i` sits behind `src/player/ring.i`'s `XF_*` mailbox in place of
|
||
`tools/bench/stream.lua`'s modelled transport. `XF_GO` is answered by a real
|
||
READ(10) to a real MB89352 and `XF_ACK` is a word the 68000 bumps when the bytes
|
||
have landed — not one a host synthesises from emulated time.
|
||
|
||
| | |
|
||
|---|---|
|
||
| records fetched by the 68000 | **120**, one READ(10) each |
|
||
| bytes into the ring | **4,488,588**, and the decode is **pixel-exact** |
|
||
| bytes off the disc | **4,548,608** — see 58.3 |
|
||
| ring | 256 KB, **18 wraps, 14.7 KB mean hole** |
|
||
| a real mid-stream seek | **pass 2 pixel-exact**, ring thrown away and rebuilt |
|
||
|
||
**The 18 wraps are the load-bearing assertion, not the byte count.** They are
|
||
the same 18 the host producer produced in 49.4 and the same 18 `ring.i`
|
||
produced against a modelled transport in 55.4 — a *third* transport, same
|
||
tiling. `ring.i` is not supposed to be able to tell which side of the mailbox
|
||
answered it, and this is the number that says it could not.
|
||
|
||
**The change above the seam is two `bsr`s.** One in `ring_poll`, one in
|
||
`ring_seek`'s quiet-wait — and the second is not optional. With the transport
|
||
*inside* the machine, the only thing that can retire an outstanding request is
|
||
that wait loop itself, so a seek issued with a request in flight spins forever
|
||
without it. A host transport retired it on its own time. That is exactly the
|
||
kind of difference the seam exists to hide, and it is the one it could not.
|
||
|
||
### 58.2 What it costs: 87.28 clocks per delivered byte, and the number is portable
|
||
|
||
`tools/bench/xfer_cost.sh` decodes the same 120 frames twice — same ring, same
|
||
`stream.s`, same `ring.i` placing every record — and changes only which side of
|
||
the mailbox answers:
|
||
|
||
| | emulated | per frame |
|
||
|---|---|---|
|
||
| decode + `ring_poll` alone | 6.7737 s | **67.7%** of a 12 fps frame |
|
||
| ...with the real transport | 45.9516 s | **459.5%** |
|
||
| **the transport** | **391,779,000 clk** | **391.8%** |
|
||
|
||
**= 87.28 clocks per delivered byte**, 86.13 per byte off the FIFO.
|
||
|
||
**AND IT IS NOT MAME'S NUMBER, WHICH IS THE HALF THAT MATTERS.** The keep loop
|
||
in `src/player/scsi.i`, priced against the 68000's own cycle table — 12 patience
|
||
reload, 16 SSTS read, 10 `btst`, 10 branch, 20 DREG read to `(a1)+`, 8 `subq`,
|
||
10 branch — is **86 clocks**, and the FIFO also carries the dropped window bytes
|
||
of 58.3, which makes it **87.15 per delivered byte**. Measured 87.28. **0.2%
|
||
apart.** So the cost is the instruction stream and not a wait on MAME's SPC
|
||
model: it is a figure a real board would also pay, and it is the first number
|
||
this rig has produced that survives leaving the emulator. The residual, +0.13
|
||
clk/B = 4,989 clocks per record, is the per-*command* cost — select, CDB,
|
||
status, message, `xf_service` — and it is the part that does not scale with the
|
||
record.
|
||
|
||
**Against the ladder, in the same units** (clocks charged to the CPU per
|
||
delivered byte, at this container's 37,405 B mean record):
|
||
|
||
| | | share of a 12 fps frame |
|
||
|---|---|---|
|
||
| `W` = 5 | single address, bus HELD | 22.4% |
|
||
| `W` = 9 | dual address, held | 40.4% |
|
||
| `W` = 12 | single address, arbitrated | 53.9% |
|
||
| `W` = 19 | dual address, arbitrated — the IPL ROM's own disk channel (52.5) | 85.3% |
|
||
| **PIO 87** | **this rig, measured** | **391.8%** |
|
||
|
||
**The PIO transport is 4.6x the worst DMA configuration this project has found
|
||
and 17.5x the best.** P4a is not an optimisation of this. It is the difference
|
||
between a player and a slideshow, and it is now the *only* thing between the
|
||
tree and M2.
|
||
|
||
**The player's own clock says the same thing, independently.** Self-paced off
|
||
V-DISP, the machine decoded 120 frames in **560 slots** of a 12 fps clock:
|
||
**2.57 fps**. That agrees with 12 / 4.595 = 2.61 from the cycle accounting
|
||
above, from a completely different instrument.
|
||
|
||
**AND "UNDERRUNS: 0/120" IS VACUOUS IN THIS RUN.** A synchronous transport
|
||
cannot underrun by construction — a frame cannot start before its record has
|
||
landed, because the decoder *is* the transport. The counter that means anything
|
||
here is `NO IDLE`: **119 of 120 frames found their slot already open, worst
|
||
overrun 441 whole ticks.** `tools/bench/stream.lua` now prints that argument
|
||
next to the zero rather than leaving the zero to be quoted. This is the same
|
||
class of error as 49.7.2's free-running ring passing at 48 KB: a rig
|
||
configuration in which the failure it tests for cannot occur.
|
||
|
||
**The resource that binds also flipped, and it is worth naming.** Against a
|
||
modelled 488 KB/s pipe this container was rate-bound and the ring never filled
|
||
(55.4). Here the ring **fills — 90 refusals for space** — while the decoder
|
||
starves. Not of bytes: of *time*. Every previous delivery result in this tree
|
||
was measured on a rig where the transport cost the CPU nothing.
|
||
|
||
### 58.3 A record is not a sector, and the cheapest fix is a re-encode
|
||
|
||
`ring.i` asks for a byte offset and a length, both 4-byte aligned (28.3);
|
||
a target answers in 512 B blocks. **On the gate container 117 of 120 records
|
||
start part way into a sector.** This is not a rounding nuisance: the bytes on
|
||
either side of a record belong to *other* records the decoder may still be
|
||
reading, and the block loop walks `a0` with no bounds check (49.2), so a
|
||
transport that reads whole sectors straight into the ring **corrupts its
|
||
neighbours** — wrong pixels, not a fault.
|
||
|
||
`tools/analysis/26_sector_align.py` prices the three ways out:
|
||
|
||
| | wire | clocks |
|
||
|---|---|---|
|
||
| **A. windowed PIO** — read the covering sectors, store only the record | **+1.34%** | 86/B on every byte off the FIFO, and **a DMAC cannot do it at all** |
|
||
| **B. bounce buffer** — DMA whole sectors elsewhere, then copy | +1.34% | **+5/B on every delivered byte** = 22.4% of the frame, on top of `W` |
|
||
| **C. sector-aligned records** — pad to 512 in the container | **+0.43%** | **zero** |
|
||
|
||
A is what shipped in `scsi.i` and what 58.1 ran, and it is free *only* because
|
||
the CPU is already touching every byte — the property that disappears the moment
|
||
P4a succeeds. B is the cost `aligned` was chosen over `split` to avoid (49.3)
|
||
arriving by a different door, and on every byte instead of on a wrap.
|
||
|
||
**C wins on both axes**: it is cheaper on the wire than A and B by 0.91 points
|
||
of the payload (40,940 B on this scene), and it is the only one of the three a
|
||
DMA channel can run without a copy. What it costs is a **container revision** —
|
||
a re-encode plus a re-measurement of every constant fitted to the gate
|
||
container. That is the class of change ROADMAP already has bundled with P2's
|
||
other half (reserve index 0 as black) and 55's two open re-encode questions.
|
||
**It should join that bundle**, and P4a should be attempted against a
|
||
sector-aligned container rather than against this one.
|
||
|
||
It also grows the largest record from 40,984 to 41,472 B, which a 256 KB ring
|
||
still holds six times over — so it costs nothing in ring size.
|
||
|
||
### 58.4 What this does and does not move
|
||
|
||
**Does not move `W`.** Again, and for the same reason 57.6 gives. Nothing here
|
||
programmed a DMAC channel; `xf_service` is the CPU standing in for one, through
|
||
the DMAC's own door (57.3).
|
||
|
||
**Does not measure a delivery rate,** and the rig now refuses to be asked. A
|
||
`DLX_XFER=scsi` run rejects a `--kbps` argument outright rather than ignoring
|
||
it, and `stream.lua` suppresses `CHANNEL IDLE`, `DEADLINE` and `REQUIRED
|
||
PREFILL` instead of printing them as zeros — a zero there reads as "the channel
|
||
never stopped", which would be a claim about a medium this tree has never timed.
|
||
|
||
**Does close P4b, and does hand P4a a sharper question than it had.** Before
|
||
this session, "get the DMAC to hold the bus" was worth 9 clk/B against 19. It is
|
||
now worth 87 against either.
|
||
|
||
**Does put a real number on the layer below every previous delivery result.**
|
||
Everything in FINDINGS 49, 51 and 55 was measured with the bytes arriving free.
|
||
They are not free, and 58.2 is the first measurement of what they cost.
|
||
|
||
---
|
||
|
||
## FINDINGS 59 — the DMAC drives the data phase, and auto-request is charged by time (session 27)
|
||
|
||
**Emulated.** MAME 0.277, `x68000 -exp1 cz6bs1 -ramsize 2M`, a blank
|
||
`scsiexrom.bin` on a private rompath (57.1's substitution, unchanged). No real
|
||
hardware ran. `./tools/bench/check.sh` was ALL GREEN before this and ALL GREEN
|
||
after, with one new stage.
|
||
|
||
**ROADMAP P4a is DONE at the transport level.** What is left before M2 is not a
|
||
DMAC question any more; it is the re-encode bundle, because 59.4 makes
|
||
sector-aligned records a precondition the transport now enforces rather than a
|
||
preference the roadmap recorded.
|
||
|
||
### 59.1 The channel drives the data phase, and the evidence is the CPU's own progress
|
||
|
||
`src/player/dma.i` programs HD63450 channel 1 and hands it the DATA IN phase;
|
||
`src/player/dmagate.s` reads **the same 2,048 B at LBA 1000 three ways** and the
|
||
host compares all three against its own copy of the image:
|
||
|
||
| | bytes | MTC one instruction after START | CPU trips round its wait loop |
|
||
|---|---|---|---|
|
||
| PIO, the path 58.2 measured | byte-exact | — | — |
|
||
| **DMA, bus HELD** (DCR `$00`, OCR `$81`) | **byte-exact** | **0 of 2048** | **1** |
|
||
| **DMA, cycle STEALING** (DCR `$80`, OCR `$80`) | **byte-exact** | 2048 of 2048 | 426 |
|
||
|
||
Both channels reported `CSR = $E0` (COC, BTC, NDT), `CER = $00`, `MTC = 0` and a
|
||
memory address exactly `+2048` from where it started.
|
||
|
||
**THE DISCRIMINATOR NEVER READS `$EA0015`, and that is the whole design.** 57.3
|
||
established that watching the data register cannot answer this question: with
|
||
the DMAC's `OWN` asserted — which it is at idle on this machine — MAME cannot
|
||
tell a CPU-driven byte there from a DMAC-driven one. So what separates the two
|
||
configurations is **whether the 68000 executed anything while the bytes were
|
||
arriving**:
|
||
|
||
move.b #CCR_START,DM_CCR ; the channel is told to go
|
||
move.w DM_MTC,d0 ; <- sampled by the VERY NEXT instruction
|
||
|
||
Held, `d0` is **zero**: the entire 2,048-byte transfer happened between two
|
||
instructions, because the CPU did not run in between. Stealing, `d0` is the full
|
||
count and the CPU then goes round its own loop 426 times while the bytes
|
||
trickle in. That is what "holds the bus" means, and it is a fact about the CPU
|
||
rather than about the data register.
|
||
|
||
**The mechanism, named so the claim is not over-read.** MAME models a held bus
|
||
by asserting `INPUT_LINE_HALT` for burst + max-rate and clearing it at
|
||
end-of-transfer (`hd63450.cpp`). It is not inventing that semantic: MC68450
|
||
§5.2.3.3.1 says of maximum-rate auto-request that "all operands in the data
|
||
block will be transferred in one burst, so that the DMAC will use **100% of the
|
||
available bus bandwidth**". The model and the datasheet agree about what this
|
||
configuration does to the CPU. **They do not agree about anything per-operand,
|
||
and no `W` is claimed here** (42.5: MAME's DMAC runs on wall-clock attotimes).
|
||
|
||
**The gate was checked against its own negative.** With the *stealing* register
|
||
pair put in the *held* slot, the run still delivers all 2,048 bytes byte-exact
|
||
— and `tools/bench/dma_run.sh` goes **red**, on the trip count and on the MTC
|
||
sample. A counter that cannot come out different is 58.3's vacuous
|
||
"UNDERRUNS: 0/120" again; this one can, and was made to.
|
||
|
||
`tools/analysis/27_dmac_config.py` decodes the four register bytes out of
|
||
`src/player/dma.i` itself, with the same MC68450 field tables
|
||
`21_iplrom_dmac.py` reads the IPL ROM with (now one copy, `mc68450.py`). So
|
||
"dual address, 8-bit port, burst, auto-request at max rate" is a decode of the
|
||
bytes the player programs, not a comment next to them — and it is directly
|
||
comparable with Sharp's own disk channel, which 52.5 read as `DCR $80 / OCR $B2`
|
||
and priced at 16..19 clk/B.
|
||
|
||
### 59.2 Three things this model cannot be asked, and they bound the result
|
||
|
||
Read out of MAME 0.277's source rather than inferred from behaviour:
|
||
|
||
1. **The card has no request line to the DMAC.** `x68k_scsiext.cpp`'s `drq_w`
|
||
only stores a flag; the expansion slot carries no request to the HD63450 at
|
||
all (`x68k.cpp` wires `drq0` from the FDC and `drq3` from ADPCM, and nothing
|
||
else). The card's flow control is **DTACK**: on a DMAC cycle with DRQ low it
|
||
negates DTACK and the channel discards that operand and retries. So **REQG =
|
||
10, external request — the mode the ladder's `W`=5 and `W`=12 rows assume —
|
||
cannot be run here at all.**
|
||
2. **Single address cannot be run either.** `hd63450.cpp` takes the implicit
|
||
path only for a channel with a device callback, and on this machine only
|
||
channel 0 (the FDC) has one. `DTYP` = 10/11 on channels 1..3 falls through to
|
||
the dual-address code.
|
||
3. **Only burst is modelled as held.** The device tests `(dcr & 0xc0) == 0`, so
|
||
XRM = 10 (cycle steal *without* hold) and XRM = 11 (cycle steal **with**
|
||
hold) are one code path.
|
||
|
||
**So of the four rows of the per-byte ladder, exactly one — dual address, bus
|
||
held, 9 clk/B — has a code path in this model, and it is the one demonstrated.**
|
||
That is a bound on the apparatus, not a result about the board. The slot's own
|
||
pinout has `#EXREQ` at B36, so a real CZ-6BS1 plausibly drives it; whether it
|
||
does is ROADMAP B3, and it is now a sharper question than "which DTYP".
|
||
|
||
### 59.3 Auto-request is charged by TIME, not by byte — and the GCR is the lever
|
||
|
||
This is the finding that outlives the emulator. Every `W` in this project is
|
||
**clocks per delivered byte**, which presumes the device asks for each one. An
|
||
auto-requested channel does not know whether the device is ready: it takes the
|
||
share of the bus it was told to take and spends it either way. So the cost of a
|
||
record scales with **how long the record takes to arrive** — halve the delivery
|
||
rate and the CPU cost of the same record **doubles**. No `W` does that.
|
||
|
||
`tools/analysis/28_autorequest_cost.py` prices it from MC68450 §3.8 and
|
||
§5.2.3.3.2, and gates its formulas against Table 5-3's sixteen printed rows
|
||
before printing anything. At the gate container's 37,405 B mean record and
|
||
**460 KB/s, an explicit rate and not a measurement** (FINDINGS 50):
|
||
|
||
| configuration | sustains | charged to the 68000 | % of a 12 fps frame |
|
||
|---|---|---|---|
|
||
| REQG 01, max rate — what 59.1 demonstrated | the wire | 21.23 clk/B | **95.3%** |
|
||
| REQG 00, LRAR, BR = 00, 50% of the bus | 534 KB/s | 10.61 clk/B | **47.6%** |
|
||
| REQG 00, LRAR, BR = 01, 25% | 267 KB/s | 5.31 | 23.8% — **does not carry the rate** |
|
||
| REQG 00, LRAR, BR = 10, 12.5% | 133 KB/s | 2.65 | 11.9% — **does not carry the rate** |
|
||
| *(ladder, for comparison)* `W`=9 dual held | — | 9 | 40.4% |
|
||
| *(measured, 58.2)* PIO | — | 87.28 | 391.8% |
|
||
|
||
**The held configuration is the cheapest per byte MOVED and the dearest per byte
|
||
DELIVERED**, and the gap between those two is the device's own slowness: 9
|
||
clocks of DMAC work inside 21.2 clocks of waiting means **42.4% of the held bus
|
||
does anything at all**. Holding the bus is only cheap when the hold is ended by
|
||
the device, which is what external request is for.
|
||
|
||
**BT and BR are two bits each and they set what fraction of the bus the player
|
||
gives away** (burst time `2^(BT+4)` clocks, sample period `2^(BT+BR+5)`, share
|
||
`2^-(BR+1)`). Nothing in this tree had named the GCR as a design choice; it is
|
||
the same kind of lever as `aligned` vs `split` and it belongs in the same list.
|
||
**At 460 KB/s only BR = 00 carries the rate**, so the fallback plan — if a real
|
||
card turns out not to drive `#EXREQ` — is 50% of the bus for the duration of
|
||
every record, or 47.6% of a frame slot per record. That is affordable and it is
|
||
not free, and it is the first cost model in this project that gets *worse* when
|
||
the disc gets slower.
|
||
|
||
**59.3's one load-bearing assumption**, stated because the whole table rests on
|
||
it: that the channel *spends* its allotted share whether or not the device has a
|
||
byte. Under auto-request a request is pending until MTC is exhausted, so the
|
||
DMAC takes the bus during every window it is entitled to; when the device is not
|
||
ready the cycle is stretched by wait states (a real card negating DTACK) or
|
||
retried later (MAME's model discards the operand), and either way the window is
|
||
gone from the CPU's point of view. **If a real CZ-6BS1 instead lets the DMAC off
|
||
the bus early when no byte is there, 59.3's figures are upper bounds.** That is
|
||
a board question, and it is B3's.
|
||
|
||
### 59.4 The window is refused, and that makes the re-encode a precondition
|
||
|
||
58.3 found that 117 of 120 records start part way into a sector, and that PIO
|
||
absorbs it for free because the CPU is already touching every byte. A channel
|
||
cannot: it writes a contiguous run and cannot be told to drop the 300 bytes in
|
||
front of the record. `sc_in_data` now **refuses** a windowed read when the data
|
||
phase is the DMAC's — a new error, `SCE_WINDOW` — rather than quietly delivering
|
||
the neighbouring records' bytes into the ring, where the block loop has no
|
||
bounds check to catch them (49.2). The gate asserts the refusal.
|
||
|
||
**So "sector-aligned records" has stopped being a preference in ROADMAP's
|
||
re-encode bundle and become the thing standing between P4a and the ring.** The
|
||
transport states its own precondition; the container does not meet it yet.
|
||
|
||
### 59.5 What this does and does not move
|
||
|
||
**Does not move `W`.** Not by one clock, for the third session running, and for
|
||
the reason 57.6 and 58.4 give.
|
||
|
||
**Does not put the DMAC behind `ring.i`'s mailbox.** 59.4 is why: `xfer.i` asks
|
||
for records, and every record but three needs a window. That work is now
|
||
downstream of the re-encode bundle rather than of a DMAC question.
|
||
|
||
**Does close the question ROADMAP called P4's first job.** A configuration that
|
||
holds the bus exists, runs, delivers the disc's bytes byte-exact, and is
|
||
demonstrated by evidence that does not come from watching `$EA0015` — which is
|
||
exactly what 57.3 said would be needed.
|
||
|
||
**Does change what the fallback looks like.** Before this session the fallback
|
||
below a held bus was `W` = 16..19, the IPL ROM's own arbitrated configuration.
|
||
It is now limited-rate auto-request at a share the player chooses, priced in a
|
||
currency the project did not have, and the arithmetic says a 50% share carries
|
||
this container at 460 KB/s.
|
||
|
||
### 59.6 One collision, and it was caught by the half of `check.sh` that runs first
|
||
|
||
`DM_USE` — the word that tells `sc_in_data` whether the data phase is the
|
||
DMAC's — was first placed at `$18300`. `scsi.i`'s trace ends at `$182FF` and the
|
||
next 160 bytes are the **ring's**: `$18300` is `ring.i`'s `XF_SLOT` mailbox, and
|
||
`tools/bench/stream.lua` reads the same addresses from outside the machine. So
|
||
the ring rig's first record request wrote a non-zero word into what the
|
||
transport now read as "use the DMAC", and **the P4b stage — a stage this session
|
||
did not otherwise touch — went red on a run that never reached its snapshot.**
|
||
|
||
Recorded because the procedure is the finding: `check.sh` was ALL GREEN before
|
||
any of this work, so the red was unambiguously new, and the failure was in a
|
||
stage nobody would have re-run on suspicion. Both halves of "green before and
|
||
green after" earned their place; the map is now `$18500`, clear of everything
|
||
the streaming rig owns.
|
||
|
||
### 59.7 What it all costs: the frame affords 6.74 clocks a byte, and a dual-address byte is 9
|
||
|
||
The sweep in `15_bus_occupancy.py` has always answered *"what does each `W`
|
||
cost"*. It never answered *"what can the frame afford"*, and after 59.2 those
|
||
stopped being the same question. The tool now answers both; every figure below
|
||
is it, on the gate container at 12 fps, and the decode term is **measured**
|
||
(C68K) while the transport terms are datasheet arithmetic — except PIO, which
|
||
58.2 measured.
|
||
|
||
frame slot 833,333 clk
|
||
decoder, MEASURED 570,958 clk 68.5% (worst frame 110.8%)
|
||
audio DMA 10,417 clk 1.25%
|
||
------------------------------------------
|
||
HEADROOM 251,958 clk 30.2% = 6.74 clk/B at a 37,403 B record
|
||
|
||
| transport | clk/B | video | CPU+audio+video |
|
||
|---|---|---|---|
|
||
| PIO — **measured**, 58.2 | 87.28 | 391.7% | **461.5%** |
|
||
| dual address, arbitrated — the IPL ROM's own (52.5) | 16..19 | 71.8..85.3% | 141.6..155.0% |
|
||
| single address, arbitrated | 12 | 53.9% | 123.6% |
|
||
| **dual address, held — and the FLOOR of every dual-address configuration** | **9** | 40.4% | **110.2%** |
|
||
| single address, held | 5 | 22.4% | **92.2%** |
|
||
|
||
**WHAT P4a BOUGHT: the transport falls from 391.7% of a frame to 40..95%**, four
|
||
to ten times. That is the whole of the gain and it is the largest single
|
||
movement in the project's cost model since the decoder was written.
|
||
|
||
**WHAT IT DID NOT BUY IS A FIT, and the reason is one line of arithmetic.** A
|
||
dual-address byte is a 4-clock read of the device and a 5-clock write to memory
|
||
— `buscost.DMA_DUAL_BYTE_CLK`, Fig 4-25 — so **9 clocks is a floor, and the
|
||
frame affords 6.74.** No GCR share goes under it and no delivery rate goes under
|
||
it: a share decides whether the channel sits *at* the floor or *above* it. So
|
||
**59.2's three bounds arrive in the budget as one sentence — the configurations
|
||
this machine can run are exactly the ones the frame cannot afford**, and the one
|
||
it can afford is single address, which needs the device to ACK, which needs the
|
||
request line ROADMAP B3 asks about.
|
||
|
||
**The rate sweep says the same thing from the other side.** At 460 KB/s
|
||
max-rate totals **165.1%** and LRAR at 50% totals **117.4%**; the smaller shares
|
||
total less but cannot carry the rate at all. A faster disc makes auto-request
|
||
cheaper — which no `W` does, and it is the first cost in this project that
|
||
improves when the medium does — but it cannot reach the floor: **a 50% share
|
||
tops out at 543 KB/s**, above which the channel is the bottleneck and the
|
||
delivered rate falls back to it, at which point the cost is exactly 9 clk/B
|
||
again.
|
||
|
||
**SO THE FIT NOW TURNS ON ONE OF TWO THINGS, AND ONE OF THEM IS OURS.**
|
||
|
||
1. **Does a real CZ-6BS1 drive `#EXREQ`/`#EXACK`?** It is the only route to
|
||
single address, 22.4%, 92.2% total, fitting with 7.8% to spare. MAME cannot
|
||
answer it — it does not connect the pin (59.2) — and the slot pinout has it
|
||
at B36/B37. **This is now the sharpest form ROADMAP B3 has ever had**, and it
|
||
is worth more than the throughput half of B1: B1 sets how much headroom the
|
||
player has, B3 decides whether there is any.
|
||
2. **Bytes.** At the 9 clk/B floor this container must come down to **27,995 B a
|
||
frame — 328 KB/s of payload**, against the 37,403 B and 438 KB/s it is now:
|
||
**34% too big.** That is an encoder target, it is entirely inside this
|
||
project, and **it is the pessimistic reading of the lever**, because a
|
||
lighter container also decodes cheaper and the decode term falls with the
|
||
byte term.
|
||
|
||
**WHAT THIS IS NOT.** It is the **gate container**, which is deliberately the
|
||
heaviest thing the encoder emits — the span-heavy scsi container encoded to the
|
||
488 recipe, chosen so that every block mode and the newest span path are
|
||
exercised (check.sh's own note). The lighter `cpufit` family exists and was NOT
|
||
measured here: `15_bus_occupancy.py` refuses it, correctly, because the C68K
|
||
measurement in `tmp/c68k_frames.csv` belongs to the gate container and the
|
||
cross-check at the top of the tool is what licenses every number under it.
|
||
**Re-deriving 59.7 against a lighter container needs that harness re-run first**,
|
||
and until it is, "34% too big" is a statement about the heaviest container and
|
||
not about the project.
|
||
|
||
---
|
||
|
||
## 60. The re-encode bundle: the container agrees with the medium, and two encoder levers turn out not to be levers (session 28)
|
||
|
||
**Emulated** — MAME 0.277, `x68000 -exp1 cz6bs1 -ramsize 2M/6M`, plus px68k's
|
||
C68K core in `tools/bench/c68k`. **No real hardware ran.** `./tools/bench/
|
||
check.sh` was ALL GREEN before this session and is ALL GREEN after it, including
|
||
a full re-encode of the gate container.
|
||
|
||
ROADMAP's re-encode bundle was four items collected under P2 because they shared
|
||
one re-measurement. All four are now closed, and **two of them close as
|
||
negatives** — which is the more useful half of the session.
|
||
|
||
### 60.1 DLX5: the container is laid out in sectors, and the disc now moves exactly the records
|
||
|
||
`tools/encoder/encode.py` pads every record up to **512 bytes** instead of 4,
|
||
and pads the scene header so the frame stream itself starts on a sector
|
||
boundary. `tools/encoder/dlx.py` reads it as `DLX5`; `rec_align` is a property
|
||
of the container version and `record_lengths()` is the one place the rule is
|
||
applied, replacing four hand-copied `4 + n + (-(4+n) % 4)` expressions in the
|
||
analysis tools.
|
||
|
||
**Why 4 was not enough** is 58.3 option C and 59.4: a SCSI target answers in
|
||
512 B blocks, 117 of 120 DLX4 records started part way into one, and
|
||
`sc_in_data` **refuses** a windowed read when the data phase belongs to the
|
||
DMAC, because a channel writes a contiguous run and cannot drop the 300 bytes in
|
||
front of a record. Windowed PIO absorbed that for free and does not survive the
|
||
move to the channel.
|
||
|
||
| on the gate container | DLX4 | **DLX5** |
|
||
|---|---:|---:|
|
||
| records starting on a sector boundary | 3/120 | **120/120** |
|
||
| bytes off the disc, for 4,488,577 B of record | 4,548,608 (+1.34%) | **4,510,208 (+0.48%)** |
|
||
| bytes into the ring | 4,488,588 | **4,510,208** |
|
||
| clocks of window or bounce copy | 0 (PIO only) / +5 clk/B (DMA) | **0, and a channel can run it** |
|
||
| largest record | 40,984 B | 41,472 B — a 256 KB ring still holds it 6 times |
|
||
|
||
**The disc figure and the ring figure are now the same number, and check.sh
|
||
gates on that identity** rather than on two constants. Both are read out of the
|
||
container at check time: the old gate had 4,488,588 and 4,548,608 written into
|
||
it as literals, and the re-encode went red on both — correctly, because the
|
||
container had changed and the expectation had not. **A gate whose expected value
|
||
is a literal tests the literal.**
|
||
|
||
### 60.2 The consumer had to be told, and the failure was a byte-exact wrong answer
|
||
|
||
`stream.s` released the ring up to the last byte it *read*, rounded to 4. Under
|
||
DLX5 that strands up to 511 bytes of pad per record, and the ring's own audit
|
||
caught it on frame 0: `RD_PTR MISMATCH: decoder released 00040F08, record ends
|
||
00041000`. The release now rounds to `RECALN` (geom.i), which is the record
|
||
alignment the container guarantees, and is correct only because the ring base is
|
||
`RECALN`-aligned too — stated where the constant is defined rather than assumed.
|
||
|
||
**This is the shape of the whole item.** Sector alignment is not a tidier
|
||
version of 4-byte alignment; it is a contract with *two* sides, and the producer
|
||
side alone would have drifted the free-space arithmetic by up to 511 B a frame
|
||
with every frame still decoding pixel-exact.
|
||
|
||
### 60.3 Reserved black at index 0 — the letterbox is true black, and it costs 0.04 dB
|
||
|
||
23.4, open since session 5, was the other half of P2. `VQ.scene_palette` now
|
||
quantises the picture into **255 entries and reserves index 0 as (0,0,0)**;
|
||
`dlxload.pack_palette` gives it `I = 0` by its own minimum-squared-error rule
|
||
with no special case, so 23.3's "the bars sit at RGB (4,4,4)" goes away for
|
||
free. GVRAM cleared to zero displays entry 0, and a free mediancut palette put a
|
||
real image colour there — on 00020 f0001, `(206,192,176)`, used by 210 image
|
||
pixels.
|
||
|
||
Black is **reserved, not withheld**: the mapper may still spend index 0 on
|
||
genuinely black pixels. What the reservation buys is that index 0 is black
|
||
whatever the scene contains, which is what the letterbox needs and what a free
|
||
palette cannot promise. Measured cost on the Singe window: **28.92 dB → 28.88 dB
|
||
at `--spans need`**, and the palette ceiling is unmoved at 31.32 dB.
|
||
|
||
### 60.4 `--spans all` as the default: MEASURED, and the recommendation is REFUSED
|
||
|
||
E2 has been "a recommendation, not a measurement" since 43.6.1. It is now a
|
||
measurement, on the DLX5 container, 120 frames, the gate recipe
|
||
(`--kbps 280 --span-kbps 488`):
|
||
|
||
| | KB/s incl. audio | PSNR | frames that miss the 12 fps deadline |
|
||
|---|---:|---:|---:|
|
||
| `--spans need` (shipped default) | **267.9** | 28.88 | 2/120 |
|
||
| `--spans all` | 448.2 | 29.07 | 1/120 |
|
||
|
||
**+67% of the wire, for +0.19 dB and one frame of 120.** Session 27 established
|
||
that the frame affords 6.69 clocks a byte and that a dual-address byte costs 9,
|
||
so the container's problem is that it is too big; spending 180 KB/s to buy a
|
||
fifth of a decibel is the wrong direction, and `need` stays the default.
|
||
|
||
**The item is closed, and the answer is no.** The gate container keeps
|
||
`--spans all` — it is a fixture chosen to exercise the newest path in the
|
||
decoder, not a shipping recipe, and check.sh says so.
|
||
|
||
### 60.5 Joint span/lam selection: implemented, measured, and it is a NO-OP for a reason
|
||
|
||
E3 (39.3 item 5) asked for span selection to be re-derived jointly with `lam`
|
||
instead of greedily after it. The argument is sound on paper: the span pass
|
||
*removes* the block payload of every block it covers, so the frame lands under
|
||
its byte allowance and the blocks that were **not** spanned were priced at a
|
||
`lam` chosen as if those bytes were still needed.
|
||
|
||
`ratectl._refit_joint` (`--joint-spans`) hands the freed bytes back to the lam
|
||
search and re-spans, to a fixed point or two rounds. A round is kept only if the
|
||
frame still fits both ceilings it was already fitting, so `lam` can only fall
|
||
and the un-spanned blocks can only improve.
|
||
|
||
**It emits byte-identical containers.** All four cells of `{need, all} x {greedy,
|
||
joint}` produce two md5s, not four — and so does `--rc-floor open`:
|
||
|
||
| | lam, 120 frames | container |
|
||
|---|---|---|
|
||
| `need` greedy / joint | 10.0 min, median, p90, max | **identical** |
|
||
| `all` greedy / joint | 10.0 min, median, p90, max | **identical** |
|
||
| `all --rc-floor open` greedy / joint | 1.0 min, median, p90, max | **identical** |
|
||
|
||
**The reason is 44.3, and it is structural rather than incidental: `lam` never
|
||
leaves its floor on any frame, at either floor the encoder offers.** The block
|
||
coder at the profile floor already lands under the per-frame byte budget, so
|
||
there is nothing for a joint re-search to spend the freed bytes on. E3 is not a
|
||
lever, and it is not a lever for the same reason `--kbps` and the bucket are
|
||
not: **the byte side of this encoder is governed by the span pass and by `mu`.**
|
||
|
||
The code stays, defaulted off, because the measurement is the finding and a
|
||
future container that moved `lam` off its floor would make the question live
|
||
again. That is 44's pattern kept deliberately: ask whether the lever is loaded
|
||
before pulling it.
|
||
|
||
### 60.6 The apparatus lied, silently, and only a byte comparison caught it
|
||
|
||
**MAME 0.277 served a compressed CHD's own file bytes as sector data.** On the
|
||
DLX5 volume, every `READ(10)` reported success and returned the wrong bytes: the
|
||
destination buffer after a 4,096 B read at LBA 0 was **byte-for-byte the first
|
||
4,096 bytes of `dlxdisk.chd`, starting `MComprHD`** — the CHD file header — while
|
||
`chdman verify` reported both SHA1s correct.
|
||
|
||
Isolated by experiment, and the trigger is the image's **content**:
|
||
|
||
| volume | compression | result |
|
||
|---|---|---|
|
||
| DLX4 gate, 8,768 sectors | default (lzma/zlib/huff/flac) | **byte-exact** |
|
||
| DLX5 gate, 8,809 sectors | default | CHD header served as LBA 0 |
|
||
| DLX5 gate, padded to 9,072 sectors, 16x63 geometry | default | same failure |
|
||
| DLX5 gate, **truncated to 8,768 sectors** — the working length | default | same failure |
|
||
| DLX5 gate | `-c zlib` alone | same failure |
|
||
| DLX5 gate | **`-c none`** | **byte-exact** |
|
||
|
||
So it is not size, not geometry and not the codec; compression decides it and
|
||
uncompressed is sound. **The MAME-side cause is not diagnosed** and is not
|
||
claimed here. `tools/bench/mkvol.sh` now builds the volume `-c none`, at 4.5 MB
|
||
in `tmp/` against 1.6 MB, with the measurement written next to the flag.
|
||
|
||
**What is worth keeping is not the workaround.** The failure was *silent at the
|
||
transport layer* — the SPC handshake completed, the phase sequence was correct,
|
||
the byte count was right, and `scsi_read` returned 0. The only thing in the tree
|
||
that could see it was `tools/bench/scsi.lua` comparing every delivered byte
|
||
against the host's copy of the same image. A transport gate that checked status
|
||
and length would have been green all session, and FINDINGS 58's "byte-exact
|
||
against the host's copy" would have been the load-bearing phrase nobody noticed.
|
||
|
||
### 60.7 What the bundle did to the budget: almost nothing, which is the result
|
||
|
||
The re-measurement the four items shared, on the rebuilt gate container, with
|
||
`tmp/c68k_frames.csv` regenerated by the same `check.sh` run:
|
||
|
||
| | before (DLX4, session 27) | **after (DLX5, session 28)** |
|
||
|---|---:|---:|
|
||
| decoder, MEASURED on C68K | 68.5% of the frame | **68.6%** |
|
||
| delivered bytes a frame | 37,403 | **37,585** (the pad is delivered, so it is charged) |
|
||
| headroom after decode + best-case audio | 6.74 clk/B | **6.69 clk/B** |
|
||
| single address held, `W`=5, total | 92.2% | **92.4%** |
|
||
| dual address held, `W`=9, total | 110.2% | **110.4%** |
|
||
| E7's byte target at the 9 clk/B floor | 328 KB/s, 34% too big | **327 KB/s, 35% too big** |
|
||
| B1's zero-prefill delivery requirement | 451.4 KB/s | **453.6 KB/s** |
|
||
| scene header (P1/53.5, 22_scene_load) | 6,164 B | **6,656 B** |
|
||
|
||
**Every conclusion in 59.7 survives unchanged**, which is what a precondition is
|
||
supposed to do: it removes an obstacle without moving the arithmetic. The two
|
||
numbers that did move are both the alignment pad being paid honestly —
|
||
`15_bus_occupancy.py` now charges the **padded** record rather than the payload,
|
||
because the disc carries the pad whether or not a frame contains it.
|
||
|
||
### 60.8 Two caches that would have tested the wrong artefact
|
||
|
||
`H.build` acquired an option this session (`reserve_black`), and two tools cache
|
||
its output in a pickle **keyed on the frames directory alone** —
|
||
`16_span_roundtrip.py`, which is the span container's round-trip gate, and
|
||
`13_cpu_ratectl.py`. A stale pickle would have let the gate round-trip a
|
||
container built from the old palette while the shipping encoder emitted the new
|
||
one: green, and testing an artefact that no longer exists. Both now store the
|
||
build parameters with the model and rebuild on a mismatch.
|
||
|
||
### 60.9 What is now true, and what P4a still owes
|
||
|
||
**M2's re-encode bundle is done and the container is what the DMA path needs.**
|
||
`dma_run.sh` still gates on the *refusal* — `WINDOWED DMA READ REFUSED, as it
|
||
must be` — which is now a negative control rather than a description of the
|
||
container, because the container no longer asks for a windowed read.
|
||
|
||
What is left of P4a is the wiring: putting the channel behind `ring.i`'s `XF_*`
|
||
mailbox in place of the PIO loop in `xfer.i`, which 59.4 blocked on exactly this
|
||
precondition. `xfer.i`'s sector arithmetic already degenerates correctly —
|
||
`SC_WSKIP` is 0 and `SC_WKEEP` is the whole record on every record — so what
|
||
changes is which loop moves the bytes, not what is asked for.
|
||
|
||
## 61. The decoder-free packed player fits the clocks the codec misses, and the codec cannot have the packing (session 29)
|
||
|
||
**Emulated.** MAME 0.277, `x68000 -bios ipl10`, and for the transport runs
|
||
`-exp1 cz6bs1` with a zero-filled `scsiexrom.bin`. No real hardware ran, and
|
||
47.4 — does buffer mode blank the layer — still wants a board.
|
||
|
||
FINDINGS 44.7 asked what a player with no codec at all would cost and answered
|
||
"it fits the clocks and dies on the medium". 46.5/47.1 found the off switch for
|
||
the 2:1 GVRAM tax, 47.2 built the packed layout and rendered it pixel-exactly on
|
||
both emulators, and **47.5 re-derived the budget on a cost model that has since
|
||
been replaced in full** — the transport was an unmeasured `c` when it was
|
||
written, and sessions 25b–28 measured it (58.2), bounded it (59.2) and priced
|
||
what a frame can afford against it (59.7/60.7). 47.6.1 filed the packed paint's
|
||
`movem` shape as an *assumption*; 47.6.2 said the DMAC "has not been near this";
|
||
47.6.4 left the codec's own survival under the layout untouched.
|
||
|
||
All four are answered here. `tools/analysis/29_packed_player.py` is the
|
||
arithmetic, `tools/bench/blit.s` V8/V9/V10 are the paint measurements, and
|
||
`src/player/dmagate.s` runs 4–6 are the transport ones.
|
||
|
||
### 61.1 The packed paint, MEASURED: it costs what the unpacked path pays to WRITE
|
||
|
||
`blit.s` V8 is V1 with one thing different — a row is 128 words instead of 256,
|
||
because R20 bit 11 lets a word carry two picture bytes — and `blit.lua` times it
|
||
in the same run as V1, V2 and V3, so it is quoted against numbers that have not
|
||
moved since session 9.
|
||
|
||
| variant | clk/frame | % of a 12 fps frame | what it moves |
|
||
|---|---:|---:|---|
|
||
| V1 unpacked `movem` blit | 446,286 | 53.6% | 96 KB read + 96 KB write |
|
||
| V2 byte-source expansion | 1,284,174 | 154.1% | 48 KB read + 96 KB write |
|
||
| V3 unpacked WRITE-ONLY floor | 225,789 | 27.1% | no source read at all |
|
||
| **V8 PACKED `movem` blit** | **227,553** | **27.3%** | **48 KB read + 48 KB write** |
|
||
|
||
All four put the same 49,152 pixels on screen. **V8 is 51.0% of V1 and 100.8% of
|
||
V3: the packed blit costs what the unpacked one pays to write alone, with its
|
||
source read thrown in free.** It is not exactly half of V1 because the 192-row
|
||
loop does not halve with the words — per word V1 is 9.080 clocks and V8 is
|
||
9.259, and the difference is the row loop amortised over half as many bursts.
|
||
|
||
V2 is worth keeping in view: it is the "send 1 byte a pixel and let the CPU
|
||
expand it" trade, and at 154.1% it is not a trade, which is why the unpacked
|
||
path has no cheap wire.
|
||
|
||
V8 is a **timing** variant and does not set bit 11. MAME's `gvram_w` carries no
|
||
timing in either arm, so the bit cannot move a cycle; what it moves is the
|
||
picture, and the picture is what `show_frame256_packed.lua` and `gvpack` already
|
||
verify pixel-exactly (47.2). The source is pre-interleaved host-side, which is
|
||
the honest half of the claim — the packing is an encoder-side transform, the
|
||
same argument 46.3 made for the text plane.
|
||
|
||
### 61.2 The channel writes GVRAM in buffer mode, and it walks the line stride itself
|
||
|
||
47.6.2 was right that nothing in this tree had ever pointed a channel at
|
||
`$C00000`. `dmagate.s` now does, three times, and `dma_run.sh` gates all three:
|
||
|
||
| run | R20 | result |
|
||
|---|---|---|
|
||
| **4. DMA → GVRAM, bus held, bit 11 SET** | `$0916` | **2,048 B byte-exact against the disc**; 1,024 landed in page 1 (high half), 1,024 in page 0 |
|
||
| **5. the SAME, bit 11 CLEAR** | `$0116` | **457 bytes lost, every one of them at an EVEN offset; not one ODD byte harmed** |
|
||
| **6. ARRAY CHAINED, 8 rows at the 1024 B stride** | `$0916` | **2,048 B byte-exact at eight separate row bases from ONE start**; `MAR` ended at `+7,424` |
|
||
|
||
In all three the discriminator reads the same as 59.1's: **`MTC` sampled by the
|
||
instruction after `START` is 0 of 2,048 and the CPU went round its wait loop
|
||
once** — the whole transfer happened between two instructions.
|
||
|
||
**Run 5 exists because the first cut of run 4 was a test that could not fail.**
|
||
It OR-ed bit 11 on top of whatever the IPL left, and the IPL leaves `$0B16`
|
||
(22.1) — bit 11 **already set**, and `COL = %11`, the 65,536-colour setup, which
|
||
writes whole words with or without it. It passed and proved nothing. Both runs
|
||
now write R20 outright and differ in exactly that bit.
|
||
|
||
**And the control's signature is a PLACE, not a count.** The obvious assertion —
|
||
"half the bytes must differ" — is wrong, and asserting it turned the gate red on
|
||
a correct run. In masked 256-colour mode `gvram_w` takes `data & 0x00ff` and
|
||
ignores `mem_mask`, so a byte written to an even address is never stored and the
|
||
high half keeps what it held; where the record is pad, the stale half matches the
|
||
disc by coincidence, and 567 of 1,024 did. What the mechanism says is that
|
||
**every odd byte survives and only even ones can be lost**, and that is what the
|
||
gate asserts.
|
||
|
||
Run 6 is the one that changes an architecture. A picture row is 256 B of a
|
||
1,024 B line stride, so a frame is **192 destinations, not one**, and a channel
|
||
writes a contiguous run — the same limitation that makes it refuse a windowed
|
||
read (58.3). 46.6 said "no stride for a DMAC to skip" about the bytes *within* a
|
||
row and never examined the rows. The MC68450 answers with sequential array
|
||
chaining, `src/player/dma.i` now programs it behind a `DM_BARV` mailbox that is
|
||
**zero unless a caller asks**, and the channel walked an 8-entry array on its
|
||
own. **The CPU does not restart the channel per row.**
|
||
|
||
### 61.3 The codec cannot have the packing — 47.6.4, closed
|
||
|
||
A 4×4 block owns four bytes at **stride 2** under the packed layout, because the
|
||
high bytes of its four words belong to the block 128 columns away. There are
|
||
exactly two ways a block decoder could live with that, and `blit.s` V9 and V10
|
||
are them:
|
||
|
||
| variant | clk/frame | % of frame | vs V4 |
|
||
|---|---:|---:|---:|
|
||
| V4 block order, UNPACKED (the shipping shape) | 637,971 | 76.6% | — |
|
||
| **V9 block order, PACKED, 16 `move.b` at stride 2** | **815,839** | **97.9%** | **+28%** |
|
||
| **V10 block order, PACKED, blocks PAIRED** | **320,456** | **38.5%** | **−50%** |
|
||
|
||
**V9 is 28% DEARER than V4, and buys nothing on the wire** — a codeword is
|
||
already one byte a pixel, so the packed layout has no bytes left to save a block
|
||
decoder. That route costs 177,868 clocks a frame for nothing.
|
||
|
||
**V10 halves the paint and pays for it in the mode map.** Pairing the block at
|
||
x with the block at x+128 gives V4's `movem` shape back, and a pair skips only
|
||
if *both* its blocks skip. On the gate container:
|
||
|
||
| | now | paired |
|
||
|---|---:|---:|
|
||
| SKIP | **66.3%** of blocks | **46.1%** of pairs |
|
||
| painted | **33.7%** | **53.9%** |
|
||
|
||
So pairing paints **1.60×** as many blocks for **0.50×** the paint per block:
|
||
**−20% on the clock, and about +60% on the BYTES**, because a coded block is
|
||
bytes in the container whether its half of the pair changed or not. E7 needs the
|
||
bytes **down 35%**.
|
||
|
||
**The packed layout is not an upgrade the codec can take. It is the thing you
|
||
get instead of the codec.**
|
||
|
||
### 61.4 The budget, re-derived on the measured model
|
||
|
||
Every cell below is CPU work plus transport plus best-case audio (10,417 clk,
|
||
1.25%, from the IPL ROM's own channel-3 setup — 21_iplrom_dmac.py). None of them
|
||
overlap: `buscost.DMA_OVERLAPS = False`.
|
||
|
||
| architecture | B/frame | W=5 | **W=9** | W=12 | W=16 | W=19 |
|
||
|---|---:|---:|---:|---:|---:|---:|
|
||
| CODEC, CPU-decoded (the shipping design) | 37,585 | 92.4% | **110.4%** | 124.0% | 142.0% | 155.5% |
|
||
| free / DMAC device→GVRAM / unpacked | 98,304 | 61.1% | 108.2% | 143.6% | 190.8% | 226.2% |
|
||
| **free / DMAC device→GVRAM / PACKED** | **49,152** | 31.6% | **55.2%** | 72.9% | 96.5% | 114.1% |
|
||
| free / CPU-painted / unpacked, 2 B/px wire | 98,304 | 113.8% | 161.0% | 196.4% | 243.5% | 278.9% |
|
||
| free / CPU-painted / unpacked, 1 B/px wire | 49,152 | 184.8% | 208.4% | 226.1% | 249.7% | 267.4% |
|
||
| **free / CPU-painted / PACKED** | **49,152** | 58.0% | **81.6%** | 99.3% | 122.9% | 140.6% |
|
||
|
||
**W=9 is the column that matters**, because 59.2 found the only configurations
|
||
this machine can be shown to run are dual-address, and a dual-address byte is a
|
||
4-clock read of the device plus a 5-clock write to memory. Everything left of it
|
||
is a hardware fact nobody here has (ROADMAP B3).
|
||
|
||
**At the floor the CODEC misses by 10.4% and the DECODER-FREE PACKED PLAYER FITS
|
||
WITH 45% TO SPARE.** Decoding 37,585 bytes costs 109% of a frame; not decoding
|
||
49,152 costs 54%.
|
||
|
||
That is not a small correction to 47.5. **44.7 said it in advance, on a cost
|
||
model that has since been thrown away: "the codec is not there to save CPU — it
|
||
is there to save the wire."** The measured model agrees and goes further: the
|
||
CPU side is not merely affordable without the codec, it is *strictly cheaper*.
|
||
|
||
### 61.5 So it is entirely a medium question, and the medium is the unmeasured one
|
||
|
||
| architecture | B/frame | KB/s | GB for 22.8 min |
|
||
|---|---:|---:|---:|
|
||
| CODEC, gate container | 37,585 | 440.4 | 0.62 |
|
||
| CODEC, `need` default | — | 267.9 | ~0.38 |
|
||
| CODEC, E7's target at the 9 clk/B floor | 27,924 | 327 | 0.46 |
|
||
| **decoder-free PACKED, either architecture** | **49,152** | **576.0** | **0.81** |
|
||
| decoder-free unpacked | 98,304 | 1,152.0 | 1.61 |
|
||
|
||
**576 KB/s, sustained, with no lever to pull.** A codec's bitrate is a lever; a
|
||
literal frame's is geometry, and no scene in the picture costs less than another.
|
||
ROADMAP B1 is unmeasured, and the 0.7–1.7 MB/s usually quoted for BlueSCSI on an
|
||
X68000 is **folklore with no published benchmark behind it**. 576 KB/s sits
|
||
inside that range, which is exactly the reason the range has to be measured
|
||
rather than cited: **a codec at 327 KB/s survives a slower answer and a literal
|
||
frame does not degrade, it drops.**
|
||
|
||
**B1 has therefore changed character.** It used to set how much headroom the
|
||
player has. It now decides which player exists.
|
||
|
||
### 61.6 And under MAME's blanking reading, the cheap architecture is the dark one
|
||
|
||
47.4's question is untouched by any of this and it still needs a board. What the
|
||
measured paint does is turn 48.3's range into numbers, and it exposes an
|
||
asymmetry nobody had stated: **R20 bit 11 only has to be set across the GVRAM
|
||
writes, and where those writes come from decides how long that is.**
|
||
|
||
| architecture | black interval | bit 11 set for |
|
||
|---|---:|---|
|
||
| free / DMAC device→GVRAM / unpacked | 60%..225% of a frame | the whole DMA |
|
||
| free / DMAC device→GVRAM / PACKED | 30%..113% of a frame | the whole DMA |
|
||
| free / CPU-painted / unpacked | 53.6% | the blit only |
|
||
| **free / CPU-painted / PACKED** | **27.3%** | **the blit only** |
|
||
|
||
The CPU-painted packed path has the smallest black window of any of them,
|
||
because its transport lands in RAM where bit 11 is irrelevant and only the
|
||
227,553-clock blit needs the bit. **The DMAC-direct path, which is cheaper in
|
||
clocks at every rung of the ladder, is the one that must hold the bit across its
|
||
whole transfer.** If MAME is right, the cheap architecture is the dark one and
|
||
the expensive one is merely dim.
|
||
|
||
Both are a strobe at the frame rate over the whole picture, and the packed layout
|
||
has no page to flip to: both 256-colour pages carry picture, which is the entire
|
||
point of it (48.3). If px68k is right, every number in 61.4 and 61.5 stands as
|
||
written. 48.1's prior — an assertion against a silence — has not moved.
|
||
|
||
### 61.7 What this does NOT establish
|
||
|
||
1. **No clock here is a transport clock.** MAME's DMAC runs on wall-clock
|
||
attotimes and models a held bus by halting the CPU (42.5), so runs 4–6 settle
|
||
which configurations *work* and not what one costs. Every `W` in 61.4 is
|
||
datasheet arithmetic except PIO, and the array-chain entry at 36 clocks is
|
||
Fig 4-25 sheet 1, not a measurement.
|
||
2. **One frame's worth of transport, not a stream.** Run 6 chained eight rows,
|
||
not 192, and nothing here ran a chained transfer back to back at 12 fps or
|
||
through `ring.i`. A decoder-free player has no ring at all in the DMAC-direct
|
||
form, which is a simplification this tree has not tested.
|
||
3. **The paint is a lower bound.** Every `blit.s` number is instruction cycles
|
||
against zero-wait-state memory; real GVRAM stalls the CPU and neither
|
||
emulator models it (`blit.lua`'s header, and 45's note on C68K).
|
||
4. **61.3's pairing cost is one container's.** 66.3% / 46.1% is the gate
|
||
container, deliberately the heaviest thing the encoder emits. A lighter
|
||
container has a different SKIP map and would pair differently — though not,
|
||
at that margin, differently enough to change the sign.
|
||
5. **No audio, no branching, no seek.** 61.4 charges the audio DMA and nothing
|
||
else. A decoder-free player still has to make 56's branch decisions and 51's
|
||
seeks, and at 576 KB/s it has less slack to make them in, not more.
|
||
|
||
### 61.8 Encoder work is parked (session 29, USER DECISION)
|
||
|
||
61.4 first closed with "nothing here is a reason to stop work on the codec — it
|
||
is the only branch that survives a slow answer to B1." **That does not survive
|
||
its own arithmetic and is withdrawn.**
|
||
|
||
It rested on 576 KB/s against **E7's 327 KB/s target, which does not exist.** The
|
||
codec that exists is **440 KB/s and 110.4% of a frame**, so the gap is **1.31x,
|
||
not 1.76x** — and reaching 327 needs a 35% byte reduction after 60.4 and 60.5
|
||
measured two of the encoder's three byte levers and found neither is a lever. The
|
||
reward on success is a design at ~100% of the frame with no margin, which is where
|
||
55.2% already is.
|
||
|
||
The two branches are not symmetric, and counting what each NEEDS is the argument:
|
||
|
||
| | facts it needs |
|
||
|---|---|
|
||
| **decoder-free packed** | buffer mode does not blank (B2); medium clears 576 KB/s (B1) |
|
||
| **codec** | E7 succeeds (unproven, two levers dead); **and** medium clears 327; **and** it ships at ~100% of a frame |
|
||
|
||
**So E7 and E4 are parked, and C1 with them.** E4 is included deliberately: it is
|
||
`H.build`'s k-means, it builds **VQ codebooks**, and a literal player has no VQ.
|
||
|
||
**What is NOT parked is the codec itself.** It stays on disk, gated by check.sh,
|
||
and nothing is built on it. 48.1's prior leans against packing — MAME asserts the
|
||
blanking semantic twice and deliberately, px68k's display path never reads the
|
||
bit at all, and an assertion against a silence is not a tie — and 48.3 stands:
|
||
if buffer mode blanks there is no version of the packed player that is merely
|
||
expensive. In that branch the codec is the only path left. **Keeping a working
|
||
decoder is inventory; building on it is work, and the work waits on B2.**
|
||
|
||
### 61.9 And the picture is BETTER — the codec is capped below the thing replacing it
|
||
|
||
Asked while scoping what a packed player would take, and it is the finding that
|
||
makes the branch worth building rather than merely affordable. The numbers are
|
||
`18_text_plane_16col.py`'s, over the same 120-frame window everything else is
|
||
measured on; 46.3 computed them to give the *text plane* something to be scored
|
||
against and never turned them on the 256-colour path itself.
|
||
|
||
| | PSNR vs the 24-bit source |
|
||
|---|---:|
|
||
| shipping container (the codec, as it ships) | **29.19 dB** |
|
||
| 256 colours, SCENE palette — **the codec's CEILING** | **31.33 dB** |
|
||
| 256 colours, **PER-FRAME** palette | **34.08 dB** |
|
||
|
||
**The middle row is a ceiling, not a rival.** Every codeword the codec emits is
|
||
an index *into* `vq.scene_palette`, so no bitrate takes it past 31.33 dB; it
|
||
spends 440 KB/s getting within 2.14 dB of it.
|
||
|
||
**A literal frame has no codebooks, so nothing forces the scene palette on it,
|
||
and per-frame palettes become legal.** 46.3 stated the constraint in as many
|
||
words while arguing the *other* side — "the 256-colour path cannot do this: its
|
||
palette is shared scene-wide because the codec's codebooks are indices INTO it."
|
||
Remove the codec and the constraint goes with it.
|
||
|
||
**So the decoder-free packed player is +4.89 dB on the shipping container and
|
||
+2.75 dB past a ceiling the codec cannot cross** — while costing 55.2% of a frame
|
||
against 110.4%. It is not a quality compromise bought with clocks. It is better
|
||
on both, and the whole of its cost is on the wire.
|
||
|
||
What the per-frame palette costs:
|
||
|
||
| | |
|
||
|---|---|
|
||
| wire | 512 B/frame → 49,664 B, **582.0 KB/s (+1.0%)** |
|
||
| clocks | ~2,370, **0.28% of a frame** if the CPU writes it — DERIVED from V8's measured 9.259 clk/word in the same `movem` shape |
|
||
| colours | **254, not 256** — the packed layout spends index 0 on the transparency key and puts black at 255 (47.2, `prep_frame.py --pack-transparent`), where `--reserve-black` spends one. A reserved entry measured 0.04 dB in 60.3, so this is noise against +4.89 |
|
||
|
||
**Two things this does NOT settle.**
|
||
|
||
1. **Whether a channel can write `$E82000`.** If the palette registers take a
|
||
byte-wide DMA the way GVRAM does in buffer mode, the palette is a 193rd array
|
||
entry and costs the CPU nothing at all — one channel start still paints a
|
||
whole frame. 61.2 only ever pointed a channel at GVRAM. **This is the next
|
||
probe, and it is the same shape as the ones that worked.**
|
||
2. **The quantiser is PIL's MEDIANCUT, not this project's.** `vq.scene_palette`
|
||
and `H.build` are what would actually ship the palette. The direction is
|
||
measured and the magnitude is about right; re-derive the per-frame figure
|
||
against the real builder before quoting it as the player's number.
|
||
|
||
## 62. One channel start paints a whole frame — the palette registers take the DMA (session 30)
|
||
|
||
**Emulated.** MAME 0.277, `x68000 -bios ipl10`, `-exp1 cz6bs1` with a
|
||
zero-filled `scsiexrom.bin`. No real hardware ran. 47.4 — does buffer mode blank
|
||
the layer — is still the board question and is still open, and **62.4 adds a
|
||
second board question this run created.**
|
||
|
||
61.9 left the decoder-free packed player one open structural item, ROADMAP K1:
|
||
a frame is a picture *and* a palette, and 61.2 had only ever pointed a channel
|
||
at GVRAM. If the palette registers at `$E82000` take a byte-wide DMA the way
|
||
GVRAM does in buffer mode, the palette is a 193rd array-chain entry and the
|
||
whole video path is **one channel start a frame**; if they do not, the CPU
|
||
writes 256 words a frame (61.9 derives ~2,370 clocks, 0.28% of a frame) and the
|
||
architecture stands anyway. It is the difference between cheap and free, and it
|
||
took one run to know which.
|
||
|
||
`src/player/dmagate.s` runs 7–9, gated by `tools/bench/dma_run.sh`, which
|
||
`check.sh` runs. **The answer is free, in this model.**
|
||
|
||
| run | what it does | result |
|
||
|---|---|---|
|
||
| **7. DMA → `$E82000`, bus held** | 512 B off the disc into the whole graphic palette | **byte-exact in 256 register words**, read back out of the registers by the 68000 |
|
||
| **8. the SAME transfer aimed at RAM** | the attribution control: `$2C000` instead | **byte-exact at `$2C000`, and 256 of 256 palette words still read the poison** |
|
||
| **9. ONE array-chained start** | 7 entries: the palette, then six picture rows at the 1,024 B line stride | **2,048 B byte-exact across BOTH kinds of destination**, `MAR` ended at `+5,376` |
|
||
|
||
In all three the discriminator reads as 59.1's and 61.2's do: **`MTC` sampled by
|
||
the instruction after `START` is 0 and the CPU went round its wait loop once** —
|
||
the transfer happened between two instructions, with the 68000 not executing.
|
||
|
||
### 62.1 The destination was POISONED first, because "it matches" has been a weak claim all along
|
||
|
||
Runs 4–6 wrote into RAM that was zero and GVRAM that was stale, against a record
|
||
that is mostly pad. A destination that could already hold the right answer
|
||
cannot distinguish a channel that wrote from a channel that did nothing — which
|
||
is run 4's could-not-fail trap wearing different clothes (61.2).
|
||
|
||
So `dg_poison` fills the palette with word *i* = `$A500|i` before each palette
|
||
run, written by the 68000 and read back by it. The host does not assume the
|
||
poison is a discriminator, it **counts**: `PALETTE POISON IS A DISCRIMINATOR:
|
||
511 of 512 positions differ from the disc's bytes`, and the gate refuses a run
|
||
where fewer than 500 do. One position coincides, and the pass does not rest on
|
||
it.
|
||
|
||
### 62.2 The control is an ATTRIBUTION control, not a mechanism one — and it says so
|
||
|
||
Run 5 could point at a mode bit; there is no mode bit here. What had to be
|
||
excluded is that run 7's palette held the disc's bytes for some reason other
|
||
than the channel having put them there — a readback that aliases somewhere else,
|
||
the SPC's own path touching the registers, the poison never having landed at
|
||
all. So run 8 is the same transfer with **one thing different, the destination
|
||
address**, and it makes two claims from one run:
|
||
|
||
- the disc's bytes appear at `$2C000`, so the transfer happened;
|
||
- **the palette still reads poison in all 256 words**, so what reached
|
||
`$E82000` in run 7 was decided by the channel's `MAR`.
|
||
|
||
The second claim is also the positive half: it shows the CPU's own writes reach
|
||
the registers the host reads back, so the readback path is not the thing under
|
||
test.
|
||
|
||
### 62.3 The 193rd entry is literal, and the array is SCENE-constant
|
||
|
||
Run 9 is the one that changes the architecture, and it is not "the palette
|
||
works" — it is that **one array chain crosses two kinds of destination**: device
|
||
registers at `$E82000` and video RAM at `$C14000`, in one start, with the CPU
|
||
halted from the first byte to the last. A frame is that shape with 192 row
|
||
entries instead of six.
|
||
|
||
**And the array does not have to be rebuilt per frame.** The row bases are
|
||
`$C00000 + row * 1024` and they do not change: the packed layout spends *both*
|
||
256-colour pages — page 0 is the low byte of a word and page 1 the high byte
|
||
(47.2, and 46.1's page masks) — so there is no spare page to flip into and no
|
||
alternate set of destinations to alternate between. The 193-entry array
|
||
(1,158 B) is built once at scene setup and started once a frame.
|
||
|
||
**What is left on the CPU in the video path is therefore the channel start and
|
||
the READ(10) that fetches the record — and neither is priced here.** The command
|
||
issue is already inside the transport's own account (58.2); the start is about a
|
||
dozen register writes and is DERIVED as small rather than measured. Do not quote
|
||
"no per-frame CPU work" without that sentence attached: it is no per-frame
|
||
*paint* work.
|
||
|
||
### 62.4 What this does NOT settle, and it is a NEW BOARD QUESTION
|
||
|
||
**MAME cannot discriminate here, and the reason is in its source.** The graphic
|
||
palette is not modelled as a register file at all: `x68k.cpp:817` maps
|
||
`$E82000-$E821FF` to `palette_device::read16/write16`, `emupal.cpp:417` forwards
|
||
to `memory_array::write16`, and `memarray.h:75` is a plain `COMBINE_DATA`. That
|
||
is **RAM that honours `mem_mask`** — so a byte write lands in its half by
|
||
construction, and a green run says *nothing in the model forbids it* rather than
|
||
*the board takes it*.
|
||
|
||
This is a different kind of bound from 61.2's. GVRAM has a real handler with a
|
||
real 256-colour arm, which is why run 5 could find a mechanism to fail on; the
|
||
palette has no handler to be wrong about. **What a byte write to a real
|
||
X68000 palette register does is UNMEASURED and this project has no figure for
|
||
it** — not folklore, not an estimate, an absence. It goes on the hardware list
|
||
as **B4**, and it is cheap: write `$A5` to `$E82000` and `$5A` to `$E82001` from
|
||
the CPU on a real board and read the word back.
|
||
|
||
**The blast radius if B4 comes back negative is small and known**: the palette
|
||
leaves the chain, the CPU writes 256 words a frame at 61.9's derived 0.28% of a
|
||
frame, and every other claim in 61 and 62 stands. Run 9's crossing would still
|
||
have to be re-asked, because it would no longer have a device-register end.
|
||
|
||
### 62.5 The chain's ORDER is a free choice with a visible consequence, and it is not decided
|
||
|
||
Run 9 puts the palette FIRST. It could as easily be 193rd, which is what 61.9
|
||
called it. The two are not equivalent on screen and neither is obviously right:
|
||
|
||
- **palette first** — the 192 rows of the *previous* frame are displayed under
|
||
the *new* palette until each is overwritten;
|
||
- **palette last** — the new frame's rows are displayed under the *old* palette
|
||
until the chain reaches the end.
|
||
|
||
The mismatch lasts one paint either way. Which is less visible depends on how
|
||
much the palette moves between consecutive frames, which is a property of the
|
||
encoder K2 has not been written yet, and the whole question is **moot if buffer
|
||
mode blanks the layer** (47.4/B2) because nothing is displayed during the paint
|
||
at all. **Filed, not answered.** It is named here so that the choice in the
|
||
final player is a decision rather than an accident of which run happened first.
|
||
|
||
## 63. The packed container, and the palette that buys 2.31 dB has a price nobody had counted (session 31)
|
||
|
||
**Emulated, and one stage of it is not emulated at all.** The quality numbers
|
||
are host arithmetic over the Blu-ray's own frames. The rendering check is
|
||
**px68k's real `x68k/gvram.c`**, linked headless the way `tools/bench/c68k`
|
||
links its CPU core. No MAME run was needed for any of this and no real hardware
|
||
ran. 47.4 — does buffer mode blank the layer — is still the board question, and
|
||
this session made it **bigger**.
|
||
|
||
ROADMAP **K2 is done**. `tools/encoder/dlxp.py` is the format,
|
||
`tools/encoder/pack.py` is the encoder, `tools/analysis/30_packed_container.py`
|
||
is the gate and the re-derivation, `tools/bench/gvpack/verify_dlxp.py` renders
|
||
the container's own bytes on the second emulator, and `check.sh` runs all four.
|
||
|
||
### 63.1 DLXP1: the container whose correctness is that nothing parses it
|
||
|
||
`dlx.py` exists because a 68000 has to **parse** the codec's container and can
|
||
get it wrong. `dlxp.py` exists for the opposite reason: a DMA channel must not
|
||
have to parse anything, and the format is what makes that true.
|
||
|
||
| | |
|
||
|---|---|
|
||
| record | **49,664 B = 97 sectors EXACTLY** — 512 B of palette, 49,152 B of picture |
|
||
| picture | 192 rows x 128 words, word `i` = `(pix[y][i+128] << 8) \| pix[y][i]` — the layout 47.2 rendered pixel-exactly on both emulators |
|
||
| index | **none, and none is possible to need**: a record's length is geometry, so record `i` is at `off_frm + i * rec_bytes` and a seek is arithmetic |
|
||
| wire | **582.0 KB/s, fixed** — 61.9 predicted 582.0 and the container is 582.0 |
|
||
| encode | **3.3 s for 120 frames**, against ~55 s for the codec, 95% of which is k-means |
|
||
|
||
DLX4's record index was invented (49.3) because a codec record's length is
|
||
content-dependent and a producer streaming off a disc cannot learn it by walking
|
||
bytes it has not fetched. **A packed record has no such problem to solve**, and
|
||
that is the same reason the packed player has no ring: there is nothing
|
||
variable-length to keep contiguous.
|
||
|
||
**Sector alignment, which cost session 28 a whole re-encode (60.1), is free
|
||
here.** 49,664 is 97 sectors because a 256x192 picture and a 256-entry palette
|
||
happen to be. It is still *checked* rather than assumed, because the thing being
|
||
protected is not tidiness: the channel copies bytes and has no opinion about
|
||
them, so a container whose geometry is a byte wrong does not fail, it paints.
|
||
|
||
### 63.2 The container's OWN BYTES, through px68k's GVRAM code, with no interleave computed
|
||
|
||
`verify_gvpack.py` (47.2) checks the **layout**: it hands the harness a picture
|
||
and the harness computes the interleave. That cannot catch an encoder whose byte
|
||
order is wrong, because a container round-trips against its own inverse either
|
||
way. `verify_dlxp.py` writes a **DLXP1 record's bytes into GVRAM verbatim** and
|
||
asks px68k what they display as.
|
||
|
||
**256x192 index-exact, letterbox on the reserved black, index 0 never on
|
||
screen** — frames 0, 60 and 119. Two negative controls, both mechanisms this
|
||
container depends on rather than decoration: **R20 bit 11 clear loses 24,576 px**
|
||
(exactly the 128 columns page 1 carries) and **page 1 unscrolled loses 48,958**.
|
||
|
||
### 63.3 The picture, RE-DERIVED against this project's own builder, and charged the hardware word
|
||
|
||
This is what the session 30 handoff asked for. 61.9's 34.08 dB was PIL's free
|
||
256-colour MEDIANCUT and was filed as "a direction, not the player's number". It
|
||
had **two** debts, not one, and the second had gone unnamed: every PSNR this
|
||
project has ever quoted — 29.19, 31.33, 34.08 — is measured in the **RGB888
|
||
palette domain**, upstream of the X68000's `GGGGGRRRRRBBBBBI` word (23.3). A
|
||
packed record carries that word and nothing else, so a player's number has to
|
||
come from the other side of it.
|
||
|
||
PSNR vs the 24-bit source, mean over the same 120-frame window, on the gate
|
||
container's own bytes:
|
||
|
||
| | RGB888 | **GRB555** |
|
||
|---|---:|---:|
|
||
| CODEC, the gate container (440.4 KB/s) | 29.07 | **28.72** |
|
||
| 256c scene palette — the codec's CEILING | 31.32 | **30.79** |
|
||
| PACKED, 254c SCENE palette *(the control)* | 31.32 | **30.79** |
|
||
| **PACKED CONTAINER, 254c PER-FRAME** | **34.05** | **33.10** |
|
||
|
||
**61.9's direction survives the real builder: 34.05 against its 34.08.** The two
|
||
entries the packed layout reserves — index 0 for the transparency key, 255 for
|
||
black — cost **+0.0003 dB**, which is to say nothing at all, and marginally the
|
||
right way. 60.3 measured one reserved entry at 0.04 dB; two is not twice that,
|
||
it is noise.
|
||
|
||
**The GRB555 word costs 0.53 dB** and it costs every row of the table, so no
|
||
comparison in this project moves because of it. It is stated because a player's
|
||
number should be a player's number.
|
||
|
||
**And the CONTROL is the finding under the headline.** A packed container with
|
||
one scene palette lands **exactly on the codec's ceiling, 30.79 dB** — as it
|
||
must, because at that point the only difference left between them is VQ. So the
|
||
whole of the packed branch's picture advantage over the codec's *ceiling*, +2.31
|
||
dB, is **the per-frame palette and nothing else**. Not the packing, not the
|
||
literal frames, not 254 colours. One mechanism, and 61.9 named it correctly.
|
||
|
||
### 63.4 62.5 is priced, and the answer is that the ORDER does not matter and the MISMATCH does
|
||
|
||
62.5 filed palette-first-or-193rd as a free choice whose severity "depends on how
|
||
much the palette moves between consecutive frames, which is a property of the
|
||
encoder K2 has not been written yet". It is written now.
|
||
|
||
| | |
|
||
|---|---:|
|
||
| palette entries that CHANGE frame to frame | **231.1 of 256 (90%)** |
|
||
| palette FIRST — old rows under the new palette | **20.32 dB** (−12.79) |
|
||
| palette LAST — new rows under the old palette | **20.33 dB** (−12.78) |
|
||
|
||
**The two orders are indistinguishable — 0.01 dB apart — so 62.5's choice is a
|
||
wash and can be made on other grounds.** The container makes it a flag
|
||
(`--palette-last`, flags bit 1) rather than an assumption, so K3 can run both.
|
||
|
||
**What the same measurement found is not a wash.** A per-frame palette is not a
|
||
small delta: 90% of the entries move every frame, and a picture under the
|
||
neighbouring frame's palette is **12.8 dB worse than the correct pairing**. The
|
||
mismatch is a wipe rather than a flash — rows arrive top to bottom, so part of
|
||
the screen is always right, and the transfer is 55.2% of a frame slot (61) —
|
||
but at 12 fps that is a colour-scrambled region present for roughly half of
|
||
every frame slot, forever.
|
||
|
||
**And it was LOOKED AT, not only scored.**
|
||
`30_packed_container.py --mismatch-png` writes the frame whose mismatch is
|
||
closest to the mean — chosen that way so the picture is not an outlier picked to
|
||
flatter the number — as correct render | same frame under the next frame's
|
||
palette | 24-bit source:
|
||
|
||

|
||
|
||
**It is chroma speckle and a shifted ground, not a scramble.** Two median-cut
|
||
palettes of adjacent frames occupy a similar gamut, so nothing goes
|
||
psychedelic — the lava turns red where it should be ochre, the dragon's scales
|
||
break into noise, and the torch survives. That is milder than 12.8 dB sounds and
|
||
worse than it looks in a still, because the still does not show it arriving and
|
||
leaving twelve times a second.
|
||
|
||
**So B2 stopped being a headroom question and became a picture question, again
|
||
and worse.** 61.6 said whether buffer mode blanks decides *which player exists*.
|
||
63.4 says that if it does **not** blank, it also decides **which packed
|
||
container** exists, because the per-frame palette is what the artefact is made
|
||
of. The codec never had this exposure: its palette is scene-constant, so its
|
||
tear is old picture against new picture and never old colours against new ones.
|
||
|
||
**And the fallback is already in the encoder, which is the useful half.**
|
||
`pack.py --scene-palette` emits the control row: **30.79 dB, zero palette churn,
|
||
no mismatch to have**, still **+2.07 dB on the shipping codec as the display
|
||
renders both**, and with `--no-palette` alongside it — a scene palette is loaded
|
||
once at scene setup, which is what the codec has always done — the record loses
|
||
its 512 B and the wire is **576.0 KB/s against 582.0**, cheaper on the one
|
||
resource the packed branch is short of. The per-frame palette is worth +2.31 dB and it is now a **priced**
|
||
+2.31 dB rather than a free one.
|
||
|
||
### 63.5 What this does not settle
|
||
|
||
- **Nothing about clocks.** 29_packed_player.py owns those, off the measured
|
||
blit, and nothing here moves them: 55.2% of a frame at the 9 clk/B
|
||
dual-address floor, unchanged.
|
||
- **Nothing about the medium.** 582.0 KB/s is geometry. Whether anything
|
||
sustains it is **B1**, and this container cannot negotiate — a codec's
|
||
bitrate is a lever and a literal frame's is arithmetic.
|
||
- **Nothing on a machine.** No packed frame has been put on screen from a
|
||
container by a 68000 or by a channel. That is **K3**, and 63.2 is the strongest
|
||
statement available without it: the bytes are right, on a second emulator's
|
||
own GVRAM model, with the harness computing nothing.
|
||
|
||
---
|
||
|
||
## 64. The packed player runs end to end off the disc — and the write window is the frame (session 32)
|
||
|
||
**ROADMAP K3.** `src/player/packed.s` is **2,898 bytes** of 68000 code that
|
||
brings up its own display, builds its own 193-entry DMA chain, keeps its own
|
||
frame clock off V-DISP and fetches every record itself with `READ(10)` off a
|
||
CZ-6BS1. `tools/bench/packed.lua` writes **no picture byte, no palette entry and
|
||
no CRTC register**; it pushes the code and eleven mailbox words and then reads.
|
||
`tools/bench/packed_run.sh` is the gate, `tools/bench/verify_packed.py` the
|
||
comparison, `tools/analysis/31_display_duty.py` the arithmetic underneath the
|
||
result.
|
||
|
||
**THE HEADLINE, and it is two facts that point opposite ways.**
|
||
|
||
**120 of 120 frames are pixel-exact**, every one of them compared, off a real
|
||
volume, on a clock the machine keeps itself — the strongest end-to-end result
|
||
this project has. And **the picture was on screen for none of the frame slot it
|
||
belongs to**, because the write window that a packed frame requires is the whole
|
||
of its transfer, and buffer mode blanks the layer it is written through.
|
||
|
||
### 64.1 What was run, and why every frame had to be checked
|
||
|
||
`tools/bench/verify_decode.py` checks **one** frame — the last — and that audits
|
||
all 120, because the codec is temporally recursive: a SKIP block is a claim that
|
||
the previous frame is still in GVRAM. **A packed frame is a LITERAL.** Frame 119
|
||
being right says nothing whatever about frame 60. The simplification that
|
||
deleted the ring, the codebooks and the decoder also deleted the gate's free
|
||
lunch, so `verify_packed.py` snapshots and compares **every frame**, letterbox
|
||
included — the 64 static rows are written once at scene setup and never touched
|
||
again, so a picture can be pixel-exact while the screen is not.
|
||
|
||
| | |
|
||
|---|---|
|
||
| container | `tmp/packed_singe.dlxp`, DLXP1, 120 records of 49,664 B = 97 sectors |
|
||
| record `i` | at LBA `1 + i*97` — **arithmetic**, no index, nothing walked |
|
||
| chain | **193 entries**, built by the 68000: `$E82000`/512 B, then 192 rows of 256 B a 1,024 B stride apart |
|
||
| result | **120 of 120 pixel-exact**, 0 frames unsampled, `FLAG=$FF` |
|
||
| passes with the palette LAST too | chain starts `$C08000`/256 instead, **120 of 120 pixel-exact** |
|
||
|
||
The array is **scene-constant** and the seek is **subtraction**: the packed
|
||
layout spends both 256-colour pages, so there is no page to flip, and a record's
|
||
length is geometry, so a new pass is `LBA0` again. That is the whole of what K3
|
||
deletes, and it deleted it without incident.
|
||
|
||
### 64.2 The write window is the frame, and the rate it needs is not the rate it costs
|
||
|
||
The free-running run — the one that asks for record *i+1* the instant record
|
||
*i* lands, which is what a 12 fps player becomes the moment the transfer is
|
||
longer than the slot — reported a number no budget in this tree has a column
|
||
for: **the write window was open on 99.5% of the host frames.** Every frame was
|
||
pixel-exact and almost none of them was visible.
|
||
|
||
It is **arithmetic, not an emulator artefact**. 256-colour GVRAM masks the high
|
||
byte of every write unless R20 bit 11 is set (46.5/47.1), and the packed
|
||
layout's entire 1.0 B/pixel claim is that one word carries two pixels — so a
|
||
packed write *requires* the bit. If buffer mode blanks the layer while the bit is
|
||
set (47.4/B2 — MAME says it does, 48.1's prior leans that way), the layer is dark
|
||
for exactly as long as the window is open, and for a DMAC-direct player the
|
||
window is open for the whole data phase. There is no second page to hide behind:
|
||
the packed layout spends both, which is the same fact that made a frame one
|
||
channel start (62).
|
||
|
||
dark fraction of a slot = record bytes / (DATA-PHASE rate x slot)
|
||
|
||
**And the rate in that expression is the BURST rate, not the sustained one.**
|
||
This is the correction the session had to make to itself. 582.0 KB/s is a
|
||
*sustained* requirement and it decides whether record *i* arrives before slot
|
||
*i*. The dark fraction is set by how fast bytes move **during the data phase**,
|
||
which for a drive with a read-ahead cache can be several times the sustained
|
||
figure. **They are independent, and a medium can pass one and fail the other:**
|
||
|
||
| requirement | figure | status |
|
||
|---|---|---|
|
||
| sustained, or frames arrive late | **≥ 582.0 KB/s** | B1, known since 63 |
|
||
| data phase, or the frame is never displayed | **see below** | **NEW — B1 has no test for it** |
|
||
|
||
| data phase | transfer | window open | picture on screen |
|
||
|---:|---:|---:|---:|
|
||
| 582.0 KB/s (= the wire) | 83.33 ms | 100.0% | **0.0%** |
|
||
| 700 KB/s | 69.29 ms | 83.1% | 16.9% |
|
||
| 1,164 KB/s | 41.67 ms | 50.0% | 50.0% |
|
||
| 2,131 KB/s | 22.75 ms | 27.3% | 72.7% |
|
||
| 3,000 KB/s | 16.17 ms | 19.4% | 80.6% |
|
||
|
||
**A medium that exactly meets the sustained requirement delivers every frame, on
|
||
time, pixel-exact, and displays none of them.**
|
||
|
||
**AND THIS REVERSES 61.5's RANKING.** There are two packed players, and the
|
||
difference between them is *when* the window is open:
|
||
|
||
- **A, DMAC-direct** (the one that is built): one channel start, the CPU halted
|
||
or nearly, window open for the whole data phase.
|
||
- **B, DMA-to-RAM plus a CPU paint**: the record lands in RAM with the window
|
||
shut, and the 68000 paints it with the packed `movem` blit — **227,553 clocks,
|
||
27.3% of a slot, MEASURED** (`blit.s` V8, 61.4) and **independent of the
|
||
medium**. On screen **72.7% of every slot at any rate that delivers the record
|
||
at all.**
|
||
|
||
They are equally visible at a data-phase rate of **2,131 KB/s, which is 3.7x the
|
||
container's own wire**. Below that — which is every rate anyone has proposed —
|
||
**the player with the CPU in the loop is on screen longer than the one without
|
||
it.** 61.5 is not wrong; it ranked them in **clocks**, and this is the column
|
||
that table does not have:
|
||
|
||
| W (clk per delivered byte) | A: DMAC-direct | B: DMA + CPU paint |
|
||
|---:|---:|---:|
|
||
| 5 | 31.0% | 58.4% |
|
||
| **9 — the dual-address floor** | **54.9%** | **82.2%** |
|
||
| 12 | 72.8% | 100.1% |
|
||
| 16 | 96.6% | 123.9% |
|
||
| 19 | 114.5% | 141.8% |
|
||
|
||
Both charge the audio DMA at 10,417 clocks (1.25%, from the IPL ROM's own
|
||
channel-3 setup, 52.5); neither has a decoder in it. **B costs 27.3% of a frame
|
||
and 99,328 B of RAM** — two record buffers, because at any rate near the wire the
|
||
delivery of record *i+1* occupies most of the slot the paint of record *i*
|
||
happens in. On a 2 MB machine that is 4.7% of memory, and memory is the resource
|
||
the packed branch has spare: the ring it deleted was 256 KB.
|
||
|
||
### 64.3 Holding the bus costs the frame clock half its ticks — and the clock cannot tell
|
||
|
||
`src/player/clock.i` counts V-DISP interrupts. A held channel **halts the
|
||
68000**. The MFP's pending bit is **one bit**, so every edge that falls inside a
|
||
transfer spanning two of them is an edge the machine can **never** count. Nothing
|
||
in this project had run a transfer and a clock at once, so nothing could have
|
||
seen it.
|
||
|
||
| configuration, 120 frames | V-DISP edges seen | host frames drawn | lost |
|
||
|---|---:|---:|---:|
|
||
| **held, paced at 12 fps** | 551 | 1,038 | **487 = 46.9%** |
|
||
| stealing, paced at 6 fps | 1,105 | 1,112 | 7 = 0.6% |
|
||
|
||
**And the player reported ZERO late frames in both.** That is not a reassurance,
|
||
it is the finding: the pace gate compares the frame index against `PACE`, and
|
||
`PACE` is advanced by the ISR the held channel stops the CPU from running — so a
|
||
clock that loses edges loses them from **both sides of the comparison**. The held
|
||
player believed it was running at 12 fps; the screen was at **6.37**. The only
|
||
thing in the run that can contradict it is the host's raster count, which is why
|
||
`packed.lua` reports both and `packed_run.sh` gates on the difference being
|
||
non-zero.
|
||
|
||
The CPU's own account says the same thing from the other end: **held, the 68000
|
||
went round its transfer wait 120 times in 120 frames — once each, meaning it
|
||
never executed during a single transfer.** Stealing, it went round 1,100,520
|
||
times. A player has to keep a clock, read a stick and feed ADPCM; **which of the
|
||
two configurations can do any of that is a design question, and this is the run
|
||
that answers it.**
|
||
|
||
### 64.4 The channel configuration does not set the transport's time
|
||
|
||
Free-running, both configurations delivered the same 49,664 B record within 0.5%
|
||
of each other: **90.72 ms stealing, 90.27 ms held** — 534.6 and 537.3 KB/s,
|
||
108.9% and 108.3% of a 12 fps slot.
|
||
|
||
**That figure is a property of the apparatus and is not `W` and not a medium.**
|
||
MAME's device models carry no transfer timing (`docs/BENCHMARK.md`, 42.5). What
|
||
the pair of runs *does* establish is a **negative that no arithmetic could
|
||
have given**: the transfer time is **not the DMAC configuration's to set**. What
|
||
a channel configuration buys is **who owns the CPU**, not when the picture
|
||
appears. The mechanism behind MAME's own ceiling is **not diagnosed** — it is not
|
||
the DMAC (the two configurations agree) and not the CPU (held, the CPU is halted
|
||
throughout) — and no MAME source tree was available on this machine to name it.
|
||
|
||
### 64.5 What this does not settle
|
||
|
||
- **`W`. Not one clock of it.** Unchanged since 59.
|
||
- **B2, whether a real board blanks in buffer mode.** Everything in 64.2 is
|
||
conditional on it, and the condition now decides **which of two packed players
|
||
is built** rather than how much headroom one has. `probe_bit11_blank.lua` is
|
||
still written and still wants a board.
|
||
- **The data-phase rate of any real medium.** This is the session's addition to
|
||
**B1**, and it is a measurement nobody has planned: throughput and seek time
|
||
were the two numbers on the list, and **the burst rate during a data phase is a
|
||
third** that decides whether a DMAC-direct packed player shows a picture.
|
||
- **Whether B is buildable as described.** It is priced off a measured blit and
|
||
a measured ladder, and no line of it has been written.
|
||
|
||
---
|
||
|
||
## 65. Audio has an encoder, and the packed container's best property is what makes it cost (session 33)
|
||
|
||
**ROADMAP P6**, everything in the item except the bus half session 20 closed.
|
||
`tools/encoder/adpcm.py` is the codec, `tools/encoder/extract_audio.py` the
|
||
extraction, `tools/bench/verify_adpcm.py` the gate, `tools/analysis/32_audio_wire.py`
|
||
the container arithmetic. All of it is **host arithmetic and one emulator
|
||
introspection**; no board ran, and the one MAME experiment that was attempted
|
||
did not work — 65.5 says so rather than leaving it out.
|
||
|
||
### 65.1 There is no reference encoder, so the gate had to be built sideways
|
||
|
||
The X68000's ADPCM is an OKI MSM6258V: 4 bits a sample, two to a byte, 12-bit
|
||
signal word, and three rates that are 8 MHz over 512, 768 and 1024. **ffmpeg has
|
||
a decoder for the format (`adpcm_ima_oki`) and no encoder**, so there is nothing
|
||
to diff an encoder against.
|
||
|
||
What `verify_adpcm.py` gates instead is the thing that can actually be wrong:
|
||
the encoder runs a decoder **inside its own loop** to choose each nibble, and
|
||
that decoder is checked **sample-exact against ffmpeg's** over 4,268 nibbles.
|
||
An encoder that agrees with its own wrong decoder is exactly the failure mode a
|
||
round-trip test cannot see.
|
||
|
||
Two facts fell out of building it, and both were **measured rather than assumed**:
|
||
|
||
| | |
|
||
|---|---|
|
||
| nibble order | **HIGH NIBBLE FIRST** — reading low-first disagrees with ffmpeg on 3,285 of 4,268 samples, and that mismatch is carried as the gate's negative control |
|
||
| the step table | 49 entries, **BUILT** as `floor(16 * 1.1**k)` and checked against the published list, so a transcription slip is not one of the things that can be wrong |
|
||
|
||
On the window this project gates everything on (00223 @539.4 s, 10.000 s, the
|
||
same seconds as `tmp/fr_singe`): **156,250 samples → 78,125 B, SNR 21.97 dB**,
|
||
and 78,125 B / 10.000 s is **7,812.5 B/s to the byte**, which is 52's figure
|
||
arriving from the other direction.
|
||
|
||
**A negative worth having: the level is not a lever.** The disc's window peaks
|
||
at −13.4 dBFS, using 435 of the 12-bit word's 2,048. Normalising it — gain ×4.7,
|
||
one sample clipped — moves the SNR from **21.97 dB to 21.97 dB**. The step
|
||
table's adaptation covers the range, so there is no headroom win to collect and
|
||
no reason to touch the disc's level.
|
||
|
||
### 65.2 The two available references DISAGREE, and it costs 25 dB
|
||
|
||
The delta a nibble contributes has two forms in circulation:
|
||
|
||
'shift' delta = ((2*(n&7) + 1) * step) >> 3
|
||
'terms' delta = step/8 + (n&4)*step + (n&2)*step/2 + (n&1)*step/4,
|
||
each term truncated independently
|
||
|
||
`'shift'` is what ffmpeg computes — **verified sample-exact here, so that is a
|
||
measurement of the decoder that ships, not a reading of its source.** `'terms'`
|
||
is the OKI datasheet's own form, the one a chip builds out of shifts and adds,
|
||
and it is what MAME's `okim6258` is understood to compute. **That last clause is
|
||
NOT verified**: no MAME source tree is on this machine (64.4).
|
||
|
||
They differ **on 1,000 of 4,268 sampled nibbles, by at most 3 in 12-bit units**,
|
||
which reads like something nobody could hear. **It is not.**
|
||
|
||
| encoded | decoded | SNR |
|
||
|---|---|---:|
|
||
| `shift` | `shift` | **21.97 dB** |
|
||
| `shift` | `terms` | **−2.88 dB** |
|
||
| `terms` | `terms` | 21.99 dB |
|
||
| `terms` | `shift` | −3.38 dB |
|
||
|
||
**The noise is louder than the signal.** Per-sample disagreement over the real
|
||
window: **max 257, mean 78.2**, against a source whose RMS is 74.4. A 3-LSB
|
||
formula difference becomes a 25 dB loss because **ADPCM is RECURSIVE** — the
|
||
delta is added to a running predictor and the nibble also moves the step index,
|
||
so a disagreement does not stay where it happens. It is the same shape as the
|
||
codec's temporal recursion, one dimension down: 64.1 used that recursion to make
|
||
one frame audit 120, and here the same property turns a rounding difference into
|
||
a broken stream.
|
||
|
||
**So "which formula does the MSM6258 run" is not a footnote. It is a
|
||
precondition on shipping any audio at all**, and it has to be answered before an
|
||
encoder's output is committed to a container.
|
||
|
||
### 65.3 The interleave, and why the obvious cadence is the wrong one
|
||
|
||
**A packed record is 49,664 B = 97 sectors and its address is `LBA0 + i*97`.
|
||
There is no index and none can be needed** — that is the format's whole claim
|
||
(63, 64.1). Audio is a stream at a rate with no arithmetic relationship to the
|
||
frame rate: at 15,625 Hz a 12 fps slot is **651.0417 B**, and the `.0417` is the
|
||
same remainder the frame clock carries (54), because 8 MHz / 512 / 2 / 12 has a
|
||
3 in the denominator that no power of two clears.
|
||
|
||
Give record *i* the audio belonging to slot *i* and the records become variable
|
||
length — and the moment records are variable length the format needs an index
|
||
and stops being the format. So audio rides a **fixed cadence**: every `F`
|
||
frames, `A` whole sectors, placed between records, leaving
|
||
|
||
LBA(i) = LBA0 + i*97 + floor(i/F)*A
|
||
|
||
which is still arithmetic. Choosing `(F, A)` is a rational approximation to
|
||
`15625/12288 = 1.271565755` from above, and **the obvious cadence is the worst
|
||
point in the space**:
|
||
|
||
| F | A | lump | needs | padding | wire adds | held (2 lumps) |
|
||
|---:|---:|---:|---:|---:|---:|---:|
|
||
| **1** — one lump a record | 2 | 1,024 B | 651.0 B | **57.29%** | **12.00 KB/s** | 2,048 B |
|
||
| 3 | 4 | 2,048 B | 1,953.1 B | 4.86% | 8.00 KB/s | 4,096 B |
|
||
| 7 | 9 | 4,608 B | 4,557.3 B | 1.11% | 7.71 KB/s | 9,216 B |
|
||
| **11** | **14** | **7,168 B** | 7,161.5 B | **0.09%** | **7.64 KB/s** | **14,336 B** |
|
||
| 81 | 103 | 52,736 B | 52,734.4 B | 0.003% | 7.63 KB/s | 105,472 B |
|
||
|
||
**F=11 is the pick.** It buys 57.2 points of padding for 12,288 B of RAM over
|
||
the naive cadence; the floor of the sweep buys the last 0.09 of a point for
|
||
91,136 B more, and on a machine where K4 already wants 99,328 B for two record
|
||
buffers that second trade is not one.
|
||
|
||
**The wire, then:** the packed container's sustained requirement was **582.0 KB/s
|
||
silent** and is **589.6 KB/s with sound** (+1.31%). A literal frame's bitrate is
|
||
geometry and cannot be talked down; the audio on top of it is 7.63 KB/s of
|
||
payload and can only be talked down by choosing a worse chip rate.
|
||
|
||
### 65.4 And this is the first price anyone has found for the packed branch's own simplification
|
||
|
||
The codec container **pays none of it**. `rc_fr_singe_scsi_span.dlx` already
|
||
carries an index and already has variable records (4,096..41,472 B, sector-aligned
|
||
since DLX5), so it can put exactly 651.0417 B of audio in record *i* and pad only
|
||
to the sector it was going to pad to anyway: **zero audio padding**, wire
|
||
440.4 → 448.1 KB/s.
|
||
|
||
"A record's length is geometry, so there is no index and none can be needed" is
|
||
what makes the packed player a page of arithmetic instead of a parser — and it
|
||
is **exactly** the property that makes a second stream at an unrelated rate cost
|
||
a cadence, a padding fraction and a 14,336 B buffer. It is a small cost and it
|
||
is not zero, and **nothing in FINDINGS 61-64 predicted it**. 64's risk list said
|
||
"a simplification that large usually hides something"; this is the first thing
|
||
it hid.
|
||
|
||
The bus half reproduces 52 exactly, which is why the tool prints it: 651.0 B a
|
||
slot at the IPL ROM's own channel-3 cost of 16..19 clk/B is **10,417..12,370
|
||
clocks = 1.25%..1.48% of a slot**. **The interaction 52 could not have had** is
|
||
with 64.2's write window: a DMAC-direct packed player holds the GVRAM window
|
||
open for the whole data phase, so an audio channel stealing the bus during that
|
||
phase makes the phase longer — audio costs **darkness**, not just clocks. It is
|
||
negligible against a dark fraction that is already 1.0, and it is not negligible
|
||
against K4's 27.3% paint. That is the third time this session the two players
|
||
have ranked differently on a column that is not clocks.
|
||
|
||
### 65.5 The experiment that did not work, stated rather than omitted
|
||
|
||
65.2's open question has an obvious apparatus: **MAME's x68000 has the chip**,
|
||
so it can be asked. Introspection got as far as fact and no further, and the
|
||
facts are worth keeping because the next attempt starts from them rather than
|
||
from folklore:
|
||
|
||
| | |
|
||
|---|---|
|
||
| device | **`:okim6258`**, shortname `okim6258` — it is there |
|
||
| registers | **`$E92001`** and **`$E92003`**, each one byte, read out of the maincpu program map by `tools/bench/probe_adpcm.lua` |
|
||
| also on the map | `$E9A000-$E9BFFF`, the PPI that carries ADPCM pan and the clock divider |
|
||
|
||
**Feeding the chip from Lua produced no audio.** `tools/bench/probe_adpcm2.lua`
|
||
writes a control byte to `$E92001` and nibble pairs to `$E92003`;
|
||
`probe_adpcm3.lua` sweeps PPI port C at `$E9A005` over all sixteen low-nibble
|
||
values with a loud burst under each. **Control 0..3 × port C 0..15: MAME's
|
||
`-wavwrite` capture is silent throughout, 0 of 567,360 samples non-zero.**
|
||
|
||
The register *semantics* are the gap — which control value starts playback,
|
||
whether port C needs the PPI's mode word set first, and whether the chip has a
|
||
clock at all until the divider is written. **None of that was guessed at further,
|
||
because guessing at it is how a rig produces a confident wrong answer.** The way
|
||
to do this is from 68000 code with the IPL ROM's own channel-3 DMAC
|
||
configuration, which `tools/analysis/21_iplrom_dmac.py` already reads out of the
|
||
ROM — the real design, and the one path in the machine that is known to be
|
||
correct because Sharp wrote it.
|
||
|
||
### 65.6 What this does not settle
|
||
|
||
- **Which delta formula the chip runs.** 65.2, and it is worth 25 dB.
|
||
- **Anything on a board.** No hardware ran. The `-wavwrite` silence is a
|
||
statement about an apparatus, not about a chip.
|
||
- **The container.** DLXP1 has no audio section; 65.3 is the arithmetic a
|
||
DLXP2 would be built from, and no byte of one has been written.
|
||
- **What audio does to a scene change.** The slack table is here, but 51.3's
|
||
refill climb with a second consumer through a real branch point is not.
|
||
|
||
---
|
||
|
||
## 66. The chip is asked, and it disagrees with the encoder on all four axes — the largest of them is not the one we were worried about (session 34)
|
||
|
||
**ROADMAP P6a.** `src/player/adpcm.i`, `src/player/adpcmgate.s`,
|
||
`tools/bench/prep_adpcm.py`, `tools/bench/adpcm.lua`,
|
||
`tools/bench/adpcm_run.sh`, `tools/bench/verify_adpcm_chip.py`,
|
||
`tools/analysis/33_adpcm_model.py`.
|
||
|
||
**Name the layer.** 68000 code drives the transport; the thing measured is
|
||
**MAME 0.277's `okim6258` device model**, end to end through the machine's real
|
||
DMA path. It settles the **rig** — an emulated audio test encoded against the
|
||
wrong model is 25 dB of nothing — and it does **not** settle the silicon. What a
|
||
real MSM6258V does is still a hardware/datasheet item.
|
||
|
||
### 66.0 A MAME source tree IS reachable from this machine, and 64.4 is struck
|
||
|
||
FINDINGS 64.4 and `tools/encoder/adpcm.py`'s header both record "no MAME source
|
||
tree is on this machine". There is no tree on disk, but the machine has network
|
||
and `raw.githubusercontent.com/mamedev/mame/mame0277/...` fetches. That is how
|
||
this session designed its experiment rather than swept blindly, and it is worth
|
||
recording because two sessions reasoned about MAME's device model as an
|
||
unopenable box when it was one `curl` away.
|
||
|
||
**It does not replace the measurement and did not become one.** The installed
|
||
binary is Ubuntu's 0.277 and the tag is upstream's; whether the two are the same
|
||
bytes is not something a fetch can say. Everything below is read out of a
|
||
capture from the binary that is actually here.
|
||
|
||
### 66.1 The transport is the IPL ROM's own, and it works first time
|
||
|
||
`src/player/adpcm.i` programs HD63450 **channel 3** with the bytes
|
||
`tools/analysis/21_iplrom_dmac.py` decodes out of the IPL ROM at `$FF0C2E` and
|
||
`$FF9A82`: `DCR = $80` (dual address, 8-bit port, cycle steal without hold),
|
||
`SCR = $04`, `MFC = DFC = $05`, `CPR = $01`, `DAR = $E92003`, `OCR = $32`
|
||
(memory→device, byte, **external request**), `CCR = $80`, then command `$02` to
|
||
`$E92001`. 839 bytes moved in **0.1074 s = 7,811.4 B/s** against the format's own
|
||
7,812.5, `CSR = $E0`, `CER = $00`, `MTC = 0`. **This is P6b's transport, not
|
||
scaffolding.**
|
||
|
||
**Two things session 33 could not have guessed and did not have to.** 65.5 fed
|
||
the chip from Lua, got silence, swept control 0..3 × port C 0..15 and stopped.
|
||
Both reasons are ordinary:
|
||
|
||
* **The PPI's port C is an INPUT until it is told otherwise.** ADPCM pan and the
|
||
sample-rate divider are port C bits; an i8255 out of reset has every port an
|
||
input, so writes to `$E9A005` move a latch nothing is reading. Control word
|
||
`$92` first, and only then does `$08` mean *pan both, ÷512*.
|
||
* **`$01` is COMMAND_STOP.** `$02` is PLAY. The sweep that "covered" 0..3 wrote
|
||
`$01` in the probe that swept port C, so the chip was never playing in it.
|
||
|
||
### 66.2 THE HEADLINE: four axes were wrong, and the expensive one is the NIBBLE ORDER
|
||
|
||
The probe is 1,678 nibbles — 16 zero nibbles of prologue, a trigger, an encoded
|
||
sine, then loud bursts — and `verify_adpcm_chip.py` searches **sixteen candidate
|
||
decoder models** (nibble feed × delta formula × clamp × initial accumulator)
|
||
crossed with three capture decimations and a prologue length. **Exactly one
|
||
reproduces the capture, over 1,678 consecutive samples, sample-exact**, and each
|
||
axis carries a negative control: flip it alone and the closest surviving
|
||
alternative disagrees on 826, 1,504, 156 and 1,522 samples respectively.
|
||
|
||
| axis | `adpcm.py` default | **the chip** | cost of getting it wrong ALONE |
|
||
|---|---|---|---:|
|
||
| nibble order | high first | **LOW first** | **−25.74 dB** |
|
||
| delta formula | `shift` | **`terms`** | −2.88 dB |
|
||
| clamp | 12-bit | **10-bit** | 0.00 dB *(on this window)* |
|
||
| accumulator at PLAY | 0 | **−2** | −0.45 dB |
|
||
| **all four at once** | | | **−10.38 dB** against 21.97 |
|
||
|
||
**65.2 named the wrong axis as the risk.** It priced the delta formula at 25 dB
|
||
and left nibble order recorded as "HIGH FIRST, **measured**". That measurement
|
||
was real and it was **against ffmpeg**, i.e. about the Dialogic VOX *file*
|
||
convention — not about what a chip does with a byte written to its data
|
||
register. The two are different questions with different answers, and the one
|
||
this port needs is the second. `verify_adpcm.py` keeps ffmpeg's parameters
|
||
deliberately: a reference check whose reference has been adjusted to agree is
|
||
not a check. `adpcm.CHIP` carries the measured set, and anything that encodes
|
||
**for the machine** passes it explicitly.
|
||
|
||
### 66.3 The clamp is 10-bit, it costs nothing here, and that is the finding that will bite
|
||
|
||
The MSM6258's D/A is 10-bit and the model clamps the **accumulator** there, so it
|
||
is inside the recursion rather than an output scaling. On the Singe window it is
|
||
free — encode for 12 bits or for 10 and the answer is **21.99 dB either way,
|
||
with zero samples on the clamp** — for one reason only: that window peaks at
|
||
**435 of 511**, i.e. **1.4 dB of headroom**, and it is a −13.4 dBFS passage.
|
||
|
||
A 10-bit accumulator is **12.1 dB smaller** than the 12-bit word the encoder was
|
||
clamping to. **65.1's "the level is not a lever" survives downward and is now
|
||
wrong upward**: normalising still buys nothing, and a passage a few dB louder
|
||
than this one does not fit. **Nothing in this project has measured the loudest
|
||
passage on the disc**, so the audio level is an open choice, not a settled one.
|
||
|
||
### 66.4 A rig fact that cost this session most of its time, and is worth the space
|
||
|
||
The 8 MHz ADPCM master clock is **CT1 in the YM2151's port register `$1B`**, in a
|
||
different device from the divider. MAME delivers that write to the ADPCM chip on
|
||
the **sound system's own schedule**, not at the instant of the store — so a
|
||
transfer started in the same breath as the setup plays its first ~17 ms at the
|
||
previous clock. The symptom is specific and misleading: the capture's first
|
||
~130 samples arrive in **exact identical pairs**, the rest do not, and **no model
|
||
fits a stream that changed rate part way through** — which reads exactly like a
|
||
broken probe. `adpcmgate.s` spins ~100 ms after `ad_setup` and says why. A
|
||
player sets its clock once at boot and never meets this.
|
||
|
||
**The general lesson is the one this tree keeps relearning**: a run that fails to
|
||
match is not evidence about the thing being measured until the apparatus has
|
||
been shown to be steady. Three quarters of the diagnosis here was spent
|
||
disproving hypotheses about the *chip* for a symptom that was about the *clock
|
||
write*.
|
||
|
||
### 66.5 What is still open, stated so it is not read as closed
|
||
|
||
* **The silicon.** Every value in 66.2 is MAME's. A real MSM6258V may differ on
|
||
any of the four, and the two published references already disagree on one.
|
||
This is a datasheet or a board, and it is cheap on a board: play a known
|
||
nibble stream and record the line out.
|
||
* **Nothing has played as audio.** The capture is a measurement instrument, not
|
||
a listening test, and no ADPCM has reached a speaker on real hardware.
|
||
* **The encoder is still greedy** (65 risk list, unchanged): exhaustive
|
||
per-sample search, no lookahead. 21.99 dB is this format's floor here.
|
||
* **P6b is now unblocked and its bytes are decided**: DLXP2 must be encoded with
|
||
`adpcm.CHIP`, and 65.3's cadence arithmetic (F=11, A=14, wire 582.0 → 589.6
|
||
KB/s) is untouched by any of this — it is a byte count, and none of the four
|
||
axes changes how many bytes a second the format needs.
|
||
|
||
---
|
||
|
||
## 67. DLXP2 — a packed container with sound in it, and the padding turns out to be drift
|
||
|
||
**ROADMAP P6b. Session 35.** `tools/encoder/dlxp.py` (DLXP2),
|
||
`tools/encoder/pack.py --audio`, `tools/analysis/34_packed_audio.py`,
|
||
`src/player/packed.s` (the third LBA term), `tools/bench/prep_packed.py`,
|
||
`tools/bench/packed.lua`.
|
||
|
||
**NAME THE LAYER.** The container is **host arithmetic**, gated against itself
|
||
and against a silent control. The one thing that ran on the emulated 68000 is
|
||
the *video* consequence: `packed.s` fetches records out of an interleaved
|
||
container and all 120 are still pixel-exact off a real volume. **No audio byte
|
||
has been fed to a chip out of this container**, on any layer — the chip gate
|
||
(66) feeds a designed stream, not this one.
|
||
|
||
### 67.1 The format, and what it costs the player: one `divu` and one `mulu`
|
||
|
||
A DLXP2 is a DLXP1 with a 64-byte header and, from sector 1, **groups**: one
|
||
audio lump of `A` sectors, then `F` records.
|
||
|
||
record i = off_frm + i*rec_bytes + (i//F)*A*512
|
||
lump k = off_aud + k*(F*rec_bytes + A*512)
|
||
|
||
Neither is a lookup. **The format still has no index and still needs none** —
|
||
which was 63/64.1's whole claim for the packed branch, and a second stream at an
|
||
unrelated rate is exactly the thing that could have ended it. The lump goes
|
||
**before** the group it feeds rather than after it (65.3's formula put it after):
|
||
a stream is read forwards, so bytes that arrive after the slot they belong to
|
||
are bytes a player had to fetch early anyway.
|
||
|
||
On the 68000 the third term is **six instructions in `pg_frame`** — a `divu`, an
|
||
`andi` to drop the remainder `divu` leaves in the high half, a `mulu` and an
|
||
`add` — and **zero instructions of parsing**, because the cadence is two numbers
|
||
in the header rather than a table in the stream.
|
||
|
||
**Measured, on the emulated machine, off a real MB89352 volume: 120 of 120
|
||
frames pixel-exact** out of the interleaved container, every one compared. The
|
||
interleave moved no picture byte — asserted against a **silent control** built
|
||
from the same frames with `--audio` off, all 120 records byte-identical.
|
||
|
||
### 67.2 The finding: the payload is not the lump
|
||
|
||
65.3 chose the cadence F=11, A=14 and called the 0.09% "padding". It is padding
|
||
on the wire. **It is drift in the player**, and that is a different thing.
|
||
|
||
A lump is `A*512` = **7,168 B of space**. Eleven frames of audio is
|
||
`11*15625/24` = **7,161.4583… B**. So the *payload* alternates **7,161 and
|
||
7,162** — the same remainder FINDINGS 54's frame clock carries, one dimension
|
||
over — and the last 6.54 B of the sector run are zero.
|
||
|
||
**A player that fed the chip the whole lump** — the obvious implementation, and
|
||
the one the phrase "14 sectors of audio every 11 frames" invites — hands it
|
||
6.54 B a group it should not have. That is **0.84 ms of extra audio every
|
||
0.9167 s**, and it does not average out:
|
||
|
||
| play | lip-sync error |
|
||
|---|---:|
|
||
| 1 min | 0.05 s |
|
||
| **22.8 min (the game)** | **1.25 s** |
|
||
|
||
A second and a quarter is a scene of dialogue arriving after the mouth that
|
||
spoke it. So `lump_bytes(k)` is **part of the format**, not a convenience, and
|
||
what a player carries is the same shape `clock.i` carries and for the same
|
||
reason — a rate whose denominator is 24 cannot be a count:
|
||
|
||
acc += 11*15625 ; = 171,875
|
||
n = acc // 24 ; the MTC for this lump's channel
|
||
acc %= 24
|
||
|
||
**The general shape, and it is the third time this tree has hit it**: 54's frame
|
||
clock, 65.3's `.0417` B a slot, and now this. A ratio with a remainder that is
|
||
rounded *once* is a rounding error; rounded *every period* it is a rate error,
|
||
and a rate error integrates.
|
||
|
||
### 67.3 The four ADPCM axes are in the header, and the gate proves they earn it
|
||
|
||
66 measured which decoder MAME's chip runs and priced getting it wrong at up to
|
||
−25.74 dB. DLXP2 carries all four — nibble order, delta formula, clamp width,
|
||
the accumulator at PLAY — as **three header fields rather than a version
|
||
number**, so a mismatch is legible in a hexdump instead of inferred from a
|
||
container's age.
|
||
|
||
`34_packed_audio.py` decodes the container's own lumps and **flips one axis at a
|
||
time**, which is the negative control that makes the fields load-bearing rather
|
||
than documentation:
|
||
|
||
| axis | header | flipped to | SNR | cost |
|
||
|---|---|---|---:|---:|
|
||
| — | | | **21.99 dB** | |
|
||
| nibble order | low | high | −10.00 dB | **−31.99 dB** |
|
||
| delta formula | terms | shift | −2.87 dB | **−24.86 dB** |
|
||
| clamp | 10 | 12 | 21.99 dB | +0.00 dB |
|
||
| accumulator at PLAY | −2 | 0 | 21.50 dB | −0.49 dB |
|
||
|
||
These are **decode-side** flips on bytes that were encoded correctly, where
|
||
66.2's table encoded and decoded on the wrong model together; the two agree on
|
||
which axes are expensive and disagree by a few dB on how expensive, which is
|
||
what different experiments on the same fact look like. The clamp is still free
|
||
**on this window only** and for 66.3's reason: it peaks at 435 of 511.
|
||
|
||
### 67.4 The failure mode this format has and the codec's does not
|
||
|
||
A DLX record is found through an index, so a player that reads the wrong entry
|
||
gets a length word that does not parse. **A packed record is found by
|
||
arithmetic and nothing parses it.** A player that drops the `(i//F)*A` term
|
||
reads 97 sectors starting 14 sectors early and *paints them*: the tail of the
|
||
previous record, then most of this one, shifted down the screen. It is a
|
||
picture. Nothing errors.
|
||
|
||
And it is invisible where a gate usually looks: **frames 0..10 are byte-identical
|
||
either way**. The gate asserts the whole shape — that a cadence-blind read is
|
||
wrong for exactly records `F..n-1`, 109 of 120, first at frame 11 — rather than
|
||
that some frame differs.
|
||
|
||
The same argument is why the gate **partitions the file** instead of reading
|
||
records back one at a time: an off-by-one that shifts everything after it reads
|
||
back fine record by record. 131 spans, no overlap, no gap, ending exactly at the
|
||
file's last byte.
|
||
|
||
### 67.5 The wire, unchanged from the prediction
|
||
|
||
video 582.0 KB/s geometry, no lever
|
||
audio 7.64 KB/s the cadence's, padding included
|
||
total 589.6 KB/s +1.31%
|
||
|
||
**65.3 predicted 589.6 and the container is 589.6** — which is what a byte count
|
||
should do, since none of 66's four axes changes how many bytes a second the
|
||
format needs. The audio figure charges the padding on purpose: the disc moves
|
||
whole sectors and the wire pays for the zero ones.
|
||
|
||
### 67.6 What is still open
|
||
|
||
* **No audio has been played out of this container**, on any layer. The chip
|
||
gate (66) fed a designed nibble stream through the IPL ROM's channel-3
|
||
configuration; wiring *this* stream to *that* transport, and running it beside
|
||
the video channel, is P6's remaining quarter.
|
||
* **The second consumer has not met a branch point.** 51.3's refill climb with
|
||
audio on the wire is arithmetic in `32_audio_wire.py` and has not been run.
|
||
* **The level is still open downward** (66.3), and the loudest passage on the
|
||
disc is still unmeasured.
|
||
* **The lump buffer is not allocated anywhere.** 65.3 charges 14,336 B for
|
||
double-buffering the cadence and no player holds it.
|
||
|
||
## 68. The container is played — and holding the bus costs the audio 463 times the seam (session 36)
|
||
|
||
**ROADMAP P6c. Session 36.** `src/player/packed.s` (the audio path),
|
||
`src/player/dma.i` (`DM_HOOK`), `src/player/scsi.i`, `tools/encoder/adpcm.py`
|
||
(`decode_state`), `tools/bench/verify_packed_audio.py`,
|
||
`tools/bench/prep_packed.py`, `tools/bench/packed.lua`,
|
||
`tools/bench/packed_run.sh`, `tools/bench/check.sh`.
|
||
|
||
**NAME THE LAYER.** 68000 code fetched an interleaved DLXP2 off a real MB89352
|
||
volume, ran the video path against it, and fed the audio lumps to the machine's
|
||
**MSM6258 as MAME models it** through the IPL ROM's own HD63450 channel 3. The
|
||
capture is MAME's `-wavwrite`. **No board ran**, and every number below is the
|
||
emulated machine's. What is new is that a byte of this container has now been
|
||
**heard** on some layer, which 67.6 recorded that none had.
|
||
|
||
### 68.1 It plays, and the gate is the speaker rather than the counters
|
||
|
||
All **78,125 B** of the container's audio reached the chip, in order, and every
|
||
sample the chip produced from them is exact against `adpcm.py` run with the four
|
||
axes **out of the container's own header** — **156,257 samples**, `signal =
|
||
sample/8` recovering the chip's stream with a worst residual of 1 count.
|
||
|
||
`packed.s` also reports 11 of 11 lumps armed, 11 fetched, 78,125 B of payload,
|
||
**0 starves and 0 re-arms with bytes still outstanding**. Those numbers are not
|
||
the gate and must not be: see 68.4.
|
||
|
||
### 68.2 THE HEADLINE. Held, the bus costs the audio 236 ms of replayed byte
|
||
|
||
The MSM6258 has no FIFO and **no starvation state**: when its DMA channel counts
|
||
out it goes on decoding nibbles out of whatever byte its data register still
|
||
holds (MAME `okim6258::sound_stream_update` reads `m_data_in` unconditionally
|
||
while PLAYING). So the interval between a channel counting out and the CPU
|
||
arming the next lump is not a gap — it is a **held nibble pair driving the
|
||
predictor**, and it is measurable in the capture as bytes that were played for
|
||
more than two nibbles.
|
||
|
||
Same container, same player, same 120 frames, one bit of DCR different:
|
||
|
||
| | stealing | held |
|
||
|---|---:|---:|
|
||
| audio service calls | **250,240** | **369** |
|
||
| ...per frame shown | 2,085 | 3.1 |
|
||
| lump boundaries with a seam | 4 of 10 | **10 of 10** |
|
||
| worst seam | 2 samples, **0.13 ms** | 1,138 samples, **72.83 ms** |
|
||
| total replayed | 8 samples, 0.51 ms | 3,694 samples, **236.42 ms** |
|
||
| ...as a share of the audio | 0.0051% | **2.31%** |
|
||
| bytes wrong | 0 | 0 |
|
||
|
||
**The bytes are identical and the sound is not.** Held is 673x fewer looks at
|
||
the channel and 463x the seam. A 72.8 ms seam is most of a 12 fps frame of one
|
||
byte pair repeating: not a dropout, a buzz, once every 0.92 s.
|
||
|
||
This is FINDINGS 64.3 arriving on the audio side. 64.3 measured a held channel
|
||
costing the frame CLOCK 47% of its ticks *without the clock being able to tell*;
|
||
the same halt costs the audio its service, and the audio can tell — in the only
|
||
place the interval exists, which is the sound.
|
||
|
||
**And it is why the seam is small in the stealing configuration at all.** The
|
||
one place a 68000 driving this video path has time is **inside the transfer
|
||
wait**. Of the 250,240 service calls, **240 are the frame loop's** two-a-frame
|
||
and the other **250,000 came from `DM_HOOK`**, dma.i's per-trip hook. Held, the
|
||
same player gets 369: the same 240, plus the 129 single trips round the wait
|
||
that 131 burst transfers (120 records and 11 lumps) still allow. Servicing the
|
||
audio only from the frame loop is a 90 ms seam by construction, in either
|
||
configuration — the hook is the whole of the difference.
|
||
|
||
### 68.3 The drift is measured now, not derived
|
||
|
||
67.2 computed that feeding the chip the whole 7,168 B lump instead of the
|
||
payload runs 0.09% fast, and priced it at 1.25 s of lip-sync over the game. The
|
||
accumulator (`acc += F*hz; n = acc/(2*fps); acc %= 2*fps`, three lines in
|
||
`pg_apay`) is now in a player and the capture prices what it avoided, over the
|
||
10 lumps that carry a whole group:
|
||
|
||
payload 71,614 B against 71,680 B of lump space
|
||
excess 66 B = 0.092% = 8.45 ms over 9.17 s
|
||
-> 1.26 s over the game's 22.8 min
|
||
|
||
**67.2 predicted 1.25 and the machine played 1.26.** The accumulator ends the
|
||
scene at 1/24, which is where the arithmetic says it should.
|
||
|
||
The gate does not merely *check* the payload lengths, it **cannot pass without
|
||
them**: the walk accounts for every byte of `DLXP.audio()`, and a whole-lump
|
||
player's stream contains 6.54 B of zero at the end of each group that are not in
|
||
that stream at all.
|
||
|
||
### 68.4 Two bugs, and neither is visible from inside the player
|
||
|
||
**(a) Clearing `DM_BARV` does not unchain a channel.** `dma.i` reads `DM_BARV`
|
||
to decide whether to write MAR and MTC, but the register the *channel* obeys is
|
||
**OCR bits 3-2**, which `pg_dmacfg` sets to %10 for the whole scene. So the
|
||
lump fetch wrote MAR and MTC and the channel ignored both and loaded them from
|
||
whatever BAR held (`hd63450.cpp` `dma_transfer_start`). It does not fail as a
|
||
bad address: the SPC's phase never completes and the read returns **POLL
|
||
TIMEOUT — on the lump and then on every record after it**. A wedged transport
|
||
that reads exactly like a disc problem.
|
||
|
||
**(b) The refill overwrote the lump that was playing.** The resident set is
|
||
`AKF - AK + 1` lumps, not `AKF - AK`: `AK` is the *next* lump to arm, so the one
|
||
channel 3 is reading is `AK-1` and its slot is not free. Off by one, the fetcher
|
||
ran three lumps ahead of a three-slot ring and lump 3 landed on top of lump 0
|
||
**while the channel was reading it**.
|
||
|
||
**Every counter in the player stayed right**: 11 of 11 armed, 11 fetched, 78,125
|
||
B of payload, no starve, no late re-arm — and the sound was wrong from **0.2 s**
|
||
in. This is 67.4's failure mode arriving on the audio side. Nothing parses a
|
||
packed container, so a wrong byte is not an error, it is a sound; the only
|
||
instrument that can see it is the stream the chip produced.
|
||
|
||
### 68.5 The rig fact that shapes the gate: a byte is not always two nibbles
|
||
|
||
MAME's `okim6258::data_w` sets `m_data_in` **and resets `m_nibble_shift` to 0**.
|
||
The DMA channel writes at 7,812.5 B/s and the stream decodes at 15,625 Hz, and
|
||
those are two independent clocks — so where a write lands inside a sound-stream
|
||
slice decides whether the byte before it was played as two nibbles, as one (the
|
||
high nibble dropped), or as three or more.
|
||
|
||
Measured over the stealing run: **78,120 of 78,125 bytes were played as exactly
|
||
two nibbles (99.994%)**, one as one, four as four. So a whole-stream
|
||
`decode(data) == capture` cannot be the check, and a tolerance would not be one
|
||
either. The gate's model is one line — *byte `b` was played as `c` nibbles from
|
||
the cycle `(b&15, b>>4)`, `c >= 1`* — and it finds `c` for every byte, requiring
|
||
all `c` samples and 32 samples of continuation to be exact. **Every `c > 2` in
|
||
both runs was the last byte of a lump**, asserted, which is what makes the same
|
||
walk both the correctness check and the seam measurement.
|
||
|
||
This is a property of the emulator's audio scheduling, not of the machine and
|
||
not of the format. On silicon the chip latches a byte and consumes both nibbles.
|
||
|
||
### 68.6 What was NOT established
|
||
|
||
* **Two slots was not tried.** Three lump slots and a prefill of two is what
|
||
ran; the minimum buffering depth is an experiment nobody has done, so 68's
|
||
49,152 B of slot (21,504 B of lump) is an upper bound and not a requirement.
|
||
The slot stride is 16,384 B for a 7,168 B lump to make the address a shift;
|
||
a shipping player should charge `A*512` and pay the `mulu`.
|
||
* **The seam figures are this apparatus's.** They are set by how often the CPU
|
||
looks at the channel, which is set by how long the emulated transport takes to
|
||
deliver a record (90.7 ms against a 12 fps slot of 83.3). The *ratio* between
|
||
held and stealing is the finding; the absolute milliseconds are not `W`.
|
||
* **Nothing has been listened to.** The capture is an instrument. 66's risk is
|
||
unchanged and so is 66.3's: **the level is still open downward and the loudest
|
||
passage on the disc is still unmeasured.**
|
||
* **A second pass does not replay the audio.** `PG_ITER > 1` repeats the video
|
||
and leaves `PG_AK` at `PG_NLUMP`; the run that matters is one pass.
|
||
* **The silicon is untouched.** All four decoder axes are still MAME's, and
|
||
session 34's fifth hardware item — play a known nibble stream on a real
|
||
MSM6258V and record the line out — is still the cheap way to settle them.
|