prosolis
07f36c2af9
ROADMAP K1, the packed player's one open structural item. A frame is a picture
AND a palette, and no run in this tree had pointed a DMA channel at the palette
registers. dmagate.s runs 7-9, gated by dma_run.sh and check.sh:
7. 512 B off the disc into $E82000, bus held -- byte-exact in 256 register
words, read back OUT OF the registers by the 68000;
8. the SAME transfer aimed at RAM -- byte-exact at $2C000, and 256 of 256
palette words still read the poison the CPU wrote, which is what attributes
run 7 to the channel's MAR rather than to the readback path;
9. ONE array-chained start across two kinds of destination -- the palette and
six picture rows at the 1,024 B line stride, 2,048 B byte-exact.
So a packed frame is one channel start: a 193-entry array, palette first, CPU
halted from the first byte to the last. The array is scene-constant, because
the packed layout spends both 256-colour pages and there is no page to flip.
What is left on the CPU per frame in the video path is the channel start and the
READ(10) -- no per-frame PAINT, which is not the same claim as no per-frame CPU.
The destination is POISONED first (62.1). Runs 4-6 wrote into RAM that was zero
and GVRAM that was stale against a record that is mostly pad; "it matches the
disc" was weaker than it read as. The host counts whether the poison actually
discriminates instead of assuming it: 511 of 512, and the gate refuses under 500.
And it opened a hardware item (62.4, ROADMAP B4). MAME maps the palette to
palette_device over memory_array, whose write16 is a plain COMBINE_DATA -- RAM
that honours mem_mask, with no handler that could refuse a byte write. Unlike
GVRAM's 256-colour arm there is nothing here to be wrong about, so the run
bounds the model and not the board. What a real palette register does with a
byte write is unmeasured. A negative costs 0.28% of a frame and nothing else.
29_packed_player.py now also prints the two rows with the per-frame palette
charged -- 55.7% of a frame on the chain, 582 KB/s -- alongside the picture-only
figures the codec comparison is quoted against.
check.sh ALL GREEN before (tmp/check_s30_start.log) and after
(tmp/check_s30_end.log).
Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
6435 lines
344 KiB
Markdown
6435 lines
344 KiB
Markdown
# Findings — session 1 (2026-08-23)
|
||
|
||
All numbers here are MEASURED unless marked ESTIMATE or FOLKLORE.
|
||
|
||
---
|
||
|
||
|
||
> **THE DELIVERY RATE HAS NO WORKING FIGURE — retired session 18 (USER
|
||
> DECISION).** Sections below written before session 18 name a "4 Mbps" pipe
|
||
> constant and score tables against it. **Read every one of those as history.**
|
||
> It was never a bus measurement: user-supplied, no provenance, 10% of SCSI-1's
|
||
> asynchronous rating (FINDINGS 42.1), and FINDINGS 49.5 caught the shipping
|
||
> candidate exceeding it while nothing in the tree was comparing the two.
|
||
>
|
||
> It is now gone as a default from every analysis tool and from
|
||
> `tools/bench/stream.lua` — `--bus` / `--kbps` / `DLX_STREAM_KBPS` are
|
||
> **required arguments** with no fallback, so no table can be scored against a
|
||
> rate its own output does not state. The one survivor is `GATE_SPAN_KBPS` in
|
||
> `tools/bench/check.sh`, which is a **container recipe**, not a delivery claim:
|
||
> the gate container was encoded with it and every per-block and span constant
|
||
> in FINDINGS 41/43/45/49 is fitted to that container, so changing it is a
|
||
> re-encode plus a re-measurement, not an edit.
|
||
>
|
||
> **What to use instead:** `tools/analysis/19_ring_stream.py` reports the
|
||
> **zero-prefill pipe** — the rate a medium must clear for a container to need no
|
||
> prefill. That is a requirement to measure a BlueSCSI against, not a constant to
|
||
> design on. For the session-14 candidate it is **513.2 KB/s**.
|
||
|
||
## 1. Source material
|
||
|
||
`DRAGONS_LAIR.iso` — 16 GB, UDF 2.x, **decrypted** (no AACS dir).
|
||
Loop-mounted read-only at `/media/reala-misaki/BDROM` via `udisksctl loop-setup -r -f`.
|
||
(7-Zip cannot read UDF 2.x; use the loop mount.)
|
||
|
||
- **224 `.m2ts` streams**, 1920x1080, **MPEG-2, progressive, 23.976 fps**
|
||
- Size histogram: 47 <5MB, 138 5-50MB, 22 50-150MB, 14 150-400MB, 3 >400MB
|
||
- The 185 sub-50MB streams are the **arcade branching scenes already split into
|
||
individual clips** — we get scene boundaries for free.
|
||
- Big streams are full-feature playthroughs: 00215 (1376s), 00216 (1151s), 00223 (566s)
|
||
- Typical scene clip ~60s (00203/00205/00199), some ~100s (00164/00212)
|
||
|
||
**Gotcha:** clip durations vary wildly. Always read `format=duration` and seek
|
||
relative to it. Seeking to a fixed offset silently yields 0 frames on short clips.
|
||
|
||
---
|
||
|
||
## 2. GVRAM layout [verified — see HARDWARE.md for source]
|
||
|
||
**One 16-bit word per pixel position in EVERY color mode.** Bit depth does not
|
||
change VRAM bandwidth; it only subdivides the word.
|
||
|
||
`addr = page_base + y*1024 + x*2` — adjacent pixels are 2 bytes apart in all modes.
|
||
|
||
Consequence: low bit depth buys **no speed**. 16-color mode is strictly worse than
|
||
256-color (same bus traffic, 1/16 the palette). Page-alias writes are hardware
|
||
auto-masked, so 16-color needs no software read-modify-write — but it's still
|
||
one word-access per pixel.
|
||
|
||
**Chosen: 256 colors, 256x192 active area.**
|
||
In 256-color mode P0=low byte, P1=high byte of each word. Sacrificing page 1 as a
|
||
double-buffer lets a `move.l` cover two pixel positions, enabling `movem.l` bursts
|
||
(12 regs = 48 bytes = 24 pixels). Identical blit cost to 65536-color mode but
|
||
**half the on-disk data**.
|
||
|
||
---
|
||
|
||
## 3. Content measurements (8 scenes sampled, 5s each at 40% into each clip)
|
||
|
||
| metric | mean | p90 |
|
||
|---|---|---|
|
||
| pixels changed / frame | 20.1% | 30.2% |
|
||
| **blit cost** | **~64k cycles** | **~97k cycles** |
|
||
| naive delta+RLE frame size | 15.5 KB | 19.6 KB |
|
||
|
||
Budget is **833,333 cycles/frame** @ 12fps on a 10MHz 68000.
|
||
|
||
### => THE CPU IS NOT THE BOTTLENECK. I/O IS.
|
||
Blit uses **under 8%** of budget. The naive row-span+RLE codec achieves only
|
||
**3.2:1**, giving **365 KB/s / 470 MB** at 24fps (~183 KB/s / 235 MB at 12fps).
|
||
|
||
Per-scene variance is extreme: static dialogue ~30 KB/s, action ~700 KB/s.
|
||
Any codec needs a hard bitrate ceiling, not just a good average.
|
||
|
||
### "Shot on twos" — ASSUMPTION FAILED
|
||
Dedupe found **zero** duplicate frames across all 8 scenes (`uniq=120/120`,
|
||
24.0 fps effective). This Blu-ray is a restoration where every frame is unique.
|
||
We do NOT get halved data for free. **Decimation to 12fps must be explicit.**
|
||
|
||
A weak alternation signature does exist (even-index pairs 40.7% vs odd 27.5%,
|
||
ratio 1.5x, with occasional true-duplicate pairs at 0.03-0.19%), but it is
|
||
irregular — Bluth mixed ones and twos; action is animated on ones.
|
||
|
||
---
|
||
|
||
## 4. MEASUREMENT TRAPS — read before trusting any pipeline number
|
||
|
||
Three separate false results were produced and caught this session. All three
|
||
looked plausible. Guard against them:
|
||
|
||
1. **Per-frame Floyd-Steinberg dithering destroys temporal coherence.**
|
||
Error diffusion is chaotic: a +/-1 input change cascades across the row and
|
||
produces a completely different index pattern. First run reported 31.5% pixels
|
||
changed with near-zero variance (median 31.6, p90 32.3, max 32.7) while source
|
||
mean-abs-diff was 0.09 — i.e. visually identical frames. That flat variance is
|
||
the tell: **real animation has scene-dependent variance; noise does not.**
|
||
Use no dithering (cel art is flat) or ordered/Bayer (spatially fixed, temporally stable).
|
||
|
||
2. **Temporal denoise smears motion.** `hqdn3d=4:3:6:4` — the `6:4` are temporal
|
||
params. It flattened real motion, which then measured as "no motion" and
|
||
produced an absurd 0.8 fps / 4 MB result. **Use spatial-only: `hqdn3d=4:3:0:0`.**
|
||
|
||
3. **Exact-match dedupe fails on a noisy source.** MPEG-2 grain means near-duplicate
|
||
frames differ by +/-1 and are never bit-exact. Use a threshold on
|
||
"% pixels differing by more than N levels", and pick the threshold from the
|
||
observed distribution, not a guess. A 2% threshold ate genuine animation when
|
||
mean consecutive change was only 0.9%.
|
||
|
||
**Sanity rule: if a result has suspiciously low variance, or is suspiciously
|
||
good, it is probably an artifact of the measurement, not a property of the content.**
|
||
|
||
Scripts kept in `tools/analysis/` — 01 and 02 are marked BROKEN deliberately as
|
||
regression references; 03 and 04 are the correct ones.
|
||
|
||
---
|
||
|
||
## 5. Storage interface — the SASI/SCSI split
|
||
|
||
> **SUPERSEDED IN PART.** The claim below that DMA means streaming "costs
|
||
> essentially no CPU" is **wrong** — see 19. The bandwidth figures here are
|
||
> folklore; the working figure was **4 Mbps = 488 KB/s** (21), and **42.1
|
||
> retires that too**: it was never a bus figure. The 50-pin SCSI-1 5 MB/s below
|
||
> is the correct bus rating. What binds is not the pipe but `W`, the clocks the
|
||
> DMA steals per word — and this section's 8 clk/word is a bracket midpoint,
|
||
> not a measurement (39.7, 42.6).
|
||
|
||
[Yasuma, X68030 internal SCSI controller]
|
||
|
||
- Interface: **SCSI-1**, 50-pin, 5 MB/s bus spec
|
||
- Controller: **Fujitsu MB89352** SPC
|
||
- Transfer mode: **DMA** (via **HD63450** DMAC)
|
||
- Bus: X68000 original bus, **16-bit @ 10MHz**
|
||
|
||
**Even on the X68030, SCSI runs at 10MHz 16-bit DMA.** Storage bandwidth does
|
||
NOT scale with CPU — the controller sits on the original bus. HD63450's 12.5MHz
|
||
official ceiling is why the X68030 runs at 25MHz. An "HSCSI" TSR forces PIO/FIFO
|
||
transfer instead of DMA but was marginal even at 25MHz.
|
||
|
||
Because it's DMA, **streaming costs essentially no CPU** — this stacks with the
|
||
8% blit utilisation. The 68000 really is nearly idle.
|
||
|
||
### Model split — IMPORTANT
|
||
**The 10MHz models (original X68000, ACE, PRO, EXPERT) use SASI, not SCSI.**
|
||
Built-in SCSI starts at the X68000 **Super** (1990) and continues through XVI,
|
||
Compact, X68030. SCSI on earlier machines needs the **Sharp CZ-6BS1** board
|
||
in an I/O slot (MAME models this: `-exp1 cz6bs1`).
|
||
|
||
| target | bandwidth | naive codec (365 KB/s) | VQ codec (~30 KB/s) |
|
||
|---|---|---|---|
|
||
| SASI (stock ACE/EXPERT) | ~300-500 KB/s FOLKLORE | infeasible | comfortable |
|
||
| SCSI (Super+, or CZ-6BS1) | ~1 MB/s FOLKLORE | tight but viable | trivial |
|
||
|
||
Derived bounds (ESTIMATE): 16-bit @10MHz with 4-clock bus cycle = 5 MB/s absolute
|
||
ceiling; HD63450 single-address DMA ~8 clocks/word => ~2.5 MB/s practical ceiling,
|
||
before SCSI-1 async handshake and drive latency.
|
||
|
||
**No measured benchmark was obtained — see STATUS.md.** The ~300-500 KB/s and
|
||
~1 MB/s figures are folklore-grade; I could not find a primary measurement.
|
||
|
||
---
|
||
|
||
## 6. Codec decision: vector quantization (Cinepak-style)
|
||
|
||
Given ~8x CPU headroom and an I/O ceiling, spend CPU to buy bandwidth.
|
||
|
||
- Split frame into 4x4 blocks, encode each as a 1-byte index into a per-scene codebook
|
||
- Decode = 16-byte copy from a lookup table: nearly free
|
||
- A **full** frame = 256*192/16 = **3,072 bytes** — a hard 16:1 floor before delta
|
||
- Add block-level delta on top; action scenes ~2-3 KB/frame
|
||
- => roughly **30 KB/s, ~40 MB total**, with a *deterministic* bitrate ceiling
|
||
|
||
Divergence from the SNES project (below): use a **per-scene codebook with delta
|
||
updates**, not a per-frame rebuild. We trade adaptivity for bandwidth because we
|
||
have 2MB RAM to keep a codebook resident and CPU to spare.
|
||
|
||
**Risk not yet evaluated:** 4x4 VQ with a 256-entry codebook will visibly soften
|
||
detail. Bluth's fine ink linework is what suffers. Prototype and eyeball before committing.
|
||
|
||
---
|
||
|
||
## 7. Comparison: astrobleem/SNES-SuperDragonsLairArcade
|
||
|
||
Reached the **same core architecture independently** — "512 tiles per frame" is
|
||
vector quantization (8x8 codebook + tilemap). Good validation.
|
||
|
||
But: the SNES PPU has **no bitmap mode**, so tiles are forced on them by display
|
||
hardware. The X68000 has a real linear framebuffer, so VQ is a *compression
|
||
choice* we can tune or drop per-scene.
|
||
|
||
**MSU-1 is a bandwidth cheat we don't have.** It's a modern flash-cart coprocessor
|
||
giving memory-mapped streaming the real SNES never had. Their budget: 512 tiles x
|
||
32 bytes (4bpp 8x8) + tilemap ~= 18 KB/frame => **~430 KB/s** at 23.976fps.
|
||
That's *higher* than the 365 KB/s we'd reject on SASI. (ESTIMATE: my arithmetic on
|
||
their stated tile budget, not a measured figure.)
|
||
|
||
Where we're ahead: 256 simultaneous colors from a 65536 palette vs their 4bpp
|
||
sub-palettes needing a tile-aware palette optimizer plus a spatial smoothing pass
|
||
to hide 8x8 palette seams. That problem doesn't exist for us. Plus 68000@10MHz
|
||
vs 65816@3.58MHz, and 2MB vs 128KB.
|
||
|
||
**Most valuable thing in that repo is NOT the codec — it's `data/events/`:**
|
||
516 chapter definitions across 29 scenes as XML, plus
|
||
`data/chapter_event_inventory.md`. That's the arcade scene graph and input-timing
|
||
structure, entirely hardware-independent — the whole game-logic layer we'd
|
||
otherwise reverse-engineer from the arcade ROM.
|
||
|
||
**TODO: check their license before planning to reuse it.**
|
||
Their 516 chapters are finer-grained than our 224 Blu-ray streams, so mapping
|
||
their event table onto our footage means subdividing streams by timecode.
|
||
|
||
Caveat: all of the above is from README/repo-tree summaries, not their source.
|
||
|
||
---
|
||
---
|
||
|
||
# Findings — session 2 (2026-08-23)
|
||
|
||
## 8. CORRECTION to session 1: halving the framerate does NOT halve the bitrate
|
||
|
||
> **PARTLY SUPERSEDED.** The framerate correction stands. The
|
||
> "changed-spans + deflate = 247 KB/s" figure is a **compression upper bound,
|
||
> not a shippable design** — deflate decode does not fit the 68000's frame
|
||
> budget. See 17.2.
|
||
|
||
Session 1 measured 365 KB/s for naive delta+RLE at 24 fps and wrote
|
||
"(~183 KB/s at 12fps)". **That extrapolation is wrong.** Decimating to 12 fps
|
||
roughly doubles the per-frame delta, so the *rate* stays nearly flat.
|
||
|
||
Re-measured directly on 12 fps decimated frames (4 scenes, 66 frames):
|
||
|
||
| codec (all LOSSLESS w.r.t. the 256-colour frame) | B/frame | KB/s @12 | 22 min | ratio |
|
||
|---|---|---|---|---|
|
||
| raw 8bpp 256x192 | 49152 | 576 | 743 MB | 1.0:1 |
|
||
| session 1 row-span + RLE | 29055 | 340 | 439 MB | 1.7:1 |
|
||
| XOR vs prev + deflate | 30196 | 354 | 456 MB | 1.6:1 |
|
||
| **changed-spans + deflate** | **21110** | **247** | **319 MB** | **2.3:1** |
|
||
| changed-spans + LZMA | 18759 | 220 | 283 MB | 2.6:1 |
|
||
|
||
Session 1's own RLE re-measured at 12 fps gives **340 KB/s, not 183**.
|
||
Any plan that assumed 183 KB/s was based on a bad number.
|
||
|
||
Deflate-class entropy coding on top of the span payload is worth **1.4x** over
|
||
hand-rolled RLE, and LZ decode is cheap on a 68000 (byte copies), so the
|
||
lossless floor is ~247 KB/s / 319 MB. That is **infeasible on SASI** and
|
||
**tight but real on SCSI**.
|
||
|
||
## 9. Flat 4x4 VQ at k=256 is NOT acceptable — confirmed by eye
|
||
|
||
The risk flagged in 6 is real. At k=256, 4x4:
|
||
|
||
| scene | palette-only PSNR | after VQ | VQ loss |
|
||
|---|---|---|---|
|
||
| 00010 | 38.35 | 29.68 | 8.67 dB |
|
||
| 00020 | 39.90 | 32.67 | 7.22 dB |
|
||
| 00146 | 35.25 | 29.35 | 5.89 dB |
|
||
| 00181 | 41.92 | 32.87 | 9.05 dB |
|
||
|
||
Visually: Dirk's face disintegrates, teeth and eyes turn to mush, ink outlines
|
||
break into 4-pixel stair-steps, colour bleeds across block boundaries.
|
||
|
||
|
||
*Left: 1080p source. Middle: 256-colour palettised 256x192 — the quality ceiling,
|
||
and it is excellent. Right: flat 4x4 VQ at k=256. This is the result that killed
|
||
the flat-VQ architecture.*
|
||
|
||
**Crucially, the 256-colour palettised frame itself looks excellent.** Flat cel
|
||
art with a per-scene median-cut palette and no dithering is near-transparent
|
||
(35-42 dB). So the palette is not the problem and 256 colours is not the
|
||
problem — **block VQ is**. The quality ceiling we should hold ourselves to is
|
||
the palettised frame, not the 1080p source.
|
||
|
||
## 10. Hybrid VQ (Cinepak V1/V4 + SKIP) — this is the codec
|
||
|
||
Per 4x4 block, choose by rate-distortion: SKIP (reuse previous frame),
|
||
V1 (one 4x4 codeword, 1 byte), or V4 (four 2x2 codewords, 4 bytes),
|
||
with a 2-bit-per-block mode header. `lam` is the lagrangian rate knob.
|
||
|
||
Measured, k1=k4=256, 4 scenes (mean of the per-scene table in the session log):
|
||
|
||
| lam | PSNR | loss vs palette | SKIP% | V1% | V4% | B/frame | KB/s @12 |
|
||
|---|---|---|---|---|---|---|---|
|
||
| 0 (max quality) | 33.9 | 4.9 | 30.8 | 18.5 | 50.8 | 7574 | 88.8 |
|
||
| 200 | 31.9 | 5.9 | 44.0 | 37.6 | 18.4 | 4183 | 49.0 |
|
||
| 1000 | 31.6 | 7.3 | 47.4 | 47.7 | 4.9 | 2841 | 33.3 |
|
||
| 5000 | 25.5 | 13.3 | 55.6 | 44.4 | 0.0 | 2134 | 25.0 |
|
||
|
||
At a **matched ~30 KB/s** the hybrid beats flat 4x4 VQ by ~1 dB, and unlike flat
|
||
VQ it keeps scaling: at 89 KB/s it reaches within **4.9 dB of the palette
|
||
ceiling**, which flat VQ cannot reach at any bitrate.
|
||
|
||
Note V4% collapses to 0 at lam=5000 — that is the knob doing exactly what it
|
||
should: under a hard ceiling, detail blocks are the first thing sacrificed.
|
||
|
||
## 11. Codebook size sweep (flat 4x4, for reference)
|
||
|
||
> **SUPERSEDED.** The k=1024 result below is an artifact of a rate model that
|
||
> charged 1 byte for a 10-bit index. k=256 ships. See 14.
|
||
|
||
| block | k | PSNR | loss | key B | changed% | KB/s @12 | codebook RAM |
|
||
|---|---|---|---|---|---|---|---|
|
||
| 4x4 | 256 | 30.46 | 8.39 | 3072 | 52.7 | 28.5 | 8K |
|
||
| 4x4 | 1024 | 32.89 | 5.96 | 3840 | 56.6 | 35.6 | 32K |
|
||
|
||
+2.4 dB for 24K more RAM and 7 KB/s. With 2 MB of RAM, a 1024-entry codebook is
|
||
cheap and clearly worth it. (RAM figure is the word-expanded form the blitter
|
||
wants: k * 16 px * 2 bytes.)
|
||
|
||
## 12. Source framing — OPEN
|
||
|
||
The Blu-ray is **full-frame 1920x1080 16:9 with no pillarboxing**. The arcade
|
||
original is 4:3. The extractor currently centre-crops 1440x1080, which is the
|
||
arcade-faithful choice but discards image the 2006 remaster added. Options are
|
||
`crop` (default), `squash`, `wide` in `tools/encoder/extract.py`.
|
||
**Not yet decided; needs an eyeball comparison against arcade reference.**
|
||
|
||
## 13. Stream inventory correction
|
||
|
||
Session 1 said "typical scene clip ~60s". Sampled directly: the ~3-5 MB streams
|
||
are **1.2-1.7 s** clips — these are the individual arcade death/action moments,
|
||
which is exactly the granularity the game logic needs. Some 60 s streams
|
||
(e.g. 00203) are **menu screens, not content**. Any survey must classify
|
||
menu vs content before averaging, or the bitrate numbers are diluted by static
|
||
menus.
|
||
|
||
## 14. A FOURTH false-good result — and the correction
|
||
|
||
Add this to the 4 list. The mechanism was new but the shape was identical.
|
||
|
||
**The false result:** flat and hybrid VQ both showed **+2.4 dB for k=1024 over
|
||
k=256** at an apparently similar bitrate, which made a 1024-entry codebook look
|
||
like an obvious win. The k=1024 quality ladder rendered from that run looked
|
||
great at "45 KB/s".
|
||
|
||
**The bug:** the rate-distortion model in `vq_hybrid.encode()` charged **1 byte**
|
||
per codebook index unconditionally. A 1024-entry codebook needs a **10-bit index,
|
||
stored as 2 bytes**. So every k=1024 measurement understated the V1 and V4
|
||
payload by exactly 2x, *and* the lagrangian mode decision was choosing V4 on the
|
||
belief that four codewords cost 4 bytes when they cost 8.
|
||
|
||
**After charging the true index cost** (`idx_bytes` is now explicit and defaults
|
||
from the codebook size), matched-bitrate comparison on scene 00020:
|
||
|
||
| KB/s | k=256 (1-byte idx) | k=1024 (2-byte idx) |
|
||
|---|---|---|
|
||
| ~32-42 | **33.87 dB** @ 32.5 | 28.91 dB @ 42.3 |
|
||
| ~44-52 | **34.80 dB** @ 44.1 | 35.13 dB @ 52.5 |
|
||
| ~72-86 | **35.87 dB** @ 72.2 | 36.51 dB @ 86.0 |
|
||
|
||
k=1024 buys +0.3 to +0.6 dB for +19% bitrate — a wash at best — and at the low
|
||
end where the SASI profile lives it is **5 dB worse**, because the 2-byte index
|
||
floor dominates once V4 is priced out.
|
||
|
||
**k=256 with 1-byte indices is the shipping choice.** It is also the better
|
||
decoder: a plain `move.b` index with no alignment case, and an 8 KB codebook
|
||
instead of 32 KB.
|
||
|
||
**The general lesson, again:** the comparison was not wrong about VQ, it was
|
||
wrong about *cost*. When a knob looks like a free win, check that the rate model
|
||
is charging for it. Same failure family as 4.1-4.3: a plausible number produced
|
||
by a pipeline that was not measuring what it claimed to measure.
|
||
|
||
## 15. Rate-distortion curve of the shipping codec (k=256, corrected)
|
||
|
||
Scene 00020 (Dirk screaming, close-up face — the hardest case for linework),
|
||
and 00146. Includes the 2-bit-per-block mode header. No entropy coding yet.
|
||
|
||
| lam | 00020 PSNR | 00020 KB/s | 00146 PSNR | 00146 KB/s | SKIP | V1 | V4 | RAW |
|
||
|---|---|---|---|---|---|---|---|---|
|
||
| 25 | 38.68 | 182.2 | 31.04 | 193.5 | ~37% | ~24% | ~13% | ~26% |
|
||
| 100 | 35.87 | 72.2 | 29.04 | 72.5 | ~41% | ~34% | ~21% | ~4% |
|
||
| 300 | 34.80 | 44.1 | 28.28 | 44.4 | ~44% | ~42% | ~14% | 0% |
|
||
| 800 | 33.87 | 32.5 | 27.77 | 36.1 | ~46% | ~48% | ~5% | 0% |
|
||
| 2000 | 27.57 | 25.5 | 24.88 | 30.2 | ~50% | ~49% | ~1% | 0% |
|
||
|
||
Palette ceilings: 00020 = 39.90 dB, 00146 = 35.25 dB.
|
||
|
||
|
||
*The shipping codec across the rate knob. Top: source, palette ceiling, lam=25.
|
||
Bottom: lam=100 (`scsi` profile), lam=300 (`sasi` profile), lam=800.
|
||
Both shipping profiles hold Bluth's linework; the failure only starts past lam=800.*
|
||
|
||
Two things to read off this table:
|
||
- **The cliff is between lam=800 and lam=2000.** That is where V4 is priced out
|
||
entirely and detail blocks have nowhere to go. Do not ship past lam~800.
|
||
- **RAW is doing real work at high bitrate** (26% of blocks at lam=25) and
|
||
vanishes by lam=300. It is what makes the top of the curve reach the palette
|
||
ceiling, and it costs the decoder nothing — RAW is the cheapest mode to blit.
|
||
|
||
## 16. Licences cleared for the game-logic layer
|
||
|
||
Both checked this session:
|
||
|
||
- **astrobleem/SNES-SuperDragonsLairArcade — MIT**, "Copyright (c) 2026 Chad
|
||
Doebelin". `data/events/` holds 516 XML chapter definitions with timing and
|
||
event data. Reusable with attribution.
|
||
- **icculus/DirkSimple — zlib.** Independent from-scratch reimplementation of
|
||
the game logic in Lua, scene/timing tables in `game.lua`. Also permissive.
|
||
|
||
Having **two independent permissively-licensed transcriptions** of the arcade
|
||
scene graph is better than one: they can be diffed against each other to catch
|
||
transcription errors before any of it is committed to 68000 tables.
|
||
|
||
## 17. The profiles were set far too low — and entropy coding is a CPU trap
|
||
|
||
> **PARTLY SUPERSEDED.** 17.1's diagnosis (the profiles were not derived from
|
||
> hardware) and 17.2's CPU analysis both stand. But 17 reasoned against a
|
||
> misread bandwidth of 4 MB/s; the correct figure was taken as **4 Mbps =
|
||
> 488 KB/s**, so the "ship pixel-exact if SCSI sustains >=800 KB/s" conclusion
|
||
> in 17.5 was withdrawn. See 18 and 21.
|
||
>
|
||
> **17.5 IS REINSTATED BY 42.3.** The 488 KB/s figure that withdrew it was
|
||
> itself unsourced, and the delivered stream is 837.4 KB/s at 0.29 dB off the
|
||
> palette ceiling — 17.5's threshold and 17.5's conclusion, arrived at from the
|
||
> other end five sessions later. Its reasoning was sound; only its bandwidth
|
||
> number was wrong, and it was wrong in the direction that made it look wrong.
|
||
|
||
Prompted by the user asking why the SCSI profile was only 75 KB/s. It should not
|
||
have been. Two separate errors, one of them serious.
|
||
|
||
### 17.1 The profile bitrates were not derived from the hardware at all
|
||
|
||
They were read off the knee of the rate-distortion curve and then presented as
|
||
though bandwidth-driven. Against the (folklore) bus figures from 5:
|
||
|
||
| profile | was | bus figure | utilisation |
|
||
|---|---|---|---|
|
||
| `sasi` | 45 KB/s | ~300-500 KB/s | **12%** |
|
||
| `scsi` | 75 KB/s | ~1 MB/s | **7%** |
|
||
|
||
Nothing justified leaving 90% of the pipe unused. Raised to `sasi` 110 KB/s
|
||
(lam=60) and `scsi` 280 KB/s (lam=10), which is 35% and 28% utilisation —
|
||
still conservative, because the bus figures are folklore.
|
||
|
||
### 17.2 CPU is NOT the reason to stay low — but entropy coding would be
|
||
|
||
Budget is 833,333 cycles/frame at 12 fps. At session 1's measured ~6.5 cycles
|
||
per GVRAM pixel write:
|
||
|
||
| work | cycles | % of budget |
|
||
|---|---|---|
|
||
| blit 20.1% of pixels (session 1's 24fps figure) | 64k | 7.7% |
|
||
| blit 40% of pixels (the same content at 12fps) | 128k | 15.3% |
|
||
| **blit the FULL frame, every frame** | **319k** | **38.3%** |
|
||
| deflate decode, ~30 KB/frame output | 1,800k | **216%** |
|
||
| LZ4/LZSS decode, ~30 KB/frame output | 450k | **54%** |
|
||
|
||
Two conclusions, and the second one corrects 8:
|
||
|
||
- **Raising the VQ bitrate is nearly free on CPU.** Even a full-frame pixel-exact
|
||
blit fits in 38% of budget, and VQ decode is table copies — RAW, the mode that
|
||
dominates at high bitrate, is the *cheapest* mode to blit, not the dearest.
|
||
- **The 247 KB/s "lossless changed-spans + deflate" figure in 8 is a compression
|
||
upper bound, NOT a shippable design.** Deflate's Huffman decode is bitwise and
|
||
costs about 2.2x the entire frame budget on a 68000. Even byte-oriented LZ4 at
|
||
54% leaves too little beside a 38% blit. **Do not plan on entropy coding.**
|
||
All profile bitrates are raw payload.
|
||
|
||
This inverts session 1's "the CPU is idle, I/O is the ceiling" for the *decode*
|
||
path specifically: the blit is cheap, but any bit-oriented decompressor is not.
|
||
VQ is the right architecture precisely because its decode is a table copy.
|
||
|
||
### 17.3 The hybrid at lam=0 IS the lossless codec
|
||
|
||
Measured, un-entropy-coded raw payload, and deflated for reference only:
|
||
|
||
| scene | lam=0 raw | lam=0 deflated | lossless changed-spans+deflate | PSNR |
|
||
|---|---|---|---|---|
|
||
| 00020 | 442.1 KB/s | 274.5 KB/s | 267.3 KB/s | 39.90 = ceiling |
|
||
| 00146 | 467.6 KB/s | 223.2 KB/s | 219.1 KB/s | 35.25 = ceiling |
|
||
|
||
The hybrid at `lam=0` converges to within 3% of the purpose-built lossless coder.
|
||
That confirms the architecture unifies: there is no separate lossless path to
|
||
maintain, just the same bitstream with the knob open.
|
||
|
||
### 17.4 Full curve in raw (shippable) bytes
|
||
|
||
| lam | 00020 PSNR | 00020 KB/s | 00146 PSNR | 00146 KB/s | RAW% |
|
||
|---|---|---|---|---|---|
|
||
| 0 | **39.90** (exact) | 442.1 | **35.25** (exact) | 467.6 | ~76% |
|
||
| 10 | 39.38 | 248.1 | 32.27 | 305.2 | ~44% |
|
||
| 25 | 38.68 | 182.2 | 31.04 | 193.5 | ~26% |
|
||
| 60 | 36.94 | 108.0 | 29.61 | 103.1 | ~10% |
|
||
| 150 | 35.31 | 55.6 | 28.63 | 56.1 | ~1% |
|
||
| 300 | 34.80 | 44.1 | 28.28 | 44.4 | 0% |
|
||
|
||
### 17.5 This makes the blocked disk benchmark critical-path
|
||
|
||
Session 1 judged it "NOT on the critical path" because VQ at 30 KB/s was correct
|
||
whether SASI did 300 or 600 KB/s. That reasoning no longer holds. The profiles
|
||
now sit at 110 and 280 KB/s, close enough to the folklore ceilings that the
|
||
error bars matter, and **if SCSI sustains >=800 KB/s the correct `scsi` profile
|
||
is lam=0 — pixel-exact video.** Whether this port ships transparent or lossy on
|
||
SCSI is now waiting on one measurement.
|
||
|
||
## 18. Peak-to-mean burstiness — the mean was hiding the problem
|
||
|
||
> **SUPERSEDED — DO NOT ACT ON THIS SECTION.** The peak-vs-sustained comparison
|
||
> below is the **wrong test**. With a ring buffer the correct test is cumulative
|
||
> demand vs cumulative supply, and both profiles pass it with **zero required
|
||
> prefill**. `scsi` at lam=10 ships. See 21. The per-frame peak numbers
|
||
> themselves are still valid data; only the conclusion drawn from them is wrong.
|
||
|
||
Prompted by the user clarifying that the bandwidth figure is **4 Mbps = 488 KB/s**,
|
||
not 4 MB/s. That is ~8x tighter than what 17 was reasoning against, and it
|
||
changes the answer.
|
||
|
||
Per-frame instantaneous rate (video + 7.8 KB/s audio), 12 fps:
|
||
|
||
| scene | lam | mean | p90 | **max** | peak/mean | max as % of 488 KB/s |
|
||
|---|---|---|---|---|---|---|
|
||
| 00010 | 60 | 95.0 | 127.3 | 138.8 | 1.46 | 28.4% |
|
||
| 00010 | 10 | 198.9 | 266.1 | 284.0 | 1.43 | 58.2% |
|
||
| 00020 | 60 | 115.8 | 155.4 | 222.3 | 1.92 | 45.5% |
|
||
| 00020 | 10 | 255.9 | 391.2 | **470.8** | 1.84 | **96.4%** |
|
||
|
||
**The `scsi` profile as committed in f0f2f80 does not fit 4 Mbps.** Its mean is a
|
||
comfortable 52% of the pipe, but it peaks at 96.4% — and a frame that arrives
|
||
late is a *dropped frame*, not a slow one. Sizing a real-time stream on the mean
|
||
is the mistake; peak/mean is 1.4-1.9x on 1.2-1.7s clips and will be worse across
|
||
a full scene.
|
||
|
||
Two ways out, and only one is good:
|
||
- Size for the peak: `lam=25`, mean 194 KB/s. Costs a full step of quality.
|
||
- **Rate-control to the mean and carry a leaky bucket:** `lam=10` fits, and buys
|
||
back +0.7 dB (00020) / +1.2 dB (00146).
|
||
|
||
`ratectl.py` was written in session 2 but **never wired into `encode.py`**. This
|
||
demotes that from a loose end to the highest-value unfinished work in the repo.
|
||
|
||
## 19. Cycle-stealing DMA is not free DMA — 5 was wrong
|
||
|
||
FINDINGS 5 concluded "because it's DMA, streaming costs essentially no CPU —
|
||
this stacks with the 8% blit utilisation. The 68000 really is nearly idle."
|
||
|
||
The HD63450 steals bus cycles from the 68000 at roughly 8 clocks per 16-bit word:
|
||
|
||
| stream | words/s | clocks/s | CPU stolen | + full-frame blit |
|
||
|---|---|---|---|---|
|
||
| 110 KB/s | 56,320 | 450,560 | 4.5% | 42.8% |
|
||
| 250 KB/s | 128,000 | 1,024,000 | 10.2% | 48.5% |
|
||
| 450 KB/s | 230,400 | 1,843,200 | 18.4% | 56.7% |
|
||
| 488 KB/s | 249,856 | 1,998,848 | 20.0% | 58.3% |
|
||
|
||
At the rates the profiles now use, streaming costs **10-20% of the machine**.
|
||
Still affordable — nothing here breaks — but **bandwidth and CPU are one budget,
|
||
not two**, and any future headroom argument has to spend from both. The
|
||
"nearly idle" framing should not be reused.
|
||
|
||
(The 8 clocks/word figure is session 1's ESTIMATE from HD63450 timing, not a
|
||
measurement. It is the weakest link in this table.)
|
||
|
||
## 20. Where the profiles should come from
|
||
|
||
`tools/encoder/profile_gen.py` now derives lam from a bandwidth figure rather
|
||
than from the shape of the RD curve, accounting for audio, peak/mean, and
|
||
reporting DMA steal. Full benchmarking methodology — and why MAME cannot answer
|
||
the bandwidth question — is in `docs/BENCHMARK.md`.
|
||
|
||
The 4 Mbps figure itself is **user-supplied and its provenance is not recorded**.
|
||
Every profile now hangs off it, so it is worth pinning down.
|
||
|
||
## 21. Correction to 18 — the peak test was the wrong test
|
||
|
||
18 flagged that `scsi` "does not fit 4 Mbps" because a frame peaked at 96.4% of
|
||
the sustained rate. **That was the wrong comparison**, and the user was right to
|
||
push back. It measured instantaneous frame demand against a sustained rate as if
|
||
they had to match frame-by-frame. They do not: the disk keeps filling *during*
|
||
the frame, and any shortfall is absorbed by a ring buffer.
|
||
|
||
The correct test is whether **cumulative** demand ever outruns cumulative supply.
|
||
Simulated at a constant 488 KB/s fill, 12 fps, using the real per-frame sizes:
|
||
|
||
| scene | lam | mean KB/s | worst frame | **required prefill** | stall tolerance @256KB |
|
||
|---|---|---|---|---|---|
|
||
| 00010 | 10 | 198.9 | 23.67 KB | **0.0 KB** | 15.4 frames |
|
||
| 00020 | 10 | 255.9 | 39.23 KB | **0.0 KB** | 12.0 frames |
|
||
| 00146 | 10 | **313.0** | 42.10 KB | **0.0 KB** | 9.8 frames |
|
||
| 00181 | 10 | 211.1 | 25.25 KB | **0.0 KB** | 14.6 frames |
|
||
| (all) | 60 | 95-116 | 11-19 KB | **0.0 KB** | 26-32 frames |
|
||
|
||
Fill delivers **40.69 KB per frame time**. Only one measured frame exceeds that
|
||
(00146, 42.10 KB) and it is recovered by the following frame. **No scene needs
|
||
any prefill at all**, and a 256 KB buffer — 12.5% of RAM — carries ~1 second of
|
||
stall tolerance at `lam=10`, which is orders of magnitude more than an SD-backed
|
||
seek requires.
|
||
|
||
`scsi` at `lam=10` stands. The hardest scene sampled (00146) runs 313 KB/s mean,
|
||
64% of the pipe, with zero underrun risk.
|
||
|
||
### Why SD-backed changes the sizing rule
|
||
The deployment target is BlueSCSI / SCSI2SD, not a period spinning drive. That
|
||
was noted as a caveat in 5 but its consequence was not carried through:
|
||
|
||
- The sustained rate is a **bus-limited constant**, not an average over variable
|
||
seek latency. There is no long tail to leave margin for.
|
||
- Seek is ~microseconds, so branch-point stalls are a non-issue against a buffer
|
||
measured in whole seconds.
|
||
- Therefore we can size much closer to the ceiling than spinning-disk practice
|
||
would suggest. Conservative margins here are cargo-culted from a constraint
|
||
this deployment does not have.
|
||
|
||
**The SASI/SCSI split is about BUS PROTOCOL, not media.** SD emulation removes
|
||
seek latency from both, but a SASI bus is still slower than a SCSI one. Two
|
||
profiles remain the right design; both are now predictable constants rather than
|
||
distributions.
|
||
|
||
### What rate control is actually for now
|
||
Its value drops from load-bearing to **insurance**. Intra-scene peaks are a
|
||
non-problem. But we have measured **4 clips of 1.2-1.7s** out of 224 streams, and
|
||
00146 already runs 23% hotter than 00020. A sustained action sequence could
|
||
plausibly exceed the pipe where a 1.7s clip does not. Rate control gives a
|
||
*deterministic ceiling* across content we have not measured yet — which was the
|
||
original reason for choosing VQ over a lossless delta in the first place.
|
||
|
||
Still worth wiring in. No longer a blocker for shipping `scsi` at `lam=10`.
|
||
|
||
## 22. The display path, measured — first real frame on the X68000
|
||
|
||
Everything before this section was Python-side or a headless `-video none` run.
|
||
This is the first time pixels reached an emulated X68000 screen, and it produced
|
||
four hardware facts and one blocker that no amount of reasoning would have found.
|
||
|
||
**Scope — read this before quoting the result.** The X68000's *video* hardware
|
||
did the rendering: CRTC, GVRAM page decoding and the 15-bit+I palette lookup are
|
||
all genuinely emulated, which is why the output is bit-exact against the
|
||
hardware's colour math. But the pixels were written into GVRAM by a MAME Lua
|
||
script calling `SP:write_u16()` — the host poking emulated memory. **No 68000
|
||
instruction was executed to draw this frame.**
|
||
|
||
The equivalent is proving a framebuffer works by writing to it from a debugger.
|
||
It says the display path is correct; it says *nothing* about whether the 68000
|
||
can fill that framebuffer in time. Lua writes cost zero 68000 cycles, so the 38%
|
||
full-frame blit estimate that the entire CPU budget rests on remains completely
|
||
unvalidated. That is next step (2), the decoder skeleton, and it is untouched.
|
||
|
||
Reproduce:
|
||
```
|
||
python3 tools/bench/prep_frame.py <framedir> tmp/frame.bin 0
|
||
cd tmp && SDL_VIDEODRIVER=dummy mame x68000 -bios ipl10 -video soft -window \
|
||
-sound none -nothrottle -plugins -autoboot_script ../tools/bench/show_frame.lua \
|
||
-snapshot_directory ./snap -snapview native -seconds_to_run 6
|
||
```
|
||
|
||
### 22.1 The blocker: CRTC R20 bit 11 hides the graphics layer
|
||
The IPL leaves **CRTC R20 (`$E80028`) = `0x0B16`**. Bit 11 is *"G-VRAM set to
|
||
buffer"*, and MAME's `x68k_v.cpp` bails out of `draw_gfx()` on it outright:
|
||
|
||
```c
|
||
if (m_crtc->gfx_layer_buffer()) // if graphic layers are set to buffer, they aren't visible
|
||
return false;
|
||
// x68k_crtc.h: bool gfx_layer_buffer() const { return BIT(m_reg[20], 11); }
|
||
```
|
||
|
||
While that bit is set, GVRAM writes still land and read back correctly — which
|
||
is exactly what makes it so misleading. Six separate attempts at the video
|
||
controller (`$E82400/$E82500/$E82600`) rendered black with every register
|
||
reading back the intended value. **The video controller was never the problem.**
|
||
|
||
`R20` bits 9-8 select the colour setup, and this determines how `$C00000` is
|
||
decoded: `0x0300` = 65536c (16 bits/word), `0x0100` = 256c (low byte),
|
||
`0x0000` = 16c (4 bits). Set `R20 = 0x0116` for our mode.
|
||
|
||
### 22.2 Monitor contrast: the IPL leaves it at 14, not 15
|
||
`$E8E001` bits 3-0 are monitor contrast; MAME does
|
||
`m_screen->set_brightness(contrast * 0x11)`. The IPL leaves it at **14**, which
|
||
scales all output to 14/15 = 93.3%. Every rendered colour came out ~7% dark
|
||
until this was set to 15. **The player must write `$E8E001 = 15` at startup.**
|
||
|
||
Contrast `0` blanks the screen entirely (`x68k_v.cpp:661`) — that is the cheap
|
||
fade-to-black for scene transitions, no palette animation required.
|
||
|
||
### 22.3 Palette format CONFIRMED (was previously an assumption)
|
||
`PALETTE(config, m_gfxpalette).set_format(2, &x68k_state::GGGGGRRRRRBBBBBI, 256)`
|
||
|
||
```
|
||
bit 15..11 10..6 5..1 0
|
||
GGGGG RRRRR BBBBB I <- I is a shared LSB for all three channels
|
||
```
|
||
Expansion is `pal6bit((field << 1) | I)`, i.e. `(v << 2) | (v >> 4)`.
|
||
With contrast at 15, **all 256 entries render exactly as this predicts** — the
|
||
frame is pixel-identical, not merely close. GVRAM line stride is confirmed as
|
||
512 words = 1024 bytes, matching `HARDWARE.md`.
|
||
|
||
### 22.4 A new quality ceiling: the 15-bit palette costs 38.88 dB
|
||
> **Superseded by 23.3.** The 38.88 dB figure assumed the shared LSB `I` is
|
||
> always 1. Choosing `I` per palette entry by minimum error lifts the ceiling to
|
||
> **40.81 dB** on the same frame. The conclusion below ("`scsi` is close to
|
||
> display-transparent") is therefore weaker than stated — there is ~2 dB more
|
||
> headroom than this section claims.
|
||
Section 3 called the 256-colour palettised frame "the real quality ceiling".
|
||
That was measured in 24-bit RGB. The hardware palette only stores 5 bits per
|
||
channel plus a shared LSB, so there is a **second** quantisation below it:
|
||
|
||
| stage | PSNR |
|
||
|---|---|
|
||
| 24-bit palettised source -> X68000 15-bit+I display | **38.88 dB** |
|
||
| `scsi` profile codec error (00020, FINDINGS 15) | 39.4 dB |
|
||
|
||
The codec's error at `scsi` is **the same order as the display's own error**.
|
||
On real hardware `scsi` is therefore close to display-transparent, and pushing
|
||
`lam` below 10 buys quality the monitor cannot show. This bounds how much the
|
||
`scsi` profile is worth raising — it does not change the profiles themselves.
|
||
|
||
Caveat: measured on one frame (00020 f0001). It is a property of the palette,
|
||
not the content, so it should generalise, but it has not been checked across
|
||
scenes.
|
||
|
||
### 22.5 Why the first frame appears twice
|
||
GVRAM is a 512-pixel-wide page while the IPL's CRTC is still in its 768-wide
|
||
text timing, so the layer repeats at exactly x=512. This is correct hardware
|
||
behaviour, not a bug. The player sets its own CRTC mode and the wrap disappears.
|
||
No CRTC timing table has been written yet — the harness deliberately keeps the
|
||
IPL's timing so that no invented CRTC values are in play.
|
||
|
||
|
||
## 23. A real CRTC mode: 256x192 inside 256x256 (session 4)
|
||
|
||
Session 3's harness borrowed the IPL's 768x512 text timing and invented no CRTC
|
||
values, which is why the frame repeated at x=512 (22.5). This session derived a
|
||
real 256x256 mode table from MAME 0.277 source and verified it by snapshot.
|
||
Table: `tools/bench/crtc_mode.lua`. Regression test: `tools/bench/verify_frame256.py`.
|
||
|
||
|
||
|
||
*Left: palettised source. Right: the emulated X68000's native 256x512 raster —
|
||
256 dots wide, 512 scanlines carrying 256 double-scanned graphics rows, with the
|
||
192-row picture letterboxed in true black.*
|
||
|
||
### 23.1 The table, and why it needed no guessing
|
||
`refresh_mode()` in `x68k_crtc.cpp` selects the dot clock as
|
||
`(reg20 bit4 ? 69.55199MHz : 38.86363MHz) / div`, with `div` from a ladder keyed
|
||
on `reg20 & 0x1f`. Three entries matter:
|
||
|
||
| `reg20 & 0x1f` | div | dot clock | mode |
|
||
|---|---|---|---|
|
||
| `0x16` | 2 | 34.776 MHz | IPL's 768 wide, 31.5kHz |
|
||
| `0x11` | 3 | 23.184 MHz | 512 wide, 31.5kHz |
|
||
| `0x10` | 6 | 11.592 MHz | **256 wide, 31.5kHz, graphics double-scanned** |
|
||
|
||
The IPL's `R00 = 137` gives `m_htotal = (137+1)*8 = 1104` dots, and
|
||
`34.776e6 / 1104 = 31500.0 Hz` **exactly**. Holding the same line rate at div 6
|
||
needs `11.592e6 / 31500 = 368` dots `= 46` chars, so `R00 = 45`.
|
||
|
||
`368 = 1104/3` exactly, so every horizontal register is the 768-mode value
|
||
divided by three, and the active window divides without remainder:
|
||
`(124-28)/3 = 32` chars `= 256` dots. **No horizontal value was recalled or
|
||
estimated.** Only the blanking split rounds: the 768 mode is sync/back/front =
|
||
14/14/14 chars, `/3 = 4.67` each, and the closest integer triple summing to
|
||
`46-32 = 14` is 5/5/4.
|
||
|
||
| reg | value | meaning |
|
||
|---|---|---|
|
||
| R00 | 45 | H total, 46 chars = 368 dots -> 31500.0 Hz |
|
||
| R01 | 5 | H sync end (3.45 us) |
|
||
| R02 | 10 | H display begin -> `hbegin = 81` |
|
||
| R03 | 42 | H display end -> `hend = 336`; inclusive width `336-81+1 = 256` |
|
||
| R04 | 567 | V total, 568 scanlines -> 55.46 Hz |
|
||
| R05 | 5 | V sync end |
|
||
| R06 | 40 | V display begin -> `vbegin = 41` |
|
||
| R07 | 552 | V display end -> 512 scanlines = 256 double-scanned rows |
|
||
| R08 | 27 | H sync adjust (MAME stores it and never reads it) |
|
||
| R20 | `0x0110` | display (not buffer), 256-colour, 31.5kHz, 256x256 |
|
||
|
||
**The vertical registers are NOT halved**, which is the one thing that looks
|
||
wrong and is not. The CRTC still generates a 568-line raster; "256 lines" is a
|
||
graphics-layer double-scan applied in `draw_gfx()` (`x68k_v.cpp:401`), not a
|
||
change to the raster. Halving R04 would ask the monitor for 110 Hz. MAME emits
|
||
a `visarea larger then reg[20]` logerror for this; it is cosmetic.
|
||
|
||
Total blanking time is identical to the 768 mode (112 dots at 11.592 MHz =
|
||
336 dots at 34.776 MHz = 9.66 us), which is the property a real monitor cares
|
||
about — so this table should be safe on hardware, though that is untested.
|
||
|
||
### 23.2 MAME's double-scan is phase-shifted by one raster line
|
||
`get_gfx_pixel()` indexes `m_gfxbitmap.pix(scanline / divisor, pixel)` using the
|
||
**absolute** scanline, and `vbegin = 41` is odd. So in the native 256x512
|
||
snapshot the identical row pairs are `(1,2), (3,4), ...` and row 0 is a lone
|
||
half-line. Even rows are graphics rows 0..255. This cost a false failure before
|
||
it was understood; the regression test now asserts the shifted pairing
|
||
explicitly so a change in MAME's behaviour is visible rather than confusing.
|
||
|
||
### 23.3 The shared LSB `I` must be chosen per palette entry — worth 1.96 dB
|
||
Session 3's `pack()` hardcoded `I = 1`. That is not free: `I` is shared by all
|
||
three channels and each renders as `pal6bit((field << 1) | I)`, so with `I = 1`
|
||
the darkest reachable value is `pal6bit(1) = 4`, and **true black does not
|
||
exist**. Choosing `I` per entry to minimise summed squared error over R,G,B:
|
||
|
||
| rule | ceiling vs 24-bit palettised (00020 f0001) | entries with I=0 |
|
||
|---|---|---|
|
||
| `I = 1` fixed (session 3) | 38.85 dB | 0 |
|
||
| `I` per entry, min squared error | **40.81 dB** | 102 / 256 |
|
||
|
||
Nearly **2 dB for free**, and 102 of 256 entries want `I = 0` — this is not a
|
||
corner case. It supersedes the ceiling in 22.4 and means `scsi` has about 2 dB
|
||
more headroom before it hits the display than that section claimed.
|
||
|
||
The encoder does not yet do this. `tools/encoder/` still emits 24-bit palettes
|
||
and the packing happens Lua-side; whatever eventually writes X68000 palette
|
||
words must use the per-entry rule.
|
||
|
||
### 23.4 Letterboxing requires a reserved black palette entry
|
||
GVRAM cleared to zero displays **palette entry 0**, and a free mediancut palette
|
||
puts a real image colour there — on 00020 f0001 it was `(206,192,176)`, used by
|
||
210 image pixels, so it cannot simply be repurposed. A 256x192 picture in a
|
||
256x256 mode has 64 blank rows, so the palette must be built with **255 colours
|
||
plus a reserved black at index 0** (`prep_frame.py --reserve-black`). Combined
|
||
with 23.3, entry 0 also needs `I = 0` or the bars sit at RGB (4,4,4).
|
||
|
||
Cost: one of 256 entries. Measured quality effect: none visible — the ceiling
|
||
figure in 23.3 is already measured on the 255-colour palette.
|
||
|
||
### 23.5 What is still not proven
|
||
GVRAM was again filled from Lua. **No 68000 instruction has drawn a pixel yet**,
|
||
and the 38% full-frame blit estimate underpinning the CPU budget remains
|
||
unvalidated. What this section adds is that the *target mode* is now real, so
|
||
68000 code has a defined geometry to write into: 256 words per visible row, a
|
||
1024-byte line stride, and rows 32..223 of a 256-row page.
|
||
|
||
---
|
||
|
||
## 24. The blit, measured on the 68000 — the 38% estimate was wrong (session 5)
|
||
|
||
**The first 68000 instructions in this project to draw a pixel.** Everything in
|
||
22 and 23 was GVRAM filled from Lua, which costs zero 68000 cycles. This section
|
||
replaces the estimate that the whole CPU budget rested on with a measurement.
|
||
|
||
Harness: `tools/bench/blit.s` + `tools/bench/blit.lua`. Four variants of a
|
||
full-frame 256x192 paint, each looped to run ~4 emulated seconds, timed from
|
||
`machine.time` between two flag writes by the 68000 itself.
|
||
|
||
| variant | what it does | cycles/frame | % of a 12fps frame |
|
||
|---|---|---:|---:|
|
||
| **V1** | `movem.l` blit from a word-expanded RAM frame (96KB read + 96KB write) | **446,286** | **53.6%** |
|
||
| V2 | naive `move.b`/`move.w` per pixel from a byte source | 1,284,174 | 154.1% |
|
||
| **V3** | write-only floor — registers preloaded, no source read at all | **225,789** | **27.1%** |
|
||
| **V4** | the same 96KB of writes issued in **4x4 block order** | **637,971** | **76.6%** |
|
||
|
||
The 12fps budget is 833,333 cycles (10.0 MHz confirmed from `x68k.cpp:1133`,
|
||
`40_MHz_XTAL / 4`).
|
||
|
||
### 24.1 The numbers are cross-checked against hand-derived cycle counts
|
||
Every variant was predicted from the MC68000 timing tables *before* the run
|
||
(`MOVEM.L` M->R `(An)+` = 12+8n, `(d16,An)` = 16+8n; R->M `(An)` = 8+8n,
|
||
`(d16,An)` = 12+8n) and then measured:
|
||
|
||
| | predicted | measured | error |
|
||
|---|---:|---:|---:|
|
||
| V1 | 447,744 | 446,286 | 0.33% |
|
||
| V2 | 1,284,096 | 1,284,174 | 0.006% |
|
||
| V3 | 225,792 | 225,789 | 0.001% |
|
||
| V4 | 640,704 | 637,971 | 0.43% |
|
||
|
||
This agreement is the point. A MAME timing number on its own would be worth
|
||
little given how many false-good results this project has produced (FINDINGS 4);
|
||
two independent derivations landing within half a percent is worth something.
|
||
The residual error is the frame-granularity of the measurement — Lua gets no
|
||
cycle counter (`luaengine.cpp` exposes `machine.time` and nothing from
|
||
`device_execute_interface`), so timing resolution is one video frame, 18.03 ms.
|
||
|
||
### 24.2 SCOPE: these are instruction cycles, and therefore a LOWER BOUND
|
||
MAME's `gvram_w`/`gvram_r` (`x68k_crtc.cpp:501,595`) contain **no timing at
|
||
all** — no wait states, no `adjust_icount`. GVRAM in MAME is as fast as main
|
||
RAM. Real X68000 GVRAM stalls the CPU on access, so every figure above is a
|
||
floor, not a prediction. **Do not quote these as hardware numbers.** Interrupts
|
||
were masked (`SR = $2700`) so the IPL's timer and VBL handlers could not steal
|
||
cycles into the measurement; a real player will take interrupts on top.
|
||
|
||
### 24.3 The 38% estimate is dead — a full-frame blit is 53.6%
|
||
The realistic "decode into a RAM frame, then blit it" design costs **53.6% of
|
||
the frame budget before decoding a single block**, and that is the zero-wait-
|
||
state floor. The estimate the CPU budget has been carrying since session 1 was
|
||
38%. It was optimistic by 41%.
|
||
|
||
The cause is visible in the V1/V3 gap: **reading the source frame is exactly
|
||
half the total cost** (221,952 of 446,286 cycles). The 68000 pays 8 cycles per
|
||
longword read and 8 per longword written, and in 256-colour mode a pixel
|
||
occupies a whole word of address space, so a frame is 96KB of traffic in each
|
||
direction rather than 48KB.
|
||
|
||
### 24.4 The high byte of every GVRAM write is discarded — confirmed from source
|
||
`gvram_w` case `0x0100` writes `data & 0x00ff` with `mem_mask 0x00ff`. So in
|
||
256-colour mode the CPU cannot pack two pixels into one word, and the odd bytes
|
||
of a word-expanded source frame never need clearing — V1 exploits this by
|
||
leaving them uninitialised. This is why 96KB, not 48KB, is the irreducible
|
||
write traffic.
|
||
|
||
### 24.5 The architecture question, and where it turns over
|
||
> **Superseded by FINDINGS 28.1/28.2 (session 7).** The two-path plan below is
|
||
> incoherent — the compose path needs a RAM reference the direct path never
|
||
> writes — and its two costs are both *copies*, so they were never comparable to
|
||
> a decode. The "76.6% x non-SKIP fraction" model is also 2.03x optimistic:
|
||
> the four block modes cost 300/448/400 cycles, not one figure. One path ships.
|
||
|
||
V4 prices the access pattern a decoder that writes codewords **straight into
|
||
GVRAM** actually has: 4 rows of 8 bytes at a 1024-byte stride per 4x4 block. The
|
||
same 96KB of writes costs **76.6%** in block order versus 53.6% row-linear — the
|
||
stride destroys the `movem.l` burst, 208 cycles per block against a theoretical
|
||
best of ~150.
|
||
|
||
But a decoder never writes every block: SKIP blocks cost **nothing at all**, and
|
||
the previous frame is already sitting in GVRAM, so **no RAM reference frame is
|
||
needed for SKIP to work**. So the two designs scale differently:
|
||
|
||
- **compose-in-RAM then blit** — flat 53.6%, independent of how much changed
|
||
- **decode-direct-to-GVRAM** — 76.6% x (fraction of non-SKIP blocks)
|
||
|
||
**They cross at 70% of blocks changed.** Below that, writing straight into GVRAM
|
||
wins, and it also drops the 96KB RAM reference frame entirely. Above it, the
|
||
flat blit wins.
|
||
|
||
**This makes the non-SKIP block fraction the single most important unmeasured
|
||
number in the project.** It is already computable from the encoder — it is a
|
||
by-product of the mode decision in `vq_hybrid.py` — and it has never been
|
||
reported. Measure it before writing any decoder inner loop, because it selects
|
||
which inner loop to write.
|
||
|
||
### 24.6 The frame the 68000 drew is pixel-exact
|
||
V1's output was snapshotted and passes `verify_frame256.py` unchanged: `256x512
|
||
native, double-scan exact, active 256x192 pixel-exact, letterbox true black`,
|
||
40.81 dB. So 68000 code drives the mode of FINDINGS 23 correctly, and 23.5 is
|
||
now closed.
|
||
|
||
---
|
||
|
||
## 25. The sustained action sequence, found and measured (session 5)
|
||
|
||
STATUS has carried "a *sustained* action sequence is the one thing that could
|
||
still break the bitrate" as the open risk since session 2. Every clip measured
|
||
before this was 1.2-1.7 s. This section closes it: **it does break the profiles,
|
||
though not the bus.**
|
||
|
||
### 25.1 The two largest streams on the disc are not game footage
|
||
A survey that sorts 224 streams by size and encodes the biggest would have
|
||
measured **live action**:
|
||
|
||
| stream | size | what it actually is |
|
||
|---|---:|---|
|
||
| 00216 | 3777 MB | the feature with a **burned-in picture-in-picture commentary** |
|
||
| 00215 | 3475 MB | the commentary itself, full-screen live action |
|
||
| **00223** | **1802 MB** | **clean animation, 9.4 min — the one to use** |
|
||
|
||
The PiP in 00216 is burned into video stream 0, not a selectable secondary
|
||
stream, so there is no ffmpeg flag that recovers a clean frame from it. This
|
||
extends FINDINGS 13's menu-vs-content warning: the classification needed is
|
||
**content / menu / bonus**, and bonus material is the one that looks most like
|
||
content by every cheap metric (size, duration, bitrate).
|
||
|
||
### 25.2 Picking the worst window by measurement, not by eye
|
||
`tools/analysis/07_motion_survey.py` scans a whole stream at 96x72 and reports
|
||
the highest-mean sliding window of inter-frame absolute difference. On 00223:
|
||
|
||
```
|
||
6793 frames @12fps = 566.1s
|
||
motion energy mean 9.40 median 5.60 p90 21.70 max 112.39
|
||
hottest sustained 10s window: t = 539.4s (2.01x stream mean)
|
||
quietest 10s window: t = 144.2s (0.19x stream mean)
|
||
```
|
||
|
||
The 10.6x spread between the quietest and hottest sustained windows is the whole
|
||
argument for not sampling clips by hand. `t = 539.4s` is the Singe endgame.
|
||
|
||
### 25.3 Both profiles overshoot on that window — rate control is now required
|
||
Encoding those 120 frames at the shipping profiles, with the fixed `lam` the CLI
|
||
currently uses:
|
||
|
||
| profile | target | measured | overshoot | PSNR | palette ceiling |
|
||
|---|---:|---:|---:|---:|---:|
|
||
| `sasi` | 110 KB/s | **129.6 KB/s** | **+18%** | 27.82 dB | 31.33 dB |
|
||
| `scsi` | 280 KB/s | **373.8 KB/s** | **+34%** | 30.81 dB | 31.33 dB |
|
||
| *(00020 baseline, `sasi`)* | 110 KB/s | 108.0 KB/s | -2% | 36.94 dB | 39.90 dB |
|
||
|
||
**This reclassifies rate control from insurance to a requirement.** STATUS has
|
||
had "wire rate control into `encode.py`" at priority 3-4 since session 2 with the
|
||
note "no longer a blocker (FINDINGS 21)". That was true of the clips measured
|
||
then. It is not true of this one. `ratectl.encode_rate_controlled()` already
|
||
exists and builds a per-frame lam ladder; it has simply never been hooked up.
|
||
|
||
Note what did **not** break: 373.8 + 7.8 = 381.6 KB/s is still under the 488 KB/s
|
||
working figure, so FINDINGS 21's ring-buffer conclusion survives — but at 78% of
|
||
the pipe sustained over ten seconds rather than the comfortable margin implied by
|
||
1.7 s clips.
|
||
|
||
### 25.4 The palette ceiling is content-dependent, and on hard content it binds
|
||
The 256-colour scene palette costs **31.33 dB** on this window against **39.90 dB**
|
||
on 00020 — 8.6 dB worse. Fire, lava and smoke gradients are exactly what a
|
||
256-entry mediancut palette handles worst.
|
||
|
||
This inverts an assumption the project has been carrying. FINDINGS 23.3 put the
|
||
X68000 display ceiling at 40.81 dB and treated it as comfortably clear of the
|
||
codec's own error. On this content the **scene palette (31.33 dB), not the
|
||
display hardware (40.81 dB), is the binding constraint** — and `scsi` is already
|
||
within 0.51 dB of it. Spending bits to close that last half-dB is spending them
|
||
against a ceiling that is not the display's.
|
||
|
||
### 25.5 `scsi` collapses to RAW under stress
|
||
Mode distribution on this window is qualitatively different from anything
|
||
measured before:
|
||
|
||
| profile | SKIP | V1 | V4 | RAW |
|
||
|---|---:|---:|---:|---:|
|
||
| `sasi` (lam=60) | 45.6% | 16.3% | 24.2% | 13.9% |
|
||
| `scsi` (lam=10) | 26.2% | 5.5% | 7.1% | **61.2%** |
|
||
| *00020, `sasi`* | 46.9% | 24.1% | 17.8% | 11.2% |
|
||
|
||
At `lam=10` the rate-distortion decision finds literal pixels cheaper than any
|
||
codeword for 61% of blocks — the codebooks are simply not describing this
|
||
content. That is the mechanism behind the +34% overshoot in 25.3, and it is a
|
||
rate-control problem, not a codec-structure problem: the RD decision is behaving
|
||
correctly for the lam it was given.
|
||
|
||
### 25.6 The decoder needs BOTH display paths, chosen per frame
|
||
> **Superseded by FINDINGS 28.1 (session 7).** Mixing the paths displays stale
|
||
> pixels on 70 of these 120 frames. The "median 37.0%, capped at 53.6%" below is
|
||
> the cost of an incorrect player; every coherent version is dearer, and plain
|
||
> direct-to-GVRAM is the cheapest of them.
|
||
|
||
Applying FINDINGS 24.5's crossover to the real per-frame distribution:
|
||
|
||
| | median non-SKIP | p90 | frames over the 70% crossover |
|
||
|---|---:|---:|---:|
|
||
| `sasi`, Singe window | 48.4% | 82.8% | 36 / 120 (30%) |
|
||
| `scsi`, Singe window | 70.8% | 92.4% | 64 / 120 (53%) |
|
||
| `sasi`, 00020 | 54.0% | 88.8% | 3 / 14 (21%) |
|
||
|
||
Neither path wins outright: **30-53% of frames want the flat blit and the rest
|
||
want direct-to-GVRAM.** A player that implements both and picks per frame — the
|
||
mode headers are parsed before any pixel is written, so the count is free — pays
|
||
a median of **37.0%** of the frame budget and is capped at **53.6%**. A player
|
||
that implements only direct-to-GVRAM pays up to 76.6% and would miss frames on
|
||
the scene cuts.
|
||
|
||
So the answer to 24.5 is "both", and the selection is a one-line comparison
|
||
against a block count the decoder already has in hand.
|
||
|
||
### 25.7 What this does not measure
|
||
One 10 s window of one stream, at fixed lam, with `_paint` still a Python loop.
|
||
The full-disc survey is still not done, and the numbers above are the *worst*
|
||
window rather than a distribution over content. What has changed is that the
|
||
worst case is now a measurement rather than a worry.
|
||
|
||
---
|
||
|
||
## 26. Rate control is unsound as written — found before wiring it up (session 5)
|
||
|
||
FINDINGS 25.3 promoted rate control from insurance to a requirement. Reading
|
||
`ratectl.py` before wiring it into `encode.py` turned up a correctness bug that
|
||
would have produced exactly the kind of plausible-looking wrong result this
|
||
project keeps catching (FINDINGS 4, 9, 14, 18).
|
||
|
||
### 26.1 The lam ladder desynchronises the encoder from the decoder
|
||
`H.encode()` is **temporally recursive**: SKIP blocks are copied from the
|
||
previous *reconstruction*, and `prev = out` closes the loop
|
||
(`vq_hybrid.py:84-109`). A frame's output therefore depends on every frame
|
||
before it in that same run.
|
||
|
||
`encode_rate_controlled()` runs `H.encode()` once per lam over the **whole
|
||
sequence**, building a ladder of independent temporal chains, then picks each
|
||
frame from whichever rung fits the budget. When frame *f* comes from rung *i*
|
||
and frame *f-1* was emitted from rung *j != i*, the SKIP blocks in *f* reference
|
||
a reconstruction **the decoder never saw**.
|
||
|
||
Measured on the Singe window (`tools/analysis/09_ratectl_drift.py`, 120 frames,
|
||
5 rungs, target 110 KB/s):
|
||
|
||
| | |
|
||
|---|---|
|
||
| rung switches | **67** over 120 frames |
|
||
| frames whose emitted output differs from what the encoder recorded | **111 / 120** |
|
||
| worst frame | **21,339 px = 43.4% of the frame** |
|
||
| encoder-vs-decoder agreement, worst frame | 27.1 dB |
|
||
| reported PSNR overstatement | **0.36 dB** |
|
||
|
||
The 0.36 dB is the least interesting number here. The encoder is reporting
|
||
quality for a reconstruction that will never exist, and 43% of a frame differing
|
||
is a visible artefact whatever the mean says.
|
||
|
||
**The fix is structural, not a tuning change:** `H.encode()` must become
|
||
frame-drivable — take `prev` and one lam, return one frame — so rate control can
|
||
feed back the frame it actually emitted. The current whole-sequence signature is
|
||
what makes the ladder tempting in the first place.
|
||
|
||
### 26.2 The ladder spans 250x past the shippable range
|
||
`lam_hi=2e5`, but FINDINGS 15 puts the quality cliff between lam=800 and
|
||
lam=2000 and says do not ship past lam~800. Every rung above ~800 is
|
||
unshippable, so a frame that only fits at lam=9457 has not been rate-controlled,
|
||
it has been destroyed. Cap `lam_hi` at 800 and let a frame that cannot fit
|
||
overrun the bucket — a visible overrun is a better failure than silent garbage.
|
||
|
||
### 26.3 The ladder is far too coarse where it matters
|
||
With `steps=5` the geomspace lands on 1 / 21 / 447 / 9457 / 200000, and **only
|
||
two rungs were ever chosen**. The budget is 8,721 B/frame; the two straddling
|
||
rungs deliver 23,183 B (lam=21) and 3,071 B (lam=447) — a **7.5x** gap across
|
||
the operating point. Rate control cannot land near a target it has to jump over.
|
||
|
||
The module docstring already describes the right approach — *"per frame we
|
||
binary-search lam to land inside a byte budget"* — but the implementation is a
|
||
fixed precomputed ladder. Doc and code disagree; the doc is correct.
|
||
|
||
### 26.4 What does work
|
||
The leaky bucket lands the mean where it should: **109.1 KB/s against a 110
|
||
target**, with 32% of frames over the per-frame budget and banked by the bucket.
|
||
That mechanism is sound and worth keeping. It is the per-frame lam *selection*
|
||
underneath it that needs rebuilding, not the bucket.
|
||
|
||
### 26.5 Cost note before starting
|
||
Each rung is a full-sequence encode and `_paint` is still a Python per-block
|
||
loop, so a 5-rung run over 120 frames takes minutes. **Vectorise `_paint`
|
||
first** — it is already on the list for the full-disc survey and it makes the
|
||
rate-control work practical rather than merely faster.
|
||
|
||
## 27. Rate control, rebuilt and wired in (session 6)
|
||
|
||
FINDINGS 26 stopped the session-5 rate controller before it shipped: it picked
|
||
frames out of independently-encoded whole-sequence runs, so 111 of 120 frames
|
||
referenced reconstructions the decoder would never see. The fix was structural,
|
||
as 26.1 said it had to be. It is now wired into `encode.py` and **on by
|
||
default** for a profile.
|
||
|
||
### 27.1 The encoder is frame-drivable, and the drift is zero by construction
|
||
`vq_hybrid` now exposes one frame at a time — `frame_ctx(m, f, prev)` /
|
||
`decide(ctx, lam)` / `paint(m, ctx, mode)` — and `encode()` is a thin loop over
|
||
that API. Rate control drives the same three calls and feeds back **the frame it
|
||
actually emitted** as the next frame's `prev`. There is no ladder to pick from,
|
||
so the desync has no way to occur.
|
||
|
||
`tools/analysis/09_ratectl_drift.py`, unchanged in what it asserts:
|
||
|
||
| | session 5 | session 6 |
|
||
|---|---|---|
|
||
| frames whose emitted output differs from what the encoder recorded | 111 / 120 | **0 / 120** |
|
||
| worst frame | 21,339 px (43.4%) | **0 px** |
|
||
| reported PSNR overstatement | 0.36 dB | **0.00 dB** |
|
||
|
||
This is the harder case for that test on purpose: it runs with `lam_lo=1.0`, so
|
||
lam moves on 117 of 119 frame boundaries. Under the old ladder, 67 rung switches
|
||
were enough to corrupt 111 frames.
|
||
|
||
### 27.2 Both overshoots are closed, and they cost under 1 dB
|
||
The Singe window (FINDINGS 25.3), which is the worst sustained window on the
|
||
disc. Totals include the 7.8 KB/s ADPCM allowance:
|
||
|
||
| profile | target | fixed lam (session 5) | rate-controlled | quality cost |
|
||
|---|---|---|---|---|
|
||
| `sasi` | 110 KB/s | 137.4 KB/s (**+25%**) | **109.5 KB/s** | 27.82 → 27.22 dB (−0.60) |
|
||
| `scsi` | 280 KB/s | 381.6 KB/s (**+36%**) | **280.0 KB/s** | 30.81 → 29.90 dB (−0.91) |
|
||
|
||
Zero frames hit the lam=800 cliff at either profile, so nothing was destroyed to
|
||
get there (26.2's failure mode did not trigger). `sasi` needed lam to reach 183
|
||
at worst against a floor of 60; `scsi` reached 58.7 against 10. The controller
|
||
is working an order of magnitude below the cliff, which is where the search
|
||
range being capped at 800 rather than 2e5 stops mattering at all — and that is
|
||
the point: a range that never needs its top is a range you can trust.
|
||
|
||
`scsi` still sits **1.43 dB** from the scene palette ceiling of 31.33 dB
|
||
(FINDINGS 25.4), against 0.51 dB before. The ceiling, not the codec, is still
|
||
what bounds this content.
|
||
|
||
The percentages differ from 25.3's +18%/+34% because those compared video
|
||
payload against the total target; the table above compares like with like
|
||
(total against total). The payload figures are unchanged: 129.6 and 373.8 KB/s.
|
||
|
||
### 27.3 Rate control makes the display path cheaper, not dearer
|
||
The decoder-architecture numbers of FINDINGS 25.6 were measured on the
|
||
fixed-lam encoder. Re-measured under rate control, on the same window, with the
|
||
player picking the cheaper of compose-then-blit and direct-to-GVRAM per frame:
|
||
|
||
| profile | median display cost | frames above the 70% crossover |
|
||
|---|---|---|
|
||
| `sasi` fixed → RC | 37.0% → **36.6%** | 30.0% → 26.7% |
|
||
| `scsi` fixed → RC | 53.6% → **47.1%** | 53.3% → 35.8% |
|
||
|
||
Raising lam moves blocks to SKIP and V1, which is fewer blocks to write. The
|
||
"implement both paths, pick per frame" conclusion is unaffected and the cap is
|
||
still 53.6%.
|
||
|
||
### 27.4 The quality floor barely matters; the prefill matters, wrongly
|
||
Two knobs were measured rather than guessed.
|
||
|
||
**`--rc-floor`** decides whether a quiet frame may spend more than the fixed-lam
|
||
profile would. On the Singe window it is worth nothing — 109.5 vs 110.0 KB/s and
|
||
**0.00 dB** — because no frame on that window is quiet enough for the bucket to
|
||
saturate. The default is `profile` (never spend more than session 5 would), so
|
||
rate control cannot regress content that already fits.
|
||
|
||
**`--prefill`** models how full the player's buffer is at scene start. It is
|
||
tempting and it is a trap, so it defaults to 0:
|
||
|
||
| clip | prefill 0.0 | 0.5 | 1.0 | target |
|
||
|---|---|---|---|---|
|
||
| Singe, 120 fr, `sasi` | 109.5 | 112.9 | **116.3** | 110 |
|
||
| Singe, 120 fr, `scsi` | 280.0 | 289.1 | **298.2** | 280 |
|
||
| 00020, 14 fr, `sasi` | 92.0 | **115.8** | **115.8** | 110 |
|
||
| 00020, 14 fr, `scsi` | 224.8 | **255.9** | **255.9** | 280 |
|
||
|
||
(`scsi` on 00020 is the one cell where prefill looks harmless: the clip fits
|
||
under 280 either way. That is the content being easy, not the knob being safe.)
|
||
|
||
Prefill buys a permission to overshoot of exactly `bucket / nframes`. At 8
|
||
frames of bucket over 120 frames that is 6.2% — measured — and on a 14-frame
|
||
clip the bucket is larger than the clip, so rate control switches itself off and
|
||
reproduces fixed-lam exactly (lam never leaves its floor: min = median = max =
|
||
60). **A prefill that makes a target look met has disabled the controller.**
|
||
|
||
### 27.5 The 00020 undershoot is a clip-length artefact, not a bug
|
||
At prefill 0 the 14-frame 00020 clip lands at 92.0 KB/s against a 110 ceiling —
|
||
0.66 dB given away for nothing. That is the leaky bucket's startup transient:
|
||
the first `bucket_frames` frames cannot draw on a bank they have not accumulated.
|
||
It is bounded by `bucket / nframes`, so it is 6% on a 10-second window and 20%
|
||
on a 1.2-second one.
|
||
|
||
The lesson is the one FINDINGS 25.3 already taught in a different costume: **a
|
||
1.2-second clip cannot be used to judge rate control.** Real scenes are tens of
|
||
seconds. Do not tune the bucket against 00020.
|
||
|
||
Worth recording separately: fixed-lam `sasi` on 00020 delivers 115.8 KB/s — the
|
||
supposedly easy clip was **already 5% over its target**, which nothing had
|
||
noticed because the profile table quotes its PSNR and not its bitrate.
|
||
|
||
### 27.6 FINDINGS 26.5's cost premise was wrong in both halves
|
||
26.5 said a rate-control experiment was minutes because `_paint` is a Python
|
||
per-block loop, and told the next session to vectorise it first. Vectorising it
|
||
was correct and it is **17.1x faster**, but it was never the bottleneck, and the
|
||
ladder was never minutes. Measured per frame, 256x192:
|
||
|
||
| | ms |
|
||
|---|---|
|
||
| `VQ.assign` x2 — codeword search | **22.83** |
|
||
| SKIP error against `prev` | 1.40 |
|
||
| `decide` — argmin at one lam | 0.06 |
|
||
| `paint`, vectorised | 0.29 |
|
||
| `paint`, old per-block loop | 4.93 |
|
||
|
||
`_paint` was 14% of a frame. A 5-rung ladder over 120 frames was ~18 s of
|
||
encoding, not minutes — the "few minutes" in the drift test's docstring was
|
||
`H.build`'s k-means (51 s), which no amount of vectorising `_paint` would have
|
||
touched.
|
||
|
||
What actually makes per-frame rate control affordable is that `VQ.assign`'s
|
||
output depends on **neither `lam` nor `prev`**, so it is computed once per frame
|
||
and a lam search only re-runs the 0.06 ms argmin:
|
||
|
||
| | |
|
||
|---|---|
|
||
| 12-step per-frame lam search, 120 frames, symbols cached | **0.31 s** |
|
||
| the same search by re-running whole-sequence encodes | 49.10 s |
|
||
|
||
That is a 158x difference, and it is the reason the controller can afford a real
|
||
bisection instead of a 5-rung ladder — which was the actual defect in 26.3.
|
||
|
||
The cache holds **one frame**. At ~133 KB of intermediates per frame, caching
|
||
the sequence would cost 900 MB on a 9.4-minute stream to save nothing: every
|
||
caller works a frame at a time.
|
||
|
||
---
|
||
|
||
## 28. The 68000 decoder exists, is pixel-exact, and does not fit (session 7)
|
||
|
||
`src/player/decode.s` parses DLX1 and draws frames on the emulated X68000. It is
|
||
**pixel-exact across a 120-frame sequential run** of the worst sustained window
|
||
on the disc (`tools/bench/verify_decode.py`), exercising all four block modes
|
||
and the full temporal recursion — the last frame is only right if every frame
|
||
before it was.
|
||
|
||
It is also **too slow**. On that window, at the shipping `sasi` profile:
|
||
|
||
| | non-SKIP blocks | measured cost |
|
||
|---|---:|---:|
|
||
| cheapest frame | 15.4% | 31.5% of a 12fps frame |
|
||
| median frame | 47.8% | 73.8% |
|
||
| p90 frame | 82.5% | **116.4%** |
|
||
| worst frame | 100.0% | **135.8%** |
|
||
| mean over the window | 47.8% | **81.7%** |
|
||
|
||
**31% of frames miss the 833,333-cycle budget**, and like every figure since
|
||
FINDINGS 24 these are instruction cycles against zero-wait-state memory, so
|
||
they are a floor. This is the first time CPU, not disk, is the binding
|
||
constraint — FINDINGS 21 retired the bandwidth worry, and this replaces it.
|
||
|
||
### 28.1 The dual-path plan of 24.5/25.6 was incoherent, and is withdrawn
|
||
FINDINGS 24.5 specified two display paths chosen per frame on the non-SKIP
|
||
count, and 25.6 costed the mix at "median 37.0%, capped at 53.6%". Two of its
|
||
premises cannot both hold:
|
||
|
||
- compose-in-RAM-then-blit exists to make the blit **row-linear**, so it must
|
||
assemble a **full** frame in RAM. The pixels it does not decode this frame —
|
||
the SKIP blocks — can only come from a RAM copy of the previous
|
||
reconstruction.
|
||
- decode-direct-to-GVRAM's stated advantage is that **"no RAM reference frame
|
||
is needed"**, because the previous frame is already in GVRAM.
|
||
|
||
So every direct frame silently invalidates the reference the next compose frame
|
||
reads. Simulated on the Singe window at the crossover the plan specifies
|
||
(`tools/analysis/10_pathmix_drift.py`): **70 of 120 frames display pixels no
|
||
correct player would display**, first at frame 2, worst frame 18.8% of the
|
||
screen. This is FINDINGS 26 in different clothing — two code paths disagreeing
|
||
about what "the previous frame" means — and it is the **sixth** false premise
|
||
this project has caught before it shipped.
|
||
|
||
Every coherent repair is worse than not mixing at all:
|
||
|
||
| strategy | median | p90 | max | correct |
|
||
|---|---:|---:|---:|---|
|
||
| mix per frame, as specified | 36.6% | 53.6% | 53.6% | **no** |
|
||
| mix, direct also writes the RAM reference | 53.6% | 68.4% | 81.4% | yes |
|
||
| mix, re-read GVRAM into RAM on each switch | 36.6% | 107.2% | 107.2% | yes, 13 frames miss |
|
||
| compose only | 53.6% | 53.6% | 53.6% | yes |
|
||
| **direct only** | **36.6%** | 62.5% | 76.6% | yes |
|
||
|
||
(Costs in that table are 24.5's own model, for like-for-like comparison; 28.2
|
||
replaces the model itself.)
|
||
|
||
**24.5 also compared the wrong two things.** Its 53.6% and 76.6% are both
|
||
*copies* measured in `blit.s` — neither includes decoding. A real compose path
|
||
costs decode-into-RAM **plus** the 53.6% blit, so it is strictly dearer than
|
||
decoding straight into GVRAM, whatever the block mix. There was never a
|
||
crossover to find.
|
||
|
||
**The decoder therefore implements one path, direct-to-GVRAM**, and drops the
|
||
96 KB RAM reference frame entirely.
|
||
|
||
### 28.2 The four block modes do not cost the same, and V4 is the expensive one
|
||
24.5's model — "76.6% of a frame x the non-SKIP fraction" — prices every
|
||
non-SKIP block as one `movem.l` burst. Measured separately, with synthetic
|
||
single-mode frames (`tools/bench/prep_dlx.py`):
|
||
|
||
| mode | cycles/block | vs the 24.5 model (207.8) |
|
||
|---|---:|---:|
|
||
| SKIP, in an all-SKIP header byte | 13.3 | model says 0 |
|
||
| SKIP, inside a mixed byte | ~45 | model says 0 |
|
||
| V1 (one 4x4 codeword) | **299.9** | 1.44x |
|
||
| V4 (four 2x2 codewords) | **448.2** | 2.16x |
|
||
| RAW (16 literal indices) | **400.4** | 1.93x |
|
||
|
||
Applied to the real per-frame histograms (`tools/analysis/11_cpu_budget.py`),
|
||
the model reproduces all four frames timed on the 68000 to within **1
|
||
percentage point**, and shows 24.5 to be **2.03x optimistic at the median**.
|
||
|
||
Where the cycles actually go over the window:
|
||
|
||
| mode | % of blocks | % of cycles |
|
||
|---|---:|---:|
|
||
| SKIP | 46.4% | 9.2% |
|
||
| V1 | 19.8% | 26.1% |
|
||
| V4 | **25.2%** | **49.7%** |
|
||
| RAW | 8.5% | 15.0% |
|
||
|
||
**V4 is a quarter of the blocks and half the cycles.** It costs 1.49x a V1 block
|
||
while the mode decision in `vq_hybrid.py` charges it only its 4x payload bytes.
|
||
The lagrangian trades distortion against *bytes*; on this machine it now has to
|
||
trade distortion against *cycles* as well.
|
||
|
||
### 28.3 The container is big-endian but not aligned, and that is an address error
|
||
The DLX1 header docstring says every multi-byte field is big-endian "so the
|
||
68000 reads them with a plain `move`". Alignment is the other half of that
|
||
sentence and the container does not have it: frame records are
|
||
`[u32 length][768-byte mode header][payload]` laid end to end with arbitrary
|
||
payload lengths, so record boundaries land on odd addresses.
|
||
|
||
`move.l (a0)+,d0` at an odd address is an **address error** on a 68000 — not a
|
||
slow read. The first run decoded frame 0 perfectly, consumed exactly its 8,715
|
||
payload bytes, then read frame 1's length at `$03220F` and vectored into the IPL
|
||
at `$FF059A`, where it sat for 59 emulated seconds looking like an infinite
|
||
loop. It was found by dumping PC and the address registers, not by reading the
|
||
source: the code was correct, the data layout was not.
|
||
|
||
The decoder now rounds each record start up to 4. **The container should carry
|
||
the padding itself** so a streaming player can DMA records into place: measured
|
||
cost on this window is **199 bytes over 120 frames — 1.66 B/frame, 20 B/s**
|
||
against a 110 KB/s budget. Until `encode.py` does it, `prep_dlx.py` realigns at
|
||
load time.
|
||
|
||
### 28.4 The measurements agree with hand-derived MC68000 timings
|
||
As in FINDINGS 24, each figure was derived from the instruction timing tables
|
||
before being believed. A V1 block, summing dispatch, index decode, the indexed
|
||
`movem.l` load and four `movem.l` stores, plus its quarter share of the header
|
||
loop: **298.5 cycles derived against 299.9 measured — 0.5%.** RAW derives to
|
||
396 against 400.4 measured (1%). V4 derives to 415 against 448 (7%, the gap
|
||
being in the indexed two-register `movem.l`, the mode this decoder uses most
|
||
heavily). So these are 68000 cycles, not a MAME artefact.
|
||
|
||
### 28.5 A full frame does not fit at 12fps in ANY mode
|
||
An all-V1 frame — the cheapest possible way to redraw all 3,072 blocks — costs
|
||
**921,187 cycles, 110.5% of the budget**. All-V4 is 165.2% and all-RAW 147.6%.
|
||
|
||
So the ceiling is structural, not a tuning problem: **at 12fps on a 10MHz 68000
|
||
no more than ~88% of the screen can change in one frame**, however cheaply it is
|
||
coded. Scene cuts change 100% of it. Either a cut gets one late frame (the
|
||
outgoing content is unrelated, so this may be free to the eye), or cuts have to
|
||
be spread across two frame times, or the framerate has to come down — at 10fps
|
||
the budget is 1,000,000 cycles and an all-V1 frame fits.
|
||
|
||
### 28.6 What this does not measure
|
||
One 10 s window of one stream at one profile, and MAME still models no GVRAM
|
||
wait states. The `scsi` profile will be worse: FINDINGS 25.5 has it collapsing
|
||
to RAW under stress, and RAW is 1.93x the old model's block. Nothing here has
|
||
been run on `00020` or on quiet content, where the median frame is far cheaper.
|
||
|
||
### 28.7 The profiles are an I/O axis; the CPU limit is the clock
|
||
`sasi` and `scsi` are two points on one rate-distortion curve, chosen against
|
||
**disk bandwidth**. They say nothing about CPU, and the locked target CPU is a
|
||
stock 10MHz 68000 for both. So both have to fit the same 833,333 cycles:
|
||
|
||
| | sasi | scsi |
|
||
|---|---:|---:|
|
||
| stock / Super, 10 MHz | median 74.4%, **31% of frames miss** | median 94.9%, **42% miss** |
|
||
| XVI, 16.67 MHz | median 44.6%, 0% miss | median 56.9%, 0% miss |
|
||
|
||
Clocks confirmed from MAME 0.277 `x68k.cpp:1133/1194/1200`: `x68000` and
|
||
`x68ksupr` are **both** `40_MHz_XTAL/4` = 10 MHz, and only `x68kxvi` is faster
|
||
at `33.33_MHz_XTAL/2`. The Super has SCSI at 10 MHz, so a faster bus does not
|
||
imply a faster CPU — the XVI column above is headroom, not a target.
|
||
|
||
`sasi` is the cheaper profile, but choosing it is not a fix: it still misses
|
||
31% of frames. The cycle ceiling has to be enforced in the encoder either way.
|
||
|
||
**How much of the miss is the encoder's to fix.** Re-coding every non-SKIP block
|
||
as V1 — the cheapest mode, quality ignored — is the floor any mode assignment
|
||
can reach:
|
||
|
||
| | frames that miss | recoverable by re-coding | impossible at 12fps |
|
||
|---|---:|---:|---:|
|
||
| `sasi` | 37/120 | 26 | **11** (from 89.8% non-SKIP up) |
|
||
| `scsi` | 51/120 | 39 | **12** (from 91.9% non-SKIP up) |
|
||
|
||
So a cost-aware mode decision can reach about **three quarters** of the misses.
|
||
The remaining ~10% of frames are 28.5's ceiling in practice: past ~90% non-SKIP
|
||
no mode assignment fits, because the blocks have to be drawn at all. Those
|
||
frames need a structural answer — a late frame at a cut, a cut spread over two
|
||
frame times, or a lower framerate — not a better encoder.
|
||
|
||
### 28.8 V4 costs more cycles than RAW, so it is CPU-dominated by it
|
||
448.2 against 400.4 cycles. RAW is also pixel-exact where V4 is lossy, so V4's
|
||
only advantage is that it costs 4 payload bytes instead of 16. **On the CPU
|
||
axis V4 is strictly dominated**, which inverts the mode preference the byte
|
||
lagrangian has: an encoder short of cycles but not of bytes should buy RAW
|
||
wherever it would have bought V4, and gain quality doing it.
|
||
|
||
That escape is only open to the byte-rich profile. `scsi` already spends 41.3%
|
||
of its blocks on RAW (FINDINGS 25.5 saw it "collapse to RAW under stress" and
|
||
read that as a failure; on the CPU axis it is the cheap direction). `sasi` at
|
||
110 KB/s cannot afford it, so its only lever is V4 -> V1 -> SKIP, every step of
|
||
which costs quality. **The CPU constraint therefore bites harder on `sasi` in
|
||
quality terms even though it bites less in cycles.**
|
||
|
||
Caveat: this ordering is a property of *this* decoder, not of the codec. V4's
|
||
cost is four indexed `movem.l` lookups; pairing sub-block rows into
|
||
`movem.l d0/d2,(a4)` would save ~16 of 448 cycles, which narrows the gap to RAW
|
||
without closing it.
|
||
|
||
---
|
||
|
||
## 29. Trading bytes for cycles: the bus has 4x the headroom the CPU has (session 7)
|
||
|
||
> **ALSO SUPERSEDED IN PART BY 38.** "The bus has 4x the headroom the CPU has"
|
||
> is about the SCSI pipe. The 68000's LOCAL bus is a different resource and the
|
||
> decoder occupies 86.7% of it, so trading cycles for bytes is not free in the
|
||
> currency that turned out to bind. 29.6's DMAC idea is costed in 39.
|
||
>
|
||
> **SUPERSEDED IN PART BY 30, which measured it.** The mode survives and the
|
||
> conclusion holds, but every number in this section moved: a span costs 43.7
|
||
> cycles + 9.152/pixel *only* in an encoder-assisted format (the obvious
|
||
> decoder is 97.9 + 10.46), spans beat V1 from runs of 4 blocks and not 2, and
|
||
> the re-priced trade-off is 52.0% median / 10 misses, not 43.0% / 8. Read 30's
|
||
> tables over 29.3's. 29.5's other three items are still open, and 29.6 stands.
|
||
>
|
||
> **STATUS AT THE TIME: DERIVED, NOT MEASURED.** No 68000 had executed a span
|
||
> decoder. The per-pixel figure it rests on *is* measured (FINDINGS 24 V1) but
|
||
> at full row width; the per-span overhead was hand-derived. FINDINGS 4 is why
|
||
> it was labelled and then tested rather than believed.
|
||
|
||
FINDINGS 28 leaves the project CPU-bound while the **bus sits 4x idle**: `sasi`
|
||
spends 110 KB/s of a 488 KB/s pipe. That asymmetry is exploitable, because the
|
||
codec was designed when bytes were the scarce thing and every one of its
|
||
decisions trades cycles to save them.
|
||
|
||
### 29.1 The decoder pays per changed PIXEL; the disk pays per BYTE
|
||
Per-pixel costs, all measured:
|
||
|
||
| what | cycles/pixel | source |
|
||
|---|---:|---|
|
||
| write-only floor (no source read) | 4.59 | FINDINGS 24 V3 |
|
||
| **row-linear copy from word-expanded RAM** | **9.08** | FINDINGS 24 V1 |
|
||
| block-order copy, same bytes | 12.98 | FINDINGS 24 V4 |
|
||
| V1 codebook block | 18.74 | FINDINGS 28.2 |
|
||
| RAW, byte literals unpacked to words | 25.03 | FINDINGS 28.2 |
|
||
| naive per-pixel byte expansion | 26.13 | FINDINGS 24 V2 |
|
||
|
||
Two structural facts fall out. **The 1024-byte stride costs 43%** — the same
|
||
bytes cost 12.98 cycles/px in 4x4 block order against 9.08 row-linear, because
|
||
the stride breaks the `movem.l` burst. And **unpacking bytes to words costs more
|
||
than the write itself**: 25.03 against 9.08.
|
||
|
||
So the two cheapest things a decoder can be handed are *word-expanded* pixels
|
||
in *row-linear runs* — and both cost bytes on disc, which is what we have.
|
||
|
||
### 29.2 Codebooks are a byte optimisation that now costs cycles
|
||
A word-expanded literal 4x4 block, `movem.l (a0)+,d0-d7` straight from the
|
||
stream buffer into GVRAM, derives to **~240 cycles** — cheaper than V1's
|
||
measured 299.9, and pixel-exact. V1 is dearer *because* it is compressed: it
|
||
pays an index decode and an indexed `movem.l` that a literal does not, and then
|
||
does exactly the same four writes. It buys 31 bytes and spends 60 cycles.
|
||
|
||
**Every codebook mode is CPU-dominated by a literal.** V4 was already dominated
|
||
by RAW (28.8); with word-expanded literals available, so is V1. The VQ codebook
|
||
earns its place only while bytes are scarce.
|
||
|
||
### 29.3 Row-linear literal spans, priced against the real mode maps
|
||
Replace the per-block escape with a per-row **span**: `(x, count, word-expanded
|
||
pixels)`, decoded with `movem.l` bursts. A run of L horizontally adjacent dirty
|
||
blocks becomes 4 spans of 4L pixels, deriving to `4 * (50 + 4L * 9.08)` cycles
|
||
against `300L` for V1 — **cheaper for any run of 2 blocks or more**, at 32 bytes
|
||
per block instead of 1.
|
||
|
||
Applied greedily (buy the best cycles-saved-per-byte until the bus budget is
|
||
gone) to the *unchanged* mode maps of the `sasi` Singe window:
|
||
|
||
| | today | + literal spans |
|
||
|---|---:|---:|
|
||
| median frame | 74.4% | **43.0%** |
|
||
| p90 frame | 115.1% | **83.6%** |
|
||
| worst frame | 136.2% | **106.2%** |
|
||
| frames missing the budget | **37/120** | **8/120** |
|
||
| bitrate | 101.7 KB/s | 453.2 KB/s (bus 488) |
|
||
|
||
And the fit is structural rather than lucky: **spans get cheaper exactly where
|
||
blocks get expensive.** A span amortises its overhead over a long run, and long
|
||
runs are what a high-change frame is made of. The frames that miss today are the
|
||
frames spans help most.
|
||
|
||
### 29.4 This reopens 28.5, which said a scene cut cannot fit
|
||
28.5 concluded that no mode assignment fits a 100%-changed frame at 12fps,
|
||
because the cheapest full redraw available — all-V1 — is 110.5%. That was true
|
||
of *the mode set the codec has*. Adding a byte-expensive, cycle-cheap mode
|
||
changes the arithmetic: mixing a fraction `x` of the frame as spans against V1
|
||
for the rest,
|
||
|
||
- CPU needs `x >= 0.19`
|
||
- the 40,977 B/frame bus budget allows `x <= 0.39`
|
||
|
||
**The interval is not empty.** A scene cut fits at 12fps if roughly a quarter to
|
||
a third of it arrives as word-expanded row-linear literals. 28.5's "structural
|
||
ceiling" was a ceiling of the bitstream, not of the machine.
|
||
|
||
### 29.5 What has to be measured before any of this is believed
|
||
1. **Span cost on the 68000.** The 50-cycle per-span overhead is derived, and
|
||
the 9.08 cycles/px is measured at *full row width* with 12-register bursts —
|
||
a short or oddly-aligned span cannot burst as well, so short spans are
|
||
flattered here. Extend `tools/bench/blit.s` with a span variant and measure
|
||
it against run length. **This is the load-bearing number.**
|
||
2. **Re-run the ring-buffer simulation at ~450 KB/s.** FINDINGS 21's zero
|
||
required prefill was established at 110 and 280 KB/s against a 488 KB/s pipe.
|
||
At 453 the margin is a tenth of what it was, and 21's own caveat was that the
|
||
test is cumulative — it needs redoing, not extrapolating.
|
||
3. **Confirm the 4 Mbps figure**, which is user-supplied with no recorded
|
||
provenance and which this design would run at 93% of. It has been a "would be
|
||
nice" since session 1; a design that leans on it makes it load-bearing.
|
||
4. **Confirm DMA, not PIO** (STATUS priority 5). At 453 KB/s a PIO fallback puts
|
||
the transfer cost on the CPU we are trying to relieve. Cheapest check
|
||
available and now the most consequential.
|
||
|
||
### 29.6 The other lever, not yet costed: let the DMAC do the copy
|
||
The X68000 has an HD63450 DMAC (4 channels, `x68k.cpp:1046`). Channel 3 is
|
||
ADPCM — confirmed, `adpcm_drq_tick` asserts `drq3_w` — but memory-to-memory
|
||
transfer on a free channel would take the GVRAM copy off the CPU entirely,
|
||
leaving it only the parsing. This is the one idea here that could move the
|
||
budget without spending a single extra byte.
|
||
|
||
It cannot be settled in MAME: like the SCSI/SASI devices (BENCHMARK.md), the
|
||
HD63450 is a functional model, so a timing number out of it would measure the
|
||
emulator's scheduler. It needs hand-derivation against the datasheet plus real
|
||
hardware — the same three-tier approach the disk benchmark already documents.
|
||
|
||
## 30. The span, measured: the mode survives, and it is an encoder format (session 8)
|
||
|
||
FINDINGS 29 priced a new decoder mode at `4 * (50 + 4L*9.08)` cycles and marked
|
||
the whole section DERIVED. This is the measurement. `tools/bench/blit.s` gained
|
||
two span variants, `tools/bench/prep_spans.py` generates one stream per span
|
||
length, `tools/bench/span.lua` times them, and `tools/bench/span.sh` runs the
|
||
lot, and the whole thing takes about 25 seconds.
|
||
|
||
Same scope as every 68000 figure since FINDINGS 24: instruction cycles against
|
||
MAME's zero-wait-state GVRAM, interrupts masked. A **lower bound**, not a
|
||
prediction.
|
||
|
||
### 30.1 What was measured
|
||
Twelve `v5` configs and eleven `v6` configs, each cutting the **same** 256x192
|
||
frame into spans of a different length, so the work differs only in how finely
|
||
it is cut. Regressing `cycles = A*spans + B*pixels` over a set reads the
|
||
per-span overhead and the per-pixel cost straight off.
|
||
|
||
Every config draws the whole picture, the picture is cleared before each run and
|
||
snapshotted after, and all 23 snapshots are checked pixel-exact by
|
||
`tools/bench/verify_frame256.py`. A config cannot time fast by writing nothing.
|
||
|
||
| | per span | per pixel | fit error |
|
||
|---|---:|---:|---:|
|
||
| **v5** — decoder handed `(x, npix)`, works the copy out | **97.9** | **10.459** | ±1.4%, and only on spans that are a whole number of bursts |
|
||
| **v6** — encoder hands it an address and a jump | **43.7** | **9.152** | **±0.3% over all 11 lengths** |
|
||
| *29's assumption* | *50.0* | *9.080* | — |
|
||
|
||
**29's arithmetic was right about a format nobody had written yet.** v6 hits it
|
||
almost exactly; v5 — the obvious decoder, and the one 29 was describing — is
|
||
2.24x dearer per span and 14% dearer per pixel.
|
||
|
||
### 30.2 Why the difference is a format difference, not an optimisation
|
||
v5's record is `(x, npix)`, so the decoder computes the destination, divides
|
||
`npix` into 16-pixel bursts, and handles the 0..15 remainder: about 122 cycles
|
||
of arithmetic and branching per span before a single pixel moves. All of it is
|
||
known at encode time.
|
||
|
||
v6's record is `{u32 absolute GVRAM address, u16 jump displacement}` and nothing
|
||
else. The displacement jumps into an unrolled chain of eleven 24-pixel copy
|
||
units, so a span of any supported length is straight-line code with no loop, no
|
||
remainder, and no address arithmetic — `move.l (a0)+,a2` / `move.w (a0)+,d0` /
|
||
`jmp v6ch(pc,d0.w)`, then `movem.l` pairs. GVRAM is at $C00000 on every X68000,
|
||
so absolute destinations are a legitimate thing to bake into a stream.
|
||
|
||
Two consequences of that format, both cheap:
|
||
- **Span lengths are multiples of 24 pixels** and a run pads up to it. The
|
||
padding costs bytes and its own pixels, nothing else, and it is *correct on
|
||
screen*: a literal span carries true pixels of the current frame, so painting
|
||
a clean neighbour is a no-op visually.
|
||
- **A span may overrun the visible 256 pixels of its row by up to 23.** Free:
|
||
the line stride is 1024 bytes and only the first 512 are displayed, so the
|
||
overrun lands in the invisible half of the line.
|
||
|
||
### 30.3 The remainder path is where a short span actually dies
|
||
v5's cost per span, measured, against its length:
|
||
|
||
| span | 4 px | 8 px | 12 px | 16 px | 20 px | 24 px | 32 px |
|
||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||
| cycles/span | 180.3 | 240.9 | 296.3 | 261.8 | 347.7 | 401.9 | 430.7 |
|
||
| cycles/pixel | 45.08 | 30.11 | 25.46 | **16.36** | 17.65 | 17.27 | **13.46** |
|
||
|
||
A 12-pixel span costs *more* than a 16-pixel one. Everything below the 16-pixel
|
||
burst width goes through `move.l`/`move.w` at roughly 10 cycles a pixel plus the
|
||
per-span overhead, and 29's warning that "short spans are flattered" was
|
||
correct — but the fix is to pad them up to a burst, not to avoid them. v6 has no
|
||
remainder path at all, which is most of why its fit is linear to 0.3%.
|
||
|
||
### 30.4 Registers are the reason the per-pixel cost moved
|
||
FINDINGS 24's 9.08 cycles/pixel came from a fixed blit with 12 registers free
|
||
for `movem.l` and no live state. A span decoder keeps a stream pointer, a
|
||
destination and counters live, so v5 can spare only 8 registers per burst — 32
|
||
bytes instead of 48 — and pays 10.46 cycles/pixel for it. v6 gets back to 12
|
||
registers precisely because the encoder holds the state instead, and lands at
|
||
9.152. **The per-pixel figure is a function of how much the decoder has to
|
||
remember**, which is not something the FINDINGS 24 measurement could have shown.
|
||
|
||
Two smaller results, both cheap and both worth having on the record:
|
||
- **Odd-`x` alignment is free.** Spans starting at an odd pixel run their bursts
|
||
at `addr mod 4 == 2` and cost 259.0 cycles/span against 261.8 aligned — inside
|
||
the timing granularity. The 68000's 16-bit bus does not care, as expected;
|
||
now it is measured rather than assumed.
|
||
- **A full-row span is 154 cycles per 4x4 block**, the floor this mode can
|
||
reach, against V1's measured 299.9.
|
||
|
||
### 30.5 Re-pricing: the trade holds, and it is smaller
|
||
`tools/analysis/12_span_tradeoff.py` now runs on measured constants. Same greedy
|
||
(buy the best cycles-saved-per-byte until the bus budget is gone), same
|
||
unmodified mode maps, same Singe window:
|
||
|
||
| | today | 29 (derived) | **30 (measured)** |
|
||
|---|---:|---:|---:|
|
||
| `sasi` median frame | 74.4% | 43.0% | **52.0%** |
|
||
| `sasi` worst frame | 136.2% | 106.2% | **108.7%** |
|
||
| `sasi` frames missing | 37/120 | 8/120 | **10/120** |
|
||
| `sasi` bitrate | 101.7 KB/s | 453.2 | **448.0 KB/s** |
|
||
| `scsi` median frame | 94.9% | 69.4% | **74.6%** |
|
||
| `scsi` frames missing | 51/120 | 18/120 | **25/120** |
|
||
|
||
And the break-even moved. Cycles per 4x4 block in a run of L blocks, v6, with
|
||
each of the run's 4 spans padded to a whole 24-pixel unit:
|
||
|
||
| L | 1 | 2 | 4 | 8 | 16 | 64 |
|
||
|---|---:|---:|---:|---:|---:|---:|
|
||
| cycles/block | 1053 | 527 | **263** | 242 | 176 | 154 |
|
||
|
||
So a run beats all-V1 (299.9) **from L=4 up**, not from L=2 as 29.3 claimed, and
|
||
runs of 1-3 blocks all cost the same 1053 cycles because they pad to the same
|
||
single unit. A cost-aware mode decision should not offer a span below 4 blocks
|
||
at all.
|
||
|
||
### 30.6 29.4 survives: a scene cut still fits at 12fps
|
||
Mixing a fraction `x` of a 100%-changed frame as full-row spans against V1 for
|
||
the rest, on measured costs (154 cycles and 33.4 bytes per block):
|
||
|
||
- CPU needs `x >= 0.196`
|
||
- the 40,977 B/frame bus budget allows `x <= 0.373`
|
||
|
||
The interval is not empty — narrower than 29.4's 0.19..0.39, same conclusion.
|
||
FINDINGS 28.5's "a scene cut cannot fit" was a ceiling of the bitstream, not of
|
||
the machine, and that now rests on a measurement. `12_span_tradeoff.py` prints
|
||
this arithmetic and will say so if it ever stops being true.
|
||
|
||
### 30.7 What this does NOT settle
|
||
The three remaining items of 29.5 are unchanged and are now **more** load-bearing,
|
||
because the measured design runs at 448 KB/s of a 488 KB/s pipe rather than 453:
|
||
re-run the ring-buffer simulation at that rate, confirm the 4 Mbps figure's
|
||
provenance, and confirm DMA rather than PIO. A PIO fallback would put a 448 KB/s
|
||
transfer back on the CPU this mode exists to relieve.
|
||
|
||
Also unmeasured: **the parse cost of a span-heavy stream**. Every figure here
|
||
times the copy. The 68000 also has to read the mode map and dispatch: the
|
||
re-priced `sasi` stream buys 8773 spans across 120 frames, a mean of 73 a frame,
|
||
and each one's three-instruction dispatch is inside the fitted 43.7 — but the
|
||
mode-map walk that decides a span exists is not. `decode.s` does not implement
|
||
spans yet.
|
||
|
||
## 31. The mode decision can see cycles now, and it costs 0.26 dB (session 8)
|
||
|
||
FINDINGS 28 left the decoder missing 31% of frames at `sasi` and 42% at `scsi`
|
||
while the mode decision minimised `D + lam*R` — distortion against BYTES — on a
|
||
machine whose binding budget is CYCLES. This is the second controller.
|
||
|
||
`vq_hybrid.decide(ctx, lam, mu)` now minimises `D + lam*bytes + mu*cycles`, and
|
||
`ratectl.encode_rate_controlled(cycle_budget=...)` bisects `mu` per frame
|
||
against 833,333 cycles with the `lam` bisection nested inside it.
|
||
`tools/analysis/13_cpu_ratectl.py` measures what it costs.
|
||
|
||
### 31.1 The result
|
||
Worst sustained window, 120 frames, same targets, same quality floors:
|
||
|
||
| | PSNR | KB/s | CPU median | CPU max | frames missing |
|
||
|---|---:|---:|---:|---:|---:|
|
||
| `sasi` bytes only | 27.22 dB | 109.5 | 74.4% | 136.2% | **37/120** |
|
||
| `sasi` + cycle ceiling | **26.95 dB** | 109.4 | 81.5% | 110.6% | **1/120** |
|
||
| `scsi` bytes only | 29.90 dB | 280.0 | 94.9% | 146.6% | **51/120** |
|
||
| `scsi` + cycle ceiling | **29.27 dB** | 278.6 | 99.6% | 110.6% | **1/120** |
|
||
|
||
**36 of 37 misses at `sasi` for 0.26 dB, 50 of 51 at `scsi` for 0.62 dB.** The
|
||
bitrate does not move: the byte controller still binds, and mu changes *which*
|
||
modes are bought rather than how many bytes.
|
||
|
||
`sasi` pays less quality than `scsi` because it had less to give up: it was
|
||
already short of bytes, so the cycle-cheap directions it takes (V4 -> V1, and
|
||
blocks it can afford to hold) were near where the byte lagrangian already sat.
|
||
28.8 predicted the shape of this and got the sign right.
|
||
|
||
Mode mix, `sasi`, bytes-only -> with the ceiling: SKIP 46.4 -> 47.1%,
|
||
V1 19.8 -> 23.0%, **V4 25.2 -> 20.3%**, RAW 8.5 -> 9.6%. At `scsi` the V4
|
||
collapse is dramatic — **15.0 -> 5.3%**, with RAW taking it at 41.3 -> 43.2%,
|
||
which is 28.8's inversion happening in practice: RAW is dearer in bytes and
|
||
cheaper in cycles, so a byte-rich profile buys its way out of V4.
|
||
|
||
Only **46 of 120 frames need any mu at all** at `sasi`; the median frame is
|
||
decided at mu=0 and is unchanged from session 6.
|
||
|
||
### 31.2 The one frame that cannot fit is the intra frame, not a hard case
|
||
Both profiles miss exactly one frame, both at 110.6% — the all-V1 floor of
|
||
FINDINGS 28.5 — and in both it is **frame 0**. It has no previous
|
||
reconstruction, so every block must be coded, which is the definition of a
|
||
100%-changed frame. A scene cut mid-stream is the same thing.
|
||
|
||
That is the correct behaviour rather than a failure, and it is worth being
|
||
explicit about why: at `MU_CLIFF` a block only becomes SKIP if holding the
|
||
previous reconstruction costs less than ~28,665 units of distortion. A frame
|
||
with nothing on screen worth holding stays fully coded and is emitted **late on
|
||
purpose**, exactly as a frame that will not fit at `LAM_CLIFF` is emitted over
|
||
budget. Freezing a cut to make a deadline is the worse failure.
|
||
|
||
### 31.3 28.7 was too pessimistic, and the reason is instructive
|
||
28.7 estimated that only ~three quarters of the misses were the encoder's to
|
||
fix — 26 of 37 at `sasi` — because re-coding every non-SKIP block as V1 still
|
||
missed 11 frames. Measured, the controller fixes **36 of 37**.
|
||
|
||
The gap is that 28.7's floor held the SKIP set fixed and asked "how cheap can
|
||
the blocks we already decided to draw be?". The real decision can also **move a
|
||
block to SKIP**, paying distortion for it, and above ~90% non-SKIP that is the
|
||
only lever left. So 28.7's floor was a floor for a fixed SKIP set, not for the
|
||
mode decision. Two conclusions of 28.7 stand: the profiles are an I/O axis and
|
||
both must fit the same 10 MHz budget.
|
||
|
||
### 31.4 SKIP is not a constant, and the way out is two cost functions
|
||
A SKIP block costs 13.25 cycles when all four blocks sharing its header byte are
|
||
SKIP (one `tst.b` clears the group) and ~45 in a mixed byte, so its price
|
||
depends on its neighbours — which a per-block lagrangian cannot see. Picking one
|
||
number is a real trade: 45 overcharges clustered SKIPs and pushes the encoder
|
||
away from the mode that saves the most cycles, 13.25 undercharges isolated ones
|
||
and lets frames overrun.
|
||
|
||
The resolution is that **the budget check does not have to use the same cost
|
||
function as the mode decision**. `decide()` uses 13.25 purely to *rank* modes
|
||
within a block, where the choice only scales the incentive (the V1-SKIP gap
|
||
moves 12% between the two candidates). The controller scores whole frames with
|
||
`vq_hybrid.cycles()`, the exact clustered rule, validated to 1 point against the
|
||
68000 — so the bisection converges on what the machine will really do, whatever
|
||
the ranking constant was. That function is now defined once and imported by
|
||
`11_cpu_budget.py`, rather than living in two places that can drift apart.
|
||
|
||
### 31.5 Both controllers are gated against decoder drift
|
||
The mu controller varies the mode map frame to frame exactly as the lam
|
||
controller does, so it is exposed to the FINDINGS 26.1 failure — an encoder
|
||
reporting a reconstruction the decoder will never produce. `09_ratectl_drift.py`
|
||
now runs **both** configurations and both report 0/120 drifting frames, 0.00 dB
|
||
overstatement. The CPU ceiling is on by default in `encode.py`
|
||
(`--no-cpu-fit` restores session 7 behaviour).
|
||
|
||
### 31.6 With spans on top, the window fits completely
|
||
Re-running the span pricing of FINDINGS 30 against a cost-aware container —
|
||
lever B first, then lever A on what it leaves:
|
||
|
||
| `sasi` | bytes only | + cycle ceiling | + ceiling + spans |
|
||
|---|---:|---:|---:|
|
||
| median frame | 74.4% | 81.5% | **56.8%** |
|
||
| worst frame | 136.2% | 110.6% | **91.5%** |
|
||
| frames missing | 37/120 | 1/120 | **0/120** |
|
||
| bitrate | 101.7 KB/s | 101.6 | 449.3 KB/s |
|
||
|
||
The intra frame lands at 91.5% — spans are what make a full redraw fit, which is
|
||
30.6's arithmetic arriving in a real container. That row is still a **model** of
|
||
a bitstream nothing implements; the two levers have never run on the 68000
|
||
together, and the ring-buffer question of 30.7 gets sharper at 449 KB/s.
|
||
|
||
## 32. SASI is dropped, and the reason is capacity, not bandwidth (session 9)
|
||
|
||
**USER DECISION**: drop the `sasi` profile. A SASI volume on this machine is
|
||
limited to 40 MB, and the game does not fit in one.
|
||
|
||
That ends the two-quality-mode decision of session 2. `scsi` is now the only
|
||
profile, and `encode.py --profile` has one choice. The retired 110 KB/s rate
|
||
point is not deleted from the record, for the reason in 32.3.
|
||
|
||
### 32.1 How much video there actually is
|
||
Measured off the source Blu-ray rather than recalled: the unique scene footage
|
||
is streams `00000`-`00201`, **1366.6 s = 22.8 min**. The longer streams
|
||
(`00215` 1376 s, `00216` 1152 s, `00223` 566 s) are compilations of the same
|
||
material and are not additional content — 00223 is the window every codec
|
||
measurement in this project has been taken on. Total across all 224 streams is
|
||
88.3 min, which is the figure to *not* quote.
|
||
|
||
22.8 min agrees with the ~22 min of laserdisc footage the arcade original is
|
||
usually credited with, which is the cross-check that the compilations really
|
||
are duplicates.
|
||
|
||
At the rates this codec has actually produced, including the 7.8 KB/s audio
|
||
allowance:
|
||
|
||
| stream | rate | whole game |
|
||
|---|---:|---:|
|
||
| retired 110 KB/s profile | 109.4 KB/s | **146.0 MiB** |
|
||
| `scsi`, measured (FINDINGS 31) | 278.6 KB/s | **371.8 MiB** |
|
||
| `scsi` + spans (MODEL, 31.6) | 449.3 KB/s | **599.6 MiB** |
|
||
|
||
### 32.2 Where the 40 MB actually comes from
|
||
It is not a bus-addressing limit. MAME 0.277's `src/mame/sharp/x68k_hdc.cpp`
|
||
builds the SASI LBA from a 6-byte Group-0 CDB as
|
||
`(cmd[1] & 0x1f) << 16 | cmd[2] << 8 | cmd[3]` — **21 bits of 256-byte blocks,
|
||
so 512 MiB is addressable per unit**. `call_create` makes a 20 MB image
|
||
(`0x13c98` blocks) because that is what a period drive was.
|
||
|
||
So the 40 MB ceiling is a **Human68k / IPL volume-format and period-drive
|
||
limit**, not something the SASI command set imposes. That distinction does not
|
||
rescue the profile: four units at 40 MB is 160 MiB, and 146.0 MiB of video
|
||
would consume essentially the entire SASI address space of the machine at the
|
||
*lowest* rate this codec has ever produced, leaving nothing for Human68k, the
|
||
player, or the game's own data.
|
||
|
||
*Scope: the 21-bit CDB and the 256-byte block are read out of MAME's
|
||
implementation. The 40 MB volume figure is the user's, and is consistent with
|
||
Human68k's SASI partitioning; it has not been measured here.*
|
||
|
||
### 32.3 The rate point may come back, under a different name
|
||
Dropping SASI removes an interface, not a bitrate, and the two are on different
|
||
axes — the profile axis has been I/O bandwidth only since FINDINGS 28.7. The
|
||
awkward part is that **capacity and bandwidth now pull in opposite directions**:
|
||
|
||
- the only period medium with room for 371.8 MiB (let alone 599.6) is
|
||
**CD-ROM** at 540-650 MB, and
|
||
- a **1x CD-ROM sustains ~150 KB/s**, which is *below* the surviving 280 KB/s
|
||
profile and much nearer the rate that was just retired.
|
||
|
||
A SCSI hard disk has the bandwidth but has to be large for the era at 372 MiB.
|
||
**The user's call was to ship `scsi` as the only profile now and settle the
|
||
medium when the pipe is measured** — the blocked disk benchmark
|
||
(`docs/BENCHMARK.md`) and the DMA-vs-PIO check of FINDINGS 29.5.
|
||
|
||
**Correction to the framing above, found after that call was made.** The
|
||
medium is less open than this section first presented it. FINDINGS 21.2 already
|
||
committed the deployment target to **SD-backed SCSI (BlueSCSI / SCSI2SD)**, in
|
||
session 2, and that is the premise the whole 488 KB/s constant rests on. On SD
|
||
there is no capacity problem at any rate this codec produces — an SD card is
|
||
gigabytes — and no seek tail either. So:
|
||
|
||
- **Capacity does not choose between the survivors.** It killed SASI, whose
|
||
40 MB ceiling is a Human68k volume-format limit that SD emulation does not
|
||
lift, and it does not bind on SD-backed SCSI at all.
|
||
- **CD-ROM is the one that capacity rules out**, not in. With spans the stream
|
||
is 487.1 KB/s = **650.1 MiB** for the whole game, past a CD's ~620 MiB
|
||
usable — and 487 KB/s is more than 3x a 1x CD-ROM's ~150 KB/s. A CD-ROM
|
||
delivery would mean giving up the span lever *and* re-deriving a profile
|
||
around 150 KB/s.
|
||
|
||
So the open question is not "which medium" but the one FINDINGS 29.5/30.7
|
||
already had: **confirm the 488 KB/s figure's provenance, and confirm DMA**.
|
||
`profile_gen.py` exists precisely to re-derive a profile from a measured
|
||
bandwidth once there is one.
|
||
|
||
### 32.4 What MAME says about the SCSI path that survives
|
||
Read out of MAME 0.277 while settling 32.2, and directly relevant because the
|
||
medium decision is now the thing gating the profile:
|
||
|
||
**The CZ-6BS1's DMA is real and fully modelled.** `x68k_scsiext.cpp` wires the
|
||
MB89352's DREQ to the expansion slot and replaces the data register at
|
||
`$EA0015` with DMA-aware glue: on a DMA cycle (`m_slot->exown()`, driven by
|
||
`m_hd63450->own()`) a read goes to `spc->dma_r()` and #DTACK is negated until
|
||
DRQ asserts. `x68k.cpp:1114` closes the loop the other way
|
||
(`out_dtack_callback -> hd63450_device::dtack_w`). That is a genuine
|
||
DMAC-driven transfer with hardware flow control, on the **stock `x68000`
|
||
driver** — the one MAME marks working. This is the configuration FINDINGS 29.5
|
||
asked about, and the answer for this board is **DMA, not PIO**.
|
||
|
||
**The internal SCSI of the Super/XVI/030 is NOT modelled that way**, and it is
|
||
a trap. `x68k.cpp:1176` reads, verbatim, `// TODO: duplicate DMA glue from
|
||
CZ-6BS1`. So MAME's internal SCSI is PIO-only. A benchmark run on `x68ksupr`
|
||
would measure a PIO fallback the real machine does not have — on top of those
|
||
drivers already being MACHINE_NOT_WORKING (FINDINGS 28.7). **Benchmark
|
||
`x68000 -exp1 cz6bs1`, not `x68ksupr`.**
|
||
|
||
**CD-ROM is a first-class SCSI device on the internal bus** — `x68k.cpp:1168`
|
||
puts an `NSCSI_CDROM` at ID 6 by default — but the CZ-6BS1 card's own device
|
||
list offers `harddisk` only. So the CD-ROM delivery route of 32.3 is
|
||
emulatable, but not on the board whose DMA is modelled, without a source
|
||
change.
|
||
|
||
None of this is a transfer RATE. `docs/BENCHMARK.md`'s split still holds and is
|
||
worth restating because 32.3 defers a decision to a measurement: MAME can
|
||
settle whether the path works and whether it is DMA, and **cannot** settle
|
||
KB/s, because its device models are functional rather than transfer-timing
|
||
accurate. The rate half of the medium question needs derivation or real
|
||
hardware, not a longer MAME run.
|
||
|
||
## 33. The container carries its own alignment: DLX1 -> DLX2 (session 9)
|
||
|
||
The encoder gap left open since session 7 (FINDINGS 28.3, STATUS item 4) is
|
||
closed. `encode.py` now emits **DLX2**, which pads every frame record up to a
|
||
4-byte boundary — the first one included, by padding the codebook tables so
|
||
`off_frm` is aligned. `dlx.py` reads both versions; DLX1 containers stay
|
||
readable because every measurement in FINDINGS 28-31 was taken on one.
|
||
|
||
Measured on the same 120-frame window:
|
||
|
||
| | record starts not 4-aligned | padding cost |
|
||
|---|---:|---:|
|
||
| DLX1 (through session 8) | **94/120** | 0 (the loader added 180 B) |
|
||
| DLX2 (now) | **0/120** | 160 B = 1.33 B/frame = **16 B/s** |
|
||
|
||
16 B/s against 278.6 KB/s is 0.006% of the stream. The thing it buys is not
|
||
speed: an odd `move.l (a0)+` on a 68000 is an **address error**, which vectors
|
||
into the IPL and presents as an infinite loop, not as a slow read. That is the
|
||
bug that cost session 7 an afternoon.
|
||
|
||
`tools/bench/prep_dlx.py` still realigns at load time and now says whether it
|
||
had to — `0/120 record starts unaligned -- the container carries its own
|
||
padding` on a DLX2 input. It is kept rather than deleted because it is what
|
||
makes the session 7-8 containers decodable, and those are the containers the
|
||
published timings belong to.
|
||
|
||
**Cross-check that this changed nothing else:** re-encoding the `scsi` window
|
||
with the DLX2 writer reproduces FINDINGS 31.1 exactly — 29.27 dB, 278.6 KB/s,
|
||
median 99.6% / max 110.6% of a 12fps frame, 1/120 frames missing. The padding
|
||
is additive; it does not touch the mode decision.
|
||
|
||
## 34. The cost model, checked against the machine on a cost-aware container (session 9)
|
||
|
||
STATUS item 1. Everything in FINDINGS 31 was the validated cost MODEL
|
||
(`vq_hybrid.cycles`) applied to a container it had never been checked against —
|
||
the 1-point validation of 28.2 belongs to the *session 7* stream. This is the
|
||
cost-aware container timed on the emulated 68000, same harness, same scope
|
||
(instruction cycles, zero-wait-state GVRAM, interrupts masked; a LOWER BOUND).
|
||
|
||
| anchor | non-SKIP | model | measured | error |
|
||
|---|---:|---:|---:|---:|
|
||
| min non-SKIP | 15.4% | 254,683 cyc / 30.6% | 262,751 / 31.5% | **-3.07%** |
|
||
| median | 53.2% | 681,199 / 81.7% | 690,251 / 82.8% | **-1.31%** |
|
||
| p90 | 75.7% | 832,116 / 99.9% | 834,213 / 100.1% | **-0.25%** |
|
||
| max non-SKIP | 100.0% | 921,293 / 110.6% | 921,187 / 110.5% | **+0.01%** |
|
||
| whole 120-frame mean | — | 649,089 / 77.9% | 657,081 / 78.8% | **-1.22%** |
|
||
|
||
The model holds, and its error is **signed**: it under-predicts by 1-3% on light
|
||
frames and converges to exact on heavy ones. That is the right direction to be
|
||
wrong in for a ceiling controller — the bisection is tightest where the model is
|
||
most accurate — but it means the median frame is ~1 point dearer than FINDINGS
|
||
31 reports, not cheaper.
|
||
|
||
The four synthetic single-mode frames reproduce session 7 exactly: all-V1
|
||
110.5%, all-V4 165.2%, all-RAW 147.6%, all-SKIP 4.9%. Those are properties of
|
||
`decode.s`, not of the container, so agreeing across two different streams is
|
||
the cross-check that the harness is measuring what it claims.
|
||
|
||
### 34.1 The 23-minute "hang" was the buffering trap again
|
||
The session-8 note said this run "was still going at 12 minutes of CPU". It was
|
||
re-run here and sat at 99.9% CPU for **23 minutes** with a 0-byte log, then was
|
||
killed. Re-launched under **`stdbuf -oL`** with `-seconds_to_run 60`, the
|
||
identical plan completed in about **25 seconds of wall time** and printed every
|
||
line as it went — MAME reports `Average speed: 528.72% (52 seconds)`, so the
|
||
whole plan needs ~52 emulated seconds and the machine runs it at 5x realtime.
|
||
|
||
The lesson is the one already in STATUS, one level deeper: it is not enough to
|
||
write MAME's output to a file instead of a pipe. **A file is block-buffered too**,
|
||
so a long MAME run is unobservable until it exits, and an unobservable run that
|
||
is merely finishing looks exactly like one that is wedged. Session 8 lost the
|
||
measurement to that, and session 9 lost 23 minutes to it before spending 25
|
||
seconds getting the answer. **`stdbuf -oL` on every MAME job that prints
|
||
progress.**
|
||
|
||
## 35. The CPU budget has never had the disk in it (session 9)
|
||
|
||
> **TESTED BY 38 AND IT STANDS.** Session 10 first argued that the flat
|
||
> subtraction here is too pessimistic -- that the disk DMA could hide in bus
|
||
> cycles the CPU was not using -- and scored the same window at 53/120 instead
|
||
> of 84/120. **That was wrong.** A 68000 has no cache and a two-word prefetch
|
||
> queue, so it stalls as soon as another master takes the bus; DMA time is
|
||
> additive, which is exactly what this section assumed. The 84/120 stands and
|
||
> 38.3 now reproduces it.
|
||
|
||
**Raised by the user: "PIO is such a CPU killer. DMA is not. I'm concerned about
|
||
us drawing the wrong conclusions."** The concern is correct, and it is larger
|
||
than the labelling question of 32.4. This is the seventh false premise this
|
||
project has caught, and the most expensive one.
|
||
|
||
Every CPU figure in FINDINGS 24 through 34 is measured against **833,333 cycles
|
||
per frame**, the full 10 MHz clock divided by 12 fps. Nothing has ever been
|
||
subtracted from it for moving the bitstream off the disk. The decoder has been
|
||
scored as though the data arrives for free.
|
||
|
||
### 35.1 What the transfer actually costs
|
||
`profile_gen.py` has carried `DMA_CLOCKS_PER_WORD = 8` since session 2 (FINDINGS
|
||
5, an **ESTIMATE** from HD63450 timing, never measured) and prints a "DMA steal"
|
||
line — but that line was only ever compared against the 38.3% *blit* figure of
|
||
FINDINGS 17, which FINDINGS 24 superseded and which was never the decoder cost.
|
||
It was never debited from the decoder budget.
|
||
|
||
At the rates that matter, on a 10 MHz 68000:
|
||
|
||
| stream | DMA @ 8 clk/word | PIO, unrolled (~12 clk/B) | PIO, byte loop (~20 clk/B) |
|
||
|---|---:|---:|---:|
|
||
| `scsi`, 278.6 KB/s | **11.4%** | 34.2% | 57.1% |
|
||
| `scsi` + spans, 487.1 KB/s | **20.0%** | 59.9% | **99.8%** |
|
||
|
||
The PIO columns are hand-derived floors, not measurements: a byte from an I/O
|
||
register plus a store is 16 cycles on a 68000 before any loop overhead. They are
|
||
here to size the risk, and the size of the risk is that **PIO at the span rate
|
||
consumes the entire machine**.
|
||
|
||
### 35.2 What that does to the conclusions of FINDINGS 31
|
||
Debiting the DMA steal — the *cheap* case, the one we are hoping for:
|
||
|
||
| | KB/s | steal | budget left | median | p90 | worst | fits? |
|
||
|---|---:|---:|---:|---:|---:|---:|:--|
|
||
| `scsi` today | 278.6 | 11.4% | 738,238 | **112.4%** | 112.9% | 124.8% | **no** |
|
||
| `scsi` + spans | 487.1 | 20.0% | 667,070 | **98.3%** | 102.8% | 114.3% | **no** |
|
||
|
||
FINDINGS 31's headline — "1 frame of 120 misses" — is measured against a budget
|
||
with no I/O in it. With DMA debited the surviving profile does not fit at all:
|
||
the *median* frame is over. And 31.6's "with spans the window fits completely"
|
||
becomes a worst frame of 114.3%, because **the span lever buys cycles by
|
||
spending bandwidth, and the bandwidth comes back out of the CPU as steal.**
|
||
Spans still help — 112.4% -> 98.3% at the median, 14 points — but they no longer
|
||
close the gap on their own.
|
||
|
||
### 35.3 Why this is not settled by the DMA finding of 32.4
|
||
32.4 established that the CZ-6BS1's DMA *path* exists and is modelled. Three
|
||
things it does not establish, and all three are load-bearing:
|
||
|
||
1. **DMA vs PIO is a property of OUR player, not of the board.** The hardware
|
||
supports DMA; if the player reads through IOCS and IOCS does PIO, we get PIO
|
||
and the table above. `docs/BENCHMARK.md` item 4 already proposed driving the
|
||
MB89352 registers directly for exactly this reason — that is now not an
|
||
optimisation but the difference between fitting and not.
|
||
2. **8 clocks per word has never been measured.** It is now the single most
|
||
load-bearing unmeasured number in the project: at 8 the port is marginal, at
|
||
12 it is dead, at 4 it is comfortable. It comes from a datasheet reading in
|
||
session 2 and nothing has checked it since.
|
||
3. **MAME cannot settle it.** Its device models are functional, not
|
||
transfer-timing accurate (BENCHMARK.md), and it models no GVRAM wait states
|
||
either — so a MAME run can confirm the transfer is a DMA cycle and cannot
|
||
price it. This needs derivation from the HD63450 and MB89352 datasheets, or
|
||
real hardware.
|
||
|
||
### 35.4 What this does and does not overturn
|
||
It does **not** overturn the decoder measurements: 300/448/400 cycles per block
|
||
and the model validation of FINDINGS 34 are properties of `decode.s` and stand
|
||
unchanged. What it overturns is every statement of the form "N frames of 120
|
||
miss the budget", because the budget was wrong. Those all need re-running
|
||
against `833,333 * (1 - steal)` once `steal` is a measurement rather than a
|
||
datasheet estimate.
|
||
|
||
It also sharpens the framerate question of STATUS item 5 considerably. At 10 fps
|
||
the budget is 1,000,000 cycles and the same DMA steal is proportionally smaller
|
||
per frame, which is now a much stronger argument for 10 fps than "one late frame
|
||
per cut" ever was.
|
||
|
||
### 35.5 `11_cpu_budget.py` now debits it, and 10 fps absorbs it
|
||
The tool takes `--io dma|pio|none` (default **dma**) and prints the budget it is
|
||
actually scoring against. On `tmp/rc_fr_singe_scsi_cpufit.dlx`:
|
||
|
||
| `--io` | budget left | median | worst | frames missing |
|
||
|---|---:|---:|---:|---:|
|
||
| `none` — the pre-session-9 premise | 833,333 | 99.6% | 110.6% | **1/120** |
|
||
| `dma` (8 clk/word, estimated) | 738,234 | 112.4% | 124.8% | **84/120** |
|
||
| `pio` (12 clk/B, floor) | 548,036 | 151.4% | 168.1% | **120/120** |
|
||
|
||
`--io none` prints a warning naming FINDINGS 35, so the old number cannot be
|
||
produced by accident.
|
||
|
||
**At 10 fps and DMA the same container goes back to 1/120** — median 93.7%,
|
||
worst 104.0%. That is conservative, because it holds the 12 fps byte rate: a
|
||
real 10 fps encode carries ~17% fewer bytes per second, so the steal falls too.
|
||
|
||
This changes what the framerate decision (STATUS item 5) is *for*. It was a
|
||
quality question about one late frame per scene cut. It is now the lever that
|
||
pays for the disk, and on current estimates it is the difference between a
|
||
stream that fits and one that misses 70% of its frames.
|
||
|
||
## 36. A `scsi` window does not fit in the machine the test rig emulates (session 9)
|
||
|
||
Swapping the decoder gate onto the surviving profile's container made it fail —
|
||
`frame 119 not pixel-exact: 49,005 px differ`. That is not a decoder bug and not
|
||
the DLX2 change. **The container does not fit in RAM.**
|
||
|
||
`tools/bench/decode.lua` loads the entire stream into emulated memory at
|
||
`STREAM = 0x30000`, and the locked target is a stock **2 MB** machine:
|
||
|
||
| container | stream | ends at | verdict |
|
||
|---|---:|---:|---|
|
||
| session 7-8 `sasi` | 1,108,888 B | 0x13EB98 = 1.25 MB | fits |
|
||
| `scsi` cost-aware | 2,840,860 B | 0x2E591C = 2.90 MB | **overruns 0x200000 by 940 KB** |
|
||
|
||
The loader wrote 940 KB past the top of memory, the decoder then parsed whatever
|
||
that reads back as, and the run neither completed its sequential pass nor drew
|
||
the right picture. Every 68000 decode verification before session 9 was done on
|
||
a container small enough to fit by accident — the `sasi` profile was a third the
|
||
bitrate, so nobody met this.
|
||
|
||
**This is a property of the test rig, not of the player.** The shipping player
|
||
streams from disk into a ring buffer and holds seconds of video, not minutes.
|
||
But it does bound what the rig can prove: at 278.6 KB/s, a 2 MB machine holds
|
||
about **6.7 seconds** of stream, so the strongest test in the tree can only ever
|
||
audit a prefix of a window.
|
||
|
||
The fix keeps the test honest rather than making it pass:
|
||
- `prep_dlx.py` truncates the frame list to what fits, **prints that it did**,
|
||
and takes `--ram` / `--all-frames`. On the `scsi` window it keeps **80 of
|
||
120** frames.
|
||
- `verify_decode.py` takes `--nframes` so the reference decoder replays exactly
|
||
the prefix the 68000 decoded, instead of running 40 frames ahead of it.
|
||
- `check.sh` reads the count back out of `decode_meta.lua` and passes it through,
|
||
and now **fails loudly if the sequential pass did not complete** — the missing
|
||
`snapshot taken` marker — instead of reporting a pixel diff against a
|
||
half-drawn screen. That guard is what turned this from a mystery into a
|
||
five-minute diagnosis.
|
||
|
||
Verifying a prefix is still a real test: SKIP blocks make every frame a claim
|
||
about the one before it, so frame 79 is only correct if all 80 were. What is
|
||
lost is coverage of the last 40 frames, and the honest way to get it back is to
|
||
gate on more than one window rather than to pretend one pass covers everything.
|
||
|
||
**The timing confirms the diagnosis.** Truncated to 80 frames the pass completes
|
||
in **8 emulated seconds** and the frame is pixel-exact; the model predicts
|
||
~6.6 s for 80 frames at this container's cost, so that is the expected number.
|
||
The 120-frame run that overran RAM could not finish the same work in **44**.
|
||
A decoder reading garbage does not run slowly for an interesting reason — it was
|
||
parsing lengths out of unmapped memory and walking wherever they pointed. Any
|
||
"the decoder is 4x slower than the model on RAW-heavy streams" conclusion drawn
|
||
from that run would have been entirely false, which is the third time in this
|
||
session that an unobservable run nearly produced a wrong finding.
|
||
|
||
## 37. A second emulator, and MAME is not running the core we thought (session 10)
|
||
|
||
Every 68000 cycle figure in FINDINGS 24-35 came from one instrument. This is a
|
||
second one, run against byte-for-byte the same `decode.bin` and the same
|
||
container.
|
||
|
||
`tools/bench/c68k/` links **px68k's C68K core** into a headless harness: a
|
||
hand-built X68000 memory map, no SDL, no ROMs, no emulated machine. The decoder
|
||
touches nothing but RAM, the control block and GVRAM, so the machine around it
|
||
was never part of the measurement.
|
||
|
||
### 37.1 What the two instruments actually are
|
||
**MAME 0.277's `M68000` is not Musashi.** `src/devices/cpu/m68000/m68000.lst`
|
||
plus `m68000gen.py`: it is the microcode core, where timing emerges from the
|
||
modelled micro-sequence and 4-clock bus cycles. C68K is a static per-instruction
|
||
cycle table (`ORI_CLOCKS_*` / `EA_CLOCKS_*` in `c68kmacro.h`), hand-transcribed
|
||
from the Motorola manual by a different author.
|
||
|
||
Those are two different ways of arriving at a number, which is what makes the
|
||
agreement worth something. It would be worth much less if both were tables.
|
||
|
||
### 37.2 The harness is self-validating
|
||
It decodes all 80 frames and dumps the screen; `verify_c68k.py` checks it
|
||
against `tools/encoder/dlx.py` **pixel for pixel, on palette indices**. That is
|
||
the licence for the cycle numbers: the harness rebuilds px68k's memory model
|
||
from scratch -- byte-swapped RAM (`mem_wrap.c:420`), GVRAM word writes that
|
||
discard the high byte -- and any of it being subtly wrong would still print
|
||
plausible cycles. It could not print a pixel-exact 80-frame temporal recursion.
|
||
|
||
It does. **`decode.s` is now pixel-exact under two independent CPU cores.**
|
||
|
||
### 37.3 The numbers
|
||
```
|
||
anchor MAME C68K delta MAME C68K of a 12fps frame
|
||
min non-SKIP 42.8% 600982 620760 +3.29% 72.1% 74.5%
|
||
median 65.2% 841038 869036 +3.33% 100.9% 104.3%
|
||
p90 72.9% 836124 856872 +2.48% 100.3% 102.8%
|
||
max non-SKIP 100.0% 921187 923090 +0.21% 110.5% 110.8%
|
||
synthetic all-SKIP 40729 40946 +0.53% 4.9% 4.9%
|
||
synthetic all-V1 921187 923090 +0.21% 110.5% 110.8%
|
||
synthetic all-V4 1376881 1420754 +3.19% 165.2% 170.5%
|
||
synthetic all-RAW -- 1273298 -- 152.8%
|
||
```
|
||
|
||
The `all-RAW` cell is empty because MAME's timed pass did not reach it. That is
|
||
an operational fact worth recording: with `-video soft -nothrottle` this box runs
|
||
`x68000` at about **0.033x realtime**, so `decode.lua`'s eight anchors plus two
|
||
full passes — ~48 emulated seconds — cost ~25 minutes of wall clock, and two runs
|
||
were killed by their own `timeout`. The C68K harness does the same work in
|
||
seconds because it emulates a CPU and not a machine. **Anchor MAME runs by wall
|
||
clock, not by `-seconds_to_run`.**
|
||
|
||
**Cycle-table error is bounded at 3.3%, and it runs against us** -- C68K reads
|
||
high on every anchor. Nothing here rescues FINDINGS 35. The disagreement is
|
||
mode-dependent (all-V1 +0.21%, all-V4 +3.19%), so it localises to the V4 path's
|
||
indexed two-register `movem.l`, not to a systematic clock difference.
|
||
|
||
**FINDINGS 28.8 is confirmed independently**: under C68K, V4 (170.5%) still
|
||
costs more than RAW (152.8%). That conclusion inverts the encoder's mode
|
||
preference, so having it from a second core matters more than most.
|
||
|
||
### 37.4 What it does not settle
|
||
px68k has no bus-timing model anywhere in `x68k/*.c` -- grep it. Neither
|
||
instrument charges GVRAM wait states, so this is **the same lower bound,
|
||
measured twice**. It bounds cycle-table error. It says nothing about the
|
||
distance to a real X68000; that is still BENCHMARK.md Tier 3.
|
||
|
||
### 37.5 One trap, recorded because it will catch the next person
|
||
C68K is 64-bit-unsafe by construction: its `MOVEM` macros do
|
||
`src = (UINT32)(&D0)` -- they truncate the host address of the register file and
|
||
dereference it -- and `C68k_Set_Fetch` keeps the opcode-fetch base in a `UINT32`.
|
||
Under the default PIE the binary loads near `0x555555550000` and the first
|
||
`movem` segfaults. The Makefile builds `-no-pie` and the harness mmaps its arena
|
||
`MAP_32BIT`. Both are load-bearing, not tidiness.
|
||
|
||
## 38. The bus, measured: the project is bus-bound, not CPU-bound (session 10)
|
||
|
||
> **This supersedes part of 29 and part of 35.** FINDINGS 29's "the bus has 4x
|
||
> the headroom the CPU has" is true of the SCSI pipe and false of the 68000's
|
||
> local bus, and they are different resources. FINDINGS 35's flat CPU debit for
|
||
> the disk charges the CPU for bus cycles it was not going to use.
|
||
|
||
Everything since FINDINGS 24 has been costed in CPU clocks. The 68000 has
|
||
another budget nobody had counted: its **memory bus**, one 4-clock cycle at a
|
||
time, carrying instruction prefetch as well as data.
|
||
|
||
### 38.1 Two sources that check each other
|
||
`tools/bench/c68k/c68k_bench` counts every Read/Write callback the C68K core
|
||
makes -- exact, because C68K splits a long access into two word calls, which is
|
||
what the 16-bit bus does. It cannot count **instruction prefetch**: C68K reads
|
||
opcodes straight through a host pointer with no callback, and MAME exposes no
|
||
fetch count either.
|
||
|
||
So `tools/analysis/15_bus_occupancy.py` derives prefetch by walking
|
||
`decode.s`'s straight-line paths in `tools/bench/decode.lst` and multiplying by
|
||
each frame's mode histogram. The same walk also predicts the data half -- and
|
||
that half is measurable:
|
||
|
||
```
|
||
measured mean 66,700 data bus cycles/frame
|
||
derived mean 66,672 error -0.04% mean, 0.06% worst
|
||
```
|
||
|
||
The walk reproduces the measurement, so its prefetch figure stands on the same
|
||
footing. `15_bus_occupancy.py` exits non-zero if that check ever stops holding.
|
||
|
||
### 38.2 The result
|
||
```
|
||
mean median worst frame
|
||
bus slots in a frame 201,497 211,013 230,772
|
||
data accesses 66,672 68,044 105,216
|
||
instruction prefetch 108,002 110,982 122,910
|
||
total bus cycles 174,674 181,998 193,248
|
||
bus OCCUPANCY 86.7% 86.8% 88.3%
|
||
slots left for a DMAC 26,823 26,618 21,115
|
||
```
|
||
|
||
**The decoder occupies 86.7% of its own bus, and prefetch is 62% of that.** A
|
||
data-only count understates occupancy by about 2x, which is exactly the mistake
|
||
an instrumented emulator would lead you into.
|
||
|
||
Per mode, bus clocks against measured clocks: V1 204/299.9 (68%), V4 308/448.2
|
||
(69%), RAW 316/400.4 (79%), and the v6 span **9.0/9.152 (98%)**.
|
||
|
||
### 38.3 What that does to the frame budget -- and one wrong turn
|
||
The first thing done with 86.7% was to argue that FINDINGS 35's flat CPU debit
|
||
for the disk is too pessimistic: the decoder leaves ~26,800 bus slots a frame
|
||
idle against the disk's ~23,000, so score it as contention,
|
||
`frame = max(CPU clocks, 4 x bus cycles)`, and the window misses 53/120 rather
|
||
than 84/120.
|
||
|
||
**That is wrong, and the MC68450 manual is what says so.** A 68000 relinquishes
|
||
the bus on BGACK and cannot execute without it -- no cache, a two-word prefetch
|
||
queue that empties immediately. Worse, the DMAC does not interleave at operand
|
||
granularity by default: limited-rate auto-request hands it the bus in *bursts*
|
||
of `2(BT+4)` clocks out of a sample period of `2(BT+BR+5)`, taking
|
||
`2^-(BR+1)` of the bandwidth in slabs (MC68450 sect 5.2.3.2, Fig 5-2). During a
|
||
slab the CPU is stopped.
|
||
|
||
So **DMA time is additive to CPU time**, which is what FINDINGS 35 assumed all
|
||
along. `14_dmac_chain.py` reproduces its 84/120 exactly in the `today` column.
|
||
|
||
What 86.7% *does* say is worse than the thing it appeared to rescue: **there is
|
||
almost no room to overlap anything.** The 13.3% of bus slots the decoder leaves
|
||
idle are single gaps inside a `movem`-heavy loop, not windows a bus master can
|
||
be handed. Any design whose case rests on DMA hiding under CPU work on this
|
||
machine should be assumed dead until measured on hardware.
|
||
|
||
The measurement still earns its place: it is what prices the span painter
|
||
against a DMAC in 39, and it is the reason the answer there came out the way it
|
||
did.
|
||
|
||
### 38.4 What is not counted
|
||
Bus arbitration. The 68000's BR/BG/BGACK handover costs cycles a cycle-steal
|
||
DMA cannot avoid, and the disk debit here embeds it only insofar as FINDINGS 5's
|
||
8 clocks/word already does. Also: no GVRAM wait states, as everywhere since 24.
|
||
Both make the real occupancy **higher** than 86.7%, not lower.
|
||
|
||
## 39. The DMAC chain against the span: the datasheet says no (session 10)
|
||
|
||
FINDINGS 29.6 named "let the DMAC do the copy" the one lever that could move the
|
||
budget without spending a byte, and left it uncosted. This costs it, and the
|
||
answer is **no** -- but only after the constants came from the MC68450 manual
|
||
rather than from bus arithmetic, which is the whole lesson of the section.
|
||
|
||
### 39.1 They are the same container
|
||
v6's record is `{u32 absolute GVRAM address, u16 jump displacement}` = 6 bytes.
|
||
An MC68450/HD63450 **array-chaining entry** is `{u32 memory address, u16
|
||
transfer count}` = 6 bytes. Set the channel dual-address, direction
|
||
device->memory, Sequence Control counting both addresses up: MAR reloads per
|
||
entry (the GVRAM destination), DAR walks the stream buffer, MTC is the span's
|
||
word count. **The chain array IS the span table.** Every byte figure in
|
||
FINDINGS 30 carries over, and this is not a fork in the format -- the encoder
|
||
emits the same thing either way, only the executor changes. That much is real
|
||
and survives everything below.
|
||
|
||
### 39.2 The first answer was wrong by a clock
|
||
Session 10 first derived the DMAC's cost from bus arithmetic: moving a pixel is
|
||
a read cycle plus a write cycle, 2 bus cycles, 8 clocks, against v6's measured
|
||
9.152 -- a 12.6% edge. On that basis the design scored 1/120 frames over budget
|
||
against v6's 10/120 and looked decisive.
|
||
|
||
The datasheet does not agree. **MC68450 Fig 4-25 sheet 4**, dual address /
|
||
operand size WORD / device size 16 bits, D->M or M->D:
|
||
`{WORD READ, WORD WRITE}` = **9 CLOCKS**. Confirmed by the long-operand row, two
|
||
of each for 18. And **Fig 4-25 note 2** says why: the DMAC's reads take four
|
||
clocks and its **writes take five**. The 68000 writes in four.
|
||
|
||
| per pixel | clocks | source |
|
||
|---|---:|---|
|
||
| DMAC, dual-address word, two 16-bit ports | **9.000** | MC68450 Fig 4-25 sheet 4 |
|
||
| v6 `movem` chain | **9.152** | MEASURED, FINDINGS 30 |
|
||
|
||
**1.7%.** One clock on every DMAC write is the entire difference between a
|
||
12.6% win and a rounding error. Per span, sequential array chaining costs
|
||
**36 clocks** (Fig 4-25 sheet 1: three word reads for the 6-byte entry, plus
|
||
reload) against v6's measured 43.7 -- the DMAC's one genuine edge, and it is
|
||
7.7 clocks.
|
||
|
||
### 39.3 Scored additively, as 38.3 requires
|
||
```
|
||
today v6 span v6 fine tail DMAC chain
|
||
bitrate KB/s 270.8 479.2 479.9 479.9
|
||
frame, median 108.1% 99.3% 96.5% 95.0%
|
||
frame, worst 114.7% 112.0% 111.4% 110.3%
|
||
frames missing 84/120 55/120 18/120 12/120
|
||
blocks spanned/frame 0 727 839 845
|
||
```
|
||
`today` reproduces FINDINGS 35's 84/120 exactly, which is the check that the
|
||
scenario lines up.
|
||
|
||
### 39.4 What the DMAC actually buys, and who else can sell it
|
||
`v6 fine tail` is the decomposition. v6 pads every span up to 24 pixels because
|
||
its copy is an unrolled chain of 12-register `movem` units; adding a second,
|
||
finer chain of 2-register units caps the padding at 3 pixels instead of 23, for
|
||
the price of some more unrolled code and **nothing per span**. Priced
|
||
conservatively (a 4-pixel unit costs 56 clocks against a full unit's 220 for 24,
|
||
so it is dearer per pixel and paid at most once a span):
|
||
|
||
| | frames over |
|
||
|---|---:|
|
||
| v6 as built | 55/120 |
|
||
| **v6 with a finer chain tail -- software only** | **18/120** |
|
||
| DMAC chain | 12/120 |
|
||
|
||
**86% of the DMAC's advantage over v6 is the 24-pixel padding quantum**, and
|
||
that is a property of v6's unrolled chain, not of the CPU. The residual is 1.7%
|
||
a pixel and 7.7 clocks a span, worth 6 frames of 120.
|
||
|
||
Break-even against all-V1 moves the same way: v6 as built needs a run of 4
|
||
blocks, v6 with the finer tail needs 3, the DMAC needs 1.
|
||
|
||
### 39.5 The verdict
|
||
**Fix the quantum in software.** Six frames of 120 does not buy a reserved DMAC
|
||
channel, a two-region container layout, and a dependency on transfer timing that
|
||
cannot be verified in either emulator on this box. The `v6 fine tail` figure is
|
||
itself DERIVED and should be measured with `span.sh` before it is believed --
|
||
that is a day's work in a tool that already exists, against a hardware
|
||
dependency that needs an actual X68000.
|
||
|
||
Keep 39.1 on the record. If a later measurement moves the DMAC's per-pixel cost
|
||
below 8 clocks -- for instance if GVRAM tolerates a four-clock DMAC write in a
|
||
way the datasheet's typical-system assumption does not model -- the container
|
||
does not have to change to take advantage of it.
|
||
|
||
### 39.6 What else would have to be true, if it is ever revisited
|
||
- **A free channel.** Four exist; channel 3 is ADPCM (`adpcm_drq_tick` asserts
|
||
`drq3_w`) and the SCSI stream needs one.
|
||
- **Two regions per frame.** Chaining fetches entries from an array while DAR
|
||
walks the pixel data, so the span table and the literal words cannot be
|
||
interleaved as v6 interleaves them.
|
||
- **The mode-map walk stays on the CPU.** 39.3 charges it; FINDINGS 30.7 flagged
|
||
that 12_span_tradeoff.py did not.
|
||
|
||
### 39.7 A number the datasheet settled on the way past
|
||
FINDINGS 5's **8 clocks/word** for the SCSI DMA has been an unsourced estimate
|
||
since session 1 and STATUS has called it the most load-bearing unmeasured number
|
||
in the project. Fig 4-25 sheet 3 gives single-address `W/B READ` 4 clocks and
|
||
`W/B WRITE` 5; a device->memory disk transfer is one memory write. So it is
|
||
**5 clocks/word if the DMAC holds the bus** and about **12 if it arbitrates per
|
||
word** (front-end 5 best case / 8 worst, sect 4.5.2.1; back-end 2, sect 4.5.2.2).
|
||
The feature list's "up to 5 Megabytes per Second at 10 MHz, no wait states"
|
||
is the held-bus case: 2 bytes per 4-clock cycle.
|
||
|
||
**8 is the midpoint of a bracket the datasheet supports, not a guess.** Which
|
||
end applies depends on how the MB89352 drives REQ and whether cycle-steal-with-
|
||
hold is used, which is a design decision the player has not made yet -- and it
|
||
is worth 7 clocks a word on a 480 KB/s stream, so it is worth making
|
||
deliberately.
|
||
|
||
---
|
||
|
||
## 40. The finer chain tail, measured: v7 (session 11)
|
||
|
||
FINDINGS 39.4 attributed **86% of the DMAC array-chain's advantage over v6** to
|
||
one thing that is not a property of the DMAC at all -- v6's 24-pixel padding
|
||
quantum -- and derived that fixing it in software would take the `scsi` window
|
||
from 55/120 frames over budget to 18/120. It labelled that figure DERIVED and
|
||
said it should not be believed until `span.sh` measured it. This measures it.
|
||
|
||
### 40.1 The result
|
||
`tools/bench/blit.s` gains **v7**: v6's 24-pixel coarse chain with a second,
|
||
finer chain appended. Measured over thirteen span lengths by
|
||
`tools/bench/span.sh`, every one of which drew a **pixel-exact** frame:
|
||
|
||
```
|
||
cycles = 66.0 per span + 9.143 per COARSE pixel + 9.978 per FINE pixel
|
||
```
|
||
|
||
fitting all 13 lengths to within **0.2%** -- and the fit is not flattered by its
|
||
own configs, because the three-term model was identified on span lengths chosen
|
||
so that every fine remainder a real span can have (0, 4, 8, 12, 16, 20) appears.
|
||
v5 and v6 re-measure to 97.9 + 10.459 and 43.7 + 9.152, reproducing FINDINGS 30
|
||
exactly, so the harness has not drifted underneath the new variant.
|
||
|
||
| clocks per 4x4 block, run of L blocks | L=1 | L=2 | L=3 | L=4 | L=8 | L=64 |
|
||
|---|---:|---:|---:|---:|---:|---:|
|
||
| v6 as built | 1053 | 527 | 351 | 263 | 241 | 154 |
|
||
| **v7** | **424** | **292** | **248** | **226** | **183** | **151** |
|
||
| DMAC chain (datasheet) | 288 | 216 | 192 | 180 | 162 | 146 |
|
||
|
||
**Break-even against all-V1 (299.9) moves from L=4 to L=2 blocks.** 39.4
|
||
predicted L=3; the measurement is better than the derivation.
|
||
|
||
### 40.2 The scoring, and a derivation that was right for the wrong reasons
|
||
Rescoring the same `scsi` window with `14_dmac_chain.py`, the same additive
|
||
model, the same mode maps:
|
||
|
||
| | frames over budget |
|
||
|---|---:|
|
||
| today | 84/120 |
|
||
| v6 span as built | 55/120 |
|
||
| **v7, MEASURED** | **18/120** |
|
||
| DMAC chain (datasheet) | 12/120 |
|
||
|
||
**18/120, exactly the derived figure.** That agreement is a coincidence of two
|
||
cancelling errors, and it is worth writing down because a match this clean would
|
||
otherwise be read as confirmation:
|
||
|
||
- 39.4 assumed a **2-register `movem` tail**, derived at 56 clocks per 4 pixels
|
||
= 14.0 clocks/pixel. The real tail costs **9.978** -- 29% cheaper.
|
||
- 39.4 assumed the second entry point costs **nothing per span**. It costs
|
||
**22.3 clocks** (66.0 against v6's 43.7), because it is a second
|
||
`move.w (a0)+,d0` and a second `jmp`.
|
||
|
||
The per-pixel win and the per-span loss are within a frame of each other over
|
||
this window. **The conclusion survives; the reasoning behind it did not.**
|
||
|
||
### 40.3 The instruction the derivation should have picked
|
||
A 2-register `movem` is the obvious "smaller unit of the same thing", and it is
|
||
the wrong instruction. Per 4 pixels:
|
||
|
||
| tail unit | bus cycles | clocks | per pixel |
|
||
|---|---:|---:|---:|
|
||
| `movem.l (a0)+,d0-d1` + `movem.l d0-d1,(a2)` + `lea` | 14 | 56 | 14.0 |
|
||
| **2 x `move.l (a0)+,(a2)+`** | **10** | **40** | **~10.0** |
|
||
|
||
`movem` pays two instruction words and a `lea` to move what two of the plainest
|
||
instructions on the machine move with post-increment on both sides. v7's fine
|
||
unit is therefore **one `move.l (a0)+,(a2)+` = 2 pixels**, which also makes the
|
||
padding quantum **2** rather than 4 -- and a span is a run of 4x4 blocks, so its
|
||
length is always a multiple of 4 and **the padding is exactly zero**. 39.4's
|
||
"caps the padding at 3 pixels" was pessimistic by three pixels.
|
||
|
||
The derived bus model predicts the measurement well once the right instruction
|
||
is in it: 5 bus cycles = 20 clocks per 2 pixels = 10.0/pixel against 9.978
|
||
measured, and 54 bus cycles = 216 clocks per 24 against 9.143*24 = 219.4.
|
||
|
||
### 40.4 Where the fine displacement lives, and why it is not in the record
|
||
Two chains need two entry points, and the second one has to survive the coarse
|
||
copy. Holding it in a register would cost a payload register -- v6's whole
|
||
reason for a 24-pixel unit is that it has exactly 12 free (`d0-d6/a1/a3-a6`).
|
||
|
||
So it is not in the span record. **It is in the stream**, after the coarse
|
||
pixels and before the fine ones. The coarse chain falls out into
|
||
`move.w (a0)+,d0 / jmp v7fh(pc,d0.w)`, and at that instant `d0` is dead payload
|
||
and `a0` is pointing exactly at it. The decoder holds nothing extra across the
|
||
copy and keeps all 12 registers.
|
||
|
||
The record is still `{u32 absolute GVRAM address, u16 coarse displacement}`;
|
||
the container costs **2 more bytes per span**, which `14_dmac_chain.py` charges.
|
||
|
||
### 40.5 The verdict, now measured rather than argued
|
||
**FINDINGS 39.5 stands: fix the quantum in software, drop the DMAC.** v7 takes
|
||
back **37 of the 43 frames** the DMAC chain would, using an instruction sequence
|
||
that needs no reserved channel, no two-region container, and no transfer timing
|
||
that neither emulator on this box can verify. 39.1 still holds if that ever
|
||
changes: the chain array and the span table are the same six bytes.
|
||
|
||
### 40.6 The 13-minute run that measured nothing
|
||
`span.sh` ran for 13 minutes producing an empty log and zero snapshots, and the
|
||
same MAME command with a shorter `-seconds_to_run` completed the identical work
|
||
in 30 seconds. The cause is still not identified. What matters is that **the
|
||
run was unobservable in both directions**: MAME's stdout did not reach the log
|
||
until exit even under `stdbuf -oL`, and the snapshots -- the one artefact that
|
||
would have shown progress -- may themselves only land at exit.
|
||
|
||
So the bisection that resolved it did not chase the hang. It **shrank the
|
||
stimulus** instead: `tmp/spans_meta.lua` carries byte offsets into a blob that
|
||
`prep_spans.py` writes once, so deleting config lines from the metadata runs any
|
||
subset in seconds against the same unmodified stream file. v5 alone, v7 alone
|
||
and the full set at a shorter run all completed; the wedge never reproduced.
|
||
|
||
This is the fourth instance of the pattern FINDINGS 34.1 named, and it is the
|
||
first where **the instrument was unobservable but the thing being measured was
|
||
fine**. `span.sh` now runs at `-seconds_to_run 200`, measured at 30 s wall for
|
||
all 36 configs, and asserts the snapshot count against the number of configs in
|
||
the generated metadata rather than a literal 23 -- so adding a config can no
|
||
longer silently weaken the pixel-exactness gate.
|
||
|
||
## 41. v7 is in the player, and the model it is scored by was 18% wrong (session 12)
|
||
|
||
> **41.2's FRAMING IS SUPERSEDED BY 42.** This section ends by naming the rate
|
||
> point as a fork the user must choose. There is no fork: the span pass
|
||
> saturates at ~837 KB/s on its own, and the 488 KB/s ceiling it is scored
|
||
> against here was never a bus figure (42.1). The measurements below stand; the
|
||
> "two byte budgets" mechanism of 41.2 is what made 42 findable.
|
||
|
||
FINDINGS 40 measured v7 in `tools/bench/blit.s` and left it there. This builds it
|
||
into `src/player/decode.s`, defines the container that carries it, and scores
|
||
what the encoder actually delivers rather than what a selection model predicts.
|
||
Three things came out of it that were not on the list.
|
||
|
||
### 41.1 The decoder, and the format
|
||
`src/player/decode.s` gains `paint_spans`, which is `blit.s` v7 verbatim -- the
|
||
same instruction sequence, deliberately, because the 66.0/9.143/9.978 fit was
|
||
measured on that sequence and a tidier rewrite would silently invalidate it.
|
||
|
||
The container is **DLX3**: the span section sits between the 768-byte mode
|
||
header and the block payload, because that is the only place the 68000 can
|
||
reach without first parsing something of variable length.
|
||
|
||
```
|
||
u32 payload length
|
||
768 B mode header spanned blocks read SKIP
|
||
u16 nspans
|
||
nspans * { u32 GVRAM address, u16 coarse disp, c*48 B,
|
||
u16 fine disp, f*4 B }
|
||
block payload V1 -> 1 B, V4 -> 4 B, RAW -> 16 B
|
||
```
|
||
|
||
Every span record is a multiple of 4 bytes (4+2+48c+2+4f), so the section needs
|
||
no internal padding and the block payload starts aligned. `a1`, the mode-header
|
||
cursor, is one of v7's twelve payload registers, so it goes on the stack across
|
||
the pass: two long accesses a frame, against the 24 pixels a register buys per
|
||
chain unit.
|
||
|
||
**Pixel-exact under both CPU cores on the first run**, over a container where
|
||
every frame carries 128-216 spans painting up to 38% of the picture, with full
|
||
temporal recursion. `tools/analysis/16_span_roundtrip.py` is the new gate and it
|
||
is in `check.sh`: encode, write the container, read it back with the reference
|
||
decoder, compare to what the encoder recorded. It asserts it emitted enough
|
||
spans to have tested anything -- a round-trip over a span-less container is
|
||
green by vacuity, which is FINDINGS 40.6's lesson about the snapshot count.
|
||
|
||
### 41.2 There are TWO byte budgets, and conflating them hides the whole win
|
||
The first measured span encode looked like a regression: at the `scsi` profile
|
||
spans fired on 5 of 120 frames and bought almost nothing. The cause is not the
|
||
codec. **The lam search had already spent the byte allowance**, so the span pass
|
||
inherited a few hundred bytes of room.
|
||
|
||
FINDINGS 40's 18/120 was never scored at 280 KB/s. `14_dmac_chain.py` defaults
|
||
to `--bus 488` -- the PIPE -- and gives each frame 40,977 bytes. The profile's
|
||
is 23,228. **Those are two different budgets and only one of them is hardware.**
|
||
The profile is a chosen quality rate point; the pipe is a ceiling. Bytes between
|
||
the two buy a better picture if spent on `lam`, the 68000's deadline if spent on
|
||
spans, and nothing at all if left unspent.
|
||
|
||
So the encoder now takes both: `--kbps` sets the quality target and
|
||
`--span-kbps` the ceiling the span pass may draw on, flat per frame and not
|
||
banked, because a pipe cannot be saved up. The quality bucket is credited with
|
||
the BLOCK payload only -- charging it the span bytes drives it to its floor on
|
||
the first spanned frame and starves every later frame of quality for a budget
|
||
the spans were never drawing on.
|
||
|
||
Spans also run BEFORE `mu`, and that ordering is the point. Both controllers
|
||
make a frame decode in time; `mu` pays in quality and a span pays in bytes, and
|
||
a span carries literal source pixels so it *removes* that run's quantisation
|
||
error. Spending bytes we already have beats spending picture.
|
||
|
||
| 120-frame `scsi` window | KB/s | over budget | PSNR |
|
||
|---|---:|---:|---:|
|
||
| no spans | 278.3 | 86/120 | 29.27 dB |
|
||
| spans, profile budget only | 280.0 | 77/120 | 29.23 dB |
|
||
| **spans on the 488 KB/s pipe** | 487.7 | **34/120** | **29.63 dB** |
|
||
|
||
Scored with `17_span_delivered.py`, which reads the emitted span section and
|
||
prices exactly those spans -- no selection model at all -- in 14's additive
|
||
model: block decode + span painting + disk DMA.
|
||
|
||
### 41.3 The blit.s fit transfers into the player, to 0.2%
|
||
`prep_dlx.py` gained two synthetic all-SPAN frames (full-row runs, and 4-block
|
||
runs at the break-even). They price v7 inside `decode.s` against the constants
|
||
`span.sh` fitted in `blit.s`:
|
||
|
||
| | predicted | MAME | C68K | error |
|
||
|---|---:|---:|---:|---:|
|
||
| all-SPAN-64 (192 spans x 256 px) | 505,636 | 506,533 | 506,824 | +0.18% |
|
||
| all-SPAN-4 (3072 spans x 16 px) | 734,193 | 735,133 | 735,304 | +0.13% |
|
||
|
||
Per 4x4 block that is **151.2 and 225.6 clocks, against FINDINGS 40's table of
|
||
151 and 226**. The mode costs what it was said to cost, in the real decoder, on
|
||
two emulators.
|
||
|
||
### 41.4 The rig had been writing past the top of RAM
|
||
`prep_dlx.py` truncated the real frames to a RAM budget and then appended its
|
||
synthetic timing frames ON TOP, 26 KB past the 0x200000 top of a 2 MB machine.
|
||
Survivable while it lasted, because the modes it overran are data-independent:
|
||
reading junk payload costs a V1 or a RAW block exactly what reading pixels
|
||
costs, so the anchors timed correctly by luck.
|
||
|
||
**A span is not data-independent.** Its two jump displacements come out of the
|
||
stream, so an out-of-RAM span record jumps into open bus. The synthetic frames
|
||
are now built first and their size comes out of the budget, with an assertion
|
||
that the stream ends below the top of RAM.
|
||
|
||
### 41.5 C_SKIP_MIXED was never measured, and it was 18% low
|
||
Chasing a 3% gap between the model and the measured decode turned up the one
|
||
constant in `vq_hybrid`'s cost table that came from a derivation rather than a
|
||
measurement: **the cost of a SKIP block sharing its header byte with a coded
|
||
block.** It was 45.0 from session 7 to session 12. It is **55.0**.
|
||
|
||
Every other constant comes from a synthetic frame of a single mode, and there
|
||
was no such frame for a mixed SKIP, *because one cannot exist* -- the byte has
|
||
to hold a coded block for the SKIP to be mixed at all. So `prep_dlx.py` now
|
||
emits four frames that bracket it, (3 SKIP + 1 V1), (1 SKIP + 3 V1), and the
|
||
same pair with RAW, each pair solving for the SKIP cost and its partner's
|
||
together:
|
||
|
||
| | MAME | C68K |
|
||
|---|---:|---:|
|
||
| mixed SKIP, from the V1 pair | 55.03 | 56.50 |
|
||
| mixed SKIP, from the RAW pair | 55.83 | 56.50 |
|
||
| V1, solved back out | 300.66 | 300.50 |
|
||
|
||
The partner solves back to its own anchored value to 0.2%, which is what says
|
||
the pair is measuring the SKIP rather than absorbing it. 55.0 is taken because
|
||
every other constant in the table is MAME's.
|
||
|
||
**The header bytes ROTATE through all four positions, and that is load-bearing.**
|
||
`decode.s` reaches a block's mode bits with `lsr.b #6/#4/#2` and no shift at all
|
||
for the last one, so a block costs 52/48/44/34 clocks of dispatch depending on
|
||
where in its byte it sits. A fixed pattern like 0x01 pins every SKIP to the
|
||
three expensive slots and every V1 to the free one, and solving two such
|
||
equations returns a number that describes no real frame. The first attempt did
|
||
exactly that, and a single-parameter fit against real frames then "confirmed"
|
||
87.7 -- collinear with the span term, and wrong.
|
||
|
||
With the constant corrected the model predicts the measured decode of a real
|
||
spanned container to **-0.06% on the mean and 0.09% worst frame**, against
|
||
-2.99% and 4.30% as it stood.
|
||
|
||
It matters more than 10 clocks a block sounds, because **a span marks its run
|
||
SKIP**: a spanned container is made largely of mixed SKIPs, so this is the
|
||
dominant population in exactly the frames spans are judged on. It is imported
|
||
now, not copied, in `spans.py` and `14_dmac_chain.py`.
|
||
|
||
*Incidental, and it resolved a false lead:* RAW and V4 read 3.2-3.5% higher on
|
||
C68K than on MAME, on mixed and pure frames alike. That is FINDINGS 37's known
|
||
table spread, not a property of mixed bytes -- but comparing a C68K-derived
|
||
solve against a MAME-derived anchor made it look like one for an hour.
|
||
|
||
### 41.6 Frames-over-budget is not a safe headline any more
|
||
The delivered 34/120 against 14's simulated 18/120 is a **1.4% difference in
|
||
mean frame cost** (786,381 clocks against 774,356). The metric is that
|
||
sensitive because the rate controller *aims* at the deadline: 55 of 120 frames
|
||
land within 5% of it, and shifting every frame by 1% moves the count from 25 to
|
||
47.
|
||
|
||
| every frame shifted by | -3% | -2% | -1% | 0 | +1% | +2% | +3% |
|
||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||
| frames over budget | 18 | 21 | 25 | **31** | 47 | 61 | 63 |
|
||
|
||
This was a fair metric when nothing controlled to the budget. It is now a
|
||
measurement of where the controller aims, and any cost-model error is amplified
|
||
into a large count change -- which is how 41.5's 18% error stayed invisible.
|
||
**Report the cost distribution; quote the count only with its sensitivity.**
|
||
Add this to the §4 measurement traps.
|
||
|
||
## 42. The rate point dissolves: bytes are nearly free, and one constant decides everything (session 13)
|
||
|
||
Session 12 ended by naming the rate point as "the first real fork since the
|
||
profile was set" and asking the user to choose one. **The question turned out to
|
||
be mis-posed, and the user is what posed it correctly**: challenged on where the
|
||
488 KB/s constant came from, the answer is that it never came from the SCSI bus
|
||
at all, and the bus has roughly eight times that in it. Once bytes are that
|
||
cheap the rate point is not a choice, it is a saturation point the encoder finds
|
||
on its own -- and what actually decides whether the game runs is **the number of
|
||
clocks the SCSI DMA steals per word**, which is still unmeasured and now worth
|
||
more frames than every optimisation since FINDINGS 24 combined.
|
||
|
||
### 42.1 The 488 KB/s constant was never a bus figure
|
||
FINDINGS 18/21 recorded it as a user-supplied "4 Mbps" with no provenance, and
|
||
29.5 item 3 has carried "confirm the 4 Mbps figure" as an open item ever since.
|
||
Checked against the standard: **SCSI-1 (ANSI X3.131-1986) is an 8-bit bus at
|
||
~1.5 MB/s asynchronous and 5 MB/s synchronous** -- MB/s, not Mbps. The working
|
||
constant is 10% of the asynchronous rating and 3% of the deployment target's
|
||
(FINDINGS 21.2 committed to SD-backed SCSI in session 2).
|
||
|
||
So the pipe was never the binding resource on the I/O side. What binds is that
|
||
**a delivered byte is charged to the 68000's frame budget**, because the DMAC
|
||
stalls the CPU rather than overlapping with it (38.3). The pipe's units are
|
||
KB/s; the real currency is clocks.
|
||
|
||
### 42.2 The cheapest way to put a pixel on this screen is to not code it
|
||
Per pixel, in frame-budget clocks, with `W` the SCSI DMA's clocks per word:
|
||
|
||
| | wire bytes | paint | disk debit | total |
|
||
|---|---:|---:|---:|---|
|
||
| v7 literal span | 2 | 9.143 | `W` | `9.143 + W` |
|
||
| RAW block | 1 | 25.03 | `W/2` | `25.03 + 0.5W` |
|
||
| V1 block | 0.25 | 18.74 | `W/8` | `18.74 + 0.125W` |
|
||
|
||
A span pixel costs **two** wire bytes, not one: X68000 GVRAM in 256-colour mode
|
||
is one pixel per WORD with the high byte discarded (`spans.py:44`, and it is the
|
||
memory model both emulators are pixel-exact against), so the span path is a
|
||
straight `movem` copy that cannot pack. That 2x on bytes is exactly what buys
|
||
9.143 clocks/pixel.
|
||
|
||
**A span beats a RAW block for any `W < 31.7`**, i.e. everywhere in the 5..12
|
||
bracket of 39.7, and by a factor of nearly 2. The codec's expensive modes exist
|
||
to save bytes, and bytes have stopped being the scarce thing.
|
||
|
||
The budget is 833,333 clocks for 49,152 pixels = **16.95 clocks/pixel**. A
|
||
full-frame literal therefore needs `9.143 + W` to fit under ~16.1 after span and
|
||
all-SKIP overhead: **lossless fits iff `W` is about 6.5 or less.**
|
||
|
||
### 42.3 Measured: the span pass saturates at ~935 KB/s, 0.13 dB off the display
|
||
Encoded with `--kbps 280` held fixed and `--span-kbps` swept, so every column
|
||
below has the same quality target and differs only in what the span pass was
|
||
allowed to draw on. Scored with `17_span_delivered.py` over the emitted span
|
||
sections:
|
||
|
||
| container | KB/s | span px | PSNR | over @5 | @8 | @12 |
|
||
|---|---:|---:|---:|---:|---:|---:|
|
||
| `s12_280_off` | 278.6 | 0% | 29.27 dB | 76/120 | 86/120 | 109/120 |
|
||
| `rc_fr_singe_scsi_span` (the gate) | 487.7 | 26.6% | 29.63 dB | 11/120 | 34/120 | 78/120 |
|
||
| `s13_280p700` | 693.8 | 48.3% | 30.28 dB | 16/120 | 28/120 | 63/120 |
|
||
| `s13_280p1000` | 815.4 | 60.2% | 30.93 dB | 7/120 | 20/120 | 64/120 |
|
||
| `s13_280p1500` | 837.4 | 62.3% | 31.04 dB | **0/120** | 21/120 | 63/120 |
|
||
| `s13_lossless` (`--kbps 2000`) | 934.6 | 70.4% | **31.19 dB** | **0/120** | 47/120 | -- |
|
||
|
||
**`s13_280p1500` and `s13_280p2200` are byte-identical files.** The span pass
|
||
stops finding spans worth taking at ~837 KB/s; asking for more bytes returns the
|
||
same container. The rate point is not a choice between quality and deadline any
|
||
more, it is a saturation the encoder reaches by itself.
|
||
|
||
Three results worth separating out:
|
||
|
||
1. **0/120 is the first time anything in this project has fitted 12fps on every
|
||
frame.** It is also the best picture yet: 31.19 dB against a **31.33 dB
|
||
palette ceiling**, 0.13 dB off exact for this display.
|
||
2. **`mu` is never spent.** At span budgets >= 1000 KB/s, 0/120 frames need the
|
||
CPU-fit lagrangian at all. The 0.62 dB FINDINGS 31 paid to make frames decode
|
||
in time is refunded in full -- spans buy the deadline with bytes, and a span
|
||
is pixel-exact, so the trade is quality-positive in both directions.
|
||
3. **Quality and deadline stopped competing.** Every earlier section on this
|
||
project trades one against the other. Above ~800 KB/s more bytes improve
|
||
both at once, until the span pass runs out of runs worth spanning.
|
||
|
||
### 42.4 So the whole result now hangs on `W`, and only on `W`
|
||
Read the columns of 42.3 sideways rather than down. At 934.6 KB/s:
|
||
|
||
| `W`, clocks/word | median frame | worst | over budget |
|
||
|---:|---:|---:|---:|
|
||
| 5 | 83.0% | 91.0% | **0/120** |
|
||
| 6 | 87.6% | 97.0% | **0/120** |
|
||
| 6.5 | -- | -- | crossover |
|
||
| 7 | 92.2% | 103.0% | 21/120 |
|
||
| 8 | 96.8% | 109.0% | 47/120 |
|
||
|
||
Moving `W` across its datasheet bracket costs more frames than moving the rate
|
||
from 280 to 935 KB/s wins. **`W` is now the most load-bearing unmeasured number
|
||
in the project**, displacing the 4 Mbps figure it just retired.
|
||
|
||
### 42.5 MAME settles which end of the bracket applies, and cannot settle the number
|
||
STATUS has carried "benchmark `x68000 -exp1 cz6bs1`, never `x68ksupr`" since
|
||
session 10, and the note reads as a hardware claim. **It is not one, and the
|
||
user was right to challenge it.** Both machines use the same MB89352 SPC --
|
||
`x68k.cpp:834` maps it at `$E96020`, `x68k_scsiext.cpp:83` at `$EA0000` -- and
|
||
neither needs a driver because SCSI IOCS is in ROM. The difference is an
|
||
emulation gap: `x68k.cpp:1176` is literally `// TODO: duplicate DMA glue from
|
||
CZ-6BS1`.
|
||
|
||
The external board's glue is modelled, and reading it answers the mode question:
|
||
|
||
- `x68k_scsiext.cpp:110-136`: a transfer is a DMA cycle when `m_slot->exown()`
|
||
-- the HD63450's **OWN** -- is asserted, so the DMAC holds the bus and the
|
||
68000 is off it. It single-address reads the data register at `$EA0015` and
|
||
**negates `#DTACK` whenever `DRQ` is not ready**, stalling rather than
|
||
arbitrating away.
|
||
- `x68k.cpp:1114-1115, 1122-1123`: the slot's DTACK feeds `hd63450::dtack_w`
|
||
and the DMAC's `own()` feeds back to the slot.
|
||
|
||
So the modelled CZ-6BS1 path is **cycle-steal with the bus held** -- the 5
|
||
clocks/word end of 39.7's bracket, not the ~12 arbitrated end. That is the end
|
||
where everything above fits.
|
||
|
||
**But MAME cannot give the constant, and asking it for one would be reading back
|
||
a hand-set table.** `x68k.cpp:1047-1048` configures the DMAC with
|
||
`set_clocks(attotime::from_usec(2), from_nsec(450), from_usec(4), ...)` and
|
||
`set_burst_clocks(...)` -- wall-clock attotimes, not the MC68450's per-operand
|
||
cycle counts. Same lesson as FINDINGS 39: the datasheet supplies constants, the
|
||
emulator supplies structure. Use MAME to confirm which handshake the player's
|
||
code actually provokes; use Fig 4-25 for what it costs.
|
||
|
||
### 42.6 Where the wait actually comes from, and why flash is the right premise
|
||
With DTACK gating the cost decomposes:
|
||
|
||
W = 5 clocks (MC68450 single-address write, Fig 4-25 sheet 3)
|
||
+ however long the drive makes the DMAC wait for DRQ
|
||
|
||
The 5 is silicon and fixed. **The rest is the device**, and it is where 8 and 12
|
||
came from -- 8 was taken as a bracket midpoint in FINDINGS 5, not measured. A
|
||
period spinning SCSI-1 drive supplies a real wait; SD-backed SCSI with a modern
|
||
controller collapses it toward zero and leaves the floor. The deployment target
|
||
has been SD since session 2 (21.2), so **the favourable end of the bracket is
|
||
the one the actual hardware is on** -- which is what makes 42.3's 0/120 worth
|
||
taking seriously rather than filing as a best case.
|
||
|
||
This is a better-founded version of the user's argument than the bus rating was.
|
||
The bus rating is true and irrelevant; the wait term is what their premise
|
||
actually buys.
|
||
|
||
### 42.7 What is NOT established
|
||
Stated plainly, because 42.3 is the most favourable table this project has ever
|
||
produced and that is exactly when it should be distrusted:
|
||
|
||
1. **No 68000 has decoded a 70%-span container.** Every figure in 42.3 is the
|
||
additive cost model. That model is validated to -0.06% mean / 0.09% worst on
|
||
spanned containers (41.5), but the heaviest one ever *run* is the gate's
|
||
26.6%. The rig loads the stream into a 2 MB machine, so a 934.6 KB/s stream
|
||
is about **5 frames** -- getting coverage back needs a different rig
|
||
(stream-in-chunks, or a larger machine as a deliberately-labelled non-target),
|
||
not a longer pass. This is FINDINGS 36's lesson pointing at a new wall.
|
||
2. **`W` is unmeasured**, and 42.4 is the whole result's sensitivity to it.
|
||
3. **The ring buffer has never been simulated near this rate.** 98 KB/frame of
|
||
span payload against 2 MB with no double buffer; `09_buffer_sim.py` last ran
|
||
at 110 and 280 KB/s (29.5/30.7).
|
||
4. **Capacity is fine but should be stated:** 934.6 KB/s x 1366.6 s = **1.22 GB**
|
||
for the whole game. Irrelevant on SD, fatal on anything period. This closes
|
||
CD-ROM permanently rather than parking it (32.3).
|
||
5. **Span selection is still greedy after `lam`** (STATUS item 2, 39.3). At
|
||
saturation that probably leaves something on the table rather than costing
|
||
anything, but it has not been checked at this rate.
|
||
6. **The `--kbps 2000` container is not a shippable configuration**, it is a
|
||
probe: it lets `lam` fall to its floor and asks what the span pass does with
|
||
an unbounded budget. `s13_280p1500` at 837.4 KB/s is the honest candidate.
|
||
|
||
### 42.8 FINDINGS 17.5 was right, and was withdrawn for the wrong reason
|
||
Session 4 concluded: **"ship pixel-exact if SCSI sustains >=800 KB/s."** Session
|
||
5 withdrew it, because 17 had reasoned against a misread 4 MB/s and the
|
||
correction to 4 Mbps = 488 KB/s put 800 out of reach.
|
||
|
||
The correction was to the wrong direction of the error. 4 MB/s was indeed a
|
||
misread of the bus, but 488 KB/s was not a measurement either -- and the bus
|
||
really does have ~1.5-5 MB/s in it (42.1). **The delivered stream is 837.4 KB/s
|
||
at 0.29 dB off the palette ceiling**: 17.5's threshold, and 17.5's conclusion,
|
||
reached from the opposite end five sessions later by a route that never cited
|
||
it.
|
||
|
||
Worth recording as a methodology result rather than a curiosity. The project's
|
||
habit of appending corrections rather than editing history is what made this
|
||
recoverable at all -- but 17.5 spent five sessions marked "not available"
|
||
because a number nobody had sourced was allowed to retire a conclusion that had
|
||
been reasoned properly. **A correction is only as good as the constant behind
|
||
it**, and this is the second time that constant is the one at fault (compare
|
||
39: bus arithmetic retired the DMAC's first costing, and the datasheet retired
|
||
the retirement).
|
||
|
||
## 43. The disk debit was denominated per word, and the SPC is a byte-wide port (session 14)
|
||
|
||
Session 13 made `W` -- the clocks the SCSI DMA steals -- the one binding unknown
|
||
in the project, and 42.4 showed the whole result swinging on where in a 5..12
|
||
bracket it landed. **The bracket was in the wrong unit.** `W` was charged per
|
||
WORD of delivered stream, and the MB89352 is an 8-bit port: the DMAC pays per
|
||
BYTE. The debit is 2x what every table since FINDINGS 5 has charged, the
|
||
favourable end of the bracket was never physically reachable, and 42.3's 0/120
|
||
does not survive.
|
||
|
||
The result is not lost, but it is re-anchored 41% lower in rate and 1.85 dB
|
||
lower in picture, and it had to be re-encoded rather than re-scored -- because
|
||
the encoder was making its decisions in the wrong units too (43.6).
|
||
|
||
### 43.1 The floor argument needs no datasheet
|
||
A 68000 bus cycle is four clocks minimum. The SPC delivers one byte per bus
|
||
cycle. Therefore **no DMA of this device can cost less than 4 clocks/byte = 8
|
||
clocks/word**, before a single clock of DMAC overhead, the write cycle that
|
||
puts the byte in RAM, or any wait the drive imposes.
|
||
|
||
The retired bracket's own midpoint, 8 clocks/word, is that floor exactly; its
|
||
favourable end, 5 clocks/word, is **2.5 clocks/byte -- 62% of a single bus
|
||
cycle**. It implied a 4 MB/s DMA on a 10 MHz bus, on a SCSI-1 link 42.1 had
|
||
just established runs at 1.5 MB/s asynchronous. The number that retired the
|
||
4 Mbps figure should have retired this one in the same paragraph.
|
||
|
||
Independent cross-check, of the kind this project files as folklore rather than
|
||
measurement: BlueSCSI-class throughput on an X68000 is discussed at 0.7-1.7
|
||
MB/s. 5 clocks/byte is a 2 MB/s ceiling and 9 is 1.11 MB/s; 2.5 clocks/byte is
|
||
4 MB/s, which no one has ever reported on this machine.
|
||
|
||
### 43.2 The datasheet, per byte, for the transfer this actually is
|
||
Device-to-memory, 8-bit device, MC68450 Fig 4-25, all while the DMAC owns the
|
||
bus (note 2: reads 4 clocks, writes 5):
|
||
|
||
| how the DMAC is programmed | clocks per BYTE | source |
|
||
|---|---:|---|
|
||
| single address, D->M | **5.0** | sheet 2 |
|
||
| dual address, byte, no packing | **9.0** | sheet 4, the note on the 9-clock word case |
|
||
| dual address, byte packed to word writes | **16.5** | sheet 3: 14 clocks then 19, per two bytes |
|
||
|
||
Byte packing is *dearer*, not cheaper: the FIFO path carries 10 clocks of
|
||
inter-cycle overhead per operand where the unpacked path carries none.
|
||
|
||
To this, per period of bus ownership, add front-end overhead of **5 clocks best
|
||
case (Fig 4-23) or 8 worst (Fig 4-24)** and back-end of **2 (4.5.2.2)**.
|
||
Amortised over a sector that is noise; taken per operand it would add 7-10
|
||
clocks to every byte. **That is what actually rules out arbitrating per byte**
|
||
-- not the OWN pin (43.3).
|
||
|
||
**Which row applies is a wiring question, and it is worth 4 clocks on every
|
||
byte of the game.** Single-address needs the SPC's DACK driven from the bus's
|
||
`#EXACK` (pin B37, and the bus really does have it, `x68kexp.h`). MAME models
|
||
the dual-address row: an `exown`-gated byte read of `$EA0015` written to memory
|
||
with `space.write_byte` (`x68k_scsiext.cpp:110-121`, `hd63450.cpp:383-385`), and
|
||
it has no DACK path at all, so **MAME cannot settle this one either way.**
|
||
|
||
### 43.3 42.5's reading of the OWN pin was over-read
|
||
42.5 concluded from `x68k_scsiext.cpp`'s `m_slot->exown()` gate that the modelled
|
||
path is "cycle steal with the bus held", i.e. the 5-clock end. It does not say
|
||
that. `hd63450.cpp:366,447` asserts `m_own(0)` before **every** `single_transfer`
|
||
and negates it after, in every request-generation mode, and `x68kexp.h:131`
|
||
inverts it. So `exown()` distinguishes **a DMAC access from a CPU access** --
|
||
DMA versus PIO -- and says nothing about hold versus arbitrate. The check was
|
||
answering a different question than the one asked of it.
|
||
|
||
Two things about the board do survive, and they are the useful half:
|
||
|
||
1. **The glue's flow control is a stalled bus cycle, not a released bus.** When
|
||
`DRQ` is low the board negates `#DTACK` and the DMAC waits mid-cycle
|
||
(`hd63450.cpp:449`, `if (!m_dtack) return;` -- the operand does not advance).
|
||
There is no path by which it hands the bus back inside a word. So whatever
|
||
the drive makes it wait, it waits **holding the bus**, which is why the wait
|
||
term of 42.6 is charged to the frame budget in full.
|
||
2. **MAME has no DRQ line from the expansion slot to the DMAC.** `x68kexp.cpp`
|
||
has no such callback; DMAC channel 0 is wired to the FDC and channel 3 to the
|
||
ADPCM (`x68k.cpp:1052-1053,159`). The real bus has `#EXREQ`/`#EXACK`
|
||
(B36/B37), so external-request modes exist on hardware and are simply absent
|
||
from the model. **Any request-generation experiment run in MAME would be
|
||
measuring the gap, not the board.**
|
||
|
||
### 43.4 What it costs, on the containers session 13 already had
|
||
`17_span_delivered.py` and `14_dmac_chain.py` now take `--disk-clk-byte`;
|
||
`--disk-clk-word` is kept and halves it, so session 13's tables reproduce
|
||
exactly. Frames over the 12fps budget, 120-frame `singe` window:
|
||
|
||
| container | KB/s | 2.5 c/B (s13) | 5 c/B | 9 c/B | 16.5 c/B |
|
||
|---|---:|---:|---:|---:|---:|
|
||
| `s12_280_off` | 278.6 | 76/120 | 100/120 | 118/120 | 120/120 |
|
||
| `rc_fr_singe_scsi_span` (the gate) | 487.7 | 11/120 | 71/120 | 119/120 | 120/120 |
|
||
| `s13_280p1000` | 815.4 | 7/120 | 45/120 | 120/120 | 120/120 |
|
||
| **`s13_280p1500`** (s13's candidate) | 837.4 | **0/120** | 44/120 | 120/120 | 120/120 |
|
||
| `s13_lossless` | 934.6 | **0/120** | 87/120 | 120/120 | 120/120 |
|
||
|
||
**Nothing session 13 emitted fits at any point in the real bracket.** The
|
||
mechanism is 42.2's own table read at the right price. Per pixel, with `c` the
|
||
clocks per delivered byte:
|
||
|
||
v7 literal span 2 bytes 9.143 + 2c
|
||
RAW block 1 byte 25.03 + c
|
||
V1 block 0.25 B 18.74 + 0.25c
|
||
|
||
A span beats RAW for `c < 15.9` -- still true everywhere real. But **a span
|
||
beats a V1 block only for `c < 5.48`**, and that crossover sits between the
|
||
single-address row and the dual-address row of 43.2. The span pass is not
|
||
robust to the wiring question; it is decided by it. The encodes in 43.6
|
||
confirm the prediction: spans paint 30.7% of the picture at `c=5` and 3.9% at
|
||
`c=9`.
|
||
|
||
### 43.5 A frame's cost is now, to three figures, its byte count
|
||
On `s13_280p1500` at 5 clocks/byte, `corr(bytes, total frame clocks) = 0.989`,
|
||
and the disk term's p10..p90 spread (281k..464k clocks) is most of the frame
|
||
total's (713k..966k). Frames that miss carry 87,203 bytes on average against
|
||
61,295 for frames that fit.
|
||
|
||
**Which breaks the rate controller's bucket.** It banks bytes across 8 frames
|
||
because the player's ring buffer can hold them -- true, and irrelevant now:
|
||
those bytes are also clocks, and FINDINGS 28 established there is no double
|
||
buffer to decode ahead into, so **a frame that borrows bytes from the bucket
|
||
borrows clocks it cannot bank.** Byte smoothing was free when bytes were free.
|
||
It is now a direct source of deadline misses.
|
||
|
||
### 43.6 The encoder was never told a byte costs anything either
|
||
This is why the session could not just rescore. `ratectl.py` bisected `mu`
|
||
against 833,333 cycles of DECODE, with no disk term, and `spans.select()`
|
||
admitted a run "only if the span beats the blocks it replaces on cycles ALONE"
|
||
-- explicitly ignoring the bytes it adds. Both now work in one currency:
|
||
|
||
- `ratectl.DISK_CLK_BYTE` (default **5.0**, `encode.py --disk-clk-byte`) is
|
||
charged inside the CPU ceiling, so every fit test is `decode + c*bytes`.
|
||
- `spans.select()` admits and ranks on **net** clocks, `(clocks won) - c*(bytes
|
||
added)`. At `c=0` both reduce exactly to the old rules, and
|
||
`--disk-clk-byte 0` re-emits `s13_280p1500` byte for byte (8,501,948 B,
|
||
31.04 dB) -- so the change is the price, not the codec.
|
||
|
||
Re-encoded honestly, 120-frame `singe` window, 12 fps:
|
||
|
||
| | KB/s | PSNR | loss vs ceiling | over budget | span px |
|
||
|---|---:|---:|---:|---:|---:|
|
||
| s13's claim (2.5 c/B) | 837.4 | 31.04 dB | 0.29 dB | 0/120 | 62.3% |
|
||
| **`s14_d5_all1500`, single address** | **496.7** | **29.19 dB** | 2.14 dB | **1/120** | 30.7% |
|
||
| `s14_d9_all1500`, dual address | 255.0 | 28.50 dB | 2.83 dB | **1/120** | 3.9% |
|
||
|
||
**The one frame over is frame 0** in both -- the intra frame, which FINDINGS
|
||
28.5/31 already established is emitted late on purpose because there is nothing
|
||
on screen to hold. **Every other frame lands at or under 100.0%**: the
|
||
controller now binds exactly on the joint budget. `17_span_delivered.py`, which
|
||
shares no code with the encoder's own accounting, reproduces both rows to the
|
||
digit.
|
||
|
||
Two smaller results fall out:
|
||
|
||
1. **`--spans all` is now the better rule, and `--spans need` the worse one**
|
||
(275.8 KB/s, 28.92 dB, 2/120). `need` stops as soon as the frame fits and
|
||
leaves profitable clock savings unbought; once profitability is measured in
|
||
one currency, spending every profitable byte *is* the optimum. My
|
||
recommendation is to make `all` the default; that is my inference, not a
|
||
measurement.
|
||
2. **`--span-kbps` has stopped binding.** 700 and 1500 produce identical files.
|
||
Session 13's saturation was the byte ceiling running out; this one is the
|
||
span pass running out of runs that pay, which is a property of the codec
|
||
rather than of a chosen number.
|
||
|
||
### 43.7 What this withdraws
|
||
- **42.3's 0/120 and "quality and deadline stopped competing".** They compete
|
||
again, and harder than before: a byte now buys picture and spends deadline.
|
||
- **42.3's "`mu` is never spent"** and the refund of FINDINGS 31's 0.62 dB.
|
||
`mu` is spent on 103 of 120 frames at `c=5` and 118 at `c=9`.
|
||
- **42.8's revival of 17.5** ("ship pixel-exact if SCSI sustains >=800 KB/s").
|
||
The stream that fits is 496.7 KB/s and 2.14 dB off the palette ceiling, so
|
||
17.5's threshold is not met and its conclusion does not return. The
|
||
methodology point in 42.8 stands and now applies to itself.
|
||
- **`docs/BENCHMARK.md`'s "~8 clocks/word => ~2.5 MB/s practical ceiling".**
|
||
The ceiling is 2 MB/s at best (5 clocks/byte) and 1.11 MB/s dual-address.
|
||
- **FINDINGS 5's 8 clocks/word**, retroactively, wherever it was used: every
|
||
I/O debit in the project before this section was charged at half rate.
|
||
|
||
### 43.8 What is NOT established
|
||
1. **Single-address versus dual-address is a hardware fact this tree cannot
|
||
check.** It is worth 242 KB/s and 0.69 dB, and MAME models only the dual
|
||
row (43.2). It needs the CZ-6BS1 schematic or a real board -- and note it is
|
||
the *board's* wiring, not our code, so unlike every previous item on this
|
||
list it is not a design decision we get to make.
|
||
2. **The 5 and the 9 are datasheet floors with the drive wait set to zero.**
|
||
42.6's argument that SD-backed SCSI collapses that term is unchanged and
|
||
still unmeasured.
|
||
3. **Front-end and back-end overhead are excluded**, which assumes at least a
|
||
sector per period of bus ownership. If the player ends up taking the bus per
|
||
operand, add 7-10 clocks to every byte and nothing fits at all.
|
||
4. **Still no 68000 has decoded any of these containers** (42.7 item 1). The
|
||
new ones are lighter in spans than the gate, so that gap is narrower than it
|
||
was, but it is the same gap.
|
||
5. **The bucket has not been fixed**, only diagnosed (43.5).
|
||
|
||
### 43.9 The unit was never written down
|
||
The error is one substitution -- bytes/2 for bytes -- and it survived from
|
||
FINDINGS 5 through nine sessions, two cost-model rewrites, a datasheet reading
|
||
that corrected the *value* of the same constant (39.7), and a section devoted
|
||
to distrusting the table it produced (42.7, which lists six things 42.3 did not
|
||
establish and does not list its own denominator).
|
||
|
||
What let it hide: `W` was carried as "clocks per word" in three tools and a
|
||
half-dozen tables, and **the device's port width was never in the same sentence
|
||
as it.** The 8-bit-ness of the MB89352 was known -- it is visible in
|
||
`install_device(..., 0x00ff00ff)` and in every register map in the project --
|
||
but it lived in the I/O notes while `W` lived in the budget arithmetic.
|
||
|
||
The rule this project already had (FINDINGS 33: a design that counts only CPU
|
||
shows a win the I/O it created takes away) needed one more clause: **check what
|
||
the unit is denominated in, on the device that supplies it.** A per-word debit
|
||
for a byte-wide port is a factor of two, and a factor of two is the difference
|
||
between this game running and not.
|
||
|
||
---
|
||
|
||
## 44. The byte-side rate controller is inert, and both inconsistencies in it are worth under 2% (session 15)
|
||
|
||
STATUS's item 2 asked for the leaky bucket to be fixed: 43.5 diagnosed it as
|
||
banking bytes that are now clocks, across a player with no double buffer to
|
||
bank clocks in, and called it "a direct source of deadline misses". The
|
||
diagnosis is correct as a mechanism. **It is not a source of anything at the
|
||
operating point this project actually recommends**, and neither is the second,
|
||
larger-looking inconsistency found next to it. Both are now implemented, both
|
||
measure as a wash or a regression, and **both ship off by default** --
|
||
`--joint-decide` and `--joint-bucket` turn them on.
|
||
|
||
Everything below is the 120-frame `singe` window, `--profile scsi --kbps 280
|
||
--span-kbps 1500`, `c = 5` clocks/byte, 12 fps, scored both by the encoder and
|
||
by `17_span_delivered.py`, which shares no code with it.
|
||
|
||
### 44.1 The mode decision was the last place a byte was free
|
||
43.6 charged the disk inside the rate controller's fit test and inside
|
||
`spans.select()`, but `vq_hybrid.decide()` still minimised `D + lam*bytes +
|
||
mu*cycles` with `cycles` meaning DECODE cycles only. So while `mu` was
|
||
enforcing a joint budget from above, the per-block lagrangian underneath it
|
||
still believed delivery was free.
|
||
|
||
**That inverts FINDINGS 28.8.** RAW is 400.4 cycles against V4's 448.2, so with
|
||
a free byte, raising `mu` buys cycles by moving V4 -> RAW -- which is exactly
|
||
what 28.8 observed and what session 8's `0c` recorded as V4 collapsing. Priced
|
||
per delivered byte, a RAW block costs `400.4 + 16c` and a V4 block `448.2 + 4c`:
|
||
|
||
| c, clocks/byte | V1 | V4 | RAW |
|
||
|---:|---:|---:|---:|
|
||
| 0 | 299.9 | 448.2 | **400.4** |
|
||
| 2.5 (the retired 5 clk/word) | 302.4 | 458.2 | **440.4** |
|
||
| **3.98** | 303.9 | **464.3** | **464.3** |
|
||
| 5 (single address) | 304.9 | **468.2** | 480.4 |
|
||
| 9 (dual address) | 308.9 | **484.2** | 544.4 |
|
||
|
||
**The crossover is c = 3.98 and 43.1's floor argument is c >= 4.** A 68000 bus
|
||
cycle is four clocks and the SPC hands over one byte per cycle, so RAW's cycle
|
||
advantage does not exist on any real machine: it was spending 12 clocks of bus
|
||
to save 47.8 of CPU. The escape hatch was an artefact of the same free byte
|
||
that FINDINGS 43 found everywhere else.
|
||
|
||
### 44.2 And correcting it changes almost nothing
|
||
`decide()` now takes `byte_clk` and prices a payload byte at `lam + mu*byte_clk`.
|
||
At `byte_clk = 0` it is the old decision exactly, and the containers of session
|
||
14 re-encode to the same MD5.
|
||
|
||
| `--spans all`, c=5 | KB/s | PSNR | mean frame clocks | over budget | worst |
|
||
|---|---:|---:|---:|---:|---:|
|
||
| the shipped decision | 496.7 | **29.19** | **740,049** | 1/120 | 112.9% |
|
||
| `--joint-decide` | 482.5 | 29.17 | 745,438 | 1/120 | 112.9% |
|
||
|
||
It **buys 6,058 clocks of disk with 17,207 clocks of block decode** -- a net
|
||
5,389 clocks a frame in the wrong direction. Scored at c=4 and c=9 it is the
|
||
marginally worse container at every price, so this is not a bet on which row of
|
||
43.2 wins. The reason is scale, not sign: at the `mu` the controller actually
|
||
settles on (median 0.74-0.91) the added byte price is `mu*c` ~ 4 against a
|
||
`lam` floor of 10, and the RAW/V4 decision is dominated by V4's distortion term
|
||
rather than by either. RAW moves 16.0% -> 15.7% of blocks.
|
||
|
||
**The inconsistency was real, the correction is right, and the effect is 0.02 dB.**
|
||
|
||
### 44.3 The bucket does not bind, and at `--spans all` nothing on the byte side does
|
||
Before fixing the bucket, measure whether it is loaded. It is not:
|
||
|
||
| `--spans all`, c=5 | KB/s | PSNR | over budget | container |
|
||
|---|---:|---:|---:|---|
|
||
| `--bucket-frames 8` (shipped) | 496.7 | 29.19 | 1/120 | baseline |
|
||
| `--bucket-frames 32` | 496.7 | 29.19 | 1/120 | **byte-identical** |
|
||
| `--bucket-frames 1` (no banking at all) | 498.0 | 29.19 | 1/120 | +1.3 KB/s |
|
||
| `--rc-floor open` (lam floor 1.0) | 503.7 | 29.21 | 1/120 | +0.02 dB |
|
||
|
||
**`lam` never leaves its floor of 10.0 on any of 120 frames**, in any of these,
|
||
and quadrupling the bucket emits the same bytes. With `--spans off` at 259.3
|
||
KB/s under a 280 KB/s target, a 1-frame bucket and an 8-frame bucket are again
|
||
byte-identical. The block coder at the profile floor simply lands under the
|
||
per-frame byte budget, so there is nothing for the bucket to lend and nothing
|
||
for the lam bisection to do. **The rate this project reports is set by the span
|
||
pass and by `mu`; `--kbps` and the bucket are not the levers.**
|
||
|
||
The one place the bucket does cost something is the mode 43.6.1 recommends
|
||
abandoning: at `--spans need` it is worth exactly one frame of 120.
|
||
|
||
### 44.4 The cap, and why capping only half a frame is worse than not capping
|
||
`--joint-bucket` caps what the bucket may lend at what the frame's clock budget
|
||
can still absorb after its own block decode -- `(cycle_budget - cycles(mode0))
|
||
/ c`, priced at the mode map the un-banked budget buys, and never below that
|
||
budget. The bucket then smooths only what is left after the disk is paid, which
|
||
is what STATUS item 2 asked for.
|
||
|
||
| c=5 | KB/s | PSNR | median frame | over budget |
|
||
|---|---:|---:|---:|---:|
|
||
| `need`, shipped | 275.8 | 28.92 | 99.8% | 2/120 |
|
||
| **`need --joint-bucket`** | 302.1 | 28.91 | 99.9% | **1/120** |
|
||
| `all`, shipped | **496.7** | **29.19** | **87.9%** | 1/120 |
|
||
| `all --joint-bucket` | 506.4 | 29.18 | 89.6% | 1/120 |
|
||
| `all --joint-bucket`, cap extended to the span section | 273.7 | 28.88 | 99.4% | 1/120 |
|
||
|
||
**It RAISES the bitrate at `--spans all`**, which is the tell: capping the block
|
||
payload does not remove those bytes, it moves them into the span section, which
|
||
draws on its own flat pipe (41.2) and is not under the cap at all. So the third
|
||
row is the honest reading of "the per-frame ceiling should be joint and hard" --
|
||
and it is the worst container here, losing 0.30 dB and 233 KB/s for no change in
|
||
the frames-over count, because the cap starves the pass that was buying the
|
||
deadline in the first place.
|
||
|
||
There is no per-frame ceiling on the whole frame record anywhere in this
|
||
encoder, and **44.3 is why adding one has not been urgent**: the joint budget is
|
||
enforced after the span pass by `mu`, which is a controller that pays in
|
||
picture, and the byte-side ceiling it would replace is not binding.
|
||
|
||
### 44.5 An encode is 95% k-means, and that is now 2.7x faster, exactly
|
||
Prompted by the user observing that this should not take a minute a scene.
|
||
Profiled, a 120-frame encode was 60.6 s of which 60.8 s was two k-means runs
|
||
(the rest of the encoder, rate control and the span pass included, is about a
|
||
second). `VQ.assign` was all of it, and three things were on the floor, none of
|
||
which changes a label:
|
||
|
||
| | 1,474,560 2x2 blocks, k=256 |
|
||
|---|---:|
|
||
| as written | 1.83 s |
|
||
| `C.T` materialised once (a view makes BLAS re-copy it per chunk) | 1.02 s |
|
||
| chunk 8192 -> 2048 (the `(chunk,k)` temporary, cache not memory; 32768 is 2.80 s) | 0.81 s |
|
||
| 8 threads over the chunk loop (numpy releases the GIL in both matmul and argmin) | **0.31 s** |
|
||
|
||
Partitioning rows cannot change an `argmin`, so the labels are bit-identical to
|
||
the serial ones and every container this encoder emits still hashes the same --
|
||
which is the assertion, not the hope: `s14_d5_all1500` re-encodes to
|
||
`d13d142b...` on both sides of the change. Whole encode 60.6 s -> 29.4 s,
|
||
`H.build` 60.9 -> 22.2 s. What is left is `np.add.at` in the centroid update
|
||
(~5 s of 22), and taking it costs the bit-exactness, so it stays.
|
||
|
||
At 0.18 s/frame the 22.8 minutes of unique scene footage is ~50 minutes of
|
||
codebook training, single machine, single pass.
|
||
|
||
### 44.6 What this does NOT establish
|
||
1. **It does not re-open the c=5 vs c=9 question** (43.8.1), which is still the
|
||
largest open number and still a hardware fact this tree cannot check.
|
||
2. **The 1/120 is the same 1/120** -- frame 0, the intra frame, emitted late on
|
||
purpose. Nothing here moved it, and 43.8's five caveats all stand.
|
||
3. **`--spans all` remains a recommendation, not a measurement** (43.6.1), and
|
||
44.3 sharpens why it matters: it is the only lever on this side of the
|
||
encoder that does anything.
|
||
4. **The pixel-exact gate now covers 37 of 120 frames**, down from 80 in session
|
||
10 (FINDINGS 36) -- the span-heavy container outgrew the 2 MB machine. The
|
||
strongest test in the tree audits under a third of the window it names. That
|
||
makes STATUS item 3's chunk-streaming rig load-bearing rather than optional.
|
||
5. **Two fixes, both correct, both defaulted off.** The pattern worth keeping is
|
||
that the second one was found by asking whether the lever was loaded before
|
||
pulling it, and 44.3 took four encodes to establish -- against a session that
|
||
could have been spent making a bucket cap work.
|
||
|
||
### 44.7 No decoder at all: what a literal frame costs, and where it dies
|
||
Asked by the user: does streaming raw preprocessed frames straight into video
|
||
memory save the CPU? On clocks, yes -- completely. It dies on the medium, and
|
||
the arithmetic is worth writing down because it also retires the last line of
|
||
42.2 still standing in the wrong units.
|
||
|
||
**Two versions, and the split is the answer.** A frame is 256x192 = 49,152
|
||
pixels; 256-colour GVRAM is one pixel per WORD with the high byte discarded
|
||
(42.2, `spans.py:44`, and the memory model both emulators are pixel-exact
|
||
against), so a literal frame is **98,304 bytes of GVRAM writes** against a
|
||
budget of 833,333 clocks = **16.95 clocks/pixel**.
|
||
|
||
*1. The CPU paints it.* This is what a v7 span already is -- literal GRB555
|
||
words, preprocessed offline, copied by `movem` -- at `9.143 + 2c` clocks/pixel:
|
||
|
||
| | clocks/pixel | vs 16.95 |
|
||
|---|---:|---|
|
||
| c = 3.906 | 16.95 | breakeven |
|
||
| **c = 4.0**, the 43.1 floor | **17.14** | **misses by 1.1%** |
|
||
| c = 5.0, single address | 19.14 | misses by 13% |
|
||
| c = 9.0, dual address | 27.14 | misses by 60% |
|
||
|
||
**A CPU-painted full-frame literal does not fit at any physically reachable
|
||
price**, and it misses at the floor itself. This supersedes 42.2's "lossless
|
||
fits iff `W` is about 6.5 or less", which was denominated per WORD: 6.5
|
||
clocks/word is 3.25 clocks/byte, below the 4-clock floor, so that threshold was
|
||
never reachable either. FINDINGS 43 withdrew the tables downstream of the unit
|
||
error but not this line; it is withdrawn here.
|
||
|
||
*2. The DMAC writes device -> GVRAM and the CPU is not in the loop.* The 9.143
|
||
disappears and a pixel costs only its two wire bytes:
|
||
|
||
| c | clocks/frame | % of budget | wire | 22.8 min of game |
|
||
|---:|---:|---:|---:|---:|
|
||
| 4.0 | 393,216 | 47.2% | 1,152 KB/s | 1.61 GB |
|
||
| **5.0** single address | **491,520** | **59.0%** | 1,152 KB/s | 1.61 GB |
|
||
| 9.0 dual address (the row MAME models) | 884,736 | **106.2%** | 1,152 KB/s | 1.61 GB |
|
||
| 16.5 packed | 1,622,016 | 194.6% | 1,152 KB/s | 1.61 GB |
|
||
|
||
**At c=5 it fits, with 41% of the frame budget left and no decoder at all.** It
|
||
is defeated by delivery, not by the 68000: 1,152 KB/s is ~79% of SCSI-1
|
||
asynchronous (42.1) with nothing left for audio or seeks, and 1.61 GB is 2.3x
|
||
the 0.70 GB the shipping 496.7 KB/s container already needs -- which is itself
|
||
already past a CD-ROM. At c=9 it does not fit the clocks either.
|
||
|
||
**The waste is specific and it is the same one the span path pays.** Because the
|
||
high byte is discarded, half of every byte pulled off the disk is thrown away by
|
||
the hardware on arrival: two bytes of disk per byte of picture. That 2:1 is why
|
||
a span pixel costs `2c` and why 43.4's span-vs-V1 crossover sits at c=5.48. The
|
||
codec is not there to save CPU -- 44.7 shows the CPU can be removed outright --
|
||
it is there to save the wire.
|
||
|
||
**The one thing that would change this answer** is a packed write path into
|
||
256-colour GVRAM, two pixels per word. That halves the wire to 576 KB/s and
|
||
806 MB and puts a decoder-free player back in play. **It is NOT established
|
||
either way.** The tree's basis for one-pixel-per-word is that MAME and px68k are
|
||
both pixel-exact against that model, which is evidence about two emulators, not
|
||
about the CRTC and the palette hardware. It is the same class of question as
|
||
43.8.1 -- a service manual or a real board settles it, an emulator cannot, and
|
||
both emulators here would model a packed path identically wrong if it exists.
|
||
|
||
Note this is a different transfer from the one FINDINGS 39 costed and rejected.
|
||
39 priced **memory -> GVRAM** (the DMAC replacing v6's `movem` chain out of a
|
||
RAM stream buffer) and it lost by a clock. This is **device -> GVRAM**, with no
|
||
RAM staging and no CPU, and it loses to the disk instead.
|
||
|
||
## 45. The strongest test in the tree was short by 83 frames, and the fix was the rig's memory (session 16)
|
||
|
||
STATUS item 1 said the pixel-exact gate "needs the chunk-streaming rig, not a
|
||
longer pass." It needed neither. It needed a bigger emulated machine, and the
|
||
reason that was not obvious is worth more than the fix.
|
||
|
||
### 45.1 The constraint was the rig's, and the gate does not measure timing
|
||
`prep_dlx.py` preloads the whole container into emulated RAM at `STREAM=0x30000`
|
||
and `check.sh` ran the machine at `-ramsize 2M`, so the span-heavy gate container
|
||
-- 5,261,814 B of stream, ending at 0x534BF6 -- was truncated to the prefix that
|
||
fit. That was **37 of 120 frames**, down from 80 in session 10 (FINDINGS 36) as
|
||
the container grew.
|
||
|
||
The 2 MB was carried over from the timing rig, where it is correct: the locked
|
||
target is a stock 2 MB machine. But `check.sh` runs this gate under
|
||
`DLX_VERIFY_ONLY=1`, which **drops the cost anchors entirely** and asserts only
|
||
pixel-exactness. A verify-only pass makes no claim about the target's memory, so
|
||
it was never the 2 MB that was load-bearing here -- and preloading a whole
|
||
container is unlike the shipping player at *any* size, because the player streams
|
||
from disk into a ring buffer and never holds a window at once.
|
||
|
||
`RIG_RAM=6` in `check.sh` (MAME's x68000 accepts 1M-12M) covers all 120 frames.
|
||
|
||
### 45.2 The raise is licensed by measurement, not by convenience
|
||
Raising the emulated RAM to make a test pass is exactly the move that should be
|
||
distrusted, so it was checked rather than asserted. The full timing pass -- not
|
||
verify-only -- was run at `-ramsize 2M` and `-ramsize 6M`, and the **five
|
||
synthetic anchors come out bit-identical**:
|
||
|
||
| synthetic frame | 2M | 6M |
|
||
|---|---:|---:|
|
||
| all-SKIP | 40,729 | 40,729 |
|
||
| all-V1 | 921,187 | 921,187 |
|
||
| all-V4 | 1,376,881 | 1,376,881 |
|
||
| all-RAW | 1,229,883 | 1,229,883 |
|
||
| all-SPAN-64 | 506,533 | 506,533 |
|
||
|
||
They sit at **different addresses** in the two layouts -- the synthetic block is
|
||
placed after a 37-frame stream in one and a 120-frame stream in the other -- and
|
||
still cost the same, so MAME's cycle model does not depend on `-ramsize` over
|
||
this range. Every per-block constant in FINDINGS 24/30/41 is measured from these
|
||
frames and is therefore unmoved by the change.
|
||
|
||
### 45.3 The 37-frame prefix was a biased sample, and the quiet end was missing
|
||
The gate is now 120/120 pixel-exact on **both** cores -- MAME's 68000 and px68k's
|
||
C68K. (The C68K harness never had a RAM ceiling at all: its arena is 16 MB and
|
||
`RAMTOP` is defined but unused, so it was short only because it reads the blob
|
||
`prep_dlx.py` truncated.) What the extra 83 frames show is that the prefix was
|
||
not representative:
|
||
|
||
| anchor | 37-frame prefix | full 120 |
|
||
|---|---:|---:|
|
||
| min non-SKIP | 25.1% of blocks, 61.9% of budget | **15.2%, 53.6%** |
|
||
| median | 45.4%, 79.7% | **41.1%, 81.1%** |
|
||
| p90 | 52.3%, 93.1% | **48.5%, 91.1%** |
|
||
| max non-SKIP | 62.5%, 91.8% | 62.5%, 91.8% (same frame) |
|
||
| C68K sequential-pass mean | 693,886 cyc, **83.3%** | **641,444 cyc, 77.0%** |
|
||
|
||
The prefix **overstated the mean cost of the window by 8.2%**. It caught the
|
||
worst frame -- the max is the same frame in both -- but it never saw the quiet
|
||
end: the true minimum is 15.2% non-SKIP against the prefix's 25.1%, and 53.6% of
|
||
budget against 61.9%. The distribution the anchors exist to sample was cut off at
|
||
one end, which is the failure mode FINDINGS 25.6 warned about in a different
|
||
guise: a prefix is not a sample.
|
||
|
||
Note the direction. The prefix was **pessimistic**, so nothing downstream of it
|
||
was flattered, and no headroom claim in this tree was resting on the missing
|
||
frames. That is luck, not design.
|
||
|
||
### 45.4 What this does NOT establish
|
||
1. **The streaming path is still untested, at either RAM size.** The rig
|
||
preloads; the player streams into a ring buffer. This gate proves the decoder
|
||
is pixel-exact over a whole window, and says nothing about the ring buffer,
|
||
the chunk boundaries, or the disk. The chunk-streaming rig STATUS item 1
|
||
called for is still unbuilt -- it was just never what the 37/120 needed.
|
||
2. **The 2 MB target is unchanged.** `RIG_RAM` is the emulated machine's memory
|
||
for a verify-only pass. `span.sh` and the session-7 timing reproduction still
|
||
run at 2M, and the shipping player's memory budget is untouched.
|
||
3. **The truncation guard stays.** `prep_dlx.py` still truncates and announces
|
||
it, and `check.sh` still reads the count back and greps for `TRUNCATED`. A
|
||
heavier container, or a lowered `RIG_RAM`, brings it straight back.
|
||
4. **The moved anchors are a re-measurement, not a regression.** No constant
|
||
changed; the frames the anchors point at did.
|
||
|
||
## 46. The 256-colour mask is defeatable, and a packed path is back in play (session 16)
|
||
|
||
STATUS item 2 asked whether 256-colour GVRAM has a PACKED write path -- two
|
||
pixels per word rather than one pixel per word with the high byte discarded. It
|
||
**The first answer was no and it was wrong** -- or rather, it was right about the
|
||
default write path and missed the register that turns the masking off. 46.1-46.3
|
||
record what was established and what was measured, and are kept as written.
|
||
**46.5 is the correction and it is the important part of this section.**
|
||
|
||
### 46.1 The answer is no, and the sub-word fields are PAGES, not pixels
|
||
Three independent lines, none of them a service manual (see 46.4 on evidence
|
||
class):
|
||
|
||
1. **A community hardware guide** (x68000-dev-guide, `docs/graphics.md`): "each
|
||
pixel occupies exactly one word (2 bytes), regardless of the color mode." In
|
||
256-colour mode the word packs **2 pages** -- page 0 is mask `$00FF` at the
|
||
`$C00000` alias, page 1 is mask `$FF00` at `$C80000`.
|
||
2. **A Japanese retro-computing writeup** (wizforest), independently: 1 word = 1
|
||
dot in every display mode, and in 256-colour mode only the lower 8 bits of a
|
||
1 MB region are valid. It states the masking as the well-known nuisance it
|
||
was, at a *different* depth: "16色モードでは 16bit を書き込んでもハード的に
|
||
マスクされてしまって 4bit しか書きこまれない" -- in 16-colour mode a 16-bit
|
||
write is hardware-masked down to 4 bits.
|
||
3. **px68k's write path**, which is the mechanism the other two describe:
|
||
|
||
```c
|
||
case 1: /* 256 colors */
|
||
if ( adr<0x100000 )
|
||
if ( !(adr&1) ) { /* the other byte of the word: discarded */
|
||
...
|
||
if (adr&0x80000) adr+=1; /* the $C80000 alias IS the other byte */
|
||
adr &= 0x7ffff;
|
||
GVRAM[adr] = data;
|
||
```
|
||
|
||
**The decisive detail is that both pages derive `line` identically**, from
|
||
`(adr&0x7ffff)>>10` -- 1024 bytes per scanline for 512 pixels. Page 0 and page 1
|
||
are the two bytes of one word **at the same screen coordinate**. The same holds
|
||
in 16-colour mode, where `page = (adr>>17)&0x0c` selects one of four nibbles of
|
||
the word: four pages, one coordinate. There is **no graphics mode in which one
|
||
16-bit word holds two horizontally adjacent pixels.**
|
||
|
||
### 46.2 The near-miss is worth writing down so it is not re-derived
|
||
Page 1 has its own scroll register, so scrolling it one pixel relative to page 0
|
||
would put its byte at screen x+1 while page 0's sits at x -- two adjacent screen
|
||
pixels from one word. It does not work, and the arithmetic is why: page 1 stores
|
||
a byte for **every** coordinate, not every other one, so to let page 0 show
|
||
through on alternate columns you must write page 1's transparent index there.
|
||
That is the same byte count. 1024 bytes per row still buys ~512 screen pixels.
|
||
The 2:1 is structural, not an addressing accident.
|
||
|
||
**So FINDINGS 44.7 stands unchanged.** The decoder-free player still needs
|
||
1,152 KB/s and 1.61 GB, and is still killed by the medium. The `2c` span pixel,
|
||
the c=5.48 span-vs-V1 crossover, and every figure resting on two disk bytes per
|
||
picture byte are unmoved.
|
||
|
||
### 46.3 But the text plane is 4bpp PLANAR, and that is 4x denser on the wire
|
||
The tax is a property of the *graphics* planes. The X68000's text plane is not
|
||
laid out that way at all -- px68k's `TVRAM_Write` addresses **four planes at
|
||
`0x20000` stride**, each 0x20000 = 131,072 B = 1024x1024 **bits**. That is 4bpp
|
||
planar: **0.5 bytes per pixel**, against 2.0 for 256-colour graphics.
|
||
|
||
The wire arithmetic, and it is only arithmetic:
|
||
|
||
| surface | B/pixel | frame (256x192) | at 12fps | 22.8 min of game |
|
||
|---|---:|---:|---:|---:|
|
||
| 256-colour graphics, literal (44.7) | 2.0 | 98,304 | 1,152 KB/s | 1.61 GB |
|
||
| **shipping DLX3 container, compressed** | -- | -- | **496.7 KB/s** | **0.70 GB** |
|
||
| **4bpp planar text plane, literal** | **0.5** | **24,576** | **288.0 KB/s** | **0.40 GB** |
|
||
|
||
**An UNCOMPRESSED 16-colour frame is 42% cheaper on the wire than this project's
|
||
compressed 256-colour stream**, and it needs no decoder at all -- the planar
|
||
conversion is an encoder-side transform, so the disk delivers plane words that
|
||
go straight out. On clocks it is not close either: 24,576 bytes/frame is
|
||
`24,576c`, which at c=5 is 122,880 clocks = **14.7% of a 12fps budget**.
|
||
|
||
**It was measured the same session, and it is dead.** `tools/analysis/18_text_plane_16col.py`
|
||
over the 120-frame `singe` window, PSNR against the 24-bit source, generous to
|
||
the 16-colour side on every axis the hardware allows -- per-frame palettes (the
|
||
text palette is 16 entries; reloading it is 16 words a frame, nothing against
|
||
833,333 clocks) which the 256-colour path *cannot* use, because its codebooks
|
||
are indices into a scene-wide palette:
|
||
|
||
| | mean PSNR | min | max |
|
||
|---|---:|---:|---:|
|
||
| 256 colours, scene palette (the tree's) | 31.33 | 27.08 | 34.08 |
|
||
| 256 colours, per-frame palette | 34.08 | 32.19 | 38.32 |
|
||
| 16 colours, scene palette | 23.17 | 17.36 | 25.87 |
|
||
| **16 colours, per-frame palette** | **25.49** | 22.99 | 28.48 |
|
||
|
||
**256 -> 16 costs 5.84 dB at each side's best.** Against the shipping container's
|
||
**29.19 dB at 496.7 KB/s**, a 16-colour literal delivers **25.49 dB at 288.0
|
||
KB/s** -- **3.70 dB worse for 58% of the bitrate.** A codec that buys 3.70 dB for
|
||
1.72x the bytes is doing its job; the wire saving does not pay for the colours.
|
||
|
||
**Closed.** The user's call was to drop the 16-colour direction outright, and the
|
||
number agrees with it, so the text plane is not pursued. FINDINGS 7's 256-colour
|
||
claim stands, and it now stands on a measurement rather than on preference.
|
||
|
||
Two honest notes on that measurement:
|
||
- **The dither row is void.** A Floyd-Steinberg run was included and came out
|
||
*bit-identical* to the undithered one, which means PIL ignored `dither=` under
|
||
`MEDIANCUT` rather than that dither is free. It is left out of the table. It
|
||
does not change the conclusion -- dither lowers PSNR by construction, and the
|
||
lead was already 3.70 dB short.
|
||
- **Everything else about the path stayed unestablished and now stays that way**:
|
||
TVRAM wait states (every `c` in this tree is a GVRAM figure), the text plane's
|
||
geometry and priority against the graphics planes, and the fact that a planar
|
||
word spans 16 pixels of one bitplane, so VQ blocks and v7 spans -- both chunky
|
||
-- would not survive the change unaltered. None of it was worth measuring once
|
||
the colour cost came in.
|
||
|
||
### 46.4 Evidence class, stated plainly
|
||
46.1 is **secondary documentation plus an emulator's mechanism**, not primary. No
|
||
service manual, CRTC databook, or real board was consulted. What changed since
|
||
44.7 is the *kind* of evidence: 44.7 rested on MAME and px68k both being
|
||
pixel-exact against one-pixel-per-word, which is evidence about two emulators
|
||
that could be identically wrong. Now two independent documents describe the same
|
||
mechanism -- sub-word fields are pages sharing a coordinate -- and px68k's code
|
||
implements exactly that mechanism, including the `$C80000` alias landing on the
|
||
adjacent byte. Agreement on a mechanism is much harder to get wrong by accident
|
||
than agreement on an output.
|
||
|
||
It is still not a board. STATUS item 3 (single- vs dual-address, 43.2/43.3) is
|
||
unaffected by any of this and remains the largest open hardware fact.
|
||
|
||
### 46.5 CORRECTION: CRTC R20 bit 11 turns the masking off
|
||
46.1 concluded there is no packed write path. That conclusion was drawn from
|
||
px68k's `GVRAM_Write` and two documents describing the default behaviour, and it
|
||
**missed a register bit that both emulators implement**. MAME's
|
||
`x68k_crtc_device::gvram_w` shows it first:
|
||
|
||
```c
|
||
if (m_reg[20] & 0x0800) /* "G-VRAM set to buffer" */
|
||
{
|
||
if (offset < 0x40000)
|
||
m_gvram_write_callback(offset, data, mem_mask); /* FULL WORD, unmasked */
|
||
}
|
||
else switch (m_reg[20] & 0x0300)
|
||
{
|
||
case 0x0100: /* 256 colour */
|
||
if (offset < 0x40000)
|
||
m_gvram_write_callback(offset, data & 0x00ff, 0x00ff);
|
||
else if (offset < 0x80000)
|
||
m_gvram_write_callback(offset - 0x40000, (data & 0x00ff) << 8, 0xff00);
|
||
```
|
||
|
||
The `case 0x0100` arm confirms 46.1 exactly -- `offset - 0x40000` is the **same
|
||
word, other byte**, and `data & 0x00ff` throws the CPU's high byte away on both
|
||
aliases. But the `m_reg[20] & 0x0800` arm bypasses the depth switch entirely and
|
||
writes the full 16 bits.
|
||
|
||
**px68k has the identical bit**, which is what makes this a mechanism and not a
|
||
MAME quirk: `if (CRTC_Regs[0x28]&8)`, commented `65536モードのVRAMアクセス
|
||
（Nemesis用）` -- "65536-mode VRAM access, for Nemesis". px68k's `CRTC_Regs` is
|
||
byte-indexed, so `[0x28]` is the **high byte of R20** and bit 3 of it is bit 11
|
||
of the register. Same bit, same effect, two independent implementations, and a
|
||
named shipping game that used it.
|
||
|
||
**So the 2:1 tax is a property of the default write path, not of the memory.**
|
||
|
||
### 46.6 A contiguous packed layout, derived and NOT yet tested
|
||
The second thing 46.1 missed is that the two 256-colour pages have **independent
|
||
scroll registers**. px68k's `Grp_DrawLine8(int page, int opaq)` indexes
|
||
`GrphScrollX[page*8]` and `GrphScrollY[page*8]`, selects the byte within the word
|
||
with `add esi, ecx` (page 0 -> low, page 1 -> high), and takes an `opaq` flag --
|
||
so the pages composite with transparency and can be offset from each other.
|
||
|
||
46.2 dismissed interleaving on byte count, and that dismissal assumed a 1-pixel
|
||
scroll. **Scrolling by 128 instead makes the used words contiguous**, which is
|
||
the whole difference:
|
||
|
||
- GVRAM row stride is fixed at 1024 bytes = 512 words (`shl esi, 10`).
|
||
- Write words 0..127 of each row, **unmasked, full 16 bits** (R20 bit 11).
|
||
- Page 0, unscrolled: `page0[i]` displays at screen column `i` -> columns 0..127.
|
||
- Page 1, X-scrolled by **+128**: `page1[i]` displays at column `i+128` ->
|
||
columns 128..255.
|
||
- Page 1 opaque and above page 0, so page 0's stale storage at 128..511 is
|
||
covered; page 1's storage at 128..511 displays at 256..383, which is off the
|
||
edge of the **real 256x256 mode this project already uses** (FINDINGS 23).
|
||
|
||
**128 contiguous words carry 256 pixels: 1.0 byte per pixel, against 2.0.** No
|
||
transparency mask to maintain, no stride for a DMAC to skip, and the writes are
|
||
`movem`-shaped. If it holds, it is exactly the halving 44.7 named:
|
||
|
||
| | B/pixel | frame | at 12fps | 22.8 min |
|
||
|---|---:|---:|---:|---:|
|
||
| 256-colour, default masked path | 2.0 | 98,304 | 1,152 KB/s | 1.61 GB |
|
||
| **packed via R20 bit 11 + page scroll** | **1.0** | **49,152** | **576 KB/s** | **0.81 GB** |
|
||
|
||
**This is a derivation, not a result.** What is CONFIRMED is the register bit
|
||
(both emulators, plus a named game) and the per-page scroll and opacity (px68k's
|
||
draw path). What is DERIVED and untested is the layout above: whether R20 bit 11
|
||
coexists with 256-colour *display* rather than forcing the 65536-colour
|
||
interpretation, whether the video controller's priority and transparency
|
||
registers can put page 1 over page 0 the way this needs, and what the 256-wide
|
||
screen does to page 1's off-edge storage.
|
||
|
||
**Unlike 43.2 and 43.8.1, this one the tree CAN answer.** It is a display-model
|
||
question, both emulators implement the mechanism, and the rig already screenshots
|
||
and compares pixel-exactly (`tools/bench/verify_frame256.py`). It is a register
|
||
setup and a snapshot, not a service manual -- and if it holds it halves the wire
|
||
for every path in this project, the shipping codec included.
|
||
|
||
## 47. The packed layout works on both emulators, and they disagree about what it costs (session 16)
|
||
|
||
46.6 derived a 1.0 byte/pixel layout and did not test it. It has now been built
|
||
and run on **both** emulators. **It renders correctly on both.** The two
|
||
disagree on two register semantics, and one of those disagreements decides
|
||
whether the thing is usable.
|
||
|
||
### 47.1 The write path, measured directly
|
||
`tools/bench/probe_packed.lua` writes one word and reads the two page aliases
|
||
back, under MAME:
|
||
|
||
| R20 | wrote | raw word | page 0 (low byte) | page 1 (high byte) |
|
||
|---|---|---:|---:|---:|
|
||
| 0x0110, bit 11 = 0 | `AB5C` | `005C` | `5C` | `00` |
|
||
| **0x0910, bit 11 = 1** | `AB5C` | `AB5C` | `5C` | **`AB`** |
|
||
|
||
**That is the 2:1 tax, and its off switch, in one table.** Masked, the CPU's high
|
||
byte is destroyed. In buffer mode one word write lands two picture bytes.
|
||
46.5's reading of `gvram_w` is confirmed by experiment, not just by code.
|
||
|
||
### 47.2 The packed layout renders correctly, on both
|
||
`tools/bench/show_frame256_packed.lua` (MAME) and `tools/bench/gvpack` (px68k):
|
||
|
||
- page 0, opaque bottom, unscrolled -> screen columns 0..127
|
||
- page 1, transparent top, X-scroll **384** -> columns 128..255, because column
|
||
`c` fetches `page1[(c+384) & 511]`
|
||
- words 0..127 of each row carry both halves: `(right << 8) | left`
|
||
- words 128..511 zeroed **once** -- page 1's storage at 384..511 sits under
|
||
columns 0..127 and must read 0 so the opaque page 0 shows through. Static
|
||
setup, not per-frame payload.
|
||
- the blob is built `--pack-transparent`: index 0 is the transparency key, so it
|
||
never appears in the picture and black lives at 255.
|
||
|
||
| | result | palette ceiling |
|
||
|---|---|---:|
|
||
| MAME, `verify_frame256.py` | **256x192 pixel-exact, letterbox true black** | 40.83 dB |
|
||
| px68k, `verify_gvpack.py` | **256x192 index-exact, letterbox on 255** | 40.83 dB |
|
||
|
||
**Per-frame payload: 128 words/row x 192 rows = 24,576 words = 49,152 bytes for
|
||
49,152 pixels. 1.0 B/pixel, against 2.0.**
|
||
|
||
`tools/bench/gvpack` links px68k's real `x68k/gvram.c` -- the address decode, the
|
||
bit-11 write path, the page-byte selection, the scroll wrap and the index-0
|
||
transparency test are px68k's own code, the way `tools/bench/c68k` links its CPU
|
||
core. The only mirrored part is `windraw.c`'s twelve-line page-ordering
|
||
dispatch, which is SDL-bound; it is quoted verbatim in `pick_order()`.
|
||
|
||
**Four negative controls, because a test that cannot fail proves nothing:**
|
||
|
||
| control | expected | got |
|
||
|---|---|---|
|
||
| ordinary unpacked 2.0 B/px path | pass | pass |
|
||
| packed, **bit 11 OFF** | fail | fail: right half is 24,576 px of index 0 |
|
||
| packed, **page-1 scroll removed** | fail | fail: 49,073 px differ |
|
||
| packed, priority `vc1=0x00` | fail (per MAME) | **PASS on px68k** -- see 47.3 |
|
||
|
||
### 47.3 Disagreement 1: the priority register, when the fields are equal
|
||
Video controller R1 (`0xE82500`) decides which page composites on top.
|
||
|
||
| `vc1` | MAME | px68k |
|
||
|---|---|---|
|
||
| `0x0000` | page 1 **not shown** -- right half black, 24,576 px differ | page 0 on top **transparently** -- renders correctly |
|
||
| `0x0002` | page 1 on top, **correct** | page 1 on top, **correct** |
|
||
|
||
They agree at `0x0002` and that is what the layout uses, so the result stands on
|
||
a setting both model identically. But the packed layout's correctness rests on a
|
||
register the two emulators do not model the same way, and neither is authority.
|
||
|
||
### 47.4 Disagreement 2: does buffer mode BLANK the display? -- and this one decides it
|
||
`tools/bench/probe_bit11_blank.lua` is the known-good 256-colour test with **one
|
||
line added**, setting bit 11:
|
||
|
||
- **MAME: the screen goes fully black.** Max channel 0, zero non-black pixels.
|
||
Buffer mode is a write *window*, not a display mode -- which is why the working
|
||
test clears bit 11 after painting.
|
||
- **px68k: it does not blank.** `Grp_DrawLine8` never reads `CRTC_Regs[0x28]`,
|
||
and `gvpack --keepbuffer` renders the frame correctly with the bit still set.
|
||
|
||
**This is the question the packed path lives or dies on.** If MAME is right, the
|
||
graphics layer is blanked for the whole time the CPU or DMAC is painting, and a
|
||
12fps full-frame player would show black for whatever fraction of each frame the
|
||
paint takes. If px68k is right, the packing is free. Both are plausible readings
|
||
of "G-VRAM set to buffer", and px68k's own comment -- `65536モードのVRAMアクセス
|
||
（Nemesis用）` -- says the bit exists for a game that blasted graphics through it,
|
||
which is at least consistent with the write-window reading.
|
||
|
||
**It is a hardware fact, and it is now the cheapest high-value one outstanding**
|
||
-- cheaper than 43.2, because a single real board plus the two-line probe above
|
||
settles it, and because the answer moves more numbers.
|
||
|
||
### 47.5 What it would be worth, DERIVED
|
||
Arithmetic on measured constants, not measurements. **Every line below is void if
|
||
47.4 goes MAME's way**, and the mode-decision cost model (44.3, 43.1) would need
|
||
re-deriving from scratch either way.
|
||
|
||
| | default masked | **packed** |
|
||
|---|---:|---:|
|
||
| bytes/frame into GVRAM | 98,304 | **49,152** |
|
||
| word writes/frame | 49,152 | **24,576** |
|
||
| wire at 12fps | 1,152 KB/s | **576 KB/s** |
|
||
| 22.8 min of game | 1.61 GB | **0.81 GB** |
|
||
| DMAC device->GVRAM, c=5 | 59.0% of budget | **29.5%** |
|
||
| CPU-painted literal, c=5 | 19.14 clocks/px (**misses** by 13%) | **9.57 (fits, 44% spare)** |
|
||
| CPU-painted literal, c=9 | 27.14 (**misses** by 60%) | **13.57 (fits)** |
|
||
|
||
The CPU-painted row is the one that overturns something. 44.7 concluded "a
|
||
CPU-painted full-frame literal does not fit at any physically reachable price,
|
||
and it misses at the floor itself." Packed, one `movem` word carries two pixels,
|
||
so the per-pixel cost becomes `(9.143 + 2c)/2 = 4.571 + c` -- and it fits at c=5
|
||
**and** at c=9. That conclusion is withdrawn, conditionally on 47.4.
|
||
|
||
### 47.6 What this does NOT establish
|
||
1. **Correctness was tested, cost was not.** Both harnesses write GVRAM
|
||
directly -- MAME through Lua's address space, `gvpack` by calling
|
||
`GVRAM_Write`. Neither runs 68000 instructions, so no clock in 47.5 is
|
||
measured. The `movem` shape of the packed writes is an assumption.
|
||
2. **The DMAC has not been near this.** 44.7's device->GVRAM transfer in buffer
|
||
mode is untested, and 43.2's single- vs dual-address question sits underneath
|
||
every `c` in 47.5.
|
||
3. **One frame, not a stream.** A static frame was painted and snapshotted.
|
||
Nothing here exercises per-frame toggling of bit 11, and if MAME is right
|
||
about blanking, that toggling is the whole problem.
|
||
4. **The codec was not considered.** VQ blocks and v7 spans address chunky
|
||
pixels; under the packed layout a word spans two columns 128 apart. Whether
|
||
the existing codec survives that is untouched -- 47 is about a literal frame.
|
||
5. **Two of 46.6's three assumptions held, one was wrong.** R20 bit 11 does
|
||
coexist with 256-colour display (after clearing it), and the off-edge storage
|
||
behaves. The priority guess was wrong: 46.6 assumed page 0 on top; it is
|
||
page 1.
|
||
|
||
## 48. The blanking disagreement is not symmetric, and the paper trail favours MAME (session 17)
|
||
|
||
47.4 filed the blanking question as two emulators disagreeing, and called both
|
||
readings of "G-VRAM set to buffer" equally plausible. **They are not equally
|
||
plausible, and the two implementations are not making the same kind of
|
||
statement.** Nothing here is a board, so 47.4 is not closed -- but the prior
|
||
moves, and it moves against the packed layout.
|
||
|
||
### 48.1 px68k is silent, not dissenting
|
||
`grep -a` matters here: `x68k/gvram.c` carries EUC-JP comments, so a plain
|
||
`grep` treats it as binary and reports **no matches at all** for any pattern.
|
||
Read with `-a`, R20's high byte appears in exactly one file:
|
||
|
||
$ for f in x68k/*.c x11/*.c; do n=$(grep -ac "CRTC_Regs\[0x28\]" $f); ...
|
||
x68k/gvram.c: 6
|
||
|
||
and all six are the address decode -- lines 150/153/158 inside `GVRAM_Read`,
|
||
211/221/225 inside `GVRAM_Write`, where `CRTC_Regs[0x28]&8` is R20 bit 11 and
|
||
line 211 carries the comment `65536モードのVRAMアクセス（Nemesis用）`.
|
||
|
||
**No px68k display code reads the bit anywhere.** `Grp_DrawLine8` and
|
||
`x11/windraw.c` never consult it. So px68k does not model buffer mode as
|
||
non-blanking; it does not model the display side of buffer mode **at all**.
|
||
Its "no" is an omission.
|
||
|
||
MAME's "yes" is the opposite -- a deliberate, commented claim, made twice:
|
||
|
||
x68k_v.cpp:407 if(m_crtc->gfx_layer_buffer()) // if graphic layers are set
|
||
return false; // to buffer, they aren't visible
|
||
x68k_v.cpp:766 if((m_video.gfx_pri == priority) && !m_crtc->gfx_layer_buffer() && ...
|
||
|
||
and the parallel bit 12 is modelled the same way for the text layer
|
||
(`x68k_v.cpp:756`, `!m_crtc->text_layer_buffer()`). Someone implemented a
|
||
semantic on purpose. **An assertion and a silence are not a tie.**
|
||
|
||
### 48.2 The register table names the bit "for display / for buffer"
|
||
Sharp's own R20 bit map, transcribed on Data Crystal's X68k IOMAP:
|
||
|
||
| bit | name | %0 | %1 |
|
||
|---|---|---|---|
|
||
| 12 | T-MEM | `表示用` | `バッファ用` |
|
||
| **11** | **G-MEM** | **`表示用`** | **`バッファ用`(bit 10〜8 は無効)**<br>`(G-VRAM が 65536 色表示時と同じ構造になる)` |
|
||
| 10 | SIZE | 512x512 | 1024x1024 |
|
||
| 9-8 | COL | %00 16 / %01 256 / %11 65536 | |
|
||
|
||
Two things in that one row:
|
||
|
||
1. **The bit is named "for display" against "for buffer"** -- not "16-bit write"
|
||
or "unmasked access". The naming is MAME's reading.
|
||
2. **`bit 10〜8 は無効` -- the colour-mode field goes invalid.** COL is what the
|
||
display side decodes a plane structure from. A layer whose colour mode is
|
||
undefined has nothing to render with, which is a *mechanism* for blanking
|
||
rather than a restatement of it.
|
||
|
||
The counter-reading survives, and it is the parenthetical: the doc says the
|
||
structure becomes the 65536-colour one, and does **not** say the screen goes
|
||
dark. That is why this is a shifted prior and not a result.
|
||
|
||
### 48.3 The cost of the MAME branch, DERIVED, and it is not a partial blank
|
||
47.4 left "measure what fraction of a frame the paint takes" as the fallback.
|
||
Bit 11 only has to be set across the GVRAM writes, so the blank interval is the
|
||
paint, not the frame. The measured blit is **53.6% of the frame budget**
|
||
(session 9) unpacked; packed halves the word count, so the floor is ~27% and the
|
||
ceiling ~54% depending on how much of the blit is stores.
|
||
|
||
**Either end is fatal for this content.** The graphics layer would be visible
|
||
between roughly half and three-quarters of each frame at 12fps, with the black
|
||
interval locked to frame rate -- a 12 Hz strobe over the whole picture, not a
|
||
tear or a partial band. And the packed layout has **no page left to flip to**:
|
||
both 256-colour pages carry picture, which is the entire point of it. There is
|
||
no version of the MAME branch where the packing is merely expensive.
|
||
|
||
### 48.4 px68k cannot testify about SCSI at all (bears on 43.2)
|
||
Checked while looking for a second opinion on single- vs dual-address. **px68k
|
||
does not emulate the MB89352.** `x68k/scsi.c` is 81 lines: it synthesises a
|
||
64-byte fake CZ-6BS1 boot ROM at `$EA0020` (the `Human68k` signature, the IOCS
|
||
`$F5` vector, `move.b d1,$e9f800`) and traps the IOCS call on the host. Its own
|
||
header says so -- `SCSI IOCS を特殊処理で対応。SPCはエミュレートしない`.
|
||
|
||
So the "second emulator" method that settled 46/47 **is not available for
|
||
43.2**, and never was. MAME models the dual-address row and has no DACK path;
|
||
px68k models no SPC. Item 2 needs the CZ-6BS1's `scsiexrom.bin` (8 KB, CRC
|
||
`7be488de`, absent here) disassembled for its DMAC DCR programming -- or a
|
||
schematic. It does not need another emulator.
|
||
|
||
**And the field to read is now pinned to the primary source.** MC68450
|
||
datasheet (Motorola ADI1216, Jul-89 printing), section 3.6.1 -- the DCR field
|
||
order is `XRM | DTYP | DPS | PCL`, MSB to LSB, and DTYP is the answer in one
|
||
two-bit field:
|
||
|
||
| DTYP | datasheet text, verbatim | addressing |
|
||
|---|---|---|
|
||
| `00` | `M68000 Compatible, Explicitly Addressed` | **dual** |
|
||
| `01` | `M6800 Compatible, Explicitly Addressed` | **dual** |
|
||
| `10` | `Device with ACK, Implicitly Addressed` | **single** |
|
||
| `11` | `Device with ACK and RDY, Implicitly Addressed` | **single** |
|
||
|
||
and 3.6.1.2 states the equivalence outright rather than leaving it to be
|
||
inferred:
|
||
|
||
> For M68000 type devices, the DMAC will use a **dual address transfer
|
||
> protocol** by running M68000 type bus cycles to transfer data to or from the
|
||
> device registers and a second bus cycle to complete the operand transfer from
|
||
> or to memory. [...] In the remaining two device protocols, the DMAC asserts
|
||
> the acknowledge signal to **implicitly address the device during a single
|
||
> address transfer** while it is explicitly addressing a memory location.
|
||
|
||
So 43.2's 5.0-vs-9.0 clocks/byte is decided by two bits in one byte the
|
||
CZ-6BS1's boot ROM writes at init. (Bit *positions* within DCR are the
|
||
conventional 7-6/5-4/3/2-0 split; the datasheet's own bit-number row did not
|
||
survive OCR, so treat the positions as unconfirmed and the field order and
|
||
encodings as quoted.)
|
||
|
||
**MAME confirms the negative half of this from the other side:**
|
||
`hd63450.cpp` contains **no DTYP handling at all** -- the only device-shape
|
||
field it decodes is `ocr & 0x30` (operand size). It cannot express an
|
||
implicitly-addressed device, which is why 43.2 was right that MAME settles
|
||
nothing here.
|
||
|
||
### 48.5 One thing the hunt did confirm independently
|
||
px68k's `kaiseki.txt` (the author's own analysis notes, 2014/2/14), on GVRAM
|
||
structure, unprompted and predating any of this:
|
||
|
||
256色の場合は、Page0の(0,0), Page1の(0,0), Page0の(1,0), Page1の(1,0)... と交互に並ぶ。
|
||
|
||
Page 0 and page 1 bytes alternate within the word at the *same* coordinate.
|
||
That is 46.1's page-not-pixel finding from a third source, and it is the
|
||
premise the packed layout is built on. **The layout's premise is solid; only
|
||
its visibility is in doubt.**
|
||
|
||
## 49. The streaming path, built and run: contiguity is the constraint, and the shipping rate does not fit the pipe (session 18)
|
||
|
||
STATUS item 3 has been open since session 7, and 45.4.1 stated the gap in as
|
||
many words: **the gate proves the decoder is pixel-exact over a whole window and
|
||
says nothing about how the bytes get to it.** `tools/bench/decode.lua` preloads
|
||
5,261,814 B of container into emulated RAM and lets `a0` walk through all of it.
|
||
The shipping player never holds a window at once.
|
||
|
||
That rig is now built. `src/player/stream.s`, `tools/bench/prep_stream.py` and
|
||
`tools/bench/stream.lua` decode the gate container **out of a bounded ring**,
|
||
with a modelled SCSI pipe as the producer, and the container living in a HOST
|
||
file rather than in emulated RAM.
|
||
|
||
### 49.1 The result: pixel-exact from a ring one twentieth the size of the stream
|
||
|
||
| ring | machine | wraps | mean hole | result |
|
||
|---|---|---:|---:|---|
|
||
| 256 KB | **stock 2 MB** | 18 | 14.7 KB (5.7%) | **120/120, final frame pixel-exact** |
|
||
| 128 KB | stock 2 MB | 37 | 10.6 KB (8.3%) | 120/120, pixel-exact |
|
||
| 96 KB | stock 2 MB | 54 | 15.6 KB (16.2%) | 120/120, pixel-exact |
|
||
| 80 KB | stock 2 MB | 60 | 8.9 KB (11.2%) | 120/120, pixel-exact |
|
||
| 64 KB | stock 2 MB | 107 | 23.4 KB (36.6%) | 120/120, pixel-exact |
|
||
| 48 KB | stock 2 MB | 111 | 8.9 KB (18.5%) | 120/120, pixel-exact |
|
||
|
||
Verified by `tools/bench/verify_decode.py`, the same comparison that gates
|
||
`decode.s`: the last frame against `tools/encoder/dlx.py`'s reconstruction, and
|
||
because a SKIP block is a claim about the previous frame still being in GVRAM,
|
||
the last frame is only right if all 120 were.
|
||
|
||
**A side effect worth naming: the rig's RAM ceiling is gone.** FINDINGS 44.6.4
|
||
audited 37/120 frames because the container did not fit 2 MB, and 45 raised
|
||
`RIG_RAM` to 6 MB to fix it. The streaming rig holds ~256 KB of stream and
|
||
reads the rest from the host, so **a stock 2 MB machine runs the whole window** —
|
||
and the machine it runs on is now the machine the player targets, rather than a
|
||
rig-shaped one.
|
||
|
||
### 49.2 The constraint is CONTIGUITY, and a byte-counting simulation cannot see it
|
||
`09_buffer_sim.py` asked whether cumulative supply ever falls behind cumulative
|
||
demand, in bytes, and FINDINGS 21 answered "zero required prefill". That test is
|
||
necessary and not sufficient. The block loop and the span chain read the stream
|
||
with a monotonically increasing `a0` and **no bounds check anywhere** —
|
||
`move.l (a0)+,d0`, `lea MODEB(a0),a0`, eleven unrolled `movem.l (a0)+`, a
|
||
`move.b (a0)+` per block index. None of it survives an address that wraps
|
||
mid-record.
|
||
|
||
So the ring needs the **whole next record resident AND contiguous**, not merely
|
||
enough bytes by the deadline. Those are different conditions and only the
|
||
second one is a byte count. `tools/analysis/19_ring_stream.py` models the ring's
|
||
addresses rather than its occupancy.
|
||
|
||
### 49.3 `aligned` beats `split`, and it is not close
|
||
Two policies can give the reader a contiguous record. Both columns below are for
|
||
**`s14_d5_all1500`**, the shipping candidate, in a 256 KB ring — the costs scale
|
||
with the container's record sizes, so they must be quoted per container:
|
||
|
||
| policy | mechanism | RAM cost | CPU cost |
|
||
|---|---|---:|---:|
|
||
| **`aligned`** | producer refuses to start a record it cannot finish; leaves a hole, restarts at 0 | 23.4 KB mean hole = **9.1%** of the ring | **0** |
|
||
| `split` | records wrap; ring's first MAXREC bytes mirrored into a shadow past its end | 0 | **46,394 clk/frame = 5.57% of the frame budget, forever** |
|
||
|
||
For the lighter gate container `rc_fr_singe_scsi_span` the same trade is
|
||
**5.7% of the ring against 3.64% of the frame budget** — same direction, same
|
||
verdict, smaller numbers.
|
||
|
||
The decoder already spends **77.0% of the budget on the mean frame and 91.1% at
|
||
p90** (FINDINGS 45). `split` puts p90 at **96.7%**. RAM is the thing this machine
|
||
has 2 MB of; clocks are the thing it has none of.
|
||
|
||
`aligned` also needs a per-record **index** on the fill side — and a branching
|
||
laserdisc game needs one anyway to seek to a branch point. **The policy that
|
||
costs no clocks reuses a structure the player cannot avoid.**
|
||
|
||
The third option — teach the block loop to wrap its own reads — is the expensive
|
||
one, and not because of the branch. A bounds test lands *inside* the instruction
|
||
sequences FINDINGS 30.4 and 40 fitted their constants to, so it does not cost a
|
||
compare: it costs **every span and per-block figure in the tree being
|
||
re-measured**.
|
||
|
||
### 49.4 Two independent implementations agree exactly
|
||
The Python simulation predicts the ring's behaviour from record sizes alone; the
|
||
Lua producer drives a real 68000 through MAME. They agree to the digit, and at
|
||
two ring sizes rather than one -- so it is not a coincidence of a single tiling:
|
||
|
||
| ring | | wraps | mean hole | usable ring |
|
||
|---|---|---:|---:|---:|
|
||
| 256 KB | `19_ring_stream.py` (from record sizes) | 18 | 14.7 KB | 94.3% |
|
||
| 256 KB | `stream.lua` (driving the 68000) | **18** | **14.7 KB** | **94.3%** |
|
||
| 128 KB | `19_ring_stream.py` | 37 | 10.6 KB | 91.7% |
|
||
| 128 KB | `stream.lua` | **37** | **10.6 KB** | **91.7%** |
|
||
|
||
They share no code. This is the same class of check as FINDINGS 46's — agreement
|
||
on a *mechanism* rather than on an output.
|
||
|
||
### 49.5 The shipping candidate does not fit the 488 KB/s pipe, and nothing was checking
|
||
`s14_d5_all1500` is the session-14 candidate: **29.19 dB at 496.7 KB/s** (43.6).
|
||
The pipe constant this tree has simulated against since session 2 is
|
||
**488 KB/s**. Those two numbers have never been put side by side.
|
||
|
||
wire demand 496.7 KB/s - pipe 488.0 KB/s = 8.7 KB/s OVER, on the MEAN
|
||
|
||
This is **not a burst a ring absorbs**. The deficit grows 744 B per frame for as
|
||
long as the scene runs — 87 KB over the 120-frame window, **523 KB per minute of
|
||
play**. No ring size fixes a sustained overrun, and quoting the window's 118.4 KB
|
||
"required prefill" for it would be the most flattering possible way to state it.
|
||
|
||
**Why it was never caught.** 42.1 established that 488 was never a bus figure and
|
||
that the binding resource is *clocks*, not KB/s — so the rate controller was
|
||
built to bind on `decode + c*bytes` and has **no pipe term at all**. That was a
|
||
defensible decision. What was not decided is that FINDINGS 21's buffer sizing,
|
||
and its "zero required prefill", would keep standing on a constant the design had
|
||
stopped enforcing. Item 4 has been open since session 7 for exactly this reason.
|
||
|
||
**The useful output is a requirement on the medium, not a verdict.** Since 488 is
|
||
unmeasured folklore and the intent is to measure a BlueSCSI directly, the tool
|
||
reports the threshold to measure *against*:
|
||
|
||
| container | wire | **zero-prefill pipe** | vs 488 |
|
||
|---|---:|---:|---:|
|
||
| `s14_d5_all1500` (the candidate) | 496.7 KB/s | **513.2 KB/s** | +5% |
|
||
| `rc_fr_singe_scsi_span` (the gate) | 446.1 KB/s | **451.4 KB/s** | -8% |
|
||
|
||
**513.2 KB/s is now a hardware acceptance test**, and it is 33% of SCSI-1's
|
||
asynchronous rating and 10% of its synchronous one (42.1). It is very likely met;
|
||
it has never been shown to be met.
|
||
|
||
### 49.6 The rig measures ARRIVAL, and the first version of it measured the wrong thing
|
||
`stream.s` has no frame clock — it asks for the next record the instant it
|
||
finishes the last. So it outruns any finite pipe, and its spin counter reports
|
||
**91 of 120 frames "stalled" at a pipe the same run shows is fast enough**. A
|
||
shipping player waits for vblank and spends that same time idle. Reporting that
|
||
count as an underrun would have been a false finding of exactly the shape this
|
||
project keeps filing.
|
||
|
||
The rig now records the emulated time at which each record becomes **resident**
|
||
and checks it against a 12 fps deadline, which is a question about arrival alone
|
||
and does not need the decoder paced:
|
||
|
||
| pipe | decoder waited | **records late** | worst | **required prefill** |
|
||
|---|---:|---:|---:|---:|
|
||
| unlimited | 0/120 | **0/120** | — | 0 |
|
||
| 520 KB/s | 85/120 | **0/120** | — | 0 |
|
||
| 488 KB/s | 91/120 | **1/120** | 4.9 ms (0.06 fr) | 2.3 KB |
|
||
| 460 KB/s | 97/120 | **2/120** | 9.7 ms (0.12 fr) | 4.4 KB |
|
||
|
||
The 2.3 KB at 488 against the simulation's 0.0 KB is a modelling difference, not
|
||
a disagreement: the Lua producer delivers in **whole records**, the simulation in
|
||
**512-byte sectors**, and whole-record granularity is the more conservative of
|
||
the two by up to one record's worth of latency.
|
||
|
||
### 49.7 What this does NOT establish
|
||
1. **The DMAC's clock debit is not modelled.** The pipe here is a constant byte
|
||
rate on the emulated clock. It is honest about *arrival order and residency*,
|
||
which is what a ring manages, and says nothing about the clocks the DMAC
|
||
steals from the 68000 while it delivers (43.2, and `W` is still undecided).
|
||
**A zero-late result from this rig means "the bytes were in time", NOT "the
|
||
frame fits".**
|
||
2. **Buffer stall-tolerance at a branch point is still untested.** Because the
|
||
decoder free-runs, the ring never backs up, so the ring-size sweep in 49.1
|
||
tests *wrap correctness* at each size and **not** the buffering that a seek
|
||
needs. At 64 KB the mean hole is 36.6% and effective capacity is one record —
|
||
it is single-buffered, and it passes anyway. Do not read 48 KB as a viable
|
||
player buffer.
|
||
3. **The producer is not an MB89352.** No sector-level command overhead, no
|
||
arbitration, no seek. `19_ring_stream.py` quantises to 512 B; the rig does not.
|
||
4. **The hole is a tiling effect and is not monotonic in ring size** — 96 KB
|
||
wastes more than 80 KB does. It depends on how record sizes tile the ring, so
|
||
a ring should be sized against the *measured* hole for the container it will
|
||
carry, not against a fraction.
|
||
5. **An unexplained cross-emulator gap, left open rather than explained away.**
|
||
The ring pass decodes the gate container at **561,532 cycles/frame under
|
||
MAME**. FINDINGS 45's figure for the same container is **641,444 cycles/frame,
|
||
and it is px68k's C68K core**, not MAME's -- so the two are not comparable and
|
||
the 12% between them is not evidence of anything yet. Getting MAME's own
|
||
full-pass number for this container needs a `decode.lua` timing run that did
|
||
not complete inside 25 minutes on this host; it was killed rather than left to
|
||
race another MAME job, which is how session 18 lost its first attempt at it.
|
||
**No figure in this section rests on the comparison.** Whoever picks it up:
|
||
run `decode.lua` on `rc_fr_singe_scsi_span.dlx` with nothing else touching
|
||
`tmp/`, and compare its "full 120-frame pass" against 561,532. If they agree,
|
||
the C68K/MAME delta is the thing to explain; if they do not, `stream.s`'s
|
||
frame loop is.
|
||
6. **`decode.s` is unchanged, and provably.** The block loop and span chain were
|
||
moved to `src/player/frame.i` and the constants to `geom.i` so both
|
||
front-ends assemble from literally the same bytes; `decode.s` still assembles
|
||
to the same **1,296 bytes** it did before the split, and `prep_dlx.py` still
|
||
emits a byte-identical blob after the loader maths moved to
|
||
`tools/bench/dlxload.py`. Both are asserted in `check.sh`.
|
||
|
||
## 50. The pipe constant is retired (session 18, USER DECISION)
|
||
|
||
**USER DECISION, after 49.5: remove the delivery-rate constant from the repo as
|
||
a live number.** 42.1 established in session 13 that it was never a bus figure —
|
||
a user-supplied "4 Mbps" with no recorded provenance, 10% of SCSI-1's
|
||
asynchronous rating — and 29.5 item 3 had carried "confirm the 4 Mbps figure" as
|
||
an open item since session 7. It was never confirmed. It was also never removed,
|
||
and 49.5 is what that cost: the shipping candidate ran 8.7 KB/s over it for five
|
||
sessions with nothing in the tree comparing the two.
|
||
|
||
### 50.1 What was actually wrong was the DEFAULT, not the number
|
||
The number being unmeasured was known and written down. What kept it load-bearing
|
||
was that **six tools defaulted to it** — `12_span_tradeoff.py`,
|
||
`14_dmac_chain.py`, `16_span_roundtrip.py`, `17_span_delivered.py`,
|
||
`19_ring_stream.py` and `tools/bench/stream.lua`. A default is how a figure gets
|
||
into a table without appearing in the sentence that reports the table. Every
|
||
span figure in FINDINGS 30–41 was scored against it; none of them had to say so.
|
||
|
||
**All six now take a REQUIRED argument with no fallback.** A tool that cannot run
|
||
without being told the rate cannot quietly assume one, and a result that had to
|
||
name its rate to exist is a result whose provenance travels with it.
|
||
|
||
A small confirmation of the same point fell out of doing it: writing the new
|
||
help text broke `--help` on four of the tools -- the text said "10%" and argparse
|
||
read it as a format spec -- and **nobody would have noticed**, because none of
|
||
these tools had ever been run with `--help` in this tree. The removal was the
|
||
first thing that made them state their own arguments out loud.
|
||
|
||
That is the transferable part: **an unmeasured constant is not made safe by
|
||
documenting that it is unmeasured.** 42.1 documented it perfectly and it went on
|
||
silently underwriting tables for five more sessions. It is made safe by deleting
|
||
the default.
|
||
|
||
### 50.2 What survives, and why it is not the same thing
|
||
`GATE_SPAN_KBPS` in `tools/bench/check.sh`. The gate container was **encoded**
|
||
with it, and every per-block and span constant in FINDINGS 41/43/45/49 is fitted
|
||
to that container. Changing it is a re-encode plus a re-measurement of all of
|
||
them, not an edit.
|
||
|
||
It is a **container recipe, not a delivery claim**, and check.sh says so at the
|
||
point of use. The distinction is the whole reason it can stay: nothing reads a
|
||
medium's throughput out of it.
|
||
|
||
### 50.3 What replaces it: a requirement, not a constant
|
||
`19_ring_stream.py` reports the **zero-prefill pipe** — the rate a medium must
|
||
clear for a given container to need no prefill at all:
|
||
|
||
| container | wire demand | **zero-prefill pipe** |
|
||
|---|---:|---:|
|
||
| `s14_d5_all1500` (the candidate) | 496.7 KB/s | **513.2 KB/s** |
|
||
| `rc_fr_singe_scsi_span` (the gate) | 446.1 KB/s | **451.4 KB/s** |
|
||
|
||
Its rate sweep is now anchored to each container's own wire demand (0.90x to
|
||
2.00x) rather than to a fixed list of absolute rates, so it privileges no
|
||
constant and stays meaningful for any container.
|
||
|
||
**This is a number to MEASURE A MEDIUM AGAINST, not one to design on**, which is
|
||
the difference that mattered. The BlueSCSI has never been benchmarked on this
|
||
machine and the intent has always been to measure it directly; it now has a
|
||
threshold to be measured against.
|
||
|
||
### 50.4 What this does NOT do
|
||
1. **It does not measure anything.** The delivery rate remains unknown. The
|
||
tree is now honest about that rather than carrying a placeholder — which is
|
||
a smaller claim than it sounds, and the right one.
|
||
2. **It does not re-derive the tables that were scored against the old figure.**
|
||
FINDINGS 30–41's span figures stand as measured *at that rate*; what changed
|
||
is that nothing new can be scored there without saying so. Anything that
|
||
needs a delivery rate to mean something is now waiting on a measurement, and
|
||
should be.
|
||
3. **It does not touch the gate container**, so the green light and every
|
||
constant fitted to it are unmoved. `./tools/bench/check.sh` is **ALL GREEN**
|
||
after the removal.
|
||
|
||
## 51. Pacing the decoder: seek slack is accumulated, not owned (session 19)
|
||
|
||
STATUS item 4, open since session 7 in one form or another and sharpened by
|
||
49.7.2. `tools/bench/stream.lua` decodes 120 frames out of a bounded ring and
|
||
the pass is pixel-exact at every ring size down to 48 KB — and that result could
|
||
not be read as a statement about buffering, because `src/player/stream.s` has no
|
||
frame clock. It asks for record *i* the instant it finishes record *i-1*, so it
|
||
outruns any finite pipe, the ring never backs up, and the producer's overlap
|
||
test is never the thing that refuses a placement. **A ring-size sweep under
|
||
those conditions tests wrap correctness at each size and nothing else.**
|
||
|
||
The rig now has a frame clock. `PACE`/`PACEON` (`$18034`/`$18038`) are written
|
||
by the producer — vblank or an MFP timer in the player — and frame *i* may not
|
||
start before tick *i*. `PACEON=0` leaves the loop free-running and is what the
|
||
green light's wrap gate still uses, so 49's figures are unmoved.
|
||
|
||
### 51.1 The ceiling: what a ring is worth once it is full
|
||
Every cell below is a full 120-frame decode on MAME's 68000, pixel-verified
|
||
against `tools/encoder/dlx.py`, on the gate container (`rc_fr_singe_scsi_span`,
|
||
446.1 KB/s wire, 36.5 KB mean record). "Ceiling" is the largest number of whole
|
||
records resident and unconsumed, i.e. **the frames the decoder could still draw
|
||
with delivery stopped dead**. Pipe `0` is unlimited, which isolates the ring's
|
||
own capacity from the rate.
|
||
|
||
| ring | unlimited | 460 KB/s | 488 KB/s | 520 KB/s | 600 KB/s |
|
||
|---:|---:|---:|---:|---:|---:|
|
||
| 64 KB | 2 | 2 | 2 | 2 | 2 |
|
||
| 96 KB | 3 | 3 | 3 | 3 | 3 |
|
||
| 128 KB | 5 | 4 | 4 | 4 | 4 |
|
||
| 192 KB | 6 | 4 | 5 | 5 | 5 |
|
||
| 256 KB | 8 | 4 | 7 | 7 | 7 |
|
||
| 384 KB | 11 | 4 | 10 | 11 | 11 |
|
||
| 512 KB | 15 | 4 | 11 | 14 | 14 |
|
||
|
||
**64 KB carries two frames and 96 KB carries three.** 49.7.2 warned that 48 KB
|
||
was single-buffered and passing anyway; this is the number, and it says the
|
||
small end of that sweep was measuring nothing about delivery.
|
||
|
||
### 51.2 Tolerance is `ceiling - 1`, and it was falsified rather than asserted
|
||
`DLX_CUT_AT`/`DLX_CUT_FR` stop the pipe dead at a chosen tick, as a seek does.
|
||
At 256 KB and 520 KB/s, with **7 records resident**:
|
||
|
||
| cut | underruns |
|
||
|---:|---:|
|
||
| 2 frame times | 0 |
|
||
| 6 | **0** |
|
||
| 7 | **1** |
|
||
| 8 | 1 |
|
||
| 10 | 1 |
|
||
|
||
**Seven resident records buy six frame times, not seven** — 500 ms, not 583.
|
||
The last one is spent covering the pipe's restart: a 36.5 KB record takes ~0.9
|
||
frame times to place at 512 KB/s of video, so the record due immediately after
|
||
the cut is still arriving when its slot opens. A design that reads the resident
|
||
count as its stall budget is over by one record, every time.
|
||
|
||
Every cut run stayed **pixel-exact**, including the ones that underran. That is
|
||
the expected shape and worth stating: under-delivery makes a frame LATE, not
|
||
wrong — `waitrec` spins and the decode is byte-identical when it resumes.
|
||
A rig that reported corruption here would be reporting its own bug.
|
||
|
||
### 51.3 The result that bears on a branching game
|
||
Slack is not a property the buffer has. It is **accumulated out of the surplus
|
||
between the pipe and the wire demand**, and a seek spends all of it.
|
||
|
||
| ring | pipe | ceiling | play needed to reach it, from empty |
|
||
|---:|---:|---:|---:|
|
||
| 256 KB | 488 KB/s | 7 fr (583 ms) | **4.83 s** |
|
||
| 256 KB | 520 KB/s | 7 fr | 2.83 s |
|
||
| 256 KB | 600 KB/s | 7 fr | 1.67 s |
|
||
| 512 KB | 520 KB/s | 14 fr (1.17 s) | **8.42 s** |
|
||
| 512 KB | 600 KB/s | 14 fr | 8.42 s |
|
||
|
||
**A bigger ring raises the ceiling and lengthens the climb to it.** The fill
|
||
rate is `pipe - wire`, which is set by the encoder and the medium; the ring only
|
||
sets where the climb stops. So the question a branch point asks is not "is the
|
||
buffer big enough" but "has there been enough play since the last branch point
|
||
to refill it" — and at 488 KB/s with a 256 KB ring the answer is **4.83 seconds
|
||
of play**. Two branch decisions closer together than that and the second one has
|
||
no buffer to spend, at any ring size. Dragon's Lair's decision points are
|
||
seconds apart.
|
||
|
||
This is the first statement in this tree about back-to-back branches, and it is
|
||
a consequence of the delivery model, not of the decoder.
|
||
|
||
### 51.4 Which resource is binding, said out loud
|
||
The producer now counts its refusals separately. A **rate** refusal (no credit
|
||
for a whole record) means a bigger ring buys nothing; a **ring** refusal (the
|
||
decoder still owns those bytes) means a faster pipe buys nothing. From the
|
||
decoder's side the two are identical — "no new record" — and they have opposite
|
||
fixes.
|
||
|
||
**At 460 KB/s every ring from 192 KB to 512 KB reports RATE-BOUND with a ceiling
|
||
of 4 and never fills inside 120 frames.** 460 is 13.9 KB/s over the container's
|
||
446.1 wire demand; that surplus fills 146 KB in the whole 10-second window. Any
|
||
ring larger than that is dead RAM in this scene. The zero-prefill pipe for this
|
||
container is 451.4 (49.5) — **clearing the arrival deadline and being able to
|
||
absorb a seek are different requirements, and the gap between them is large.**
|
||
|
||
### 51.5 An independent implementation agrees, within one record
|
||
`tools/analysis/20_seek_slack.py` is the same model written from record sizes in
|
||
Python, sharing no code with the Lua producer — the 49.4 pattern. Over all 35
|
||
cells of the grid:
|
||
|
||
- **35/35 of the rig's ceilings fall inside the sim's bracket**, and 33/35 sit
|
||
at the top of it.
|
||
|
||
The bracket is one record wide and is reported as a range rather than a number,
|
||
deliberately. At these rates the pipe delivers almost exactly one record per
|
||
frame slot, so "records resident at slot *i*" differs by one depending on
|
||
whether the sample is taken before or after that slot's delivery. Sampled after,
|
||
the sim matched the rig's ceiling in 33/35 cells; sampled before, it was exactly
|
||
one low in 33/35. Picking the sampling point that matched, and then reporting
|
||
the match as a cross-check, would have been fitting the model to the
|
||
measurement. Both are returned and the caller prints the range.
|
||
|
||
### 51.6 Two rig defects the pacing exposed
|
||
Both were in the producer, both were invisible while the decoder free-ran, and
|
||
both would have made the first paced result wrong in a plausible direction.
|
||
|
||
1. **The `RD_PTR` cross-check was really a test of how often `reap()` ran.** It
|
||
asserted, for every record it retired, that the decoder's released-to pointer
|
||
equalled that record's end. `RD_PTR` is a single pointer and names the end of
|
||
record `tail-1`; retiring several records in one pass — normal the moment the
|
||
decoder is paced and the pipe stops — made it fire a false MISMATCH on the
|
||
earlier ones. It now asserts on `tail-1` only, which is the invariant that
|
||
actually holds.
|
||
2. **`reap()` was skipped for the duration of a cut.** The cut returned early
|
||
from `produce()`, so the ring looked full for the whole seek and the producer
|
||
could not restart against space the decoder had long since released. A seek
|
||
stops delivery; it does not stop the decoder. Only credit and placement stop
|
||
now.
|
||
|
||
### 51.7 What this does NOT establish
|
||
1. **The pipe is still a model and the medium is still unmeasured.** Every rate
|
||
in 51.1 is a chosen input, not a measurement — FINDINGS 50 stands, and
|
||
`pace_run.sh` requires the rate for the same reason. The columns are a
|
||
sensitivity table; none of them is a claim about a BlueSCSI.
|
||
2. **The DMAC's clock debit is still not modelled** (49.7.1). A paced decoder
|
||
makes the ring's behaviour honest and does nothing about the clocks the DMAC
|
||
steals while it fills. "Survives a 500 ms cut" means the bytes were there,
|
||
not that the frames fit.
|
||
3. **One container, one scene.** The ceilings are in whole records, so they move
|
||
with record size; a scene with heavier frames has a lower ceiling in the same
|
||
ring. Nothing here is a universal per-KB figure and the tools take the
|
||
container as an argument for that reason.
|
||
4. **The pace gate costs the rig a `tst.l`/`beq.s` per frame** — `stream.s` is
|
||
1,418 bytes against 1,396. It is outside `src/player/frame.i`, so every
|
||
per-block and span constant in FINDINGS 24/30/40/41 is untouched, and
|
||
`decode.bin` is still 1,296 bytes at the same MD5 (asserted in `check.sh`).
|
||
5. **Seek TIME itself is not modelled.** The cut is a chosen duration. What a
|
||
real seek costs on the target medium is part of the same unmeasured
|
||
question as the rate, and it is the other half of what item 1 buys.
|
||
|
||
---
|
||
|
||
## 52. The DMAC configuration was never a mystery: it is in the IPL ROM (session 20)
|
||
|
||
ROADMAP called the ADPCM stream "the largest unpriced risk left in the project"
|
||
and asked for one cheap thing first — **put the audio DMA on the bus and see
|
||
what it does to the 86.7%.** Doing that needs a clocks-per-byte figure for the
|
||
audio channel, and the tree did not have one: `11_cpu_budget.py` charged audio
|
||
bytes `--dma-clocks-per-byte`, the *disk's* rate, defaulting to 5 and described
|
||
in its own help text as "single-address, bus held, no drive wait". That is a
|
||
description of the SCSI channel, and it is the favourable end of ROADMAP B3, an
|
||
open question worth 242 KB/s. **Audio was being charged the disk's guess.**
|
||
|
||
It did not have to be a guess for either of them. The X68000's IPL ROM programs
|
||
all four HD63450 channels itself, and **the ROM is on this machine** — MAME
|
||
boots the player rig with `-bios ipl10`. `tools/analysis/21_iplrom_dmac.py`
|
||
reads the configuration out of the image and decodes the MC68450 register
|
||
fields. It is a gate, not a report: every value is (address, expected bytes,
|
||
meaning), eight sites, and a mismatch or an unrecognised ROM revision exits
|
||
non-zero rather than decoding some other code. It is in `check.sh`, needs no
|
||
emulator, and runs in milliseconds.
|
||
|
||
**NAME THE LAYER.** This is not a measurement of a running machine and not real
|
||
hardware. It is the shipping ROM image (IPL 1.0, md5 `7fd4caab…`, 131,072 B),
|
||
read statically — evidence about what Sharp's engineers configured this board
|
||
to do, from the vendor, for these exact devices. Field layouts are SOURCED from
|
||
MC68450, Motorola, Jul 1989, the document `buscost.py` already cites.
|
||
|
||
### 52.1 What the ROM programs
|
||
|
||
Boot, `$FF0CCA`–`$FF0C58`: a ten-pair table at `$FF0D8E` initialises channels 0
|
||
and 1, then two inline runs do channels 2 and 3.
|
||
|
||
| ch | device | DAR | DCR | decoded |
|
||
|---|---|---|---|---|
|
||
| 0 | FDC | `$E94003` | `$80` | dual address, 8-bit port, cycle steal **without hold** |
|
||
| 1 | **SASI** | `$E96001` | `$80` | dual address, 8-bit port, cycle steal without hold |
|
||
| 2 | IOCS `_DMAMOVE` | per call | `$08` | dual address, **16-bit** port, burst |
|
||
| 3 | **ADPCM** | `$E92003` | `$80` | dual address, 8-bit port, cycle steal without hold |
|
||
|
||
Per transfer, `$FF9A82` (ADPCM play) and `$FF9944` (SASI): **OCR = `$32`** for
|
||
memory→device, `$B2` for device→memory. Both decode to `SIZE = 11` (byte),
|
||
`CHAIN = 00` (none) and — the load-bearing field — **`REQG = 10`, external
|
||
request: one operand per device request.**
|
||
|
||
### 52.2 Audio is dual-address and cannot hold the bus. 16..19 clocks a byte.
|
||
`DTYP = 00` is *explicitly addressed*, so every ADPCM byte is a memory read
|
||
followed by a device write: 4 + 5 = **9 clocks** (Fig 4-25 sheet 4, note 2 —
|
||
already in `buscost.py` for the span work). `XRM = 10` is cycle steal without
|
||
hold and `REQG = 10` is external request, so **the DMAC arbitrates for the bus
|
||
once per byte and hands it straight back.** There is no burst to amortise the
|
||
front-end (5..8 clocks, §4.5.2.1) and back-end (2, §4.5.2.2) over. The audio
|
||
byte costs **16 clocks best case, 19 worst** — not 5.
|
||
|
||
### 52.3 The byte rate, derived rather than restated
|
||
`ratectl.AUDIO_KBPS = 7.8` had no derivation next to it, and ROADMAP flagged it
|
||
as exactly the kind of figure that cost the project a 2x error in FINDINGS 43.
|
||
It survives, with a correction of units: 15.6 kHz is the MSM6258V's 8 MHz clock
|
||
÷512 = **15,625 samples/s**, 4 bits each, two to a byte = **7,812.5 B/s**. The
|
||
7.8 is that in *decimal* kB; `11_cpu_budget.py` was multiplying it by 1024, so
|
||
it read 2.4% high. Harmless, and now derived from the sample rate in
|
||
`buscost.ADPCM_BYTES_PER_S` instead of typed in.
|
||
|
||
**The request count does not halve.** 7,812.5 B/s is 7,812.5 DMA requests/s,
|
||
because the port is 8 bits and the operand is a byte. That is the FINDINGS 43
|
||
trap in the other stream, and it does not spring: nobody had denominated audio
|
||
per word.
|
||
|
||
### 52.4 So: what the audio does to the bus. Almost nothing.
|
||
651.0 B/frame at 12 fps × 16..19 clocks = **10,417..12,370 clocks of 833,333 —
|
||
1.25% to 1.48% of the frame.** The decoder's measured mean is 68.5% of the frame
|
||
period on the gate container, so audio takes about **4% of what the decoder
|
||
leaves**, and about 1.7% of it on the worst frame (which is already at 110.8%
|
||
and misses with or without audio).
|
||
|
||
**P6's bus risk does not materialise.** The concern was sound and the answer is
|
||
that a second DMA consumer at 7.8 kB/s is not what a bus at 88% occupancy is
|
||
short of. The debit was 3.2x..3.8x understated, and it is still small.
|
||
|
||
### 52.5 THE ONE THAT MOVES SOMETHING: the disk channel is programmed the same
|
||
ch1, SASI, `DCR = $80`, `OCR = $B2` — **dual address, 8-bit port, cycle steal
|
||
without hold, external request.** Byte by byte, full arbitration each time. ch0
|
||
(FDC) too. Sharp programs every explicitly-addressed 8-bit device on this board
|
||
identically, and by 52.2's arithmetic that is **16..19 clocks per delivered
|
||
byte**.
|
||
|
||
**A CORRECTION TO THIS SECTION AS FIRST WRITTEN, made in the same session.**
|
||
It cited 42.4's sensitivity table — `W <= 6` fits 0/120 frames, `W = 8` misses
|
||
47/120 — as though those figures were in clocks per BYTE. **They are per WORD,
|
||
and FINDINGS 43 voided them**: 43 is the section that caught `W` being charged
|
||
per word to a byte-wide port, and it says in terms that 42.3's 0/120 was never
|
||
physically reachable. Quoting them here would have re-imported the exact 2x unit
|
||
error 43 exists to have corrected, one section after using the same trap as a
|
||
warning. They are struck, and nothing below depends on them.
|
||
|
||
In the corrected unit the ladder is a per-byte cost of the DMAC's own
|
||
configuration, and it is the ladder `buscost.py` already carries:
|
||
|
||
| configuration | clk/byte |
|
||
|---|---:|
|
||
| single address, bus held | 5 |
|
||
| dual address, bus held | 9 |
|
||
| single address, arbitrated per byte | 12 |
|
||
| **dual address, arbitrated per byte — what the ROM programs** | **16..19** |
|
||
|
||
`15_bus_occupancy.py` sweeps it on the gate container (37,403 B/frame, measured
|
||
mean decode 570,958 clocks):
|
||
|
||
| W clk/B | video clk/frame | % of frame | CPU + audio + video |
|
||
|---:|---:|---:|---:|
|
||
| 5 | 187,017 | 22.4% | 92.2% |
|
||
| 8 | 299,228 | 35.9% | 105.7% |
|
||
| 12 | 448,842 | 53.9% | 123.6% |
|
||
| **16** | 598,455 | 71.8% | **141.6%** |
|
||
| **19** | 710,666 | 85.3% | **155.0%** |
|
||
|
||
**This table is the statement, and it is not corroborated by 42.4.** The
|
||
resemblance between the `W = 8` row here and 42.4's 47/120 is a coincidence of
|
||
two different units on two different containers at two different rates, and
|
||
calling it a cross-check — as this section did when first written — was
|
||
manufacturing agreement out of a unit error.
|
||
|
||
**This does not close B3.** `scsiexrom.bin` drives an MB89352, not the SASI
|
||
port, and a different ROM may configure it differently. What changed is the
|
||
prior and the framing: 42.6 says the handshake "is ours to choose, not to
|
||
receive", and that is still true — but **nothing in this tree has shown a
|
||
cheaper configuration is reachable for an explicitly-addressed 8-bit port, and
|
||
the vendor's own answer is the expensive one.** Holding the bus, which is what
|
||
separates 9 from 16..19, is a *requirement on the player's DMAC programming*
|
||
rather than a range the hardware hands us. It is now the largest open number in
|
||
the project, ahead of the rate.
|
||
|
||
### 52.6 Audio outranks the disk at the arbiter
|
||
CPR: FDC 0, **ADPCM 1**, SASI 2, `_DMAMOVE` 3 — lower is higher priority. With
|
||
the ROM's arrangement, when both channels want the bus in the same slot ADPCM is
|
||
served first. An audio byte is never the thing that waits; **a video byte is.**
|
||
Relevant to 51's underrun analysis, which models delivery as a smooth rate.
|
||
|
||
### 52.7 What this does NOT establish
|
||
1. **Static read of a ROM image, not a running machine.** No emulator executed
|
||
this code for the purpose; the claim is about bytes in the shipping image.
|
||
Real hardware would confirm the registers, not the timings.
|
||
2. **The timings are datasheet, not measured.** 9 clocks for the transfer and
|
||
5..8 / 2 for the arbitration come from MC68450 Fig 4-25 and §4.5.2. They are
|
||
the same source `buscost.py` already rests on for the span work, and they
|
||
have never been checked against a board.
|
||
3. **IPL 1.0 is a pre-SCSI machine.** ch1 is SASI. B3 stands.
|
||
4. **Our player is not obliged to copy the ROM.** It programs these registers
|
||
itself. 52.5 is a prior and a warning, not a measured ceiling — and the
|
||
experiment that would settle it is P4, not another reading.
|
||
5. **Nothing here is an audio implementation.** P6's other risks — extraction,
|
||
encode, container interleave, the second stream's effect on `wire` and hence
|
||
on 51.3's refill climb — are untouched. Only the bus question is answered.
|
||
6. **The frame-period accounting assumes the DMAC does not overlap the CPU**
|
||
(`buscost.DMA_OVERLAPS = False`), which is FINDINGS 35's premise: no cache,
|
||
a two-word prefetch queue. A stolen bus cycle is a stopped 68000.
|
||
|
||
---
|
||
|
||
## 53. The loader moves onto the 68000, and a scene change finally has a price (session 21)
|
||
|
||
ROADMAP P1 and P2. Since session 1 the two load-time transforms have been done
|
||
**host-side**, in `tools/bench/dlxload.py`, with the rigs pushing the *result*
|
||
into emulated RAM: the codebooks expanded to word-per-pixel form (CB1 to 32 B an
|
||
entry, CB4 to 8 B) and the 24-bit palette packed to `GGGGGRRRRRBBBBBI` with the
|
||
shared LSB chosen per entry. That was the right call while the inner loop was
|
||
what was being measured — charging a once-per-scene cost to the per-frame path
|
||
would have flattered or damned it for no reason — but **a player has no host.**
|
||
|
||
`src/player/load.i` does both on the 68000, out of the raw container header as
|
||
it comes off the disc. `tools/bench/loadgate.s` is its front-end, the way
|
||
`decode.s` is `frame.i`'s.
|
||
|
||
**NAME THE LAYER.** Everything here is **emulated**: MAME 0.277 `x68000`,
|
||
`-bios ipl10`, stock 10 MHz / 2 MB, cross-checked on px68k's C68K core. Nothing
|
||
has run on real hardware.
|
||
|
||
### 53.1 It reproduces dlxload.py exactly, on both cores
|
||
|
||
`dlxload.py` stays the reference — what changed is **where the transforms run,
|
||
not what they produce** — so the gate is byte-for-byte, not "close enough":
|
||
|
||
- **CB1 8,192 B, CB4 2,048 B, palette 512 B: identical.** A wrong codebook byte
|
||
is a wrong colour in every block that uses that codeword, in every frame of
|
||
the scene, and a wrong shared LSB is a *slightly* wrong colour, which is
|
||
exactly the sort of defect that gets attributed to the codec.
|
||
- **The palette half is read back out of the palette registers at `$E82000`**,
|
||
not out of a RAM shadow, so "the words reached the hardware" is part of what
|
||
passes.
|
||
- **The darkest-entry index agrees too** (255 on the gate container). It comes
|
||
out of an `argmin` whose tie-break has to match numpy's — first index at the
|
||
minimum wins — and it is what the letterbox is filled with.
|
||
- **Both CPU cores produced the same 10,752 bytes**, and the same as the host.
|
||
|
||
`tools/bench/load_run.sh` runs it and `check.sh` gates it.
|
||
|
||
### 53.2 What it costs, measured on two cores
|
||
|
||
| stage | MAME clocks | C68K clocks | Δ | data bus (C68K) |
|
||
|---|---|---|---|---|
|
||
| scratch tables (boot only) | 52,919 | 61,622 | +16.4% | 6.5% |
|
||
| **P1** codebook expansion | 92,609 | 95,304 | +2.9% | **43.2%** |
|
||
| **P2** palette entries | 97,019 | 97,348 | +0.4% | 14.3% |
|
||
| BOOT: all three | 246,957 | 253,614 | +2.7% | 23.2% |
|
||
| **SCENE CHANGE: P1 + P2** | **189,627** | 192,322 | +1.4% | 28.6% |
|
||
|
||
**A scene change costs 18.96 ms of 68000 time — 22.8% of one 12 fps frame.**
|
||
Boot costs 24.70 ms. Bus occupancy is data accesses only (C68K does not see
|
||
prefetch), so it is a lower bound; the expansion is the bus-heaviest thing here
|
||
because it is a pure copy, and it still runs alone.
|
||
|
||
**The stages are exactly additive on the exact core.** `P1 + P2 - SCENE = 330`
|
||
clocks, and `tables + P1 + P2 - 2x330 = 253,614 = BOOT`, to the clock — 330 is
|
||
the front-end's own per-pass overhead. On MAME the same identity closes to 1.8%,
|
||
which is one tick of its host clock over the 0.99 s run. Two instruments,
|
||
two granularities, one arithmetic. (That tick was written here as 1/55.46 s and
|
||
is **1/56.69 s** — MAME's raster, not the hardware's, 54.5. 17.64 ms over 990 ms
|
||
is 1.78%, so the sentence was right and the label was wrong.)
|
||
|
||
### 53.3 The scratch tables are scene-independent, so they are not in the scene path
|
||
|
||
Packing a palette entry needs the squared error of both choices of the shared
|
||
LSB, per channel. That is three table reads and a sign test here, out of three
|
||
tables — the 6-bit-to-8-bit rendering the CRTC performs, its square, and the
|
||
per-channel error difference — and **not one of them describes the scene.** They
|
||
describe the machine. `pal_tables` is therefore a separate entry point from
|
||
`pal_pack`, built once at boot: **5.29 ms saved on every scene change**, 22% of
|
||
what a naive port of `dlxload.py` would have charged per scene.
|
||
|
||
### 53.4 The two cores disagree only where the multiplies are, and C68K is wrong in kind
|
||
|
||
The table build is the only code in this tree that multiplies, and it is the
|
||
only stage where the two cores disagree by more than 3%. **px68k's C68K charges
|
||
a flat 50 clocks for `MULU` and `MULS`** regardless of the operand
|
||
(`c68kmacro.h:1869/1883`, `RET(50 + EA_CLOCKS_...)`); the 68000 charges
|
||
**38 + 2n**, n counting bits in the source. For the 576 multiplies this code
|
||
executes, the real total is 24,192 clocks against C68K's 28,800: **the flat rate
|
||
explains 4,608 of the 8,703 clock gap, and 4,095 clocks — 7.7% of the stage —
|
||
are NOT explained.** Recorded as open rather than rounded away; the residual is
|
||
somewhere else in the two cycle tables and this stage is not worth the hunt.
|
||
|
||
**The consequence is general and belongs in the reader's head:** where a future
|
||
measurement contains multiplies, C68K over-charges them, and it is the second
|
||
opinion this tree leans on for every cycle figure. Nothing else in
|
||
`src/player/` multiplies — index scaling is `lsl.w #5`/`#3` by construction —
|
||
so no figure in FINDINGS 24-52 is affected.
|
||
|
||
### 53.5 Where the cost actually lands: the scene change, priced
|
||
|
||
`tools/analysis/22_scene_load.py`, cycle counts parsed out of the rig's own log
|
||
rather than pasted in as constants. Three costs in three units, and **the third
|
||
is the one that compounds**:
|
||
|
||
- **BYTES.** The header region is **5,920 B** (palette 768 + CB1 4,096 + CB4
|
||
1,024 + 32) and it must arrive before frame 0 can be decoded. It is not part
|
||
of any frame record, so **no rate table in this tree has ever counted it.**
|
||
- **CLOCKS.** 189,627, from 53.2.
|
||
- **ACCUMULATED SLACK.** Those bytes are bytes the pipe did not spend filling
|
||
the ring, so they cost play-time at the surplus rate `pipe - wire` — the
|
||
currency FINDINGS 51.3 established a branch point spends.
|
||
|
||
| pipe KB/s | header ms | + load ms | total | frame slots | surplus KB/s | slack cost |
|
||
|---|---|---|---|---|---|---|
|
||
| 451.4 | 12.81 | 18.96 | 31.77 | 0.38 | 5.3 | **1.099 s** |
|
||
| 488 | 11.85 | 18.96 | 30.81 | 0.37 | 41.9 | 0.138 s |
|
||
| 513.2 | 11.27 | 18.96 | 30.23 | 0.36 | 67.1 | 0.086 s |
|
||
| 600 | 9.64 | 18.96 | 28.60 | 0.34 | 153.9 | 0.038 s |
|
||
|
||
(Rates are explicit arguments with no default, FINDINGS 50. `wire` is 446.1
|
||
KB/s on the gate container, audio included.)
|
||
|
||
**Two readings, and the second is the finding.** First: the whole fixed cost of
|
||
a scene change is **about a third of one frame slot** — it is not what makes a
|
||
branch point expensive, the seek and the refill climb are. Second: **the slack
|
||
cost is hypersensitive to the rate**, because it is divided by a surplus that
|
||
goes to zero. At 488 KB/s the header lengthens the climb by 138 ms; at 451.4
|
||
KB/s — the *arrival-deadline* rate for this same container, 49.5 — the same
|
||
5,920 bytes cost **1.1 seconds of play**. The header is cheap only where the
|
||
pipe already has room, which is the same place everything else in this project
|
||
is cheap.
|
||
|
||
### 53.6 The alternative that was not taken
|
||
|
||
The encoder could ship the codebooks pre-expanded and P1 would not exist. That
|
||
trades **9.26 ms of 68000 time for 5,120 more bytes in every scene header** —
|
||
10.5 ms of pipe at 488 KB/s, and 5,120 bytes that lengthen the climb again by
|
||
the arithmetic above. **Derived, not measured**, from the two figures either
|
||
side of it. It is close to a wash in milliseconds and it is not a wash in
|
||
*kind*: the CPU is idle during a seek and the pipe is the resource this project
|
||
is short of. The transform stays on the 68000.
|
||
|
||
### 53.7 What is still open in P2
|
||
|
||
**The encoder still does not reserve a black entry** (23.4), so the letterbox
|
||
gets the palette's closest thing to black — index 255 here — rather than a true
|
||
black with `I = 0`. That half of P2 is encoder-side, it changes the container,
|
||
and it moves every constant fitted to the gate container, so it is a re-encode
|
||
plus a re-measurement rather than an edit. `load.i` is ready for it: it reads
|
||
whatever the palette section holds and reports the darkest index either way.
|
||
|
||
---
|
||
|
||
## 54. The frame clock moves onto the 68000, and the 12 fps frame turns out never to have existed (session 22)
|
||
|
||
ROADMAP P3, and the item was phrased "needs MFP timer or VBL" — which quietly
|
||
assumes one of those can do it. Neither can, and finding out why produced a
|
||
better clock than either and a correction to an instrument the whole tree reads.
|
||
|
||
`src/player/clock.i` is the clock; `src/player/clockgate.s` and
|
||
`tools/bench/clock.lua` measure it; `tools/analysis/23_frame_clock.py`
|
||
enumerates the space it was chosen from and prices its cadence. Two new stages
|
||
in `tools/bench/check.sh` gate it.
|
||
|
||
**Layer: MAME 0.277's emulated X68000, not real hardware.** The MFP, the CRTC
|
||
and the interrupt sequence are all the emulator's. Where the emulator and the
|
||
registers disagree — and they do, 54.5 — the code is built on the registers.
|
||
|
||
### 54.1 No MFP timer can tick at 12 Hz, and none can tick as slowly as a frame
|
||
|
||
The MC68901's timer clock on this board is 16 MHz / 4 = 4 MHz
|
||
(`sharp/x68k.cpp:1027-1028`), its prescaler ladder is `{4, 10, 16, 50, 64, 100,
|
||
200}` (`machine/mc68901.cpp:173`) and its data register is 8 bits. So:
|
||
|
||
* the **slowest** tick a single timer can produce is 4e6/(200·256) =
|
||
**78.125 Hz**, which is 6.5× faster than a 12 fps frame — a software divider
|
||
is required whatever the source;
|
||
* **4e6/12 = 333,333.33 is not an integer**, so no prescale/data pair divides to
|
||
12 Hz at all. `23_frame_clock.py` walks all 7 × 256 of them and finds zero.
|
||
|
||
A timer clock is therefore not "the simple option". It is a divider *plus* an
|
||
interrupt rate 3.6× higher than the raster's, at an arbitrary phase against the
|
||
scan.
|
||
|
||
### 54.2 The raster cannot do it by whole division either — and the fix is exact
|
||
|
||
V-DISP is on MFP GPIP4 (`x68k.cpp:1139`), and the same pin is Timer A's event
|
||
input (`mc68901.cpp:167`, `GPIO_TIMER = {GPIP_4, GPIP_3}`); its interrupt is
|
||
channel 6, `IR_GPIP_4 = $40` (`mc68901.cpp:76`). The raster is 31,500/568 =
|
||
**55.4577 Hz** exactly. Every whole divide misses:
|
||
|
||
| Timer A event count | fps | error |
|
||
|---:|---:|---:|
|
||
| 4 | 13.8644 | +15.54% |
|
||
| 5 | 11.0915 | −7.57% |
|
||
|
||
12 fps needs **4.6215 refreshes per frame**. So the divider keeps a remainder:
|
||
|
||
```
|
||
each V-DISP: acc += fps*VTOTAL ; 12*568 = 6816
|
||
if acc >= 31500: acc -= 31500 ; PACE += 1
|
||
```
|
||
|
||
Long-run rate is `fps·VTOTAL/VTOTAL` = **12.000000 fps exactly**, with a
|
||
remainder that never accumulates. Both constants are **read out of the CRTC at
|
||
init** — `R04+1` for VTOTAL, `R20` bit 4 checked for the 31.5 kHz mode — so the
|
||
clock is derived from the registers that generate the raster it counts, and the
|
||
two cannot drift apart. The accumulator peaks at 38,316, so it is 16-bit
|
||
arithmetic on a 68000; `clk_init` refuses rather than overflow (the ceiling is
|
||
fps < 59.9 at this VTOTAL).
|
||
|
||
**Measured over 3,000 refreshes: 3,000 interrupts, 649 ticks, where 649.1429
|
||
were exactly due — an error of −0.14 ticks, i.e. the remainder still held.** The
|
||
gate is stated in ticks and not in ppm on purpose: a remainder-keeping divider
|
||
is off by at most one tick over *any* window, so quoting ppm would let a longer
|
||
window advertise a tighter clock for nothing.
|
||
|
||
### 54.3 It costs 181.35 clocks per V-DISP — 838 per frame, 0.10% of the budget
|
||
|
||
The host cannot time this: MAME's Lua sees the machine once per screen frame,
|
||
17.64 ms, and the interrupt costs microseconds. So the **68000 times it
|
||
itself**. `clockgate.s` runs a one-instruction loop for a window of 3,000
|
||
refreshes with the clock off and again with it armed:
|
||
|
||
```
|
||
clock off: iters0·L = clocks in the window -> L
|
||
clock on: iters1·L + ints·H = clocks in the window -> H
|
||
```
|
||
|
||
`L` is **calibrated, not looked up** — the point is to price the clock on the
|
||
machine rather than against `buscost.py`, which is the table being checked.
|
||
|
||
| | |
|
||
|---|---:|
|
||
| loop iteration, calibrated over 13,926,121 of them | **38.000002 clocks** |
|
||
| per V-DISP interrupt, over 3,000 | **181.35 clocks** |
|
||
| per 12 fps frame (4.6215 interrupts) | **838 clocks = 0.1006%** |
|
||
|
||
`L` landing on a whole number to seven digits is the check that licenses the
|
||
subtraction, and it is also an independent confirmation of `buscost.py`'s model:
|
||
`addq.l #1,(xxx).L` is 3 instruction words + 4 data accesses = 7 bus cycles = 28
|
||
clocks, plus 10 for the `bra.s`.
|
||
|
||
**The 181.35 decomposes exactly.** By the same model the handler body is 130
|
||
clocks on a V-DISP that emits no tick and 164 on one that does; over the measured
|
||
649/3,000 mix that is 137.355, leaving **43.99 clocks for the interrupt exception
|
||
sequence** — the textbook 44, measured here rather than recalled.
|
||
|
||
For comparison, the cheapest exact MFP-timer clock would interrupt 16.7 times a
|
||
frame instead of 4.6: **3.6× the cost, for a tick with no fixed relationship to
|
||
the scan.**
|
||
|
||
### 54.4 THE ONE THAT MOVES SOMETHING: there is no 83.33 ms frame, and there never was
|
||
|
||
12 fps on a 55.4577 Hz raster is 4.6215 refreshes, so a frame is shown for **4
|
||
refreshes (72.13 ms) or 5 (90.16 ms)** — 37.9% of them short. The 833,333-clock
|
||
budget every figure in this project is priced against is the **mean** slot, and
|
||
the short slot is **13.4% under it**.
|
||
|
||
With the per-frame decode costs (`tmp/c68k_frames.csv`, 120 frames of the gate
|
||
container) run through the actual divider and the actual pace gate:
|
||
|
||
| tick source | short slot | frames over it | no idle left |
|
||
|---|---:|---:|---:|
|
||
| nominal 1/fps model (no raster has it) | 833,333 | 1/120 | **1/120** |
|
||
| the host tick, as `stream.lua` really emits it | 705,590 | 10/120 | **4/120** |
|
||
| the 68000's clock, hardware raster | 721,270 | 10/120 | **4/120** |
|
||
| the 68000's clock, MAME's raster | 705,590 | 10/120 | **4/120** |
|
||
|
||
**The cadence was already there and nothing had named it.** `stream.lua`'s tick
|
||
is `floor((t - t_rel) * fps)` — which *looks* uniform and is not, because Lua
|
||
only sees the machine at frame boundaries, so its ticks land on refreshes and its
|
||
gaps are the same two whole numbers. Every host-paced result in FINDINGS 49 and
|
||
51 already carried a 4/5 cadence. **P3 did not introduce it. It moved who
|
||
produces it onto the machine and made it visible.**
|
||
|
||
**A short slot is not a dropped frame.** The pace gate says only "not before
|
||
tick i", so a frame that overruns spends the next frame's idle and the clock
|
||
recovers itself; the cost is one frame presented a refresh late. What the table
|
||
counts is frames with no idle left, and the difference between the nominal row
|
||
and the raster rows — **1 against 4** — is the entire price of the cadence on
|
||
this container.
|
||
|
||
**The expensive frame is frame 0, at 923,146 clocks = 111% of the nominal
|
||
budget**: the first frame of a scene has nothing to SKIP against, so it is the
|
||
whole picture in one slot. Most of what follows it in those counts is that
|
||
transient draining. It also means the cost lands **at a scene change**, next to
|
||
FINDINGS 53.2's 18.96 ms of loader and the seek — not spread over the window.
|
||
|
||
`src/player/stream.s` now counts this itself (`LATEFR`/`LATEMAX`/`LATE1ST`), and
|
||
the rig's count matches the offline model **exactly**: 4/120, first at frame 1,
|
||
on both tick sources. The counter sits ahead of the wait loop and the
|
||
free-running path executes none of it, so FINDINGS 49's figures are untouched.
|
||
|
||
### 54.5 MAME's raster runs 2.22% fast, and the whole tree has been sampling it
|
||
|
||
`x68k_crtc.cpp refresh_mode()` computes the frame period as
|
||
`(scr.max_x * scr.max_y)` dots over the dot clock, with
|
||
`scr.max_x = m_htotal - 8` — one character cell short, and an **inclusive
|
||
rectangle bound used as a count**. In the 256-wide mode that is 360 where the
|
||
registers say 368, so MAME's refresh is fast by **368/360 = 1.02222**:
|
||
|
||
* registers: 31,500/568 = **55.4577 Hz**
|
||
* MAME, measured by `clock.lua` over 3,000 frames: **56.6901 Hz**
|
||
|
||
The two agree to six digits with `clock_69m()/6 / (360·568)`, so this is the
|
||
mechanism and not a coincidence. Consequences, and the third one is why it is
|
||
worth this much space:
|
||
|
||
1. **Every "1/55.46 s granularity" note in this tree was wrong** — it is
|
||
1/56.69 s, 17.64 ms. Corrected in `decode.lua`, `load.lua`, `span.lua`,
|
||
`blit.lua`, `loadgate.s` and `check.sh`, with the derivation put once in
|
||
`crtc_mode.lua`. No conclusion changes: 53.2's "one tick over the 0.99 s run"
|
||
is 1.78% at the corrected figure and was quoted as 1.8%.
|
||
2. **68000 cycle figures are untouched.** The CPU clock is 40 MHz/4 and has
|
||
nothing to do with the screen. Nothing in FINDINGS 24–53 moves.
|
||
3. **A raster-paced player runs 2.22% fast under MAME**, so the rig measures
|
||
12.267 fps where the hardware would give 12.000. `clock.lua` reports both and
|
||
de-skews, and `clock_run.sh` prices the interrupt against the **hardware**
|
||
refresh count — charging the player the emulator's extra interrupts would
|
||
overstate the cost by that same 2.2%.
|
||
|
||
**Do not "fix" 55.4577 to match the measurement.** It is the hardware's, derived
|
||
from the dot clocks, and it is what the divider is built on.
|
||
|
||
### 54.6 What had to be turned off, and why it is in the file
|
||
|
||
The rigs launch the 68000 at `SR=$2700` into a machine the IPL ROM has already
|
||
booted, so the MFP arrives with whatever IOCS enabled on it and vectors pointing
|
||
into IOCS. Lowering the mask without disarming it would vector into code we did
|
||
not put there. `clk_init` writes `IERA = IERB = 0` first — which on the MC68901
|
||
clears the matching pending bits with them (`mc68901.cpp REGISTER_IERA/B`,
|
||
`m_ipr &= m_ier`) — then arms GPIP4 alone, takes the falling edge (AER bit 4
|
||
clear: the **start of vertical blanking**, which is when a player would present),
|
||
and drops to `SR=$2500`. Levels 1–5 stay masked, so the DMAC (IRQ3) and the SCC
|
||
(IRQ5) cannot get in. `VR` is written with **S clear**, so an acknowledge clears
|
||
the pending bit by itself and the handler needs no end-of-interrupt write.
|
||
|
||
The handler saves only the **low word** of `d0`, because every operation in it is
|
||
a word operation — which is legal precisely because `clk_init` proved the
|
||
accumulator fits 16 bits. `decode.s` and `frame.i` were checked for stack tricks
|
||
before the mask was lowered: the only `a7` use in either is one `move.l a1,-(sp)`
|
||
pair, so an interrupt cannot corrupt decoder state. The 120-frame self-paced
|
||
decode being pixel-exact is the test of that, and it is gated.
|
||
|
||
---
|
||
|
||
## 55. The 68000 fills its own ring, and the player's request loop costs more than the medium does (session 23)
|
||
|
||
ROADMAP P5, the last M2 item this tree could build. FINDINGS 49 and 51 measured
|
||
a ring that a HOST filled: `tools/bench/stream.lua` held the record index, chose
|
||
where every record went, wrote the descriptor and advertised it. The 68000 only
|
||
consumed. That is the same shape session 21 found in the loader and session 22
|
||
in the frame clock — a policy living outside the machine that has to run inside
|
||
it — and it was the last one in the delivery path.
|
||
|
||
`src/player/ring.i` is that policy on the 68000: `aligned` placement, the
|
||
descriptor ring, a prefill, an accumulated-slack rule and a seek. The rig keeps
|
||
only what is genuinely not the CPU's — a transport that answers one request at a
|
||
time at a modelled rate, which is what an SPC and one DMAC channel are.
|
||
|
||
### 55.1 The container had to change: DLX4 carries a record index
|
||
|
||
`aligned` asks whether the NEXT record fits before the end of the ring, which is
|
||
a question about a record's length asked **before it is fetched**. Every reader
|
||
in this tree learned record boundaries by **walking** the frame stream — reading
|
||
each record's length word to find the next — and that is exactly what a player
|
||
streaming off a disc cannot do: the length word of record *i+1* is one of the
|
||
bytes it has not fetched. A branch point needs the same table a second time, to
|
||
seek to record *j* without reading what lies between.
|
||
|
||
DLX4 adds `nframes` u16 longword-counts to the scene header, ahead of the frame
|
||
stream. Costs, measured on the gate container:
|
||
|
||
| | DLX3 | DLX4 |
|
||
|---|---:|---:|
|
||
| scene header | 5,920 B | **6,164 B** (+240 index, +4 header) |
|
||
| frame payloads | — | **byte-identical**, all 120 |
|
||
|
||
The payloads being byte-identical is asserted rather than assumed: the same
|
||
encode was written both ways and compared record for record, so **no constant
|
||
fitted to the gate container moves**. `dlx.py` cross-checks the index against
|
||
its own walk of the stream and refuses a container where they disagree, and
|
||
`prep_stream.py` checks it again against the disk image it lays out. Lengths
|
||
rather than offsets: 2 bytes a frame instead of 4, and the disc offsets are a
|
||
running sum the player builds once at scene load (`ROFF`, 4 B/record of RAM).
|
||
|
||
In 53.5's currency the 244 bytes are small — 0.5 ms of pipe at 488 KB/s — but
|
||
they are on the same side of the ledger as the 5,920 that section priced, and
|
||
the scene header is now **6,164 B that must arrive before frame 0**.
|
||
|
||
### 55.2 It reproduces the host producer's tiling exactly
|
||
|
||
Third independent implementation of `aligned`, on the gate container in a 256 KB
|
||
ring:
|
||
|
||
| producer | wraps | mean hole | pixel-exact |
|
||
|---|---:|---:|:--:|
|
||
| `19_ring_stream.py` (Python, from record sizes) | 18 | 14.7 KB | — |
|
||
| `stream.lua` (host, driving the 68000) | 18 | 14.7 KB | yes |
|
||
| **`ring.i` (the 68000 itself)** | **18** | **14.7 KB** | **yes** |
|
||
|
||
The host now **audits** rather than produces: every placement the machine makes
|
||
is checked against the host's own index and its own list of records the decoder
|
||
has not consumed, and the run is refused on the first disagreement. That is what
|
||
makes the pixel-exact result a statement about `ring.i` and not about a new rig.
|
||
|
||
### 55.3 THE ONE THAT MOVES SOMETHING: the channel is idle whenever the player is not asking
|
||
|
||
A channel only moves bytes while it has a request, and only the CPU can give it
|
||
one. Between the completion of record *i* and the issue of record *i+1* the disc
|
||
**stands still**, and the length of that gap is a property of the player's loop,
|
||
not of the medium. No host-filled run could see it — the host producer placed
|
||
records whenever it liked — so **no rate table in this tree contains it**.
|
||
|
||
Measured on the machine, same container, same 256 KB ring, same 488 KB/s, the
|
||
only difference being how many requests the player may have outstanding:
|
||
|
||
| queue | channel idle | gaps | underruns | slack ceiling | mean slack | bound by |
|
||
|---:|---:|---:|---:|---:|---:|---|
|
||
| **1 request** | **669.0 ms, 6.8%** | 119 | **59/120** | 2 | 1.0 | rate |
|
||
| **2 requests** | **317.5 ms, 3.4%** | 9 | **0/120** | 5 | 3.5 | ring |
|
||
|
||
The surplus this container has over the wire at 488 KB/s is 8.7% of the pipe,
|
||
and a one-deep request loop spends 6.8% of it on nothing. **That is most of the
|
||
surplus 51.3's lookahead is accumulated out of**, which is why the same ring at
|
||
the same rate goes from rate-bound with a ceiling of 2 to ring-bound with a
|
||
ceiling of 5 on a change with no bytes in it at all.
|
||
|
||
A second queued slot costs the 68000 nothing per frame and is available on the
|
||
hardware: the HD63450 has four channels and the IPL programs all of them (52.1).
|
||
|
||
### 55.4 Prefill is the weaker lever, and now there is a number for it
|
||
|
||
Prefill in whole records, at 488 KB/s in a 256 KB ring, every cell pixel-exact:
|
||
|
||
| prefill | 1 | 2 | 3 | 4 | 6 |
|
||
|---|---:|---:|---:|---:|---:|
|
||
| underruns, 1-deep queue | 66 | 59 | 49 | 47 | 24 |
|
||
| underruns, 2-deep queue | 1 | **0** | **0** | **0** | **0** |
|
||
|
||
**A prefill buys a one-off cushion that a rate-bound pipe spends immediately; a
|
||
queued request buys the rate back every frame.** Six records of prefill is half
|
||
a second of black screen at the start of every scene and still leaves 24
|
||
underruns; a second slot leaves none for nothing. The policy `ring_prefill`
|
||
implements is therefore small — 2 records — and the reason it is not 1 is 51.2:
|
||
*n* resident records buy *n-1* frame times, so releasing at 1 starts a scene
|
||
with a stall budget of zero.
|
||
|
||
### 55.5 The slack rule is in the player now, and so is a seek
|
||
|
||
`ring_may_seek` answers 51.2's rule as arithmetic the player can run — "resident
|
||
minus one, against the frames this branch will cost" — instead of a line in a
|
||
rig's log. `ring_seek` takes a record number, waits the channel quiet (an
|
||
outstanding transfer is bytes already on their way to an address about to be
|
||
declared free), takes the disc address out of the index, and empties the ring.
|
||
|
||
Rehearsed as a second pass over the same scene: **240 records placed, the seek
|
||
at 12.87 s, the ring refilled from empty, 0 underruns after it, and the final
|
||
frame of the second pass pixel-exact**. The seek's cost shows up exactly where
|
||
55.3 says it would — as the worst channel gap of the run, **397.5 ms** — and
|
||
that is the disc idle, not a mechanical seek, which is still unmodelled (51.7.5).
|
||
|
||
### 55.6 An independent model, and where it does and does not agree
|
||
|
||
`tools/analysis/24_ring_owner.py` is the same producer written from record
|
||
sizes and per-frame decode costs, sharing no code with the rig — the 49.4/51.5
|
||
arrangement. It reads the DLX4 index the machine reads, and it reproduces 54.4's
|
||
4-or-5-refresh cadence rather than averaging it away.
|
||
|
||
| 488 KB/s | rig Q=1 | model Q=1 | rig Q=2 | model Q=2 |
|
||
|---|---:|---:|---:|---:|
|
||
| channel idle | 6.8% | 8.8% | 3.4% | 5.3% |
|
||
| slack ceiling | 2 | 3 | 5 | 6 |
|
||
| mean slack | 1.0 | 1.1 | 3.5 | 4.2 |
|
||
| underruns | 59/120 | 11/120 | 0/120 | 0/120 |
|
||
|
||
The model runs **one record ahead** of the rig, which is the same one-record
|
||
bracket 51.5 recorded and reported rather than tuned away. The underrun count at
|
||
Q=1 is the one number that disagrees badly, and it is a threshold statistic on a
|
||
quantity sitting at 1: with a mean slack of one record, whether each individual
|
||
frame's record lands before or after its tick is decided by details neither model
|
||
has. **The agreement that matters is the resource statement** — a one-deep queue
|
||
loses 7-9% of the pipe and two-thirds of the lookahead — and on that they agree.
|
||
|
||
The model's mean decode cost, 561,126 clk/frame, lands within 0.07% of the
|
||
561,532 the ring pass measured under MAME (49.7.5), from the encoder's own
|
||
constants.
|
||
|
||
### 55.7 Three bugs and one instrument correction, recorded because they were all silent
|
||
|
||
1. **The reader's wrap rule was not the writer's.** Stepping the read cursor
|
||
past record *i* lands on the end of record *i*, which is where record *i+1*
|
||
went only if it FITTED there. Using the wrong record's length left the cursor
|
||
inside the hole, and one more retirement pushed it past the end of the ring
|
||
and wrapped it to an address unrelated to any record. The live span computed
|
||
from that is *shorter* than the truth, so the producer places over a record
|
||
the decoder has not read. Symptom: a bitstream desync, not a fault.
|
||
2. **The free-space test decided the wrap before it knew the shape.** When the
|
||
LIVE span is the one that wraps, the ring base is not free and `aligned` may
|
||
not restart there. Deciding from `WCUR + len > SZ` alone overwrote live
|
||
records. Same symptom.
|
||
3. **The queue was gated on completion instead of retirement.** A slot stays in
|
||
use until the descriptor has been read out of it, which happens one poll after
|
||
the ack at the earliest. Gating on the ack let the CPU overwrite a slot whose
|
||
descriptor had not been published; `DESC` for that frame stayed zero and the
|
||
decoder decoded address zero. This one only exists at a queue depth above 1,
|
||
and it is why the two-deep result took three attempts to obtain.
|
||
4. **The rig was capturing a torn screen.** MAME renders a screen line by line
|
||
and the machine-frame notifier fires at the END of that frame, so a bitmap for
|
||
a frame in which GVRAM changed holds lines from before and after the change.
|
||
Snapshotting it captures a tear, which reads as a pixel-exactness failure in
|
||
the bottom blocks plus a broken double-scan pairing. It only bites when the
|
||
decoder finishes its last frame late in a screen frame, so it appeared for the
|
||
first time in a run with underruns. The rig now waits one whole frame before
|
||
the capture. **No previously reported result is affected** — every one of them
|
||
finished its last frame with idle to spare — but the check.sh gates would have
|
||
been flaky under any future run that did not.
|
||
|
||
An instrument note, not a bug: under the self-clock the host's "records late"
|
||
report grades arrivals against the tick times it OBSERVED, which are up to
|
||
17.64 ms late (54.5), so it understates lateness — 6 records late where the
|
||
68000 itself counted 59 frames that had to wait. The decoder's own stall counter
|
||
is the sharp instrument. Host-paced runs take their deadlines from a host model
|
||
and are exact, so 49.6's table is unaffected.
|
||
|
||
### 55.8 What this does NOT establish
|
||
|
||
1. **The transport is still a model.** It delivers at a chosen byte rate with an
|
||
exact clock; it is not an MB89352. No command overhead, no arbitration, no
|
||
sector granularity, no mechanical seek. `W` — the clocks the DMAC steals per
|
||
delivered byte — is still undecided and still unmeasurable here (P4, 52.5).
|
||
**A zero-underrun result means the bytes were in time, not that the frames
|
||
fit.**
|
||
2. **The rates are chosen inputs.** FINDINGS 50 stands: every column is a
|
||
sensitivity, not a claim about a BlueSCSI.
|
||
3. **One container, one scene, one ring size.** The ceilings are in whole
|
||
records and move with record size (51.7.3).
|
||
4. **The seek is a rewind, not a branch.** It exercises the machinery — quiet
|
||
the channel, empty the ring, address record *j* out of the index, refill —
|
||
against a container that has one scene in it. What the worst gap between two
|
||
real decision points is still needs the scene graph (ROADMAP G1).
|
||
5. **`decode.s` and `frame.i` are unchanged** and `decode.bin` is still 1,296 B
|
||
at the same MD5. `stream.s` grew to 2,814 B: the ring producer, plus a
|
||
two-instruction test at the top of the pace wait that routes a self-filled
|
||
run into a polling wait loop. The legacy wait loops are byte for byte the ones
|
||
FINDINGS 51 measured and a host-filled run executes none of the new code.
|
||
|
||
---
|
||
|
||
# Findings — session 24 (2026-08-24)
|
||
|
||
## 56. The scene graph is in, and the worst gap between two decision points is zero (session 24)
|
||
|
||
ROADMAP G1, and it was scheduled early *because it is a measurement input*.
|
||
FINDINGS 51.3 established that a ring's lookahead is accumulated out of
|
||
`pipe - wire` and that a seek spends all of it, so what a branch point costs is
|
||
set by the rate and by **the time since the last branch**. 55.5 rehearsed a seek
|
||
on the machine and then said out loud that it could not ask the question that
|
||
matters, because nothing in this tree knew where the branch points *are*.
|
||
|
||
Now it does, in two files with a hard line between them (USER DECISION,
|
||
session 24):
|
||
|
||
- **`tools/import/scenegraph.py` is the only file in this tree that knows
|
||
anything about somebody else's source.** Their file layout, table names,
|
||
timing formulas and magic constants are wired into it and nowhere else. It
|
||
writes **`DLXSCENE1`**, this project's own schema — a clip's length, its exits
|
||
and the earliest instant each of them can fire — into gitignored `tmp/`.
|
||
- **`tools/analysis/25_scene_graph.py` reads only DLXSCENE1** and could not name
|
||
an outside project if it wanted to.
|
||
|
||
Nothing is vendored and nothing outside-derived is committed. That line is worth
|
||
drawing before the game-logic layer exists rather than after: when one of those
|
||
projects moves, exactly one file in this tree breaks, and when a scene table is
|
||
eventually committed rather than regenerated, there is one place the attribution
|
||
already lives. Restructuring changed no number in this section — the split was
|
||
made after the measurement and the whole output was re-run byte for byte.
|
||
|
||
### 56.1 What was imported, and what gates it
|
||
|
||
**40 scenes, 516 sequences, 906 input windows**, plus `scene_manager.rows` — the
|
||
13x3 scene order the arcade walks. **282 of the 516 sequences (54.7%) are
|
||
entered by a seek**; the rest play on from wherever the disc already is, which
|
||
is the distinction the whole finding rests on.
|
||
|
||
The source table is Lua, and there is no Lua interpreter on this box, so the
|
||
importer contains a **deliberately small Lua-subset parser**: table constructors,
|
||
literals, bare identifiers, the file's four timing helpers and `+`/`-` between
|
||
them. Anything else is a parse error rather than a silent skip. Three gates
|
||
stand behind that choice:
|
||
|
||
1. the four timing helpers are **matched against the text that defines them** —
|
||
`frame / 23.976`, the `- 6297.0` ROM offset, `noseek` returning `-1`,
|
||
`(seconds * 1000) + ms`. If upstream changes a formula, the tool stops
|
||
rather than keeps evaluating the old one.
|
||
2. the parse must reach **516 sequences and 906 input windows** exactly. A
|
||
parser that quietly dropped a branch would produce a *smaller* graph and a
|
||
*longer* worst gap, i.e. it would fail in the flattering direction.
|
||
3. `check.sh` runs the import and then the analysis, and **skips rather than
|
||
fails** when there is no checkout, like every other outside-this-repo input
|
||
in the tree.
|
||
|
||
### 56.2 CORRECTION to FINDINGS 16: there is only one transcription
|
||
|
||
> 16 cleared two permissively licensed transcriptions and planned to **diff them
|
||
> against each other to catch transcription errors**. That plan does not work.
|
||
|
||
The SNES project's own `data/events/README.md` states its 516 chapter XMLs are
|
||
"**derived from DirkSimple game data**". They are a *conversion* of the same
|
||
transcription, not a second one. The diff below is still worth running — it
|
||
catches conversion errors — but it **cannot** catch a transcription error,
|
||
because there is nothing independent to compare against. DirkSimple is the
|
||
single source, and its own provenance is the arcade ROM's data table.
|
||
|
||
That correction was cheap to make and it was one sentence of a README away from
|
||
never being made at all. It belongs with FINDINGS 42.1: a plausible source with
|
||
no provenance is folklore, and so is a plausible cross-check.
|
||
|
||
### 56.3 THE MEASUREMENT: the worst gap is zero, and 5.4% of branches are
|
||
|
||
Chaining play across non-seeking sequences and taking the **earliest** moment an
|
||
input window opens (the least play a clip can deliver before the branch it leads
|
||
to), over 612 distinct transitions into a seek, excluding attract mode:
|
||
|
||
| | seconds of play |
|
||
|---|---:|
|
||
| worst | **0.000** |
|
||
| p10 | 0.950 |
|
||
| p25 | 1.966 |
|
||
| median | 3.473 |
|
||
| p75 | 5.800 |
|
||
| p90 | 9.548 |
|
||
| best | 82.497 |
|
||
|
||
**33 of the 612 (5.4%) are zero**: an input window that opens at t=0 of a clip
|
||
the disc *seeked to*, so two seeks can fall back to back with no play between
|
||
them at all. `flaming_ropes.enter_room -> fall_to_death` is one — press right on
|
||
the frame the clip starts and the player dies immediately.
|
||
|
||
This is not an edge case to be designed around; it is the game. **A rule of the
|
||
form "has there been enough play since the last branch" — 51.2's slack rule,
|
||
`ring_may_seek` in `src/player/ring.i` — can be answered NO by the content,
|
||
not by the buffer**, and no amount of ring is a defence.
|
||
|
||
**203 of the 612 end the scene**, which in this design is also a container
|
||
change and needs 6,164 header bytes before frame 0 (53, 55.1). The worst
|
||
scene-change gap is **0.541 s** and the median is 2.501 s.
|
||
|
||
### 56.4 The median branch point arrives before the ring has refilled
|
||
|
||
51.3's climb, against the game's own gap distribution. Gate container
|
||
(`rc_fr_singe_scsi_span.dlx`, mean record 36.5 KB, wire 446.1 KB/s); rates are
|
||
explicit arguments and every one of them is a sensitivity, not a claim:
|
||
|
||
| ring KB | pipe KB/s | ceiling | climb s | branch points under the climb |
|
||
|---:|---:|---:|---:|---:|
|
||
| 256 | 451.4 | 3 | 20.83 | 601/612 (98%) |
|
||
| 256 | 488.0 | 7 | 6.11 | 468/612 (76%) |
|
||
| 256 | 513.2 | 7 | 3.81 | 370/612 (60%) |
|
||
| 256 | 600.0 | 7 | 1.66 | 129/612 (21%) |
|
||
| 512 | 488.0 | 11 | 9.60 | 551/612 (90%) |
|
||
| 512 | 513.2 | 14 | 7.63 | 512/612 (84%) |
|
||
| 512 | 600.0 | 14 | 3.32 | 301/612 (49%) |
|
||
|
||
Two things fall out, and the second is the one that costs something.
|
||
|
||
1. **At every rate this tree has considered, most branch points arrive with less
|
||
lookahead than the one before them.** At 488 KB/s in a 256 KB ring — the
|
||
configuration `check.sh` gates — that is 76%.
|
||
2. **A bigger ring makes this metric worse, and now content says so too.** 51.3
|
||
derived it from the surplus alone; here the same rate goes from 76% under the
|
||
climb at 256 KB to 90% at 512 KB, because doubling the ring doubles the
|
||
ceiling without touching `pipe - wire`. **The ring is not the lever. The
|
||
surplus is.**
|
||
|
||
### 56.5 What a branch actually costs when the gap bought nothing
|
||
|
||
The climb is what a player needs to *tolerate* the next branch. What it *pays*
|
||
at one is the prefill, because the ring is empty after a seek and 55.4's shipped
|
||
policy releases the decoder at 2 records:
|
||
|
||
| pipe KB/s | 2-record prefill | as a scene change (+6,164 B) |
|
||
|---:|---:|---:|
|
||
| 451.4 | 161.8 ms (1.94 frame slots) | 175.2 ms (2.10) |
|
||
| 488.0 | 149.7 ms (1.80) | 162.0 ms (1.94) |
|
||
|
||
So a zero-play branch is **not** a failure: it costs about two frame slots of
|
||
black, every time. What it removes is margin. A player that branches at p10
|
||
(0.950 s) has spent its whole lookahead and rebuilt almost none of it, and the
|
||
next slow record has nothing behind it. **The finding is not "this breaks", it
|
||
is "this design runs permanently at minimum lookahead, and the arcade content is
|
||
what puts it there."** 22_scene_load.py prices the scene-change case properly,
|
||
clocks included; the mechanical seek is still unmodelled (B1) and is charged on
|
||
top of all of it.
|
||
|
||
### 56.6 What the cross-check IS worth, now that it is not a cross-check
|
||
|
||
Run anyway, against 518 SNES chapter XMLs, with the 40 scene abbreviations
|
||
matched to DirkSimple scenes **from the data** — the abbreviation's letters must
|
||
be a subsequence of the scene name, ranked by sequence-name overlap, with a
|
||
reversed-scene rule for the thirteen mirrored scenes. (Overlap alone mapped
|
||
`snkr` to `black_knight`, because two scenes full of `seqN` names look alike.)
|
||
|
||
- **Start times do not compare at all.** The SNES XMLs are on their own
|
||
extraction's timeline; DirkSimple is on arcade ROM frames minus 6,297 ms. The
|
||
difference is not a constant and **does not even keep its sign** (spread
|
||
-8.0 s .. +6.4 s). Reported as a spread rather than as a check, because a
|
||
check it is not.
|
||
- **Durations do compare** — they are offset-invariant. 388/505 agree within one
|
||
laserdisc frame (76.8%), median difference 0 ms.
|
||
- **Branch structure compares**: 470/505 chapters (93.1%) carry the identical
|
||
set of (input -> target) edges. Of the 35 that differ, **16 are renames**
|
||
(`captured_by_ghouls` -> `ghoul_capture`) and **18 of the remaining 19 are the
|
||
SNES conversion dropping the arcade's diagonals**. The 19th is
|
||
`intr_castle_exterior`, where the SNES *added* a skip.
|
||
|
||
**Zero transcription discrepancies were found, and none could have been.** What
|
||
the diff produced is one useful fact about our own input layer, below.
|
||
|
||
### 56.7 Two constraints on the input layer, from the same import
|
||
|
||
1. **The arcade uses eight directions plus action and start.** Counted over the
|
||
906 windows: left 233, right 217, up 209, down 152, action 72, **upleft 13,
|
||
upright 4, downleft 3, downright 1**, start 2. The diagonals are 21 windows
|
||
out of 906 — rare enough to be dropped by a port that had to (the SNES one
|
||
did) and **not** droppable by one aiming at the arcade.
|
||
2. **The shortest input window is 98 ms.** Median 950 ms (11.4 frame slots), p10
|
||
393 ms, but the floor is 98 ms — and by 54.4 a 12 fps frame slot is 72.13 ms
|
||
or 90.16 ms, never 83.33. **A 98 ms window is one or two frames wide.**
|
||
Polling input on the frame tick is therefore marginal by construction: the
|
||
input layer has to run off something faster than the frame clock, and
|
||
`src/player/clock.i` already owns an MFP interrupt at 181.35 clocks a
|
||
V-DISP (54.2) that is 8.6x faster and costs 0.1% of the budget.
|
||
|
||
### 56.8 What this does NOT establish
|
||
|
||
1. **No rate here is measured.** Every column is a sensitivity across explicit
|
||
rates (FINDINGS 50). B1 is still open and the mechanical seek still has no
|
||
figure at all.
|
||
2. **The gap model is a lower bound by construction.** It takes the earliest
|
||
instant an input window opens, so it is what an *expert* player can force,
|
||
not what a typical one produces. That is the right bound for a buffer
|
||
design and the wrong one for describing play.
|
||
3. **It is the arcade's graph, not this port's.** The 612 transitions assume
|
||
the port reproduces every branch. Nothing has been mapped onto our 224
|
||
Blu-ray streams yet — the SNES project's 516 chapters are finer-grained than
|
||
our streams, and that mapping is still C1's problem.
|
||
4. **Nothing ran on the 68000 this session.** This is host-side analysis of an
|
||
imported table. `decode.s`, `stream.s`, `ring.i`, `clock.i` and `load.i` are
|
||
untouched, `decode.bin` is still 1,296 B at the same MD5, and the green light
|
||
was ALL GREEN before and after.
|
||
5. **The scene graph is not vendored, and the coupling is contained.** Neither
|
||
repo ships here and neither is redistributable from this tree; both are
|
||
permissive (DirkSimple zlib, Ryan C. Gordon; SNES project MIT, Chad
|
||
Doebelin) and both are cloned by the reader. The `sources` block of every
|
||
generated table carries the attribution. **`tmp/scenegraph.json` is
|
||
generated, gitignored and derived data**: committing it, or any table built
|
||
from it, is redistribution and the attribution has to travel with it.
|
||
|
||
---
|
||
|
||
## FINDINGS 57 — the 68000 reads the disc itself, and P4 was never blocked
|
||
|
||
Session 25. ROADMAP P4, first half. Green light ALL GREEN before and after.
|
||
**Emulated** — MAME 0.277, `x68000 -exp1 cz6bs1`. No real hardware.
|
||
|
||
### 57.1 The blocker was a missing FILE, not a missing MODEL, and the tree already knew
|
||
Session 21 recorded P4 as "blocked in this tree", re-checked rather than
|
||
assumed, on three grounds. One of them is wrong:
|
||
|
||
> there is still no `scsiexrom.bin` anywhere on this machine ... **MAME's
|
||
> `x68000` has no MB89352 path**, and `hd63450.cpp` decodes no DTYP.
|
||
|
||
MAME 0.277's `x68000` **does** have one. `-listslots` offers `cz6bs1` on `exp1`
|
||
and `exp2`; `-listdevices x68000 -exp1 cz6bs1` shows a *Fujitsu MB89352 SCSI
|
||
controller @ 5.00 MHz* on a SCSI bus with a hard disk at ID 0, alongside the
|
||
HD63450. **FINDINGS 32.4 had already established this in session 9** — the
|
||
CZ-6BS1's DMA glue, `$EA0000`, the data register at `$EA0015` — and 42.5 built
|
||
on it in session 14. The session-21 note is a regression in the record, not a
|
||
discovery.
|
||
|
||
What is genuinely missing is only the 8 KB `scsiexrom.bin` (CRC `7be488de`),
|
||
which MAME **requires** to instantiate the card: without it the machine refuses
|
||
to start at all. **That is the entire blocker, and it is not one**, because the
|
||
player drives the SPC registers directly and never executes that ROM — which was
|
||
already the plan in `docs/BENCHMARK.md` item 4, written in session 2, long
|
||
before the file turned out to be absent. `tools/bench/scsi_run.sh` supplies a
|
||
zero-filled placeholder on its own rompath, leaves the user's romset untouched,
|
||
and lets MAME print WRONG CHECKSUMS as it should.
|
||
|
||
**The substitution is honest here and would not be everywhere.** Anything that
|
||
boots from the card, or calls SCSI IOCS, *does* execute that ROM. Do not reuse
|
||
the rompath for those. **B3 is untouched**: it wants the ROM's bytes
|
||
disassembled for the DCR it writes, and a blank one has none.
|
||
|
||
### 57.2 The register map, measured rather than inferred
|
||
32.4 quoted one address. `src/player/scsigate.s` probes `$EA0000..$EA003F` one
|
||
address at a time, with a bus-error handler that records the fault, steps the
|
||
index and re-enters the loop — so a hole costs an entry in the map rather than
|
||
the rest of the run. **60 of 64 addresses answer.**
|
||
|
||
| | |
|
||
|---|---|
|
||
| registers | odd bytes, `$EA0001 + 2n`, `n = 0..14` |
|
||
| `$EA0007` (n=3, TMOD) | **BUS ERROR** |
|
||
| `$EA001F` (n=15, EXBF) | **BUS ERROR** |
|
||
| `$EA0017` (n=11, TEMP) | wrote `$A5`, read back `$A5` |
|
||
| `$EA0015` (n=10, DREG) | as 32.4 said |
|
||
|
||
The two holes are exactly the two registers the **MB89352 omits and the MB87030
|
||
has**, which independently confirms which part MAME is modelling. It also
|
||
**corrects the device's own documentation**: the summary of `mb87030.cpp` says
|
||
the MB89351/352 "skip TMOD and EXBF, shifting subsequent indices accordingly",
|
||
and the machine says MAME leaves *holes* and shifts nothing — which is what
|
||
keeps DREG at index 10 and at `$EA0015`. The bytes win. The first version of
|
||
this probe walked upward with `move.b (a0)+`, took a bus error at `$EA0006`, and
|
||
knew one address was dead and nothing about the other 57; **a sequential dump
|
||
reports the first hole as the answer**.
|
||
|
||
### 57.3 The data register is DMA-only on this card, and a PIO write vanishes
|
||
`x68k_scsiext.cpp` installs its own handler on `$EA0015` **and on no other
|
||
address**:
|
||
|
||
write: if (exown()) { if (!drq) dtack_w(1); else dma_w(data); }
|
||
else dreg_w(data);
|
||
|
||
On this machine `exown()` — the HD63450's OWN, fed back to the slot by
|
||
`x68k.cpp` — is asserted where a PIO write needs it not to be. The `else` arm is
|
||
unreachable, and **a byte written to `$EA0015` with the SPC in PROGRAM transfer
|
||
mode is discarded silently**: no error bit, no status change, no interrupt.
|
||
Quieting all four DMAC channels (`CCR = 0`, `CSR = $FF`) does not change it.
|
||
|
||
It was **measured, not reasoned about** — the gate writes `$5A` to `$EA0015` and
|
||
reads it straight back, and gets `$00` with the FIFO still empty — because ten
|
||
command bytes vanishing without trace looks exactly like a target refusing a
|
||
command, and that is how it first presented.
|
||
|
||
So every transfer issues `SCMD` **without** the PROGRAM bit, which puts the SPC
|
||
in DMA mode and makes it raise DRQ, and the CPU then moves the bytes through
|
||
`$EA0015` itself, in via `dma_w` and out via `dma_r`. **The CPU stands in for
|
||
the DMAC, through the DMAC's own door.**
|
||
|
||
**What that costs the argument, stated because it is easy to overclaim.** With
|
||
`exown` asserted at idle, MAME **cannot distinguish a CPU-driven byte at
|
||
`$EA0015` from a DMAC-driven one**. This rig therefore demonstrates the DATA
|
||
PATH and cannot, on its own, demonstrate that the HD63450 is the thing driving
|
||
it — which is precisely what ROADMAP calls P4's first job. Whether a real
|
||
CZ-6BS1 also refuses PIO here is **not settled**: it is a property of MAME's
|
||
model and it wants a board.
|
||
|
||
### 57.4 The result
|
||
`src/player/scsi.i`, 68000 code, no IOCS and no host in the transfer path:
|
||
selects the target, and issues **READ(10)** twice.
|
||
|
||
READ(10) OK: 4096 B from LBA 0 match the host's image byte for byte
|
||
READ(10) OK: 2048 B from LBA 1000 match too
|
||
|
||
The second one is the half that matters: **a driver that emits a malformed LBA
|
||
field still passes LBA 0**, because zero is what a malformed field usually is.
|
||
|
||
The volume is `tmp/stream_disk.bin` — the *same* file `prep_stream.py` already
|
||
writes for the ring rig — so the SCSI volume and the host-file pipe carry
|
||
byte-identical bytes, and a difference between the two rigs cannot be a
|
||
difference in what they are reading.
|
||
|
||
### 57.5 Five bugs, and four of them were silent
|
||
Recorded because the pattern is the finding: **nothing in a SCSI bring-up tells
|
||
you what you did wrong.** Every one of these presented as a phase that never
|
||
arrived.
|
||
|
||
1. **`sc_settc` wrote the wrong three bytes.** Three chained `rol.l #8` put the
|
||
original bits 31..24, 23..16 and 15..8 into TCH/TCM/TCL, so a count of 10
|
||
loaded a transfer counter of **zero**. MAME completed the TRANSFER instantly
|
||
and silently, and the bus sat in command phase. Now written low-byte-first
|
||
with `lsr`. The same trap was live in the CDB's block-count field.
|
||
2. **A byte handed to a FIFO is not a byte on the bus.** The driver returned
|
||
from the command phase with the last byte still in the SPC, asked what phase
|
||
the bus was in, and got COMMAND — which reads as a target refusing the
|
||
command. Fixed by waiting for `XFER IN PROGRESS` to clear.
|
||
3. **A fixed phase sequence is wrong.** The first version ran select → command →
|
||
data → status → message; the target came up in MESSAGE OUT with ATN asserted
|
||
and the driver called it an unexpected phase. **The bus decides the order.**
|
||
`scsi_read` is now a phase loop, which is both shorter and correct.
|
||
4. **The discarded PIO write** of 57.3.
|
||
5. **The initiator must let go of the bus, in two steps.** After the final
|
||
message byte the SPC still holds ACK — `PSNS` reads `$4F`, REQ low and ACK
|
||
high — and a target cannot drop BSY into that. It needs `SCMD` reset-ACK/REQ
|
||
**and then** bus release. **This one only appeared once there were two
|
||
reads**: one read passed byte-exact and every conclusion drawn from it was
|
||
sound, and the second could not select. A player issues one command per
|
||
record, so the failure would have been universal in the ring and invisible in
|
||
the demonstration.
|
||
|
||
### 57.6 What this does and does not move
|
||
**Does not move `W`.** Not by one clock. MAME's device models are functional,
|
||
not transfer-timing accurate (`docs/BENCHMARK.md`), and 42.5 reads its DMAC
|
||
configured in wall-clock attotimes rather than per-operand cycles. `W` remains
|
||
the project's largest open number and still wants a board.
|
||
|
||
**Does not finish P4.** What is done is the correctness half — BENCHMARK's Tier
|
||
1, "does our read path work at all". What is left is the half ROADMAP calls
|
||
P4's first job: **a DMAC configuration that holds the bus**, and then the driver
|
||
behind `ring.i`'s `XF_*` mailbox in place of `stream.lua`'s modelled transport,
|
||
gated on the same pixel-exact 120 frames. 57.3 is a warning about the first of
|
||
those: this apparatus cannot tell a DMAC-driven byte from a CPU-driven one at
|
||
`$EA0015`, so "the DMAC held the bus" will need evidence that does not come from
|
||
watching that address.
|
||
|
||
**Does move the premise of every delivery rig in the tree.** Until now the
|
||
bytes came from a host. They now come off a disc, on the machine's own
|
||
instructions, byte-exact at two different LBAs.
|
||
|
||
---
|
||
|
||
## FINDINGS 58 — the player runs off the disc, and PIO costs 87 clocks a byte (session 25b/26)
|
||
|
||
**Emulated.** MAME 0.277, `x68000 -exp1 cz6bs1 -ramsize 2M`, a blank
|
||
`scsiexrom.bin` on a private rompath (57.1's substitution, unchanged). No real
|
||
hardware ran. `./tools/bench/check.sh` was ALL GREEN before this and ALL GREEN
|
||
after, with two new stages.
|
||
|
||
**ROADMAP P4b is DONE. P4a is not, and 58.2 is why it is now the item that
|
||
decides the project rather than one of two that do.**
|
||
|
||
### 58.1 The seam closed: 120 pixel-exact frames, off a real volume
|
||
|
||
`src/player/xfer.i` sits behind `src/player/ring.i`'s `XF_*` mailbox in place of
|
||
`tools/bench/stream.lua`'s modelled transport. `XF_GO` is answered by a real
|
||
READ(10) to a real MB89352 and `XF_ACK` is a word the 68000 bumps when the bytes
|
||
have landed — not one a host synthesises from emulated time.
|
||
|
||
| | |
|
||
|---|---|
|
||
| records fetched by the 68000 | **120**, one READ(10) each |
|
||
| bytes into the ring | **4,488,588**, and the decode is **pixel-exact** |
|
||
| bytes off the disc | **4,548,608** — see 58.3 |
|
||
| ring | 256 KB, **18 wraps, 14.7 KB mean hole** |
|
||
| a real mid-stream seek | **pass 2 pixel-exact**, ring thrown away and rebuilt |
|
||
|
||
**The 18 wraps are the load-bearing assertion, not the byte count.** They are
|
||
the same 18 the host producer produced in 49.4 and the same 18 `ring.i`
|
||
produced against a modelled transport in 55.4 — a *third* transport, same
|
||
tiling. `ring.i` is not supposed to be able to tell which side of the mailbox
|
||
answered it, and this is the number that says it could not.
|
||
|
||
**The change above the seam is two `bsr`s.** One in `ring_poll`, one in
|
||
`ring_seek`'s quiet-wait — and the second is not optional. With the transport
|
||
*inside* the machine, the only thing that can retire an outstanding request is
|
||
that wait loop itself, so a seek issued with a request in flight spins forever
|
||
without it. A host transport retired it on its own time. That is exactly the
|
||
kind of difference the seam exists to hide, and it is the one it could not.
|
||
|
||
### 58.2 What it costs: 87.28 clocks per delivered byte, and the number is portable
|
||
|
||
`tools/bench/xfer_cost.sh` decodes the same 120 frames twice — same ring, same
|
||
`stream.s`, same `ring.i` placing every record — and changes only which side of
|
||
the mailbox answers:
|
||
|
||
| | emulated | per frame |
|
||
|---|---|---|
|
||
| decode + `ring_poll` alone | 6.7737 s | **67.7%** of a 12 fps frame |
|
||
| ...with the real transport | 45.9516 s | **459.5%** |
|
||
| **the transport** | **391,779,000 clk** | **391.8%** |
|
||
|
||
**= 87.28 clocks per delivered byte**, 86.13 per byte off the FIFO.
|
||
|
||
**AND IT IS NOT MAME'S NUMBER, WHICH IS THE HALF THAT MATTERS.** The keep loop
|
||
in `src/player/scsi.i`, priced against the 68000's own cycle table — 12 patience
|
||
reload, 16 SSTS read, 10 `btst`, 10 branch, 20 DREG read to `(a1)+`, 8 `subq`,
|
||
10 branch — is **86 clocks**, and the FIFO also carries the dropped window bytes
|
||
of 58.3, which makes it **87.15 per delivered byte**. Measured 87.28. **0.2%
|
||
apart.** So the cost is the instruction stream and not a wait on MAME's SPC
|
||
model: it is a figure a real board would also pay, and it is the first number
|
||
this rig has produced that survives leaving the emulator. The residual, +0.13
|
||
clk/B = 4,989 clocks per record, is the per-*command* cost — select, CDB,
|
||
status, message, `xf_service` — and it is the part that does not scale with the
|
||
record.
|
||
|
||
**Against the ladder, in the same units** (clocks charged to the CPU per
|
||
delivered byte, at this container's 37,405 B mean record):
|
||
|
||
| | | share of a 12 fps frame |
|
||
|---|---|---|
|
||
| `W` = 5 | single address, bus HELD | 22.4% |
|
||
| `W` = 9 | dual address, held | 40.4% |
|
||
| `W` = 12 | single address, arbitrated | 53.9% |
|
||
| `W` = 19 | dual address, arbitrated — the IPL ROM's own disk channel (52.5) | 85.3% |
|
||
| **PIO 87** | **this rig, measured** | **391.8%** |
|
||
|
||
**The PIO transport is 4.6x the worst DMA configuration this project has found
|
||
and 17.5x the best.** P4a is not an optimisation of this. It is the difference
|
||
between a player and a slideshow, and it is now the *only* thing between the
|
||
tree and M2.
|
||
|
||
**The player's own clock says the same thing, independently.** Self-paced off
|
||
V-DISP, the machine decoded 120 frames in **560 slots** of a 12 fps clock:
|
||
**2.57 fps**. That agrees with 12 / 4.595 = 2.61 from the cycle accounting
|
||
above, from a completely different instrument.
|
||
|
||
**AND "UNDERRUNS: 0/120" IS VACUOUS IN THIS RUN.** A synchronous transport
|
||
cannot underrun by construction — a frame cannot start before its record has
|
||
landed, because the decoder *is* the transport. The counter that means anything
|
||
here is `NO IDLE`: **119 of 120 frames found their slot already open, worst
|
||
overrun 441 whole ticks.** `tools/bench/stream.lua` now prints that argument
|
||
next to the zero rather than leaving the zero to be quoted. This is the same
|
||
class of error as 49.7.2's free-running ring passing at 48 KB: a rig
|
||
configuration in which the failure it tests for cannot occur.
|
||
|
||
**The resource that binds also flipped, and it is worth naming.** Against a
|
||
modelled 488 KB/s pipe this container was rate-bound and the ring never filled
|
||
(55.4). Here the ring **fills — 90 refusals for space** — while the decoder
|
||
starves. Not of bytes: of *time*. Every previous delivery result in this tree
|
||
was measured on a rig where the transport cost the CPU nothing.
|
||
|
||
### 58.3 A record is not a sector, and the cheapest fix is a re-encode
|
||
|
||
`ring.i` asks for a byte offset and a length, both 4-byte aligned (28.3);
|
||
a target answers in 512 B blocks. **On the gate container 117 of 120 records
|
||
start part way into a sector.** This is not a rounding nuisance: the bytes on
|
||
either side of a record belong to *other* records the decoder may still be
|
||
reading, and the block loop walks `a0` with no bounds check (49.2), so a
|
||
transport that reads whole sectors straight into the ring **corrupts its
|
||
neighbours** — wrong pixels, not a fault.
|
||
|
||
`tools/analysis/26_sector_align.py` prices the three ways out:
|
||
|
||
| | wire | clocks |
|
||
|---|---|---|
|
||
| **A. windowed PIO** — read the covering sectors, store only the record | **+1.34%** | 86/B on every byte off the FIFO, and **a DMAC cannot do it at all** |
|
||
| **B. bounce buffer** — DMA whole sectors elsewhere, then copy | +1.34% | **+5/B on every delivered byte** = 22.4% of the frame, on top of `W` |
|
||
| **C. sector-aligned records** — pad to 512 in the container | **+0.43%** | **zero** |
|
||
|
||
A is what shipped in `scsi.i` and what 58.1 ran, and it is free *only* because
|
||
the CPU is already touching every byte — the property that disappears the moment
|
||
P4a succeeds. B is the cost `aligned` was chosen over `split` to avoid (49.3)
|
||
arriving by a different door, and on every byte instead of on a wrap.
|
||
|
||
**C wins on both axes**: it is cheaper on the wire than A and B by 0.91 points
|
||
of the payload (40,940 B on this scene), and it is the only one of the three a
|
||
DMA channel can run without a copy. What it costs is a **container revision** —
|
||
a re-encode plus a re-measurement of every constant fitted to the gate
|
||
container. That is the class of change ROADMAP already has bundled with P2's
|
||
other half (reserve index 0 as black) and 55's two open re-encode questions.
|
||
**It should join that bundle**, and P4a should be attempted against a
|
||
sector-aligned container rather than against this one.
|
||
|
||
It also grows the largest record from 40,984 to 41,472 B, which a 256 KB ring
|
||
still holds six times over — so it costs nothing in ring size.
|
||
|
||
### 58.4 What this does and does not move
|
||
|
||
**Does not move `W`.** Again, and for the same reason 57.6 gives. Nothing here
|
||
programmed a DMAC channel; `xf_service` is the CPU standing in for one, through
|
||
the DMAC's own door (57.3).
|
||
|
||
**Does not measure a delivery rate,** and the rig now refuses to be asked. A
|
||
`DLX_XFER=scsi` run rejects a `--kbps` argument outright rather than ignoring
|
||
it, and `stream.lua` suppresses `CHANNEL IDLE`, `DEADLINE` and `REQUIRED
|
||
PREFILL` instead of printing them as zeros — a zero there reads as "the channel
|
||
never stopped", which would be a claim about a medium this tree has never timed.
|
||
|
||
**Does close P4b, and does hand P4a a sharper question than it had.** Before
|
||
this session, "get the DMAC to hold the bus" was worth 9 clk/B against 19. It is
|
||
now worth 87 against either.
|
||
|
||
**Does put a real number on the layer below every previous delivery result.**
|
||
Everything in FINDINGS 49, 51 and 55 was measured with the bytes arriving free.
|
||
They are not free, and 58.2 is the first measurement of what they cost.
|
||
|
||
---
|
||
|
||
## FINDINGS 59 — the DMAC drives the data phase, and auto-request is charged by time (session 27)
|
||
|
||
**Emulated.** MAME 0.277, `x68000 -exp1 cz6bs1 -ramsize 2M`, a blank
|
||
`scsiexrom.bin` on a private rompath (57.1's substitution, unchanged). No real
|
||
hardware ran. `./tools/bench/check.sh` was ALL GREEN before this and ALL GREEN
|
||
after, with one new stage.
|
||
|
||
**ROADMAP P4a is DONE at the transport level.** What is left before M2 is not a
|
||
DMAC question any more; it is the re-encode bundle, because 59.4 makes
|
||
sector-aligned records a precondition the transport now enforces rather than a
|
||
preference the roadmap recorded.
|
||
|
||
### 59.1 The channel drives the data phase, and the evidence is the CPU's own progress
|
||
|
||
`src/player/dma.i` programs HD63450 channel 1 and hands it the DATA IN phase;
|
||
`src/player/dmagate.s` reads **the same 2,048 B at LBA 1000 three ways** and the
|
||
host compares all three against its own copy of the image:
|
||
|
||
| | bytes | MTC one instruction after START | CPU trips round its wait loop |
|
||
|---|---|---|---|
|
||
| PIO, the path 58.2 measured | byte-exact | — | — |
|
||
| **DMA, bus HELD** (DCR `$00`, OCR `$81`) | **byte-exact** | **0 of 2048** | **1** |
|
||
| **DMA, cycle STEALING** (DCR `$80`, OCR `$80`) | **byte-exact** | 2048 of 2048 | 426 |
|
||
|
||
Both channels reported `CSR = $E0` (COC, BTC, NDT), `CER = $00`, `MTC = 0` and a
|
||
memory address exactly `+2048` from where it started.
|
||
|
||
**THE DISCRIMINATOR NEVER READS `$EA0015`, and that is the whole design.** 57.3
|
||
established that watching the data register cannot answer this question: with
|
||
the DMAC's `OWN` asserted — which it is at idle on this machine — MAME cannot
|
||
tell a CPU-driven byte there from a DMAC-driven one. So what separates the two
|
||
configurations is **whether the 68000 executed anything while the bytes were
|
||
arriving**:
|
||
|
||
move.b #CCR_START,DM_CCR ; the channel is told to go
|
||
move.w DM_MTC,d0 ; <- sampled by the VERY NEXT instruction
|
||
|
||
Held, `d0` is **zero**: the entire 2,048-byte transfer happened between two
|
||
instructions, because the CPU did not run in between. Stealing, `d0` is the full
|
||
count and the CPU then goes round its own loop 426 times while the bytes
|
||
trickle in. That is what "holds the bus" means, and it is a fact about the CPU
|
||
rather than about the data register.
|
||
|
||
**The mechanism, named so the claim is not over-read.** MAME models a held bus
|
||
by asserting `INPUT_LINE_HALT` for burst + max-rate and clearing it at
|
||
end-of-transfer (`hd63450.cpp`). It is not inventing that semantic: MC68450
|
||
§5.2.3.3.1 says of maximum-rate auto-request that "all operands in the data
|
||
block will be transferred in one burst, so that the DMAC will use **100% of the
|
||
available bus bandwidth**". The model and the datasheet agree about what this
|
||
configuration does to the CPU. **They do not agree about anything per-operand,
|
||
and no `W` is claimed here** (42.5: MAME's DMAC runs on wall-clock attotimes).
|
||
|
||
**The gate was checked against its own negative.** With the *stealing* register
|
||
pair put in the *held* slot, the run still delivers all 2,048 bytes byte-exact
|
||
— and `tools/bench/dma_run.sh` goes **red**, on the trip count and on the MTC
|
||
sample. A counter that cannot come out different is 58.3's vacuous
|
||
"UNDERRUNS: 0/120" again; this one can, and was made to.
|
||
|
||
`tools/analysis/27_dmac_config.py` decodes the four register bytes out of
|
||
`src/player/dma.i` itself, with the same MC68450 field tables
|
||
`21_iplrom_dmac.py` reads the IPL ROM with (now one copy, `mc68450.py`). So
|
||
"dual address, 8-bit port, burst, auto-request at max rate" is a decode of the
|
||
bytes the player programs, not a comment next to them — and it is directly
|
||
comparable with Sharp's own disk channel, which 52.5 read as `DCR $80 / OCR $B2`
|
||
and priced at 16..19 clk/B.
|
||
|
||
### 59.2 Three things this model cannot be asked, and they bound the result
|
||
|
||
Read out of MAME 0.277's source rather than inferred from behaviour:
|
||
|
||
1. **The card has no request line to the DMAC.** `x68k_scsiext.cpp`'s `drq_w`
|
||
only stores a flag; the expansion slot carries no request to the HD63450 at
|
||
all (`x68k.cpp` wires `drq0` from the FDC and `drq3` from ADPCM, and nothing
|
||
else). The card's flow control is **DTACK**: on a DMAC cycle with DRQ low it
|
||
negates DTACK and the channel discards that operand and retries. So **REQG =
|
||
10, external request — the mode the ladder's `W`=5 and `W`=12 rows assume —
|
||
cannot be run here at all.**
|
||
2. **Single address cannot be run either.** `hd63450.cpp` takes the implicit
|
||
path only for a channel with a device callback, and on this machine only
|
||
channel 0 (the FDC) has one. `DTYP` = 10/11 on channels 1..3 falls through to
|
||
the dual-address code.
|
||
3. **Only burst is modelled as held.** The device tests `(dcr & 0xc0) == 0`, so
|
||
XRM = 10 (cycle steal *without* hold) and XRM = 11 (cycle steal **with**
|
||
hold) are one code path.
|
||
|
||
**So of the four rows of the per-byte ladder, exactly one — dual address, bus
|
||
held, 9 clk/B — has a code path in this model, and it is the one demonstrated.**
|
||
That is a bound on the apparatus, not a result about the board. The slot's own
|
||
pinout has `#EXREQ` at B36, so a real CZ-6BS1 plausibly drives it; whether it
|
||
does is ROADMAP B3, and it is now a sharper question than "which DTYP".
|
||
|
||
### 59.3 Auto-request is charged by TIME, not by byte — and the GCR is the lever
|
||
|
||
This is the finding that outlives the emulator. Every `W` in this project is
|
||
**clocks per delivered byte**, which presumes the device asks for each one. An
|
||
auto-requested channel does not know whether the device is ready: it takes the
|
||
share of the bus it was told to take and spends it either way. So the cost of a
|
||
record scales with **how long the record takes to arrive** — halve the delivery
|
||
rate and the CPU cost of the same record **doubles**. No `W` does that.
|
||
|
||
`tools/analysis/28_autorequest_cost.py` prices it from MC68450 §3.8 and
|
||
§5.2.3.3.2, and gates its formulas against Table 5-3's sixteen printed rows
|
||
before printing anything. At the gate container's 37,405 B mean record and
|
||
**460 KB/s, an explicit rate and not a measurement** (FINDINGS 50):
|
||
|
||
| configuration | sustains | charged to the 68000 | % of a 12 fps frame |
|
||
|---|---|---|---|
|
||
| REQG 01, max rate — what 59.1 demonstrated | the wire | 21.23 clk/B | **95.3%** |
|
||
| REQG 00, LRAR, BR = 00, 50% of the bus | 534 KB/s | 10.61 clk/B | **47.6%** |
|
||
| REQG 00, LRAR, BR = 01, 25% | 267 KB/s | 5.31 | 23.8% — **does not carry the rate** |
|
||
| REQG 00, LRAR, BR = 10, 12.5% | 133 KB/s | 2.65 | 11.9% — **does not carry the rate** |
|
||
| *(ladder, for comparison)* `W`=9 dual held | — | 9 | 40.4% |
|
||
| *(measured, 58.2)* PIO | — | 87.28 | 391.8% |
|
||
|
||
**The held configuration is the cheapest per byte MOVED and the dearest per byte
|
||
DELIVERED**, and the gap between those two is the device's own slowness: 9
|
||
clocks of DMAC work inside 21.2 clocks of waiting means **42.4% of the held bus
|
||
does anything at all**. Holding the bus is only cheap when the hold is ended by
|
||
the device, which is what external request is for.
|
||
|
||
**BT and BR are two bits each and they set what fraction of the bus the player
|
||
gives away** (burst time `2^(BT+4)` clocks, sample period `2^(BT+BR+5)`, share
|
||
`2^-(BR+1)`). Nothing in this tree had named the GCR as a design choice; it is
|
||
the same kind of lever as `aligned` vs `split` and it belongs in the same list.
|
||
**At 460 KB/s only BR = 00 carries the rate**, so the fallback plan — if a real
|
||
card turns out not to drive `#EXREQ` — is 50% of the bus for the duration of
|
||
every record, or 47.6% of a frame slot per record. That is affordable and it is
|
||
not free, and it is the first cost model in this project that gets *worse* when
|
||
the disc gets slower.
|
||
|
||
**59.3's one load-bearing assumption**, stated because the whole table rests on
|
||
it: that the channel *spends* its allotted share whether or not the device has a
|
||
byte. Under auto-request a request is pending until MTC is exhausted, so the
|
||
DMAC takes the bus during every window it is entitled to; when the device is not
|
||
ready the cycle is stretched by wait states (a real card negating DTACK) or
|
||
retried later (MAME's model discards the operand), and either way the window is
|
||
gone from the CPU's point of view. **If a real CZ-6BS1 instead lets the DMAC off
|
||
the bus early when no byte is there, 59.3's figures are upper bounds.** That is
|
||
a board question, and it is B3's.
|
||
|
||
### 59.4 The window is refused, and that makes the re-encode a precondition
|
||
|
||
58.3 found that 117 of 120 records start part way into a sector, and that PIO
|
||
absorbs it for free because the CPU is already touching every byte. A channel
|
||
cannot: it writes a contiguous run and cannot be told to drop the 300 bytes in
|
||
front of the record. `sc_in_data` now **refuses** a windowed read when the data
|
||
phase is the DMAC's — a new error, `SCE_WINDOW` — rather than quietly delivering
|
||
the neighbouring records' bytes into the ring, where the block loop has no
|
||
bounds check to catch them (49.2). The gate asserts the refusal.
|
||
|
||
**So "sector-aligned records" has stopped being a preference in ROADMAP's
|
||
re-encode bundle and become the thing standing between P4a and the ring.** The
|
||
transport states its own precondition; the container does not meet it yet.
|
||
|
||
### 59.5 What this does and does not move
|
||
|
||
**Does not move `W`.** Not by one clock, for the third session running, and for
|
||
the reason 57.6 and 58.4 give.
|
||
|
||
**Does not put the DMAC behind `ring.i`'s mailbox.** 59.4 is why: `xfer.i` asks
|
||
for records, and every record but three needs a window. That work is now
|
||
downstream of the re-encode bundle rather than of a DMAC question.
|
||
|
||
**Does close the question ROADMAP called P4's first job.** A configuration that
|
||
holds the bus exists, runs, delivers the disc's bytes byte-exact, and is
|
||
demonstrated by evidence that does not come from watching `$EA0015` — which is
|
||
exactly what 57.3 said would be needed.
|
||
|
||
**Does change what the fallback looks like.** Before this session the fallback
|
||
below a held bus was `W` = 16..19, the IPL ROM's own arbitrated configuration.
|
||
It is now limited-rate auto-request at a share the player chooses, priced in a
|
||
currency the project did not have, and the arithmetic says a 50% share carries
|
||
this container at 460 KB/s.
|
||
|
||
### 59.6 One collision, and it was caught by the half of `check.sh` that runs first
|
||
|
||
`DM_USE` — the word that tells `sc_in_data` whether the data phase is the
|
||
DMAC's — was first placed at `$18300`. `scsi.i`'s trace ends at `$182FF` and the
|
||
next 160 bytes are the **ring's**: `$18300` is `ring.i`'s `XF_SLOT` mailbox, and
|
||
`tools/bench/stream.lua` reads the same addresses from outside the machine. So
|
||
the ring rig's first record request wrote a non-zero word into what the
|
||
transport now read as "use the DMAC", and **the P4b stage — a stage this session
|
||
did not otherwise touch — went red on a run that never reached its snapshot.**
|
||
|
||
Recorded because the procedure is the finding: `check.sh` was ALL GREEN before
|
||
any of this work, so the red was unambiguously new, and the failure was in a
|
||
stage nobody would have re-run on suspicion. Both halves of "green before and
|
||
green after" earned their place; the map is now `$18500`, clear of everything
|
||
the streaming rig owns.
|
||
|
||
### 59.7 What it all costs: the frame affords 6.74 clocks a byte, and a dual-address byte is 9
|
||
|
||
The sweep in `15_bus_occupancy.py` has always answered *"what does each `W`
|
||
cost"*. It never answered *"what can the frame afford"*, and after 59.2 those
|
||
stopped being the same question. The tool now answers both; every figure below
|
||
is it, on the gate container at 12 fps, and the decode term is **measured**
|
||
(C68K) while the transport terms are datasheet arithmetic — except PIO, which
|
||
58.2 measured.
|
||
|
||
frame slot 833,333 clk
|
||
decoder, MEASURED 570,958 clk 68.5% (worst frame 110.8%)
|
||
audio DMA 10,417 clk 1.25%
|
||
------------------------------------------
|
||
HEADROOM 251,958 clk 30.2% = 6.74 clk/B at a 37,403 B record
|
||
|
||
| transport | clk/B | video | CPU+audio+video |
|
||
|---|---|---|---|
|
||
| PIO — **measured**, 58.2 | 87.28 | 391.7% | **461.5%** |
|
||
| dual address, arbitrated — the IPL ROM's own (52.5) | 16..19 | 71.8..85.3% | 141.6..155.0% |
|
||
| single address, arbitrated | 12 | 53.9% | 123.6% |
|
||
| **dual address, held — and the FLOOR of every dual-address configuration** | **9** | 40.4% | **110.2%** |
|
||
| single address, held | 5 | 22.4% | **92.2%** |
|
||
|
||
**WHAT P4a BOUGHT: the transport falls from 391.7% of a frame to 40..95%**, four
|
||
to ten times. That is the whole of the gain and it is the largest single
|
||
movement in the project's cost model since the decoder was written.
|
||
|
||
**WHAT IT DID NOT BUY IS A FIT, and the reason is one line of arithmetic.** A
|
||
dual-address byte is a 4-clock read of the device and a 5-clock write to memory
|
||
— `buscost.DMA_DUAL_BYTE_CLK`, Fig 4-25 — so **9 clocks is a floor, and the
|
||
frame affords 6.74.** No GCR share goes under it and no delivery rate goes under
|
||
it: a share decides whether the channel sits *at* the floor or *above* it. So
|
||
**59.2's three bounds arrive in the budget as one sentence — the configurations
|
||
this machine can run are exactly the ones the frame cannot afford**, and the one
|
||
it can afford is single address, which needs the device to ACK, which needs the
|
||
request line ROADMAP B3 asks about.
|
||
|
||
**The rate sweep says the same thing from the other side.** At 460 KB/s
|
||
max-rate totals **165.1%** and LRAR at 50% totals **117.4%**; the smaller shares
|
||
total less but cannot carry the rate at all. A faster disc makes auto-request
|
||
cheaper — which no `W` does, and it is the first cost in this project that
|
||
improves when the medium does — but it cannot reach the floor: **a 50% share
|
||
tops out at 543 KB/s**, above which the channel is the bottleneck and the
|
||
delivered rate falls back to it, at which point the cost is exactly 9 clk/B
|
||
again.
|
||
|
||
**SO THE FIT NOW TURNS ON ONE OF TWO THINGS, AND ONE OF THEM IS OURS.**
|
||
|
||
1. **Does a real CZ-6BS1 drive `#EXREQ`/`#EXACK`?** It is the only route to
|
||
single address, 22.4%, 92.2% total, fitting with 7.8% to spare. MAME cannot
|
||
answer it — it does not connect the pin (59.2) — and the slot pinout has it
|
||
at B36/B37. **This is now the sharpest form ROADMAP B3 has ever had**, and it
|
||
is worth more than the throughput half of B1: B1 sets how much headroom the
|
||
player has, B3 decides whether there is any.
|
||
2. **Bytes.** At the 9 clk/B floor this container must come down to **27,995 B a
|
||
frame — 328 KB/s of payload**, against the 37,403 B and 438 KB/s it is now:
|
||
**34% too big.** That is an encoder target, it is entirely inside this
|
||
project, and **it is the pessimistic reading of the lever**, because a
|
||
lighter container also decodes cheaper and the decode term falls with the
|
||
byte term.
|
||
|
||
**WHAT THIS IS NOT.** It is the **gate container**, which is deliberately the
|
||
heaviest thing the encoder emits — the span-heavy scsi container encoded to the
|
||
488 recipe, chosen so that every block mode and the newest span path are
|
||
exercised (check.sh's own note). The lighter `cpufit` family exists and was NOT
|
||
measured here: `15_bus_occupancy.py` refuses it, correctly, because the C68K
|
||
measurement in `tmp/c68k_frames.csv` belongs to the gate container and the
|
||
cross-check at the top of the tool is what licenses every number under it.
|
||
**Re-deriving 59.7 against a lighter container needs that harness re-run first**,
|
||
and until it is, "34% too big" is a statement about the heaviest container and
|
||
not about the project.
|
||
|
||
---
|
||
|
||
## 60. The re-encode bundle: the container agrees with the medium, and two encoder levers turn out not to be levers (session 28)
|
||
|
||
**Emulated** — MAME 0.277, `x68000 -exp1 cz6bs1 -ramsize 2M/6M`, plus px68k's
|
||
C68K core in `tools/bench/c68k`. **No real hardware ran.** `./tools/bench/
|
||
check.sh` was ALL GREEN before this session and is ALL GREEN after it, including
|
||
a full re-encode of the gate container.
|
||
|
||
ROADMAP's re-encode bundle was four items collected under P2 because they shared
|
||
one re-measurement. All four are now closed, and **two of them close as
|
||
negatives** — which is the more useful half of the session.
|
||
|
||
### 60.1 DLX5: the container is laid out in sectors, and the disc now moves exactly the records
|
||
|
||
`tools/encoder/encode.py` pads every record up to **512 bytes** instead of 4,
|
||
and pads the scene header so the frame stream itself starts on a sector
|
||
boundary. `tools/encoder/dlx.py` reads it as `DLX5`; `rec_align` is a property
|
||
of the container version and `record_lengths()` is the one place the rule is
|
||
applied, replacing four hand-copied `4 + n + (-(4+n) % 4)` expressions in the
|
||
analysis tools.
|
||
|
||
**Why 4 was not enough** is 58.3 option C and 59.4: a SCSI target answers in
|
||
512 B blocks, 117 of 120 DLX4 records started part way into one, and
|
||
`sc_in_data` **refuses** a windowed read when the data phase belongs to the
|
||
DMAC, because a channel writes a contiguous run and cannot drop the 300 bytes in
|
||
front of a record. Windowed PIO absorbed that for free and does not survive the
|
||
move to the channel.
|
||
|
||
| on the gate container | DLX4 | **DLX5** |
|
||
|---|---:|---:|
|
||
| records starting on a sector boundary | 3/120 | **120/120** |
|
||
| bytes off the disc, for 4,488,577 B of record | 4,548,608 (+1.34%) | **4,510,208 (+0.48%)** |
|
||
| bytes into the ring | 4,488,588 | **4,510,208** |
|
||
| clocks of window or bounce copy | 0 (PIO only) / +5 clk/B (DMA) | **0, and a channel can run it** |
|
||
| largest record | 40,984 B | 41,472 B — a 256 KB ring still holds it 6 times |
|
||
|
||
**The disc figure and the ring figure are now the same number, and check.sh
|
||
gates on that identity** rather than on two constants. Both are read out of the
|
||
container at check time: the old gate had 4,488,588 and 4,548,608 written into
|
||
it as literals, and the re-encode went red on both — correctly, because the
|
||
container had changed and the expectation had not. **A gate whose expected value
|
||
is a literal tests the literal.**
|
||
|
||
### 60.2 The consumer had to be told, and the failure was a byte-exact wrong answer
|
||
|
||
`stream.s` released the ring up to the last byte it *read*, rounded to 4. Under
|
||
DLX5 that strands up to 511 bytes of pad per record, and the ring's own audit
|
||
caught it on frame 0: `RD_PTR MISMATCH: decoder released 00040F08, record ends
|
||
00041000`. The release now rounds to `RECALN` (geom.i), which is the record
|
||
alignment the container guarantees, and is correct only because the ring base is
|
||
`RECALN`-aligned too — stated where the constant is defined rather than assumed.
|
||
|
||
**This is the shape of the whole item.** Sector alignment is not a tidier
|
||
version of 4-byte alignment; it is a contract with *two* sides, and the producer
|
||
side alone would have drifted the free-space arithmetic by up to 511 B a frame
|
||
with every frame still decoding pixel-exact.
|
||
|
||
### 60.3 Reserved black at index 0 — the letterbox is true black, and it costs 0.04 dB
|
||
|
||
23.4, open since session 5, was the other half of P2. `VQ.scene_palette` now
|
||
quantises the picture into **255 entries and reserves index 0 as (0,0,0)**;
|
||
`dlxload.pack_palette` gives it `I = 0` by its own minimum-squared-error rule
|
||
with no special case, so 23.3's "the bars sit at RGB (4,4,4)" goes away for
|
||
free. GVRAM cleared to zero displays entry 0, and a free mediancut palette put a
|
||
real image colour there — on 00020 f0001, `(206,192,176)`, used by 210 image
|
||
pixels.
|
||
|
||
Black is **reserved, not withheld**: the mapper may still spend index 0 on
|
||
genuinely black pixels. What the reservation buys is that index 0 is black
|
||
whatever the scene contains, which is what the letterbox needs and what a free
|
||
palette cannot promise. Measured cost on the Singe window: **28.92 dB → 28.88 dB
|
||
at `--spans need`**, and the palette ceiling is unmoved at 31.32 dB.
|
||
|
||
### 60.4 `--spans all` as the default: MEASURED, and the recommendation is REFUSED
|
||
|
||
E2 has been "a recommendation, not a measurement" since 43.6.1. It is now a
|
||
measurement, on the DLX5 container, 120 frames, the gate recipe
|
||
(`--kbps 280 --span-kbps 488`):
|
||
|
||
| | KB/s incl. audio | PSNR | frames that miss the 12 fps deadline |
|
||
|---|---:|---:|---:|
|
||
| `--spans need` (shipped default) | **267.9** | 28.88 | 2/120 |
|
||
| `--spans all` | 448.2 | 29.07 | 1/120 |
|
||
|
||
**+67% of the wire, for +0.19 dB and one frame of 120.** Session 27 established
|
||
that the frame affords 6.69 clocks a byte and that a dual-address byte costs 9,
|
||
so the container's problem is that it is too big; spending 180 KB/s to buy a
|
||
fifth of a decibel is the wrong direction, and `need` stays the default.
|
||
|
||
**The item is closed, and the answer is no.** The gate container keeps
|
||
`--spans all` — it is a fixture chosen to exercise the newest path in the
|
||
decoder, not a shipping recipe, and check.sh says so.
|
||
|
||
### 60.5 Joint span/lam selection: implemented, measured, and it is a NO-OP for a reason
|
||
|
||
E3 (39.3 item 5) asked for span selection to be re-derived jointly with `lam`
|
||
instead of greedily after it. The argument is sound on paper: the span pass
|
||
*removes* the block payload of every block it covers, so the frame lands under
|
||
its byte allowance and the blocks that were **not** spanned were priced at a
|
||
`lam` chosen as if those bytes were still needed.
|
||
|
||
`ratectl._refit_joint` (`--joint-spans`) hands the freed bytes back to the lam
|
||
search and re-spans, to a fixed point or two rounds. A round is kept only if the
|
||
frame still fits both ceilings it was already fitting, so `lam` can only fall
|
||
and the un-spanned blocks can only improve.
|
||
|
||
**It emits byte-identical containers.** All four cells of `{need, all} x {greedy,
|
||
joint}` produce two md5s, not four — and so does `--rc-floor open`:
|
||
|
||
| | lam, 120 frames | container |
|
||
|---|---|---|
|
||
| `need` greedy / joint | 10.0 min, median, p90, max | **identical** |
|
||
| `all` greedy / joint | 10.0 min, median, p90, max | **identical** |
|
||
| `all --rc-floor open` greedy / joint | 1.0 min, median, p90, max | **identical** |
|
||
|
||
**The reason is 44.3, and it is structural rather than incidental: `lam` never
|
||
leaves its floor on any frame, at either floor the encoder offers.** The block
|
||
coder at the profile floor already lands under the per-frame byte budget, so
|
||
there is nothing for a joint re-search to spend the freed bytes on. E3 is not a
|
||
lever, and it is not a lever for the same reason `--kbps` and the bucket are
|
||
not: **the byte side of this encoder is governed by the span pass and by `mu`.**
|
||
|
||
The code stays, defaulted off, because the measurement is the finding and a
|
||
future container that moved `lam` off its floor would make the question live
|
||
again. That is 44's pattern kept deliberately: ask whether the lever is loaded
|
||
before pulling it.
|
||
|
||
### 60.6 The apparatus lied, silently, and only a byte comparison caught it
|
||
|
||
**MAME 0.277 served a compressed CHD's own file bytes as sector data.** On the
|
||
DLX5 volume, every `READ(10)` reported success and returned the wrong bytes: the
|
||
destination buffer after a 4,096 B read at LBA 0 was **byte-for-byte the first
|
||
4,096 bytes of `dlxdisk.chd`, starting `MComprHD`** — the CHD file header — while
|
||
`chdman verify` reported both SHA1s correct.
|
||
|
||
Isolated by experiment, and the trigger is the image's **content**:
|
||
|
||
| volume | compression | result |
|
||
|---|---|---|
|
||
| DLX4 gate, 8,768 sectors | default (lzma/zlib/huff/flac) | **byte-exact** |
|
||
| DLX5 gate, 8,809 sectors | default | CHD header served as LBA 0 |
|
||
| DLX5 gate, padded to 9,072 sectors, 16x63 geometry | default | same failure |
|
||
| DLX5 gate, **truncated to 8,768 sectors** — the working length | default | same failure |
|
||
| DLX5 gate | `-c zlib` alone | same failure |
|
||
| DLX5 gate | **`-c none`** | **byte-exact** |
|
||
|
||
So it is not size, not geometry and not the codec; compression decides it and
|
||
uncompressed is sound. **The MAME-side cause is not diagnosed** and is not
|
||
claimed here. `tools/bench/mkvol.sh` now builds the volume `-c none`, at 4.5 MB
|
||
in `tmp/` against 1.6 MB, with the measurement written next to the flag.
|
||
|
||
**What is worth keeping is not the workaround.** The failure was *silent at the
|
||
transport layer* — the SPC handshake completed, the phase sequence was correct,
|
||
the byte count was right, and `scsi_read` returned 0. The only thing in the tree
|
||
that could see it was `tools/bench/scsi.lua` comparing every delivered byte
|
||
against the host's copy of the same image. A transport gate that checked status
|
||
and length would have been green all session, and FINDINGS 58's "byte-exact
|
||
against the host's copy" would have been the load-bearing phrase nobody noticed.
|
||
|
||
### 60.7 What the bundle did to the budget: almost nothing, which is the result
|
||
|
||
The re-measurement the four items shared, on the rebuilt gate container, with
|
||
`tmp/c68k_frames.csv` regenerated by the same `check.sh` run:
|
||
|
||
| | before (DLX4, session 27) | **after (DLX5, session 28)** |
|
||
|---|---:|---:|
|
||
| decoder, MEASURED on C68K | 68.5% of the frame | **68.6%** |
|
||
| delivered bytes a frame | 37,403 | **37,585** (the pad is delivered, so it is charged) |
|
||
| headroom after decode + best-case audio | 6.74 clk/B | **6.69 clk/B** |
|
||
| single address held, `W`=5, total | 92.2% | **92.4%** |
|
||
| dual address held, `W`=9, total | 110.2% | **110.4%** |
|
||
| E7's byte target at the 9 clk/B floor | 328 KB/s, 34% too big | **327 KB/s, 35% too big** |
|
||
| B1's zero-prefill delivery requirement | 451.4 KB/s | **453.6 KB/s** |
|
||
| scene header (P1/53.5, 22_scene_load) | 6,164 B | **6,656 B** |
|
||
|
||
**Every conclusion in 59.7 survives unchanged**, which is what a precondition is
|
||
supposed to do: it removes an obstacle without moving the arithmetic. The two
|
||
numbers that did move are both the alignment pad being paid honestly —
|
||
`15_bus_occupancy.py` now charges the **padded** record rather than the payload,
|
||
because the disc carries the pad whether or not a frame contains it.
|
||
|
||
### 60.8 Two caches that would have tested the wrong artefact
|
||
|
||
`H.build` acquired an option this session (`reserve_black`), and two tools cache
|
||
its output in a pickle **keyed on the frames directory alone** —
|
||
`16_span_roundtrip.py`, which is the span container's round-trip gate, and
|
||
`13_cpu_ratectl.py`. A stale pickle would have let the gate round-trip a
|
||
container built from the old palette while the shipping encoder emitted the new
|
||
one: green, and testing an artefact that no longer exists. Both now store the
|
||
build parameters with the model and rebuild on a mismatch.
|
||
|
||
### 60.9 What is now true, and what P4a still owes
|
||
|
||
**M2's re-encode bundle is done and the container is what the DMA path needs.**
|
||
`dma_run.sh` still gates on the *refusal* — `WINDOWED DMA READ REFUSED, as it
|
||
must be` — which is now a negative control rather than a description of the
|
||
container, because the container no longer asks for a windowed read.
|
||
|
||
What is left of P4a is the wiring: putting the channel behind `ring.i`'s `XF_*`
|
||
mailbox in place of the PIO loop in `xfer.i`, which 59.4 blocked on exactly this
|
||
precondition. `xfer.i`'s sector arithmetic already degenerates correctly —
|
||
`SC_WSKIP` is 0 and `SC_WKEEP` is the whole record on every record — so what
|
||
changes is which loop moves the bytes, not what is asked for.
|
||
|
||
## 61. The decoder-free packed player fits the clocks the codec misses, and the codec cannot have the packing (session 29)
|
||
|
||
**Emulated.** MAME 0.277, `x68000 -bios ipl10`, and for the transport runs
|
||
`-exp1 cz6bs1` with a zero-filled `scsiexrom.bin`. No real hardware ran, and
|
||
47.4 — does buffer mode blank the layer — still wants a board.
|
||
|
||
FINDINGS 44.7 asked what a player with no codec at all would cost and answered
|
||
"it fits the clocks and dies on the medium". 46.5/47.1 found the off switch for
|
||
the 2:1 GVRAM tax, 47.2 built the packed layout and rendered it pixel-exactly on
|
||
both emulators, and **47.5 re-derived the budget on a cost model that has since
|
||
been replaced in full** — the transport was an unmeasured `c` when it was
|
||
written, and sessions 25b–28 measured it (58.2), bounded it (59.2) and priced
|
||
what a frame can afford against it (59.7/60.7). 47.6.1 filed the packed paint's
|
||
`movem` shape as an *assumption*; 47.6.2 said the DMAC "has not been near this";
|
||
47.6.4 left the codec's own survival under the layout untouched.
|
||
|
||
All four are answered here. `tools/analysis/29_packed_player.py` is the
|
||
arithmetic, `tools/bench/blit.s` V8/V9/V10 are the paint measurements, and
|
||
`src/player/dmagate.s` runs 4–6 are the transport ones.
|
||
|
||
### 61.1 The packed paint, MEASURED: it costs what the unpacked path pays to WRITE
|
||
|
||
`blit.s` V8 is V1 with one thing different — a row is 128 words instead of 256,
|
||
because R20 bit 11 lets a word carry two picture bytes — and `blit.lua` times it
|
||
in the same run as V1, V2 and V3, so it is quoted against numbers that have not
|
||
moved since session 9.
|
||
|
||
| variant | clk/frame | % of a 12 fps frame | what it moves |
|
||
|---|---:|---:|---|
|
||
| V1 unpacked `movem` blit | 446,286 | 53.6% | 96 KB read + 96 KB write |
|
||
| V2 byte-source expansion | 1,284,174 | 154.1% | 48 KB read + 96 KB write |
|
||
| V3 unpacked WRITE-ONLY floor | 225,789 | 27.1% | no source read at all |
|
||
| **V8 PACKED `movem` blit** | **227,553** | **27.3%** | **48 KB read + 48 KB write** |
|
||
|
||
All four put the same 49,152 pixels on screen. **V8 is 51.0% of V1 and 100.8% of
|
||
V3: the packed blit costs what the unpacked one pays to write alone, with its
|
||
source read thrown in free.** It is not exactly half of V1 because the 192-row
|
||
loop does not halve with the words — per word V1 is 9.080 clocks and V8 is
|
||
9.259, and the difference is the row loop amortised over half as many bursts.
|
||
|
||
V2 is worth keeping in view: it is the "send 1 byte a pixel and let the CPU
|
||
expand it" trade, and at 154.1% it is not a trade, which is why the unpacked
|
||
path has no cheap wire.
|
||
|
||
V8 is a **timing** variant and does not set bit 11. MAME's `gvram_w` carries no
|
||
timing in either arm, so the bit cannot move a cycle; what it moves is the
|
||
picture, and the picture is what `show_frame256_packed.lua` and `gvpack` already
|
||
verify pixel-exactly (47.2). The source is pre-interleaved host-side, which is
|
||
the honest half of the claim — the packing is an encoder-side transform, the
|
||
same argument 46.3 made for the text plane.
|
||
|
||
### 61.2 The channel writes GVRAM in buffer mode, and it walks the line stride itself
|
||
|
||
47.6.2 was right that nothing in this tree had ever pointed a channel at
|
||
`$C00000`. `dmagate.s` now does, three times, and `dma_run.sh` gates all three:
|
||
|
||
| run | R20 | result |
|
||
|---|---|---|
|
||
| **4. DMA → GVRAM, bus held, bit 11 SET** | `$0916` | **2,048 B byte-exact against the disc**; 1,024 landed in page 1 (high half), 1,024 in page 0 |
|
||
| **5. the SAME, bit 11 CLEAR** | `$0116` | **457 bytes lost, every one of them at an EVEN offset; not one ODD byte harmed** |
|
||
| **6. ARRAY CHAINED, 8 rows at the 1024 B stride** | `$0916` | **2,048 B byte-exact at eight separate row bases from ONE start**; `MAR` ended at `+7,424` |
|
||
|
||
In all three the discriminator reads the same as 59.1's: **`MTC` sampled by the
|
||
instruction after `START` is 0 of 2,048 and the CPU went round its wait loop
|
||
once** — the whole transfer happened between two instructions.
|
||
|
||
**Run 5 exists because the first cut of run 4 was a test that could not fail.**
|
||
It OR-ed bit 11 on top of whatever the IPL left, and the IPL leaves `$0B16`
|
||
(22.1) — bit 11 **already set**, and `COL = %11`, the 65,536-colour setup, which
|
||
writes whole words with or without it. It passed and proved nothing. Both runs
|
||
now write R20 outright and differ in exactly that bit.
|
||
|
||
**And the control's signature is a PLACE, not a count.** The obvious assertion —
|
||
"half the bytes must differ" — is wrong, and asserting it turned the gate red on
|
||
a correct run. In masked 256-colour mode `gvram_w` takes `data & 0x00ff` and
|
||
ignores `mem_mask`, so a byte written to an even address is never stored and the
|
||
high half keeps what it held; where the record is pad, the stale half matches the
|
||
disc by coincidence, and 567 of 1,024 did. What the mechanism says is that
|
||
**every odd byte survives and only even ones can be lost**, and that is what the
|
||
gate asserts.
|
||
|
||
Run 6 is the one that changes an architecture. A picture row is 256 B of a
|
||
1,024 B line stride, so a frame is **192 destinations, not one**, and a channel
|
||
writes a contiguous run — the same limitation that makes it refuse a windowed
|
||
read (58.3). 46.6 said "no stride for a DMAC to skip" about the bytes *within* a
|
||
row and never examined the rows. The MC68450 answers with sequential array
|
||
chaining, `src/player/dma.i` now programs it behind a `DM_BARV` mailbox that is
|
||
**zero unless a caller asks**, and the channel walked an 8-entry array on its
|
||
own. **The CPU does not restart the channel per row.**
|
||
|
||
### 61.3 The codec cannot have the packing — 47.6.4, closed
|
||
|
||
A 4×4 block owns four bytes at **stride 2** under the packed layout, because the
|
||
high bytes of its four words belong to the block 128 columns away. There are
|
||
exactly two ways a block decoder could live with that, and `blit.s` V9 and V10
|
||
are them:
|
||
|
||
| variant | clk/frame | % of frame | vs V4 |
|
||
|---|---:|---:|---:|
|
||
| V4 block order, UNPACKED (the shipping shape) | 637,971 | 76.6% | — |
|
||
| **V9 block order, PACKED, 16 `move.b` at stride 2** | **815,839** | **97.9%** | **+28%** |
|
||
| **V10 block order, PACKED, blocks PAIRED** | **320,456** | **38.5%** | **−50%** |
|
||
|
||
**V9 is 28% DEARER than V4, and buys nothing on the wire** — a codeword is
|
||
already one byte a pixel, so the packed layout has no bytes left to save a block
|
||
decoder. That route costs 177,868 clocks a frame for nothing.
|
||
|
||
**V10 halves the paint and pays for it in the mode map.** Pairing the block at
|
||
x with the block at x+128 gives V4's `movem` shape back, and a pair skips only
|
||
if *both* its blocks skip. On the gate container:
|
||
|
||
| | now | paired |
|
||
|---|---:|---:|
|
||
| SKIP | **66.3%** of blocks | **46.1%** of pairs |
|
||
| painted | **33.7%** | **53.9%** |
|
||
|
||
So pairing paints **1.60×** as many blocks for **0.50×** the paint per block:
|
||
**−20% on the clock, and about +60% on the BYTES**, because a coded block is
|
||
bytes in the container whether its half of the pair changed or not. E7 needs the
|
||
bytes **down 35%**.
|
||
|
||
**The packed layout is not an upgrade the codec can take. It is the thing you
|
||
get instead of the codec.**
|
||
|
||
### 61.4 The budget, re-derived on the measured model
|
||
|
||
Every cell below is CPU work plus transport plus best-case audio (10,417 clk,
|
||
1.25%, from the IPL ROM's own channel-3 setup — 21_iplrom_dmac.py). None of them
|
||
overlap: `buscost.DMA_OVERLAPS = False`.
|
||
|
||
| architecture | B/frame | W=5 | **W=9** | W=12 | W=16 | W=19 |
|
||
|---|---:|---:|---:|---:|---:|---:|
|
||
| CODEC, CPU-decoded (the shipping design) | 37,585 | 92.4% | **110.4%** | 124.0% | 142.0% | 155.5% |
|
||
| free / DMAC device→GVRAM / unpacked | 98,304 | 61.1% | 108.2% | 143.6% | 190.8% | 226.2% |
|
||
| **free / DMAC device→GVRAM / PACKED** | **49,152** | 31.6% | **55.2%** | 72.9% | 96.5% | 114.1% |
|
||
| free / CPU-painted / unpacked, 2 B/px wire | 98,304 | 113.8% | 161.0% | 196.4% | 243.5% | 278.9% |
|
||
| free / CPU-painted / unpacked, 1 B/px wire | 49,152 | 184.8% | 208.4% | 226.1% | 249.7% | 267.4% |
|
||
| **free / CPU-painted / PACKED** | **49,152** | 58.0% | **81.6%** | 99.3% | 122.9% | 140.6% |
|
||
|
||
**W=9 is the column that matters**, because 59.2 found the only configurations
|
||
this machine can be shown to run are dual-address, and a dual-address byte is a
|
||
4-clock read of the device plus a 5-clock write to memory. Everything left of it
|
||
is a hardware fact nobody here has (ROADMAP B3).
|
||
|
||
**At the floor the CODEC misses by 10.4% and the DECODER-FREE PACKED PLAYER FITS
|
||
WITH 45% TO SPARE.** Decoding 37,585 bytes costs 109% of a frame; not decoding
|
||
49,152 costs 54%.
|
||
|
||
That is not a small correction to 47.5. **44.7 said it in advance, on a cost
|
||
model that has since been thrown away: "the codec is not there to save CPU — it
|
||
is there to save the wire."** The measured model agrees and goes further: the
|
||
CPU side is not merely affordable without the codec, it is *strictly cheaper*.
|
||
|
||
### 61.5 So it is entirely a medium question, and the medium is the unmeasured one
|
||
|
||
| architecture | B/frame | KB/s | GB for 22.8 min |
|
||
|---|---:|---:|---:|
|
||
| CODEC, gate container | 37,585 | 440.4 | 0.62 |
|
||
| CODEC, `need` default | — | 267.9 | ~0.38 |
|
||
| CODEC, E7's target at the 9 clk/B floor | 27,924 | 327 | 0.46 |
|
||
| **decoder-free PACKED, either architecture** | **49,152** | **576.0** | **0.81** |
|
||
| decoder-free unpacked | 98,304 | 1,152.0 | 1.61 |
|
||
|
||
**576 KB/s, sustained, with no lever to pull.** A codec's bitrate is a lever; a
|
||
literal frame's is geometry, and no scene in the picture costs less than another.
|
||
ROADMAP B1 is unmeasured, and the 0.7–1.7 MB/s usually quoted for BlueSCSI on an
|
||
X68000 is **folklore with no published benchmark behind it**. 576 KB/s sits
|
||
inside that range, which is exactly the reason the range has to be measured
|
||
rather than cited: **a codec at 327 KB/s survives a slower answer and a literal
|
||
frame does not degrade, it drops.**
|
||
|
||
**B1 has therefore changed character.** It used to set how much headroom the
|
||
player has. It now decides which player exists.
|
||
|
||
### 61.6 And under MAME's blanking reading, the cheap architecture is the dark one
|
||
|
||
47.4's question is untouched by any of this and it still needs a board. What the
|
||
measured paint does is turn 48.3's range into numbers, and it exposes an
|
||
asymmetry nobody had stated: **R20 bit 11 only has to be set across the GVRAM
|
||
writes, and where those writes come from decides how long that is.**
|
||
|
||
| architecture | black interval | bit 11 set for |
|
||
|---|---:|---|
|
||
| free / DMAC device→GVRAM / unpacked | 60%..225% of a frame | the whole DMA |
|
||
| free / DMAC device→GVRAM / PACKED | 30%..113% of a frame | the whole DMA |
|
||
| free / CPU-painted / unpacked | 53.6% | the blit only |
|
||
| **free / CPU-painted / PACKED** | **27.3%** | **the blit only** |
|
||
|
||
The CPU-painted packed path has the smallest black window of any of them,
|
||
because its transport lands in RAM where bit 11 is irrelevant and only the
|
||
227,553-clock blit needs the bit. **The DMAC-direct path, which is cheaper in
|
||
clocks at every rung of the ladder, is the one that must hold the bit across its
|
||
whole transfer.** If MAME is right, the cheap architecture is the dark one and
|
||
the expensive one is merely dim.
|
||
|
||
Both are a strobe at the frame rate over the whole picture, and the packed layout
|
||
has no page to flip to: both 256-colour pages carry picture, which is the entire
|
||
point of it (48.3). If px68k is right, every number in 61.4 and 61.5 stands as
|
||
written. 48.1's prior — an assertion against a silence — has not moved.
|
||
|
||
### 61.7 What this does NOT establish
|
||
|
||
1. **No clock here is a transport clock.** MAME's DMAC runs on wall-clock
|
||
attotimes and models a held bus by halting the CPU (42.5), so runs 4–6 settle
|
||
which configurations *work* and not what one costs. Every `W` in 61.4 is
|
||
datasheet arithmetic except PIO, and the array-chain entry at 36 clocks is
|
||
Fig 4-25 sheet 1, not a measurement.
|
||
2. **One frame's worth of transport, not a stream.** Run 6 chained eight rows,
|
||
not 192, and nothing here ran a chained transfer back to back at 12 fps or
|
||
through `ring.i`. A decoder-free player has no ring at all in the DMAC-direct
|
||
form, which is a simplification this tree has not tested.
|
||
3. **The paint is a lower bound.** Every `blit.s` number is instruction cycles
|
||
against zero-wait-state memory; real GVRAM stalls the CPU and neither
|
||
emulator models it (`blit.lua`'s header, and 45's note on C68K).
|
||
4. **61.3's pairing cost is one container's.** 66.3% / 46.1% is the gate
|
||
container, deliberately the heaviest thing the encoder emits. A lighter
|
||
container has a different SKIP map and would pair differently — though not,
|
||
at that margin, differently enough to change the sign.
|
||
5. **No audio, no branching, no seek.** 61.4 charges the audio DMA and nothing
|
||
else. A decoder-free player still has to make 56's branch decisions and 51's
|
||
seeks, and at 576 KB/s it has less slack to make them in, not more.
|
||
|
||
### 61.8 Encoder work is parked (session 29, USER DECISION)
|
||
|
||
61.4 first closed with "nothing here is a reason to stop work on the codec — it
|
||
is the only branch that survives a slow answer to B1." **That does not survive
|
||
its own arithmetic and is withdrawn.**
|
||
|
||
It rested on 576 KB/s against **E7's 327 KB/s target, which does not exist.** The
|
||
codec that exists is **440 KB/s and 110.4% of a frame**, so the gap is **1.31x,
|
||
not 1.76x** — and reaching 327 needs a 35% byte reduction after 60.4 and 60.5
|
||
measured two of the encoder's three byte levers and found neither is a lever. The
|
||
reward on success is a design at ~100% of the frame with no margin, which is where
|
||
55.2% already is.
|
||
|
||
The two branches are not symmetric, and counting what each NEEDS is the argument:
|
||
|
||
| | facts it needs |
|
||
|---|---|
|
||
| **decoder-free packed** | buffer mode does not blank (B2); medium clears 576 KB/s (B1) |
|
||
| **codec** | E7 succeeds (unproven, two levers dead); **and** medium clears 327; **and** it ships at ~100% of a frame |
|
||
|
||
**So E7 and E4 are parked, and C1 with them.** E4 is included deliberately: it is
|
||
`H.build`'s k-means, it builds **VQ codebooks**, and a literal player has no VQ.
|
||
|
||
**What is NOT parked is the codec itself.** It stays on disk, gated by check.sh,
|
||
and nothing is built on it. 48.1's prior leans against packing — MAME asserts the
|
||
blanking semantic twice and deliberately, px68k's display path never reads the
|
||
bit at all, and an assertion against a silence is not a tie — and 48.3 stands:
|
||
if buffer mode blanks there is no version of the packed player that is merely
|
||
expensive. In that branch the codec is the only path left. **Keeping a working
|
||
decoder is inventory; building on it is work, and the work waits on B2.**
|
||
|
||
### 61.9 And the picture is BETTER — the codec is capped below the thing replacing it
|
||
|
||
Asked while scoping what a packed player would take, and it is the finding that
|
||
makes the branch worth building rather than merely affordable. The numbers are
|
||
`18_text_plane_16col.py`'s, over the same 120-frame window everything else is
|
||
measured on; 46.3 computed them to give the *text plane* something to be scored
|
||
against and never turned them on the 256-colour path itself.
|
||
|
||
| | PSNR vs the 24-bit source |
|
||
|---|---:|
|
||
| shipping container (the codec, as it ships) | **29.19 dB** |
|
||
| 256 colours, SCENE palette — **the codec's CEILING** | **31.33 dB** |
|
||
| 256 colours, **PER-FRAME** palette | **34.08 dB** |
|
||
|
||
**The middle row is a ceiling, not a rival.** Every codeword the codec emits is
|
||
an index *into* `vq.scene_palette`, so no bitrate takes it past 31.33 dB; it
|
||
spends 440 KB/s getting within 2.14 dB of it.
|
||
|
||
**A literal frame has no codebooks, so nothing forces the scene palette on it,
|
||
and per-frame palettes become legal.** 46.3 stated the constraint in as many
|
||
words while arguing the *other* side — "the 256-colour path cannot do this: its
|
||
palette is shared scene-wide because the codec's codebooks are indices INTO it."
|
||
Remove the codec and the constraint goes with it.
|
||
|
||
**So the decoder-free packed player is +4.89 dB on the shipping container and
|
||
+2.75 dB past a ceiling the codec cannot cross** — while costing 55.2% of a frame
|
||
against 110.4%. It is not a quality compromise bought with clocks. It is better
|
||
on both, and the whole of its cost is on the wire.
|
||
|
||
What the per-frame palette costs:
|
||
|
||
| | |
|
||
|---|---|
|
||
| wire | 512 B/frame → 49,664 B, **582.0 KB/s (+1.0%)** |
|
||
| clocks | ~2,370, **0.28% of a frame** if the CPU writes it — DERIVED from V8's measured 9.259 clk/word in the same `movem` shape |
|
||
| colours | **254, not 256** — the packed layout spends index 0 on the transparency key and puts black at 255 (47.2, `prep_frame.py --pack-transparent`), where `--reserve-black` spends one. A reserved entry measured 0.04 dB in 60.3, so this is noise against +4.89 |
|
||
|
||
**Two things this does NOT settle.**
|
||
|
||
1. **Whether a channel can write `$E82000`.** If the palette registers take a
|
||
byte-wide DMA the way GVRAM does in buffer mode, the palette is a 193rd array
|
||
entry and costs the CPU nothing at all — one channel start still paints a
|
||
whole frame. 61.2 only ever pointed a channel at GVRAM. **This is the next
|
||
probe, and it is the same shape as the ones that worked.**
|
||
2. **The quantiser is PIL's MEDIANCUT, not this project's.** `vq.scene_palette`
|
||
and `H.build` are what would actually ship the palette. The direction is
|
||
measured and the magnitude is about right; re-derive the per-frame figure
|
||
against the real builder before quoting it as the player's number.
|
||
|
||
## 62. One channel start paints a whole frame — the palette registers take the DMA (session 30)
|
||
|
||
**Emulated.** MAME 0.277, `x68000 -bios ipl10`, `-exp1 cz6bs1` with a
|
||
zero-filled `scsiexrom.bin`. No real hardware ran. 47.4 — does buffer mode blank
|
||
the layer — is still the board question and is still open, and **62.4 adds a
|
||
second board question this run created.**
|
||
|
||
61.9 left the decoder-free packed player one open structural item, ROADMAP K1:
|
||
a frame is a picture *and* a palette, and 61.2 had only ever pointed a channel
|
||
at GVRAM. If the palette registers at `$E82000` take a byte-wide DMA the way
|
||
GVRAM does in buffer mode, the palette is a 193rd array-chain entry and the
|
||
whole video path is **one channel start a frame**; if they do not, the CPU
|
||
writes 256 words a frame (61.9 derives ~2,370 clocks, 0.28% of a frame) and the
|
||
architecture stands anyway. It is the difference between cheap and free, and it
|
||
took one run to know which.
|
||
|
||
`src/player/dmagate.s` runs 7–9, gated by `tools/bench/dma_run.sh`, which
|
||
`check.sh` runs. **The answer is free, in this model.**
|
||
|
||
| run | what it does | result |
|
||
|---|---|---|
|
||
| **7. DMA → `$E82000`, bus held** | 512 B off the disc into the whole graphic palette | **byte-exact in 256 register words**, read back out of the registers by the 68000 |
|
||
| **8. the SAME transfer aimed at RAM** | the attribution control: `$2C000` instead | **byte-exact at `$2C000`, and 256 of 256 palette words still read the poison** |
|
||
| **9. ONE array-chained start** | 7 entries: the palette, then six picture rows at the 1,024 B line stride | **2,048 B byte-exact across BOTH kinds of destination**, `MAR` ended at `+5,376` |
|
||
|
||
In all three the discriminator reads as 59.1's and 61.2's do: **`MTC` sampled by
|
||
the instruction after `START` is 0 and the CPU went round its wait loop once** —
|
||
the transfer happened between two instructions, with the 68000 not executing.
|
||
|
||
### 62.1 The destination was POISONED first, because "it matches" has been a weak claim all along
|
||
|
||
Runs 4–6 wrote into RAM that was zero and GVRAM that was stale, against a record
|
||
that is mostly pad. A destination that could already hold the right answer
|
||
cannot distinguish a channel that wrote from a channel that did nothing — which
|
||
is run 4's could-not-fail trap wearing different clothes (61.2).
|
||
|
||
So `dg_poison` fills the palette with word *i* = `$A500|i` before each palette
|
||
run, written by the 68000 and read back by it. The host does not assume the
|
||
poison is a discriminator, it **counts**: `PALETTE POISON IS A DISCRIMINATOR:
|
||
511 of 512 positions differ from the disc's bytes`, and the gate refuses a run
|
||
where fewer than 500 do. One position coincides, and the pass does not rest on
|
||
it.
|
||
|
||
### 62.2 The control is an ATTRIBUTION control, not a mechanism one — and it says so
|
||
|
||
Run 5 could point at a mode bit; there is no mode bit here. What had to be
|
||
excluded is that run 7's palette held the disc's bytes for some reason other
|
||
than the channel having put them there — a readback that aliases somewhere else,
|
||
the SPC's own path touching the registers, the poison never having landed at
|
||
all. So run 8 is the same transfer with **one thing different, the destination
|
||
address**, and it makes two claims from one run:
|
||
|
||
- the disc's bytes appear at `$2C000`, so the transfer happened;
|
||
- **the palette still reads poison in all 256 words**, so what reached
|
||
`$E82000` in run 7 was decided by the channel's `MAR`.
|
||
|
||
The second claim is also the positive half: it shows the CPU's own writes reach
|
||
the registers the host reads back, so the readback path is not the thing under
|
||
test.
|
||
|
||
### 62.3 The 193rd entry is literal, and the array is SCENE-constant
|
||
|
||
Run 9 is the one that changes the architecture, and it is not "the palette
|
||
works" — it is that **one array chain crosses two kinds of destination**: device
|
||
registers at `$E82000` and video RAM at `$C14000`, in one start, with the CPU
|
||
halted from the first byte to the last. A frame is that shape with 192 row
|
||
entries instead of six.
|
||
|
||
**And the array does not have to be rebuilt per frame.** The row bases are
|
||
`$C00000 + row * 1024` and they do not change: the packed layout spends *both*
|
||
256-colour pages — page 0 is the low byte of a word and page 1 the high byte
|
||
(47.2, and 46.1's page masks) — so there is no spare page to flip into and no
|
||
alternate set of destinations to alternate between. The 193-entry array
|
||
(1,158 B) is built once at scene setup and started once a frame.
|
||
|
||
**What is left on the CPU in the video path is therefore the channel start and
|
||
the READ(10) that fetches the record — and neither is priced here.** The command
|
||
issue is already inside the transport's own account (58.2); the start is about a
|
||
dozen register writes and is DERIVED as small rather than measured. Do not quote
|
||
"no per-frame CPU work" without that sentence attached: it is no per-frame
|
||
*paint* work.
|
||
|
||
### 62.4 What this does NOT settle, and it is a NEW BOARD QUESTION
|
||
|
||
**MAME cannot discriminate here, and the reason is in its source.** The graphic
|
||
palette is not modelled as a register file at all: `x68k.cpp:817` maps
|
||
`$E82000-$E821FF` to `palette_device::read16/write16`, `emupal.cpp:417` forwards
|
||
to `memory_array::write16`, and `memarray.h:75` is a plain `COMBINE_DATA`. That
|
||
is **RAM that honours `mem_mask`** — so a byte write lands in its half by
|
||
construction, and a green run says *nothing in the model forbids it* rather than
|
||
*the board takes it*.
|
||
|
||
This is a different kind of bound from 61.2's. GVRAM has a real handler with a
|
||
real 256-colour arm, which is why run 5 could find a mechanism to fail on; the
|
||
palette has no handler to be wrong about. **What a byte write to a real
|
||
X68000 palette register does is UNMEASURED and this project has no figure for
|
||
it** — not folklore, not an estimate, an absence. It goes on the hardware list
|
||
as **B4**, and it is cheap: write `$A5` to `$E82000` and `$5A` to `$E82001` from
|
||
the CPU on a real board and read the word back.
|
||
|
||
**The blast radius if B4 comes back negative is small and known**: the palette
|
||
leaves the chain, the CPU writes 256 words a frame at 61.9's derived 0.28% of a
|
||
frame, and every other claim in 61 and 62 stands. Run 9's crossing would still
|
||
have to be re-asked, because it would no longer have a device-register end.
|
||
|
||
### 62.5 The chain's ORDER is a free choice with a visible consequence, and it is not decided
|
||
|
||
Run 9 puts the palette FIRST. It could as easily be 193rd, which is what 61.9
|
||
called it. The two are not equivalent on screen and neither is obviously right:
|
||
|
||
- **palette first** — the 192 rows of the *previous* frame are displayed under
|
||
the *new* palette until each is overwritten;
|
||
- **palette last** — the new frame's rows are displayed under the *old* palette
|
||
until the chain reaches the end.
|
||
|
||
The mismatch lasts one paint either way. Which is less visible depends on how
|
||
much the palette moves between consecutive frames, which is a property of the
|
||
encoder K2 has not been written yet, and the whole question is **moot if buffer
|
||
mode blanks the layer** (47.4/B2) because nothing is displayed during the paint
|
||
at all. **Filed, not answered.** It is named here so that the choice in the
|
||
final player is a decision rather than an accident of which run happened first.
|