Commit Graph
9 Commits
Author SHA1 Message Date
prosolis 34f9ee341d A real 256x256 CRTC mode, derived not recalled; palette ceiling was 2 dB low
Session 3 left the harness on the IPL's 768x512 text timing because no CRTC
values had been derived and guessing them was the failure mode to avoid. This
derives them from MAME 0.277's divisor ladder instead, and the derivation is
self-checking: the 256-wide mode runs at div 6 against the 768 mode's div 2, so
htotal is exactly 1104/3 = 368 dots and every horizontal register divides by
three with no remainder. Only the blanking split rounds. Verified by snapshot:
native 256x512, active area pixel-exact, x=512 wrap gone.

Two things fell out that change numbers elsewhere:

- The palette's shared LSB I must be chosen per entry, not hardcoded to 1.
  Doing so lifts the display ceiling from 38.85 to 40.81 dB and is the only way
  to reach true black at all, since pal6bit(1) = 4. 102 of 256 entries want
  I = 0, so this is not a corner case. Supersedes FINDINGS 22.4; scsi has ~2 dB
  more headroom than that section claimed. The encoder does not do this yet.

- Letterboxing costs a palette entry: GVRAM cleared to zero shows entry 0, and
  a free mediancut palette puts a real image colour there. 255 colours plus a
  reserved black, via prep_frame.py --reserve-black.

MAME's graphics double-scan is phase-shifted one raster line (it halves the
absolute scanline and vbegin is odd), which produced a false failure before it
was understood; the regression test now asserts the shifted pairing explicitly.

Still Lua-side. No 68000 instruction has drawn a pixel; the 38% blit estimate
remains unvalidated. What this buys is a defined geometry for the decoder to
write into: 256 words per row, 1024-byte stride, rows 32..223.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
2026-08-23 13:27:17 -07:00
prosolis 966417893b Scope the display-path result: no 68000 code drew that frame
The session-3 milestone was written in a way that reads as "the port renders",
which it does not. The video hardware is genuinely emulated and the output is
bit-exact, but GVRAM was filled by a MAME Lua script poking emulated memory,
not by 68000 instructions.

The distinction is load-bearing: Lua writes cost zero 68000 cycles, so nothing
here tests whether the CPU can decode and blit inside 833,333 cycles. The 38%
full-frame blit estimate that the entire budget rests on is still unvalidated.

Only the "Not yet started" list carried this caveat, which was too buried for
a claim this easy to over-read.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
2026-08-23 13:18:22 -07:00
prosolis b322e84cd4 Get a real Dragon's Lair frame onto the emulated X68000
First pixels on an actual X68000 screen. Everything up to now was Python-side
or a headless -video none run, which cannot snapshot at all.

The blocker was not the video controller. The IPL leaves CRTC R20 = 0x0B16,
and bit 11 is "G-VRAM set to buffer", which makes MAME's draw_gfx() return
early. GVRAM writes still land and read back correctly while the layer is
invisible, so six attempts at $E82400/$E82500/$E82600 all rendered black with
every register holding the value I intended.

Two more facts, both confirmed against MAME 0.277 source rather than assumed:

- $E8E001 monitor contrast is left at 14 by the IPL, scaling all output to
  93.3%. The player must set it to 15. Contrast 0 blanks the screen, which is
  a free fade-to-black for scene transitions.
- The palette word is GGGGGRRRRRBBBBBI with a shared LSB, expanded as
  pal6bit((field<<1)|I). With contrast at 15 the render is pixel-exact, not
  merely close, which also confirms the 1024-byte GVRAM line stride.

That exactness gives a new quality ceiling: the 15-bit+I palette alone costs
38.88 dB against the 24-bit palettised source, the same order as the scsi
profile's own codec error. scsi is close to display-transparent on hardware,
which bounds how much further it is worth raising.

Unblocks next step 2, the 68000 decoder skeleton, which now has a known-good
reference image to diff against.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
2026-08-23 13:12:27 -07:00
prosolis 64cd1ffd72 Handoff: reconcile docs and tooling with the corrections made this session
Session 2 reversed several of its own conclusions. The docs are append-only, so
a reader could land on a superseded section and act on it. This pass makes the
repo internally consistent.

Defects found and fixed in STATUS.md:
- claimed "Hybrid VQ with k=1024: no" as the answer to the linework question,
  directly contradicting FINDINGS 14, which rejected k=1024. Both profiles are
  k=256.
- malformed profile table (six column separators, five columns).
- next-steps list had two items numbered 3 and listed the full-disc survey
  twice.
- the disk-benchmark section still read CRITICAL-PATH with "if SCSI sustains
  >=800 KB/s, ship pixel-exact". That was written while the bandwidth figure
  was misread as 4 MB/s. At 4 Mbps pixel-exact needs 92-97% of the pipe and is
  not available, and the ring-buffer result means the design no longer hangs on
  the benchmark at all. Rewritten with what it IS still worth doing: confirming
  the 4 Mbps provenance, and confirming DMA is used rather than PIO.

FINDINGS now carries supersession blockquotes on 5, 8, 11, 17 and 18 pointing
at the sections that correct them. 18 is the dangerous one -- its peak-vs-
sustained test is reversed by 21 -- so it is marked DO NOT ACT ON THIS SECTION
while noting the per-frame data itself remains valid.

profile_gen.py had the same problem in code: it defaulted to the superseded
peak sizing and returned lam=25 where the docs say lam=10. The buffered test is
now the default and peak sizing is behind --size-for-peak as a bound only. A
tool that contradicts the findings is worse than no tool.

Also preserves the five measurement scripts that produced this session's
numbers as tools/analysis/05-09, following the session 1 precedent, and adds an
"explicitly abandoned -- do not re-propose" list to STATUS covering entropy
coding, k=1024 codebooks and flat 4x4 VQ.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
2026-08-23 12:28:41 -07:00
prosolis fb8a1462b0 Correct the peak analysis: both profiles fit 4 Mbps with zero prefill
The previous commit warned that scsi "does not fit 4 Mbps" because a frame
peaked at 96.4% of the sustained rate. That was the wrong test, and the user
was right to push back on it. It compared instantaneous frame demand against a
sustained rate as if they had to match frame-by-frame; the disk keeps filling
during the frame, and a ring buffer absorbs any shortfall.

Correct test is cumulative demand vs cumulative supply. Simulated on the real
per-frame sizes at a constant 488 KB/s fill:

  required prefill = 0.0 KB for EVERY scene at both profiles

Fill delivers 40.69 KB per frame time; only one measured frame (00146,
42.10 KB) exceeds it and the next frame recovers it. A 256 KB buffer -- 12.5%
of RAM -- carries ~1 second of stall tolerance, orders of magnitude more than
an SD-backed seek needs. scsi at lam=10 stands; the hardest sampled scene runs
313 KB/s mean, 64% of the pipe.

Also carries through a consequence of SD-backed deployment that session 1 noted
as a caveat but never applied: with BlueSCSI/SCSI2SD the sustained rate is a
bus-limited CONSTANT, not an average over variable seek latency, so there is no
long tail to leave margin for and we can size much closer to the ceiling.
Conservative margins here were cargo-culted from a constraint this deployment
does not have. The SASI/SCSI split remains correct because it is about bus
protocol, not media.

Rate control drops from load-bearing to insurance: intra-scene peaks are a
non-problem, but only 4 clips of 1.2-1.7s out of 224 streams have been measured
and 00146 already runs 23% hotter than 00020. The full-disc survey is now the
highest-value measurement, ahead of wiring up ratectl.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
2026-08-23 12:23:08 -07:00
prosolis 48e912de8b Size against 4 Mbps: peaks break the scsi profile; DMA steal is not free
User clarified the bandwidth figure is 4 Mbps (488 KB/s), not 4 MB/s -- ~8x
tighter than the previous commit reasoned against. Two consequences, plus a
correction to session 1.

1. The scsi profile committed in f0f2f80 DOES NOT FIT. Its mean is a
   comfortable 52% of the pipe but it PEAKS at 96.4% (470.8 KB/s on scene
   00020), and a frame that arrives late is a dropped frame, not a slow one.
   Peak/mean is 1.4-1.9x even on 1.2-1.7s clips. Sizing a real-time stream on
   the mean was the error. Flagged in STATUS rather than silently retuned,
   because the fix is rate control, not a lower lam.

   This promotes ratectl.py -- written in session 2, never wired into
   encode.py -- from a loose end to the highest-value work in the repo. It is
   worth a full step on the quality ladder (lam=25 -> lam=10, +0.7/+1.2 dB)
   because it allows sizing for the mean instead of the peak.

2. Pixel-exact is off the table at this bandwidth: lam=0 needs 92-97% of the
   pipe. The previous commit's "if SCSI sustains >=800 KB/s, ship transparent"
   conclusion only applies at roughly double the user's figure.

3. FINDINGS 5 said that because transfers are DMA, streaming "costs essentially
   no CPU" and the 68000 is "nearly idle". That is wrong. The HD63450 steals
   ~8 clocks per 16-bit word: 10-20% of the machine at the rates the profiles
   now use, on top of a 38% full-frame blit. Bandwidth and CPU are one budget.

Adds tools/encoder/profile_gen.py, which derives lam FROM a bandwidth figure
(accounting for audio, peak/mean and DMA steal) instead of reading it off the
knee of the RD curve, and docs/BENCHMARK.md covering how to actually measure
the subsystem -- including why MAME cannot answer the bandwidth question and
would be the same class of error as the FINDINGS 4 traps.

The 4 Mbps figure is user-supplied and its provenance is not recorded; every
profile now hangs off it.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
2026-08-23 12:12:34 -07:00
prosolis f0f2f807a4 Raise both quality profiles; rule out entropy coding on CPU grounds
The profiles shipped in e4062ed were set far too low. 45 KB/s (sasi) and
75 KB/s (scsi) are 12% and 7% of the respective folklore bus figures. They had
been read off the knee of the rate-distortion curve and then presented as
though bandwidth-derived, which they were not.

Raised to sasi 110 KB/s (lam=60) and scsi 280 KB/s (lam=10) -- 35% and 28%
utilisation. scsi is now within 0.52 dB of the palette ceiling on scene 00020.

Checking the CPU side, which nobody had done for the decode path, produces a
second and more important result. Against the 833k cycle/frame budget at 12fps:

  full-frame blit, every frame     319k   38%   affordable
  LZ4/LZSS decode ~30KB/frame      450k   54%
  deflate decode  ~30KB/frame     1800k  216%   infeasible

So raising the VQ bitrate is nearly free -- RAW, the mode that dominates at
high rate, is the cheapest mode to blit -- but entropy coding is not viable at
all. That demotes the "247 KB/s lossless changed-spans+deflate" figure from
FINDINGS 8 to a compression upper bound rather than a shippable design, and
removes entropy coding from the roadmap. VQ is the right architecture precisely
because its decode is a table copy.

Also confirms the architecture unifies: the hybrid at lam=0 lands within 3% of
the purpose-built lossless coder, so there is no separate lossless path.

Consequence for planning: the blocked disk benchmark is now critical-path, not
optional. If SCSI sustains >=800 KB/s the correct scsi profile is lam=0 --
pixel-exact video at ~450 KB/s and 38% CPU. Whether this port ships transparent
or lossy on SCSI is waiting on one measurement.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
2026-08-23 12:06:52 -07:00
prosolis e4062ed294 Session 2: hybrid VQ codec, two quality profiles, three corrections
Answers session 1's critical-path question. Flat 4x4 VQ at k=256 was prototyped
and REJECTED by eye: Dirk's face disintegrates and ink outlines break into
4-pixel stair-steps. The 256-colour palettised frame is excellent, so the
palette was never the problem -- block VQ was.

Replaced it with a Cinepak-style hybrid: each 4x4 block is SKIP, one 4x4
codeword, four 2x2 codewords, or RAW literal pixels, chosen per block by
rate-distortion. The RAW escape makes lam=0 pixel-exact (measured 0.00 dB loss),
so the quality knob spans lossless to heavily-compressed in one bitstream.

Per the user's decision, ships TWO quality profiles from that one codec, one
decoder and one bitstream -- only the rate knob differs:
  sasi  45 KB/s  lam=300  34.8 dB   stock 10MHz ACE/EXPERT
  scsi  75 KB/s  lam=100  35.9 dB   Super/XVI or CZ-6BS1

Three corrections to earlier numbers:

1. Session 1's "183 KB/s at 12fps" was a bad extrapolation. Halving the
   framerate does not halve the bitrate -- decimation roughly doubles the
   per-frame delta. Re-measured directly: 340 KB/s for session 1's own RLE,
   247 KB/s for changed-spans+deflate. The lossless floor is 319 MB.

2. A FOURTH false-good result, same family as the three in FINDINGS 4:
   k=1024 codebooks appeared to buy +2.4 dB free, because the rate model
   charged 1 byte for a 10-bit index. Charging the true cost reverses the
   verdict -- k=256 wins at every matched bitrate, and by 5 dB at the low end
   where the SASI profile lives. k=256 ships.

3. Stream inventory: the ~3-5MB clips are 1.2-1.7s, not ~60s, and some 60s
   streams are menus, not content. Any survey must classify before averaging.

Also cleared both candidate sources for the game-logic layer: the SNES project
is MIT and DirkSimple is zlib, so the arcade scene graph can be imported and
the two transcriptions diffed against each other.

Encoder is working end-to-end: extract.py -> vq/vq_hybrid/ratectl -> encode.py,
emitting a big-endian DLX1 container the 68000 can parse with plain moves.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
2026-08-23 11:56:08 -07:00
reala-misaki 65112b9305 Session 1: hardware research, content measurement, codec decision, MAME harness
Verified GVRAM is one word-access per pixel in ALL color modes; chose 256-color
256x192 with movem.l bursts (page 1 sacrificed as double-buffer).

Measured 8 scenes from the Blu-ray source: blit costs under 8% of the 12fps
cycle budget, so I/O is the bottleneck, not CPU. Naive delta+RLE reaches only
3.2:1 (365 KB/s, 470MB) -> decision to use 4x4 vector quantization (~30 KB/s).

"Shot on twos" assumption failed: the transfer has zero duplicate frames, so
12fps requires explicit decimation.

Documents three false measurement results and their root causes (per-frame
Floyd-Steinberg dithering, temporal denoise, exact-match dedupe on noisy source).

MAME Lua injection harness works and is reusable for cycle-cost measurement;
the IOCS _B_READ disk benchmark is blocked returning -1.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
2026-08-23 11:23:49 -07:00