A real 256x256 CRTC mode, derived not recalled; palette ceiling was 2 dB low

Session 3 left the harness on the IPL's 768x512 text timing because no CRTC
values had been derived and guessing them was the failure mode to avoid. This
derives them from MAME 0.277's divisor ladder instead, and the derivation is
self-checking: the 256-wide mode runs at div 6 against the 768 mode's div 2, so
htotal is exactly 1104/3 = 368 dots and every horizontal register divides by
three with no remainder. Only the blanking split rounds. Verified by snapshot:
native 256x512, active area pixel-exact, x=512 wrap gone.

Two things fell out that change numbers elsewhere:

- The palette's shared LSB I must be chosen per entry, not hardcoded to 1.
  Doing so lifts the display ceiling from 38.85 to 40.81 dB and is the only way
  to reach true black at all, since pal6bit(1) = 4. 102 of 256 entries want
  I = 0, so this is not a corner case. Supersedes FINDINGS 22.4; scsi has ~2 dB
  more headroom than that section claimed. The encoder does not do this yet.

- Letterboxing costs a palette entry: GVRAM cleared to zero shows entry 0, and
  a free mediancut palette puts a real image colour there. 255 colours plus a
  reserved black, via prep_frame.py --reserve-black.

MAME's graphics double-scan is phase-shifted one raster line (it halves the
absolute scanline and vbegin is odd), which produced a false failure before it
was understood; the regression test now asserts the shifted pairing explicitly.

Still Lua-side. No 68000 instruction has drawn a pixel; the 38% blit estimate
remains unvalidated. What this buys is a defined geometry for the decoder to
write into: 256 words per row, 1024-byte stride, rows 32..223.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6

docs/FINDINGS.md

@@ -659,6 +659,11 @@ frame is pixel-identical, not merely close. GVRAM line stride is confirmed as
 512 words = 1024 bytes, matching `HARDWARE.md`.
 
 ### 22.4 A new quality ceiling: the 15-bit palette costs 38.88 dB
+> **Superseded by 23.3.** The 38.88 dB figure assumed the shared LSB `I` is
+> always 1. Choosing `I` per palette entry by minimum error lifts the ceiling to
+> **40.81 dB** on the same frame. The conclusion below ("`scsi` is close to
+> display-transparent") is therefore weaker than stated — there is ~2 dB more
+> headroom than this section claims.
 Section 3 called the 256-colour palettised frame "the real quality ceiling".
 That was measured in 24-bit RGB. The hardware palette only stores 5 bits per
 channel plus a shared LSB, so there is a **second** quantisation below it:
@@ -683,3 +688,107 @@ text timing, so the layer repeats at exactly x=512. This is correct hardware
 behaviour, not a bug. The player sets its own CRTC mode and the wrap disappears.
 No CRTC timing table has been written yet — the harness deliberately keeps the
 IPL's timing so that no invented CRTC values are in play.
+
+
+## 23. A real CRTC mode: 256x192 inside 256x256 (session 4)
+
+Session 3's harness borrowed the IPL's 768x512 text timing and invented no CRTC
+values, which is why the frame repeated at x=512 (22.5). This session derived a
+real 256x256 mode table from MAME 0.277 source and verified it by snapshot.
+Table: `tools/bench/crtc_mode.lua`. Regression test: `tools/bench/verify_frame256.py`.
+
+![256x256 mode](images/x68k_256x256_mode.png)
+
+*Left: palettised source. Right: the emulated X68000's native 256x512 raster —
+256 dots wide, 512 scanlines carrying 256 double-scanned graphics rows, with the
+192-row picture letterboxed in true black.*
+
+### 23.1 The table, and why it needed no guessing
+`refresh_mode()` in `x68k_crtc.cpp` selects the dot clock as
+`(reg20 bit4 ? 69.55199MHz : 38.86363MHz) / div`, with `div` from a ladder keyed
+on `reg20 & 0x1f`. Three entries matter:
+
+| `reg20 & 0x1f` | div | dot clock | mode |
+|---|---|---|---|
+| `0x16` | 2 | 34.776 MHz | IPL's 768 wide, 31.5kHz |
+| `0x11` | 3 | 23.184 MHz | 512 wide, 31.5kHz |
+| `0x10` | 6 | 11.592 MHz | **256 wide, 31.5kHz, graphics double-scanned** |
+
+The IPL's `R00 = 137` gives `m_htotal = (137+1)*8 = 1104` dots, and
+`34.776e6 / 1104 = 31500.0 Hz` **exactly**. Holding the same line rate at div 6
+needs `11.592e6 / 31500 = 368` dots `= 46` chars, so `R00 = 45`.
+
+`368 = 1104/3` exactly, so every horizontal register is the 768-mode value
+divided by three, and the active window divides without remainder:
+`(124-28)/3 = 32` chars `= 256` dots. **No horizontal value was recalled or
+estimated.** Only the blanking split rounds: the 768 mode is sync/back/front =
+14/14/14 chars, `/3 = 4.67` each, and the closest integer triple summing to
+`46-32 = 14` is 5/5/4.
+
+| reg | value | meaning |
+|---|---|---|
+| R00 | 45 | H total, 46 chars = 368 dots -> 31500.0 Hz |
+| R01 | 5 | H sync end (3.45 us) |
+| R02 | 10 | H display begin -> `hbegin = 81` |
+| R03 | 42 | H display end -> `hend = 336`; inclusive width `336-81+1 = 256` |
+| R04 | 567 | V total, 568 scanlines -> 55.46 Hz |
+| R05 | 5 | V sync end |
+| R06 | 40 | V display begin -> `vbegin = 41` |
+| R07 | 552 | V display end -> 512 scanlines = 256 double-scanned rows |
+| R08 | 27 | H sync adjust (MAME stores it and never reads it) |
+| R20 | `0x0110` | display (not buffer), 256-colour, 31.5kHz, 256x256 |
+
+**The vertical registers are NOT halved**, which is the one thing that looks
+wrong and is not. The CRTC still generates a 568-line raster; "256 lines" is a
+graphics-layer double-scan applied in `draw_gfx()` (`x68k_v.cpp:401`), not a
+change to the raster. Halving R04 would ask the monitor for 110 Hz. MAME emits
+a `visarea larger then reg[20]` logerror for this; it is cosmetic.
+
+Total blanking time is identical to the 768 mode (112 dots at 11.592 MHz =
+336 dots at 34.776 MHz = 9.66 us), which is the property a real monitor cares
+about — so this table should be safe on hardware, though that is untested.
+
+### 23.2 MAME's double-scan is phase-shifted by one raster line
+`get_gfx_pixel()` indexes `m_gfxbitmap.pix(scanline / divisor, pixel)` using the
+**absolute** scanline, and `vbegin = 41` is odd. So in the native 256x512
+snapshot the identical row pairs are `(1,2), (3,4), ...` and row 0 is a lone
+half-line. Even rows are graphics rows 0..255. This cost a false failure before
+it was understood; the regression test now asserts the shifted pairing
+explicitly so a change in MAME's behaviour is visible rather than confusing.
+
+### 23.3 The shared LSB `I` must be chosen per palette entry — worth 1.96 dB
+Session 3's `pack()` hardcoded `I = 1`. That is not free: `I` is shared by all
+three channels and each renders as `pal6bit((field << 1) | I)`, so with `I = 1`
+the darkest reachable value is `pal6bit(1) = 4`, and **true black does not
+exist**. Choosing `I` per entry to minimise summed squared error over R,G,B:
+
+| rule | ceiling vs 24-bit palettised (00020 f0001) | entries with I=0 |
+|---|---|---|
+| `I = 1` fixed (session 3) | 38.85 dB | 0 |
+| `I` per entry, min squared error | **40.81 dB** | 102 / 256 |
+
+Nearly **2 dB for free**, and 102 of 256 entries want `I = 0` — this is not a
+corner case. It supersedes the ceiling in 22.4 and means `scsi` has about 2 dB
+more headroom before it hits the display than that section claimed.
+
+The encoder does not yet do this. `tools/encoder/` still emits 24-bit palettes
+and the packing happens Lua-side; whatever eventually writes X68000 palette
+words must use the per-entry rule.
+
+### 23.4 Letterboxing requires a reserved black palette entry
+GVRAM cleared to zero displays **palette entry 0**, and a free mediancut palette
+puts a real image colour there — on 00020 f0001 it was `(206,192,176)`, used by
+210 image pixels, so it cannot simply be repurposed. A 256x192 picture in a
+256x256 mode has 64 blank rows, so the palette must be built with **255 colours
+plus a reserved black at index 0** (`prep_frame.py --reserve-black`). Combined
+with 23.3, entry 0 also needs `I = 0` or the bars sit at RGB (4,4,4).
+
+Cost: one of 256 entries. Measured quality effect: none visible — the ceiling
+figure in 23.3 is already measured on the 255-colour palette.
+
+### 23.5 What is still not proven
+GVRAM was again filled from Lua. **No 68000 instruction has drawn a pixel yet**,
+and the 38% full-frame blit estimate underpinning the CPU budget remains
+unvalidated. What this section adds is that the *target mode* is now real, so
+68000 code has a defined geometry to write into: 256 words per visible row, a
+1024-byte line stride, and rows 32..223 of a 256-row page.

docs/STATUS.md

@@ -1,4 +1,4 @@
-# Status & next-session handoff — end of session 3 (2026-08-23)
+# Status & next-session handoff — session 4 in progress (2026-08-23)
 
 ## Decisions locked
 
@@ -72,6 +72,27 @@ rate-distortion curve, not two codecs.
    the working-setup section below. They cost ~1.5 h of wall clock and a wedged
    CPU core, and one of them was hit again this session.
 
+## What session 4 settled (in progress)
+
+1. **A real 256x256 CRTC mode exists and is verified.** `crtc_mode.lua`, derived
+   from `x68k_crtc.cpp`'s divisor ladder rather than recalled — the derivation is
+   self-checking (368 = 1104/3 exactly, so the horizontal registers divide by
+   three with no remainder). Snapshot is native 256x512, active area pixel-exact,
+   letterbox true black. FINDINGS 23. The x=512 wrap of FINDINGS 22.5 is gone.
+2. **The palette ceiling was wrong by 2 dB, in our favour.** The shared LSB `I`
+   must be chosen **per palette entry**, not hardcoded to 1. Doing so lifts the
+   display ceiling from 38.85 to **40.81 dB** and is the only way to get true
+   black at all (`pal6bit(1) = 4`). 102 of 256 entries want `I = 0`. This
+   supersedes FINDINGS 22.4 and gives `scsi` ~2 dB more headroom than believed.
+   **The encoder does not do this yet** — see the encoder-gaps list.
+3. **Letterboxing costs one palette entry.** 255 colours + a reserved black at
+   index 0, with `I = 0` on it. `prep_frame.py --reserve-black`. FINDINGS 23.4.
+4. **MAME's graphics double-scan is phase-shifted one raster line** — pairs are
+   (1,2),(3,4),..., not (0,1), because `get_gfx_pixel` halves the *absolute*
+   scanline and `vbegin = 41` is odd. Cost a false failure. FINDINGS 23.2.
+
+---
+
 ## What session 2 settled
 
 1. **The critical-path question is answered.** "Does VQ soften Bluth's linework
@@ -113,6 +134,10 @@ multi-byte fields are **big-endian** so the 68000 reads them with a plain `move`
   builds a lam-ladder per frame; it needs hooking up and validating.
 - **Payload is deliberately NOT entropy-coded** — deflate decode does not fit in
   the 68000's frame budget (FINDINGS 17.2). Do not "optimise" this later.
+- **Palette packing is not implemented in the encoder.** It still emits 24-bit
+  palettes; the X68000 word packing happens Lua-side. Whatever writes real
+  palette words must pick `I` per entry by minimum squared error (FINDINGS 23.3,
+  worth 1.96 dB) and reserve index 0 as black with `I = 0` (FINDINGS 23.4).
 - Codebooks are per-scene and rebuilt from scratch; no inter-scene reuse.
 - `_paint` is a Python per-block loop — fine for prototyping, slow for a full
   disc encode. Vectorise before the 224-stream run.
@@ -177,7 +202,7 @@ functional models, not timing-accurate; a KB/s figure from MAME measures the
 emulator's scheduler. `docs/BENCHMARK.md` covers the three-tier approach
 (MAME validates the path, derivation bounds it, real hardware settles it).
 
-## Display path — VERIFIED (session 3). CPU path — still unproven.
+## Display path — VERIFIED (session 3), in a real mode (session 4). CPU path — still unproven.
 
 The first real frame is on screen: `docs/images/x68k_first_frame_compare.png`.
 
@@ -228,19 +253,19 @@ SDL_VIDEODRIVER=dummy mame x68000 -bios ipl10 -video soft -window \
    **Vectorise `_paint` before this run** — it is a Python per-block loop.
 2. **68000 decoder skeleton.** Parse `DLX1`, expand codebooks to word-per-pixel,
    blit SKIP/V1/V4/RAW. Measure real cycles with the existing MAME Lua harness.
-   **Now unblocked** — the display path is verified (FINDINGS 22) and
-   `tools/bench/show_frame.lua` gives a known-good reference image to diff the
-   68000's output against. Validates the 38% full-frame blit estimate that the
-   whole CPU budget rests on. Still needs a real CRTC mode table for 256x256;
-   the harness deliberately borrows the IPL's timing and invents nothing.
-2a. **CRTC mode table for 256x192-in-256x256.** Prerequisite for (2) and the
-   smallest well-defined unit of work available right now. Needs real R00-R08
-   timing values. **Do not write these from memory** — session 3 lost time to
-   exactly that failure mode on the video registers. Derive them from the CRTC
-   dividers in `x68k_crtc.cpp` (`m_reg[20] & 0x1f` selects the dot-clock
-   divisor; the IPL's `0x16` gives /2 off the 69MHz clock), or lift a known-good
-   set from a real X68000 title and verify by snapshot. The harness makes this
-   cheap to iterate: change values, snapshot, look.
+   **Fully unblocked** — the display path is verified (FINDINGS 22) AND the
+   target CRTC mode is now real (FINDINGS 23), so 68000 code has a defined
+   geometry to write into: 256 words per row, 1024-byte line stride, picture in
+   rows 32..223 of a 256-row page. `tools/bench/show_frame256.lua` gives a
+   known-good reference image to diff the 68000's output against. This is what
+   validates the 38% full-frame blit estimate the whole CPU budget rests on.
+   **This is now the top priority** — it is the only remaining unknown that can
+   still invalidate the design.
+2a. ~~CRTC mode table for 256x192-in-256x256.~~ **DONE, session 4.** Derived from
+   the CRTC divisor ladder (not recalled), verified by snapshot, pixel-exact.
+   `tools/bench/crtc_mode.lua`; write-up in FINDINGS 23; regression test
+   `tools/bench/verify_frame256.py`. Untested on real hardware, but the blanking
+   timing is identical to the IPL's 768 mode, which is what a monitor cares about.
 
 3. **Wire rate control into `encode.py`.** No longer a blocker (FINDINGS 21), but
    it is what gives a deterministic ceiling over content not yet measured, which
@@ -268,14 +293,24 @@ SDL_VIDEODRIVER=dummy mame x68000 -bios ipl10 -video soft -window \
 ## Not yet started
 - **Any 68000 player code.** `src/player/` is still empty. The display path is
   proven, but proven *from Lua* — no 68000 instruction has yet drawn a pixel.
-- **A real CRTC mode table.** The harness deliberately borrows the IPL's 768x512
-  text timing and invents no CRTC values, which is why the frame repeats at
-  x=512 (FINDINGS 22.5). A 256x256 mode needs real R00-R08 values, and those
-  must be derived or measured, NOT recalled from memory — see the note below.
 - ADPCM audio extraction/encoding
 - Disk image packaging
 - Game logic (scene branching, input windows, death clips)
 
+## Reproducing the 256x256 mode result (session 4)
+
+```
+python3 tools/encoder/extract.py 00020 tmp/fr_00020 12 crop
+python3 tools/bench/prep_frame.py tmp/fr_00020 tmp/frame256.bin 0 --reserve-black
+mkdir -p tmp/snap256 && cd tmp && SDL_VIDEODRIVER=dummy timeout -k 5 90 mame x68000 \
+  -bios ipl10 -video soft -window -sound none -nothrottle -plugins \
+  -autoboot_script ../tools/bench/show_frame256.lua \
+  -snapshot_directory ./snap256 -snapview native -seconds_to_run 6
+cd .. && python3 tools/bench/verify_frame256.py
+```
+Exits non-zero on any drift. Expected: `256x512 native, double-scan exact,
+active 256x192 pixel-exact, letterbox true black`, ceiling 40.81 dB.
+
 ## Reproducing the display result
 
 ```