Put the frame in a container with no decoder, and find the palette is not free

ROADMAP K2. DLXP1: a 49,664 B record that is 97 sectors exactly, no index and
no length word, because a packed record's length is geometry rather than
content. 582.0 KB/s, which is what FINDINGS 61.9 predicted to the tenth, and it
encodes in 3.3 s because there is no k-means in it.

px68k's own x68k/gvram.c renders the container's bytes index-exact with the
harness computing no interleave -- the only test that can catch an encoder whose
byte order is wrong, since a container round-trips against its own inverse
either way. Both negative controls fail as they must.

The picture is re-derived against this project's builder rather than PIL's
(34.05 dB against 61.9's 34.08) and the GGGGGRRRRRBBBBBI word is charged for the
first time in this tree: 0.53 dB, on every row, so it moves no comparison.

What the control found is the finding. A packed container on a SCENE palette
lands exactly on the codec's ceiling, so the whole +2.31 dB is the per-frame
palette and nothing else -- and 231 of 256 entries change every frame, which
makes a mismatched paint 12.8 dB worse than the correct pairing, on screen for
roughly half of every frame slot if buffer mode does not blank. So B2 now
decides which packed CONTAINER ships, not only which player. The fallback is
already a flag: --scene-palette --no-palette is 30.79 dB, zero churn, 576.0 KB/s
and still +2.07 dB on the shipping codec.

62.5 is priced and is a wash: palette first 20.32 dB, palette last 20.33.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6

tools/bench/check.sh

@@ -662,4 +662,45 @@ python3 tools/analysis/29_packed_player.py "$DLX" > tmp/packed_player.log 2>&1 \
 grep -aE "SKIP block PAIRS|free / DMAC->GVRAM / PACKED|^  CODEC, gate" \
   tmp/packed_player.log
 
+echo "--- session 31: the PACKED container, and the picture re-derived (FINDINGS 63) ---"
+# ROADMAP K2. The container is REBUILT every run rather than reused when
+# present, the way the codec's gate container is: a packed encode is 3 seconds
+# because there is no k-means in it, so there is no reason to let a stale file
+# stand between the encoder and the gate.
+python3 tools/encoder/pack.py tmp/fr_singe tmp/packed_singe.dlxp \
+  --nframes "$NF" > tmp/pack_encode.log 2>&1 \
+  || { cat tmp/pack_encode.log; exit 1; }
+grep -aE "^  (record|wire)" tmp/pack_encode.log
+# WHAT IS GATED. Four format invariants that a DMA channel cannot check for
+# itself -- it copies bytes and has no opinion about them (FINDINGS 62) -- and
+# the three quality claims FINDINGS 61.9 rests the whole packed branch on:
+#   round-trip, sector geometry, index 0 unused, palette words agree
+#   packed > the shipping codec / > the codec's CEILING / > a SCENE-palette control
+# A tree where any of the last three flipped has a different answer to ROADMAP K
+# and should say so out loud rather than let the branch keep building.
+python3 tools/analysis/30_packed_container.py tmp/packed_singe.dlxp \
+  --codec "$DLX" > tmp/packed_container.log 2>&1 \
+  || { cat tmp/packed_container.log; exit 1; }
+sed -n '/^PSNR/,$p' tmp/packed_container.log | grep -aE "RGB888|CODEC|CEILING|CONTROL|PER-FRAME|packed vs|worth|costs"
+# And the container's OWN BYTES through px68k's real gvram.c, with the harness
+# computing no interleave -- the only test that can catch an encoder whose byte
+# order is wrong, because the container round-trips against its own inverse
+# either way. Same skip-not-fail rule as the C68K stage: px68k is not in here.
+if [ -f "$PX68K/x68k/gvram.c" ]; then
+  make -s -C tools/bench/gvpack PX68K="$PX68K"
+  # Frame 0 carries the two negative controls; the other two are there because
+  # one frame rendering does not say the container's 120th record is placed
+  # right, and record placement is arithmetic this format has no index to check.
+  python3 tools/bench/gvpack/verify_dlxp.py tmp/packed_singe.dlxp 0 --controls || exit 1
+  for f in $((NF / 2)) $((NF - 1)); do
+    # NOT piped into head: `set -e` reads a pipeline's status from its LAST
+    # command, so a piped verifier that failed would be reported by head's zero.
+    python3 tools/bench/gvpack/verify_dlxp.py tmp/packed_singe.dlxp "$f" \
+      > "tmp/dlxp_f$f.log" 2>&1 || { cat "tmp/dlxp_f$f.log"; exit 1; }
+    head -1 "tmp/dlxp_f$f.log"
+  done
+else
+  echo "  SKIPPED: no px68k at $PX68K -- the container's bytes were not rendered"
+fi
+
 echo "ALL GREEN"

tools/bench/dlxload.py

@@ -36,11 +36,18 @@ def pack_palette(d):
     which is the point: the verifier and the loader must agree or a colour bug
     reads as a decoder bug.
 
-    Returns (palette bytes 256x2 big-endian, index of the darkest entry).  The
-    encoder does not yet reserve a black entry (docs/STATUS.md, encoder gaps),
-    so the letterbox gets the closest thing to black the palette has.
+    Returns (palette bytes 256x2 big-endian, index of the darkest entry, and the
+    RGB888 the hardware actually RENDERS from those words).  The encoder does not
+    reserve a black entry in the CODEC container (docs/STATUS.md, encoder gaps),
+    so the letterbox gets the closest thing to black the palette has; the PACKED
+    container does reserve one (tools/encoder/dlxp.py, index 255).
+
+    `d` is a DLX container OR a bare (256,3) uint8 palette.  The packed path has
+    no codebooks and so no DLX object to carry a palette on, and this had to stay
+    the ONE copy of the GRB555+I maths -- the verifier, the loader and now the
+    packed encoder all have to agree or a colour bug reads as a decoder bug.
     """
-    pal = d.pal.astype(int)
+    pal = (d if isinstance(d, np.ndarray) else d.pal).astype(int)
     p6 = lambda v: ((v << 2) | (v >> 4)) & 0xFF
     f = pal >> 3
     render = lambda I: p6((f << 1) | I[:, None])

tools/bench/gvpack/harness.c

@@ -97,6 +97,17 @@ int main(int argc, char **argv)
     int iw = (d[4] << 8) | d[5], ih = (d[6] << 8) | d[7];
     if (n < (size_t)(8 + 768 + iw * ih)) { fprintf(stderr, "short blob\n"); return 2; }
     const BYTE *pix = d + 8 + 768;
+    /* 'DLXQ' -- the blob is PRE-INTERLEAVED: `pix` is already the bytes a DLXP1
+     * record carries, in GVRAM order.  The ordinary 'DLXR' path computes the
+     * interleave here, which tests the LAYOUT; this path tests the CONTAINER,
+     * by writing its bytes verbatim and asking px68k's own gvram.c what they
+     * display as.  The two agreeing is the claim ROADMAP K2 has to make: the
+     * encoder's byte order is the one 47.2 verified as a picture. */
+    int prepacked = (d[3] == 'Q');
+    if (prepacked && !packed) {
+        fprintf(stderr, "a pre-interleaved blob has no unpacked form\n");
+        return 2;
+    }
     int yoff = (H - ih) / 2;
     const BYTE BLACK = 255;
 
@@ -121,8 +132,23 @@ int main(int argc, char **argv)
         for (int y = 0; y < H; y++) {
             DWORD base = 0xC00000 + y * 1024;
             for (int i = 128; i < 512; i++) wr16(base + i * 2, 0);
-            for (int i = 0; i < 128; i++)
-                wr16(base + i * 2, (WORD)((PIX(y, i + 128) << 8) | PIX(y, i)));
+            for (int i = 0; i < 128; i++) {
+                WORD w;
+                if (prepacked) {
+                    /* The letterbox rows are STATIC SETUP and are not in a
+                     * record (dlxp.py), so they are supplied here, the way a
+                     * player's scene setup supplies them: both halves BLACK. */
+                    if (y < yoff || y >= yoff + ih)
+                        w = (WORD)((BLACK << 8) | BLACK);
+                    else {
+                        const BYTE *row = pix + (y - yoff) * iw;
+                        w = (WORD)((row[i * 2] << 8) | row[i * 2 + 1]);
+                    }
+                } else {
+                    w = (WORD)((PIX(y, i + 128) << 8) | PIX(y, i));
+                }
+                wr16(base + i * 2, w);
+            }
         }
         if (!keepbuf) set_r20(R20_DISPLAY);       /* back to display */
     } else {

tools/bench/gvpack/verify_dlxp.py

@@ -0,0 +1,87 @@
+#!/usr/bin/env python3
+"""The PACKED CONTAINER's own bytes, through px68k's real GVRAM code.
+
+    python3 tools/bench/gvpack/verify_dlxp.py [packed.dlxp] [frame] [--controls]
+
+`verify_gvpack.py` checks the LAYOUT: it hands the harness a picture and lets
+the harness compute the interleave, so what it proves is that FINDINGS 47.2's
+scheme renders.  This checks the CONTAINER: it writes a DLXP1 record's bytes
+into GVRAM VERBATIM -- no interleave computed anywhere in the harness -- and
+asks px68k what they display as.  That is the only way to test a format whose
+whole design is that nothing parses it (dlxp.py): if the encoder's byte order
+were wrong, every check upstream of the display would still pass, because the
+container round-trips against its own inverse.
+
+It is the same second-emulator argument tools/bench/c68k makes for cycles: the
+address decode, the R20 bit-11 write path, the page-byte selection, the scroll
+wrap and the index-0 transparency test are px68k's own `x68k/gvram.c`.
+
+Two negative controls, because a test that cannot fail proves nothing, and both
+are mechanisms this container depends on rather than decoration:
+  --nobuffer   R20 bit 11 CLEAR -- the high byte of every word is masked away,
+               so page 1 (columns 128..255) never gets written
+  --noscroll   page 1 unscrolled -- its storage sits under the wrong columns
+"""
+import os, struct, subprocess, sys
+sys.path.insert(0, "tools/encoder")
+import numpy as np
+from dlxp import DLXP
+
+args = [x for x in sys.argv[1:] if not x.startswith("--")]
+path = args[0] if args else "tmp/packed_singe.dlxp"
+frame = int(args[1]) if len(args) > 1 else 0
+controls = "--controls" in sys.argv
+
+d = DLXP(path)
+_, pic = d._split(frame)
+blob = b"DLXQ" + struct.pack(">HH", d.W, d.H) + b"\0" * 768 + pic
+open("tmp/dlxp_gvpack.bin", "wb").write(blob)
+
+BIN = "tools/bench/gvpack/gvpack"
+if not os.path.exists(BIN):
+    sys.exit(f"{BIN} not built -- make -C tools/bench/gvpack PX68K=...")
+
+
+def run(extra=None):
+    cmd = [BIN, "tmp/dlxp_gvpack.bin", "tmp/dlxp_gvpack.raw", "--packed", "0x02"]
+    if extra:
+        cmd.append(extra)
+    subprocess.run(cmd, check=True, stderr=subprocess.DEVNULL)
+    g = np.frombuffer(open("tmp/dlxp_gvpack.raw", "rb").read(), np.uint8)
+    return g.reshape(256, 256)
+
+
+want = d.indices(frame)
+yoff = (256 - d.H) // 2
+g = run()
+act = g[yoff:yoff + d.H]
+
+fail = []
+if not np.array_equal(act, want):
+    bad = act != want
+    fail.append(f"{bad.sum()} px differ (left half {bad[:, :128].sum()}, "
+                f"right half {bad[:, 128:].sum()})")
+bars = np.concatenate([g[:yoff], g[yoff + d.H:]])
+if bars.size and (bars != 255).any():
+    fail.append(f"letterbox is not index 255: {(bars != 255).sum()} px")
+if (act == 0).any():
+    fail.append(f"index 0 reached the screen: {(act == 0).sum()} px")
+
+if controls and not fail:
+    for flag, why in (("--nobuffer", "R20 bit 11 clear"),
+                      ("--noscroll", "page 1 unscrolled")):
+        c = run(flag)[yoff:yoff + d.H]
+        n = int((c != want).sum())
+        print(f"    control {flag:<11s} ({why}): {n:,} px differ"
+              + ("" if n else "   <-- IT DID NOT FAIL"))
+        if not n:
+            fail.append(f"control {flag} passed -- the test cannot fail on it")
+
+for x in fail:
+    print("FAIL " + x)
+if fail:
+    sys.exit(1)
+print(f"OK  {os.path.basename(path)} frame {frame}: px68k's own gvram.c renders "
+      f"the container's {d.pic_bytes:,} bytes index-exact over {d.W}x{d.H},")
+print(f"    letterbox on the reserved black, and the transparency key never "
+      f"reaches the screen. The harness computed no interleave.")