From cb05e77a425564c033a86c8c6dc4d1c1a563d60d Mon Sep 17 00:00:00 2001 From: prosolis <5590409+prosolis@users.noreply.github.com> Date: Sun, 23 Aug 2026 15:21:12 -0700 Subject: [PATCH] Handoff: reconcile the docs with a decoder that works and does not fit Session 7 handoff. check.sh green end to end, including both new stages. STATUS now leads with the cost-aware mode decision, and the stale statements that session 7 overturned are struck rather than left to be re-read as current: - next-steps item 1 still announced "implement both display paths and pick per frame" as a settled answer. Withdrawn, with the reason, and the part that did hold up (report the distribution, not the mean) kept. - the profile table's "machine" column reads as a CPU claim and is not one. It is the bus. Both profiles target the same stock 10 MHz 68000 and neither fits; the Super has SCSI at 10 MHz. - the display-path section stopped at "68000 code copied a frame". It parses one now, and the blit figures are no longer the display-path budget. - the green-light section still described a 2-minute, three-stage check, and did not warn that 10_pathmix_drift.py is SUPPOSED to exit non-zero. Sharpened the one trap in the next session's first step: SKIP is not a per-block constant. It costs 13.25 cycles inside an all-SKIP header byte and ~45 in a mixed one, so its price depends on its neighbours and a per-block lagrangian cannot see that. Picking either number is wrong in a different direction; the budget check can use the exact clustered cost function while the per-block term only has to rank modes. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 --- docs/STATUS.md | 75 ++++++++++++++++++++++++++++++++++++++------------ 1 file changed, 57 insertions(+), 18 deletions(-) diff --git a/docs/STATUS.md b/docs/STATUS.md index bd7dc40..b81ecda 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -26,9 +26,19 @@ worth 4 bytes, with no idea what it costs to draw. **The work, in order:** 1. **Add a cycle term to the mode decision.** `decide()` already builds a cost - matrix of `error + lam * bytes` per mode per block; add `+ mu * cycles`, with - the cycles vector `[13, 300, 448, 400]` measured in FINDINGS 28.2. One extra - row of arithmetic in a function that is already vectorised. + matrix of `error + lam * bytes` per mode per block; add `+ mu * cycles`, + with the per-mode cycles measured in FINDINGS 28.2. + **SKIP is not a constant and this is the one trap here.** A SKIP block costs + 13.25 cycles when all four blocks in its header byte are SKIP (one `tst.b` + clears the group) and ~45 when it sits in a mixed byte — so SKIP's price + depends on its *neighbours*, which a per-block lagrangian cannot see. Do not + pick one number and move on: 45 overcharges clustered SKIPs and pushes the + encoder away from the mode that saves cycles, 13.25 undercharges isolated + ones and lets frames overrun. The way out is that the **budget check does not + have to use the same cost function as the mode decision** — score frames with + the exact clustered cost (`cycles()` in `tools/analysis/11_cpu_budget.py`, + validated to 1 point against the 68000) and let the bisection converge on + that, while the per-block term uses a constant purely to *rank* modes. 2. **Then bisect `mu` per frame against the 833,333-cycle budget**, exactly as session 6 bisects `lam` against the byte budget. The machinery is already there and already gated: `ratectl.encode_rate_controlled` is frame-driven and @@ -43,6 +53,7 @@ worth 4 bytes, with no idea what it costs to draw. cliff? `tools/analysis/11_cpu_budget.py` scores a container without needing MAME, so the search loop is cheap; confirm the winner on the 68000 with `tools/bench/decode.lua`. + 3b. **Know which misses are yours to fix before starting.** Re-coding every non-SKIP block as V1 is the floor any mode assignment can reach, and it still misses 11 frames at `sasi` and 12 at `scsi` — every frame above ~90% @@ -154,12 +165,20 @@ saves ~16 of 448 cycles. ``` ./tools/bench/check.sh ``` -~2 min, needs the Blu-ray mounted. Re-runs both display regression tests from -source media **and the rate-control drift test** (session 6), then prints -`ALL GREEN`. Verified green at end of session 6. +~3 min, needs the Blu-ray mounted. From source media it re-runs both display +regression tests, the rate-control drift test (session 6), the display-path +coherency counterexample and a **120-frame 68000 decode** (session 7), then +prints `ALL GREEN`. Verified green at end of session 7. If it fails, fix that before doing anything else — everything downstream assumes the display path is pixel-exact. +The two session-7 stages are worth knowing the shape of before they fail on you: +- `10_pathmix_drift.py` is expected to exit **non-zero**; `check.sh` fails if it + ever starts passing, because that would mean the counterexample behind the + one-path decoder had stopped reproducing. +- the decode stage needs `tmp/rc_fr_singe_sasi_rcprofile.dlx` and will spend + ~55 s encoding it if it is missing, nearly all of that k-means in `H.build`. + ## Decisions locked | decision | value | why | @@ -184,6 +203,13 @@ bitrate ceiling is a build parameter in `tools/encoder/ratectl.py`: | `sasi` | 110 KB/s | 60 (floor) | 36.9 / 29.6 dB | stock 10MHz ACE/EXPERT | | `scsi` | 280 KB/s | 10 (floor) | 39.4 / 32.3 dB | Super/XVI, or CZ-6BS1 board | +**That "machine" column is about the BUS, not the CPU.** The profiles are an +I/O-bandwidth axis and say nothing about clock speed: the X68000 Super has +built-in SCSI at 10 MHz (`x68k.cpp:1194`, `40_MHz_XTAL/4`, same as the base +machine), and only the XVI is faster. **Both profiles target the same stock +10 MHz 68000**, so both must fit the same 833,333-cycle frame budget — and as of +session 7 neither does. FINDINGS 28.7. + **As of session 6 `lam` is a floor, not a setting.** The target is a ceiling and the encoder bisects lam per frame to stay under it; the profile's lam is the best quality it is allowed to spend on a quiet frame. On the worst sustained @@ -434,10 +460,18 @@ functional models, not timing-accurate; a KB/s figure from MAME measures the emulator's scheduler. `docs/BENCHMARK.md` covers the three-tier approach (MAME validates the path, derivation bounds it, real hardware settles it). -## Display path — VERIFIED (session 3), in a real mode (session 4), by 68000 code (session 5). +## Display path — VERIFIED (session 3), in a real mode (session 4), by 68000 code (session 5), by a 68000 DECODER (session 7). The first real frame is on screen: `docs/images/x68k_first_frame_compare.png`. +**Session 7 went from copying a frame to parsing one.** `src/player/decode.s` +reads DLX1, dispatches all four block modes and writes straight into GVRAM; +120 frames decoded in sequence are pixel-exact against `tools/encoder/dlx.py` +(`tools/bench/verify_decode.py`, in `check.sh`). The blit numbers below are +still correct for what they measured — a *copy* — but they are no longer the +display-path budget: the decoder costs 300/448/400 cycles per V1/V4/RAW block +and misses the 12fps budget on 31% of frames. FINDINGS 28. + **Session 5 closed the gap this paragraph used to describe.** GVRAM is now filled by 68000 instructions and the result is still pixel-exact, and the blit cost is measured rather than estimated: **53.6% of a 12fps frame**, not 38% @@ -483,11 +517,16 @@ SDL_VIDEODRIVER=dummy mame x68000 -bios ipl10 -video soft -window \ ## Next steps, in priority order -1. ~~**Measure the non-SKIP block fraction.**~~ **DONE, session 5** — FINDINGS - 25.6. Answer: implement **both** display paths and pick per frame; median - 37.0% of the frame budget, capped at 53.6%. Reporting is wired into - `encode.py`. Original framing kept below because the reasoning still governs - the decoder's inner loop: +1. ~~**Measure the non-SKIP block fraction.**~~ **DONE, session 5**, and its + answer **WITHDRAWN in session 7** — FINDINGS 28.1/28.2. It concluded + "implement both display paths and pick per frame, median 37.0%, capped at + 53.6%". Mixing the paths is incoherent (the compose path needs a RAM + reference the direct path never writes) and the two costs it compared were + both copies with no decode in either. **The shipping decoder has one path.** + The non-SKIP fraction is still reported by `encode.py` and is still the right + thing to look at — it is just no longer a switch. Original framing kept + below, because its instruction to report the distribution rather than the + mean is the part that held up: FINDINGS 24.5: compose-in-RAM-then-blit costs a flat 53.6% of the frame budget; decode-direct-to-GVRAM costs 76.6% x (fraction of blocks that are not SKIP) and needs no RAM reference frame. **They cross at 70%.** Which side of @@ -514,12 +553,6 @@ SDL_VIDEODRIVER=dummy mame x68000 -bios ipl10 -video soft -window \ of a 12fps frame, 31% of frames over 100%. FINDINGS 28. The follow-on is priority 0 at the top of this file. -2b. **Pad frame records to 4 bytes in `encode.py`.** Not optional: unaligned - records are an address error on a 68000 (FINDINGS 28.3), and `prep_dlx.py` - currently repairs it at load time, which the shipping player streaming from - disc cannot do. The padding is real bytes on disc, so it has to be inside - the rate controller's accounting, not added after it. 20 B/s at 12fps. - 2a. **Re-budget everything against the MEASURED per-mode costs**, not 53.6% and not 38%. Session 7 replaced the model twice over (FINDINGS 28.2): the display path is not one number times a block fraction, and the median frame is 74.4% @@ -534,6 +567,12 @@ SDL_VIDEODRIVER=dummy mame x68000 -bios ipl10 -video soft -window \ rejection, which was argued as "54% LZ4 with no room beside a 38% blit". The conclusion gets *stronger*, not weaker, but the arithmetic should be restated. +2b. **Pad frame records to 4 bytes in `encode.py`.** Not optional: unaligned + records are an address error on a 68000 (FINDINGS 28.3), and `prep_dlx.py` + currently repairs it at load time, which the shipping player streaming from + disc cannot do. The padding is real bytes on disc, so it has to be inside + the rate controller's accounting, not added after it. 20 B/s at 12fps. + 3. **Full-disc survey.** Now scoped by session 5 rather than open-ended: the worst *sustained* window is measured (FINDINGS 25), so what remains is the distribution over content, not the worst case.