Handoff: reconcile the docs with a decoder that works and does not fit
Session 7 handoff. check.sh green end to end, including both new stages. STATUS now leads with the cost-aware mode decision, and the stale statements that session 7 overturned are struck rather than left to be re-read as current: - next-steps item 1 still announced "implement both display paths and pick per frame" as a settled answer. Withdrawn, with the reason, and the part that did hold up (report the distribution, not the mean) kept. - the profile table's "machine" column reads as a CPU claim and is not one. It is the bus. Both profiles target the same stock 10 MHz 68000 and neither fits; the Super has SCSI at 10 MHz. - the display-path section stopped at "68000 code copied a frame". It parses one now, and the blit figures are no longer the display-path budget. - the green-light section still described a 2-minute, three-stage check, and did not warn that 10_pathmix_drift.py is SUPPOSED to exit non-zero. Sharpened the one trap in the next session's first step: SKIP is not a per-block constant. It costs 13.25 cycles inside an all-SKIP header byte and ~45 in a mixed one, so its price depends on its neighbours and a per-block lagrangian cannot see that. Picking either number is wrong in a different direction; the budget check can use the exact clustered cost function while the per-block term only has to rank modes. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
This commit is contained in:
+57
-18
@@ -26,9 +26,19 @@ worth 4 bytes, with no idea what it costs to draw.
|
|||||||
**The work, in order:**
|
**The work, in order:**
|
||||||
|
|
||||||
1. **Add a cycle term to the mode decision.** `decide()` already builds a cost
|
1. **Add a cycle term to the mode decision.** `decide()` already builds a cost
|
||||||
matrix of `error + lam * bytes` per mode per block; add `+ mu * cycles`, with
|
matrix of `error + lam * bytes` per mode per block; add `+ mu * cycles`,
|
||||||
the cycles vector `[13, 300, 448, 400]` measured in FINDINGS 28.2. One extra
|
with the per-mode cycles measured in FINDINGS 28.2.
|
||||||
row of arithmetic in a function that is already vectorised.
|
**SKIP is not a constant and this is the one trap here.** A SKIP block costs
|
||||||
|
13.25 cycles when all four blocks in its header byte are SKIP (one `tst.b`
|
||||||
|
clears the group) and ~45 when it sits in a mixed byte — so SKIP's price
|
||||||
|
depends on its *neighbours*, which a per-block lagrangian cannot see. Do not
|
||||||
|
pick one number and move on: 45 overcharges clustered SKIPs and pushes the
|
||||||
|
encoder away from the mode that saves cycles, 13.25 undercharges isolated
|
||||||
|
ones and lets frames overrun. The way out is that the **budget check does not
|
||||||
|
have to use the same cost function as the mode decision** — score frames with
|
||||||
|
the exact clustered cost (`cycles()` in `tools/analysis/11_cpu_budget.py`,
|
||||||
|
validated to 1 point against the 68000) and let the bisection converge on
|
||||||
|
that, while the per-block term uses a constant purely to *rank* modes.
|
||||||
2. **Then bisect `mu` per frame against the 833,333-cycle budget**, exactly as
|
2. **Then bisect `mu` per frame against the 833,333-cycle budget**, exactly as
|
||||||
session 6 bisects `lam` against the byte budget. The machinery is already
|
session 6 bisects `lam` against the byte budget. The machinery is already
|
||||||
there and already gated: `ratectl.encode_rate_controlled` is frame-driven and
|
there and already gated: `ratectl.encode_rate_controlled` is frame-driven and
|
||||||
@@ -43,6 +53,7 @@ worth 4 bytes, with no idea what it costs to draw.
|
|||||||
cliff? `tools/analysis/11_cpu_budget.py` scores a container without needing
|
cliff? `tools/analysis/11_cpu_budget.py` scores a container without needing
|
||||||
MAME, so the search loop is cheap; confirm the winner on the 68000 with
|
MAME, so the search loop is cheap; confirm the winner on the 68000 with
|
||||||
`tools/bench/decode.lua`.
|
`tools/bench/decode.lua`.
|
||||||
|
|
||||||
3b. **Know which misses are yours to fix before starting.** Re-coding every
|
3b. **Know which misses are yours to fix before starting.** Re-coding every
|
||||||
non-SKIP block as V1 is the floor any mode assignment can reach, and it
|
non-SKIP block as V1 is the floor any mode assignment can reach, and it
|
||||||
still misses 11 frames at `sasi` and 12 at `scsi` — every frame above ~90%
|
still misses 11 frames at `sasi` and 12 at `scsi` — every frame above ~90%
|
||||||
@@ -154,12 +165,20 @@ saves ~16 of 448 cycles.
|
|||||||
```
|
```
|
||||||
./tools/bench/check.sh
|
./tools/bench/check.sh
|
||||||
```
|
```
|
||||||
~2 min, needs the Blu-ray mounted. Re-runs both display regression tests from
|
~3 min, needs the Blu-ray mounted. From source media it re-runs both display
|
||||||
source media **and the rate-control drift test** (session 6), then prints
|
regression tests, the rate-control drift test (session 6), the display-path
|
||||||
`ALL GREEN`. Verified green at end of session 6.
|
coherency counterexample and a **120-frame 68000 decode** (session 7), then
|
||||||
|
prints `ALL GREEN`. Verified green at end of session 7.
|
||||||
If it fails, fix that before doing anything else — everything downstream assumes
|
If it fails, fix that before doing anything else — everything downstream assumes
|
||||||
the display path is pixel-exact.
|
the display path is pixel-exact.
|
||||||
|
|
||||||
|
The two session-7 stages are worth knowing the shape of before they fail on you:
|
||||||
|
- `10_pathmix_drift.py` is expected to exit **non-zero**; `check.sh` fails if it
|
||||||
|
ever starts passing, because that would mean the counterexample behind the
|
||||||
|
one-path decoder had stopped reproducing.
|
||||||
|
- the decode stage needs `tmp/rc_fr_singe_sasi_rcprofile.dlx` and will spend
|
||||||
|
~55 s encoding it if it is missing, nearly all of that k-means in `H.build`.
|
||||||
|
|
||||||
## Decisions locked
|
## Decisions locked
|
||||||
|
|
||||||
| decision | value | why |
|
| decision | value | why |
|
||||||
@@ -184,6 +203,13 @@ bitrate ceiling is a build parameter in `tools/encoder/ratectl.py`:
|
|||||||
| `sasi` | 110 KB/s | 60 (floor) | 36.9 / 29.6 dB | stock 10MHz ACE/EXPERT |
|
| `sasi` | 110 KB/s | 60 (floor) | 36.9 / 29.6 dB | stock 10MHz ACE/EXPERT |
|
||||||
| `scsi` | 280 KB/s | 10 (floor) | 39.4 / 32.3 dB | Super/XVI, or CZ-6BS1 board |
|
| `scsi` | 280 KB/s | 10 (floor) | 39.4 / 32.3 dB | Super/XVI, or CZ-6BS1 board |
|
||||||
|
|
||||||
|
**That "machine" column is about the BUS, not the CPU.** The profiles are an
|
||||||
|
I/O-bandwidth axis and say nothing about clock speed: the X68000 Super has
|
||||||
|
built-in SCSI at 10 MHz (`x68k.cpp:1194`, `40_MHz_XTAL/4`, same as the base
|
||||||
|
machine), and only the XVI is faster. **Both profiles target the same stock
|
||||||
|
10 MHz 68000**, so both must fit the same 833,333-cycle frame budget — and as of
|
||||||
|
session 7 neither does. FINDINGS 28.7.
|
||||||
|
|
||||||
**As of session 6 `lam` is a floor, not a setting.** The target is a ceiling and
|
**As of session 6 `lam` is a floor, not a setting.** The target is a ceiling and
|
||||||
the encoder bisects lam per frame to stay under it; the profile's lam is the
|
the encoder bisects lam per frame to stay under it; the profile's lam is the
|
||||||
best quality it is allowed to spend on a quiet frame. On the worst sustained
|
best quality it is allowed to spend on a quiet frame. On the worst sustained
|
||||||
@@ -434,10 +460,18 @@ functional models, not timing-accurate; a KB/s figure from MAME measures the
|
|||||||
emulator's scheduler. `docs/BENCHMARK.md` covers the three-tier approach
|
emulator's scheduler. `docs/BENCHMARK.md` covers the three-tier approach
|
||||||
(MAME validates the path, derivation bounds it, real hardware settles it).
|
(MAME validates the path, derivation bounds it, real hardware settles it).
|
||||||
|
|
||||||
## Display path — VERIFIED (session 3), in a real mode (session 4), by 68000 code (session 5).
|
## Display path — VERIFIED (session 3), in a real mode (session 4), by 68000 code (session 5), by a 68000 DECODER (session 7).
|
||||||
|
|
||||||
The first real frame is on screen: `docs/images/x68k_first_frame_compare.png`.
|
The first real frame is on screen: `docs/images/x68k_first_frame_compare.png`.
|
||||||
|
|
||||||
|
**Session 7 went from copying a frame to parsing one.** `src/player/decode.s`
|
||||||
|
reads DLX1, dispatches all four block modes and writes straight into GVRAM;
|
||||||
|
120 frames decoded in sequence are pixel-exact against `tools/encoder/dlx.py`
|
||||||
|
(`tools/bench/verify_decode.py`, in `check.sh`). The blit numbers below are
|
||||||
|
still correct for what they measured — a *copy* — but they are no longer the
|
||||||
|
display-path budget: the decoder costs 300/448/400 cycles per V1/V4/RAW block
|
||||||
|
and misses the 12fps budget on 31% of frames. FINDINGS 28.
|
||||||
|
|
||||||
**Session 5 closed the gap this paragraph used to describe.** GVRAM is now
|
**Session 5 closed the gap this paragraph used to describe.** GVRAM is now
|
||||||
filled by 68000 instructions and the result is still pixel-exact, and the blit
|
filled by 68000 instructions and the result is still pixel-exact, and the blit
|
||||||
cost is measured rather than estimated: **53.6% of a 12fps frame**, not 38%
|
cost is measured rather than estimated: **53.6% of a 12fps frame**, not 38%
|
||||||
@@ -483,11 +517,16 @@ SDL_VIDEODRIVER=dummy mame x68000 -bios ipl10 -video soft -window \
|
|||||||
|
|
||||||
## Next steps, in priority order
|
## Next steps, in priority order
|
||||||
|
|
||||||
1. ~~**Measure the non-SKIP block fraction.**~~ **DONE, session 5** — FINDINGS
|
1. ~~**Measure the non-SKIP block fraction.**~~ **DONE, session 5**, and its
|
||||||
25.6. Answer: implement **both** display paths and pick per frame; median
|
answer **WITHDRAWN in session 7** — FINDINGS 28.1/28.2. It concluded
|
||||||
37.0% of the frame budget, capped at 53.6%. Reporting is wired into
|
"implement both display paths and pick per frame, median 37.0%, capped at
|
||||||
`encode.py`. Original framing kept below because the reasoning still governs
|
53.6%". Mixing the paths is incoherent (the compose path needs a RAM
|
||||||
the decoder's inner loop:
|
reference the direct path never writes) and the two costs it compared were
|
||||||
|
both copies with no decode in either. **The shipping decoder has one path.**
|
||||||
|
The non-SKIP fraction is still reported by `encode.py` and is still the right
|
||||||
|
thing to look at — it is just no longer a switch. Original framing kept
|
||||||
|
below, because its instruction to report the distribution rather than the
|
||||||
|
mean is the part that held up:
|
||||||
FINDINGS 24.5: compose-in-RAM-then-blit costs a flat 53.6% of the frame
|
FINDINGS 24.5: compose-in-RAM-then-blit costs a flat 53.6% of the frame
|
||||||
budget; decode-direct-to-GVRAM costs 76.6% x (fraction of blocks that are not
|
budget; decode-direct-to-GVRAM costs 76.6% x (fraction of blocks that are not
|
||||||
SKIP) and needs no RAM reference frame. **They cross at 70%.** Which side of
|
SKIP) and needs no RAM reference frame. **They cross at 70%.** Which side of
|
||||||
@@ -514,12 +553,6 @@ SDL_VIDEODRIVER=dummy mame x68000 -bios ipl10 -video soft -window \
|
|||||||
of a 12fps frame, 31% of frames over 100%. FINDINGS 28. The follow-on is
|
of a 12fps frame, 31% of frames over 100%. FINDINGS 28. The follow-on is
|
||||||
priority 0 at the top of this file.
|
priority 0 at the top of this file.
|
||||||
|
|
||||||
2b. **Pad frame records to 4 bytes in `encode.py`.** Not optional: unaligned
|
|
||||||
records are an address error on a 68000 (FINDINGS 28.3), and `prep_dlx.py`
|
|
||||||
currently repairs it at load time, which the shipping player streaming from
|
|
||||||
disc cannot do. The padding is real bytes on disc, so it has to be inside
|
|
||||||
the rate controller's accounting, not added after it. 20 B/s at 12fps.
|
|
||||||
|
|
||||||
2a. **Re-budget everything against the MEASURED per-mode costs**, not 53.6% and
|
2a. **Re-budget everything against the MEASURED per-mode costs**, not 53.6% and
|
||||||
not 38%. Session 7 replaced the model twice over (FINDINGS 28.2): the display
|
not 38%. Session 7 replaced the model twice over (FINDINGS 28.2): the display
|
||||||
path is not one number times a block fraction, and the median frame is 74.4%
|
path is not one number times a block fraction, and the median frame is 74.4%
|
||||||
@@ -534,6 +567,12 @@ SDL_VIDEODRIVER=dummy mame x68000 -bios ipl10 -video soft -window \
|
|||||||
rejection, which was argued as "54% LZ4 with no room beside a 38% blit". The
|
rejection, which was argued as "54% LZ4 with no room beside a 38% blit". The
|
||||||
conclusion gets *stronger*, not weaker, but the arithmetic should be restated.
|
conclusion gets *stronger*, not weaker, but the arithmetic should be restated.
|
||||||
|
|
||||||
|
2b. **Pad frame records to 4 bytes in `encode.py`.** Not optional: unaligned
|
||||||
|
records are an address error on a 68000 (FINDINGS 28.3), and `prep_dlx.py`
|
||||||
|
currently repairs it at load time, which the shipping player streaming from
|
||||||
|
disc cannot do. The padding is real bytes on disc, so it has to be inside
|
||||||
|
the rate controller's accounting, not added after it. 20 B/s at 12fps.
|
||||||
|
|
||||||
3. **Full-disc survey.** Now scoped by session 5 rather than open-ended: the
|
3. **Full-disc survey.** Now scoped by session 5 rather than open-ended: the
|
||||||
worst *sustained* window is measured (FINDINGS 25), so what remains is the
|
worst *sustained* window is measured (FINDINGS 25), so what remains is the
|
||||||
distribution over content, not the worst case.
|
distribution over content, not the worst case.
|
||||||
|
|||||||
Reference in New Issue
Block a user