Price cycles in the mode decision: 37 misses become 1, for 0.26 dB
The decoder has been CPU-bound since FINDINGS 28 while the mode decision
minimised D + lam*R -- distortion against BYTES. decide() now minimises
D + lam*bytes + mu*cycles, and ratectl bisects mu per frame against the
833,333-cycle budget with the lam bisection nested inside it. On the worst
sustained window:
sasi 27.22 -> 26.95 dB, 109.5 -> 109.4 KB/s, 37/120 misses -> 1
scsi 29.90 -> 29.27 dB, 280.0 -> 278.6 KB/s, 51/120 misses -> 1
Bitrate does not move: the byte controller still binds, and mu changes WHICH
modes are bought. V4 is what it stops buying -- 25.2 -> 20.3% of blocks at sasi
and 15.0 -> 5.3% at scsi, where RAW takes it. That is 28.8's inversion in
practice: RAW is dearer in bytes and cheaper in cycles, so only the byte-rich
profile can buy its way out of V4.
Three things worth knowing beyond the headline:
- The one frame that still misses, at both profiles, is FRAME 0 -- no previous
reconstruction, so 100% changed by definition, which is also what a scene
cut is. It comes out at the all-V1 floor of 110.6% and is emitted late on
purpose. Freezing a cut to make a deadline is the worse failure.
- 28.7's "11 frames are impossible" was too pessimistic. That floor held the
SKIP set fixed and asked how cheaply the drawn blocks could be drawn; the
real decision can also MOVE a block to SKIP, which above ~90% non-SKIP is
the only lever left.
- SKIP's price depends on its neighbours (13.25 cycles clustered, 45 mixed),
which a per-block lagrangian cannot see. The way out is that the two uses
need not share a cost function: a ranking constant inside decide(), the
exact clustered rule for the frame-level bisection. vq_hybrid.cycles() is
now the one definition of that rule and 11_cpu_budget.py imports it.
Gated: 09_ratectl_drift.py runs both controllers, both 0/120 drifting frames.
The cost-aware container decodes pixel-exact on the 68000 (120 frames). ON by
default in encode.py; --no-cpu-fit restores session 7. check.sh ALL GREEN.
Still a model, not a measurement, for THIS container: FINDINGS 31's cycle
figures come from vq_hybrid.cycles (within 1 point of the 68000 on four frames
of the session-7 container). Timing this one on the machine is step 1 of the
next session -- it was started and killed for time, and it is slow.
FINDINGS 31. tools/analysis/13_cpu_ratectl.py.
Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
This commit is contained in:
prosolis
@@ -39,7 +39,9 @@ tools/analysis/ measurement scripts, numbered in the order they were written
|
||||
11 scores a container against the MEASURED per-mode block
|
||||
costs without needing MAME; 12 prices the literal-span mode of
|
||||
FINDINGS 30 against those same mode maps, and prints whether a
|
||||
scene cut still fits at 12fps.
|
||||
scene cut still fits at 12fps; 13 measures what fitting the
|
||||
CPU budget costs in dB (FINDINGS 31) and caches H.build so the
|
||||
search loop is seconds, not minutes.
|
||||
tools/bench/ MAME Lua injection harness + 68000 benchmark sources.
|
||||
`check.sh` re-runs both display regression tests (~40 s).
|
||||
`blit.s`/`blit.lua` time the full-frame GVRAM blit on the
|
||||
@@ -70,7 +72,14 @@ python3 tools/encoder/encode.py /tmp/fr out.dlx --profile sasi --preview p.png
|
||||
Two quality profiles ship from one codec and one decoder — `sasi` (110 KB/s) and
|
||||
`scsi` (280 KB/s) are two points on the same rate-distortion curve. Both are
|
||||
**ceilings**: lam is bisected per frame under a leaky bucket, so the profile's
|
||||
`lam` is a quality floor rather than a setting (`--fixed-lam` opts out). The codec is
|
||||
`lam` is a quality floor rather than a setting (`--fixed-lam` opts out).
|
||||
|
||||
There are **two** ceilings, on two different axes. The second is the 68000's
|
||||
decode budget: `mu` is bisected per frame against 833,333 cycles so the frame
|
||||
also *decodes* in time, which takes the worst sustained window from 37 frames
|
||||
over budget to 1 for 0.26 dB (FINDINGS 31). It is on by default; `--no-cpu-fit`
|
||||
restores session 7 behaviour. Unlike bytes, cycles have no bucket — there is no
|
||||
double buffer to decode ahead into, so it is a hard per-frame ceiling. The codec is
|
||||
a Cinepak-style hybrid: each 4x4 block is coded as SKIP, one 4x4 codeword, four
|
||||
2x2 codewords, or RAW literal pixels, chosen per block by rate-distortion.
|
||||
|
||||
|
||||
@@ -1678,3 +1678,108 @@ re-priced `sasi` stream buys 8773 spans across 120 frames, a mean of 73 a frame,
|
||||
and each one's three-instruction dispatch is inside the fitted 43.7 — but the
|
||||
mode-map walk that decides a span exists is not. `decode.s` does not implement
|
||||
spans yet.
|
||||
|
||||
## 31. The mode decision can see cycles now, and it costs 0.26 dB (session 8)
|
||||
|
||||
FINDINGS 28 left the decoder missing 31% of frames at `sasi` and 42% at `scsi`
|
||||
while the mode decision minimised `D + lam*R` — distortion against BYTES — on a
|
||||
machine whose binding budget is CYCLES. This is the second controller.
|
||||
|
||||
`vq_hybrid.decide(ctx, lam, mu)` now minimises `D + lam*bytes + mu*cycles`, and
|
||||
`ratectl.encode_rate_controlled(cycle_budget=...)` bisects `mu` per frame
|
||||
against 833,333 cycles with the `lam` bisection nested inside it.
|
||||
`tools/analysis/13_cpu_ratectl.py` measures what it costs.
|
||||
|
||||
### 31.1 The result
|
||||
Worst sustained window, 120 frames, same targets, same quality floors:
|
||||
|
||||
| | PSNR | KB/s | CPU median | CPU max | frames missing |
|
||||
|---|---:|---:|---:|---:|---:|
|
||||
| `sasi` bytes only | 27.22 dB | 109.5 | 74.4% | 136.2% | **37/120** |
|
||||
| `sasi` + cycle ceiling | **26.95 dB** | 109.4 | 81.5% | 110.6% | **1/120** |
|
||||
| `scsi` bytes only | 29.90 dB | 280.0 | 94.9% | 146.6% | **51/120** |
|
||||
| `scsi` + cycle ceiling | **29.27 dB** | 278.6 | 99.6% | 110.6% | **1/120** |
|
||||
|
||||
**36 of 37 misses at `sasi` for 0.26 dB, 50 of 51 at `scsi` for 0.62 dB.** The
|
||||
bitrate does not move: the byte controller still binds, and mu changes *which*
|
||||
modes are bought rather than how many bytes.
|
||||
|
||||
`sasi` pays less quality than `scsi` because it had less to give up: it was
|
||||
already short of bytes, so the cycle-cheap directions it takes (V4 -> V1, and
|
||||
blocks it can afford to hold) were near where the byte lagrangian already sat.
|
||||
28.8 predicted the shape of this and got the sign right.
|
||||
|
||||
Mode mix, `sasi`, bytes-only -> with the ceiling: SKIP 46.4 -> 47.1%,
|
||||
V1 19.8 -> 23.0%, **V4 25.2 -> 20.3%**, RAW 8.5 -> 9.6%. At `scsi` the V4
|
||||
collapse is dramatic — **15.0 -> 5.3%**, with RAW taking it at 41.3 -> 43.2%,
|
||||
which is 28.8's inversion happening in practice: RAW is dearer in bytes and
|
||||
cheaper in cycles, so a byte-rich profile buys its way out of V4.
|
||||
|
||||
Only **46 of 120 frames need any mu at all** at `sasi`; the median frame is
|
||||
decided at mu=0 and is unchanged from session 6.
|
||||
|
||||
### 31.2 The one frame that cannot fit is the intra frame, not a hard case
|
||||
Both profiles miss exactly one frame, both at 110.6% — the all-V1 floor of
|
||||
FINDINGS 28.5 — and in both it is **frame 0**. It has no previous
|
||||
reconstruction, so every block must be coded, which is the definition of a
|
||||
100%-changed frame. A scene cut mid-stream is the same thing.
|
||||
|
||||
That is the correct behaviour rather than a failure, and it is worth being
|
||||
explicit about why: at `MU_CLIFF` a block only becomes SKIP if holding the
|
||||
previous reconstruction costs less than ~28,665 units of distortion. A frame
|
||||
with nothing on screen worth holding stays fully coded and is emitted **late on
|
||||
purpose**, exactly as a frame that will not fit at `LAM_CLIFF` is emitted over
|
||||
budget. Freezing a cut to make a deadline is the worse failure.
|
||||
|
||||
### 31.3 28.7 was too pessimistic, and the reason is instructive
|
||||
28.7 estimated that only ~three quarters of the misses were the encoder's to
|
||||
fix — 26 of 37 at `sasi` — because re-coding every non-SKIP block as V1 still
|
||||
missed 11 frames. Measured, the controller fixes **36 of 37**.
|
||||
|
||||
The gap is that 28.7's floor held the SKIP set fixed and asked "how cheap can
|
||||
the blocks we already decided to draw be?". The real decision can also **move a
|
||||
block to SKIP**, paying distortion for it, and above ~90% non-SKIP that is the
|
||||
only lever left. So 28.7's floor was a floor for a fixed SKIP set, not for the
|
||||
mode decision. Two conclusions of 28.7 stand: the profiles are an I/O axis and
|
||||
both must fit the same 10 MHz budget.
|
||||
|
||||
### 31.4 SKIP is not a constant, and the way out is two cost functions
|
||||
A SKIP block costs 13.25 cycles when all four blocks sharing its header byte are
|
||||
SKIP (one `tst.b` clears the group) and ~45 in a mixed byte, so its price
|
||||
depends on its neighbours — which a per-block lagrangian cannot see. Picking one
|
||||
number is a real trade: 45 overcharges clustered SKIPs and pushes the encoder
|
||||
away from the mode that saves the most cycles, 13.25 undercharges isolated ones
|
||||
and lets frames overrun.
|
||||
|
||||
The resolution is that **the budget check does not have to use the same cost
|
||||
function as the mode decision**. `decide()` uses 13.25 purely to *rank* modes
|
||||
within a block, where the choice only scales the incentive (the V1-SKIP gap
|
||||
moves 12% between the two candidates). The controller scores whole frames with
|
||||
`vq_hybrid.cycles()`, the exact clustered rule, validated to 1 point against the
|
||||
68000 — so the bisection converges on what the machine will really do, whatever
|
||||
the ranking constant was. That function is now defined once and imported by
|
||||
`11_cpu_budget.py`, rather than living in two places that can drift apart.
|
||||
|
||||
### 31.5 Both controllers are gated against decoder drift
|
||||
The mu controller varies the mode map frame to frame exactly as the lam
|
||||
controller does, so it is exposed to the FINDINGS 26.1 failure — an encoder
|
||||
reporting a reconstruction the decoder will never produce. `09_ratectl_drift.py`
|
||||
now runs **both** configurations and both report 0/120 drifting frames, 0.00 dB
|
||||
overstatement. The CPU ceiling is on by default in `encode.py`
|
||||
(`--no-cpu-fit` restores session 7 behaviour).
|
||||
|
||||
### 31.6 With spans on top, the window fits completely
|
||||
Re-running the span pricing of FINDINGS 30 against a cost-aware container —
|
||||
lever B first, then lever A on what it leaves:
|
||||
|
||||
| `sasi` | bytes only | + cycle ceiling | + ceiling + spans |
|
||||
|---|---:|---:|---:|
|
||||
| median frame | 74.4% | 81.5% | **56.8%** |
|
||||
| worst frame | 136.2% | 110.6% | **91.5%** |
|
||||
| frames missing | 37/120 | 1/120 | **0/120** |
|
||||
| bitrate | 101.7 KB/s | 101.6 | 449.3 KB/s |
|
||||
|
||||
The intra frame lands at 91.5% — spans are what make a full redraw fit, which is
|
||||
30.6's arithmetic arriving in a real container. That row is still a **model** of
|
||||
a bitstream nothing implements; the two levers have never run on the 68000
|
||||
together, and the ring-buffer question of 30.7 gets sharper at 449 KB/s.
|
||||
|
||||
+100
-94
@@ -1,115 +1,116 @@
|
||||
# Status & next-session handoff — session 8 (2026-08-23)
|
||||
# Status & next-session handoff — end of session 8 (2026-08-23)
|
||||
|
||||
## Where this stands
|
||||
|
||||
The decoder exists, it is pixel-exact, and **it does not fit**: mean 81.7% of a
|
||||
12fps frame on the worst sustained window at `sasi`, 31% of frames over budget
|
||||
(`scsi`: 94.9% median, 42% miss). FINDINGS 28. CPU is the binding constraint.
|
||||
Session 7 left the decoder pixel-exact and **31% of frames over the CPU budget**
|
||||
at `sasi`, 42% at `scsi` (FINDINGS 28), with two levers proposed and neither
|
||||
measured. Session 8 did both.
|
||||
|
||||
Session 7 proposed two levers and session 8 measured the cheap one first.
|
||||
**Lever A, spans: measured.** A row-linear span of word-expanded literals costs
|
||||
**43.7 cycles per span + 9.152 per pixel** — but only in an encoder-assisted
|
||||
format, `{u32 absolute GVRAM address, u16 jump displacement}` into an unrolled
|
||||
copy chain. The obvious decoder, handed `(x, npix)`, is 97.9 + 10.46.
|
||||
FINDINGS 30, `tools/bench/span.sh` (~25 s).
|
||||
|
||||
**Lever A — spend bandwidth to buy cycles — is real, and it is an encoder
|
||||
format.** A row-linear span of word-expanded literals measures **43.7 cycles per
|
||||
span + 9.152 per pixel** (FINDINGS 30, `tools/bench/span.sh`), which is what
|
||||
FINDINGS 29 assumed — but only when the *encoder* hands the decoder an absolute
|
||||
GVRAM address and a jump displacement into an unrolled copy chain. The obvious
|
||||
decoder, handed `(x, npix)` and left to work the copy out, is 97.9 + 10.46 and
|
||||
2.2x dearer on a short span. Re-priced against the unchanged mode maps:
|
||||
**Lever B, the cost-aware mode decision: implemented, measured, and ON by
|
||||
default.** `decide()` minimises `D + lam*bytes + mu*cycles`; `mu` is bisected
|
||||
per frame against 833,333 cycles with the `lam` bisection nested inside it.
|
||||
FINDINGS 31, `tools/analysis/13_cpu_ratectl.py`.
|
||||
|
||||
| | today | 29 (derived) | **30 (measured)** |
|
||||
|---|---:|---:|---:|
|
||||
| `sasi` median frame | 74.4% | 43.0% | **52.0%** |
|
||||
| `sasi` worst frame | 136.2% | 106.2% | **108.7%** |
|
||||
| `sasi` frames missing | 37/120 | 8/120 | **10/120** |
|
||||
| bitrate | 101.7 KB/s | 453.2 | **448.0 KB/s** (bus 488) |
|
||||
| `sasi`, worst sustained window | PSNR | KB/s | CPU median | CPU max | missing |
|
||||
|---|---:|---:|---:|---:|---:|
|
||||
| bytes only (session 7) | 27.22 dB | 109.5 | 74.4% | 136.2% | **37/120** |
|
||||
| + cycle ceiling (now the default) | 26.95 dB | 109.4 | 81.5% | 110.6% | **1/120** |
|
||||
| + ceiling + spans (MODEL, nothing implements it) | — | 449.3 | 56.8% | 91.5% | **0/120** |
|
||||
|
||||
Break-even moved with it: a run beats all-V1 **from 4 blocks up**, not 2. And
|
||||
29.4 survives — a scene cut needs `x >= 0.196` of the frame as spans and the bus
|
||||
allows `x <= 0.373`, so it fits at 12fps.
|
||||
`scsi`: 51/120 -> 1/120 for 0.62 dB. The one remaining miss at either profile is
|
||||
**frame 0**, which has no previous reconstruction and so is 100% changed by
|
||||
definition — the same case as a scene cut. It is emitted late on purpose.
|
||||
|
||||
**Lever B — stop buying modes the CPU cannot afford — is untouched.**
|
||||
`vq_hybrid.decide()` still minimises `D + lam*R`, distortion against BYTES, on a
|
||||
machine whose binding budget is CYCLES:
|
||||
The cost-aware container is verified pixel-exact on the 68000 (120 frames,
|
||||
`tools/bench/verify_decode.py`).
|
||||
|
||||
| mode | payload bytes | cycles | cycles per byte |
|
||||
|---|---:|---:|---:|
|
||||
| SKIP | 0 | 13 (clustered) | — |
|
||||
| V1 | 1 | 300 | 300 |
|
||||
| V4 | 4 | 448 | 112 |
|
||||
| RAW | 16 | 400 | 25 |
|
||||
| **span, per 4x4 block in a run of L** | **32** | **1053/L, floor 154** | **~5** |
|
||||
## NEXT SESSION, in order
|
||||
|
||||
V4 is 25% of blocks and 50% of the cycles. The lagrangian charges it 4x a V1
|
||||
block; the CPU charges it 1.49x.
|
||||
0. **Green light first.** `./tools/bench/check.sh` (~3 min, Blu-ray mounted).
|
||||
The drift stage now runs BOTH controllers; both must report 0/120.
|
||||
|
||||
## The work, in order
|
||||
1. **Time the cost-aware container on the 68000.** Everything in FINDINGS 31 is
|
||||
the validated cost MODEL (`vq_hybrid.cycles`, within 1 point of the machine
|
||||
on four frames of the session-7 container), not a measurement of this one.
|
||||
The full timing pass was started and killed for time:
|
||||
```
|
||||
python3 tools/bench/prep_dlx.py tmp/rc_fr_singe_sasi_cpufit.dlx
|
||||
tools/vasm/vasmm68k_mot -Fbin -o tmp/decode.bin src/player/decode.s
|
||||
( cd tmp && SDL_VIDEODRIVER=dummy mame x68000 -bios ipl10 -ramsize 2M \
|
||||
-video soft -window -sound none -nothrottle -plugins \
|
||||
-autoboot_script ../tools/bench/decode.lua -snapshot_directory ./snap_decode \
|
||||
-snapview native -seconds_to_run 300 > decode_cpufit.log 2>&1 )
|
||||
```
|
||||
**Budget real time for it: the run was still going at 12 minutes of CPU.**
|
||||
MAME's stdout is block-buffered to a file, so there is no progress to watch
|
||||
— wait on the PID, never on a `pgrep -f` match (see the shell traps below).
|
||||
Confirm the four anchors against `11_cpu_budget.py` on the same container,
|
||||
and update FINDINGS 31 with measured-vs-model errors.
|
||||
|
||||
1. **Add a cycle term to the mode decision.** `decide()` already builds a cost
|
||||
matrix of `error + lam * bytes` per mode per block; add `+ mu * cycles`,
|
||||
with the per-mode cycles measured in FINDINGS 28.2.
|
||||
**SKIP is not a constant and this is the one trap here.** A SKIP block costs
|
||||
13.25 cycles when all four blocks in its header byte are SKIP (one `tst.b`
|
||||
clears the group) and ~45 when it sits in a mixed byte — so SKIP's price
|
||||
depends on its *neighbours*, which a per-block lagrangian cannot see. Do not
|
||||
pick one number and move on: 45 overcharges clustered SKIPs and pushes the
|
||||
encoder away from the mode that saves cycles, 13.25 undercharges isolated
|
||||
ones and lets frames overrun. The way out is that the **budget check does not
|
||||
have to use the same cost function as the mode decision** — score frames with
|
||||
the exact clustered cost (`cycles()` in `tools/analysis/11_cpu_budget.py`,
|
||||
validated to 1 point against the 68000) and let the bisection converge on
|
||||
that, while the per-block term uses a constant purely to *rank* modes.
|
||||
2. **Then bisect `mu` per frame against the 833,333-cycle budget**, exactly as
|
||||
session 6 bisects `lam` against the byte budget. The machinery is already
|
||||
there and already gated: `ratectl.encode_rate_controlled` is frame-driven and
|
||||
feeds back the frame it emitted. **But cycles have NO bucket.** Bytes can be
|
||||
banked in the ring buffer; a frame that misses its decode deadline is just
|
||||
late, because there is no double buffer to decode ahead into. So this is a
|
||||
hard per-frame ceiling, not a leaky bucket — simpler than rate control, and
|
||||
the two controllers have to run together (raising `mu` moves blocks to SKIP
|
||||
and V1, which also *lowers* the bitrate, so the byte controller must see it).
|
||||
3. **Measure the quality cost.** Everything session 6 did for bytes: what does
|
||||
fitting 100% of frames in the CPU budget cost in dB, and does any frame hit a
|
||||
cliff? `tools/analysis/11_cpu_budget.py` scores a container without needing
|
||||
MAME, so the search loop is cheap; confirm the winner on the 68000 with
|
||||
`tools/bench/decode.lua`.
|
||||
2. **Put spans in the bitstream.** This is the big one and it is now fully
|
||||
specified by measurement: format in FINDINGS 30.2, costs in 30.5, and the
|
||||
scene-cut arithmetic in 30.6. It touches `encode.py` (a fifth mode and a
|
||||
run-aware decision), `dlx.py` (the reference decoder), and `decode.s`. The
|
||||
24-pixel quantisation and the free row overrun are part of the format, not
|
||||
optimisations to add later. Order it AFTER item 1 so the model that prices it
|
||||
has been checked against the machine once more.
|
||||
|
||||
3b. **Know which misses are yours to fix before starting.** Re-coding every
|
||||
non-SKIP block as V1 is the floor any mode assignment *of the current mode
|
||||
set* can reach, and it still misses 11 frames at `sasi` and 12 at `scsi` —
|
||||
every frame above ~90% non-SKIP. So the cost-aware decision can reach about
|
||||
three quarters of the misses (26 of 37 at `sasi`) and the rest need item 4.
|
||||
FINDINGS 28.7.
|
||||
3. **The three open items of FINDINGS 29.5/30.7**, now load-bearing because a
|
||||
span design runs at ~449 KB/s of a 488 KB/s pipe: re-run the ring-buffer
|
||||
simulation at that rate (FINDINGS 21 was established at 110 and 280), confirm
|
||||
the provenance of the user's 4 Mbps figure, and **confirm DMA rather than
|
||||
PIO** — a PIO fallback puts a 449 KB/s transfer back on the CPU the whole
|
||||
lever exists to relieve. The DMA check is the cheapest of the three and the
|
||||
most consequential.
|
||||
|
||||
3c. **Buy RAW, not V4, wherever the bytes allow.** RAW is 400 cycles against
|
||||
V4's 448 *and* is pixel-exact, so on the CPU axis V4 is strictly dominated —
|
||||
the byte lagrangian's preference inverts. `scsi` can take that escape and
|
||||
`sasi` cannot afford it, so expect the cycle ceiling to cost `sasi` more
|
||||
quality even though it costs `sasi` fewer cycles. FINDINGS 28.8.
|
||||
4. **Encoder gap, still open from session 7:** `encode.py` should pad frame
|
||||
records to 4 bytes. Frame boundaries land on odd addresses and a 68000 takes
|
||||
an address error, not a slow read (FINDINGS 28.3). `prep_dlx.py` pads at load
|
||||
time, which is why the decoder works; the container itself does not.
|
||||
Measured cost of fixing it: 1.5 B/frame = 18 B/s.
|
||||
|
||||
4. **Put spans in the bitstream** — the mode is measured and nothing implements
|
||||
it. This is a container change (`encode.py`, `dlx.py`, `decode.s`), a mode
|
||||
decision that can see runs rather than blocks, and the 24-pixel quantisation
|
||||
and row-overrun rules of FINDINGS 30.2. It subsumes item 4 of session 7's
|
||||
plan: with spans, a scene cut fits.
|
||||
|
||||
5. **The three things 30.7 leaves open, now more load-bearing than before**,
|
||||
because the span design runs at 448 KB/s of a 488 KB/s pipe: re-run the
|
||||
ring-buffer simulation at that rate (FINDINGS 21 was established at 110 and
|
||||
280), confirm the provenance of the 4 Mbps figure, and **confirm DMA rather
|
||||
than PIO** — a PIO fallback puts a 448 KB/s transfer back on the CPU this
|
||||
whole lever exists to relieve. The DMA check is the cheapest of the three.
|
||||
5. **A quality-vs-framerate question that is the user's, not the encoder's.**
|
||||
Every miss is now one frame per cut. The options remain: one late frame at
|
||||
each cut (the outgoing content is unrelated, so it may be invisible), a cut
|
||||
spread over two frame times, or 10fps. Spans (item 2) make the question go
|
||||
away if they land as modelled.
|
||||
|
||||
**Do not start by hand-optimising `decode.s`.** The hand-derived timings agree
|
||||
with the measurements to 0.5% on V1 and 1% on RAW (FINDINGS 28.4), so the
|
||||
inner loop is close to what the instruction set allows; the plausible wins are
|
||||
single-digit percentages against a 36-point gap. The V4 write pattern is the one
|
||||
place worth a look afterwards — pairing sub-block rows into `movem.l d0/d2,(a4)`
|
||||
saves ~16 of 448 cycles.
|
||||
with the measurements to 0.5% on V1 and 1% on RAW (FINDINGS 28.4), so the inner
|
||||
loop is close to what the instruction set allows.
|
||||
|
||||
---
|
||||
|
||||
|
||||
## What session 8 settled
|
||||
|
||||
0. **The mode decision can see cycles, it is on by default, and it costs
|
||||
0.26 dB.** `decide(ctx, lam, mu)` minimises `D + lam*bytes + mu*cycles`;
|
||||
`ratectl` bisects mu per frame against a HARD 833,333-cycle ceiling (bytes
|
||||
bank in the ring buffer, cycles cannot — there is no double buffer to decode
|
||||
ahead into). `sasi` 37/120 misses -> 1, `scsi` 51 -> 1. Bitrate does not
|
||||
move: mu changes which modes are bought, not how many bytes. FINDINGS 31,
|
||||
`tools/analysis/13_cpu_ratectl.py`.
|
||||
0b. **28.7's "11 frames are impossible" was too pessimistic — it is 1.** That
|
||||
floor held the SKIP set fixed; the real decision can also move a block to
|
||||
SKIP, which above ~90% non-SKIP is the only lever left. FINDINGS 31.3.
|
||||
0c. **V4 collapses when cycles are priced**, as 28.8 predicted: 25.2 -> 20.3%
|
||||
of blocks at `sasi` and **15.0 -> 5.3%** at `scsi`, where RAW takes it. RAW
|
||||
is dearer in bytes and cheaper in cycles, so the byte lagrangian's preference
|
||||
inverts and only the byte-rich profile can take the escape.
|
||||
0d. **SKIP's price depends on its neighbours, and the way out is two cost
|
||||
functions**: a ranking constant inside the per-block lagrangian, the exact
|
||||
clustered rule (`vq_hybrid.cycles`, validated to 1 point against the 68000)
|
||||
for the frame-level bisection. That function is now defined once and imported
|
||||
by `11_cpu_budget.py`. FINDINGS 31.4.
|
||||
0e. **Both controllers are gated against decoder drift.**
|
||||
`09_ratectl_drift.py` runs bytes-only AND bytes+cycles; both 0/120.
|
||||
1. **The span is measured: 43.7 cycles/span + 9.152/pixel, fitted to 0.3% over
|
||||
eleven span lengths.** `tools/bench/blit.s` v5/v6, `prep_spans.py`,
|
||||
`span.lua`, driven by `tools/bench/span.sh` (~25 s, not in `check.sh`
|
||||
@@ -217,10 +218,15 @@ saves ~16 of 448 cycles.
|
||||
```
|
||||
./tools/bench/check.sh
|
||||
```
|
||||
~3 min, needs the Blu-ray mounted. From source media it re-runs both display
|
||||
regression tests, the rate-control drift test (session 6), the display-path
|
||||
coherency counterexample and a **120-frame 68000 decode** (session 7), then
|
||||
prints `ALL GREEN`. Verified green at end of session 7.
|
||||
~4 min, needs the Blu-ray mounted. From source media it re-runs both display
|
||||
regression tests, the rate-control drift test (session 6, now covering BOTH
|
||||
controllers -- bytes, and bytes+cycles), the display-path coherency
|
||||
counterexample and a **120-frame 68000 decode** (session 7), then prints
|
||||
`ALL GREEN`. Verified green at end of session 8.
|
||||
|
||||
Do not run two of these at once, and do not run one alongside a MAME timing
|
||||
job: they share `tmp/` snapshot directories and log files, and the second run
|
||||
silently truncates the first one's output.
|
||||
If it fails, fix that before doing anything else — everything downstream assumes
|
||||
the display path is pixel-exact.
|
||||
|
||||
|
||||
@@ -19,6 +19,10 @@ paint), so drift is zero by construction rather than by tuning.
|
||||
This replays what a real decoder does -- SKIP copies the ACTUALLY EMITTED
|
||||
previous frame -- and compares it to the reconstruction ratectl recorded.
|
||||
|
||||
Session 8 added a SECOND controller (mu, the per-frame 68000 decode ceiling)
|
||||
that also varies the mode map frame to frame, so it is exposed to exactly the
|
||||
same failure and is tested here too. Both configurations must show zero drift.
|
||||
|
||||
Needs tmp/fr_singe (see docs/STATUS.md, reproducing the sustained-action
|
||||
result). ~55 s, nearly all of it the k-means in H.build; the rate-controlled
|
||||
encode of 120 frames is ~2 s.
|
||||
@@ -29,49 +33,63 @@ import numpy as np
|
||||
import vq as VQ, vq_hybrid as H, ratectl as RC
|
||||
|
||||
m = H.build("tmp/fr_singe", k1=256, k4=256, iters=16)
|
||||
# lam_lo=1.0: let quiet frames spend the whole allowance, which is the
|
||||
# harder case for this test -- it maximises how often lam moves frame to frame.
|
||||
enc = RC.encode_rate_controlled(m, target_kbps=110, lam_lo=1.0)
|
||||
|
||||
lam = enc["lam"]
|
||||
sw = int((np.diff(lam) != 0).sum())
|
||||
print(f"frames={len(lam)} distinct lam used={len(set(lam.tolist()))} "
|
||||
f"lam changes frame-to-frame={sw} "
|
||||
f"overruns={int(enc['overrun'].sum())}")
|
||||
|
||||
pal, nbx = m["pal"], m["W"] // 4
|
||||
emitted = []
|
||||
drift_px, drift_db = [], []
|
||||
for f, (rec, mode) in enumerate(zip(enc["recon"], enc["modes"])):
|
||||
out = rec.copy()
|
||||
if f > 0:
|
||||
prev_true = emitted[-1]
|
||||
for b in np.flatnonzero(mode == 0): # SKIP blocks
|
||||
by, bx = divmod(int(b), nbx)
|
||||
y, x = by*4, bx*4
|
||||
out[y:y+4, x:x+4] = prev_true[y:y+4, x:x+4]
|
||||
emitted.append(out)
|
||||
d = (out != rec).sum()
|
||||
drift_px.append(d)
|
||||
drift_db.append(VQ.psnr(pal[rec], pal[out]))
|
||||
def check(label, cycle_budget):
|
||||
"""Encode, replay as a decoder would, and return the drift in pixels."""
|
||||
print(f"\n=== {label} ===")
|
||||
m.pop("_sym", None)
|
||||
# lam_lo=1.0: let quiet frames spend the whole allowance, which is the
|
||||
# harder case for this test -- it maximises how often lam moves frame to
|
||||
# frame.
|
||||
enc = RC.encode_rate_controlled(m, target_kbps=110, lam_lo=1.0,
|
||||
cycle_budget=cycle_budget)
|
||||
lam = enc["lam"]
|
||||
sw = int((np.diff(lam) != 0).sum())
|
||||
print(f"frames={len(lam)} distinct lam used={len(set(lam.tolist()))} "
|
||||
f"lam changes frame-to-frame={sw} "
|
||||
f"overruns={int(enc['overrun'].sum())}")
|
||||
|
||||
drift_px = np.array(drift_px)
|
||||
print(f"pixels differing from what the encoder recorded:")
|
||||
print(f" frames with ANY drift: {int((drift_px>0).sum())}/{len(drift_px)}")
|
||||
print(f" max {drift_px.max()} px ({100*drift_px.max()/(m['H']*m['W']):.1f}% of frame)")
|
||||
print(f" mean {drift_px.mean():.0f} px")
|
||||
fin = [d for d in drift_db if np.isfinite(d)]
|
||||
if fin:
|
||||
print(f" encoder-vs-decoder agreement: min {min(fin):.1f} dB "
|
||||
f"(inf = identical on {len(drift_db)-len(fin)} frames)")
|
||||
pal, nbx = m["pal"], m["W"] // 4
|
||||
emitted = []
|
||||
drift_px, drift_db = [], []
|
||||
for f, (rec, mode) in enumerate(zip(enc["recon"], enc["modes"])):
|
||||
out = rec.copy()
|
||||
if f > 0:
|
||||
prev_true = emitted[-1]
|
||||
for b in np.flatnonzero(mode == 0): # SKIP blocks
|
||||
by, bx = divmod(int(b), nbx)
|
||||
y, x = by*4, bx*4
|
||||
out[y:y+4, x:x+4] = prev_true[y:y+4, x:x+4]
|
||||
emitted.append(out)
|
||||
d = (out != rec).sum()
|
||||
drift_px.append(d)
|
||||
drift_db.append(VQ.psnr(pal[rec], pal[out]))
|
||||
|
||||
drift_px = np.array(drift_px)
|
||||
print(f"pixels differing from what the encoder recorded:")
|
||||
print(f" frames with ANY drift: {int((drift_px>0).sum())}/{len(drift_px)}")
|
||||
print(f" max {drift_px.max()} px ({100*drift_px.max()/(m['H']*m['W']):.1f}% of frame)")
|
||||
print(f" mean {drift_px.mean():.0f} px")
|
||||
fin = [d for d in drift_db if np.isfinite(d)]
|
||||
if fin:
|
||||
print(f" encoder-vs-decoder agreement: min {min(fin):.1f} dB "
|
||||
f"(inf = identical on {len(drift_db)-len(fin)} frames)")
|
||||
|
||||
r = RC.summarise(m, enc, 110)
|
||||
print(f"\nratectl reports PSNR {r['psnr']:.2f} dB, {r['kbps']:.1f} KB/s "
|
||||
f"(target 110), {r['over']:.0f}% of frames over budget")
|
||||
tp = np.mean([VQ.psnr(o, pal[e]) for o, e in zip(m["rgb"], emitted)])
|
||||
print(f"what a decoder actually reconstructs: {tp:.2f} dB "
|
||||
f"-> overstated by {r['psnr']-tp:.2f} dB")
|
||||
return drift_px
|
||||
|
||||
r = RC.summarise(m, enc, 110)
|
||||
print(f"\nratectl reports PSNR {r['psnr']:.2f} dB, {r['kbps']:.1f} KB/s "
|
||||
f"(target 110), {r['over']:.0f}% of frames over budget")
|
||||
tp = np.mean([VQ.psnr(o, pal[e]) for o, e in zip(m["rgb"], emitted)])
|
||||
print(f"what a decoder actually reconstructs: {tp:.2f} dB "
|
||||
f"-> overstated by {r['psnr']-tp:.2f} dB")
|
||||
|
||||
# Acceptance criterion for the fix: a decoder replaying the emitted stream must
|
||||
# reconstruct exactly what the encoder recorded.
|
||||
sys.exit(1 if (drift_px > 0).any() else 0)
|
||||
# reconstruct exactly what the encoder recorded -- under either controller.
|
||||
bad = 0
|
||||
for label, cb in (("bytes only (session 6)", None),
|
||||
("bytes + CPU ceiling (session 8)", RC.FRAME_CYCLES)):
|
||||
d = check(label, cb)
|
||||
bad += int((d > 0).any())
|
||||
sys.exit(1 if bad else 0)
|
||||
|
||||
@@ -22,6 +22,7 @@ import sys, os, argparse
|
||||
sys.path.insert(0, "tools/encoder")
|
||||
import numpy as np
|
||||
from dlx import DLX
|
||||
import vq_hybrid as H
|
||||
|
||||
# Machine clocks, confirmed from MAME 0.277 src/mame/sharp/x68k.cpp:1133/1194/
|
||||
# 1200 -- not recalled. x68000 and x68ksupr are BOTH 40_MHz_XTAL/4 = 10 MHz;
|
||||
@@ -30,10 +31,13 @@ from dlx import DLX
|
||||
CLOCKS = {"stock": 10.0, "super": 10.0, "xvi": 33.33 / 2, "x68030": 25.0}
|
||||
FPS = 12
|
||||
|
||||
# cycles per block, measured on the emulated 68000 (synthetic single-mode frames)
|
||||
C_V1, C_V4, C_RAW = 299.9, 448.2, 400.4
|
||||
C_SKIP_FAST = 53.0 / 4 # all-SKIP header byte: one tst.b for 4
|
||||
C_SKIP_MIXED = 45.0 # a SKIP block inside a mixed byte
|
||||
# Cycles per block, measured on the emulated 68000 (synthetic single-mode
|
||||
# frames). Defined in tools/encoder/vq_hybrid.py, which is where the mode
|
||||
# decision needs them too -- one copy, not two, so a re-measurement cannot
|
||||
# leave the encoder and the scorer disagreeing.
|
||||
C_V1, C_V4, C_RAW = H.C_V1, H.C_V4, H.C_RAW
|
||||
C_SKIP_FAST, C_SKIP_MIXED = H.C_SKIP_CLUSTERED, H.C_SKIP_MIXED
|
||||
cycles = H.cycles
|
||||
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("container", nargs="?",
|
||||
@@ -50,17 +54,6 @@ if not os.path.exists(a.container):
|
||||
|
||||
d = DLX(a.container)
|
||||
|
||||
def cycles(mode):
|
||||
g = mode.reshape(-1, 4) # one header byte = four blocks
|
||||
allskip = (g == 0).all(1)
|
||||
c = allskip.sum() * 4 * C_SKIP_FAST
|
||||
m = g[~allskip]
|
||||
c += (m == 0).sum() * C_SKIP_MIXED
|
||||
c += (m == 1).sum() * C_V1
|
||||
c += (m == 2).sum() * C_V4
|
||||
c += (m == 3).sum() * C_RAW
|
||||
return c
|
||||
|
||||
modes = [d.modes(f) for f in range(d.nframes)]
|
||||
cyc = np.array([cycles(m) for m in modes])
|
||||
pct = 100 * cyc / FRAME
|
||||
|
||||
@@ -0,0 +1,101 @@
|
||||
#!/usr/bin/env python3
|
||||
"""What does fitting the CPU budget cost in quality? (session 8, lever B)
|
||||
|
||||
python3 tools/analysis/13_cpu_ratectl.py [frames_dir] [--profiles sasi,scsi]
|
||||
|
||||
Session 6 made the BYTE budget a ceiling by bisecting `lam` per frame. FINDINGS
|
||||
28 then showed the binding budget is CYCLES, not bytes, and that the mode
|
||||
decision cannot see them: it minimises `D + lam*R` on a machine that charges V4
|
||||
1.49x a V1 block while the lagrangian charges it 4x.
|
||||
|
||||
`ratectl.encode_rate_controlled(cycle_budget=...)` adds the second controller --
|
||||
`mu` bisected per frame against 833,333 cycles, with the lam bisection nested
|
||||
inside it. This measures what that costs: PSNR, bitrate, and how many frames
|
||||
still miss, against the same encode with the ceiling off.
|
||||
|
||||
The cycle budget is HARD, not a bucket. Bytes bank in the player's ring buffer;
|
||||
there is no double buffer to decode ahead into, so a frame that misses its
|
||||
decode deadline is simply late (FINDINGS 28).
|
||||
|
||||
Both controllers score frames with the exact clustered cost `vq_hybrid.cycles`,
|
||||
validated to 1 point against the 68000 (FINDINGS 28.2) -- not with the per-block
|
||||
ranking constant the mode decision uses. See vq_hybrid's note on SKIP.
|
||||
"""
|
||||
import argparse, os, pickle, sys, time
|
||||
sys.path.insert(0, "tools/encoder")
|
||||
import numpy as np
|
||||
import vq as VQ, vq_hybrid as H, ratectl as RC
|
||||
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("frames_dir", nargs="?", default="tmp/fr_singe")
|
||||
ap.add_argument("--profiles", default="sasi,scsi")
|
||||
ap.add_argument("--fps", type=int, default=12)
|
||||
ap.add_argument("--cache", default=None, help="pickle of H.build (auto by dir)")
|
||||
a = ap.parse_args()
|
||||
if not os.path.isdir(a.frames_dir):
|
||||
sys.exit(f"missing {a.frames_dir} -- see tools/bench/check.sh for extraction")
|
||||
|
||||
BUDGET = RC.FRAME_CYCLES
|
||||
|
||||
# H.build is ~55 s, nearly all k-means, and it does not depend on the profile:
|
||||
# both ship k1=k4=256. One build, cached, serves every row of the table.
|
||||
cache = a.cache or f"tmp/model_{os.path.basename(a.frames_dir.rstrip('/'))}.pkl"
|
||||
if os.path.exists(cache):
|
||||
m = pickle.load(open(cache, "rb"))
|
||||
print(f"model from {cache}")
|
||||
else:
|
||||
t = time.time()
|
||||
m = H.build(a.frames_dir, k1=256, k4=256, iters=16)
|
||||
pickle.dump(m, open(cache, "wb"))
|
||||
print(f"built model in {time.time()-t:.0f} s -> {cache}")
|
||||
print(f"{a.frames_dir}: {len(m['idx'])} frames, {m['nb']} blocks, "
|
||||
f"budget {BUDGET:,.0f} cycles/frame at {a.fps}fps\n")
|
||||
|
||||
|
||||
def run(prof_name, cycle_budget):
|
||||
p = RC.PROFILES[prof_name]
|
||||
m.pop("_sym", None) # the frame-symbol cache holds one frame
|
||||
t = time.time()
|
||||
enc = RC.encode_rate_controlled(m, p["kbps"], fps=a.fps, lam_lo=p["lam"],
|
||||
cycle_budget=cycle_budget)
|
||||
s = RC.summarise(m, enc, p["kbps"], fps=a.fps)
|
||||
s["secs"] = time.time() - t
|
||||
s["ns"] = float(np.mean([100*(mm != 0).mean() for mm in enc["modes"]]))
|
||||
return s, enc
|
||||
|
||||
|
||||
rows = []
|
||||
for name in a.profiles.split(","):
|
||||
for label, cb in (("bytes only", None), ("bytes + cycles", BUDGET)):
|
||||
s, enc = run(name, cb)
|
||||
rows.append((name, label, s))
|
||||
print(f"{name:5s} {label:<15s} {s['secs']:5.1f} s "
|
||||
f"PSNR {s['psnr']:.2f} dB {s['kbps']:6.1f} KB/s "
|
||||
f"CPU med {100*s['cyc_med']/BUDGET:5.1f}% p90 "
|
||||
f"{100*s['cyc_p90']/BUDGET:5.1f}% max {100*s['cyc_max']/BUDGET:5.1f}% "
|
||||
f"miss {s['cpu_miss']:3d} late {s['late']:2d} "
|
||||
f"mu med {s['mu_med']:.4f} max {s['mu_max']:.3f}")
|
||||
|
||||
print()
|
||||
hdr = f"{'':<22}{'PSNR':>8}{'KB/s':>9}{'CPU med':>10}{'CPU max':>10}{'miss':>7}"
|
||||
for name in a.profiles.split(","):
|
||||
r = {lab: s for n, lab, s in rows if n == name}
|
||||
b, c = r["bytes only"], r["bytes + cycles"]
|
||||
print(f"--- {name} (target {RC.PROFILES[name]['kbps']} KB/s) ---")
|
||||
print(hdr)
|
||||
for lab, s in (("bytes only", b), ("bytes + cycles", c)):
|
||||
print(f" {lab:<20}{s['psnr']:>7.2f} {s['kbps']:>8.1f} "
|
||||
f"{100*s['cyc_med']/BUDGET:>9.1f}%{100*s['cyc_max']/BUDGET:>9.1f}%"
|
||||
f"{s['cpu_miss']:>6d}")
|
||||
print(f" {'cost of fitting':<20}{c['psnr']-b['psnr']:>+7.2f} dB, "
|
||||
f"{c['kbps']-b['kbps']:+.1f} KB/s, "
|
||||
f"{b['cpu_miss']-c['cpu_miss']} fewer misses, "
|
||||
f"{c['late']} frames unfixable at mu={RC.MU_CLIFF:g}")
|
||||
print(f" {'modes % (b/c)':<20}SKIP {b['skip']:.1f}/{c['skip']:.1f} "
|
||||
f"V1 {b['v1']:.1f}/{c['v1']:.1f} V4 {b['v4']:.1f}/{c['v4']:.1f} "
|
||||
f"RAW {b['raw']:.1f}/{c['raw']:.1f}")
|
||||
print()
|
||||
|
||||
print("FINDINGS 28.7: re-coding every non-SKIP block as V1 is the floor the "
|
||||
"CURRENT mode set\nallows, and it still misses 11 frames at sasi / 12 at "
|
||||
"scsi. Misses above that floor\nare item 4 (spans), not item 1.")
|
||||
@@ -157,6 +157,11 @@ SUB = emu.add_machine_frame_notifier(function()
|
||||
M.video:snapshot()
|
||||
P("snapshot taken after the sequential pass -- last frame, 68000-decoded")
|
||||
step = step + 1
|
||||
-- DLX_VERIFY_ONLY leaves nothing after the correctness pass, and this
|
||||
-- used to walk off the end of PLAN and raise a Lua error AFTER the
|
||||
-- snapshot was already on disk -- harmless to check.sh, and exactly the
|
||||
-- kind of thing that gets mistaken for a decoder failure later.
|
||||
if not PLAN[step] then st = "finish"; return end
|
||||
launch(PLAN[step].off, PLAN[step].nfr, PLAN[step].iter)
|
||||
st, t0 = "running", nil; return
|
||||
end
|
||||
|
||||
+37
-15
@@ -93,6 +93,9 @@ def main():
|
||||
help="quality floor for rate control")
|
||||
ap.add_argument("--bucket-frames", type=int, default=8,
|
||||
help="leaky-bucket depth, in frame budgets")
|
||||
ap.add_argument("--no-cpu-fit", action="store_true",
|
||||
help="drop the per-frame 68000 decode ceiling (session 7 "
|
||||
"behaviour: 31%% of frames on hard content do not fit)")
|
||||
ap.add_argument("--prefill", type=float, default=0.0,
|
||||
help="how full the player's buffer is assumed to be at "
|
||||
"scene start, as a fraction of the bucket (0 = cold "
|
||||
@@ -108,12 +111,19 @@ def main():
|
||||
# An explicit --lam is a request for that lam, so it implies --fixed-lam.
|
||||
rc = not (a.fixed_lam or a.lam is not None)
|
||||
lam_lo = lam if a.rc_floor == "profile" else 1.0
|
||||
# The CPU ceiling is hardware, not taste: without it 31%% of frames on the
|
||||
# worst sustained window do not decode in time on a stock 68000, and with
|
||||
# it that is one frame -- the intra frame -- for 0.26 dB. FINDINGS 31.
|
||||
cyc_budget = None if a.no_cpu_fit else RC.FRAME_CYCLES
|
||||
|
||||
print(f"profile {a.profile}: {prof['desc']}")
|
||||
if rc:
|
||||
print(f" target {prof['kbps']} KB/s CEILING, rate-controlled: "
|
||||
f"lam bisected per frame in [{lam_lo:g}, {RC.LAM_CLIFF:g}], "
|
||||
f"{a.bucket_frames}-frame bucket")
|
||||
print(f" CPU ceiling: " + (f"mu bisected per frame against "
|
||||
f"{RC.FRAME_CYCLES:,.0f} cycles (12fps, stock 68000)"
|
||||
if cyc_budget else "OFF (--no-cpu-fit)"))
|
||||
else:
|
||||
print(f" target {prof['kbps']} KB/s, FIXED lam={lam} (no rate control)")
|
||||
print(f" k1={k1} k4={k4}, {_IDX_BYTES}-byte indices")
|
||||
@@ -122,7 +132,8 @@ def main():
|
||||
if rc:
|
||||
enc = RC.encode_rate_controlled(m, prof["kbps"], fps=a.fps,
|
||||
bucket_frames=a.bucket_frames,
|
||||
lam_lo=lam_lo, prefill=a.prefill)
|
||||
lam_lo=lam_lo, prefill=a.prefill,
|
||||
cycle_budget=cyc_budget)
|
||||
else:
|
||||
enc = H.encode(m, lam=lam)
|
||||
r = H.evaluate(m, enc, fps=a.fps)
|
||||
@@ -186,23 +197,34 @@ def main():
|
||||
print(f" frames that could not fit even at the lam={RC.LAM_CLIFF:g} "
|
||||
f"cliff: {rr['overrun']}/{len(lm)}")
|
||||
|
||||
# PER-FRAME non-SKIP distribution. The mean above cannot answer the
|
||||
# decoder-architecture question (FINDINGS 24.5): decode-direct-to-GVRAM
|
||||
# costs 76.6% of a 12fps frame budget x (non-SKIP fraction), while
|
||||
# compose-in-RAM-then-blit is a flat 53.6% regardless. They cross at 70%,
|
||||
# and that is a decision taken FRAME BY FRAME -- a scene cut is ~100%
|
||||
# non-SKIP and a held frame near 0%, so their mean describes no real frame.
|
||||
# PER-FRAME DECODE COST, from the measured per-mode block costs
|
||||
# (FINDINGS 28.2, vq_hybrid.cycles). The mean cannot answer this: a scene
|
||||
# cut is ~100% non-SKIP and a held frame near 0%, so their mean describes
|
||||
# no real frame. What matters is how many frames MISS, and by how much.
|
||||
#
|
||||
# This replaces the per-frame blit-vs-direct path choice that used to be
|
||||
# printed here. That plan is withdrawn -- mixing the two paths displays
|
||||
# stale pixels on 70 of 120 frames, and there was never a crossover to
|
||||
# begin with, because the compose path pays the blit ON TOP of decoding.
|
||||
# FINDINGS 28.1/28.4. The player has one path and no reference frame.
|
||||
ns = np.array([100 * (mm != 0).mean() for mm in enc["modes"]])
|
||||
over = int((ns > CROSSOVER_PCT).sum())
|
||||
cyc = np.array([H.cycles(mm) for mm in enc["modes"]])
|
||||
pct = 100 * cyc / RC.FRAME_CYCLES
|
||||
miss = int((pct > 100).sum())
|
||||
print(f" non-SKIP blocks/frame: median {np.median(ns):.1f}% "
|
||||
f"p90 {np.percentile(ns, 90):.1f}% max {ns.max():.1f}%")
|
||||
print(f" frames above the {CROSSOVER_PCT:.0f}% blit crossover: "
|
||||
f"{over}/{len(ns)} ({100*over/len(ns):.1f}%) -> "
|
||||
f"{'compose+blit wins on those' if over else 'direct-to-GVRAM wins throughout'}")
|
||||
cost = np.minimum(BLIT_PCT, DIRECT_PCT * ns / 100)
|
||||
print(f" display cost if the player picks the cheaper path per frame: "
|
||||
f"median {np.median(cost):.1f}% p90 {np.percentile(cost, 90):.1f}% "
|
||||
f"max {cost.max():.1f}% of a 12fps frame")
|
||||
print(f" decode cost: median {np.median(pct):.1f}% "
|
||||
f"p90 {np.percentile(pct, 90):.1f}% max {pct.max():.1f}% "
|
||||
f"of a {a.fps}fps frame")
|
||||
print(f" frames that do NOT decode in time: {miss}/{len(pct)} "
|
||||
f"({100*miss/len(pct):.0f}%)"
|
||||
+ (f" -- worst {pct.max():.1f}%" if miss else ""))
|
||||
if rc and cyc_budget:
|
||||
rr2 = RC.summarise(m, enc, prof["kbps"], fps=a.fps)
|
||||
print(f" mu: median {rr2['mu_med']:.4f} max {rr2['mu_max']:.3f} "
|
||||
f"frames needing any mu at all: {int((enc['mu'] > 0).sum())}/{len(pct)}")
|
||||
print(f" frames that cannot fit even at mu={RC.MU_CLIFF:g} "
|
||||
f"(emitted late on purpose): {rr2['late']}")
|
||||
|
||||
if a.preview:
|
||||
from PIL import Image
|
||||
|
||||
+101
-10
@@ -81,6 +81,27 @@ PROFILES = {
|
||||
# instead (FINDINGS 26.2). The old ladder ran to lam=2e5, 250x past shippable.
|
||||
LAM_CLIFF = 800.0
|
||||
|
||||
# Ceiling on the CYCLE search. mu prices a cycle in the same units lam prices a
|
||||
# byte, so the scale that matters is set by their ratio: at the `sasi` floor of
|
||||
# lam=60, mu=0.2 makes a V1 block's 300 cycles cost what its 1 payload byte
|
||||
# costs. MU_CLIFF=100 is three decades past that: a V1 block priced at 30,000
|
||||
# distortion units.
|
||||
#
|
||||
# It does NOT freeze the picture, and that is the point. At MU_CLIFF a block
|
||||
# only becomes SKIP if holding the previous reconstruction costs less than
|
||||
# 28,665 units of distortion, so a frame with nothing on screen to hold -- the
|
||||
# first frame of a stream, or a scene cut -- stays fully coded and comes out at
|
||||
# the all-V1 floor of 110.6% (FINDINGS 28.5). Such a frame is emitted LATE on
|
||||
# purpose, exactly as a frame that will not fit at LAM_CLIFF is emitted over
|
||||
# budget. Freezing a cut to make the deadline would be the worse failure.
|
||||
MU_CLIFF = 100.0
|
||||
MU_FLOOR = 1e-4 # bisection is geometric, so lo must be > 0
|
||||
|
||||
# The hard per-frame decode budget. NOT a bucket: bytes can be banked in the
|
||||
# player's ring buffer, but there is no double buffer to decode ahead into, so
|
||||
# a frame that misses its deadline is simply late. FINDINGS 28.
|
||||
FRAME_CYCLES = 10_000_000 / 12.0
|
||||
|
||||
AUDIO_KBPS = 7.8 # MSM6258 ADPCM 15.6kHz mono -- comes out of the same budget
|
||||
|
||||
|
||||
@@ -89,7 +110,7 @@ def frame_budget(kbps, fps=12, audio=AUDIO_KBPS):
|
||||
return (kbps - audio) * 1024.0 / fps
|
||||
|
||||
|
||||
def _search_lam(ctx, allow, lam_lo, lam_hi, iters=12):
|
||||
def _search_lam(ctx, allow, lam_lo, lam_hi, iters=12, mu=0.0):
|
||||
"""Smallest lam (=> best quality) whose frame fits `allow` bytes.
|
||||
|
||||
Payload size is non-increasing in lam -- raising lam can only move a block
|
||||
@@ -100,17 +121,17 @@ def _search_lam(ctx, allow, lam_lo, lam_hi, iters=12):
|
||||
not fit: that frame is emitted over budget on purpose. Past the FINDINGS 15
|
||||
cliff a frame is not rate-controlled, it is destroyed, so a visible overrun
|
||||
is the better failure (FINDINGS 26.2)."""
|
||||
mode, sz = H.decide(ctx, lam_lo)
|
||||
mode, sz = H.decide(ctx, lam_lo, mu)
|
||||
if sz <= allow:
|
||||
return lam_lo, mode, sz, False
|
||||
mode_hi, sz_hi = H.decide(ctx, lam_hi)
|
||||
mode_hi, sz_hi = H.decide(ctx, lam_hi, mu)
|
||||
if sz_hi > allow:
|
||||
return lam_hi, mode_hi, sz_hi, True
|
||||
lo, hi = lam_lo, lam_hi # lo does not fit, hi does
|
||||
best = (lam_hi, mode_hi, sz_hi)
|
||||
for _ in range(iters):
|
||||
mid = float(np.sqrt(lo * hi))
|
||||
mode_m, sz_m = H.decide(ctx, mid)
|
||||
mode_m, sz_m = H.decide(ctx, mid, mu)
|
||||
if sz_m <= allow:
|
||||
hi = mid; best = (mid, mode_m, sz_m)
|
||||
else:
|
||||
@@ -118,9 +139,55 @@ def _search_lam(ctx, allow, lam_lo, lam_hi, iters=12):
|
||||
return best[0], best[1], best[2], False
|
||||
|
||||
|
||||
def _search_mu(ctx, allow, lam_lo, lam_hi, cyc_budget, iters=10):
|
||||
"""Smallest mu whose frame fits BOTH budgets: `allow` bytes and
|
||||
`cyc_budget` 68000 cycles.
|
||||
|
||||
Two controllers, one nested inside the other, because the constraints are
|
||||
not separable. Raising mu moves blocks to cheaper-to-DECODE modes, which
|
||||
usually also shrinks the frame -- but not always: RAW is 400 cycles against
|
||||
V4's 448 and 16 bytes against 4, so mu can buy cycles by SPENDING bytes
|
||||
(FINDINGS 28.8). So every mu step re-runs the lam bisection and the byte
|
||||
budget is enforced at the mu that is actually chosen.
|
||||
|
||||
Cost is scored with H.cycles(), the exact clustered rule, NOT with the
|
||||
per-block ranking constant the decision uses -- see vq_hybrid's note on
|
||||
SKIP. The controller therefore converges on what the 68000 will really do.
|
||||
|
||||
Monotonicity: at a fixed lam, raising mu can only move a block to a mode
|
||||
that costs no more cycles, and it can only ADD to a SKIP cluster, so frame
|
||||
cycles are non-increasing in mu. The nested lam re-search can perturb that
|
||||
at the margin (a smaller frame permits a smaller lam, which buys quality
|
||||
back and can cost a few cycles), so the bisection keeps the best FEASIBLE
|
||||
point it has actually seen rather than trusting the invariant.
|
||||
|
||||
Returns (mu, lam, mode, size, cyc, over_bytes, over_cycles)."""
|
||||
lam, mode, sz, ovr = _search_lam(ctx, allow, lam_lo, lam_hi, mu=0.0)
|
||||
cyc = H.cycles(mode)
|
||||
if cyc <= cyc_budget:
|
||||
return 0.0, lam, mode, sz, cyc, ovr, False
|
||||
|
||||
lam_h, mode_h, sz_h, ovr_h = _search_lam(ctx, allow, lam_lo, lam_hi, mu=MU_CLIFF)
|
||||
cyc_h = H.cycles(mode_h)
|
||||
if cyc_h > cyc_budget: # cannot fit even frozen: emit late
|
||||
return MU_CLIFF, lam_h, mode_h, sz_h, cyc_h, ovr_h, True
|
||||
|
||||
lo, hi = MU_FLOOR, MU_CLIFF # lo overruns, hi fits
|
||||
best = (MU_CLIFF, lam_h, mode_h, sz_h, cyc_h, ovr_h)
|
||||
for _ in range(iters):
|
||||
mid = float(np.sqrt(lo * hi))
|
||||
lam_m, mode_m, sz_m, ovr_m = _search_lam(ctx, allow, lam_lo, lam_hi, mu=mid)
|
||||
cyc_m = H.cycles(mode_m)
|
||||
if cyc_m <= cyc_budget:
|
||||
hi = mid; best = (mid, lam_m, mode_m, sz_m, cyc_m, ovr_m)
|
||||
else:
|
||||
lo = mid
|
||||
return (*best, False)
|
||||
|
||||
|
||||
def encode_rate_controlled(m, target_kbps, fps=12, bucket_frames=8,
|
||||
lam_lo=1.0, lam_hi=LAM_CLIFF, prefill=0.0,
|
||||
steps=None, verbose=False):
|
||||
steps=None, verbose=False, cycle_budget=None):
|
||||
"""Per-frame lam search under a leaky bucket, driving the encoder ONE FRAME
|
||||
AT A TIME and feeding back the frame actually emitted.
|
||||
|
||||
@@ -149,6 +216,12 @@ def encode_rate_controlled(m, target_kbps, fps=12, bucket_frames=8,
|
||||
109.5 to 116.3 KB/s against a 110 ceiling, and on a 14-frame clip it
|
||||
disables rate control entirely because the bucket is larger than the clip.
|
||||
|
||||
`cycle_budget` adds the SECOND controller (session 8): a hard per-frame
|
||||
68000 decode ceiling, bisected on `mu` inside the lam search. None (the
|
||||
default) leaves it off and reproduces session 6 exactly, which is what
|
||||
keeps tools/analysis/09_ratectl_drift.py comparable. Pass
|
||||
FRAME_CYCLES for the 12fps stock-68000 budget.
|
||||
|
||||
`steps` is accepted and ignored -- there is no ladder any more.
|
||||
"""
|
||||
if steps is not None and verbose:
|
||||
@@ -156,25 +229,35 @@ def encode_rate_controlled(m, target_kbps, fps=12, bucket_frames=8,
|
||||
budget = frame_budget(target_kbps, fps)
|
||||
cap = bucket_frames * budget
|
||||
bucket = prefill * cap # banked bytes; bounded by the player's buffer both ways
|
||||
out = dict(recon=[], modes=[], sizes=[], lam=[], l1=[], l4g=[], overrun=[])
|
||||
out = dict(recon=[], modes=[], sizes=[], lam=[], l1=[], l4g=[], overrun=[],
|
||||
mu=[], cycles=[], late=[])
|
||||
prev = None
|
||||
for f in range(len(m["idx"])):
|
||||
ctx = H.frame_ctx(m, f, prev)
|
||||
allow = budget + bucket
|
||||
lam, mode, sz, ovr = _search_lam(ctx, allow, lam_lo, lam_hi)
|
||||
if cycle_budget is None:
|
||||
lam, mode, sz, ovr = _search_lam(ctx, allow, lam_lo, lam_hi)
|
||||
mu, cyc, late = 0.0, H.cycles(mode), False
|
||||
else:
|
||||
mu, lam, mode, sz, cyc, ovr, late = _search_mu(
|
||||
ctx, allow, lam_lo, lam_hi, cycle_budget)
|
||||
rec = H.paint(m, ctx, mode)
|
||||
bucket = float(np.clip(bucket + budget - sz, -cap, cap))
|
||||
out["recon"].append(rec); out["modes"].append(mode)
|
||||
out["sizes"].append(sz); out["lam"].append(lam); out["overrun"].append(ovr)
|
||||
out["mu"].append(mu); out["cycles"].append(cyc); out["late"].append(late)
|
||||
out["l1"].append(ctx["sym"]["l1"]); out["l4g"].append(ctx["sym"]["l4g"])
|
||||
prev = rec
|
||||
if verbose:
|
||||
print(f" f{f:04d} lam={lam:8.2f} {sz:7.0f} B "
|
||||
f"(allow {allow:7.0f}){' OVER' if ovr else ''}")
|
||||
print(f" f{f:04d} lam={lam:8.2f} mu={mu:8.4f} {sz:7.0f} B "
|
||||
f"(allow {allow:7.0f}) {100*cyc/FRAME_CYCLES:5.1f}% cpu"
|
||||
f"{' OVER' if ovr else ''}{' LATE' if late else ''}")
|
||||
return dict(recon=out["recon"], modes=out["modes"],
|
||||
sizes=np.array(out["sizes"]), lam=np.array(out["lam"]),
|
||||
l1=out["l1"], l4g=out["l4g"], overrun=np.array(out["overrun"]),
|
||||
nb=m["nb"], budget=budget, cap=cap)
|
||||
mu=np.array(out["mu"]), cycles=np.array(out["cycles"]),
|
||||
late=np.array(out["late"]),
|
||||
nb=m["nb"], budget=budget, cap=cap, cycle_budget=cycle_budget)
|
||||
|
||||
|
||||
def summarise(m, enc, target_kbps, fps=12):
|
||||
@@ -192,6 +275,14 @@ def summarise(m, enc, target_kbps, fps=12):
|
||||
over=100.0 * np.mean(sz > enc.get("budget", np.inf)),
|
||||
skip=100 * (mo == 0).mean(), v1=100 * (mo == 1).mean(),
|
||||
v4=100 * (mo == 2).mean(), raw=100 * (mo == 3).mean())
|
||||
if "cycles" in enc:
|
||||
cy = np.asarray(enc["cycles"])
|
||||
d.update(cyc_med=float(np.median(cy)), cyc_max=float(cy.max()),
|
||||
cyc_p90=float(np.percentile(cy, 90)),
|
||||
cpu_miss=int((cy > FRAME_CYCLES).sum()),
|
||||
mu_med=float(np.median(enc["mu"])),
|
||||
mu_max=float(np.asarray(enc["mu"]).max()),
|
||||
late=int(np.asarray(enc.get("late", [])).sum()))
|
||||
if "lam" in enc:
|
||||
lam = enc["lam"]
|
||||
d.update(lam_med=float(np.median(lam)), lam_max=float(lam.max()),
|
||||
|
||||
+60
-10
@@ -42,6 +42,49 @@ LUMA = VQ.LUMA
|
||||
_HDR_BYTES_PER_BLOCK = 2 / 8.0
|
||||
RAW_BYTES = 16.0 # literal palette bytes, never indices
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# CYCLE cost of each mode, per block, MEASURED on the 68000 (FINDINGS 28.2,
|
||||
# tools/bench/decode.lua). This is the other axis: `lam` prices bytes, `mu`
|
||||
# prices cycles, and the two are not proportional -- V4 is 4x a V1 block in
|
||||
# bytes and 1.49x in cycles.
|
||||
#
|
||||
# SKIP IS NOT A CONSTANT, and it is the one trap in here. A SKIP block costs
|
||||
# 13.25 cycles when all four blocks sharing its header byte are SKIP (one
|
||||
# `tst.b` clears the group) and ~45 when it sits in a mixed byte -- so its
|
||||
# price depends on its NEIGHBOURS, which a per-block lagrangian cannot see.
|
||||
# The way out is that the two uses do not need the same number:
|
||||
# * `decide` uses C_SKIP_RANK purely to RANK modes within a block. SKIP is
|
||||
# the cheapest mode either way, so the choice only scales the incentive:
|
||||
# the V1-SKIP gap moves 12% between the two candidates.
|
||||
# * `cycles()` scores a WHOLE frame with the exact clustered rule, and that
|
||||
# is what the rate controller bisects against. Nothing downstream of the
|
||||
# mode decision uses the ranking constant.
|
||||
C_V1, C_V4, C_RAW = 299.9, 448.2, 400.4
|
||||
C_SKIP_CLUSTERED = 53.0 / 4 # all-SKIP header byte: one tst.b for four
|
||||
C_SKIP_MIXED = 45.0 # a SKIP block inside a mixed byte
|
||||
C_SKIP_RANK = C_SKIP_CLUSTERED # ranking only -- see above
|
||||
MODE_CYCLES = np.array([C_SKIP_RANK, C_V1, C_V4, C_RAW], dtype=np.float64)
|
||||
|
||||
FRAME_CYCLES_12FPS = 10_000_000 / 12.0 # 833,333, x68k.cpp:1133
|
||||
|
||||
|
||||
def cycles(mode):
|
||||
"""Exact decode cost of one frame's mode map, in 68000 cycles.
|
||||
|
||||
Single source of truth: tools/analysis/11_cpu_budget.py imports this, and
|
||||
it reproduces the four frames timed on the 68000 to within 1 point
|
||||
(FINDINGS 28.2). Instruction cycles against zero-wait-state memory, so a
|
||||
LOWER BOUND like every 68000 figure since FINDINGS 24."""
|
||||
g = np.asarray(mode).reshape(-1, 4) # one header byte = four blocks
|
||||
allskip = (g == 0).all(1)
|
||||
c = allskip.sum() * 4 * C_SKIP_CLUSTERED
|
||||
mm = g[~allskip]
|
||||
c += (mm == 0).sum() * C_SKIP_MIXED
|
||||
c += (mm == 1).sum() * C_V1
|
||||
c += (mm == 2).sum() * C_V4
|
||||
c += (mm == 3).sum() * C_RAW
|
||||
return float(c)
|
||||
|
||||
|
||||
def blocks_of(idx, pal, bw, bh):
|
||||
return VQ.blockify(idx, pal, bw, bh)
|
||||
@@ -149,17 +192,24 @@ def frame_ctx(m, f, prev, idx_bytes=None):
|
||||
idx_bytes=default_idx_bytes(m) if idx_bytes is None else idx_bytes)
|
||||
|
||||
|
||||
def decide(ctx, lam):
|
||||
"""Lagrangian mode decision at one lam. Returns (mode, payload bytes).
|
||||
def decide(ctx, lam, mu=0.0):
|
||||
"""Lagrangian mode decision at one lam and one mu. Returns (mode, bytes).
|
||||
|
||||
Cheap by design: no painting, no image-sized work. A lam search calls this
|
||||
a dozen times per frame and paints once."""
|
||||
Minimises `distortion + lam*bytes + mu*cycles` per block. `mu=0` is the
|
||||
byte-only decision every session before 8 made; the machine's binding
|
||||
budget is cycles, and bytes and cycles do not rank the modes the same way
|
||||
(V4 is 4x V1 in bytes, 1.49x in cycles; RAW is dearer than V4 in bytes and
|
||||
CHEAPER in cycles, so mu inverts that preference -- FINDINGS 28.8).
|
||||
|
||||
Cheap by design: no painting, no image-sized work. A search calls this a
|
||||
dozen times per lam step and paints once."""
|
||||
ib = ctx["idx_bytes"]
|
||||
s = ctx["sym"]
|
||||
cost = np.stack([ctx["eS"],
|
||||
s["e1"] + lam * (1.0 * ib),
|
||||
s["e4"] + lam * (4.0 * ib),
|
||||
np.full(ctx["nb"], lam * RAW_BYTES)])
|
||||
mc = mu * MODE_CYCLES
|
||||
cost = np.stack([ctx["eS"] + mc[0],
|
||||
s["e1"] + lam * (1.0 * ib) + mc[1],
|
||||
s["e4"] + lam * (4.0 * ib) + mc[2],
|
||||
np.full(ctx["nb"], lam * RAW_BYTES + mc[3])])
|
||||
mode = np.argmin(cost, axis=0).astype(np.uint8)
|
||||
return mode, frame_bytes(mode, ctx["nb"], ib)
|
||||
|
||||
@@ -193,10 +243,10 @@ def paint(m, ctx, mode):
|
||||
return from_blocks(ob, nbx, nby)
|
||||
|
||||
|
||||
def encode_frame(m, f, prev, lam, idx_bytes=None):
|
||||
def encode_frame(m, f, prev, lam, idx_bytes=None, mu=0.0):
|
||||
"""One frame at one lam against one previous reconstruction."""
|
||||
ctx = frame_ctx(m, f, prev, idx_bytes)
|
||||
mode, sz = decide(ctx, lam)
|
||||
mode, sz = decide(ctx, lam, mu)
|
||||
return dict(recon=paint(m, ctx, mode), mode=mode, size=sz,
|
||||
l1=ctx["sym"]["l1"], l4g=ctx["sym"]["l4g"], ctx=ctx)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user