ed172c2da297c7f6931364373e16c45b5d13bb7d
29
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 prosolis
|
ed172c2da2 |
Rewrite the README as a description, and strike a unit error it exposed
The README had become a changelog: which session found what, which constant was retired and by whom, which figure was overturned. That is what STATUS is for, and it made the front page a poor read for anyone who has not been following. Rewritten to say what the project is, what it looks like, where it stands and how to reproduce it, present tense, no session numbers, no em dashes. Restating the standing facts caught one of my own from this session. FINDINGS 52.5 cited 42.4's W sensitivity table -- W <= 6 fits 0/120 frames, W = 8 misses 47/120 -- as though those were clocks per BYTE. They are per WORD. FINDINGS 43 is the section that caught W being charged per word to a byte-wide port and says in terms that 42.3's 0/120 was never physically reachable, so quoting it in byte units re-imported the exact 2x error 43 exists to have corrected, one section after using that same trap as a warning. Struck in 52.5 and in ROADMAP P4, and replaced with the ladder buscost.py already carries, which is in the right unit: 5 clk/B single address with the bus held, 9 dual address held, 12 single address arbitrated per byte, 16..19 dual address arbitrated per byte, the last being what the IPL ROM programs. Nothing else in 52 depended on the struck figures -- 15_bus_occupancy.py's sweep is in byte units and is unaffected. The "cross-check" claim goes with it. 52.5 read the agreement between its W = 8 row and 42.4's 47/120 as two independent models landing in the same place. It is two different units on two different containers at two different rates, and calling it corroboration was manufacturing agreement out of the unit error. What replaces "W <= 12 is a requirement on the player's DMAC programming" is the sharper version: getting the DMAC to HOLD THE BUS is what separates 9 from 19, and that is the thing P4 has to demonstrate. Docs only; no code changed and no gate output moves. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
e565dfbbab |
Take the disc mount out of the source, so someone else's copy of the media works
Three files hardcoded /media/reala-misaki/BDROM -- extract.py, 07_motion_survey.py and check.sh -- which made the pipeline unrunnable for anyone whose disc mounts anywhere else. DLX_BDROM now overrides it everywhere, defaulting to /media/$USER/BDROM, so this box is unchanged and another one works. Verified by running extract.py against a symlinked mount at a different path. check.sh's failure message names the path it looked at and the variable to set, instead of assuming udisks put it where this machine puts it. README gains a "Reproducing this" section: no media ships here and none of it is redistributable, so it says what you have to bring (the disc) and what is already packaged (vasm is vendored as a binary with its source tarball; the k-means is hand-rolled, so numpy and Pillow are the whole Python dependency). It also names the two gates that SKIP rather than fail -- the px68k second-core pass and the IPL ROM DMAC gate -- because both live outside this repo and a silent skip is worth reading as a skip. One trap called out rather than left to bite: scene selection is a hard-coded stream number, not a search. A different pressing that numbers its .m2ts files differently will extract the wrong footage and the green light will PASS on it. check.sh ALL GREEN. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
2f9f5cc995 |
Pace the ring, then read the DMAC config out of the IPL ROM: audio is cheap and the disk is not
Two sessions that were never separated in the working tree, so they land as one commit. check.sh ALL GREEN before and after both. SESSION 19 -- the ring rig gets a frame clock (FINDINGS 51). src/player/stream.s had no frame clock: it asked for record i the instant it finished i-1, outran any finite pipe, and never let the ring back up. The 49.1 sweep passing at 48 KB was therefore a wrap-correctness result and nothing else. PACE/PACEON ($18034/$18038) hold the decoder to 12 fps, so FR_HEAD-FR_TAIL finally means what it reads as: whole frames the decoder could still draw with delivery stopped dead. PACEON=0 free-runs and is what the wrap gate still uses, so every figure in 49 is unmoved. Paced, on the gate container: 64 KB holds 2 frames, 256 KB holds 7-8, 512 KB holds 14-15, all pixel-exact. Tolerance is ceiling-1, measured by cutting the pipe: 256 KB buys 500 ms of dead pipe, not 583. SLACK IS ACCUMULATED, NOT OWNED. It is built out of pipe-wire and a seek spends all of it. At 488 KB/s a 256 KB ring needs 4.83 s of play to reach its ceiling from empty; 512 KB needs 8.42 s to reach 14. A bigger ring raises the ceiling AND lengthens the climb, so a branch point does not ask "is the buffer big enough" but "has there been enough play since the last one" -- and Dragon's Lair's decision points are seconds apart. The rig now also says WHICH resource is binding: at 460 KB/s every ring from 192 KB to 512 KB is rate-bound at ceiling 4 and never fills, so larger rings are dead RAM in that scene. 20_seek_slack.py is the same model rewritten in Python from record sizes, sharing no code with the Lua producer: 35/35 ceilings inside its bracket. SESSION 20 -- the DMAC configuration was in the IPL ROM the whole time (FINDINGS 52). ROADMAP's "do this first" was to put the ADPCM stream on the bus. That needs a clocks-per-byte figure for the audio channel, and 11_cpu_budget.py was charging audio the DISK's rate -- 5 clk/B, its own help text calling it "single-address, bus held". Audio was being charged the favourable end of B3, a 242 KB/s open question. It never had to be a guess. The IPL ROM programs all four HD63450 channels itself and MAME boots the rig with it, so 21_iplrom_dmac.py reads the configuration out of the image and decodes the MC68450 fields. Eight (address, expected bytes, meaning) sites; a mismatch or an unknown revision exits non-zero. In check.sh, no emulator, milliseconds. ch3 DCR=$80, OCR=$32: dual address, 8-bit port, cycle steal WITHOUT hold, REQG=10 external request. The DMAC arbitrates once per byte with no burst to amortise the 5..8 + 2 over, so an audio byte is 16..19 clocks, not 5 -- the old debit was 3.2x..3.8x small. And on the bus it is still nothing: 651 B/frame is 1.25%..1.48% of a frame, about 4% of what the decoder leaves. P6's bus risk does not materialise. The unit worry was worth checking and nearly right: 15.6 kHz is 8 MHz/512 = 15,625 samples/s, two 4-bit samples to a byte = 7,812.5 B/s exactly, and AUDIO_KBPS=7.8 is that in decimal kB while the tool multiplied by 1024. THE DISK CHANNEL IS PROGRAMMED IDENTICALLY. ch1 (SASI) is DCR=$80 too, and so is ch0. That is 16..19 clocks per delivered byte, where 42.4 brackets W at 5..12 and 42.5 has W=8 already missing 47/120 frames. The only worked example of a disk DMA configuration on this machine sits above the entire bracket, and at that price nothing fits at any container size. It is not scsiexrom.bin so B3 stays open -- what changed is that a cheap configuration is now the thing that has to be SHOWN. W <= 12 is a requirement on the player's DMAC programming, not a range the hardware hands us, and it is now the largest open number in the project, ahead of the rate. An unforced cross-check fell out: 15_bus_occupancy.py's new W sweep puts W=8 at 105.7% of the frame, agreeing with 42.5's 47/120, from mode histograms and bus clocks respectively, two models sharing no code. Also: ADPCM outranks the disk at the arbiter (CPR 1 against 2), so an audio byte never waits and a video byte does -- relevant to 51's smooth-rate delivery model. README MEDIA. stream.lua gains DLX_SNAP_EVERY=1 (needs DLX_PACE, off by default, on no path check.sh takes) and tools/media/make_readme_media.py turns the PNGs into docs/img/. The stills and both clips are MAME's own screen pixels. Building it turned up something worth recording. 116 of 119 captured frames are pixel-exact against dlx.py; three are TORN -- frame n on top, frame n-1 below the tear line -- because MAME captured the screen while the block loop was partway down it. decode.s writes straight to the displayed page (one display path, 28.1), so a real player tears the same way, and this is the first time that consequence has been visible rather than argued. The script ASSERTS the tear and refuses to build otherwise, rather than trimming three frames and reporting "every frame I kept is exact". Second correction the capture forced: the snapshot fires before frame n is decoded, so the obvious reading is that it holds frame n-1 -- it does not, because MAME renders the screen at the end of the machine frame, by which time the 68000 has finished frame n. 11_cpu_budget.py's "validated to within 1 pt" line is also corrected: the model reads 2..10 pt HIGH and by more as the frame gets harder, which was already true before either session. src/player/decode.s is unchanged; decode.bin is still 1,296 B at the same MD5. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
b49bbdc939 |
Build v7 into the player, and find the cost model 18% wrong on the block it made commonest
src/player/decode.s now paints v7 literal spans, pixel-exact under MAME and px68k's C68K core over a container where every frame carries 128-216 spans covering up to 38% of the picture. The span pass is blit.s v7 verbatim: the 66.0/9.143/9.978 fit was measured on that instruction sequence. The container is DLX3 -- a span section between the mode header and the block payload, since that is the only place the 68000 can reach without first parsing something of variable length. 16_span_roundtrip.py gates it in check.sh, and asserts it emitted enough spans to have tested anything. Two synthetic all-SPAN anchors price v7 inside decode.s at 151.2 and 225.6 clocks per 4x4 block, against FINDINGS 40's table of 151 and 226 -- 0.2% on both emulators. The measured mode costs what it was said to cost. Two things that were not on the list: TWO BYTE BUDGETS. FINDINGS 40's 18/120 was scored against the 488 KB/s PIPE, not the 280 KB/s profile, and at the profile rate the lam search has already spent the allowance -- spans fired on 5 frames of 120 and looked like a regression. The profile is a chosen quality rate point; the pipe is hardware. --kbps and --span-kbps are now separate and spans run before mu, because a span pays in bytes and mu pays in picture. Delivered: 86/120 over budget without spans, 77/120 at the profile budget, 34/120 on the pipe for +0.36 dB. C_SKIP_MIXED WAS NEVER MEASURED, and it was 18% low -- 45.0, now 55.0. It is the one constant in the table that came from a derivation, because the synthetic frame that would measure it cannot exist: a byte needs a coded block for its SKIP to be mixed. Four bracketing anchors measure it on both emulators with the header byte rotated through all four positions, and the partner mode solves back to its own anchored value to 0.2%. With it corrected the model predicts a real spanned decode to -0.06% mean / 0.09% worst, against -2.99% / 4.30%. It matters because a span marks its run SKIP, so mixed SKIPs dominate exactly the frames spans are judged on. Also: the rig had been writing its synthetic timing frames 26 KB past the top of a 2 MB machine, and got away with it because the modes it overran are data-independent. A span's jump displacements come out of the stream, so it is not. And frames-over-budget is no longer a safe headline -- the controller aims at the deadline, so 55 of 120 frames sit within 5% of it and a 1% cost shift moves 22 frames. FINDINGS 41. check.sh ALL GREEN, now gating on a span-heavy DLX3 container. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
c520a89e14 |
Measure the finer chain tail: 84/120 becomes 18/120, and the derivation was right by cancellation
blit.s gains v7 -- v6's 24-pixel movem chain plus a second chain whose unit is
one `move.l (a0)+,(a2)+`. Measured over 13 span lengths by span.sh, every config
pixel-exact:
cycles = 66.0 per span + 9.143 per COARSE pixel + 9.978 per FINE pixel
fitting all 13 to within 0.2%. v5 and v6 re-measure to FINDINGS 30 exactly, so
the harness has not drifted underneath the new variant.
Rescored against the same scsi window and the same additive model, v7 takes
84/120 frames over budget to 18/120 -- exactly what FINDINGS 39.4 derived, and
that agreement is two cancelling errors: the derivation's 2-register movem tail
is 29% too dear per pixel, and its "nothing per span" for the second chain entry
is 22.3 clocks too cheap. The plain post-incrementing move.l is the right tail
instruction, and it makes the padding quantum 2 pixels, which a run of 4x4
blocks pads to exactly zero.
The DMAC stays dropped on a measurement now rather than an argument: v7 takes
back 37 of the 43 frames the array chain would, with no reserved channel and no
timing neither emulator here can verify. Break-even against all-V1 moves from
L=4 blocks to L=2.
The fine displacement is carried mid-stream rather than in the span record, so
the decoder holds nothing across the copy and keeps all 12 payload registers --
which is the whole reason the coarse unit is 24 pixels.
span.sh is now -seconds_to_run 200 (30 s wall, 36 configs) and takes its
expected snapshot count from the generated metadata instead of a literal 23.
Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
|
||
|
prosolis
|
c5ca56330e |
A second emulator agrees, the bus was never counted, and the DMAC loses by one clock
Three things, and the last one reversed itself when the datasheet arrived.
A SECOND EMULATOR. tools/bench/c68k/ links px68k's C68K core into a headless
harness -- no SDL, no ROMs, no emulated machine, because the decoder touches
nothing but RAM, the control block and GVRAM. decode.s is now pixel-exact under
two independent CPU cores, and cycle-table error against MAME is bounded at
3.3%, running against us. MAME 0.277's M68000 turns out to be the MICROCODE
core, not Musashi (m68000.lst + m68000gen.py), so this is two structurally
different timing models agreeing rather than two tables. FINDINGS 28.8's "V4
costs more than RAW" reproduces independently. FINDINGS 37.
THE BUS. Nothing since FINDINGS 24 had counted the 68000's local memory bus --
one 4-clock cycle at a time, carrying instruction prefetch as well as data. The
decoder occupies 86.7% of it and PREFETCH IS 62% OF THAT TRAFFIC, so a data-only
count understates occupancy by 2x. Two sources check each other: c68k_bench
counts every bus callback exactly, and a static walk of decode.lst supplies the
prefetch no emulator here can report. The walk reproduces the measured data half
to 0.04%, which is what licenses its prefetch half, and 15_bus_occupancy.py is a
gate rather than a report because every bus figure depends on that check.
FINDINGS 38.
THE DMAC CHAIN LOSES. FINDINGS 29.6 named it the one uncosted lever. Costed from
bus arithmetic -- a read cycle plus a write cycle, 8 clocks a pixel -- it scored
1/120 frames over budget against the v6 span's 10/120 and looked decisive. Then
the MC68450 manual (Motorola Jul 1989, now at ~/src/mc68450.pdf): Fig 4-25 sheet
4 puts a dual-address word between two 16-bit ports at 9 CLOCKS, because note 2
gives the DMAC 4-clock reads and 5-clock WRITES. The 68000 writes in 4.
DMAC 9.000 clocks/pixel datasheet
v6 9.152 clocks/pixel measured, FINDINGS 30
1.7%. Scored additively, 86% of what remains of the DMAC's advantage is v6's
24-pixel padding quantum -- a property of its unrolled movem chain, fixable in
software with a finer tail chain, worth 55/120 -> 18/120 against the DMAC's
12/120. Recommendation: fix the quantum, drop the DMAC. Six frames does not buy
a reserved channel, a two-region container layout and a timing dependency
neither emulator here can verify. The container is identical either way -- v6's
record and an HD63450 chaining entry are both 6 bytes, so the chain array IS the
span table -- so nothing is foreclosed. FINDINGS 39.
TWO CORRECTIONS TO MY OWN WORK IN THE SAME SESSION:
- I argued FINDINGS 35's flat CPU debit for the disk was too pessimistic and
rescored the window at 53/120 with max(CPU, bus). Wrong. A 68000 has no cache
and a two-word prefetch queue, so it stalls the moment another master takes
the bus, and the MC68450 hands the bus over in SLABS under limited-rate
auto-request rather than interleaving per operand. DMA is additive. 84/120
stands and 14_dmac_chain.py reproduces it exactly. What 86.7% occupancy really
says is that there is almost no room to overlap anything. FINDINGS 38.3.
- The first DMAC costing was derived where a primary source existed. Both wrong
answers were confident and both were caught by reading the manual.
Also landed:
- FINDINGS 5's 8 clocks/word for the SCSI DMA, STATUS's own "most load-bearing
unmeasured number", is now bracketed by the datasheet: 5 clk/word with the bus
held, ~12 if the DMAC arbitrates per word. 8 is a supported midpoint, and
which end applies is a player design decision worth 7 clocks a word on a
480 KB/s stream. FINDINGS 39.7.
- check.sh gains two gates: the C68K pixel-exact decode (seconds, no MAME) and
the bus-model self-check. Both skip cleanly without a px68k checkout.
- spanned blocks are now charged their mode-map dispatch, which FINDINGS 30.7
flagged as uncounted in 12_span_tradeoff.py.
- MAME timed runs must be budgeted by WALL CLOCK, not -seconds_to_run: this box
runs x68000 at ~0.033x realtime and two runs were killed by their own timeout.
That is why the all-RAW cell in 37.3 is empty. The C68K harness does the same
work in seconds because it emulates a CPU and not a machine.
Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
|
||
|
prosolis
|
7d365b3ff5 |
Drop SASI on capacity, then find the budget never had the disk in it
USER DECISION: drop the `sasi` profile. Not on bandwidth -- on capacity. A SASI volume is 40 MB, and the 22.8 min of unique scene footage on the source Blu-ray (streams 00000-00201, measured, not recalled) is 146 MiB at the LOWEST rate this codec makes -- more than the machine's whole 4-unit SASI space. `scsi` is the only profile now. FINDINGS 32. Then the user asked whether we were drawing the wrong conclusions about PIO vs DMA, and we were, more broadly than the question implied. Every CPU figure in FINDINGS 24-34 is scored against the full 833,333 cycles/frame with nothing subtracted for moving the bitstream off disk. Debiting the HD63450 cycle-steal at the long-standing 8 clk/word ESTIMATE, "1 frame of 120 misses" becomes 84 of 120, median 112.4%. PIO at the span rate is 99.8% of the machine. Spans buy cycles by spending bandwidth and the bandwidth returns as steal, so 31.6's "fits completely" becomes a worst frame of 114.3%. 10 fps absorbs it: median 93.7%, 1/120. FINDINGS 35. `11_cpu_budget.py` takes --io dma|pio|none, defaults to dma, and warns if asked for none. Also landed: - item 1 done: the cost model checked against the 68000 on a cost-aware container, -3.07% to +0.01%, whole-window mean -1.22%. FINDINGS 34. - item 4 done: the container carries its own 4-byte record alignment (DLX2). 94/120 record starts were on odd addresses -- an address error, not a slow read -- now 0/120 for 16 B/s. Re-encoding reproduces 31.1 exactly. FINDINGS 33. - a `scsi` window does not fit the 2 MB machine the rig emulates (2.84 MB of stream past a 0x200000 ceiling). The gate now verifies 80 of 120 frames and SAYS so, and fails loudly when the pass does not complete, instead of reporting a phantom 49,005-pixel diff. FINDINGS 36. Three near-misses this session had one shape: an unobservable run nearly produced a false finding. stdbuf -oL on any MAME job that prints progress -- a file is block-buffered too, and a run that is merely finishing looks exactly like one that is wedged. check.sh ALL GREEN. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
06b98d4b47 |
Price cycles in the mode decision: 37 misses become 1, for 0.26 dB
The decoder has been CPU-bound since FINDINGS 28 while the mode decision
minimised D + lam*R -- distortion against BYTES. decide() now minimises
D + lam*bytes + mu*cycles, and ratectl bisects mu per frame against the
833,333-cycle budget with the lam bisection nested inside it. On the worst
sustained window:
sasi 27.22 -> 26.95 dB, 109.5 -> 109.4 KB/s, 37/120 misses -> 1
scsi 29.90 -> 29.27 dB, 280.0 -> 278.6 KB/s, 51/120 misses -> 1
Bitrate does not move: the byte controller still binds, and mu changes WHICH
modes are bought. V4 is what it stops buying -- 25.2 -> 20.3% of blocks at sasi
and 15.0 -> 5.3% at scsi, where RAW takes it. That is 28.8's inversion in
practice: RAW is dearer in bytes and cheaper in cycles, so only the byte-rich
profile can buy its way out of V4.
Three things worth knowing beyond the headline:
- The one frame that still misses, at both profiles, is FRAME 0 -- no previous
reconstruction, so 100% changed by definition, which is also what a scene
cut is. It comes out at the all-V1 floor of 110.6% and is emitted late on
purpose. Freezing a cut to make a deadline is the worse failure.
- 28.7's "11 frames are impossible" was too pessimistic. That floor held the
SKIP set fixed and asked how cheaply the drawn blocks could be drawn; the
real decision can also MOVE a block to SKIP, which above ~90% non-SKIP is
the only lever left.
- SKIP's price depends on its neighbours (13.25 cycles clustered, 45 mixed),
which a per-block lagrangian cannot see. The way out is that the two uses
need not share a cost function: a ranking constant inside decide(), the
exact clustered rule for the frame-level bisection. vq_hybrid.cycles() is
now the one definition of that rule and 11_cpu_budget.py imports it.
Gated: 09_ratectl_drift.py runs both controllers, both 0/120 drifting frames.
The cost-aware container decodes pixel-exact on the 68000 (120 frames). ON by
default in encode.py; --no-cpu-fit restores session 7. check.sh ALL GREEN.
Still a model, not a measurement, for THIS container: FINDINGS 31's cycle
figures come from vq_hybrid.cycles (within 1 point of the 68000 on four frames
of the session-7 container). Timing this one on the machine is step 1 of the
next session -- it was started and killed for time, and it is slow.
FINDINGS 31. tools/analysis/13_cpu_ratectl.py.
Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
|
||
|
prosolis
|
29eb78a599 |
Measure the span: the mode survives, and it is an encoder format
FINDINGS 29 priced a literal-span mode at 4*(50 + 4L*9.08) cycles and labelled
the whole section DERIVED. Session 8 step 0 was to measure it before optimising
over the mode set it implies. Two variants in blit.s, one stream per span length
from prep_spans.py, timed by span.lua, driven by span.sh in ~25 s:
v5, handed (x, npix) and left to work the copy out: 97.9/span + 10.459/px
v6, handed an address and a jump displacement: 43.7/span + 9.152/px
29 assumed 50.0/span + 9.080/px
So 29's arithmetic was right about a format nobody had written. The difference
is not tuning: v5 spends ~122 cycles a span computing a destination, dividing
npix into bursts and handling a 0..15 remainder, all of which the encoder knows
at build time. v6's record is {u32 absolute GVRAM address, u16 jump
displacement} into an unrolled chain of 24-pixel copy units -- no loop, no
remainder, no arithmetic -- and it fits 11 span lengths to 0.3%.
Three things that measurement showed and derivation could not:
- The per-pixel cost is a function of REGISTER PRESSURE. FINDINGS 24's 9.08
was a fixed blit with 12 registers free; v5 can spare 8 and pays 10.46; v6
gets 12 back only because the encoder holds the state.
- Short spans die in the remainder path -- a 12-pixel span costs MORE than a
16-pixel one -- and the fix is padding, not avoidance.
- Odd-x alignment is free (259.0 vs 261.8 cycles/span), as a 16-bit bus
implies but nobody had checked.
Re-priced against the unchanged mode maps, sasi: median 74.4% -> 52.0% (29 said
43.0), misses 37 -> 10/120 (29 said 8), 448.0 KB/s. Break-even moved from runs
of 2 blocks to runs of 4. 29.4 survives: a scene cut needs x >= 0.196 of the
frame as spans and the bus allows x <= 0.373, so it fits at 12fps.
All 23 timing configs are also checked pixel-exact, so none of this was timed
against a decoder that quietly skipped work.
FINDINGS 30. Next: lever B, the cost-aware mode decision.
Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
|
||
|
prosolis
|
3641f37e28 |
The bus is 4x idle while the CPU is pinned: price the trade
The codec was designed when bytes were scarce, so every decision in it trades
cycles to save bytes. That is now backwards: sasi spends 110 KB/s of a 488 KB/s
pipe while missing 31% of frames on CPU.
The cheapest thing a 68000 can be handed is the most expensive thing to store.
Measured, per pixel: row-linear copy from word-expanded memory 9.08 cycles,
block-order 12.98, V1 codebook 18.74, RAW byte literals 25.03. So the 1024-byte
stride costs 43% and unpacking bytes to words costs more than the write itself.
Pricing one new mode -- a per-row span of word-expanded literals movem.l'd
straight from the stream buffer -- against the UNCHANGED mode maps:
sasi median 74.4% -> 43.0%, worst 136.2% -> 106.2%, misses 37 -> 8/120,
101.7 -> 453.2 KB/s
scsi median 94.9% -> 69.4%, misses 51 -> 18/120, 272 -> 479.7 KB/s
scsi gains less precisely because it has less idle bandwidth left to trade.
Two consequences worth flagging. A word-expanded literal block derives to ~240
cycles, cheaper than V1's measured 299.9 and pixel-exact -- so every codebook
mode is CPU-dominated by a literal, and the codebook is a byte optimisation
that now costs cycles. And 28.5's "a scene cut cannot fit at 12fps" reopens:
CPU needs >=19% of the frame as spans, the bus allows <=39%, and that interval
is not empty.
DERIVED, NOT MEASURED, and labelled as such everywhere. The 9.08 cycles/pixel
is real but was measured at full row width with 12-register bursts, so short
spans are flattered. Measuring one span on the 68000 is now step 0 of the next
session, ahead of the cost-aware mode decision, because it changes the mode set
that decision optimises over.
FINDINGS 29. tools/analysis/12_span_tradeoff.py.
Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
|
||
|
prosolis
|
cb05e77a42 |
Handoff: reconcile the docs with a decoder that works and does not fit
Session 7 handoff. check.sh green end to end, including both new stages. STATUS now leads with the cost-aware mode decision, and the stale statements that session 7 overturned are struck rather than left to be re-read as current: - next-steps item 1 still announced "implement both display paths and pick per frame" as a settled answer. Withdrawn, with the reason, and the part that did hold up (report the distribution, not the mean) kept. - the profile table's "machine" column reads as a CPU claim and is not one. It is the bus. Both profiles target the same stock 10 MHz 68000 and neither fits; the Super has SCSI at 10 MHz. - the display-path section stopped at "68000 code copied a frame". It parses one now, and the blit figures are no longer the display-path budget. - the green-light section still described a 2-minute, three-stage check, and did not warn that 10_pathmix_drift.py is SUPPOSED to exit non-zero. Sharpened the one trap in the next session's first step: SKIP is not a per-block constant. It costs 13.25 cycles inside an all-SKIP header byte and ~45 in a mixed one, so its price depends on its neighbours and a per-block lagrangian cannot see that. Picking either number is wrong in a different direction; the budget check can use the exact clustered cost function while the per-block term only has to rank modes. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
31c4c1aba1 |
Separate the two axes: profiles are I/O, the CPU ceiling is one target for both
The profiles were chosen against disk bandwidth and say nothing about CPU. The locked CPU target is a stock 10MHz 68000 for both of them, so both must fit 833,333 cycles -- picking sasi does not rescue it, it still misses 31% of frames against scsi's 42%. Splits the miss into what the encoder can fix and what it cannot: re-coding every non-SKIP block as V1 is the floor, and it still misses 11 frames at sasi and 12 at scsi, all of them above ~90% non-SKIP. So a cost-aware mode decision can reach about three quarters of the misses; the rest need a structural answer, not a better encoder. Also: V4 is 448 cycles against RAW's 400, and RAW is pixel-exact. On the CPU axis V4 is strictly dominated and the byte lagrangian's mode preference inverts. Only the byte-rich profile can take that escape, so the cycle ceiling should cost sasi MORE quality than scsi despite costing it fewer cycles. FINDINGS 28.7/28.8. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
ed353d24a9 |
Budget both profiles against both clocks: the CPU limit is the clock, not the profile
11_cpu_budget.py takes --machine. Clocks confirmed from MAME 0.277
x68k.cpp:1133/1194/1200, not recalled: x68000 AND x68ksupr are both
40_MHz_XTAL/4 = 10 MHz; only the XVI is faster at 33.33_MHz_XTAL/2.
sasi scsi
stock 10MHz 31% miss 42% miss
XVI 16.7MHz 0% miss 0% miss
sasi is the cheaper profile but it does not fit either at 10 MHz. The XVI
column is headroom, not a target: the profiles are an I/O-bandwidth axis and
say nothing about CPU, and the locked target CPU is a stock 10 MHz 68000 for
both of them. So both profiles have to fit the same 833,333-cycle budget, and
the cycle ceiling has to be enforced in the encoder regardless of which one
ships.
Model comparisons are now gated to the clock and framerate they were stated
at: quoting 24.5's 76.6% or the stock-machine 68000 timings against an XVI
budget compares a model to a measurement of a different machine.
Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
|
||
|
prosolis
|
e1aa26bb57 |
The 68000 decoder draws pixel-exact frames, and does not fit
src/player/decode.s parses DLX1 and decodes straight into GVRAM. Verified pixel-exact over a 120-frame sequential run of the worst sustained window on the disc -- all four block modes, full temporal recursion, so the last frame is only right if all 120 were. In check.sh. It costs a mean of 81.7% of a 12fps frame budget, and 31% of frames exceed 100% (42% at scsi). CPU is now the binding constraint. FINDINGS 28. Three things that were believed and are not true: - The dual-display-path plan of FINDINGS 24.5/25.6 is incoherent. The compose path needs a RAM copy of the previous reconstruction; the direct path's selling point is that it keeps none. Mixing them shows stale pixels on 70 of 120 frames, worst frame 18.8% of the screen. Every coherent repair is dearer than not mixing, and 24.5's two figures were both copies with no decode in either, so there was never a crossover to find. One path ships, and the 96KB reference frame is gone. tools/analysis/10_pathmix_drift.py keeps the counterexample runnable; check.sh asserts it still reproduces. - The four block modes do not cost the same. V1 300, V4 448, RAW 400 cycles against the old model's flat 207.8. V4 is 25% of blocks and 50% of the cycles, and the mode decision charges it bytes it does not charge cycles for. tools/analysis/11_cpu_budget.py reproduces all four frames timed on the 68000 to within 1 point. Hand-derived timings agree to 0.5% on V1. - The container is big-endian but not aligned. Variable-length records laid end to end put frame 1's length field at an odd address, and move.l (a0)+ there is an address error: frame 0 decoded perfectly and then vectored into the IPL for 59 emulated seconds looking like a hang. Found by dumping PC, not by reading the source. Also: an all-V1 frame, the cheapest possible full redraw, is 110.5% of budget. No mode assignment fits a scene cut at 12fps. That one needs a decision, not a measurement. Next: charge cycles in the mode decision and bisect against 833,333 per frame, the way session 6 bisects lam against bytes -- but with no bucket, because a late frame cannot be banked. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
497f88b945 |
Rate control: rebuilt per-frame, wired in, and gated at zero drift
FINDINGS 26 stopped the session-5 rate controller before it shipped: it built a lam-ladder of independent whole-sequence encodes and picked frames off it, so SKIP blocks referenced reconstructions the decoder never saw -- 111 of 120 frames drifted. The fix is the structural one 26.1 said it had to be. vq_hybrid is now frame-drivable -- frame_ctx / decide / paint -- and encode() is a thin loop over it. Rate control drives the same three calls, bisects lam per frame under the leaky bucket, and feeds back the frame it actually emitted. The desync has no way to occur, and 09_ratectl_drift.py goes 111/120 -> 0/120. That test is now part of check.sh, which is ~2 min rather than ~40 s. Both overshoots on the worst sustained window are closed for under 1 dB, totals including audio: sasi 137.4 -> 109.5 KB/s (-0.60 dB), scsi 381.6 -> 280.0 KB/s (-0.91 dB). Zero frames hit the lam=800 cliff, so nothing was destroyed to get there. Rate control also makes the display path cheaper -- scsi's median drops 53.6% -> 47.1% -- because raising lam moves blocks to SKIP and V1. Two knobs measured rather than guessed. --rc-floor is worth 0.00 dB on that window and defaults to the profile lam, so rate control cannot regress content that already fits. --prefill defaults to 0 and is documented as a trap: it buys a permission to overshoot of exactly bucket/nframes, and on a 14-frame clip it disables the controller outright. FINDINGS 26.5 was wrong in both halves and 27.6 records it. _paint was not the bottleneck (14% of a frame, though vectorising it was still right at 17.1x) and the ladder was never "minutes" -- those were k-means in build(). What makes per-frame rate control affordable is that VQ.assign depends on neither lam nor prev, so it is cached one frame deep: a 12-step search over 120 frames costs 0.31 s against 49.1 s. Also caught: fixed-lam sasi was already 5% over target on 00020, the clip everyone called easy. Nothing noticed because the profile table quotes PSNR and not bitrate. check.sh: ALL GREEN. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
145753c0bf |
Handoff: rate control is next, and it is unsound as written
Session 5 handoff. The user has chosen rate control as the next session's work, so this reads ratectl.py properly before that session starts rather than discovering the problem mid-implementation. FINDINGS 26: encode_rate_controlled() is not sound. H.encode() is temporally recursive -- SKIP blocks copy the previous RECONSTRUCTION -- but rate control builds a ladder of independent whole-sequence encodes and picks each frame from whichever rung fits the budget. Frames then reference reconstructions the decoder never saw. Measured on the Singe window: 67 rung switches, 111 of 120 frames drift, worst frame 43.4% of pixels, reported PSNR overstated by 0.36 dB. It would have wired up cleanly and reported a plausible wrong answer. Two further defects in the same function: the lam ladder runs to 2e5, 250x past the FINDINGS 15 cliff, so a frame that only fits up there is destroyed rather than rate-controlled; and with 5 rungs only two are ever chosen, straddling the operating point by 7.5x. The docstring describes a per-frame binary search, which is the right design -- the implementation is a fixed ladder. The leaky bucket does work and should be kept: 109.1 KB/s against a 110 target. tools/analysis/09_ratectl_drift.py is the regression test and the acceptance criterion: it exits non-zero until zero frames drift. Also corrected the stale 38% blit figure in ratectl.py's profile commentary, which session 5 measured at 53.6% (FINDINGS 24), and recorded the pgrep -f self-kill trap again -- four times across three sessions now. check.sh ALL GREEN. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
e00264a058 |
Find the sustained action sequence: it breaks both profiles
The open risk since session 2 was "a sustained action sequence could still break the bitrate", with every clip measured so far being 1.2-1.7 s. Closed by measurement rather than by sampling clips by hand. 07_motion_survey.py scans a whole stream at 96x72 for the hottest sliding window of inter-frame difference. On 00223 the spread between the quietest and hottest sustained 10 s windows is 10.6x, which is the argument for not eyeballing it. Hottest is t=539.4s, the Singe endgame. There, with the fixed lam the CLI uses, sasi overshoots 110 -> 129.6 KB/s (+18%) and scsi 280 -> 373.8 KB/s (+34%). Rate control moves from "insurance, not a fix" to required, and is promoted above the full-disc survey. The bus is not broken -- 381.6 KB/s still fits the 488 KB/s figure -- so FINDINGS 21 survives, at 78% of the pipe instead of a comfortable margin. Three further corrections fall out: - The two largest streams on the disc are bonus material. 00216 is the feature with a burned-in commentary PiP; 00215 is the commentary. 00223 is the clean 9.4 min. A size-ranked survey would have encoded live action. - On hard content the 256-colour scene palette (31.33 dB) binds well before the X68000 display (40.81 dB); scsi is already within 0.51 dB of it. - FINDINGS 24.5's architecture question resolves to "both paths, chosen per frame": 30-53% of frames sit above the 70% crossover. Picking per frame costs a median 37.0% of the frame budget and caps at 53.6%. Reporting for this is wired into encode.py, which previously only printed a mean over all frames -- the one statistic that cannot answer a per-frame question. extract.py takes optional start/dur; 08_mode_map.py renders source | decoded | block-mode map to .webm. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
09a5a50065 |
Measure the blit on the 68000: the 38% estimate was 53.6%
First 68000 instructions in this project to draw a pixel. Everything before this was GVRAM filled from Lua, which costs zero 68000 cycles, so the blit figure the whole CPU budget rests on had never been validated. Four variants of a full-frame 256x192 paint, timed in MAME and each also hand-derived from the MC68000 timing tables beforehand; the two agree to 0.006-0.43%, which is what makes the result trustworthy after this project's history of false-good measurements. V1 movem.l blit from a word-expanded RAM frame 446,286 cyc 53.6% V2 naive move.b/move.w per pixel 1,284,174 cyc 154.1% V3 write-only floor, no source read 225,789 cyc 27.1% V4 same writes in 4x4 block order 637,971 cyc 76.6% Scope: MAME's gvram_w/gvram_r carry no timing at all, so these are instruction cycles against zero-wait-state memory -- a floor, not a hardware prediction. V1's output snapshots pixel-exact through verify_frame256.py, closing FINDINGS 23.5. The V1/V3 gap shows reading the source frame is exactly half the cost, which makes the architecture question live: decode-direct-to-GVRAM needs no RAM reference frame and scales with the non-SKIP block fraction, crossing compose-then-blit at 70% of blocks changed. That fraction is now the top priority and is already a by-product of vq_hybrid.py's mode decision. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
7ba979a236 |
Handoff: green-light script, reconciled figures, a smaller first step for the decoder
Prepares session 4 for handoff. No new measurements; this reconciles the docs with what session 4 changed and makes the next session's entry point cheaper. - tools/bench/check.sh re-runs both display regression tests from the Blu-ray in ~40 s and prints ALL GREEN. Verified green cold, after wiping tmp/ and re-extracting. STATUS and README both open with it, because everything downstream assumes the display path is pixel-exact and nothing previously checked that in one command. - Reconciled the figures session 4 invalidated. Session 3's 38.88 dB ceiling is struck through in STATUS with a pointer to 40.81; the "three facts the player must honour" table no longer quotes R20 = 0x0116, which was the 768-wide IPL timing and would have been copied into the player as if it were the shipping value. crtc_mode.lua is now named as the single source of truth for CRTC registers, in both STATUS and README. The 38.88 dB in the session-3 reproduce section is left alone and annotated instead: it is correct for that test, which still packs I = 1. The two numbers disagree for a reason and a reader should be able to see which is which. - Next-step 2 now leads with something smaller than "write the decoder": a dumb full-frame RAM->GVRAM blit in 68000 code, timed. That number alone confirms or kills the 38% estimate, and needs no bitstream, codebooks, or DLX1 parsing. The reference image and its checker already exist. - Parked the user's Cliff Hanger / Lupin III follow-on in STATUS so it is not lost and not mistaken for scheduled work. Cheaper than this project on every axis except media prep, which is where it would actually stall. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
34f9ee341d |
A real 256x256 CRTC mode, derived not recalled; palette ceiling was 2 dB low
Session 3 left the harness on the IPL's 768x512 text timing because no CRTC values had been derived and guessing them was the failure mode to avoid. This derives them from MAME 0.277's divisor ladder instead, and the derivation is self-checking: the 256-wide mode runs at div 6 against the 768 mode's div 2, so htotal is exactly 1104/3 = 368 dots and every horizontal register divides by three with no remainder. Only the blanking split rounds. Verified by snapshot: native 256x512, active area pixel-exact, x=512 wrap gone. Two things fell out that change numbers elsewhere: - The palette's shared LSB I must be chosen per entry, not hardcoded to 1. Doing so lifts the display ceiling from 38.85 to 40.81 dB and is the only way to reach true black at all, since pal6bit(1) = 4. 102 of 256 entries want I = 0, so this is not a corner case. Supersedes FINDINGS 22.4; scsi has ~2 dB more headroom than that section claimed. The encoder does not do this yet. - Letterboxing costs a palette entry: GVRAM cleared to zero shows entry 0, and a free mediancut palette puts a real image colour there. 255 colours plus a reserved black, via prep_frame.py --reserve-black. MAME's graphics double-scan is phase-shifted one raster line (it halves the absolute scanline and vbegin is odd), which produced a false failure before it was understood; the regression test now asserts the shifted pairing explicitly. Still Lua-side. No 68000 instruction has drawn a pixel; the 38% blit estimate remains unvalidated. What this buys is a defined geometry for the decoder to write into: 256 words per row, 1024-byte stride, rows 32..223. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
966417893b |
Scope the display-path result: no 68000 code drew that frame
The session-3 milestone was written in a way that reads as "the port renders", which it does not. The video hardware is genuinely emulated and the output is bit-exact, but GVRAM was filled by a MAME Lua script poking emulated memory, not by 68000 instructions. The distinction is load-bearing: Lua writes cost zero 68000 cycles, so nothing here tests whether the CPU can decode and blit inside 833,333 cycles. The 38% full-frame blit estimate that the entire budget rests on is still unvalidated. Only the "Not yet started" list carried this caveat, which was too buried for a claim this easy to over-read. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
3265bf2740 |
Handoff: reconcile docs with the verified display path
Session 3 summary in STATUS.md, plus the things a cold start needs. - Reproduce section for the display result, verified cold from the Blu-ray at end of session: extract -> prep -> MAME -> verify, exact match, 38.88 dB. The frames are not in the repo and the old ones lived in /tmp, so the chain starts from extract.py rather than assuming a scratch directory survives. - tools/bench/verify_frame.py turns FINDINGS 22 into a regression check. It is deliberately an exact test rather than a PSNR threshold, since the whole point of that section is that the render is bit-for-bit predictable. It prints the three registers to check when it fails. - Recorded where the MAME source now lives, and why to read it first: six register-poking attempts failed against a gate that one grep found. - Split the CRTC mode table out as its own next step. It is the prerequisite for the decoder skeleton and the smallest well-defined task available, with an explicit warning not to write the timing values from memory. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
b322e84cd4 |
Get a real Dragon's Lair frame onto the emulated X68000
First pixels on an actual X68000 screen. Everything up to now was Python-side or a headless -video none run, which cannot snapshot at all. The blocker was not the video controller. The IPL leaves CRTC R20 = 0x0B16, and bit 11 is "G-VRAM set to buffer", which makes MAME's draw_gfx() return early. GVRAM writes still land and read back correctly while the layer is invisible, so six attempts at $E82400/$E82500/$E82600 all rendered black with every register holding the value I intended. Two more facts, both confirmed against MAME 0.277 source rather than assumed: - $E8E001 monitor contrast is left at 14 by the IPL, scaling all output to 93.3%. The player must set it to 15. Contrast 0 blanks the screen, which is a free fade-to-black for scene transitions. - The palette word is GGGGGRRRRRBBBBBI with a shared LSB, expanded as pal6bit((field<<1)|I). With contrast at 15 the render is pixel-exact, not merely close, which also confirms the 1024-byte GVRAM line stride. That exactness gives a new quality ceiling: the 15-bit+I palette alone costs 38.88 dB against the 24-bit palettised source, the same order as the scsi profile's own codec error. scsi is close to display-transparent on hardware, which bounds how much further it is worth raising. Unblocks next step 2, the 68000 decoder skeleton, which now has a known-good reference image to diff against. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
22c67f1cb8 |
Record the two shell traps that wedged session 2's background jobs
Eleven watcher shells and one MAME instance were left running for over an hour. Both had the same shape: a wait that can never be satisfied. - `until ! pgrep -f foo.py` matches the watching shell's own command line, so the loop never terminates. Wait on a PID or a sentinel file instead. - `timeout N mame` sends a SIGTERM that MAME ignores when its autoboot script is blocked; without `-k` the process spins at 100% CPU forever. Also gitignore vasm's default `a.out` output. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
64cd1ffd72 |
Handoff: reconcile docs and tooling with the corrections made this session
Session 2 reversed several of its own conclusions. The docs are append-only, so a reader could land on a superseded section and act on it. This pass makes the repo internally consistent. Defects found and fixed in STATUS.md: - claimed "Hybrid VQ with k=1024: no" as the answer to the linework question, directly contradicting FINDINGS 14, which rejected k=1024. Both profiles are k=256. - malformed profile table (six column separators, five columns). - next-steps list had two items numbered 3 and listed the full-disc survey twice. - the disk-benchmark section still read CRITICAL-PATH with "if SCSI sustains >=800 KB/s, ship pixel-exact". That was written while the bandwidth figure was misread as 4 MB/s. At 4 Mbps pixel-exact needs 92-97% of the pipe and is not available, and the ring-buffer result means the design no longer hangs on the benchmark at all. Rewritten with what it IS still worth doing: confirming the 4 Mbps provenance, and confirming DMA is used rather than PIO. FINDINGS now carries supersession blockquotes on 5, 8, 11, 17 and 18 pointing at the sections that correct them. 18 is the dangerous one -- its peak-vs- sustained test is reversed by 21 -- so it is marked DO NOT ACT ON THIS SECTION while noting the per-frame data itself remains valid. profile_gen.py had the same problem in code: it defaulted to the superseded peak sizing and returned lam=25 where the docs say lam=10. The buffered test is now the default and peak sizing is behind --size-for-peak as a bound only. A tool that contradicts the findings is worse than no tool. Also preserves the five measurement scripts that produced this session's numbers as tools/analysis/05-09, following the session 1 precedent, and adds an "explicitly abandoned -- do not re-propose" list to STATUS covering entropy coding, k=1024 codebooks and flat 4x4 VQ. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
fb8a1462b0 |
Correct the peak analysis: both profiles fit 4 Mbps with zero prefill
The previous commit warned that scsi "does not fit 4 Mbps" because a frame peaked at 96.4% of the sustained rate. That was the wrong test, and the user was right to push back on it. It compared instantaneous frame demand against a sustained rate as if they had to match frame-by-frame; the disk keeps filling during the frame, and a ring buffer absorbs any shortfall. Correct test is cumulative demand vs cumulative supply. Simulated on the real per-frame sizes at a constant 488 KB/s fill: required prefill = 0.0 KB for EVERY scene at both profiles Fill delivers 40.69 KB per frame time; only one measured frame (00146, 42.10 KB) exceeds it and the next frame recovers it. A 256 KB buffer -- 12.5% of RAM -- carries ~1 second of stall tolerance, orders of magnitude more than an SD-backed seek needs. scsi at lam=10 stands; the hardest sampled scene runs 313 KB/s mean, 64% of the pipe. Also carries through a consequence of SD-backed deployment that session 1 noted as a caveat but never applied: with BlueSCSI/SCSI2SD the sustained rate is a bus-limited CONSTANT, not an average over variable seek latency, so there is no long tail to leave margin for and we can size much closer to the ceiling. Conservative margins here were cargo-culted from a constraint this deployment does not have. The SASI/SCSI split remains correct because it is about bus protocol, not media. Rate control drops from load-bearing to insurance: intra-scene peaks are a non-problem, but only 4 clips of 1.2-1.7s out of 224 streams have been measured and 00146 already runs 23% hotter than 00020. The full-disc survey is now the highest-value measurement, ahead of wiring up ratectl. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
48e912de8b |
Size against 4 Mbps: peaks break the scsi profile; DMA steal is not free
User clarified the bandwidth figure is 4 Mbps (488 KB/s), not 4 MB/s -- ~8x
tighter than the previous commit reasoned against. Two consequences, plus a
correction to session 1.
1. The scsi profile committed in
|
||
|
prosolis
|
f0f2f807a4 |
Raise both quality profiles; rule out entropy coding on CPU grounds
The profiles shipped in
|
||
|
prosolis
|
e4062ed294 |
Session 2: hybrid VQ codec, two quality profiles, three corrections
Answers session 1's critical-path question. Flat 4x4 VQ at k=256 was prototyped and REJECTED by eye: Dirk's face disintegrates and ink outlines break into 4-pixel stair-steps. The 256-colour palettised frame is excellent, so the palette was never the problem -- block VQ was. Replaced it with a Cinepak-style hybrid: each 4x4 block is SKIP, one 4x4 codeword, four 2x2 codewords, or RAW literal pixels, chosen per block by rate-distortion. The RAW escape makes lam=0 pixel-exact (measured 0.00 dB loss), so the quality knob spans lossless to heavily-compressed in one bitstream. Per the user's decision, ships TWO quality profiles from that one codec, one decoder and one bitstream -- only the rate knob differs: sasi 45 KB/s lam=300 34.8 dB stock 10MHz ACE/EXPERT scsi 75 KB/s lam=100 35.9 dB Super/XVI or CZ-6BS1 Three corrections to earlier numbers: 1. Session 1's "183 KB/s at 12fps" was a bad extrapolation. Halving the framerate does not halve the bitrate -- decimation roughly doubles the per-frame delta. Re-measured directly: 340 KB/s for session 1's own RLE, 247 KB/s for changed-spans+deflate. The lossless floor is 319 MB. 2. A FOURTH false-good result, same family as the three in FINDINGS 4: k=1024 codebooks appeared to buy +2.4 dB free, because the rate model charged 1 byte for a 10-bit index. Charging the true cost reverses the verdict -- k=256 wins at every matched bitrate, and by 5 dB at the low end where the SASI profile lives. k=256 ships. 3. Stream inventory: the ~3-5MB clips are 1.2-1.7s, not ~60s, and some 60s streams are menus, not content. Any survey must classify before averaging. Also cleared both candidate sources for the game-logic layer: the SNES project is MIT and DirkSimple is zlib, so the arcade scene graph can be imported and the two transcriptions diffed against each other. Encoder is working end-to-end: extract.py -> vq/vq_hybrid/ratectl -> encode.py, emitting a big-endian DLX1 container the 68000 can parse with plain moves. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |