6dd3fb3597ab080365fbb975af9ad2a5b1447822
29
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 prosolis
|
6dd3fb3597 |
Ask the chip which decoder it is, and find four wrong axes where one was expected
ROADMAP P6a, on the machine. 68000 code programs HD63450 channel 3 with the IPL ROM's own ADPCM bytes -- dual address, 8-bit port, cycle steal, external request -- and feeds the MSM6258 a designed 1,678-nibble stream at the chip's own pace: 839 B in 0.1074 s = 7,811.4 B/s against the format's 7,812.5, CER=$00. That transport is P6b's, not scaffolding. Sixteen candidate decoder models, three capture decimations and a searched prologue are fitted to MAME's capture. Exactly one reproduces it sample-exact over all 1,678 samples, and every axis carries a negative control: flip it alone and the closest survivor disagrees on 826, 1,504, 156 and 1,522 samples. The chip runs 'terms', takes the LOW nibble of a byte first, clamps the accumulator at 10 bits and starts it at -2. tools/encoder/adpcm.py defaulted to the opposite of all four, and 65.2 named the wrong axis as the risk: the delta formula is worth -2.88 dB and the NIBBLE ORDER is worth -25.74 dB. 65.1's "high first, measured" was a measurement of ffmpeg, i.e. of the VOX file convention, which is a different question from what a chip does with a byte in its data register. The 10-bit clamp is free on the Singe window and only because that window peaks at 435 of 511 -- 1.4 dB of headroom on a -13.4 dBFS passage, 12.1 dB below where the encoder was clamping, and inside the recursion. So the audio level is an open choice again, downward, and the loudest passage on the disc is unmeasured. Session 33's silence had two ordinary causes: the PPI's port C is an input until control word $92 says otherwise, and $01 is COMMAND_STOP. And a rig fact worth the space: the 8 MHz ADPCM clock is CT1 in the YM2151's $1B, delivered on the sound system's schedule rather than at the store, so a transfer started in the same breath as the setup plays its first ~17 ms at the old clock and no model fits a stream that changed rate part way through. Name the layer: this is MAME 0.277's okim6258 device model measured end to end through the machine's real transport. It settles the rig and not the silicon. Also struck: 64.4's "no MAME source tree is on this machine" -- there is none on disk, but the machine has network and the upstream tag fetches. check.sh ALL GREEN before (tmp/check_s34_start.log) and after (tmp/check_s34_end.log), with one new stage. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
f925a1dd9a |
Put sound on the wire, and find three LSBs are worth 25 dB
ROADMAP P6, everything in the item except the bus half session 20 closed. tools/encoder/adpcm.py is an MSM6258 codec, tools/encoder/extract_audio.py takes the same seconds of the same stream the frames come from, tools/bench/verify_adpcm.py is the gate, tools/analysis/32_audio_wire.py the container arithmetic. There is no reference encoder -- ffmpeg has a decoder for this format and none the other way -- so what is gated is the decoder the encoder runs INSIDE its own nibble search, sample-exact against ffmpeg's over 4,268 nibbles. An encoder that agrees with its own wrong decoder is what that catches. The Singe window: 156,250 samples -> 78,125 B at 21.97 dB, which is 7,812.5 B/s to the byte. Normalising the disc's -13.4 dBFS level moves the SNR 21.97 -> 21.97, so the level is not a lever. And the two published delta formulas are not the same codec. They differ by at most 3 in 12-bit units; encode for one and decode on the other and the SNR goes 21.97 -> -2.88 dB, the noise louder than the signal, because ADPCM is recursive the way the video codec is temporally recursive. Which one the chip runs is now P6a and it is a precondition on shipping any audio. And audio is the first thing the packed branch's simplification has cost anything for. A record has no index BY DESIGN, so audio cannot be per-record without making records variable; it rides a fixed cadence (F, A), the obvious F=1 wastes 57.3% of every audio sector, and the pick is F=11 A=14 -- 0.09% padding, 14,336 B held, wire 582.0 -> 589.6 KB/s. The codec container, which kept its index, pays zero. The MAME experiment did not work and 65.5 says so: :okim6258 is there at $E92001/$E92003, read out of the machine's own program map, and feeding it from Lua recorded silence across control 0..3 x port C 0..15. The register semantics were not guessed at further. FINDINGS 65. check.sh ALL GREEN before and after, with a new stage. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
6f698ca226 |
Put the player on a real volume, and find the write window is the frame
ROADMAP K3. src/player/packed.s (2,898 B) brings up its own display, builds
its own 193-entry DMA chain, keeps its own frame clock off V-DISP and fetches
every record itself with READ(10) off a CZ-6BS1. The rig writes no picture
byte, no palette entry and no CRTC register.
120 of 120 frames pixel-exact, every one compared, in both palette orders --
the gate had to grow to do it, because a packed frame is a LITERAL and the
codec's recursion was what made one comparison audit 120.
And the write window turns out to be the frame. A packed write requires R20
bit 11, buffer mode blanks the layer, and a DMAC-direct player holds the
window open for the whole data phase, so
dark fraction of a slot = record bytes / (DATA-PHASE rate x slot)
which is 1.0 at the container's own 582.0 KB/s: every frame delivered, on
time, pixel-exact, and none of them displayed. The rate in that expression is
the BURST rate, a third hardware number B1 has no test for. It reverses 61.5's
ranking -- a packed player that DMAs to RAM and paints with the measured 27.3%
blit is on screen 72.7% of every slot at any rate, and the two are equal only
at 2,131 KB/s = 3.7x the wire.
And a held channel costs the frame clock half its ticks without the clock
being able to tell: 487 of 1,038 V-DISP edges lost, zero late frames reported,
the player believing 12 fps while the screen ran at 6.37.
FINDINGS 64. ROADMAP K4 opened and fenced behind B2.
check.sh ALL GREEN before and after.
Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
|
||
|
prosolis
|
f1007a0dbc |
Put the frame in a container with no decoder, and find the palette is not free
ROADMAP K2. DLXP1: a 49,664 B record that is 97 sectors exactly, no index and no length word, because a packed record's length is geometry rather than content. 582.0 KB/s, which is what FINDINGS 61.9 predicted to the tenth, and it encodes in 3.3 s because there is no k-means in it. px68k's own x68k/gvram.c renders the container's bytes index-exact with the harness computing no interleave -- the only test that can catch an encoder whose byte order is wrong, since a container round-trips against its own inverse either way. Both negative controls fail as they must. The picture is re-derived against this project's builder rather than PIL's (34.05 dB against 61.9's 34.08) and the GGGGGRRRRRBBBBBI word is charged for the first time in this tree: 0.53 dB, on every row, so it moves no comparison. What the control found is the finding. A packed container on a SCENE palette lands exactly on the codec's ceiling, so the whole +2.31 dB is the per-frame palette and nothing else -- and 231 of 256 entries change every frame, which makes a mismatched paint 12.8 dB worse than the correct pairing, on screen for roughly half of every frame slot if buffer mode does not blank. So B2 now decides which packed CONTAINER ships, not only which player. The fallback is already a flag: --scene-palette --no-palette is 30.79 dB, zero churn, 576.0 KB/s and still +2.07 dB on the shipping codec. 62.5 is priced and is a wash: palette first 20.32 dB, palette last 20.33. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
07f36c2af9 |
Put the palette on the channel, and find one start paints a whole frame
ROADMAP K1, the packed player's one open structural item. A frame is a picture
AND a palette, and no run in this tree had pointed a DMA channel at the palette
registers. dmagate.s runs 7-9, gated by dma_run.sh and check.sh:
7. 512 B off the disc into $E82000, bus held -- byte-exact in 256 register
words, read back OUT OF the registers by the 68000;
8. the SAME transfer aimed at RAM -- byte-exact at $2C000, and 256 of 256
palette words still read the poison the CPU wrote, which is what attributes
run 7 to the channel's MAR rather than to the readback path;
9. ONE array-chained start across two kinds of destination -- the palette and
six picture rows at the 1,024 B line stride, 2,048 B byte-exact.
So a packed frame is one channel start: a 193-entry array, palette first, CPU
halted from the first byte to the last. The array is scene-constant, because
the packed layout spends both 256-colour pages and there is no page to flip.
What is left on the CPU per frame in the video path is the channel start and the
READ(10) -- no per-frame PAINT, which is not the same claim as no per-frame CPU.
The destination is POISONED first (62.1). Runs 4-6 wrote into RAM that was zero
and GVRAM that was stale against a record that is mostly pad; "it matches the
disc" was weaker than it read as. The host counts whether the poison actually
discriminates instead of assuming it: 511 of 512, and the gate refuses under 500.
And it opened a hardware item (62.4, ROADMAP B4). MAME maps the palette to
palette_device over memory_array, whose write16 is a plain COMBINE_DATA -- RAM
that honours mem_mask, with no handler that could refuse a byte write. Unlike
GVRAM's 256-colour arm there is nothing here to be wrong about, so the run
bounds the model and not the board. What a real palette register does with a
byte write is unmeasured. A negative costs 0.28% of a frame and nothing else.
29_packed_player.py now also prints the two rows with the per-frame palette
charged -- 55.7% of a frame on the chain, 582 KB/s -- alongside the picture-only
figures the codec comparison is quoted against.
check.sh ALL GREEN before (tmp/check_s30_start.log) and after
(tmp/check_s30_end.log).
Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
|
||
|
prosolis
|
1be428c270 |
Align the container to the disc, and find the decoder-free packed player fits
Two sessions, unrecorded until now, committed together because their edits share files and cannot be split cleanly after the fact. Session 28 (FINDINGS 60): the container is DLX5 -- every record sector-aligned, 120/120 starting on a boundary where 3/120 did, +0.48% on the wire and zero clocks -- and the ring's release rounds to RECALN so no pad is stranded. Two encoder levers measured and refused: `--spans all` buys +0.19 dB for +67% of the wire, and joint span/lam selection emits byte-identical containers because `lam` never leaves its floor on any of 120 frames. Session 29 (FINDINGS 61): the packed full-frame blit is 27.3% of a 12 fps frame, a channel fills GVRAM in buffer mode off the disc with the CPU halted, and it walks the 1,024 B line stride itself through array chaining. At the 9 clk/B dual-address floor the codec is 110.4% of a frame and a decoder-free packed literal player is 55.2%, at +4.89 dB -- 2.75 dB past a ceiling the codec's scene-wide palette cannot cross. Encoder work is parked; the codec is kept and not built on. check.sh is ALL GREEN before and after, plus one new stage that gates the ORDER of the measured paint costs rather than their values. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
8800d8f8c0 |
Price the transport against the frame, and find dual address cannot fit
FINDINGS 59.7. tools/analysis/15_bus_occupancy.py has always answered "what does each W cost" and never "what can the frame afford", and after 59.2 those are not the same question. It now answers both, and takes an optional --kbps for the auto-request rows -- the only rows whose cost depends on how long the record takes to arrive. On the gate container at 12 fps, decode term MEASURED: decode 68.5%, audio 1.25%, HEADROOM 30.2% = 6.74 clocks per byte at a 37,403 B record. Against that, P4a cut the transport from 391.7% of a frame to 40..95% -- four to ten times, the largest movement in this project's cost model since the decoder was written -- and it still does not fit. A dual-address byte is a 4-clock read of the device plus a 5-clock write to memory, so 9 clk/B is a FLOOR and the frame affords 6.74. No GCR share goes under it and no delivery rate goes under it: a share decides whether the channel sits at the floor or above it. At 460 KB/s max-rate totals 165.1% and LRAR at 50% totals 117.4%, and a 50% share tops out at 543 KB/s, above which the channel is the bottleneck and the rate falls back to exactly that floor. So 59.2's three bounds arrive in the budget as one sentence: the configurations this machine can run are the ones the frame cannot afford, and the one it can afford -- single address, 5 clk/B, 92.2% total, 7.8% spare -- needs the device to ACK the DMAC, which needs a request line MAME does not connect and the slot pinout does have at B36/B37. ROADMAP re-ranks accordingly. B3 stops being a constant to look up and becomes DOES THE CARD DRIVE #EXREQ, ahead of B1: B1 sets how much headroom the player has, B3 decides whether there is any. New E7 carries the other branch -- if the answer is no, the container must reach 27,995 B a frame, 328 KB/s of payload, against 438 now. The dependency diagram is redrawn around that fork. The scope is stated rather than buried: this is the GATE container, deliberately the heaviest thing the encoder emits, and the lighter cpufit family was NOT priced -- 15_bus_occupancy.py refuses it, correctly, because the C68K measurement it cross-checks against belongs to the gate container. E7 therefore begins with a harness re-run, and until then "34% too big" is a statement about the fixture and not about the project. check.sh ALL GREEN before and after. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
621a5bb457 |
Put the data phase on the DMAC, and find auto-request is charged by time
ROADMAP P4a. src/player/dma.i programs HD63450 channel 1 and takes the SCSI DATA IN phase off the CPU; src/player/dmagate.s reads the same 2,048 B at LBA 1000 three ways -- PIO, the channel with the bus held, the channel stealing cycles -- and all three are byte-exact against the host's copy of the volume. The evidence never reads $EA0015, because 57.3 established that it cannot: with the DMAC's OWN asserted MAME cannot tell a CPU-driven byte there from a DMAC-driven one. The discriminator is the CPU's own progress. MTC is sampled by the INSTRUCTION AFTER the one that starts the channel, and held it reads 0 of 2,048 -- the whole transfer happened between two instructions, because the 68000 did not execute in between -- against the full count and 426 loop trips for the stealing configuration. Put the stealing registers in the held slot and every byte still arrives and tools/bench/dma_run.sh goes RED, which is what says the counter can come out different; 58.3's vacuous "UNDERRUNS: 0/120" is the trap being avoided. tools/analysis/27_dmac_config.py decodes the four register bytes out of the player's own source, with the MC68450 field tables now in one copy (tools/analysis/mc68450.py) shared with 21_iplrom_dmac.py, so the player's configuration and the IPL ROM's 16..19 clk/B one are the same decoding. Three bounds on the apparatus, read out of MAME 0.277 rather than inferred: the CZ-6BS1 has NO request line to the DMAC (its flow control is DTACK), so external request cannot be run; single address cannot be run either, because only channel 0 has device callbacks; and only burst is modelled as held. Of the four rows of the W ladder exactly one -- dual address held -- has a code path here, and it is the one demonstrated. W did not move by one clock, for the third session running. What outlives the emulator is the currency. Every W in this project is clocks per DELIVERED byte, which presumes the device asks; an auto-requested channel spends its share of the bus whether or not a byte is there, so a record costs what it costs to ARRIVE -- halve the delivery rate and the CPU cost of the same record doubles. tools/analysis/28_autorequest_cost.py prices it from MC68450 3.8 and 5.2.3.3.2, gating its formulas against Table 5-3's sixteen rows first. At 37,405 B and an explicit 460 KB/s: max rate costs the whole 95.3% of a frame the record takes to land, and of the GCR's four bus shares only BR=00, 50%, carries the rate -- 10.61 clk/B, 47.6% of a frame, against 40.4% for the W=9 row and 391.8% measured for PIO. The GCR is a design lever nothing in this tree had named. 59.4 changes what is left. sc_in_data now REFUSES a windowed read when the data phase is the channel's (SCE_WINDOW), because a channel writes a contiguous run and cannot drop the 300 B in front of a record. 117 of 120 records need one, so sector-aligned records have gone from a preference in ROADMAP's re-encode bundle to the precondition the transport enforces -- and that bundle is now the only thing between this tree and M2. One collision, recorded because the procedure is the finding: DM_USE first sat at $18300, which is ring.i's XF_SLOT mailbox, and the P4b stage -- untouched by this work -- went red on a run that never reached its snapshot. check.sh was ALL GREEN before any of this, which is what made that red unambiguous. ALL GREEN after too, with one new stage. decode.bin is unchanged at 1,296 B and the same MD5. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
5921fab118 |
Put the transport on the 68000, and find PIO costs 87 clocks a byte
ROADMAP P4b. src/player/xfer.i answers src/player/ring.i's XF_* mailbox with a real READ(10) to a real MB89352 in place of tools/bench/stream.lua's modelled transport: 120 records, 4,488,588 B, pixel-exact out of a 256 KB ring, with a real mid-stream seek in a second pass. The tiling is the SAME 18 wraps and 14.7 KB mean hole that 49.4's host producer and 55.4's modelled transport produced -- a third transport, same placement, which is the assertion that ring.i could not tell which side of the seam answered it. What it costs is the finding. tools/bench/xfer_cost.sh subtracts the same 120 frames run twice and gets 87.28 clocks per delivered byte, against the 68000's own cycle table for the loop, which says 87.15 -- 0.2% apart, so the cost is the instruction stream and not MAME's device model, and it is the first number this rig has produced that survives leaving the emulator. That is 391.8% of a 12 fps frame; the machine's own V-DISP clock agrees from the other end at 2.57 fps. Against the ladder, W=5 held is 22.4% of a frame and W=19 is 85.3%, so P4a is worth 4.6x the worst DMA configuration in this tree and 17.5x the best -- where before this session it was worth 9 against 19. W itself did not move by a clock. "UNDERRUNS: 0/120" is vacuous with a synchronous transport, and stream.lua now prints that argument next to the zero: a frame cannot start before its record has landed because the decoder IS the transport. The counter that means something is NO IDLE, 119/120 with a worst overrun of 441 whole ticks. Same class of error as 49.7.2's free-running ring passing at 48 KB. 58.3: a record is not a sector -- 117 of 120 start part way into one, and reading whole sectors into the ring corrupts the neighbours rather than wasting bytes (49.2, no bounds check). scsi.i reads the covering sectors and stores only the window, which is free in PIO and stops being free the moment P4a succeeds. tools/analysis/26_sector_align.py prices the three ways out and sector-aligned records win on both axes: +0.43% wire and zero clocks, against +1.34% and a bounce copy at +5 clk/B. ROADMAP now carries a four-item re-encode bundle and P4a should be attempted against a sector-aligned container. check.sh gains two stages and was ALL GREEN before and after. decode.bin is unchanged at 1,296 B and the same MD5. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
00232bb22b |
Import the scene graph, and find the worst gap between two branch points is zero
ROADMAP G1, scheduled early because it is a measurement input, and it paid for that immediately. 51.3 established that a ring's lookahead is accumulated out of pipe - wire and that a seek spends all of it, so what a branch point costs is set by the time since the last one. 55.5 rehearsed a seek on the machine and said out loud that it could not ask the question, because nothing here knew where the branch points are. They are everywhere, and 5.4% of them are free of charge to the player and not to us. Over 612 distinct transitions into a seek, taking the earliest instant each input window opens: worst 0.000 s, p10 0.950, median 3.473, best 82.497. 33 open on the first frame of a clip the disc SEEKED to -- press right as flaming_ropes.enter_room appears and you are in fall_to_death, two seeks with no play between them. 51.2's slack rule can therefore be answered NO by the content rather than by the buffer, and no amount of ring is a defence. It does not break the design. A branch on an empty ring costs the 2-record prefill, 149.7 ms at 488 KB/s, not the climb. What it removes is margin: at that rate in a 256 KB ring, 76% of this game's branch points arrive before the ring has refilled, and a 512 KB ring makes that 90%, because doubling the ceiling does not touch the surplus. The ring is not the lever; the surplus is. CORRECTION to FINDINGS 16: there is only one transcription. The SNES chapter set says in its own README that it is derived from DirkSimple, so the planned diff of two independent sources catches conversion errors only. Run anyway: durations agree 388/505 within a frame, branch structure 470/505, and of the 35 differences 16 are renames and 18 of the other 19 are that port dropping the arcade's diagonals. Zero transcription discrepancies, and none were findable. Two constraints on the input layer come free: the arcade needs eight directions, and the shortest input window is 98 ms against 54.4's 72.13/90.16 ms frame slot, so input cannot be polled on the frame tick. The coupling to outside source is contained to one file (USER DECISION). tools/import/scenegraph.py is the only code here that knows those projects exist -- their paths, table names, timing formulas, constants -- and it writes DLXSCENE1, this project's own schema, into gitignored tmp/ with the sources' licences inside it. tools/analysis/25_scene_graph.py reads only that. Nothing is vendored and nothing outside-derived is committed. The split was made after the measurement and the whole output was re-run byte for byte to show it moved no number. Both import gates are negative-tested: deleting one sequence upstream fails the 516/906 count, and closing the table early fails the constructor-end check, which replaced one that was vacuous. No 68000 code ran or changed; decode.bin is still 1,296 B at the same MD5. check.sh gains an import stage that skips when there is no checkout. ALL GREEN before and after. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
2676f3b835 |
Put the ring on the 68000, and find the disc stops whenever the player is not asking
ROADMAP P5. The loader moved in session 21 and the frame clock in 22; the ring producer was the last policy living outside the machine. src/player/ring.i does `aligned` placement, the descriptor ring, a prefill, 51.2's slack rule and a seek, and the host keeps only the transport. It needed a container change. `aligned` asks whether the next record fits before the end of the ring -- a length asked BEFORE the record is fetched -- and every reader in this tree answered that by walking the frame stream, which is exactly what a player streaming off a disc cannot do. DLX4 carries nframes u16 record lengths in the scene header. Frame payloads are byte-identical to the DLX3 encode, so no fitted constant moves; the scene header goes 5,920 to 6,164 B. The producer reproduces the host's tiling exactly: 18 wraps, 14.7 KB mean hole, pixel-exact, a third independent implementation of the same policy. What it exposed is bigger than the item. A channel only moves bytes while it has a request and only the CPU can issue one, so the disc stands still between records by an amount the PLAYER sets, not the medium -- and no host-filled run could see it. At 488 KB/s in a 256 KB ring a one-deep request queue gives away 6.8% of the pipe and underruns 59 of 120 frames; two-deep gives away 3.4% and underruns none. The container's whole surplus over the wire is 8.7%, so the player's own loop was spending most of the slack a branch point saves up. Prefill is the weaker lever: six records of it still leaves 24 underruns. Three silent bugs are recorded in FINDINGS 55.7 -- all produced wrong pixels or a desync rather than a fault -- plus a rig one: MAME renders a screen line by line, so snapshotting the frame the decoder finished in captures a tear that reads exactly like a decoder bug. check.sh gains the machine-owned ring and a seek with the decode after it. decode.bin is unchanged at 1,296 B and a host-filled run executes none of the new code, so every FINDINGS 49/51 figure stands. ALL GREEN before and after. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
c419251266 |
Put the frame clock on the 68000, and find that the 12 fps frame does not exist
ROADMAP P3 said "needs MFP timer or VBL" and neither can do it. The MFP's timer clock is 16 MHz/4, its prescalers stop at 200 and its data register is 8 bits, so the slowest tick any single timer can make is 78.125 Hz -- 6.5x faster than a frame -- and 4e6/12 is not an integer, so no setting reaches 12 Hz at all. The raster has no whole divide near 12 either: 4 refreshes is 13.86 fps and 5 is 11.09. tools/analysis/23_frame_clock.py walks all 7x256 timer settings rather than asserting it. src/player/clock.i takes the V-DISP falling edge on MFP GPIP4 -- the start of vertical blanking, which is when a player would present -- and adds fps*VTOTAL per edge to a 16-bit accumulator, emitting a tick at 31,500 and keeping the remainder. The long-run rate is fps*VTOTAL/VTOTAL = 12.000000 fps exactly, and both constants are read out of the CRTC at init, so the clock is derived from the registers that generate the raster it counts. Measured over 3,000 refreshes: 3,000 interrupts, 649 ticks where 649.1429 were due. It costs 181.35 clocks per V-DISP, 838 per frame, 0.1006% of the budget -- timed by the 68000 itself, because the host's granularity is 17.64 ms and the interrupt is microseconds. The loop's own cost was calibrated rather than looked up and landed on 38.000002 clocks, which both licenses the subtraction and confirms buscost.py's model; the 181.35 then decomposes exactly, leaving 43.99 clocks for the interrupt exception -- the textbook 44, measured. THE ONE THAT MOVES SOMETHING: 12 fps on a 55.4577 Hz raster is 4.6215 refreshes, so a frame is shown for 4 refreshes (72.13 ms) or 5 (90.16 ms), 37.9% of them short. The 833,333-clock budget every figure in this project is priced against is the MEAN slot, and the short one is 13.4% under it. The cadence was already in the tree unnamed: stream.lua's tick is sampled at frame boundaries, so its gaps were always 4 or 5, and every host-paced result in FINDINGS 49/51 carried it. P3 moved who produces it onto the machine and made it visible. It is not a dropped frame -- the pace gate lets an overrun eat the next frame's idle -- and on the gate container it costs 4 frames of 120 their idle against 1 for the nominal model, most of that the frame-0 transient at 111% of budget. stream.s counts it now, and the rig matches an offline model of the divider exactly. Also struck: MAME's raster runs 2.22% fast. refresh_mode() builds the frame period from scr.max_x*scr.max_y with scr.max_x = m_htotal - 8, one character cell short and an inclusive bound used as a count, so it runs at 56.6901 Hz where the registers say 55.4577 -- agreeing to six digits with the arithmetic. Every "1/55.46 s granularity" note in this tree was wrong and is 1/56.69 s, corrected in six files with the derivation put once in crtc_mode.lua. No conclusion changes and no 68000 cycle figure moves; the CPU clock is unrelated to the screen. But anything paced by the raster runs fast under MAME, so the rig reports both rates and prices the interrupt against the hardware's. decode.s and frame.i are unchanged; decode.bin is still 1,296 B at the same MD5. The pace gate's wait loop is byte-for-byte the one FINDINGS 51 measured and the free-running path executes none of the new code. check.sh gains two stages: the clock's own measurement, and 120 frames decoded pixel-exact with nothing outside the machine deciding when a frame may start. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
7179339bd2 |
Move the loader onto the 68000, and find 5,920 bytes nobody counted
src/player/load.i expands both codebooks to word-per-pixel form and packs the palette to GGGGGRRRRRBBBBBI out of the RAW container header, byte-exact against tools/bench/dlxload.py on both CPU cores. The palette half is gated on words read back out of the palette registers at $E82000, so "the words reached the hardware" is part of what passes. ROADMAP P1 is done; P2's encoder half (a reserved black entry, 23.4) is not, and is a re-encode rather than an edit. A scene change costs 18.96 ms of 68000 time, 22.8% of one 12 fps frame; boot costs 24.70 ms. The scratch tables describe the CRTC, not the scene, so pal_tables is a separate entry point built once at boot -- 5.29 ms off every scene change. The one that moves something: the scene header is 5,920 B that no rate table in this tree included, because it belongs to no frame record. In FINDINGS 51.3's currency it is divided by the surplus pipe - wire, so it is hypersensitive: 138 ms of extra refill climb at 488 KB/s and 1.099 s at 451.4 KB/s, for the same bytes. tools/analysis/22_scene_load.py prices it across explicit rates. Recorded as open: the two CPU cores agree to <3% on every stage but the table build, where they differ by 16.4%. px68k's C68K charges a flat 50 clocks for MULU/MULS (c68kmacro.h:1869) where the 68000 charges 38+2n, which explains 4,608 of the 8,703 clock gap. 4,095 clocks are unexplained. Nothing else in src/player/ multiplies, so no figure in FINDINGS 24-52 is affected. decode.s and stream.s are untouched; decode.bin is still 1,296 B at the same MD5. check.sh gains a stage that gates byte-exactness on both cores and deliberately does not gate the cycle counts -- MAME's clock is 1/55.46 s and a wall timing would make the green light host-sensitive. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
e565dfbbab |
Take the disc mount out of the source, so someone else's copy of the media works
Three files hardcoded /media/reala-misaki/BDROM -- extract.py, 07_motion_survey.py and check.sh -- which made the pipeline unrunnable for anyone whose disc mounts anywhere else. DLX_BDROM now overrides it everywhere, defaulting to /media/$USER/BDROM, so this box is unchanged and another one works. Verified by running extract.py against a symlinked mount at a different path. check.sh's failure message names the path it looked at and the variable to set, instead of assuming udisks put it where this machine puts it. README gains a "Reproducing this" section: no media ships here and none of it is redistributable, so it says what you have to bring (the disc) and what is already packaged (vasm is vendored as a binary with its source tarball; the k-means is hand-rolled, so numpy and Pillow are the whole Python dependency). It also names the two gates that SKIP rather than fail -- the px68k second-core pass and the IPL ROM DMAC gate -- because both live outside this repo and a silent skip is worth reading as a skip. One trap called out rather than left to bite: scene selection is a hard-coded stream number, not a search. A different pressing that numbers its .m2ts files differently will extract the wrong footage and the green light will PASS on it. check.sh ALL GREEN. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
2f9f5cc995 |
Pace the ring, then read the DMAC config out of the IPL ROM: audio is cheap and the disk is not
Two sessions that were never separated in the working tree, so they land as one commit. check.sh ALL GREEN before and after both. SESSION 19 -- the ring rig gets a frame clock (FINDINGS 51). src/player/stream.s had no frame clock: it asked for record i the instant it finished i-1, outran any finite pipe, and never let the ring back up. The 49.1 sweep passing at 48 KB was therefore a wrap-correctness result and nothing else. PACE/PACEON ($18034/$18038) hold the decoder to 12 fps, so FR_HEAD-FR_TAIL finally means what it reads as: whole frames the decoder could still draw with delivery stopped dead. PACEON=0 free-runs and is what the wrap gate still uses, so every figure in 49 is unmoved. Paced, on the gate container: 64 KB holds 2 frames, 256 KB holds 7-8, 512 KB holds 14-15, all pixel-exact. Tolerance is ceiling-1, measured by cutting the pipe: 256 KB buys 500 ms of dead pipe, not 583. SLACK IS ACCUMULATED, NOT OWNED. It is built out of pipe-wire and a seek spends all of it. At 488 KB/s a 256 KB ring needs 4.83 s of play to reach its ceiling from empty; 512 KB needs 8.42 s to reach 14. A bigger ring raises the ceiling AND lengthens the climb, so a branch point does not ask "is the buffer big enough" but "has there been enough play since the last one" -- and Dragon's Lair's decision points are seconds apart. The rig now also says WHICH resource is binding: at 460 KB/s every ring from 192 KB to 512 KB is rate-bound at ceiling 4 and never fills, so larger rings are dead RAM in that scene. 20_seek_slack.py is the same model rewritten in Python from record sizes, sharing no code with the Lua producer: 35/35 ceilings inside its bracket. SESSION 20 -- the DMAC configuration was in the IPL ROM the whole time (FINDINGS 52). ROADMAP's "do this first" was to put the ADPCM stream on the bus. That needs a clocks-per-byte figure for the audio channel, and 11_cpu_budget.py was charging audio the DISK's rate -- 5 clk/B, its own help text calling it "single-address, bus held". Audio was being charged the favourable end of B3, a 242 KB/s open question. It never had to be a guess. The IPL ROM programs all four HD63450 channels itself and MAME boots the rig with it, so 21_iplrom_dmac.py reads the configuration out of the image and decodes the MC68450 fields. Eight (address, expected bytes, meaning) sites; a mismatch or an unknown revision exits non-zero. In check.sh, no emulator, milliseconds. ch3 DCR=$80, OCR=$32: dual address, 8-bit port, cycle steal WITHOUT hold, REQG=10 external request. The DMAC arbitrates once per byte with no burst to amortise the 5..8 + 2 over, so an audio byte is 16..19 clocks, not 5 -- the old debit was 3.2x..3.8x small. And on the bus it is still nothing: 651 B/frame is 1.25%..1.48% of a frame, about 4% of what the decoder leaves. P6's bus risk does not materialise. The unit worry was worth checking and nearly right: 15.6 kHz is 8 MHz/512 = 15,625 samples/s, two 4-bit samples to a byte = 7,812.5 B/s exactly, and AUDIO_KBPS=7.8 is that in decimal kB while the tool multiplied by 1024. THE DISK CHANNEL IS PROGRAMMED IDENTICALLY. ch1 (SASI) is DCR=$80 too, and so is ch0. That is 16..19 clocks per delivered byte, where 42.4 brackets W at 5..12 and 42.5 has W=8 already missing 47/120 frames. The only worked example of a disk DMA configuration on this machine sits above the entire bracket, and at that price nothing fits at any container size. It is not scsiexrom.bin so B3 stays open -- what changed is that a cheap configuration is now the thing that has to be SHOWN. W <= 12 is a requirement on the player's DMAC programming, not a range the hardware hands us, and it is now the largest open number in the project, ahead of the rate. An unforced cross-check fell out: 15_bus_occupancy.py's new W sweep puts W=8 at 105.7% of the frame, agreeing with 42.5's 47/120, from mode histograms and bus clocks respectively, two models sharing no code. Also: ADPCM outranks the disk at the arbiter (CPR 1 against 2), so an audio byte never waits and a video byte does -- relevant to 51's smooth-rate delivery model. README MEDIA. stream.lua gains DLX_SNAP_EVERY=1 (needs DLX_PACE, off by default, on no path check.sh takes) and tools/media/make_readme_media.py turns the PNGs into docs/img/. The stills and both clips are MAME's own screen pixels. Building it turned up something worth recording. 116 of 119 captured frames are pixel-exact against dlx.py; three are TORN -- frame n on top, frame n-1 below the tear line -- because MAME captured the screen while the block loop was partway down it. decode.s writes straight to the displayed page (one display path, 28.1), so a real player tears the same way, and this is the first time that consequence has been visible rather than argued. The script ASSERTS the tear and refuses to build otherwise, rather than trimming three frames and reporting "every frame I kept is exact". Second correction the capture forced: the snapshot fires before frame n is decoded, so the obvious reading is that it holds frame n-1 -- it does not, because MAME renders the screen at the end of the machine frame, by which time the 68000 has finished frame n. 11_cpu_budget.py's "validated to within 1 pt" line is also corrected: the model reads 2..10 pt HIGH and by more as the frame gets harder, which was already true before either session. src/player/decode.s is unchanged; decode.bin is still 1,296 B at the same MD5. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
b49bbdc939 |
Build v7 into the player, and find the cost model 18% wrong on the block it made commonest
src/player/decode.s now paints v7 literal spans, pixel-exact under MAME and px68k's C68K core over a container where every frame carries 128-216 spans covering up to 38% of the picture. The span pass is blit.s v7 verbatim: the 66.0/9.143/9.978 fit was measured on that instruction sequence. The container is DLX3 -- a span section between the mode header and the block payload, since that is the only place the 68000 can reach without first parsing something of variable length. 16_span_roundtrip.py gates it in check.sh, and asserts it emitted enough spans to have tested anything. Two synthetic all-SPAN anchors price v7 inside decode.s at 151.2 and 225.6 clocks per 4x4 block, against FINDINGS 40's table of 151 and 226 -- 0.2% on both emulators. The measured mode costs what it was said to cost. Two things that were not on the list: TWO BYTE BUDGETS. FINDINGS 40's 18/120 was scored against the 488 KB/s PIPE, not the 280 KB/s profile, and at the profile rate the lam search has already spent the allowance -- spans fired on 5 frames of 120 and looked like a regression. The profile is a chosen quality rate point; the pipe is hardware. --kbps and --span-kbps are now separate and spans run before mu, because a span pays in bytes and mu pays in picture. Delivered: 86/120 over budget without spans, 77/120 at the profile budget, 34/120 on the pipe for +0.36 dB. C_SKIP_MIXED WAS NEVER MEASURED, and it was 18% low -- 45.0, now 55.0. It is the one constant in the table that came from a derivation, because the synthetic frame that would measure it cannot exist: a byte needs a coded block for its SKIP to be mixed. Four bracketing anchors measure it on both emulators with the header byte rotated through all four positions, and the partner mode solves back to its own anchored value to 0.2%. With it corrected the model predicts a real spanned decode to -0.06% mean / 0.09% worst, against -2.99% / 4.30%. It matters because a span marks its run SKIP, so mixed SKIPs dominate exactly the frames spans are judged on. Also: the rig had been writing its synthetic timing frames 26 KB past the top of a 2 MB machine, and got away with it because the modes it overran are data-independent. A span's jump displacements come out of the stream, so it is not. And frames-over-budget is no longer a safe headline -- the controller aims at the deadline, so 55 of 120 frames sit within 5% of it and a 1% cost shift moves 22 frames. FINDINGS 41. check.sh ALL GREEN, now gating on a span-heavy DLX3 container. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
c520a89e14 |
Measure the finer chain tail: 84/120 becomes 18/120, and the derivation was right by cancellation
blit.s gains v7 -- v6's 24-pixel movem chain plus a second chain whose unit is
one `move.l (a0)+,(a2)+`. Measured over 13 span lengths by span.sh, every config
pixel-exact:
cycles = 66.0 per span + 9.143 per COARSE pixel + 9.978 per FINE pixel
fitting all 13 to within 0.2%. v5 and v6 re-measure to FINDINGS 30 exactly, so
the harness has not drifted underneath the new variant.
Rescored against the same scsi window and the same additive model, v7 takes
84/120 frames over budget to 18/120 -- exactly what FINDINGS 39.4 derived, and
that agreement is two cancelling errors: the derivation's 2-register movem tail
is 29% too dear per pixel, and its "nothing per span" for the second chain entry
is 22.3 clocks too cheap. The plain post-incrementing move.l is the right tail
instruction, and it makes the padding quantum 2 pixels, which a run of 4x4
blocks pads to exactly zero.
The DMAC stays dropped on a measurement now rather than an argument: v7 takes
back 37 of the 43 frames the array chain would, with no reserved channel and no
timing neither emulator here can verify. Break-even against all-V1 moves from
L=4 blocks to L=2.
The fine displacement is carried mid-stream rather than in the span record, so
the decoder holds nothing across the copy and keeps all 12 payload registers --
which is the whole reason the coarse unit is 24 pixels.
span.sh is now -seconds_to_run 200 (30 s wall, 36 configs) and takes its
expected snapshot count from the generated metadata instead of a literal 23.
Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
|
||
|
prosolis
|
c5ca56330e |
A second emulator agrees, the bus was never counted, and the DMAC loses by one clock
Three things, and the last one reversed itself when the datasheet arrived.
A SECOND EMULATOR. tools/bench/c68k/ links px68k's C68K core into a headless
harness -- no SDL, no ROMs, no emulated machine, because the decoder touches
nothing but RAM, the control block and GVRAM. decode.s is now pixel-exact under
two independent CPU cores, and cycle-table error against MAME is bounded at
3.3%, running against us. MAME 0.277's M68000 turns out to be the MICROCODE
core, not Musashi (m68000.lst + m68000gen.py), so this is two structurally
different timing models agreeing rather than two tables. FINDINGS 28.8's "V4
costs more than RAW" reproduces independently. FINDINGS 37.
THE BUS. Nothing since FINDINGS 24 had counted the 68000's local memory bus --
one 4-clock cycle at a time, carrying instruction prefetch as well as data. The
decoder occupies 86.7% of it and PREFETCH IS 62% OF THAT TRAFFIC, so a data-only
count understates occupancy by 2x. Two sources check each other: c68k_bench
counts every bus callback exactly, and a static walk of decode.lst supplies the
prefetch no emulator here can report. The walk reproduces the measured data half
to 0.04%, which is what licenses its prefetch half, and 15_bus_occupancy.py is a
gate rather than a report because every bus figure depends on that check.
FINDINGS 38.
THE DMAC CHAIN LOSES. FINDINGS 29.6 named it the one uncosted lever. Costed from
bus arithmetic -- a read cycle plus a write cycle, 8 clocks a pixel -- it scored
1/120 frames over budget against the v6 span's 10/120 and looked decisive. Then
the MC68450 manual (Motorola Jul 1989, now at ~/src/mc68450.pdf): Fig 4-25 sheet
4 puts a dual-address word between two 16-bit ports at 9 CLOCKS, because note 2
gives the DMAC 4-clock reads and 5-clock WRITES. The 68000 writes in 4.
DMAC 9.000 clocks/pixel datasheet
v6 9.152 clocks/pixel measured, FINDINGS 30
1.7%. Scored additively, 86% of what remains of the DMAC's advantage is v6's
24-pixel padding quantum -- a property of its unrolled movem chain, fixable in
software with a finer tail chain, worth 55/120 -> 18/120 against the DMAC's
12/120. Recommendation: fix the quantum, drop the DMAC. Six frames does not buy
a reserved channel, a two-region container layout and a timing dependency
neither emulator here can verify. The container is identical either way -- v6's
record and an HD63450 chaining entry are both 6 bytes, so the chain array IS the
span table -- so nothing is foreclosed. FINDINGS 39.
TWO CORRECTIONS TO MY OWN WORK IN THE SAME SESSION:
- I argued FINDINGS 35's flat CPU debit for the disk was too pessimistic and
rescored the window at 53/120 with max(CPU, bus). Wrong. A 68000 has no cache
and a two-word prefetch queue, so it stalls the moment another master takes
the bus, and the MC68450 hands the bus over in SLABS under limited-rate
auto-request rather than interleaving per operand. DMA is additive. 84/120
stands and 14_dmac_chain.py reproduces it exactly. What 86.7% occupancy really
says is that there is almost no room to overlap anything. FINDINGS 38.3.
- The first DMAC costing was derived where a primary source existed. Both wrong
answers were confident and both were caught by reading the manual.
Also landed:
- FINDINGS 5's 8 clocks/word for the SCSI DMA, STATUS's own "most load-bearing
unmeasured number", is now bracketed by the datasheet: 5 clk/word with the bus
held, ~12 if the DMAC arbitrates per word. 8 is a supported midpoint, and
which end applies is a player design decision worth 7 clocks a word on a
480 KB/s stream. FINDINGS 39.7.
- check.sh gains two gates: the C68K pixel-exact decode (seconds, no MAME) and
the bus-model self-check. Both skip cleanly without a px68k checkout.
- spanned blocks are now charged their mode-map dispatch, which FINDINGS 30.7
flagged as uncounted in 12_span_tradeoff.py.
- MAME timed runs must be budgeted by WALL CLOCK, not -seconds_to_run: this box
runs x68000 at ~0.033x realtime and two runs were killed by their own timeout.
That is why the all-RAW cell in 37.3 is empty. The C68K harness does the same
work in seconds because it emulates a CPU and not a machine.
Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
|
||
|
prosolis
|
7d365b3ff5 |
Drop SASI on capacity, then find the budget never had the disk in it
USER DECISION: drop the `sasi` profile. Not on bandwidth -- on capacity. A SASI volume is 40 MB, and the 22.8 min of unique scene footage on the source Blu-ray (streams 00000-00201, measured, not recalled) is 146 MiB at the LOWEST rate this codec makes -- more than the machine's whole 4-unit SASI space. `scsi` is the only profile now. FINDINGS 32. Then the user asked whether we were drawing the wrong conclusions about PIO vs DMA, and we were, more broadly than the question implied. Every CPU figure in FINDINGS 24-34 is scored against the full 833,333 cycles/frame with nothing subtracted for moving the bitstream off disk. Debiting the HD63450 cycle-steal at the long-standing 8 clk/word ESTIMATE, "1 frame of 120 misses" becomes 84 of 120, median 112.4%. PIO at the span rate is 99.8% of the machine. Spans buy cycles by spending bandwidth and the bandwidth returns as steal, so 31.6's "fits completely" becomes a worst frame of 114.3%. 10 fps absorbs it: median 93.7%, 1/120. FINDINGS 35. `11_cpu_budget.py` takes --io dma|pio|none, defaults to dma, and warns if asked for none. Also landed: - item 1 done: the cost model checked against the 68000 on a cost-aware container, -3.07% to +0.01%, whole-window mean -1.22%. FINDINGS 34. - item 4 done: the container carries its own 4-byte record alignment (DLX2). 94/120 record starts were on odd addresses -- an address error, not a slow read -- now 0/120 for 16 B/s. Re-encoding reproduces 31.1 exactly. FINDINGS 33. - a `scsi` window does not fit the 2 MB machine the rig emulates (2.84 MB of stream past a 0x200000 ceiling). The gate now verifies 80 of 120 frames and SAYS so, and fails loudly when the pass does not complete, instead of reporting a phantom 49,005-pixel diff. FINDINGS 36. Three near-misses this session had one shape: an unobservable run nearly produced a false finding. stdbuf -oL on any MAME job that prints progress -- a file is block-buffered too, and a run that is merely finishing looks exactly like one that is wedged. check.sh ALL GREEN. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
06b98d4b47 |
Price cycles in the mode decision: 37 misses become 1, for 0.26 dB
The decoder has been CPU-bound since FINDINGS 28 while the mode decision
minimised D + lam*R -- distortion against BYTES. decide() now minimises
D + lam*bytes + mu*cycles, and ratectl bisects mu per frame against the
833,333-cycle budget with the lam bisection nested inside it. On the worst
sustained window:
sasi 27.22 -> 26.95 dB, 109.5 -> 109.4 KB/s, 37/120 misses -> 1
scsi 29.90 -> 29.27 dB, 280.0 -> 278.6 KB/s, 51/120 misses -> 1
Bitrate does not move: the byte controller still binds, and mu changes WHICH
modes are bought. V4 is what it stops buying -- 25.2 -> 20.3% of blocks at sasi
and 15.0 -> 5.3% at scsi, where RAW takes it. That is 28.8's inversion in
practice: RAW is dearer in bytes and cheaper in cycles, so only the byte-rich
profile can buy its way out of V4.
Three things worth knowing beyond the headline:
- The one frame that still misses, at both profiles, is FRAME 0 -- no previous
reconstruction, so 100% changed by definition, which is also what a scene
cut is. It comes out at the all-V1 floor of 110.6% and is emitted late on
purpose. Freezing a cut to make a deadline is the worse failure.
- 28.7's "11 frames are impossible" was too pessimistic. That floor held the
SKIP set fixed and asked how cheaply the drawn blocks could be drawn; the
real decision can also MOVE a block to SKIP, which above ~90% non-SKIP is
the only lever left.
- SKIP's price depends on its neighbours (13.25 cycles clustered, 45 mixed),
which a per-block lagrangian cannot see. The way out is that the two uses
need not share a cost function: a ranking constant inside decide(), the
exact clustered rule for the frame-level bisection. vq_hybrid.cycles() is
now the one definition of that rule and 11_cpu_budget.py imports it.
Gated: 09_ratectl_drift.py runs both controllers, both 0/120 drifting frames.
The cost-aware container decodes pixel-exact on the 68000 (120 frames). ON by
default in encode.py; --no-cpu-fit restores session 7. check.sh ALL GREEN.
Still a model, not a measurement, for THIS container: FINDINGS 31's cycle
figures come from vq_hybrid.cycles (within 1 point of the 68000 on four frames
of the session-7 container). Timing this one on the machine is step 1 of the
next session -- it was started and killed for time, and it is slow.
FINDINGS 31. tools/analysis/13_cpu_ratectl.py.
Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
|
||
|
prosolis
|
29eb78a599 |
Measure the span: the mode survives, and it is an encoder format
FINDINGS 29 priced a literal-span mode at 4*(50 + 4L*9.08) cycles and labelled
the whole section DERIVED. Session 8 step 0 was to measure it before optimising
over the mode set it implies. Two variants in blit.s, one stream per span length
from prep_spans.py, timed by span.lua, driven by span.sh in ~25 s:
v5, handed (x, npix) and left to work the copy out: 97.9/span + 10.459/px
v6, handed an address and a jump displacement: 43.7/span + 9.152/px
29 assumed 50.0/span + 9.080/px
So 29's arithmetic was right about a format nobody had written. The difference
is not tuning: v5 spends ~122 cycles a span computing a destination, dividing
npix into bursts and handling a 0..15 remainder, all of which the encoder knows
at build time. v6's record is {u32 absolute GVRAM address, u16 jump
displacement} into an unrolled chain of 24-pixel copy units -- no loop, no
remainder, no arithmetic -- and it fits 11 span lengths to 0.3%.
Three things that measurement showed and derivation could not:
- The per-pixel cost is a function of REGISTER PRESSURE. FINDINGS 24's 9.08
was a fixed blit with 12 registers free; v5 can spare 8 and pays 10.46; v6
gets 12 back only because the encoder holds the state.
- Short spans die in the remainder path -- a 12-pixel span costs MORE than a
16-pixel one -- and the fix is padding, not avoidance.
- Odd-x alignment is free (259.0 vs 261.8 cycles/span), as a 16-bit bus
implies but nobody had checked.
Re-priced against the unchanged mode maps, sasi: median 74.4% -> 52.0% (29 said
43.0), misses 37 -> 10/120 (29 said 8), 448.0 KB/s. Break-even moved from runs
of 2 blocks to runs of 4. 29.4 survives: a scene cut needs x >= 0.196 of the
frame as spans and the bus allows x <= 0.373, so it fits at 12fps.
All 23 timing configs are also checked pixel-exact, so none of this was timed
against a decoder that quietly skipped work.
FINDINGS 30. Next: lever B, the cost-aware mode decision.
Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
|
||
|
prosolis
|
3641f37e28 |
The bus is 4x idle while the CPU is pinned: price the trade
The codec was designed when bytes were scarce, so every decision in it trades
cycles to save bytes. That is now backwards: sasi spends 110 KB/s of a 488 KB/s
pipe while missing 31% of frames on CPU.
The cheapest thing a 68000 can be handed is the most expensive thing to store.
Measured, per pixel: row-linear copy from word-expanded memory 9.08 cycles,
block-order 12.98, V1 codebook 18.74, RAW byte literals 25.03. So the 1024-byte
stride costs 43% and unpacking bytes to words costs more than the write itself.
Pricing one new mode -- a per-row span of word-expanded literals movem.l'd
straight from the stream buffer -- against the UNCHANGED mode maps:
sasi median 74.4% -> 43.0%, worst 136.2% -> 106.2%, misses 37 -> 8/120,
101.7 -> 453.2 KB/s
scsi median 94.9% -> 69.4%, misses 51 -> 18/120, 272 -> 479.7 KB/s
scsi gains less precisely because it has less idle bandwidth left to trade.
Two consequences worth flagging. A word-expanded literal block derives to ~240
cycles, cheaper than V1's measured 299.9 and pixel-exact -- so every codebook
mode is CPU-dominated by a literal, and the codebook is a byte optimisation
that now costs cycles. And 28.5's "a scene cut cannot fit at 12fps" reopens:
CPU needs >=19% of the frame as spans, the bus allows <=39%, and that interval
is not empty.
DERIVED, NOT MEASURED, and labelled as such everywhere. The 9.08 cycles/pixel
is real but was measured at full row width with 12-register bursts, so short
spans are flattered. Measuring one span on the 68000 is now step 0 of the next
session, ahead of the cost-aware mode decision, because it changes the mode set
that decision optimises over.
FINDINGS 29. tools/analysis/12_span_tradeoff.py.
Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
|
||
|
prosolis
|
ed353d24a9 |
Budget both profiles against both clocks: the CPU limit is the clock, not the profile
11_cpu_budget.py takes --machine. Clocks confirmed from MAME 0.277
x68k.cpp:1133/1194/1200, not recalled: x68000 AND x68ksupr are both
40_MHz_XTAL/4 = 10 MHz; only the XVI is faster at 33.33_MHz_XTAL/2.
sasi scsi
stock 10MHz 31% miss 42% miss
XVI 16.7MHz 0% miss 0% miss
sasi is the cheaper profile but it does not fit either at 10 MHz. The XVI
column is headroom, not a target: the profiles are an I/O-bandwidth axis and
say nothing about CPU, and the locked target CPU is a stock 10 MHz 68000 for
both of them. So both profiles have to fit the same 833,333-cycle budget, and
the cycle ceiling has to be enforced in the encoder regardless of which one
ships.
Model comparisons are now gated to the clock and framerate they were stated
at: quoting 24.5's 76.6% or the stock-machine 68000 timings against an XVI
budget compares a model to a measurement of a different machine.
Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
|
||
|
prosolis
|
e1aa26bb57 |
The 68000 decoder draws pixel-exact frames, and does not fit
src/player/decode.s parses DLX1 and decodes straight into GVRAM. Verified pixel-exact over a 120-frame sequential run of the worst sustained window on the disc -- all four block modes, full temporal recursion, so the last frame is only right if all 120 were. In check.sh. It costs a mean of 81.7% of a 12fps frame budget, and 31% of frames exceed 100% (42% at scsi). CPU is now the binding constraint. FINDINGS 28. Three things that were believed and are not true: - The dual-display-path plan of FINDINGS 24.5/25.6 is incoherent. The compose path needs a RAM copy of the previous reconstruction; the direct path's selling point is that it keeps none. Mixing them shows stale pixels on 70 of 120 frames, worst frame 18.8% of the screen. Every coherent repair is dearer than not mixing, and 24.5's two figures were both copies with no decode in either, so there was never a crossover to find. One path ships, and the 96KB reference frame is gone. tools/analysis/10_pathmix_drift.py keeps the counterexample runnable; check.sh asserts it still reproduces. - The four block modes do not cost the same. V1 300, V4 448, RAW 400 cycles against the old model's flat 207.8. V4 is 25% of blocks and 50% of the cycles, and the mode decision charges it bytes it does not charge cycles for. tools/analysis/11_cpu_budget.py reproduces all four frames timed on the 68000 to within 1 point. Hand-derived timings agree to 0.5% on V1. - The container is big-endian but not aligned. Variable-length records laid end to end put frame 1's length field at an odd address, and move.l (a0)+ there is an address error: frame 0 decoded perfectly and then vectored into the IPL for 59 emulated seconds looking like a hang. Found by dumping PC, not by reading the source. Also: an all-V1 frame, the cheapest possible full redraw, is 110.5% of budget. No mode assignment fits a scene cut at 12fps. That one needs a decision, not a measurement. Next: charge cycles in the mode decision and bisect against 833,333 per frame, the way session 6 bisects lam against bytes -- but with no bucket, because a late frame cannot be banked. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
497f88b945 |
Rate control: rebuilt per-frame, wired in, and gated at zero drift
FINDINGS 26 stopped the session-5 rate controller before it shipped: it built a lam-ladder of independent whole-sequence encodes and picked frames off it, so SKIP blocks referenced reconstructions the decoder never saw -- 111 of 120 frames drifted. The fix is the structural one 26.1 said it had to be. vq_hybrid is now frame-drivable -- frame_ctx / decide / paint -- and encode() is a thin loop over it. Rate control drives the same three calls, bisects lam per frame under the leaky bucket, and feeds back the frame it actually emitted. The desync has no way to occur, and 09_ratectl_drift.py goes 111/120 -> 0/120. That test is now part of check.sh, which is ~2 min rather than ~40 s. Both overshoots on the worst sustained window are closed for under 1 dB, totals including audio: sasi 137.4 -> 109.5 KB/s (-0.60 dB), scsi 381.6 -> 280.0 KB/s (-0.91 dB). Zero frames hit the lam=800 cliff, so nothing was destroyed to get there. Rate control also makes the display path cheaper -- scsi's median drops 53.6% -> 47.1% -- because raising lam moves blocks to SKIP and V1. Two knobs measured rather than guessed. --rc-floor is worth 0.00 dB on that window and defaults to the profile lam, so rate control cannot regress content that already fits. --prefill defaults to 0 and is documented as a trap: it buys a permission to overshoot of exactly bucket/nframes, and on a 14-frame clip it disables the controller outright. FINDINGS 26.5 was wrong in both halves and 27.6 records it. _paint was not the bottleneck (14% of a frame, though vectorising it was still right at 17.1x) and the ladder was never "minutes" -- those were k-means in build(). What makes per-frame rate control affordable is that VQ.assign depends on neither lam nor prev, so it is cached one frame deep: a 12-step search over 120 frames costs 0.31 s against 49.1 s. Also caught: fixed-lam sasi was already 5% over target on 00020, the clip everyone called easy. Nothing noticed because the profile table quotes PSNR and not bitrate. check.sh: ALL GREEN. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
145753c0bf |
Handoff: rate control is next, and it is unsound as written
Session 5 handoff. The user has chosen rate control as the next session's work, so this reads ratectl.py properly before that session starts rather than discovering the problem mid-implementation. FINDINGS 26: encode_rate_controlled() is not sound. H.encode() is temporally recursive -- SKIP blocks copy the previous RECONSTRUCTION -- but rate control builds a ladder of independent whole-sequence encodes and picks each frame from whichever rung fits the budget. Frames then reference reconstructions the decoder never saw. Measured on the Singe window: 67 rung switches, 111 of 120 frames drift, worst frame 43.4% of pixels, reported PSNR overstated by 0.36 dB. It would have wired up cleanly and reported a plausible wrong answer. Two further defects in the same function: the lam ladder runs to 2e5, 250x past the FINDINGS 15 cliff, so a frame that only fits up there is destroyed rather than rate-controlled; and with 5 rungs only two are ever chosen, straddling the operating point by 7.5x. The docstring describes a per-frame binary search, which is the right design -- the implementation is a fixed ladder. The leaky bucket does work and should be kept: 109.1 KB/s against a 110 target. tools/analysis/09_ratectl_drift.py is the regression test and the acceptance criterion: it exits non-zero until zero frames drift. Also corrected the stale 38% blit figure in ratectl.py's profile commentary, which session 5 measured at 53.6% (FINDINGS 24), and recorded the pgrep -f self-kill trap again -- four times across three sessions now. check.sh ALL GREEN. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
e00264a058 |
Find the sustained action sequence: it breaks both profiles
The open risk since session 2 was "a sustained action sequence could still break the bitrate", with every clip measured so far being 1.2-1.7 s. Closed by measurement rather than by sampling clips by hand. 07_motion_survey.py scans a whole stream at 96x72 for the hottest sliding window of inter-frame difference. On 00223 the spread between the quietest and hottest sustained 10 s windows is 10.6x, which is the argument for not eyeballing it. Hottest is t=539.4s, the Singe endgame. There, with the fixed lam the CLI uses, sasi overshoots 110 -> 129.6 KB/s (+18%) and scsi 280 -> 373.8 KB/s (+34%). Rate control moves from "insurance, not a fix" to required, and is promoted above the full-disc survey. The bus is not broken -- 381.6 KB/s still fits the 488 KB/s figure -- so FINDINGS 21 survives, at 78% of the pipe instead of a comfortable margin. Three further corrections fall out: - The two largest streams on the disc are bonus material. 00216 is the feature with a burned-in commentary PiP; 00215 is the commentary. 00223 is the clean 9.4 min. A size-ranked survey would have encoded live action. - On hard content the 256-colour scene palette (31.33 dB) binds well before the X68000 display (40.81 dB); scsi is already within 0.51 dB of it. - FINDINGS 24.5's architecture question resolves to "both paths, chosen per frame": 30-53% of frames sit above the 70% crossover. Picking per frame costs a median 37.0% of the frame budget and caps at 53.6%. Reporting for this is wired into encode.py, which previously only printed a mean over all frames -- the one statistic that cannot answer a per-frame question. extract.py takes optional start/dur; 08_mode_map.py renders source | decoded | block-mode map to .webm. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
64cd1ffd72 |
Handoff: reconcile docs and tooling with the corrections made this session
Session 2 reversed several of its own conclusions. The docs are append-only, so a reader could land on a superseded section and act on it. This pass makes the repo internally consistent. Defects found and fixed in STATUS.md: - claimed "Hybrid VQ with k=1024: no" as the answer to the linework question, directly contradicting FINDINGS 14, which rejected k=1024. Both profiles are k=256. - malformed profile table (six column separators, five columns). - next-steps list had two items numbered 3 and listed the full-disc survey twice. - the disk-benchmark section still read CRITICAL-PATH with "if SCSI sustains >=800 KB/s, ship pixel-exact". That was written while the bandwidth figure was misread as 4 MB/s. At 4 Mbps pixel-exact needs 92-97% of the pipe and is not available, and the ring-buffer result means the design no longer hangs on the benchmark at all. Rewritten with what it IS still worth doing: confirming the 4 Mbps provenance, and confirming DMA is used rather than PIO. FINDINGS now carries supersession blockquotes on 5, 8, 11, 17 and 18 pointing at the sections that correct them. 18 is the dangerous one -- its peak-vs- sustained test is reversed by 21 -- so it is marked DO NOT ACT ON THIS SECTION while noting the per-frame data itself remains valid. profile_gen.py had the same problem in code: it defaulted to the superseded peak sizing and returned lam=25 where the docs say lam=10. The buffered test is now the default and peak sizing is behind --size-for-peak as a bound only. A tool that contradicts the findings is worse than no tool. Also preserves the five measurement scripts that produced this session's numbers as tools/analysis/05-09, following the session 1 precedent, and adds an "explicitly abandoned -- do not re-propose" list to STATUS covering entropy coding, k=1024 codebooks and flat 4x4 VQ. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
reala-misaki
|
65112b9305 |
Session 1: hardware research, content measurement, codec decision, MAME harness
Verified GVRAM is one word-access per pixel in ALL color modes; chose 256-color 256x192 with movem.l bursts (page 1 sacrificed as double-buffer). Measured 8 scenes from the Blu-ray source: blit costs under 8% of the 12fps cycle budget, so I/O is the bottleneck, not CPU. Naive delta+RLE reaches only 3.2:1 (365 KB/s, 470MB) -> decision to use 4x4 vector quantization (~30 KB/s). "Shot on twos" assumption failed: the transfer has zero duplicate frames, so 12fps requires explicit decimation. Documents three false measurement results and their root causes (per-frame Floyd-Steinberg dithering, temporal denoise, exact-match dedupe on noisy source). MAME Lua injection harness works and is reusable for cycle-cost measurement; the IOCS _B_READ disk benchmark is blocked returning -1. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |