1be428c2700515f27d5e8b150021d9ca6605395e
33
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 prosolis
|
1be428c270 |
Align the container to the disc, and find the decoder-free packed player fits
Two sessions, unrecorded until now, committed together because their edits share files and cannot be split cleanly after the fact. Session 28 (FINDINGS 60): the container is DLX5 -- every record sector-aligned, 120/120 starting on a boundary where 3/120 did, +0.48% on the wire and zero clocks -- and the ring's release rounds to RECALN so no pad is stranded. Two encoder levers measured and refused: `--spans all` buys +0.19 dB for +67% of the wire, and joint span/lam selection emits byte-identical containers because `lam` never leaves its floor on any of 120 frames. Session 29 (FINDINGS 61): the packed full-frame blit is 27.3% of a 12 fps frame, a channel fills GVRAM in buffer mode off the disc with the CPU halted, and it walks the 1,024 B line stride itself through array chaining. At the 9 clk/B dual-address floor the codec is 110.4% of a frame and a decoder-free packed literal player is 55.2%, at +4.89 dB -- 2.75 dB past a ceiling the codec's scene-wide palette cannot cross. Encoder work is parked; the codec is kept and not built on. check.sh is ALL GREEN before and after, plus one new stage that gates the ORDER of the measured paint costs rather than their values. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
8800d8f8c0 |
Price the transport against the frame, and find dual address cannot fit
FINDINGS 59.7. tools/analysis/15_bus_occupancy.py has always answered "what does each W cost" and never "what can the frame afford", and after 59.2 those are not the same question. It now answers both, and takes an optional --kbps for the auto-request rows -- the only rows whose cost depends on how long the record takes to arrive. On the gate container at 12 fps, decode term MEASURED: decode 68.5%, audio 1.25%, HEADROOM 30.2% = 6.74 clocks per byte at a 37,403 B record. Against that, P4a cut the transport from 391.7% of a frame to 40..95% -- four to ten times, the largest movement in this project's cost model since the decoder was written -- and it still does not fit. A dual-address byte is a 4-clock read of the device plus a 5-clock write to memory, so 9 clk/B is a FLOOR and the frame affords 6.74. No GCR share goes under it and no delivery rate goes under it: a share decides whether the channel sits at the floor or above it. At 460 KB/s max-rate totals 165.1% and LRAR at 50% totals 117.4%, and a 50% share tops out at 543 KB/s, above which the channel is the bottleneck and the rate falls back to exactly that floor. So 59.2's three bounds arrive in the budget as one sentence: the configurations this machine can run are the ones the frame cannot afford, and the one it can afford -- single address, 5 clk/B, 92.2% total, 7.8% spare -- needs the device to ACK the DMAC, which needs a request line MAME does not connect and the slot pinout does have at B36/B37. ROADMAP re-ranks accordingly. B3 stops being a constant to look up and becomes DOES THE CARD DRIVE #EXREQ, ahead of B1: B1 sets how much headroom the player has, B3 decides whether there is any. New E7 carries the other branch -- if the answer is no, the container must reach 27,995 B a frame, 328 KB/s of payload, against 438 now. The dependency diagram is redrawn around that fork. The scope is stated rather than buried: this is the GATE container, deliberately the heaviest thing the encoder emits, and the lighter cpufit family was NOT priced -- 15_bus_occupancy.py refuses it, correctly, because the C68K measurement it cross-checks against belongs to the gate container. E7 therefore begins with a harness re-run, and until then "34% too big" is a statement about the fixture and not about the project. check.sh ALL GREEN before and after. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
621a5bb457 |
Put the data phase on the DMAC, and find auto-request is charged by time
ROADMAP P4a. src/player/dma.i programs HD63450 channel 1 and takes the SCSI DATA IN phase off the CPU; src/player/dmagate.s reads the same 2,048 B at LBA 1000 three ways -- PIO, the channel with the bus held, the channel stealing cycles -- and all three are byte-exact against the host's copy of the volume. The evidence never reads $EA0015, because 57.3 established that it cannot: with the DMAC's OWN asserted MAME cannot tell a CPU-driven byte there from a DMAC-driven one. The discriminator is the CPU's own progress. MTC is sampled by the INSTRUCTION AFTER the one that starts the channel, and held it reads 0 of 2,048 -- the whole transfer happened between two instructions, because the 68000 did not execute in between -- against the full count and 426 loop trips for the stealing configuration. Put the stealing registers in the held slot and every byte still arrives and tools/bench/dma_run.sh goes RED, which is what says the counter can come out different; 58.3's vacuous "UNDERRUNS: 0/120" is the trap being avoided. tools/analysis/27_dmac_config.py decodes the four register bytes out of the player's own source, with the MC68450 field tables now in one copy (tools/analysis/mc68450.py) shared with 21_iplrom_dmac.py, so the player's configuration and the IPL ROM's 16..19 clk/B one are the same decoding. Three bounds on the apparatus, read out of MAME 0.277 rather than inferred: the CZ-6BS1 has NO request line to the DMAC (its flow control is DTACK), so external request cannot be run; single address cannot be run either, because only channel 0 has device callbacks; and only burst is modelled as held. Of the four rows of the W ladder exactly one -- dual address held -- has a code path here, and it is the one demonstrated. W did not move by one clock, for the third session running. What outlives the emulator is the currency. Every W in this project is clocks per DELIVERED byte, which presumes the device asks; an auto-requested channel spends its share of the bus whether or not a byte is there, so a record costs what it costs to ARRIVE -- halve the delivery rate and the CPU cost of the same record doubles. tools/analysis/28_autorequest_cost.py prices it from MC68450 3.8 and 5.2.3.3.2, gating its formulas against Table 5-3's sixteen rows first. At 37,405 B and an explicit 460 KB/s: max rate costs the whole 95.3% of a frame the record takes to land, and of the GCR's four bus shares only BR=00, 50%, carries the rate -- 10.61 clk/B, 47.6% of a frame, against 40.4% for the W=9 row and 391.8% measured for PIO. The GCR is a design lever nothing in this tree had named. 59.4 changes what is left. sc_in_data now REFUSES a windowed read when the data phase is the channel's (SCE_WINDOW), because a channel writes a contiguous run and cannot drop the 300 B in front of a record. 117 of 120 records need one, so sector-aligned records have gone from a preference in ROADMAP's re-encode bundle to the precondition the transport enforces -- and that bundle is now the only thing between this tree and M2. One collision, recorded because the procedure is the finding: DM_USE first sat at $18300, which is ring.i's XF_SLOT mailbox, and the P4b stage -- untouched by this work -- went red on a run that never reached its snapshot. check.sh was ALL GREEN before any of this, which is what made that red unambiguous. ALL GREEN after too, with one new stage. decode.bin is unchanged at 1,296 B and the same MD5. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
5921fab118 |
Put the transport on the 68000, and find PIO costs 87 clocks a byte
ROADMAP P4b. src/player/xfer.i answers src/player/ring.i's XF_* mailbox with a real READ(10) to a real MB89352 in place of tools/bench/stream.lua's modelled transport: 120 records, 4,488,588 B, pixel-exact out of a 256 KB ring, with a real mid-stream seek in a second pass. The tiling is the SAME 18 wraps and 14.7 KB mean hole that 49.4's host producer and 55.4's modelled transport produced -- a third transport, same placement, which is the assertion that ring.i could not tell which side of the seam answered it. What it costs is the finding. tools/bench/xfer_cost.sh subtracts the same 120 frames run twice and gets 87.28 clocks per delivered byte, against the 68000's own cycle table for the loop, which says 87.15 -- 0.2% apart, so the cost is the instruction stream and not MAME's device model, and it is the first number this rig has produced that survives leaving the emulator. That is 391.8% of a 12 fps frame; the machine's own V-DISP clock agrees from the other end at 2.57 fps. Against the ladder, W=5 held is 22.4% of a frame and W=19 is 85.3%, so P4a is worth 4.6x the worst DMA configuration in this tree and 17.5x the best -- where before this session it was worth 9 against 19. W itself did not move by a clock. "UNDERRUNS: 0/120" is vacuous with a synchronous transport, and stream.lua now prints that argument next to the zero: a frame cannot start before its record has landed because the decoder IS the transport. The counter that means something is NO IDLE, 119/120 with a worst overrun of 441 whole ticks. Same class of error as 49.7.2's free-running ring passing at 48 KB. 58.3: a record is not a sector -- 117 of 120 start part way into one, and reading whole sectors into the ring corrupts the neighbours rather than wasting bytes (49.2, no bounds check). scsi.i reads the covering sectors and stores only the window, which is free in PIO and stops being free the moment P4a succeeds. tools/analysis/26_sector_align.py prices the three ways out and sector-aligned records win on both axes: +0.43% wire and zero clocks, against +1.34% and a bounce copy at +5 clk/B. ROADMAP now carries a four-item re-encode bundle and P4a should be attempted against a sector-aligned container. check.sh gains two stages and was ALL GREEN before and after. decode.bin is unchanged at 1,296 B and the same MD5. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
e935d8661c |
Put the SPC on the 68000, and find P4 was blocked on a file nobody needed
ROADMAP P4, first half. Every byte the player has ever consumed was placed in emulated RAM by a host: decode.lua preloaded a container, stream.lua answered a mailbox at a modelled rate. src/player/scsi.i selects a SCSI target on a real MB89352 and issues READ(10) itself -- 4,096 B from LBA 0 and 2,048 B from LBA 1000, both byte-for-byte against the host's copy of the same volume, with no IOCS and no host in the transfer path. The non-zero LBA is the half that matters: a driver that emits a malformed LBA field still passes block 0, because zero is what a malformed field usually is. P4 was recorded as blocked in this tree and was not. Session 21's handoff said MAME's x68000 has no MB89352 path; -exp1 cz6bs1 instantiates one next to the HD63450, and FINDINGS 32.4 had read that card's DMA glue in session 9. The session-21 note is a regression in the record, not a discovery. What is genuinely absent is the 8 KB scsiexrom.bin MAME requires to INSTANTIATE the card and the player never executes -- driving the SPC registers directly has been the plan since BENCHMARK item 4 in session 2 -- so scsi_run.sh supplies a zero-filled placeholder on its own rompath, leaves the user's romset alone, and lets MAME print WRONG CHECKSUMS as it should. B3 is untouched: it wants that ROM's bytes disassembled and a blank one has none. The register map is measured, not inferred, and it corrects MAME's own documentation. The probe walks $EA0000..$EA003F one address at a time with a bus-error handler that records the fault and steps the index, because a sequential dump reports the first hole as the answer -- the earlier version took a bus error at $EA0006 and knew nothing about the other 57. 60 of 64 answer; the two holes are exactly the TMOD and EXBF the MB89352 omits and the MB87030 has. MAME leaves HOLES and does not shift the later indices down, which its own device summary claims it does, and that is what keeps DREG at $EA0015. The data register is DMA-only here and a PIO write vanishes. x68k_scsiext.cpp glues $EA0015 and nothing else, and with exown() asserted and DRQ low the byte is discarded: no error bit, no status change, no interrupt. Quieting all four DMAC channels does not change it. Measured rather than reasoned about -- write $5A, read back $00 with the FIFO still empty -- because ten command bytes vanishing without trace looks exactly like a target refusing a command, which is how it first presented. So every transfer runs the SPC in DMA mode and the CPU moves the bytes through the DMAC's own door. That costs the argument something, and it is easy to overclaim here: with exown asserted at idle MAME cannot distinguish a CPU-driven byte at $EA0015 from a DMAC-driven one. This shows the DATA PATH and cannot by itself show that the HD63450 is driving it, which is precisely what ROADMAP calls P4's first job. Whether a real CZ-6BS1 also refuses PIO there is not settled; it is a property of MAME's model and it wants a board. W did not move by one clock, and could not have. MAME's device models are functional rather than transfer-timing accurate and 42.5 reads its DMAC configured in wall-clock attotimes, so this is BENCHMARK Tier 1 -- does the read path work -- and never Tier 2. W is still the largest open number here. Five bugs, four of them silent, recorded in 57.5 because the pattern is the finding: a chain of rol.l #8 that loaded a transfer counter of ZERO from a count of 10; a byte handed to a FIFO mistaken for a byte on the bus; a fixed phase sequence where the bus decides the order; the discarded PIO write; and an initiator that must drop ACK and only then release the bus. The last appeared only once there were TWO reads -- one passed byte-exact and every conclusion from it was sound, and the second could not select. A player issues one command per record, so that failure would have been universal in the ring and invisible in a one-read demonstration. No decoder code changed; decode.bin is still 1,296 B at the same MD5. check.sh gains a SCSI stage that builds the volume out of the same stream_disk.bin the ring rig reads, gates the register window at 60 of 64 and both reads byte-exact, and skips when chdman is absent. ALL GREEN before and after. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
00232bb22b |
Import the scene graph, and find the worst gap between two branch points is zero
ROADMAP G1, scheduled early because it is a measurement input, and it paid for that immediately. 51.3 established that a ring's lookahead is accumulated out of pipe - wire and that a seek spends all of it, so what a branch point costs is set by the time since the last one. 55.5 rehearsed a seek on the machine and said out loud that it could not ask the question, because nothing here knew where the branch points are. They are everywhere, and 5.4% of them are free of charge to the player and not to us. Over 612 distinct transitions into a seek, taking the earliest instant each input window opens: worst 0.000 s, p10 0.950, median 3.473, best 82.497. 33 open on the first frame of a clip the disc SEEKED to -- press right as flaming_ropes.enter_room appears and you are in fall_to_death, two seeks with no play between them. 51.2's slack rule can therefore be answered NO by the content rather than by the buffer, and no amount of ring is a defence. It does not break the design. A branch on an empty ring costs the 2-record prefill, 149.7 ms at 488 KB/s, not the climb. What it removes is margin: at that rate in a 256 KB ring, 76% of this game's branch points arrive before the ring has refilled, and a 512 KB ring makes that 90%, because doubling the ceiling does not touch the surplus. The ring is not the lever; the surplus is. CORRECTION to FINDINGS 16: there is only one transcription. The SNES chapter set says in its own README that it is derived from DirkSimple, so the planned diff of two independent sources catches conversion errors only. Run anyway: durations agree 388/505 within a frame, branch structure 470/505, and of the 35 differences 16 are renames and 18 of the other 19 are that port dropping the arcade's diagonals. Zero transcription discrepancies, and none were findable. Two constraints on the input layer come free: the arcade needs eight directions, and the shortest input window is 98 ms against 54.4's 72.13/90.16 ms frame slot, so input cannot be polled on the frame tick. The coupling to outside source is contained to one file (USER DECISION). tools/import/scenegraph.py is the only code here that knows those projects exist -- their paths, table names, timing formulas, constants -- and it writes DLXSCENE1, this project's own schema, into gitignored tmp/ with the sources' licences inside it. tools/analysis/25_scene_graph.py reads only that. Nothing is vendored and nothing outside-derived is committed. The split was made after the measurement and the whole output was re-run byte for byte to show it moved no number. Both import gates are negative-tested: deleting one sequence upstream fails the 516/906 count, and closing the table early fails the constructor-end check, which replaced one that was vacuous. No 68000 code ran or changed; decode.bin is still 1,296 B at the same MD5. check.sh gains an import stage that skips when there is no checkout. ALL GREEN before and after. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
2676f3b835 |
Put the ring on the 68000, and find the disc stops whenever the player is not asking
ROADMAP P5. The loader moved in session 21 and the frame clock in 22; the ring producer was the last policy living outside the machine. src/player/ring.i does `aligned` placement, the descriptor ring, a prefill, 51.2's slack rule and a seek, and the host keeps only the transport. It needed a container change. `aligned` asks whether the next record fits before the end of the ring -- a length asked BEFORE the record is fetched -- and every reader in this tree answered that by walking the frame stream, which is exactly what a player streaming off a disc cannot do. DLX4 carries nframes u16 record lengths in the scene header. Frame payloads are byte-identical to the DLX3 encode, so no fitted constant moves; the scene header goes 5,920 to 6,164 B. The producer reproduces the host's tiling exactly: 18 wraps, 14.7 KB mean hole, pixel-exact, a third independent implementation of the same policy. What it exposed is bigger than the item. A channel only moves bytes while it has a request and only the CPU can issue one, so the disc stands still between records by an amount the PLAYER sets, not the medium -- and no host-filled run could see it. At 488 KB/s in a 256 KB ring a one-deep request queue gives away 6.8% of the pipe and underruns 59 of 120 frames; two-deep gives away 3.4% and underruns none. The container's whole surplus over the wire is 8.7%, so the player's own loop was spending most of the slack a branch point saves up. Prefill is the weaker lever: six records of it still leaves 24 underruns. Three silent bugs are recorded in FINDINGS 55.7 -- all produced wrong pixels or a desync rather than a fault -- plus a rig one: MAME renders a screen line by line, so snapshotting the frame the decoder finished in captures a tear that reads exactly like a decoder bug. check.sh gains the machine-owned ring and a seek with the decode after it. decode.bin is unchanged at 1,296 B and a host-filled run executes none of the new code, so every FINDINGS 49/51 figure stands. ALL GREEN before and after. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
c419251266 |
Put the frame clock on the 68000, and find that the 12 fps frame does not exist
ROADMAP P3 said "needs MFP timer or VBL" and neither can do it. The MFP's timer clock is 16 MHz/4, its prescalers stop at 200 and its data register is 8 bits, so the slowest tick any single timer can make is 78.125 Hz -- 6.5x faster than a frame -- and 4e6/12 is not an integer, so no setting reaches 12 Hz at all. The raster has no whole divide near 12 either: 4 refreshes is 13.86 fps and 5 is 11.09. tools/analysis/23_frame_clock.py walks all 7x256 timer settings rather than asserting it. src/player/clock.i takes the V-DISP falling edge on MFP GPIP4 -- the start of vertical blanking, which is when a player would present -- and adds fps*VTOTAL per edge to a 16-bit accumulator, emitting a tick at 31,500 and keeping the remainder. The long-run rate is fps*VTOTAL/VTOTAL = 12.000000 fps exactly, and both constants are read out of the CRTC at init, so the clock is derived from the registers that generate the raster it counts. Measured over 3,000 refreshes: 3,000 interrupts, 649 ticks where 649.1429 were due. It costs 181.35 clocks per V-DISP, 838 per frame, 0.1006% of the budget -- timed by the 68000 itself, because the host's granularity is 17.64 ms and the interrupt is microseconds. The loop's own cost was calibrated rather than looked up and landed on 38.000002 clocks, which both licenses the subtraction and confirms buscost.py's model; the 181.35 then decomposes exactly, leaving 43.99 clocks for the interrupt exception -- the textbook 44, measured. THE ONE THAT MOVES SOMETHING: 12 fps on a 55.4577 Hz raster is 4.6215 refreshes, so a frame is shown for 4 refreshes (72.13 ms) or 5 (90.16 ms), 37.9% of them short. The 833,333-clock budget every figure in this project is priced against is the MEAN slot, and the short one is 13.4% under it. The cadence was already in the tree unnamed: stream.lua's tick is sampled at frame boundaries, so its gaps were always 4 or 5, and every host-paced result in FINDINGS 49/51 carried it. P3 moved who produces it onto the machine and made it visible. It is not a dropped frame -- the pace gate lets an overrun eat the next frame's idle -- and on the gate container it costs 4 frames of 120 their idle against 1 for the nominal model, most of that the frame-0 transient at 111% of budget. stream.s counts it now, and the rig matches an offline model of the divider exactly. Also struck: MAME's raster runs 2.22% fast. refresh_mode() builds the frame period from scr.max_x*scr.max_y with scr.max_x = m_htotal - 8, one character cell short and an inclusive bound used as a count, so it runs at 56.6901 Hz where the registers say 55.4577 -- agreeing to six digits with the arithmetic. Every "1/55.46 s granularity" note in this tree was wrong and is 1/56.69 s, corrected in six files with the derivation put once in crtc_mode.lua. No conclusion changes and no 68000 cycle figure moves; the CPU clock is unrelated to the screen. But anything paced by the raster runs fast under MAME, so the rig reports both rates and prices the interrupt against the hardware's. decode.s and frame.i are unchanged; decode.bin is still 1,296 B at the same MD5. The pace gate's wait loop is byte-for-byte the one FINDINGS 51 measured and the free-running path executes none of the new code. check.sh gains two stages: the clock's own measurement, and 120 frames decoded pixel-exact with nothing outside the machine deciding when a frame may start. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
7179339bd2 |
Move the loader onto the 68000, and find 5,920 bytes nobody counted
src/player/load.i expands both codebooks to word-per-pixel form and packs the palette to GGGGGRRRRRBBBBBI out of the RAW container header, byte-exact against tools/bench/dlxload.py on both CPU cores. The palette half is gated on words read back out of the palette registers at $E82000, so "the words reached the hardware" is part of what passes. ROADMAP P1 is done; P2's encoder half (a reserved black entry, 23.4) is not, and is a re-encode rather than an edit. A scene change costs 18.96 ms of 68000 time, 22.8% of one 12 fps frame; boot costs 24.70 ms. The scratch tables describe the CRTC, not the scene, so pal_tables is a separate entry point built once at boot -- 5.29 ms off every scene change. The one that moves something: the scene header is 5,920 B that no rate table in this tree included, because it belongs to no frame record. In FINDINGS 51.3's currency it is divided by the surplus pipe - wire, so it is hypersensitive: 138 ms of extra refill climb at 488 KB/s and 1.099 s at 451.4 KB/s, for the same bytes. tools/analysis/22_scene_load.py prices it across explicit rates. Recorded as open: the two CPU cores agree to <3% on every stage but the table build, where they differ by 16.4%. px68k's C68K charges a flat 50 clocks for MULU/MULS (c68kmacro.h:1869) where the 68000 charges 38+2n, which explains 4,608 of the 8,703 clock gap. 4,095 clocks are unexplained. Nothing else in src/player/ multiplies, so no figure in FINDINGS 24-52 is affected. decode.s and stream.s are untouched; decode.bin is still 1,296 B at the same MD5. check.sh gains a stage that gates byte-exactness on both cores and deliberately does not gate the cycle counts -- MAME's clock is 1/55.46 s and a wall timing would make the green light host-sensitive. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
ed172c2da2 |
Rewrite the README as a description, and strike a unit error it exposed
The README had become a changelog: which session found what, which constant was retired and by whom, which figure was overturned. That is what STATUS is for, and it made the front page a poor read for anyone who has not been following. Rewritten to say what the project is, what it looks like, where it stands and how to reproduce it, present tense, no session numbers, no em dashes. Restating the standing facts caught one of my own from this session. FINDINGS 52.5 cited 42.4's W sensitivity table -- W <= 6 fits 0/120 frames, W = 8 misses 47/120 -- as though those were clocks per BYTE. They are per WORD. FINDINGS 43 is the section that caught W being charged per word to a byte-wide port and says in terms that 42.3's 0/120 was never physically reachable, so quoting it in byte units re-imported the exact 2x error 43 exists to have corrected, one section after using that same trap as a warning. Struck in 52.5 and in ROADMAP P4, and replaced with the ladder buscost.py already carries, which is in the right unit: 5 clk/B single address with the bus held, 9 dual address held, 12 single address arbitrated per byte, 16..19 dual address arbitrated per byte, the last being what the IPL ROM programs. Nothing else in 52 depended on the struck figures -- 15_bus_occupancy.py's sweep is in byte units and is unaffected. The "cross-check" claim goes with it. 52.5 read the agreement between its W = 8 row and 42.4's 47/120 as two independent models landing in the same place. It is two different units on two different containers at two different rates, and calling it corroboration was manufacturing agreement out of the unit error. What replaces "W <= 12 is a requirement on the player's DMAC programming" is the sharper version: getting the DMAC to HOLD THE BUS is what separates 9 from 19, and that is the thing P4 has to demonstrate. Docs only; no code changed and no gate output moves. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
2f9f5cc995 |
Pace the ring, then read the DMAC config out of the IPL ROM: audio is cheap and the disk is not
Two sessions that were never separated in the working tree, so they land as one commit. check.sh ALL GREEN before and after both. SESSION 19 -- the ring rig gets a frame clock (FINDINGS 51). src/player/stream.s had no frame clock: it asked for record i the instant it finished i-1, outran any finite pipe, and never let the ring back up. The 49.1 sweep passing at 48 KB was therefore a wrap-correctness result and nothing else. PACE/PACEON ($18034/$18038) hold the decoder to 12 fps, so FR_HEAD-FR_TAIL finally means what it reads as: whole frames the decoder could still draw with delivery stopped dead. PACEON=0 free-runs and is what the wrap gate still uses, so every figure in 49 is unmoved. Paced, on the gate container: 64 KB holds 2 frames, 256 KB holds 7-8, 512 KB holds 14-15, all pixel-exact. Tolerance is ceiling-1, measured by cutting the pipe: 256 KB buys 500 ms of dead pipe, not 583. SLACK IS ACCUMULATED, NOT OWNED. It is built out of pipe-wire and a seek spends all of it. At 488 KB/s a 256 KB ring needs 4.83 s of play to reach its ceiling from empty; 512 KB needs 8.42 s to reach 14. A bigger ring raises the ceiling AND lengthens the climb, so a branch point does not ask "is the buffer big enough" but "has there been enough play since the last one" -- and Dragon's Lair's decision points are seconds apart. The rig now also says WHICH resource is binding: at 460 KB/s every ring from 192 KB to 512 KB is rate-bound at ceiling 4 and never fills, so larger rings are dead RAM in that scene. 20_seek_slack.py is the same model rewritten in Python from record sizes, sharing no code with the Lua producer: 35/35 ceilings inside its bracket. SESSION 20 -- the DMAC configuration was in the IPL ROM the whole time (FINDINGS 52). ROADMAP's "do this first" was to put the ADPCM stream on the bus. That needs a clocks-per-byte figure for the audio channel, and 11_cpu_budget.py was charging audio the DISK's rate -- 5 clk/B, its own help text calling it "single-address, bus held". Audio was being charged the favourable end of B3, a 242 KB/s open question. It never had to be a guess. The IPL ROM programs all four HD63450 channels itself and MAME boots the rig with it, so 21_iplrom_dmac.py reads the configuration out of the image and decodes the MC68450 fields. Eight (address, expected bytes, meaning) sites; a mismatch or an unknown revision exits non-zero. In check.sh, no emulator, milliseconds. ch3 DCR=$80, OCR=$32: dual address, 8-bit port, cycle steal WITHOUT hold, REQG=10 external request. The DMAC arbitrates once per byte with no burst to amortise the 5..8 + 2 over, so an audio byte is 16..19 clocks, not 5 -- the old debit was 3.2x..3.8x small. And on the bus it is still nothing: 651 B/frame is 1.25%..1.48% of a frame, about 4% of what the decoder leaves. P6's bus risk does not materialise. The unit worry was worth checking and nearly right: 15.6 kHz is 8 MHz/512 = 15,625 samples/s, two 4-bit samples to a byte = 7,812.5 B/s exactly, and AUDIO_KBPS=7.8 is that in decimal kB while the tool multiplied by 1024. THE DISK CHANNEL IS PROGRAMMED IDENTICALLY. ch1 (SASI) is DCR=$80 too, and so is ch0. That is 16..19 clocks per delivered byte, where 42.4 brackets W at 5..12 and 42.5 has W=8 already missing 47/120 frames. The only worked example of a disk DMA configuration on this machine sits above the entire bracket, and at that price nothing fits at any container size. It is not scsiexrom.bin so B3 stays open -- what changed is that a cheap configuration is now the thing that has to be SHOWN. W <= 12 is a requirement on the player's DMAC programming, not a range the hardware hands us, and it is now the largest open number in the project, ahead of the rate. An unforced cross-check fell out: 15_bus_occupancy.py's new W sweep puts W=8 at 105.7% of the frame, agreeing with 42.5's 47/120, from mode histograms and bus clocks respectively, two models sharing no code. Also: ADPCM outranks the disk at the arbiter (CPR 1 against 2), so an audio byte never waits and a video byte does -- relevant to 51's smooth-rate delivery model. README MEDIA. stream.lua gains DLX_SNAP_EVERY=1 (needs DLX_PACE, off by default, on no path check.sh takes) and tools/media/make_readme_media.py turns the PNGs into docs/img/. The stills and both clips are MAME's own screen pixels. Building it turned up something worth recording. 116 of 119 captured frames are pixel-exact against dlx.py; three are TORN -- frame n on top, frame n-1 below the tear line -- because MAME captured the screen while the block loop was partway down it. decode.s writes straight to the displayed page (one display path, 28.1), so a real player tears the same way, and this is the first time that consequence has been visible rather than argued. The script ASSERTS the tear and refuses to build otherwise, rather than trimming three frames and reporting "every frame I kept is exact". Second correction the capture forced: the snapshot fires before frame n is decoded, so the obvious reading is that it holds frame n-1 -- it does not, because MAME renders the screen at the end of the machine frame, by which time the 68000 has finished frame n. 11_cpu_budget.py's "validated to within 1 pt" line is also corrected: the model reads 2..10 pt HIGH and by more as the frame gets harder, which was already true before either session. src/player/decode.s is unchanged; decode.bin is still 1,296 B at the same MD5. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
b49bbdc939 |
Build v7 into the player, and find the cost model 18% wrong on the block it made commonest
src/player/decode.s now paints v7 literal spans, pixel-exact under MAME and px68k's C68K core over a container where every frame carries 128-216 spans covering up to 38% of the picture. The span pass is blit.s v7 verbatim: the 66.0/9.143/9.978 fit was measured on that instruction sequence. The container is DLX3 -- a span section between the mode header and the block payload, since that is the only place the 68000 can reach without first parsing something of variable length. 16_span_roundtrip.py gates it in check.sh, and asserts it emitted enough spans to have tested anything. Two synthetic all-SPAN anchors price v7 inside decode.s at 151.2 and 225.6 clocks per 4x4 block, against FINDINGS 40's table of 151 and 226 -- 0.2% on both emulators. The measured mode costs what it was said to cost. Two things that were not on the list: TWO BYTE BUDGETS. FINDINGS 40's 18/120 was scored against the 488 KB/s PIPE, not the 280 KB/s profile, and at the profile rate the lam search has already spent the allowance -- spans fired on 5 frames of 120 and looked like a regression. The profile is a chosen quality rate point; the pipe is hardware. --kbps and --span-kbps are now separate and spans run before mu, because a span pays in bytes and mu pays in picture. Delivered: 86/120 over budget without spans, 77/120 at the profile budget, 34/120 on the pipe for +0.36 dB. C_SKIP_MIXED WAS NEVER MEASURED, and it was 18% low -- 45.0, now 55.0. It is the one constant in the table that came from a derivation, because the synthetic frame that would measure it cannot exist: a byte needs a coded block for its SKIP to be mixed. Four bracketing anchors measure it on both emulators with the header byte rotated through all four positions, and the partner mode solves back to its own anchored value to 0.2%. With it corrected the model predicts a real spanned decode to -0.06% mean / 0.09% worst, against -2.99% / 4.30%. It matters because a span marks its run SKIP, so mixed SKIPs dominate exactly the frames spans are judged on. Also: the rig had been writing its synthetic timing frames 26 KB past the top of a 2 MB machine, and got away with it because the modes it overran are data-independent. A span's jump displacements come out of the stream, so it is not. And frames-over-budget is no longer a safe headline -- the controller aims at the deadline, so 55 of 120 frames sit within 5% of it and a 1% cost shift moves 22 frames. FINDINGS 41. check.sh ALL GREEN, now gating on a span-heavy DLX3 container. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
c520a89e14 |
Measure the finer chain tail: 84/120 becomes 18/120, and the derivation was right by cancellation
blit.s gains v7 -- v6's 24-pixel movem chain plus a second chain whose unit is
one `move.l (a0)+,(a2)+`. Measured over 13 span lengths by span.sh, every config
pixel-exact:
cycles = 66.0 per span + 9.143 per COARSE pixel + 9.978 per FINE pixel
fitting all 13 to within 0.2%. v5 and v6 re-measure to FINDINGS 30 exactly, so
the harness has not drifted underneath the new variant.
Rescored against the same scsi window and the same additive model, v7 takes
84/120 frames over budget to 18/120 -- exactly what FINDINGS 39.4 derived, and
that agreement is two cancelling errors: the derivation's 2-register movem tail
is 29% too dear per pixel, and its "nothing per span" for the second chain entry
is 22.3 clocks too cheap. The plain post-incrementing move.l is the right tail
instruction, and it makes the padding quantum 2 pixels, which a run of 4x4
blocks pads to exactly zero.
The DMAC stays dropped on a measurement now rather than an argument: v7 takes
back 37 of the 43 frames the array chain would, with no reserved channel and no
timing neither emulator here can verify. Break-even against all-V1 moves from
L=4 blocks to L=2.
The fine displacement is carried mid-stream rather than in the span record, so
the decoder holds nothing across the copy and keeps all 12 payload registers --
which is the whole reason the coarse unit is 24 pixels.
span.sh is now -seconds_to_run 200 (30 s wall, 36 configs) and takes its
expected snapshot count from the generated metadata instead of a literal 23.
Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
|
||
|
prosolis
|
c5ca56330e |
A second emulator agrees, the bus was never counted, and the DMAC loses by one clock
Three things, and the last one reversed itself when the datasheet arrived.
A SECOND EMULATOR. tools/bench/c68k/ links px68k's C68K core into a headless
harness -- no SDL, no ROMs, no emulated machine, because the decoder touches
nothing but RAM, the control block and GVRAM. decode.s is now pixel-exact under
two independent CPU cores, and cycle-table error against MAME is bounded at
3.3%, running against us. MAME 0.277's M68000 turns out to be the MICROCODE
core, not Musashi (m68000.lst + m68000gen.py), so this is two structurally
different timing models agreeing rather than two tables. FINDINGS 28.8's "V4
costs more than RAW" reproduces independently. FINDINGS 37.
THE BUS. Nothing since FINDINGS 24 had counted the 68000's local memory bus --
one 4-clock cycle at a time, carrying instruction prefetch as well as data. The
decoder occupies 86.7% of it and PREFETCH IS 62% OF THAT TRAFFIC, so a data-only
count understates occupancy by 2x. Two sources check each other: c68k_bench
counts every bus callback exactly, and a static walk of decode.lst supplies the
prefetch no emulator here can report. The walk reproduces the measured data half
to 0.04%, which is what licenses its prefetch half, and 15_bus_occupancy.py is a
gate rather than a report because every bus figure depends on that check.
FINDINGS 38.
THE DMAC CHAIN LOSES. FINDINGS 29.6 named it the one uncosted lever. Costed from
bus arithmetic -- a read cycle plus a write cycle, 8 clocks a pixel -- it scored
1/120 frames over budget against the v6 span's 10/120 and looked decisive. Then
the MC68450 manual (Motorola Jul 1989, now at ~/src/mc68450.pdf): Fig 4-25 sheet
4 puts a dual-address word between two 16-bit ports at 9 CLOCKS, because note 2
gives the DMAC 4-clock reads and 5-clock WRITES. The 68000 writes in 4.
DMAC 9.000 clocks/pixel datasheet
v6 9.152 clocks/pixel measured, FINDINGS 30
1.7%. Scored additively, 86% of what remains of the DMAC's advantage is v6's
24-pixel padding quantum -- a property of its unrolled movem chain, fixable in
software with a finer tail chain, worth 55/120 -> 18/120 against the DMAC's
12/120. Recommendation: fix the quantum, drop the DMAC. Six frames does not buy
a reserved channel, a two-region container layout and a timing dependency
neither emulator here can verify. The container is identical either way -- v6's
record and an HD63450 chaining entry are both 6 bytes, so the chain array IS the
span table -- so nothing is foreclosed. FINDINGS 39.
TWO CORRECTIONS TO MY OWN WORK IN THE SAME SESSION:
- I argued FINDINGS 35's flat CPU debit for the disk was too pessimistic and
rescored the window at 53/120 with max(CPU, bus). Wrong. A 68000 has no cache
and a two-word prefetch queue, so it stalls the moment another master takes
the bus, and the MC68450 hands the bus over in SLABS under limited-rate
auto-request rather than interleaving per operand. DMA is additive. 84/120
stands and 14_dmac_chain.py reproduces it exactly. What 86.7% occupancy really
says is that there is almost no room to overlap anything. FINDINGS 38.3.
- The first DMAC costing was derived where a primary source existed. Both wrong
answers were confident and both were caught by reading the manual.
Also landed:
- FINDINGS 5's 8 clocks/word for the SCSI DMA, STATUS's own "most load-bearing
unmeasured number", is now bracketed by the datasheet: 5 clk/word with the bus
held, ~12 if the DMAC arbitrates per word. 8 is a supported midpoint, and
which end applies is a player design decision worth 7 clocks a word on a
480 KB/s stream. FINDINGS 39.7.
- check.sh gains two gates: the C68K pixel-exact decode (seconds, no MAME) and
the bus-model self-check. Both skip cleanly without a px68k checkout.
- spanned blocks are now charged their mode-map dispatch, which FINDINGS 30.7
flagged as uncounted in 12_span_tradeoff.py.
- MAME timed runs must be budgeted by WALL CLOCK, not -seconds_to_run: this box
runs x68000 at ~0.033x realtime and two runs were killed by their own timeout.
That is why the all-RAW cell in 37.3 is empty. The C68K harness does the same
work in seconds because it emulates a CPU and not a machine.
Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
|
||
|
prosolis
|
7d365b3ff5 |
Drop SASI on capacity, then find the budget never had the disk in it
USER DECISION: drop the `sasi` profile. Not on bandwidth -- on capacity. A SASI volume is 40 MB, and the 22.8 min of unique scene footage on the source Blu-ray (streams 00000-00201, measured, not recalled) is 146 MiB at the LOWEST rate this codec makes -- more than the machine's whole 4-unit SASI space. `scsi` is the only profile now. FINDINGS 32. Then the user asked whether we were drawing the wrong conclusions about PIO vs DMA, and we were, more broadly than the question implied. Every CPU figure in FINDINGS 24-34 is scored against the full 833,333 cycles/frame with nothing subtracted for moving the bitstream off disk. Debiting the HD63450 cycle-steal at the long-standing 8 clk/word ESTIMATE, "1 frame of 120 misses" becomes 84 of 120, median 112.4%. PIO at the span rate is 99.8% of the machine. Spans buy cycles by spending bandwidth and the bandwidth returns as steal, so 31.6's "fits completely" becomes a worst frame of 114.3%. 10 fps absorbs it: median 93.7%, 1/120. FINDINGS 35. `11_cpu_budget.py` takes --io dma|pio|none, defaults to dma, and warns if asked for none. Also landed: - item 1 done: the cost model checked against the 68000 on a cost-aware container, -3.07% to +0.01%, whole-window mean -1.22%. FINDINGS 34. - item 4 done: the container carries its own 4-byte record alignment (DLX2). 94/120 record starts were on odd addresses -- an address error, not a slow read -- now 0/120 for 16 B/s. Re-encoding reproduces 31.1 exactly. FINDINGS 33. - a `scsi` window does not fit the 2 MB machine the rig emulates (2.84 MB of stream past a 0x200000 ceiling). The gate now verifies 80 of 120 frames and SAYS so, and fails loudly when the pass does not complete, instead of reporting a phantom 49,005-pixel diff. FINDINGS 36. Three near-misses this session had one shape: an unobservable run nearly produced a false finding. stdbuf -oL on any MAME job that prints progress -- a file is block-buffered too, and a run that is merely finishing looks exactly like one that is wedged. check.sh ALL GREEN. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
06b98d4b47 |
Price cycles in the mode decision: 37 misses become 1, for 0.26 dB
The decoder has been CPU-bound since FINDINGS 28 while the mode decision
minimised D + lam*R -- distortion against BYTES. decide() now minimises
D + lam*bytes + mu*cycles, and ratectl bisects mu per frame against the
833,333-cycle budget with the lam bisection nested inside it. On the worst
sustained window:
sasi 27.22 -> 26.95 dB, 109.5 -> 109.4 KB/s, 37/120 misses -> 1
scsi 29.90 -> 29.27 dB, 280.0 -> 278.6 KB/s, 51/120 misses -> 1
Bitrate does not move: the byte controller still binds, and mu changes WHICH
modes are bought. V4 is what it stops buying -- 25.2 -> 20.3% of blocks at sasi
and 15.0 -> 5.3% at scsi, where RAW takes it. That is 28.8's inversion in
practice: RAW is dearer in bytes and cheaper in cycles, so only the byte-rich
profile can buy its way out of V4.
Three things worth knowing beyond the headline:
- The one frame that still misses, at both profiles, is FRAME 0 -- no previous
reconstruction, so 100% changed by definition, which is also what a scene
cut is. It comes out at the all-V1 floor of 110.6% and is emitted late on
purpose. Freezing a cut to make a deadline is the worse failure.
- 28.7's "11 frames are impossible" was too pessimistic. That floor held the
SKIP set fixed and asked how cheaply the drawn blocks could be drawn; the
real decision can also MOVE a block to SKIP, which above ~90% non-SKIP is
the only lever left.
- SKIP's price depends on its neighbours (13.25 cycles clustered, 45 mixed),
which a per-block lagrangian cannot see. The way out is that the two uses
need not share a cost function: a ranking constant inside decide(), the
exact clustered rule for the frame-level bisection. vq_hybrid.cycles() is
now the one definition of that rule and 11_cpu_budget.py imports it.
Gated: 09_ratectl_drift.py runs both controllers, both 0/120 drifting frames.
The cost-aware container decodes pixel-exact on the 68000 (120 frames). ON by
default in encode.py; --no-cpu-fit restores session 7. check.sh ALL GREEN.
Still a model, not a measurement, for THIS container: FINDINGS 31's cycle
figures come from vq_hybrid.cycles (within 1 point of the 68000 on four frames
of the session-7 container). Timing this one on the machine is step 1 of the
next session -- it was started and killed for time, and it is slow.
FINDINGS 31. tools/analysis/13_cpu_ratectl.py.
Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
|
||
|
prosolis
|
29eb78a599 |
Measure the span: the mode survives, and it is an encoder format
FINDINGS 29 priced a literal-span mode at 4*(50 + 4L*9.08) cycles and labelled
the whole section DERIVED. Session 8 step 0 was to measure it before optimising
over the mode set it implies. Two variants in blit.s, one stream per span length
from prep_spans.py, timed by span.lua, driven by span.sh in ~25 s:
v5, handed (x, npix) and left to work the copy out: 97.9/span + 10.459/px
v6, handed an address and a jump displacement: 43.7/span + 9.152/px
29 assumed 50.0/span + 9.080/px
So 29's arithmetic was right about a format nobody had written. The difference
is not tuning: v5 spends ~122 cycles a span computing a destination, dividing
npix into bursts and handling a 0..15 remainder, all of which the encoder knows
at build time. v6's record is {u32 absolute GVRAM address, u16 jump
displacement} into an unrolled chain of 24-pixel copy units -- no loop, no
remainder, no arithmetic -- and it fits 11 span lengths to 0.3%.
Three things that measurement showed and derivation could not:
- The per-pixel cost is a function of REGISTER PRESSURE. FINDINGS 24's 9.08
was a fixed blit with 12 registers free; v5 can spare 8 and pays 10.46; v6
gets 12 back only because the encoder holds the state.
- Short spans die in the remainder path -- a 12-pixel span costs MORE than a
16-pixel one -- and the fix is padding, not avoidance.
- Odd-x alignment is free (259.0 vs 261.8 cycles/span), as a 16-bit bus
implies but nobody had checked.
Re-priced against the unchanged mode maps, sasi: median 74.4% -> 52.0% (29 said
43.0), misses 37 -> 10/120 (29 said 8), 448.0 KB/s. Break-even moved from runs
of 2 blocks to runs of 4. 29.4 survives: a scene cut needs x >= 0.196 of the
frame as spans and the bus allows x <= 0.373, so it fits at 12fps.
All 23 timing configs are also checked pixel-exact, so none of this was timed
against a decoder that quietly skipped work.
FINDINGS 30. Next: lever B, the cost-aware mode decision.
Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
|
||
|
prosolis
|
3641f37e28 |
The bus is 4x idle while the CPU is pinned: price the trade
The codec was designed when bytes were scarce, so every decision in it trades
cycles to save bytes. That is now backwards: sasi spends 110 KB/s of a 488 KB/s
pipe while missing 31% of frames on CPU.
The cheapest thing a 68000 can be handed is the most expensive thing to store.
Measured, per pixel: row-linear copy from word-expanded memory 9.08 cycles,
block-order 12.98, V1 codebook 18.74, RAW byte literals 25.03. So the 1024-byte
stride costs 43% and unpacking bytes to words costs more than the write itself.
Pricing one new mode -- a per-row span of word-expanded literals movem.l'd
straight from the stream buffer -- against the UNCHANGED mode maps:
sasi median 74.4% -> 43.0%, worst 136.2% -> 106.2%, misses 37 -> 8/120,
101.7 -> 453.2 KB/s
scsi median 94.9% -> 69.4%, misses 51 -> 18/120, 272 -> 479.7 KB/s
scsi gains less precisely because it has less idle bandwidth left to trade.
Two consequences worth flagging. A word-expanded literal block derives to ~240
cycles, cheaper than V1's measured 299.9 and pixel-exact -- so every codebook
mode is CPU-dominated by a literal, and the codebook is a byte optimisation
that now costs cycles. And 28.5's "a scene cut cannot fit at 12fps" reopens:
CPU needs >=19% of the frame as spans, the bus allows <=39%, and that interval
is not empty.
DERIVED, NOT MEASURED, and labelled as such everywhere. The 9.08 cycles/pixel
is real but was measured at full row width with 12-register bursts, so short
spans are flattered. Measuring one span on the 68000 is now step 0 of the next
session, ahead of the cost-aware mode decision, because it changes the mode set
that decision optimises over.
FINDINGS 29. tools/analysis/12_span_tradeoff.py.
Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
|
||
|
prosolis
|
31c4c1aba1 |
Separate the two axes: profiles are I/O, the CPU ceiling is one target for both
The profiles were chosen against disk bandwidth and say nothing about CPU. The locked CPU target is a stock 10MHz 68000 for both of them, so both must fit 833,333 cycles -- picking sasi does not rescue it, it still misses 31% of frames against scsi's 42%. Splits the miss into what the encoder can fix and what it cannot: re-coding every non-SKIP block as V1 is the floor, and it still misses 11 frames at sasi and 12 at scsi, all of them above ~90% non-SKIP. So a cost-aware mode decision can reach about three quarters of the misses; the rest need a structural answer, not a better encoder. Also: V4 is 448 cycles against RAW's 400, and RAW is pixel-exact. On the CPU axis V4 is strictly dominated and the byte lagrangian's mode preference inverts. Only the byte-rich profile can take that escape, so the cycle ceiling should cost sasi MORE quality than scsi despite costing it fewer cycles. FINDINGS 28.7/28.8. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
e1aa26bb57 |
The 68000 decoder draws pixel-exact frames, and does not fit
src/player/decode.s parses DLX1 and decodes straight into GVRAM. Verified pixel-exact over a 120-frame sequential run of the worst sustained window on the disc -- all four block modes, full temporal recursion, so the last frame is only right if all 120 were. In check.sh. It costs a mean of 81.7% of a 12fps frame budget, and 31% of frames exceed 100% (42% at scsi). CPU is now the binding constraint. FINDINGS 28. Three things that were believed and are not true: - The dual-display-path plan of FINDINGS 24.5/25.6 is incoherent. The compose path needs a RAM copy of the previous reconstruction; the direct path's selling point is that it keeps none. Mixing them shows stale pixels on 70 of 120 frames, worst frame 18.8% of the screen. Every coherent repair is dearer than not mixing, and 24.5's two figures were both copies with no decode in either, so there was never a crossover to find. One path ships, and the 96KB reference frame is gone. tools/analysis/10_pathmix_drift.py keeps the counterexample runnable; check.sh asserts it still reproduces. - The four block modes do not cost the same. V1 300, V4 448, RAW 400 cycles against the old model's flat 207.8. V4 is 25% of blocks and 50% of the cycles, and the mode decision charges it bytes it does not charge cycles for. tools/analysis/11_cpu_budget.py reproduces all four frames timed on the 68000 to within 1 point. Hand-derived timings agree to 0.5% on V1. - The container is big-endian but not aligned. Variable-length records laid end to end put frame 1's length field at an odd address, and move.l (a0)+ there is an address error: frame 0 decoded perfectly and then vectored into the IPL for 59 emulated seconds looking like a hang. Found by dumping PC, not by reading the source. Also: an all-V1 frame, the cheapest possible full redraw, is 110.5% of budget. No mode assignment fits a scene cut at 12fps. That one needs a decision, not a measurement. Next: charge cycles in the mode decision and bisect against 833,333 per frame, the way session 6 bisects lam against bytes -- but with no bucket, because a late frame cannot be banked. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
497f88b945 |
Rate control: rebuilt per-frame, wired in, and gated at zero drift
FINDINGS 26 stopped the session-5 rate controller before it shipped: it built a lam-ladder of independent whole-sequence encodes and picked frames off it, so SKIP blocks referenced reconstructions the decoder never saw -- 111 of 120 frames drifted. The fix is the structural one 26.1 said it had to be. vq_hybrid is now frame-drivable -- frame_ctx / decide / paint -- and encode() is a thin loop over it. Rate control drives the same three calls, bisects lam per frame under the leaky bucket, and feeds back the frame it actually emitted. The desync has no way to occur, and 09_ratectl_drift.py goes 111/120 -> 0/120. That test is now part of check.sh, which is ~2 min rather than ~40 s. Both overshoots on the worst sustained window are closed for under 1 dB, totals including audio: sasi 137.4 -> 109.5 KB/s (-0.60 dB), scsi 381.6 -> 280.0 KB/s (-0.91 dB). Zero frames hit the lam=800 cliff, so nothing was destroyed to get there. Rate control also makes the display path cheaper -- scsi's median drops 53.6% -> 47.1% -- because raising lam moves blocks to SKIP and V1. Two knobs measured rather than guessed. --rc-floor is worth 0.00 dB on that window and defaults to the profile lam, so rate control cannot regress content that already fits. --prefill defaults to 0 and is documented as a trap: it buys a permission to overshoot of exactly bucket/nframes, and on a 14-frame clip it disables the controller outright. FINDINGS 26.5 was wrong in both halves and 27.6 records it. _paint was not the bottleneck (14% of a frame, though vectorising it was still right at 17.1x) and the ladder was never "minutes" -- those were k-means in build(). What makes per-frame rate control affordable is that VQ.assign depends on neither lam nor prev, so it is cached one frame deep: a 12-step search over 120 frames costs 0.31 s against 49.1 s. Also caught: fixed-lam sasi was already 5% over target on 00020, the clip everyone called easy. Nothing noticed because the profile table quotes PSNR and not bitrate. check.sh: ALL GREEN. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
145753c0bf |
Handoff: rate control is next, and it is unsound as written
Session 5 handoff. The user has chosen rate control as the next session's work, so this reads ratectl.py properly before that session starts rather than discovering the problem mid-implementation. FINDINGS 26: encode_rate_controlled() is not sound. H.encode() is temporally recursive -- SKIP blocks copy the previous RECONSTRUCTION -- but rate control builds a ladder of independent whole-sequence encodes and picks each frame from whichever rung fits the budget. Frames then reference reconstructions the decoder never saw. Measured on the Singe window: 67 rung switches, 111 of 120 frames drift, worst frame 43.4% of pixels, reported PSNR overstated by 0.36 dB. It would have wired up cleanly and reported a plausible wrong answer. Two further defects in the same function: the lam ladder runs to 2e5, 250x past the FINDINGS 15 cliff, so a frame that only fits up there is destroyed rather than rate-controlled; and with 5 rungs only two are ever chosen, straddling the operating point by 7.5x. The docstring describes a per-frame binary search, which is the right design -- the implementation is a fixed ladder. The leaky bucket does work and should be kept: 109.1 KB/s against a 110 target. tools/analysis/09_ratectl_drift.py is the regression test and the acceptance criterion: it exits non-zero until zero frames drift. Also corrected the stale 38% blit figure in ratectl.py's profile commentary, which session 5 measured at 53.6% (FINDINGS 24), and recorded the pgrep -f self-kill trap again -- four times across three sessions now. check.sh ALL GREEN. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
e00264a058 |
Find the sustained action sequence: it breaks both profiles
The open risk since session 2 was "a sustained action sequence could still break the bitrate", with every clip measured so far being 1.2-1.7 s. Closed by measurement rather than by sampling clips by hand. 07_motion_survey.py scans a whole stream at 96x72 for the hottest sliding window of inter-frame difference. On 00223 the spread between the quietest and hottest sustained 10 s windows is 10.6x, which is the argument for not eyeballing it. Hottest is t=539.4s, the Singe endgame. There, with the fixed lam the CLI uses, sasi overshoots 110 -> 129.6 KB/s (+18%) and scsi 280 -> 373.8 KB/s (+34%). Rate control moves from "insurance, not a fix" to required, and is promoted above the full-disc survey. The bus is not broken -- 381.6 KB/s still fits the 488 KB/s figure -- so FINDINGS 21 survives, at 78% of the pipe instead of a comfortable margin. Three further corrections fall out: - The two largest streams on the disc are bonus material. 00216 is the feature with a burned-in commentary PiP; 00215 is the commentary. 00223 is the clean 9.4 min. A size-ranked survey would have encoded live action. - On hard content the 256-colour scene palette (31.33 dB) binds well before the X68000 display (40.81 dB); scsi is already within 0.51 dB of it. - FINDINGS 24.5's architecture question resolves to "both paths, chosen per frame": 30-53% of frames sit above the 70% crossover. Picking per frame costs a median 37.0% of the frame budget and caps at 53.6%. Reporting for this is wired into encode.py, which previously only printed a mean over all frames -- the one statistic that cannot answer a per-frame question. extract.py takes optional start/dur; 08_mode_map.py renders source | decoded | block-mode map to .webm. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
09a5a50065 |
Measure the blit on the 68000: the 38% estimate was 53.6%
First 68000 instructions in this project to draw a pixel. Everything before this was GVRAM filled from Lua, which costs zero 68000 cycles, so the blit figure the whole CPU budget rests on had never been validated. Four variants of a full-frame 256x192 paint, timed in MAME and each also hand-derived from the MC68000 timing tables beforehand; the two agree to 0.006-0.43%, which is what makes the result trustworthy after this project's history of false-good measurements. V1 movem.l blit from a word-expanded RAM frame 446,286 cyc 53.6% V2 naive move.b/move.w per pixel 1,284,174 cyc 154.1% V3 write-only floor, no source read 225,789 cyc 27.1% V4 same writes in 4x4 block order 637,971 cyc 76.6% Scope: MAME's gvram_w/gvram_r carry no timing at all, so these are instruction cycles against zero-wait-state memory -- a floor, not a hardware prediction. V1's output snapshots pixel-exact through verify_frame256.py, closing FINDINGS 23.5. The V1/V3 gap shows reading the source frame is exactly half the cost, which makes the architecture question live: decode-direct-to-GVRAM needs no RAM reference frame and scales with the non-SKIP block fraction, crossing compose-then-blit at 70% of blocks changed. That fraction is now the top priority and is already a by-product of vq_hybrid.py's mode decision. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
34f9ee341d |
A real 256x256 CRTC mode, derived not recalled; palette ceiling was 2 dB low
Session 3 left the harness on the IPL's 768x512 text timing because no CRTC values had been derived and guessing them was the failure mode to avoid. This derives them from MAME 0.277's divisor ladder instead, and the derivation is self-checking: the 256-wide mode runs at div 6 against the 768 mode's div 2, so htotal is exactly 1104/3 = 368 dots and every horizontal register divides by three with no remainder. Only the blanking split rounds. Verified by snapshot: native 256x512, active area pixel-exact, x=512 wrap gone. Two things fell out that change numbers elsewhere: - The palette's shared LSB I must be chosen per entry, not hardcoded to 1. Doing so lifts the display ceiling from 38.85 to 40.81 dB and is the only way to reach true black at all, since pal6bit(1) = 4. 102 of 256 entries want I = 0, so this is not a corner case. Supersedes FINDINGS 22.4; scsi has ~2 dB more headroom than that section claimed. The encoder does not do this yet. - Letterboxing costs a palette entry: GVRAM cleared to zero shows entry 0, and a free mediancut palette puts a real image colour there. 255 colours plus a reserved black, via prep_frame.py --reserve-black. MAME's graphics double-scan is phase-shifted one raster line (it halves the absolute scanline and vbegin is odd), which produced a false failure before it was understood; the regression test now asserts the shifted pairing explicitly. Still Lua-side. No 68000 instruction has drawn a pixel; the 38% blit estimate remains unvalidated. What this buys is a defined geometry for the decoder to write into: 256 words per row, 1024-byte stride, rows 32..223. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
966417893b |
Scope the display-path result: no 68000 code drew that frame
The session-3 milestone was written in a way that reads as "the port renders", which it does not. The video hardware is genuinely emulated and the output is bit-exact, but GVRAM was filled by a MAME Lua script poking emulated memory, not by 68000 instructions. The distinction is load-bearing: Lua writes cost zero 68000 cycles, so nothing here tests whether the CPU can decode and blit inside 833,333 cycles. The 38% full-frame blit estimate that the entire budget rests on is still unvalidated. Only the "Not yet started" list carried this caveat, which was too buried for a claim this easy to over-read. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
b322e84cd4 |
Get a real Dragon's Lair frame onto the emulated X68000
First pixels on an actual X68000 screen. Everything up to now was Python-side or a headless -video none run, which cannot snapshot at all. The blocker was not the video controller. The IPL leaves CRTC R20 = 0x0B16, and bit 11 is "G-VRAM set to buffer", which makes MAME's draw_gfx() return early. GVRAM writes still land and read back correctly while the layer is invisible, so six attempts at $E82400/$E82500/$E82600 all rendered black with every register holding the value I intended. Two more facts, both confirmed against MAME 0.277 source rather than assumed: - $E8E001 monitor contrast is left at 14 by the IPL, scaling all output to 93.3%. The player must set it to 15. Contrast 0 blanks the screen, which is a free fade-to-black for scene transitions. - The palette word is GGGGGRRRRRBBBBBI with a shared LSB, expanded as pal6bit((field<<1)|I). With contrast at 15 the render is pixel-exact, not merely close, which also confirms the 1024-byte GVRAM line stride. That exactness gives a new quality ceiling: the 15-bit+I palette alone costs 38.88 dB against the 24-bit palettised source, the same order as the scsi profile's own codec error. scsi is close to display-transparent on hardware, which bounds how much further it is worth raising. Unblocks next step 2, the 68000 decoder skeleton, which now has a known-good reference image to diff against. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
64cd1ffd72 |
Handoff: reconcile docs and tooling with the corrections made this session
Session 2 reversed several of its own conclusions. The docs are append-only, so a reader could land on a superseded section and act on it. This pass makes the repo internally consistent. Defects found and fixed in STATUS.md: - claimed "Hybrid VQ with k=1024: no" as the answer to the linework question, directly contradicting FINDINGS 14, which rejected k=1024. Both profiles are k=256. - malformed profile table (six column separators, five columns). - next-steps list had two items numbered 3 and listed the full-disc survey twice. - the disk-benchmark section still read CRITICAL-PATH with "if SCSI sustains >=800 KB/s, ship pixel-exact". That was written while the bandwidth figure was misread as 4 MB/s. At 4 Mbps pixel-exact needs 92-97% of the pipe and is not available, and the ring-buffer result means the design no longer hangs on the benchmark at all. Rewritten with what it IS still worth doing: confirming the 4 Mbps provenance, and confirming DMA is used rather than PIO. FINDINGS now carries supersession blockquotes on 5, 8, 11, 17 and 18 pointing at the sections that correct them. 18 is the dangerous one -- its peak-vs- sustained test is reversed by 21 -- so it is marked DO NOT ACT ON THIS SECTION while noting the per-frame data itself remains valid. profile_gen.py had the same problem in code: it defaulted to the superseded peak sizing and returned lam=25 where the docs say lam=10. The buffered test is now the default and peak sizing is behind --size-for-peak as a bound only. A tool that contradicts the findings is worse than no tool. Also preserves the five measurement scripts that produced this session's numbers as tools/analysis/05-09, following the session 1 precedent, and adds an "explicitly abandoned -- do not re-propose" list to STATUS covering entropy coding, k=1024 codebooks and flat 4x4 VQ. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
fb8a1462b0 |
Correct the peak analysis: both profiles fit 4 Mbps with zero prefill
The previous commit warned that scsi "does not fit 4 Mbps" because a frame peaked at 96.4% of the sustained rate. That was the wrong test, and the user was right to push back on it. It compared instantaneous frame demand against a sustained rate as if they had to match frame-by-frame; the disk keeps filling during the frame, and a ring buffer absorbs any shortfall. Correct test is cumulative demand vs cumulative supply. Simulated on the real per-frame sizes at a constant 488 KB/s fill: required prefill = 0.0 KB for EVERY scene at both profiles Fill delivers 40.69 KB per frame time; only one measured frame (00146, 42.10 KB) exceeds it and the next frame recovers it. A 256 KB buffer -- 12.5% of RAM -- carries ~1 second of stall tolerance, orders of magnitude more than an SD-backed seek needs. scsi at lam=10 stands; the hardest sampled scene runs 313 KB/s mean, 64% of the pipe. Also carries through a consequence of SD-backed deployment that session 1 noted as a caveat but never applied: with BlueSCSI/SCSI2SD the sustained rate is a bus-limited CONSTANT, not an average over variable seek latency, so there is no long tail to leave margin for and we can size much closer to the ceiling. Conservative margins here were cargo-culted from a constraint this deployment does not have. The SASI/SCSI split remains correct because it is about bus protocol, not media. Rate control drops from load-bearing to insurance: intra-scene peaks are a non-problem, but only 4 clips of 1.2-1.7s out of 224 streams have been measured and 00146 already runs 23% hotter than 00020. The full-disc survey is now the highest-value measurement, ahead of wiring up ratectl. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
prosolis
|
48e912de8b |
Size against 4 Mbps: peaks break the scsi profile; DMA steal is not free
User clarified the bandwidth figure is 4 Mbps (488 KB/s), not 4 MB/s -- ~8x
tighter than the previous commit reasoned against. Two consequences, plus a
correction to session 1.
1. The scsi profile committed in
|
||
|
prosolis
|
f0f2f807a4 |
Raise both quality profiles; rule out entropy coding on CPU grounds
The profiles shipped in
|
||
|
prosolis
|
e4062ed294 |
Session 2: hybrid VQ codec, two quality profiles, three corrections
Answers session 1's critical-path question. Flat 4x4 VQ at k=256 was prototyped and REJECTED by eye: Dirk's face disintegrates and ink outlines break into 4-pixel stair-steps. The 256-colour palettised frame is excellent, so the palette was never the problem -- block VQ was. Replaced it with a Cinepak-style hybrid: each 4x4 block is SKIP, one 4x4 codeword, four 2x2 codewords, or RAW literal pixels, chosen per block by rate-distortion. The RAW escape makes lam=0 pixel-exact (measured 0.00 dB loss), so the quality knob spans lossless to heavily-compressed in one bitstream. Per the user's decision, ships TWO quality profiles from that one codec, one decoder and one bitstream -- only the rate knob differs: sasi 45 KB/s lam=300 34.8 dB stock 10MHz ACE/EXPERT scsi 75 KB/s lam=100 35.9 dB Super/XVI or CZ-6BS1 Three corrections to earlier numbers: 1. Session 1's "183 KB/s at 12fps" was a bad extrapolation. Halving the framerate does not halve the bitrate -- decimation roughly doubles the per-frame delta. Re-measured directly: 340 KB/s for session 1's own RLE, 247 KB/s for changed-spans+deflate. The lossless floor is 319 MB. 2. A FOURTH false-good result, same family as the three in FINDINGS 4: k=1024 codebooks appeared to buy +2.4 dB free, because the rate model charged 1 byte for a 10-bit index. Charging the true cost reverses the verdict -- k=256 wins at every matched bitrate, and by 5 dB at the low end where the SASI profile lives. k=256 ships. 3. Stream inventory: the ~3-5MB clips are 1.2-1.7s, not ~60s, and some 60s streams are menus, not content. Any survey must classify before averaging. Also cleared both candidate sources for the game-logic layer: the SNES project is MIT and DirkSimple is zlib, so the arcade scene graph can be imported and the two transcriptions diffed against each other. Encoder is working end-to-end: extract.py -> vq/vq_hybrid/ratectl -> encode.py, emitting a big-endian DLX1 container the 68000 can parse with plain moves. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |
||
|
reala-misaki
|
65112b9305 |
Session 1: hardware research, content measurement, codec decision, MAME harness
Verified GVRAM is one word-access per pixel in ALL color modes; chose 256-color 256x192 with movem.l bursts (page 1 sacrificed as double-buffer). Measured 8 scenes from the Blu-ray source: blit costs under 8% of the 12fps cycle budget, so I/O is the bottleneck, not CPU. Naive delta+RLE reaches only 3.2:1 (365 KB/s, 470MB) -> decision to use 4x4 vector quantization (~30 KB/s). "Shot on twos" assumption failed: the transfer has zero duplicate frames, so 12fps requires explicit decimation. Documents three false measurement results and their root causes (per-frame Floyd-Steinberg dithering, temporal denoise, exact-match dedupe on noisy source). MAME Lua injection harness works and is reusable for cycle-cost measurement; the IOCS _B_READ disk benchmark is blocked returning -1. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6 |