drwily/Dragon-s-Lair-X68k
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: drwily/Dragon-s-Lair-X68k:31c4c1aba12e46dc68a15048b4b87f7272c70c64
Branches Tags
drwily/Dragon-s-Lair-X68k:main
...
compare: drwily/Dragon-s-Lair-X68k:main
Branches Tags
drwily/Dragon-s-Lair-X68k:main
32 Commits
31c4c1aba1 ... main
Author SHA1 Message Date
prosolis 91e2e9803d Take the project's own story out of the README, starting with a claim about elapsed time 
USER CORRECTION: the README is a description, not a chronicle, and the intro
had it narrating an arc -- "for most of this project's life that was read as a
video codec problem" -- which is also a claim about elapsed time that nothing
measured supports. The tree is days old.

The intro now states the design: it is a delivery problem, the live design has
no decoder in it, and the comparison that settles it is a measurement (37,585
decoded costs more than 49,152 not decoded; 110.4% against 54.9% at the 9 clk/B
floor). The codec is "in the tree and parked (USER DECISION)" -- no session
number attached, because which sitting it happened in is STATUS's business.

Four more of the same, found by grepping rather than by eye:
  - B2 "is not a headroom question any more" -> it decides which player exists
    rather than how much headroom one has.
  - "Other things the machine has said" -> "What else a player has to carry".
  - ring.i/xfer.i/stream.s "left the video path when a DMAC-direct player
    deleted the ring" -> a DMAC-direct player has no use for them, because it
    has no ring. Same edit in the parked list and in Layout.
  - "Why it is parked" kept the standing reason and dropped the retelling.

And one where the story was carrying a fact that had to stay: the sasi profile
was "dropped on capacity rather than bandwidth" with the numbers left in
FINDINGS. They are in the sentence now -- a 40 MB volume against 146 MiB of
game -- so the clause survives without the reader needing the episode.

One "session" reference stays, at the STATUS.md bullet: that file IS ordered
newest session first, and saying so is navigation for another document rather
than this one telling a story.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
 2026-08-25 16:59:04 -07:00
prosolis 3b7f2af27e Lead the README with the player that has no decoder, and give it a clip with sound 
The README still opened by calling this a video codec problem and led with the
codec player's stills and clips, which have been PARKED since session 29
(FINDINGS 61.8). Everything measured since says the opposite: at the 9 clk/B
dual-address floor the codec is 110.4% of a 12 fps frame and the decoder-free
packed player is 54.9%, so decoding 37,585 bytes costs more than not decoding
49,152. The document now says that in its first paragraph.

871 -> 473 lines. "Where it stands" was an append-only pile of session
headlines -- which is what docs/STATUS.md is for -- and is now a STATE: what
runs end to end, the four resources every item is priced in, the one open
question that decides which player gets built (A vs B, blocked on B2), and the
machine facts a player has to carry. The codec keeps a section as the parked
branch, with its media and the reason it is parked.

Drift struck while restructuring:
  - "Current encode: 496.7 KB/s at 29.19 dB" was session 14's candidate, not
    the DLX5 gate container, and was presented as the project's current state.
  - The Encoder section documented only the parked encoder. It now leads with
    pack.py/dlxp.py/adpcm.py and uses the gate window the tree actually
    encodes (00223, 539.4 s, 10.0 s) rather than 00020.
  - Analysis scripts 25, 30 and 34 were described under tools/bench/gvpack/
    and tools/import/. They are tools/analysis/.
  - make_packed_media.py was never mentioned.
  - check.sh was "~6 min" in prose and "~3 min" in the code block, and its own
    header says ~2 min. No figure is quoted now rather than a fourth one.

AND THE HERO IS A SEQUENCE NOW, BECAUSE A PACKED FRAME IS A LITERAL. A still
cannot show the property that separates this branch from the codec's: there is
no recursion here, so frame 119 says nothing about frame 60 and 120 frames are
120 separate claims. make_packed_media.py grew a clip path that gates all of
them before writing any -- and refuses to write if one differs, because a clip
of the player is a claim that the player drew it.

The clip is a COMPOSITE of two runs of packed_run.sh and it says so on the
page: the picture is run 1, paced at half rate so the snapshot lands inside
the write window; the sound is run 5, MAME's own -wavwrite capture off the
speaker at 12 fps. They cannot be one run -- audio cut at 12 fps played at 6
is not this scene. What it is NOT is a real-time capture of the shipping
configuration, which at this container's own burst rate would show a blank
layer for 99.5% of every slot (FINDINGS 64.2).

The audio cut is GATED, not eyeballed. The capture opens with the machine
booting, so the stream starts at the first non-zero sample -- exactly the kind
of alignment that is off by one byte forever with every counter agreeing
(71.5). Lump 0 is decoded with the four ADPCM axes out of the container's own
header and required to be sample-exact from the cut: 50,603 samples in, 3.24 s
of boot dropped, terms/low, 10-bit, init -2.

check.sh is ALL GREEN with this in the tree (tmp/check_s40.log, exit 0, 101 OK
lines). It does not gate the README or the media builder, so that run confirms
the tree the README describes and not the description.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
 2026-08-25 15:48:00 -07:00
prosolis ba966efe7e Take the player through a branch with sound, and find the predictor does not seek 
FINDINGS 71, ROADMAP P6d.  70.3 named exactly what was missing -- packed.s starts
PG_AK/PG_AKF at lump 0 and has no audio seek path -- and priced its absence at a
mean 416.5 ms of silence over the arcade's 409 within-container seek targets.
pg_aseek is that path: the lump index, the stream position, the remainder
accumulator and the byte offset into the group, then the second READ(10) at the
lump's own LBA and a re-arm part way into the buffer.  Measured off a real volume:
132,162 B of spliced stream accounted for byte by byte in MAME's own capture,
across a branch at frame 37 -- four frames into lump 3, deliberately NOT on a
group boundary -- in both chip configurations.  Skip computed 2,604 B, cadence
says 2,604.

THE PREDICTOR DOES NOT SEEK, AND THE ERROR IS DC.  The MSM6258's accumulator is a
pure integrator with no leakage term, so a branch that hands the chip bytes chosen
for a state it is not in produces an offset that does not decay.  Playing through:
DC -355 of 511 with AC 0.00 -- the right shape from the wrong ground -- still -108
four seconds later.  STOP and re-PLAY: all 62,500 post-seek samples are EXACTLY a
decode from the container's own init, and the whole error is the single constant
-65.  A re-PLAY is 5.5x better and neither is zero, so PG_ARST is a mailbox with a
number under it.  The host computes -65 out of the container's bytes and the gate
asserts the equality rather than printing both.

AND THE ONLY FIX THAT REACHES ZERO IS THE ENCODER'S.  A player cannot set the
chip's accumulator, only reset it.  Resetting the encoder's predictor every frame
makes all 119 of the container's branch points exact for 0.33 dB (21.99 -> 21.66),
because the step table's floor is a constant 16.  That is a DLXP3 and it is
deliberately not in tools/encoder.

TWO SILENT BUGS, BOTH CAUGHT BY THE CAPTURE.  pg_udiv32 trashes d4 and pg_aseek
held hz there, so the offset came out 1 byte instead of 2,604 -- 166 ms of the
wrong part of the scene at exactly the right rate, every counter agreeing.  And
one already in the tree that had passed this gate three times: pg_ainit waited on
a READ-BACK MTC before PLAY, which is the same test as "a byte has left RAM" only
if no byte leaves in between.  One does, and the chip then plays the scene one
byte in, forever.  Found by locating the capture's opening samples in the
container image: sector 1 + 1.  The witness is now the count that was written.

ALL GREEN, two new stages included.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
 2026-08-25 15:10:20 -07:00
prosolis 8b5f51704c Take the climb to a real branch point, and find the cadence has a third column 
FINDINGS 70, and it closes ROADMAP P6.  65.6 recorded that "the slack table is
here, but 51.3's refill climb with a second consumer through a real branch point
is not", and 67.6 repeated it verbatim two sessions later.
tools/analysis/36_branch_audio.py is that run: 51.3's surplus model, 56.3's 612
real branch points, 32's cadence arithmetic, both containers' headers and the
buffer constants of the player 68 measured, in one place.  No machine -- all
five were already numbers.

AUDIO IS 1.7% OF THE CODEC WIRE AND UP TO 3.30x OF THE CLIMB.  Slack accrues out
of pipe - wire, which is a small difference of two large numbers, so a second
consumer is not priced by its share of the wire -- which is how every budget in
this tree since FINDINGS 8 has quoted it.  At 451.4 KB/s (the container's own
zero-prefill pipe, 49.5) in a 256 KB ring the climb goes 6.70 s -> 22.11 s and
the branch points arriving under it go 482/612 (79%) -> 603/612 (99%).  At 488
it is 1.19x, at 600 it is 1.05x.  Correction to 56.4: it charged audio at
ratectl.AUDIO_KBPS = 7.8, which is 7,812.5 B/s in decimal kB; in binary KB the
figure is 7.6294, so the placeholder was 2.24% high and every column moved in
the flattering direction by less than one part in six hundred of the wire.

AND ON THE PACKED BRANCH THERE IS NO CLIMB AT ALL.  A record goes straight into
GVRAM, so the video lookahead is zero records and 56.4's alarming column is
vacuous when every frame arrives with less lookahead than the one before it.
Acceptance there is a per-frame deadline, not an average, which is why B1 has to
name a burst rate.  The only consumer on that branch with any lookahead is the
audio one: 1.833 s of sound held against 0.000 s of picture.  And at 589.6 KB/s
-- the acceptance figure this project quotes -- the sounded container's surplus
is exactly zero: it plays and never banks a byte, while the silent one climbs
its record in 6.38 s.

THE COST NOBODY HAD COUNTED.  A DLXP2 group is `lump k, then F records`, so lump
k is at a LOWER address than every record of its group but the first, and a seek
to record i finds its audio behind it.  Measured on the 409 within-container
seek targets of the arcade's own graph at the shipped F=11: mean 416.5 ms of
silence entering a branch, median 416.7, p90 750.0, worst 833.3, and only 36 of
409 land on a group boundary.  The other 203 -- the scene changes -- are free by
construction, because lump 0 sits between the header and record 0.  The
container's own start is the one branch point the cadence costs nothing at, and
it was the only one anybody had looked at.

SO THE CADENCE PICK IS REOPENED, on a third column.  32_audio_wire chose F=11 on
padding and RAM; F=1, the cadence it called the worst one, has no group to enter
off-boundary at all, and gives back 12,288 B of RAM for +4.36 KB/s of wire
(0.74% of the acceptance figure).  The alternative is a second read: lump k is
up to 503,808 B behind, so it is a separate command of 7,168 B -- 11.7 ms
against 416, 36x cheaper in time -- and src/player/packed.s starts PG_AK/PG_AKF
at lump 0 and has no audio seek path at all.  Nothing here decides it; the
deciding number is what one more SCSI command costs, and that is B1's.  The
content is not uniform mod F either: the measured mean runs 0.67x..1.23x of a
uniform assumption across the sweep, and the 1.00x at F=11 is a coincidence.

New check.sh stage, ~10 s, and it gates the STRUCTURE and deliberately not the
milliseconds: audio never shortens a climb, the shipped cadence strands most
within-container branches, F=1 strands none, and the lump read is an order of
magnitude cheaper than the silence it removes.

The one thing 70.3 and 70.4 rest on is a DESIGN ASSUMPTION and is printed in the
tool's own output: one container per SCENE (53, 55.1, 56.3).  One container per
sequence makes every seek land on frame 0 and empties both sections.

No encoder, player or container byte moved.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
 2026-08-25 14:21:32 -07:00
prosolis ab15c0749a Measure the level off the whole disc, and find the headroom is not worth buying 
FINDINGS 69, ROADMAP P6 -- the item 66.3 reopened in session 34 and sessions 35
and 36 both deferred.  The chip clamps its accumulator at 10 bits INSIDE the
recursion, and the ten seconds every audio figure in this tree is quoted on peak
at 435 of 511: it fits, and it fits by accident, because that window is a
-13.4 dBFS passage.  Nothing knew what the loudest passage of the game was.

tools/analysis/35_audio_level.py reads every stream of the unique scene footage
(00000-00201) through extract_audio.py's own chain -- 1,291.6 s, 201 of 202
streams -- and encodes windows of it with adpcm.CHIP.  The disc peaks at 946 of
2048 = -6.71 dBFS (00200 @ 2.11 s), which is 5.35 dB over the clamp, and the
census behind that peak is 687 samples of 20,182,000 (0.0034%) in 402 events,
44.0 ms, longest 0.90 ms.

THE HEADLINE IS A NEGATIVE: THE LEVEL DOES NOT CHANGE.  Forty 2 s windows drawn
over the game's timeline at six gains -- the disc's own level (gain 1.0) has the
best mean SNR (22.03 dB) and the best median, and loses the worst-window column
to -3 dB by 0.04 dB.  The gain that guarantees zero clamping disc-wide (0.5402)
costs 0.85 dB of mean SNR across the whole game to buy back 1.90 dB on the
2.11 s that clamp, because the OKI step table's floor is a constant 16 and does
not scale with the signal.

AND 66.3's MECHANISM DOES NOT SURVIVE A CONTROL.  Error after a clamp run is
elevated ~5x -- and so is the same window at a gain that never clamps, read at
the same indices, because those samples are simply loud.  Worst ratio 1.28 over
64 offsets, and the clamped encode's whole-window mean |error| is the LOWER of
the two (4.71 vs 5.05).  adpcm.encode runs the chip's clamp inside its own
sixteen-way search, so it never loses the chip's state.  The worry was right
about the mechanism and aimed one layer too late: an encoder clamping at 12 bits
while the chip clamps at 10 is exactly that divergence, and 66 closed it.

pack.py gains --audio-gain (default 1.0) so the level is a named parameter with
a measurement behind it instead of a shift buried in a list comprehension, and
prints the encoded window's peak against the clamp.  tmp/packed_singe.dlxp
rebuilds byte-identical, all 6,039,040 B.  New check.sh stage, ~18 s.

Three rig facts in 69.4, because a shipping encoder meets all three: 00176 has
no audio track at all; 00199 is 61.31 s of video with 1.25 s of audio; and 18
stream pairs share duration, peak and RMS, 7 of them byte-identical.

The 10-bit clamp is a DRIVER SETTING, not a chip constant -- x68k.cpp:1089 sets
OUTPUT_10BITS -- so it is MAME's reading of the board, and hardware item 5 is
what settles it.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
 2026-08-25 14:20:55 -07:00
prosolis 191f2b47bb Put the container on the chip, and find the held bus costs 463 times the seam 
ROADMAP P6c, FINDINGS 68. 78,125 B of a DLXP2's audio out of channel 3,
sample-exact, while the video channel fetches records off the same disc.

The two pieces 67.6 said were missing: the lump buffer (pg_afill/pg_afetch,
three slots and the minimum is unmeasured) and 67.2's remainder accumulator
(pg_apay). The capture prices what the accumulator avoided at 1.26 s of
lip-sync over the game, against 67.2's predicted 1.25.

The finding is the third piece, which nothing had named: the MSM6258 has no
starvation state, so the gap between a channel counting out and the next arm
is a held nibble pair driving the predictor. Stealing, the seam is 0.51 ms
over ten seconds because dma.i's new DM_HOOK services the chip from inside
the transfer wait -- 250,000 of 250,240 looks. Held, the 68000 is halted and
gets 369: every one of the ten lump boundaries has a seam, worst 72.8 ms,
2.31% of the audio. Identical bytes, different sound. 64.3 reaching the audio.

Two bugs, and no counter in the player could see either. Clearing DM_BARV does
not unchain a channel -- OCR bits 3-2 are what it obeys -- and the symptom is
POLL TIMEOUT on the lump and every record after it. And the refill ran one lump
ahead of its ring and overwrote the buffer the channel was reading: 11 of 11
armed, 11 fetched, no starve, and the sound wrong from 0.2 s in. Which is why
the gate is a WAV: verify_packed_audio.py walks the stream one delivered byte
at a time, because MAME's okim6258 resets the nibble select on every write and
a byte is two nibbles only 99.994% of the time.

check.sh ALL GREEN before (tmp/check_s36_start.log) and after
(tmp/check_s36_end.log), with the new stage.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
 2026-08-25 13:11:37 -07:00
prosolis e3778f62b0 Put sound in the packed container, and find the padding is a rate error 
ROADMAP P6b, FINDINGS 67.  DLXP2: a 64-byte header and then groups -- one
audio lump of A sectors, then F records -- so record i is at
off_frm + i*rec + (i//F)*A*512 and lump k at off_aud + k*(F*rec + A*512).
Still no index and still none needed, which is the packed branch's whole
claim surviving the one change that could have ended it.  The player carries
the third term in six instructions once a frame and zero parsing, and 120 of
120 records are still pixel-exact off a real MB89352 volume with the
interleave in, against a silent control that says no picture byte moved.

The finding is what 65.3 called padding.  A lump is 7,168 B of SPACE; eleven
frames of audio is 7,161.4583... B, so the payload alternates 7,161 and
7,162 and the rest is zero.  A player that fed the chip the whole lump --
which is what "14 sectors every 11 frames" invites -- runs 0.09% fast, and
that is not waste, it is drift: 0.84 ms a group, 1.25 s of lip-sync over the
game's 22.8 minutes.  What a player carries is one accumulator,
acc += 11*15625; n = acc//24; acc %= 24, which is clock.i's shape for
clock.i's reason and the third time this tree has met the pattern.

The four ADPCM axes ride in the header as fields rather than a version
number, and the gate flips each one to prove they earn it: nibble order
-31.99 dB, delta formula -24.86, clamp 0.00, accumulator -0.49.  Nothing
parses a packed container, so the gate partitions the whole file -- 131
spans, no overlap, no gap -- and asserts what a cadence-blind player would
read: exactly records 11..119 wrong, and frames 0..10 identical either way,
which is how an off-by-one like that survives a rig that checks frame 0.

Wire 582.0 + 7.64 = 589.6 KB/s, 65.3's prediction to the tenth.

Green light ALL GREEN before (tmp/check_s35_start.log) and after
(tmp/check_s35_end.log), with the new stage in it.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
 2026-08-25 11:22:38 -07:00
prosolis 6dd3fb3597 Ask the chip which decoder it is, and find four wrong axes where one was expected 
ROADMAP P6a, on the machine. 68000 code programs HD63450 channel 3 with the
IPL ROM's own ADPCM bytes -- dual address, 8-bit port, cycle steal, external
request -- and feeds the MSM6258 a designed 1,678-nibble stream at the chip's
own pace: 839 B in 0.1074 s = 7,811.4 B/s against the format's 7,812.5, CER=$00.
That transport is P6b's, not scaffolding.

Sixteen candidate decoder models, three capture decimations and a searched
prologue are fitted to MAME's capture. Exactly one reproduces it sample-exact
over all 1,678 samples, and every axis carries a negative control: flip it
alone and the closest survivor disagrees on 826, 1,504, 156 and 1,522 samples.

The chip runs 'terms', takes the LOW nibble of a byte first, clamps the
accumulator at 10 bits and starts it at -2. tools/encoder/adpcm.py defaulted to
the opposite of all four, and 65.2 named the wrong axis as the risk: the delta
formula is worth -2.88 dB and the NIBBLE ORDER is worth -25.74 dB. 65.1's "high
first, measured" was a measurement of ffmpeg, i.e. of the VOX file convention,
which is a different question from what a chip does with a byte in its data
register.

The 10-bit clamp is free on the Singe window and only because that window peaks
at 435 of 511 -- 1.4 dB of headroom on a -13.4 dBFS passage, 12.1 dB below where
the encoder was clamping, and inside the recursion. So the audio level is an
open choice again, downward, and the loudest passage on the disc is unmeasured.

Session 33's silence had two ordinary causes: the PPI's port C is an input until
control word $92 says otherwise, and $01 is COMMAND_STOP. And a rig fact worth
the space: the 8 MHz ADPCM clock is CT1 in the YM2151's $1B, delivered on the
sound system's schedule rather than at the store, so a transfer started in the
same breath as the setup plays its first ~17 ms at the old clock and no model
fits a stream that changed rate part way through.

Name the layer: this is MAME 0.277's okim6258 device model measured end to end
through the machine's real transport. It settles the rig and not the silicon.
Also struck: 64.4's "no MAME source tree is on this machine" -- there is none on
disk, but the machine has network and the upstream tag fetches.

check.sh ALL GREEN before (tmp/check_s34_start.log) and after
(tmp/check_s34_end.log), with one new stage.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
 2026-08-25 09:50:03 -07:00
prosolis f925a1dd9a Put sound on the wire, and find three LSBs are worth 25 dB 
ROADMAP P6, everything in the item except the bus half session 20 closed.
tools/encoder/adpcm.py is an MSM6258 codec, tools/encoder/extract_audio.py
takes the same seconds of the same stream the frames come from,
tools/bench/verify_adpcm.py is the gate, tools/analysis/32_audio_wire.py the
container arithmetic.

There is no reference encoder -- ffmpeg has a decoder for this format and none
the other way -- so what is gated is the decoder the encoder runs INSIDE its
own nibble search, sample-exact against ffmpeg's over 4,268 nibbles. An
encoder that agrees with its own wrong decoder is what that catches. The Singe
window: 156,250 samples -> 78,125 B at 21.97 dB, which is 7,812.5 B/s to the
byte. Normalising the disc's -13.4 dBFS level moves the SNR 21.97 -> 21.97, so
the level is not a lever.

And the two published delta formulas are not the same codec. They differ by at
most 3 in 12-bit units; encode for one and decode on the other and the SNR
goes 21.97 -> -2.88 dB, the noise louder than the signal, because ADPCM is
recursive the way the video codec is temporally recursive. Which one the chip
runs is now P6a and it is a precondition on shipping any audio.

And audio is the first thing the packed branch's simplification has cost
anything for. A record has no index BY DESIGN, so audio cannot be per-record
without making records variable; it rides a fixed cadence (F, A), the obvious
F=1 wastes 57.3% of every audio sector, and the pick is F=11 A=14 -- 0.09%
padding, 14,336 B held, wire 582.0 -> 589.6 KB/s. The codec container, which
kept its index, pays zero.

The MAME experiment did not work and 65.5 says so: :okim6258 is there at
$E92001/$E92003, read out of the machine's own program map, and feeding it
from Lua recorded silence across control 0..3 x port C 0..15. The register
semantics were not guessed at further.

FINDINGS 65. check.sh ALL GREEN before and after, with a new stage.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
 2026-08-25 09:11:03 -07:00
prosolis 6f698ca226 Put the player on a real volume, and find the write window is the frame 
ROADMAP K3. src/player/packed.s (2,898 B) brings up its own display, builds
its own 193-entry DMA chain, keeps its own frame clock off V-DISP and fetches
every record itself with READ(10) off a CZ-6BS1. The rig writes no picture
byte, no palette entry and no CRTC register.

120 of 120 frames pixel-exact, every one compared, in both palette orders --
the gate had to grow to do it, because a packed frame is a LITERAL and the
codec's recursion was what made one comparison audit 120.

And the write window turns out to be the frame. A packed write requires R20
bit 11, buffer mode blanks the layer, and a DMAC-direct player holds the
window open for the whole data phase, so

    dark fraction of a slot = record bytes / (DATA-PHASE rate x slot)

which is 1.0 at the container's own 582.0 KB/s: every frame delivered, on
time, pixel-exact, and none of them displayed. The rate in that expression is
the BURST rate, a third hardware number B1 has no test for. It reverses 61.5's
ranking -- a packed player that DMAs to RAM and paints with the measured 27.3%
blit is on screen 72.7% of every slot at any rate, and the two are equal only
at 2,131 KB/s = 3.7x the wire.

And a held channel costs the frame clock half its ticks without the clock
being able to tell: 487 of 1,038 V-DISP edges lost, zero late frames reported,
the player believing 12 fps while the screen ran at 6.37.

FINDINGS 64. ROADMAP K4 opened and fenced behind B2.
check.sh ALL GREEN before and after.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
 2026-08-25 09:10:48 -07:00
prosolis f1007a0dbc Put the frame in a container with no decoder, and find the palette is not free 
ROADMAP K2. DLXP1: a 49,664 B record that is 97 sectors exactly, no index and
no length word, because a packed record's length is geometry rather than
content. 582.0 KB/s, which is what FINDINGS 61.9 predicted to the tenth, and it
encodes in 3.3 s because there is no k-means in it.

px68k's own x68k/gvram.c renders the container's bytes index-exact with the
harness computing no interleave -- the only test that can catch an encoder whose
byte order is wrong, since a container round-trips against its own inverse
either way. Both negative controls fail as they must.

The picture is re-derived against this project's builder rather than PIL's
(34.05 dB against 61.9's 34.08) and the GGGGGRRRRRBBBBBI word is charged for the
first time in this tree: 0.53 dB, on every row, so it moves no comparison.

What the control found is the finding. A packed container on a SCENE palette
lands exactly on the codec's ceiling, so the whole +2.31 dB is the per-frame
palette and nothing else -- and 231 of 256 entries change every frame, which
makes a mismatched paint 12.8 dB worse than the correct pairing, on screen for
roughly half of every frame slot if buffer mode does not blank. So B2 now
decides which packed CONTAINER ships, not only which player. The fallback is
already a flag: --scene-palette --no-palette is 30.79 dB, zero churn, 576.0 KB/s
and still +2.07 dB on the shipping codec.

62.5 is priced and is a wash: palette first 20.32 dB, palette last 20.33.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
 2026-08-25 07:31:05 -07:00
prosolis 07f36c2af9 Put the palette on the channel, and find one start paints a whole frame 
ROADMAP K1, the packed player's one open structural item.  A frame is a picture
AND a palette, and no run in this tree had pointed a DMA channel at the palette
registers.  dmagate.s runs 7-9, gated by dma_run.sh and check.sh:

  7. 512 B off the disc into $E82000, bus held -- byte-exact in 256 register
     words, read back OUT OF the registers by the 68000;
  8. the SAME transfer aimed at RAM -- byte-exact at $2C000, and 256 of 256
     palette words still read the poison the CPU wrote, which is what attributes
     run 7 to the channel's MAR rather than to the readback path;
  9. ONE array-chained start across two kinds of destination -- the palette and
     six picture rows at the 1,024 B line stride, 2,048 B byte-exact.

So a packed frame is one channel start: a 193-entry array, palette first, CPU
halted from the first byte to the last.  The array is scene-constant, because
the packed layout spends both 256-colour pages and there is no page to flip.
What is left on the CPU per frame in the video path is the channel start and the
READ(10) -- no per-frame PAINT, which is not the same claim as no per-frame CPU.

The destination is POISONED first (62.1).  Runs 4-6 wrote into RAM that was zero
and GVRAM that was stale against a record that is mostly pad; "it matches the
disc" was weaker than it read as.  The host counts whether the poison actually
discriminates instead of assuming it: 511 of 512, and the gate refuses under 500.

And it opened a hardware item (62.4, ROADMAP B4).  MAME maps the palette to
palette_device over memory_array, whose write16 is a plain COMBINE_DATA -- RAM
that honours mem_mask, with no handler that could refuse a byte write.  Unlike
GVRAM's 256-colour arm there is nothing here to be wrong about, so the run
bounds the model and not the board.  What a real palette register does with a
byte write is unmeasured.  A negative costs 0.28% of a frame and nothing else.

29_packed_player.py now also prints the two rows with the per-frame palette
charged -- 55.7% of a frame on the chain, 582 KB/s -- alongside the picture-only
figures the codec comparison is quoted against.

check.sh ALL GREEN before (tmp/check_s30_start.log) and after
(tmp/check_s30_end.log).

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
 2026-08-25 07:06:44 -07:00
prosolis 1be428c270 Align the container to the disc, and find the decoder-free packed player fits 
Two sessions, unrecorded until now, committed together because their edits
share files and cannot be split cleanly after the fact.

Session 28 (FINDINGS 60): the container is DLX5 -- every record sector-aligned,
120/120 starting on a boundary where 3/120 did, +0.48% on the wire and zero
clocks -- and the ring's release rounds to RECALN so no pad is stranded.  Two
encoder levers measured and refused: `--spans all` buys +0.19 dB for +67% of
the wire, and joint span/lam selection emits byte-identical containers because
`lam` never leaves its floor on any of 120 frames.

Session 29 (FINDINGS 61): the packed full-frame blit is 27.3% of a 12 fps
frame, a channel fills GVRAM in buffer mode off the disc with the CPU halted,
and it walks the 1,024 B line stride itself through array chaining.  At the
9 clk/B dual-address floor the codec is 110.4% of a frame and a decoder-free
packed literal player is 55.2%, at +4.89 dB -- 2.75 dB past a ceiling the
codec's scene-wide palette cannot cross.  Encoder work is parked; the codec is
kept and not built on.

check.sh is ALL GREEN before and after, plus one new stage that gates the ORDER
of the measured paint costs rather than their values.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
 2026-08-25 06:54:27 -07:00
prosolis 8800d8f8c0 Price the transport against the frame, and find dual address cannot fit 
FINDINGS 59.7. tools/analysis/15_bus_occupancy.py has always answered "what does
each W cost" and never "what can the frame afford", and after 59.2 those are not
the same question. It now answers both, and takes an optional --kbps for the
auto-request rows -- the only rows whose cost depends on how long the record
takes to arrive.

On the gate container at 12 fps, decode term MEASURED: decode 68.5%, audio 1.25%,
HEADROOM 30.2% = 6.74 clocks per byte at a 37,403 B record. Against that, P4a
cut the transport from 391.7% of a frame to 40..95% -- four to ten times, the
largest movement in this project's cost model since the decoder was written --
and it still does not fit. A dual-address byte is a 4-clock read of the device
plus a 5-clock write to memory, so 9 clk/B is a FLOOR and the frame affords 6.74.
No GCR share goes under it and no delivery rate goes under it: a share decides
whether the channel sits at the floor or above it. At 460 KB/s max-rate totals
165.1% and LRAR at 50% totals 117.4%, and a 50% share tops out at 543 KB/s, above
which the channel is the bottleneck and the rate falls back to exactly that floor.

So 59.2's three bounds arrive in the budget as one sentence: the configurations
this machine can run are the ones the frame cannot afford, and the one it can
afford -- single address, 5 clk/B, 92.2% total, 7.8% spare -- needs the device to
ACK the DMAC, which needs a request line MAME does not connect and the slot
pinout does have at B36/B37.

ROADMAP re-ranks accordingly. B3 stops being a constant to look up and becomes
DOES THE CARD DRIVE #EXREQ, ahead of B1: B1 sets how much headroom the player
has, B3 decides whether there is any. New E7 carries the other branch -- if the
answer is no, the container must reach 27,995 B a frame, 328 KB/s of payload,
against 438 now. The dependency diagram is redrawn around that fork.

The scope is stated rather than buried: this is the GATE container, deliberately
the heaviest thing the encoder emits, and the lighter cpufit family was NOT
priced -- 15_bus_occupancy.py refuses it, correctly, because the C68K measurement
it cross-checks against belongs to the gate container. E7 therefore begins with a
harness re-run, and until then "34% too big" is a statement about the fixture and
not about the project.

check.sh ALL GREEN before and after.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
 2026-08-25 00:17:29 -07:00
prosolis 621a5bb457 Put the data phase on the DMAC, and find auto-request is charged by time 
ROADMAP P4a. src/player/dma.i programs HD63450 channel 1 and takes the SCSI DATA
IN phase off the CPU; src/player/dmagate.s reads the same 2,048 B at LBA 1000
three ways -- PIO, the channel with the bus held, the channel stealing cycles --
and all three are byte-exact against the host's copy of the volume.

The evidence never reads $EA0015, because 57.3 established that it cannot: with
the DMAC's OWN asserted MAME cannot tell a CPU-driven byte there from a
DMAC-driven one. The discriminator is the CPU's own progress. MTC is sampled by
the INSTRUCTION AFTER the one that starts the channel, and held it reads 0 of
2,048 -- the whole transfer happened between two instructions, because the 68000
did not execute in between -- against the full count and 426 loop trips for the
stealing configuration. Put the stealing registers in the held slot and every
byte still arrives and tools/bench/dma_run.sh goes RED, which is what says the
counter can come out different; 58.3's vacuous "UNDERRUNS: 0/120" is the trap
being avoided. tools/analysis/27_dmac_config.py decodes the four register bytes
out of the player's own source, with the MC68450 field tables now in one copy
(tools/analysis/mc68450.py) shared with 21_iplrom_dmac.py, so the player's
configuration and the IPL ROM's 16..19 clk/B one are the same decoding.

Three bounds on the apparatus, read out of MAME 0.277 rather than inferred: the
CZ-6BS1 has NO request line to the DMAC (its flow control is DTACK), so external
request cannot be run; single address cannot be run either, because only channel
0 has device callbacks; and only burst is modelled as held. Of the four rows of
the W ladder exactly one -- dual address held -- has a code path here, and it is
the one demonstrated. W did not move by one clock, for the third session running.

What outlives the emulator is the currency. Every W in this project is clocks per
DELIVERED byte, which presumes the device asks; an auto-requested channel spends
its share of the bus whether or not a byte is there, so a record costs what it
costs to ARRIVE -- halve the delivery rate and the CPU cost of the same record
doubles. tools/analysis/28_autorequest_cost.py prices it from MC68450 3.8 and
5.2.3.3.2, gating its formulas against Table 5-3's sixteen rows first. At 37,405
B and an explicit 460 KB/s: max rate costs the whole 95.3% of a frame the record
takes to land, and of the GCR's four bus shares only BR=00, 50%, carries the
rate -- 10.61 clk/B, 47.6% of a frame, against 40.4% for the W=9 row and 391.8%
measured for PIO. The GCR is a design lever nothing in this tree had named.

59.4 changes what is left. sc_in_data now REFUSES a windowed read when the data
phase is the channel's (SCE_WINDOW), because a channel writes a contiguous run
and cannot drop the 300 B in front of a record. 117 of 120 records need one, so
sector-aligned records have gone from a preference in ROADMAP's re-encode bundle
to the precondition the transport enforces -- and that bundle is now the only
thing between this tree and M2.

One collision, recorded because the procedure is the finding: DM_USE first sat at
$18300, which is ring.i's XF_SLOT mailbox, and the P4b stage -- untouched by this
work -- went red on a run that never reached its snapshot. check.sh was ALL GREEN
before any of this, which is what made that red unambiguous. ALL GREEN after too,
with one new stage. decode.bin is unchanged at 1,296 B and the same MD5.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
 2026-08-24 23:55:07 -07:00
prosolis 5921fab118 Put the transport on the 68000, and find PIO costs 87 clocks a byte 
ROADMAP P4b. src/player/xfer.i answers src/player/ring.i's XF_* mailbox with a
real READ(10) to a real MB89352 in place of tools/bench/stream.lua's modelled
transport: 120 records, 4,488,588 B, pixel-exact out of a 256 KB ring, with a
real mid-stream seek in a second pass. The tiling is the SAME 18 wraps and
14.7 KB mean hole that 49.4's host producer and 55.4's modelled transport
produced -- a third transport, same placement, which is the assertion that
ring.i could not tell which side of the seam answered it.

What it costs is the finding. tools/bench/xfer_cost.sh subtracts the same 120
frames run twice and gets 87.28 clocks per delivered byte, against the 68000's
own cycle table for the loop, which says 87.15 -- 0.2% apart, so the cost is the
instruction stream and not MAME's device model, and it is the first number this
rig has produced that survives leaving the emulator. That is 391.8% of a 12 fps
frame; the machine's own V-DISP clock agrees from the other end at 2.57 fps.
Against the ladder, W=5 held is 22.4% of a frame and W=19 is 85.3%, so P4a is
worth 4.6x the worst DMA configuration in this tree and 17.5x the best -- where
before this session it was worth 9 against 19. W itself did not move by a clock.

"UNDERRUNS: 0/120" is vacuous with a synchronous transport, and stream.lua now
prints that argument next to the zero: a frame cannot start before its record
has landed because the decoder IS the transport. The counter that means
something is NO IDLE, 119/120 with a worst overrun of 441 whole ticks. Same
class of error as 49.7.2's free-running ring passing at 48 KB.

58.3: a record is not a sector -- 117 of 120 start part way into one, and
reading whole sectors into the ring corrupts the neighbours rather than wasting
bytes (49.2, no bounds check). scsi.i reads the covering sectors and stores only
the window, which is free in PIO and stops being free the moment P4a succeeds.
tools/analysis/26_sector_align.py prices the three ways out and sector-aligned
records win on both axes: +0.43% wire and zero clocks, against +1.34% and a
bounce copy at +5 clk/B. ROADMAP now carries a four-item re-encode bundle and
P4a should be attempted against a sector-aligned container.

check.sh gains two stages and was ALL GREEN before and after. decode.bin is
unchanged at 1,296 B and the same MD5.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
 2026-08-24 23:15:53 -07:00
prosolis e935d8661c Put the SPC on the 68000, and find P4 was blocked on a file nobody needed 
ROADMAP P4, first half. Every byte the player has ever consumed was placed in
emulated RAM by a host: decode.lua preloaded a container, stream.lua answered a
mailbox at a modelled rate. src/player/scsi.i selects a SCSI target on a real
MB89352 and issues READ(10) itself -- 4,096 B from LBA 0 and 2,048 B from LBA
1000, both byte-for-byte against the host's copy of the same volume, with no
IOCS and no host in the transfer path. The non-zero LBA is the half that
matters: a driver that emits a malformed LBA field still passes block 0,
because zero is what a malformed field usually is.

P4 was recorded as blocked in this tree and was not. Session 21's handoff said
MAME's x68000 has no MB89352 path; -exp1 cz6bs1 instantiates one next to the
HD63450, and FINDINGS 32.4 had read that card's DMA glue in session 9. The
session-21 note is a regression in the record, not a discovery. What is
genuinely absent is the 8 KB scsiexrom.bin MAME requires to INSTANTIATE the
card and the player never executes -- driving the SPC registers directly has
been the plan since BENCHMARK item 4 in session 2 -- so scsi_run.sh supplies a
zero-filled placeholder on its own rompath, leaves the user's romset alone, and
lets MAME print WRONG CHECKSUMS as it should. B3 is untouched: it wants that
ROM's bytes disassembled and a blank one has none.

The register map is measured, not inferred, and it corrects MAME's own
documentation. The probe walks $EA0000..$EA003F one address at a time with a
bus-error handler that records the fault and steps the index, because a
sequential dump reports the first hole as the answer -- the earlier version
took a bus error at $EA0006 and knew nothing about the other 57. 60 of 64
answer; the two holes are exactly the TMOD and EXBF the MB89352 omits and the
MB87030 has. MAME leaves HOLES and does not shift the later indices down, which
its own device summary claims it does, and that is what keeps DREG at $EA0015.

The data register is DMA-only here and a PIO write vanishes. x68k_scsiext.cpp
glues $EA0015 and nothing else, and with exown() asserted and DRQ low the byte
is discarded: no error bit, no status change, no interrupt. Quieting all four
DMAC channels does not change it. Measured rather than reasoned about -- write
$5A, read back $00 with the FIFO still empty -- because ten command bytes
vanishing without trace looks exactly like a target refusing a command, which
is how it first presented. So every transfer runs the SPC in DMA mode and the
CPU moves the bytes through the DMAC's own door.

That costs the argument something, and it is easy to overclaim here: with exown
asserted at idle MAME cannot distinguish a CPU-driven byte at $EA0015 from a
DMAC-driven one. This shows the DATA PATH and cannot by itself show that the
HD63450 is driving it, which is precisely what ROADMAP calls P4's first job.
Whether a real CZ-6BS1 also refuses PIO there is not settled; it is a property
of MAME's model and it wants a board.

W did not move by one clock, and could not have. MAME's device models are
functional rather than transfer-timing accurate and 42.5 reads its DMAC
configured in wall-clock attotimes, so this is BENCHMARK Tier 1 -- does the
read path work -- and never Tier 2. W is still the largest open number here.

Five bugs, four of them silent, recorded in 57.5 because the pattern is the
finding: a chain of rol.l #8 that loaded a transfer counter of ZERO from a
count of 10; a byte handed to a FIFO mistaken for a byte on the bus; a fixed
phase sequence where the bus decides the order; the discarded PIO write; and an
initiator that must drop ACK and only then release the bus. The last appeared
only once there were TWO reads -- one passed byte-exact and every conclusion
from it was sound, and the second could not select. A player issues one command
per record, so that failure would have been universal in the ring and invisible
in a one-read demonstration.

No decoder code changed; decode.bin is still 1,296 B at the same MD5. check.sh
gains a SCSI stage that builds the volume out of the same stream_disk.bin the
ring rig reads, gates the register window at 60 of 64 and both reads
byte-exact, and skips when chdman is absent. ALL GREEN before and after.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
 2026-08-24 22:53:58 -07:00
prosolis 00232bb22b Import the scene graph, and find the worst gap between two branch points is zero 
ROADMAP G1, scheduled early because it is a measurement input, and it paid for
that immediately. 51.3 established that a ring's lookahead is accumulated out of
pipe - wire and that a seek spends all of it, so what a branch point costs is
set by the time since the last one. 55.5 rehearsed a seek on the machine and
said out loud that it could not ask the question, because nothing here knew
where the branch points are.

They are everywhere, and 5.4% of them are free of charge to the player and not
to us. Over 612 distinct transitions into a seek, taking the earliest instant
each input window opens: worst 0.000 s, p10 0.950, median 3.473, best 82.497.
33 open on the first frame of a clip the disc SEEKED to -- press right as
flaming_ropes.enter_room appears and you are in fall_to_death, two seeks with no
play between them. 51.2's slack rule can therefore be answered NO by the
content rather than by the buffer, and no amount of ring is a defence.

It does not break the design. A branch on an empty ring costs the 2-record
prefill, 149.7 ms at 488 KB/s, not the climb. What it removes is margin: at that
rate in a 256 KB ring, 76% of this game's branch points arrive before the ring
has refilled, and a 512 KB ring makes that 90%, because doubling the ceiling
does not touch the surplus. The ring is not the lever; the surplus is.

CORRECTION to FINDINGS 16: there is only one transcription. The SNES chapter
set says in its own README that it is derived from DirkSimple, so the planned
diff of two independent sources catches conversion errors only. Run anyway:
durations agree 388/505 within a frame, branch structure 470/505, and of the 35
differences 16 are renames and 18 of the other 19 are that port dropping the
arcade's diagonals. Zero transcription discrepancies, and none were findable.

Two constraints on the input layer come free: the arcade needs eight directions,
and the shortest input window is 98 ms against 54.4's 72.13/90.16 ms frame slot,
so input cannot be polled on the frame tick.

The coupling to outside source is contained to one file (USER DECISION).
tools/import/scenegraph.py is the only code here that knows those projects
exist -- their paths, table names, timing formulas, constants -- and it writes
DLXSCENE1, this project's own schema, into gitignored tmp/ with the sources'
licences inside it. tools/analysis/25_scene_graph.py reads only that. Nothing is
vendored and nothing outside-derived is committed. The split was made after the
measurement and the whole output was re-run byte for byte to show it moved no
number.

Both import gates are negative-tested: deleting one sequence upstream fails the
516/906 count, and closing the table early fails the constructor-end check,
which replaced one that was vacuous.

No 68000 code ran or changed; decode.bin is still 1,296 B at the same MD5.
check.sh gains an import stage that skips when there is no checkout. ALL GREEN
before and after.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
 2026-08-24 22:11:04 -07:00
prosolis 2676f3b835 Put the ring on the 68000, and find the disc stops whenever the player is not asking 
ROADMAP P5. The loader moved in session 21 and the frame clock in 22; the ring
producer was the last policy living outside the machine. src/player/ring.i does
`aligned` placement, the descriptor ring, a prefill, 51.2's slack rule and a
seek, and the host keeps only the transport.

It needed a container change. `aligned` asks whether the next record fits
before the end of the ring -- a length asked BEFORE the record is fetched -- and
every reader in this tree answered that by walking the frame stream, which is
exactly what a player streaming off a disc cannot do. DLX4 carries nframes u16
record lengths in the scene header. Frame payloads are byte-identical to the
DLX3 encode, so no fitted constant moves; the scene header goes 5,920 to 6,164 B.

The producer reproduces the host's tiling exactly: 18 wraps, 14.7 KB mean hole,
pixel-exact, a third independent implementation of the same policy.

What it exposed is bigger than the item. A channel only moves bytes while it has
a request and only the CPU can issue one, so the disc stands still between
records by an amount the PLAYER sets, not the medium -- and no host-filled run
could see it. At 488 KB/s in a 256 KB ring a one-deep request queue gives away
6.8% of the pipe and underruns 59 of 120 frames; two-deep gives away 3.4% and
underruns none. The container's whole surplus over the wire is 8.7%, so the
player's own loop was spending most of the slack a branch point saves up.
Prefill is the weaker lever: six records of it still leaves 24 underruns.

Three silent bugs are recorded in FINDINGS 55.7 -- all produced wrong pixels or
a desync rather than a fault -- plus a rig one: MAME renders a screen line by
line, so snapshotting the frame the decoder finished in captures a tear that
reads exactly like a decoder bug.

check.sh gains the machine-owned ring and a seek with the decode after it.
decode.bin is unchanged at 1,296 B and a host-filled run executes none of the
new code, so every FINDINGS 49/51 figure stands. ALL GREEN before and after.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
 2026-08-24 21:45:05 -07:00
prosolis c419251266 Put the frame clock on the 68000, and find that the 12 fps frame does not exist 
ROADMAP P3 said "needs MFP timer or VBL" and neither can do it.  The MFP's
timer clock is 16 MHz/4, its prescalers stop at 200 and its data register is 8
bits, so the slowest tick any single timer can make is 78.125 Hz -- 6.5x faster
than a frame -- and 4e6/12 is not an integer, so no setting reaches 12 Hz at
all.  The raster has no whole divide near 12 either: 4 refreshes is 13.86 fps
and 5 is 11.09.  tools/analysis/23_frame_clock.py walks all 7x256 timer settings
rather than asserting it.

src/player/clock.i takes the V-DISP falling edge on MFP GPIP4 -- the start of
vertical blanking, which is when a player would present -- and adds fps*VTOTAL
per edge to a 16-bit accumulator, emitting a tick at 31,500 and keeping the
remainder.  The long-run rate is fps*VTOTAL/VTOTAL = 12.000000 fps exactly, and
both constants are read out of the CRTC at init, so the clock is derived from
the registers that generate the raster it counts.  Measured over 3,000
refreshes: 3,000 interrupts, 649 ticks where 649.1429 were due.

It costs 181.35 clocks per V-DISP, 838 per frame, 0.1006% of the budget -- timed
by the 68000 itself, because the host's granularity is 17.64 ms and the
interrupt is microseconds.  The loop's own cost was calibrated rather than
looked up and landed on 38.000002 clocks, which both licenses the subtraction
and confirms buscost.py's model; the 181.35 then decomposes exactly, leaving
43.99 clocks for the interrupt exception -- the textbook 44, measured.

THE ONE THAT MOVES SOMETHING: 12 fps on a 55.4577 Hz raster is 4.6215 refreshes,
so a frame is shown for 4 refreshes (72.13 ms) or 5 (90.16 ms), 37.9% of them
short.  The 833,333-clock budget every figure in this project is priced against
is the MEAN slot, and the short one is 13.4% under it.  The cadence was already
in the tree unnamed: stream.lua's tick is sampled at frame boundaries, so its
gaps were always 4 or 5, and every host-paced result in FINDINGS 49/51 carried
it.  P3 moved who produces it onto the machine and made it visible.  It is not a
dropped frame -- the pace gate lets an overrun eat the next frame's idle -- and
on the gate container it costs 4 frames of 120 their idle against 1 for the
nominal model, most of that the frame-0 transient at 111% of budget.  stream.s
counts it now, and the rig matches an offline model of the divider exactly.

Also struck: MAME's raster runs 2.22% fast.  refresh_mode() builds the frame
period from scr.max_x*scr.max_y with scr.max_x = m_htotal - 8, one character
cell short and an inclusive bound used as a count, so it runs at 56.6901 Hz
where the registers say 55.4577 -- agreeing to six digits with the arithmetic.
Every "1/55.46 s granularity" note in this tree was wrong and is 1/56.69 s,
corrected in six files with the derivation put once in crtc_mode.lua.  No
conclusion changes and no 68000 cycle figure moves; the CPU clock is unrelated
to the screen.  But anything paced by the raster runs fast under MAME, so the
rig reports both rates and prices the interrupt against the hardware's.

decode.s and frame.i are unchanged; decode.bin is still 1,296 B at the same MD5.
The pace gate's wait loop is byte-for-byte the one FINDINGS 51 measured and the
free-running path executes none of the new code.  check.sh gains two stages: the
clock's own measurement, and 120 frames decoded pixel-exact with nothing outside
the machine deciding when a frame may start.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
 2026-08-24 20:55:34 -07:00
prosolis 7179339bd2 Move the loader onto the 68000, and find 5,920 bytes nobody counted 
src/player/load.i expands both codebooks to word-per-pixel form and packs the
palette to GGGGGRRRRRBBBBBI out of the RAW container header, byte-exact against
tools/bench/dlxload.py on both CPU cores.  The palette half is gated on words
read back out of the palette registers at $E82000, so "the words reached the
hardware" is part of what passes.  ROADMAP P1 is done; P2's encoder half (a
reserved black entry, 23.4) is not, and is a re-encode rather than an edit.

A scene change costs 18.96 ms of 68000 time, 22.8% of one 12 fps frame; boot
costs 24.70 ms.  The scratch tables describe the CRTC, not the scene, so
pal_tables is a separate entry point built once at boot -- 5.29 ms off every
scene change.

The one that moves something: the scene header is 5,920 B that no rate table in
this tree included, because it belongs to no frame record.  In FINDINGS 51.3's
currency it is divided by the surplus pipe - wire, so it is hypersensitive:
138 ms of extra refill climb at 488 KB/s and 1.099 s at 451.4 KB/s, for the
same bytes.  tools/analysis/22_scene_load.py prices it across explicit rates.

Recorded as open: the two CPU cores agree to <3% on every stage but the table
build, where they differ by 16.4%.  px68k's C68K charges a flat 50 clocks for
MULU/MULS (c68kmacro.h:1869) where the 68000 charges 38+2n, which explains
4,608 of the 8,703 clock gap.  4,095 clocks are unexplained.  Nothing else in
src/player/ multiplies, so no figure in FINDINGS 24-52 is affected.

decode.s and stream.s are untouched; decode.bin is still 1,296 B at the same
MD5.  check.sh gains a stage that gates byte-exactness on both cores and
deliberately does not gate the cycle counts -- MAME's clock is 1/55.46 s and a
wall timing would make the green light host-sensitive.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
 2026-08-24 20:20:40 -07:00
prosolis ed172c2da2 Rewrite the README as a description, and strike a unit error it exposed 
The README had become a changelog: which session found what, which constant was
retired and by whom, which figure was overturned. That is what STATUS is for,
and it made the front page a poor read for anyone who has not been following.
Rewritten to say what the project is, what it looks like, where it stands and
how to reproduce it, present tense, no session numbers, no em dashes.

Restating the standing facts caught one of my own from this session. FINDINGS
52.5 cited 42.4's W sensitivity table -- W <= 6 fits 0/120 frames, W = 8 misses
47/120 -- as though those were clocks per BYTE. They are per WORD. FINDINGS 43
is the section that caught W being charged per word to a byte-wide port and says
in terms that 42.3's 0/120 was never physically reachable, so quoting it in byte
units re-imported the exact 2x error 43 exists to have corrected, one section
after using that same trap as a warning.

Struck in 52.5 and in ROADMAP P4, and replaced with the ladder buscost.py
already carries, which is in the right unit: 5 clk/B single address with the bus
held, 9 dual address held, 12 single address arbitrated per byte, 16..19 dual
address arbitrated per byte, the last being what the IPL ROM programs. Nothing
else in 52 depended on the struck figures -- 15_bus_occupancy.py's sweep is in
byte units and is unaffected.

The "cross-check" claim goes with it. 52.5 read the agreement between its W = 8
row and 42.4's 47/120 as two independent models landing in the same place. It is
two different units on two different containers at two different rates, and
calling it corroboration was manufacturing agreement out of the unit error.

What replaces "W <= 12 is a requirement on the player's DMAC programming" is the
sharper version: getting the DMAC to HOLD THE BUS is what separates 9 from 19,
and that is the thing P4 has to demonstrate.

Docs only; no code changed and no gate output moves.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
 2026-08-24 18:30:42 -07:00
prosolis e565dfbbab Take the disc mount out of the source, so someone else's copy of the media works 
Three files hardcoded /media/reala-misaki/BDROM -- extract.py, 07_motion_survey.py
and check.sh -- which made the pipeline unrunnable for anyone whose disc mounts
anywhere else. DLX_BDROM now overrides it everywhere, defaulting to
/media/$USER/BDROM, so this box is unchanged and another one works. Verified by
running extract.py against a symlinked mount at a different path.

check.sh's failure message names the path it looked at and the variable to set,
instead of assuming udisks put it where this machine puts it.

README gains a "Reproducing this" section: no media ships here and none of it is
redistributable, so it says what you have to bring (the disc) and what is
already packaged (vasm is vendored as a binary with its source tarball; the
k-means is hand-rolled, so numpy and Pillow are the whole Python dependency).
It also names the two gates that SKIP rather than fail -- the px68k second-core
pass and the IPL ROM DMAC gate -- because both live outside this repo and a
silent skip is worth reading as a skip.

One trap called out rather than left to bite: scene selection is a hard-coded
stream number, not a search. A different pressing that numbers its .m2ts files
differently will extract the wrong footage and the green light will PASS on it.

check.sh ALL GREEN.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
 2026-08-24 18:19:54 -07:00
prosolis 2f9f5cc995 Pace the ring, then read the DMAC config out of the IPL ROM: audio is cheap and the disk is not 
Two sessions that were never separated in the working tree, so they land as one
commit. check.sh ALL GREEN before and after both.

SESSION 19 -- the ring rig gets a frame clock (FINDINGS 51).

src/player/stream.s had no frame clock: it asked for record i the instant it
finished i-1, outran any finite pipe, and never let the ring back up. The 49.1
sweep passing at 48 KB was therefore a wrap-correctness result and nothing else.
PACE/PACEON ($18034/$18038) hold the decoder to 12 fps, so FR_HEAD-FR_TAIL
finally means what it reads as: whole frames the decoder could still draw with
delivery stopped dead. PACEON=0 free-runs and is what the wrap gate still uses,
so every figure in 49 is unmoved.

Paced, on the gate container: 64 KB holds 2 frames, 256 KB holds 7-8, 512 KB
holds 14-15, all pixel-exact. Tolerance is ceiling-1, measured by cutting the
pipe: 256 KB buys 500 ms of dead pipe, not 583.

SLACK IS ACCUMULATED, NOT OWNED. It is built out of pipe-wire and a seek spends
all of it. At 488 KB/s a 256 KB ring needs 4.83 s of play to reach its ceiling
from empty; 512 KB needs 8.42 s to reach 14. A bigger ring raises the ceiling
AND lengthens the climb, so a branch point does not ask "is the buffer big
enough" but "has there been enough play since the last one" -- and Dragon's
Lair's decision points are seconds apart. The rig now also says WHICH resource
is binding: at 460 KB/s every ring from 192 KB to 512 KB is rate-bound at
ceiling 4 and never fills, so larger rings are dead RAM in that scene.
20_seek_slack.py is the same model rewritten in Python from record sizes,
sharing no code with the Lua producer: 35/35 ceilings inside its bracket.

SESSION 20 -- the DMAC configuration was in the IPL ROM the whole time
(FINDINGS 52).

ROADMAP's "do this first" was to put the ADPCM stream on the bus. That needs a
clocks-per-byte figure for the audio channel, and 11_cpu_budget.py was charging
audio the DISK's rate -- 5 clk/B, its own help text calling it "single-address,
bus held". Audio was being charged the favourable end of B3, a 242 KB/s open
question.

It never had to be a guess. The IPL ROM programs all four HD63450 channels
itself and MAME boots the rig with it, so 21_iplrom_dmac.py reads the
configuration out of the image and decodes the MC68450 fields. Eight
(address, expected bytes, meaning) sites; a mismatch or an unknown revision
exits non-zero. In check.sh, no emulator, milliseconds.

ch3 DCR=$80, OCR=$32: dual address, 8-bit port, cycle steal WITHOUT hold,
REQG=10 external request. The DMAC arbitrates once per byte with no burst to
amortise the 5..8 + 2 over, so an audio byte is 16..19 clocks, not 5 -- the old
debit was 3.2x..3.8x small. And on the bus it is still nothing: 651 B/frame is
1.25%..1.48% of a frame, about 4% of what the decoder leaves. P6's bus risk
does not materialise. The unit worry was worth checking and nearly right: 15.6
kHz is 8 MHz/512 = 15,625 samples/s, two 4-bit samples to a byte = 7,812.5 B/s
exactly, and AUDIO_KBPS=7.8 is that in decimal kB while the tool multiplied by
1024.

THE DISK CHANNEL IS PROGRAMMED IDENTICALLY. ch1 (SASI) is DCR=$80 too, and so
is ch0. That is 16..19 clocks per delivered byte, where 42.4 brackets W at 5..12
and 42.5 has W=8 already missing 47/120 frames. The only worked example of a
disk DMA configuration on this machine sits above the entire bracket, and at
that price nothing fits at any container size. It is not scsiexrom.bin so B3
stays open -- what changed is that a cheap configuration is now the thing that
has to be SHOWN. W <= 12 is a requirement on the player's DMAC programming, not
a range the hardware hands us, and it is now the largest open number in the
project, ahead of the rate.

An unforced cross-check fell out: 15_bus_occupancy.py's new W sweep puts W=8 at
105.7% of the frame, agreeing with 42.5's 47/120, from mode histograms and bus
clocks respectively, two models sharing no code.

Also: ADPCM outranks the disk at the arbiter (CPR 1 against 2), so an audio byte
never waits and a video byte does -- relevant to 51's smooth-rate delivery model.

README MEDIA.

stream.lua gains DLX_SNAP_EVERY=1 (needs DLX_PACE, off by default, on no path
check.sh takes) and tools/media/make_readme_media.py turns the PNGs into
docs/img/. The stills and both clips are MAME's own screen pixels.

Building it turned up something worth recording. 116 of 119 captured frames are
pixel-exact against dlx.py; three are TORN -- frame n on top, frame n-1 below
the tear line -- because MAME captured the screen while the block loop was
partway down it. decode.s writes straight to the displayed page (one display
path, 28.1), so a real player tears the same way, and this is the first time
that consequence has been visible rather than argued. The script ASSERTS the
tear and refuses to build otherwise, rather than trimming three frames and
reporting "every frame I kept is exact". Second correction the capture forced:
the snapshot fires before frame n is decoded, so the obvious reading is that it
holds frame n-1 -- it does not, because MAME renders the screen at the end of
the machine frame, by which time the 68000 has finished frame n.

11_cpu_budget.py's "validated to within 1 pt" line is also corrected: the model
reads 2..10 pt HIGH and by more as the frame gets harder, which was already true
before either session.

src/player/decode.s is unchanged; decode.bin is still 1,296 B at the same MD5.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
 2026-08-24 18:14:01 -07:00
prosolis b49bbdc939 Build v7 into the player, and find the cost model 18% wrong on the block it made commonest 
src/player/decode.s now paints v7 literal spans, pixel-exact under MAME and
px68k's C68K core over a container where every frame carries 128-216 spans
covering up to 38% of the picture. The span pass is blit.s v7 verbatim: the
66.0/9.143/9.978 fit was measured on that instruction sequence.

The container is DLX3 -- a span section between the mode header and the block
payload, since that is the only place the 68000 can reach without first parsing
something of variable length. 16_span_roundtrip.py gates it in check.sh, and
asserts it emitted enough spans to have tested anything.

Two synthetic all-SPAN anchors price v7 inside decode.s at 151.2 and 225.6
clocks per 4x4 block, against FINDINGS 40's table of 151 and 226 -- 0.2% on
both emulators. The measured mode costs what it was said to cost.

Two things that were not on the list:

TWO BYTE BUDGETS. FINDINGS 40's 18/120 was scored against the 488 KB/s PIPE,
not the 280 KB/s profile, and at the profile rate the lam search has already
spent the allowance -- spans fired on 5 frames of 120 and looked like a
regression. The profile is a chosen quality rate point; the pipe is hardware.
--kbps and --span-kbps are now separate and spans run before mu, because a span
pays in bytes and mu pays in picture. Delivered: 86/120 over budget without
spans, 77/120 at the profile budget, 34/120 on the pipe for +0.36 dB.

C_SKIP_MIXED WAS NEVER MEASURED, and it was 18% low -- 45.0, now 55.0. It is
the one constant in the table that came from a derivation, because the
synthetic frame that would measure it cannot exist: a byte needs a coded block
for its SKIP to be mixed. Four bracketing anchors measure it on both emulators
with the header byte rotated through all four positions, and the partner mode
solves back to its own anchored value to 0.2%. With it corrected the model
predicts a real spanned decode to -0.06% mean / 0.09% worst, against -2.99% /
4.30%. It matters because a span marks its run SKIP, so mixed SKIPs dominate
exactly the frames spans are judged on.

Also: the rig had been writing its synthetic timing frames 26 KB past the top
of a 2 MB machine, and got away with it because the modes it overran are
data-independent. A span's jump displacements come out of the stream, so it is
not. And frames-over-budget is no longer a safe headline -- the controller aims
at the deadline, so 55 of 120 frames sit within 5% of it and a 1% cost shift
moves 22 frames.

FINDINGS 41. check.sh ALL GREEN, now gating on a span-heavy DLX3 container.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
 2026-08-23 20:02:03 -07:00
prosolis c520a89e14 Measure the finer chain tail: 84/120 becomes 18/120, and the derivation was right by cancellation 
blit.s gains v7 -- v6's 24-pixel movem chain plus a second chain whose unit is
one `move.l (a0)+,(a2)+`. Measured over 13 span lengths by span.sh, every config
pixel-exact:

    cycles = 66.0 per span + 9.143 per COARSE pixel + 9.978 per FINE pixel

fitting all 13 to within 0.2%. v5 and v6 re-measure to FINDINGS 30 exactly, so
the harness has not drifted underneath the new variant.

Rescored against the same scsi window and the same additive model, v7 takes
84/120 frames over budget to 18/120 -- exactly what FINDINGS 39.4 derived, and
that agreement is two cancelling errors: the derivation's 2-register movem tail
is 29% too dear per pixel, and its "nothing per span" for the second chain entry
is 22.3 clocks too cheap. The plain post-incrementing move.l is the right tail
instruction, and it makes the padding quantum 2 pixels, which a run of 4x4
blocks pads to exactly zero.

The DMAC stays dropped on a measurement now rather than an argument: v7 takes
back 37 of the 43 frames the array chain would, with no reserved channel and no
timing neither emulator here can verify. Break-even against all-V1 moves from
L=4 blocks to L=2.

The fine displacement is carried mid-stream rather than in the span record, so
the decoder holds nothing across the copy and keeps all 12 payload registers --
which is the whole reason the coarse unit is 24 pixels.

span.sh is now -seconds_to_run 200 (30 s wall, 36 configs) and takes its
expected snapshot count from the generated metadata instead of a literal 23.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
 2026-08-23 19:02:23 -07:00
prosolis c5ca56330e A second emulator agrees, the bus was never counted, and the DMAC loses by one clock 
Three things, and the last one reversed itself when the datasheet arrived.

A SECOND EMULATOR. tools/bench/c68k/ links px68k's C68K core into a headless
harness -- no SDL, no ROMs, no emulated machine, because the decoder touches
nothing but RAM, the control block and GVRAM. decode.s is now pixel-exact under
two independent CPU cores, and cycle-table error against MAME is bounded at
3.3%, running against us. MAME 0.277's M68000 turns out to be the MICROCODE
core, not Musashi (m68000.lst + m68000gen.py), so this is two structurally
different timing models agreeing rather than two tables. FINDINGS 28.8's "V4
costs more than RAW" reproduces independently. FINDINGS 37.

THE BUS. Nothing since FINDINGS 24 had counted the 68000's local memory bus --
one 4-clock cycle at a time, carrying instruction prefetch as well as data. The
decoder occupies 86.7% of it and PREFETCH IS 62% OF THAT TRAFFIC, so a data-only
count understates occupancy by 2x. Two sources check each other: c68k_bench
counts every bus callback exactly, and a static walk of decode.lst supplies the
prefetch no emulator here can report. The walk reproduces the measured data half
to 0.04%, which is what licenses its prefetch half, and 15_bus_occupancy.py is a
gate rather than a report because every bus figure depends on that check.
FINDINGS 38.

THE DMAC CHAIN LOSES. FINDINGS 29.6 named it the one uncosted lever. Costed from
bus arithmetic -- a read cycle plus a write cycle, 8 clocks a pixel -- it scored
1/120 frames over budget against the v6 span's 10/120 and looked decisive. Then
the MC68450 manual (Motorola Jul 1989, now at ~/src/mc68450.pdf): Fig 4-25 sheet
4 puts a dual-address word between two 16-bit ports at 9 CLOCKS, because note 2
gives the DMAC 4-clock reads and 5-clock WRITES. The 68000 writes in 4.

    DMAC   9.000 clocks/pixel   datasheet
    v6     9.152 clocks/pixel   measured, FINDINGS 30

1.7%. Scored additively, 86% of what remains of the DMAC's advantage is v6's
24-pixel padding quantum -- a property of its unrolled movem chain, fixable in
software with a finer tail chain, worth 55/120 -> 18/120 against the DMAC's
12/120. Recommendation: fix the quantum, drop the DMAC. Six frames does not buy
a reserved channel, a two-region container layout and a timing dependency
neither emulator here can verify. The container is identical either way -- v6's
record and an HD63450 chaining entry are both 6 bytes, so the chain array IS the
span table -- so nothing is foreclosed. FINDINGS 39.

TWO CORRECTIONS TO MY OWN WORK IN THE SAME SESSION:

- I argued FINDINGS 35's flat CPU debit for the disk was too pessimistic and
  rescored the window at 53/120 with max(CPU, bus). Wrong. A 68000 has no cache
  and a two-word prefetch queue, so it stalls the moment another master takes
  the bus, and the MC68450 hands the bus over in SLABS under limited-rate
  auto-request rather than interleaving per operand. DMA is additive. 84/120
  stands and 14_dmac_chain.py reproduces it exactly. What 86.7% occupancy really
  says is that there is almost no room to overlap anything. FINDINGS 38.3.
- The first DMAC costing was derived where a primary source existed. Both wrong
  answers were confident and both were caught by reading the manual.

Also landed:
- FINDINGS 5's 8 clocks/word for the SCSI DMA, STATUS's own "most load-bearing
  unmeasured number", is now bracketed by the datasheet: 5 clk/word with the bus
  held, ~12 if the DMAC arbitrates per word. 8 is a supported midpoint, and
  which end applies is a player design decision worth 7 clocks a word on a
  480 KB/s stream. FINDINGS 39.7.
- check.sh gains two gates: the C68K pixel-exact decode (seconds, no MAME) and
  the bus-model self-check. Both skip cleanly without a px68k checkout.
- spanned blocks are now charged their mode-map dispatch, which FINDINGS 30.7
  flagged as uncounted in 12_span_tradeoff.py.
- MAME timed runs must be budgeted by WALL CLOCK, not -seconds_to_run: this box
  runs x68000 at ~0.033x realtime and two runs were killed by their own timeout.
  That is why the all-RAW cell in 37.3 is empty. The C68K harness does the same
  work in seconds because it emulates a CPU and not a machine.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
 2026-08-23 18:30:23 -07:00
prosolis 7d365b3ff5 Drop SASI on capacity, then find the budget never had the disk in it 
USER DECISION: drop the `sasi` profile. Not on bandwidth -- on capacity. A SASI
volume is 40 MB, and the 22.8 min of unique scene footage on the source Blu-ray
(streams 00000-00201, measured, not recalled) is 146 MiB at the LOWEST rate this
codec makes -- more than the machine's whole 4-unit SASI space. `scsi` is the
only profile now. FINDINGS 32.

Then the user asked whether we were drawing the wrong conclusions about PIO vs
DMA, and we were, more broadly than the question implied. Every CPU figure in
FINDINGS 24-34 is scored against the full 833,333 cycles/frame with nothing
subtracted for moving the bitstream off disk. Debiting the HD63450 cycle-steal
at the long-standing 8 clk/word ESTIMATE, "1 frame of 120 misses" becomes 84 of
120, median 112.4%. PIO at the span rate is 99.8% of the machine. Spans buy
cycles by spending bandwidth and the bandwidth returns as steal, so 31.6's "fits
completely" becomes a worst frame of 114.3%. 10 fps absorbs it: median 93.7%,
1/120. FINDINGS 35. `11_cpu_budget.py` takes --io dma|pio|none, defaults to dma,
and warns if asked for none.

Also landed:
- item 1 done: the cost model checked against the 68000 on a cost-aware
  container, -3.07% to +0.01%, whole-window mean -1.22%. FINDINGS 34.
- item 4 done: the container carries its own 4-byte record alignment (DLX2).
  94/120 record starts were on odd addresses -- an address error, not a slow
  read -- now 0/120 for 16 B/s. Re-encoding reproduces 31.1 exactly. FINDINGS 33.
- a `scsi` window does not fit the 2 MB machine the rig emulates (2.84 MB of
  stream past a 0x200000 ceiling). The gate now verifies 80 of 120 frames and
  SAYS so, and fails loudly when the pass does not complete, instead of
  reporting a phantom 49,005-pixel diff. FINDINGS 36.

Three near-misses this session had one shape: an unobservable run nearly
produced a false finding. stdbuf -oL on any MAME job that prints progress -- a
file is block-buffered too, and a run that is merely finishing looks exactly
like one that is wedged.

check.sh ALL GREEN.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
 2026-08-23 17:09:47 -07:00
prosolis 06b98d4b47 Price cycles in the mode decision: 37 misses become 1, for 0.26 dB 
The decoder has been CPU-bound since FINDINGS 28 while the mode decision
minimised D + lam*R -- distortion against BYTES. decide() now minimises
D + lam*bytes + mu*cycles, and ratectl bisects mu per frame against the
833,333-cycle budget with the lam bisection nested inside it. On the worst
sustained window:

  sasi  27.22 -> 26.95 dB, 109.5 -> 109.4 KB/s, 37/120 misses -> 1
  scsi  29.90 -> 29.27 dB, 280.0 -> 278.6 KB/s, 51/120 misses -> 1

Bitrate does not move: the byte controller still binds, and mu changes WHICH
modes are bought. V4 is what it stops buying -- 25.2 -> 20.3% of blocks at sasi
and 15.0 -> 5.3% at scsi, where RAW takes it. That is 28.8's inversion in
practice: RAW is dearer in bytes and cheaper in cycles, so only the byte-rich
profile can buy its way out of V4.

Three things worth knowing beyond the headline:

  - The one frame that still misses, at both profiles, is FRAME 0 -- no previous
    reconstruction, so 100% changed by definition, which is also what a scene
    cut is. It comes out at the all-V1 floor of 110.6% and is emitted late on
    purpose. Freezing a cut to make a deadline is the worse failure.
  - 28.7's "11 frames are impossible" was too pessimistic. That floor held the
    SKIP set fixed and asked how cheaply the drawn blocks could be drawn; the
    real decision can also MOVE a block to SKIP, which above ~90% non-SKIP is
    the only lever left.
  - SKIP's price depends on its neighbours (13.25 cycles clustered, 45 mixed),
    which a per-block lagrangian cannot see. The way out is that the two uses
    need not share a cost function: a ranking constant inside decide(), the
    exact clustered rule for the frame-level bisection. vq_hybrid.cycles() is
    now the one definition of that rule and 11_cpu_budget.py imports it.

Gated: 09_ratectl_drift.py runs both controllers, both 0/120 drifting frames.
The cost-aware container decodes pixel-exact on the 68000 (120 frames). ON by
default in encode.py; --no-cpu-fit restores session 7. check.sh ALL GREEN.

Still a model, not a measurement, for THIS container: FINDINGS 31's cycle
figures come from vq_hybrid.cycles (within 1 point of the 68000 on four frames
of the session-7 container). Timing this one on the machine is step 1 of the
next session -- it was started and killed for time, and it is slow.

FINDINGS 31. tools/analysis/13_cpu_ratectl.py.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
 2026-08-23 16:24:22 -07:00
prosolis 29eb78a599 Measure the span: the mode survives, and it is an encoder format 
FINDINGS 29 priced a literal-span mode at 4*(50 + 4L*9.08) cycles and labelled
the whole section DERIVED. Session 8 step 0 was to measure it before optimising
over the mode set it implies. Two variants in blit.s, one stream per span length
from prep_spans.py, timed by span.lua, driven by span.sh in ~25 s:

  v5, handed (x, npix) and left to work the copy out:  97.9/span + 10.459/px
  v6, handed an address and a jump displacement:       43.7/span +  9.152/px
  29 assumed                                           50.0/span +  9.080/px

So 29's arithmetic was right about a format nobody had written. The difference
is not tuning: v5 spends ~122 cycles a span computing a destination, dividing
npix into bursts and handling a 0..15 remainder, all of which the encoder knows
at build time. v6's record is {u32 absolute GVRAM address, u16 jump
displacement} into an unrolled chain of 24-pixel copy units -- no loop, no
remainder, no arithmetic -- and it fits 11 span lengths to 0.3%.

Three things that measurement showed and derivation could not:

  - The per-pixel cost is a function of REGISTER PRESSURE. FINDINGS 24's 9.08
    was a fixed blit with 12 registers free; v5 can spare 8 and pays 10.46; v6
    gets 12 back only because the encoder holds the state.
  - Short spans die in the remainder path -- a 12-pixel span costs MORE than a
    16-pixel one -- and the fix is padding, not avoidance.
  - Odd-x alignment is free (259.0 vs 261.8 cycles/span), as a 16-bit bus
    implies but nobody had checked.

Re-priced against the unchanged mode maps, sasi: median 74.4% -> 52.0% (29 said
43.0), misses 37 -> 10/120 (29 said 8), 448.0 KB/s. Break-even moved from runs
of 2 blocks to runs of 4. 29.4 survives: a scene cut needs x >= 0.196 of the
frame as spans and the bus allows x <= 0.373, so it fits at 12fps.

All 23 timing configs are also checked pixel-exact, so none of this was timed
against a decoder that quietly skipped work.

FINDINGS 30. Next: lever B, the cost-aware mode decision.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
 2026-08-23 15:45:51 -07:00
prosolis 3641f37e28 The bus is 4x idle while the CPU is pinned: price the trade 
The codec was designed when bytes were scarce, so every decision in it trades
cycles to save bytes. That is now backwards: sasi spends 110 KB/s of a 488 KB/s
pipe while missing 31% of frames on CPU.

The cheapest thing a 68000 can be handed is the most expensive thing to store.
Measured, per pixel: row-linear copy from word-expanded memory 9.08 cycles,
block-order 12.98, V1 codebook 18.74, RAW byte literals 25.03. So the 1024-byte
stride costs 43% and unpacking bytes to words costs more than the write itself.

Pricing one new mode -- a per-row span of word-expanded literals movem.l'd
straight from the stream buffer -- against the UNCHANGED mode maps:

  sasi   median 74.4% -> 43.0%, worst 136.2% -> 106.2%, misses 37 -> 8/120,
         101.7 -> 453.2 KB/s
  scsi   median 94.9% -> 69.4%, misses 51 -> 18/120, 272 -> 479.7 KB/s

scsi gains less precisely because it has less idle bandwidth left to trade.

Two consequences worth flagging. A word-expanded literal block derives to ~240
cycles, cheaper than V1's measured 299.9 and pixel-exact -- so every codebook
mode is CPU-dominated by a literal, and the codebook is a byte optimisation
that now costs cycles. And 28.5's "a scene cut cannot fit at 12fps" reopens:
CPU needs >=19% of the frame as spans, the bus allows <=39%, and that interval
is not empty.

DERIVED, NOT MEASURED, and labelled as such everywhere. The 9.08 cycles/pixel
is real but was measured at full row width with 12-register bursts, so short
spans are flattered. Measuring one span on the 68000 is now step 0 of the next
session, ahead of the cost-aware mode decision, because it changes the mode set
that decision optimises over.

FINDINGS 29. tools/analysis/12_span_tradeoff.py.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
 2026-08-23 15:28:29 -07:00
prosolis cb05e77a42 Handoff: reconcile the docs with a decoder that works and does not fit 
Session 7 handoff. check.sh green end to end, including both new stages.

STATUS now leads with the cost-aware mode decision, and the stale statements
that session 7 overturned are struck rather than left to be re-read as current:

- next-steps item 1 still announced "implement both display paths and pick per
  frame" as a settled answer. Withdrawn, with the reason, and the part that did
  hold up (report the distribution, not the mean) kept.
- the profile table's "machine" column reads as a CPU claim and is not one. It
  is the bus. Both profiles target the same stock 10 MHz 68000 and neither
  fits; the Super has SCSI at 10 MHz.
- the display-path section stopped at "68000 code copied a frame". It parses
  one now, and the blit figures are no longer the display-path budget.
- the green-light section still described a 2-minute, three-stage check, and
  did not warn that 10_pathmix_drift.py is SUPPOSED to exit non-zero.

Sharpened the one trap in the next session's first step: SKIP is not a
per-block constant. It costs 13.25 cycles inside an all-SKIP header byte and
~45 in a mixed one, so its price depends on its neighbours and a per-block
lagrangian cannot see that. Picking either number is wrong in a different
direction; the budget check can use the exact clustered cost function while
the per-block term only has to rank modes.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
 2026-08-23 15:21:12 -07:00
136 changed files with 34353 additions and 525 deletions
Expand all files Collapse all files
.gitignore
+2
View File
@@ -11,5 +11,7 @@ roms/
__pycache__/
*.pyc
*.dlx
*.dlxp
a.out
tmp/
tools/bench/c68k/c68k_bench
README.md
+465 -79
View File
@@ -1,90 +1,476 @@
# Dragon's Lair — Sharp X68000 port
# Dragon's Lair: Sharp X68000 port
Porting Dragon's Lair to a stock X68000 (68000 @ 10MHz, 2MB, SASI/SCSI).
Porting Dragon's Lair to a stock X68000 (68000 @ 10MHz, 2MB, SCSI).
This is fundamentally a **video codec problem**, not a game-logic problem: the
game logic is a scene table with branching input windows; the difficulty is
pushing ~22 minutes of Don Bluth animation through a 10MHz 68000.
The game logic is a scene table with branching input windows. The difficulty is
pushing ~22 minutes of Don Bluth animation through a 10MHz 68000, and it is a
**delivery** problem rather than a compression one — bus cycles and bytes off a
disc.
**Green-light check:** `./tools/bench/check.sh` (~3 min, needs the Blu-ray
mounted) re-runs both display regression tests, the rate-control drift test, the
display-path coherency counterexample and a 120-frame 68000 decode, then prints
`ALL GREEN`.
**The live design has no decoder in it.** The disc holds packed 8-bit frames and
a DMA channel walks them into GVRAM with the CPU halted. Measured on the machine,
decoding 37,585 bytes costs more than not decoding 49,152: at the 9 clk/B
dual-address floor the codec is 110.4% of a 12 fps frame and the decoder-free
packed player is 54.9%.
## Read first
- **`docs/FINDINGS.md`** — measured hardware facts, content statistics, codec
decision, and a section on measurement traps that produced three separate
false results. Read §4 before trusting any pipeline number.
- **`docs/STATUS.md`** — current state, working setup, blockers, next steps.
**Start here.** It also lists what has been explicitly abandoned, so old ideas
do not get re-proposed.
- **`docs/BENCHMARK.md`** — how to measure the storage subsystem, and why a
A working codec is in the tree and is **parked** (USER DECISION). Both branches
are described below; the packed one is where the work goes.
## What it looks like
**The packed player, running off a real volume, with sound.** `src/player/packed.s`
on an emulated stock X68000: the 68000 brings up its own display, builds its own
193-entry DMA chain, keeps its own frame clock off the CRTC's V-DISP, and fetches
every record itself with `READ(10)` from a real MB89352. **There is no decoder in
the machine.** Source on the left; on the right, MAME's own snapshots of what the
68000 put on screen. **120 of 120 frames are pixel-exact against the container**,
every one compared, and the clip refuses to build otherwise.
<video src="docs/img/packed-player.webm" controls muted loop width="100%"></video>
[`docs/img/packed-player.webm`](docs/img/packed-player.webm) (120 frames, 12 fps,
VP9, with the chip's own audio)
**The clip is a composite of two runs and that is the point.** The picture is the
gate run — paced at half rate so each snapshot lands inside the write window,
cycle stealing, no sound. The sound is the audio run — the same container at
12 fps with the MSM6258 on channel 3, captured by MAME off the speaker, cut at
the first sample the chip produced and gated sample-exact against lump 0 before
anything is written. **What it is not is a real-time capture of the shipping
configuration**, which at this container's own burst rate would show a blank
layer for 99.5% of every slot (FINDINGS 64.2, below).
![Blu-ray source next to the packed player's own screen](docs/img/packed-player.png)
The still is the frame whose PSNR is closest to the mean over the gated window,
so it is not the flattering one: **33.13 dB against the window's mean of 33.10**,
which is the packed container's predicted GRB555 figure to the digit.
**Name the layer.** Everything here is **emulated**: MAME 0.277 `x68000`,
`-bios ipl10`, stock 10 MHz / 2 MB, with every 68000 cycle figure cross-checked
on a second CPU core (px68k's C68K) and the display path cross-checked against
px68k's own `gvram.c`. **Nothing in this project has run on real hardware yet.**
## How the packed player works
**256-colour GVRAM throws away the high byte of every word a CPU writes**, so a
picture byte normally costs two disc bytes. **CRTC R20 bit 11 turns the masking
off**, and with the two 256-colour pages scrolled apart one word carries two
pixels (FINDINGS 46/47). That makes a frame 1.0 B/pixel — 49,152 bytes — and it
makes the frame a *literal*: no codebook, no recursion, no decode.
A record is that picture plus its own 256-entry palette: **49,664 bytes, which is
97 sectors exactly**. A packed record's length is geometry, so **the container
carries no index and no length word** — record *i* is at `off + i*rec` and a seek
is arithmetic. A DMA channel fills GVRAM in buffer mode straight off the disc and
**walks the 1,024-byte line stride itself** through array chaining, so a frame is
**one channel start and not 192** — and the chain crosses from the palette
registers at `$E82000` into GVRAM in the same start (FINDINGS 62). The array is
scene-constant, because the packed layout spends both 256-colour pages and there
is no page to flip.
**Audio rides a fixed cadence, not the record.** The MSM6258 wants 7,812.5 B/s
and 12 fps wants 651.0417 B a slot, a ratio with no arithmetic relationship, so
audio in the record would make records variable-length and bring the index back.
DLXP2 groups instead: **one 14-sector audio lump, then 11 records**, so
`record i = off_frm + i*rec + (i//11)*7168` — six instructions once a frame. The
lump is 7,168 bytes of space and eleven frames of audio is 7,161.4583…, so **the
payload alternates 7,161 and 7,162** and a player that fed the chip whole lumps
runs 0.09% fast: **1.25 s of lip-sync drift over the game's 22.8 minutes**,
predicted, and played at 1.26 (FINDINGS 67.2/68.3). The player carries one
accumulator instead, which is the frame clock's shape for the frame clock's
reason.
**The wire is fixed and there is no lever on it: 582.0 KB/s of picture +
7.64 KB/s of audio cadence = 589.6 KB/s.** A codec's bitrate is adjustable; a
literal frame's is geometry, and no scene costs less than another.
| | |
|---|---:|
| record | 49,664 B = 97 sectors |
| wire | **589.6 KB/s** |
| picture, as the display renders it | **33.10 dB** mean over the gate window |
| CPU per frame, video path | one channel start and one `READ(10)` — no paint |
| cost at the 9 clk/B dual-address floor | **54.9% of a 12 fps frame** (FINDINGS 64.2) |
| the codec, at the same floor | 110.4% (FINDINGS 61.4) |
## Where it stands
**What runs, end to end, on the emulated machine off a real volume:**
- **120 of 120 frames pixel-exact**, in both palette orders, every frame compared
rather than the last — a packed frame is a literal, so frame 119 says nothing
about frame 60 (FINDINGS 64.1).
- **All 78,125 B of a container's audio, read back off the speaker**, sample-exact
against the four ADPCM axes carried in the container's own header, one
delivered byte at a time — because a player's counters all stay right through
a bug that overwrites the buffer the channel is reading (FINDINGS 68.1).
- **A mid-stream seek with sound on it**: `pg_aseek` rebuilds the lump index, the
stream position, the remainder accumulator and the byte offset into the group
and issues a second read — **132,162 B of spliced stream accounted for byte by
byte** across a branch deliberately not on a group boundary (FINDINGS 71.1).
- **The scene graph**: 40 scenes, 516 sequences, 906 input windows, imported from
the arcade's own graph into this project's schema (FINDINGS 56).
**The binding resource is the 68000's local BUS, not its clock**, and every item
above is priced in one of four units:
| resource | state |
|---|---|
| **68000 local bus** | the binding one. The codec's decoder occupies 86.7% of it, and 52 of the 53 frames that miss the 12 fps budget miss on the bus (FINDINGS 38). |
| **68000 clocks** | measured, on two independent cores. |
| **delivery rate** | **no working figure, deliberately** (FINDINGS 50, USER DECISION). `--bus`, `--kbps` and `DLX_STREAM_KBPS` are required arguments with no defaults, so no table can be scored against a rate its own output does not state. |
| **seek time** | **no figure at all, and never had one.** |
| **W, clocks stolen per delivered byte** | 5 single-address held, 9 dual held, 12 single arbitrated, 16..19 for the IPL ROM's own disk channel. **Still the largest open number.** The CPU doing the transfer itself is **87.28 clocks a byte, measured** — 4.6x the worst DMA configuration found here and 17.5x the best (FINDINGS 58). |
### The open question that decides which player gets built
**The write window is the frame.** A packed write *requires* R20 bit 11; buffer
mode blanks the layer while it is set; and a DMAC-direct player holds the window
open for the whole data phase, because there is no second page to hide behind.
dark fraction of a slot = record bytes / (DATA-PHASE rate x slot)
**The rate in that expression is the BURST rate, not the sustained one**, and at
the container's own 582.0 KB/s the dark fraction is **1.0**: every frame
delivered, on time, pixel-exact, and none of them displayed. Free-running, the
run reported exactly that — **the window was open on 99.5% of host frames**.
It **reverses the ranking** in clocks, so there are two packed players and the
difference between them is *when* the window is open:
| | clocks (W=9) | on screen | RAM |
|---|---:|---:|---:|
| **A — DMAC-direct** (built, FINDINGS 64) | **54.9%** of a slot | 0% at the container's wire, 72.7% only at 2,131 KB/s | none |
| **B — DMA to RAM + CPU paint** (K4, not built) | 82.2% | **72.7% at any rate** | 99,328 B |
B's paint is **measured**: the packed `movem` blit is 227,553 clocks, 27.3% of a
slot, independent of the medium. They are equally visible only at **3.7x the
wire**. So B2 — *does buffer mode blank the display?* — decides which player
exists rather than how much headroom one has, and **K4 is not built until it is
answered** (FINDINGS 64.2, ROADMAP K4).
### What else a player has to carry
**A held channel costs the frame clock half its ticks, without the clock being
able to tell.** `clock.i` counts V-DISP interrupts; a held channel halts the
68000; the MFP's pending bit is one bit. Held at 12 fps, **487 of 1,038 edges are
lost** and the player reports **zero late frames**, because the tick it grades
itself against is advanced by the interrupt the channel stopped it from taking.
It believed it was at 12 fps; the screen was at **6.37** (FINDINGS 64.3).
**Held, the bus costs the audio 463 times the seam.** The MSM6258 has no
starvation state — it goes on decoding nibbles out of whatever byte its data
register still holds — so the interval between a channel counting out and the CPU
arming the next lump is a held nibble pair driving the predictor. Stealing, that
seam is **0.51 ms over ten seconds**, because the audio service runs from inside
`dma.i`'s transfer wait. Held, it is **236 ms, 2.31% of the audio**, with every
byte still correct (FINDINGS 68.2).
**The chip is not the datasheet and not ffmpeg, and it was asked.** Sixteen
candidate decoder models were fitted to what the emulated MSM6258 produced from a
designed nibble stream; **exactly one reproduces it sample-exact**, with a
negative control on every axis. The chip runs the datasheet's truncation, takes
the **low** nibble first, clamps its accumulator at **10 bits** and starts it at
**−2**. Getting the delta formula wrong costs −2.88 dB; **getting the nibble order
wrong costs −25.74 dB** (FINDINGS 66). The four axes ride in the container header
as fields rather than a version number.
**The audio level is measured off the whole disc and does not change.** All 201
streams that have audio, 21.5 minutes: the **disc peaks at 946 of 2048, 5.35 dB
over the chip's clamp**, in 402 events totalling 44.0 ms. Forty windows encoded at
six gains price the choice, and the disc's own level has the **best mean SNR
(22.03 dB)** — the gain that guarantees zero clamping costs 0.85 dB across the
game to buy back 1.90 dB on the 2.11 seconds that clamp, because the OKI step
table's floor is a constant and does not scale (FINDINGS 69).
**The predictor does not seek.** The MSM6258's accumulator is a pure integrator
with no leakage term, so a branch that hands the chip bytes chosen for a state it
is not in produces **a DC offset that does not decay**: playing through, −355 of
511 with AC 0.00, still −108 four seconds later; stopping and re-PLAYing, a single
permanent constant of −65. **A re-PLAY is 5.5x better and neither is zero.** The
only fix that reaches zero is the encoder's — reset the predictor where a branch
can land — and it costs **0.33 dB** (21.99 → 21.66) because the step table
re-converges in a few samples (FINDINGS 71).
**Branch points do not wait for the buffer.** 5.4% of the game's 612 branch
transitions open an input window on the first frame of a clip the disc *seeked
to*, so two seeks can fall back to back with no play between them. On the packed
branch there is **no refill climb at all** — a record goes straight into GVRAM and
the video lookahead is zero records — so the only consumer with any lookahead is
the audio one: **1.833 s of sound held against 0.000 s of picture** (FINDINGS
56/70.2). And a DLXP2 group puts its lump *in front* of its records, so a seek
lands with its audio behind it: **mean 416.5 ms of silence entering a branch**
over the arcade's 409 within-container targets, which is what `pg_aseek`'s second
read removes for 11.7 ms (FINDINGS 70.3).
### The parked branch: the codec
The codec is a Cinepak-style hybrid — each 4x4 block coded as SKIP, one 4x4
codeword, four 2x2 codewords, or RAW literal pixels, chosen per block by
rate-distortion, with a v7 literal-span mode over the top. It works, it is
measured, and it is **pixel-exact on the 68000 under two independent CPU cores**;
`stream.s` decodes a 120-frame window out of a 256 KB ring on a stock 2 MB
machine with the container in a file rather than in RAM.
<video src="docs/img/player.webm" controls muted loop width="100%"></video>
[`docs/img/player.webm`](docs/img/player.webm) — the codec player, 119 frames,
12 fps. 116 are pixel-exact against the reference decoder; **three are torn**,
frame *n* on top of frame *n-1*, because MAME captured the screen while the block
loop was partway down it. `decode.s` writes straight to the displayed page, so a
real player tears the same way, and the media builder **asserts the tear** rather
than trimming it. [`docs/img/modes.webm`](docs/img/modes.webm) is the same window
with the block-mode map beside it — black SKIP, blue V1, amber V4, red RAW —
which is what every cost table in FINDINGS is really about.
**Why it is parked.** Its remaining path is a conjunction and the packed one is
not: reaching a fitting rate needs a 35% byte reduction, two of its three levers
measure inert, and the reward on success is a design at ~100% of the frame
against the packed player's 54.9%. **It is kept on disk and not built on** — if
B2 comes back "buffer mode blanks", it is the only thing left (FINDINGS 61.8,
48.3).
## What is open
**Hardware — this list is the user's, and nothing here can be settled by an
emulator.**
- **B1. Measure the medium.** Three thresholds, not one: **sustained ≥ 589.6 KB/s**
or frames arrive late; **the data-phase BURST rate**, which decides how much of
the slot the picture is on screen; and **seek time**, which has no figure at
all. Plus what one extra SCSI command costs, which decides the audio cadence.
The 0.7–1.7 MB/s usually quoted for BlueSCSI on an X68000 is **folklore with no
published benchmark behind it**.
- **B2. Does buffer mode blank the display?** `probe_bit11_blank.lua` is written
and settles it in minutes on a real board. It decides A vs B above.
- **B3. Does a real CZ-6BS1 drive `#EXREQ`?** MAME's card has no request line to
the DMAC at all, so every configuration that can be run against it is
auto-requested and **charged by time rather than by byte**. A real request line
is what single-address 5 clk/B needs.
- **B4. Does a real palette register take a byte write?** A negative answer costs
0.28% of a frame and nothing else.
- **The MSM6258V, on silicon**: the four axes; whether it resets accumulator, step
index and nibble select on PLAY only when it was not already playing; and
whether it goes on asserting `#DRQ` while STOPped.
**Software, in order.**
1. **The predictor-reset container (DLXP3).** The cheapest thing that takes a
measured cost to zero rather than down. The player half already exists
(`PG_ARST`).
2. **The audio buffering depth.** `PG_ANBUF` is 3; two slots is one constant and
one run. The audio buffer is the packed branch's *only* buffer.
3. **The silent-clip and short-audio cases**: a scene whose audio is shorter than
its frames, and a scene with no audio track at all.
4. **The cadence pick.** With a working seek path the silence F=11 costs is zero,
so the trade is padding against RAM — plus one SCSI command per branch, which
is B1's.
**Parked, so it is not re-proposed:** the codec's remaining encoder work (E7, E4,
C1); `ring.i`, `xfer.i` and most of `stream.s`, which a DMAC-direct player has no
use for because it has no ring, and P4a's wiring with them. **K4 is blocked
on B2**, not parked.
## Reproducing this
**No media ships in this repo and none of it is redistributable.** Bring your own
Dragon's Lair Blu-ray. Everything else needed to rebuild every number and every
picture above is either here or is packaged.
| | |
|---|---|
| the disc | loop-mounted read-only: `udisksctl loop-setup -r -f DRAGONS_LAIR.iso`. Built against a decrypted UDF 2.x image; 7-Zip cannot read UDF 2.x, so use the loop mount |
| `python3` | plus **numpy** and **Pillow**, and nothing else. The k-means is hand-rolled rather than pulling in sklearn |
| `ffmpeg` / `ffprobe` | frame extraction, and the clips above |
| **MAME** | tested on 0.277, with the `x68000` ROM set. The rigs drive it headless via `-autoboot_script` |
| vasm (m68k, Motorola syntax) | **vendored**: `tools/vasm/vasmm68k_mot` is a Linux x86-64 binary, with the source tarball beside it to rebuild elsewhere |
```sh
export DLX_BDROM=/path/to/your/mounted/bluray # if not /media/$USER/BDROM
./tools/bench/check.sh # the green light, prints ALL GREEN
```
`check.sh` re-runs everything above that a host can re-run: both display
regression tests, the rate-control drift gate, the display-path coherency
counterexample, a 120-frame 68000 decode on two CPU cores, the ring and paced-ring
passes, the DMAC configuration gate, the load-time transforms on both cores, the
scene-graph import when a DirkSimple checkout is present, the packed container
(with audio, and a silent control beside it) rendered through px68k's own GVRAM
model, **the packed player for 120 frames off a real volume with every frame
compared**, the audio encoder against ffmpeg's decoder, the sixteen-way decoder
identification against the emulated chip, the DLXP2 container and each of its four
axes with a negative control on each, the audio level off every stream of the
game's own footage, and the refill climb against the arcade's own 612 branch
points. `DLX_BDROM` is honoured by every tool that reads the disc.
Two stages **skip rather than fail** when their input is absent, because both live
outside this repo: `PX68K=/path/to/px68k` for the second-CPU-core gate — the
cheapest strong test in the tree, and what licenses the bus and cycle figures —
and `IPLROM=/path/to/iplrom.dat` for the DMAC configuration gate.
The player runs and the media are rebuilt from them:
```sh
bash tools/bench/packed_run.sh # the packed player, 7 runs
python3 tools/media/make_packed_media.py # docs/img/packed-player.{png,webm}
python3 tools/media/make_readme_media.py <c.dlx> # the parked codec's stills and clips
```
Both media builders **gate before they write**: a still or a clip of the player is
a claim that the player drew it, so every frame is checked pixel-exact against the
container first and the audio cut is checked sample-exact against lump 0. A README
that illustrated a pixel-exact player with an approximate picture would be a small
lie about the one property this project keeps testing.
**Scene selection is a hard-coded stream number, not a search.** The gates use
streams `00020` and `00223` of the disc's 224 `.m2ts` files. A different pressing
may number them differently, and if so the green light will extract the wrong
footage rather than fail, so check that `tmp/fr_singe/` looks like the Singe
encounter before trusting any figure. **Not every large stream is game footage**:
`00216` is the feature with a burned-in commentary picture-in-picture and `00215`
is the commentary itself, the two largest files on the disc. The clean
9.4-minute animation is **`00223`** (FINDINGS 25.1).
**Nothing outside-derived is committed here.** The scene graph is regenerated from
a reader's own clones into gitignored `tmp/`, and `tools/import/scenegraph.py` is
the single file in the repo coupled to those projects — everything downstream reads
`DLXSCENE1`, this project's own schema, with the sources' attribution carried in
it. DirkSimple is zlib (Ryan C. Gordon); the SNES chapter set is MIT (Chad
Doebelin) and, by its own README, *derived* from DirkSimple rather than an
independent transcription.
## Encoders
**The live one is the packed container**, and it has no rate control in it because
there is no rate lever:
```sh
# the gate window: stream 00223, 10.0 s from 539.4 s, the same seconds both times
python3 tools/encoder/extract.py 00223 tmp/fr_singe 12 crop 539.4 10.0
python3 tools/encoder/extract_audio.py 00223 tmp/au_singe.raw 15625 539.4 10.0
python3 tools/encoder/pack.py tmp/fr_singe out.dlxp --audio tmp/au_singe.raw
```
`dlxp.py` is the format and the one place the interleave, the 97-sector record,
the audio cadence and the lump payload are stated. It encodes in ~3 seconds
because there is no k-means in it. `--scene-palette --no-palette` is the fallback
container: **30.79 dB, zero palette churn, 576.0 KB/s**, still ahead of the codec
as the display renders both, and it is what ships if B2 says the layer blanks —
because a per-frame palette changes **231 of 256 entries every frame**, and a
picture under the neighbouring frame's palette is **12.8 dB worse**
([`docs/img/palette-mismatch.png`](docs/img/palette-mismatch.png)).
`adpcm.py` is the MSM6258 codec and it carries **two decoders on purpose**: the
module defaults are ffmpeg's, so `tools/bench/verify_adpcm.py` stays a check
against an independent implementation, and `adpcm.CHIP` is the set measured out of
the machine's own chip. Anything that encodes *for* the machine passes `CHIP`
explicitly.
**The parked codec's encoder is kept and still runs:**
```sh
python3 tools/encoder/encode.py tmp/fr_singe out.dlx --profile scsi --preview p.png
```
`--kbps` is the quality rate point and `--span-kbps` the ceiling the span pass may
draw on; `mu` is bisected per frame against the 68000's own decode budget so a
frame also *decodes* in time. One profile, `scsi`; the 110 KB/s `sasi` profile was
dropped on capacity rather than bandwidth — a SASI volume is 40 MB and the
game is 146 MiB even at that rate. The RAW escape means `lam=0` is
pixel-exact against the palettised frame. See FINDINGS 31/41/44 before changing
any of it.
## Documentation
- **`docs/STATUS.md`** is the current state, working setup, blockers and next
steps, newest session first. **Start here.** It also lists what has been
explicitly abandoned, so old ideas do not get re-proposed.
- **`docs/ROADMAP.md`** is the remaining work to a completion target. Read it with
STATUS rather than instead of it: STATUS holds the measurements, ROADMAP holds
the shape and goes stale first.
- **`docs/FINDINGS.md`** is measured hardware facts, content statistics, the codec
decision, and a section on measurement traps that produced three separate false
results. Read §4 before trusting any pipeline number. It is append-only and
later sections overturn earlier ones; superseded sections carry a blockquote
pointing at the correction.
- **`docs/BENCHMARK.md`** is how to measure the storage subsystem, and why a
bandwidth figure out of MAME would be meaningless.
- **`docs/HARDWARE.md`** — X68000 GVRAM/CRTC reference.
- **`docs/HARDWARE.md`** is the X68000 GVRAM/CRTC reference.
## Layout
```
docs/ findings, status, hardware reference
tools/analysis/ measurement scripts, numbered in the order they were written
(01/02 marked BROKEN deliberately, kept as regression refs).
Run from the repo root — they import from tools/encoder/.
07 finds the hottest sustained window in a stream; 08 renders
source | decoded | block-mode map as .webm; 09 is the
rate-control drift gate (FINDINGS 26/27) and is part of
check.sh -- it exits non-zero if the encoder ever again
reports a reconstruction no decoder would produce.
10 is a COUNTEREXAMPLE, and exits non-zero by design: it
demonstrates that the two-display-path plan of FINDINGS
24.5/25.6 corrupts 70 of 120 frames (FINDINGS 28.1).
11 scores a container against the MEASURED per-mode block
costs without needing MAME.
tools/bench/ MAME Lua injection harness + 68000 benchmark sources.
`check.sh` re-runs both display regression tests (~40 s).
`blit.s`/`blit.lua` time the full-frame GVRAM blit on the
68000 itself (FINDINGS 24) — not part of check.sh, because
wall timings would make the green-light check host-sensitive.
`crtc_mode.lua` is the single source of truth for CRTC R00-R08
and R20 — do not write CRTC values anywhere else.
`prep_dlx.py`/`decode.lua`/`verify_decode.py` load, time and
verify `src/player/decode.s`; the verify pass is in check.sh.
tools/vasm/ vasm m68k assembler (built from source)
tools/encoder/ hybrid VQ encoder + DLX1 container writer (working).
dlx.py is the reference DECODER -- ground truth for the 68000.
src/player/ decode.s: the 68000 DLX1 decoder. Pixel-exact, and 31% of
frames over the 12fps CPU budget. See FINDINGS 28.
assets/ extracted frames/audio (gitignored)
```
## Encoder
```
python3 tools/encoder/extract.py 00020 /tmp/fr 12 crop
python3 tools/encoder/encode.py /tmp/fr out.dlx --profile sasi --preview p.png
src/player/ packed.s THE LIVE PLAYER: display bring-up, the 193-entry DMA
chain, its own V-DISP clock, READ(10) off a real
volume, and the MSM6258 on channel 3 -- a lump ring,
the remainder accumulator that stops the padding
becoming drift, a service routine that runs from
INSIDE dma.i's transfer wait (0.51 ms of seam
against 236), and pg_aseek.
dma.i the HD63450 channel: array chaining, held and
stealing, and DM_HOOK.
scsi.i the MB89352: selection, READ(10), the data phase.
adpcm.i the audio transport, every register byte of it
decoded out of the IPL ROM rather than recalled.
clock.i the frame clock, off the CRTC's V-DISP.
geom.i the constants, in one place.
decode.s the parked codec's 68000 decoder (1,296 bytes,
stream.s asserted), the same decoder behind a bounded ring,
ring.i the ring producer, the transport under it, and the
xfer.i load-time codebook/palette transforms. OUT OF THE
load.i VIDEO PATH: a DMAC-direct player has no ring.
frame.i the block loop and span chain, included by both, so
there is exactly ONE copy of the bytes every cycle
constant is fitted to. The span pass is blit.s v7
verbatim -- do not tidy it.
tools/encoder/ dlxp.py THE LIVE CONTAINER (DLXP2) and pack.py writes it.
pack.py Nothing is shared with the codec's writer on purpose.
adpcm.py the MSM6258 codec, carrying ffmpeg's decoder and the
chip's measured one side by side.
dlx.py the codec's container and its REFERENCE DECODER,
encode.py ground truth for the 68000; the encoder, rate
spans.py control, and the v7 span geometry. Parked, kept.
extract.py frames and audio off the disc.
tools/bench/ check.sh the green light.
packed_run.sh the packed player: gate, rate, audio, held, seek.
verify_packed.py / verify_packed_audio.py
the two gates that matter. The audio one reads the
SPEAKER, one delivered byte at a time, because every
counter in the player stayed right through a bug
that overwrote the buffer the channel was reading.
crtc_mode.lua the SINGLE SOURCE OF TRUTH for CRTC R00-R08 and
R20. Do not write CRTC values anywhere else.
probe_bit11_blank.lua B2, ready to run on a real board.
c68k/ a SECOND emulator for every 68000 cycle figure:
px68k's CPU core, no SDL, no ROMs. It also counts
BUS cycles, which MAME cannot report.
gvpack/ the same argument for the DISPLAY: px68k's own
x68k/gvram.c, so the address decode, the R20 bit-11
write path and the scroll wrap are its code and not
a model of it.
blit.s the full-frame GVRAM blit, timed on the 68000
itself. V8 is the packed one: 227,553 clocks.
tools/analysis/ measurement scripts, numbered in the order they were written.
Run from the repo root. 01 and 02 are marked BROKEN
deliberately and kept as regression references; 10 is a
COUNTEREXAMPLE and exits non-zero by design, demonstrating that
the two-display-path plan corrupts 70 of 120 frames. 15 is the
bus occupancy model, 19 the ring's ADDRESSES (not its
occupancy), 21 the IPL ROM's DMAC configuration, 25 the scene
graph, 30/34 the packed containers, 31 the display duty, 35 the
audio level off the whole disc, 36 the climb against real
branch points, 37 what a branch costs the chip. buscost.py is
the shared bus-cycle table; the per-block constants live in
vq_hybrid.py and are imported, never copied.
tools/import/ scenegraph.py -- the ONLY code in this tree coupled to somebody
else's source. Output is gitignored derived data.
tools/media/ make_packed_media.py builds the live player's still and clip;
make_readme_media.py builds the parked codec's. Both gate
before they write.
tools/vasm/ vasm m68k assembler, binary plus source tarball.
docs/img/ the stills and clips above, built from real emulated runs.
assets/, tmp/ extracted frames, audio and run output (gitignored).
```
Two quality profiles ship from one codec and one decoder — `sasi` (110 KB/s) and
`scsi` (280 KB/s) are two points on the same rate-distortion curve. Both are
**ceilings**: lam is bisected per frame under a leaky bucket, so the profile's
`lam` is a quality floor rather than a setting (`--fixed-lam` opts out). The codec is
a Cinepak-style hybrid: each 4x4 block is coded as SKIP, one 4x4 codeword, four
2x2 codewords, or RAW literal pixels, chosen per block by rate-distortion.
The RAW escape means `lam=0` is pixel-exact against the palettised frame, so the
quality knob spans lossless to heavily-compressed without changing the bitstream.
Profiles are derived from a bandwidth figure, not chosen by eye:
```
python3 tools/encoder/profile_gen.py --bw-mbps 4 --name scsi
```
> **On reading `docs/FINDINGS.md`:** it is append-only and several later sections
> overturn earlier ones. Superseded sections carry a blockquote at the top
> pointing to the correction — heed those, especially 18 (reversed by 21).
Source media (`DRAGONS_LAIR.iso`) and ROMs are gitignored — supply your own.
**Not every large stream is game footage.** `00216` is the feature with a
burned-in commentary picture-in-picture and `00215` is the commentary itself —
the two largest files on the disc. The clean 9.4-minute animation is **`00223`**.
See FINDINGS 25.1 before running any size-ranked survey.
Source media (`DRAGONS_LAIR.iso`) and ROMs are gitignored. Supply your own.
docs/BENCHMARK.md
+23 -5
View File
@@ -53,13 +53,31 @@ was used. **Do not record KB/s and treat it as a hardware figure.**
Already partly in FINDINGS 5. Bounds worth tightening from datasheets:
- 68000 bus cycle: 4 clocks @ 10MHz, 16-bit => **5 MB/s** absolute ceiling
- HD63450 single-address DMA, ~8 clocks/word => **~2.5 MB/s** practical ceiling
- HD63450 single-address DMA, **5 clocks/BYTE** => **2.0 MB/s** practical ceiling
(dual-address is 9 clocks/byte => 1.11 MB/s). CORRECTED session 14: this line
read "~8 clocks/word => ~2.5 MB/s", which charged a byte-wide SPC per word.
FINDINGS 43.
- SCSI-1 asynchronous REQ/ACK handshake per byte, plus MB89352 FIFO depth
=> the real limiter, and the number we do not have from a primary source
The user's working figure is **4 Mbps = 488 KB/s**, which sits sensibly between
the derived DMA ceiling and observed period-drive rates. **Provenance not yet
recorded — worth pinning down, because every profile now hangs off it.**
**RETIRED, session 18 (USER DECISION).** This document used to name a working
figure of "4 Mbps" here and note that every profile hung off it. It was never a
bus measurement — user-supplied, no provenance, and 10% of SCSI-1's asynchronous
rating (FINDINGS 42.1). It has been removed as a default from every analysis
tool and from `tools/bench/stream.lua`; the tools now REQUIRE an explicit rate,
so nothing can be scored against a figure the scorer never restates.
**There is no working delivery figure. That is the honest state, and it is the
point:** the rate is a property of the medium, the medium is a BlueSCSI, and it
has not been measured. `tools/analysis/19_ring_stream.py` reports the
**zero-prefill pipe** — the rate a medium must clear for a given container to
need no prefill at all — which is the threshold a measurement should be taken
against. For the session-14 candidate that is **513.2 KB/s** (FINDINGS 49.5).
One place still carries the old number: `GATE_SPAN_KBPS` in
`tools/bench/check.sh`, because the gate container was *encoded* with it and
every per-block and span constant in FINDINGS 41/43/45/49 is fitted to that
container. It is a container recipe, not a claim about any medium.
### The coupling nobody had counted
Cycle-stealing DMA is not free DMA. At ~8 clocks per 16-bit word:
@@ -69,7 +87,7 @@ Cycle-stealing DMA is not free DMA. At ~8 clocks per 16-bit word:
| 110 KB/s | 4.5% | 42.8% |
| 250 KB/s | 10.2% | 48.5% |
| 450 KB/s | 18.4% | 56.7% |
| 488 KB/s | 20.0% | 58.3% |
| ~490 KB/s | 20.0% | 58.3% |
FINDINGS 5 concluded that because transfers are DMA, "streaming costs
essentially no CPU". **That is wrong.** It costs up to a fifth of the machine at
docs/FINDINGS.md
+6493 -4
View File
File diff suppressed because it is too large Load Diff
docs/ROADMAP.md
+1035
View File
File diff suppressed because it is too large Load Diff
docs/STATUS.md
+3512 -91
View File
File diff suppressed because it is too large Load Diff
docs/img/decoded-frame.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 64 KiB

docs/img/modes.webm
BIN
View File
Binary file not shown.
docs/img/packed-player.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 200 KiB

docs/img/packed-player.webm
BIN
View File
Binary file not shown.
docs/img/palette-mismatch.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 224 KiB

docs/img/player.webm
BIN
View File
Binary file not shown.
docs/img/source-vs-decoded.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 182 KiB

src/player/adpcm.i
+131
View File
@@ -0,0 +1,131 @@
; ---------------------------------------------------------------------------
; adpcm.i -- the MSM6258V, driven the way the machine's own ROM drives it.
; ROADMAP P6a, and it is P6b's transport arriving early rather than scaffolding.
;
; NOTHING HERE IS INVENTED. Every register value below is one that
; tools/analysis/21_iplrom_dmac.py decodes OUT OF THE IPL ROM's own bytes, at
; the addresses it prints: channel 3's DCR/SCR/MFC/CPR/DFC/DAR at $FF0C2E and
; the per-transfer OCR = $32 plus command $02 at $FF9A82. That is the one
; ADPCM path on this board that is known-correct because Sharp wrote it.
;
; WHY THIS FILE EXISTS AT ALL. Session 33 fed the chip from Lua and got
; silence, swept control 0..3 against port C 0..15, and stopped rather than
; guess (65.5). Two of the reasons are visible from here and neither is a
; register semantic anybody had to guess:
;
; * THE PPI'S PORT C IS NOT AN OUTPUT UNTIL IT IS TOLD TO BE. The ADPCM pan
; and the sample-rate divider are port C bits, and an i8255 in its reset
; state has every port an INPUT -- so a write to $E9A005 changes a latch
; nobody is reading and the pan never leaves wherever it was. Control word
; $92 (mode 0, A and B input, both halves of C OUTPUT) is what makes the
; other write mean anything.
; * AND FEEDING IT SLOWLY IS NOT FEEDING IT. The chip has no FIFO and no
; starvation state: it consumes a nibble every sample period out of whatever
; its data register last held, forever. A byte per host frame is not a
; quiet chip, it is the same two nibbles 130 times, which saturates in six
; samples. The feed has to be paced by the chip, which is what channel 3
; and its request line are FOR.
;
; THE CLOCK IS TWO WRITES AND THEY ARE IN DIFFERENT DEVICES. 15,625 Hz is
; 8 MHz / 512: the 8 MHz comes from CT1 in the YM2151's port register $1B, and
; the /512 from port C bits 3,2 = 10. Neither is readable, so the rate is
; verified from the OTHER end -- the capture's own sample count.
AD_CTRLR = $E92001 ; W: command R: status (bit7 = NOT playing)
AD_DATAR = $E92003 ; W: the byte the chip takes two nibbles from
AD_PLAY = $02 ; COMMAND_PLAY -- session 33's probes wrote $01,
AD_STOP = $01 ; which is COMMAND_STOP
PPI_PC = $E9A005
PPI_CTL = $E9A007
PPI_COUT = $92 ; mode 0, A/B input, BOTH halves of C output
PPI_RATE = $08 ; pan 00 = both, rate 10 = /512 = 15,625 Hz
YM_ADDR = $E90001
YM_DATA = $E90003
YM_CT = $1B ; CT1 in bit 1: 0 = ADPCM master clock 8 MHz
AD_DMAC = $E840C0 ; HD63450 channel 3 -- the ADPCM channel, and
A3_CSR = AD_DMAC+$00 ; the one the ROM points at $E92003
A3_CER = AD_DMAC+$01
A3_DCR = AD_DMAC+$04
A3_OCR = AD_DMAC+$05
A3_SCR = AD_DMAC+$06
A3_CCR = AD_DMAC+$07
A3_MTC = AD_DMAC+$0A
A3_MAR = AD_DMAC+$0C
A3_DAR = AD_DMAC+$14
A3_MFC = AD_DMAC+$29
A3_CPR = AD_DMAC+$2D
A3_DFC = AD_DMAC+$31
A3_DCRV = $80 ; XRM 10 cycle steal w/o hold, DTYP 00 dual
; address, DPS 0 8-bit port (IPL $FF0C2E)
A3_OCRV = $32 ; DIR memory->device, SIZE 11 byte unpacked,
; CHAIN 00, REQG 10 EXTERNAL REQUEST (IPL
; $FF9A82). External request is what makes the
; chip the pacemaker: one byte per #DRQ3, and
; #DRQ3 ticks at half the sample rate.
A3_SCRV = $04 ; MAC 01 memory increment, DAC 00 -- the device
; address is a REGISTER and must not walk
A3_CCRST = $80
; --------------------------------------------------------------- ad_setup
; The clock and the pan. No arguments, no result; trashes d0.
ad_setup:
move.b #YM_CT,YM_ADDR
moveq #60,d0 ; the YM2151 wants settling between the
.ymw: subq.l #1,d0 ; address write and the data write
bne.s .ymw
move.b #$00,YM_DATA ; CT1 = 0 -> ADPCM master clock 8 MHz
move.b #PPI_COUT,PPI_CTL ; ...and NOW port C drives something
move.b #PPI_RATE,PPI_PC ; pan both, /512
rts
; ----------------------------------------------------------------- ad_arm
; Arm channel 3 to feed (a1) for d1 bytes and start it. Trashes d0.
; The channel is started BEFORE the chip is told to play (see ad_play), so that
; byte 0 is already in the data register when the accumulator is reset.
ad_arm:
move.b #$FF,A3_CSR ; CSR is write-one-to-clear: a stale COC
; would pass the wait loop instantly
move.b #A3_DCRV,A3_DCR
move.b #A3_SCRV,A3_SCR
move.b #$05,A3_MFC
move.b #$05,A3_DFC
move.b #$01,A3_CPR ; the ROM's own priority: ADPCM outranks
; the disk at the arbiter (52.5 item 5)
move.l #AD_DATAR,A3_DAR
move.b #A3_OCRV,A3_OCR
move.l a1,A3_MAR
move.w d1,A3_MTC
move.b #A3_CCRST,A3_CCR
rts
ad_play:
move.b #AD_PLAY,AD_CTRLR
rts
ad_halt:
move.b #AD_STOP,AD_CTRLR
rts
; --------------------------------------------------------------- ad_abort
; STOP channel 3 where it stands, and clear what stopping it posts.
;
; A SEEK IS THE ONLY THING IN THIS PLAYER THAT NEEDS THIS. Every other re-arm
; happens at COC, where the channel has already counted itself out and there is
; nothing to stop; a seek arrives MID-LUMP, because a branch is a frame index
; and a frame does not know about the cadence (FINDINGS 70.3).
;
; SAB is CCR bit 4. CSR is then written $FF -- write-one-to-clear -- to take
; down COC and ERR together, because the abort posts a channel error (CER $11)
; and pg_aserv's whole test is "did the channel count out": a stale COC would
; make the very next service call arm a lump that is already playing.
;
; NAME THE LAYER. MAME's hd63450 does not NEED the abort -- its
; dma_transfer_start reloads MAR/MTC and restarts the timer whatever the channel
; was doing. The MC68450 does: STR written to an active channel is an operation
; timing error, and the transfer that is running is not the one that was asked
; for. The abort is here for the silicon, and the run below cannot tell.
ad_abort:
move.b #$10,A3_CCR ; SAB -- software abort
move.b #$FF,A3_CSR ; ...and the COC/ERR it posts
rts
src/player/adpcmgate.s
+112
View File
@@ -0,0 +1,112 @@
; Front-end for the ADPCM transport (ROADMAP P6a), for the rig.
;
; It plays ONE buffer of nibbles the host pushed into RAM and reports what the
; channel did. What is being measured is not this code -- it is the CHIP: which
; delta formula, which nibble of a byte first, where the accumulator clamps, and
; what it starts at. tools/bench/verify_adpcm_chip.py reads all four out of
; MAME's own -wavwrite capture.
;
; THE ONE THING THIS FILE HAS TO GET RIGHT is the order of the two starts. The
; chip resets its accumulator, its step index AND its nibble select when it is
; told to PLAY, and it begins consuming immediately out of whatever its data
; register holds. So the channel goes first and the CPU waits for MTC to move
; -- proof that a byte has actually been taken -- before the PLAY. The stream
; still has a prologue of unknown length, because the gap between PLAY and the
; NEXT #DRQ3 is not ours to set; the prologue is 16 zero nibbles for exactly
; that reason and the verifier reads its length off the capture.
include "src/player/geom.i"
AD_FLAG = $18600 ; u32 0 idle, 1 armed, 2 playing, $FF done, $EE error
AD_BUF = $18604 ; u32 where the nibble bytes are
AD_LEN = $18608 ; u32 how many bytes
AD_MTC0 = $1860C ; u32 MTC at the instant PLAY was written
AD_CSRF = $18610 ; u32 CSR at completion
AD_CERF = $18614 ; u32 CER with it
AD_MTCF = $18618 ; u32 MTC with it
AD_MARF = $1861C ; u32 MAR with it -- where the channel stopped
AD_SPIN = $18620 ; u32 trips round the wait loop
AD_STAT = $18624 ; u32 the chip's own status byte while playing
AD_PATIENCE = 60000000
AD_SETTLE = 60000 ; ~100 ms at 10 MHz, 18 clocks a trip ; the wait is bounded like every other
org $10000
start:
bsr ad_setup
; ---- SETTLE, and it is not superstition. The 8 MHz ADPCM clock is
; CT1 in the YM2151's port register, and this machine delivers that
; write to the ADPCM chip on the SOUND system's own schedule rather
; than at the instant of the store -- so a transfer started in the same
; breath as ad_setup plays its first ~17 ms at the PREVIOUS clock. The
; symptom is exact: the capture's first 130-odd samples come out in
; identical PAIRS, because the chip is clocking half as fast as the
; capture, and every model then fails to fit a stream that changed rate
; part way through. ~100 ms of nothing costs the gate nothing and a
; player sets its clock once at boot.
move.l #AD_SETTLE,d0
.settle:subq.l #1,d0
bne.s .settle
movea.l AD_BUF.l,a1
move.l AD_LEN.l,d1
bsr ad_arm
move.l #1,AD_FLAG.l
; ---- wait for the channel to actually take byte 0. Not a delay loop:
; the condition is MTC having moved, which is the channel's own account.
move.l #AD_PATIENCE,d3
.first: move.w A3_MTC,d0
andi.l #$FFFF,d0
cmp.l AD_LEN.l,d0
bne.s .go
subq.l #1,d3
bne.s .first
bra bad
.go: move.l d0,AD_MTC0.l
bsr ad_play
move.l #2,AD_FLAG.l
moveq #0,d0
move.b AD_CTRLR,d0 ; bit 7 clear = the chip says it is playing
move.l d0,AD_STAT.l
clr.l AD_SPIN.l
move.l #AD_PATIENCE,d3
.wait: addq.l #1,AD_SPIN.l
move.b A3_CSR,d4
btst #4,d4 ; ERR -- CER says which
bne.s bad
btst #7,d4 ; COC
bne.s .fin
subq.l #1,d3
bne.s .wait
bra.s bad
.fin: bsr report
; The chip is left PLAYING deliberately: it goes on replaying the last
; byte it was given, which the verifier ignores. Stopping here would
; put a silence in the capture at a point the host would then have to
; find, and the capture already has a length it knows.
move.l #$FF,AD_FLAG.l
hold: bra.s hold
bad: bsr report
move.l #$EE,AD_FLAG.l
bra.s hold
report:
moveq #0,d0
move.b A3_CSR,d0
move.l d0,AD_CSRF.l
moveq #0,d0
move.b A3_CER,d0
move.l d0,AD_CERF.l
move.w A3_MTC,d0
andi.l #$FFFF,d0
move.l d0,AD_MTCF.l
move.l A3_MAR,d0
move.l d0,AD_MARF.l
rts
include "src/player/adpcm.i"
src/player/clock.i
+202
View File
@@ -0,0 +1,202 @@
; ---------------------------------------------------------------------------
; clock.i -- the FRAME CLOCK, on the 68000 itself. ROADMAP P3.
;
; src/player/stream.s has a pace gate: frame i may not START before tick i, and
; PACE is the tick counter. Until now PACE was written by tools/bench/
; stream.lua, i.e. by the host, off the host's idea of what 12 fps means. That
; was honest for what FINDINGS 49/51 were measuring -- arrival times against a
; deadline -- and it is not a player. A player has no host. These are the
; bytes that replace it.
;
; WHAT THE MACHINE ACTUALLY OFFERS, because "use the MFP timer or vblank" hides
; a real constraint. The MC68901's timer clock on this board is 16 MHz / 4 =
; 4 MHz (MAME 0.277 src/mame/sharp/x68k.cpp:1027-1028), its prescaler ladder is
; {4, 10, 16, 50, 64, 100, 200} (src/devices/machine/mc68901.cpp:173) and its
; data register is 8 bits. The SLOWEST tick a single MFP timer can produce is
; therefore 4e6 / (200*256) = 78.125 Hz, and 4e6/12 = 333,333.33 is not even an
; integer -- so no prescaler/data pair ticks at 12 Hz, and no timer at any
; setting ticks as slowly as a 12 fps frame. A frame clock needs a divider in
; software whichever source it is built on. tools/analysis/23_frame_clock.py
; enumerates the whole space rather than asserting this.
;
; So the source is the RASTER, and that is a better answer than a timer anyway.
; GPIP4 on the MFP is V-DISP (x68k.cpp:1139, `m_crtc->vdisp_cb().set(i4_w)`),
; high while the display is active; the same signal is the MFP's Timer A event
; input (mc68901.cpp:167, GPIO_TIMER = {GPIP_4, GPIP_3}). Its interrupt is
; channel 6, IR_GPIP_4 = $40 in IERB/IPRB/IMRB (mc68901.cpp:76). We take the
; FALLING edge (AER bit 4 = 0), which is the start of vertical blanking -- the
; instant a player would present a finished frame, so the clock and the flip
; are the same event rather than two events with a phase between them.
;
; THE DIVIDER IS EXACT, AND IT IS EXACT BY CONSTRUCTION. The raster is
;
; 31,500 lines/s / (R04 + 1) lines/frame
;
; and 31,500 / 568 = 55.4577 Hz is not a multiple of 12, so a whole-number
; divide cannot do it: 4 refreshes is 13.87 fps and 5 is 11.09 fps. Instead
; each V-DISP adds `fps * (R04+1)` to an accumulator and a frame tick is emitted
; whenever it reaches 31,500, keeping the remainder:
;
; acc += fps*VTOTAL ; if acc >= HFREQ: acc -= HFREQ ; PACE += 1
;
; Over VTOTAL/gcd raster frames that emits exactly fps*VTOTAL/gcd ticks, so the
; long-run rate is fps*VTOTAL/VTOTAL = fps EXACTLY, with a bounded remainder and
; ZERO accumulated drift -- not 12.0001, not 11.9998. It holds for any fps and
; any vertical geometry, which is why the two constants are READ OUT OF THE
; CRTC at init rather than assembled in: the clock is derived from the same
; registers that generate the raster it is counting, so the two cannot disagree.
;
; WHAT IT COSTS IN CADENCE, WHICH IS THE PART THAT IS NOT FREE. 12 fps on a
; 55.4577 Hz raster is 4.6215 refreshes per frame, so a frame is shown for
; either 4 or 5 refreshes -- 72.13 ms or 90.16 ms. Nothing can change that;
; it is the display's quantisation, not the clock's error, and a timer-derived
; clock would have exactly the same cadence with an arbitrary phase against the
; raster on top. It does mean the slot a frame gets is NOT always the 83.33 ms
; every budget in this project is priced against, and the short slot is 13.4%
; under it. tools/analysis/23_frame_clock.py prices that; do not read this file
; as a claim that the clock made the budget bigger.
;
; INTERRUPTS, AND WHAT HAD TO BE TURNED OFF. The rigs launch the 68000 at
; SR=$2700 with everything masked, into a machine the IPL ROM has already booted
; -- so the MFP arrives with whatever IOCS enabled on it (keyboard receive,
; Timer C, its own V-DISP handler) and vectors pointing into IOCS. Lowering the
; mask without disarming the MFP would vector into code we did not put there.
; clk_init therefore writes IERA = IERB = 0 first, which on the MC68901 also
; clears the matching pending bits (mc68901.cpp REGISTER_IERA/B: `m_ipr &=
; m_ier`), and only then arms GPIP4 alone. Levels 1-5 stay masked at SR=$2500,
; so the DMAC (IRQ3) and the SCC (IRQ5) cannot get in either; level 7 is the
; front-panel NMI and is not ours to mask.
;
; The vector is the MFP's own: VR is written with the S bit CLEAR, so the
; in-service register is not used and an acknowledge clears the pending bit by
; itself (mc68901.cpp get_vector). No end-of-interrupt write in the handler.
; ---------------------------------------------------------------------------
; --- MFP registers. The device sits on D0-D7 of a 16-bit bus (x68k.cpp:793,
; `.umask16(0x00ff)`), so register n is one BYTE at $E88001 + 2n.
MFP = $E88001
MFP_GPIP = MFP+0*2
MFP_AER = MFP+1*2
MFP_DDR = MFP+2*2
MFP_IERA = MFP+3*2
MFP_IERB = MFP+4*2
MFP_IPRA = MFP+5*2
MFP_IPRB = MFP+6*2
MFP_ISRA = MFP+7*2
MFP_ISRB = MFP+8*2
MFP_IMRA = MFP+9*2
MFP_IMRB = MFP+10*2
MFP_VR = MFP+11*2
MFP_GPIP4 = 4 ; bit number of V-DISP in GPIP/AER/DDR
MFP_IVDISP = $40 ; IR_GPIP_4, channel 6, in IERB/IPRB/IMRB
MFP_VBASE = $40 ; vector base; S clear -> no in-service register
CLK_VEC = (MFP_VBASE+6)*4 ; $118: MFP channel 6 vector, as an ADDRESS
CRTC = $E80000
CRTC_R04 = CRTC+4*2 ; V total, in scanlines, minus one
CRTC_R20 = CRTC+20*2 ; mode; bit 4 = 31.5 kHz
HFREQ = 31500 ; lines/s in the 31.5 kHz modes. Exact: the
; 768-wide IPL mode is 34.776 MHz / 1104 dots
; and the 256-wide mode 11.592 MHz / 368, both
; 31500.0 (tools/bench/crtc_mode.lua).
; --- state. PACE is stream.s's, deliberately: the whole point is that the
; 68000 now writes the word the host used to write, and the pace gate that
; reads it does not change by a single byte.
CLK_PACE = $18034 ; == stream.s PACE
CLK_ACC = $18060 ; word: Bresenham remainder, < HFREQ
CLK_INCR = $18062 ; word: fps * (R04+1), computed by clk_init
CLK_VDISP = $18064 ; long: V-DISP edges taken. An INSTRUMENT --
; it is what lets a rig check that the tick
; count and the raster count are the same clock.
CLK_FPS = $18068 ; long: requested fps, an argument to clk_init
CLK_ERR = $1806C ; long: 0 ok / 1 not a 31.5 kHz mode
; / 2 fps*VTOTAL would overflow 16 bits
; ---------------------------------------------------------------------------
; clk_init -- arm the frame clock. Reads CLK_FPS, leaves CLK_ERR.
; Clobbers d0-d2. Leaves the CPU at SR=$2500 on success.
; ---------------------------------------------------------------------------
clk_init:
clr.l CLK_ERR.l
clr.l CLK_VDISP.l
clr.w CLK_ACC.l
clr.l CLK_PACE.l
; The mode has to be the one HFREQ describes. A 15 kHz mode would halve the
; line rate and the divider would run at double speed while looking correct,
; which is the failure this test exists to prevent.
move.w CRTC_R20.l,d0
btst #4,d0
bne.s .modeok
move.l #1,CLK_ERR.l
rts
.modeok:
; VTOTAL and the increment. Both out of the CRTC, so a change of mode changes
; the clock with it. acc is 16-bit and reaches at most HFREQ-1+incr, so incr
; must leave room: 65536 - 31500 = 34036. At VTOTAL=568 that is fps < 59.9,
; which is every rate this machine can display anyway -- but it is checked
; rather than argued.
move.w CRTC_R04.l,d0
addq.w #1,d0 ; VTOTAL scanlines
move.w d0,d1
move.w CLK_FPS+2.l,d2 ; low word of the long
mulu d2,d1 ; fps * VTOTAL (see FINDINGS 53.4 on
; C68K's flat MULU charge; this is boot
; code and is not cost-measured there)
cmp.l #65536-HFREQ,d1
bcs.s .fitok
move.l #2,CLK_ERR.l
rts
.fitok:
move.w d1,CLK_INCR.l
; The vector, before the source is armed.
move.l #clk_isr,CLK_VEC.w
; Disarm everything the IPL left running, then arm GPIP4 alone. Order matters:
; IER first (which clears IPR with it), then the edge, then the mask.
move.b #0,MFP_IERA.l
move.b #0,MFP_IERB.l
move.b #0,MFP_IMRA.l
move.b #MFP_VBASE,MFP_VR.l ; S clear: acknowledge clears pending
bclr #MFP_GPIP4,MFP_DDR.l ; V-DISP is an input
bclr #MFP_GPIP4,MFP_AER.l ; interrupt on the FALLING edge, i.e.
; at the start of vertical blanking
move.b #MFP_IVDISP,MFP_IERB.l
move.b #MFP_IVDISP,MFP_IMRB.l
move.w #$2500,sr ; let level 6 in; 1-5 stay masked
rts
; ---------------------------------------------------------------------------
; clk_stop -- disarm, and put the mask back where the rigs expect it.
; ---------------------------------------------------------------------------
clk_stop:
move.w #$2700,sr
move.b #0,MFP_IERB.l
move.b #0,MFP_IMRB.l
rts
; ---------------------------------------------------------------------------
; clk_isr -- one V-DISP. Every instruction here is charged to every frame the
; decoder draws, so it is deliberately the shortest thing that is still exact:
; four word operations and one long increment.
;
; Only the LOW WORD of d0 is touched, so only the low word is saved. The
; accumulator, the increment and the threshold all fit in 16 bits by the check
; in clk_init, which is what makes that legal.
; ---------------------------------------------------------------------------
clk_isr:
move.w d0,-(sp)
addq.l #1,CLK_VDISP.l
move.w CLK_ACC.l,d0
add.w CLK_INCR.l,d0
cmp.w #HFREQ,d0
bcs.s .nf
sub.w #HFREQ,d0
addq.l #1,CLK_PACE.l
.nf:
move.w d0,CLK_ACC.l
move.w (sp)+,d0
rte
src/player/clockgate.s
+59
View File
@@ -0,0 +1,59 @@
; Front-end for the frame clock (ROADMAP P3), for the rig.
;
; It exists to answer two questions that the streaming rig cannot answer on its
; own, because there the clock is buried under a decoder:
;
; 1. does the tick actually come from the raster, and at exactly the rate
; asked for -- measured over thousands of refreshes, not four;
; 2. WHAT IT COSTS, in clocks, per interrupt. This project's currency is
; 68000 clocks and the decoder already occupies 86.7% of the bus, so a
; frame clock is not free until someone has priced it.
;
; THE INSTRUMENT, and why it is a busy loop. MAME's Lua only sees the machine
; at frame boundaries, so it can time to 1/55.46 s and no finer -- 18 ms, where
; the whole per-frame cost of this clock is microseconds. Differencing two
; wall timings would measure nothing. So the 68000 counts instead: a loop with
; ONE instruction in its body runs for a fixed number of refreshes, and the
; iteration count is read out at both ends.
;
; clock off: iters0 * L = clocks in the window -> L
; clock on: iters1 * L + ints * H = clocks in the window -> H
;
; The window is an exact number of raster frames, so its length in clocks is
; exact and does not depend on the host at all. L is calibrated out by the
; first run rather than assumed from a cycle table, which matters: the point of
; the exercise is to price this code on the machine that will run it, and a
; table is the thing being checked. With ~2.7e7 iterations behind it, L carries
; enough digits that the interrupt cost -- 0.08% of the window -- survives the
; subtraction.
;
; The body is `addq.l #1,CGCNT.l` and nothing else: no compare, no counter in a
; register that an interrupt could be accused of disturbing, and a value that
; the host can read at any moment without stopping the CPU.
;
; The gate does NOT decode anything. What the clock does to a real frame is
; tools/bench/stream.lua's question, with DLX_PACE=2.
CGFLAG = $18070 ; 0 idle / 1 running / $EE clk_init refused
CGON = $18074 ; 1 = arm the frame clock, 0 = leave it off
CGCNT = $18078 ; <- loop iterations, read by the host at both
; ends of the window
org $10000
start:
clr.l CGCNT.l
move.l CGON.l,d0
beq.s noclk
bsr clk_init
tst.l CLK_ERR.l
bne.s bad
noclk:
move.l #1,CGFLAG.l ; the host starts its window here
loop:
addq.l #1,CGCNT.l
bra.s loop
bad:
move.l #$EE,CGFLAG.l
hold: bra.s hold
include "src/player/clock.i"
src/player/decode.s
+37 -116
View File
@@ -38,6 +38,39 @@
; paid only by blocks that are NOT all-SKIP: a header byte of zero clears four
; blocks with one tst.b, and SKIP is the median block.
;
; LITERAL SPANS (v7, FINDINGS 40). A run of horizontally adjacent dirty blocks
; is cheaper to paint as four ROW-LINEAR runs of word-expanded literal pixels
; than as blocks: 226 clocks per 4x4 block at a run of 4, against V1's 299.9,
; and the break-even is a run of 2. The run's blocks read SKIP in the mode
; header and the span section paints them instead, so the block loop below is
; unchanged -- it sees a SKIP and advances, exactly as it does for a genuinely
; held block.
;
; The section sits BETWEEN the mode header and the block payload because that is
; the only place the 68000 can reach without first parsing something of variable
; length: the header is a fixed 768 bytes. Per span the record is {u32 absolute
; GVRAM address, u16 coarse displacement}, then the coarse pixels, then a u16
; FINE displacement, then the fine pixels.
;
; The two displacements are jumps into two unrolled copy chains -- 24 pixels per
; coarse unit (a 12-register movem pair) and 2 per fine unit (one
; `move.l (a0)+,(a2)+`) -- so a span of any length is straight-line code with no
; loop, no remainder and no address arithmetic. A run of 4x4 blocks is always a
; multiple of 4 pixels long, and 4 is a multiple of the 2-pixel fine quantum, so
; NOTHING is padded (FINDINGS 40.3).
;
; The fine displacement is in the STREAM rather than in the span record because
; that is what pays for the second dispatch: when the coarse chain falls out
; into `move.w (a0)+,d0 / jmp`, d0 is dead payload and a0 is already pointing at
; it, so the decoder holds nothing extra across the copy and keeps all twelve
; payload registers (FINDINGS 40.4). Twelve is why the coarse unit is 24 pixels
; and not V5's 16, and it is the whole reason the per-pixel cost is 9.143 rather
; than 10.459 (FINDINGS 30.4).
;
; a1 (the mode header cursor) is one of those twelve, so it goes on the stack
; across the span pass. Two long accesses per frame, against the 24 pixels a
; register buys per chain unit.
;
; ALIGNMENT. Frame records are [u32 length][768-byte mode header][payload] laid
; end to end, and payload lengths are arbitrary -- so record boundaries land on
; odd addresses, and `move.l (a0)+,d0` on an odd address is an ADDRESS ERROR on
@@ -55,14 +88,8 @@ FPTR = $18010 ; -> first frame record
SCR_N = $18014 ; frames remaining this pass
SCR_END = $18018 ; expected end of the current payload
CB1 = $20000 ; expanded 4x4 codebook
CB4 = $22000 ; expanded 2x2 codebook
include "src/player/geom.i"
DST0 = $C08000 ; GVRAM + 32*1024 (first picture row)
DSTE = $C38000 ; GVRAM + 224*1024 (one past last)
BROW = 4096 ; bytes per block row (4 picture rows)
ROWLEN = 512 ; bytes per block row of blocks (64 * 8)
MODEB = 768 ; packed mode header, 3072 blocks * 2 bits
org $10000
start:
@@ -75,7 +102,8 @@ frameloop:
lea 0(a0,d0.l),a1
move.l a1,SCR_END.l ; where the payload must end
move.l a0,a1 ; a1 = packed mode header
lea MODEB(a0),a0 ; a0 = payload
lea MODEB(a0),a0 ; a0 = span section
bsr paint_spans ; -> a0 = block payload, a1 preserved
bsr decode_frame
cmpa.l SCR_END.l,a0 ; bitstream desync is silent otherwise
bne desync
@@ -92,111 +120,4 @@ hold: bra.s hold
desync: move.l #$EE,FLAG.l
bra.s hold
; ---------------------------------------------------------------- one block
; \1 = right-shift needed to bring this block's 2 mode bits to bits 1-0.
BLOCK macro
move.b (a1),d0
ifne \1
lsr.b #\1,d0
endc
and.w #3,d0
beq .sk\@ ; 00 SKIP -- the median block
subq.w #1,d0
beq .v1\@ ; 01 V1
subq.w #1,d0
bne .rw\@ ; 11 RAW, else 10 V4
; -- V4: four 2x2 codewords, sub-block order TL TR BL BR (vq_hybrid.paint)
moveq #0,d0
move.b (a0)+,d0
lsl.w #3,d0
movem.l (a3,d0.w),d0-d1
move.l d0,(a4)
move.l d1,1024(a4)
moveq #0,d0
move.b (a0)+,d0
lsl.w #3,d0
movem.l (a3,d0.w),d0-d1
move.l d0,4(a4)
move.l d1,1028(a4)
moveq #0,d0
move.b (a0)+,d0
lsl.w #3,d0
movem.l (a3,d0.w),d0-d1
move.l d0,2048(a4)
move.l d1,3072(a4)
moveq #0,d0
move.b (a0)+,d0
lsl.w #3,d0
movem.l (a3,d0.w),d0-d1
move.l d0,2052(a4)
move.l d1,3076(a4)
bra .sk\@
; -- V1: one 4x4 codeword, 32 bytes, straight out of the expanded codebook
.v1\@:
moveq #0,d0
move.b (a0)+,d0
lsl.w #5,d0
movem.l (a2,d0.w),d0-d7 ; EA is resolved before the load
movem.l d0-d1,(a4)
movem.l d2-d3,1024(a4)
movem.l d4-d5,2048(a4)
movem.l d6-d7,3072(a4)
bra .sk\@
; -- RAW: 16 literal palette indices. Two indices are assembled into one long
; via swap, so each pair of pixels costs one write instead of two; the high
; byte of each word is left as zero because the hardware discards it anyway.
.rw\@:
RAWPAIR 0
RAWPAIR 4
RAWPAIR 1024
RAWPAIR 1028
RAWPAIR 2048
RAWPAIR 2052
RAWPAIR 3072
RAWPAIR 3076
.sk\@:
addq.l #8,a4
endm
RAWPAIR macro
moveq #0,d0
move.b (a0)+,d0
swap d0
move.b (a0)+,d0
move.l d0,\1(a4)
endm
; ------------------------------------------------------------- one frame
; in: a0 = payload, a1 = packed mode header
; out: a0 = one past the last payload byte consumed
decode_frame:
lea CB1,a2
lea CB4,a3
lea DST0,a6
rowloop:
move.l a6,a4
lea ROWLEN(a6),a5
byteloop:
tst.b (a1) ; four SKIPs in one test
beq allskip
BLOCK 6
BLOCK 4
BLOCK 2
BLOCK 0
addq.l #1,a1
cmpa.l a5,a4
bne byteloop
bra rowdone
allskip:
addq.l #1,a1
lea 32(a4),a4
cmpa.l a5,a4
bne byteloop
rowdone:
lea BROW(a6),a6
cmpa.l #DSTE,a6
bne rowloop
rts
include "src/player/frame.i"
src/player/dma.i
+273
View File
@@ -0,0 +1,273 @@
; The HD63450 driving the SCSI data phase. ROADMAP P4a.
;
; WHAT P4a HAS TO SHOW, and why it needed a new kind of evidence. FINDINGS 58
; measured the CPU moving every byte itself at 87.28 clocks per delivered byte
; -- 391.8% of a 12 fps frame, against 22.4% for the cheapest DMA row of the
; ladder and 85.3% for the dearest. So the whole of what is left before M2 is a
; DMAC configuration that HOLDS THE BUS. 57.3 is why it could not simply be
; watched into existence: x68k_scsiext.cpp glues $EA0015 so that with the DMAC's
; OWN asserted -- which it is at idle on this machine -- MAME CANNOT DISTINGUISH
; a CPU-driven byte at that address from a DMAC-driven one. Watching the data
; register cannot answer the question it looks like it answers.
;
; THE DISCRIMINATOR USED HERE IS THE CPU'S OWN PROGRESS, and it never reads
; $EA0015 at all. A DMAC that holds the bus is one the CPU is not running
; against; so the witness is a single instruction:
;
; move.b #CCR_START,DM_CCR ; the channel is told to go
; move.w DM_MTC,d0 ; <- sampled by the VERY NEXT instruction
;
; If the bus was held for the transfer, the whole transfer happened between
; those two instructions and d0 reads ZERO. If it was not, d0 reads very nearly
; the full count and the CPU goes on to spin thousands of times while the
; channel trickles. Both configurations deliver the same bytes; what separates
; them is whether the 68000 got to execute anything meanwhile, which is exactly
; what "holds the bus" means and is not a fact about $EA0015.
;
; WHAT MAME CAN AND CANNOT BE ASKED, stated here because it bounds the claim and
; it is not obvious from the outside:
;
; * THE CARD HAS NO EXREQ PATH. x68k_scsiext.cpp's drq_w only stores a flag;
; the expansion slot has no request line to the DMAC at all (x68k.cpp wires
; drq0 from the FDC and drq3 from ADPCM, and nothing else). The card's flow
; control is DTACK: on a DMAC cycle with DRQ low the card NEGATES DTACK and
; the HD63450 discards that operand and retries. So every configuration
; below is AUTO-REQUEST; REQG=10, external request -- the mode the ladder's
; W=5 and W=12 rows assume -- has no wiring in this model and cannot be run.
; * SINGLE ADDRESS CANNOT BE RUN EITHER. hd63450.cpp only takes the implicit
; path when a channel has a dma_read/dma_write callback, and on this machine
; only channel 0 (the FDC) has one. DTYP=10/11 on channels 1..3 falls
; through to the dual-address code.
; * ONLY BURST IS MODELLED AS HELD. The device tests `(dcr & 0xc0) == 0`, so
; XRM=10 (cycle steal without hold) and XRM=11 (cycle steal WITH hold) are
; one code path. The bus is held, and the CPU halted, only for XRM=00 burst
; with REQG=01 max rate.
;
; So of the four rows of the per-byte ladder, exactly ONE -- dual address, bus
; held, 9 clk/B -- has a code path in this model, and it is the one demonstrated
; below. That is a bound on the apparatus and not a result about the board.
;
; AND IT IS STILL NOT A RATE. MAME's DMAC is configured in wall-clock attotimes
; (42.5), not per-operand cycles: set_burst_clocks gives channel 1 450 ns an
; operand no matter what the 68000 is doing. `W` is untouched by every line in
; this file and still wants a board (ROADMAP B1/B3).
; ---- the channel. 1, not 0: channel 0 is the FDC's and is the one channel
; with device callbacks, which would silently take the implicit-address path.
; Channel 1 is also the channel the IPL ROM points at the SASI data register
; (52.5), so this is the machine's own disk channel programmed differently.
DMA_CH = 1
DMACB = DMAC+DMA_CH*DMAC_CH ; $E84040
DM_CSR = DMACB+$00 ; channel status (write 1s to clear)
DM_CER = DMACB+$01 ; channel error (read only)
DM_DCR = DMACB+$04 ; device control
DM_OCR = DMACB+$05 ; operation control
DM_SCR = DMACB+$06 ; sequence control
DM_CCR = DMACB+$07 ; channel control
DM_MTC = DMACB+$0A ; memory transfer count, WORD
DM_MAR = DMACB+$0C ; memory address, LONG
DM_DAR = DMACB+$14 ; device address, LONG
DM_BTC = DMACB+$1A ; base transfer count, WORD (array chain)
DM_BAR = DMACB+$1C ; base address, LONG (array chain)
DM_MFC = DMACB+$29
DM_CPR = DMACB+$2D
DM_DFC = DMACB+$31
; CSR bits
CSR_COC = $80 ; channel operation complete
CSR_BTC = $40
CSR_NDT = $20 ; normal device termination
CSR_ERR = $10 ; channel error -- CER says which
CSR_ACT = $08 ; channel active
CCR_START = $80
; ---- the two configurations, as (DCR, OCR) pairs. Both are decoded by
; tools/analysis/27_dmac_config.py out of THESE bytes, using the same MC68450
; field tables 21_iplrom_dmac.py reads the IPL ROM's channels with -- so what
; the run claims it programmed and what it programmed cannot drift apart.
;
; HELD : DCR $00 = XRM 00 burst, DTYP 00 dual address, DPS 0 8-bit port
; OCR $81 = DIR device->memory, SIZE byte, no chain, REQG 01 max rate
; STEAL: DCR $80 = XRM 10 cycle steal WITHOUT hold, otherwise identical
; OCR $80 = REQG 00 auto-request at limited rate
DM_HELD_DCR = $00
DM_HELD_OCR = $81
DM_STEAL_DCR = $80
DM_STEAL_OCR = $80
; ---- what the run reports. Every one of these is a DMAC register or a count
; of the CPU's own instructions; none of them is a read of $EA0015.
; $18500 AND NOT $18300, WHICH IS WHERE THIS FIRST WENT. scsi.i's trace ends at
; $182FF and the next 160 bytes are the RING's: $18300 is ring.i's XF_SLOT
; mailbox, and tools/bench/stream.lua reads the same addresses from outside.
; dma.i is included by stream.s as well as by the gate, so DM_USE landed on the
; transfer request slot and the ring rig's first record request read as "use the
; DMAC" -- P4b's stage went red on a run that never reached its snapshot. The
; symptom was in a stage this session did not touch, which is the whole argument
; for check.sh being run before and after rather than only after.
DM_USE = $18500 ; u32 0 = PIO data phase, 1 = this file
DM_DCRV = $18504 ; u32 the DCR byte to program
DM_OCRV = $18508 ; u32 the OCR byte to program
DM_MTC0 = $1850C ; u32 MTC one instruction after START
DM_SPIN = $18510 ; u32 times the CPU went round the wait
DM_CSRF = $18514 ; u32 CSR when the channel finished
DM_CERF = $18518 ; u32 CER with it
DM_MTCF = $1851C ; u32 MTC with it
DM_MARF = $18520 ; u32 MAR with it -- where it stopped
DM_LEN = $18524 ; u32 bytes the channel was asked for
; ---- SEQUENTIAL ARRAY CHAINING, and it is OFF unless a caller asks for it.
; A device->GVRAM transfer cannot be one contiguous run: a picture row is 256 B
; of a 1024 B line stride, so 192 rows want 192 destinations. The MC68450 walks
; an array of 6-byte {u32 MAR, u16 MTC} entries for exactly this, and MAME's
; hd63450 implements it (`(ocr & 0x0c) == 0x08`, dma_transfer_start and
; dma_transfer_continue). DM_BARV = 0 means no chaining and NOTHING below
; changes, which is what stream.s gets: this file is included by the player as
; well as by the gate, and a mailbox that defaults to a new behaviour is how
; DM_USE landed on ring.i's slot and turned a stage red (above).
DM_BARV = $18528 ; u32 array base address, 0 = no chain
DM_BTCV = $1852C ; u32 array entry count (BTC)
; ---- THE SERVICE HOOK, and it is the whole of what a SECOND consumer needs
; from this file. ROADMAP P6c. A player that feeds ADPCM has to look at the
; audio channel more often than once a frame: the MSM6258 has no FIFO, so the
; instant its channel counts out the chip goes on replaying whatever byte its
; data register still holds -- and at 12 fps a once-a-frame re-arm makes that
; replay 90 ms long, which is not a gap, it is a buzz.
;
; THE ONLY PLACE A 68000 HAS TO SPARE IS INSIDE THIS WAIT. In the STEALING
; configuration the CPU goes round the loop below thousands of times per record
; (measured: 1,100,520 trips over 120 frames) and every one of them is time the
; disc is delivering and the CPU is not. So the hook is called from there, and
; the second consumer costs the video path nothing it was using.
;
; AND IN THE HELD CONFIGURATION THE HOOK CANNOT RUN AT ALL, which is not a bug
; in it: a burst channel HALTS the 68000 (dma_transfer_start asserts
; INPUT_LINE_HALT) and the CPU does not execute the loop, or anything else,
; until the record has landed. That asymmetry is the measurement -- FINDINGS
; 64.3 showed a held channel costs the frame CLOCK half its ticks, and this is
; the same fact reaching the audio.
;
; ZERO BY DEFAULT and every other front-end in this tree leaves it zero, so the
; cost to them is a `move.l` and a `beq` per trip. A mailbox that defaulted to
; a new behaviour is how DM_USE landed on ring.i's slot (above).
DM_HOOK = $18530 ; u32 0 = none, else a routine to call
; on every trip round the transfer wait.
; d0 is dead here and a0 is saved round
; the call, so the hook may trash both;
; it must preserve EVERYTHING else,
; because sc_in_dma's own d3/d4/d5 and
; scsi_read's a1 are live across it.
DM_PATIENCE = 4000000 ; the wait is bounded like every other
; ---------------------------------------------------------------- sc_in_dma
; Receive d1 bytes into (a1) in phase d2, WITHOUT the CPU touching one of them.
; Entered from sc_in_data when DM_USE is set; same registers, same contract.
;
; ORDER MATTERS AND IT IS NOT THE OBVIOUS ONE. The SPC is put into DMA transfer
; BEFORE the channel is started, because in the held configuration the 68000
; stops executing at the CCR write and does not run again until the transfer is
; over -- so anything the SPC needs to be told has to have been told already.
sc_in_dma:
movem.l d3-d5,-(sp)
move.l d1,d5 ; keep the length for the report
move.l d5,DM_LEN.l
move.b d2,SC_PCTL
move.l d1,d0
bsr sc_settc ; the SPC counts the same bytes down
; ---- the channel, quiet first: CSR is write-one-to-clear and a stale
; COC from a previous record would pass the wait loop instantly.
move.b #$FF,DM_CSR
move.l DM_DCRV.l,d0
move.b d0,DM_DCR
move.l DM_OCRV.l,d0
move.b d0,DM_OCR
move.b #$04,DM_SCR ; MAC 01 memory increment, DAC 00 none:
; the device address is a REGISTER and
; must not walk off it.
move.b #$05,DM_MFC ; the function codes the IPL ROM uses
move.b #$05,DM_DFC
move.b #$01,DM_CPR
move.l DM_BARV.l,d0
bne.s .chain
move.w d5,DM_MTC
move.l a1,DM_MAR
bra.s .darset
.chain:
; MAR and MTC are NOT written: the channel loads both from the array's
; first entry when it starts, and reloads them from the next entry at
; every count-out. Writing them here would be writing registers the
; hardware is about to overwrite, which reads like a contract and is not.
move.l d0,DM_BAR
move.l DM_BTCV.l,d0
move.w d0,DM_BTC
.darset:
move.l #SC_DREG,DM_DAR ; $EA0015 -- the DMAC's door, and now
; the DMAC is the one going through it
move.b #SCMD_XFER,SC_SCMD ; no PROGRAM bit: the SPC raises DRQ
move.l #11,SC_TAG.l ; 11 = channel armed, SPC in DMA mode
bsr sc_snap
; ---- START, and the witness immediately after it
move.b #CCR_START,DM_CCR
move.w DM_MTC,d0 ; THE DISCRIMINATOR. Held: zero.
andi.l #$FFFF,d0
move.l d0,DM_MTC0.l
; ---- wait for the channel, counting the CPU's own trips round the loop.
; In the held configuration this is one trip, because the CPU did not
; get to run until the transfer was over. In the stealing one it is
; thousands, and every one of them is a 68000 instruction that executed
; while the disc was delivering -- which is the whole point of P4a.
clr.l DM_SPIN.l
move.l #DM_PATIENCE,d3
.wait: addq.l #1,DM_SPIN.l
move.l DM_HOOK.l,d0 ; the second consumer's slot -- see above
beq.s .nohook
move.l a0,-(sp)
movea.l d0,a0
jsr (a0)
movea.l (sp)+,a0
.nohook:
move.b DM_CSR,d4
btst #4,d4 ; ERR
bne.s .err
btst #7,d4 ; COC
bne.s .fin
subq.l #1,d3
bne.s .wait
bsr .report
movem.l (sp)+,d3-d5
move.l #SCE_TIMEOUT,SC_ERR.l
moveq #-1,d0
rts
.err: bsr .report
movem.l (sp)+,d3-d5
move.l #SCE_TIMEOUT,SC_ERR.l ; a channel error is a dead transport
moveq #-1,d0
rts
.fin: bsr .report
move.b #$FF,DM_CSR ; leave the channel as we found it
move.l #12,SC_TAG.l ; 12 = channel reported COC
bsr sc_snap
movem.l (sp)+,d3-d5
bsr sc_xferend ; the SPC's own transfer, not the DMAC's
move.l d0,-(sp)
move.l #9,SC_TAG.l
bsr sc_snap
move.l (sp)+,d0
rts
; ---- the channel's own account of what it did, read out of its registers
.report:
moveq #0,d0
move.b DM_CSR,d0
move.l d0,DM_CSRF.l
moveq #0,d0
move.b DM_CER,d0
move.l d0,DM_CERF.l
move.w DM_MTC,d0
andi.l #$FFFF,d0
move.l d0,DM_MTCF.l
move.l DM_MAR,d0
move.l d0,DM_MARF.l
rts
src/player/dmagate.s
+403
View File
@@ -0,0 +1,403 @@
; Front-end for the HD63450 DATA PHASE (ROADMAP P4a), for the rig.
;
; THE QUESTION. FINDINGS 58 put the transport on the 68000 and priced it: the
; CPU moving every byte itself costs 87.28 clocks per delivered byte, 391.8% of
; a 12 fps frame. Against that, the cheapest DMA row of the ladder is 22.4% and
; the dearest is 85.3%, so everything left before M2 turns on getting the DMAC
; to drive the data phase with the bus HELD. 57.3 is why it cannot be shown by
; watching the data register: with the DMAC's OWN asserted, which it is at idle
; here, MAME cannot tell a CPU-driven byte at $EA0015 from a DMAC-driven one.
;
; THE EVIDENCE THIS GATE PRODUCES, and none of it is a read of $EA0015:
;
; 1. THE SAME BYTES. The same sectors are read three times -- once by the PIO
; path FINDINGS 58 measured, once by the channel with the bus held, once by
; the channel stealing cycles -- and the HOST compares all three against its
; own copy of the image. A transport that returns the wrong bytes without
; saying so is the failure a checksum-free ring cannot survive (49.2).
; 2. THE CPU'S OWN PROGRESS. MTC is sampled by the INSTRUCTION AFTER the one
; that starts the channel. Held, it reads zero: the entire transfer
; happened between two instructions, because the 68000 did not execute in
; between. Stealing, it reads nearly the full count and the CPU then goes
; round its wait loop thousands of times while the bytes arrive. That
; difference IS "the DMAC held the bus", and it is a fact about the CPU.
; 3. THE CHANNEL'S OWN ACCOUNT. CSR, CER, the final MTC and the final MAR:
; the channel says it completed without error, moved every byte, and left
; its memory pointer exactly one transfer-length past where it started.
; 4. THE WINDOW IS REFUSED. A windowed read (58.3: 117 of 120 records start
; part way into a sector) is rejected by the transport rather than silently
; delivering the neighbouring records' bytes into the ring. P4a's
; precondition is stated by the code that has it, not by a comment.
;
; WHAT IT DOES NOT SHOW. Not `W`. Not one clock of it. MAME's DMAC runs on
; wall-clock attotimes (42.5) and its burst mode halts the CPU outright rather
; than costing it cycles per operand, so this gate settles WHICH CONFIGURATION
; WORKS and not what one costs. See src/player/dma.i for the three ways this
; model bounds the question -- no EXREQ wiring, no single-address path, and only
; burst modelled as held.
DGFLAG = $18600 ; 0 idle / 1 done
DGREC = $18800 ; 9 x 32 B: rc, err, mtc0, spin, csr, cer, mtc, mar
; $18800 AND NOT $18610, WHERE THIS LIVED: nine
; records of 32 B run to $188FF, and from $18610
; they would have run over DGWIN at $18700 --
; the window run's own result, which run 10 then
; writes back. A silent overlap between two
; runs' evidence is the kind of thing that makes
; a gate report the wrong run's numbers.
DGREC_SZ = 32
DGWIN = $18700 ; u32 return of the WINDOWED dma read (want -1)
DGWERR = $18704 ; u32 SC_ERR after it (want SCE_WINDOW)
DGR20 = $18708 ; u32 R20 as it stood during run 4
DGR20N = $1870C ; u32 R20 as it stood during run 5 (the control)
DGR20C = $18710 ; u32 R20 as it stood during run 6 (chained)
DGR20P = $18714 ; u32 R20 as it stood during run 9 (palette+rows)
R20_BUF = $0916 ; 256-colour, 31.5 kHz, G-VRAM SET TO BUFFER
R20_MSK = $0116 ; the SAME, bit 11 CLEAR -- the negative control
DGDST4 = $C0C000 ; where the control writes
DGCHA = $19000 ; the array-chain table: 8 x {u32 MAR, u16 MTC}
DGCHN = 8 ; entries
DGCROW = 256 ; bytes an entry carries -- one packed picture row
DGDST5 = $C10000 ; and the 8 row bases, at the 1024 B line stride
CRTC20 = $E80028 ; CRTC R20
DGDST3 = $C08000 ; DMA straight into GVRAM, BUFFER MODE (47.6.2)
DGLBA = 1000 ; a NON-ZERO LBA throughout: a driver that emits
; a malformed LBA field still passes LBA 0
DGBLK = 4 ; 4 x 512 = 2,048 B
DGDST0 = $20000 ; PIO
DGDST1 = $24000 ; DMA, bus held
DGDST2 = $28000 ; DMA, cycle stealing
; ---- runs 7-9: THE PALETTE (ROADMAP K1, FINDINGS 61.9's first open item).
DGPAL = $E82000 ; the GRAPHIC palette: 256 words, GGGGGRRRRRBBBBBI
DGPALN = 256 ; words in it -- and 512 B is exactly ONE sector
DGPBLK = 1 ; so the whole palette is one block off the disc
DGPOIS = $A500 ; the poison: word i = DGPOIS|i. A palette that
; still reads this was not written by anything.
DGDST6 = $2C000 ; run 8's destination: RAM, so the palette is
; left alone and must still read poison
DGDST7 = $C14000 ; run 9's six GVRAM rows, at the 1024 B stride
DGCHA2 = $19100 ; run 9's array: 7 x {u32 MAR, u16 MTC}
DGCHN2 = 7 ; palette 512 B + 6 rows x 256 B = 2,048 B
DGCROW2 = 256
DGPS7 = $1A000 ; the palette as it stood after run 7 ...
DGPS8 = $1A200 ; ... after run 8 (the control: poison) ...
DGPS9 = $1A400 ; ... and after run 9 (chained). SNAPSHOTS, not
; a late read: each run overwrites the previous
; run's palette, so a host that looks once at
; the end sees only the last of the three.
org $10000
start:
clr.l DGFLAG.l
; ---- 1. the PIO path, unchanged, as the reference the other two are measured
; against. It is here so that a DMA failure cannot be confused with a SCSI
; protocol failure: if this one is wrong, nothing below is about the DMAC.
bsr scsi_init
move.l #DGLBA,d3
moveq #DGBLK,d4
lea DGDST0,a1
bsr scsi_read
lea DGREC,a0
bsr dg_save
; ---- 2. the channel, WITH THE BUS HELD
bsr scsi_init
move.l #DM_HELD_DCR,DM_DCRV.l
move.l #DM_HELD_OCR,DM_OCRV.l
move.l #1,DM_USE.l ; after scsi_init, which clears it
move.l #DGLBA,d3
moveq #DGBLK,d4
lea DGDST1,a1
bsr scsi_read
lea DGREC+DGREC_SZ,a0
bsr dg_save
; ---- 3. the channel, STEALING CYCLES. Same bytes, same code, two register
; values different -- which is what makes the comparison a comparison.
bsr scsi_init
move.l #DM_STEAL_DCR,DM_DCRV.l
move.l #DM_STEAL_OCR,DM_OCRV.l
move.l #1,DM_USE.l
move.l #DGLBA,d3
moveq #DGBLK,d4
lea DGDST2,a1
bsr scsi_read
lea DGREC+2*DGREC_SZ,a0
bsr dg_save
; ---- 4. THE CHANNEL WRITING GVRAM, IN BUFFER MODE. 47.6.2: "the DMAC has not
; been near this" -- 44.7 costed a device->GVRAM transfer and 47 built the packed
; layout, and no run in this tree has ever pointed a channel at $C00000. Two
; separate things are being asked at once and both are write-path questions:
;
; a. can a channel write GVRAM AT ALL? Nothing says it cannot -- the DMAC
; writes through the same program address space the CPU does -- but a
; transport that silently drops its writes into a device handler is exactly
; the failure this gate exists to catch, and the SPC's own bytes make a
; better witness than a fill pattern.
; b. does a BYTE-wide channel fill the PACKED layout? 47.1 measured the write
; path with word writes from the CPU. A dual-address channel with an 8-bit
; device port writes BYTES, and MAME's gvram_w in buffer mode passes
; mem_mask straight through -- so an even byte should land in the HIGH half
; of its word (page 1) and an odd byte in the LOW half (page 0). If it
; does, a linear DMA of a stream interleaved (right<<8)|left -- which is
; exactly show_frame256_packed.lua's layout -- fills the screen with no CPU
; in the loop at all.
;
; R20 bit 11 is left SET across the readback on purpose: gvram_r returns the raw
; word in buffer mode and the masked byte outside it, so clearing it first would
; hide half of what is being measured.
;
; R20 IS WRITTEN OUTRIGHT AND NOT OR-ED INTO. The first cut of this run set bit
; 11 on top of whatever the IPL left, and the IPL leaves $0B16 (22.1) -- bit 11
; ALREADY SET, and COL = %11, the 65,536-colour setup, which writes whole words
; with or without the bit. The run passed and proved nothing: it was a test that
; could not fail. Run 5 below is the control that makes this one mean something,
; and the two differ in EXACTLY BIT 11.
bsr scsi_init
move.w #R20_BUF,CRTC20.l
move.l #R20_BUF,DGR20.l
move.l #DM_HELD_DCR,DM_DCRV.l
move.l #DM_HELD_OCR,DM_OCRV.l
move.l #1,DM_USE.l
move.l #DGLBA,d3
moveq #DGBLK,d4
lea DGDST3,a1
bsr scsi_read
lea DGREC+3*DGREC_SZ,a0
bsr dg_save
; ---- 5. THE NEGATIVE CONTROL: the same transfer with bit 11 CLEAR. In masked
; 256-colour mode gvram_w takes `data & 0x00ff` and IGNORES mem_mask, so a byte
; written to an EVEN address -- where the 68000 puts the MS byte, and where every
; other disc byte lands -- contributes nothing and cannot be read back. Half the
; transfer must be lost, and if it is not, run 4 was not measuring the bit.
bsr scsi_init
move.w #R20_MSK,CRTC20.l
move.l #R20_MSK,DGR20N.l
move.l #DM_HELD_DCR,DM_DCRV.l
move.l #DM_HELD_OCR,DM_OCRV.l
move.l #1,DM_USE.l
move.l #DGLBA,d3
moveq #DGBLK,d4
lea DGDST4,a1
bsr scsi_read
lea DGREC+4*DGREC_SZ,a0
bsr dg_save
move.w #R20_BUF,CRTC20.l ; back to buffer mode, so the host reads
; RAW WORDS out of both destinations
; ---- 6. THE ROW STRIDE, WHICH IS THE REAL SHAPE OF THE TRANSFER. Runs 4 and 5
; wrote 2,048 contiguous bytes, and no picture is contiguous: a packed row is
; 256 B of a 1024 B line stride, so a frame is 192 destinations and not one.
; 46.6 said "no stride for a DMAC to skip" about the bytes WITHIN a row and left
; the rows themselves unexamined; a channel cannot skip 768 B any more than it
; could skip the 300 B in front of a record (run 10 below).
;
; The MC68450 answers this with SEQUENTIAL ARRAY CHAINING -- an array of 6-byte
; {u32 MAR, u16 MTC} entries it walks by itself -- and MAME implements it. So
; the question is not whether the CPU can restart the channel 192 times a frame;
; it is whether it has to at all. Eight rows here, out of the same LBA and the
; same 2,048 B, so the ONLY difference from run 4 is where the bytes land.
bsr scsi_init
move.w #R20_BUF,CRTC20.l
move.l #R20_BUF,DGR20C.l
lea DGCHA,a0
lea DGDST5,a1
moveq #DGCHN-1,d5
dg_mkch:
move.l a1,(a0)+ ; MAR: this row's base
move.w #DGCROW,(a0)+ ; MTC: 256 bytes of it
lea 1024(a1),a1 ; the next row is a line stride away
dbra d5,dg_mkch
move.l #DM_HELD_DCR,DM_DCRV.l
move.l #DM_HELD_OCR|$08,DM_OCRV.l ; OCR CHAIN = %10, array chain
move.l #DGCHA,DM_BARV.l
move.l #DGCHN,DM_BTCV.l
move.l #1,DM_USE.l
move.l #DGLBA,d3
moveq #DGBLK,d4
lea DGDST5,a1 ; ignored under chaining; passed so the
; call site reads the same as the others
bsr scsi_read
lea DGREC+5*DGREC_SZ,a0
bsr dg_save
clr.l DM_BARV.l ; and OFF again, so run 7 is unchained
; ---- 7. THE PALETTE. Can a channel write $E82000? (FINDINGS 61.9, ROADMAP
; K1.) Runs 4-6 put the PICTURE on the channel; a packed frame is a picture AND
; a palette, and if the palette registers take a byte-wide DMA the way GVRAM
; does in buffer mode then the palette is a 193rd array-chain entry and ONE
; channel start paints a whole frame -- no per-frame CPU work in the video path
; at all. If they do not, the CPU writes 256 words a frame (61.9 derives that
; at ~2,370 clocks, 0.28% of a frame) and the architecture still stands; this is
; the difference between cheap and free, and it is worth one run to know which.
;
; 512 B IS THE WHOLE PALETTE AND EXACTLY ONE SECTOR, which is why this run reads
; one block where the others read four: a transfer that ran long would write
; $E82200 (the TEXT palette) and then $E82400 (the video controller's own
; registers, priority included), and a probe that reconfigures the video
; controller as a side effect is not a probe.
;
; THE PALETTE IS POISONED FIRST, and that is what stops this being run 4's trap
; a second time. A destination that already holds the right bytes cannot tell a
; channel that wrote them from a channel that did nothing; RAM at $20000 was
; zero and the record is mostly pad, so "it matches" has been a weak claim all
; session. Word i is set to $A500|i, which no 512 B of container matches by
; accident, and the host reports how many of the 512 positions the poison and
; the disc actually differ in rather than assuming all of them.
bsr scsi_init
bsr dg_poison
move.l #DM_HELD_DCR,DM_DCRV.l
move.l #DM_HELD_OCR,DM_OCRV.l
move.l #1,DM_USE.l
move.l #DGLBA,d3
moveq #DGPBLK,d4
lea DGPAL,a1
bsr scsi_read
lea DGPS7,a1
bsr dg_palsnap ; before run 8 overwrites it
lea DGREC+6*DGREC_SZ,a0
bsr dg_save
; ---- 8. THE NEGATIVE CONTROL, and it is an ATTRIBUTION control rather than a
; mechanism one. Run 5's control could point at a mode bit; there is no mode
; bit here, so what has to be excluded is that run 7's palette held the disc's
; bytes for some reason OTHER than the channel having written them there --
; a readback that aliases somewhere else, the SPC's own path touching the
; registers, the poison never having landed.
;
; Same transfer, same channel, same bytes; ONE thing different, the destination
; address. The disc's bytes must appear at $2C000, and the palette must still
; read poison in all 256 words. Two claims from one run, and the second is the
; one that makes run 7 mean anything: it also proves the CPU's poison writes
; reach the registers the host reads back, which is the positive half.
bsr scsi_init
bsr dg_poison
move.l #DM_HELD_DCR,DM_DCRV.l
move.l #DM_HELD_OCR,DM_OCRV.l
move.l #1,DM_USE.l
move.l #DGLBA,d3
moveq #DGPBLK,d4
lea DGDST6,a1
bsr scsi_read
lea DGPS8,a1
bsr dg_palsnap ; must be poison, word for word
lea DGREC+7*DGREC_SZ,a0
bsr dg_save
; ---- 9. THE 193rd ENTRY: one start, the palette AND the picture rows. This is
; the run K1 exists for. Runs 7 and 8 only show that a channel can write the
; palette registers; what the architecture needs is that ONE array chain can
; cross from a hardware register area into GVRAM without the CPU between them,
; because a frame is a palette entry followed by 192 row entries and the whole
; claim is that the CPU starts the channel once.
;
; Seven entries, 2,048 B, out of the same LBA as everything else: 512 B into the
; palette and then six rows of 256 B at the 1024 B line stride. The destination
; regions are of two different KINDS -- device registers and video RAM in buffer
; mode -- which is exactly the crossing that has never been run.
bsr scsi_init
bsr dg_poison
move.w #R20_BUF,CRTC20.l
move.l #R20_BUF,DGR20P.l
lea DGCHA2,a0
move.l #DGPAL,(a0)+ ; entry 0: the palette, a whole sector
move.w #DGPALN*2,(a0)+
lea DGDST7,a1
moveq #DGCHN2-2,d5 ; the remaining six are picture rows
dg_mkch2:
move.l a1,(a0)+
move.w #DGCROW2,(a0)+
lea 1024(a1),a1
dbra d5,dg_mkch2
move.l #DM_HELD_DCR,DM_DCRV.l
move.l #DM_HELD_OCR|$08,DM_OCRV.l ; OCR CHAIN = %10, array chain
move.l #DGCHA2,DM_BARV.l
move.l #DGCHN2,DM_BTCV.l
move.l #1,DM_USE.l
move.l #DGLBA,d3
moveq #DGBLK,d4
lea DGDST7,a1 ; ignored under chaining, as in run 6
bsr scsi_read
lea DGPS9,a1
bsr dg_palsnap
lea DGREC+8*DGREC_SZ,a0
bsr dg_save
clr.l DM_BARV.l ; and OFF again, so run 10 is unchained
; ---- 10. and a WINDOWED read through the channel, which must be REFUSED. This
; is the one test here that is expected to fail, and it has to fail LOUDLY: the
; alternative is a channel writing a whole sector into a ring that has room for
; a record, over the top of records the decoder has not finished with.
bsr scsi_init
move.l #DM_HELD_DCR,DM_DCRV.l
move.l #DM_HELD_OCR,DM_OCRV.l
move.l #1,DM_USE.l
move.l #300,SC_WSKIP.l ; a record that starts 300 B into a sector
move.l #1024,SC_WKEEP.l
move.l #DGLBA,d3
moveq #DGBLK,d4
lea DGDST2,a1
bsr scsi_read_win
move.l d0,DGWIN.l
move.l SC_ERR.l,DGWERR.l
move.l #1,DGFLAG.l
hold: bra.s hold
; ---- one config's result, copied out of the shared reporting words before the
; next run overwrites them. d0 = scsi_read's return; a0 = where it goes.
dg_save:
move.l d0,(a0)+
move.l SC_ERR.l,(a0)+
move.l DM_MTC0.l,(a0)+
move.l DM_SPIN.l,(a0)+
move.l DM_CSRF.l,(a0)+
move.l DM_CERF.l,(a0)+
move.l DM_MTCF.l,(a0)+
move.l DM_MARF.l,(a0)+
; and clear them, so a config that never reached the channel reports
; zeros of its own rather than the previous config's numbers.
clr.l DM_MTC0.l
clr.l DM_SPIN.l
clr.l DM_CSRF.l
clr.l DM_CERF.l
clr.l DM_MTCF.l
clr.l DM_MARF.l
rts
; ---- the poison. Word i of the graphic palette <- $A500|i, written by the
; 68000 itself. Two jobs: a destination that cannot hold the right answer by
; accident, and a known pattern the control run reads back out of the registers
; to show the readback path shows what is actually in them.
dg_poison:
movem.l d0-d2/a0,-(sp)
lea DGPAL,a0
moveq #0,d1 ; i
move.w #DGPALN-1,d0
.p: move.w d1,d2
ori.w #DGPOIS,d2 ; $A500|i, and i never exceeds 255
move.w d2,(a0)+
addq.w #1,d1
dbra d0,.p
movem.l (sp)+,d0-d2/a0
rts
; ---- the palette as it stands, copied to (a1) by the 68000 READING THE
; REGISTERS. A snapshot and not a late host read: each palette run overwrites
; the last one's result, so all three have to be kept while they are true. It
; is also the same shape of evidence 53.3 used for pal_pack -- the words come
; back out of $E82000 rather than out of the buffer they were built in.
dg_palsnap:
movem.l d0/a0-a1,-(sp)
lea DGPAL,a0
move.w #DGPALN-1,d0
.s: move.w (a0)+,(a1)+
dbra d0,.s
movem.l (sp)+,d0/a0-a1
rts
include "src/player/scsi.i"
src/player/frame.i
+183
View File
@@ -0,0 +1,183 @@
; ---------------------------------------------------------------- one block
; \1 = right-shift needed to bring this block's 2 mode bits to bits 1-0.
BLOCK macro
move.b (a1),d0
ifne \1
lsr.b #\1,d0
endc
and.w #3,d0
beq .sk\@ ; 00 SKIP -- the median block
subq.w #1,d0
beq .v1\@ ; 01 V1
subq.w #1,d0
bne .rw\@ ; 11 RAW, else 10 V4
; -- V4: four 2x2 codewords, sub-block order TL TR BL BR (vq_hybrid.paint)
moveq #0,d0
move.b (a0)+,d0
lsl.w #3,d0
movem.l (a3,d0.w),d0-d1
move.l d0,(a4)
move.l d1,1024(a4)
moveq #0,d0
move.b (a0)+,d0
lsl.w #3,d0
movem.l (a3,d0.w),d0-d1
move.l d0,4(a4)
move.l d1,1028(a4)
moveq #0,d0
move.b (a0)+,d0
lsl.w #3,d0
movem.l (a3,d0.w),d0-d1
move.l d0,2048(a4)
move.l d1,3072(a4)
moveq #0,d0
move.b (a0)+,d0
lsl.w #3,d0
movem.l (a3,d0.w),d0-d1
move.l d0,2052(a4)
move.l d1,3076(a4)
bra .sk\@
; -- V1: one 4x4 codeword, 32 bytes, straight out of the expanded codebook
.v1\@:
moveq #0,d0
move.b (a0)+,d0
lsl.w #5,d0
movem.l (a2,d0.w),d0-d7 ; EA is resolved before the load
movem.l d0-d1,(a4)
movem.l d2-d3,1024(a4)
movem.l d4-d5,2048(a4)
movem.l d6-d7,3072(a4)
bra .sk\@
; -- RAW: 16 literal palette indices. Two indices are assembled into one long
; via swap, so each pair of pixels costs one write instead of two; the high
; byte of each word is left as zero because the hardware discards it anyway.
.rw\@:
RAWPAIR 0
RAWPAIR 4
RAWPAIR 1024
RAWPAIR 1028
RAWPAIR 2048
RAWPAIR 2052
RAWPAIR 3072
RAWPAIR 3076
.sk\@:
addq.l #8,a4
endm
RAWPAIR macro
moveq #0,d0
move.b (a0)+,d0
swap d0
move.b (a0)+,d0
move.l d0,\1(a4)
endm
; ------------------------------------------------------- the span section
; in: a0 = span section, a1 = mode header (preserved across the call)
; out: a0 = one past the section, i.e. the block payload
;
; This is tools/bench/blit.s v7 verbatim, and deliberately so: the 66.0 clocks
; per span + 9.143 per coarse pixel + 9.978 per fine pixel of FINDINGS 40 were
; measured on exactly this instruction sequence, over thirteen span lengths, and
; a "tidier" rewrite here would silently invalidate every span figure in
; FINDINGS 39/40 and in tools/analysis/14_dmac_chain.py.
;
; The fine chain is entered by FALLING OUT of the coarse one, so a span with no
; coarse units enters at v7cx with d0 already reloaded -- which is why the
; coarse displacement for c=0 is SPCN*SPCU, one past the last coarse unit,
; rather than a special case.
paint_spans:
move.w (a0)+,d7 ; spans in this frame
subq.w #1,d7
bmi spnone ; a frame may legitimately have none (the
; chain is far past a short branch)
move.l a1,-(sp) ; a1 is a payload register below
spspan: move.l (a0)+,a2 ; absolute GVRAM destination
move.w (a0)+,d0 ; (SPCN - coarse) * SPCU
jmp spch(pc,d0.w)
spch:
movem.l (a0)+,d0-d6/a1/a3-a6
movem.l d0-d6/a1/a3-a6,(a2)
lea 48(a2),a2
movem.l (a0)+,d0-d6/a1/a3-a6
movem.l d0-d6/a1/a3-a6,(a2)
lea 48(a2),a2
movem.l (a0)+,d0-d6/a1/a3-a6
movem.l d0-d6/a1/a3-a6,(a2)
lea 48(a2),a2
movem.l (a0)+,d0-d6/a1/a3-a6
movem.l d0-d6/a1/a3-a6,(a2)
lea 48(a2),a2
movem.l (a0)+,d0-d6/a1/a3-a6
movem.l d0-d6/a1/a3-a6,(a2)
lea 48(a2),a2
movem.l (a0)+,d0-d6/a1/a3-a6
movem.l d0-d6/a1/a3-a6,(a2)
lea 48(a2),a2
movem.l (a0)+,d0-d6/a1/a3-a6
movem.l d0-d6/a1/a3-a6,(a2)
lea 48(a2),a2
movem.l (a0)+,d0-d6/a1/a3-a6
movem.l d0-d6/a1/a3-a6,(a2)
lea 48(a2),a2
movem.l (a0)+,d0-d6/a1/a3-a6
movem.l d0-d6/a1/a3-a6,(a2)
lea 48(a2),a2
movem.l (a0)+,d0-d6/a1/a3-a6
movem.l d0-d6/a1/a3-a6,(a2)
lea 48(a2),a2
movem.l (a0)+,d0-d6/a1/a3-a6
movem.l d0-d6/a1/a3-a6,(a2)
lea 48(a2),a2
spcx: move.w (a0)+,d0 ; (SPFN - fine) * SPFU, from mid-stream
jmp spfh(pc,d0.w)
spfh:
move.l (a0)+,(a2)+
move.l (a0)+,(a2)+
move.l (a0)+,(a2)+
move.l (a0)+,(a2)+
move.l (a0)+,(a2)+
move.l (a0)+,(a2)+
move.l (a0)+,(a2)+
move.l (a0)+,(a2)+
move.l (a0)+,(a2)+
move.l (a0)+,(a2)+
move.l (a0)+,(a2)+
dbra d7,spspan
move.l (sp)+,a1
spnone: rts
; ------------------------------------------------------------- one frame
; in: a0 = payload, a1 = packed mode header
; out: a0 = one past the last payload byte consumed
decode_frame:
lea CB1,a2
lea CB4,a3
lea DST0,a6
rowloop:
move.l a6,a4
lea ROWLEN(a6),a5
byteloop:
tst.b (a1) ; four SKIPs in one test
beq allskip
BLOCK 6
BLOCK 4
BLOCK 2
BLOCK 0
addq.l #1,a1
cmpa.l a5,a4
bne byteloop
bra rowdone
allskip:
addq.l #1,a1
lea 32(a4),a4
cmpa.l a5,a4
bne byteloop
rowdone:
lea BROW(a6),a6
cmpa.l #DSTE,a6
bne rowloop
rts
src/player/geom.i
+41
View File
@@ -0,0 +1,41 @@
; Geometry and codebook constants shared by every front-end in src/player/.
;
; Split out of decode.s in session 18 so that decode.s (the preloaded-stream
; rig, gated by tools/bench/check.sh) and stream.s (the ring-buffer streaming
; rig, FINDINGS 49) assemble from LITERALLY THE SAME BYTES for the block loop
; and the span chain. Those bytes are not incidental: the 66.0 clocks/span,
; 9.143 clocks/coarse pixel and 9.978 clocks/fine pixel of FINDINGS 40, and
; every per-block constant in FINDINGS 24/30/41, are fitted to this exact
; instruction sequence. Two hand-maintained copies of it would drift, and the
; drift would be invisible -- both would still decode correctly, and only the
; cost model would be wrong.
;
; The split is a no-op by construction: tools/bench/check.sh asserts that
; decode.s still assembles to the same 1,296 bytes it did before it.
CB1 = $20000 ; expanded 4x4 codebook
CB4 = $22000 ; expanded 2x2 codebook
DST0 = $C08000 ; GVRAM + 32*1024 (first picture row)
DSTE = $C38000 ; GVRAM + 224*1024 (one past last)
BROW = 4096 ; bytes per block row (4 picture rows)
ROWLEN = 512 ; bytes per block row of blocks (64 * 8)
MODEB = 768 ; packed mode header, 3072 blocks * 2 bits
SPCU = 12 ; bytes of code per COARSE span unit (24 px)
SPCN = 11 ; coarse units: 11*24 = 264 px >= one row
SPFU = 2 ; bytes of code per FINE span unit (2 px)
SPFN = 11 ; fine units: 11*2 = 22 px > one coarse unit
; RECORD ALIGNMENT, and it is a property of the CONTAINER rather than of the
; 68000. DLX2 padded each record up to 4, which is all `move.l (a0)+` needs
; (FINDINGS 28.3). DLX5 pads up to a 512 B SECTOR, so that a DMA channel can
; read a record as whole sectors straight into the ring with no window and no
; bounce copy -- `sc_in_data` REFUSES a windowed read when the data phase is
; the channel's (59.4), and 117 of 120 records needed one under DLX4.
;
; The consumer has to know it too: the decoder releases the ring up to the end
; of the record it was handed, and a decoder that released only the bytes it
; READ would leave the pad unreclaimed and drift the producer's free-space
; arithmetic by up to RECALN-1 per record. The ring base must therefore be
; RECALN-aligned, which is asserted where the ring is placed.
RECALN = 512
src/player/load.i
+256
View File
@@ -0,0 +1,256 @@
; ---------------------------------------------------------------------------
; load.i -- the two LOAD-TIME transforms, on the 68000 itself. ROADMAP P1+P2.
;
; Until now both of these were done host-side, in tools/bench/dlxload.py, and
; the rigs pushed the RESULT into emulated RAM. That was the right call while
; the inner loop was the thing being measured -- charging a once-per-scene cost
; to the per-frame path would have flattered or damned it for no reason -- but
; a player has no host. These are the bytes that replace it.
;
; The reference is tools/bench/dlxload.py and it stays the reference: this code
; is gated BYTE-FOR-BYTE against it (tools/bench/verify_load.py), palette words
; and darkest-entry index included. If the two ever disagree, the symptom in a
; rig would be wrong colours rather than a crash, which is exactly the class of
; bug the split was made to prevent.
;
; WHAT IT READS. The RAW container as it comes off the disc. The DLX header is
; fixed-layout and big-endian (tools/encoder/dlx.py):
; +0 magic 'DLX3' +12 k1 u16 +16 off_pal u32
; +4 W u16 +14 k4 u16 +20 off_cb1 u32
; +6 H u16 +24 off_cb4 u32
; +8 fps u16 +28 off_frm u32
; +10 nframes u16
; The three offsets are container-relative, so every one of them is an add of
; the base the loader was handed. Nothing here parses a frame record.
;
; WHAT IT WRITES. CB1 (8 KB) and CB4 (2 KB) expanded to one WORD per pixel at
; the addresses geom.i names, and 256 packed palette words straight into the
; graphics palette at $E82000. It also reports the darkest entry, which is what
; the letterbox is filled with until the encoder reserves a black one (23.4,
; still open).
;
; WHY WORD-PER-PIXEL. The block loop movems codebook entries straight into
; GVRAM with no unpacking, and the high byte of a GVRAM word write is discarded
; by the hardware, so the high byte is left zero and never has to be cleared.
; It also makes index scaling a shift rather than a multiply (lsl.w #5 / #3).
;
; SCRATCH. Three tables, built here and dead the moment the palette is packed:
; P6TAB 64 B 6-bit level -> the 8-bit value the hardware renders it as
; SQTAB 256 B the square of that, so the darkest-entry search has no muls
; DTAB 512 B err(v, I=0) - err(v, I=1) per 8-bit channel value, signed
; DTAB is what turns P2's per-entry minimum-squared-error choice of the shared
; LSB into three table reads and a sign test. Choosing I per entry rather than
; fixing it is worth 1.96 dB (FINDINGS 23.3), and it is a per-ENTRY decision
; across three channels, so it cannot be folded into a per-channel table alone.
; ---------------------------------------------------------------------------
LFLAG = $18040 ; 0 idle / 1 running / $FF done / $EE bad header
LHDR = $18044 ; -> raw container base
LDARK = $18048 ; <- index of the darkest palette entry
LK1 = $1804C ; <- k1, as the 68000 read it out of the header
LK4 = $18050 ; <- k4
LMODE = $18054 ; bit0 codebooks, bit1 palette entries,
; bit2 the three scratch tables
LITER = $18058 ; repeat count, so a 55 Hz host clock can time it
P6TAB = $19000 ; 64 bytes
SQTAB = $19040 ; 64 longs
DTAB = $19140 ; 256 words
GPAL = $E82000 ; graphics palette, 256 words
; ---------------------------------------------------------------- do_load
; in: a0 = container base, d1 = mode bits: 1 codebooks, 2 palette entries,
; 4 the scratch tables. A player builds the tables ONCE at boot (they
; describe the hardware's colour rendering and nothing about the scene) and
; then loads each scene with 3.
; out: d0 = 0 ok, -1 not a DLX3/DLX4 container. a0-a4 clobbered, a5 = base.
;
; The magic is accepted as 'DLX' plus a version byte of '3' OR ABOVE rather than
; as one constant. DLX4 (ROADMAP P5) adds the per-record index and a fifth
; header offset at +32; every field this routine reads is at its DLX3 place, so
; the transforms are version-independent and the check should be too. A version
; this loader has never seen is still refused -- '3' or above, not "anything
; that begins DLX".
do_load:
movea.l a0,a5
move.l (a5),d0
andi.l #$FFFFFF00,d0
cmpi.l #$444C5800,d0 ; 'DLX'
bne .bad
cmpi.b #'3',3(a5) ; ... version 3 or above
bcs .bad
move.w 12(a5),d0
ext.l d0
move.l d0,LK1.l
move.w 14(a5),d0
ext.l d0
move.l d0,LK4.l
btst #2,d1
beq.s .notab
move.l d1,-(sp)
bsr pal_tables
move.l (sp)+,d1
.notab:
btst #0,d1
beq.s .nocb
moveq #0,d2 ; the count is built as a LONG and the
move.w 12(a5),d2 ; high word must not carry junk into it
lsl.l #4,d2 ; k1 entries x 16 source bytes
movea.l 20(a5),a0
adda.l a5,a0
lea CB1,a1
bsr expand
moveq #0,d2
move.w 14(a5),d2
lsl.l #2,d2 ; k4 entries x 4 source bytes
movea.l 24(a5),a0
adda.l a5,a0
lea CB4,a1
bsr expand
.nocb:
btst #1,d1
beq.s .nopal
bsr pal_pack
.nopal:
moveq #0,d0
rts
.bad: moveq #-1,d0
rts
; ---------------------------------------------------------------- expand
; One source byte -> one destination word, high byte zero.
; in: a0 src, a1 dst, d2 = source byte count. Always a multiple of 4: CB1 is
; k1*16 and CB4 is k4*4, so no remainder case can exist and none is written.
; A junk high word here is not a slow path, it is a WRONG one: `lsr.l #2` walks
; two of its bits down into the low word and the dbra count comes out long.
expand:
lsr.l #2,d2
subq.l #1,d2 ; k<=256, so the count fits a dbra
moveq #0,d0
.e1: move.b (a0)+,d0
move.w d0,(a1)+
move.b (a0)+,d0
move.w d0,(a1)+
move.b (a0)+,d0
move.w d0,(a1)+
move.b (a0)+,d0
move.w d0,(a1)+
dbra d2,.e1
rts
; ---------------------------------------------------------------- pal_tables
; The three scratch tables. SCENE-INDEPENDENT, every one of them: they describe
; how the CRTC renders a 5-bit channel plus the shared LSB, which is a property
; of the machine. A player builds them once at boot and never again, which is
; why they are a separate entry point rather than the head of pal_pack -- see
; FINDINGS 53.3 for what that is worth.
pal_tables:
; -- P6TAB[x] = ((x<<2)|(x>>4)) & $FF, and SQTAB[x] = P6TAB[x]^2
lea P6TAB,a0
lea SQTAB,a1
moveq #0,d1
.p1: move.w d1,d0
lsl.w #2,d0
move.w d1,d2
lsr.w #4,d2
or.w d2,d0
andi.w #$FF,d0
move.b d0,(a0)+
move.w d0,d2
mulu d2,d2
move.l d2,(a1)+
addq.w #1,d1
cmpi.w #64,d1
bne.s .p1
; -- DTAB[v] = (render(v,0)-v)^2 - (render(v,1)-v)^2, signed
lea P6TAB,a0
lea DTAB,a1
moveq #0,d1
.p2: move.w d1,d2
lsr.w #2,d2
andi.w #$3E,d2 ; x0 = (v>>3)<<1
moveq #0,d3
move.b 0(a0,d2.w),d3
sub.w d1,d3
muls d3,d3
moveq #0,d4
move.b 1(a0,d2.w),d4
sub.w d1,d4
muls d4,d4
sub.l d4,d3
move.w d3,(a1)+
addq.w #1,d1
cmpi.w #256,d1
bne.s .p2
rts
; ---------------------------------------------------------------- pal_pack
; 24-bit RGB -> GGGGGRRRRRBBBBBI, the shared LSB chosen per entry by minimum
; squared error, written to the palette registers. Identical arithmetic to
; dlxload.pack_palette, including its tie-breaks: I stays 0 when the two errors
; are equal, and the darkest entry is the FIRST index at the minimum.
; in: a5 = container base, and pal_tables already run.
pal_pack:
movea.l 16(a5),a0
adda.l a5,a0 ; -> 256 x RGB888
lea GPAL,a1
lea DTAB,a2
lea SQTAB,a4 ; P6TAB is not needed here: the rendered
; value is only ever wanted SQUARED
move.l #$7FFFFFFF,d6
clr.l LDARK.l
moveq #0,d7
.p3: moveq #0,d1
move.b (a0)+,d1 ; R
moveq #0,d2
move.b (a0)+,d2 ; G
moveq #0,d3
move.b (a0)+,d3 ; B
move.w d1,d0
add.w d0,d0
move.w 0(a2,d0.w),d4
move.w d2,d0
add.w d0,d0
add.w 0(a2,d0.w),d4
move.w d3,d0
add.w d0,d0
add.w 0(a2,d0.w),d4 ; sum of err0-err1 over the three
moveq #0,d5
tst.w d4
ble.s .p4
moveq #1,d5 ; I=1 only when it is STRICTLY better
.p4: lsr.w #3,d1 ; fR
lsr.w #3,d2 ; fG
lsr.w #3,d3 ; fB
move.w d2,d4
lsl.w #5,d4
or.w d1,d4
lsl.w #6,d4 ; (fG<<11)|(fR<<6)
move.w d3,d0
add.w d0,d0
or.w d0,d4
or.w d5,d4
move.w d4,(a1)+ ; -> the palette register
add.w d1,d1 ; x = (f<<1)|I, per channel
or.w d5,d1
add.w d2,d2
or.w d5,d2
add.w d3,d3
or.w d5,d3
lsl.w #2,d1 ; SQTAB holds longs
move.l 0(a4,d1.w),d0
lsl.w #2,d2
add.l 0(a4,d2.w),d0
lsl.w #2,d3
add.l 0(a4,d3.w),d0 ; squared distance from black
cmp.l d6,d0
bge.s .p5
move.l d0,d6
move.l d7,LDARK.l ; first index at the minimum wins
.p5: addq.w #1,d7
cmpi.w #256,d7
bne .p3
rts
src/player/loadgate.s
+39
View File
@@ -0,0 +1,39 @@
; Front-end for the load-time transforms (ROADMAP P1+P2), for the rig.
;
; It is to load.i what decode.s is to frame.i: a timing and control wrapper that
; does nothing the shipping player would not do, so that the bytes being
; measured are the bytes that will ship. The player's own boot path will call
; do_load once with the mode bits set to 3; this repeats it LITER times so a
; host clock with 1/56.69 s granularity (tools/bench/crtc_mode.lua) can time a
; job that takes milliseconds,
; and splits it by LMODE so the codebook expansion and the palette pack can be
; priced apart. A player calls do_load with mode 7 once at boot -- the three
; scratch tables describe the machine, not the scene -- and with mode 3 at every
; scene change after that.
;
; Repeating is honest here in a way it would not be for a frame: nothing in
; do_load is temporally recursive. Pass n writes exactly what pass n-1 wrote,
; over the top of it, out of the same source bytes.
include "src/player/geom.i"
org $10000
start:
move.l LMODE.l,d1
move.l LITER.l,d3
move.l #1,LFLAG.l ; timer starts here
loop:
movem.l d1/d3,-(sp)
movea.l LHDR.l,a0
bsr do_load
movem.l (sp)+,d1/d3
tst.l d0
bne.s bad
subq.l #1,d3
bne.s loop
move.l #$FF,LFLAG.l ; timer stops here
hold: bra.s hold
bad: move.l #$EE,LFLAG.l
bra.s hold
include "src/player/load.i"
src/player/packed.s
+1308
View File
File diff suppressed because it is too large Load Diff
src/player/ring.i
+506
View File
@@ -0,0 +1,506 @@
; ---------------------------------------------------------------- ring.i
; The RING PRODUCER, on the 68000. ROADMAP P5.
;
; WHAT MOVED. FINDINGS 49 and 51 measured a ring that a HOST filled:
; tools/bench/stream.lua held the record index, decided where each record went,
; wrote the descriptor and advertised it. The 68000 only ever consumed. That
; is the same shape session 21 found in the loader and session 22 in the frame
; clock -- a policy living outside the machine that has to run inside it -- and
; it is the last one in the delivery path. A player has no host to place its
; records.
;
; So the placement policy is here now, and the host keeps only the part that is
; genuinely not the CPU's: moving bytes off a disc at a rate. What the rig
; supplies is a TRANSPORT, one request at a time, which is what a single SPC and
; one DMAC channel are (FINDINGS 52.5); what this file supplies is every
; decision about WHICH record, WHERE in the ring, and WHEN it is safe to start.
;
; THE POLICY IS `aligned`, and it is the same one 19_ring_stream.py scored and
; 49.3 chose: never start a record that will not finish before the end of the
; ring; leave the hole, restart at the base. It costs a mean hole of 5.7% of
; the ring on the gate container and ZERO clocks in the block loop, against
; `split`'s 3.64% of every frame budget forever.
;
; WHY IT NEEDS AN INDEX, and why that is a container change (DLX4). `aligned`
; asks "does the NEXT record fit before the end of the ring", which is a
; question about a record's length asked BEFORE it is fetched. Every reader in
; this tree learned record lengths by walking the frame stream -- reading each
; record's length word to find the next -- and that is exactly what a player
; streaming off a disc cannot do: the length word of record i+1 is one of the
; bytes it has not fetched yet. DLX4 puts nframes u16 longword-counts in the
; scene header for this, and the same table gives a branch point the disc
; address of an arbitrary record without reading what lies between (`ring_seek`).
;
; THE HANDSHAKE WITH THE DECODER IS UNCHANGED, deliberately. FR_HEAD/FR_TAIL/
; DESC[] are the same words src/player/stream.s already reads, written in the
; same order, so the decoder cannot tell a host-filled ring from a self-filled
; one -- which is what makes the self-filled run a test of THIS file and not of
; a new rig. Two monotonic counters, single reader, single writer, no atomics.
;
; THE TRANSPORT MAILBOX, XF_QD outstanding requests:
; XF_SLOT[seq & 1] what to fetch, where to put it, how much, and which
; record it is
; XF_GO requests issued -- bumped LAST, after the slot
; XF_ACK requests completed, in order, by the transport
; In the player XF_* is an MB89352 command and a DMAC channel; here it is
; tools/bench/stream.lua delivering at a modelled rate. Either way the CPU
; issues and polls, and the bytes arrive on somebody else's time.
;
; AND THAT IS WHERE THE COST IS. The channel only moves bytes while a request
; is outstanding, and only the CPU can issue the next one. Between the
; completion of record i and the issue of record i+1 the disc is IDLE, and the
; length of that gap is a property of the PLAYER's loop, not of the medium.
; `ring_poll` is therefore called from the pace wait -- the idle the frame clock
; already creates -- rather than once a frame: once a frame would cap the fill
; at one record per slot, which is the wire rate exactly, and a ring that can
; only just keep up can never accumulate the slack a branch point spends
; (51.3). A frame that uses its whole slot does not merely present late
; (54.4); it stops the disc for a frame time. The rig counts that gap.
; ---- transport mailbox. TWO REQUEST SLOTS, and the depth is a knob.
; A channel only moves bytes while it has a request, and only the CPU can give
; it one. With ONE slot the disc stands still from the moment a transfer
; completes until the player next polls -- and a player polls in its idle, which
; is the end of a frame slot, so the gap is up to a whole frame's decode. With
; TWO the next request is already queued when the current one lands and the
; channel need never stop. XF_QD selects which, so the cost of the first is
; measurable against the second in one rig rather than argued about.
XF_SLOT = $18300 ; 2 x 16 B: u32 disc offset, u32 destination,
; u32 length, u32 record index
XF_SLSZ = 16
XF_SLM = 1 ; slot = sequence & XF_SLM
XF_GO = $18320 ; u32 requests ISSUED, written by the 68000
XF_ACK = $18324 ; u32 requests COMPLETED, written by transport
XF_QD = $18328 ; u32 queue depth, 1 or 2 (input)
; ---- producer state
RINGOWN = $1832C ; 1 = the 68000 owns placement (this file)
RNG_B = $18330 ; ring base address
RNG_SZ = $18334 ; ring size in bytes
IDX_B = $18338 ; base of the DLX4 record index, nframes u16
RQ_NEXT = $1833C ; next record to REQUEST
WCUR = $18340 ; write cursor, a ring OFFSET
RCUR = $18344 ; read cursor: ring offset of the oldest record
; the decoder has not finished with
RTAILN = $18348 ; records RCUR has stepped over; chases FR_TAIL
NRETIRE = $1834C ; requests this file has published; chases XF_ACK
DOFF = $18350 ; running disc offset of record RQ_NEXT
; ---- instruments. None of these is read by the policy.
N_HOLE = $18354 ; wraps that left a hole
N_HOLEB = $18358 ; total bytes in those holes
N_FULL = $1835C ; polls that refused for SPACE (ring-bound)
N_POLL = $18360 ; ring_poll calls
N_ISSUE = $18364 ; requests issued
SLK_MIN = $18368 ; least slack seen at a frame start, in records
SLK_AT = $1836C ; and the frame it was seen at
PF_REC = $18370 ; prefill target, in whole records (input)
PF_DONE = $18374 ; records resident when the prefill released
N_SEEK = $18378 ; ring_seek calls
SK_WAIT = $1837C ; polls spent waiting for the channel to go
; quiet before the last seek could start
ROFF = $19400 ; u32 per record: disc offset, built at load.
; The seek half of the index -- a running sum
; is enough to PLAY, but a branch point needs
; record j's address without summing to it.
ROFFMAX = 1024 ; entries; $19400..$1A400, below CB1 at $20000
RPOLLMAX = 4000000 ; ring_poll calls with no progress before the
; producer is declared wedged
; ---------------------------------------------------------------- ring_init
; in: IDX_B, RNG_B, RNG_SZ, NFR set by the caller.
; out: d0 = 0 ok, -1 the index is longer than ROFF can hold. Builds the disc
; offset table and leaves the ring empty at record 0.
; Clobbers d0-d2/a0-a1.
ring_init:
move.l NFR.l,d0
cmpi.l #ROFFMAX,d0
bhi .toobig
movea.l IDX_B.l,a0
lea ROFF.l,a1
moveq #0,d1 ; running disc offset
move.l d0,d2
beq.s .noidx
.sum: move.l d1,(a1)+
moveq #0,d0
move.w (a0)+,d0 ; longwords in this padded record
lsl.l #2,d0
add.l d0,d1
subq.l #1,d2
bne.s .sum
.noidx:
clr.l NRETIRE.l
clr.l N_HOLE.l
clr.l N_HOLEB.l
clr.l N_FULL.l
clr.l N_POLL.l
clr.l N_ISSUE.l
clr.l N_SEEK.l
clr.l SK_WAIT.l
clr.l XF_GO.l
clr.l XF_ACK.l
move.l #$7FFFFFFF,SLK_MIN.l
move.l #-1,SLK_AT.l
moveq #0,d0
bsr ring_seek ; a scene starts with a seek to record 0
moveq #0,d0
rts
.toobig:
moveq #-1,d0
rts
; ---------------------------------------------------------------- ring_seek
; in: d0 = record index to play from.
; out: the ring is empty, the cursors are at its base, and the next request
; will be for record d0. FR_HEAD is reset; the CALLER must reset FR_TAIL
; (it is the decoder's word, and this file never writes the decoder's).
;
; A SEEK CANNOT START WHILE THE CHANNEL IS BUSY. An outstanding request is
; bytes already on their way to an address this routine is about to declare
; free, so it is waited out and thrown away rather than cancelled -- a real
; SPC would need the transfer aborted and the bus handed back before a new
; command, and waiting is the version of that a rig can be honest about. What
; it costs is up to one record's delivery time, charged to the seek, and
; SK_WAIT counts the polls it took.
; Clobbers d0-d2/a0.
ring_seek:
movem.l d0-d2/a0,-(sp)
addq.l #1,N_SEEK.l
clr.l SK_WAIT.l
.wait: move.l XF_ACK.l,d1
cmp.l XF_GO.l,d1
beq.s .quiet
addq.l #1,SK_WAIT.l
bsr xf_service ; the transport hook, and here it is not
; optional: with the transport INSIDE
; the machine (src/player/xfer.i) the
; only thing that can retire the
; outstanding request is this loop, so
; without it a seek issued with a
; request in flight spins forever. A
; host transport retired it on its own
; time, which is exactly the kind of
; difference the seam exists to hide and
; this one it could not.
bra.s .wait
.quiet:
move.l XF_GO.l,NRETIRE.l ; whatever landed belongs to the scene
; we came FROM, and is discarded
move.l d0,RQ_NEXT.l
lsl.l #2,d0
lea ROFF.l,a0
move.l (a0,d0.l),DOFF.l ; the index's second job: record j's
; disc address without reading to it
clr.l WCUR.l
clr.l RCUR.l
clr.l RTAILN.l
clr.l FR_HEAD.l
movem.l (sp)+,d0-d2/a0
rts
; ---------------------------------------------------------------- ring_poll
; Advance the producer by at most one step: retire a completed request, catch
; the read cursor up with the decoder, and issue the next request if one fits.
; Preserves every register -- it is called from inside the decoder's wait loops
; and must be invisible to them.
ring_poll:
movem.l d0-d3/a0-a1,-(sp)
addq.l #1,N_POLL.l
; ---- 0. the TRANSPORT, if it lives in this machine. src/player/xfer.i answers
; at most one outstanding request per call and preserves every register;
; with XF_SCSI = 0 it is a tst and a branch, and the host is the transport
; exactly as it was in FINDINGS 55. It goes BEFORE the retire step so that
; a transfer completed here is published in the same poll.
bsr xf_service
; ---- 1. retire. The descriptor is written BEFORE the count that advertises
; it, which is the same order tools/bench/stream.lua used and the reason
; src/player/stream.s reads them the other way round.
move.l NRETIRE.l,d3
.retire:
cmp.l XF_ACK.l,d3
bcc.s .retired ; d3 >= XF_ACK: nothing new has landed
move.l d3,d0
and.l #XF_SLM,d0
lsl.l #4,d0 ; * XF_SLSZ
lea XF_SLOT.l,a1
adda.l d0,a1 ; a1 = the completed request's slot
move.l 12(a1),d0 ; its record index
lsl.l #2,d0
and.w #DESCM,d0
lea DESC.l,a0
move.l 4(a1),(a0,d0.w) ; its destination -> the descriptor
addq.l #1,FR_HEAD.l ; ...advertised only after the address
addq.l #1,d3
bra.s .retire
.retired:
move.l d3,NRETIRE.l
; ---- 2. catch the read cursor up. The decoder publishes FR_TAIL and nothing
; else the producer needs: RCUR walks the SAME placement rule the writer
; used, so it steps over the holes in exactly the places they were left.
; That is what makes the free space a single circular gap rather than a
; list of live records -- the host producer kept a list because it could
; afford to.
; THE RULE MUST BE THE WRITER'S, APPLIED TO THE SAME RECORD. Stepping
; the reader past record i lands on the END of record i, which is where
; record i+1 went only if i+1 FITTED there -- and if it did not, the writer
; put it at the ring base and left a hole. So the wrap is decided by the
; length of the record being stepped ONTO, exactly as the placement was.
;
; Deciding it with the wrong record's length was a real bug and not a
; conservative one: it left RCUR pointing into the hole, and one more
; retirement then pushed it past the end of the ring and wrapped it to a
; low address unrelated to any record. The live span computed from that is
; SHORTER than the truth, so the producer places on top of a record the
; decoder has not finished, and the symptom is a bitstream desync -- the
; decoder's a0 walking off the end of a record that changed underneath it.
move.l RTAILN.l,d1
.catch: cmp.l FR_TAIL.l,d1
bcc.s .caught
movea.l IDX_B.l,a0
move.l d1,d0
add.l d0,d0
moveq #0,d2
move.w (a0,d0.l),d2
lsl.l #2,d2 ; length of the record being retired
move.l RCUR.l,d0
add.l d2,d0 ; d0 = one past its end
addq.l #1,d1
cmp.l NFR.l,d1
bcc.s .last ; nothing follows it in this scene
move.l d1,d2
add.l d2,d2
moveq #0,d3
move.w (a0,d2.l),d3
lsl.l #2,d3 ; length of the record after it
add.l d0,d3
cmp.l RNG_SZ.l,d3
bls.s .last
moveq #0,d0 ; it did not fit: the writer restarted
; at the base, so the reader does too
.last:
move.l d0,RCUR.l
bra.s .catch
.caught:
move.l d1,RTAILN.l
; ---- 3. issue, if there is anything left and it fits.
move.l RQ_NEXT.l,d1
cmp.l NFR.l,d1
bcc .out ; whole scene requested
; ---- MEASURED AGAINST WHAT HAS BEEN RETIRED, NOT WHAT HAS BEEN ACKED, and
; the difference is a slot. A request's slot stays in use until this file
; has read the record index and destination out of it -- which happens in
; step 1 above, one poll later than the ack at the earliest. Gating on
; XF_ACK let the CPU write a slot whose descriptor had not been published
; yet: the transport had finished the transfer, the retire loop then read
; the OVERWRITTEN slot, and DESC for that frame stayed zero. The decoder
; duly decoded address zero and reported a bitstream desync.
move.l XF_GO.l,d0
sub.l NRETIRE.l,d0 ; slots still spoken for
cmp.l XF_QD.l,d0
bcc .out ; the queue is as deep as it may go
move.l d1,d0
add.l d0,d0
movea.l IDX_B.l,a0
moveq #0,d2
move.w (a0,d0.l),d2
lsl.l #2,d2 ; d2 = length to place
; ---- WHERE IT GOES, AND WHETHER IT MAY. The live bytes are the circular
; interval [RCUR, WCUR) -- oldest record the decoder has not finished with,
; up to the write cursor -- so the FREE bytes are its complement, and a
; record has to fit in ONE piece of it because the block loop reads with a
; monotonically increasing a0 (49.2).
;
; There are three shapes and they are not symmetric, which is the trap:
; empty the whole ring is free
; RCUR <= WCUR live is one run; free is [WCUR, SZ) THEN [0, RCUR),
; so a record that will not fit before the end may
; restart at the base -- this is `aligned`, and the
; skipped bytes are the hole
; RCUR > WCUR LIVE is the one that wraps; free is only [WCUR, RCUR)
; and the ring base is NOT ours -- a record that will
; not fit must simply wait
; Deciding the wrap from `WCUR + len > SZ` alone, before knowing which
; shape it is, was the second bug in this file: in the third shape it
; restarted at a base that was live and overwrote records the decoder had
; not read, and the symptom was a bitstream desync rather than a fault.
move.l WCUR.l,d3 ; d3 = candidate offset
moveq #0,d0 ; d0 = hole bytes, if any
move.l RQ_NEXT.l,d1
cmp.l RTAILN.l,d1
beq.s .isempty
move.l RCUR.l,d1
cmp.l d3,d1
beq .full ; RCUR == WCUR and not empty: the ring
; is completely full
bhi.s .freehi
; RCUR < WCUR: free is [WCUR, SZ) then [0, RCUR).
move.l d3,d0
add.l d2,d0
cmp.l RNG_SZ.l,d0
bls.s .nohole2 ; fits before the end of the ring
move.l RCUR.l,d0
cmp.l d2,d0
bcs .full ; it will not fit at the base either
move.l RNG_SZ.l,d0
sub.l d3,d0 ; the hole `aligned` is about to leave.
; Charged here, where the record is
; actually PLACED, and not where the
; wrap is decided: charging it at the
; decision counts one hole per retry
; while the decoder still owns the base,
; which is every poll of a fast pipe,
; and reported 105 wraps where there
; are 18.
moveq #0,d3
bra.s .place
.nohole2:
moveq #0,d0
bra.s .place
.freehi:
; RCUR > WCUR: the LIVE span wraps, so the only free run is [WCUR, RCUR).
move.l d3,d0
add.l d2,d0
cmp.l RCUR.l,d0
bhi .full
moveq #0,d0
bra.s .place
.isempty:
; Nothing live, so the whole ring is free and the live span is about to start
; here. Moving RCUR is what keeps the invariant true across a drained ring;
; without it the reader's cursor would still point at the last consumed record.
move.l d3,d0
add.l d2,d0
cmp.l RNG_SZ.l,d0
bls.s .enohole
move.l RNG_SZ.l,d0
sub.l d3,d0
moveq #0,d3
bra.s .esetr
.enohole:
moveq #0,d0
.esetr:
move.l d3,RCUR.l
.place:
tst.l d0
beq.s .nohole
addq.l #1,N_HOLE.l
add.l d0,N_HOLEB.l
.nohole:
move.l XF_GO.l,d1
and.l #XF_SLM,d1
lsl.l #4,d1 ; * XF_SLSZ
lea XF_SLOT.l,a1
adda.l d1,a1
move.l DOFF.l,(a1)
move.l RNG_B.l,d0
add.l d3,d0
move.l d0,4(a1)
move.l d2,8(a1)
move.l RQ_NEXT.l,12(a1)
add.l d2,d3
move.l d3,WCUR.l
add.l d2,DOFF.l
addq.l #1,RQ_NEXT.l
addq.l #1,N_ISSUE.l
addq.l #1,XF_GO.l ; LAST: the three words above must be
; visible before the request is
bra.s .out
.full:
addq.l #1,N_FULL.l
.out:
movem.l (sp)+,d0-d3/a0-a1
rts
; ---------------------------------------------------------------- ring_prefill
; Fill until PF_REC whole records are resident, then return. THE POLICY, not a
; convenience: the decoder must not be released at slack 1, because 51.2
; measured that n resident records buy n-1 frame times of stall -- the last one
; is spent covering the pipe's restart. Releasing at 1 therefore starts a scene
; with a stall budget of zero, and the first hiccup is an underrun.
;
; It is also the ONLY place a player can buy lookahead cheaply. 51.3: slack is
; accumulated out of `pipe - wire` over seconds of play, so a scene that starts
; empty climbs for 4.83 s at 488 KB/s before it can afford a branch. Bytes
; bought here are bought before the frame clock starts and cost nothing but the
; wait -- which is the one moment in a scene when the decoder has nothing else
; to do anyway.
;
; out: d0 = 0 ok, -1 the transport never delivered. PF_DONE = records resident.
ring_prefill:
movem.l d1-d2,-(sp)
moveq #0,d1
.loop: bsr ring_poll
move.l FR_HEAD.l,d0
sub.l FR_TAIL.l,d0
cmp.l PF_REC.l,d0
bcc.s .done
addq.l #1,d1
cmp.l #RPOLLMAX,d1
bcs.s .loop
movem.l (sp)+,d1-d2
moveq #-1,d0
rts
.done: move.l d0,PF_DONE.l
movem.l (sp)+,d1-d2
moveq #0,d0
rts
; ---------------------------------------------------------------- ring_slack
; out: d0 = whole records resident and unconsumed.
;
; This is the number FINDINGS 51 spent a session establishing the meaning of,
; and it is only worth what it is worth when the decoder is PACED: free-running,
; the decoder outruns any pipe and the ring never backs up, so the difference is
; a statement about earliness (49.7.2).
ring_slack:
move.l FR_HEAD.l,d0
sub.l FR_TAIL.l,d0
rts
; ---------------------------------------------------------------- ring_may_seek
; in: d0 = the stall a branch would cost, in whole frame times.
; out: d0 = 0 the ring can cover it, -1 it cannot. Z set on ok.
;
; THE RULE, from 51.2, measured and not assumed: n resident records buy n-1
; frame times, because the record due immediately after the pipe restarts is
; still arriving when its slot opens. A design that reads the resident count
; as its stall budget is over by one record every time.
;
; What a player does with a `no` is not this file's business -- delay the
; branch, take the outcome that needs no seek, or accept a late present -- but
; it must be able to ASK, and until now the answer only existed in the rig's
; log.
ring_may_seek:
move.l d1,-(sp)
move.l FR_HEAD.l,d1
sub.l FR_TAIL.l,d1
beq.s .no ; nothing resident: the subtraction
; below would wrap to $FFFFFFFF and an
; unsigned compare would then answer YES
; to any request, from an empty ring
subq.l #1,d1 ; the restart record is not spendable
cmp.l d0,d1
bcs.s .no
move.l (sp)+,d1
moveq #0,d0
rts
.no: move.l (sp)+,d1
moveq #-1,d0
rts
; ---------------------------------------------------------------- ring_mark
; Sample the slack at a frame boundary, for the instruments only.
; in: d0 = frame number. Clobbers nothing.
ring_mark:
movem.l d0-d1,-(sp)
move.l FR_HEAD.l,d1
sub.l FR_TAIL.l,d1
cmp.l SLK_MIN.l,d1
bcc.s .out
move.l d1,SLK_MIN.l
move.l d0,SLK_AT.l
.out: movem.l (sp)+,d0-d1
rts
src/player/scsi.i
+695
View File
@@ -0,0 +1,695 @@
; ---------------------------------------------------------------- scsi.i
; The MB89352 TRANSPORT, on the 68000. ROADMAP P4.
;
; WHAT THIS REPLACES. src/player/ring.i decides which record to fetch, where in
; the ring to put it and when it is safe; it hands that to a TRANSPORT through
; the XF_* mailbox and polls for completion. Until now the transport was
; tools/bench/stream.lua -- a host moving bytes at a modelled rate. A player has
; no host. This file is the transport: a Fujitsu MB89352 SPC and, later, one
; HD63450 channel.
;
; THE REGISTER MAP IS MEASURED, NOT ASSUMED. src/player/scsigate.s probes
; $EA0000..$EA003F one address at a time and survives the bus errors, and the
; map below is what answered:
;
; registers sit on the ODD bytes, $EA0001 + 2n, n = 0..14
; n=3 (TMOD) and n=15 (EXBF) BUS ERROR -- the MB89352 omits both, where the
; MB87030 has them, and MAME leaves HOLES rather than shifting the
; later indices down. DREG is index 10 at $EA0015 either way, which is the
; one address FINDINGS 32.4 had quoted.
; TEMP ($EA0017) took $A5 and gave it back, so these are registers and not a
; mirror of something.
;
; That last point is why the probe was worth a run: MAME's own device summary
; says the MB89352 "shifts subsequent indices", and the machine says it does
; not. The bytes win.
;
; THE DATA REGISTER IS DMA-ONLY, AND THAT IS NOT A CHOICE THIS CODE MADE.
; x68k_scsiext.cpp puts its own glue on $EA0015 and on no other address:
;
; write: if (exown()) { if (!drq) dtack_w(1); else dma_w(data); }
; else dreg_w(data);
;
; On this machine `exown()` -- the HD63450's OWN, fed back to the slot by
; x68k.cpp -- is asserted where a PIO write needs it not to be, so the `else`
; arm is unreachable and a byte written to $EA0015 with the SPC in PROGRAM
; transfer mode is DISCARDED. Silently: no error bit, no status change, no
; interrupt. It was measured rather than reasoned about -- scsigate.s writes
; $5A to $EA0015 and reads it straight back, and gets $00 with the FIFO still
; empty -- because ten command bytes vanishing without trace looks exactly like
; a target refusing a command.
;
; So every transfer here issues SCMD WITHOUT the PROGRAM bit, which puts the SPC
; in DMA mode and makes it raise DRQ; the CPU then moves the bytes through
; $EA0015 itself and they go in via `dma_w`/`dma_r`. The CPU is standing in for
; the DMAC, through the DMAC's own door.
;
; WHAT THAT COSTS THE ARGUMENT, stated because it is easy to overclaim here:
; with `exown` asserted at idle, MAME cannot distinguish a CPU-driven byte at
; $EA0015 from a DMAC-driven one. So this rig demonstrates THE DATA PATH and
; cannot, on its own, demonstrate that the HD63450 is the thing driving it.
; Whether a real CZ-6BS1 also refuses PIO here is NOT settled by this -- it is a
; property of MAME's model, and it wants a board (ROADMAP B1/B3).
;
; PIO FIRST, DMA SECOND, DELIBERATELY. The thing P4 has to demonstrate is a
; DMAC configuration that HOLDS THE BUS (ROADMAP: "getting the DMAC to hold the
; bus is the difference between 9 and 19 clocks per byte, and demonstrating a
; configuration that does it is P4's first job"). But a DMA bring-up that fails
; cannot tell "the SCSI protocol is wrong" from "the DMAC is misprogrammed". So
; the protocol is settled in PIO, where every byte is the CPU's and nothing else
; can be blamed, and only then does the data phase move to the channel.
;
; NOTHING HERE IS A RATE MEASUREMENT, and it cannot become one. MAME's device
; models are functional, not transfer-timing accurate (docs/BENCHMARK.md), and
; 42.5 reads its DMAC configured in wall-clock attotimes rather than per-operand
; cycles. `W` -- clocks stolen per delivered byte -- is untouched by every line
; below. What this settles is which handshake the player's own code provokes.
; ---- the SPC, at the CZ-6BS1's decode
SPCB = $EA0001 ; register 0; stride 2, odd lane
SC_BDID = SPCB+0 ; own ID (write the NUMBER; reads a MASK)
SC_SCTL = SPCB+2
SC_SCMD = SPCB+4
; SPCB+6 = TMOD, ABSENT on the MB89352 -- reading it BUS ERRORS
SC_INTS = SPCB+8
SC_PSNS = SPCB+10
SC_SSTS = SPCB+12
SC_SERR = SPCB+14
SC_PCTL = SPCB+16
SC_MBC = SPCB+18
SC_DREG = SPCB+20 ; $EA0015, and the DMAC's single address
SC_TEMP = SPCB+22
SC_TCH = SPCB+24
SC_TCM = SPCB+26
SC_TCL = SPCB+28
; SPCB+30 = EXBF, ABSENT -- reading it BUS ERRORS
; SCTL
SCTL_RESET = $80 ; reset & disable
; SCMD, command in bits 7-5
SCMD_RELEASE = $00 ; command 000, let go of the bus
SCMD_RSTACK = $C0 ; command 110, drop ACK/REQ
SCMD_SELECT = $20
SCMD_RSTATN = $40 ; command 010, drop ATN
SCMD_XFER = $80
SCMD_PROGRAM = $04 ; set = PIO, clear = DMA. NOT USED, and
; the reason is the whole of 57.x -- see
; "THE DATA REGISTER IS DMA-ONLY" above.
; INTS
INTS_RESET = $01
INTS_HARDERR = $02
INTS_TIMEOUT = $04
INTS_SERVICE = $08
INTS_CMDCOMP = $10
INTS_DISCON = $20
; SSTS
SSTS_DREG_E = $01 ; DREG empty
SSTS_DREG_F = $02 ; DREG full
SSTS_TC0 = $04
SSTS_BUSY = $20
SSTS_INITCON = $80
; SCSI bus phases, as PSNS bits 2..0 and as PCTL's low three
PH_DATAOUT = 0
PH_DATAIN = 1
PH_CMD = 2
PH_STATUS = 3
PH_MSGOUT = 6
PH_MSGIN = 7
SCSI_ID = 7 ; the player is the initiator
SCSI_TGT = 0 ; the disc
; ---- error codes, reported through SC_ERR
SCE_OK = 0
SCE_SELTMO = 1 ; the target never answered selection
SCE_PHASE = 2 ; the bus went somewhere unexpected
SCE_TIMEOUT = 3 ; a poll loop ran out of patience
SCE_STATUS = 4 ; the target returned non-zero status
SCE_WINDOW = 5 ; a WINDOWED read was asked of the DMAC,
; which cannot drop bytes (58.3/P4a)
SC_ERR = $18200 ; u32 last error
SC_STAT = $18204 ; u32 SCSI status byte from the last cmd
SC_PH = $18208 ; u32 phase we were in when it went wrong
SC_CDB = $18210 ; 12 B command block, built here
SC_MSG = $1821C ; 4 B message byte, either direction
; ---- THE RECORD WINDOW, and why a transport needs one. ROADMAP P4b.
; src/player/ring.i asks for a RECORD: a byte offset into the scene's frame
; stream and a length, both of them 4-byte aligned and neither of them a
; multiple of 512. A SCSI target deals in BLOCKS. On the gate container 117
; of 120 records start part way into a sector, so a transport that reads only
; whole sectors delivers the record plus up to 511 bytes in front of it and up
; to 511 behind, and those neighbouring bytes belong to records the decoder may
; still be reading -- the block loop walks a0 with no bounds check (49.2), so
; landing them in the ring is a corruption, not a waste.
;
; IN PIO THE FIX IS FREE, and that is the only reason this is affordable here:
; the CPU is already touching every byte, so it simply does not STORE the ones
; outside the window. SC_WSKIP bytes are pulled from the FIFO and dropped, the
; next SC_WKEEP are stored, the rest are pulled and dropped. Three loops rather
; than one steered loop, deliberately: the middle one is then byte-for-byte as
; tight as the un-windowed sc_in_pio, so the per-byte cost this rig reports is
; the transport's and not the window's.
;
; UNDER A DMAC IT IS NOT FREE, and that is P4a's problem arriving early. A
; channel writes a contiguous run to a contiguous address; it cannot be told to
; drop the first 300 bytes. So when the data phase moves to the HD63450 the
; choice is a bounce buffer plus a copy of every byte (the cost `aligned` was
; chosen to avoid, 49.3) or sector-aligned records in the container -- which is
; a re-encode. 57 measured which is cheaper; see FINDINGS 58.3.
SC_WSKIP = $18220 ; u32 bytes to drop before the window
SC_WKEEP = $18224 ; u32 bytes of window to store
; ---- a TRACE, because a SCSI bring-up cannot be debugged from one error code.
; Four registers at each interesting instant: SSTS, PSNS, INTS, SERR. MAME's
; SCMD_CMD_TRANSFER is a NO-OP unless SSTS_INIT_CONNECTED is set -- it `break`s
; out of the switch without complaint -- so "the transfer did nothing" and "the
; transfer went wrong" look identical from the outside. The trace separates
; them.
SC_TAG = $1822C ; u32 where the next snapshot came from
SC_TRN = $18230 ; u32 trace entries used
SC_TR = $18240 ; 24 x 8 B: SSTS PSNS INTS SERR TCH TCM TCL TAG
; A poll bound. Every wait in this file is bounded, because a SCSI phase that
; never arrives must be REPORTED -- an unbounded spin is indistinguishable from
; a wedged emulator, and 34.1 already cost this project fifteen minutes to that
; exact confusion.
SC_PATIENCE = 200000
; ---------------------------------------------------------------- sc_snap
; Append SSTS/PSNS/INTS/SERR to the trace. Clobbers nothing the callers use.
sc_snap:
movem.l d0/a0,-(sp)
move.l SC_TRN.l,d0
cmp.l #24,d0
bge.s sn_out
lea SC_TR.l,a0
lsl.l #3,d0
adda.l d0,a0
move.b SC_SSTS,(a0)+
move.b SC_PSNS,(a0)+
move.b SC_INTS,(a0)+
move.b SC_SERR,(a0)+
move.b SC_TCH,(a0)+
move.b SC_TCM,(a0)+
move.b SC_TCL,(a0)+
move.b SC_TAG+3,(a0)+ ; WHERE this snapshot was taken
addq.l #1,SC_TRN.l
sn_out: movem.l (sp)+,d0/a0
rts
; ---- the HD63450, so that PIO through the card's data register works at all.
; x68k_scsiext.cpp puts DMA-AWARE GLUE on $EA0015 and nowhere else:
;
; write: if (exown()) { if (!drq) dtack_w(1); else dma_w(data); }
; else dreg_w(data);
;
; With OWN asserted and DRQ low the byte is DROPPED, silently. That is the
; bring-up's fourth bug and the least guessable: ten command bytes went into
; $EA0015, the FIFO stayed empty, the transfer counter stayed at 10, and every
; register the SPC has said "waiting". Nothing reports a discarded write.
; OWN is the DMAC's, and the IPL ROM has been running for three seconds before
; the player's first instruction (52.5 reads its channel setup out of the ROM),
; so the player does not inherit a quiet DMAC -- it has to make one.
DMAC = $E84000
DMAC_CH = $40 ; channels are 64 B apart
dmac_quiet:
lea DMAC,a0
moveq #3,d1
dq1: move.b #0,7(a0) ; CCR: no operation
move.b #$FF,0(a0) ; CSR: write-one-to-clear
adda.w #DMAC_CH,a0
dbra d1,dq1
rts
; ---------------------------------------------------------------- scsi_init
; Reset the SPC and claim an initiator ID. Leaves interrupts DISABLED: the
; player polls, because the ring producer is already a polling loop living in
; the pace wait (ring.i) and an interrupt would buy it nothing it does not
; already have.
scsi_init:
bsr dmac_quiet
move.b #SCTL_RESET,SC_SCTL ; reset & disable
moveq #40,d0
sci1: nop
dbra d0,sci1
move.b #SCSI_ID,SC_BDID
move.b #0,SC_SCTL ; out of reset; no arbitration, no ints
move.b #$FF,SC_INTS ; INTS is cleared by writing its bits
move.b #0,SC_PCTL
clr.l SC_ERR.l
clr.l DM_USE.l ; PIO unless a caller asks otherwise,
; AFTER this call (src/player/dma.i)
clr.l DM_HOOK.l ; ...and no second consumer is being
; serviced until one installs itself,
; for the same reason and in the same
; direction: a mailbox that defaults to
; a behaviour is how DM_USE landed on
; ring.i's slot.
clr.l SC_TRN.l
move.l #0,SC_TAG.l
bsr sc_snap
rts
; ---------------------------------------------------------------- sc_settc
; d0 = 24-bit transfer count -> TCH/TCM/TCL
; Written LOW BYTE FIRST with lsr, not as a chain of rol.l #8. The rol version
; was the bring-up's second bug: three rotations put the ORIGINAL bits 31..24,
; 23..16 and 15..8 into TCH/TCM/TCL, so a count of 10 loaded a transfer counter
; of ZERO. MAME then completed the TRANSFER instantly and silently -- SSTS came
; back $85, TC0 set and XFER_IN_PROGRESS clear -- and the bus sat in command
; phase, which surfaced as the same `UNEXPECTED PHASE` as a protocol error.
sc_settc:
move.l d0,-(sp)
move.b d0,SC_TCL ; bits 7..0
lsr.l #8,d0
move.b d0,SC_TCM ; bits 15..8
lsr.l #8,d0
move.b d0,SC_TCH ; bits 23..16
move.l (sp)+,d0
rts
; ---------------------------------------------------------------- sc_waitreq
; Wait until the SPC reports a REQ with a phase, or patience runs out.
; Returns the phase in d0; sets SC_ERR and returns -1 on timeout.
sc_waitreq:
move.l #SC_PATIENCE,d1
swr1: move.b SC_PSNS,d0
btst #7,d0 ; REQ
bne.s swr2
subq.l #1,d1
bne.s swr1
move.l #SCE_TIMEOUT,SC_ERR.l
moveq #-1,d0
rts
swr2: and.l #7,d0
rts
; ---------------------------------------------------------------- sc_waitfree
; Wait for BUS FREE. A command is not over when its last message byte has been
; read: the target still has BSY asserted, and an initiator that starts
; arbitrating into that gets a selection timeout.
;
; This is the bring-up's fifth bug, and it only appeared once there were TWO
; reads. One read passed, byte-exact, and every conclusion drawn from it was
; sound; the SECOND could not select, because nothing had waited for the first
; to let go of the bus. A player issues one of these per record, so the failure
; would have been universal in the ring and invisible in the demonstration.
sc_waitfree:
move.l #SC_PATIENCE,d1
swf1: move.b SC_PSNS,d0
btst #3,d0 ; BSY
beq.s swf2
subq.l #1,d1
bne.s swf1
move.l #SCE_TIMEOUT,SC_ERR.l
moveq #-1,d0
rts
swf2: moveq #0,d0
rts
; ---------------------------------------------------------------- sc_select
; Select SCSI_TGT. The selection bitmask goes in TEMP -- both IDs, ours and
; theirs -- and the transfer counter doubles as the selection timeout (MAME:
; SelectionWaitBSY is derived from TC's upper bits, which is the datasheet's
; behaviour too).
sc_select:
move.b #$FF,SC_INTS
move.b #(1<<SCSI_ID)|(1<<SCSI_TGT),SC_TEMP
move.l #$002000,d0
bsr sc_settc
move.b #0,SC_PCTL
move.b #SCMD_SELECT,SC_SCMD
move.l #SC_PATIENCE,d1
ssel1: move.b SC_INTS,d0
btst #4,d0 ; COMMAND COMPLETE = selection won
bne.s ssel_ok
btst #2,d0 ; TIMEOUT = nobody there
bne.s ssel_tmo
subq.l #1,d1
bne.s ssel1
move.l #SCE_TIMEOUT,SC_ERR.l
moveq #-1,d0
rts
ssel_tmo:
move.b #$FF,SC_INTS
move.l #SCE_SELTMO,SC_ERR.l
moveq #-1,d0
rts
ssel_ok:
move.l #1,SC_TAG.l
bsr sc_snap
move.b #$FF,SC_INTS
move.l #2,SC_TAG.l
bsr sc_snap
moveq #0,d0
rts
; ---------------------------------------------------------------- sc_xferend
; Wait for the SPC to finish the TRANSFER it was given, rather than for the last
; byte to have been HANDED to it.
;
; This is the bring-up's one real bug and it is worth recording. Without it,
; sc_out_pio wrote all ten command bytes and returned, the caller immediately
; asked what phase the bus was in, and the answer was STILL COMMAND -- because
; the SPC had the last byte in its FIFO and had not yet run the REQ/ACK for it.
; The symptom was `UNEXPECTED PHASE, phase=2` at the DATA-IN check, which reads
; like a target refusing the command and is nothing of the kind. A byte handed
; to a FIFO is not a byte on the bus.
sc_xferend:
move.l #SC_PATIENCE,d3
sxe1: move.b SC_SSTS,d0
btst #4,d0 ; XFER IN PROGRESS
beq.s sxe2
subq.l #1,d3
bne.s sxe1
move.l #SCE_TIMEOUT,SC_ERR.l
moveq #-1,d0
rts
sxe2: moveq #0,d0
rts
; ---------------------------------------------------------------- sc_out_pio
; Send d1 bytes from (a0) in phase d2. Command blocks and nothing else, so it
; is the small, simple one.
sc_out_pio:
move.b d2,SC_PCTL
move.l d1,d0
bsr sc_settc
move.b #SCMD_XFER,SC_SCMD
move.l #4,SC_TAG.l ; 4 = TRANSFER issued for an OUT phase
bsr sc_snap
sop1: move.l #SC_PATIENCE,d3
sop2: move.b SC_SSTS,d0
btst #1,d0 ; DREG FULL -- wait for room
beq.s sop3
subq.l #1,d3
bne.s sop2
move.l #SCE_TIMEOUT,SC_ERR.l
moveq #-1,d0
rts
sop3: move.b (a0)+,SC_DREG
subq.l #1,d1
bne.s sop1
move.l #5,SC_TAG.l ; 5 = every byte handed to the FIFO
bsr sc_snap
bsr sc_xferend
move.l d0,-(sp)
move.l #6,SC_TAG.l ; 6 = after waiting for the transfer
bsr sc_snap
move.l (sp)+,d0
rts
; ---------------------------------------------------------------- sc_in_pio
; Receive d1 bytes into (a1) in phase d2. This is the path the DMA version
; replaces; it stays because it is what makes a DMA failure diagnosable.
sc_in_pio:
move.b d2,SC_PCTL
move.l d1,d0
bsr sc_settc
move.b #SCMD_XFER,SC_SCMD
move.l #7,SC_TAG.l ; 7 = TRANSFER issued for an IN phase
bsr sc_snap
sip1: move.l #SC_PATIENCE,d3
sip2: move.b SC_SSTS,d0
btst #0,d0 ; DREG EMPTY -- wait for a byte
beq.s sip3
subq.l #1,d3
bne.s sip2
move.l #SCE_TIMEOUT,SC_ERR.l
moveq #-1,d0
rts
sip3: move.b SC_DREG,(a1)+
subq.l #1,d1
bne.s sip1
move.l #8,SC_TAG.l ; 8 = every byte taken from the FIFO
bsr sc_snap
bsr sc_xferend
move.l d0,-(sp)
move.l #9,SC_TAG.l ; 9 = after waiting for the IN transfer
bsr sc_snap
move.l (sp)+,d0
rts
; ---------------------------------------------------------------- sc_in_data
; Receive d1 bytes in phase d2, storing only the WINDOW: drop SC_WSKIP, store
; SC_WKEEP at (a1), drop whatever is left. This is the DATA IN path; STATUS and
; MESSAGE IN keep sc_in_pio, which is one byte and has no window.
;
; THREE LOOPS, NOT ONE STEERED LOOP. A single loop with a `which third am I in`
; test per byte would cost ~20 clocks on every byte of every record, and the
; number this rig exists to produce is the transport's per-byte cost -- so the
; middle loop is byte-for-byte sc_in_pio's and the window is paid for once at
; each boundary instead of once per byte.
;
; A SPLIT DATA PHASE WOULD RE-SKIP. The counters are re-read from memory on
; every entry, so a target that broke one READ(10) across two DATA IN phases
; would drop the head of the second phase as well. This one does not split --
; the same limitation sc_in_pio's caller already carries -- and the fix is the
; same one: d5 has to become what each phase actually delivered.
sc_in_data:
; ---- P4a: the DATA IN phase can be handed to the HD63450 instead, and
; when it is, the CPU touches none of these bytes. src/player/dma.i.
; The window is REFUSED rather than ignored: a channel writes a
; contiguous run and cannot be told to drop the first 300 bytes, so a
; windowed DMA read would deliver the neighbours' bytes into the ring
; and the block loop has no bounds check to catch it (49.2, 58.3).
; Refusing it here is what makes "sector-aligned records" a PRECONDITION
; the transport states rather than an assumption it carries.
tst.l DM_USE.l
beq.s .pio
tst.l SC_WSKIP.l
bne.s .nowin
move.l SC_WKEEP.l,d0
cmp.l d1,d0
bne.s .nowin
bra sc_in_dma
.nowin: move.l #SCE_WINDOW,SC_ERR.l
moveq #-1,d0
rts
.pio:
movem.l d6-d7,-(sp)
move.b d2,SC_PCTL
move.l d1,d0
bsr sc_settc
move.b #SCMD_XFER,SC_SCMD
move.l #7,SC_TAG.l ; 7 = TRANSFER issued for an IN phase
bsr sc_snap
move.l SC_WSKIP.l,d6
move.l SC_WKEEP.l,d7
sub.l d6,d1
sub.l d7,d1 ; d1 = trailing bytes to drop
tst.l d6
beq.s .keep
.drop1: move.l #SC_PATIENCE,d3
.dw1: move.b SC_SSTS,d0
btst #0,d0 ; DREG EMPTY -- wait for a byte
beq.s .dg1
subq.l #1,d3
bne.s .dw1
bra .tmo
.dg1: tst.b SC_DREG ; popped and thrown away
subq.l #1,d6
bne.s .drop1
.keep: tst.l d7
beq.s .tail
.keep1: move.l #SC_PATIENCE,d3
.kw1: move.b SC_SSTS,d0
btst #0,d0
beq.s .kg1
subq.l #1,d3
bne.s .kw1
bra .tmo
.kg1: move.b SC_DREG,(a1)+
subq.l #1,d7
bne.s .keep1
.tail: tst.l d1
beq.s .fin
.tail1: move.l #SC_PATIENCE,d3
.tw1: move.b SC_SSTS,d0
btst #0,d0
beq.s .tg1
subq.l #1,d3
bne.s .tw1
bra.s .tmo
.tg1: tst.b SC_DREG
subq.l #1,d1
bne.s .tail1
.fin: movem.l (sp)+,d6-d7
move.l #8,SC_TAG.l ; 8 = every byte taken from the FIFO
bsr sc_snap
bsr sc_xferend
move.l d0,-(sp)
move.l #9,SC_TAG.l
bsr sc_snap
move.l (sp)+,d0
rts
.tmo: movem.l (sp)+,d6-d7
move.l #SCE_TIMEOUT,SC_ERR.l
moveq #-1,d0
rts
; ---------------------------------------------------------------- scsi_read
; READ(10) of d4 blocks from LBA d3 into (a1). READ(10) rather than READ(6)
; because a 21-bit LBA and a 256-block ceiling are limits this container will
; reach -- 4,488,588 B of frame records is already 8,767 sectors, and a full
; disc is 1.09 GiB (ROADMAP C3).
;
; DRIVEN BY THE PHASE, NOT BY A SCRIPT, and that is the third thing the bring-up
; taught. The first version ran a fixed sequence -- select, command, data,
; status, message -- and broke the moment the target asked for something else:
; it came up in MESSAGE OUT with ATN asserted and the driver, which "knew" the
; next phase was COMMAND, called it an unexpected phase and gave up. The bus
; decides the order. A driver that reads the phase and services whatever it
; finds is both shorter and correct, and it is what the target is entitled to.
scsi_read:
movem.l d3-d5/a1,-(sp)
move.l d4,d5
lsl.l #8,d5
lsl.l #1,d5 ; blocks * 512
clr.l SC_WSKIP.l ; no window: keep the whole transfer
move.l d5,SC_WKEEP.l
bra.s scr_body
; ---- the same read, delivering only SC_WSKIP..SC_WSKIP+SC_WKEEP of it. The
; caller sets the two words; everything else is identical, which is the point --
; a windowed read and a whole one must not be able to differ in the protocol.
scsi_read_win:
movem.l d3-d5/a1,-(sp)
move.l d4,d5
lsl.l #8,d5
lsl.l #1,d5
scr_body:
; ---- the command block, built before anything is on the bus
lea SC_CDB.l,a0
move.b #$28,(a0)+ ; READ(10)
clr.b (a0)+
move.l d3,d0 ; LBA, big-endian u32
rol.l #8,d0
move.b d0,(a0)+ ; 31..24
rol.l #8,d0
move.b d0,(a0)+ ; 23..16
rol.l #8,d0
move.b d0,(a0)+ ; 15..8
rol.l #8,d0
move.b d0,(a0)+ ; 7..0
clr.b (a0)+
move.l d4,d0 ; block count, big-endian u16. Same
lsr.l #8,d0 ; trap as sc_settc had: a rol chain here
move.b d0,(a0)+ ; would have emitted bits 31..24/23..16
move.b d4,(a0)+ ; of a count that lives in 15..0.
clr.b (a0)+
clr.l SC_STAT.l
bsr sc_select
tst.l d0
bmi scr_out
; Drop ATN. We have no message to send, so asking the target not to ask
; for one is cheaper than answering. The MSGOUT arm below still exists,
; because "cheaper" is not "guaranteed".
move.b #SCMD_RSTATN,SC_SCMD
; ---- service whatever the bus asks for, until the target ends the command
scr_ph:
bsr sc_waitreq
tst.l d0
bmi scr_out
move.l d0,-(sp)
move.l #3,SC_TAG.l ; 3 = the phase loop saw a REQ
bsr sc_snap
move.l (sp)+,d0
cmp.l #PH_CMD,d0
beq.s scr_cmd
cmp.l #PH_DATAIN,d0
beq.s scr_din
cmp.l #PH_STATUS,d0
beq.s scr_st
cmp.l #PH_MSGIN,d0
beq.s scr_min
cmp.l #PH_MSGOUT,d0
beq.s scr_mout
bra scr_phase
scr_cmd:
lea SC_CDB.l,a0
moveq #10,d1
moveq #PH_CMD,d2
bsr sc_out_pio
tst.l d0
bmi scr_out
bra scr_ph
scr_din:
; KNOWN LIMITATION, harmless here and not harmless forever: this asks
; for the WHOLE remaining count every time the bus enters DATA IN. A
; target that split one READ(10) across two data phases would be served
; the full length twice and overrun the caller's buffer. This one does
; not split -- 4,096 B and 2,048 B both arrive in a single phase -- but
; a real drive may, and P4b's mailbox integration is where d5 has to
; start being decremented by what each phase actually delivered.
move.l 12(sp),a1 ; the caller's destination. movem.l
; d3-d5/a1,-(sp) lays them out ASCENDING
; from sp as d3,d4,d5,a1 -- a1 is at 12.
move.l d5,d1
moveq #PH_DATAIN,d2
bsr sc_in_data
tst.l d0
bmi scr_out
bra scr_ph
scr_st:
lea SC_STAT.l,a1
addq.l #3,a1 ; the byte lands in the u32's low end
moveq #1,d1
moveq #PH_STATUS,d2
bsr sc_in_pio
tst.l d0
bmi scr_out
bra scr_ph
scr_min:
lea SC_MSG.l,a1
moveq #1,d1
moveq #PH_MSGIN,d2
bsr sc_in_pio
tst.l d0
bmi scr_out
; A message in ends the command. Anything non-zero in the status byte
; is the target refusing, and a transport that ignored it would hand the
; ring a buffer of stale bytes and call it a record.
move.l SC_STAT.l,d0
beq.s scr_ok
move.l #SCE_STATUS,SC_ERR.l
moveq #-1,d0
bra.s scr_out
scr_mout:
; Nothing to say: IDENTIFY, no disconnect, LUN 0.
lea SC_MSG.l,a1
move.b #$80,(a1)
move.l a1,a0
moveq #1,d1
moveq #PH_MSGOUT,d2
bsr sc_out_pio
tst.l d0
bmi scr_out
bra scr_ph
scr_ok: ; END OF COMMAND, and it takes two steps rather than one. After the
; final message byte the SPC is still holding ACK -- PSNS reads $4F,
; REQ low and ACK high -- and a target cannot drop BSY into that. So
; ACK is dropped explicitly, and only then is the bus released.
move.b #SCMD_RSTACK,SC_SCMD
move.b #SCMD_RELEASE,SC_SCMD
move.l #10,SC_TAG.l ; 10 = after the bus release command
bsr sc_snap
bsr sc_waitfree ; leave the bus as we found it
tst.l d0
bmi scr_out
moveq #0,d0
scr_out:
movem.l (sp)+,d3-d5/a1
rts
scr_phase:
move.l d0,SC_PH.l
move.l #SCE_PHASE,SC_ERR.l
moveq #-1,d0
bra.s scr_out
; ---- the DMAC side of the data phase, ROADMAP P4a. Included unconditionally
; so that there is ONE transport file: sc_in_data dispatches on DM_USE, which
; scsi_init clears, so a front-end that never sets it assembles the same PIO
; path FINDINGS 58 measured and executes not one instruction of the following.
include "src/player/dma.i"
src/player/scsigate.s
+150
View File
@@ -0,0 +1,150 @@
; Front-end for the MB89352 PROBE (ROADMAP P4, first step), for the rig.
;
; WHY A PROBE AND NOT A DRIVER. P4 replaces tools/bench/stream.lua's modelled
; transport with a real SPC behind the XF_* mailbox src/player/ring.i already
; talks to. Before any of that can be written, the register map has to be a
; FACT on the emulated machine rather than a reading of somebody's datasheet.
; FINDINGS 32.4 quotes MAME mapping the data register at $EA0015 -- register
; index 10 at a stride of 2 from $EA0001, i.e. registers on the ODD bytes. That
; is an inference from ONE address, and every access the driver makes rests on
; it.
;
; WHY IT ENUMERATES INSTEAD OF DUMPING. The first version of this walked
; $EA0000 upwards with a plain `move.b (a0)+`, and took a bus error at $EA0006 --
; at which point it knew one address was dead and nothing about the other 57.
; A sequential dump stops at the first hole and reports the hole as the answer.
; So this probes ONE ADDRESS AT A TIME with the index in MEMORY, and a bus error
; handler that records the fault, steps the index and re-enters the loop. A
; dead address costs an entry in the map rather than the rest of the run.
;
; The 68000 cannot resume a faulted instruction -- RTE re-runs it and faults
; again -- so the handler does not try. It restores a stack pointer saved
; before the loop and jumps back to the loop head, which reloads everything it
; needs from memory. Nothing lives in a register across a fault.
;
; This gate drives no SCSI bus and moves no data. It is the smallest thing that
; can turn "MAME instantiates an MB89352" into "the 68000 can reach it, HERE".
SCFLAG = $18080 ; 0 idle / 1 done
SCN = 64 ; addresses probed, from SPCBASE up
SCIDX = $18084 ; u32 probe index, lives in memory across faults
SCSAVSP = $18088 ; u32 stack pointer saved before the loop
SCVAL = $18100 ; SCN bytes: what each address read
SCOK = $18140 ; SCN bytes: 1 = answered, 0 = bus error
SCTMP = $180D0 ; u32: TEMP writeback -- $A5 written, read back
SCTMPOK = $180D4 ; u32: 1 = the writeback completed without fault
SCRD = $180D8 ; u32: scsi_read's return, 0 = the read worked
SCDRG = $180DC ; u32: $5A written to DREG then read straight back
SCDRS = $180E0 ; u32: SSTS between that write and that read
SCDST = $20000 ; where the read lands
SCBLKS = 8 ; 8 x 512 B, enough to cross a sector boundary
SCRD2 = $180E4 ; u32: the second read's return
SCER1 = $180E8 ; u32: SC_ERR as it stood after the first read
SCER2 = $180EC ; u32: ...and after the second
SCDST2 = $28000 ; where the second read lands
SCLBA2 = 1000 ; a NON-ZERO LBA: block 0 would pass even if the
; LBA bytes of the command block were ignored
SCBLK2 = 4
SPCBASE = $EA0000
org $10000
start:
move.l #buserr,$8.w ; vector 2
clr.l SCFLAG.l
clr.l SCTMP.l
clr.l SCTMPOK.l
clr.l SCIDX.l
move.l sp,SCSAVSP.l
; ---- probe SCN addresses, one at a time, surviving each fault
ploop:
move.l SCIDX.l,d0
cmp.l #SCN,d0
bge.s pdone
lea SPCBASE,a0
adda.l d0,a0
lea SCVAL.l,a1
lea SCOK.l,a2
move.b #1,0(a2,d0.l) ; assume it answers; the handler undoes
move.b (a0),d1 ; <- the access under test
move.b d1,0(a1,d0.l)
addq.l #1,SCIDX.l
bra.s ploop
pdone:
; ---- TEMP (register 11 on the believed map, $EA0017) is a scratch latch on a
; real MB89352. Writing a pattern and reading it back separates "these odd
; bytes are registers" from "these odd bytes are a mirror of something".
; Guarded the same way: if it faults, the handler lands in ploop with SCIDX
; already past the end, falls through here again, and SCTMPOK stays 0.
move.b #$A5,SPCBASE+23
moveq #0,d0
move.b SPCBASE+23,d0
move.l d0,SCTMP.l
move.l #1,SCTMPOK.l
; ---- DOES A WRITE TO THE DATA REGISTER REACH THE CHIP AT ALL?
; $EA0015 is the one address x68k_scsiext.cpp puts its own glue on, and that
; glue DROPS a write when the DMAC's OWN is asserted and DRQ is low. A dropped
; write is invisible: no error, no status bit, nothing. So it is tested
; directly, before any SCSI protocol can be blamed for it. dreg_w enqueues into
; the FIFO, so DREG_EMPTY must fall between the write and the read, and the read
; must give the byte back.
bsr scsi_init
move.b #$5A,SPCBASE+21
moveq #0,d0
move.b SPCBASE+13,d0 ; SSTS: is the FIFO still empty?
move.l d0,SCDRS.l
moveq #0,d0
move.b SPCBASE+21,d0
move.l d0,SCDRG.l
; ---- the SPC is reachable; now make it fetch something. A read of the first
; SCBLKS sectors, in PIO, verified BY THE HOST against the same bytes in
; tmp/dlxdisk.img. That is the whole of P4's correctness half in one line: the
; player's own code selected a target, issued a READ(10) and got the disc's
; bytes back, with no IOCS and no host in the path.
bsr scsi_init
moveq #0,d3 ; LBA 0
moveq #SCBLKS,d4
lea SCDST,a1
bsr scsi_read
move.l d0,SCRD.l
move.l SC_ERR.l,SCER1.l ; SC_ERR is the LAST error, so it is
; captured per read: reading it once at
; the end reported the second read's
; failure against the first read's name.
; ---- and again, somewhere else on the disc. A read of LBA 0 is passed by a
; driver that emits a malformed LBA field, because zero is what a malformed
; field usually is. This one is not.
move.l #SCLBA2,d3
moveq #SCBLK2,d4
lea SCDST2,a1
bsr scsi_read
move.l d0,SCRD2.l
move.l SC_ERR.l,SCER2.l
move.l #1,SCFLAG.l
hold: bra.s hold
; ---- bus error. Mark the address dead, step past it, re-enter the loop with a
; stack pointer that is known good. The stacked frame is abandoned deliberately:
; there is nothing in it worth more than the next 57 addresses.
buserr:
move.l SCSAVSP.l,sp
move.l SCIDX.l,d0
cmp.l #SCN,d0
bge.s btmp
lea SCOK.l,a2
clr.b 0(a2,d0.l)
lea SCVAL.l,a1
move.b #$FF,0(a1,d0.l)
addq.l #1,SCIDX.l
jmp ploop
btmp: ; the fault was the TEMP writeback
move.l #1,SCFLAG.l
be: bra.s be
include "src/player/scsi.i"
src/player/stream.s
+348
View File
@@ -0,0 +1,348 @@
; DLX3 frame decoder, RING-BUFFER front-end -- STATUS item 3, FINDINGS 49.
;
; src/player/decode.s decodes a stream that is ALREADY WHOLLY IN RAM: the rig
; preloads 5,261,814 bytes at $30000 and walks a0 forward through all of it.
; That gate proves the decoder is pixel-exact over a 120-frame window
; (FINDINGS 45) and says NOTHING about how the bytes got there. The shipping
; player never holds a window at once; it streams from a SCSI disk into a ring
; a fraction of the size. Nothing in this tree has ever tested that path.
;
; WHAT IS ACTUALLY HARD ABOUT IT. The block loop and the span chain read the
; stream with a monotonically increasing a0 and no bounds check anywhere --
; `move.l (a0)+,d0`, `lea MODEB(a0),a0`, eleven unrolled `movem.l (a0)+`, a
; `move.b (a0)+` per block index. None of it can survive an address that wraps
; mid-record. So the ring does not merely need ENOUGH BYTES resident by the
; deadline -- 09_buffer_sim.py's question, and FINDINGS 21's answer -- it needs
; the WHOLE NEXT RECORD resident and CONTIGUOUS.
;
; THE WRAP POLICY IS `aligned`, and it was chosen on measurement, not taste
; (tools/analysis/19_ring_stream.py). The producer refuses to start a record it
; cannot finish before the end of the ring: it leaves a hole and restarts at 0.
;
; aligned costs RAM -- a mean hole of 23.4 KB in a 256 KB ring, 9.1% of it --
; and ZERO CPU.
; split lets records wrap and mirrors the ring's first MAXREC bytes into a
; shadow past its end, so any record start reads linearly. Costs
; zero RAM and 46,394 clocks/frame of memcpy -- 5.57% of the frame
; budget, forever.
;
; Both figures are for s14_d5_all1500, the shipping candidate, in a 256 KB ring;
; they scale with record size, so they are per container, not universal. The
; lighter gate container makes it 5.7% of the ring against 3.64% of the budget --
; same direction, same verdict.
;
; The decoder already spends 77.0% of the budget on the mean frame and 91.1% at
; p90 (FINDINGS 45). 5.57% more puts p90 at 96.7%. RAM is the resource this
; machine has 2 MB of and clocks are the one it has none of, so the trade is not
; close. `aligned` also needs a per-record INDEX on the fill side, which a
; BRANCHING laserdisc game needs anyway to seek to a branch point -- so the
; policy that costs no clocks also reuses a structure the player cannot avoid.
;
; The third option -- teach the block loop to wrap its own reads -- is the
; expensive one and not because of the branch. A bounds test lands INSIDE the
; instruction sequences FINDINGS 30.4 and 40 fitted their constants to, so it
; does not cost a compare, it costs every span and per-block figure in the tree
; being re-measured.
;
; THE PRODUCER IS OUTSIDE THIS FILE. Here it is tools/bench/stream.lua playing
; a SCSI disk at a modelled byte rate; in the player it is the MB89352 and a
; DMAC channel. The handshake is deliberately the same either way:
;
; producer -> FR_HEAD count of records made wholly resident (monotonic)
; DESC[] ring of record base addresses, DESCN entries
; decoder -> FR_TAIL count of records consumed (monotonic)
; RD_PTR one past the last byte read; everything below is free
;
; Two monotonic counters and a released-to pointer -- no lock, no shared cursor,
; single reader and single writer, so it is correct on a 68000 with no atomics
; provided each side only ever writes its own words. That is why FR_TAIL is
; the decoder's and FR_HEAD is the producer's rather than one shared index.
;
; STALLS ARE COUNTED, NOT HIDDEN. A frame whose record is not resident when the
; decoder wants it spins in `waitrec`, and STALLS counts the FRAMES that had to
; wait at all (not the polls). A rig that silently absorbed an underrun would
; report a pixel-exact decode of a stream that arrived late, which is precisely
; the failure this front-end exists to make visible. The spin is bounded:
; SPINMAX polls without progress sets FLAG=$E1, so a wedged producer fails as a
; wedged producer instead of as a MAME timeout with no diagnosis (FINDINGS 34.1).
FLAG = $18000 ; 0 idle / 1 running / $FF done / $EE desync
; / $E1 producer stalled out
ITER = $18008 ; outer repeat count, written by Lua
NFR = $1800C ; frames per pass
SCR_N = $18014 ; frames remaining this pass
SCR_END = $18018 ; expected end of the current payload
RD_PTR = $18020 ; decoder -> producer: released up to here
FR_HEAD = $18024 ; producer -> decoder: records resident
FR_TAIL = $18028 ; decoder -> producer: records consumed
STALLS = $1802C ; frames that had to wait for their record
SPINS = $18030 ; total poll iterations spent waiting
PACE = $18034 ; producer -> decoder: frame ticks elapsed since
; release. Frame i may not START before tick i.
PACEON = $18038 ; 1 = obey PACE. 0 leaves the loop free-running,
; byte for byte the loop FINDINGS 49 measured.
LATEFR = $18080 ; frames that reached the pace gate with their
; tick ALREADY ARRIVED, i.e. did not idle for a
; single poll -- the previous frame used its
; whole slot. See the gate below.
LATEMAX = $18084 ; the worst of those, in WHOLE ticks overrun
LATE1ST = $18088 ; index of the FIRST such frame, so that a
; count can be told apart from a start-up
; transient without re-running anything
CLKON = $1803C ; 1 = the 68000 paces ITSELF: src/player/clock.i
; drives PACE off the CRTC's V-DISP instead of
; the host writing it. Needs PACEON=1; the gate
; below cannot tell the two apart and must not.
DESC = $18100 ; DESCN x u32, record base addresses
; The producer's own words -- RINGOWN, the transport mailbox and its
; instruments -- are in src/player/ring.i, at $18300 and up, clear of DESC's
; 256 bytes.
DESCN = 64 ; power of two; the index is masked, not compared
DESCM = (DESCN-1)*4 ; mask for a BYTE offset into DESC
SPINMAX = 2000000 ; polls with no progress before giving up
include "src/player/geom.i"
org $10000
start:
; ---- the ring producer, if this run is asking the 68000 to fill its own ring
; (ROADMAP P5, src/player/ring.i). It goes FIRST because it only builds tables
; and touches no hardware: a run that cannot build them should not have armed an
; interrupt source first.
tst.l RINGOWN.l
beq.s noring
; ---- the transport, before the producer that will ask it for something.
; ring_init ends in a ring_seek and a seek WAITS for the channel to go quiet, so
; the thing that makes the channel quiet has to exist first. With XF_SCSI = 0
; this brings up nothing and the host is the transport (FINDINGS 55).
bsr xf_init
bsr ring_init
tst.l d0
bpl.s noring
move.l #$E3,FLAG.l ; the record index is longer than ROFF
bra hold
noring:
; ---- the frame clock, if this run is asking the 68000 to keep its own time.
; It goes here rather than inside the frame loop because clk_init CLEARS PACE:
; tick 0 has to be the instant the decoder was released, exactly as it is when
; the host writes PACE, or the first frame's deadline moves.
tst.l CLKON.l
beq.s noclk
bsr clk_init
tst.l CLK_ERR.l
beq.s noclk
move.l #$E2,FLAG.l ; the clock refused; CLK_ERR says why
bra hold
noclk:
move.l #1,FLAG.l ; timer starts here
outer:
move.l NFR.l,SCR_N.l
clr.l FR_TAIL.l
clr.l STALLS.l
clr.l SPINS.l
clr.l LATEFR.l
clr.l LATEMAX.l
move.l #-1,LATE1ST.l
; ---- SEEK AND PREFILL. Every pass starts with a seek to record 0 -- which on
; the first pass is just "start of scene" and on any later one is a REAL seek:
; the channel has to go quiet, the ring is declared empty, and the whole
; lookahead 51.3 says takes seconds of play to accumulate is thrown away and
; rebuilt from the prefill up. That is the branch point rehearsed with the one
; thing a rig can check afterwards -- the decode has to still be pixel-exact.
;
; The clock is REBASED here rather than at clk_init, because tick 0 must be the
; instant the decoder is released and the prefill happens before that. Under a
; host-written PACE this word belongs to the host, so it is only touched when
; the 68000 is keeping its own time; ITER>1 therefore needs CLKON.
tst.l RINGOWN.l
beq.s noseek
moveq #0,d0
bsr ring_seek
bsr ring_prefill
tst.l d0
bpl.s .pfok
move.l #$E1,FLAG.l ; the transport never delivered
bra hold
.pfok:
tst.l CLKON.l
beq.s noseek
clr.l PACE.l ; at most one tick is lost to a V-DISP
; landing between the ISR and here
noseek:
frameloop:
tst.l RINGOWN.l
beq.s nomark
move.l FR_TAIL.l,d0
bsr ring_mark
nomark:
; ---- PACE GATE (FINDINGS 49.7.2, and it is the whole point of this session).
; Free-running, this loop asks for record i the instant it finishes record i-1,
; so it outruns any finite pipe, the ring NEVER backs up, and the producer's
; overlap test never refuses a placement. A ring-size sweep under those
; conditions tests WRAP CORRECTNESS at each size and nothing about BUFFERING:
; 48 KB passes while holding one record. A shipping player does not do this --
; it draws frame i, waits for its slot, and spends the rest of the frame time
; idle while the disk fills the ring behind it.
;
; So the rig gets a frame clock. PACE is bumped by the producer (in the player,
; vblank or an MFP timer) and frame i is forbidden to start before tick i. With
; the decoder held to 12 fps the ring fills, the producer starts hitting its own
; overlap test, and FR_HEAD-FR_TAIL becomes what it claims to be: the number of
; whole frames the decoder could run on if delivery stopped dead -- which is the
; branch-point seek question stated in frames.
;
; It also makes STALLS mean something. Free-running, a stall is EARLINESS
; (49.6); paced, a frame that has to wait for its record is a real underrun.
;
; AND IT COUNTS THE FRAMES THAT WERE ALREADY LATE, which is a question only a
; REAL frame clock raises. 12 fps on a 55.4577 Hz raster is 4.6215 refreshes
; per frame, so the divider hands out slots of 4 refreshes (72.13 ms) and 5
; (90.16 ms), 37.9% of them short -- and the SHORT one is 13.4% under the
; 83.33 ms every budget in this project is priced against (FINDINGS 54). A
; frame that does not fit its slot does not fail here: PACE has already moved
; on, so the next frame starts the instant this one finishes and the clock
; catches up by itself. What it costs is one late PRESENT, and nothing in this
; tree counted those because until now the tick was a host model with no
; cadence in it at all.
;
; The test is "did this frame have to WAIT", not "is it a whole tick behind".
; Frame i waits while PACE < i; so PACE >= i on arrival means the decoder came
; to the gate with slot i already open and idled for zero polls, which is the
; same statement as "frame i-1 ran to the end of its slot". A whole tick of
; overrun -- PACE - FR_TAIL >= 1 -- is the much rarer case where it ran past
; the end of the NEXT one, and is reported separately as the worst seen.
;
; Frame 0 is excluded: it starts at tick 0 by definition and has no predecessor
; to have overrun. The wait loop below is untouched -- all of this is ahead of
; it, and the free-running path executes none of it.
tst.l PACEON.l
beq.s nopace
move.l PACE.l,d0
cmp.l FR_TAIL.l,d0 ; PACE < FR_TAIL: the slot has not come
bcs.s pacesel ; round yet, so this frame is EARLY
tst.l FR_TAIL.l
beq.s pacesel ; frame 0 starts AT tick 0 by definition
tst.l LATEFR.l
bne.s .nf1
move.l FR_TAIL.l,LATE1ST.l
.nf1:
addq.l #1,LATEFR.l
sub.l FR_TAIL.l,d0 ; whole ticks overrun; 0 = inside the
cmp.l LATEMAX.l,d0 ; slot but with nothing left of it
bls.s pacewait
move.l d0,LATEMAX.l
pacesel:
; Both branches above -- the early frame and frame 0 -- come here rather than
; jumping straight into the legacy wait, because the early frame is the COMMON
; case and it is the one with idle in it. Routing it past this test was a real
; bug and not a tidy-up: the producer then only ever ran from the record wait,
; about once a frame, and the disc spent most of the scene stopped.
tst.l RINGOWN.l
beq.s pacewait
; ---- THE IDLE IS WHERE THE DISC RUNS. ring_poll retires the completed
; request and issues the next one, and this loop is the only place in a paced
; player with time to spare. Polling once a FRAME instead would cap the fill at
; one record per slot -- the wire rate exactly -- and a ring that can only keep
; up never accumulates the slack a branch point spends (51.3).
pacewaitR:
bsr ring_poll
move.l PACE.l,d0
cmp.l FR_TAIL.l,d0
bcs.s pacewaitR
bra.s nopace
pacewait:
move.l PACE.l,d0
cmp.l FR_TAIL.l,d0 ; d0 - FR_TAIL; carry = tick not reached
bcs.s pacewait
nopace:
; ---- wait until the producer has made this record wholly resident.
; d1 counts polls for this frame; a nonzero d1 on exit means the frame stalled.
moveq #0,d1
move.l FR_TAIL.l,d2
tst.l RINGOWN.l
beq.s waitrec
; ---- the same wait, with the producer inside it. Here d1 counts POLLS rather
; than spins, so STALLS still means "frames that had to wait" and SPINS is not
; comparable with a host-filled run's. A frame that waits here is a real
; underrun either way: paced, its slot has already opened.
waitrecR:
bsr ring_poll
move.l FR_HEAD.l,d0
cmp.l d2,d0
bhi gotrec
addq.l #1,d1
cmp.l #SPINMAX,d1
bcs.s waitrecR
move.l #$E1,FLAG.l
bra hold
waitrec:
move.l FR_HEAD.l,d0
cmp.l d2,d0
bhi.s gotrec ; HEAD > TAIL: at least one record ready
addq.l #1,d1
cmp.l #SPINMAX,d1
bcs.s waitrec
move.l #$E1,FLAG.l ; producer never delivered
bra hold
gotrec:
tst.l d1
beq.s nostall
addq.l #1,STALLS.l
add.l d1,SPINS.l
nostall:
; ---- pop the descriptor. DESCN is a power of two, so the wrap is an and.
move.l d2,d0
lsl.l #2,d0
and.w #DESCM,d0
lea DESC,a1 ; DESC is absolute; (d0.w) needs a base
move.l (a1,d0.w),a0 ; a1 is reloaded from a0 two lines below
; ---- from here to the release, byte for byte what decode.s does. a0 is
; inside the ring rather than inside a preloaded blob, and the block loop
; cannot tell the difference -- which is the whole claim being tested.
move.l (a0)+,d0 ; u32 payload length, big-endian
lea 0(a0,d0.l),a1
move.l a1,SCR_END.l ; where the payload must end
move.l a0,a1 ; a1 = packed mode header
lea MODEB(a0),a0 ; a0 = span section
bsr paint_spans ; -> a0 = block payload, a1 preserved
bsr decode_frame
cmpa.l SCR_END.l,a0 ; bitstream desync is silent otherwise
bne desync
; ---- release. Round up to RECALN: the producer lays records on the
; container's own record boundaries (geom.i), so the byte one past this
; record's PADDED end is the first byte it may reuse. Releasing only the
; bytes actually read would strand up to RECALN-1 of pad per record and
; the producer's free-space arithmetic would drift by that much a frame.
; This is an absolute address, so it is only the record's padded end
; because the ring base is RECALN-aligned as well.
move.l a0,d0
addi.l #RECALN-1,d0
andi.l #~(RECALN-1),d0
move.l d0,RD_PTR.l
addq.l #1,FR_TAIL.l
subq.l #1,SCR_N.l
bne frameloop
subq.l #1,ITER.l
bne outer
; ---- leave the MFP as it was found. A rig that exits with a live interrupt
; source and a lowered mask hands the next thing that runs an interrupt it
; has no vector for, and the failure would land somewhere else entirely.
tst.l CLKON.l
beq.s noclk2
bsr clk_stop
noclk2:
move.l #$FF,FLAG.l ; timer stops here
hold: bra.s hold
desync: move.l #$EE,FLAG.l
bra.s hold
include "src/player/frame.i"
include "src/player/clock.i"
include "src/player/ring.i"
include "src/player/scsi.i"
include "src/player/xfer.i"
src/player/xfer.i
+148
View File
@@ -0,0 +1,148 @@
; ---------------------------------------------------------------- xfer.i
; THE TRANSPORT BEHIND THE MAILBOX. ROADMAP P4b.
;
; src/player/ring.i has always ended at a seam: it decides which record to
; fetch, where in the ring to put it and when that is safe, writes those four
; words into XF_SLOT and bumps XF_GO, and then polls XF_ACK. On the other side
; of that seam, until now, was tools/bench/stream.lua -- a host moving bytes at
; a MODELLED rate, with XF_ACK synthesised out of emulated time. A player has
; no host. This file is the other side: XF_GO is answered by src/player/scsi.i
; issuing a real READ(10) to a real MB89352, and XF_ACK is a word the 68000
; bumps when the bytes have landed.
;
; NOTHING ABOVE THE SEAM CHANGED, and that is deliberate for the same reason it
; was in sessions 22 and 23: ring.i cannot tell which transport answered it, so
; a green run here is a test of THIS file rather than of a new producer. The
; two hooks in ring.i (one in ring_poll, one in ring_seek's quiet-wait) are the
; whole of the change on that side, and with XF_SCSI = 0 they are a tst and a
; branch.
;
; IT IS SYNCHRONOUS, AND THAT IS NOT A SHORTCUT -- IT IS THE FINDING. The
; modelled transport overlapped: a request issued at time t landed at t + len/
; rate while the 68000 got on with decoding, which is what a DMAC channel does.
; Here the CPU moves every byte itself through $EA0015 (57.3: a PIO write to
; that address is discarded, so even "PIO" runs the SPC in DMA mode with the CPU
; standing in for the channel), so `bsr xf_service` does not start a transfer,
; it PERFORMS one. A two-deep request queue therefore buys nothing at all: the
; second slot is filled and drained by the same instruction stream that would
; have been decoding. FINDINGS 55.3's whole result -- that a one-deep queue
; gives away 6.8% of the pipe -- is about a transport that runs in parallel with
; the CPU, and this one does not.
;
; So what this file is FOR is not to be the shipping transport. It is to make
; the shipping transport's cost measurable: the same 120 pixel-exact frames,
; delivered by the machine off a real volume, with the CPU cost of doing it
; charged where a rate model cannot hide it. P4a -- the HD63450 holding the bus
; -- is what makes the transfer overlap again, and until it exists this is the
; honest floor.
; ---- state. Above src/player/ring.i's instruments (last: SK_WAIT at $1837C)
; and below the disc-offset table at $19400.
XF_SCSI = $18380 ; 1 = the 68000 is the transport (input)
XS_LBA0 = $18384 ; LBA of byte 0 of the scene's frame stream
XS_NXFER = $18388 ; transfers completed
XS_NBYTE = $1838C ; record bytes delivered into the ring
XS_NWIRE = $18390 ; bytes actually read off the disc, sectors and
; all -- the two differ and 58.3 is why
XS_ERR = $18394 ; SC_ERR of the FIRST failure, 0 = none
XS_ERRAT = $18398 ; ...and the request index it failed on
; ---------------------------------------------------------------- xf_init
; Clears the instruments and brings the SPC up, if this run has one. Called
; before ring_init, because ring_init ends in a ring_seek and a seek waits on
; the transport.
xf_init:
clr.l XS_NXFER.l
clr.l XS_NBYTE.l
clr.l XS_NWIRE.l
clr.l XS_ERR.l
move.l #-1,XS_ERRAT.l
tst.l XF_SCSI.l
beq.s .out
bsr scsi_init
.out: rts
; ---------------------------------------------------------------- xf_service
; Answer at most ONE outstanding request, then return. Preserves every
; register: it is called from inside ring_poll, which is itself called from
; inside the decoder's wait loops and must be invisible to them.
;
; ONE PER CALL, not "drain the queue". ring_poll retires exactly one completed
; request per call as well, and a transport that answered both queued requests
; in one visit would hand the retire loop two acks it can only take one poll at
; a time -- which is legal, but it also means the decoder's wait loop would
; disappear for two record times instead of one. One per call keeps the two
; sides stepping at the same rate.
;
; A RECORD IS NOT A SECTOR, and this is where that is dealt with. ring.i asks
; for a byte offset and a length; the target answers in 512 B blocks. So the
; command covers the sectors the record lies in, and SC_WSKIP/SC_WKEEP tell
; src/player/scsi.i's DATA IN loop which of those bytes to store. The ones
; outside the window are pulled from the FIFO and dropped -- they are NOT
; written past the ends of the destination, because the bytes on either side of
; a record in the stream belong to records the decoder may still be reading and
; the block loop has no bounds check (49.2).
;
; XS_NWIRE counts what the disc actually moved and XS_NBYTE what the ring got.
; They are not the same number and the gap is a delivery cost, not an accounting
; detail: it is bytes on the wire that no frame contains.
xf_service:
tst.l XF_SCSI.l
beq.s .idle
move.l XF_ACK.l,d0
cmp.l XF_GO.l,d0
bcs.s .work ; XF_ACK < XF_GO: something outstanding
.idle: rts
.work:
movem.l d0-d7/a0-a2,-(sp)
move.l XF_ACK.l,d0
move.l d0,d1
and.l #XF_SLM,d1
lsl.l #4,d1 ; * XF_SLSZ
lea XF_SLOT.l,a0
adda.l d1,a0
move.l (a0),d1 ; disc byte offset within the stream
movea.l 4(a0),a1 ; destination in the ring
move.l 8(a0),d2 ; length
; ---- sector arithmetic
move.l d1,d3
and.l #511,d3 ; bytes of the first sector to drop
move.l d1,d4
lsr.l #8,d4
lsr.l #1,d4 ; sector index within the stream
add.l XS_LBA0.l,d4 ; ...and where the stream begins
move.l d3,d5
add.l d2,d5
addi.l #511,d5
lsr.l #8,d5
lsr.l #1,d5 ; sectors the record lies in
move.l d3,SC_WSKIP.l
move.l d2,SC_WKEEP.l
add.l d2,XS_NBYTE.l
move.l d5,d0
lsl.l #8,d0
lsl.l #1,d0
add.l d0,XS_NWIRE.l
move.l d4,d3 ; d3 = LBA
move.l d5,d4 ; d4 = blocks
bsr scsi_read_win
tst.l d0
bmi.s .err
addq.l #1,XS_NXFER.l
addq.l #1,XF_ACK.l ; LAST: the bytes are all in the ring
; before the request is called done
movem.l (sp)+,d0-d7/a0-a2
rts
; ---- a failed read is NOT acked. The record never becomes resident, the
; decoder spins out in waitrec and reports a stalled producer, and XS_ERR says
; which request failed and why. Acking a failed transfer would publish a
; descriptor for a buffer full of whatever was there before -- and the decoder
; would find a plausible-looking length word in it and desync somewhere else
; entirely.
.err:
tst.l XS_ERR.l
bne.s .err2
move.l SC_ERR.l,XS_ERR.l
move.l XF_ACK.l,XS_ERRAT.l
.err2: movem.l (sp)+,d0-d7/a0-a2
rts
tools/analysis/07_motion_survey.py
+4 -2
View File
@@ -13,10 +13,12 @@ window it picks then gets encoded for real.
Usage: python3 tools/analysis/07_motion_survey.py 00223 [window_seconds]
"""
import subprocess, sys
import subprocess, sys, os, getpass
import numpy as np
STREAM_DIR = "/media/reala-misaki/BDROM/BDMV/STREAM"
# See tools/encoder/extract.py: DLX_BDROM overrides where the disc is mounted.
BDROM = os.environ.get("DLX_BDROM") or f"/media/{getpass.getuser()}/BDROM"
STREAM_DIR = f"{BDROM}/BDMV/STREAM"
W, H, FPS = 96, 72, 12
def frames(stream):
tools/analysis/08_mode_map.py
+2 -2
View File
@@ -16,7 +16,7 @@ the mode headers would exploit.
RAW 16 literal palette indices -- the escape that makes lam=0 pixel-exact
Usage: python3 tools/analysis/08_mode_map.py <frames_dir> <out.webm>
[--profile sasi|scsi] [--scale N] [--lossless] [--fixed-lam]
[--profile scsi] [--scale N] [--lossless] [--fixed-lam]
--fixed-lam renders the pre-session-6 encoder (no rate control) instead.
Output format follows the extension. Prefer .webm: GIF re-quantises to 256
@@ -45,7 +45,7 @@ def main():
if "--scale" in sys.argv:
SCALE = int(sys.argv[sys.argv.index("--scale")+1])
prof = RC.PROFILES[sys.argv[sys.argv.index("--profile")+1]
if "--profile" in sys.argv else "sasi"]
if "--profile" in sys.argv else "scsi"]
m = H.build(src, k1=prof["k1"], k4=prof["k4"])
# Rate-controlled by default, so the map shows the mode decisions that
# actually ship. --fixed-lam renders the pre-session-6 encoder instead;
tools/analysis/08_peaks.py
+1 -1
View File
@@ -1,4 +1,4 @@
"""At a 488 KB/s (4 Mbps) ceiling and ~52% mean utilisation, the mean is not the
"""At any fixed delivery ceiling and ~52% mean utilisation, the mean is not the
risk -- the peaks are. Measure per-frame peak-to-mean, then check whether the
leaky-bucket rate controller actually holds the ceiling."""
import sys; sys.path.insert(0,'tools/encoder')
tools/analysis/09_ratectl_drift.py
+58 -40
View File
@@ -19,6 +19,10 @@ paint), so drift is zero by construction rather than by tuning.
This replays what a real decoder does -- SKIP copies the ACTUALLY EMITTED
previous frame -- and compares it to the reconstruction ratectl recorded.
Session 8 added a SECOND controller (mu, the per-frame 68000 decode ceiling)
that also varies the mode map frame to frame, so it is exposed to exactly the
same failure and is tested here too. Both configurations must show zero drift.
Needs tmp/fr_singe (see docs/STATUS.md, reproducing the sustained-action
result). ~55 s, nearly all of it the k-means in H.build; the rate-controlled
encode of 120 frames is ~2 s.
@@ -29,49 +33,63 @@ import numpy as np
import vq as VQ, vq_hybrid as H, ratectl as RC
m = H.build("tmp/fr_singe", k1=256, k4=256, iters=16)
# lam_lo=1.0: let quiet frames spend the whole allowance, which is the
# harder case for this test -- it maximises how often lam moves frame to frame.
enc = RC.encode_rate_controlled(m, target_kbps=110, lam_lo=1.0)
lam = enc["lam"]
sw = int((np.diff(lam) != 0).sum())
print(f"frames={len(lam)} distinct lam used={len(set(lam.tolist()))} "
f"lam changes frame-to-frame={sw} "
f"overruns={int(enc['overrun'].sum())}")
pal, nbx = m["pal"], m["W"] // 4
emitted = []
drift_px, drift_db = [], []
for f, (rec, mode) in enumerate(zip(enc["recon"], enc["modes"])):
out = rec.copy()
if f > 0:
prev_true = emitted[-1]
for b in np.flatnonzero(mode == 0): # SKIP blocks
by, bx = divmod(int(b), nbx)
y, x = by*4, bx*4
out[y:y+4, x:x+4] = prev_true[y:y+4, x:x+4]
emitted.append(out)
d = (out != rec).sum()
drift_px.append(d)
drift_db.append(VQ.psnr(pal[rec], pal[out]))
def check(label, cycle_budget):
"""Encode, replay as a decoder would, and return the drift in pixels."""
print(f"\n=== {label} ===")
m.pop("_sym", None)
# lam_lo=1.0: let quiet frames spend the whole allowance, which is the
# harder case for this test -- it maximises how often lam moves frame to
# frame.
enc = RC.encode_rate_controlled(m, target_kbps=110, lam_lo=1.0,
cycle_budget=cycle_budget)
lam = enc["lam"]
sw = int((np.diff(lam) != 0).sum())
print(f"frames={len(lam)} distinct lam used={len(set(lam.tolist()))} "
f"lam changes frame-to-frame={sw} "
f"overruns={int(enc['overrun'].sum())}")
drift_px = np.array(drift_px)
print(f"pixels differing from what the encoder recorded:")
print(f" frames with ANY drift: {int((drift_px>0).sum())}/{len(drift_px)}")
print(f" max {drift_px.max()} px ({100*drift_px.max()/(m['H']*m['W']):.1f}% of frame)")
print(f" mean {drift_px.mean():.0f} px")
fin = [d for d in drift_db if np.isfinite(d)]
if fin:
print(f" encoder-vs-decoder agreement: min {min(fin):.1f} dB "
f"(inf = identical on {len(drift_db)-len(fin)} frames)")
pal, nbx = m["pal"], m["W"] // 4
emitted = []
drift_px, drift_db = [], []
for f, (rec, mode) in enumerate(zip(enc["recon"], enc["modes"])):
out = rec.copy()
if f > 0:
prev_true = emitted[-1]
for b in np.flatnonzero(mode == 0): # SKIP blocks
by, bx = divmod(int(b), nbx)
y, x = by*4, bx*4
out[y:y+4, x:x+4] = prev_true[y:y+4, x:x+4]
emitted.append(out)
d = (out != rec).sum()
drift_px.append(d)
drift_db.append(VQ.psnr(pal[rec], pal[out]))
drift_px = np.array(drift_px)
print(f"pixels differing from what the encoder recorded:")
print(f" frames with ANY drift: {int((drift_px>0).sum())}/{len(drift_px)}")
print(f" max {drift_px.max()} px ({100*drift_px.max()/(m['H']*m['W']):.1f}% of frame)")
print(f" mean {drift_px.mean():.0f} px")
fin = [d for d in drift_db if np.isfinite(d)]
if fin:
print(f" encoder-vs-decoder agreement: min {min(fin):.1f} dB "
f"(inf = identical on {len(drift_db)-len(fin)} frames)")
r = RC.summarise(m, enc, 110)
print(f"\nratectl reports PSNR {r['psnr']:.2f} dB, {r['kbps']:.1f} KB/s "
f"(target 110), {r['over']:.0f}% of frames over budget")
tp = np.mean([VQ.psnr(o, pal[e]) for o, e in zip(m["rgb"], emitted)])
print(f"what a decoder actually reconstructs: {tp:.2f} dB "
f"-> overstated by {r['psnr']-tp:.2f} dB")
return drift_px
r = RC.summarise(m, enc, 110)
print(f"\nratectl reports PSNR {r['psnr']:.2f} dB, {r['kbps']:.1f} KB/s "
f"(target 110), {r['over']:.0f}% of frames over budget")
tp = np.mean([VQ.psnr(o, pal[e]) for o, e in zip(m["rgb"], emitted)])
print(f"what a decoder actually reconstructs: {tp:.2f} dB "
f"-> overstated by {r['psnr']-tp:.2f} dB")
# Acceptance criterion for the fix: a decoder replaying the emitted stream must
# reconstruct exactly what the encoder recorded.
sys.exit(1 if (drift_px > 0).any() else 0)
# reconstruct exactly what the encoder recorded -- under either controller.
bad = 0
for label, cb in (("bytes only (session 6)", None),
("bytes + CPU ceiling (session 8)", RC.FRAME_CYCLES)):
d = check(label, cb)
bad += int((d > 0).any())
sys.exit(1 if bad else 0)
tools/analysis/11_cpu_budget.py
+97 -23
View File
@@ -22,6 +22,15 @@ import sys, os, argparse
sys.path.insert(0, "tools/encoder")
import numpy as np
from dlx import DLX
import vq_hybrid as H
import ratectl as RC
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import buscost as B
# The audio byte rate is now DERIVED, not restated: 15.6 kHz mono MSM6258V is
# 15,625 4-bit samples/s, two to a byte. RC.AUDIO_KBPS's 7.8 is that figure in
# DECIMAL kB, and was being multiplied by 1024 here -- a 2.4% overstatement,
# harmless, but it hid which unit the constant was in.
RC_AUDIO_BPS = B.ADPCM_BYTES_PER_S
# Machine clocks, confirmed from MAME 0.277 src/mame/sharp/x68k.cpp:1133/1194/
# 1200 -- not recalled. x68000 and x68ksupr are BOTH 40_MHz_XTAL/4 = 10 MHz;
@@ -30,10 +39,13 @@ from dlx import DLX
CLOCKS = {"stock": 10.0, "super": 10.0, "xvi": 33.33 / 2, "x68030": 25.0}
FPS = 12
# cycles per block, measured on the emulated 68000 (synthetic single-mode frames)
C_V1, C_V4, C_RAW = 299.9, 448.2, 400.4
C_SKIP_FAST = 53.0 / 4 # all-SKIP header byte: one tst.b for 4
C_SKIP_MIXED = 45.0 # a SKIP block inside a mixed byte
# Cycles per block, measured on the emulated 68000 (synthetic single-mode
# frames). Defined in tools/encoder/vq_hybrid.py, which is where the mode
# decision needs them too -- one copy, not two, so a re-measurement cannot
# leave the encoder and the scorer disagreeing.
C_V1, C_V4, C_RAW = H.C_V1, H.C_V4, H.C_RAW
C_SKIP_FAST, C_SKIP_MIXED = H.C_SKIP_CLUSTERED, H.C_SKIP_MIXED
cycles = H.cycles
ap = argparse.ArgumentParser()
ap.add_argument("container", nargs="?",
@@ -41,6 +53,37 @@ ap.add_argument("container", nargs="?",
ap.add_argument("--machine", default="stock", choices=list(CLOCKS),
help="which X68000's clock to budget against (default stock)")
ap.add_argument("--fps", type=float, default=FPS)
# FINDINGS 35: the frame budget has never had the disk in it. The bitstream has
# to be moved off SCSI into the ring buffer, and on this machine that costs CPU
# whether it is DMA (the HD63450 cycle-steals) or PIO (the 68000 moves every
# byte). Default ON, because scoring a decoder against a budget that assumes the
# data arrives for free is exactly the mistake 35 was raised to stop.
ap.add_argument("--io", default="dma", choices=["dma", "pio", "none"],
help="how the bitstream reaches RAM (default dma)")
ap.add_argument("--dma-clocks-per-word", type=float, default=8.0,
help="HD63450 cycle-steal. ESTIMATE from FINDINGS 5, NEVER "
"MEASURED, and the most load-bearing unmeasured number "
"in the project (FINDINGS 35.3)")
ap.add_argument("--dma-clocks-per-byte", type=float, default=5.0,
help="what the SCSI DMA costs per DELIVERED BYTE. The MB89352 "
"is an 8-bit port, so the DMAC pays per byte and the "
"per-word denominator of FINDINGS 5/39.7 was half the "
"real debit (FINDINGS 43). 5 = single-address, bus held, "
"no drive wait; 9 = dual-address")
ap.add_argument("--pio-clocks-per-byte", type=float, default=12.0,
help="hand-derived floor for a 68000 register-to-RAM copy")
# Audio is NOT the disk, and charging it the disk's rate was charging it the
# favourable side of an open question. tools/analysis/21_iplrom_dmac.py reads
# the IPL ROM's own HD63450 setup: channel 3 is dual address, 8-bit port, cycle
# steal WITHOUT hold, external request -- one full arbitration per byte, no
# burst to amortise it over. 16 is the datasheet best case, 19 the worst.
ap.add_argument("--adpcm-clocks-per-byte", type=float,
default=B.ADPCM_CLK_BYTE_BEST,
help="what an ADPCM byte costs. READ OUT OF THE IPL ROM's DMAC "
"configuration (21_iplrom_dmac.py), not assumed: dual "
"address + per-byte arbitration = 16 best, 19 worst. The "
"audio stream always DMAs, whatever --io says about the "
"disk")
a = ap.parse_args()
CPUHZ = CLOCKS[a.machine] * 1e6
FPS = a.fps
@@ -50,25 +93,48 @@ if not os.path.exists(a.container):
d = DLX(a.container)
def cycles(mode):
g = mode.reshape(-1, 4) # one header byte = four blocks
allskip = (g == 0).all(1)
c = allskip.sum() * 4 * C_SKIP_FAST
m = g[~allskip]
c += (m == 0).sum() * C_SKIP_MIXED
c += (m == 1).sum() * C_V1
c += (m == 2).sum() * C_V4
c += (m == 3).sum() * C_RAW
return c
# --- what the transfer costs, from the container's own byte rate
vid_bps = sum(n + 4 for (_, n) in d.frames) / d.nframes * d.fps
io_bps = vid_bps + RC_AUDIO_BPS
aud_cycles_per_s = RC_AUDIO_BPS * a.adpcm_clocks_per_byte
if a.io == "dma":
io_cycles_per_s = vid_bps * a.dma_clocks_per_byte + aud_cycles_per_s
elif a.io == "pio":
io_cycles_per_s = vid_bps * a.pio_clocks_per_byte + aud_cycles_per_s
else:
io_cycles_per_s = 0.0
io_pct = 100 * io_cycles_per_s / CPUHZ
aud_pct = 100 * aud_cycles_per_s / CPUHZ
FRAME_NET = FRAME * (1 - io_pct / 100)
modes = [d.modes(f) for f in range(d.nframes)]
cyc = np.array([cycles(m) for m in modes])
pct = 100 * cyc / FRAME
pct = 100 * cyc / FRAME_NET
ns = np.array([100 * (m != 0).mean() for m in modes])
print(f"{a.container}: {d.nframes} frames, {d.nb} blocks/frame")
print(f"budget: {a.machine} @ {CLOCKS[a.machine]:.2f} MHz, {FPS:g} fps "
f"-> {FRAME:,.0f} cycles/frame")
print(f" I/O ({a.io}): {io_bps/1024:.1f} KB/s costs {io_pct:.1f}% of the CPU "
f"-> {FRAME_NET:,.0f} cycles/frame left for decoding")
if a.io != "none":
print(f" video {vid_bps/1024:6.1f} KB/s x "
f"{(a.dma_clocks_per_byte if a.io=='dma' else a.pio_clocks_per_byte):g}"
f" clk/B = {io_pct-aud_pct:5.2f}% "
f"(W: still open, ROADMAP B3 / FINDINGS 42.4)\n"
f" audio {RC_AUDIO_BPS/1024:6.2f} KB/s x {a.adpcm_clocks_per_byte:g}"
f" clk/B = {aud_pct:5.2f}% "
f"(SETTLED: read out of the IPL ROM, FINDINGS 52)")
if a.io == "dma":
print(f" {a.dma_clocks_per_byte:g} clocks/BYTE, the MC68450 datasheet "
f"floor for an 8-bit port (FINDINGS 43).\n It is not measured on "
f"hardware; what IS settled is that the per-word denominator this\n"
f" used before session 14 was physically impossible -- 2.5 "
f"clocks/byte is below\n the 68000's 4-clock minimum bus cycle.")
elif a.io == "none":
print(" WARNING: --io none scores the decoder as if the disk were free. "
"That is the\n premise FINDINGS 35 overturned; every 'N frames miss' "
"figure before session 9\n was computed this way.")
if a.machine != "stock":
print(" (derived: scaled by clock from cycles measured on the 10 MHz core.\n"
" MAME 0.277 marks x68ksupr/x68kxvi/x68030 MACHINE_NOT_WORKING, so\n"
@@ -85,15 +151,20 @@ TIMED_FRAMES = (("min non-SKIP", 15.4, 31.5), ("median", 48.1, 73.8),
("p90", 82.5, 116.4), ("max non-SKIP", 100.0, 135.8))
if (os.path.abspath(a.container) == os.path.abspath(TIMED)
and a.machine == "stock" and a.fps == 12):
print("model vs the frames actually timed on the 68000:")
print("model vs the frames actually timed on the 68000 "
"(the model reads HIGH, and by more\n as the frame gets harder -- "
"so a 'does not fit' from it is the safe direction):")
for label, frac, meas in TIMED_FRAMES:
i = int(np.argmin(abs(ns - frac)))
print(f" {label:<14} non-SKIP {ns[i]:5.1f}% model {pct[i]:6.1f}% "
f"measured {meas:5.1f}% error {pct[i]-meas:+.1f} pt")
else:
print(f"(no 68000 timings for this container/machine -- the model was "
f"validated to\n within 1 pt on {TIMED} at stock/12fps;\n"
f" run tools/bench/decode.lua to time another container)")
print(f"(no 68000 timings for this container/machine. The model is "
f"validated against four\n frames timed on the 68000, and only on "
f"{TIMED}\n at stock/12fps -- run it on that container to see the "
f"errors, which are a few points\n CONSERVATIVE and grow with the "
f"non-SKIP fraction. Run tools/bench/decode.lua to\n time another "
f"container.)")
print(f"\nper-frame cost, % of a {FPS:g}fps frame budget:")
print(f" measured-cost model: median {np.median(pct):5.1f} "
@@ -107,10 +178,11 @@ if a.machine == "stock" and a.fps == 12:
f"(optimistic by {np.median(pct)/np.median(old):.2f}x at the median)")
miss = pct > 100
print(f"\nframes that do NOT fit {FRAME:,.0f} cycles: {miss.sum()}/{d.nframes} "
print(f"\nframes that do NOT fit {FRAME_NET:,.0f} cycles: {miss.sum()}/{d.nframes} "
f"({100*miss.mean():.0f}%)")
print(f" sustainable framerate if EVERY frame must fit: "
f"{CPUHZ/cyc.max():.1f} fps; at the mean frame {CPUHZ/cyc.mean():.1f} fps")
f"{CPUHZ*(1-io_pct/100)/cyc.max():.1f} fps; at the mean frame "
f"{CPUHZ*(1-io_pct/100)/cyc.mean():.1f} fps")
if miss.any():
print(f" worst {pct.max():.1f}% -- {(pct.max()-100)/100*1000/FPS:.0f} ms late "
f"on an {1000/FPS:.0f} ms frame")
@@ -123,5 +195,7 @@ print(f"\nwhere the cycles go, over the whole window:")
for k, n in enumerate(("SKIP", "V1", "V4", "RAW")):
print(f" {n:<5} {100*tot[k]/tot.sum():5.1f}% of blocks "
f"{100*spend[k]/spend.sum():5.1f}% of the cycles")
print(f"\nV4 is {C_V4/C_V1:.2f}x a V1 block for {4}x the payload bytes -- the mode "
f"decision\nin vq_hybrid.py charges it the bytes but not the cycles.")
print(f"\nV4 is {C_V4/C_V1:.2f}x a V1 block for {4}x the payload bytes. Since "
f"session 8 the mode\ndecision charges it BOTH (decide(ctx, lam, mu), "
f"FINDINGS 31), which is why V4 is now\nthe rarest non-SKIP mode here -- "
f"a byte-rich profile buys its way out to RAW instead.")
tools/analysis/12_span_tradeoff.py
+150
View File
@@ -0,0 +1,150 @@
#!/usr/bin/env python3
"""What does spending the idle bus bandwidth buy back in CPU cycles?
python3 tools/analysis/12_span_tradeoff.py [container.dlx] --bus <KB/s>
FINDINGS 28 leaves the decoder CPU-bound at 110 KB/s on a much wider pipe. Every
codec decision was made when bytes were scarce, so each one trades cycles to
save them -- and the cheapest thing a 68000 can be handed is the most expensive
thing to store: word-expanded pixels in row-linear runs.
This prices ONE new mode against the real mode maps: a per-row SPAN of
word-expanded literals, `movem.l`-ed straight from the stream buffer into GVRAM.
A run of L horizontally adjacent dirty blocks becomes 4 spans of 4L pixels.
MEASURED as of session 8 (FINDINGS 30), on the 68000, with the span decoder in
tools/bench/blit.s v6 and the streams in tools/bench/prep_spans.py:
43.7 cycles per span + 9.152 per pixel, fitting eleven span lengths to within
0.3%. That is the ENCODER-ASSISTED format: the record is an absolute GVRAM
address and a jump displacement into an unrolled copy chain, so the decoder does
no arithmetic per span. The obvious decoder -- handed (x, npix) and left to work
the copy out -- measures 97.9 + 10.46 and is 2.2x dearer on a 24-pixel span (v5).
Span length is therefore a multiple of 24 pixels, and a run pads up to it; the
padding is free of cycles beyond its pixels and correct on screen, because a
literal span carries true pixels of the current frame.
The mode maps are NOT re-optimised: this only re-codes regions the encoder
already chose to redraw, so it is a lower bound on what a cost-aware encoder
would find.
"""
import sys, os, argparse
sys.path.insert(0, "tools/encoder")
import numpy as np
from dlx import DLX
FRAME_CYC = 833333.0 # 12fps at 10 MHz
AUDIO_KBPS = 7.8
C_V1, C_V4, C_RAW = 299.9, 448.2, 400.4 # FINDINGS 28.2 (measured)
C_SKIP_CLUSTERED, C_SKIP_MIXED = 13.25, 45.0
SPAN_OVERHEAD = 43.7 # per span, MEASURED, FINDINGS 30
CYC_PX_ROWLIN = 9.152 # per pixel, MEASURED, FINDINGS 30
SPAN_UNIT_PX = 24 # 12 registers of movem.l, one chain unit
SPAN_BYTES_PX = 2 # word-expanded: 1 pixel = 1 word
SPAN_HDR = 6 # u32 GVRAM address + u16 jump displacement
def span_px(npix): # a span is a whole number of units
return -(-npix // SPAN_UNIT_PX) * SPAN_UNIT_PX
ap = argparse.ArgumentParser()
ap.add_argument("container", nargs="?",
default="tmp/rc_fr_singe_sasi_rcprofile.dlx")
ap.add_argument("--bus", type=float, required=True,
help="REQUIRED. There is no default: the delivery rate is a property of the medium and this project has never measured it. FINDINGS 42.1 -- the figure this tool used to default to was a user-supplied '4 Mbps' with no provenance, was a tenth of SCSI-1's asynchronous rating, and was never a bus measurement at all. A default let every table in FINDINGS 30-49 be scored against it without anyone restating it. Pass one explicitly.")
ap.add_argument("--fps", type=float, default=12.0)
a = ap.parse_args()
if not os.path.exists(a.container):
sys.exit(f"missing {a.container}")
BYTE_BUD = (a.bus - AUDIO_KBPS) * 1024 / a.fps
d = DLX(a.container)
BLK_C = {1: C_V1, 2: C_V4, 3: C_RAW}
BLK_B = {1: 1, 2: 4, 3: 16}
rows = []
for f in range(d.nframes):
mode = d.modes(f)
g = mode.reshape(-1, 4)
allskip = (g == 0).all(1)
base = allskip.sum() * 4 * C_SKIP_CLUSTERED
mm = g[~allskip]
base += (mm == 0).sum() * C_SKIP_MIXED
for k, c in BLK_C.items():
base += (mm == k).sum() * c
base_b = d.mode_bytes + sum(BLK_B.get(int(x), 0) for x in mode)
m = mode.reshape(d.nby, d.nbx)
cand = []
for by in range(d.nby):
dirty = m[by] != 0
i = 0
while i < d.nbx:
if not dirty[i]:
i += 1
continue
j = i
while j < d.nbx and dirty[j]:
j += 1
L = j - i
cur_c = sum(BLK_C[int(b)] for b in m[by][i:j])
cur_b = sum(BLK_B[int(b)] for b in m[by][i:j])
sp = span_px(4 * L) # padded to the chain's 24-pixel unit
span_c = 4 * (SPAN_OVERHEAD + sp * CYC_PX_ROWLIN)
span_b = 4 * (SPAN_HDR + sp * SPAN_BYTES_PX)
if span_c < cur_c:
cand.append((cur_c - span_c, span_b - cur_b, L))
i = j
cand.sort(key=lambda s: -(s[0] / max(s[1], 1))) # best cycles per byte
cyc, byt, taken = base, base_b, 0
for dc, db, L in cand:
if byt + db <= BYTE_BUD:
cyc -= dc; byt += db; taken += 1
rows.append((base, cyc, base_b, byt, len(cand), taken))
base, new, bb, nb, ncand, ntaken = map(np.array, list(zip(*rows)))
pc = lambda v: 100 * v / FRAME_CYC
print(f"{a.container}: {d.nframes} frames")
print(f"bus {a.bus:.0f} KB/s - {AUDIO_KBPS} audio -> {BYTE_BUD:,.0f} B/frame "
f"at {a.fps:g}fps\n")
print(f"{'':<26}{'today':>12}{'+ literal spans':>18}")
for label, fn in (("median frame", np.median),
("p90 frame", lambda v: np.percentile(v, 90)),
("worst frame", np.max)):
print(f" {label:<24}{pc(fn(base)):>11.1f}%{pc(fn(new)):>17.1f}%")
print(f" {'frames missing budget':<24}{int((base>FRAME_CYC).sum()):>8}/{d.nframes}"
f"{int((new>FRAME_CYC).sum()):>14}/{d.nframes}")
print(f" {'bitrate':<24}{bb.mean()*a.fps/1024:>10.1f} KB/s"
f"{nb.mean()*a.fps/1024:>13.1f} KB/s")
print(f"\nspans taken: {ntaken.sum()} of {ncand.sum()} candidate runs "
f"({100*ntaken.sum()/max(ncand.sum(),1):.0f}%) -- the rest priced out by the bus")
brk = next(L for L in range(1, 65)
if 4*(SPAN_OVERHEAD + span_px(4*L)*CYC_PX_ROWLIN) < L*C_V1)
print(f"\nspan cost MEASURED (FINDINGS 30): {SPAN_OVERHEAD:.1f}/span + "
f"{CYC_PX_ROWLIN:.3f}/pixel, {SPAN_UNIT_PX}-pixel units.")
print(f"a run of L blocks beats all-V1 from L={brk} blocks up "
f"({4*(SPAN_OVERHEAD + span_px(4*brk)*CYC_PX_ROWLIN)/brk:.0f} vs {C_V1:.0f} "
f"cycles/block); the floor at a full row is "
f"{4*(SPAN_OVERHEAD + span_px(256)*CYC_PX_ROWLIN)/64:.0f}.")
print("The mode maps are NOT re-optimised, so this is a lower bound on a "
"cost-aware encoder.")
# FINDINGS 28.5 said a scene cut cannot fit at 12fps: the cheapest full redraw
# the codec's mode set allows is all-V1 at 110.5% of budget. 29.4 reopened that
# on derived span costs; this is the same arithmetic on measured ones. Mix a
# fraction x of a 100%-changed frame as full-row spans, V1 for the rest.
NB = d.nb
row_c = 4 * (SPAN_OVERHEAD + span_px(4 * d.nbx) * CYC_PX_ROWLIN) / d.nbx
row_b = 4 * (SPAN_HDR + span_px(4 * d.nbx) * SPAN_BYTES_PX) / d.nbx
x_cpu = (NB * C_V1 - FRAME_CYC) / (NB * (C_V1 - row_c))
x_bus = (BYTE_BUD - d.mode_bytes - NB * BLK_B[1]) / (NB * (row_b - BLK_B[1]))
print(f"\nscene cut (100% of blocks change), spans at full row width "
f"({row_c:.0f} cyc, {row_b:.1f} B per block):")
print(f" all-V1 costs {100*NB*C_V1/FRAME_CYC:.1f}% of the frame -- FINDINGS 28.5")
print(f" CPU needs x >= {x_cpu:.3f} of the frame as spans; "
f"the bus allows x <= {x_bus:.3f}")
print(" " + ("the interval is NOT empty: a cut fits at 12fps (FINDINGS 29.4 holds)"
if x_cpu <= x_bus else
"the interval IS empty: a cut does not fit (FINDINGS 28.5 stands)"))
tools/analysis/13_cpu_ratectl.py
+115
View File
@@ -0,0 +1,115 @@
#!/usr/bin/env python3
"""What does fitting the CPU budget cost in quality? (session 8, lever B)
python3 tools/analysis/13_cpu_ratectl.py [frames_dir] [--profiles scsi]
Session 6 made the BYTE budget a ceiling by bisecting `lam` per frame. FINDINGS
28 then showed the binding budget is CYCLES, not bytes, and that the mode
decision cannot see them: it minimises `D + lam*R` on a machine that charges V4
1.49x a V1 block while the lagrangian charges it 4x.
`ratectl.encode_rate_controlled(cycle_budget=...)` adds the second controller --
`mu` bisected per frame against 833,333 cycles, with the lam bisection nested
inside it. This measures what that costs: PSNR, bitrate, and how many frames
still miss, against the same encode with the ceiling off.
The cycle budget is HARD, not a bucket. Bytes bank in the player's ring buffer;
there is no double buffer to decode ahead into, so a frame that misses its
decode deadline is simply late (FINDINGS 28).
Both controllers score frames with the exact clustered cost `vq_hybrid.cycles`,
validated to 1 point against the 68000 (FINDINGS 28.2) -- not with the per-block
ranking constant the mode decision uses. See vq_hybrid's note on SKIP.
"""
import argparse, os, pickle, sys, time
sys.path.insert(0, "tools/encoder")
import numpy as np
import vq as VQ, vq_hybrid as H, ratectl as RC
ap = argparse.ArgumentParser()
ap.add_argument("frames_dir", nargs="?", default="tmp/fr_singe")
ap.add_argument("--profiles", default="scsi")
ap.add_argument("--fps", type=int, default=12)
ap.add_argument("--cache", default=None, help="pickle of H.build (auto by dir)")
a = ap.parse_args()
if not os.path.isdir(a.frames_dir):
sys.exit(f"missing {a.frames_dir} -- see tools/bench/check.sh for extraction")
BUDGET = RC.FRAME_CYCLES
# H.build is ~55 s, nearly all k-means, and it does not depend on the profile:
# both ship k1=k4=256. One build, cached, serves every row of the table.
cache = a.cache or f"tmp/model_{os.path.basename(a.frames_dir.rstrip('/'))}.pkl"
# The build parameters are stored with the model and a mismatch rebuilds: the
# cache is keyed on the frames directory alone, and once H.build acquired an
# option (session 28's reserved black entry, 23.4) a stale pickle would quietly
# serve a model the shipping encoder no longer builds. Same guard as
# tools/analysis/16_span_roundtrip.py.
SIG = dict(k1=256, k4=256, iters=16, reserve_black=True)
m = None
if os.path.exists(cache):
m = pickle.load(open(cache, "rb"))
if m.get("sig") != SIG:
print(f"{cache}: built with {m.get('sig')}, wanted {SIG} -- rebuilding")
m = None
else:
print(f"model from {cache}")
if m is None:
t = time.time()
m = H.build(a.frames_dir, **SIG)
m["sig"] = SIG
pickle.dump(m, open(cache, "wb"))
print(f"built model in {time.time()-t:.0f} s -> {cache}")
print(f"{a.frames_dir}: {len(m['idx'])} frames, {m['nb']} blocks, "
f"budget {BUDGET:,.0f} cycles/frame at {a.fps}fps\n")
def run(prof_name, cycle_budget):
p = RC.PROFILES[prof_name]
m.pop("_sym", None) # the frame-symbol cache holds one frame
t = time.time()
enc = RC.encode_rate_controlled(m, p["kbps"], fps=a.fps, lam_lo=p["lam"],
cycle_budget=cycle_budget)
s = RC.summarise(m, enc, p["kbps"], fps=a.fps)
s["secs"] = time.time() - t
s["ns"] = float(np.mean([100*(mm != 0).mean() for mm in enc["modes"]]))
return s, enc
rows = []
for name in a.profiles.split(","):
for label, cb in (("bytes only", None), ("bytes + cycles", BUDGET)):
s, enc = run(name, cb)
rows.append((name, label, s))
print(f"{name:5s} {label:<15s} {s['secs']:5.1f} s "
f"PSNR {s['psnr']:.2f} dB {s['kbps']:6.1f} KB/s "
f"CPU med {100*s['cyc_med']/BUDGET:5.1f}% p90 "
f"{100*s['cyc_p90']/BUDGET:5.1f}% max {100*s['cyc_max']/BUDGET:5.1f}% "
f"miss {s['cpu_miss']:3d} late {s['late']:2d} "
f"mu med {s['mu_med']:.4f} max {s['mu_max']:.3f}")
print()
hdr = f"{'':<22}{'PSNR':>8}{'KB/s':>9}{'CPU med':>10}{'CPU max':>10}{'miss':>7}"
for name in a.profiles.split(","):
r = {lab: s for n, lab, s in rows if n == name}
b, c = r["bytes only"], r["bytes + cycles"]
print(f"--- {name} (target {RC.PROFILES[name]['kbps']} KB/s) ---")
print(hdr)
for lab, s in (("bytes only", b), ("bytes + cycles", c)):
print(f" {lab:<20}{s['psnr']:>7.2f} {s['kbps']:>8.1f} "
f"{100*s['cyc_med']/BUDGET:>9.1f}%{100*s['cyc_max']/BUDGET:>9.1f}%"
f"{s['cpu_miss']:>6d}")
print(f" {'cost of fitting':<20}{c['psnr']-b['psnr']:>+7.2f} dB, "
f"{c['kbps']-b['kbps']:+.1f} KB/s, "
f"{b['cpu_miss']-c['cpu_miss']} fewer misses, "
f"{c['late']} frames unfixable at mu={RC.MU_CLIFF:g}")
print(f" {'modes % (b/c)':<20}SKIP {b['skip']:.1f}/{c['skip']:.1f} "
f"V1 {b['v1']:.1f}/{c['v1']:.1f} V4 {b['v4']:.1f}/{c['v4']:.1f} "
f"RAW {b['raw']:.1f}/{c['raw']:.1f}")
print()
print("FINDINGS 28.7: re-coding every non-SKIP block as V1 is the floor the "
"CURRENT mode set\nallows, and it still missed 11 frames at the retired "
"110 KB/s profile / 12 at scsi.\nMisses above that floor are spans, not "
"the mode decision -- and 31.3 showed the\nfloor itself was too "
"pessimistic, because the real decision can move a block to SKIP.")
tools/analysis/14_dmac_chain.py
+233
View File
@@ -0,0 +1,233 @@
#!/usr/bin/env python3
"""Would letting the HD63450 paint the spans beat letting the 68000 do it?
python3 tools/analysis/14_dmac_chain.py [container.dlx] --bus <KB/s>
[--dma-px-bus 2] [--disk-bus-byte 1]
FINDINGS 29.6 called this the one lever that could move the CPU budget without
spending a byte, and left it uncosted. FINDINGS 30 measured the alternative --
the 68000 painting spans itself, 43.7 cycles per span + 9.152 per pixel. This
prices the two against each other, and the answer turns on a resource neither
section costed: the 68000's own LOCAL BUS.
FINDINGS 29's "the bus has 4x the headroom the CPU has" is about the SCSI pipe,
110 KB/s of the delivery pipe. That is a different bus. The 68000's memory bus runs one 4-clock
cycle at a time and carries instruction prefetch as well as data, and
tools/analysis/15_bus_occupancy.py measures the decoder using 86.7% of it.
THE TWO DESIGNS ARE THE SAME CONTAINER. v6's record is {u32 absolute GVRAM
address, u16 jump displacement} = 6 bytes; an MC68450/HD63450 array-chaining
entry is {u32 memory address, u16 transfer count} = 6 bytes. Set the channel to
dual-address, direction device->memory, Sequence Control counting both addresses
up: MAR reloads per entry (the GVRAM destination), DAR walks the stream buffer,
MTC is the span's word count. The chain array IS the span table.
THE DMAC CONSTANTS ARE NOW SOURCED, and they killed the first answer. From the
MC68450 manual (Motorola, Jul 1989, bitsavers), Fig 4-25 sheet 4: a dual-address
WORD operand between two 16-bit ports is **9 clocks**, because note 2 gives the
DMAC 4-clock reads and **5-clock writes**. The 68000 writes in 4. So:
DMAC 9.000 clocks/pixel (datasheet)
v6 9.152 clocks/pixel (measured, FINDINGS 30)
A 1.7% difference. Session 10's first pass guessed 2 bus cycles = 8 clocks from
bus arithmetic and was 12% optimistic; the extra clock on every DMAC write is
the whole story. Per span, sequential array chaining costs 36 clocks (Fig 4-25
sheet 1) against v6's measured 43.7 -- the DMAC's one real edge, and it is small.
AND DMA DOES NOT OVERLAP. The 68000 has no cache and a two-word prefetch queue,
so it stalls as soon as another master takes the bus. Frame time is therefore
CPU + DMA, additive. Session 10's first pass used max(CPU, bus) and got 53/120
where the additive model gives 84/120; FINDINGS 35's flat debit was right.
So the only material difference left is v6's 24-pixel padding quantum -- and
that is a property of v6's unrolled chain, not of the CPU. The `v7 fine tail`
column prices fixing it in software instead, and as of session 11 that column
is MEASURED on the 68000 (blit.s v7, tools/bench/span.sh, FINDINGS 40) rather
than derived: 66.0 clocks per span + 9.143 per coarse pixel + 9.978 per fine
pixel, with a 2-pixel quantum that a run of 4x4 blocks pads to exactly.
"""
import sys, os, argparse
sys.path.insert(0, "tools/encoder")
sys.path.insert(0, "tools/analysis")
import numpy as np
from dlx import DLX
import buscost as B
FRAME_CYC = 833333.0
AUDIO_KBPS = 7.8
import vq_hybrid as _H
C_V1, C_V4, C_RAW = _H.C_V1, _H.C_V4, _H.C_RAW # FINDINGS 28.2 (MEASURED)
# 45.0 until session 12 measured it at 55.0 (FINDINGS 41.5) -- imported now, so
# the correction cannot be undone by a stale copy.
C_SKIP_CLUSTERED, C_SKIP_MIXED = _H.C_SKIP_CLUSTERED, _H.C_SKIP_MIXED
SPAN_BYTES_PX, SPAN_HDR = 2, 6
ap = argparse.ArgumentParser()
ap.add_argument("container", nargs="?", default="tmp/rc_fr_singe_scsi_cpufit.dlx")
ap.add_argument("--bus", type=float, required=True,
help="REQUIRED. There is no default: the delivery rate is a property of the medium and this project has never measured it. FINDINGS 42.1 -- the figure this tool used to default to was a user-supplied '4 Mbps' with no provenance, was a tenth of SCSI-1's asynchronous rating, and was never a bus measurement at all. A default let every table in FINDINGS 30-49 be scored against it without anyone restating it. Pass one explicitly.")
ap.add_argument("--fps", type=float, default=12.0)
ap.add_argument("--dma-px-clk", type=float, default=B.DMA_PX_CLK,
help="clocks the DMAC spends per pixel, dual-address word "
"between two 16-bit ports. 9 is the DATASHEET figure "
"(MC68450 Fig 4-25 sheet 4).")
ap.add_argument("--disk-clk-byte", type=float, default=5.0,
help="clocks the SCSI DMA steals per BYTE delivered. The SPC is "
"an 8-bit port, so the DMAC pays per byte, not per word "
"(FINDINGS 43). 5, the default, is the OPTIMISTIC end and "
"what ratectl encodes against: single-address, bus held, no "
"drive wait (Fig 4-25 sheet 2). 9 is dual-address, which is "
"what MAME models and what applies if the board does not "
"drive DACK. Score both.")
ap.add_argument("--disk-clk-word", type=float, default=None,
help="DEPRECATED denominator of FINDINGS 39.7/42, kept so the "
"old tables reproduce: sets --disk-clk-byte to half this")
a = ap.parse_args()
if a.disk_clk_word is not None:
a.disk_clk_byte = a.disk_clk_word / 2.0
if not os.path.exists(a.container):
sys.exit(f"missing {a.container}")
BYTE_BUD = (a.bus - AUDIO_KBPS) * 1024 / a.fps
BUS_SLOTS = FRAME_CYC / B.BUS_CLK
d = DLX(a.container)
BLK_C = {1: C_V1, 2: C_V4, 3: C_RAW}
BLK_B = {1: 1, 2: 4, 3: 16}
def runs(m, by):
dirty = m[by] != 0
i = 0
while i < d.nbx:
if not dirty[i]:
i += 1; continue
j = i
while j < d.nbx and dirty[j]:
j += 1
yield i, j
i = j
def span_cost(design, L):
"""(pixels carried, clocks charged to the frame) for a run of L blocks,
as 4 rows of 4L pixels. Every design is charged additively: the 68000
cannot execute while the DMAC owns the bus."""
if design == "v6":
px = B.pad24(4 * L)
return 4 * px, 4 * (B.V6_SPAN_CYC + px * B.V6_PX_CYC)
if design == "v7":
px, c = B.v7_span(4 * L)
return 4 * px, 4 * c
px = 4 * L
return 4 * px, 4 * (B.DMA_CHAIN_CLK + px * a.dma_px_clk)
def score(design):
"""Greedy, as 12_span_tradeoff.py: buy the best clocks-saved per byte spent
until the frame's byte budget is gone. Unlike 12, a spanned block still pays
its mode-map dispatch, which FINDINGS 30.7 flagged as uncounted."""
out = []
for f in range(d.nframes):
m = d.modes(f).reshape(d.nby, d.nbx)
byt = d.mode_bytes + sum(BLK_B.get(int(x), 0) for x in m.ravel())
spanned = np.zeros_like(m, bool)
span_clk = 0.0
cand = []
if design != "none":
for by in range(d.nby):
for i, j in runs(m, by):
L = j - i
cur_c = sum(BLK_C[int(b)] for b in m[by][i:j])
cur_b = sum(BLK_B[int(b)] for b in m[by][i:j])
px, sc = span_cost(design, L)
sc += L * C_SKIP_MIXED # the dispatch still happens
# v7 carries a second u16 (the fine displacement) per span.
hdr = B.V7_SPAN_HDR if design == "v7" else SPAN_HDR
span_b = 4 * hdr + px * SPAN_BYTES_PX
if sc < cur_c:
cand.append((cur_c - sc, span_b - cur_b, by, i, j, sc, L))
cand.sort(key=lambda s: -(s[0] / max(s[1], 1)))
for dc, db, by, i, j, sc, L in cand:
if byt + db <= BYTE_BUD:
byt += db
spanned[by][i:j] = True
span_clk += sc - L * C_SKIP_MIXED
g = m.copy()
g[spanned] = 0
gg = g.reshape(-1, 4)
allskip = (gg == 0).all(1)
cpu = allskip.sum() * 4 * C_SKIP_CLUSTERED
mm = gg[~allskip]
cpu += (mm == 0).sum() * C_SKIP_MIXED
for k, c in BLK_C.items():
cpu += (mm == k).sum() * c
pref, data = B.block_bus(m, spanned)
disk = byt * a.disk_clk_byte
# additive: CPU work, then span painting, then the disk stealing the bus
out.append((cpu + span_clk + disk, (pref + data) * B.BUS_CLK, byt,
spanned.sum()))
return np.array(out).T
DESIGNS = [("today", "none"), ("v6 span", "v6"),
("v7 fine tail", "v7"), ("DMAC chain", "dmac")]
res = {n: score(k) for n, k in DESIGNS}
print(f"{a.container}: {d.nframes} frames, {d.nb} blocks, {a.fps:g} fps")
print(f"SCSI pipe {a.bus:.0f} KB/s -> {BYTE_BUD:,.0f} B/frame; "
f"68000 bus {BUS_SLOTS:,.0f} cycles/frame; CPU {FRAME_CYC:,.0f} clocks\n")
print("PER PIXEL AND PER SPAN -- datasheet against measurement")
print(f" DMAC dual-address word, two 16-bit ports {B.DMA_PX_CLK:.3f} clocks "
f"MC68450 Fig 4-25 sheet 4")
print(f" v6 movem chain {B.V6_PX_CYC:.3f} clocks "
f"MEASURED, FINDINGS 30")
print(f" -> the DMAC is {100*(B.V6_PX_CYC-B.DMA_PX_CLK)/B.V6_PX_CYC:+.1f}% per pixel. "
f"The 68000 writes in 4 clocks; the DMAC takes 5.")
print(f" per span: DMAC array chaining {B.DMA_CHAIN_CLK} clocks against v6's "
f"{B.V6_SPAN_CYC:.1f}\n")
w = 15
print(f"{'':<26}" + "".join(f"{n:>{w}}" for n, _ in DESIGNS))
def row(label, fmt, get):
print(f" {label:<24}" + "".join(f"{fmt(get(res[n])):>{w}}" for n, _ in DESIGNS))
row("bitrate KB/s", lambda v: f"{v:.1f}", lambda r: r[2].mean() * a.fps / 1024)
row("frame, median", lambda v: f"{v:.1f}%", lambda r: 100*np.median(r[0])/FRAME_CYC)
row("frame, worst", lambda v: f"{v:.1f}%", lambda r: 100*r[0].max()/FRAME_CYC)
row("frames missing", lambda v: f"{v}/{d.nframes}",
lambda r: int((r[0] > FRAME_CYC).sum()))
row("blocks spanned/frame", lambda v: f"{v:,.0f}", lambda r: r[3].mean())
print(f"\n ADDITIVE: frame = CPU + span painting + disk DMA. The 68000 has no"
f"\n cache and a two-word prefetch queue, so it stalls the moment another"
f"\n master takes the bus. Disk debited at {a.disk_clk_byte:g} clocks/byte.")
# What is left of the case, isolated.
v6m = int((res["v6 span"][0] > FRAME_CYC).sum())
finem = int((res["v7 fine tail"][0] > FRAME_CYC).sum())
dmam = int((res["DMAC chain"][0] > FRAME_CYC).sum())
print(f"\nWHAT THE DMAC ACTUALLY BUYS, decomposed")
print(f" v6 as built {v6m}/{d.nframes} frames over")
print(f" v7, a finer chain tail (MEASURED) {finem}/{d.nframes}")
print(f" DMAC chain {dmam}/{d.nframes}")
print(f" -> of the gap between v6 and the DMAC, "
f"{100*(v6m-finem)/max(v6m-dmam,1):.0f}% is the 24-pixel padding")
print(f" quantum, which is a property of v6's unrolled chain and fixable")
print(f" in software. The rest is 1.7% a pixel and 7.7 clocks a span.")
# The additive model here IS FINDINGS 35's flat debit, and reproduces its
# 84/120 exactly in the "today" column. Session 10's first pass replaced it with
# max(CPU, bus) and got 53/120; that was wrong, because a 68000 cannot execute
# while the DMAC holds the bus.
print(f"\nbreak-even against all-V1 ({C_V1:.1f} cycles/block), clocks per block")
print(f" {'L':<16}" + "".join(f"{L:>8}" for L in (1, 2, 3, 4, 8, 16, 64)))
for nm, dz in (("v6 as built", "v6"), ("v7 fine tail", "v7"), ("DMAC chain", "dmac")):
print(f" {nm:<16}" + "".join(f"{span_cost(dz, L)[1]/L:>8.0f}"
for L in (1, 2, 3, 4, 8, 16, 64)))
for nm, dz in (("v6 as built", "v6"), ("v7 fine tail", "v7"), ("DMAC chain", "dmac")):
brk = next((L for L in range(1, 65) if span_cost(dz, L)[1] < L * C_V1), None)
print(f" {nm:<16} beats all-V1 from L={brk} blocks up")
tools/analysis/15_bus_occupancy.py
+298
View File
@@ -0,0 +1,298 @@
#!/usr/bin/env python3
"""How much of the bus does the 68000 decoder actually leave for a DMAC?
python3 tools/analysis/15_bus_occupancy.py [container.dlx] [--nframes N]
FINDINGS 29.6's DMAC idea only pays if the DMAC can find bus slots the CPU is
not using. That is not a cycle count, it is a BUS count, and nothing in the tree
had one.
Two sources, and the point is that they check each other:
DATA accesses MEASURED by tools/bench/c68k/c68k_bench, which counts every
Read/Write callback the C68K core makes. Exact.
INSTRUCTION DERIVED here by walking src/player/decode.s's straight-line
prefetch paths in tools/bench/decode.lst and multiplying by the mode
histogram. Not measurable from either emulator: MAME's core
does not expose a fetch count and C68K reads opcodes straight
through a host pointer with no callback.
If the derived DATA figure matches the measured one, the derived PREFETCH figure
from the same walk is trustworthy too. That check is the first thing printed,
and this script exits non-zero if it fails.
A 68000 bus cycle is 4 clocks, so a frame of C clocks holds C/4 bus slots.
"""
import sys, os, argparse, csv
sys.path.insert(0, "tools/encoder")
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import numpy as np
from dlx import DLX
import buscost as B
from buscost import V7_FRAME_PREF, V7_FRAME_DATA
BUS_CLK = 4
# --- straight-line path costs, read off tools/bench/decode.lst -------------
# (instruction words, data bus cycles). A long access is two bus cycles on the
# 68000's 16-bit bus; movem.l of N registers is 2N.
#
# dispatch move.b (a1),d0 / lsr.b / and.w #3 / beq .sk 6w, 1 read
# + subq / beq .v1 -> 8w
# + subq / bne .rw -> 10w
# V4 body $10090..$100E2 = 82 B = 41w; 4 x (1 byte read
# + movem.l 2 regs = 4 reads + 2 move.l = 4 writes) = 36
# V1 body $100E2..$10106 = 36 B = 18w; 1 byte read
# + movem.l 8 regs = 16 reads + 4 x movem.l 2 = 16 w = 33
# RAW body $10106..$10164 = 94 B = 47w; 8 x (2 byte reads
# + 1 move.l = 2 writes) = 32
# .sk tail addq.l #8,a4 1w
# BLOCK 0 has no lsr.b, so one of the four dispatches in a group is 1w cheaper.
DISPATCH_SK, DISPATCH_V1, DISPATCH_V4 = 6, 8, 10
BODY = {0: (0, 0), 1: (18, 33), 2: (41, 36), 3: (47, 32)}
DISPATCH = {0: DISPATCH_SK, 1: DISPATCH_V1, 2: DISPATCH_V4, 3: DISPATCH_V4}
SK_TAIL = 1
GROUP_HEAD = 3 # tst.b (a1) 1w + beq allskip 2w
GROUP_TAIL = 4 # addq.l #1,a1 / cmpa.l a5,a4 / bne byteloop
ALLSKIP = 9 # the whole four-block fast path, tst.b included
ROW_HEAD, ROW_TAIL = 3, 7
ap = argparse.ArgumentParser()
ap.add_argument("container", nargs="?", default="tmp/rc_fr_singe_scsi_cpufit.dlx")
ap.add_argument("--csv", default="tmp/c68k_frames.csv",
help="per-frame output of tools/bench/c68k/run.sh")
ap.add_argument("--nframes", type=int, default=None)
ap.add_argument("--kbps", type=float, default=None,
help="delivery rate in KB/s. OPTIONAL and there is no default "
"(FINDINGS 50): supply it and the AUTO-REQUEST rows are "
"added, which are the only rows whose cost depends on how "
"long the record takes to arrive (59.3).")
a = ap.parse_args()
if not os.path.exists(a.container):
sys.exit(f"missing {a.container}")
d = DLX(a.container)
meas = {}
if os.path.exists(a.csv):
for r in csv.DictReader(open(a.csv)):
meas[int(r["frame"])] = (int(r["cycles"]),
int(r["bus_reads"]) + int(r["bus_writes"]))
NF = a.nframes or (max(meas) + 1 if meas else d.nframes)
pref_t, data_t, cyc_t = [], [], []
for f in range(NF):
m = d.modes(f).reshape(d.nby, d.nbx)
pref = d.nby * (ROW_HEAD + ROW_TAIL)
data = 0
for by in range(d.nby):
row = m[by]
for gi in range(0, d.nbx, 4):
g = row[gi:gi+4]
if (g == 0).all():
pref += ALLSKIP; data += 1
continue
pref += GROUP_HEAD + GROUP_TAIL - 1 # BLOCK 0 has no lsr.b
data += 1
for b in g:
b = int(b)
pw, pd = BODY[b]
pref += DISPATCH[b] + pw + SK_TAIL
data += 1 + pd
# The span section is bus traffic too, and it is most of the frame's data
# accesses in a span-heavy container: 48 per 24-pixel chain unit. Leaving it
# out would not merely understate the total -- it would break the CHECK
# below, which is the whole licence for the prefetch figure.
sp, _ = d.spans(f)
if sp:
pref += V7_FRAME_PREF; data += V7_FRAME_DATA
for _, _, px in sp:
sp_p, sp_d = B.v7_span_split(len(px))
pref += sp_p; data += sp_d
pref_t.append(pref); data_t.append(data)
cyc_t.append(meas.get(f, (0, 0))[0])
pref_t, data_t, cyc_t = map(np.array, (pref_t, data_t, cyc_t))
print(f"{a.container}: {NF} frames, {d.nb} blocks/frame\n")
if meas:
md = np.array([meas[f][1] for f in range(NF)])
err = 100 * (data_t - md) / md
print("CHECK -- derived DATA bus cycles against the C68K harness's measurement")
print(f" measured mean {md.mean():>10,.0f} /frame")
print(f" derived mean {data_t.mean():>10,.0f} /frame "
f"error {err.mean():+.2f}% mean, {np.abs(err).max():.2f}% worst")
if np.abs(err).max() > 2.0:
sys.exit("\nFAIL: the path walk does not reproduce the measured data "
"accesses, so its prefetch figure cannot be trusted either.")
print(" the walk reproduces the measurement, so its prefetch count stands\n")
slots = cyc_t / BUS_CLK
tot = pref_t + data_t
print(f"{'':<22}{'mean':>12}{'median':>12}{'worst frame':>14}")
for label, v in (("bus slots in a frame", slots),
(" data accesses", data_t),
(" instruction prefetch", pref_t),
(" total bus cycles", tot)):
print(f"{label:<22}{v.mean():>12,.0f}{np.median(v):>12,.0f}{v.max():>14,.0f}")
occ = 100 * tot / slots
print(f"{'bus OCCUPANCY':<22}{occ.mean():>11.1f}%{np.median(occ):>11.1f}%"
f"{occ.max():>13.1f}%")
free = slots - tot
print(f"{'slots left for a DMAC':<22}{free.mean():>12,.0f}{np.median(free):>12,.0f}"
f"{free.min():>14,.0f} (worst = fewest)")
print(f"\nprefetch is {100*pref_t.sum()/tot.sum():.0f}% of the decoder's bus traffic: "
f"the data-only\nfigure the harness prints understates occupancy by about 2x.")
print(f"A DMAC painting spans at 8 clocks (2 bus cycles) per pixel could use at\n"
f"most {free.mean()/2:,.0f} pixels' worth of the mean frame's spare slots "
f"-- against {d.nb*16:,} pixels\nin a whole screen.")
# ---------------------------------------------------------------------------
# THE OTHER TWO MASTERS. Everything above is the 68000's own traffic, and it
# was the whole of this tool until session 20. The frame also has to carry the
# bitstream in off the disk and a byte of ADPCM out to $E92003 every 128 us,
# and neither has ever appeared in a bus figure -- FINDINGS 35's lesson, which
# was about the CLOCK budget, had never been applied to the BUS one.
#
# The DMAC does not overlap with the CPU (buscost.DMA_OVERLAPS = False): the
# 68000 has no cache and a two-word prefetch queue that empties at once, so a
# stolen bus cycle is a stopped CPU. The three demands therefore ADD.
#
# Audio's per-byte figure is SETTLED, not bracketed by taste:
# tools/analysis/21_iplrom_dmac.py reads the IPL ROM's own HD63450 setup and
# finds channel 3 dual-address, 8-bit port, cycle steal without hold, external
# request -- one arbitration per byte, no burst. Video's is NOT settled: it is
# ROADMAP B3 / FINDINGS 42.4-42.6's W, so it is swept rather than picked.
print("\n" + "=" * 72)
print("THE OTHER TWO MASTERS -- what the DMAC takes out of the same frame\n")
FPS = d.fps
CPUHZ = 10e6 # stock X68000, MAME 0.277 x68k.cpp:1133
FRAME_CLK = CPUHZ / FPS
# WHAT THE TRANSPORT MOVES, which is the PADDED record and not the payload.
# Under DLX2/3/4 the pad was 0..3 B and the distinction was noise; under DLX5 it
# is 0..511 B, and charging the payload would price the channel for bytes it
# does not carry while the disc carries them anyway. A budget that debits only
# the bytes a frame CONTAINS is the same incomplete accounting this project has
# been caught by before -- the pad is delivered, so the pad is charged.
vid_bpf = sum(d.record_lengths()[:NF]) / NF
aud_bpf = B.ADPCM_BYTES_PER_S / FPS
a_lo = aud_bpf * B.ADPCM_CLK_BYTE_BEST
a_hi = aud_bpf * B.ADPCM_CLK_BYTE_WORST
cpu_clk = cyc_t.mean() if cyc_t.any() else float("nan")
print(f"frame period at {FPS:g} fps on a 10 MHz 68000: {FRAME_CLK:,.0f} clocks")
if cyc_t.any():
print(f" decoder, MEASURED (C68K) {cpu_clk:>10,.0f} clk "
f"{100*cpu_clk/FRAME_CLK:5.1f}% worst frame "
f"{100*cyc_t.max()/FRAME_CLK:.1f}%")
print(f" audio DMA, {aud_bpf:,.1f} B/frame {a_lo:>10,.0f} clk "
f"{100*a_lo/FRAME_CLK:5.2f}% .. {a_hi:,.0f} clk "
f"({100*a_hi/FRAME_CLK:.2f}%)")
print(f" {B.ADPCM_CLK_BYTE_BEST}..{B.ADPCM_CLK_BYTE_WORST} clk/byte, "
f"from the ROM's own DCR/OCR (21_iplrom_dmac.py). NOT a guess, and\n"
f" not the disk's rate: audio arbitrates for the bus once per byte "
f"and cannot burst.")
print(f"\n video DMA, {vid_bpf:,.0f} B/frame, swept over W -- ROADMAP B3 is "
f"still open:")
print(f" {'W (clk/byte)':<16}{'clk/frame':>12}{'% of frame':>12} "
f"{'CPU+audio+video':>18}")
for W, note in ((5.0, "single address, bus held (11_cpu_budget.py default)"),
(8.0, "FINDINGS 5's long-standing per-word ESTIMATE"),
(9.0, "DUAL address, bus held -- and the FLOOR of every "
"dual-address\n "
" configuration, auto-request included (59.3)"),
(12.0, "single address, arbitrated per byte"),
(16.0, "what the ROM programs for SASI (best case)"),
(19.0, "what the ROM programs for SASI (worst case)"),
(87.28, "PIO -- MEASURED, FINDINGS 58.2, the CPU doing it itself")):
v = vid_bpf * W
tot_clk = (cpu_clk if cyc_t.any() else 0) + a_lo + v
print(f" {W:<16.6g}{v:>12,.0f}{100*v/FRAME_CLK:>11.1f}% "
f"{100*tot_clk/FRAME_CLK:>17.1f}% {note}")
print(f"\n (the last column adds the MEASURED mean decode and the BEST-CASE "
f"audio, so it is\n the optimistic end of every row. 100% is the frame "
f"deadline at {FPS:g} fps.)")
print(f"""
Audio is {100*a_lo/FRAME_CLK:.2f}%..{100*a_hi/FRAME_CLK:.2f}% of the frame and video is {vid_bpf*5/FRAME_CLK*100:.0f}%..{vid_bpf*19/FRAME_CLK*100:.0f}% over the ladder, against
{vid_bpf*87.28/FRAME_CLK*100:.0f}% for the PIO transport FINDINGS 58.2 measured. The unpriced audio stream
was never the risk P6 called it -- ON THE BUS.""")
# --- HEADROOM, AND THE FLOOR UNDER THE LADDER -----------------------------
# Added session 27. The sweep above answers "what does each W cost"; it never
# answered "what can this frame afford", and the two are not the same question.
# FINDINGS 59.2 is why it matters now: with no external request line the only
# configurations that can be run are dual-address, and a dual-address byte has
# a FLOOR -- one 4-clock read of the device plus one 5-clock write to memory,
# buscost.DMA_DUAL_BYTE_CLK. No GCR share and no delivery rate goes under it.
print("\n" + "=" * 72)
print("WHAT THE FRAME CAN AFFORD, AND THE FLOOR UNDER THE LADDER\n")
head_clk = FRAME_CLK - (cpu_clk if cyc_t.any() else 0) - a_lo
head_wb = head_clk / vid_bpf
print(f" headroom after the MEASURED decode and best-case audio: "
f"{head_clk:,.0f} clk = {100*head_clk/FRAME_CLK:.1f}%")
print(f" at {vid_bpf:,.0f} B a frame that is {head_wb:.2f} CLOCKS PER BYTE, and "
f"that is the number\n a transport has to come in under.\n")
floor = B.DMA_DUAL_BYTE_CLK
print(f" dual-address floor {floor} clk/B ({B.DMA_READ_CLK} read of the "
f"device + {B.DMA_WRITE_CLK} write to memory, Fig 4-25)")
print(f" single-address held {B.DMA_DISK_CLK_WORD_HELD} clk/B (one memory "
f"write; needs the device to ACK, i.e. a REQUEST LINE)")
if head_wb < floor:
print(f"""
SO DUAL ADDRESS DOES NOT FIT THIS CONTAINER AT {FPS:g} fps -- not at any
delivery rate and not at any GCR share, because {head_wb:.2f} < {floor}. A share
decides whether the channel sits AT the floor or above it; it cannot
go under it. That is FINDINGS 59.2's three bounds arriving in the
budget: the configurations this machine can run are exactly the ones
the frame cannot afford, and the one it can afford -- single address,
{B.DMA_DISK_CLK_WORD_HELD} clk/B, {100*vid_bpf*B.DMA_DISK_CLK_WORD_HELD/FRAME_CLK:.1f}% -- needs the request line ROADMAP B3 asks about.""")
for w, what in ((floor, "dual address"), (B.DMA_DISK_CLK_WORD_HELD, "single address")):
tgt = head_clk / w
print(f"\n TO FIT AT {w} clk/B ({what}) THIS CONTAINER MUST COME DOWN TO")
print(f" {tgt:,.0f} B a frame = {tgt*FPS/1024:,.0f} KB/s of payload "
f"(it is {vid_bpf:,.0f} B, {vid_bpf*FPS/1024:,.0f} KB/s)"
+ (" -- already met" if vid_bpf <= tgt else
f" -- {100*(vid_bpf/tgt-1):.0f}% too big"))
print(f"""
AND THAT IS THE PESSIMISTIC READING OF THE ENCODER LEVER: a lighter
container also DECODES cheaper, so the decode term above falls with
the byte term. The figure to re-derive it against is this tool run on
the lighter container -- with its OWN C68K measurement, because the
cross-check at the top is what licenses every number below it.""")
else:
print(f"\n The frame affords {head_wb:.2f} clk/B, which is at or above the "
f"{floor} clk/B dual-address floor.")
# --- AUTO-REQUEST, and only when a rate is supplied ------------------------
# These are the rows 59.3 added and they are the only ones here whose cost is
# not a property of the transfer: an auto-requested channel spends its share of
# the bus whether or not a byte is there, so what a record costs depends on how
# long it takes to ARRIVE. No default rate, deliberately (FINDINGS 50).
if a.kbps:
RATE = a.kbps * 1024.0
wire_clk = vid_bpf / RATE * CPUHZ
cap = 0.5 * CPUHZ / B.DMA_DUAL_BYTE_CLK # the 50% share's ceiling
print("\n" + "=" * 72)
print(f"AUTO-REQUEST AT {a.kbps:g} KB/s -- charged by TIME, not by byte "
f"(59.3)\n")
print(f" the record takes {wire_clk:,.0f} clk to arrive = "
f"{100*wire_clk/FRAME_CLK:.1f}% of a frame\n")
print(f" {'configuration':<34}{'clk/B':>8}{'% of frame':>12}"
f"{'CPU+audio+video':>18}")
rows = [("REQG 01, max rate (100% of the bus)", 1.0, None)]
for br, share in ((0, .5), (1, .25), (2, .125), (3, .0625)):
rows.append((f"REQG 00, LRAR BR={br:02b}, {share*100:g}% share", share,
share * CPUHZ / B.DMA_DUAL_BYTE_CLK))
for name, share, sustains in rows:
v = share * wire_clk
tot = (cpu_clk if cyc_t.any() else 0) + a_lo + v
flag = ""
if sustains is not None and sustains < RATE:
flag = f" cannot carry the rate ({sustains/1024:.0f} KB/s max)"
print(f" {name:<34}{v/vid_bpf:>8.2f}{100*v/FRAME_CLK:>11.1f}%"
f"{100*tot/FRAME_CLK:>17.1f}%{flag}")
print(f"""
A FASTER DISC MAKES AUTO-REQUEST CHEAPER, which no W does -- the share is
spent over a shorter wire time. But it cannot reach the floor: a 50% share
tops out at {cap/1024:,.0f} KB/s, above which the CHANNEL is the bottleneck and the
delivered rate falls back to it. At that ceiling the cost is exactly the
{B.DMA_DUAL_BYTE_CLK} clk/B floor, which is where the section above already put it.""")
tools/analysis/16_span_roundtrip.py
+129
View File
@@ -0,0 +1,129 @@
#!/usr/bin/env python3
"""GATE for the DLX3 span container: does the reference decoder reproduce the
encoder's own reconstruction, from the emitted bytes?
python3 tools/analysis/16_span_roundtrip.py [frames_dir] --kbps <KB/s>
Exits non-zero if any frame differs by a single pixel.
WHY THIS EXISTS SEPARATELY FROM 09. `09_ratectl_drift.py` replays SKIP
semantics in Python against the mode maps the encoder returned; it never reads
a container. A span breaks exactly that shortcut: a spanned block reads SKIP
in the mode header and is painted by the span section instead, so a replay that
knows only about mode maps reports drift where there is none, and -- far worse
-- a container whose span section is malformed would still pass, because 09
never parses one. This gate closes that: encode, WRITE THE CONTAINER, read it
back with tools/encoder/dlx.py (the byte-for-byte reference decoder the 68000
is checked against), and compare to what ratectl recorded.
It also has to prove it tested something. A round-trip over a container with
no spans in it is green by vacuity, which is the failure mode FINDINGS 40.6
named for the snapshot count: a gate must take its expected work from the
generated artefact, not from an assumption. So the thresholds below are
asserted, not printed.
The `--kbps` default is the BUS rate, not the `scsi` profile's 280: spans are
bought with bytes, and 14_dmac_chain.py scores them against the delivery pipe.
At the profile rate the lam search has already spent the allowance and there is
nothing left to buy a span with -- which is a real finding about the encoder
(FINDINGS 41.2), not a reason for the gate to test nothing.
"""
import argparse, os, pickle, sys, time
sys.path.insert(0, "tools/encoder")
import numpy as np
import vq_hybrid as H, ratectl as RC, encode as E
from dlx import DLX
ap = argparse.ArgumentParser()
ap.add_argument("frames_dir", nargs="?", default="tmp/fr_singe")
ap.add_argument("--kbps", type=float, required=True,
help="REQUIRED. There is no default: the delivery rate is a property of the medium and this project has never measured it. FINDINGS 42.1 -- the figure this tool used to default to was a user-supplied '4 Mbps' with no provenance, was a tenth of SCSI-1's asynchronous rating, and was never a bus measurement at all. A default let every table in FINDINGS 30-49 be scored against it without anyone restating it. Pass one explicitly.")
ap.add_argument("--out", default="tmp/s12_roundtrip")
ap.add_argument("--cache", default=None)
a = ap.parse_args()
cache = a.cache or f"tmp/model_{os.path.basename(a.frames_dir.rstrip('/'))}.pkl"
# The cache is keyed on the frames directory ALONE, which was fine while
# H.build had no options and became a trap the moment it did: session 28's
# reserved black entry (23.4) changes the palette, the codebooks and every
# index in the model, and a pickle from before it would have let this gate
# round-trip a container the shipping encoder no longer emits -- green, and
# testing the wrong artefact. So the build parameters are stored WITH the
# model and a mismatch rebuilds.
SIG = dict(k1=256, k4=256, iters=16, reserve_black=True)
m = None
if os.path.exists(cache):
m = pickle.load(open(cache, "rb"))
if m.get("sig") != SIG:
print(f"{cache}: built with {m.get('sig')}, wanted {SIG} -- rebuilding")
m = None
else:
print(f"model from {cache}")
if m is None:
t = time.time()
m = H.build(a.frames_dir, **SIG)
m["sig"] = SIG
pickle.dump(m, open(cache, "wb"))
print(f"built model in {time.time()-t:.0f} s -> {cache}")
bad = 0
for span_mode in ("need", "all"):
print(f"\n=== spans={span_mode}, {a.kbps:g} KB/s ===")
m.pop("_sym", None)
enc = RC.encode_rate_controlled(m, target_kbps=a.kbps, lam_lo=1.0,
cycle_budget=RC.FRAME_CYCLES,
span_mode=span_mode)
recs = E.build_records(m, enc, span_mode)
path = f"{a.out}_{span_mode}.dlx"
total, vid, _ = E.write_container(path, m, recs, 12, m["k1"], m["k4"],
span_mode)
nsp = sum(len(x) for x in enc["spans"])
nfr = sum(1 for x in enc["spans"] if x)
px = sum(len(p) for x in enc["spans"] for _, _, p in x)
print(f"{path}: {total:,} B, {len(recs)} frames, "
f"{nsp:,} spans on {nfr} frames, {px:,} pixels painted by one "
f"({100*px/(len(recs)*m['H']*m['W']):.1f}% of all pixels)")
d = DLX(path)
if not d.has_spans:
print(f"FAIL: container is DLX{d.version}, which has no span section")
bad += 1; continue
# DLX4 adds the record index and DLX() cross-checks it against its own walk
# of the frame stream, so simply constructing it above has already gated
# that. Said out loud here because it is easy to read this as version drift.
if d.has_index:
print(f" DLX{d.version}: record index agrees with the frame stream on all "
f"{d.nframes} records ({2*d.nframes:,} B of scene header)")
# The decoder's own walk of the span section must land exactly where the
# block payload starts, and blocks() already raises if the payload does not
# consume the record -- so this reads the spans back through the same code
# path the 68000 is modelled on rather than trusting the writer.
got = d.decode_all()
diff = np.array([(g != r).sum() for g, r in zip(got, enc["recon"])])
print(f"pixels differing from the encoder's reconstruction: "
f"{diff.sum()} total, worst frame {diff.max()}, "
f"frames with any: {int((diff>0).sum())}/{len(diff)}")
if diff.sum():
f = int(np.argmax(diff))
ys, xs = np.where(got[f] != enc["recon"][f])
print(f"FAIL: frame {f} differs at {diff[f]} px, first (x={xs[0]}, "
f"y={ys[0]}), block (bx={xs[0]//4}, by={ys[0]//4}), "
f"mode there = {d.modes(f)[(ys[0]//4)*d.nbx + xs[0]//4]}")
bad += 1
# A green round-trip over a container with no spans in it proves nothing.
if span_mode == "all":
if nsp < 1000:
print(f"FAIL: only {nsp} spans emitted -- this gate did not "
f"exercise the span path"); bad += 1
if not (px and max(len(x) for x in enc["spans"]) > 50):
print(f"FAIL: no frame carries a substantial span table"); bad += 1
print()
if bad:
print(f"FAILED: {bad} check(s)")
sys.exit(1)
print("OK the span container round-trips: the reference decoder rebuilds "
"the\n encoder's reconstruction exactly, from the emitted bytes.")
tools/analysis/17_span_delivered.py
+112
View File
@@ -0,0 +1,112 @@
#!/usr/bin/env python3
"""What do the spans the ENCODER actually emitted cost, and what do they buy?
python3 tools/analysis/17_span_delivered.py a.dlx [b.dlx ...] --bus <KB/s>
Every span figure before this one -- FINDINGS 29 through 40, and
tools/analysis/12 and 14 -- was scored by SIMULATING span selection over mode
maps that were chosen without spans available. FINDINGS 39.3 flagged that as a
lower bound on what a span-aware encoder would find, and docs/STATUS.md's item 2
asks for the figures to be re-run "against a container the encoder actually
emits with spans in it". This is that script: it reads the span section out of
a DLX3 container and prices exactly those spans, with no selection model at all.
THE MODEL IS 14_dmac_chain.py's, deliberately unchanged, so the columns are
comparable:
frame clocks = block decode + span painting + disk DMA
additive, because a 68000 has no cache and a two-word prefetch queue and stalls
the moment another master takes the bus (FINDINGS 38.3). Block cost is
vq_hybrid.cycles(), which reads a spanned block as SKIP -- correct, because the
span section is what paints it, and its cost is the second term.
The span term is the MEASURED v7 fit (FINDINGS 40), and as of session 12 that
fit is confirmed inside src/player/decode.s itself rather than only in
tools/bench/blit.s: the synthetic all-SPAN anchors of tools/bench/prep_dlx.py
reproduce it to 0.23% on both emulators (FINDINGS 41.3).
"""
import argparse, os, sys
sys.path.insert(0, "tools/encoder")
sys.path.insert(0, "tools/analysis")
import numpy as np
import vq_hybrid as H
import spans as SP
import buscost as B
from dlx import DLX
FRAME_CYC = 833333.0
AUDIO_KBPS = 7.8
ap = argparse.ArgumentParser()
ap.add_argument("containers", nargs="+")
ap.add_argument("--bus", type=float, required=True,
help="REQUIRED. There is no default: the delivery rate is a property of the medium and this project has never measured it. FINDINGS 42.1 -- the figure this tool used to default to was a user-supplied '4 Mbps' with no provenance, was a tenth of SCSI-1's asynchronous rating, and was never a bus measurement at all. A default let every table in FINDINGS 30-49 be scored against it without anyone restating it. Pass one explicitly.")
ap.add_argument("--fps", type=float, default=12.0)
ap.add_argument("--disk-clk-byte", type=float, default=5.0,
help="clocks the SCSI DMA steals per BYTE delivered. The SPC is "
"an 8-bit port, so the DMAC pays per byte, not per word "
"(FINDINGS 43). 5, the default, is the OPTIMISTIC end and "
"what ratectl encodes against: single-address, bus held, no "
"drive wait (Fig 4-25 sheet 2). 9 is dual-address, which is "
"what MAME models and what applies if the board does not "
"drive DACK. Score both.")
ap.add_argument("--disk-clk-word", type=float, default=None,
help="DEPRECATED denominator of FINDINGS 39.7/42, kept so the "
"old tables reproduce: sets --disk-clk-byte to half this")
a = ap.parse_args()
if a.disk_clk_word is not None:
a.disk_clk_byte = a.disk_clk_word / 2.0
def score(path):
d = DLX(path)
rows = []
for f in range(d.nframes):
mode = d.modes(f)
sp, _ = d.spans(f)
_, n = d.frames[f]
blk = H.cycles(mode)
spc = sum(SP.clocks(len(p)) for _, _, p in sp)
disk = n * a.disk_clk_byte
rows.append((blk, spc, disk, n, len(sp),
sum(len(p) for _, _, p in sp)))
return d, np.array(rows).T
print(f"{'container':<34}{'KB/s':>8}{'spans':>9}{'span px':>9}"
f"{'median':>9}{'worst':>9}{'over':>9}")
print(f"{'':<34}{'':>8}{'/frame':>9}{'%':>9}"
f"{'% frame':>9}{'% frame':>9}{'budget':>9}")
for path in a.containers:
if not os.path.exists(path):
print(f"{path:<34} missing"); continue
d, r = score(path)
blk, spc, disk, byt, nsp, spx = r
tot = blk + spc + disk
kbps = byt.mean() * a.fps / 1024 + AUDIO_KBPS
print(f"{os.path.basename(path):<34}{kbps:>8.1f}{nsp.mean():>9.0f}"
f"{100*spx.mean()/(d.W*d.H):>9.1f}"
f"{100*np.median(tot)/FRAME_CYC:>9.1f}"
f"{100*tot.max()/FRAME_CYC:>9.1f}"
f"{int((tot > FRAME_CYC).sum()):>6}/{d.nframes:<3}")
print(f"\n ADDITIVE: frame = block decode + span painting + disk DMA, the model"
f"\n of 14_dmac_chain.py. Disk debited at {a.disk_clk_byte:g} clocks/byte "
f"over the\n container's own byte count; CPU budget {FRAME_CYC:,.0f} "
f"clocks at {a.fps:g} fps.")
# The decomposition is the point: a span moves work out of the block loop and
# into the span section, and it pays for it in bytes -- which the disk term
# then charges back. A design that only counted the CPU would show a win that
# the I/O it created takes away again (docs/FINDINGS.md 33).
print(f"\nWHERE EACH FRAME'S CLOCKS GO, mean over the container")
print(f" {'container':<34}{'blocks':>12}{'spans':>12}{'disk':>12}{'total':>12}")
for path in a.containers:
if not os.path.exists(path):
continue
d, r = score(path)
blk, spc, disk = r[0], r[1], r[2]
print(f" {os.path.basename(path):<34}{blk.mean():>12,.0f}{spc.mean():>12,.0f}"
f"{disk.mean():>12,.0f}{(blk+spc+disk).mean():>12,.0f}")
tools/analysis/18_text_plane_16col.py
+88
View File
@@ -0,0 +1,88 @@
#!/usr/bin/env python3
"""What 256 -> 16 colours actually costs, on real frames.
python3 tools/analysis/18_text_plane_16col.py [frames_dir]
FINDINGS 46.3 opened a lead and could not price it: the X68000 text plane is
4bpp planar -- 0.5 bytes/pixel against the graphics planes' 2.0 -- so a LITERAL
uncompressed 16-colour frame is 288.0 KB/s against the shipping compressed
256-colour container's 496.7 KB/s. 42% cheaper on the wire, with no decoder.
The whole lead turns on one number nobody had computed: the quality cost of 16
colours. This computes it, and it is deliberately generous to the 16-colour
side on every axis where the hardware allows it:
* PER-FRAME palettes are legitimate here. The text palette is 16 entries and
reloading it is 16 words a frame -- nothing, against a 833,333-clock budget.
The 256-colour path cannot do this: its palette is shared scene-wide
(vq.scene_palette) because the codec's codebooks are indices INTO it.
* DITHERING is free here, and only here. The tree does not dither (vq.py:32,
"cel art is flat") because dither destroys the inter-frame coherence SKIP
blocks and v7 spans are built on. A literal frame has no codec to wreck, so
Floyd-Steinberg is available to this path at zero runtime cost.
Both are measured, so the comparison cannot be accused of hobbling the option it
is testing. Reported against the 256-colour scene-palette ceiling (the tree's
existing "palette ceiling" figure) and against the shipping container's PSNR.
"""
import sys, os
sys.path.insert(0, "tools/encoder")
import numpy as np
from PIL import Image
import vq as VQ
FRAMES = sys.argv[1] if len(sys.argv) > 1 else "tmp/fr_singe"
SHIPPED_PSNR = 29.19 # docs/STATUS.md, --spans all, c=5, 496.7 KB/s
rgb = VQ.load_frames(FRAMES)
H, W = rgb[0].shape[:2]
n = len(rgb)
print(f"{FRAMES}: {n} frames, {W}x{H}")
print()
def recon_scene(colors, dither):
"""One palette for the whole scene -- what the 256 path is forced to do."""
d = Image.FLOYDSTEINBERG if dither else Image.NONE
samp = np.concatenate([r.reshape(-1, 3) for r in rgb[::3]])
ref = Image.fromarray(samp.reshape(-1, 1, 3)).quantize(
colors=colors, method=Image.MEDIANCUT, dither=Image.NONE)
pal = np.array(ref.getpalette()[:colors * 3], np.uint8).reshape(-1, 3)
return [pal[np.asarray(Image.fromarray(r).quantize(palette=ref, dither=d),
np.uint8)] for r in rgb]
def recon_perframe(colors, dither):
"""A fresh palette every frame -- what the text plane can afford."""
d = Image.FLOYDSTEINBERG if dither else Image.NONE
out = []
for r in rgb:
q = Image.fromarray(r).quantize(colors=colors, method=Image.MEDIANCUT,
dither=d)
pal = np.array(q.getpalette()[:colors * 3], np.uint8).reshape(-1, 3)
out.append(pal[np.asarray(q, np.uint8)])
return out
def report(name, recon):
per = np.array([VQ.psnr(a, b) for a, b in zip(rgb, recon)])
print(f" {name:<42s} {per.mean():6.2f} dB "
f"(min {per.min():5.2f} max {per.max():5.2f})")
return per.mean()
print("PSNR vs the 24-bit source, mean over frames:")
c256 = report("256 colours, scene palette [the tree's]", recon_scene(256, False))
report("256 colours, per-frame palette", recon_perframe(256, False))
print()
s16 = report("16 colours, scene palette", recon_scene(16, False))
p16 = report("16 colours, per-frame palette", recon_perframe(16, False))
p16d = report("16 colours, per-frame + FS dither", recon_perframe(16, True))
print()
print(f" the 16-colour ceiling is the best of those: {max(s16, p16, p16d):.2f} dB")
print(f" cost of 256 -> 16, at each side's best: "
f"{c256 - max(s16, p16, p16d):.2f} dB")
print()
print(f" for scale, the shipping container delivers {SHIPPED_PSNR:.2f} dB "
f"at 496.7 KB/s")
print(f" a 16-colour literal would deliver "
f"{max(s16, p16, p16d):.2f} dB at 288.0 KB/s")
delta = max(s16, p16, p16d) - SHIPPED_PSNR
print(f" so the text-plane path is {abs(delta):.2f} dB "
f"{'BETTER' if delta > 0 else 'WORSE'} at 58% of the bitrate")
tools/analysis/19_ring_stream.py
+275
View File
@@ -0,0 +1,275 @@
"""Ring-buffer streaming simulation, against the CONTIGUITY constraint (STATUS 3/4).
09_buffer_sim.py asked one question -- does cumulative supply ever fall behind
cumulative demand -- and answered it in BYTES. FINDINGS 21 got "zero required
prefill" out of it at 110 and 280 KB/s. That test is necessary and not
sufficient, and the missing half is the whole of STATUS item 3:
src/player/decode.s reads a frame record with a MONOTONICALLY INCREASING a0
and no bounds check anywhere. `move.l (a0)+,d0` for the length, `lea
MODEB(a0),a0` for the span section, eleven unrolled `movem.l (a0)+` chains,
`move.b (a0)+` per block index. Nothing in it can survive an address that
wraps mid-record. So the buffer does not merely need ENOUGH BYTES resident
by the deadline -- it needs the WHOLE NEXT RECORD resident and CONTIGUOUS.
Having enough bytes and having them contiguous are different conditions, and a
byte-counting simulation cannot tell them apart. This one models the ring's
addresses, not just its occupancy.
THREE WRAP POLICIES, and the point of the tool is that they are not equivalent:
split the writer wraps mid-record; the reader cannot. Requires a SHADOW of
the ring's first MAXREC bytes mirrored past its end, so any record
start can be read linearly for MAXREC bytes. Every byte landing in
that first MAXREC is written twice. Costs 68000 CLOCKS, forever, at a
rate set by MAXREC/ring -- and those clocks come out of the same
budget the decoder is already spending 77.0% of (FINDINGS 45).
aligned the writer refuses to start a record it cannot finish before the end
of the ring; it leaves a hole and restarts at 0. Costs RAM (the mean
hole) and nothing else -- no copy, no per-byte work. Needs a frame
INDEX so the fill side knows record boundaries, which a branching
laserdisc game needs anyway to seek to a branch point.
none the decoder handles the wrap itself. Priced here only to show what it
would cost: a bounds test in the block loop is inside the sequence
FINDINGS 30.4/40 fitted, so it does not cost a branch -- it costs
every span and per-block constant in the tree being re-measured.
Not simulated; see the note printed at the end.
DEADLINE MODEL, and it is the conservative one: record i must be wholly
resident when frame i's decode BEGINS. The decoder in fact reads a record
progressively over ~77% of a frame time, so a byte arriving mid-frame would in
practice be in time -- but that is a race between the DMAC's fill address and
a0, and this tool refuses to certify a design on a race it cannot see.
Fill is quantised to 512-byte SCSI blocks: a partial sector is not resident.
python3 tools/analysis/19_ring_stream.py [container ...] --kbps R [--ring KB]
`--kbps` is REQUIRED and has no default -- see the argument's help text.
"""
import sys, os, argparse
sys.path.insert(0, "tools/encoder")
import numpy as np
from dlx import DLX
import ratectl as RC
SECTOR = 512
# 5 clocks/byte for a 68000 `move.l (a0)+,(a1)+` copy: 20 clocks moves 4 bytes
# on a 16-bit bus (2 read + 2 write bus cycles at 4 clocks, plus the fetch it
# shares with the loop). Deliberately the OPTIMISTIC figure -- a movem-shaped
# copy is what the shadow would really use, and it is the same 5.0.
COPY_CLK_PER_BYTE = 5.0
CPUHZ = 10_000_000
def records(path):
"""Padded record sizes, exactly as the 68000 walks them.
prep_dlx.py rounds each record START up to 4 (FINDINGS 28.3), so the bytes
the ring must hold per frame are the padded ones, not the payload.
"""
d = DLX(path)
rec = np.array(d.record_lengths(), np.int64)
return d, rec
def simulate(rec, fill_per_frame, ring, policy, maxrec):
"""Address-level ring simulation. Returns a dict of results.
The ring is modelled as a write cursor and a read cursor over `ring` bytes.
Supply arrives at `fill_per_frame` bytes per frame time, sector-quantised.
Record i is due at the start of frame i.
"""
n = len(rec)
resident = 0.0 # bytes fully arrived and not yet consumed
carry = 0.0 # sub-sector remainder of the fill
wcur = 0 # write cursor within the ring
holes = [] # bytes wasted per wrap, `aligned` policy
shadow_bytes = 0 # bytes double-written, `split` policy
occ = []
prefill = 0.0
late = []
free = ring
# Required prefill is solved rather than searched: run once with an infinite
# head start to find the worst deficit, exactly as 09_buffer_sim does, then
# assert the ring can hold it.
deficit = np.maximum.accumulate(np.cumsum(rec - fill_per_frame))
prefill = float(max(0.0, deficit.max()))
for i, r in enumerate(rec):
# --- supply for this frame time, sector-quantised
avail = carry + fill_per_frame
sectors = int(avail // SECTOR)
got = sectors * SECTOR
carry = avail - got
# --- placement: does this frame's arriving data cross the ring end?
if policy == "aligned":
# The writer will not start a record it cannot finish. Charge the
# hole when the NEXT record would not fit in the tail.
if wcur + r > ring:
holes.append(ring - wcur)
wcur = 0
wcur += r
else: # split
end = wcur + r
if end > ring:
wcur = end - ring
# every byte that landed in the first MAXREC of the ring is
# mirrored into the shadow
shadow_bytes += min(wcur, maxrec)
else:
wcur = end
if wcur <= maxrec:
shadow_bytes += r
elif wcur - r < maxrec:
shadow_bytes += maxrec - (wcur - r)
resident += got
if resident + 1e-9 < r:
late.append((i, float(r - resident)))
resident -= r
occ.append(resident)
hole_mean = float(np.mean(holes)) if holes else 0.0
usable = ring - hole_mean if policy == "aligned" else ring
copy_clk = shadow_bytes * COPY_CLK_PER_BYTE / max(1, n)
return dict(prefill=prefill, late=late, occ=np.array(occ),
holes=holes, hole_mean=hole_mean, usable=usable,
shadow_bytes=shadow_bytes, copy_clk_per_frame=copy_clk,
wraps=len(holes) if policy == "aligned" else None)
def main():
ap = argparse.ArgumentParser()
ap.add_argument("containers", nargs="*",
default=["tmp/s14_d5_all1500.dlx",
"tmp/rc_fr_singe_scsi_span.dlx"])
ap.add_argument("--kbps", type=float, required=True,
help="delivered pipe, KB/s. REQUIRED, and deliberately has "
"no default: the delivery rate is a property of the "
"medium and this project has never measured it. The "
"figure that used to sit here was a user-supplied "
"'4 Mbps' with no provenance and was never a bus "
"measurement (FINDINGS 42.1); leaving it as a default "
"let table after table be scored against it without "
"anyone restating what it was.")
ap.add_argument("--ring", type=float, default=256.0,
help="ring size in KB (default 256, FINDINGS 21's sizing)")
a = ap.parse_args()
FPS = 12
print(f"ring {a.ring:.0f} KB sector {SECTOR} B "
f"audio {RC.AUDIO_KBPS} KB/s debited from the pipe\n")
for path in a.containers:
if not os.path.exists(path):
print(f"{path}: MISSING -- skipped\n"); continue
d, rec = records(path)
maxrec = int(rec.max())
ring = int(a.ring * 1024)
wire = rec.mean() * FPS / 1024 + RC.AUDIO_KBPS
print(f"=== {path}")
print(f" {d.nframes} frames @ {d.fps}fps, record bytes "
f"min {rec.min():,} median {int(np.median(rec)):,} max {maxrec:,}")
print(f" wire demand {wire:.1f} KB/s "
f"(video {rec.mean()*FPS/1024:.1f} + audio {RC.AUDIO_KBPS}), "
f"including the u32 length and the 4-byte record pad")
# A required prefill is only a startup cost if the window's MEAN demand
# is under the pipe. If the mean is over, the deficit grows for as long
# as the scene runs and the prefill this window reports is just how far
# it got in 120 frames -- no ring size fixes that, and quoting a KB
# figure for it would be the most flattering possible way to state a
# sustained overrun. FINDINGS 21's "zero prefill" never had to make
# this distinction because it ran far under the pipe it assumed.
if wire > a.kbps:
over = wire - a.kbps
print(f" !! SUSTAINED OVERRUN at the {a.kbps:.0f} KB/s pipe: "
f"demand exceeds supply by {over:.1f} KB/s on the MEAN, not "
f"on a burst.")
print(f" The deficit grows {over*1024/FPS:,.0f} B per frame "
f"for as long as the scene runs -- {over*1024*120/FPS/1024:.0f} "
f"KB over this 120-frame window, {over*60:.0f} KB per minute "
f"of play. Prefill below is where it got in 120 frames, NOT a "
f"startup cost that fixes it.")
if maxrec > ring:
print(f" !! MAXREC {maxrec:,} > ring {ring:,}: no policy works. "
f"decode.s needs one whole record contiguous.\n")
continue
# --- the requirement on the medium, which is the useful output, and
# the reason this tool takes no default rate. There is no measured
# pipe figure to score against (42.1), and the intent is to measure
# a BlueSCSI directly -- so the tool reports the THRESHOLD to
# measure against. The sweep is anchored to the container's own
# wire demand rather than to a list of fixed rates, so it stays
# meaningful for any container and privileges no constant.
print(f" {'pipe KB/s':>10} {'vs wire':>8} {'prefill KB':>11} "
f"{'records':>8} {'seek slack':>11}")
for mult in (0.90, 0.95, 1.00, 1.02, 1.05, 1.10, 1.25, 1.50, 2.00):
kbps = wire * mult
fill = (kbps - RC.AUDIO_KBPS) * 1024 / FPS
r = simulate(rec, fill, ring, "aligned", maxrec)
pf = r["prefill"]
# Branch-point seek slack, STATICALLY: with the ring FULL, how many
# frame times can the fill be zero before the next record is not
# resident? It is an upper bound and it assumes the premise that
# FINDINGS 51.3 took apart -- the ring is NOT full at a branch
# point, it is empty, and refilling it takes seconds of play. For
# the measured figure use tools/analysis/20_seek_slack.py, or the
# rig itself (tools/bench/pace_run.sh). Kept here as the ceiling
# this container's record sizes allow, which is what the rest of
# this row is about.
slack = (r["usable"] - maxrec) / rec.mean()
flag = ""
if pf + maxrec > r["usable"]:
flag = " <- does not fit the ring"
print(f" {kbps:>10.1f} {mult:>7.2f}x {pf/1024:>11.1f} "
f"{pf/rec.mean():>8.2f} {slack:>8.1f} fr{flag}")
# smallest pipe needing zero prefill, to 0.1 KB/s
lo, hi = wire, wire + 400
for _ in range(40):
mid = (lo + hi) / 2
f = (mid - RC.AUDIO_KBPS) * 1024 / FPS
if simulate(rec, f, ring, "aligned", maxrec)["prefill"] > 0:
lo = mid
else:
hi = mid
print(f" ZERO-PREFILL PIPE: {hi:.1f} KB/s "
f"({hi - wire:+.1f} KB/s over the wire demand, "
f"{100*hi/wire - 100:+.1f}%)")
print(f" ^ this is the number to measure a medium against. It is a "
f"REQUIREMENT, not a verdict.")
# --- the policy trade, at the default pipe
fill = (a.kbps - RC.AUDIO_KBPS) * 1024 / FPS
print(f" wrap policy, at pipe {a.kbps:.0f} KB/s:")
for policy in ("aligned", "split"):
r = simulate(rec, fill, ring, policy, maxrec)
if policy == "aligned":
print(f" aligned wraps {r['wraps']:3} mean hole "
f"{r['hole_mean']/1024:6.1f} KB usable ring "
f"{r['usable']/1024:6.1f} KB "
f"({100*r['usable']/ring:.1f}%) CPU cost 0")
else:
pct = 100 * r["copy_clk_per_frame"] / (CPUHZ / FPS)
print(f" split shadow {r['shadow_bytes']/1024:8.1f} KB "
f"= {r['copy_clk_per_frame']:8.0f} clk/frame = "
f"{pct:.2f}% of the frame budget, forever RAM cost 0")
print()
print("The `none` policy -- decoder wraps its own reads -- is not simulated.")
print("It has no RAM or copy cost and it is still the expensive one: the")
print("bounds test lands inside the exact instruction sequences FINDINGS")
print("30.4 and 40 fitted, so it does not cost a branch, it costs every span")
print("and per-block constant in the tree being re-measured. FINDINGS 28.3.")
if __name__ == "__main__":
main()
tools/analysis/20_seek_slack.py
+146
View File
@@ -0,0 +1,146 @@
"""Seek slack: how long a branch point can stop delivery (STATUS 4, FINDINGS 51).
19_ring_stream.py asks whether a container ARRIVES in time, and prints one
"seek slack" column derived statically as (usable ring - maxrec)/mean record.
That is a capacity estimate and it quietly assumes the ring is full when the
seek happens. It is not, and the difference is the whole finding:
A ring's slack is ACCUMULATED, not owned. It is built out of the surplus
between the pipe and the wire demand, at (pipe - wire) bytes per second, and
a seek spends all of it. How long a branch point can stall is a property of
the ring; how soon the NEXT branch point can be afforded is a property of the
surplus, and a bigger ring makes that one WORSE.
This is the paced-rig model (tools/bench/stream.lua with DLX_PACE=1) written
independently, and it exists to be compared against it, not to replace it. The
rig drives a real 68000 through a real ring and is the measurement; this is the
cheap sweep that says where to point it. Where they disagree, the rig wins.
python3 tools/analysis/20_seek_slack.py [container ...] --kbps R [R ...]
[--ring KB [KB ...]]
`--kbps` is REQUIRED and takes no default, for the reason FINDINGS 50 gives.
"""
import sys, os, argparse
sys.path.insert(0, "tools/encoder")
import numpy as np
from dlx import DLX
import ratectl as RC
SECTOR = 512
def records(path):
d = DLX(path)
rec = np.array(d.record_lengths(), np.int64)
return d, rec
def paced_sim(rec, ring, fill_per_frame, ticks_per_frame=8):
"""Paced-decoder ring sim. Returns TWO per-tick lookahead series.
THE ANSWER IS BRACKETED TO ONE RECORD AND IS NOT SHARPER THAN THAT. At
these rates the pipe delivers almost exactly one record per frame slot, so
"how many records are resident at slot i" depends on whether you look before
or after that slot's delivery -- and the two answers differ by one, every
time. Sampled after, this agreed with the rig's ceiling in 33 of 35 cells;
sampled before, it was exactly one record lower in 33 of 35. Neither is
wrong. Picking the one that matched would have been fitting the model to
the measurement and then reporting the agreement as a cross-check, so both
are returned and the caller prints the range. The rig sits at the top of it.
The producer is `aligned` (19_ring_stream.py): it will not start a record it
cannot finish before the end of the ring, and it will not place one over
bytes the decoder still owns. The decoder consumes exactly one record per
frame time and releases it whole.
Sub-stepping matters. Delivery and consumption interleave inside a frame
time on the rig -- the producer runs on MAME's machine-frame notifier, ~5x
per 12fps slot -- and a model that delivers a whole frame's bytes at once
can place a record into space the decoder has not released yet, or refuse
one it has. Eight sub-steps is well past the point the answer stops moving.
"""
n = len(rec)
live = [] # [idx, off, len] still owned by the decoder
wcur, nsent, credit = 0, 0, 0.0
lo, hi, ring_ref, rate_ref = [], [], 0, 0
def overlaps(off, ln):
return any(off < r[1] + r[2] and r[1] < off + ln for r in live)
for i in range(n):
if nsent < n:
lo.append(sum(1 for r in live if r[0] >= i))
for _ in range(ticks_per_frame):
credit += fill_per_frame / ticks_per_frame
while nsent < n:
r = int(rec[nsent])
if credit < r:
rate_ref += 1
break
w, hole = wcur, 0
if w + r > ring:
w, hole = 0, ring - wcur
if overlaps(w, r):
ring_ref += 1
break
# sector quantisation: a partial sector is not resident
credit -= r
live.append([nsent, w, r])
wcur, nsent = w + r, nsent + 1
if nsent < n:
hi.append(sum(1 for r in live if r[0] >= i))
# the decoder consumed record i during the slot and releases it whole
live = [r for r in live if r[0] > i]
return np.array(lo), np.array(hi), ring_ref, rate_ref
def main():
ap = argparse.ArgumentParser()
ap.add_argument("containers", nargs="*",
default=["tmp/rc_fr_singe_scsi_span.dlx"])
ap.add_argument("--kbps", type=float, nargs="+", required=True,
help="delivered pipe rates, KB/s. REQUIRED, no default "
"(FINDINGS 50): this project has never measured the "
"delivery pipe and a default is how the last unmeasured "
"one stayed load-bearing for five sessions.")
ap.add_argument("--ring", type=float, nargs="+",
default=[64, 96, 128, 192, 256, 384, 512])
a = ap.parse_args()
FPS = 12
for path in a.containers:
if not os.path.exists(path):
print(f"{path}: MISSING -- skipped\n"); continue
d, rec = records(path)
wire = rec.mean() * FPS / 1024 + RC.AUDIO_KBPS
print(f"=== {path}: {d.nframes} frames @ {d.fps}fps, mean record "
f"{rec.mean()/1024:.1f} KB, wire {wire:.1f} KB/s")
print(f"{'ring KB':>8} {'pipe':>8} {'ceiling':>9} {'build s':>8} "
f"{'mean':>11} bound")
for ring_kb in a.ring:
ring = int(ring_kb * 1024)
if rec.max() > ring:
print(f"{ring_kb:>8.0f} maxrec {rec.max():,} does not fit")
continue
for kbps in a.kbps:
fill = ((kbps - RC.AUDIO_KBPS) * 1024 / FPS) if kbps > 0 else 1e12
lo, hi, ring_ref, rate_ref = paced_sim(rec, ring, fill)
c_lo, c_hi = int(lo.max()), int(hi.max())
build = int(np.argmax(hi >= c_hi)) if len(hi) else -1
print(f"{ring_kb:>8.0f} {kbps:>8.0f} "
f"{f'{c_lo}-{c_hi}':>9} {build/FPS:>8.2f} "
f"{f'{lo.mean():.1f}-{hi.mean():.1f}':>11} "
f"{'ring' if ring_ref else 'rate'}")
# The surplus model, stated so it can be checked against the sweep
# above rather than believed: slack accrues at (pipe - wire) and a
# full ring holds `ceiling` records, so a branch point costs about
# ceiling*mean_record/(pipe - wire) seconds of play to earn back.
print()
print("Slack is accumulated, not owned. A bigger ring raises the ceiling AND")
print("lengthens the climb to it: the surplus (pipe - wire) is what fills it,")
print("and that is set by the encoder and the medium, not by the buffer.")
if __name__ == "__main__":
main()
tools/analysis/21_iplrom_dmac.py
+213
View File
@@ -0,0 +1,213 @@
#!/usr/bin/env python3
"""What the X68000's own ROM programs into the DMAC -- read out of the bytes.
python3 tools/analysis/21_iplrom_dmac.py [iplrom.dat]
FINDINGS 48.4 / ROADMAP B3 left the single-address vs dual-address question
open for the disk, priced it at 242 KB/s and 0.69 dB, and blocked it on
sourcing `scsiexrom.bin` so its DMAC init could be disassembled. The same
question was open for AUDIO and nobody had asked it: ROADMAP P6 budgets ADPCM
at 7.8 KB/s and `11_cpu_budget.py` charges those bytes the DISK's per-byte
rate, which is a guess about a channel whose configuration was never read.
It does not have to be a guess. **The IPL ROM is on this machine** -- MAME runs
the player rig with `-bios ipl10` -- and it programs all four HD63450 channels
itself. This script reads the configuration straight out of the ROM image and
decodes the MC68450 register fields, so every claim below is a byte at a named
address rather than a recollection about a chip.
It is a GATE, not a report: each piece of evidence is (address, expected bytes,
what it means), and a mismatch exits non-zero. If a different ROM revision is
pointed at it, it says so instead of quietly decoding something else.
SOURCED for the field layouts: MC68450 Direct Memory Access Controller,
Motorola, Jul 1989 (bitsavers) -- the same document FINDINGS 39 already cites
for the transfer timings in tools/analysis/buscost.py.
NOTE THE LAYER: this is the ROM's own choice of configuration, read from the
shipping image. It is not a measurement of a running machine, and it is not
proof that a different configuration is impossible -- our player programs these
registers itself. It is evidence about what Sharp's engineers could get the
board to do, from the vendor, for these exact devices.
"""
import sys, os, argparse, hashlib
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
BASE = 0xFE0000 # where the IPL ROM is mapped (and its 0xFF0000 alias)
# The image this was decoded against. A different revision is a different
# machine's answer, so it is named rather than assumed.
KNOWN = {
"7fd4caabac1d9169e289f0f7bbf71d8e":
"IPL 1.0 (MAME x68000 -bios ipl10), 131,072 B",
}
from mc68450 import REG, XRM, DTYP, DPS, PCL, SIZE, CHAIN, REQG, dcr, ocr, scr
# --- the evidence ----------------------------------------------------------
# (address, expected bytes, one-line description). Every register value quoted
# anywhere below comes out of one of these; nothing is typed in twice.
EV = [
(0xFF0BEA, "49f900e84080197c00080004197c0005",
"boot: lea $E84080,a4 (ch2) ; DCR=$08 ; SCR=$05..."),
(0xFF0C2E, "49f900e840c0197c00800004197c00040006197c00050029197c0001002d"
"197c00050031197c00050039297c00e92003",
"boot: lea $E840C0,a4 (ch3, ADPCM) ; DCR=$80 SCR=$04 MFC=$05 CPR=$01 "
"DFC=$05 BFC=$05 DAR=$E92003"),
(0xFF0D8E, "0480060429052d0031054480460469056d027105",
"boot: the ch0/ch1 init TABLE, ten (offset,value) pairs, written by the "
"loop at $FF0CD8"),
(0xFF0CE4, "217c00e940030014217c00e960010054",
"boot: DAR ch0 = $E94003 (FDC data) ; DAR ch1 = $E96001 (SASI data)"),
(0xFF9A82, "13fc003200e840c5610a13fc000200e920014e75",
"IOCS ADPCM PLAY: OCR(ch3) = $32 ; then command $02 to $E92001"),
(0xFF9A5E, "13fc00b200e840c5612e13fc000400e920014e75",
"IOCS ADPCM RECORD: OCR(ch3) = $B2 ; then command $04 to $E92001"),
(0xFF9A96, "13fc00ff00e840c023c900e840cc33c200e840ca",
"IOCS ADPCM arm: CSR=$FF ; MAR = a1 ; MTC = d2 (DCR/SCR untouched)"),
(0xFF9944, "13fc00ff00e8404013fc00b200e84045601013fc00ff00e8404013fc003200"
"e8404523c900e8404c33c300e8404a13fc008000e840474e75",
"IOCS SASI: OCR(ch1) = $B2 read / $32 write ; MAR ; MTC ; CCR = $80"),
]
ap = argparse.ArgumentParser()
ap.add_argument("rom", nargs="?",
default=os.path.expanduser("~/mame/roms/iplrom.dat"))
a = ap.parse_args()
if not os.path.exists(a.rom):
sys.exit(f"missing {a.rom} -- point this at the IPL ROM MAME boots the rig "
f"with (-bios ipl10).")
d = open(a.rom, "rb").read()
md5 = hashlib.md5(d).hexdigest()
print(f"{a.rom}: {len(d):,} B, md5 {md5}")
if md5 in KNOWN:
print(f" {KNOWN[md5]}\n")
else:
sys.exit(f"\nUNKNOWN ROM. Every field decoded below was read out of\n"
f" {list(KNOWN.values())[0]}\n"
f"and a different revision is a different machine's answer, not a "
f"detail. Add its\nmd5 to KNOWN only after re-reading the sites -- "
f"the addresses are revision-specific.")
print("EVIDENCE -- each line is bytes at an address, not a recollection")
bad = 0
for addr, hx, what in EV:
want = bytes.fromhex(hx)
got = d[addr - BASE: addr - BASE + len(want)]
ok = got == want
bad += not ok
print(f" {'OK ' if ok else 'FAIL'} ${addr:06X} {what}")
if not ok:
print(f" expected {want.hex()}\n got {got.hex()}")
if bad:
sys.exit(f"\nFAIL: {bad} evidence site(s) do not hold. The decode below "
"would be about\nsome other code, so it is not printed.")
# The ch0/ch1 table, decoded from the bytes rather than restated.
tbl = d[0xFF0D8E - BASE: 0xFF0D8E - BASE + 20]
init = {}
for i in range(0, len(tbl), 2):
off, val = tbl[i], tbl[i + 1]
init[(off >> 6, off & 0x3F)] = val
init[(2, 0x04)] = 0x08 # from the inline moves at $FF0BEA
init[(2, 0x06)] = 0x05
init[(2, 0x2D)] = 0x03
init[(3, 0x04)] = 0x80 # ...and at $FF0C2E
init[(3, 0x06)] = 0x04
init[(3, 0x2D)] = 0x01
DEV = {0: ("FDC", "$E94003"), 1: ("SASI", "$E96001"),
2: ("IOCS _DMAMOVE (general purpose)", "set per call"),
3: ("ADPCM MSM6258V", "$E92003")}
print("\nWHAT THE ROM PROGRAMS, per channel")
for ch in range(4):
name, dar = DEV[ch]
print(f"\n ch{ch} base $E840{ch*0x40:02X} {name} DAR = {dar}")
v = init[(ch, 0x04)]
print(f" DCR = ${v:02X}")
for line in dcr(v):
print(f" {line}")
v = init[(ch, 0x06)]
print(f" SCR = ${v:02X} " + " ; ".join(scr(v)))
print(f" CPR = ${init[(ch,0x2D)]:02X} channel priority "
f"({init[(ch,0x2D)]}, 0 = highest)")
print("\nAND THE PER-TRANSFER OCR, written every time a transfer is armed")
for label, ch, v in (("ADPCM playback", 3, 0x32), ("ADPCM record", 3, 0xB2),
("SASI write", 1, 0x32), ("SASI read", 1, 0xB2)):
print(f"\n {label:<15} ch{ch} OCR = ${v:02X}")
for line in ocr(v):
print(f" {line}")
print(f"""
WHAT THIS SETTLES
1. AUDIO IS DUAL ADDRESS, AND IT CANNOT HOLD THE BUS. ch3 DCR = $80: DTYP =
00, explicitly addressed, so every ADPCM byte is a MEMORY READ FOLLOWED BY A
DEVICE WRITE -- not the single-address 5 clocks the disk debit is written in.
XRM = 10 is cycle steal WITHOUT hold and OCR REQG = 10 is external request,
so the DMAC arbitrates for the bus ONCE PER BYTE and gives it straight back.
There is no burst to amortise the arbitration over.
2. THE PORT IS 8 BITS AND THE OPERAND IS A BYTE. DCR DPS = 0, OCR SIZE = 11.
One MSM6258V byte is two 4-bit samples, so 15.6 kHz is 7,812.5 BYTES/s and
7,812.5 DMA REQUESTS/s -- the request count does not halve the way a 16-bit
port's would. That is the FINDINGS 43 unit trap, in the other stream.
3. THE DISK CHANNEL IS PROGRAMMED IDENTICALLY, AND THAT IS THE BIGGER NEWS.
ch1 (SASI, DAR = $E96001) gets DCR = $80 and OCR = $B2 -- dual address,
8-bit port, cycle steal WITHOUT hold, external request. Byte by byte, with a
full arbitration each time, exactly like the audio. ch0 (FDC) too. Sharp
programs every explicitly-addressed 8-bit device on this board the same way.
This is not scsiexrom.bin and it does not close ROADMAP B3 -- a different
ROM drives a different SPC. But it is the same vendor, the same DMAC and the
same class of device, and it lands on the EXPENSIVE side of B3's 242 KB/s.
4. AND IT IS OUTSIDE THE BRACKET THE PROJECT HAS BEEN COSTING P4 IN.
FINDINGS 42.4-42.6 brackets W, the clocks stolen per delivered byte, at
5..12, and reports that W <= 6 fits 0/120 frames while W = 8 misses 47/120.
The ROM's own disk configuration costs 16..19. It is still true that the
player programs these registers itself and the choice is ours (42.6) -- but
the only worked example on the machine sits ABOVE the whole bracket, and
nothing in this tree has yet shown that a cheaper configuration is reachable
for an explicitly-addressed port. Treat W <= 12 as a REQUIREMENT ON THE
PLAYER'S DMAC PROGRAMMING, not as a range the hardware hands us.
5. AUDIO OUTRANKS THE DISK AT THE ARBITER. CPR: FDC 0, ADPCM 1, SASI 2,
_DMAMOVE 3, lower being higher priority. When both channels want the bus in
the same slot, the ROM's arrangement serves ADPCM first. An audio byte is
never the thing that waits; a video byte is.
""")
# --- what it costs ---------------------------------------------------------
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import buscost as B
ADPCM_HZ = 15625.0 # 8 MHz MSM6258V clock / 512
ADPCM_BPS = ADPCM_HZ / 2 # 4-bit samples, two to a byte
FPS, CPUHZ = 12.0, 10e6
lo = B.DMA_DUAL_BYTE_CLK + B.DMA_FRONT_CLK + B.DMA_BACK_CLK
hi = B.DMA_DUAL_BYTE_CLK + B.DMA_FRONT_CLK_WORST + B.DMA_BACK_CLK
bpf = ADPCM_BPS / FPS
print(f"WHAT IT COSTS, at the configuration above\n"
f" 15.6 kHz mono = {ADPCM_HZ:,.0f} samples/s = {ADPCM_BPS:,.1f} B/s "
f"= {ADPCM_BPS/1024:.2f} KiB/s\n"
f" (ratectl.AUDIO_KBPS is 7.8, which is this figure in DECIMAL kB; "
f"as KiB it is {ADPCM_BPS/1024:.2f})\n"
f" dual-address byte transfer {B.DMA_DUAL_BYTE_CLK} clk "
f"(read {B.DMA_READ_CLK} + write {B.DMA_WRITE_CLK}, Fig 4-25 sheet 4 note 2)\n"
f" + arbitration, EVERY byte {B.DMA_FRONT_CLK}..{B.DMA_FRONT_CLK_WORST}"
f" front + {B.DMA_BACK_CLK} back (sect 4.5.2.1/4.5.2.2)\n"
f" = {lo}..{hi} clocks per audio byte\n\n"
f" per frame at {FPS:g} fps: {bpf:,.1f} B costs {bpf*lo:,.0f}..{bpf*hi:,.0f} "
f"clocks of {CPUHZ/FPS:,.0f}\n"
f" = {100*bpf*lo/(CPUHZ/FPS):.2f}%..{100*bpf*hi/(CPUHZ/FPS):.2f}% of the "
f"frame, stolen from the 68000\n\n"
f" 11_cpu_budget.py charges audio --dma-clocks-per-byte, default 5, "
f"described\n as 'single-address, bus held, no drive wait'. The ROM says "
f"audio is neither\n single-address nor able to hold the bus, so that "
f"debit is {lo/5:.1f}x..{hi/5:.1f}x too small.\n"
f" In absolute terms it is small -- but it is small IN THE RESOURCE THE "
f"PROJECT IS\n SHORT OF, and it was being taken from the wrong side of "
f"an open question.")
tools/analysis/22_scene_load.py
+106
View File
@@ -0,0 +1,106 @@
#!/usr/bin/env python3
"""What a scene change costs, now that the loader runs on the 68000 (FINDINGS 53).
python3 tools/analysis/22_scene_load.py [container ...] --kbps R [R ...]
ROADMAP P1 asked for the codebook expansion to be priced "against the refill
climb, not treated as free setup", and that is the whole job of this file. A
scene change is the one moment where every cost in this project lands at once:
the ring is empty because of the seek, the header has to arrive before a single
frame can be drawn, and the 68000 cannot decode anything until it has expanded
the codebooks out of that header.
THREE COSTS, IN THREE DIFFERENT UNITS, and they are not interchangeable:
* BYTES. The container's header region -- palette, CB1, CB4 -- must be
delivered before frame 0 can be decoded. It is not part of any frame
record, so no rate table in this tree has ever counted it.
* CLOCKS. What src/player/load.i costs to turn that header into what the
block loop reads, MEASURED on the emulated 68000 by tools/bench/load.lua
and parsed out of its log rather than copied in here as a constant.
* ACCUMULATED SLACK. The bytes above are bytes the pipe did not spend
filling the ring, so they cost play-time at the surplus rate (pipe - wire),
which is the currency FINDINGS 51.3 established a branch point spends.
This is the one that compounds: it is charged on top of the seek itself.
`--kbps` is REQUIRED and takes no default, for the reason FINDINGS 50 gives.
Rates are decimal-KB per the rest of the tree's tooling; sizes are KiB.
"""
import sys, os, re, argparse
sys.path.insert(0, "tools/encoder")
import numpy as np
from dlx import DLX
import ratectl as RC
FPS = 12
CPUHZ = 10_000_000
def rig_cycles(path):
"""The measured per-stage cost, out of tools/bench/load.lua's own log.
Parsed rather than pasted: a constant copied in here would go stale the
first time load.i changed, and it would go stale SILENTLY -- the arithmetic
below would keep working and keep being wrong.
"""
if not os.path.exists(path):
sys.exit(f"no rig log at {path} -- run tools/bench/load_run.sh first")
out = {}
for line in open(path, "rb").read().decode("utf-8", "replace").splitlines():
m = re.search(r"^\[LOD\]\s+(\S.*?)\s{2,}(\d+) cyc", line)
if m:
out[m.group(1).strip()] = int(m.group(2))
need = ("SCENE CHANGE: codebooks + palette", "scratch tables only (boot, once)")
for k in need:
if k not in out:
sys.exit(f"{path} has no '{k}' line -- is it a load.lua summary?")
return out
ap = argparse.ArgumentParser()
ap.add_argument("containers", nargs="*", default=["tmp/rc_fr_singe_scsi_span.dlx"])
ap.add_argument("--kbps", type=float, nargs="+", required=True,
help="delivered pipe rates, KB/s. REQUIRED, no default (FINDINGS 50)")
ap.add_argument("--log", default="tmp/load_check.log",
help="tools/bench/load.lua's log, for the measured cycle counts")
a = ap.parse_args()
cyc = rig_cycles(a.log)
scene_cyc = cyc["SCENE CHANGE: codebooks + palette"]
boot_cyc = cyc["scratch tables only (boot, once)"]
frame_cyc = CPUHZ / FPS
print(f"measured on the emulated 68000 ({a.log}):")
print(f" per scene change {scene_cyc:>8,} clocks = {1000*scene_cyc/CPUHZ:6.2f} ms "
f"= {100*scene_cyc/frame_cyc:.1f}% of one {FPS}fps frame")
print(f" once at boot {boot_cyc:>8,} clocks = {1000*boot_cyc/CPUHZ:6.2f} ms "
f" (the three scratch tables: scene-independent)")
for path in a.containers:
d = DLX(path)
hdr = int.from_bytes(d.raw[28:32], "big")
rec = np.array(d.record_lengths(), np.int64)
wire = rec.mean() * FPS / 1024 + RC.AUDIO_KBPS
print(f"\n=== {path}: header region {hdr:,} B "
f"(pal 768 + cb1 {d.k1*16:,} + cb4 {d.k4*4:,} + 32), wire {wire:.1f} KB/s")
print(f"{'pipe':>6} {'header ms':>10} {'+load ms':>9} {'total':>7} "
f"{'frames':>7} {'surplus':>9} {'slack s':>9}")
for kbps in a.kbps:
hdr_ms = 1000 * hdr / (kbps * 1024)
load_ms = 1000 * scene_cyc / CPUHZ
total = hdr_ms + load_ms
surplus = kbps - wire
# What the header costs in the currency of 51.3: play-time at the
# surplus rate. A negative surplus means the container does not fit the
# pipe at all and no amount of play buys the bytes back.
slack = f"{hdr/(surplus*1024):8.3f}" if surplus > 0 else " NEVER"
print(f"{kbps:>6.0f} {hdr_ms:>10.2f} {load_ms:>9.2f} {total:>7.2f} "
f"{total/(1000/FPS):>7.2f} {surplus:>9.1f} {slack:>9}")
print("""
Reading it. The 'frames' column is the scene change's FIXED cost in 12fps
frame slots, before the ring has been given a single frame of lookahead -- so it
is a floor under the black gap at a branch point, not the gap itself. The
'slack s' column is the one that compounds with FINDINGS 51.3: the header's
bytes are bytes that did not go into the ring, so they lengthen the climb back
to the seek-slack ceiling by that much play-time, every time.""")
tools/analysis/23_frame_clock.py
+254
View File
@@ -0,0 +1,254 @@
#!/usr/bin/env python3
"""What a real frame clock can be built from, and what its cadence costs.
python3 tools/analysis/23_frame_clock.py [--fps 12] [--vtotal 568]
[--csv tmp/c68k_frames.csv]
ROADMAP P3 says "needs MFP timer or VBL", which hides the fact that ONE OF
THOSE CANNOT DO IT and the other cannot do it either without a divider. This
file enumerates the space rather than asserting a conclusion, the way FINDINGS
47 had to be re-done once a hardware "no" turned out to be a claim about a whole
configuration space nobody had walked.
EVERY CONSTANT HERE IS SOURCED, and from a file on this machine:
MFP timer clock 16 MHz / 4 MAME 0.277 sharp/x68k.cpp:1027-1028
prescaler ladder 4,10,16,50,64,100,200 machine/mc68901.cpp:173
timer data reg 8 bits, 0 means 256 machine/mc68901.cpp (TCDR/TADR)
V-DISP -> GPIP4 x68k.cpp:1139, and it is also Timer A's event input,
mc68901.cpp:167 GPIO_TIMER = {GPIP_4, GPIP_3}
line rate 31,500 Hz exactly in both 31.5 kHz modes, derived in
tools/bench/crtc_mode.lua from the dot clocks
interrupt cost MEASURED, not tabled: tools/bench/clock_run.sh
THE PART THAT IS NOT A CLOCK PROBLEM AT ALL. 12 fps on a 55.4577 Hz raster is
4.6215 refreshes per frame, so every frame is shown for 4 refreshes or 5 --
72.13 ms or 90.16 ms -- and 37.9% of them get the short one. That is the
display's quantisation and no choice of clock changes it. What it changes is
the BUDGET: 833,333 clocks is the mean slot, not the slot, and the short slot is
721,270. With the per-frame decode costs in hand this file says exactly how
many frames do not fit theirs, which is a thing this project has never had to
ask because until now the tick came from a host that could not miss.
"""
import argparse
import csv
import os
import sys
MFP_HZ = 16_000_000 // 4 # x68k.cpp:1027-1028
PRESCALER = [4, 10, 16, 50, 64, 100, 200] # mc68901.cpp:173
HFREQ = 31500 # lines/s, crtc_mode.lua
CPUHZ = 10_000_000 # 40 MHz / 4, x68k.cpp:1133
def timer_space(fps):
"""Every (prescale, data) the MFP can be set to, against a target fps."""
slowest = MFP_HZ / (PRESCALER[-1] * 256)
print(f"\n=== 1. THE MFP TIMER, WHICH CANNOT DO IT ALONE")
print(f" timer clock {MFP_HZ:,} Hz, prescalers {PRESCALER}, data 1..256")
print(f" slowest tick any single timer can produce: "
f"{MFP_HZ}/({PRESCALER[-1]}*256) = {slowest:.3f} Hz")
print(f" a {fps} fps frame needs {fps} Hz, which is {slowest/fps:.1f}x "
f"slower than that -- so a software divider is REQUIRED whatever the "
f"source, and 'use an MFP timer' is not by itself an answer.")
exact = [(p, d) for p in PRESCALER for d in range(1, 257)
if (MFP_HZ * d * p) and (MFP_HZ % (p * d) == 0)
and (MFP_HZ // (p * d)) % fps == 0]
print(f" settings whose tick rate is a whole multiple of {fps} Hz, so that "
f"a plain counter would be exact: {len(exact)}")
if exact:
best = min(exact, key=lambda pd: MFP_HZ / (pd[0] * pd[1]))
p, d = best
tick = MFP_HZ / (p * d)
print(f" slowest of them: prescale /{p} data {d} = {tick:.4f} Hz, "
f"{tick/fps:.0f} ticks per frame")
print(f" -> {tick/fps:.0f} interrupts per frame, against 4.6215 for "
f"the raster: {tick/fps/(HFREQ/(fps*568)):.1f}x the cost, and its "
f"phase against the raster is arbitrary, so a frame would be "
f"presented mid-scan.")
else:
print(f" NONE. {MFP_HZ}/{fps} = {MFP_HZ/fps:,.2f} is not an "
f"integer, so no prescale/data pair divides to {fps} Hz at all.")
def raster_space(fps, vtotal):
hz = HFREQ / vtotal
print(f"\n=== 2. THE RASTER, WHICH IS THE RIGHT SOURCE AND IS ALSO NOT "
f"A WHOLE DIVIDE")
print(f" V-DISP is {HFREQ}/{vtotal} = {hz:.4f} Hz, and it is BOTH the "
f"GPIP4 interrupt and Timer A's event-count input")
print(f" whole divides -- all the MFP can do in hardware, no software:")
for n in (3, 4, 5, 6):
f = hz / n
print(f" Timer A event count = {n}: {f:7.4f} fps "
f"({100*(f/fps-1):+6.2f}% from {fps})")
print(f" {fps} fps needs {hz/fps:.4f} refreshes per frame, which is not a "
f"whole number, so no event-count setting is exact either.")
print(f"\n THE DIVIDER THAT IS EXACT: add fps*VTOTAL = {fps*vtotal} per "
f"V-DISP, emit a tick at {HFREQ}, keep the remainder.")
print(f" long-run rate = {fps}*{vtotal}/{vtotal} = {fps} fps EXACTLY, "
f"with a remainder that never accumulates")
print(f" the accumulator stays under {HFREQ + fps*vtotal:,}, so it is "
f"16-bit arithmetic on a 68000 (the ceiling is fps < "
f"{(65536-HFREQ)/vtotal:.1f} at this VTOTAL, and clk_init checks it)")
def divider_gaps(fps, vtotal, n):
"""The tick sequence src/player/clock.i emits, in refreshes per tick."""
acc, gaps, since = 0, [], 0
while len(gaps) < n:
acc += fps * vtotal
since += 1
if acc >= HFREQ:
acc -= HFREQ
gaps.append(since)
since = 0
return gaps
def host_gaps(fps, refresh_hz, n):
"""The tick sequence tools/bench/stream.lua's HOST clock emits.
It looks uniform in the source -- `floor((t - t_rel) * fps)` -- and is not.
Lua only sees the machine at frame boundaries, so tick k lands on the first
refresh at or after k/fps, and the gaps between ticks come out as the same
two whole numbers of refreshes the divider produces. The host-paced runs of
FINDINGS 49 and 51 therefore already had this cadence in them; what ROADMAP
P3 changes is who produces it, not whether it exists.
"""
at = [-(-int(k * refresh_hz * 1000000 // fps) // 1000000) for k in range(n + 1)]
return [at[k + 1] - at[k] for k in range(n)]
def cadence(fps, vtotal, costs, label, refresh_hz, gaps, uniform=False):
"""Charge each frame the slot it really gets, and run the pace gate."""
rpf = 1.0 if uniform else HFREQ / (fps * vtotal)
lo, hi = (1, 1) if uniform else (int(rpf), int(rpf) + 1)
slot_lo = lo / refresh_hz * CPUHZ
slot_hi = hi / refresh_hz * CPUHZ
nominal = CPUHZ / fps
n_lo = sum(1 for g in gaps[:len(costs)] if g == lo)
print(f"\n --- {label}: refresh {refresh_hz:.4f} Hz")
print(f" slots are {lo} refreshes = {slot_lo:,.0f} clk "
f"({1000*lo/refresh_hz:.2f} ms) or {hi} = {slot_hi:,.0f} clk "
f"({1000*hi/refresh_hz:.2f} ms)")
print(f" the nominal {fps} fps budget every figure in this project is "
f"priced against is {nominal:,.0f} clk; the SHORT slot is "
f"{100*(slot_lo/nominal-1):+.1f}% of it")
over_lo = sum(1 for c in costs if c > slot_lo)
over_hi = sum(1 for c in costs if c > slot_hi)
over_nom = sum(1 for c in costs if c > nominal)
print(f" frames that do not fit: {over_lo}/{len(costs)} the short "
f"slot, {over_hi}/{len(costs)} the long one, {over_nom}/{len(costs)} "
f"the nominal budget")
# The schedule, with the catch-up the pace gate actually performs: frame i
# starts at max(finish of i-1, tick i). A frame that overruns does not fail
# -- it eats the next frame's idle, and the clock catches up by itself.
t, tick, late, worst, first = 0.0, 0.0, 0, 0.0, None
for i, c in enumerate(costs):
if t <= tick:
t = tick # idled: on time
else:
if i: # frame 0 has no predecessor
late += 1
if first is None:
first = i
worst = max(worst, t - tick)
t += c
tick += gaps[i] / refresh_hz * CPUHZ
print(f" through the pace gate: {late}/{len(costs)} frames found "
f"their slot already open (first at frame {first}), worst start "
f"{worst:,.0f} clk = {1000*worst/CPUHZ:.1f} ms behind its tick")
if not uniform:
print(f" {n_lo}/{len(costs)} slots were the short one "
f"({100*n_lo/len(costs):.1f}%; the exact share is "
f"{100*(hi-rpf):.1f}%)")
return late
def main():
ap = argparse.ArgumentParser()
ap.add_argument("--fps", type=int, default=12)
ap.add_argument("--vtotal", type=int, default=568,
help="CRTC R04+1; 568 is the 31.5 kHz 256-line mode")
ap.add_argument("--csv", default="tmp/c68k_frames.csv",
help="per-frame decode cost, from tools/bench/c68k/run.sh")
a = ap.parse_args()
timer_space(a.fps)
raster_space(a.fps, a.vtotal)
print(f"\n=== 3. WHAT THE CADENCE COSTS")
if not os.path.exists(a.csv):
print(f" {a.csv} not found -- run tools/bench/c68k/run.sh first. The "
f"cadence question CANNOT be answered from percentiles: it needs "
f"the per-frame series, because what matters is whether an "
f"expensive frame lands in a short slot and how long the catch-up "
f"takes afterwards.")
return 1
costs = [float(r["cycles"]) for r in csv.DictReader(open(a.csv))]
print(f" {len(costs)} frames from {a.csv}: mean {sum(costs)/len(costs):,.0f} "
f"clk, max {max(costs):,.0f} (frame {costs.index(max(costs))})")
hw = HFREQ / a.vtotal
n = len(costs)
# MAME's screen is fast by htotal/(htotal-8): refresh_mode() builds the
# frame period from scr.max_x*scr.max_y with scr.max_x = m_htotal - 8. Both
# rasters are run, because the rig measures against the fast one and the
# player will run on the other -- reporting only one would leave the rig's
# count and this file's differing with nobody able to say which was wrong.
htotal = 368
mame = hw * htotal / (htotal - 8)
# The budget model every figure in FINDINGS assumes: a slot of exactly
# 1/fps. No machine has this; it is the yardstick, run through the same
# schedule so that what the cadence ADDS can be read off.
cadence(a.fps, a.vtotal, costs, "the NOMINAL model (a slot of exactly "
"1/fps, which no raster produces)", float(a.fps),
[1] * (n + 1), uniform=True)
cadence(a.fps, a.vtotal, costs, "the HOST tick, as tools/bench/stream.lua "
"actually emits it", mame, host_gaps(a.fps, mame, n + 1))
cadence(a.fps, a.vtotal, costs, "the 68000's own clock on the HARDWARE "
"raster", hw, divider_gaps(a.fps, a.vtotal, n + 1))
cadence(a.fps, a.vtotal, costs, f"the 68000's own clock on MAME's raster "
f"(fast by {htotal}/{htotal-8})", mame,
divider_gaps(a.fps, a.vtotal, n + 1))
print(f"""
Reading it.
THE CADENCE WAS ALREADY THERE. The nominal row is the model every budget in
this project is priced against -- a slot of exactly 1/fps -- and no raster
produces it. The host row is what tools/bench/stream.lua has been emitting all
along: `floor((t - t_rel) * fps)` looks uniform, but Lua only sees the machine at
frame boundaries, so its ticks land on refreshes and its gaps are the same two
whole numbers. The host-paced results of FINDINGS 49 and 51 therefore already
carried a 4/5 cadence that nothing named. ROADMAP P3 did not introduce it; it
moved who produces it onto the machine, where it belongs, and made it visible.
THE SHORT SLOT IS REAL AND IT IS NOT A FAILURE. {sum(1 for c in costs if c > 721270)}/{len(costs)} frames do not fit
721,270 clocks. The pace gate only says "not before tick i", so a frame that
overruns spends the next frame's idle and the clock recovers by itself; the cost
is one frame presented a refresh late, not a dropped frame. What the counts
above measure is frames with no idle left, and the difference between the
nominal row and the raster rows -- 1 against 4 -- is the whole price of the
cadence on this container.
THE EXPENSIVE FRAME IS FRAME 0, at {max(costs)/(CPUHZ/a.fps)*100:.0f}% of the nominal budget: the first
frame of a scene has nothing to SKIP against, so it is the whole picture in one
slot. Most of what follows it in these counts is that transient draining, which
is why the first index is printed next to the total. It also means the cost is
paid AT A SCENE CHANGE, alongside the 18.96 ms of loader (FINDINGS 53.2) and the
seek -- not spread over the window.
SCOPE. Decode costs are C68K's, on zero-wait-state memory, so they are a lower
bound; real DRAM moves every row here in the same direction. The MAME rows are
the emulator's fast raster and exist to be compared with tools/bench/pace_run.sh,
not to describe hardware.""")
return 0
if __name__ == "__main__":
sys.exit(main())
tools/analysis/24_ring_owner.py
+305
View File
@@ -0,0 +1,305 @@
"""What the player's own fill loop costs the pipe (ROADMAP P5, FINDINGS 55).
19_ring_stream.py and 20_seek_slack.py model a ring whose producer is free to
act whenever it likes: bytes arrive at a rate and the only questions are where
they go and whether the ring can hold them. That is what a HOST-filled ring is,
and it is what every delivery figure in FINDINGS 49 and 51 was measured on.
A player has no host. The 68000 owns the ring (src/player/ring.i), and it can
only act when it is not decoding -- which turns the producer into a consumer of
the same resource the decoder is short of, and puts an idle CHANNEL between
every pair of records:
a transfer ends -> the disc has nothing to do -> the CPU next polls
-> it issues -> the disc starts again
The gap in the middle is bytes the medium could have delivered and did not, and
no rate table in this tree contains it. Its size is set by the PLAYER: how many
requests it may have outstanding (a DMAC channel takes one at a time; two slots
mean the next is already queued when the current lands), and when it polls.
This is that model, written from record sizes and per-frame decode costs, and
sharing no code with the Lua rig it is compared against -- the same arrangement
as 49.4 and 51.5. The rig drives a real 68000 through a real ring and is the
measurement; this says where to point it and what to expect.
python3 tools/analysis/24_ring_owner.py <container.dlx> --kbps R [R ...]
[--ring KB] [--qdepth N [N ...]] [--prefill RECORDS]
[--cadence raster|nominal]
`--kbps` is REQUIRED and has no default, for the reason FINDINGS 50 gives.
THE INDEX IS READ FROM THE CONTAINER, not derived by walking it. A DLX4
container carries nframes u16 record lengths in its scene header precisely
because the producer needs the length of a record it has not fetched; this model
reads the same table the 68000 does, so a container whose index disagreed with
its stream would be caught here as well as in dlx.py's constructor.
"""
import sys, os, argparse
sys.path.insert(0, "tools/encoder")
import numpy as np
from dlx import DLX
import vq_hybrid as H
import spans as SP
CPUHZ = 10_000_000
HFREQ = 31500 # lines/s in the 31.5 kHz modes (src/player/clock.i)
VTOTAL = 568 # CRTC R04+1 in the 256x256 mode (tools/bench/crtc_mode.lua)
CLK_ISR = 181.35 # clocks per V-DISP, MEASURED (FINDINGS 54.3)
AUDIO_KBPS = 7.8 # ratectl.AUDIO_KBPS; the pipe carries it too
def frame_costs(d):
"""Per-frame decode cost in 68000 clocks: blocks plus v7 spans.
Both halves come from the encoder's own measured constants -- H.cycles is
the single source 11_cpu_budget.py uses, and SP.clocks is the v7 fit of
FINDINGS 40 -- so this is the same cost model the rate controller fits `mu`
against, applied to the emitted container rather than to a candidate.
"""
out = []
for f in range(d.nframes):
c = H.cycles(d.modes(f))
for _, _, px in d.spans(f)[0]:
c += SP.clocks(len(px))
out.append(c)
return np.array(out)
def ticks(n, fps):
"""Frame tick times from src/player/clock.i's divider, or a nominal clock.
The player's clock is the raster with a remainder: a frame gets 4 refreshes
(72.13 ms) or 5 (90.16 ms) and there is no 83.33 ms frame (FINDINGS 54.4).
A model that hands out uniform slots gives every frame 13.4% more time than
the short one really has, so the cadence is reproduced here rather than
averaged away.
"""
R = VTOTAL / HFREQ # one refresh, seconds
acc, out, t = 0, [0.0], 0.0
while len(out) < n:
t += R
acc += fps * VTOTAL
if acc >= HFREQ:
acc -= HFREQ
out.append(t)
return np.array(out)
def simulate(rec, dec, tick, bps, ringsz, qdepth, prefill):
"""One pass of the machine-owned ring. Returns a dict of instruments.
The rules are src/player/ring.i's, stated as events:
* the CPU polls whenever it is NOT decoding -- the pace wait and the
record wait both call ring_poll, and nothing else in the frame does;
* a request occupies a slot until it is RETIRED, which happens at a poll,
so the queue is measured against retirement and not against completion;
* placement is `aligned`: a record that will not fit before the end of the
ring restarts at the base, and only if the base is free;
* the channel serves one transfer at a time, in order.
"""
n = len(rec)
# ring state, in ring offsets
wcur = rcur = 0
rq = 0 # next record to request
retired = 0 # requests retired (== FR_HEAD)
consumed = 0 # records the decoder has finished (== FR_TAIL)
inflight = [] # [(record, done_time)] in issue order
chan_free = 0.0 # when the channel finishes what it has
gaps, gap_tot, gap_max = 0, 0.0, 0.0
busy = 0.0
full_refusals = 0
started = False # the first transfer has no gap before it
def live_empty():
return rq == consumed
def place(length):
"""Where the next record goes: (offset, hole) or None if it cannot."""
nonlocal full_refusals
if live_empty():
if wcur + length <= ringsz:
return wcur, 0
return 0, ringsz - wcur
if rcur == wcur:
return None # completely full
if rcur < wcur: # free is [wcur, SZ) then [0, rcur)
if wcur + length <= ringsz:
return wcur, 0
if length <= rcur:
return 0, ringsz - wcur
return None
if wcur + length <= rcur: # live wraps; free is [wcur, rcur)
return wcur, 0
return None
def issue(now):
"""Issue as many requests as the queue and the ring allow, at `now`."""
nonlocal wcur, rcur, rq, chan_free, gaps, gap_tot, gap_max, busy
nonlocal full_refusals, started
while rq < n and (rq - retired) < qdepth:
p = place(rec[rq])
if p is None:
full_refusals += 1
return
off, _hole = p
if live_empty():
rcur = off
start = max(now, chan_free)
if started:
g = start - chan_free
if g > 1e-12:
gaps += 1
gap_tot += g
gap_max = max(gap_max, g)
started = True
dur = rec[rq] / bps
busy += dur
chan_free = start + dur
inflight.append((rq, chan_free))
wcur = off + rec[rq]
rq += 1
def retire(now):
"""Publish every transfer that has landed by `now`. In order."""
nonlocal retired
while inflight and inflight[0][1] <= now:
inflight.pop(0)
retired += 1
def advance_reader():
"""Step rcur over the records the decoder has finished with."""
nonlocal rcur
i = consumed_seen[0]
while i < consumed:
end = rcur + rec[i]
if i + 1 < n and end + rec[i + 1] > ringsz:
end = 0
rcur = end
i += 1
consumed_seen[0] = i
consumed_seen = [0]
# ---- prefill. The decoder is not running, so the CPU polls continuously
# and the channel never waits for it: this is the one part of a scene where
# the request loop costs nothing.
now = 0.0
while retired < prefill and rq < n:
issue(now)
if not inflight:
break
now = inflight[0][1]
retire(now)
prefill_done = now
t0 = now
underruns, worst_late, noidle = 0, 0.0, 0
slack_series = []
for i in range(n):
deadline = t0 + tick[i]
# TWO WAYS A FRAME CAN START LATE, AND THEY ARE NOT THE SAME FAILURE.
# The decoder reaches the record wait at max(its own finish, the tick):
# if it got there after the tick, the PREVIOUS frame used its whole slot
# and this is the CPU (54.4's cadence). If it got there on time and the
# record was not resident, that is the PIPE. The rig counts them
# separately -- NO IDLE and UNDERRUNS -- so conflating them here would
# have made the model disagree with it for a reason that is not about
# delivery at all.
if now > deadline + 1e-9:
noidle += 1
arrive = max(now, deadline)
now = arrive
# the record wait: the CPU polls, so it retires and issues while it waits
starved = retired <= i
while retired <= i:
issue(now)
if not inflight:
break
now = max(now, inflight[0][1])
retire(now)
if starved:
underruns += 1
worst_late = max(worst_late, now - arrive)
slack_series.append(retired - consumed)
issue(now)
# ---- decode. No polls: whatever the channel finishes now waits.
now += dec[i] / CPUHZ
consumed += 1
advance_reader()
retire(now)
issue(now)
# ---- idle until the next tick. The CPU polls throughout, so every
# completion is retired and every free slot is refilled at once.
nxt = t0 + tick[i + 1] if i + 1 < n else now
while inflight and inflight[0][1] < nxt:
now = max(now, inflight[0][1])
retire(now)
issue(now)
now = max(now, min(nxt, now))
span = max(now - t0, 1e-9)
return dict(underruns=underruns, worst_late=worst_late, gaps=gaps,
noidle=noidle,
gap_tot=gap_tot, gap_max=gap_max, busy=busy, span=span,
ceiling=max(slack_series), mean_slack=float(np.mean(slack_series)),
full=full_refusals, prefill_s=prefill_done)
ap = argparse.ArgumentParser()
ap.add_argument("container")
ap.add_argument("--kbps", type=float, nargs="+", required=True,
help="delivery rates to model. REQUIRED: this tree has no "
"default rate (FINDINGS 50)")
ap.add_argument("--ring", type=int, default=256, help="ring size, KB")
ap.add_argument("--qdepth", type=int, nargs="+", default=[1, 2],
help="requests the player may have outstanding")
ap.add_argument("--prefill", type=int, default=2, help="records before release")
ap.add_argument("--cadence", choices=["raster", "nominal"], default="raster")
a = ap.parse_args()
d = DLX(a.container)
if not d.has_index:
sys.exit(f"{a.container} is DLX{d.version}: this model reads the record "
f"index the player reads, and only DLX4 carries one.")
rec = np.array([q * 4 for q in d.index], np.int64)
dec = frame_costs(d) + CLK_ISR * (HFREQ / VTOTAL) / d.fps # + the clock's own
if a.cadence == "raster":
tick = ticks(d.nframes + 1, d.fps)
else:
tick = np.arange(d.nframes + 1) / d.fps
wire = rec.mean() * d.fps / 1024
print(f"{a.container}: {d.nframes} records, {rec.mean()/1024:.1f} KB mean, "
f"{rec.max()/1024:.1f} KB max, wire {wire:.1f} KB/s")
print(f" index: {2*d.nframes:,} B of scene header -- read, not walked")
print(f" decode: mean {dec.mean():,.0f} clk/frame ({100*dec.mean()/(CPUHZ/d.fps):.1f}% "
f"of a mean slot), p90 {np.percentile(dec,90):,.0f}")
print(f" cadence: {a.cadence}"
+ (" (4 or 5 refreshes a frame, 72.13/90.16 ms -- FINDINGS 54.4)"
if a.cadence == "raster" else " (uniform 1/fps slots)"))
print(f" ring {a.ring} KB, prefill {a.prefill} records\n")
hdr = (f"{'pipe':>8} {'Q':>2} {'idle':>9} {'gaps':>5} {'worst':>8} "
f"{'under':>7} {'late by':>8} {'noidl':>5} {'ceil':>5} {'mean':>5} "
f"{'refus':>6}")
print(hdr)
print("-" * len(hdr))
for kb in a.kbps:
bps = (kb - AUDIO_KBPS) * 1024
for q in a.qdepth:
r = simulate(rec, dec, tick, bps, a.ring * 1024, q, a.prefill)
print(f"{kb:8.0f} {q:2d} {100*r['gap_tot']/r['span']:8.1f}% "
f"{r['gaps']:5d} {r['gap_max']*1000:7.1f}ms "
f"{r['underruns']:3d}/{d.nframes:<3d} {r['worst_late']*1000:7.1f}ms "
f"{r['noidle']:5d} {r['ceiling']:5d} {r['mean_slack']:5.1f} "
f"{r['full']:6d}")
print()
print("idle = the channel with no request to work on, as a fraction of the")
print(" window. Bytes the medium could have delivered and did not.")
print("under = frames whose record was not resident when the decoder asked")
print(" for it. The PIPE.")
print("noidl = frames that reached the gate after their tick, because the one")
print(" before used its whole slot. The CPU, and 54.4's cadence.")
print("ceil = most records resident and unconsumed at a frame start: what a")
print(" branch point could spend, minus one for the restart (51.2).")
print("refus = placements refused for SPACE. Nonzero means the ring filled.")
tools/analysis/25_scene_graph.py
+363
View File
@@ -0,0 +1,363 @@
"""The worst gap between two decision points, out of the scene graph (G1).
FINDINGS 51.3 measured that a ring's lookahead is ACCUMULATED out of
`pipe - wire` and that a seek spends all of it, so what a branch point costs is
set by the rate and by the time since the last branch. 55.5 rehearsed a seek on
the machine and could not ask the question that matters, because nothing in this
tree knew where the branch points ARE:
what is the WORST gap, in seconds of play, between two consecutive
decision points, and does the refill climb survive it?
Only the arcade scene graph knows. This answers it in the currency 51.3
established.
python3 tools/import/scenegraph.py # writes tmp/scenegraph.json
python3 tools/analysis/25_scene_graph.py --kbps R [R ...] [--ring KB [KB ...]]
`--kbps` is REQUIRED and takes no default, for the reason FINDINGS 50 gives.
THIS FILE KNOWS NOTHING ABOUT WHERE THE TABLE CAME FROM, deliberately. It reads
`DLXSCENE1`, which is this project's own schema; `tools/import/scenegraph.py` is
the single file in the tree that knows anything about the outside projects the
table is built from, and it carries their attribution. Nothing is vendored.
"""
import sys, os, json, argparse, importlib.util
sys.path.insert(0, "tools/encoder")
TABLE = os.environ.get("DLX_SCENEGRAPH", "tmp/scenegraph.json")
LD_FPS = 23.976 # the medium's frame rate; one frame is the comparison floor
# ------------------------------------------------------------------- graph
class Node:
"""One (scene, sequence): a clip, its exits, and whether entering it seeks."""
def __init__(self, scene, name, seq):
self.scene, self.name, self.seq = scene, name, seq
self.start = seq["start_ms"] # ms, or -1 for no seek
self.seeks = self.start >= 0
self.timeout_ms = seq["timeout_ms"]
self.exits = [(seq["timeout_ms"], "timeout", seq["timeout_next"])]
for a in seq["actions"]:
# The player may press as early as `from_ms`, so that is the least
# play this clip can deliver before the branch it leads to.
self.exits.append((a["from_ms"], "action", a["next"]))
@property
def key(self):
return f"{self.scene}.{self.name}"
def build_graph(scenes):
nodes = {}
for scene, seqs in scenes.items():
for name, seq in seqs.items():
n = Node(scene, name, seq)
nodes[n.key] = n
return nodes
def worst_gap(nodes):
"""Least play time, in ms, between one seek and the next.
A seek is entering a sequence whose start is >= 0; a negative start means
the disc keeps playing, so play ACCUMULATES across such sequences and the
gap is a shortest path over them. Bellman-Ford rather than Dijkstra
because a zero-length timeout is common (`start_alive` chains) and the
graph has cycles; all weights are non-negative so it terminates.
A null exit ends the scene: the player moves to another scene entirely,
which is a seek AND a container change (FINDINGS 53), so it counts as a
seek and is flagged.
"""
INF = float("inf")
dist = {k: (0.0 if n.seeks else INF) for k, n in nodes.items()}
for _ in range(len(nodes) + 1):
changed = False
for n in nodes.values():
if dist[n.key] == INF:
continue
for elapsed, kind, tgt in n.exits:
if tgt is None:
continue
tk = f"{n.scene}.{tgt}"
if tk not in nodes:
continue
d = dist[n.key] + max(0.0, float(elapsed))
if not nodes[tk].seeks and d < dist[tk] - 1e-9:
dist[tk], changed = d, True
if not changed:
break
best = {}
for n in nodes.values():
if dist[n.key] == INF:
continue
for elapsed, kind, tgt in n.exits:
tk = f"{n.scene}.{tgt}" if tgt is not None else None
ends_scene = tgt is None
if ends_scene or (tk in nodes and nodes[tk].seeks):
g = dist[n.key] + max(0.0, float(elapsed))
# One entry per (source, destination): several input windows can
# lead to the same clip and only the earliest of them binds.
k = (n.key, tgt)
if k not in best or g < best[k][0]:
best[k] = (g, n.key, tgt if tgt else "<end of scene>",
kind, ends_scene, n.scene)
return sorted(best.values()), dist
# ------------------------------------------------------ 51.3's currency
def slack_model(gap_s, kbps, wire_kbps, mean_rec_b):
"""Records of lookahead accrued in `gap_s` seconds of play at `kbps`, and
the seconds one record of lookahead costs.
This is 51.3's surplus model and nothing more: slack accrues at
(pipe - wire) bytes per second. The paced rig is the measurement; this
says whether the gap is even in the right order of magnitude.
"""
surplus = (kbps - wire_kbps) * 1024.0
if surplus <= 0:
return 0.0, float("inf")
return surplus * gap_s / mean_rec_b, mean_rec_b / surplus
def main():
ap = argparse.ArgumentParser()
ap.add_argument("--kbps", type=float, nargs="+", required=True,
help="delivered pipe rates, KB/s. REQUIRED, no default "
"(FINDINGS 50).")
ap.add_argument("--table", default=TABLE, help="DLXSCENE1 scene table")
ap.add_argument("--container", default="tmp/rc_fr_singe_scsi_span.dlx",
help="container the wire demand and mean record come from")
ap.add_argument("--ring", type=float, nargs="+", default=[256, 512])
ap.add_argument("--top", type=int, default=12)
a = ap.parse_args()
if not os.path.exists(a.table):
print(f"no scene table at {a.table} -- run:\n"
f" python3 tools/import/scenegraph.py")
return 2
doc = json.load(open(a.table))
if doc.get("format") != "DLXSCENE1":
print(f"{a.table}: not a DLXSCENE1 table")
return 2
nodes = build_graph(doc["scenes"])
c = doc["counts"]
nseek = sum(1 for n in nodes.values() if n.seeks)
print(f"=== the scene graph ({a.table})")
for s in doc["sources"]:
print(f" from {s['name']} ({s['licence']}, {s['holder']}): {s['role']}")
print(f" scenes {c['scenes']}, sequences {c['sequences']}, "
f"input windows {c['windows']}")
print(f" sequences entered by a SEEK: {nseek}/{c['sequences']} "
f"({100*nseek/c['sequences']:.1f}%); the rest play on from where the "
f"disc is")
print(f" scene order: {len(doc['rows'])} rows x {len(doc['rows'][0])}")
# Gates. A parser that quietly dropped a branch would produce a SMALLER
# graph and a LONGER worst gap -- it would fail in the flattering direction.
assert c["sequences"] == 516, f"expected 516 sequences, got {c['sequences']}"
assert c["windows"] == 906, f"expected 906 input windows, got {c['windows']}"
# ---- the measurement
gaps, dist = worst_gap(nodes)
play = [g for g in gaps if g[5] != "attract_mode"]
zero = [g for g in play if g[0] <= 1e-9]
print()
print("=== the worst gap between two consecutive decision points")
print(" A 'gap' is the LEAST play time the disc delivers between one seek")
print(" and the next: the earliest an input window opens, chained across")
print(" sequences the disc plays through without seeking. One entry per")
print(" (source, destination); attract mode is excluded and reported")
print(" separately, because nothing branches there under a 12 fps budget.")
print(f" {'gap s':>7} from -> to")
for g, src, tgt, kind, ends, scene in play[:a.top]:
print(f" {g/1000:>7.3f} {src} -{kind}-> {tgt}"
+ (" [SCENE CHANGE]" if ends else ""))
sc_gaps = [g for g in play if g[4]]
print(f" ... {len(play)} distinct transitions into a seek "
f"({len(gaps)-len(play)} more in attract mode)")
worst = play[0][0] / 1000.0
med = play[len(play) // 2][0] / 1000.0
print(f" WORST {worst:.3f} s, median {med:.3f} s, "
f"best {play[-1][0]/1000:.3f} s")
print(f" SCENE CHANGES specifically ({len(sc_gaps)} of them, and each also")
print(f" needs a header before its frame 0, FINDINGS 53/55.1): worst "
f"{sc_gaps[0][0]/1000:.3f} s, median "
f"{sc_gaps[len(sc_gaps)//2][0]/1000:.3f} s")
print(f" ZERO-PLAY BRANCHES: {len(zero)} of {len(play)} "
f"({100*len(zero)/len(play):.1f}%) open an input window at t=0 of a")
print(" clip the disc SEEKED to, so two seeks can fall back to back with no")
print(" play between them at all. A rule of the form 'has there been")
print(" enough play since the last branch' (51.2's ring_may_seek) can be")
print(" answered NO by the content, not by the buffer.")
# ---- what the input layer has to survive, from the same table
inputs, windows = {}, []
for n in nodes.values():
for x in n.seq["actions"]:
inputs[x["input"]] = inputs.get(x["input"], 0) + 1
windows.append(x["to_ms"] - x["from_ms"])
windows.sort()
print()
print("=== what the input layer has to survive")
print(" " + ", ".join(f"{k} {v}" for k, v in
sorted(inputs.items(), key=lambda x: -x[1])))
diag = sum(v for k, v in inputs.items()
if k in ("upleft", "upright", "downleft", "downright"))
print(f" diagonals are {diag} windows of {len(windows)}: rare enough for a "
f"port to drop\n and not droppable by one aiming at the arcade")
print(f" window length: shortest {windows[0]:.0f} ms "
f"({windows[0]/(1000/12):.2f} frame slots at 12 fps), p10 "
f"{windows[len(windows)//10]:.0f} ms, median "
f"{windows[len(windows)//2]:.0f} ms")
print(" the floor is one to two frames wide (54.4: a slot is 72.13 or")
print(" 90.16 ms, never 83.33), so input cannot be polled on the frame tick")
# ---- 51.3's currency
if os.path.exists(a.container):
spec = importlib.util.spec_from_file_location(
"seek_slack", "tools/analysis/20_seek_slack.py")
m = importlib.util.module_from_spec(spec)
spec.loader.exec_module(m)
import ratectl as RC
d, rec = m.records(a.container)
mean_rec = float(rec.mean())
wire = mean_rec * 12 / 1024 + RC.AUDIO_KBPS
gs = [g[0] / 1000.0 for g in play]
print()
print(f"=== what that gap buys, at explicit rates "
f"({os.path.basename(a.container)}: mean record "
f"{mean_rec/1024:.1f} KB, wire {wire:.1f} KB/s)")
print(f" {'ring KB':>8} {'pipe':>7} {'ceiling':>8} {'climb s':>8} "
f"{'median gap':>11} {'accrued':>8} {'under climb':>12}")
for ring_kb in a.ring:
ring = int(ring_kb * 1024)
for kbps in a.kbps:
fill = (kbps - RC.AUDIO_KBPS) * 1024 / 12
lo, hi, ring_ref, rate_ref = m.paced_sim(rec, ring, fill)
ceiling = int(hi.max())
accrued, per_rec = slack_model(med, kbps, wire, mean_rec)
climb = ceiling * per_rec
under = sum(1 for g in gs if g < climb)
print(f" {ring_kb:>8.0f} {kbps:>7.1f} {ceiling:>8} "
f"{climb:>8.2f} {med:>11.3f} {accrued:>8.2f} "
f"{f'{under}/{len(gs)}':>12} {100*under/len(gs):.0f}%")
# What a branch costs when the gap before it bought nothing. The ring
# is empty after a seek and the decoder is released at the prefill depth
# (55.4's shipped policy is 2 records), so this is the stall the player
# eats every time -- not the climb to the ceiling, which is what it
# needs in order to TOLERATE the next one. A scene change additionally
# needs its header before frame 0; 22_scene_load.py prices that case
# properly, clocks included.
PREFILL_REC, HDR_B = 2, 6164
print()
print(f" A branch taken on an empty ring, at the shipped prefill of "
f"{PREFILL_REC} records:")
for kbps in a.kbps:
b = PREFILL_REC * mean_rec
ms = b / (kbps * 1024) * 1000
hms = (b + HDR_B) / (kbps * 1024) * 1000
print(f" {kbps:>7.1f} KB/s: {ms:>7.1f} ms "
f"({ms/(1000/12):.2f} frame slots), and {hms:>7.1f} ms "
f"({hms/(1000/12):.2f}) if it is a scene change carrying "
f"{HDR_B:,} header bytes")
print()
print(" 'climb s' is 51.3's: seconds of play to refill from empty to")
print(" the ceiling. 'under climb' is how many of this game's own")
print(" branch points arrive sooner than that, i.e. are reached with")
print(" LESS lookahead than the one before them. The worst gap is")
print(f" {worst:.3f} s and buys nothing at any rate in this table.")
else:
print(f"\n{a.container}: MISSING -- rate half skipped")
# ---- the cross-check, and what it is worth
print()
print("=== the second table, and why it is not a second transcription")
cross, meta = doc.get("crosscheck"), doc.get("crosscheck_meta")
if not cross:
print(" none in this table -- the import ran without it")
return 0
print(f" {meta['chapters']} chapters, {meta['scenes_mapped']} scenes")
print(f" PROVENANCE: {meta['provenance']}")
print(" FINDINGS 16 planned to diff two INDEPENDENT transcriptions to")
print(" catch transcription errors. There is only one transcription.")
print(" This diff catches CONVERSION errors and nothing more.")
FRAME_MS = 1000.0 / LD_FPS
offs, durs, same, diff, missing = [], [], 0, 0, 0
renames, inputs_differ, examples = 0, [], []
for scene, seqs in sorted(cross.items()):
for seq, ch in sorted(seqs.items()):
n = nodes.get(f"{scene}.{seq}")
if n is None:
missing += 1
continue
if n.seeks:
offs.append(ch["start_ms"] - n.start)
durs.append(((ch["end_ms"] - ch["start_ms"]) - n.timeout_ms,
ch["chapter"]))
ce = sorted((x["input"], x["next"]) for x in ch["actions"])
de = sorted((x["input"], x["next"]) for x in n.seq["actions"])
if ce == de:
same += 1
continue
diff += 1
ci = sorted(i for i, _ in ce)
di = sorted(i for i, _ in de)
if ci != di:
inputs_differ.append((ch["chapter"], ci, di))
else:
renames += 1
if len(examples) < 3:
examples.append((ch["chapter"], ce, de))
if offs:
offs.sort()
print(f" START TIMES are on different timelines and do not compare: "
f"{len(offs)} seeking chapters,")
print(f" offset spread {min(offs)/1000:,.1f} s .. "
f"{max(offs)/1000:,.1f} s, median {offs[len(offs)//2]/1000:,.1f} s"
f" -- not a constant, and not even one sign.")
if durs:
dd = sorted(x for x, _ in durs)
agree = sum(1 for x in dd if abs(x) <= FRAME_MS)
print(f" DURATIONS compare (offset-invariant): {agree}/{len(dd)} "
f"within one frame of the medium ({100*agree/len(dd):.1f}%), "
f"median {dd[len(dd)//2]:,.0f} ms")
for x, ch in sorted(durs, key=lambda x: -abs(x[0]))[:3]:
print(f" widest {ch}: {x/1000:+.3f} s")
if same + diff:
print(f" BRANCH STRUCTURE compares: {same}/{same+diff} chapters have "
f"the identical set of (input -> target) edges "
f"({100*same/(same+diff):.1f}%)")
if diff:
print(f" {renames} of the {diff} differ only in what a target "
f"sequence is NAMED")
DIAG = ("upleft", "upright", "downleft", "downright")
lost = [ch for ch, ci, di in inputs_differ
if set(di) - set(ci) and all(x in DIAG for x in set(di) - set(ci))]
print(f" {len(inputs_differ)} differ in the INPUT SET, and "
f"{len(lost)} of those are the other table dropping the arcade's")
print(" DIAGONALS: a controller decision, not a transcription "
"difference.")
for ch, ci, di in inputs_differ:
if ch not in lost:
print(f" the remaining one: {ch}, {ci} vs {di}")
for ch, ce, de in examples:
print(f" e.g. {ch}\n cross {ce}\n graph {de}")
if missing:
print(f" {missing} chapters have no sequence in the graph "
f"(the other project added them)")
return 0
if __name__ == "__main__":
sys.exit(main())
tools/analysis/26_sector_align.py
+150
View File
@@ -0,0 +1,150 @@
"""A record is not a sector: what the mismatch costs, three ways (P4b, 58.3).
src/player/ring.i asks the transport for a RECORD -- a byte offset into the
scene's frame stream and a length, both 4-byte aligned because that is what
`move.l (a0)+` needs (28.3) and neither of them a multiple of 512. A SCSI
target answers in 512 B BLOCKS. On the gate container 117 of 120 records start
part way into a sector, so something has to reconcile the two, and the three
ways of doing it are not close.
WHY IT IS NOT AN IMPLEMENTATION DETAIL. The bytes on either side of a record in
the stream belong to OTHER records -- ones the decoder may still be reading --
and the block loop walks a0 with no bounds check at all (49.2). So a transport
that reads whole sectors straight into the ring does not waste 500 bytes, it
CORRUPTS the neighbours, and the symptom is wrong pixels rather than a fault.
A. WINDOWED PIO. Read the sectors the record lies in, store only the record.
src/player/scsi.i does this and it is what FINDINGS 58 measured. It costs
nothing in clocks -- the CPU is touching every byte anyway -- and it costs
the extra sectors on the wire. It CANNOT be done by a DMAC: a channel
writes a contiguous run to a contiguous address and cannot be told to drop
the first 300 bytes.
B. BOUNCE BUFFER. Let the DMAC write whole sectors somewhere else, then copy
the record into the ring. Works under DMA, and costs a copy of every
delivered byte -- which is precisely the cost `aligned` was chosen over
`split` to avoid (49.3, 19_ring_stream.py).
C. SECTOR-ALIGNED RECORDS. Pad each record up to 512 in the container
instead of up to 4. Costs bytes on the disc and in every delivery, and
nothing else at all; the transport becomes a whole-sector read into the
ring with no window and no copy. It is a CONTAINER change -- a re-encode
and a re-measurement of every constant fitted to the gate container, which
is the class of change ROADMAP already has bundled with P2's other half.
python3 tools/analysis/26_sector_align.py <in.dlx> [--ring KB]
No rate is taken and none is needed: every figure here is a fraction of the
delivered bytes or a count of clocks, and both are rate-free. What a given
delivery rate does with them is 15_bus_occupancy.py's question.
"""
import sys, os, argparse
sys.path.insert(0, "tools/encoder")
from dlx import DLX
SECTOR = 512
CPUHZ = 10_000_000
# 5.0 clocks/byte, and it is 19_ring_stream.py's constant rather than a new one:
# a 68000 `move.l (a0)+,(a1)+` moves 4 bytes in 20 clocks on a 16-bit bus. It
# is the OPTIMISTIC figure there and it is the optimistic figure here.
COPY_CLK_PER_BYTE = 5.0
# The windowed PIO loop in src/player/scsi.i, from the 68000's cycle table:
# 12 move.l #SC_PATIENCE,d3 patience reload
# 16 move.b SC_SSTS,d0 (xxx).L -> Dn
# 10 btst #0,d0
# 10 beq.s taken
# 20 move.b SC_DREG,(a1)+ (xxx).L -> (An)+
# 8 subq.l #1,d7
# 10 bne.s taken
# FINDINGS 58.2 measured 87.28 clocks per delivered byte against this loop's 86
# plus 1.15 for the dropped window bytes -- 0.2% apart, which is what says the
# cost is the instruction stream and not MAME's device model.
PIO_CLK_PER_BYTE = 86.0
ap = argparse.ArgumentParser()
ap.add_argument("container")
ap.add_argument("--ring", type=int, default=256, help="ring size in KB")
a = ap.parse_args()
d = DLX(a.container)
# The disc layout the 68000 walks: [u32 len][body], each record padded up to the
# container's own alignment -- 4 on DLX2/3/4, 512 on DLX5. Exactly
# tools/bench/prep_stream.py's, and it comes from the reader rather than from a
# second copy of the rule here, so pointing this tool at a DLX5 container asks
# it the RIGHT question: what does the mismatch still cost once the container
# has been changed to remove it? (The answer had better be nothing.)
off, recs = 0, []
for ln in d.record_lengths():
recs.append((off, ln))
off += ln
# The DENOMINATOR is the record bytes the decoder actually reads -- [u32 len]
# plus payload -- and NOT the padded length, because on a DLX5 container the
# padding IS the cost being measured. Scoring against the padded length would
# make an already-aligned container report +0.00% and look free.
payload = sum(4 + n for _, n in d.frames)
nfr = len(recs)
budget = CPUHZ / d.fps
print(f"{a.container}: {nfr} records, {payload:,} B, {d.fps} fps")
print(f" mean record {payload/nfr:,.0f} B; a {d.fps} fps frame is "
f"{budget:,.0f} clocks")
aligned0 = sum(1 for o, _ in recs if o % SECTOR == 0)
print(f" records that already start on a sector boundary: {aligned0}/{nfr}")
print()
# ---- A. windowed PIO: the sectors the record lies in, and only the record kept
wire_a = sum(((o % SECTOR) + ln + SECTOR - 1) // SECTOR for o, ln in recs) * SECTOR
drop_a = wire_a - payload
print("A. WINDOWED PIO (src/player/scsi.i, what FINDINGS 58 ran)")
print(f" wire {wire_a:,} B for {payload:,} B of record "
f"= +{100*drop_a/payload:.2f}%")
print(f" clocks {PIO_CLK_PER_BYTE:.0f}/B on EVERY byte off the FIFO, "
f"dropped ones included:")
print(f" {PIO_CLK_PER_BYTE*wire_a/nfr:,.0f} clk/frame "
f"= {100*PIO_CLK_PER_BYTE*wire_a/nfr/budget:.0f}% of the frame")
print( " and it does not survive the move to the DMAC at all: a channel "
"cannot drop bytes.")
print()
# ---- B. bounce buffer: DMA whole sectors elsewhere, copy the record in
print("B. BOUNCE BUFFER (whole sectors by DMA, then a copy)")
print(f" wire {wire_a:,} B, the same +{100*drop_a/payload:.2f}% -- the "
f"command is identical")
print(f" clocks {COPY_CLK_PER_BYTE:g}/B of copy on every DELIVERED byte, "
f"on top of whatever W the")
print(f" channel steals: {COPY_CLK_PER_BYTE*payload/nfr:,.0f} clk/frame "
f"= {100*COPY_CLK_PER_BYTE*payload/nfr/budget:.1f}% of the frame")
print( " which is the cost `aligned` was chosen over `split` to avoid "
"(49.3), arriving")
print( " by a different door and on every byte instead of on a wrap.")
print()
# ---- C. sector-aligned records in the container
cur, pad = 0, 0
for _, ln in recs:
if cur % SECTOR:
pad += SECTOR - (cur % SECTOR)
cur += SECTOR - (cur % SECTOR)
cur += ln
print("C. SECTOR-ALIGNED RECORDS (a container change; a re-encode)"
+ (" -- THIS CONTAINER ALREADY IS ONE" if d.sector_aligned else ""))
print(f" wire {cur:,} B for {payload:,} B of record = "
f"+{100*(cur-payload)/payload:.2f}%")
print( " clocks ZERO: the read is a whole-sector read straight into the "
"ring, no window,")
print( " no copy, and the DMAC can do it.")
print()
ringsz = a.ring * 1024
print(f" VERDICT, in the currency this project prices delivery in. C is "
f"cheaper on the wire")
print(f" than A and B by {100*(wire_a-cur)/payload:.2f} points of the payload "
f"({wire_a-cur:,} B on this scene),")
print(f" and it is the only one of the three a DMA channel can run without a "
f"copy. What it")
print(f" costs is a container revision and the re-measurement that comes with "
f"one.")
maxrec = max(ln for _, ln in recs)
maxpad = maxrec + (-maxrec) % SECTOR
print(f" It also grows the largest record from {maxrec:,} to {maxpad:,} B, "
f"which a {a.ring} KB")
print(f" ring still holds {ringsz//maxpad} times over.")
tools/analysis/27_dmac_config.py
+111
View File
@@ -0,0 +1,111 @@
#!/usr/bin/env python3
"""What the PLAYER programs into the DMAC -- read out of the assembler source.
python3 tools/analysis/27_dmac_config.py [src/player/dma.i]
ROADMAP P4a asks for a DMAC configuration that HOLDS THE BUS, and the whole
weight of the claim is in four register bytes. tools/analysis/21_iplrom_dmac.py
already reads the IPL ROM's four channels the same way, out of the shipping
image, and found Sharp's own disk channel at 16..19 clocks a byte (FINDINGS
52.5) -- above the entire bracket this project costs P4 in. This is the other
half of that comparison: the same MC68450 field tables (tools/analysis/
mc68450.py, one copy) applied to the bytes src/player/dma.i actually programs.
IT PARSES THE SOURCE RATHER THAN RESTATING IT. A constant typed into this file
would be a claim about the player that the player could quietly stop honouring;
the equates are read out of src/player/dma.i, so a change there changes what is
printed here and a mismatch between the two is not expressible.
IT IS A GATE. Each configuration is checked against what it is FOR -- held
must decode as a mode that keeps the bus, stealing must decode as one that does
not -- and a disagreement exits non-zero rather than printing a paragraph.
WHAT IT IS NOT: a rate. Nothing here is a measurement of anything. It says
which mode the player asks the chip for; tools/bench/dma_run.sh shows the
machine doing it, and `W` -- the clocks it costs on real silicon -- remains the
project's largest open number (ROADMAP B1/B3).
"""
import sys, os, re
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from mc68450 import dcr, ocr, scr, XRM, DTYP, REQG
src = sys.argv[1] if len(sys.argv) > 1 else "src/player/dma.i"
if not os.path.exists(src):
sys.exit(f"missing {src} -- run from the repo root.")
text = open(src).read()
def equ(name):
m = re.search(rf"^{name}\s*=\s*\$([0-9A-Fa-f]+)", text, re.M)
if not m:
sys.exit(f"{src} no longer defines {name}. This script reads the "
f"player's own equates; it does not keep a copy of them.")
return int(m.group(1), 16)
# The SCR the channel is given is written inline rather than equated, because it
# is the same for both configurations and there is nothing to choose about it.
m = re.search(r"move\.b\s+#\$([0-9A-Fa-f]+),DM_SCR", text)
if not m:
sys.exit(f"{src} no longer writes DM_SCR with a literal.")
SCR = int(m.group(1), 16)
CFG = [("BUS HELD", "DM_HELD_DCR", "DM_HELD_OCR"),
("CYCLE STEALING", "DM_STEAL_DCR", "DM_STEAL_OCR")]
print(f"WHAT src/player/dma.i PROGRAMS -- decoded from {src}\n")
bad = 0
for label, dn, on in CFG:
D, O = equ(dn), equ(on)
print(f" {label} DCR = ${D:02X} OCR = ${O:02X} SCR = ${SCR:02X}")
for line in dcr(D):
print(f" {line}")
for line in ocr(O):
print(f" {line}")
for line in scr(SCR):
print(f" {line}")
holds = (D >> 6 & 3) in (0, 3) # burst, or cycle steal WITH hold
dual = (D >> 4 & 3) in (0, 1)
tomem = bool(O & 0x80)
checks = [
(dual, "DTYP must be explicitly addressed: only channel 0 has device "
"callbacks in this machine, so an implicit-address DTYP on "
"channel 1 falls through to the dual-address path anyway"),
(tomem, "OCR DIR must be device -> memory; this is a READ"),
((O >> 4 & 3) == 0, "OCR SIZE must be byte: the SPC's port is 8 bits"),
((O >> 2 & 3) == 0, "OCR CHAIN must be none until P5a picks a chaining "
"scheme for the two-deep request queue (FINDINGS 55.3)"),
((SCR & 3) == 0, "SCR DAC must not count: the device address is a "
"REGISTER at $EA0015 and must not walk off it"),
((SCR >> 2 & 3) == 1, "SCR MAC must increment: the record is contiguous"),
((O & 3) in (0, 1), "OCR REQG must be an AUTO-request mode: the "
"expansion slot has no request line to the DMAC in "
"this machine, so external request cannot be run"),
]
if label == "BUS HELD":
checks.append((holds, "the held configuration must decode as a mode "
"that KEEPS the bus between operands"))
checks.append(((O & 3) == 1, "and as max-rate auto-request: MAME models "
"a held bus only for burst + REQG 01"))
else:
checks.append((not holds, "the stealing configuration must decode as a "
"mode that RELEASES the bus between operands "
"-- otherwise the two have no contrast"))
for ok, why in checks:
if not ok:
print(f" FAIL: {why}")
bad += 1
print()
print("""AGAINST THE MACHINE'S OWN DISK CHANNEL (21_iplrom_dmac.py, FINDINGS 52.5)
IPL ROM ch1, SASI DCR $80 OCR $B2 dual address, 8-bit port, cycle steal
WITHOUT hold, EXTERNAL request
-> a full arbitration per byte, 16..19
player, held DCR $00 OCR $81 dual address, 8-bit port, BURST,
auto-request at max rate
-> the ladder's dual-address held row, 9
Sharp's own configuration and the player's differ in exactly the field that
decides the project. That is 52.5's finding read the other way round: a cheaper
configuration IS reachable for an explicitly-addressed 8-bit port, and what it
costs on real silicon is still ROADMAP B3's question and not this file's.""")
sys.exit(1 if bad else 0)
tools/analysis/28_autorequest_cost.py
+172
View File
@@ -0,0 +1,172 @@
#!/usr/bin/env python3
"""What AUTO-REQUEST DMA costs the 68000, when there is no request line.
python3 tools/analysis/28_autorequest_cost.py --kbps 460 [--record 37405]
WHY THIS EXISTS. The project's per-byte ladder -- W = 5 single-address held, 9
dual held, 12 single arbitrated, 16..19 dual arbitrated (FINDINGS 42.4, 52.5) --
prices a transfer that the DEVICE asks for: one external request, one operand,
a known number of stolen clocks per delivered byte. Session 27 found that the
CZ-6BS1 as MAME models it has NO REQUEST LINE to the DMAC at all (FINDINGS
59.2): the card's flow control is DTACK, and every configuration that can be run
against it is AUTO-REQUEST, where the channel transfers because its own counter
says so and not because a byte has arrived.
THAT CHANGES THE CURRENCY, and it is the reason this file is not a line in
another one. An externally requested transfer is charged PER DELIVERED BYTE.
An auto-requested one is charged PER UNIT OF TIME THE CHANNEL IS ACTIVE, because
the channel has no way to know the device is not ready: it takes its allotted
share of the bus and spends it whether or not a byte comes back. So the cost of
delivering a record depends on HOW LONG THE RECORD TAKES TO ARRIVE -- i.e. on
the delivery rate, the figure this tree deliberately has no default for (FINDINGS
50) -- and the tool REQUIRES one rather than assuming it.
SOURCED: MC68450 Direct Memory Access Controller, Motorola, Jul 1989
(bitsavers), sections 3.8 and 5.2.3.3, the same document buscost.py's transfer
timings come from. Section 5.2.3.3.1: under maximum-rate auto-request "all
operands in the data block will be transferred in one burst, so that the DMAC
will use 100% of the available bus bandwidth" -- which is the datasheet saying,
in its own words, what session 27 measured MAME's model doing when it HALTED the
68000 for the whole data phase (FINDINGS 59.1).
THE ONE LOAD-BEARING ASSUMPTION, stated because the whole table rests on it:
that the channel SPENDS its allotted share whether or not the device has a byte.
Under auto-request a request is pending until MTC is exhausted, so the DMAC
takes the bus during every burst window it is entitled to; when the device is
not ready the cycle is stretched by wait states (a real CZ-6BS1 negating DTACK)
or retried later (MAME's model discards the operand), and either way the window
is gone from the CPU's point of view. If a real card instead lets the DMAC off
the bus early when no byte is there, these figures are UPPER BOUNDS. That is a
board question and it is ROADMAP B3's.
NOT A MEASUREMENT. Every figure below is arithmetic over datasheet constants
and an explicit rate. `W` is still unmeasured and still wants a board.
"""
import sys, os, argparse
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from buscost import DMA_DUAL_BYTE_CLK, DMA_READ_CLK, DMA_WRITE_CLK
CPU_HZ = 10_000_000.0 # the X68000 the whole tree is costed against
FPS = 12.0
ap = argparse.ArgumentParser()
ap.add_argument("--kbps", type=float, required=True,
help="delivery rate in KB/s. REQUIRED: this tree has no default "
"rate and the whole answer scales with it (FINDINGS 50).")
ap.add_argument("--record", type=int, default=37405,
help="mean record size in bytes (default: the gate container's)")
a = ap.parse_args()
RATE = a.kbps * 1024.0
# --- 3.8 GENERAL CONTROL REGISTER, decoded from the formulas in 5.2.3.3.2 ---
# burst time = 2^(BT + 4) clocks
# sample period = 2^(BT + BR + 5) clocks
# DMAC's share = 2^-(BR + 1)
# and Table 5-3 prints all sixteen combinations, so the formulas are GATED
# against the table rather than trusted.
TABLE = { # (BR, BT): (burst, MPU period, share, sample period)
(0,0):(16,16,.5,32), (0,1):(32,32,.5,64), (0,2):(64,64,.5,128), (0,3):(128,128,.5,256),
(1,0):(16,48,.25,64), (1,1):(32,96,.25,128), (1,2):(64,192,.25,256),(1,3):(128,384,.25,512),
(2,0):(16,112,.125,128),(2,1):(32,224,.125,256),(2,2):(64,448,.125,512),(2,3):(128,896,.125,1024),
(3,0):(16,240,.0625,256),(3,1):(32,480,.0625,512),(3,2):(64,960,.0625,1024),
(3,3):(128,1920,.0625,2048),
}
bad = 0
for (br, bt), (burst, mpu, share, sample) in sorted(TABLE.items()):
f_burst, f_sample, f_share = 2**(bt+4), 2**(bt+br+5), 2.0**-(br+1)
for got, want, what in ((f_burst, burst, "burst time"),
(f_sample, sample, "sample period"),
(f_share, share, "bandwidth share"),
(f_sample - f_burst, mpu, "MPU period")):
if got != want:
print(f" FAIL BR={br:02b} BT={bt:02b} {what}: formula {got}, "
f"Table 5-3 {want}")
bad += 1
if bad:
sys.exit(f"\n{bad} disagreements between 5.2.3.3.2's formulas and Table 5-3. "
"Everything below\nis those formulas, so it is not printed.")
print(f"MC68450 5.2.3.3.2's formulas reproduce all 16 rows of Table 5-3.\n")
BYTE_CLK = DMA_DUAL_BYTE_CLK # dual address, 8-bit port: a 4-clock read of
# $EA0015 and a 5-clock write to the ring
frame_clk = CPU_HZ / FPS
wire_s = a.record / RATE # how long the record takes to land
wire_clk = wire_s * CPU_HZ # ...in 68000 clocks
per_byte_wire = wire_clk / a.record # clocks of wall time per byte
print(f"THE RECORD: {a.record:,} B at {a.kbps:g} KB/s = {wire_s*1000:.2f} ms "
f"= {wire_clk:,.0f} clocks = {100*wire_clk/frame_clk:.1f}% of a "
f"{FPS:g} fps frame")
print(f" one byte of WIRE TIME is {per_byte_wire:.2f} clocks; one byte of DMAC "
f"WORK is {BYTE_CLK} ({DMA_READ_CLK} read + {DMA_WRITE_CLK} write, "
f"buscost.py)\n")
print("REQG 01, AUTO-REQUEST AT MAXIMUM RATE -- what session 27 demonstrated")
print(f" The channel holds the bus until MTC is exhausted (5.2.3.3.1: 100% of "
f"the\n bandwidth), so the CPU gets NOTHING for the whole delivery:")
print(f" cost to the 68000 = the whole {100*wire_clk/frame_clk:.1f}% of a "
f"frame, or {per_byte_wire:.2f} clk/B")
print(f" It is the cheapest configuration per BYTE MOVED and the dearest per "
f"byte\n DELIVERED, and the gap between those is the device's own "
f"slowness:\n {BYTE_CLK} clocks of work in {per_byte_wire:.1f} clocks "
f"of waiting = {100*BYTE_CLK/per_byte_wire:.1f}% of the held bus does "
f"anything.\n")
print("REQG 00, LIMITED-RATE AUTO-REQUEST -- the lever the GCR actually gives")
print(" The DMAC takes its programmed share of the bus and spends it whether "
"or not\n a byte is there, so the CPU pays the SHARE for the WHOLE "
"delivery -- and the\n share must also be big enough to carry the rate. "
"Both, or it does not fit.\n")
print(" BR share sustains clk/B charged % of a frame fits "
f"{a.kbps:g} KB/s?")
fits_any = []
for br in range(4):
burst, mpu, share, sample = TABLE[(br, 3)] # BT=11, the longest burst
# bytes the channel can move inside one burst window, and how often that
# window comes round
bytes_per_burst = burst // BYTE_CLK
sustains = bytes_per_burst * CPU_HZ / sample
charged = share * per_byte_wire # clocks the CPU loses per
# DELIVERED byte
pct = 100 * share * wire_clk / frame_clk
ok = sustains >= RATE
if ok:
fits_any.append((br, share, charged, pct))
print(f" {br:02b} {share*100:5.2f}% {sustains/1024:7.1f} KB/s "
f"{charged:9.2f} {pct:8.1f}% {'yes' if ok else 'NO'}")
print(f"\n (BT = 11 throughout: the longest burst, 128 clocks, which is the "
f"most\n favourable row -- a shorter burst moves fewer bytes per window "
f"at the same\n share and sustains proportionally less.)")
if not fits_any:
print(f"\n NOTHING FITS. At {a.kbps:g} KB/s no limited-rate share can "
f"carry the record,\n so the only auto-request configuration that "
f"delivers is maximum rate --\n and that one stops the CPU for the "
f"whole {100*wire_clk/frame_clk:.1f}% of a frame the record takes.")
else:
br, share, charged, pct = fits_any[0]
print(f"\n CHEAPEST THAT FITS: BR = {br:02b}, {share*100:g}% of the bus, "
f"{charged:.2f} clk/B charged to the\n 68000 -- {pct:.1f}% of a frame "
f"per record.")
print(f" Against the ladder: W=5 held costs {5*a.record/frame_clk*100:.1f}%, "
f"W=9 dual held {9*a.record/frame_clk*100:.1f}%,\n W=19 the IPL ROM's "
f"own {19*a.record/frame_clk*100:.1f}%, and PIO measured "
f"{87.28*a.record/frame_clk*100:.1f}% (FINDINGS 58.2).")
print(f"""
WHAT THIS SETTLES, AND WHAT IT DOES NOT
1. AUTO-REQUEST IS CHARGED BY TIME, NOT BY BYTE. Every W in this project is
clocks per DELIVERED byte, which presumes the device asks. With no request
line the channel spends its share of the bus at a rate it was told, so the
record's cost scales with how long the disc takes -- halve the delivery rate
and the CPU cost of the same record DOUBLES. No W does that.
2. THE GCR IS A DESIGN LEVER NOBODY HAD NAMED. BT and BR are two bits each and
they set what fraction of the bus the player gives away. That is the same
kind of choice as `aligned` vs `split` and it belongs in the same list.
3. IT IS STILL NOT A MEASUREMENT. These are datasheet constants and an explicit
rate. Whether the real CZ-6BS1 drives #EXREQ (pin B36 exists on the slot, and
MAME's model simply does not connect it) is ROADMAP B3's question, and if it
does, the ladder applies and this file is the fallback rather than the plan.""")
tools/analysis/29_packed_player.py
+405
View File
@@ -0,0 +1,405 @@
#!/usr/bin/env python3
"""The DECODER-FREE PACKED player, priced against the measured cost model.
python3 tools/analysis/29_packed_player.py [container.dlx] [--kbps R]
THE QUESTION, and why it is being asked again. FINDINGS 44.7 removed the codec
and asked what a player that just puts literal frames on screen would cost. It
answered "it fits the clocks and dies on the medium": 1,152 KB/s and 1.61 GB,
because 256-colour GVRAM's default write path throws away the high byte of every
word and a picture byte therefore costs two disc bytes. 46.5/47.1 then found
the off switch -- CRTC R20 bit 11 -- and 47.2 built the layout and rendered it
pixel-exactly on both emulators at 1.0 B/pixel. 47.5 re-derived the budget on
that and withdrew 44.7's conclusion CONDITIONALLY.
Everything in 47.5 is arithmetic over a cost model that has since been REPLACED.
When it was written the transport was an unmeasured `c`; sessions 25b-28 put the
transport on the 68000 and measured it (58.2: PIO is 87.28 clk/B), put it on the
DMAC and bounded it (59.2: this machine can run dual-address only, and a
dual-address byte has a 9 clk/B FLOOR), and re-derived what a frame can afford
(59.7/60.7: 6.69 clk/B on the gate container). 47.6.1 also filed the CPU paint
cost as an ASSUMPTION -- "the `movem` shape of the packed writes is an
assumption", no clock in 47.5 measured.
So this tool re-asks 44.7's question with:
* the paint MEASURED, not assumed -- tools/bench/blit.s V8 is V1 with 128
words a row instead of 256, and tools/bench/blit.lua times it next to V1,
V2 and V3 in the same run, so the packed number is quoted against a
variant whose value (53.6%) is a session-9 result that has not moved;
* the transport swept over the SAME `W` ladder 15_bus_occupancy.py uses,
every rung of it sourced or measured (buscost.py);
* the audio DMA charged, at the rate the IPL ROM's own channel-3 setup
implies (21_iplrom_dmac.py) -- 60.x's rule that a budget debits I/O;
* and the wire and the volume stated for each, because 44.7's answer was
never about clocks.
WHAT IT DOES NOT DO. It does not settle 47.4 -- whether buffer mode BLANKS the
graphics layer, which MAME asserts and px68k is silent about (48.1), and which
needs a real board. It PRICES both branches instead, and the blanking section
is where the measured paint earns its keep: the black interval is the paint, and
until now the paint was a range read off an unpacked measurement ("~27% to ~54%",
48.3) rather than a number.
"""
import sys, os, re, argparse, csv
sys.path.insert(0, "tools/encoder")
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import numpy as np
from dlx import DLX
import buscost as B
CPUHZ = 10e6 # stock X68000, MAME 0.277 x68k.cpp:1133
GAME_S = 22.8 * 60 # the full-disc survey's runtime (ROADMAP C1)
ap = argparse.ArgumentParser()
ap.add_argument("container", nargs="?", default="tmp/rc_fr_singe_scsi_span.dlx",
help="the CODEC baseline this is compared against")
ap.add_argument("--csv", default="tmp/c68k_frames.csv",
help="per-frame C68K measurement of that container")
ap.add_argument("--blit-log", default="tmp/blit_v8.log",
help="tools/bench/blit.lua's log -- where the MEASURED paint "
"costs are read from. No defaults are compiled in.")
ap.add_argument("--kbps", type=float, default=None,
help="a delivery rate to score the wire against. OPTIONAL and "
"there is no default (FINDINGS 50).")
a = ap.parse_args()
# --- the measured paint, read out of the run's own log ---------------------
# NOT transcribed into this file. A constant copied out of a log is a constant
# that stops tracking the log, and this project has been caught by a stale
# number twice (60.8). If the log is not there the tool refuses rather than
# substituting a plausible one.
if not os.path.exists(a.blit_log):
sys.exit(f"missing {a.blit_log} -- run tools/bench/blit.lua first:\n"
f" cd tmp && mame x68000 -bios ipl10 -ramsize 2M -video soft "
f"-window -sound none -nothrottle -plugins \\\n"
f" -autoboot_script ../tools/bench/blit.lua -seconds_to_run 60")
blit = {}
for line in open(a.blit_log, errors="replace"):
m = re.search(r"V(\d+)\s+(\d+) cyc", line)
if m:
blit[int(m.group(1))] = int(m.group(2))
for v in (1, 2, 3, 4, 8, 9, 10):
if v not in blit:
sys.exit(f"{a.blit_log} has no V{v} result -- the summary is incomplete, "
f"so the run did not finish and nothing here can be quoted.")
# --- the codec baseline: the container, and its MEASURED decode ------------
d = DLX(a.container)
FPS = d.fps
FRAME_CLK = CPUHZ / FPS
meas = {}
if os.path.exists(a.csv):
for r in csv.DictReader(open(a.csv)):
meas[int(r["frame"])] = int(r["cycles"])
if not meas:
sys.exit(f"missing {a.csv} -- the codec row's decode term is MEASURED and "
f"there is no derived stand-in for it.")
NF = max(meas) + 1
codec_decode = np.mean([meas[f] for f in range(NF)])
codec_bpf = sum(d.record_lengths()[:NF]) / NF # the PADDED record (60.7)
# --- geometry, which is where the decoder-free rows come from -------------
W_PX, H_PX = d.W, d.H
NPX = W_PX * H_PX
UNPACKED_BPF = NPX * 2 # one pixel per word, high byte discarded
PACKED_BPF = NPX * 1 # R20 bit 11 + page scroll (47.2, measured)
aud_bpf = B.ADPCM_BYTES_PER_S / FPS
AUD_CLK = aud_bpf * B.ADPCM_CLK_BYTE_BEST # best case, so every row is
# the optimistic end
# A device->GVRAM channel cannot walk a 1024-byte line stride inside one
# transfer: it writes a contiguous run. 192 rows therefore need 192 array-chain
# entries -- and SESSION 29 RAN THAT, off the disc, through src/player/dma.i's
# DM_BARV/DM_BTCV: eight rows at the 1024 B stride landed from ONE channel start
# with the CPU halted throughout (tools/bench/dma_run.sh, `[chain]`). So the
# MECHANISM is demonstrated and the CPU does not restart the channel per row.
# The COST is still datasheet arithmetic -- 36 clocks an entry, Fig 4-25 sheet 1,
# buscost.DMA_CHAIN_CLK -- because MAME's DMAC runs on wall-clock attotimes and
# cannot be asked what anything costs (42.5).
CHAIN_CLK = H_PX * B.DMA_CHAIN_CLK
print(f"""{a.container}: {NF} frames of {W_PX}x{H_PX} at {FPS:g} fps
frame slot on a 10 MHz 68000: {FRAME_CLK:,.0f} clocks
paint costs MEASURED by tools/bench/blit.lua, read from {a.blit_log}:
V1 unpacked movem blit {blit[1]:>9,} clk {100*blit[1]/FRAME_CLK:5.1f}% (96 KB read + 96 KB write)
V2 byte-source expansion {blit[2]:>9,} clk {100*blit[2]/FRAME_CLK:5.1f}% (48 KB read + 96 KB write)
V3 write-only floor {blit[3]:>9,} clk {100*blit[3]/FRAME_CLK:5.1f}% (no source read at all)
V8 PACKED movem blit {blit[8]:>9,} clk {100*blit[8]/FRAME_CLK:5.1f}% (48 KB read + 48 KB write)
V8 is {100*blit[8]/blit[1]:.1f}% of V1 and {100*blit[8]/blit[3]:.1f}% of V3 -- so PACKED PAINT COSTS WHAT THE
UNPACKED PATH PAYS TO WRITE ALONE, with its source read thrown in free.
It is not exactly half of V1 because the 192-row loop does not halve with
the words: per word V1 is {blit[1]/(NPX):.3f} clk and V8 is {blit[8]/(NPX//2):.3f}.""")
# --- the architectures ----------------------------------------------------
# Each is (label, bytes on the wire per frame, CPU clocks per frame that are
# NOT the transport, and whether the transport lands in GVRAM or in RAM).
ARCH = [
("CODEC, CPU-decoded (the shipping design)", codec_bpf, codec_decode, "ring"),
("free / DMAC device->GVRAM / unpacked", UNPACKED_BPF, CHAIN_CLK, "gvram"),
("free / DMAC device->GVRAM / PACKED", PACKED_BPF, CHAIN_CLK, "gvram"),
("free / CPU-painted / unpacked, 2 B/px wire", UNPACKED_BPF, blit[1], "ring"),
("free / CPU-painted / unpacked, 1 B/px wire", PACKED_BPF, blit[2], "ring"),
("free / CPU-painted / PACKED", PACKED_BPF, blit[8], "ring"),
]
LADDER = [
(5.0, "single address, held -- needs a request line (B3)"),
(9.0, "dual address, held -- the FLOOR (59.2/59.7)"),
(12.0, "single address, arbitrated"),
(16.0, "what the ROM programs for SASI, best"),
(19.0, "what the ROM programs for SASI, worst"),
(87.28, "PIO -- MEASURED, 58.2"),
]
print("\n" + "=" * 78)
print("WHAT EACH ARCHITECTURE COSTS A FRAME, over the transport ladder\n")
print(f" audio DMA is charged in every row at {AUD_CLK:,.0f} clk "
f"({100*AUD_CLK/FRAME_CLK:.2f}%), best case.\n")
hdr = f" {'architecture':<44}{'B/frame':>9}" + "".join(f"{f'W={w:g}':>9}" for w, _ in LADDER)
print(hdr)
print(" " + "-" * (len(hdr) - 2))
for label, bpf, cpu, dest in ARCH:
cells = []
for w, _ in LADDER:
tot = bpf * w + cpu + AUD_CLK
pct = 100 * tot / FRAME_CLK
cells.append(f"{pct:>8.1f}%" if pct < 1000 else f"{pct:>8.0f}%")
print(f" {label:<44}{bpf:>9,.0f}" + "".join(cells))
print(f"""
100% is the frame deadline. Every cell is CPU work plus transport plus
best-case audio; none of them overlap, because the 68000 has no cache and a
two-word prefetch queue that empties at once (buscost.DMA_OVERLAPS = False).
THE TWO ROWS THAT MATTER ARE THE FLOOR COLUMN, W=9, because 59.2 found that
the only configurations this machine can be shown to run are dual-address,
and a dual-address byte is a 4-clock read of the device plus a 5-clock write
to memory. Every column left of it is a hardware fact nobody here has.""")
# --- the wire, which is what 44.7 actually died on ------------------------
print("\n" + "=" * 78)
print("THE WIRE AND THE MEDIUM -- 44.7's real objection\n")
print(f" {'architecture':<44}{'B/frame':>9}{'KB/s':>9}{'GB for 22.8 min':>18}")
print(" " + "-" * 78)
seen = set()
for label, bpf, cpu, dest in ARCH:
kbs = bpf * FPS / 1024
gb = bpf * FPS * GAME_S / 1e9
print(f" {label:<44}{bpf:>9,.0f}{kbs:>9.1f}{gb:>18.2f}")
print(f"""
The codec row is the gate container, which is deliberately the heaviest thing
the encoder emits (59.7). The default `need` recipe is 267.9 KB/s and E7's
byte target at the 9 clk/B floor is 327 KB/s (60.7).
SO THE PACKED DECODER-FREE PLAYER ASKS FOR {PACKED_BPF*FPS/1024:.0f} KB/s -- {PACKED_BPF*FPS/1024/327:.2f}x E7's target and
{PACKED_BPF*FPS/1024/(codec_bpf*FPS/1024):.2f}x the gate container -- and it asks for it AT A FIXED RATE. A codec's
bitrate is a lever; a literal frame's is geometry, and there is no scene in
the picture that costs less than another.""")
if a.kbps:
R = a.kbps * 1024
print(f"\n against a supplied {a.kbps:g} KB/s:")
for label, bpf, cpu, dest in ARCH:
need = bpf * FPS
print(f" {label:<44}{'FITS' if need <= R else 'SHORT BY '}"
f"{'' if need <= R else f'{(need-R)/1024:.0f} KB/s'}"
f" ({need/1024:.0f} KB/s wanted)")
# --- 47.4's two branches, priced -----------------------------------------
print("\n" + "=" * 78)
print("IF BUFFER MODE BLANKS THE LAYER (47.4 / 48, MAME's reading)\n")
print(""" R20 bit 11 only has to be SET across the GVRAM writes, so the black
interval is the paint and not the frame -- and which paint depends on where
the transport lands. That asymmetry has not been stated before:\n""")
print(f" {'architecture':<44}{'black interval':>16} {'set for':<14}")
print(" " + "-" * 78)
for label, bpf, cpu, dest in ARCH[1:]:
if dest == "gvram":
# the channel writes GVRAM, so the bit is set for the whole transfer
# DMA rungs only: a PIO transport is not a channel writing GVRAM, so
# 87.28 has no meaning in a device->GVRAM row.
rows = [bpf * w + CHAIN_CLK for w, _ in LADDER if w < 20]
span = f"{100*min(rows)/FRAME_CLK:.0f}%..{100*max(rows)/FRAME_CLK:.0f}%"
note = "the whole DMA"
else:
span = f"{100*cpu/FRAME_CLK:.1f}%"
note = "the blit only"
print(f" {label:<44}{span:>16} {note:<14}")
print(f"""
THE CPU-PAINTED PACKED PATH HAS THE SMALLEST BLACK WINDOW OF ANY OF THEM --
{100*blit[8]/FRAME_CLK:.1f}% -- because its transport lands in RAM, where bit 11 is irrelevant,
and only the {blit[8]:,}-clock blit needs the bit set. The DMAC-direct path,
which is cheaper in clocks at every rung of the ladder, is the one that must
hold the bit across its whole transfer. Under MAME's reading the cheap
architecture is the dark one.
Both are a strobe at the frame rate over the whole picture, and the packed
layout has no page to flip to: both 256-colour pages carry picture, which is
the entire point of it (48.3). {100*blit[8]/FRAME_CLK:.1f}% black at 12 Hz is not a tear.
IF PX68K IS RIGHT AND IT DOES NOT BLANK, every number above stands as
written. Neither emulator is authority and 48.1 is why the prior leans
MAME's way: MAME asserts the semantic twice and deliberately, px68k's
display path never reads the bit at all. That is an assertion against a
silence, not a tie, and it is settled by a board and the two-line probe in
tools/bench/probe_bit11_blank.lua.""")
# --- 47.6.4: does the CODEC survive the packed layout? --------------------
# Open since session 16 and never touched: "under the packed layout a word spans
# two columns 128 apart. Whether the existing codec survives that is untouched."
# There are exactly two ways it could, and blit.s V9 and V10 are them.
sk_blocks = sk_tot = pair_sk = pair_tot = 0
for f in range(NF):
m = d.modes(f).reshape(d.nby, d.nbx)
L, R = m[:, :d.nbx // 2], m[:, d.nbx // 2:]
sk_blocks += int((m == 0).sum()); sk_tot += m.size
pair_sk += int(((L == 0) & (R == 0)).sum()); pair_tot += L.size
paint_now = 1 - sk_blocks / sk_tot
paint_pair = 1 - pair_sk / pair_tot
print("\n" + "=" * 78)
print("CAN THE CODEC BE PACKED TOO? -- 47.6.4, open since session 16\n")
print(f""" A 4x4 block owns four bytes at STRIDE 2 under the packed layout, because
the high bytes of its four words belong to the block 128 columns away. So a
block decoder has two options and neither is free:
{'V4 block order, UNPACKED (the shipping shape)':<52}{blit[4]:>9,} clk {100*blit[4]/FRAME_CLK:5.1f}%
{'V9 block order, PACKED, 16 move.b at stride 2':<52}{blit[9]:>9,} clk {100*blit[9]/FRAME_CLK:5.1f}%
{'V10 block order, PACKED, blocks PAIRED (movem back)':<52}{blit[10]:>9,} clk {100*blit[10]/FRAME_CLK:5.1f}%
V9 IS {100*blit[9]/blit[4]-100:.0f}% DEARER THAN V4, not cheaper. Packing buys a block decoder
nothing on the wire either -- a codeword is already one byte a pixel -- so
that route buys NOTHING and costs {blit[9]-blit[4]:,} clocks a frame to buy it.
V10 halves the paint, and pays for it in the mode map. A pair skips only if
BOTH its blocks skip, and on this container:
SKIP blocks now {100*sk_blocks/sk_tot:5.1f}% painted now {100*paint_now:5.1f}%
SKIP block PAIRS {100*pair_sk/pair_tot:5.1f}% painted paired {100*paint_pair:5.1f}%
So pairing paints {paint_pair/paint_now:.2f}x as many blocks for {blit[10]/blit[4]:.2f}x the paint per block --
{100*(paint_pair/paint_now)*(blit[10]/blit[4])-100:+.0f}% on the clock, and about {100*(paint_pair/paint_now-1):+.0f}% on the BYTES, because a coded
block is bytes in the container whether its half of the pair changed or not.
E7 needs the bytes DOWN {100*(codec_bpf*FPS/1024)/327-100:.0f}%.
SO PACKING BELONGS TO THE LITERAL PLAYER AND ONLY TO IT. 47.6.4 is closed:
the packed layout is not an upgrade the existing codec can take, it is the
thing you get INSTEAD of the codec.""")
# --- the palette, which is where the literal player stops being a compromise --
# 46.3 measured these while pricing the TEXT PLANE and the 256-colour rows were
# only there for scale. They answer a question nobody put to them: a literal
# player has no codebooks, so it is not tied to a scene-wide palette the way the
# codec is (vq.scene_palette exists BECAUSE codewords are indices into it), and
# per-frame palettes become legal. Re-run 18_text_plane_16col.py to reproduce.
PSNR_SHIPPED = 29.19 # docs/STATUS.md, --spans all, c=5, 496.7 KB/s
PSNR_SCENE_256 = 31.33 # 18_text_plane_16col.py, tmp/fr_singe, 120 frames
PSNR_FRAME_256 = 34.08 # the same window, per-frame palettes
PAL_BYTES = 512 # 256 entries x 1 word
pal_bpf = PACKED_BPF + PAL_BYTES
# The palette write, DERIVED from a MEASURED per-word constant: V8 moves a word
# into GVRAM for blit[8]/(NPX//2) clocks and the palette is 256 consecutive
# words at $E82000 in the same movem shape.
pal_clk = 256 * blit[8] / (NPX // 2)
print("\n" + "=" * 78)
print("AND THE PICTURE IS BETTER, WHICH NOBODY HAD ASKED\n")
print(f""" PSNR against the 24-bit source, 18_text_plane_16col.py over the same
120-frame window the whole tree is measured on:
{'shipping container (the codec, as it ships)':<48}{PSNR_SHIPPED:6.2f} dB
{'256 colours, SCENE palette -- the codec CEILING':<48}{PSNR_SCENE_256:6.2f} dB
{'256 colours, PER-FRAME palette':<48}{PSNR_FRAME_256:6.2f} dB
THE MIDDLE ROW IS A CEILING AND NOT A RIVAL. Every codeword the codec emits
is an index INTO the scene palette, so no amount of bitrate takes it past
{PSNR_SCENE_256:.2f} dB; it spends {codec_bpf*FPS/1024:.0f} KB/s to get within {PSNR_SCENE_256-PSNR_SHIPPED:.2f} dB of it.
A LITERAL FRAME HAS NO CODEBOOKS, so the scene palette is not forced on it,
and the bottom row is what it simply IS -- {PSNR_FRAME_256-PSNR_SHIPPED:+.2f} dB on the shipping
container and {PSNR_FRAME_256-PSNR_SCENE_256:+.2f} dB past the ceiling the codec cannot cross.
WHAT THE PER-FRAME PALETTE COSTS:
on the wire {PAL_BYTES} B a frame -> {pal_bpf:,} B, {pal_bpf*FPS/1024:.1f} KB/s (+{100*PAL_BYTES/PACKED_BPF:.1f}%)
in clocks ~{pal_clk:,.0f} ({100*pal_clk/FRAME_CLK:.2f}% of a frame) if the CPU writes it, DERIVED
from V8's measured {blit[8]/(NPX//2):.3f} clk/word in the same movem shape
in colours 254, not 256: the packed layout spends index 0 on the
transparency key and puts black at 255 (47.2,
prep_frame.py --pack-transparent), against --reserve-black's
one entry. The tree has already measured a reserved entry at
0.04 dB (60.3), so this is noise against {PSNR_FRAME_256-PSNR_SHIPPED:+.2f}.
SETTLED IN SESSION 30, AND THE ANSWER IS YES (FINDINGS 62): a channel writes
the palette registers at $E82000 byte-exact, and ONE array-chained start
crosses from those registers into GVRAM -- so the palette IS a 193rd chain
entry and the clocks row above is what the CPU pays only if it does the write
itself. dmagate.s runs 7-9. What that does NOT settle is the board: MAME maps
the palette to palette_device over memory_array, whose write16 is a plain
COMBINE_DATA, so there is no handler that could refuse a byte write and the
model cannot discriminate. ROADMAP B4.
AND THE PSNR FIGURES ARE PIL's MEDIANCUT, not this project's own palette
builder (vq.scene_palette / H.build). The DIRECTION is measured and the
magnitude is about right; if the packed player gets built, re-derive the
per-frame number against the builder that will actually ship it.""")
# --- the answer ----------------------------------------------------------
w9 = 9.0
free_packed_dma = PACKED_BPF * w9 + CHAIN_CLK + AUD_CLK
free_packed_cpu = PACKED_BPF * w9 + blit[8] + AUD_CLK
# ... and the same two rows with the PER-FRAME PALETTE actually charged, which
# is what a player ships. The picture rows above are the comparison against the
# codec and are left alone so the published 55.2% / 81.6% do not drift; these
# are the shipping figures. Session 30 (FINDINGS 62) made the DMAC row's
# version legal: the palette is a 193rd chain ENTRY, so it costs 512 more
# delivered bytes and one more entry rather than 256 CPU word writes.
pal_dma = (PACKED_BPF + PAL_BYTES) * w9 + CHAIN_CLK + B.DMA_CHAIN_CLK + AUD_CLK
pal_cpu = (PACKED_BPF + PAL_BYTES) * w9 + blit[8] + pal_clk + AUD_CLK
codec_9 = codec_bpf * w9 + codec_decode + AUD_CLK
print("\n" + "=" * 78)
print(f"""THE ANSWER, AT THE ONE RUNG THIS MACHINE CAN BE SHOWN TO RUN (W=9)
CODEC, gate container {100*codec_9/FRAME_CLK:6.1f}% of the frame -- DOES NOT FIT
free / DMAC->GVRAM / PACKED {100*free_packed_dma/FRAME_CLK:6.1f}% -- FITS, with {100-100*free_packed_dma/FRAME_CLK:.0f}% to spare
free / CPU-painted / PACKED {100*free_packed_cpu/FRAME_CLK:6.1f}% -- FITS, with {100-100*free_packed_cpu/FRAME_CLK:.0f}% to spare
WITH THE PER-FRAME PALETTE CHARGED, which is what would ship:
DMAC-direct, palette on the CHAIN (62) {100*pal_dma/FRAME_CLK:6.1f}% of the frame, {(PACKED_BPF+PAL_BYTES)*FPS/1024:.0f} KB/s
CPU-painted, palette written by the CPU {100*pal_cpu/FRAME_CLK:6.1f}% of the frame, {(PACKED_BPF+PAL_BYTES)*FPS/1024:.0f} KB/s
The palette costs the same on the WIRE either way -- {PAL_BYTES} B a frame,
+{100*PAL_BYTES/PACKED_BPF:.1f}% -- and the wire is where this design is expensive. The gap
between the two rows is the PAINT, not the palette.
What session 30 bought is smaller than either and is worth stating exactly:
{pal_clk:,.0f} CPU clocks of palette writing replaced by one more chain entry at
{B.DMA_CHAIN_CLK} clocks, a net {100*(pal_clk-B.DMA_CHAIN_CLK)/FRAME_CLK:.2f}% of a frame -- plus the structural half,
which is that the video path then contains no per-frame PAINT at all. The
CPU still issues the READ(10) and starts the channel, and neither of those
is priced anywhere in this tree.
THE DECODER-FREE PACKED PLAYER FITS THE CLOCK BUDGET THAT THE CODEC MISSES.
That is not a small correction to 47.5, it is the reverse of the reason the
codec exists. 44.7 said it in advance and on a different cost model: "the
codec is not there to save CPU -- it is there to save the wire." The
measured model agrees, and now says the CPU side is not merely affordable
but strictly cheaper WITHOUT the codec: at the floor, decoding {codec_bpf:,.0f} bytes
costs {100*(codec_bpf*w9+codec_decode)/FRAME_CLK:.0f}% of a frame and NOT decoding {PACKED_BPF:,} costs {100*(PACKED_BPF*w9+blit[8])/FRAME_CLK:.0f}%.
SO THE QUESTION IS ENTIRELY A MEDIUM QUESTION, and it has two halves:
1. {PACKED_BPF*FPS/1024:.0f} KB/s SUSTAINED, with no lever to pull. ROADMAP B1 is
unmeasured; the 0.7-1.7 MB/s usually quoted for BlueSCSI on an X68000
is folklore with no published benchmark behind it. {PACKED_BPF*FPS/1024:.0f} KB/s sits
inside that range, which is exactly why the range has to be measured
rather than cited. A codec at 327 KB/s survives a slower answer; a
literal frame does not degrade, it drops.
2. {PACKED_BPF*FPS*GAME_S/1e9:.2f} GB for the whole game, against the codec's {codec_bpf*FPS*GAME_S/1e9:.2f} GB at the gate
recipe and ~{327*1024*GAME_S/1e9:.2f} GB at E7's target. That is a packaging fact (C3),
not a performance one.
AND 47.4 STILL SITS OVER ALL OF IT. Everything above assumes the layer is
visible while it is written. If it is not, the packed player is a {100*blit[8]/FRAME_CLK:.0f}% duty
strobe at best and there is no version of it that is merely expensive.""")
tools/analysis/30_packed_container.py
+290
View File
@@ -0,0 +1,290 @@
#!/usr/bin/env python3
"""The PACKED container: does it hold, and what is the picture actually worth?
python3 tools/analysis/30_packed_container.py [packed.dlxp]
[--frames tmp/fr_singe] [--codec tmp/rc_fr_singe_scsi_span.dlx]
ROADMAP K2. Two jobs, and they are different kinds of claim.
1. THE FORMAT HOLDS. A packed record is written into the palette registers and
GVRAM by a DMA channel with no bounds test anywhere -- the channel has no
opinion about what it is copying (FINDINGS 62) -- so "the geometry is right"
is not a tidiness check, it is the whole of the container's correctness.
Round-trip, sector geometry, and the two reserved indices are gated here.
2. THE PICTURE IS RE-DERIVED, and this is the number session 30 asked for.
FINDINGS 61.9 measured the packed player at 34.08 dB against the codec's
29.19 and filed TWO caveats: the quantiser was PIL's free 256-colour
MEDIANCUT rather than this project's builder, and the figure was quoted in
the RGB888 palette domain. Both are paid here:
* `vq.frame_palette` is what ships it -- 254 colours, because the packed
layout spends index 0 on the transparency key and 255 on black (47.2).
* the GRB555+I WORD is charged. A palette entry in a packed record is
already a hardware word; the display renders 5 bits a channel with one
shared LSB (23.3). Every PSNR in this project's encoder is measured
upstream of that, so the codec is charged it here too and the comparison
stays like for like.
And the scene-palette CONTROL is built and scored, because "per-frame
palettes became legal" is the mechanism 61.9 credits and an unrun control is
an assumption. The codec cannot take this row: every codeword it emits is an
index INTO `vq.scene_palette`, so 31.33 dB is its ceiling at any bitrate.
WHAT THIS DOES NOT DO. It does not put a packed frame on a machine -- that is
K3, and the layout itself was already rendered pixel-exactly on both emulators
in 47.2. It does not price clocks: 29_packed_player.py owns that, off the
MEASURED blit, and nothing here moves it. And it settles nothing about the
medium: 582.0 KB/s is geometry, and whether anything sustains it is B1.
"""
import argparse, glob, os, sys
sys.path.insert(0, "tools/encoder")
sys.path.insert(0, "tools/bench")
import numpy as np
from PIL import Image
import vq as VQ
import dlxp as P
from dlx import DLX
from dlxload import pack_palette
ap = argparse.ArgumentParser()
ap.add_argument("packed", nargs="?", default="tmp/packed_singe.dlxp")
ap.add_argument("--frames", default="tmp/fr_singe")
ap.add_argument("--mismatch-png", default=None,
help="write the 62.5 mismatch as a picture: correct render | the "
"same frame under the NEXT frame's palette | the 24-bit "
"source. A dB is not a look, and this claim is about a look.")
ap.add_argument("--codec", default="tmp/rc_fr_singe_scsi_span.dlx",
help="the shipping container this replaces. Its PSNR is COMPUTED "
"from its own bytes, not transcribed from docs (60.8).")
a = ap.parse_args()
fail = []
d = P.DLXP(a.packed) # every format invariant is checked in here
print(f"{a.packed}: DLXP{d.version} {d.W}x{d.H} {d.fps}fps {d.nframes} frames")
print()
# --- 1. the format -----------------------------------------------------------
print("THE FORMAT, and why each line is a gate and not a courtesy check:")
print(f" record {d.rec_bytes:,} B = {d.rec_bytes // P.SECTOR} sectors exactly, "
f"palette {d.pal_bytes} B "
f"{'LAST' if d.palette_last else 'first'}, picture {d.pic_bytes:,} B")
print(f" 1.0 B/pixel: {d.pic_bytes} bytes carry {d.W * d.H} pixels "
f"(the unpacked path needs {2 * d.W * d.H:,})")
zero = black = 0
for f in range(d.nframes):
idx = d.indices(f)
# The channel copies bytes; a container whose interleave is a byte out does
# not fail, it paints. So the round trip is the assertion that the bytes in
# the record ARE the picture, in the order GVRAM wants them.
if P.pack_picture(idx).tobytes() != d._split(f)[1]:
fail.append(f"frame {f}: the record does not round-trip through the "
f"interleave -- the container is not what it says it is")
break
zero += int((idx == 0).sum())
black += int((idx == 255).sum())
if zero:
fail.append(f"index 0 appears in the picture {zero:,} times -- it is the "
f"TRANSPARENCY KEY of the top page and must stay unused (47.2)")
print(f" round-trip: {d.nframes} records unpack and re-pack byte-identical")
print(f" index 0 (transparency key) used {zero} times; "
f"index 255 (black) {black:,} times in the picture")
# THE PICTURE'S wire, and it is the one this file is about. DLXP2 puts audio on
# the same wire at a cadence (65.3, 67) and `d.kbps()` is both; what is asserted
# here is that the PICTURE's share is still exactly geometry, because that is
# 61.6's claim and a second stream is exactly the thing that could quietly
# dilute it.
kbps = d.video_kbps()
geom = d.rec_bytes * d.fps / 1024
if abs(kbps - geom) > 1e-6:
fail.append(f"wire {kbps} != geometry {geom}")
print(f" wire {kbps:.1f} KB/s = {d.rec_bytes:,} B x {d.fps} fps. FIXED. A codec's "
f"bitrate is a lever and a literal frame's is geometry (61.6)"
+ (f"\n ...and {d.audio_kbps():.2f} KB/s of audio rides beside it on the "
f"F={d.cad_f}/A={d.cad_a} cadence, for {d.kbps():.1f} KB/s total "
f"(tools/analysis/34_packed_audio.py)" if d.has_audio else ""))
print()
# --- 2. the picture ----------------------------------------------------------
files = sorted(glob.glob(f"{a.frames}/f*.png"))[:d.nframes]
if len(files) < d.nframes:
sys.exit(f"{a.frames}: {len(files)} frames, container has {d.nframes}")
src = [np.asarray(Image.open(f).convert("RGB")) for f in files]
def rendered(pal):
"""RGB888 as the DISPLAY produces it, from the same maths the loader uses."""
return pack_palette(np.asarray(pal, np.uint8))[2]
def score(name, pal_rgb, idx_frames, note=""):
"""Two columns: the palette domain every encoder PSNR in this tree is
quoted in, and the hardware word the display actually renders."""
ren = rendered(pal_rgb)
p_pal = np.mean([VQ.psnr(s, np.asarray(pal_rgb)[i])
for s, i in zip(src, idx_frames)])
p_hw = np.mean([VQ.psnr(s, ren[i]) for s, i in zip(src, idx_frames)])
print(f" {name:<44s} {p_pal:6.2f} {p_hw:6.2f} {note}")
return p_pal, p_hw
print("PSNR vs the 24-bit source, mean over frames:")
print(f" {'':<44s} {'RGB888':>6} {'GRB555':>6}")
codec_pal = codec_hw = None
if os.path.exists(a.codec):
c = DLX(a.codec)
if c.nframes < d.nframes:
print(f" (the codec container has {c.nframes} frames and this has "
f"{d.nframes} -- its row is skipped rather than compared over a "
f"different window)")
else:
# Its rate is printed with it because this is the GATE container -- the
# heaviest stream the encoder emits, `--kbps 280 --span-kbps 488
# --spans all` (check.sh) -- and NOT the 496.7 KB/s / 29.19 dB "current
# encode" of the README. Two containers, two numbers; a row that named
# neither would invite the difference to be read as a drift.
ckbps = sum(c.record_lengths()) * c.fps / c.nframes / 1024
codec_pal, codec_hw = score("CODEC, the GATE container", c.pal,
c.decode_all()[:d.nframes],
f"{ckbps:.1f} KB/s, "
f"{os.path.basename(a.codec)}")
else:
print(f" (no codec container at {a.codec} -- its row is skipped)")
# The codec's CEILING: 256 colours, one palette for the scene, no VQ loss at
# all. Not a rival, a bound -- no bitrate takes the codec past this row.
ref, spal = VQ.scene_palette(src, reserve_black=True)
sidx = VQ.palettise(src, ref)
ceil_pal, ceil_hw = score("256c SCENE palette -- the CODEC'S CEILING",
spal, sidx, "no bitrate crosses this")
# The control for the mechanism 61.9 credits: same LAYOUT and the same 254
# picture colours, one palette for the scene instead of one per frame. It is
# built to 255 with black reserved and then black is MOVED from 0 to 255, which
# is the packed layout's convention (47.2) rather than the codec's -- so the
# only variable between this row and the container's is per-frame vs scene-wide.
cref, c255 = VQ.scene_palette(src, colors=255, reserve_black=True)
cpal = np.vstack([np.zeros((1, 3), np.uint8), c255[1:],
np.zeros((1, 3), np.uint8)])
cidx = [np.where(i == 0, np.uint8(255), i)
for i in VQ.palettise(src, cref)]
ctl_pal, ctl_hw = score("PACKED, 254c SCENE palette [the CONTROL]", cpal, cidx)
# And the container itself. The right-hand column is read out of the CONTAINER'S
# OWN BYTES -- `DLXP.render` unpacks the GRB555 words the record carries -- and
# the left-hand one is recomputed from the encoder, because a packed record has
# no RGB888 palette in it to score. The two are tied together by a gate rather
# than by trust: the palettes the encoder builds here must reproduce the
# container's indices exactly, or the left column is describing a different file.
pk_idx, pk_pal_rgb, mismatch, palbad = [], [], 0, 0
for n, s in enumerate(src):
pal, idx = VQ.frame_palette(s)
if not np.array_equal(idx, d.indices(n)):
mismatch += 1
# And the WORD. The encoder packed GRB555+I with `dlxload.pack_palette` and
# `DLXP.palette_rgb` unpacks it: two separate pieces of maths over the same
# 23.3 rule, and a container is the only place they meet. Required to agree,
# not assumed to -- a wrong shared LSB is a 1.96 dB bug that still renders.
if not np.array_equal(rendered(pal), d.palette_rgb(n)):
palbad += 1
pk_idx.append(idx)
pk_pal_rgb.append(pal)
if palbad:
fail.append(f"{palbad} of {d.nframes} records carry palette words that do "
f"not unpack to the RGB the encoder packed -- pack_palette and "
f"DLXP.palette_rgb disagree about GRB555+I")
if mismatch:
fail.append(f"{mismatch} of {d.nframes} frames re-quantise to different "
f"indices than the container holds -- the RGB888 column would "
f"be scoring a file that is not this one")
pk_pal = np.mean([VQ.psnr(s, p[i]) for s, p, i in zip(src, pk_pal_rgb, pk_idx)])
pk_hw = np.mean([VQ.psnr(s, d.render(f)) for f, s in enumerate(src)])
print(f" {'PACKED CONTAINER, 254c PER-FRAME':<44s} {pk_pal:6.2f} {pk_hw:6.2f} "
f"GRB555 read out of {os.path.basename(a.packed)}")
print()
print(" The right-hand column is the PLAYER'S number. Every PSNR this project")
print(" has quoted -- 29.19, 31.33, 34.08 -- lives in the left one, upstream of")
print(" the 5-bit hardware word (23.3), and 61.9's 34.08 is directly comparable")
print(" to the packed row's left-hand entry and to nothing else.")
print()
# --- 3. what it means --------------------------------------------------------
if codec_hw is not None:
print(f" packed vs the codec gate container, as the DISPLAY renders both: "
f"{pk_hw - codec_hw:+.2f} dB")
print(f" packed vs the codec's CEILING: "
f"{pk_hw - ceil_hw:+.2f} dB")
print(f" what the PER-FRAME palette is worth (vs the control): "
f"{pk_hw - ctl_hw:+.2f} dB")
print(f" what the GRB555 word costs the ceiling row: "
f"{ceil_hw - ceil_pal:+.2f} dB")
# 60.3 measured ONE reserved entry at 0.04 dB; the packed layout spends two.
# Scored here at scene scale, where the control makes it a clean subtraction.
print(f" what the packed layout's TWO reserved entries cost: "
f"{ctl_pal - ceil_pal:+.4f} dB (256c -> 254c, scene palette, RGB888)")
print()
# The three claims 61.9 makes, restated as gates. A tree where any of these
# flipped has a different answer to ROADMAP K and should say so out loud.
if codec_hw is not None and pk_hw <= codec_hw:
fail.append(f"the packed container is {pk_hw:.2f} dB and the codec it "
f"replaces is {codec_hw:.2f} -- 61.9's headline is inverted")
if pk_hw <= ceil_hw:
fail.append(f"the packed container is {pk_hw:.2f} dB and the codec's own "
f"CEILING is {ceil_hw:.2f} -- the per-frame palette bought "
f"nothing, and 61.9's reason for building this branch is gone")
if pk_hw <= ctl_hw:
fail.append(f"per-frame {pk_hw:.2f} dB is not better than the SCENE-palette "
f"control {ctl_hw:.2f} -- the mechanism 61.9 credits is absent")
# --- 4. FINDINGS 62.5, which needed this encoder to exist ---------------------
# 62.5 filed the chain's order -- palette first or 193rd -- as a free choice with
# a visible consequence, and said the severity "depends on how much the palette
# moves between consecutive frames, which is a property of the encoder K2 has
# not been written yet". It is written now, so the number exists.
#
# The mismatch is a WIPE, not a flash: rows arrive top to bottom, so at any
# instant part of the screen is right. What is bounded here is the WORST
# instant of each order -- the whole screen wrong -- which is the start of the
# transfer for palette-first and the end of it for palette-last. The mean over
# the transfer is about half of each, because the wipe is linear in rows.
print("FINDINGS 62.5 PRICED -- palette FIRST vs LAST, at the worst instant of each:")
churn = np.mean([int((d.palette_words(n) != d.palette_words(n - 1)).sum())
for n in range(1, d.nframes)])
first = np.mean([VQ.psnr(src[n - 1], d.palette_rgb(n)[d.indices(n - 1)])
for n in range(1, d.nframes)])
last = np.mean([VQ.psnr(src[n], d.palette_rgb(n - 1)[d.indices(n)])
for n in range(1, d.nframes)])
correct = np.mean([VQ.psnr(src[n], d.render(n)) for n in range(1, d.nframes)])
print(f" palette entries that CHANGE frame to frame: {churn:.1f} of 256 "
f"({100 * churn / 256:.0f}%) -- a per-frame palette is not a small delta")
print(f" palette FIRST, old rows under the new palette: {first:6.2f} dB "
f"({first - correct:+.2f} against the correct pairing)")
print(f" palette LAST, new rows under the old palette: {last:6.2f} dB "
f"({last - correct:+.2f})")
print(f" the container is currently palette "
f"{'LAST' if d.palette_last else 'FIRST'} (dlxp.py, --palette-last)")
if a.mismatch_png:
# The frame whose mismatch is CLOSEST TO THE MEAN, so the picture is not an
# outlier picked to make the point look worse than the number.
mis = np.array([VQ.psnr(src[n - 1], d.palette_rgb(n)[d.indices(n - 1)])
for n in range(1, d.nframes)])
n = int(np.argmin(np.abs(mis - mis.mean()))) + 1
z = lambda x: np.repeat(np.repeat(x, 2, 0), 2, 1)
gap = np.full((d.H * 2, 6, 3), 30, np.uint8)
Image.fromarray(np.concatenate(
[z(d.render(n - 1)), gap, z(d.palette_rgb(n)[d.indices(n - 1)]), gap,
z(src[n - 1])], axis=1)).save(a.mismatch_png)
print(f" wrote {a.mismatch_png}: frame {n-1} correct | frame {n-1} under "
f"frame {n}'s palette ({mis[n-1]:.2f} dB) | the 24-bit source")
print(" Both are one paint, and both are MOOT if buffer mode blanks the layer")
print(" (47.4/B2). This bounds the cost of being wrong; it does not decide it,")
print(" because dB over a whole frame is not what an eye sees in a wipe.")
print()
for x in fail:
print("FAIL " + x)
sys.exit(1 if fail else 0)
tools/analysis/31_display_duty.py
+215
View File
@@ -0,0 +1,215 @@
#!/usr/bin/env python3
"""HOW LONG IS THE PICTURE ACTUALLY ON SCREEN? ROADMAP K3, FINDINGS 64.
python3 tools/analysis/31_display_duty.py [container.dlxp] [--rate KB/s ...]
THE QUESTION NOTHING IN THIS TREE HAD ASKED. Every budget in docs/FINDINGS.md
asks what a frame COSTS -- clocks, bus cycles, bytes on the wire. Session 32
built the packed player and ran it (src/player/packed.s), and the run reported a
number no budget has a column for: the write window was open on 99.5% of the
host frames, so the graphics layer was DARK for 99.5% of the scene. Every frame
was pixel-exact and almost none of them was visible.
WHY THAT IS ARITHMETIC AND NOT AN EMULATOR ARTEFACT. 256-colour GVRAM masks the
high byte of every write unless CRTC R20 bit 11 is set (46.5/47.1), and the
packed layout's whole 1.0 B/pixel claim is that one word carries two pixels --
so a packed write REQUIRES the bit. If buffer mode blanks the layer while the
bit is set (47.4/ROADMAP B2 -- MAME says it does, and 48.1's prior leans that
way), then the layer is dark for exactly as long as the window is open, and for
a DMAC-direct player the window is open for the whole transfer. There is no
second page to hide behind: the packed layout SPENDS both 256-colour pages,
which is the same fact that made a frame one channel start (FINDINGS 62).
dark fraction of a slot = record bytes / (data-phase rate x slot)
AND THE RATE IN THAT EXPRESSION IS THE BURST RATE, NOT THE SUSTAINED ONE. This
is the correction the session had to make to itself. The container's 582.0 KB/s
is a SUSTAINED requirement -- it decides whether record i arrives before slot i.
The dark fraction is set by how fast bytes move DURING THE DATA PHASE, which for
a drive with a read-ahead cache can be several times the sustained figure. The
two are independent, and a medium can pass one and fail the other:
sustained >= 582.0 KB/s or frames arrive late (B1, known)
data phase >> 582.0 KB/s or the frame is never displayed (NEW, and B1 has
no test for it)
THE OTHER PLAYER IN THE FAMILY DOES NOT HAVE THIS PROPERTY. A packed player
that DMAs the record into RAM and paints it with the CPU opens the window only
for the paint -- tools/bench/blit.s V8, MEASURED, not assumed -- which is a
fixed share of the slot no matter what the medium does. It costs more clocks
and 49 KB of RAM and it buys a picture that is on screen. FINDINGS 61.5 already
priced both in CLOCKS and ranked DMAC-direct first; this file is the column that
was missing from that table, and it reverses the ranking under B2-blanks.
"""
import argparse, os, re, sys
sys.path.insert(0, "tools/encoder")
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from dlxp import DLXP
import buscost as B
CPUHZ = 10e6 # stock X68000, MAME 0.277 x68k.cpp:1133
ap = argparse.ArgumentParser()
ap.add_argument("container", nargs="?", default="tmp/packed_singe.dlxp")
ap.add_argument("--blit-log", default="tmp/blit_v8.log",
help="tools/bench/blit.lua's log -- the MEASURED packed paint. "
"Not a constant in this file: 47.6.1 filed the movem shape "
"as an assumption and session 29 measured it, and a second "
"copy of a measured number is how one of them goes stale.")
ap.add_argument("--rate", type=float, nargs="*", default=None,
help="data-phase rates to price, KB/s. REQUIRED to mean "
"anything: this project has no delivery figure and will "
"not default to one (FINDINGS 50).")
ap.add_argument("--run-log", default="tmp/packed_free_steal.log",
help="a free-running tools/bench/packed.lua log, for the "
"measured corroboration section")
a = ap.parse_args()
d = DLXP(a.container)
SLOT_S = 1.0 / d.fps
FRAME_CLK = CPUHZ * SLOT_S
wire = d.kbps() # KB/s, and fixed by geometry
blit = {}
if os.path.exists(a.blit_log):
for line in open(a.blit_log, errors="replace"):
m = re.search(r"V(\d+)\s+(\d+) cyc", line)
if m:
blit[int(m.group(1))] = int(m.group(2))
if 8 not in blit:
sys.exit(f"{a.blit_log} has no V8 result. The packed paint is a MEASUREMENT "
f"(tools/bench/blit.lua) and this tool will not substitute a "
f"constant for it -- run the blit bench, or point --blit-log at "
f"its log.")
PAINT_CLK = blit[8]
PAINT_FRAC = PAINT_CLK / FRAME_CLK
print(f"""THE CONTAINER (tools/encoder/dlxp.py)
{a.container}: {d.W}x{d.H} {d.fps} fps, {d.nframes} frames
record {d.rec_bytes:,} B = {d.rec_bytes // 512} sectors, palette \
{'LAST' if d.palette_last else 'FIRST'}
slot {SLOT_S*1000:.2f} ms = {FRAME_CLK:,.0f} clocks
wire {wire:.1f} KB/s -- FIXED by geometry. A codec's bitrate is a lever and a
literal frame's is not (61.6), so nothing an encoder does moves this.
THE TWO PACKED PLAYERS, and the difference is WHEN the write window is open
A. DMAC-DIRECT (src/player/packed.s, ROADMAP K3, and the one that is built).
One channel start, 193 destinations, the CPU halted or nearly. The window
must be open for the WHOLE data phase, because the channel writes when the
bytes arrive and the CPU cannot know when that is -- and a packed write
that lands with the bit clear is masked to its low byte and silently wrong.
B. DMA-TO-RAM + CPU PAINT. The record lands in RAM with the window shut; the
68000 then paints it with the packed movem blit. The window is open for the
PAINT and nothing else: {PAINT_CLK:,} clocks, {100*PAINT_FRAC:.1f}% of a slot,
MEASURED by tools/bench/blit.lua (V8), and INDEPENDENT of the medium.
Under 47.4-blanks the dark interval IS the open window, so B is on screen for
{100*(1-PAINT_FRAC):.1f}% of every slot at ANY rate that delivers the record at all,
and A's visibility is a function of the rate.
""")
rates = a.rate
if not rates:
print("""NO RATES GIVEN, so no table. This project retired its delivery
constant outright (FINDINGS 50, USER DECISION) and every tool requires an
explicit rate; a default here would be the same mistake in a new place. Pass
--rate with the figures you want priced. The three thresholds already derived
elsewhere, for reference and NOT as defaults:
453.6 KB/s the DLX5 codec gate container needs no prefill (49.5/60)
576.0 KB/s a packed container with no per-frame palette (61.5)
582.0 KB/s THIS container, palette included (63)
and note that all three are SUSTAINED figures. The dark fraction below is set by
the DATA-PHASE rate, which is a different measurement nothing has taken.""")
sys.exit(0)
print("A's VISIBILITY, against the DATA-PHASE rate\n")
print(f" {'data phase':>12} | {'transfer':>9} | {'window open':>11} | "
f"{'PICTURE ON SCREEN':>17} | vs B")
print(f" {'KB/s':>12} | {'ms':>9} | {'% of slot':>11} | "
f"{'% of slot':>17} |")
print(" " + "-"*12 + "-+-" + "-"*9 + "-+-" + "-"*11 + "-+-" + "-"*17 + "-+-----")
for R in sorted(rates):
t_ms = d.rec_bytes / (R * 1024) * 1000
openf = min(1.0, t_ms / (SLOT_S * 1000))
vis = 1.0 - openf
verdict = ("A wins" if vis > 1 - PAINT_FRAC else
"B wins" if vis < 1 - PAINT_FRAC else "equal")
late = " LATE" if R < wire else ""
print(f" {R:>12.1f} | {t_ms:>9.2f} | {100*openf:>11.1f} | "
f"{100*vis:>17.1f} | {verdict}{late}")
# The crossover, stated as a rate rather than left to be read off the table: it
# is the one number in here a hardware acceptance test can be written against.
cross = d.rec_bytes / (PAINT_FRAC * SLOT_S) / 1024
print(f"""
A and B show the picture for the same share of the slot at a data-phase rate
of {cross:,.0f} KB/s. Below that, THE PLAYER WITH THE CPU IN THE LOOP IS ON
SCREEN LONGER than the one without it -- which is the reverse of FINDINGS
61.5's ranking, and 61.5 is not wrong: it ranked them in CLOCKS, and this is
the column that table does not have.
{cross:,.0f} KB/s is {cross/wire:.1f}x the container's own wire. So a medium that exactly
meets the sustained requirement puts the DMAC-direct player's picture on
screen for {100*max(0.0, 1-wire/wire):.0f}% of every slot: it delivers every frame, on time,
pixel-exact, and displays none of them.""")
print(f"""
THE CPU SIDE, so the trade is priced on both axes (FINDINGS 61.5's ladder)
W is clocks stolen per delivered byte. Only the dual-address rungs have a code
path on this machine (59.2), and 9 is the floor: a 4-clock read of the device
plus a 5-clock write to memory.
""")
print(f" {'W':>3} | {'A: DMAC-direct':>15} | {'B: DMA + CPU paint':>19}")
print(" " + "-"*3 + "-+-" + "-"*15 + "-+-" + "-"*19)
AUDIO = B.ADPCM_BYTES_PER_S / d.fps * B.ADPCM_CLK_BYTE_BEST
for W in (5, 9, 12, 16, 19):
xfer = d.rec_bytes * W
ca = (xfer + AUDIO) / FRAME_CLK
cb = (xfer + AUDIO + PAINT_CLK) / FRAME_CLK
print(f" {W:>3} | {100*ca:>14.1f}% | {100*cb:>18.1f}%")
print(f"""
Both include the audio DMA at {AUDIO:,.0f} clocks a frame ({100*AUDIO/FRAME_CLK:.2f}%), charged from
the IPL ROM's own channel-3 setup (21_iplrom_dmac.py, 52.5). Neither includes
a decoder, because neither has one.
So B costs the paint -- {100*PAINT_FRAC:.1f}% of a frame -- and TWO record buffers,
{2*d.rec_bytes:,} B of RAM. Two and not one: at any rate near the wire the delivery
of record i+1 occupies most of the slot the paint of record i happens in, so
they overlap by construction. On a 2 MB machine that is {200*d.rec_bytes/(2*1024*1024):.1f}% of memory and
it is the resource this design has spare -- the ring the packed branch deleted
was 256 KB (FINDINGS 49). That is what a visible picture costs if 47.4 blanks.""")
# ---- the measured corroboration. It is a SEPARATE section and it is bounded
# on purpose: MAME's device models carry no transfer timing (docs/BENCHMARK.md,
# 42.5), so the run cannot supply a rate for the table above -- what it can do
# is show that the mechanism is real and that the arithmetic predicts it.
if os.path.exists(a.run_log):
txt = open(a.run_log, errors="replace").read()
m_rate = re.search(r"record lands in ([\d.]+) ms, i\.e\. ([\d.]+) KB/s", txt)
m_open = re.search(r"WRITE WINDOW OPEN on (\d+) of (\d+) host frames", txt)
if m_rate and m_open:
ms, kbps = float(m_rate.group(1)), float(m_rate.group(2))
op, tot = int(m_open.group(1)), int(m_open.group(2))
pred = min(1.0, d.rec_bytes / (kbps * 1024) / SLOT_S)
print(f"""
MEASURED, on the emulated machine (tools/bench/packed_run.sh, free-running)
{a.run_log}: a {d.rec_bytes:,} B record landed in {ms:.2f} ms = {kbps:.1f} KB/s, and the
write window was open on {op} of {tot} host frames = {100*op/tot:.1f}%.
The expression above predicts {100*pred:.1f}% at that rate.
THIS IS NOT A RATE MEASUREMENT AND {kbps:.0f} KB/s IS NOT A MEDIUM. MAME's
device models carry no transfer timing (42.5); the figure is a property of the
apparatus. What the run DOES establish is that the mechanism is the one the
arithmetic describes -- and one thing more that no arithmetic could have
given: the DMAC CONFIGURATION DOES NOT MOVE IT. Held and stealing delivered
the same record within 0.5% of each other, so what a channel configuration
buys is who owns the CPU, not when the picture appears.""")
else:
print(f"""
NO MEASURED SECTION: {a.run_log} is absent. Run
tools/bench/packed_run.sh to produce it. The arithmetic above stands without
it -- it is geometry -- but the run is what showed the effect was there to be
derived at all.""")
tools/analysis/32_audio_wire.py
+227
View File
@@ -0,0 +1,227 @@
#!/usr/bin/env python3
"""WHAT DOES AUDIO DO TO THE CONTAINER? ROADMAP P6, the half that is not the bus.
python3 tools/analysis/32_audio_wire.py [packed.dlxp] [--audio tmp/au_singe.raw]
[--rate KB/s ...]
Session 20 (FINDINGS 52) closed the bus half of P6: a second DMA consumer at
7,812.5 B/s is 1.25%..1.48% of a frame, about 4% of what the decoder leaves, and
the 7.8 kB/s figure survived with a unit correction. ROADMAP P6 then says, in
as many words, that EVERYTHING ELSE in the item is open: extraction, an encoder,
the container interleave, and what a second stream does to `wire` and therefore
to `pipe - wire` and therefore to 51.3's refill climb.
This file is the container interleave and the wire. It is arithmetic over the
real container's real geometry -- no MAME run, no board.
THE THING THAT MAKES IT INTERESTING, and it is a property of DLXP1 rather than
of audio: **a packed container has no index and cannot have one.** A record's
address is `LBA0 + i*97` because a literal frame's length is geometry (FINDINGS
63, 64.1). Audio is a stream at a rate that has nothing to do with the frame
rate, so the naive interleave -- give record i the audio bytes belonging to slot
i -- makes records VARIABLE LENGTH, and the moment records are variable length
the format needs an index and stops being the format.
So the interleave has to be a FIXED CADENCE: every F frames, A whole sectors of
audio, placed between records. Then
LBA(i) = LBA0 + i*RECSEC + floor(i/F)*A
which is still two multiplies and a divide -- arithmetic, no index, nothing
walked -- and the only cost is that A*512 must be at least F frames' worth of
audio, so the padding is whatever A*512 exceeds it by. Choosing (F, A) is a
rational-approximation problem and the answer is NOT the obvious cadence.
"""
import argparse, os, sys
from fractions import Fraction
sys.path.insert(0, "tools/encoder")
sys.path.insert(0, "tools/analysis")
import buscost as B
from dlxp import DLXP, SECTOR
ap = argparse.ArgumentParser()
ap.add_argument("container", nargs="?", default="tmp/packed_singe.dlxp")
ap.add_argument("--audio", default="tmp/au_singe.raw",
help="raw s16le mono at the chip rate, from extract_audio.py")
ap.add_argument("--codec", default="tmp/rc_fr_singe_scsi_span.dlx",
help="the codec container, for the same arithmetic on the other branch")
ap.add_argument("--rate", type=float, nargs="*",
default=[453.6, 500.0, 582.0, 600.0, 650.0, 700.0],
help="explicit sustained delivery rates, KB/s")
a = ap.parse_args()
d = DLXP(a.container)
SLOT_S = 1.0 / d.fps
FRAME_CLK = B.CPU_HZ * SLOT_S if hasattr(B, "CPU_HZ") else 10_000_000 * SLOT_S
RECSEC = d.rec_bytes // SECTOR
print(f"""
=== THE STREAM =========================================================
The chip is an MSM6258V on an 8 MHz clock and it has three rates and no
others. Every budget in this tree is written against the first one.""")
RATES = {512: 15625.0, 768: 8_000_000/768, 1024: 7812.5}
print(f"\n {'divisor':>8} {'samples/s':>11} {'bytes/s':>10} {'B per 1/%d s slot' % d.fps:>19} exact?")
for div, hz in RATES.items():
bps = hz / 2
per = bps / d.fps
print(f" 8MHz/{div:<4} {hz:11,.1f} {bps:10,.1f} {per:19,.4f} "
f"{'yes' if per == int(per) else 'NO -- a remainder, like the frame clock (54)'}")
HZ = 15625.0
AU_BPS = HZ / 2 # 4 bits a sample, two samples to a byte
AU_FRAME = AU_BPS / d.fps # 651.0416... B, and the point is the dots
print(f"""
The shipping rate's per-slot figure is {AU_FRAME:,.4f} B and it is NOT an
integer -- 8 MHz / 512 / 2 / {d.fps} has a 12 in the denominator that 2**k
cannot clear. That is the same shape as FINDINGS 54's frame clock: what a
player carries is a remainder, not a count, and a container that rounds it
either drifts or underruns.""")
if os.path.exists(a.audio):
n16 = os.path.getsize(a.audio) // 2
secs = n16 / HZ
print(f"""
MEASURED, on the window this project gates everything on (00223 @539.4s,
{secs:.3f} s, tools/encoder/extract_audio.py):
{n16:,} samples -> {n16//2:,} B of ADPCM = {n16/2/secs:,.1f} B/s
which is {AU_BPS:,.1f} to the byte, so the rate is the rate.""")
print(f"""
=== THE INTERLEAVE, AND WHY THE OBVIOUS CADENCE IS THE WRONG ONE =======
A packed record is {d.rec_bytes:,} B = {RECSEC} sectors EXACTLY and its address is
arithmetic. Audio rides between records at a fixed cadence -- every F frames,
A whole sectors -- so that LBA(i) stays arithmetic. A must satisfy
A * {SECTOR} >= F * {AU_FRAME:,.4f} i.e. A/F >= {Fraction(int(AU_BPS*2), int(2*SECTOR*d.fps))} = {AU_FRAME/SECTOR:.9f}
and everything above that ratio is PADDING that the wire pays for and nothing
plays. Here is the whole small-F space, best A for each F:""")
target = Fraction(int(round(AU_BPS * 2)), 2 * SECTOR * d.fps) # sectors per frame, exact
rows, floor = [], None
for F in range(1, 241):
A = -(-(target.numerator * F) // target.denominator) # ceil(F * target)
have, need = A * SECTOR, F * AU_FRAME
waste = (have - need) / need
add = have / F * d.fps / 1024 # what the cadence puts on the wire, KB/s
rows.append((waste, F, A, have, need, add))
print(f"\n {'F':>4} {'A':>4} {'A*512 B':>10} {'needs':>12} {'padding':>9} {'waste':>7}"
f" {'wire adds':>10} {'player RAM':>11}")
seen = None
for waste, F, A, have, need, add in rows:
show = F <= 4 or seen is None or waste < seen - 1e-12
if seen is None or waste < seen: seen = waste
if show:
print(f" {F:4d} {A:4d} {have:10,} {need:12,.1f} {have-need:9,.1f} "
f"{100*waste:6.2f}% {add:9.2f} KB/s {have:9,} B")
best = sorted(rows)
w, F, A, have, need, add = best[0]
f1 = next(x for x in rows if x[1] == 1)
print(f""" THE FLOOR OF THAT SWEEP is F={F}, A={A}: {100*w:.3f}% padding, {add:.2f} KB/s of
wire for {AU_BPS/1024:.2f} KB/s of audio.
THE OBVIOUS CADENCE IS THE WORST ONE. F=1 -- one audio lump per record, which
is what "interleave the audio into the frame" means if nobody does the
arithmetic -- needs A={f1[2]} and costs {100*f1[0]:.1f}% padding: {AU_FRAME:,.1f} B rounded up to
{f1[3]:,}, so {f1[3]-AU_FRAME:,.1f} B of every record is nothing at all, and the wire pays
{f1[5]:.2f} KB/s for {AU_BPS/1024:.2f} KB/s of audio. That is {f1[5]-add:.2f} KB/s thrown away for
no reason but the cadence.
=== WHAT IT DOES TO THE WIRE ===========================================""")
vid_kbs = d.video_kbps()
for label, cad in (("F=1 (one lump a record)", f1), (f"F={F} (the floor)", best[0])):
tot = vid_kbs + cad[5]
print(f" {label:26s} video {vid_kbs:7.1f} + audio {cad[5]:5.2f} = {tot:7.1f} KB/s "
f"({100*(tot/vid_kbs-1):+.2f}%)")
print(f"""
And this is what B1's acceptance test becomes. The packed container's
sustained requirement was {vid_kbs:.1f} KB/s SILENT (FINDINGS 61.5, 63) and it is
{vid_kbs + add:.1f} KB/s with sound. A literal frame's bitrate is geometry and cannot
be talked down; the audio on top of it is {add:.2f} KB/s and can only be talked down
by choosing a worse chip rate.""")
f11 = next(x for x in rows if x[1] == 11)
print(f"""
AND THE CADENCE HAS A SECOND PRICE, WHICH IS RAM. A cadence of F frames means
the player is holding F frames of audio, and holding it TWICE -- the channel
fills lump n+1 while the chip drains lump n, the same reason K4 needs two
record buffers (64.2). So the floor of the sweep is not the answer:
F={f1[1]:<3} {f1[3]:>7,} B a lump, {2*f1[3]:>7,} B held {100*f1[0]:6.2f}% padding {f1[5]:5.2f} KB/s
F={f11[1]:<3} {f11[3]:>7,} B a lump, {2*f11[3]:>7,} B held {100*f11[0]:6.2f}% padding {f11[5]:5.2f} KB/s <- the pick
F={F:<3} {have:>7,} B a lump, {2*have:>7,} B held {100*w:6.2f}% padding {add:5.2f} KB/s
F={f11[1]} buys {100*(f1[0]-f11[0]):.1f} points of padding for {2*f11[3]-2*f1[3]:,} B of RAM, and F={F} buys the
last {100*(f11[0]-w):.2f} of a point for {2*have-2*f11[3]:,} B more. On a machine where K4 already
wants 99,328 B for two record buffers, the second trade is not one.
=== THE ASYMMETRY: THE CODEC CONTAINER PAYS NONE OF THIS ===============""")
if os.path.exists(a.codec):
sys.path.insert(0, "tools/encoder")
from dlx import DLX
c = DLX(a.codec)
lens = c.record_lengths() if callable(getattr(c, "record_lengths", None)) else c.record_lengths
cwire = sum(lens) / len(lens) * c.fps / 1024
print(f""" {os.path.basename(a.codec)}: {c.nframes} records, index {'PRESENT' if c.has_index else 'absent'},
records already VARIABLE ({min(lens):,}..{max(lens):,} B, mean {sum(lens)/len(lens):,.0f}) and
sector-aligned since DLX5 (60.1). A container that already carries an index
and already has variable records can put EXACTLY {AU_FRAME:,.1f} B of audio in record i
and pad only to the sector it was going to pad to anyway -- so its audio
padding is not 57.3% and not 1.11%, it is ZERO, and its wire goes
{cwire:.1f} -> {cwire + AU_BPS/1024:.1f} KB/s ({100*(AU_BPS/1024)/cwire:+.2f}%).
THAT IS THE FIRST COST THIS PROJECT HAS FOUND FOR THE PACKED BRANCH'S OWN
SIMPLIFICATION. "A record's length is geometry, so there is no index and none
can be needed" (63, 64.1) is what makes the packed player a page of arithmetic
instead of a parser -- and it is exactly the property that makes a second
stream at an unrelated rate cost padding, a cadence, and a buffer. It is a
small cost ({f11[5]-AU_BPS/1024:.2f} KB/s at the pick, {2*f11[3]:,} B of RAM) and it is not zero, and
nothing in FINDINGS 61-64 predicted it.""")
else:
print(f" SKIPPED: no codec container at {a.codec}")
print(f"""
=== WHAT IT DOES TO SLACK (51.3) =======================================
Slack is ACCUMULATED out of pipe - wire, so a second consumer does not cost a
fixed amount -- it costs the accumulation rate, and what a branch point costs is
set by that (51.3, 55.4). Silent vs sounded, at explicit rates:
{'pipe':>8} {'silent':>14} {'sounded':>14} what a second of play banks""")
for kbps in a.rate:
s_sl, a_sl = kbps - vid_kbs, kbps - (vid_kbs + add)
def fmt(x): return f"{x:+8.1f} KB/s" if x >= 0 else f"{x:+8.1f} KB/s"
print(f" {kbps:8.1f} {fmt(s_sl):>14} {fmt(a_sl):>14} "
+ ("both starve" if a_sl < 0 and s_sl < 0
else "SOUND IS WHAT BREAKS IT" if s_sl >= 0 > a_sl
else f"{a_sl/s_sl*100:.0f}% of the silent rate" if s_sl > 0 else ""))
AUCLK_LO = AU_FRAME * B.ADPCM_CLK_BYTE_BEST
AUCLK_HI = AU_FRAME * B.ADPCM_CLK_BYTE_WORST
print(f"""
=== AND WHAT IT DOES TO THE FRAME (the half session 20 already closed) ==
{AU_FRAME:,.1f} B a slot at {B.ADPCM_CLK_BYTE_BEST}..{B.ADPCM_CLK_BYTE_WORST} clocks a byte (the IPL ROM's OWN channel-3
configuration, read out of the ROM by 21_iplrom_dmac.py, not chosen here) is
{AUCLK_LO:,.0f}..{AUCLK_HI:,.0f} clocks = {100*AUCLK_LO/FRAME_CLK:.2f}%..{100*AUCLK_HI/FRAME_CLK:.2f}% of a {SLOT_S*1000:.2f} ms slot.
That reproduces FINDINGS 52 exactly, which is the point of printing it.
THE INTERACTION 52 COULD NOT HAVE HAD is with 64.2's write window. A
DMAC-direct packed player holds the GVRAM window open for the whole data
phase, and an audio channel stealing the bus during that phase makes the phase
LONGER -- so audio does not merely cost clocks, it costs DARKNESS:
extra dark per slot = {100*AUCLK_LO/FRAME_CLK:.2f}%..{100*AUCLK_HI/FRAME_CLK:.2f}% of the slot, on top of
record/(burst x slot), which is already 1.0 at the wire
It is small against a dark fraction that is already 1.0, and it is not small
against K4's {100*227553/FRAME_CLK:.1f}% paint. For the CPU-painted player the audio steals
from the paint and not from the picture, which is the third time this session
the two players have ranked differently on a column that is not clocks.
""")
tools/analysis/33_adpcm_model.py
+123
View File
@@ -0,0 +1,123 @@
#!/usr/bin/env python3
"""What the chip's own decoder model costs the encoder. ROADMAP P6a, after it.
tools/bench/adpcm_run.sh MEASURED four things about the MSM6258 as this machine
models it, and tools/encoder/adpcm.py had a different value for every one:
axis encoder default the chip how it was measured
feed both-high-first both-LOW-first
formula shift terms 1,678 samples, sample-exact
clamp 12-bit 10-bit one model of sixteen matched
init 0 -2
This file prices them, on the same ten seconds of the same stream every audio
figure in this project is quoted against (tmp/au_singe.raw, FINDINGS 65). It
takes an explicit source file rather than defaulting to one, for the same reason
every rate in this tree is an explicit argument (FINDINGS 50).
THE ONE THAT IS NOT A UNIT SLIP is the CLAMP. The other three are conventions:
get one wrong and the decode is wrong, get it right and nothing is lost. A
10-bit accumulator is a smaller container, and it is INSIDE the recursion -- the
predictor cannot represent what will not fit -- so it costs SNR even when the
encoder knows about it and encodes for it. That is a ceiling on this format on
this machine and it is not recoverable by encoding harder.
"""
import math, os, sys
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "encoder"))
import adpcm
RAW = sys.argv[1] if len(sys.argv) > 1 else "tmp/au_singe.raw"
CHIP = dict(variant="terms", order="low", bits=10, init=-2)
ENC = dict(variant="shift", order="high", bits=12, init=0)
def snr_db(ref, got):
num = sum(float(s) * s for s in ref)
den = sum((float(a) - b) ** 2 for a, b in zip(ref, got))
if den == 0: return float("inf")
return 10.0 * math.log10(num / den) if num else float("-inf")
def main():
import struct
if not os.path.exists(RAW):
print(f"no {RAW} -- run tools/encoder/extract_audio.py first")
return 2
pcm = struct.unpack("<%dh" % (os.path.getsize(RAW) // 2), open(RAW, "rb").read())
src12 = [max(-2048, min(2047, x >> 4)) for x in pcm]
print(f"{RAW}: {len(src12):,} samples, peak {max(abs(v) for v in src12)} "
f"in 12-bit units")
print()
print("1. THE COST OF ENCODING FOR THE WRONG CHIP, all four axes at once")
print(" Encode under the encoder's defaults; play it on the chip. The")
print(" nibble ORDER is not a decode parameter -- it decides which nibble")
print(" of each byte the chip takes -- so it is applied by re-reading the")
print(" encoder's own packed bytes the way the chip reads them.")
nib = adpcm.encode(src12, ENC["variant"], init=ENC["init"], bits=ENC["bits"])
same = adpcm.decode(nib, ENC["variant"], init=ENC["init"], bits=ENC["bits"])
data = adpcm.pack(nib, ENC["order"])
asread = list(adpcm.unpack(data, len(nib), CHIP["order"]))
cross = adpcm.decode(asread, CHIP["variant"], init=CHIP["init"], bits=CHIP["bits"])
print(f" encoded and decoded on the encoder's model : {snr_db(src12, same):7.2f} dB")
print(f" encoded on the encoder's, played on the chip: {snr_db(src12, cross):7.2f} dB")
print()
print("2. ONE AXIS AT A TIME, so the bill is itemised rather than lumped")
for name, key, val in (("nibble order", "order", CHIP["order"]),
("delta formula", "variant", CHIP["variant"]),
("clamp", "bits", CHIP["bits"]),
("initial accumulator", "init", CHIP["init"])):
m = dict(ENC); m[key] = val
d = adpcm.pack(nib, ENC["order"])
rd = list(adpcm.unpack(d, len(nib), m["order"]))
got = adpcm.decode(rd, m["variant"], init=m["init"], bits=m["bits"])
print(f" {name:22s} wrong only here: {snr_db(src12, got):7.2f} dB")
print()
print("3. AND THE ONE THAT IS NOT A CONVENTION. Encode FOR the chip -- the")
print(" encoder knows the model and searches against it -- and compare a")
print(" 10-bit accumulator with a 12-bit one on the same seconds.")
for bits in (12, 10):
n = adpcm.encode(src12, CHIP["variant"], init=CHIP["init"], bits=bits)
r = adpcm.decode(n, CHIP["variant"], init=CHIP["init"], bits=bits)
clip = sum(1 for v in r if v in adpcm.clamp_bounds(bits))
print(f" encoded and decoded at {bits}-bit: {snr_db(src12, r):7.2f} dB"
f" ({clip:,} of {len(r):,} samples sit ON the clamp)")
print()
print("4. WHAT THE LEVEL DOES NOW, and it did nothing before (65.1).")
print(" At 12 bits the disc's -13.4 dBFS peak had headroom to spare and")
print(" normalising bought 0.00 dB. A 10-bit accumulator is 4x smaller,")
print(" so the same signal is no longer comfortably inside it.")
peak = max(abs(v) for v in src12)
for name, g in (("as recorded", 1.0),
("scaled to fit 10 bits", 500.0 / peak),
("half of that", 250.0 / peak)):
sc = [max(-512, min(511, int(round(v * g)))) for v in src12]
n = adpcm.encode(sc, CHIP["variant"], init=CHIP["init"], bits=CHIP["bits"])
r = adpcm.decode(n, CHIP["variant"], init=CHIP["init"], bits=CHIP["bits"])
print(f" {name:24s} x{g:5.2f} peak {max(abs(v) for v in sc):4d} "
f"{snr_db(sc, r):7.2f} dB")
print()
print("5. THE HEADROOM, which is the part of this that will bite later.")
hd = 20 * math.log10(511.0 / peak)
print(f" This window peaks at {peak} of the 10-bit accumulator's 511, so it")
print(f" has {hd:.1f} dB of headroom left -- and it is a QUIET passage: the")
print(" disc peaks at -13.4 dBFS here (65.1). A 10-bit accumulator is")
print(f" {20*math.log10(2047.0/511.0):.1f} dB smaller than the 12-bit word the encoder was")
print(" clamping to, so a passage only a few dB louder than this one does")
print(" not fit and the predictor CLIPS inside the recursion. Nothing in")
print(" this project has measured the loudest passage on the disc; until")
print(" something does, the audio level is an OPEN choice and not a")
print(" settled one, and 65.1's `the level is not a lever` is now wrong")
print(" in one direction: it is not a lever UPWARD.")
print()
print(" The rows in 4 are NOT comparable as absolute quality")
print(" -- each is scored against its OWN scaled reference, so what they")
print(" compare is how well the format tracks a signal of that size.")
return 0
if __name__ == "__main__":
sys.exit(main())
tools/analysis/34_packed_audio.py
+217
View File
@@ -0,0 +1,217 @@
#!/usr/bin/env python3
"""DLXP2's GATE: a packed container with sound in it. ROADMAP P6b.
python3 tools/analysis/34_packed_audio.py [packed.dlxp] [--audio tmp/au_singe.raw]
WHAT THIS IS FOR. FINDINGS 65.3 did the arithmetic of putting audio in the
packed container and wrote no byte of one; 66 measured which of sixteen decoder
models the machine's chip runs and priced the axes at up to 25.7 dB. This is
the container those two produce, and the reason it needs a gate of its own is
that NOTHING PARSES A PACKED CONTAINER. A DMA channel copies bytes and has no
opinion about them (62), so a container whose lump is one sector out does not
fail -- it plays 512 B of picture as audio and 512 B of audio as picture, both
of which are things, and a gate that only checked for errors would pass it.
THE FOUR CLAIMS, and each is checked against something that is not the writer:
1. THE FILE IS ITS OWN ARITHMETIC. Every byte of the container is accounted
for by `off_frm + i*rec + (i//F)*A*512` and `off_aud + k*(F*rec + A*512)`
with no byte left over and no byte claimed twice. A per-record read cannot
catch an off-by-one that shifts everything after it; a partition can.
2. THE PICTURE DID NOT MOVE. Interleaving a second stream into a format whose
whole claim is "record i is at LBA0 + i*97" is exactly the change that can
break that claim, so every record is compared against a re-encode of the
same frames with `--audio` off. The silent container is the control.
3. THE BYTES ARE THE ENCODER'S. The lumps, concatenated, are byte-exact
against `adpcm.encode` run again on the same PCM with the same four axes.
4. THE HEADER'S AXES ARE LOAD-BEARING. The stream decodes to the source at
the SNR the encoder reported, and flipping any ONE of the four axes the
header carries collapses it. A header field nothing would notice being
wrong is a comment.
AND THE FINDING IT REPORTS (FINDINGS 67). The padding is not where a reader of
65.3 would put it. A lump is A*512 B of SPACE; F frames of audio is
F*hz/(2*fps) B, which at F=11 is 7,161.4583..., so the PAYLOAD alternates 7,161
and 7,162 and the sector run is 7,168 either way. A player that handed the chip
the whole lump -- the obvious implementation, and the one the phrase "14 sectors
of audio every 11 frames" invites -- would be feeding it 6.54 B a group too
much. That is not waste, which is what padding usually is. It is DRIFT.
"""
import argparse, math, os, struct, sys
sys.path.insert(0, "tools/encoder")
sys.path.insert(0, "tools/analysis")
import adpcm
import dlxp as P
from dlxp import DLXP, SECTOR
ap = argparse.ArgumentParser()
ap.add_argument("container", nargs="?", default="tmp/packed_singe.dlxp")
ap.add_argument("--audio", default="tmp/au_singe.raw")
ap.add_argument("--silent", default="tmp/packed_singe_silent.dlxp",
help="the control: the same frames with --audio off. Built by "
"check.sh; skipped rather than faked when absent")
ap.add_argument("--game-min", type=float, default=22.8,
help="the game's running length, for what the drift comes to")
a = ap.parse_args()
fails = []
def ck(ok, msg):
print((" OK " if ok else " FAIL ") + msg)
if not ok:
fails.append(msg)
d = DLXP(a.container) # every format invariant is checked here
print(f"{a.container}: DLXP{d.version} {d.W}x{d.H} {d.fps}fps {d.nframes} frames, "
f"{'AUDIO' if d.has_audio else 'SILENT'}")
if not d.has_audio:
sys.exit(f"{a.container} carries no audio -- this gate has nothing to check. "
f"Build it with tools/encoder/pack.py --audio")
grp = d.cad_f * d.aud_hz / (2 * d.fps)
print(f"""
=== THE LAYOUT =========================================================
record {d.rec_bytes:,} B = {d.rec_bytes//SECTOR} sectors, lump {d.cad_a*SECTOR:,} B = {d.cad_a} sectors,
cadence F={d.cad_f} A={d.cad_a}, {d.n_lumps} lumps, {d.aud_bytes:,} B of ADPCM at {d.aud_hz:,} Hz
record i = {d.off_frm:,} + i*{d.rec_bytes:,} + (i//{d.cad_f})*{d.cad_a*SECTOR:,}
lump k = {d.off_aud:,} + k*{d.cad_f*d.rec_bytes + d.cad_a*SECTOR:,}
and NEITHER of those is a lookup. A packed record's length is geometry and a
lump's is a cadence, so DLXP2 still has no index and still needs none.""")
# --- 1. the file is its own arithmetic ------------------------------------
# Every byte, partitioned. Not "does record 7 read back" -- an off-by-one that
# shifts the whole stream reads back fine one record at a time.
spans = [(d.frame_off(i), d.rec_bytes, f"record {i}") for i in range(d.nframes)]
spans += [(d.lump_off(k), d.cad_a * SECTOR, f"lump {k}") for k in range(d.n_lumps)]
spans.sort()
pos, overlap, gap = SECTOR, [], []
for off, n, what in spans:
if off < pos: overlap.append(what)
elif off > pos: gap.append((pos, off, what))
pos = max(pos, off + n)
ck(not overlap, f"nothing overlaps ({len(spans)} spans: {d.nframes} records "
f"+ {d.n_lumps} lumps)" + (f" -- {overlap[:3]}" if overlap else ""))
ck(not gap, "no byte between the header and the end belongs to nothing"
+ (f" -- {gap[:3]}" if gap else ""))
ck(pos == len(d.raw), f"the arithmetic ends at {pos:,} and the file is "
f"{len(d.raw):,} B")
ck(all(off % SECTOR == 0 for off, _, _ in spans),
"every record and every lump starts on a 512 B sector -- 58.3/60.1's "
"precondition survives the interleave")
# --- 1b. and the cadence term is load-bearing -----------------------------
# THE FAILURE MODE THIS FORMAT HAS AND THE CODEC'S DOES NOT. A DLX record is
# found through an index and a player that read the wrong entry gets a length
# word that does not parse. A packed record is found by ARITHMETIC and nothing
# parses it, so a player that drops the `(i//F)*A` term reads 97 sectors
# starting 14 sectors early and paints them: the last 14 sectors of the previous
# record, then 83 of this one, shifted down the screen. It is a picture. Here
# is what the gate would be comparing if the term were missing, and it is only
# WRONG from frame F on -- the first group is exempt, which is how an off-by-one
# like this survives a rig that checks frame 0.
blind = [i for i in range(d.nframes)
if d.raw[d.off_frm + i*d.rec_bytes:d.off_frm + (i+1)*d.rec_bytes]
!= d.record(i)]
ck(blind == list(range(d.cad_f, d.nframes)),
f"a cadence-blind player reads the wrong bytes for {len(blind)} of "
f"{d.nframes} records, first at frame {blind[0] if blind else '-'} -- and "
f"frames 0..{d.cad_f-1} are IDENTICAL either way, so frame 0 proves nothing")
# --- 2. the picture did not move ------------------------------------------
if os.path.exists(a.silent):
q = DLXP(a.silent)
same = (q.nframes == d.nframes
and all(q.record(i) == d.record(i) for i in range(d.nframes)))
ck(same, f"all {d.nframes} records byte-exact against the SILENT control "
f"({os.path.basename(a.silent)}) -- interleaving audio moved no "
f"picture byte")
ck(q.has_audio is False and q.off_frm == SECTOR,
"and the control really is silent: no audio flag, record 0 at sector 1")
else:
print(f" SKIPPED: no silent control at {a.silent}")
# --- 3. the bytes are the encoder's ---------------------------------------
if os.path.exists(a.audio):
raw = open(a.audio, "rb").read()
pcm = struct.unpack("<%dh" % (len(raw) // 2), raw)
src = [max(-2048, min(2047, x >> 4)) for x in pcm]
axes = d.decoder()
ck(axes == adpcm.CHIP, f"the header's four axes ARE adpcm.CHIP: {axes}")
nib = adpcm.encode(src, variant=axes["variant"], init=axes["init"],
bits=axes["bits"])
want = adpcm.pack(nib, order=axes["order"])[:d.aud_bytes]
got = d.audio()
ck(got == want, f"the {len(got):,} B the lumps carry are byte-exact against "
f"adpcm.encode on the same PCM")
ck(all(d.lump(k, padding=True)[len(d.lump(k)):] == b"\0" * (
d.cad_a * SECTOR - len(d.lump(k))) for k in range(d.n_lumps)),
"and every lump's padding is zero, so a player that overruns the payload "
"feeds the chip silence rather than the next lump's first sample")
# --- 4. the header's axes are load-bearing ----------------------------
def snr(axes_):
rec = adpcm.decode(adpcm.unpack(got, len(src), order=axes_["order"]),
variant=axes_["variant"], init=axes_["init"],
bits=axes_["bits"])
n = min(len(rec), len(src))
e = sum((x - y) ** 2 for x, y in zip(src[:n], rec[:n]))
s = sum(x * x for x in src[:n])
return 10 * math.log10(s / e) if e else float("inf")
right = snr(axes)
ck(right > 20.0, f"decoded on the axes the header names: {right:.2f} dB")
print(f"\n AND EVERY AXIS IS A NEGATIVE CONTROL -- flip ONE and this is what\n"
f" a player that ignored the header would hear:\n")
print(f" {'axis':<12} {'header':>8} {'flipped to':>11} {'SNR':>9} cost")
flips = [("order", "high" if axes["order"] == "low" else "low"),
("variant", "terms" if axes["variant"] == "shift" else "shift"),
("bits", 12 if axes["bits"] == 10 else 10),
("init", 0 if axes["init"] else -2)]
for k, v in flips:
w = dict(axes); w[k] = v
s2 = snr(w)
print(f" {k:<12} {str(axes[k]):>8} {str(v):>11} {s2:9.2f} dB "
f"{s2-right:+.2f} dB")
if k in ("order", "variant"):
ck(s2 < right - 2.0, f"axis '{k}' is load-bearing: {s2-right:+.2f} dB")
else:
print(f" SKIPPED: no PCM at {a.audio} -- the bytes were not re-derived")
# --- the finding ----------------------------------------------------------
per = d.cad_a * SECTOR - grp
print(f"""
=== THE PAYLOAD IS NOT THE LUMP (FINDINGS 67) ==========================
A lump is {d.cad_a*SECTOR:,} B of SPACE. {d.cad_f} frames of audio is {grp:,.4f} B, so the
PAYLOAD is {P.lump_bytes(0, d.cad_f, d.fps, d.aud_hz):,} or {P.lump_bytes(2, d.cad_f, d.fps, d.aud_hz):,} -- the same remainder FINDINGS 54's frame
clock carries, one dimension over -- and the last {per:.4f} B are zero.
A PLAYER THAT FED THE CHIP THE WHOLE LUMP would hand it {per:.2f} B a group it
should not have. At {d.aud_hz:,} Hz that is {2*per/d.aud_hz*1000:.2f} ms of audio every
{d.cad_f/d.fps:.4f} s, which is {100*per/grp:.3f}% -- and it does not average out, it ACCUMULATES:""")
for mins in (1.0, a.game_min):
print(f" {mins:5.1f} min of play -> {mins*60*(per/grp):.2f} s of lip-sync error")
print(f""" so the cadence's {100*per/grp:.3f}% is not the waste figure 65.3 called it and left
at that. It is waste ON THE WIRE and DRIFT IN THE PLAYER, and the second is
the expensive one: {a.game_min:.1f} minutes is {a.game_min*60*(per/grp):.2f} s, which is a scene of dialogue
arriving after the mouth that spoke it.
WHAT A PLAYER CARRIES INSTEAD IS ONE ACCUMULATOR, and it is three
instructions rather than a table:
acc += {d.cad_f}*{d.aud_hz:,} ; = {d.cad_f*d.aud_hz:,}
n = acc // {2*d.fps} ; the MTC for this lump's channel
acc %= {2*d.fps}
which is exactly clock.i's shape (54) and for exactly the same reason: a rate
with a denominator of {2*d.fps} cannot be a count, so it is a remainder.
=== THE WIRE ===========================================================
video {d.video_kbps():7.1f} KB/s FIXED by geometry
audio {d.audio_kbps():7.2f} KB/s the CADENCE's, padding included -- the disc moves
whole sectors and the wire pays for the zero ones
total {d.kbps():7.1f} KB/s ({100*(d.kbps()/d.video_kbps()-1):+.2f}%), and 65.3 predicted {589.6:.1f}
""")
print(f"{'FAIL' if fails else 'OK'} 34_packed_audio: {len(fails)} failure(s)")
sys.exit(1 if fails else 0)
tools/analysis/35_audio_level.py
+496
View File
@@ -0,0 +1,496 @@
#!/usr/bin/env python3
"""HOW LOUD IS THE DISC? ROADMAP P6, the item FINDINGS 66.3 reopened.
python3 tools/analysis/35_audio_level.py [--streams 00000-00201] [--json out]
FINDINGS 66 asked MAME's MSM6258 which decoder it is and got four axes back.
The one with a bill attached is the CLAMP: the chip's accumulator is **10 bits**
and it clamps INSIDE the recursion, so the reachable set of reconstructed
samples is [-512, 511] in the 12-bit units everything in this project counts in
-- a quarter of the 12-bit word `adpcm.py` used to clamp at.
`pack.py` hands the encoder `s16 >> 4`, i.e. it maps the disc's full scale onto
the 12-bit word, and 66.3 measured the Singe window peaking at **435 of 511**.
That fit with 1.4 dB to spare, and it fit BY ACCIDENT: the window is a -13.4
dBFS passage. Any passage more than 1.4 dB louder does not merely distort at the
top, it drives the predictor -- a clamped accumulator is a WRONG STATE that the
next nibble is applied to, so the error outlives the loud sample.
So the level cannot be chosen from the ten seconds this project gates on. It has
to be chosen from the loudest thing the game will ever play, and this file
measures that: every stream of the unique scene footage -- `00000`-`00201`,
1366.6 s, FINDINGS 32.1 -- through the SAME chain `extract_audio.py` uses (AC-3
5.1, ffmpeg's default downmix matrix, mono, 15,625 Hz), because a level measured
through a different resampler is a level for a different encoder.
Two statistics, and the difference between them is the whole argument:
PEAK max |x| over the disc. What must fit under 511 for NOTHING to clamp.
PASSAGE the loudest ~1 s window's peak and RMS. What the ear gets. A single
sample 6 dB above everything else is a click and costs one clamp; a
passage 6 dB above the gate window is where the recursion lives for
fifteen thousand samples.
It prints the attenuation each choice implies, in dB and as the shift `pack.py`
would have to make, and it does NOT choose. Choosing needs the other half --
what attenuation costs at the quiet end, where the OKI step table's floor of 16
(12-bit units) does not scale with the signal -- and that is `--ladder`, which
encodes real passages at real gains with `adpcm.CHIP` and reports the SNR.
"""
import argparse, getpass, json, os, subprocess, sys
import numpy as np
sys.path.insert(0, "tools/encoder")
import adpcm
BDROM = os.environ.get("DLX_BDROM") or f"/media/{getpass.getuser()}/BDROM"
STREAM_DIR = f"{BDROM}/BDMV/STREAM"
HZ = 15625 # the chip's rate, and the only one budgeted for
FPS = 12
LUMP_FRAMES = 11 # FINDINGS 65.3's cadence: 11 frames of audio
WIN = LUMP_FRAMES * HZ // FPS # 14,322 samples ~ 0.917 s -- one audio lump
HOP = HZ // 4 # 0.25 s blocks; the window is 4 of them (rounded)
CLAMP_LO, CLAMP_HI = adpcm.clamp_bounds(adpcm.CHIP["bits"]) # -512, 511
FULL12 = 2048 # what `s16 >> 4` maps full scale to
def db(x, ref=FULL12):
return -np.inf if x <= 0 else 20 * np.log10(x / ref)
_PCM_CACHE = {}
def pcm12(stream, start=None, dur=None):
"""One stream as 12-bit signed samples, through extract_audio.py's chain.
Cached, because the scan, the census and the event walk are three passes
over the same 20 million samples and the whole game is 40 MB of int16.
"""
ck = (stream, start, dur)
if ck in _PCM_CACHE:
return _PCM_CACHE[ck]
cmd = ["ffmpeg", "-v", "error"]
if start is not None: cmd += ["-ss", str(start)]
if dur is not None: cmd += ["-t", str(dur)]
cmd += ["-i", f"{STREAM_DIR}/{stream}.m2ts", "-vn", "-ac", "1",
"-ar", str(HZ), "-f", "s16le", "-acodec", "pcm_s16le", "-"]
p = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
if p.returncode:
# 00176 is 3.0 s of mpeg2video with NO AUDIO TRACK AT ALL. That is a
# fact about the disc, not a failure here, so it is reported rather
# than swallowed -- but anything else is a real error.
if b"does not contain any stream" not in p.stderr:
raise SystemExit(f"ffmpeg failed on {stream}: "
f"{p.stderr.decode(errors='replace')[:400]}")
_PCM_CACHE[ck] = None
return None
x = np.frombuffer(p.stdout, "<i2").astype(np.int32)
# The SAME requantisation pack.py makes. It is a shift and not a divide, so
# it floors toward -inf, and that asymmetry is real: -1 >> 4 is -1.
out = np.clip(x >> 4, -FULL12, FULL12 - 1).astype(np.int16)
_PCM_CACHE[ck] = out
return out
def scan(streams):
"""Per-stream peak and loudest-passage statistics, in 12-bit units."""
rows, mute = [], []
for s in streams:
x = pcm12(s)
if x is None:
mute.append(s)
continue
if x.size == 0:
continue
a = np.abs(x).astype(np.float64)
nb = a.size // HOP
if nb:
bmax = a[:nb * HOP].reshape(nb, HOP).max(1)
bsq = (a[:nb * HOP].reshape(nb, HOP) ** 2).sum(1)
k = max(1, round(WIN / HOP))
if nb >= k:
# sliding sum over k blocks == the ~1 s lump window
cs = np.concatenate(([0.0], np.cumsum(bsq)))
wrms = np.sqrt((cs[k:] - cs[:-k]) / (k * HOP))
wpk = np.array([bmax[i:i + k].max() for i in range(nb - k + 1)])
else:
wrms = np.array([np.sqrt((a ** 2).mean())])
wpk = np.array([a.max()])
else:
wrms = np.array([np.sqrt((a ** 2).mean())])
wpk = np.array([a.max()])
ipk = int(np.argmax(a))
irms = int(np.argmax(wrms))
rows.append(dict(stream=s, n=int(x.size), secs=x.size / HZ,
peak=float(a.max()), peak_t=ipk / HZ,
rms=float(np.sqrt((a ** 2).mean())),
wpeak=float(wpk.max()),
wrms=float(wrms.max()), wrms_t=irms * HOP / HZ))
return rows, mute
def report(rows, mute):
tot = sum(r["secs"] for r in rows)
peak = max(rows, key=lambda r: r["peak"])
loud = max(rows, key=lambda r: r["wrms"])
disc_peak = peak["peak"]
print(f"=== THE DISC, {len(rows)} streams, {tot:,.1f} s = {tot/60:.1f} min "
f"(FINDINGS 32.1 says 1,366.6) ===\n")
if mute:
print(f" {len(mute)} stream(s) carry NO AUDIO TRACK: {', '.join(mute)}"
f" -- a fact about the disc, and a case a shipping encoder has\n"
f" to have an answer for (silence of the right length).\n")
print(f'{"stream":>8}{"secs":>8}{"peak":>7}{"dBFS":>8}{"passage pk":>12}'
f'{"passage rms":>13}{"dBFS":>8} at')
for r in sorted(rows, key=lambda r: -r["wrms"])[:12]:
print(f'{r["stream"]:>8}{r["secs"]:8.1f}{r["peak"]:7.0f}{db(r["peak"]):8.2f}'
f'{r["wpeak"]:12.0f}{r["wrms"]:13.1f}{db(r["wrms"]):8.2f}'
f' {r["wrms_t"]:6.2f} s')
print(" (the twelve loudest PASSAGES; the table is sorted by the window "
"RMS, not the peak)\n")
print(f" DISC PEAK {disc_peak:.0f} of {FULL12} = {db(disc_peak):.2f} dBFS"
f" ({peak['stream']} @ {peak['peak_t']:.2f} s)")
print(f" LOUDEST PASSAGE peak {loud['wpeak']:.0f}, rms {loud['wrms']:.1f}"
f" = {db(loud['wrms']):.2f} dBFS ({loud['stream']} @ {loud['wrms_t']:.2f} s)")
print(f" THE CLAMP +{CLAMP_HI} / {CLAMP_LO} (it is not symmetric), "
f"{db(CLAMP_HI):.2f} dBFS in the same units\n")
need = disc_peak / CLAMP_HI
print("=== WHAT THAT COSTS, AS A LEVEL ===\n")
print(f" `s16 >> 4` is what pack.py does today and it puts the disc's own")
print(f" peak at {disc_peak:.0f} against a clamp of {CLAMP_HI}: "
f"{'OVER by' if need > 1 else 'under by'} {abs(20*np.log10(need)):.2f} dB.")
print(f" Fitting the whole disc under the clamp with no sample clamped at")
print(f" all needs a gain of {1/need:.4f} = {-20*np.log10(need):.2f} dB, i.e.")
for sh in (4, 5, 6, 7):
pk = disc_peak / (1 << (sh - 4))
mark = " <- fits" if pk <= CLAMP_HI else ""
# ~ because a further right shift floors again and this halves; the
# difference is one count and the column is a signpost, not a spec.
print(f" s16 >> {sh} disc peak ~{pk:7.1f} "
f"{'clamps' if pk > CLAMP_HI else 'clear':>6} by "
f"{abs(20*np.log10(pk/CLAMP_HI)):5.2f} dB{mark}")
print()
# How much of the disc is actually above the clamp at today's level: the
# number that decides whether this is a level question or a limiter question.
return dict(rows=rows, disc_peak=disc_peak, peak_stream=peak["stream"],
peak_t=peak["peak_t"], loud_stream=loud["stream"],
loud_t=loud["wrms_t"], loud_rms=loud["wrms"],
loud_wpeak=loud["wpeak"], clamp=CLAMP_HI, mute=mute)
def clip_census(rows, streams, gains):
"""At each candidate gain, how many samples of the WHOLE DISC clamp?
A peak is one number and this is the distribution behind it. A gain that
clamps 12 samples in 22 minutes is a different object from one that clamps
thousands, and the peak alone cannot tell them apart.
"""
print("=== THE CENSUS: how much of the disc is ABOVE the clamp, by gain ===\n")
print(f'{"gain":>8}{"dB":>8}{"samples over":>14}{"of":>12}{"share":>10}'
f'{"worst over":>12}')
tot = 0
over = {g: 0 for g in gains}
worst = {g: 0.0 for g in gains}
for s in streams:
x = pcm12(s)
if x is None:
continue
a = np.abs(x).astype(np.float64)
tot += a.size
for g in gains:
# ROUNDED, exactly as the ladder and pack.py requantise. Comparing
# the float product instead makes 511/946 report one sample over
# its own clamp, which is arithmetic about floats and not about
# the disc.
v = np.round(a * g)
m = v > CLAMP_HI
over[g] += int(m.sum())
if m.any():
worst[g] = max(worst[g], float(v.max() / CLAMP_HI))
for g in gains:
w = f"{20*np.log10(worst[g]):.2f} dB" if worst[g] else "-"
print(f'{g:8.4f}{20*np.log10(g):8.2f}{over[g]:14,}{tot:12,}'
f'{100*over[g]/tot:9.4f}%{w:>12}')
print()
return dict(total=tot, over={f"{g:.4f}": over[g] for g in gains})
def clamp_events(streams, gain=1.0):
"""WHERE the over-clamp samples are, not just how many.
687 isolated samples in 22 minutes and one sustained 44 ms burst are the
same census row and completely different sounds, and a clamp inside a
recursion is not a clipped sample -- it is a wrong predictor state that the
next nibble is applied to. So the run lengths are the statistic.
"""
runs = []
for st in streams:
x = pcm12(st)
if x is None:
continue
m = np.round(np.abs(x).astype(np.float64) * gain) > CLAMP_HI
if not m.any():
continue
d = np.diff(np.concatenate(([0], m.view(np.int8), [0])))
beg = np.where(d == 1)[0]
end = np.where(d == -1)[0]
for b, e in zip(beg, end):
runs.append((int(e - b), st, b / HZ))
runs.sort(reverse=True)
n = sum(r[0] for r in runs)
print(f"=== WHERE THE CLAMPS ARE at gain {gain:.4f} "
f"({len(runs)} events, {n:,} samples = {1000*n/HZ:.1f} ms) ===\n")
print(f'{"run":>6}{"ms":>8} stream at')
for r, st, t in runs[:10]:
print(f'{r:6}{1000*r/HZ:8.2f} {st} {t:7.2f} s')
if runs:
print(f" longest run {runs[0][0]} samples = {1000*runs[0][0]/HZ:.2f} ms; "
f"median run {sorted(r[0] for r in runs)[len(runs)//2]}")
print()
return dict(events=len(runs), samples=n,
longest=runs[0][0] if runs else 0)
def ladder(where, gains, dur, label):
"""Encode a real passage at each gain with adpcm.CHIP and report the SNR.
This is the half a peak measurement cannot do. Attenuation buys headroom at
the top and spends resolution at the bottom, because the OKI step table's
floor is a constant 16 in 12-bit units and does not scale with the signal.
The SNR is reported against the SCALED source, which is the honest
comparison: the encoder's job is to reproduce what it was handed, and the
listener's volume knob is not this project's problem.
"""
stream, start = where
x = pcm12(stream, start, dur)
print(f"=== THE LADDER: {label} -- {stream} @ {start:.2f} s, {dur:.2f} s, "
f"{x.size:,} samples ===\n")
print(f'{"gain":>8}{"dB":>8}{"src peak":>10}{"clamped":>9}{"SNR dB":>9}'
f'{"vs 1.0":>8}')
base = None
out = []
for g in gains:
src = np.clip(np.round(x * g), -FULL12, FULL12 - 1).astype(int).tolist()
nib = adpcm.encode(src, variant=adpcm.CHIP["variant"],
init=adpcm.CHIP["init"], bits=adpcm.CHIP["bits"])
rec = adpcm.decode(nib, variant=adpcm.CHIP["variant"],
init=adpcm.CHIP["init"], bits=adpcm.CHIP["bits"])
s = np.array(src, dtype=np.float64)
r = np.array(rec, dtype=np.float64)
e = ((s - r) ** 2).sum()
snr = 10 * np.log10((s ** 2).sum() / e) if e else np.inf
nclamp = int((np.abs(s) > CLAMP_HI).sum())
if base is None:
base = snr
print(f'{g:8.4f}{20*np.log10(g):8.2f}{np.abs(s).max():10.0f}{nclamp:9,}'
f'{snr:9.2f}{snr-base:+8.2f}')
out.append(dict(gain=g, snr=snr, clamped=nclamp,
peak=float(np.abs(s).max())))
print()
return out
def survey(rows, gains, n, dur, seed=20260825):
"""THE DISC, not three passages of it.
Three hand-picked passages can be argued with; a sample cannot. `n` windows
are drawn uniformly over the game's own timeline -- weighted by stream
length, so a 24 s stream gets twenty times the draws of a 1.2 s one -- and
every one is encoded at every gain with `adpcm.CHIP`. What is reported is
the distribution: the mean SNR is what the level costs on average, and the
WORST window is what it costs where it matters, because a level is chosen
for the passage it fails on.
"""
rng = np.random.default_rng(seed)
pool = [r for r in rows if r["secs"] >= dur]
w = np.array([r["secs"] for r in pool], dtype=np.float64)
w /= w.sum()
picks = []
for _ in range(n):
r = pool[int(rng.choice(len(pool), p=w))]
t = float(rng.uniform(0, r["secs"] - dur))
picks.append((r["stream"], t))
print(f"=== THE SURVEY: {n} windows of {dur:.1f} s drawn over the whole "
f"{sum(r['secs'] for r in rows)/60:.1f} min, encoded at every gain ===\n")
src = [pcm12(st, t, dur) for st, t in picks]
print(f'{"gain":>8}{"dB":>8}{"mean SNR":>10}{"median":>9}{"WORST":>8}'
f'{"windows w/ clamp":>18}{"samples":>9}')
out = []
for g in gains:
snrs, nclamp, ncw = [], 0, 0
for x in src:
v = np.clip(np.round(x * g), -FULL12, FULL12 - 1).astype(int)
k = int((np.abs(v) > CLAMP_HI).sum())
nclamp += k
ncw += 1 if k else 0
nib = adpcm.encode(v.tolist(), variant=adpcm.CHIP["variant"],
init=adpcm.CHIP["init"], bits=adpcm.CHIP["bits"])
rec = np.array(adpcm.decode(nib, variant=adpcm.CHIP["variant"],
init=adpcm.CHIP["init"],
bits=adpcm.CHIP["bits"]), dtype=np.float64)
f = v.astype(np.float64)
e = ((f - rec) ** 2).sum()
snrs.append(10 * np.log10((f ** 2).sum() / e) if e else np.inf)
a = np.array(snrs)
print(f'{g:8.4f}{20*np.log10(g):8.2f}{a.mean():10.2f}'
f'{np.median(a):9.2f}{a.min():8.2f}{ncw:14} of {len(src)}{nclamp:9,}',
flush=True)
out.append(dict(gain=g, mean=float(a.mean()), median=float(np.median(a)),
worst=float(a.min()), clamped=nclamp, windows=ncw))
print()
return out
def recover(stream, gain, control, K=64):
"""DOES A CLAMP OUTLIVE THE SAMPLE IT HAPPENS ON? 66.3 said it would.
The worry was exact and it is the right worry for a recursive codec: a
clamped accumulator is a WRONG STATE and the next nibble is applied to it,
so the error should persist after the loud sample has gone. Measuring the
error after a clamp run does show it elevated -- and that is not evidence,
because the samples after a clamp run are LOUD samples, where the step is
large and the error is large anyway.
So the control is the same window at the gain that never clamps, rescaled
to the same units and read at the SAME sample indices. What the ratio
isolates is the clamp and nothing else.
"""
x = pcm12(stream).astype(np.float64)
def enc(g):
src = np.clip(np.round(x * g), -FULL12, FULL12 - 1).astype(int)
nib = adpcm.encode(src.tolist(), variant=adpcm.CHIP["variant"],
init=adpcm.CHIP["init"], bits=adpcm.CHIP["bits"])
rec = adpcm.decode(nib, variant=adpcm.CHIP["variant"],
init=adpcm.CHIP["init"], bits=adpcm.CHIP["bits"])
return src.astype(np.float64), np.array(rec, dtype=np.float64)
s1, r1 = enc(gain)
s2, r2 = enc(control)
e1 = np.abs(s1 - r1)
e2 = np.abs(s2 - r2) / control * gain # the control, in gain's units
m = np.abs(s1) > CLAMP_HI
d = np.diff(np.concatenate(([0], m.view(np.int8), [0])))
ends = [e for e in np.where(d == -1)[0] if e + K <= e1.size]
p1 = np.array([e1[e:e + K] for e in ends], dtype=np.float64).mean(0)
p2 = np.array([e2[e:e + K] for e in ends], dtype=np.float64).mean(0)
print(f"=== DOES THE CLAMP OUTLIVE THE SAMPLE? {stream}, gain {gain:g} "
f"against a control at {control:g} ===\n")
print(f" {int(m.sum())} samples clamp in {len(ends)} runs; the profile is "
f"the mean |error| at each\n offset after a run ENDS, in 12-bit units, "
f"against the same offsets of a\n window that never clamps at all.\n")
print(f'{"after":>7}{"clamped":>10}{"control":>10}{"ratio":>8}')
for i in (0, 1, 2, 4, 8, 16, 32, K - 1):
print(f'{"+" + str(i):>7}{p1[i]:10.2f}{p2[i]:10.2f}{p1[i]/p2[i]:8.2f}')
off = ~m
print(f'\n off-clamp mean |err| {e1[off].mean():.2f} vs {e2[off].mean():.2f}')
print(f' whole-window mean |err| {e1.mean():.2f} vs {e2.mean():.2f}')
print(f' worst ratio over the {K} offsets: {(p1/p2).max():.2f}\n')
return dict(stream=stream, gain=gain, control=control,
runs=len(ends), clamped=int(m.sum()),
worst_ratio=float((p1 / p2).max()),
mean_err=float(e1.mean()), mean_err_control=float(e2.mean()))
def main():
ap = argparse.ArgumentParser()
ap.add_argument("--first", type=int, default=0)
ap.add_argument("--last", type=int, default=201,
help="the unique scene footage is 00000-00201 (FINDINGS 32.1); "
"00215/00216/00223 are compilations of the same material")
ap.add_argument("--ladder", action="store_true",
help="also encode the loudest and a quiet passage at each gain")
ap.add_argument("--dur", type=float, default=4.0, help="ladder passage seconds")
ap.add_argument("--survey", type=int, default=0,
help="encode N windows drawn over the whole game at each gain")
ap.add_argument("--survey-dur", type=float, default=2.0)
ap.add_argument("--recover", action="store_true",
help="does a clamp outlive its sample? 66.3 said it would")
ap.add_argument("--gate", action="store_true",
help="assert FINDINGS 69's headline numbers, exit 1 if not")
ap.add_argument("--json")
a = ap.parse_args()
streams = [f"{i:05d}" for i in range(a.first, a.last + 1)]
streams = [s for s in streams if os.path.exists(f"{STREAM_DIR}/{s}.m2ts")]
if not streams:
sys.exit(f"no streams under {STREAM_DIR} -- is the Blu-ray mounted? "
f"(DLX_BDROM)")
rows, mute = scan(streams)
summary = report(rows, mute)
# The odd one is not a round number and is not meant to be: it is
# CLAMP/disc peak, the gain at which the disc's own loudest sample lands
# EXACTLY on the clamp, computed from the scan rather than typed in.
exact = round(CLAMP_HI / summary["disc_peak"], 4)
gains = sorted({1.0, 0.7071, exact, 0.5, 0.3536, 0.25}, reverse=True)
summary["exact_gain"] = exact
summary["census"] = clip_census(rows, streams, gains)
summary["events"] = clamp_events(streams, 1.0)
if a.ladder:
# Three passages, because they answer three different questions.
# PEAK what CLAMPING costs, since this is the only place on the disc
# that clamps at today's level.
# LOUD the loudest sustained window that is long enough to encode.
# QUIET what ATTENUATION costs, which is the other end of the same
# decision and the reason -15 dB is not free.
long = [r for r in rows if r["secs"] >= a.dur]
pk = max(rows, key=lambda r: r["peak"]) # the DISC peak, however short
loud = max(long, key=lambda r: r["wrms"])
quiet = min(long, key=lambda r: r["wrms"])
at = lambda r, t: (r["stream"], min(max(0.0, t - a.dur / 2),
max(0.0, r["secs"] - a.dur)))
pkdur = min(a.dur, pk["secs"])
summary["ladder_peak"] = ladder(
(pk["stream"], min(max(0.0, pk["peak_t"] - pkdur / 2),
max(0.0, pk["secs"] - pkdur))),
gains, pkdur, "THE DISC PEAK ITSELF")
summary["ladder_loud"] = ladder(at(loud, loud["wrms_t"]), gains, a.dur,
"THE LOUDEST SUSTAINED PASSAGE")
summary["ladder_quiet"] = ladder(at(quiet, quiet["wrms_t"]), gains, a.dur,
"A QUIET PASSAGE, for the other end")
if a.survey:
summary["survey"] = survey(rows, gains, a.survey, a.survey_dur)
if a.recover:
summary["recover"] = recover(summary["peak_stream"], 1.0, exact)
if a.gate:
expect = dict(disc_peak=946.0, peak_stream="00200", clamp=511,
events=402, over=687)
bad = []
for k, v in expect.items():
got = (summary["events"]["events"] if k == "events" else
summary["events"]["samples"] if k == "over" else summary[k])
if got != v:
bad.append(f"{k}: expected {v}, measured {got}")
if bad:
print("LEVEL GATE RED -- the disc does not measure as FINDINGS 69 "
"recorded it:")
for b in bad:
print(" " + b)
print(" (a different pressing is a legitimate cause; a different "
"ffmpeg downmix is not)")
sys.exit(1)
print("LEVEL GATE GREEN: disc peak 946 of 2048 at 00200, 5.35 dB over "
"the chip's 511,\n 687 samples in 402 events = 44.0 ms of the "
"game's 21.5 min of audio.")
if a.json:
json.dump(summary, open(a.json, "w"), indent=1)
print(f"-> {a.json}")
if __name__ == "__main__":
main()
tools/analysis/36_branch_audio.py
+440
View File
@@ -0,0 +1,440 @@
#!/usr/bin/env python3
"""51.3's REFILL CLIMB WITH A SECOND CONSUMER, THROUGH A REAL BRANCH POINT.
python3 tools/analysis/36_branch_audio.py --kbps R [R ...]
[--ring KB ...] [--gate]
The oldest open item in ROADMAP P6, named by FINDINGS 65.6 and again by 67.6:
"the slack table is here, but 51.3's refill climb with a second consumer
through a real branch point is not."
Everything it needs already exists and none of it has ever been put in the same
room:
* 51.3 -- slack is ACCUMULATED out of `pipe - wire`, at `pipe - wire` bytes a
second, and a seek spends all of it. `tools/analysis/20_seek_slack.py`.
* 56.3 -- where the branch points ARE: 612 distinct transitions into a seek
over the arcade's own graph, worst gap 0.000 s, median 3.473 s.
`tools/analysis/25_scene_graph.py`, reading only DLXSCENE1.
* 56.4 -- the climb against that distribution, for the codec container. It
charged audio as `ratectl.AUDIO_KBPS`, a flat 7.8 KB/s placeholder that
predates any of the audio work.
* 65.3/67.1 -- what a second consumer ACTUALLY costs a container: a fixed
cadence of F frames per A sectors, because a packed record's address is
arithmetic and cannot be an index. `tools/analysis/32_audio_wire.py`.
* 68 -- the player that holds both streams at once, and its buffers.
Three questions, and the tree has never asked any of them:
1. What does the SECOND CONSUMER do to the climb? Not to the wire -- 32
answered that and it is 1.3% -- but to `pipe - wire`, which is a small
difference of two large numbers and is the thing the climb is made of.
2. What is the climb on the PACKED branch? This is the branch the player
runs (68) and the one B1's acceptance is written against.
3. What does the CADENCE do at a branch point? A group is `lump k, then F
records`, so lump k sits at a LOWER address than every record in its group
but the first. A seek to record i lands inside a group whose audio is
BEHIND it. Nobody has ever priced entering a group off-boundary, and the
game's own seek targets say how often it happens.
THE FRAME INDEX OF A SEEK TARGET IS A DESIGN ASSUMPTION AND IS LABELLED ONE.
DLXSCENE1 carries positions on the laserdisc timeline in ms. This tree's design
puts ONE CONTAINER PER SCENE -- 53 and 55.1 charge a scene change 6,164 header
bytes, and 56.3 counts 203 of the 612 transitions as container changes for
exactly that reason -- so a seek target's frame index inside its container is
`(target start - the scene's own earliest start) * fps / 1000`. If the design
ever puts one container per SEQUENCE instead, every seek lands on frame 0, the
group offset is always zero and section 3 collapses to nothing. That is the
assumption, said out loud, in the one place the answer depends on it.
"""
import sys, os, json, argparse, importlib.util
sys.path.insert(0, "tools/encoder")
sys.path.insert(0, "tools/analysis")
TABLE = os.environ.get("DLX_SCENEGRAPH", "tmp/scenegraph.json")
FPS = 12
CHIP_HZ = 15625.0 # MSM6258V, 8 MHz / 512 (FINDINGS 32, 65)
AU_BPS = CHIP_HZ / 2 # 4 bits a sample, two samples to a byte
AU_FRAME = AU_BPS / FPS # 651.0416... B a slot, and the dots are 65.3
SECTOR = 512
def load(path, name):
spec = importlib.util.spec_from_file_location(name, path)
m = importlib.util.module_from_spec(spec)
spec.loader.exec_module(m)
return m
def cadence(F):
"""Best A for this F: the fewest whole sectors that hold F frames of chip."""
num, den = int(round(AU_BPS * 2)), 2 * SECTOR * FPS
A = -(-(num * F) // den) # ceil(F * bytes/sector)
return A, A * SECTOR
def climb_s(records, rec_b, pipe_kbps, wire_kbps):
"""Seconds of play to accumulate `records` records of lookahead (51.3)."""
surplus = (pipe_kbps - wire_kbps) * 1024.0
return float("inf") if surplus <= 0 else records * rec_b / surplus
# --------------------------------------------------------------- the branches
def branch_points(doc, nodes, sg):
"""The 612 transitions into a seek, plus each one's target frame index.
Returns (gaps_s, within, changes). `within` entries carry the frame index
the seek lands on inside its container; `changes` are scene changes, which
land on frame 0 of a new one.
"""
gaps, _ = sg.worst_gap(nodes)
play = [g for g in gaps if g[5] != "attract_mode"]
scene_start = {}
for scene, seqs in doc["scenes"].items():
st = [s["start_ms"] for s in seqs.values() if s["start_ms"] >= 0]
scene_start[scene] = min(st) if st else None
within, changes = [], []
for g, src, tgt, kind, ends, scene in play:
if ends:
changes.append((g / 1000.0, src))
continue
n = nodes.get(f"{scene}.{tgt}")
if n is None or not n.seeks or scene_start[scene] is None:
continue
i = int(round((n.start - scene_start[scene]) / 1000.0 * FPS))
within.append((g / 1000.0, max(0, i), f"{scene}.{tgt}"))
return sorted(g[0] / 1000.0 for g in play), within, changes
def silences(within, F):
"""Ms of silence entering each branch's group off-boundary, at cadence F.
A group is `lump k, then F records`. Seek to record i, take the next lump
that lies AHEAD of the read point -- lump k+1, which arrives at frame
(k+1)*F -- and the frames from i to (k+1)*F-1 have no audio. The chip's
second is a real second (15,625 samples, 2 to a byte), so the missing
time is exactly `(F - i mod F) mod F` frames of 1/12 s and none of
FINDINGS 54's frame-clock remainder gets into it.
"""
return sorted(((F - (i % F)) % F) / FPS * 1000.0 for _, i, _ in within)
def pct(xs, p):
return xs[min(len(xs) - 1, int(p * len(xs)))] if xs else float("nan")
def main():
ap = argparse.ArgumentParser()
ap.add_argument("--kbps", type=float, nargs="+",
help="delivered pipe rates, KB/s. REQUIRED (FINDINGS 50) "
"unless --gate, which carries its own.")
ap.add_argument("--ring", type=float, nargs="+", default=[256, 512])
ap.add_argument("--pkbps", type=float, nargs="+",
default=[589.6, 600.0, 650.0, 700.0, 900.0],
help="pipe rates for the PACKED table. Its own list, "
"because a packed record is 1.3x the codec's and "
"every rate that serves one starves the other.")
ap.add_argument("--table", default=TABLE)
ap.add_argument("--packed", default="tmp/packed_singe.dlxp")
ap.add_argument("--codec", default="tmp/rc_fr_singe_scsi_span.dlx")
ap.add_argument("--fsweep", type=int, default=12,
help="highest cadence F in the pick table")
ap.add_argument("--gate", action="store_true",
help="check.sh mode: fixed rates, and assert the structural "
"results rather than print the essay")
a = ap.parse_args()
if a.gate and not a.kbps:
a.kbps = [451.4, 488.0, 600.0]
if not a.kbps:
ap.error("--kbps is required and has no default (FINDINGS 50)")
if not os.path.exists(a.table):
print(f"no scene table at {a.table} -- run:\n"
f" python3 tools/import/scenegraph.py")
return 2
doc = json.load(open(a.table))
if doc.get("format") != "DLXSCENE1":
print(f"{a.table}: not a DLXSCENE1 table")
return 2
sg = load("tools/analysis/25_scene_graph.py", "scene_graph")
ss = load("tools/analysis/20_seek_slack.py", "seek_slack")
from dlxp import DLXP
import ratectl as RC
nodes = sg.build_graph(doc["scenes"])
gaps, within, changes = branch_points(doc, nodes, sg)
med = gaps[len(gaps) // 2]
d = DLXP(a.packed)
P_REC = d.rec_bytes
P_VID = P_REC * FPS / 1024
F0, A0 = d.cad_f, d.cad_a
P_AUD = A0 * SECTOR / F0 * FPS / 1024
print(f"""
=== WHAT EACH BRANCH ACTUALLY HOLDS ====================================
51.3's climb is a statement about an ACCUMULATOR. The two branches of this
project do not have the same one, and one of them does not have one at all.
codec ({os.path.basename(a.codec)})
a {a.ring[0]:.0f} KB ring of variable records with an index in front of it;
lookahead is whole records and the ceiling is what 20_seek_slack.py
simulates.
packed ({os.path.basename(a.packed)})
a record is {P_REC:,} B of literal picture and the channel puts it
STRAIGHT INTO GVRAM (FINDINGS 61, 62, 64). There is no record
buffer, so the VIDEO lookahead is ZERO records and there is nothing
to climb. The only consumer on that branch with any lookahead at
all is the AUDIO one: {d.cad_a * SECTOR:,} B a lump, PG_ANBUF={3} slots and
PG_APRE prefilled at {2} for the run 68 measured (it is a mailbox,
not a constant: src/player/packed.s, 68.6).
That is {2 * F0 / FPS:.3f} s of sound held against {0.0:.3f} s of picture.
The CPU-painted packed variant (64.2's column B) is the one that holds two
record buffers and 99,328 B, and it is the only packed configuration the
word "climb" applies to. Both are priced below.
=== THE BRANCH POINTS, OUT OF THE ARCADE'S OWN GRAPH ====================
{len(gaps)} transitions into a seek (attract mode excluded, 56.3)
worst {gaps[0]:.3f} s p10 {pct(gaps,.10):.3f} median {med:.3f} p90 {pct(gaps,.90):.3f}
{len(changes)} of them END THE SCENE and are container changes;
{len(within)} land INSIDE a container, at frame indices 0..{max(i for _, i, _ in within)}""")
# ---------------------------------------------------------------- 1
dc, rec = ss.records(a.codec)
C_REC = float(rec.mean())
C_VID = C_REC * FPS / 1024
print(f"""
=== 1. THE CLIMB WITH THE SECOND CONSUMER (the codec branch) ============
Audio is {AU_BPS/1024:.3f} KB/s and this container pays no padding for it (65.4: it
already has an index and already has variable records). Against a video wire
of {C_VID:.1f} KB/s that is {100*(AU_BPS/1024)/C_VID:+.2f}% -- and the climb is not built out of the wire,
it is built out of `pipe - wire`, so that is not the number that matters.
{'ring':>5} {'pipe':>7} {'ceil':>5} {'climb SILENT':>13} {'SOUNDED':>9} {'x':>6}"""
f" {'under, silent':>13} {'under, sounded':>14}")
tab1 = []
for ring_kb in a.ring:
ring = int(ring_kb * 1024)
for kbps in a.kbps:
fill = (kbps - AU_BPS / 1024) * 1024 / FPS
lo, hi, ring_ref, rate_ref = ss.paced_sim(rec, ring, fill)
ceil = int(hi.max())
cs = climb_s(ceil, C_REC, kbps, C_VID)
ca = climb_s(ceil, C_REC, kbps, C_VID + AU_BPS / 1024)
us = sum(1 for g in gaps if g < cs)
ua = sum(1 for g in gaps if g < ca)
ratio = ca / cs if cs not in (0.0, float("inf")) else float("inf")
tab1.append((ring_kb, kbps, ceil, cs, ca, ratio, us, ua))
print(f" {ring_kb:5.0f} {kbps:7.1f} {ceil:5d} {cs:12.2f}s "
f"{ca:8.2f}s {ratio:5.2f}x {f'{us}/{len(gaps)}':>13} "
f"{f'{ua}/{len(gaps)}':>14}")
worst = max(tab1, key=lambda r: r[5])
print(f"""
THE SECOND CONSUMER IS {100*(AU_BPS/1024)/C_VID:.1f}% OF THE WIRE AND UP TO {worst[5]:.2f}x OF THE CLIMB.
At {worst[1]:.1f} KB/s in a {worst[0]:.0f} KB ring the climb goes {worst[3]:.2f} s -> {worst[4]:.2f} s and the
branch points that arrive under it go {worst[6]}/{len(gaps)} -> {worst[7]}/{len(gaps)}
({100*worst[6]/len(gaps):.0f}% -> {100*worst[7]/len(gaps):.0f}%). Nothing about audio got bigger; the
DIFFERENCE it is subtracted from got smaller, and the climb is made of the
difference. This is why 51.4's rate/ring distinction matters more with a
second consumer than without one, and why quoting audio as a share of the
wire (32, and every budget before it) understates it at every rate close to
the wire.
A CORRECTION TO 56.4, and it is small: that table charged audio at
ratectl.AUDIO_KBPS = {RC.AUDIO_KBPS} KB/s, which is {AU_BPS:.1f} B/s expressed in decimal
kB (21_iplrom_dmac.py says so). In binary KB the figure is {AU_BPS/1024:.4f}, i.e.
{100*(RC.AUDIO_KBPS-AU_BPS/1024)/(AU_BPS/1024):+.2f}%. Every column of 56.4 moves in the flattering direction by
less than one part in six hundred of the wire. It is recorded because a
placeholder that turns out to be right is still a placeholder.""")
# ---------------------------------------------------------------- 2
print(f"""
=== 2. THE PACKED BRANCH: THERE IS NO CLIMB, AND THAT IS THE FINDING ====
video {P_VID:.1f} + audio {P_AUD:.4f} (F={F0}, A={A0}, {A0*SECTOR:,} B a lump) = {P_VID+P_AUD:.1f} KB/s,
which is B1's acceptance figure and is where it comes from.
DMAC-direct (what src/player/packed.s runs, FINDINGS 64/68):
video lookahead 0 records. The climb does not exist, the ceiling does not
exist, and the {len(gaps)} gaps buy it NOTHING -- there is no accumulator for
play to fill. Its acceptance is a PER-FRAME deadline: {P_REC:,} B must land
inside every slot, and a rate that averages {P_VID+P_AUD:.1f} KB/s over a second is
not the same claim. 56.4's alarming column -- most branch points arrive
with less lookahead than the one before them -- does not apply to it,
because every frame arrives with less lookahead than the one before it.
CPU-painted (64.2 column B, 99,328 B, two record buffers -> 1 record of
lookahead):
{'pipe':>7} {'climb SILENT':>13} {'SOUNDED':>9} {'x':>6} {'under, silent':>13} {'under, sounded':>14}""")
tab2 = []
def secs(x, w):
return f"{x:{w}.2f}s" if x != float("inf") else f"{'never':>{w+1}}"
for kbps in a.pkbps:
cs = climb_s(1, P_REC, kbps, P_VID)
ca = climb_s(1, P_REC, kbps, P_VID + P_AUD)
us = sum(1 for g in gaps if g < cs)
ua = sum(1 for g in gaps if g < ca)
r = ca / cs if cs not in (0.0, float("inf")) else float("inf")
tab2.append((kbps, cs, ca, r, us, ua))
print(f" {kbps:7.1f} {secs(cs,12)} {secs(ca,8)} "
f"{(f'{r:5.2f}x' if r != float('inf') else ' inf ')} "
f"{f'{us}/{len(gaps)}':>13} {f'{ua}/{len(gaps)}':>14}")
print(f" -- and EVERY rate in section 1's table is below {P_VID:.1f} KB/s, so "
f"none of\n them serves this container at all.")
print(f"""
READ THE FIRST ROW. At {P_VID+P_AUD:.1f} KB/s -- the acceptance figure this project
quotes -- the SILENT container still climbs its one record in {tab2[0][1]:.2f} s and
the SOUNDED one NEVER DOES, because {P_VID+P_AUD:.1f} is where its surplus is exactly
zero. The acceptance figure is the rate at which the sounded container has
no lookahead at any amount of play, which is a different thing from the rate
at which it plays.
A packed record is 1.3x the codec's mean record and the packed wire is 1.3x
the codec's, so a rate that is generous to one is tight for the other and the
same audio debit costs the packed climb more. ONE RECORD of lookahead is
1/12 s of tolerance and it takes seconds of play to earn.
THE TWO BRANCHES DIFFER HERE ON A COLUMN THAT IS NOT CLOCKS, which is the
third time (61.9, 64.2, and this). The packed branch spent its ring to
delete a decoder; what it bought with the RAM is a player with no tolerance
for a slow record at ANY time, not merely after a branch.""")
# ---------------------------------------------------------------- 3
sil = silences(within, F0)
free = sum(1 for x in sil if x == 0.0)
P_PIPE = a.pkbps[1] if len(a.pkbps) > 1 else a.pkbps[0]
lump_ms = A0 * SECTOR / (P_PIPE * 1024) * 1000
behind = (F0 - 1) * P_REC + A0 * SECTOR
print(f"""
=== 3. THE COST NOBODY HAD COUNTED: entering a group off-boundary =======
A DLXP2 group is `lump k, then F records` (dlxp.py), so lump k is at a LOWER
address than every record of its group except the first. Reading forward from
record i, the next lump to arrive is k+1, and it carries frame (k+1)*F. The
frames from i to (k+1)*F-1 therefore have picture and no sound.
Measured on the {len(within)} within-container seek targets of the arcade's own
graph, at the shipped cadence F={F0}:
mean {sum(sil)/len(sil):7.1f} ms of silence entering the branch
median {pct(sil,.50):7.1f} p90 {pct(sil,.90):7.1f} worst {sil[-1]:7.1f}
free {free}/{len(within)} land on a group boundary and cost nothing
The other {len(changes)} branch points -- the scene changes -- are FREE, and by
construction: lump 0 sits at sector 1 and record 0 at {d.off_frm:,}, so a
container's own first bytes are header, lump, record and a scene change reads
them in one forward pass. **The container's start is the one branch point
the cadence costs nothing at, and it is the only one anybody had looked at.**
THE FIX IS A SECOND READ AND NOBODY HAS ONE. Lump k is {behind:,} B behind
record i at worst, so it cannot be picked up by reading early -- it is a
separate command at a separate LBA, of {A0*SECTOR:,} B, which at {P_PIPE:.1f} KB/s is
{lump_ms:.1f} ms against a mean {sum(sil)/len(sil):.0f} ms of silence -- {sum(sil)/len(sil)/lump_ms:.0f}x cheaper in TIME,
one more command per branch, and the command overhead is B1's and unmeasured.
src/player/packed.s starts PG_AK and PG_AKF at lump 0 and has no audio seek
path at all; the player that branches needs one.
=== 4. THE CADENCE PICK, WITH THE THIRD COLUMN IT DID NOT HAVE ==========
32_audio_wire.py chose F={F0} on two columns, padding and RAM. Here is the same
sweep with the branch column, measured on the game's own seek targets rather
than assumed uniform:
{'F':>3} {'A':>3} {'lump B':>8} {'pad%':>7} {'aud KB/s':>9} {'RAM x2':>8} {'mean sil':>9} {'p90':>8} {'worst':>8} {'free':>10} {'vs uniform':>11}""")
for F in range(1, a.fsweep + 1):
A, lump = cadence(F)
need = F * AU_FRAME
s = silences(within, F)
uni = (F - 1) / 2 / FPS * 1000.0
mean = sum(s) / len(s)
mark = " <- shipped" if F == F0 else ""
print(f" {F:3d} {A:3d} {lump:8,} {100*(lump-need)/need:6.2f}% "
f"{lump/F*FPS/1024:8.3f} {2*lump:8,} {mean:8.1f} "
f"{pct(s,.90):8.1f} {s[-1]:8.1f} "
f"{f'{sum(1 for x in s if x == 0)}/{len(s)}':>10} "
f"{(mean/uni if uni else 1.0):10.2f}x{mark}")
ratios = []
for F in range(2, a.fsweep + 1):
sF_ = silences(within, F)
ratios.append((sum(sF_) / len(sF_)) / ((F - 1) / 2 / FPS * 1000.0))
min_r, max_r = min(ratios), max(ratios)
A1, l1 = cadence(1)
AF, lF = cadence(F0)
s1, sF = silences(within, 1), silences(within, F0)
print(f"""
F=1 -- "one lump a record", the cadence 32 called THE WORST ONE -- has no
group to enter off-boundary, no second read, no audio seek path and 2,048 B
of held lump instead of {2*lF:,}. It costs {l1/1*FPS/1024 - lF/F0*FPS/1024:+.3f} KB/s of wire, which is
{100*(l1/1*FPS/1024 - lF/F0*FPS/1024)/(P_VID+P_AUD):+.2f}% of the packed acceptance figure, and it BUYS BACK {2*lF-2*l1:,} B
of RAM on the branch whose whole argument is that RAM is what it has spare.
THE PICK IS THEREFORE REOPENED, and it is a real trade rather than an error:
padding is what F={F0} minimises and padding is not the only thing F sets.
A player that gets its audio seek right is indifferent; a player that does
not pays a mean {sum(sF)/len(sF):.0f} ms of silence at {len(within)} of the game's {len(gaps)} branch
points. Nothing here decides it -- the deciding number is the SCSI command
overhead of the extra read, and that is B1's.
AND THE CONTENT IS NOT UNIFORM MOD F. A uniform assumption would put the
mean at (F-1)/2 frames; the arcade's seek targets land where they land, and
the ratio column above runs {min_r:.2f}x..{max_r:.2f}x over the sweep, so a design
that assumed uniform would be out by a quarter at F=3. At the shipped F={F0} it is
{(sum(sF)/len(sF))/((F0-1)/2/FPS*1000):.2f}x, which is a coincidence and is reported as one.
=== 5. WHAT THIS DOES NOT ESTABLISH ====================================
1. NO RATE HERE IS MEASURED. Every pipe column is a sensitivity (FINDINGS
50), and B1 -- sustained AND data-phase burst -- is still the user's.
2. THE FRAME INDEX OF A SEEK TARGET IS A DESIGN ASSUMPTION. One container
per SCENE (53, 55.1, 56.3). One container per SEQUENCE makes section 3
zero and section 4 moot; nothing else in the file changes.
3. NOTHING RAN ON THE MACHINE. This is arithmetic over a scene table, two
containers and a player's own constants. 68's player has never seeked.
4. THE MECHANICAL SEEK IS STILL UNMODELLED (51.7.5) and is charged on top of
every millisecond here.
5. THE SILENCE IS A CONTAINER PROPERTY, NOT A CHIP ONE. What the MSM6258
does when it is not fed -- hold the last sample, or click -- is a board
question and belongs with session 34's fifth hardware item.""")
if a.gate:
# Structural assertions. Not the milliseconds -- those move with the
# scene table -- but the ORDER and the SIGNS, which are the finding.
ok = True
def check(cond, msg):
nonlocal ok
print(f" {'OK ' if cond else 'FAIL'} {msg}")
ok = ok and bool(cond)
print("\n=== GATE ===============================================")
check(len(gaps) == 612, f"612 transitions into a seek, got {len(gaps)}")
check(len(within) + len(changes) == len(gaps),
f"{len(within)} within + {len(changes)} scene changes = {len(gaps)}")
check(all(r[5] >= 1.0 for r in tab1),
"audio never SHORTENS the codec climb")
check(max(r[5] for r in tab1) > 1.5,
f"and at some rate it more than 1.5x's it "
f"({max(r[5] for r in tab1):.2f}x)")
check(all(r[7] >= r[6] for r in tab1),
"and never lowers the count of branch points under the climb")
check(free < len(within) // 2,
f"most within-container branches enter a group off-boundary "
f"({len(within)-free}/{len(within)})")
check(silences(within, 1) == [0.0] * len(within),
"F=1 has no off-boundary case at all")
check(sum(sil) / len(sil) > 10 * lump_ms,
f"the silence F={F0} costs is >10x the lump read that removes it "
f"({sum(sil)/len(sil):.0f} ms vs {lump_ms:.1f} ms)")
print(" " + ("BRANCH-AUDIO GATE GREEN" if ok else "BRANCH-AUDIO GATE RED"))
return 0 if ok else 1
return 0
if __name__ == "__main__":
sys.exit(main())
tools/analysis/37_audio_seek.py
+238
View File
@@ -0,0 +1,238 @@
#!/usr/bin/env python3
"""What a BRANCH costs the chip, and what an encoder could do about it.
python3 tools/analysis/37_audio_seek.py [container.dlxp] [--raw au.raw]
python3 tools/analysis/37_audio_seek.py --gate # the check.sh stage
FINDINGS 71. Session 39 put an audio seek path in src/player/packed.s and ran
it: 132,162 B across a real branch, every byte accounted for in MAME's own
capture. That settled the BYTES. This settles what is left, which is a
property of the codec rather than of the player and which no counter in the
player can reach.
THE MSM6258'S ACCUMULATOR HAS NO LEAKAGE TERM. It is a pure integrator of
deltas, clamped, and nothing pulls it back toward zero. So a branch that hands
the chip bytes chosen for a state it is not in does not produce a transient with
a time constant -- it produces a DC OFFSET THAT NEVER DECAYS. The machine run
measures both designs at one branch point; this measures the CENSUS, over every
frame boundary of the container, and prices the only fix that is worth anything,
which is in the encoder and not in the player.
* PLAY THROUGH the branch: the chip keeps whatever accumulator and step index
the previous scene's audio left it in. Unbounded, and its decay is the
signal's own clamping rather than the recursion forgetting.
* STOP and re-PLAY: the accumulator goes to the container's `init` and the
step index to 0 -- a state this script knows exactly, so the error is
EXACTLY `init - acc(target)`, constant, forever.
* ...and the third option is the ENCODER'S: encode the stream with the
predictor RESET at every point a branch can land on. Then a re-PLAYing
player is not close, it is exact. What that costs is a codec question and
is measured below.
NAME THE LAYER. Everything here is host arithmetic over one container and its
source PCM. The chip's four axes are the ones FINDINGS 66 measured on the
machine and 67.3 put in the header; the branch behaviour is the one session 39
ran. No emulator is involved and no rate is claimed.
"""
import argparse, math, os, statistics, sys
HERE = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, os.path.join(HERE, "..", "encoder"))
import adpcm
from dlxp import DLXP
def acc_trajectory(nibbles, dec):
"""The decoder's accumulator after every sample. This IS the encoder's
assumed state, because adpcm.encode runs its decoder inside its own search
loop -- the encoder cannot hold a state the decoder will not reach."""
lo, hi = adpcm.clamp_bounds(dec["bits"])
sig, idx = dec["init"], 0
out = []
for n in nibbles:
sig += adpcm.delta(n, adpcm.STEP[idx], dec["variant"])
sig = lo if sig < lo else (hi if sig > hi else sig)
idx += adpcm.INDEX_ADJUST[n & 7]
idx = 0 if idx < 0 else (48 if idx > 48 else idx)
out.append((sig, idx))
return out
def encode_reset(src12, dec, period):
"""adpcm.encode with the predictor forced back to (init, 0) every `period`
samples. period=None is the ordinary encode.
THIS CHANGES THE BYTES, so it is a container property and not a flag a
player can set. It is written here rather than in tools/encoder/adpcm.py
because nothing has decided to ship it: 71.5 is the trade and the deciding
number is a hardware one."""
if period is None:
return adpcm.encode(src12, variant=dec["variant"], init=dec["init"],
bits=dec["bits"])
out = bytearray()
for i in range(0, len(src12), period):
out += adpcm.encode(src12[i:i + period], variant=dec["variant"],
init=dec["init"], bits=dec["bits"])
return bytes(out)
def snr(ref, got):
n = min(len(ref), len(got))
sig = sum(x * x for x in ref[:n])
err = sum((ref[i] - got[i]) ** 2 for i in range(n))
if err == 0:
return float("inf")
return 10 * math.log10(sig / err) if sig else float("-inf")
def decode_reset(nib, dec, period):
if period is None:
return list(adpcm.decode_state(nib, variant=dec["variant"],
init=dec["init"], bits=dec["bits"])[0])
out = []
for i in range(0, len(nib), period):
out += list(adpcm.decode_state(nib[i:i + period], variant=dec["variant"],
init=dec["init"], bits=dec["bits"])[0])
return out
def main():
ap = argparse.ArgumentParser()
ap.add_argument("container", nargs="?", default="tmp/packed_singe.dlxp")
ap.add_argument("--raw", default="tmp/au_singe.raw")
ap.add_argument("--gate", action="store_true")
a = ap.parse_args()
d = DLXP(a.container)
if not d.has_audio:
sys.exit("this container is silent -- there is no branch to price")
dec = d.decoder()
lo, hi = adpcm.clamp_bounds(dec["bits"])
data = d.audio()
nib = adpcm.unpack(data, len(data) * 2, order=dec["order"])
traj = acc_trajectory(nib, dec)
fails = []
def ck(ok, msg):
print(("OK " if ok else "FAIL ") + msg)
if not ok:
fails.append(msg)
# ---- 1. THE CENSUS. What a re-PLAY costs at every frame boundary the
# container has, which is every point a branch in this design can land on:
# 56.3's targets are frame indices and this player seeks to a frame.
den = 2 * d.fps
# THE INDEX IS A NIBBLE INDEX AND THE POSITION IS A BYTE ONE, which is the
# one conversion in this file and it is worth the line: getting it wrong
# reads the trajectory at HALF the target and produces a census that is
# entirely plausible -- a distribution of the right shape over the wrong
# instants. The cross-check against the machine's own branch point below is
# what caught it.
pos = lambda f: 2 * (f * d.aud_hz // den) - 1
frames = [f for f in range(1, d.nframes) if pos(f) < len(traj)]
dcs = [abs(dec["init"] - traj[pos(f)][0]) for f in frames]
idxs = [traj[pos(f)][1] for f in frames]
dcs_s = sorted(dcs)
p = lambda q: dcs_s[min(len(dcs_s) - 1, int(q * len(dcs_s)))]
print(f"--- 1. A RE-PLAYED BRANCH COSTS `init - acc(target)`, EXACTLY AND "
f"FOREVER. {len(frames)} frame boundaries of {a.container}:")
print(f" |DC| against the {hi}-unit clamp: mean {statistics.mean(dcs):.1f} "
f"({statistics.mean(dcs)*100/hi:.1f}%), median {statistics.median(dcs):.0f}, "
f"p90 {p(0.90)}, worst {max(dcs)} ({max(dcs)*100/hi:.1f}%)")
print(f" ...and the step index the encoder assumed at those points runs "
f"{min(idxs)}..{max(idxs)} of 48, median {statistics.median(idxs):.0f} "
f"-- a re-PLAY sets it to 0, so a branch into a LOUD passage gets the "
f"offset AND a step index that has to climb back")
# The machine run's own branch, so the two layers are checked against each
# other rather than merely agreeing in prose.
F37 = 37
dc37 = dec["init"] - traj[pos(F37)][0]
print(f" frame {F37}, the branch tools/bench/packed_run.sh runs on the "
f"machine: DC {dc37} -- and MAME's capture measured the chip at "
f"exactly that, constant over 62,500 samples (FINDINGS 71.3)")
ck(abs(dc37) == 65,
f"the host's arithmetic for the machine's own branch point is {abs(dc37)} "
f"and the capture said 65 -- one number, two layers")
# ---- 2. THE DECAY THAT ISN'T. A re-PLAY's error is constant BY
# CONSTRUCTION -- same step index, same nibbles, one offset -- and playing
# through is not, because the step indices differ too. The point of
# measuring it here is that the constancy is a PROPERTY OF THE PREDICTOR
# and not of the ten seconds this container happens to hold.
print(f"--- 2. AND IT DOES NOT DECAY. The accumulator is an integrator with "
f"no leak: a re-PLAY changes the STARTING value and nothing else, so "
f"the same nibbles produce the same deltas and the offset is carried "
f"to the end of the stream. The machine agrees -- AC 0.00 over four "
f"seconds (FINDINGS 71.3). Playing THROUGH the branch is worse and is "
f"not constant, because the step index differs as well: -355 falling "
f"to -108 over four seconds, which is clamping and not forgetting.")
# ---- 3. THE ENCODER'S FIX, PRICED. Reset the predictor where a branch can
# land and a re-PLAYing player is EXACT rather than close.
if not os.path.exists(a.raw):
print(f" (no {a.raw}: the encoder trade below needs the source PCM)")
return 1 if fails else 0
import struct
pcm = struct.unpack("<%dh" % (os.path.getsize(a.raw) // 2),
open(a.raw, "rb").read())
src12 = [max(-2048, min(2047, x >> 4)) for x in pcm][:len(nib)]
per_frame = d.aud_hz // den * 2 # samples in one frame slot
print(f"--- 3. THE ONLY FIX THAT MAKES A BRANCH FREE IS THE ENCODER'S, and "
f"here is its bill. Reset the predictor every N frames when encoding; "
f"a player that re-PLAYs at a branch landing on one of those points is "
f"then EXACT, not close:")
print(f" {"reset every":>24} {'SNR dB':>8} {'vs shipped':>10} "
f"{'branch points made free':>24}")
base = None
rows = []
for label, period in [("never (shipped)", None),
(f"{d.cad_f} frames (the cadence)", d.cad_f * per_frame),
("1 frame", per_frame)]:
nb = encode_reset(src12, dec, period)
got = decode_reset(nb, dec, period)
v = snr(src12, got)
if base is None:
base = v
free = (0 if period is None
else (len(frames) // d.cad_f if period != per_frame
else len(frames)))
rows.append((label, v, v - base, free))
print(f" {label:>24} {v:8.2f} {v-base:+10.2f} "
f"{free:>15} of {len(frames)}")
# THE ASSERTION IS THE ORDER AND THE SIGN, not the decibel: the source PCM
# is a property of the disc and the encoder is greedy, so the exact figures
# move with the window. What must not move is that resetting COSTS SNR and
# that resetting more often costs more -- if it ever came out free, the
# predictor would not be doing anything and the codec would be pointless.
ck(rows[1][1] <= rows[0][1] + 1e-9 and rows[2][1] <= rows[1][1] + 1e-9,
f"resetting the predictor costs SNR, and resetting it more often costs "
f"more: {rows[0][1]:.2f} -> {rows[1][1]:.2f} -> {rows[2][1]:.2f} dB")
ck(rows[2][1] > rows[0][1] - 3.0,
f"...and a reset EVERY FRAME is {rows[0][1]-rows[2][1]:.2f} dB, which is "
f"the price of making all {len(frames)} of this container's branch points "
f"exact. The step table's floor is a constant 16 and the recursion "
f"re-converges in a few samples, which is why twelve resets a second is "
f"not twelve times anything")
print(f"--- 4. WHAT THIS DOES NOT SETTLE.")
print(f" * Nothing here is a rate and nothing here ran on silicon. The "
f"branch behaviour is MAME's okim6258 -- PLAY sets the accumulator to "
f"-2, the step index to 0 and the nibble select to 0 -- which is the "
f"model FINDINGS 66 fitted to the machine and NOT a measurement of an "
f"MSM6258V. It joins session 34's fifth hardware item.")
print(f" * The census is ONE container, ten seconds, one passage at "
f"-13.4 dBFS (FINDINGS 69). The offset a re-PLAY costs is the signal's "
f"own value at the cut, so a louder passage costs more, up to the "
f"clamp -- and the disc peaks at 946 of 2048 (69.2).")
print(f" * The reset-every-frame encode is NOT in tools/encoder. It is "
f"a container change (a DLXP3), it costs bytes nothing and SNR "
f"something, and what decides it is whether a branch is allowed to "
f"land anywhere or only on frames the encoder was told about.")
print("AUDIO SEEK GATE " + ("GREEN" if not fails else f"RED: {len(fails)}"))
return 1 if fails else 0
if __name__ == "__main__":
sys.exit(main())
tools/analysis/buscost.py
+215
View File
@@ -0,0 +1,215 @@
"""Bus-cycle cost of src/player/decode.s and of tools/bench/blit.s's v6 spans.
A 68000 bus cycle is 4 clocks (S0-S7) with no wait states, and the 68000
prefetches every instruction word over the same bus. So a block's bus cost is
`instruction words + data accesses`, a long access counting twice on the 16-bit
bus and `movem.l` of N registers counting 2N.
The per-path word counts are read off tools/bench/decode.lst and
tools/bench/blit.s. tools/analysis/15_bus_occupancy.py checks the DATA half of
this table against tools/bench/c68k/c68k_bench, which counts every bus callback
the C68K core makes: they agree to 0.04%. The prefetch half cannot be measured
from either emulator -- MAME does not expose a fetch count and C68K reads
opcodes through a host pointer with no callback -- so it rests on that check.
"""
BUS_CLK = 4
# --- decode.s, per block ---------------------------------------------------
# dispatch move.b (a1),d0 / lsr.b / and.w #3 / beq .sk 6w, 1 read
# + subq / beq .v1 -> 8w
# + subq / bne .rw -> 10w
# V4 body $10090..$100E2 = 82 B = 41w; 4 x (1 byte read
# + movem.l 2 = 4 reads + 2 move.l = 4 writes) = 36
# V1 body $100E2..$10106 = 36 B = 18w; 1 byte read
# + movem.l 8 = 16 reads + 4 x movem.l 2 = 16 wr = 33
# RAW body $10106..$10164 = 94 B = 47w; 8 x (2 byte reads
# + 1 move.l = 2 writes) = 32
BODY = {0: (0, 0), 1: (18, 33), 2: (41, 36), 3: (47, 32)}
DISPATCH = {0: 6, 1: 8, 2: 10, 3: 10}
SK_TAIL = 1 # addq.l #8,a4
GROUP_HEAD = 3 # tst.b (a1) + beq allskip
GROUP_TAIL = 4 # addq.l #1,a1 / cmpa.l a5,a4 / bne byteloop
ALLSKIP = 9 # the whole four-block fast path, tst.b included
ROW_HEAD, ROW_TAIL = 3, 7
# --- blit.s v6 spans -------------------------------------------------------
# One chain unit moves 12 registers = 48 B = 24 pixels:
# movem.l (a0)+,12 = 2w instr + 24 word reads = 26
# movem.l 12,(a2) = 2w instr + 24 word writes = 26
# lea 48(a2),a2 = 2w instr = 2
# Per span: move.l (a0)+,a2 (1w + 2 reads) + move.w (a0)+,d0 (1w + 1 read)
# + jmp v6ch(pc,d0.w) (2w) + dbra (2w) = 9
V6_UNIT_PX = 24
V6_UNIT_BUS = 54
V6_SPAN_BUS = 9
V6_SPAN_CYC = 43.7 # MEASURED, FINDINGS 30
V6_PX_CYC = 9.152 # MEASURED, FINDINGS 30
# --- a DMAC array-chaining span -------------------------------------------
# SOURCED, MC68450 Direct Memory Access Controller, Motorola, Jul 1989
# (bitsavers). These replace session-10's first pass, which guessed 2 bus
# cycles a pixel from bus arithmetic and was 12% optimistic.
#
# Fig 4-25 sheet 4, DUAL ADDRESS / OPERAND SIZE IS WORD / DEVICE SIZE IS
# 16-BITS, D->M or M->D: {WORD READ, WORD WRITE} = 9 CLOCKS.
# Confirmed by the long-operand row: two of each = 18 clocks.
# Fig 4-25 note 2: reads are 4 clocks and WRITES ARE 5. That extra clock on
# every write is the whole story -- it is why the DMAC does not beat a 68000
# movem chain, which writes in 4.
DMA_PX_CLK = 9
# Fig 4-25 sheet 1, SEQUENTIAL ARRAY CHAINING: 36 CLOCKS per entry (three
# word reads to fetch the 6-byte entry, plus reload).
DMA_CHAIN_CLK = 36
# Sect 4.5.2.1 front-end overhead 5 clocks best case, 8 worst; 4.5.2.2
# back-end 2 clocks best. Once per period of bus ownership, not per span.
DMA_FRONT_CLK, DMA_BACK_CLK = 5, 2
DMA_FRONT_CLK_WORST = 8
# Fig 4-25 note 2 again, split out because the ADPCM channel needs the halves
# apart: a DMAC READ is 4 clocks and a WRITE is 5, on either bus width. A
# dual-address BYTE transfer is therefore one 4 and one 5.
DMA_READ_CLK, DMA_WRITE_CLK = 4, 5
DMA_DUAL_BYTE_CLK = DMA_READ_CLK + DMA_WRITE_CLK
# Fig 4-25 sheet 3, SINGLE ADDRESS: W/B READ 4 clocks, W/B WRITE 5 clocks.
# A device->memory disk transfer is one memory WRITE = 5 clocks if the DMAC
# holds the bus, or 5 + front + back = 12 if it arbitrates per word.
# FINDINGS 5's long-standing 8 clk/word ESTIMATE sits inside that range.
DMA_DISK_CLK_WORD_HELD, DMA_DISK_CLK_WORD_ARB = 5, 12
# --- the ADPCM stream, as the IPL ROM actually programs it -----------------
# READ OUT OF THE ROM, not recalled: tools/analysis/21_iplrom_dmac.py decodes
# the HD63450 registers Sharp's own IPL 1.0 writes, and gates on the bytes still
# being there. Channel 3, DCR = $80, OCR = $32 for playback:
#
# DTYP = 00 explicitly addressed -> DUAL ADDRESS (memory read, device write)
# DPS = 0 8-bit port -> one byte per operand
# XRM = 10 cycle steal WITHOUT hold, and REQG = 10 external request
# -> the DMAC arbitrates ONCE PER BYTE. No burst to amortise over.
#
# So an audio byte costs the dual-address transfer PLUS a full arbitration,
# every time -- unlike a disk record, which can at least be argued to hold the
# bus for a run of bytes. This is the number the audio side of the I/O debit
# should be denominated in; DISK_CLK_BYTE is not it.
ADPCM_SAMPLE_HZ = 15625.0 # MSM6258V, 8 MHz clock / 512 (the 15.6 kHz mode)
ADPCM_BYTES_PER_S = ADPCM_SAMPLE_HZ / 2 # 4-bit samples, two to a byte
ADPCM_CLK_BYTE_BEST = DMA_DUAL_BYTE_CLK + DMA_FRONT_CLK + DMA_BACK_CLK # 16
ADPCM_CLK_BYTE_WORST = DMA_DUAL_BYTE_CLK + DMA_FRONT_CLK_WORST + DMA_BACK_CLK # 19
# The 68000 cannot execute while another master owns the bus: no cache, and a
# two-word prefetch queue that empties immediately. So DMA time is ADDITIVE to
# CPU time, not overlapped -- which is what FINDINGS 35's flat debit assumed
# and session 10's first pass wrongly "refined".
DMA_OVERLAPS = False
def pad24(npix):
return -(-npix // V6_UNIT_PX) * V6_UNIT_PX
def block_bus(mode_map, spanned=None):
"""(instruction words, data accesses) for one frame's CPU block decode.
`spanned` is a boolean array the same shape as mode_map marking blocks a
span will paint instead; those blocks still cost their dispatch, because
the mode map is walked either way, but not their body."""
nby, nbx = mode_map.shape
pref = nby * (ROW_HEAD + ROW_TAIL)
data = 0
for by in range(nby):
row = mode_map[by]
sp = spanned[by] if spanned is not None else None
for gi in range(0, nbx, 4):
g = row[gi:gi + 4]
if (g == 0).all():
pref += ALLSKIP
data += 1
continue
pref += GROUP_HEAD + GROUP_TAIL - 1 # BLOCK 0 has no lsr.b
data += 1
for k, b in enumerate(g):
b = int(b)
if sp is not None and sp[gi + k]:
b = 0 # the span paints it
pw, pd = BODY[b]
pref += DISPATCH[b] + pw + SK_TAIL
data += 1 + pd
return pref, data
# --- v7: v6 with a finer tail (MEASURED, session 11, FINDINGS 40) ----------
# v6 pads every span up to 24 pixels because its unrolled chain is built from
# 12-register movem units, and FINDINGS 39.3 attributed 86% of the DMAC array
# chain's advantage over v6 to exactly that padding. v7 keeps the coarse chain
# and appends a second chain whose unit is one `move.l (a0)+,(a2)+` -- 2 pixels,
# so the quantum is 2 and a run of 4x4 blocks pads to NOTHING.
#
# Session 10 proposed a 2-REGISTER MOVEM tail (4 pixels, derived at 56 clocks)
# and that would have been the wrong instruction: movem.l (a0)+,d0-d1 plus
# movem.l d0-d1,(a2) plus the lea is 14 bus cycles for 4 pixels, where two plain
# move.l are 10. The plainest instruction on the machine wins the tail.
#
# The second entry point needs a second dispatch, and the fine displacement is
# carried MID-STREAM (after the coarse pixels, before the fine ones) rather than
# in the span record, so the decoder holds nothing extra across the copy and
# keeps all 12 payload registers. Costed as 2 more bytes per span.
#
# MEASURED by tools/bench/span.sh (blit.s v7, 13 span lengths, every config
# pixel-exact): cycles = 66.0/span + 9.143/coarse pixel + 9.978/fine pixel,
# fitting all 13 to within 0.2%.
V7_SPAN_CYC = 66.0 # MEASURED, FINDINGS 40
V7_CPX_CYC = 9.143 # MEASURED, FINDINGS 40 (24-pixel coarse unit)
V7_FPX_CYC = 9.978 # MEASURED, FINDINGS 40 (2-pixel fine unit)
V7_FINE_PX = 2
# Bus: per span v6's 9 plus a second {move.w (a0)+,d0 ; jmp} = 2 + 2.
# Per fine unit: move.l (a0)+,(a2)+ = 1 instruction word + 2 reads + 2 writes.
V7_SPAN_BUS = 13
V7_FINE_BUS = 5
V7_SPAN_HDR = 8 # {u32 address, u16 coarse disp} + u16 fine disp
def pad2(npix):
return -(-npix // V7_FINE_PX) * V7_FINE_PX
def v7_span(npix):
"""(pixels carried, CPU clocks) for a v7 span of npix pixels."""
k, r = divmod(pad2(npix), V6_UNIT_PX)
return (k * V6_UNIT_PX + r,
V7_SPAN_CYC + k * V6_UNIT_PX * V7_CPX_CYC + r * V7_FPX_CYC)
def v7_span_bus(npix):
"""Bus CYCLES a v7 span occupies -- instruction words plus data accesses."""
p, d = v7_span_split(npix)
return p + d
def v7_span_split(npix):
"""(instruction words, data accesses) for one v7 span, separately.
15_bus_occupancy.py needs the two apart, because the DATA half is what the
C68K harness can check and the PREFETCH half is what rides on that check.
per span move.l (a0)+,a2 1 word + 2 reads
move.w (a0)+,d0 1 word + 1 read (coarse displacement)
jmp (pc,d0.w) 2 words
move.w (a0)+,d0 1 word + 1 read (fine, from mid-stream)
jmp (pc,d0.w) 2 words
dbra 2 words -> 9 words, 4 accesses
per coarse 2 movem.l of 12 + lea = 6 words, 24 reads + 24 writes
per fine move.l (a0)+,(a2)+ = 1 word, 2 reads + 2 writes
"""
k, r = divmod(pad2(npix), V6_UNIT_PX)
f = r // V7_FINE_PX
return (9 + k * 6 + f * 1,
4 + k * 48 + f * 4)
# Per FRAME, decode.s's paint_spans entry and exit: the span count read, the
# guard branch, and the push/pop of a1 that buys back a twelfth payload
# register. Two long accesses a frame against 24 pixels a chain unit.
V7_FRAME_PREF, V7_FRAME_DATA = 7, 7
def v6_span_bus(npix):
k = pad24(npix) // V6_UNIT_PX
return V6_SPAN_BUS + k * V6_UNIT_BUS
tools/analysis/mc68450.py
+59
View File
@@ -0,0 +1,59 @@
"""MC68450 / HD63450 register field layouts, in ONE copy.
Read by tools/analysis/21_iplrom_dmac.py, which decodes what the X68000's IPL
ROM programs into the DMAC, and by tools/analysis/27_dmac_config.py, which
decodes what src/player/dma.i programs into it. The two exist to be COMPARED
-- the ROM's own disk channel costs 16..19 clocks a byte (FINDINGS 52.5) and
the player's job is to be cheaper -- and a comparison between two decodings
that used two copies of these tables would not be one. This tree has already
paid twice for a transform with two copies of itself (FINDINGS 49.7.5).
SOURCED: MC68450 Direct Memory Access Controller, Motorola, Jul 1989
(bitsavers) -- the same document FINDINGS 39 cites for the transfer timings in
tools/analysis/buscost.py.
"""
# --- MC68450 register map, by offset inside a channel's 0x40 block ----------
REG = {0x00: "CSR", 0x01: "CER", 0x04: "DCR", 0x05: "OCR", 0x06: "SCR",
0x07: "CCR", 0x0A: "MTC", 0x0C: "MAR", 0x14: "DAR", 0x1A: "BTC",
0x1C: "BAR", 0x25: "NIV", 0x27: "EIV", 0x29: "MFC", 0x2D: "CPR",
0x31: "DFC", 0x39: "BFC"}
XRM = {0: "burst",
1: "UNDEFINED",
2: "cycle steal WITHOUT hold (bus released between operands)",
3: "cycle steal with hold"}
DTYP = {0: "68000-compatible, EXPLICITLY addressed -> DUAL ADDRESS",
1: "6800-compatible, EXPLICITLY addressed -> DUAL ADDRESS",
2: "device with ACK, implicitly addressed -> SINGLE ADDRESS",
3: "device with ACK and RDY, implicit -> SINGLE ADDRESS"}
DPS = {0: "8-bit port", 1: "16-bit port"}
PCL = {0: "status input", 1: "status input with interrupt",
2: "start pulse", 3: "abort input"}
SIZE = {0: "byte", 1: "word", 2: "long word", 3: "byte, unpacked"}
CHAIN= {0: "none", 1: "UNDEFINED", 2: "array", 3: "linked array"}
REQG = {0: "auto-request at limited rate", 1: "auto-request at max rate",
2: "EXTERNAL request (one operand per device request)",
3: "auto-request first operand, external thereafter"}
def dcr(v):
return [f"XRM = {v>>6&3:02b} {XRM[v>>6&3]}",
f"DTYP = {v>>4&3:02b} {DTYP[v>>4&3]}",
f"DPS = {v>>3&1:b} {DPS[v>>3&1]}",
f"PCL = {v&3:02b} {PCL[v&3]}"]
def ocr(v):
return [f"DIR = {v>>7&1:b} " +
("device -> memory (read)" if v & 0x80 else "memory -> device (write)"),
f"SIZE = {v>>4&3:02b} {SIZE[v>>4&3]}",
f"CHAIN= {v>>2&3:02b} {CHAIN[v>>2&3]}",
f"REQG = {v&3:02b} {REQG[v&3]}"]
def scr(v):
m = {0: "no count", 1: "increment", 2: "decrement", 3: "UNDEFINED"}
return [f"MAC = {v>>2&3:02b} memory address {m[v>>2&3]}",
f"DAC = {v&3:02b} device address {m[v&3]}"]
tools/bench/adpcm.lua
+88
View File
@@ -0,0 +1,88 @@
-- Play one buffer of ADPCM nibbles on the emulated MSM6258V, from 68000 code
-- (ROADMAP P6a). The Lua here does what Lua is allowed to do in this tree:
-- push bytes in, start the CPU, read the mailbox out. It is NOT in the feed
-- path -- session 33's probe was, and a host that writes the data register at
-- host-frame rate is not feeding a chip that consumes at 15,625 Hz (65.5).
--
-- WHAT THE MEASUREMENT IS. MAME's -wavwrite capture, at a sample rate chosen
-- to EQUAL the chip's stream rate so nothing resamples it, is the chip's own
-- output. tools/bench/verify_adpcm_chip.py reads the four model axes out of it.
M = manager.machine
SP = M.devices[":maincpu"].spaces["program"]
local META = loadfile("adpcm_meta.lua")()
local AD_FLAG, AD_BUF, AD_LEN = 0x18600, 0x18604, 0x18608
local AD_MTC0, AD_CSRF, AD_CERF = 0x1860C, 0x18610, 0x18614
local AD_MTCF, AD_MARF, AD_SPIN, AD_STAT = 0x18618, 0x1861C, 0x18620, 0x18624
local code do local f=io.open("adpcmgate.bin","rb"); code=f:read("a"); f:close() end
local data do local f=io.open("adpcm_data.bin","rb"); data=f:read("a"); f:close() end
local function P(s) print("[ADP] "..s) end
local function T() local t=M.time; return t.seconds + t.attoseconds/1e18 end
local st, t0, tplay = "boot", nil, nil
SUB = emu.add_machine_frame_notifier(function()
local ok, err = pcall(function()
local t = T()
if st == "boot" then
if t < 3.0 then return end
for i = 1, #data do SP:write_u8(META.buf + i - 1, string.byte(data, i)) end
for i = 1, #code do SP:write_u8(0x10000 + i - 1, string.byte(code, i)) end
SP:write_u32(AD_FLAG, 0)
SP:write_u32(AD_BUF, META.buf)
SP:write_u32(AD_LEN, META.nbytes)
local cpu = M.devices[":maincpu"]
cpu.state["SR"].value = 0x2700 -- supervisor, ALL interrupts masked
cpu.state["SP"].value = 0x8000
cpu.state["PC"].value = 0x10000
P(string.format("pushed %d B of code and %d B of nibbles at 0x%X",
#code, META.nbytes, META.buf))
-- THE CAPTURE'S OWN CLOCK. The wav starts at t=0 of the run, so the host
-- has to know when PLAY happened to find the stream in it -- but it is
-- NOT used as the alignment: the verifier searches a small window around
-- it, because a host frame is 17.6 ms and a sample is 64 us.
st, t0 = "running", t
return
end
if st == "running" then
local fl = SP:read_u32(AD_FLAG)
if fl == 2 and not tplay then
tplay = t
P(string.format("PLAY at t=%.4f s, chip status $%02X (bit7 clear = playing), "
.."MTC then = %d of %d", t, SP:read_u32(AD_STAT),
SP:read_u32(AD_MTC0), META.nbytes))
end
if fl == 0xFF or fl == 0xEE then
P(string.format("channel finished: CSR=$%02X CER=$%02X MTC=%d MAR=$%06X "
.."spin=%d", SP:read_u32(AD_CSRF), SP:read_u32(AD_CERF),
SP:read_u32(AD_MTCF), SP:read_u32(AD_MARF),
SP:read_u32(AD_SPIN)))
local dt = t - (tplay or t)
P(string.format("%d bytes took %.4f s = %.1f B/s "
.."(15,625 nibbles/s wants 7,812.5)",
META.nbytes, dt, META.nbytes/dt))
if fl == 0xEE then P("ERROR: the gate flagged a channel error or a timeout") end
local f = io.open("adpcm_run.lua", "w")
f:write(string.format("return { tplay = %.9f, ok = %s, nbytes = %d,\n"
.." csr = %d, cer = %d, mtc = %d, spin = %d }\n",
tplay or -1, tostring(fl == 0xFF), META.nbytes,
SP:read_u32(AD_CSRF), SP:read_u32(AD_CERF),
SP:read_u32(AD_MTCF), SP:read_u32(AD_SPIN)))
f:close()
st = "drain"; t0 = t
return
end
if t - t0 > 60 then P("TIMEOUT flag="..string.format("%08X", fl)); M:exit() end
return
end
if st == "drain" then
-- let the capture run past the end of the stream, so a truncated wav is
-- never mistaken for a short stream
if t - t0 < 0.3 then return end
P("done")
M:exit()
end
end)
if not ok then print("[ADP] LUA ERROR: "..tostring(err)); M:exit() end
end)
tools/bench/adpcm_run.sh
Executable
+56
View File
@@ -0,0 +1,56 @@
#!/bin/bash
# ROADMAP P6a: ask the machine's own MSM6258V which decoder it is.
#
# tools/bench/adpcm_run.sh
#
# WHAT A GREEN RUN MEANS: 68000 code programmed HD63450 channel 3 exactly as the
# IPL ROM programs it -- dual address, 8-bit port, cycle steal, EXTERNAL request
# -- fed the chip a designed nibble stream at the chip's own pace, and exactly
# ONE of sixteen candidate decoder models reproduces MAME's capture of the
# result SAMPLE-EXACT, with every one of the four axes shown to matter.
#
# WHAT IT DOES NOT MEAN: anything about an MSM6258. This is MAME's device model
# measured end to end through the machine's real transport. It settles the RIG.
# The silicon stays on the hardware list.
set -e
cd "$(dirname "$0")/../.."
tools/vasm/vasmm68k_mot -Fbin -o tmp/adpcmgate.bin src/player/adpcmgate.s > /dev/null
python3 tools/bench/prep_adpcm.py
# -samplerate 15625 is not a preference: it is the chip's own stream rate
# (8 MHz / 512), and equal rates are what keep MAME's resampler from filtering
# the thing being measured. The first cut of this ran at the default 48000 and
# every reconstructed sample arrived as an interpolated pair.
( cd tmp && SDL_VIDEODRIVER=dummy stdbuf -oL timeout -k 5 300 \
mame x68000 -bios ipl10 -ramsize 2M -video soft -window \
-samplerate 15625 -wavwrite adpcm.wav -nothrottle -plugins \
-autoboot_script ../tools/bench/adpcm.lua \
-seconds_to_run 12 > adpcm_run.log 2>&1 )
grep -aq "^\[ADP\] done" tmp/adpcm_run.log || {
echo "FAIL: the ADPCM gate did not finish -- no completion marker."
tail -8 tmp/adpcm_run.log; exit 1; }
grep -a "^\[ADP\]" tmp/adpcm_run.log | sed 's/^\[ADP\] / /'
fail() { echo "FAIL: $1"; exit 1; }
if grep -aq "^\[ADP\] ERROR" tmp/adpcm_run.log; then
fail "the channel reported an error or the gate timed out -- see CSR/CER above."
fi
grep -aq "bit7 clear = playing" tmp/adpcm_run.log || \
fail "the chip never reported itself playing."
# THE FEED RATE IS A GATE, not a note. The chip is the pacemaker: one byte per
# #DRQ3 and #DRQ3 at half the sample rate. If the bytes went out at some other
# rate then the channel was NOT being paced by the device, and every sample
# below is of a stream that arrived faster or slower than it was consumed --
# which is precisely the failure session 33 hit from Lua.
RATE=$(sed -n 's/.*= \([0-9.]*\) B\/s .*/\1/p' tmp/adpcm_run.log | head -1)
python3 - "$RATE" <<'PY' || fail "the feed was not paced by the chip (see above)."
import sys
r = float(sys.argv[1])
want = 7812.5
print(f" feed rate {r:,.1f} B/s against the chip's own {want:,.1f} B/s "
f"({100*(r-want)/want:+.2f}%)")
sys.exit(0 if abs(r - want) / want < 0.02 else 1)
PY
python3 tools/bench/verify_adpcm_chip.py tmp/adpcm.wav tmp/adpcm_seq.json
tools/bench/blit.lua
+25 -2
View File
@@ -11,7 +11,8 @@
-- a LOWER BOUND, not a prediction. Interrupts are masked (SR=$2700) so the
-- IPL's timer and VBL handlers cannot steal cycles into the measurement.
--
-- Timing resolution is one video frame (1/55.46 s = 18.03 ms), because Lua
-- Timing resolution is one video frame (1/56.69 s = 17.64 ms -- MAME's, not
-- the hardware's 55.46; see crtc_mode.lua), because Lua
-- gets no cycle counter -- luaengine.cpp exposes machine.time and nothing
-- from device_execute_interface. Each variant therefore loops enough times
-- to run ~4 emulated seconds, putting the granularity error near 0.4%.
@@ -28,7 +29,7 @@ end
local MODE = load_mode()
local FLAG, VAR, ITER = 0x18000, 0x18004, 0x18008
local SRCW, SRCB = 0x60000, 0x80000
local SRCW, SRCB, SRCP = 0x60000, 0x80000, 0x90000
local GVRAM, GPAL = 0xC00000, 0xE82000
local CPUHZ = 10000000 -- x68k.cpp:1133, 40_MHz_XTAL/4
local FRAME12 = CPUHZ / 12 -- 833333 cycles at 12 fps
@@ -39,6 +40,16 @@ local PLAN = {
{var=2, iter= 50, name="V2 naive byte-source expansion (move.b/move.w per pixel)"},
{var=3, iter=200, name="V3 write-only floor (no source read at all)"},
{var=4, iter= 60, name="V4 same 96KB of writes, issued in 4x4 BLOCK order (decoder access pattern)"},
-- V8 is V1 with R20 bit 11's packing: 48KB read + 48KB write for the SAME
-- 49,152 pixels. It is the per-frame work of a decoder-free packed player
-- (FINDINGS 44.7 / 46.6 / 47.5), and 47.6.1 filed its `movem` shape as an
-- ASSUMPTION -- this is the measurement that assumption was standing in for.
{var=8, iter=200, name="V8 PACKED movem.l blit (48KB read + 48KB write, same 49,152 pixels as V1)"},
-- V9/V10 are the two ways a BLOCK decoder could survive the packed layout
-- (47.6.4, open since session 16): sixteen move.b at stride 2 per block, or
-- pair the blocks 128 columns apart in the encoder and get V4's movem back.
{var= 9, iter= 40, name="V9 PACKED block order, 16 move.b at stride 2 per 4x4 block"},
{var=10, iter=120, name="V10 PACKED block order, blocks PAIRED so a movem writes whole words"},
}
local code do
@@ -96,6 +107,18 @@ local function setup()
SP:write_u8 (SRCB + y*256 + x, px)
end
end
-- SRCP: the PACKED frame, interleaved the way tools/bench/show_frame256_packed.lua
-- lays it out -- word i of a row is (column i+128) << 8 | (column i), because
-- page 0 is the low byte at screen column i and page 1 the high byte at i+128.
-- Only V8 reads it, and only its SIZE (128 words a row) affects the timing;
-- the interleave is written correctly so the buffer is the real artefact and
-- not a same-sized stand-in.
for y = 0, H-1 do
local row = PIX0 + y*W
for i = 0, (W//2)-1 do
SP:write_u16(SRCP + y*(W//2)*2 + i*2, (B(row+i+W//2) << 8) | B(row+i))
end
end
for i = 1, #code do SP:write_u8(0x10000+i-1, string.byte(code,i)) end
P(string.format("loaded blit.bin=%d bytes, source frame %dx%d at yoff=%d", #code, W, H, YOFF))
end
tools/bench/blit.s
+366
View File
@@ -31,6 +31,64 @@
; the block needs only one base pointer. V4 deliberately scrambles the
; picture (it reads a row-linear source in block order); it is a timing
; probe, which is why the correctness snapshot is taken after V1.
; V5 ROW-LINEAR LITERAL SPANS, the mode priced in FINDINGS 29 and never
; measured. Walks a stream of per-row span records
; row: u16 nspans, then nspans * { u16 x, u16 npix, npix*u16 pixels }
; for 192 rows, copying each span's word-expanded pixels straight from
; the stream buffer into GVRAM. Unlike V1-V4 the work per call is set by
; the STREAM, not by the code, so one variant measures every span length:
; tools/bench/prep_spans.py generates a stream per span length and
; tools/bench/span.lua times them and fits cycles = A*spans + B*pixels.
; The point of the measurement is A -- the per-span overhead FINDINGS 29
; guessed at 50 cycles -- and how much B degrades from V1's 9.08 when a
; span is too short to burst. Every config covers the whole frame, so
; V5 draws the SAME picture V1 does and can be verified, not just timed.
;
; Bursts are 8 registers (d0-d3/a3-a6 = 32 bytes = 16 pixels), not V1's
; 12: a0/a1/a2 and d4-d7 are all live across a span (stream, row base,
; destination, and three counters). The remainder is copied move.l at a
; time with a leading move.w when it is odd, so a 4-pixel span never
; reaches a movem at all -- which is exactly the case FINDINGS 29's
; full-row-width extrapolation flatters.
;
; V6 the SAME spans with the arithmetic moved into the encoder. V5 measures
; a decoder that is handed (x, npix) and has to work out how to copy it;
; most of its per-span cost is that working-out, and an encoder can do it
; once at build time instead of 12 times a second. V6's record is
; { u32 absolute GVRAM address, u16 jump displacement } -- no row
; structure, no counters, no remainder logic -- and the displacement
; jumps into an unrolled chain of 24-pixel copy units, so a span of any
; supported length is straight-line code with no loop at all.
; GVRAM sits at a fixed $C00000 on every X68000, so absolute destinations
; are a legitimate thing for an encoder to bake in.
;
; Two consequences of the format. Span lengths are multiples of 24
; pixels, and a span may overrun the 256 visible pixels of its row by up
; to 23 -- harmless, because the line stride is 1024 bytes and only the
; first 512 are displayed, so the overrun lands in the invisible half.
; And with row and remainder handling gone, 12 registers are free again
; (d0-d6/a1/a3-a6), which is why the unit is 24 pixels and not V5's 16.
;
; V7 v6 with a SECOND, finer chain for the tail (FINDINGS 39.4). v6 pays for
; its 24-pixel quantum in padding: an average span wastes ~11 pixels, and
; FINDINGS 39.3 attributes 86% of the DMAC array-chain's advantage over v6
; to exactly that. V7 keeps the 24-pixel coarse chain and appends a chain
; of 2-pixel units, so a span is 24*c + 2*f pixels and the padding is at
; most one pixel -- ZERO for the real case, where a span is a run of 4x4
; blocks and its length is a multiple of 4.
;
; The fine unit is `move.l (a0)+,(a2)+` (20 cycles, 2 pixels), NOT a
; 2-register movem: movem.l (a0)+,d0-d1 plus movem.l d0-d1,(a2) plus the
; lea is 52+8 cycles for 4 pixels, so the obvious "smaller movem" tail is
; 50% dearer per pixel than the plainest instruction on the machine.
;
; The second entry point costs a second dispatch, and the trick that pays
; for it is that the fine displacement is NOT in the span record: it sits
; in the STREAM, after the coarse pixels and before the fine ones. The
; coarse chain falls out into `move.w (a0)+,d0 / jmp`, by which point d0
; is dead payload and a0 is pointing exactly at it. So v7 holds nothing
; extra across the copy and keeps all 12 payload registers -- a record is
; still {u32 address, u16 displacement}, with one more u16 mid-span.
;
; 12 registers per movem burst (d0-d7/a2-a5 = 48 bytes) is the maximum
; available: a0=src, a1=dst, a6=end sentinel. The row counter lives in the
@@ -43,10 +101,21 @@
FLAG = $18000 ; 0 idle / 1 running / $FF done
VAR = $18004 ; variant selector, written by Lua
ITER = $18008 ; iteration count, written by Lua
SPTR = $1800C ; V5 span stream pointer, written by Lua
SRCW = $60000 ; word-expanded frame 192*512 = 96KB
SRCB = $80000 ; byte-per-pixel frame 192*256 = 48KB
SRCP = $90000 ; PACKED frame 192*256 = 48KB (V8): two picture
; bytes per word, already interleaved by the
; encoder, so the blit is a straight copy
DST0 = $C08000 ; GVRAM + 32*1024 (first picture row)
DSTE = $C38000 ; GVRAM + 224*1024 (one past last)
ROWS = 192 ; picture rows a V5 stream describes
V6UNIT = 12 ; bytes of code per V6 chain unit
V6MAX = 11 ; chain units = 11*24 = 264 pixels >= one row
V7CU = 12 ; bytes of code per V7 COARSE unit (24 px)
V7CN = 11 ; coarse units: 11*24 = 264 px >= one row
V7FU = 2 ; bytes of code per V7 FINE unit (2 px)
V7FN = 11 ; fine units: 11*2 = 22 px > one coarse unit
org $10000
start:
@@ -58,6 +127,18 @@ start:
beq v2
cmp.l #4,d0
beq v4
cmp.l #5,d0
beq v5
cmp.l #6,d0
beq v6
cmp.l #7,d0
beq v7
cmp.l #8,d0
beq v8
cmp.l #9,d0
beq v9
cmp.l #10,d0
beq v10
bra v3
; ---------------------------------------------------------------- V1
@@ -152,5 +233,290 @@ v4blk: movem.l (a0)+,d0-d7 ; 32 bytes = one 4x4 block, expanded
bne v4
bra done
; ---------------------------------------------------------------- V5
; a0 stream, a1 row base, a2 span destination, d7 rows, d6 spans, d5 pixels,
; d4 burst/tail counter. Everything else (d0-d3/a3-a6) is burst payload.
v5: move.l SPTR.l,a0
lea DST0,a1
move.w #ROWS-1,d7
v5row: move.w (a0)+,d6 ; spans in this row
subq.w #1,d6
bmi.s v5eor ; a row may legitimately have none
v5span: move.w (a0)+,d0 ; x, in pixels
add.w d0,d0 ; one pixel = one word
lea 0(a1,d0.w),a2
move.w (a0)+,d5 ; pixels in this span
move.w d5,d4
lsr.w #4,d4 ; 16-pixel bursts
beq.s v5tail
subq.w #1,d4
v5burst: movem.l (a0)+,d0-d3/a3-a6 ; 32 bytes straight out of the stream
movem.l d0-d3/a3-a6,(a2)
lea 32(a2),a2
dbra d4,v5burst
v5tail: moveq #15,d4
and.w d5,d4 ; 0..15 pixels left
beq.s v5eos
lsr.w #1,d4 ; C = odd pixel count
bcc.s v5t2
move.w (a0)+,(a2)+
v5t2: subq.w #1,d4
bmi.s v5eos
v5tl: move.l (a0)+,(a2)+
dbra d4,v5tl
v5eos: dbra d6,v5span
v5eor: lea 1024(a1),a1
dbra d7,v5row
subq.l #1,ITER.l
bne v5
bra done
; ---------------------------------------------------------------- V6
; a0 stream, a2 destination, d7 spans remaining; everything else is payload.
v6: move.l SPTR.l,a0
move.w (a0)+,d7 ; total spans in the frame
subq.w #1,d7
v6span: move.l (a0)+,a2 ; absolute GVRAM destination
move.w (a0)+,d0 ; (V6MAX - units) * V6UNIT, from the encoder
jmp v6ch(pc,d0.w)
v6ch:
movem.l (a0)+,d0-d6/a1/a3-a6
movem.l d0-d6/a1/a3-a6,(a2)
lea 48(a2),a2
movem.l (a0)+,d0-d6/a1/a3-a6
movem.l d0-d6/a1/a3-a6,(a2)
lea 48(a2),a2
movem.l (a0)+,d0-d6/a1/a3-a6
movem.l d0-d6/a1/a3-a6,(a2)
lea 48(a2),a2
movem.l (a0)+,d0-d6/a1/a3-a6
movem.l d0-d6/a1/a3-a6,(a2)
lea 48(a2),a2
movem.l (a0)+,d0-d6/a1/a3-a6
movem.l d0-d6/a1/a3-a6,(a2)
lea 48(a2),a2
movem.l (a0)+,d0-d6/a1/a3-a6
movem.l d0-d6/a1/a3-a6,(a2)
lea 48(a2),a2
movem.l (a0)+,d0-d6/a1/a3-a6
movem.l d0-d6/a1/a3-a6,(a2)
lea 48(a2),a2
movem.l (a0)+,d0-d6/a1/a3-a6
movem.l d0-d6/a1/a3-a6,(a2)
lea 48(a2),a2
movem.l (a0)+,d0-d6/a1/a3-a6
movem.l d0-d6/a1/a3-a6,(a2)
lea 48(a2),a2
movem.l (a0)+,d0-d6/a1/a3-a6
movem.l d0-d6/a1/a3-a6,(a2)
lea 48(a2),a2
movem.l (a0)+,d0-d6/a1/a3-a6
movem.l d0-d6/a1/a3-a6,(a2)
lea 48(a2),a2
dbra d7,v6span
subq.l #1,ITER.l
bne v6
bra done
; ---------------------------------------------------------------- V7
; a0 stream, a2 destination, d7 spans remaining; everything else is payload.
; Stream per span: u32 dest, u16 coarse disp, c*48 B pixels,
; u16 fine disp, f*4 B pixels.
v7: move.l SPTR.l,a0
move.w (a0)+,d7 ; total spans in the frame
subq.w #1,d7
v7span: move.l (a0)+,a2 ; absolute GVRAM destination
move.w (a0)+,d0 ; (V7CN - coarse) * V7CU
jmp v7ch(pc,d0.w)
v7ch:
movem.l (a0)+,d0-d6/a1/a3-a6
movem.l d0-d6/a1/a3-a6,(a2)
lea 48(a2),a2
movem.l (a0)+,d0-d6/a1/a3-a6
movem.l d0-d6/a1/a3-a6,(a2)
lea 48(a2),a2
movem.l (a0)+,d0-d6/a1/a3-a6
movem.l d0-d6/a1/a3-a6,(a2)
lea 48(a2),a2
movem.l (a0)+,d0-d6/a1/a3-a6
movem.l d0-d6/a1/a3-a6,(a2)
lea 48(a2),a2
movem.l (a0)+,d0-d6/a1/a3-a6
movem.l d0-d6/a1/a3-a6,(a2)
lea 48(a2),a2
movem.l (a0)+,d0-d6/a1/a3-a6
movem.l d0-d6/a1/a3-a6,(a2)
lea 48(a2),a2
movem.l (a0)+,d0-d6/a1/a3-a6
movem.l d0-d6/a1/a3-a6,(a2)
lea 48(a2),a2
movem.l (a0)+,d0-d6/a1/a3-a6
movem.l d0-d6/a1/a3-a6,(a2)
lea 48(a2),a2
movem.l (a0)+,d0-d6/a1/a3-a6
movem.l d0-d6/a1/a3-a6,(a2)
lea 48(a2),a2
movem.l (a0)+,d0-d6/a1/a3-a6
movem.l d0-d6/a1/a3-a6,(a2)
lea 48(a2),a2
movem.l (a0)+,d0-d6/a1/a3-a6
movem.l d0-d6/a1/a3-a6,(a2)
lea 48(a2),a2
v7cx: move.w (a0)+,d0 ; (V7FN - fine) * V7FU, from mid-stream
jmp v7fh(pc,d0.w)
v7fh:
move.l (a0)+,(a2)+
move.l (a0)+,(a2)+
move.l (a0)+,(a2)+
move.l (a0)+,(a2)+
move.l (a0)+,(a2)+
move.l (a0)+,(a2)+
move.l (a0)+,(a2)+
move.l (a0)+,(a2)+
move.l (a0)+,(a2)+
move.l (a0)+,(a2)+
move.l (a0)+,(a2)+
dbra d7,v7span
subq.l #1,ITER.l
bne v7
bra done
; ---------------------------------------------------------------- V8
; THE PACKED FULL-FRAME BLIT (FINDINGS 46.6/47.2). Identical in shape to V1 --
; a row-linear movem.l chain out of a RAM frame into GVRAM -- and different in
; exactly one thing: a row is 128 WORDS, not 256, because R20 bit 11 lets one
; word carry two picture bytes. 256 = 5*48 + 16, so five 12-register bursts
; and a 4-register tail, against V1's ten and one.
;
; TIMING ONLY, and it does not set bit 11. MAME's gvram_w carries no timing in
; either arm (blit.lua's header), so the bit cannot move a cycle here; what it
; moves is the PICTURE, and the picture is what tools/bench/show_frame256_packed.lua
; and tools/bench/gvpack already verify pixel-exactly. Setting it here would
; make this variant's snapshot right and its measurement no different, and
; would put a display-mode change inside a timing loop for no gain.
;
; The source is PRE-INTERLEAVED by the host, which is the honest half of the
; claim: the packing is an encoder-side transform (46.3's argument for the text
; plane, and the same one here), so the decoder-free player's per-frame work is
; this copy and nothing else. If the interleave had to happen at run time this
; variant would be V2, not V1.
v8: lea SRCP,a0
lea DST0,a1
lea DSTE,a6
v8row: movem.l (a0)+,d0-d7/a2-a5
movem.l d0-d7/a2-a5,(a1)
movem.l (a0)+,d0-d7/a2-a5
movem.l d0-d7/a2-a5,48(a1)
movem.l (a0)+,d0-d7/a2-a5
movem.l d0-d7/a2-a5,96(a1)
movem.l (a0)+,d0-d7/a2-a5
movem.l d0-d7/a2-a5,144(a1)
movem.l (a0)+,d0-d7/a2-a5
movem.l d0-d7/a2-a5,192(a1)
movem.l (a0)+,d0-d3
movem.l d0-d3,240(a1)
lea 1024(a1),a1
cmpa.l a6,a1
bne v8row
subq.l #1,ITER.l
bne v8
bra done
; ---------------------------------------------------------------- V9
; WHAT THE PACKED LAYOUT COSTS A BLOCK DECODER (FINDINGS 47.6.4, open).
;
; V4 is the access pattern of a decoder that writes 4x4 codewords straight into
; GVRAM: 4 rows of 8 contiguous bytes at a 1024-byte stride, so each row is one
; `movem.l` of two registers. Under the packed layout that pattern is GONE.
; A block at columns x..x+3 owns the LOW bytes of four consecutive words -- four
; bytes at STRIDE 2 -- and the high bytes of those same words belong to the
; block 128 columns away. There is no burst that writes every other byte, so
; the block is sixteen `move.b`s.
;
; V9 does the pair together, low block then high block off one base, so it
; writes every byte it touches and covers the same 49,152 pixels V1/V4/V8 do.
; It is the HONEST version of "keep the codec and pack the screen": the mode
; map is unchanged, SKIP still works per block, and the writes go byte at a
; time. V10 below is the other option, and the comparison is the point.
v9: lea SRCB,a0
lea DST0,a3
lea DSTE,a4
v9brow: move.l a3,a1
lea 256(a3),a5 ; 32 block PAIRS * 8 bytes
v9blk:
move.b (a0)+,(a1)
move.b (a0)+,2(a1)
move.b (a0)+,4(a1)
move.b (a0)+,6(a1)
move.b (a0)+,1024(a1)
move.b (a0)+,1026(a1)
move.b (a0)+,1028(a1)
move.b (a0)+,1030(a1)
move.b (a0)+,2048(a1)
move.b (a0)+,2050(a1)
move.b (a0)+,2052(a1)
move.b (a0)+,2054(a1)
move.b (a0)+,3072(a1)
move.b (a0)+,3074(a1)
move.b (a0)+,3076(a1)
move.b (a0)+,3078(a1)
move.b (a0)+,1(a1)
move.b (a0)+,3(a1)
move.b (a0)+,5(a1)
move.b (a0)+,7(a1)
move.b (a0)+,1025(a1)
move.b (a0)+,1027(a1)
move.b (a0)+,1029(a1)
move.b (a0)+,1031(a1)
move.b (a0)+,2049(a1)
move.b (a0)+,2051(a1)
move.b (a0)+,2053(a1)
move.b (a0)+,2055(a1)
move.b (a0)+,3073(a1)
move.b (a0)+,3075(a1)
move.b (a0)+,3077(a1)
move.b (a0)+,3079(a1)
addq.l #8,a1
cmpa.l a5,a1
bne v9blk
lea 4096(a3),a3
cmpa.l a4,a3
bne v9brow
subq.l #1,ITER.l
bne v9
bra done
; ---------------------------------------------------------------- V10
; THE OTHER OPTION: PAIR THE BLOCKS IN THE ENCODER. If the codec codes the
; block at x and the block at x+128 as ONE unit, the destination is whole words
; again and V4's `movem.l` shape comes straight back -- the same instructions,
; the same 32 bytes of source per unit, and TWICE the pixels, because a word now
; carries two of them. So V10 is V4's inner loop run half as many times.
;
; WHAT IT COSTS IS NOT IN THIS MEASUREMENT. A pair skips only if BOTH of its
; blocks skip, and the two are 128 columns apart with nothing in the picture
; relating them. That is a CONTAINER question -- what fraction of the mode map
; survives pairing -- and 08_mode_map.py has the data to answer it. V10 prices
; the paint; it does not price the SKIPs the pairing loses.
v10: lea SRCP,a0
lea DST0,a3
lea DSTE,a4
v10brow: move.l a3,a1
lea 256(a3),a5 ; 32 block PAIRS * 8 bytes
v10blk: movem.l (a0)+,d0-d7 ; 32 bytes = one PAIR of 4x4 blocks
movem.l d0-d1,(a1)
movem.l d2-d3,1024(a1)
movem.l d4-d5,2048(a1)
movem.l d6-d7,3072(a1)
addq.l #8,a1
cmpa.l a5,a1
bne.s v10blk
lea 4096(a3),a3
cmpa.l a4,a3
bne v10brow
subq.l #1,ITER.l
bne v10
bra done
done: move.l #$FF,FLAG.l ; timer stops here
halt: bra.s halt
tools/bench/c68k/Makefile
+19
View File
@@ -0,0 +1,19 @@
# Build the headless C68K cycle harness. PX68K points at a px68k checkout;
# only m68000/c68k.c and the two header dirs are used -- no SDL, no ROMs.
PX68K ?= $(HOME)/src/px68k
# -no-pie is LOAD-BEARING, not a tidy-up. C68K is 64-bit-unsafe on purpose:
# its MOVEM macros do `src = (UINT32)(&D0)` -- they truncate the host address of
# the CPU register file to 32 bits and dereference it -- and C68k_Set_Fetch
# stores the opcode-fetch base in a UINT32 too. Under the default PIE the
# binary loads near 0x555555550000 and the first movem segfaults. -no-pie puts
# the image at 0x400000, and the harness mmaps its arena with MAP_32BIT, so
# every pointer C68K truncates still round-trips.
CFLAGS = -O2 -fno-strict-aliasing -no-pie -Wall -Wno-unused-result \
-Wno-int-to-pointer-cast -Wno-pointer-to-int-cast \
-I$(PX68K)/m68000 -I$(PX68K)/x11 -I$(PX68K)/win32api
c68k_bench: harness.c $(PX68K)/m68000/c68k.c
$(CC) $(CFLAGS) -no-pie -o $@ harness.c $(PX68K)/m68000/c68k.c
clean:
rm -f c68k_bench
tools/bench/c68k/compare.py
+78
View File
@@ -0,0 +1,78 @@
#!/usr/bin/env python3
"""Two emulators, one decoder: MAME's cycles against px68k's C68K core.
python3 tools/bench/c68k/compare.py [--mame tmp/mame_timed.log]
[--c68k tmp/c68k.log]
WHY THIS EXISTS. Every 68000 cycle figure in FINDINGS 24-35 comes from one
instrument. This puts a second, structurally different one next to it:
MAME 0.277 M68000 is the microcode core (src/devices/cpu/m68000/m68000.lst
+ m68000gen.py), NOT Musashi -- timing emerges from the 68000's
modelled micro-sequence and 4-clock bus cycles.
C68K a static per-instruction cycle table hand-transcribed from the
Motorola manual (ORI_CLOCKS_* / EA_CLOCKS_* in c68kmacro.h).
Those are two different ways of being right, so agreement is evidence and
disagreement localises to whichever instruction the anchors separate. NEITHER
charges GVRAM wait states, so both are the same lower bound on real hardware.
"""
import argparse, re, sys
ap = argparse.ArgumentParser()
ap.add_argument("--mame", default="tmp/mame_timed.log")
ap.add_argument("--c68k", default="tmp/c68k.log")
ap.add_argument("--meta", default="tmp/decode_meta.lua")
a = ap.parse_args()
meta = open(a.meta).read()
fps = int(re.search(r"fps=(\d+)", meta).group(1))
budget = 10_000_000 / fps
# anchor name -> stream offset, so the two logs can be joined: decode.lua
# reports by name, the C68K harness by offset.
names = {int(o): n for n, o in re.findall(r'name="([^"]+)", off=(\d+)', meta)}
mame = {}
txt = open(a.mame, errors="replace").read()
for nm, cyc in re.findall(r"\[DEC\] frame @ (.+?)\n.*?->\s+(\d+) cycles/frame", txt):
mame[nm.strip()] = int(cyc)
m_seq = re.search(r"full \d+-frame pass.*?\n.*?->\s+(\d+) cycles/frame", txt)
c68k, c_seq = {}, None
for line in open(a.c68k, errors="replace"):
m = re.search(r"anchor off=(\d+)\s+(\d+) cyc", line)
if m and int(m.group(1)) in names:
c68k[names[int(m.group(1))]] = int(m.group(2))
m = re.search(r"sequential pass = (\d+) cyc, mean (\d+)", line)
if m:
c_seq = int(m.group(2))
if not mame:
sys.exit(f"no MAME anchor timings in {a.mame} -- run decode.lua WITHOUT "
f"DLX_VERIFY_ONLY=1 and give -seconds_to_run enough to finish")
w = max(len(n) for n in c68k) + 2
print(f"{'anchor':<{w}}{'MAME':>10}{'C68K':>10}{'delta':>9} {'MAME':>7}{'C68K':>7} of a {fps}fps frame")
rows = []
for nm, c in c68k.items():
m = mame.get(nm)
if m is None:
print(f"{nm:<{w}}{'--':>10}{c:>10}{'':>9} {'--':>7}{100*c/budget:>6.1f}% (MAME run did not reach it)")
continue
d = 100 * (c - m) / m
rows.append(d)
print(f"{nm:<{w}}{m:>10}{c:>10}{d:>+8.2f}% {100*m/budget:>6.1f}%{100*c/budget:>6.1f}%")
if m_seq and c_seq:
m, c = int(m_seq.group(1)), c_seq
d = 100 * (c - m) / m
print(f"{'MEAN over the window':<{w}}{m:>10}{c:>10}{d:>+8.2f}% "
f"{100*m/budget:>6.1f}%{100*c/budget:>6.1f}%")
if rows:
print(f"\nspread over {len(rows)} anchors: {min(rows):+.2f}% .. {max(rows):+.2f}%")
print("C68K reads HIGH throughout." if min(rows) > 0 else
"C68K reads high on some anchors and low on others.")
print("Neither instrument charges GVRAM wait states, so both are the same\n"
"LOWER BOUND: this bounds cycle-table error, not the distance to a\n"
"real X68000 (docs/BENCHMARK.md Tier 3).")
tools/bench/c68k/harness.c
+450
View File
@@ -0,0 +1,450 @@
/* Headless C68K cycle harness -- an independent second opinion on every
* 68000 cycle figure in FINDINGS 24-35.
*
* WHY. Every one of those numbers comes from ONE instrument: MAME 0.277's
* Musashi core, timed host-side from manager.machine.time. A cycle table is a
* hand-transcribed artefact; if Musashi's is wrong for our instruction mix, the
* 833,333-cycle budget is wrong by the same amount and nothing in the tree
* would show it. This runs the SAME decode.bin against the SAME
* decode_data.bin under px68k's C68K core, which has a completely separate
* cycle table (ORI_CLOCKS_* + EA_CLOCKS_* in c68kmacro.h) written by a
* different author from the same Motorola manual.
*
* WHAT IT DOES AND DOES NOT SETTLE. C68K, like MAMEs x68000, charges NO
* GVRAM wait states -- grep the px68k tree, there is no bus-timing model
* anywhere in x68k/*.c. So this is the same LOWER BOUND, measured twice. It
* cross-checks the cycle table. It says nothing about real-hardware wait
* states; that needs XM6 TypeG or an actual X68000 (docs/BENCHMARK.md Tier 3).
*
* WHY NOT JUST RUN px68k. The decoder touches nothing but RAM, the control
* block and GVRAM: no IPL, no CRTC, no MFP, no interrupts (the MAME rig masks
* them with SR=$2700). Booting a whole emulated machine would add SDL, ROMs
* and a 55Hz sampling clock to a measurement that wants none of them. Linking
* the core alone also buys EXACTNESS: the stop cycle is captured inside the
* write callback, so a frame's cost is known to within one instruction rather
* than MAME's 1/55.46 s. That is why the anchors here run iter=1 -- decode.lua
* only iterates to beat its own timing granularity.
*
* MEMORY MODEL mirrors px68k exactly, because the core requires it: RAM is
* stored BYTE-SWAPPED (MEM[addr ^ 1], mem_wrap.c:420) so C68K's
* READ_IMM_16() = *(UINT16 *)PC works with no swap on a little-endian host.
* GVRAM word writes discard the high byte, as the hardware and MAME's
* gvram_w case 0x0100 both do.
*
* The harness is self-validating: --dump writes the decoded screen and
* verify_c68k.py checks it pixel-for-pixel against tools/encoder/dlx.py. If
* the byte-swap or the memory map were wrong the decode could not come out
* exact, so a green verify is what licenses the cycle numbers next to it.
*/
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/mman.h>
#include "c68k.h"
/* c68k.c declares these extern and tests BusErrHandling every instruction. */
unsigned int BusErrHandling = 0;
unsigned int BusErrAdr = 0;
void Error(const char *s) { fprintf(stderr, "c68k: %s\n", s); exit(3); }
void p6logd(const char *fmt, ...) { (void)fmt; }
#define ADRMASK 0xFFFFFFu
#define ARENA (16u << 20)
#define RAMTOP 0x200000u
#define GV_LO 0xC00000u
#define GV_HI 0xC80000u
#define FLAG 0x18000u
#define LFLAG 0x18040u /* src/player/load.i's control block */
#define LHDR 0x18044u
#define LDARK 0x18048u
#define LMODE 0x18054u
#define LITER 0x18058u
#define GPAL 0xE82000u
#define ITER 0x18008u
#define NFR 0x1800Cu
#define FPTR 0x18010u
#define CB1 0x20000u
#define CB4 0x22000u
#define STREAM 0x30000u
#define CODE 0x10000u
#define STACK 0x8000u
#define GVBASE 0xC00000u
#define ROWBYTES 1024u
#define CPUHZ 10000000.0
static unsigned char *buf; /* byte-swapped, px68k convention */
/* Data bus cycles the 68000 issues. Every callback below is exactly one
* 68000 bus cycle -- C68K splits a long access into two word calls, which is
* what the 16-bit bus does too -- so counting calls counts bus cycles. This
* does NOT include instruction prefetch, which C68K reads straight through the
* fetch pointer with no callback; the count is therefore a LOWER BOUND on the
* CPU's bus occupancy, and the headroom it implies is an UPPER BOUND.
* It is still the measurement that matters for FINDINGS 29.6: if the decoder's
* data accesses alone left no room, a DMAC could not overlap with it at all. */
static long long bus_r, bus_w;
static int in_exec = 0;
/* Cycle capture. A single C68k_Exec slice runs the whole pass; the FLAG
* writes inside it record where the timed region starts and ends, so the
* count excludes nothing and includes no spin-loop tail. */
static long long slice;
static long long cyc_start = -1, cyc_stop = -1;
static int desync = 0;
static unsigned char rd8 (unsigned int a){ if (in_exec) bus_r++; return buf[(a & ADRMASK) ^ 1]; }
static unsigned short rd16(unsigned int a){ if (in_exec) bus_r++; a &= ADRMASK; return (unsigned short)(buf[a] | (buf[a+1] << 8)); }
static unsigned short peek16(unsigned int a){ a &= ADRMASK; return (unsigned short)(buf[a] | (buf[a+1] << 8)); }
static unsigned int rd32(unsigned int a){ return ((unsigned int)peek16(a) << 16) | peek16(a+2); }
static void wr8(unsigned int a, unsigned char d)
{
if (in_exec) bus_w++;
a &= ADRMASK;
if (a >= GV_LO && a < GV_HI) { if (a & 1) buf[a ^ 1] = d; return; } /* high byte discarded */
buf[a ^ 1] = d;
}
/* Only writes made BY the 68000 mean anything here. The harness sets FLAG
* itself during setup, and a `move.l` to FLAG arrives as two word writes, so
* the hook sees a half-updated long in between -- clearing FLAG from $FF to 0
* momentarily reads back as $FF again. Without in_exec that transient
* recorded a run's stop cycle before the run had started, and every frame
* after the first came out as the whole slice. */
/* Which flag word the run watches. decode.s and stream.s use FLAG; the
* load-time transforms of src/player/load.i use their own, so that a player
* could eventually contain both without one clearing the other's state. The
* VALUES mean the same thing in both (1 running, $FF done, $EE failed), which
* is why one hook serves both. */
static unsigned int flag_adr = FLAG;
static void note_flag(void)
{
unsigned int v = rd32(flag_adr);
long long now = slice - C68K.ICount;
if (!in_exec) return;
if (v == 1 && cyc_start < 0) cyc_start = now;
else if (v == 0xFF || v == 0xEE) {
if (cyc_stop < 0) { cyc_stop = now; desync = (v == 0xEE); }
C68K.ICount = 0; /* stop the slice; we keep our own count */
}
}
static void wr16(unsigned int a, unsigned short d)
{
if (in_exec) bus_w++;
a &= ADRMASK;
if (a >= GV_LO && a < GV_HI) { buf[a] = (unsigned char)d; buf[a+1] = 0; return; }
buf[a] = (unsigned char)d; buf[a+1] = (unsigned char)(d >> 8);
if (a >= flag_adr && a < flag_adr + 4) note_flag();
}
static void wr32(unsigned int a, unsigned int d){ wr16(a, (unsigned short)(d >> 16)); wr16(a+2, (unsigned short)d); }
static void push(unsigned int a, const unsigned char *s, size_t n)
{
for (size_t i = 0; i < n; i++) wr8((unsigned int)(a + i), s[i]);
}
/* Prime the screen exactly as decode.lua's setup() does: active area at index
* 0, letterbox at the darkest palette entry. A SKIP block in frame 0 is a
* claim about THIS, so it is part of the decode contract. Pass 2 re-primes,
* because pass 1 left one frame's worth of residue on the screen and frame 0's
* SKIP blocks would otherwise inherit it. */
static void prime(unsigned int W, unsigned int H, unsigned int yoff, unsigned int dark)
{
for (unsigned int y = 0; y < 256; y++) {
unsigned short v = (y < yoff || y >= yoff + H) ? (unsigned short)dark : 0;
for (unsigned int x = 0; x < W; x++) wr16(GVBASE + y*ROWBYTES + x*2, v);
}
}
static unsigned char *slurp(const char *p, size_t *n)
{
FILE *f = fopen(p, "rb");
if (!f) { fprintf(stderr, "cannot open %s\n", p); exit(2); }
fseek(f, 0, SEEK_END); long L = ftell(f); fseek(f, 0, SEEK_SET);
unsigned char *b = malloc((size_t)L);
if (fread(b, 1, (size_t)L, f) != (size_t)L) { fprintf(stderr, "short read %s\n", p); exit(2); }
fclose(f); *n = (size_t)L; return b;
}
/* Run one pass and return its exact cycle count. */
static long long run(unsigned int off, unsigned int nfr, unsigned int iter)
{
cyc_start = cyc_stop = -1; desync = 0; bus_r = bus_w = 0;
wr32(FLAG, 0); wr32(ITER, iter); wr32(NFR, nfr); wr32(FPTR, STREAM + off);
C68k_Reset(&C68K);
C68k_Set_Reg(&C68K, C68K_SR, 0x2700); /* supervisor, all IRQs masked */
C68k_Set_Reg(&C68K, C68K_A7, STACK);
C68k_Set_Reg(&C68K, C68K_PC, CODE);
slice = 2000000000LL;
in_exec = 1;
C68k_Exec(&C68K, (INT32)slice);
in_exec = 0;
if (cyc_stop < 0) { fprintf(stderr, "TIMEOUT off=%u nfr=%u -- decoder never set FLAG\n", off, nfr); exit(4); }
if (desync) { fprintf(stderr, "BITSTREAM DESYNC off=%u nfr=%u\n", off, nfr); exit(5); }
/* A runaway is not a slow frame. Without this a bad record walk reports a
* two-billion-cycle "frame" as if it were a measurement. */
if (cyc_stop - cyc_start > 40LL * nfr * iter * 833333LL) {
fprintf(stderr, "RUNAWAY off=%u nfr=%u: %lld cyc (start=%lld stop=%lld) "
"PC=%06X FLAG=%08X SCR_N=%08X SCR_END=%08X len=%u\n",
off, nfr, cyc_stop - cyc_start, cyc_start, cyc_stop,
C68k_Get_Reg(&C68K, C68K_PC) & 0xFFFFFF, rd32(FLAG),
rd32(0x18014), rd32(0x18018), rd32(STREAM + off));
exit(6);
}
return cyc_stop - cyc_start;
}
/* ---- the load-time transforms (ROADMAP P1+P2, FINDINGS 53) --------------
* The same question this harness asks of the decoder, asked of the loader: does
* a SECOND 68000 core, with its own cycle table and its own memory model,
* produce the same bytes and agree about what they cost? It also counts BUS
* cycles, which MAME cannot report -- and the bus is the resource this project
* established is the binding one (FINDINGS 38).
*/
static int run_load(const char *fcode, const char *fraw, const char *dump,
unsigned int mode, unsigned int iter,
unsigned int cb1_len, unsigned int cb4_len)
{
size_t nc, nr;
unsigned char *code = slurp(fcode, &nc), *raw = slurp(fraw, &nr);
push(STREAM, raw, nr); /* the RAW container header */
push(CODE, code, nc);
/* Poison every destination, so that a transform which writes NOTHING
* cannot pass by leaving the harness's own zeros in place. */
for (unsigned int a = CB1; a < CB1 + cb1_len; a += 2) wr16(a, 0xDEAD);
for (unsigned int a = CB4; a < CB4 + cb4_len; a += 2) wr16(a, 0xDEAD);
for (unsigned int c = 0; c < 256; c++) wr16(GPAL + c*2, 0xDEAD);
wr32(LDARK, 0xFFFFFFFFu);
/* The three scratch tables are poisoned only before a run that claims to
* build them. A run that only PACKS the palette is entitled to find them
* already built -- that is the point of pricing it separately -- so when
* this process is asked for one, it does the boot pass first, untimed,
* exactly as a player would have done at boot. Without that the pack runs
* on zeros: every entry then takes the same branch and the darkest entry
* comes out 0, which is a measurement of nothing. */
if (mode & 4)
for (unsigned int a = 0x19000; a < 0x19340; a += 2) wr16(a, 0xDEAD);
flag_adr = LFLAG;
if ((mode & 2) && !(mode & 4)) {
cyc_start = cyc_stop = -1; desync = 0;
wr32(LFLAG, 0); wr32(LHDR, STREAM); wr32(LMODE, 4); wr32(LITER, 1);
C68k_Reset(&C68K);
C68k_Set_Reg(&C68K, C68K_SR, 0x2700);
C68k_Set_Reg(&C68K, C68K_A7, STACK);
C68k_Set_Reg(&C68K, C68K_PC, CODE);
slice = 2000000000LL; in_exec = 1;
C68k_Exec(&C68K, (INT32)slice);
in_exec = 0;
if (cyc_stop < 0) { fprintf(stderr, "TIMEOUT in the table pre-pass\n"); return 4; }
}
cyc_start = cyc_stop = -1; desync = 0; bus_r = bus_w = 0;
wr32(LFLAG, 0); wr32(LHDR, STREAM); wr32(LMODE, mode); wr32(LITER, iter);
C68k_Reset(&C68K);
C68k_Set_Reg(&C68K, C68K_SR, 0x2700);
C68k_Set_Reg(&C68K, C68K_A7, STACK);
C68k_Set_Reg(&C68K, C68K_PC, CODE);
slice = 2000000000LL;
in_exec = 1;
C68k_Exec(&C68K, (INT32)slice);
in_exec = 0;
if (cyc_stop < 0) { fprintf(stderr, "TIMEOUT -- loader never set LFLAG\n"); return 4; }
if (desync) { fprintf(stderr, "BAD HEADER -- load.i found no 'DLX3' magic\n"); return 5; }
long long cyc = (cyc_stop - cyc_start) / (iter ? iter : 1);
fprintf(stderr, "[C68K] load mode %u: %lld cyc/pass (%.2f ms at 10MHz, "
"%.1f%% of a 12fps frame), dark=%u\n", mode, cyc, cyc / 10000.0,
100.0 * cyc / (10000000.0 / 12), rd32(LDARK));
/* A 68000 bus cycle is 4 clocks. Prefetch is not counted (C68K reads
* opcodes straight through the fetch pointer), so this is a LOWER bound on
* occupancy and the headroom it implies is an UPPER bound -- same caveat as
* the decoder's figure above. */
{
double slots = (double)cyc / 4.0;
double used = (double)(bus_r + bus_w) / (iter ? iter : 1);
fprintf(stderr, "[C68K] data bus: %.0f reads + %.0f writes = %.0f of "
"%.0f cycles = %.1f%% occupied (prefetch NOT counted)\n",
(double)bus_r / iter, (double)bus_w / iter, used, slots,
100.0 * used / slots);
}
if (dump) {
FILE *g = fopen(dump, "wb");
if (!g) { perror(dump); return 2; }
for (unsigned int a = CB1; a < CB1 + cb1_len; a++) { unsigned char b = rd8(a); fwrite(&b,1,1,g); }
for (unsigned int a = CB4; a < CB4 + cb4_len; a++) { unsigned char b = rd8(a); fwrite(&b,1,1,g); }
for (unsigned int c = 0; c < 256; c++) {
unsigned short w = rd16(GPAL + c*2);
unsigned char b[2] = { (unsigned char)(w >> 8), (unsigned char)w };
fwrite(b, 1, 2, g);
}
fclose(g);
fprintf(stderr, "[C68K] load output dumped to %s (%u B)\n",
dump, cb1_len + cb4_len + 512);
}
return 0;
}
int main(int argc, char **argv)
{
const char *fcode = "tmp/decode.bin", *fdata = "tmp/decode_data.bin", *dump = NULL;
unsigned int cb1_len=0, cb4_len=0, pal_len=0, stream_len=0, nframes=0, H=192, W=256, fps=12;
unsigned int dark = 255;
unsigned int anch[32]; int nanch = 0;
const char *fraw = NULL, *loaddump = NULL;
unsigned int loadmode = 7, loaditer = 1;
for (int i = 1; i < argc; i++) {
if (!strcmp(argv[i], "--code")) fcode = argv[++i];
else if (!strcmp(argv[i], "--data")) fdata = argv[++i];
else if (!strcmp(argv[i], "--dump")) dump = argv[++i];
else if (!strcmp(argv[i], "--cb1")) cb1_len = (unsigned)atoi(argv[++i]);
else if (!strcmp(argv[i], "--cb4")) cb4_len = (unsigned)atoi(argv[++i]);
else if (!strcmp(argv[i], "--pal")) pal_len = (unsigned)atoi(argv[++i]);
else if (!strcmp(argv[i], "--stream")) stream_len = (unsigned)atoi(argv[++i]);
else if (!strcmp(argv[i], "--nframes"))nframes = (unsigned)atoi(argv[++i]);
else if (!strcmp(argv[i], "--W")) W = (unsigned)atoi(argv[++i]);
else if (!strcmp(argv[i], "--H")) H = (unsigned)atoi(argv[++i]);
else if (!strcmp(argv[i], "--fps")) fps = (unsigned)atoi(argv[++i]);
else if (!strcmp(argv[i], "--dark")) dark = (unsigned)atoi(argv[++i]);
else if (!strcmp(argv[i], "--loadraw")) fraw = argv[++i];
else if (!strcmp(argv[i], "--loaddump")) loaddump = argv[++i];
else if (!strcmp(argv[i], "--loadmode")) loadmode = (unsigned)atoi(argv[++i]);
else if (!strcmp(argv[i], "--loaditer")) loaditer = (unsigned)atoi(argv[++i]);
else if (!strcmp(argv[i], "--anchor")) { if (nanch < 32) anch[nanch++] = (unsigned)strtoul(argv[++i], NULL, 10); }
else { fprintf(stderr, "unknown arg %s\n", argv[i]); return 2; }
}
if (!fraw && (!nframes || !stream_len)) {
fprintf(stderr, "need --nframes and --stream (from decode_meta.lua)\n"); return 2; }
/* MAP_32BIT: C68K keeps its fetch base in a UINT32, so the arena must live
* below 4 GB or every opcode fetch reads a truncated pointer. */
buf = mmap(NULL, ARENA, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_ANONYMOUS|MAP_32BIT, -1, 0);
if (buf == MAP_FAILED) { perror("mmap MAP_32BIT"); return 2; }
fprintf(stderr, "[C68K] arena at %p\n", (void *)buf);
if (fraw) {
C68k_Init(&C68K);
C68k_Set_ReadB (&C68K, rd8);
C68k_Set_ReadW (&C68K, rd16);
C68k_Set_WriteB(&C68K, wr8);
C68k_Set_WriteW(&C68K, wr16);
C68k_Set_Fetch (&C68K, 0x000000, 0xFFFFFF, (UINT32)(unsigned long)buf);
return run_load(fcode, fraw, loaddump, loadmode, loaditer,
cb1_len ? cb1_len : 8192, cb4_len ? cb4_len : 2048);
}
size_t nc, nd;
unsigned char *code = slurp(fcode, &nc), *data = slurp(fdata, &nd);
size_t need = (size_t)cb1_len + cb4_len + pal_len + stream_len;
if (nd < need) { fprintf(stderr, "data blob %zu B < meta's %zu B\n", nd, need); return 2; }
size_t o = 0;
push(CB1, data + o, cb1_len); o += cb1_len;
push(CB4, data + o, cb4_len); o += cb4_len;
o += pal_len; /* palette: display only */
push(STREAM, data + o, stream_len);
push(CODE, code, nc);
/* Prime the screen exactly as decode.lua's setup() does: the active area
* starts at index 0 and the letterbox gets the darkest palette entry.
* A SKIP block in frame 0 is a claim about THIS, so it is part of the
* decode contract, not decoration. */
unsigned int yoff = (256u - H) / 2;
prime(W, H, yoff, dark);
C68k_Init(&C68K);
C68k_Set_ReadB (&C68K, rd8);
C68k_Set_ReadW (&C68K, rd16);
C68k_Set_WriteB(&C68K, wr8);
C68k_Set_WriteW(&C68K, wr16);
C68k_Set_Fetch (&C68K, 0x000000, 0xFFFFFF, (UINT32)(unsigned long)buf);
double frame_budget = CPUHZ / fps;
fprintf(stderr, "[C68K] %u frames, stream %u B, budget %.0f cyc/frame @ %u fps\n",
nframes, stream_len, frame_budget, fps);
/* Pass 1 -- every frame timed on its own. MAME could only afford eight
* anchor frames because its clock is 1/55.46 s; here each frame is exact,
* so the whole distribution comes out, which is what FINDINGS 31/35 score
* against. Record layout: [u32 len][768 mode][payload], next record start
* rounded up to 4 (FINDINGS 28.3). `len` counts the mode header TOO --
* decode.s sets SCR_END from the address AFTER the length word, so the
* record is 4 + len bytes, not 4 + 768 + len. */
printf("frame,offset,cycles,pct_of_frame,bus_reads,bus_writes,bus_pct\n");
unsigned int off = 0;
long long sum = 0, busr_tot = 0, busw_tot = 0;
for (unsigned int f = 0; f < nframes; f++) {
long long c = run(off, 1, 1);
sum += c;
long long br = bus_r, bw = bus_w;
busr_tot += br; busw_tot += bw;
printf("%u,%u,%lld,%.2f,%lld,%lld,%.2f\n", f, off, c,
100.0 * c / frame_budget, br, bw, 100.0 * 4.0 * (br + bw) / c);
unsigned int len = rd32(STREAM + off);
off = (off + 4 + len + 3) & ~3u;
}
fprintf(stderr, "[C68K] per-frame sum = %lld cyc, mean %.0f (%.1f%% of a %u fps frame)\n",
sum, (double)sum / nframes, 100.0 * sum / nframes / frame_budget, fps);
/* The number FINDINGS 29.6 needs. A 68000 bus cycle is 4 clocks, so a
* frame of `sum/nframes` clocks has room for a quarter that many bus
* cycles. What the decoder's DATA accesses do not use is the headroom a
* DMAC could paint spans in -- minus instruction prefetch, which is not
* counted here, so this OVERSTATES the headroom. */
{
double mean_cyc = (double)sum / nframes;
double slots = mean_cyc / 4.0;
double used = (double)(busr_tot + busw_tot) / nframes;
fprintf(stderr, "[C68K] data bus: %.0f reads + %.0f writes = %.0f cycles/frame "
"of %.0f slots = %.1f%% occupied\n",
(double)busr_tot / nframes, (double)busw_tot / nframes, used, slots,
100.0 * used / slots);
fprintf(stderr, "[C68K] headroom >= %.0f bus cycles/frame "
"(%.1f%%), MINUS instruction prefetch, which is not counted\n",
slots - used, 100.0 * (slots - used) / slots);
}
/* Pass 2 -- one sequential run of the whole window. Two jobs: it is the
* only honest correctness test (SKIP makes every frame a claim about the
* one before it), and its total against pass 1's sum prices the outer
* frame-loop overhead the per-frame runs each pay once. */
prime(W, H, yoff, dark);
long long seq = run(0, nframes, 1);
fprintf(stderr, "[C68K] sequential pass = %lld cyc, mean %.0f (%.1f%%); "
"per-frame sum is %+.3f%% of it\n",
seq, (double)seq / nframes, 100.0 * seq / nframes / frame_budget,
100.0 * (sum - seq) / seq);
/* Dump BEFORE the anchors run. They decode single frames onto this same
* screen, so anything after them is not the sequential reconstruction and
* verify_c68k.py would report every pixel wrong. */
if (dump) {
/* Active area only, one byte per pixel -- the low byte of each GVRAM
* word, which is all the hardware keeps. */
FILE *g = fopen(dump, "wb");
if (!g) { perror(dump); return 2; }
for (unsigned int y = 0; y < H; y++)
for (unsigned int x = 0; x < W; x++) {
unsigned char p = (unsigned char)rd16(GVBASE + (yoff + y)*ROWBYTES + x*2);
fwrite(&p, 1, 1, g);
}
fclose(g);
fprintf(stderr, "[C68K] screen dumped to %s (%ux%u indices)\n", dump, W, H);
}
/* Pass 3 -- decode.lua's timing anchors, at the same stream offsets, so the
* two instruments are quoted on the same eight frames. The four synthetic
* single-mode frames live past the end of the real stream and so are not
* reachable by the record walk in pass 1; they are the ones that price the
* modes separately (prep_dlx.py), which is where two cycle tables are most
* likely to disagree. */
for (int i = 0; i < nanch; i++) {
long long c = run(anch[i], 1, 1);
fprintf(stderr, "[C68K] anchor off=%-8u %8lld cyc %5.1f%% of a %u fps frame\n",
anch[i], c, 100.0 * c / frame_budget, fps);
}
return 0;
}
tools/bench/c68k/run.sh
Executable
+21
View File
@@ -0,0 +1,21 @@
#!/bin/bash
# Run the C68K harness against whatever tools/bench/prep_dlx.py last laid out,
# so it measures byte-for-byte the same code and container MAME did.
# tools/bench/c68k/run.sh [out.csv]
set -e
cd "$(dirname "$0")/../../.."
M=tmp/decode_meta.lua
[ -f "$M" ] || { echo "no $M -- run tools/bench/prep_dlx.py first"; exit 2; }
g() { sed -n "s/.*[ ,{]$1=\([0-9]*\).*/\1/p" "$M" | head -1; }
# Same anchor offsets decode.lua times, so the two instruments are quoted on the
# same frames -- including the four synthetic single-mode ones, which sit past
# the end of the real stream and price each block mode on its own.
ANCH=()
while read -r o; do ANCH+=(--anchor "$o"); done < <(sed -n 's/.*off=\([0-9]*\).*/\1/p' "$M")
tools/bench/c68k/c68k_bench \
--code tmp/decode.bin --data tmp/decode_data.bin \
--cb1 "$(g cb1_len)" --cb4 "$(g cb4_len)" --pal "$(g pal_len)" \
--stream "$(g stream_len)" --nframes "$(g nframes)" \
--W "$(g W)" --H "$(g H)" --fps "$(g fps)" --dark "$(g dark)" \
"${ANCH[@]}" \
--dump tmp/c68k_screen.bin > "${1:-tmp/c68k_frames.csv}" 2> >(tee tmp/c68k.log >&2)
tools/bench/c68k/verify_c68k.py
+52
View File
@@ -0,0 +1,52 @@
#!/usr/bin/env python3
"""Is the C68K harness's decode pixel-exact against the reference decoder?
python3 tools/bench/c68k/verify_c68k.py <in.dlx> --nframes N
This is the licence for every cycle number the harness prints. The harness
rebuilds px68k's memory model from scratch -- byte-swapped RAM, GVRAM word
writes that discard the high byte, a hand-rolled 24-bit map -- and any of that
being subtly wrong would still produce plausible-looking cycle counts. It could
not produce a pixel-exact 80-frame temporal recursion.
Unlike tools/bench/verify_decode.py this compares palette INDICES, not rendered
RGB: the harness dumps the low byte of each GVRAM word directly, so there is no
palette round-trip to model and no snapshot geometry to unpick.
"""
import argparse, sys
sys.path.insert(0, "tools/encoder")
import numpy as np
from dlx import DLX
ap = argparse.ArgumentParser()
ap.add_argument("container")
ap.add_argument("--dump", default="tmp/c68k_screen.bin")
ap.add_argument("--nframes", type=int, default=None)
a = ap.parse_args()
d = DLX(a.container)
NF = a.nframes if a.nframes is not None else d.nframes
if NF > d.nframes:
sys.exit(f"--nframes {NF} exceeds the container's {d.nframes}")
canvas = np.zeros((d.H, d.W), np.uint8)
for f in range(NF):
d.paint(canvas, f)
got = np.fromfile(a.dump, np.uint8)
if got.size != d.H * d.W:
sys.exit(f"FAIL 1. dump is {got.size} B, expected {d.H*d.W}")
got = got.reshape(d.H, d.W)
if not np.array_equal(got, canvas):
bad = got != canvas
by, bx = np.where(bad)
blocks = sorted(set(zip((by // 4).tolist(), (bx // 4).tolist())))
sys.exit(f"FAIL 2. frame {NF-1} not pixel-exact under C68K: {bad.sum()} px in "
f"{len(blocks)} blocks differ; first block "
f"(by={blocks[0][0]}, bx={blocks[0][1]})")
print(f"OK {NF} frames decoded on px68k's C68K core, final frame pixel-exact "
f"against tools/encoder/dlx.py")
print(f" {d.W}x{d.H}, {d.nb} blocks/frame, k1={d.k1} k4={d.k4}; the memory "
f"model (byte-swapped RAM, high-byte-discarding GVRAM) is therefore right")
tools/bench/check.sh
+843 -9
View File
@@ -4,12 +4,37 @@
# exit means something drifted.
set -e
cd "$(dirname "$0")/../.."
[ -d /media/reala-misaki/BDROM ] || {
echo "Blu-ray not mounted. udisksctl loop-setup -r -f DRAGONS_LAIR.iso"; exit 2; }
# No media ships with this repo. Bring your own disc; DLX_BDROM overrides the
# mount point, and every tool that reads the disc honours the same variable.
DLX_BDROM=${DLX_BDROM:-/media/${USER:-$(id -un)}/BDROM}
export DLX_BDROM
[ -d "$DLX_BDROM" ] || {
echo "Blu-ray not mounted at $DLX_BDROM."
echo " udisksctl loop-setup -r -f DRAGONS_LAIR.iso"
echo " or set DLX_BDROM to where yours is mounted."; exit 2; }
python3 tools/encoder/extract.py 00020 tmp/fr_00020 12 crop
mkdir -p tmp/snap_verify tmp/snap256
# ---------------------------------------------------------------------------
# THE ONE PLACE THE RETIRED PIPE FIGURE STILL LIVES. Session 18 removed it as
# a default from every analysis tool and from tools/bench/stream.lua, because it
# was never a bus measurement -- a user-supplied "4 Mbps" with no provenance,
# 10% of SCSI-1's asynchronous rating (FINDINGS 42.1) -- and a default let table
# after table be scored against it without anyone restating what it was.
#
# It survives HERE and only here because the gate container was ENCODED with it,
# and every per-block and span constant in FINDINGS 41/43/45/49 is fitted to that
# container. Changing this number is not an edit, it is a re-encode plus a
# re-measurement of all of them.
#
# It is a CONTAINER RECIPE, not a claim about any medium. Do not read a delivery
# rate out of it, do not copy it into a tool, and do not add a default anywhere
# that would resurrect it. When the pipe is finally measured, this becomes an
# ordinary encoder setting and the comment goes.
GATE_SPAN_KBPS=488
# ---------------------------------------------------------------------------
run() { # run <script> <snapdir>
rm -f "tmp/$2/x68000"/*.png
( cd tmp && SDL_VIDEODRIVER=dummy timeout -k 5 120 mame x68000 -bios ipl10 \
@@ -40,6 +65,19 @@ python3 tools/analysis/09_ratectl_drift.py > tmp/drift_check.log 2>&1 \
|| { cat tmp/drift_check.log; exit 1; }
tail -9 tmp/drift_check.log
echo "--- session 12: the DLX3 span container round-trips (FINDINGS 41) ---"
# 09 above replays SKIP semantics in Python and never reads a container. A v7
# span breaks exactly that shortcut -- a spanned block reads SKIP in the mode
# header and is painted by the span section instead -- so this encodes, WRITES
# the container, reads it back with the reference decoder and compares. It also
# asserts that it emitted enough spans to have tested anything.
# --kbps is required now (session 18): the tool has no default rate, so the gate
# has to say which one it is testing at. Same recipe constant as the container.
python3 tools/analysis/16_span_roundtrip.py --kbps $GATE_SPAN_KBPS \
> tmp/span_roundtrip.log 2>&1 \
|| { cat tmp/span_roundtrip.log; exit 1; }
tail -4 tmp/span_roundtrip.log
echo "--- session 7: display-path coherency (FINDINGS 28.1) ---"
# 10_pathmix_drift.py is a COUNTEREXAMPLE, kept runnable: the dual-path plan of
# FINDINGS 24.5/25.6 must still be shown to corrupt frames, and the strategy the
@@ -57,17 +95,813 @@ echo "--- session 7: 68000 decoder is pixel-exact (FINDINGS 28) ---"
# 68000 code, every block mode, full temporal recursion. A SKIP block is a claim
# about the previous frame still being on screen, so the last frame is only
# right if all 120 were.
DLX=tmp/rc_fr_singe_sasi_rcprofile.dlx
[ -f "$DLX" ] || python3 tools/encoder/encode.py tmp/fr_singe "$DLX" --profile sasi
python3 tools/bench/prep_dlx.py "$DLX" > tmp/prep_dlx.log
# The gate container is the HEAVIEST stream the encoder emits: the scsi mode
# decision (the only profile left after session 9 dropped sasi on capacity,
# FINDINGS 32) with the span pass drawing on a byte ceiling wide enough that
# every frame carries a span table and all four block modes are still exercised.
# That ceiling is GATE_SPAN_KBPS above -- a recipe, not a delivery rate.
# Spans are the newest and least-proven path in decode.s; gating on a container
# where they are rare would be gating on the old decoder. FINDINGS 41.
DLX=tmp/rc_fr_singe_scsi_span.dlx
[ -f "$DLX" ] || python3 tools/encoder/encode.py tmp/fr_singe "$DLX" --profile scsi \
--kbps 280 --span-kbps $GATE_SPAN_KBPS --spans all
# RIG_RAM is the EMULATED MACHINE's memory, and it is not a claim about the
# target. The rig preloads the whole container into RAM at 0x30000; the shipping
# player streams from disk into a ring buffer and never holds a window at once,
# so preloading is unlike the player at ANY size. At the 2 MB of a stock machine
# this gate covered 37 of 120 frames (FINDINGS 44.6.4) -- the span-heavy
# container is 5,261,814 B of stream, ending at 0x534BF6. 6 MB covers all 120.
#
# Raising it is licensed by measurement, not by convenience: at 2M and 6M the
# five synthetic anchors come out BIT-IDENTICAL (40,729 / 921,187 / 1,376,881 /
# 1,229,883 / 506,533 cycles) despite sitting at different addresses in the two
# layouts, so MAME's cycle model does not depend on ramsize over this range.
# FINDINGS 45. What is still NOT tested, at either size, is the streaming path.
RIG_RAM=${RIG_RAM:-6}
python3 tools/bench/prep_dlx.py "$DLX" --ram $((RIG_RAM * 0x100000)) > tmp/prep_dlx.log
# Verify against exactly the frame list prep_dlx emitted. It no longer truncates
# at the default RIG_RAM, but the guard stays: lower RIG_RAM, or a heavier
# container, brings truncation straight back and it must stay announced.
NF=$(sed -n 's/.*nframes=\([0-9]*\),.*/\1/p' tmp/decode_meta.lua)
grep -a "TRUNCATED" tmp/prep_dlx.log || true
tools/vasm/vasmm68k_mot -Fbin -o tmp/decode.bin src/player/decode.s > /dev/null
mkdir -p tmp/snap_decode
rm -f tmp/snap_decode/x68000/*.png
( cd tmp && DLX_VERIFY_ONLY=1 SDL_VIDEODRIVER=dummy timeout -k 5 300 mame x68000 \
-bios ipl10 -ramsize 2M -video soft -window -sound none -nothrottle -plugins \
# stdbuf -oL: a FILE is block-buffered too, so without it a long MAME run is
# unobservable until it exits and a run that is merely finishing looks exactly
# like one that is wedged (FINDINGS 34.1).
# -seconds_to_run must cover the WHOLE sequential pass. The scsi container is
# 2.7x the payload of the session-7 one this gate used to run on, and at 20 s
# the pass was truncated -- MAME exited mid-decode and verify_decode.py then
# compared a partially drawn screen and reported 49,005 differing pixels, which
# reads as a decoder bug and is not one.
( cd tmp && DLX_VERIFY_ONLY=1 SDL_VIDEODRIVER=dummy stdbuf -oL timeout -k 5 300 mame x68000 \
-bios ipl10 -ramsize ${RIG_RAM}M -video soft -window -sound none -nothrottle -plugins \
-autoboot_script ../tools/bench/decode.lua \
-snapshot_directory ./snap_decode -snapview native -seconds_to_run 20 \
-snapshot_directory ./snap_decode -snapview native -seconds_to_run 60 \
> decode_check.log 2>&1 )
python3 tools/bench/verify_decode.py "$DLX"
# A truncated run must fail as a truncated run. Without this the only symptom is
# a pixel diff against a half-drawn frame.
grep -q "snapshot taken" tmp/decode_check.log || {
echo "FAIL: the 68000 sequential pass did not complete -- no snapshot marker."
echo " Raise -seconds_to_run; the pass needs the whole container decoded."
tail -5 tmp/decode_check.log; exit 1; }
python3 tools/bench/verify_decode.py "$DLX" --nframes "$NF"
echo "--- session 10: the same decode on a second CPU core (FINDINGS 37) ---"
# A SECOND emulator, and the cheapest strong test in the tree: seconds, no MAME,
# no ROMs. px68k's C68K core has its own cycle table and its own memory model,
# so a pass here says decode.s is pixel-exact under two independent cores and
# that the harness's byte-swapped RAM / high-byte-discarding GVRAM is right --
# which is what licenses its cycle and bus numbers.
# Skipped rather than failed when px68k is not checked out: it is an external
# tree, not part of this repo.
PX68K=${PX68K:-$HOME/src/px68k}
if [ -f "$PX68K/m68000/c68k.c" ]; then
make -s -C tools/bench/c68k PX68K="$PX68K"
bash tools/bench/c68k/run.sh tmp/c68k_frames.csv 2>tmp/c68k.log
grep -a "sequential pass" tmp/c68k.log
python3 tools/bench/c68k/verify_c68k.py "$DLX" --nframes "$NF"
echo "--- session 10: the bus model still matches the machine (FINDINGS 38) ---"
# 15_bus_occupancy.py derives instruction prefetch, which no emulator here can
# report, and validates itself against the DATA accesses the harness counts.
# If that check ever stops holding, every bus figure in FINDINGS 38/39 is
# unfounded -- so it is a gate, not a report.
python3 tools/analysis/15_bus_occupancy.py "$DLX" | sed -n '3,7p'
else
echo " SKIPPED: no px68k at $PX68K (set PX68K= to point at a checkout)"
fi
echo "--- session 20: the DMAC config, read out of the IPL ROM (FINDINGS 52) ---"
# The audio and disk per-byte debits are no longer a recollection about the
# HD63450: they are bytes at named addresses in the ROM MAME boots this rig
# with. This gate re-reads them. It is cheap, it needs no emulator, and if a
# different ROM revision is ever pointed at it, it says so rather than decoding
# some other code and reporting a number.
# Skipped rather than failed when the ROM is not where MAME keeps it: that is a
# path outside this repo.
IPLROM=${IPLROM:-$HOME/mame/roms/iplrom.dat}
if [ -f "$IPLROM" ]; then
python3 tools/analysis/21_iplrom_dmac.py "$IPLROM" > tmp/iplrom_dmac.log 2>&1 \
|| { cat tmp/iplrom_dmac.log; exit 1; }
grep -ac "^ OK " tmp/iplrom_dmac.log | xargs printf " %s evidence sites hold; "
sed -n 's/^ = \(.*clocks per audio byte\)/audio is \1/p' tmp/iplrom_dmac.log
else
echo " SKIPPED: no IPL ROM at $IPLROM (set IPLROM= to point at it)"
fi
echo "--- session 18: the shared-body split is a no-op (FINDINGS 49.7.5) ---"
# src/player/decode.s and src/player/stream.s assemble from ONE copy of the block
# loop and the span chain (src/player/frame.i) so that the two front-ends cannot
# drift apart. The drift would be silent -- both would still decode correctly,
# and only the cost model would be wrong, because the 66.0 clocks/span, 9.143
# clocks/coarse pixel and every per-block constant in FINDINGS 24/30/40/41 are
# fitted to those exact bytes. So the split is asserted to be a no-op rather than
# assumed to be one.
DECODE_MD5=7a7a06f8c6d097ee0041bca4aefa3eb2 # decode.bin before the split, 1296 B
GOT=$(md5sum tmp/decode.bin | cut -d" " -f1)
[ "$GOT" = "$DECODE_MD5" ] || {
echo "FAIL: decode.bin is $GOT, expected $DECODE_MD5 ($(stat -c%s tmp/decode.bin) B)."
echo " The block loop or the span chain changed. That is allowed -- but"
echo " every cycle constant in FINDINGS 24/30/40/41 is fitted to the old"
echo " bytes, so re-measure them and move this hash, do not just move it."
exit 1; }
echo " decode.bin unchanged at $(stat -c%s tmp/decode.bin) B ($DECODE_MD5)"
# Same argument for the loader maths, which prep_dlx.py and prep_stream.py now
# share via tools/bench/dlxload.py: a second copy of the palette packing would
# drift and the symptom would be wrong colours in one rig only.
python3 tools/bench/prep_dlx.py "$DLX" --ram $((RIG_RAM * 0x100000)) --out tmp/_pdchk > /dev/null
cmp -s tmp/_pdchk_data.bin tmp/decode_data.bin || {
echo "FAIL: prep_dlx.py is not reproducible"; exit 1; }
echo " prep_dlx.py blob reproducible ($(stat -c%s tmp/decode_data.bin) B)"
rm -f tmp/_pdchk_data.bin tmp/_pdchk_meta.lua
echo "--- session 18: 120 frames through a bounded RING (FINDINGS 49) ---"
# The gate above preloads the whole container into RAM and proves the DECODER.
# This proves the DELIVERY path: the same 120 frames decoded out of a 256 KB
# ring on a STOCK 2 MB machine, with the container in a host file. The block
# loop reads with a monotonically increasing a0 and no bounds check, so a record
# placed wrongly by the wrap policy corrupts pixels rather than faulting -- which
# is why this is gated on the same pixel-exact comparison and not on a checksum.
tools/vasm/vasmm68k_mot -Fbin -o tmp/stream.bin src/player/stream.s > /dev/null
python3 tools/bench/prep_stream.py "$DLX" > tmp/prep_stream.log
mkdir -p tmp/snap_stream
rm -f tmp/snap_stream/x68000/*.png
( cd tmp && DLX_STREAM_KBPS=0 SDL_VIDEODRIVER=dummy stdbuf -oL timeout -k 5 600 \
mame x68000 -bios ipl10 -ramsize 2M -video soft -window -sound none \
-nothrottle -plugins -autoboot_script ../tools/bench/stream.lua \
-snapshot_directory ./snap_stream -snapview native -seconds_to_run 90 \
> stream_check.log 2>&1 )
# Same truncation trap as the decode stage: without this, a run that exited
# mid-decode is compared against a half-drawn screen and reads as a wrap bug.
grep -q "snapshot taken" tmp/stream_check.log || {
echo "FAIL: the ring-buffer pass did not complete -- no snapshot marker."
tail -5 tmp/stream_check.log; exit 1; }
grep -a "ring: \|DEADLINE" tmp/stream_check.log | sed "s/\[STR\] / /"
python3 tools/bench/verify_decode.py "$DLX" --snap tmp/snap_stream
echo "--- session 19: the PACED ring, and what a branch point costs (FINDINGS 51) ---"
# The stage above runs the ring FREE-RUNNING, which is right for what it gates:
# an unlimited pipe removes delivery as a variable and leaves the wrap policy
# alone under test. It cannot see buffering, because a decoder that never waits
# never lets the ring back up -- 49.7.2, and it is why 48 KB passed while
# holding one record. This runs the same 120 frames with the decoder held to
# 12 fps, which is the only configuration in which FR_HEAD-FR_TAIL means what
# it is read to mean.
#
# Gated on: pixel-exact, zero UNDERRUNS, and a ceiling that has not moved. The
# ceiling is a property of THIS container in a 256 KB ring; it is asserted
# rather than printed because a change in it is a change in how much a branch
# point can afford, and that should not slip through as a line in a log.
bash tools/bench/pace_run.sh 256 0 > tmp/pace_check.log 2>&1 || {
echo "FAIL: the paced ring pass did not complete."; tail -8 tmp/pace_check.log
exit 1; }
grep -aE "SEEK SLACK|UNDERRUNS" tmp/pace_check.log
grep -q "UNDERRUNS: 0/120" tmp/pace_check.log || {
echo "FAIL: the paced decoder underran -- a frame's slot arrived before its"
echo " record did. Free-running this is earliness (49.6); paced it is not."
exit 1; }
grep -q "ceiling 8 frames" tmp/pace_check.log || {
echo "FAIL: the 256 KB seek-slack ceiling is no longer 8 frames (FINDINGS 51)."
echo " Re-run tools/bench/pace_sweep.sh and re-derive 51 before editing"
echo " this number -- it is what a branch point can spend."
exit 1; }
grep -q "^OK" tmp/pace_check.log || { echo "FAIL: paced pass not pixel-exact";
tail -4 tmp/pace_check.log; exit 1; }
echo "--- session 21: the 68000 builds its own codebooks and palette (FINDINGS 53) ---"
# ROADMAP P1+P2. Until now tools/bench/dlxload.py expanded the codebooks and
# packed the palette HOST-SIDE and the rigs pushed the result into emulated RAM.
# A player has no host. src/player/load.i does both on the 68000, out of the RAW
# container header, and this gates it byte-for-byte against dlxload.py -- which
# stays the reference, because what changed is where the transforms RUN, not
# what they produce.
#
# Byte-for-byte and not "close enough": a wrong codebook byte is a wrong colour
# in every block that uses that codeword, and a wrong shared LSB is a slightly
# wrong colour that looks like a codec artefact rather than a loader bug.
# The palette half is read back out of the PALETTE REGISTERS at $E82000, so
# "the words reached the hardware" is part of what passes.
#
# NOT gated on the cycle counts, and the reason is NOT the one blit.s has. These
# are emulated time and reproduce exactly run to run; what they are not is
# sharp, because MAME samples them on a 1/56.69 s clock and the job takes
# milliseconds. Nothing in the tree's cost models depends on them either. A
# change in them is a re-derivation in FINDINGS 53, not a red light here.
bash tools/bench/load_run.sh "$DLX" > tmp/load_gate.log 2>&1 || {
echo "FAIL: the load-time transforms did not pass."; tail -12 tmp/load_gate.log
exit 1; }
grep -aE "^ *OK|both CPU cores|SCENE CHANGE" tmp/load_gate.log | sed 's/^ *//;s/^/ /'
echo "--- session 22: the 68000 keeps its own frame clock (FINDINGS 54) ---"
# ROADMAP P3. Until now the 12 fps tick came from tools/bench/stream.lua -- a
# host writing a word into emulated RAM. A player has no host. src/player/
# clock.i derives the tick from the CRTC's own V-DISP output through the MFP,
# with a remainder-keeping divider whose two constants are READ OUT OF THE CRTC
# at init, so the clock and the raster it counts cannot disagree.
#
# WHAT IS GATED, and it is deliberately structural rather than numeric:
# * the interrupt count equals the raster frame count -- the tick IS the
# raster, not something that merely resembles it;
# * the divider does not accumulate drift, stated in TICKS (a remainder can
# hold back at most one) rather than in ppm, which would let a longer
# window advertise a tighter clock for free;
# * every frame tick waits 4 or 5 refreshes and nothing else, which is what a
# remainder-keeping divider can produce and a broken one cannot.
# The interrupt COST is printed and not gated, for the same reason FINDINGS 53's
# cycle counts are not: it is a measurement, and a change in it is a
# re-derivation in FINDINGS 54 rather than a red light here.
bash tools/bench/clock_run.sh 3000 12 > tmp/clock_gate.log 2>&1 || {
echo "FAIL: the frame clock did not pass."; tail -12 tmp/clock_gate.log
exit 1; }
grep -aE "INTERRUPT:|PER FRAME:|DRIFT:|CADENCE:" tmp/clock_gate.log
grep -q "V-DISP interrupts 3000" tmp/clock_gate.log || {
echo "FAIL: the tick is not the raster -- the interrupt count and the frame"
echo " count disagree. Everything else in FINDINGS 54 rests on that."
exit 1; }
echo "--- session 22: 120 frames decoded on the machine's own clock (FINDINGS 54) ---"
# The strongest form of the claim: the same pixel-exact 120-frame decode out of
# the same 256 KB ring, with NOTHING outside the machine deciding when a frame
# may start. The pace gate in src/player/stream.s is byte-for-byte the one
# FINDINGS 51 measured -- it cannot tell a host-written tick from a machine-
# written one, which is why this is a test of the clock and not of a new rig.
DLX_PACE=2 bash tools/bench/pace_run.sh 256 0 > tmp/selfpace_check.log 2>&1 || {
echo "FAIL: the self-paced pass did not complete."; tail -8 tmp/selfpace_check.log
exit 1; }
grep -aE "decoder SELF-PACED|FRAME CLOCK|UNDERRUNS|NO IDLE" tmp/selfpace_check.log
grep -q "UNDERRUNS: 0/120" tmp/selfpace_check.log || {
echo "FAIL: the self-paced decoder underran."; exit 1; }
grep -q "^OK" tmp/selfpace_check.log || {
echo "FAIL: the self-paced pass was not pixel-exact. The clock changed WHEN"
echo " frames start; if it changed WHAT they draw, the interrupt is"
echo " corrupting decoder state."; tail -4 tmp/selfpace_check.log; exit 1; }
echo "--- session 23: the 68000 fills its own ring (FINDINGS 55) ---"
# ROADMAP P5. Until now the RING was filled by tools/bench/stream.lua: the host
# held the record index, chose where each record went, wrote the descriptor and
# advertised it. A player has no host. src/player/ring.i does all of that on the
# 68000, out of the DLX4 record index in the scene header, and this script keeps
# only the part that is not the CPU's -- a transport that answers one request at
# a time at a modelled rate.
#
# WHAT IS GATED:
# * pixel-exact, which is the only test that can see a wrong placement: the
# block loop reads with a monotonically increasing a0 and no bounds check,
# so a record placed over one the decoder has not finished corrupts pixels
# rather than faulting (49.2);
# * the host AUDITS every placement against its own index and its own list of
# live records, and refuses the run on the first disagreement;
# * the wrap policy still produces the SAME 18 wraps the
# host producer produced in FINDINGS 49.4 -- a third independent
# implementation of `aligned` landing on the same tiling;
# * zero underruns at a two-deep request queue, which is the finding: a
# one-deep queue leaves the channel idle between records and underran 59 of
# 120 frames on this same container and rate.
DLX_PACE=2 DLX_RINGOWN=1 DLX_QDEPTH=2 bash tools/bench/pace_run.sh 256 488 \
> tmp/ringown_check.log 2>&1 || {
echo "FAIL: the machine-owned ring pass did not complete."
tail -10 tmp/ringown_check.log; exit 1; }
grep -aE "MACHINE-OWNED|PREFILL:|CHANNEL IDLE|UNDERRUNS|SEEK SLACK" tmp/ringown_check.log \
| sed "s/\[STR\] / /"
grep -q "MISPLACED" tmp/ringown_check.log && {
echo "FAIL: the 68000 placed a record over one the decoder still owned."
exit 1; }
grep -q "UNDERRUNS: 0/120" tmp/ringown_check.log || {
echo "FAIL: the machine-owned ring underran at a two-deep queue. That is the"
echo " configuration FINDINGS 55 says keeps the channel busy; if it no"
echo " longer does, the poll site in src/player/stream.s moved."
exit 1; }
grep -q "ring: 18 wraps" tmp/ringown_check.log || {
echo "FAIL: the machine's own \`aligned\` no longer tiles this container the"
echo " way FINDINGS 49.4's host producer did (18 wraps). The policy is"
echo " meant to be the SAME policy in a different place."
exit 1; }
grep -q "^OK" tmp/ringown_check.log || {
echo "FAIL: the machine-owned ring pass was not pixel-exact -- a record was"
echo " placed or described wrongly."; tail -4 tmp/ringown_check.log
exit 1; }
echo "--- session 23: a seek, and the decode after it (FINDINGS 55) ---"
# The branch point rehearsed. A second pass over the scene begins with a real
# seek in src/player/ring.i: the channel is waited quiet, the ring is declared
# empty, the disc address of record 0 comes out of the index rather than from a
# walk, and the whole lookahead 51.3 says takes seconds of play to accumulate is
# thrown away and rebuilt from the prefill. What is gated afterwards is the one
# thing that can see a wrong seek: the last frame of the SECOND pass has to be
# pixel-exact, and a SKIP block is a claim about the previous frame, so it is
# only right if every frame after the seek was.
DLX_PACE=2 DLX_RINGOWN=1 DLX_QDEPTH=2 DLX_ITER=2 \
bash tools/bench/pace_run.sh 256 488 > tmp/ringseek_check.log 2>&1 || {
echo "FAIL: the seek pass did not complete."; tail -10 tmp/ringseek_check.log
exit 1; }
grep -aE "SEEK PASS|CHANNEL IDLE|UNDERRUNS" tmp/ringseek_check.log | sed "s/\[STR\] / /"
grep -q "SEEK PASS 2" tmp/ringseek_check.log || {
echo "FAIL: no second pass -- the seek never happened, so this gated nothing."
exit 1; }
grep -q "UNDERRUNS: 0/120" tmp/ringseek_check.log || {
echo "FAIL: the pass after the seek underran."; exit 1; }
grep -q "^OK" tmp/ringseek_check.log || {
echo "FAIL: the decode after the seek was not pixel-exact."
tail -4 tmp/ringseek_check.log; exit 1; }
echo "--- session 25: the 68000 reads the disc itself (FINDINGS 57) ---"
# ROADMAP P4, first half. Until now every byte the player consumed was placed in
# emulated RAM by a host: decode.lua preloaded a container, stream.lua answered a
# mailbox at a modelled rate. A player has no host. src/player/scsi.i selects a
# SCSI target on a real MB89352 and issues READ(10) itself.
#
# Session 21 recorded this as blocked -- "MAME's x68000 has no MB89352 path" --
# and that was wrong: `-exp1 cz6bs1` instantiates one, and FINDINGS 32.4 had
# already read its DMA glue in session 9. What was actually missing was the
# card's 8 KB boot ROM, which MAME requires to instantiate the device and which
# the player never executes. scsi_run.sh supplies a blank one on its own rompath.
#
# WHAT IS GATED: the register window (60 of 64 addresses -- the two holes ARE the
# MB89352's missing TMOD and EXBF, and they are what put DREG at $EA0015), and
# two READ(10)s verified byte-for-byte against the host's copy of the same image,
# one of them at a NON-ZERO LBA. Nothing here is gated on rate, and nothing here
# can be: MAME's device models are functional, not transfer-timing accurate.
# Skipped rather than failed when chdman is absent -- it ships with mame-tools.
if command -v chdman > /dev/null; then
bash tools/bench/scsi_run.sh "$DLX" > tmp/scsi_gate.log 2>&1 || {
echo "FAIL: the 68000 could not read the disc."; tail -14 tmp/scsi_gate.log
exit 1; }
grep -aE "ANSWERED|READ\(10\) OK" tmp/scsi_gate.log
else
echo " SKIPPED: no chdman (ships with mame-tools) -- cannot build the volume"
fi
echo "--- session 26: the ring is filled off a real SCSI volume (FINDINGS 58) ---"
# ROADMAP P4b. The stage above shows the 68000 can READ the disc. This shows it
# can RUN off it: src/player/xfer.i sits behind src/player/ring.i's XF_* mailbox
# in place of tools/bench/stream.lua's modelled transport, and the same 120
# frames are decoded out of the same 256 KB ring with NOTHING outside the
# machine in the transfer path -- no host file, no modelled rate, no synthesised
# ack.
#
# WHAT IS GATED, and it is correctness rather than rate on purpose:
# * pixel-exact, which is the only test that can see a wrong record: the
# window in scsi.i decides which of a sector's bytes reach the ring, and a
# window off by one byte desyncs the bitstream rather than faulting (49.2);
# * the SAME 18 wraps -- ring.i's placement policy must not be able to tell
# which transport answered it, and this is the assertion that says it could
# not. The WRAP COUNT is gated and the mean hole is only reported: DLX5's
# records are up to 511 B longer than DLX4's, so the hole moved (14.7 KB ->
# 13.5 KB) while the tiling did not. Gating a number that the container's
# record lengths move would gate the container, not the policy;
# * every record accounted for: 120 READ(10)s, and the bytes into the ring
# EQUAL to the bytes off the disc -- both read out of the container rather
# than written here. Under DLX4 they differed by 1.34% because a record was
# not a sector (58.3); DLX5 aligns records to sectors and the covering-sector
# read disappears, so the gate is now their IDENTITY;
# * a real mid-stream SEEK with the real transport, in the second pass. This
# is the one path that could not exist before: ring_seek waits for the
# channel to go quiet, and with the transport INSIDE the machine the only
# thing that can retire an outstanding request is that wait loop itself.
#
# NOTHING HERE IS GATED ON RATE and nothing here can be. What the run DOES cost
# is printed by tools/bench/xfer_cost.sh and recorded in FINDINGS 58.2; it is a
# measurement, and a change in it is a re-derivation there rather than a red
# light here. Skipped rather than failed when chdman is absent.
if command -v chdman > /dev/null; then
DLX_PACE=0 DLX_RINGOWN=1 DLX_QDEPTH=2 DLX_XFER=scsi \
bash tools/bench/pace_run.sh 256 0 > tmp/p4b_check.log 2>&1 || {
echo "FAIL: the 68000 could not run the ring off a real SCSI volume."
tail -12 tmp/p4b_check.log; exit 1; }
grep -aE "REAL TRANSPORT:|SECTOR OVERHEAD|ring: " tmp/p4b_check.log \
| sed "s/^ *//;s/^/ /"
grep -aq "TRANSPORT FAILED" tmp/p4b_check.log && {
echo "FAIL: a record's READ(10) reported an error."; exit 1; }
# THE BYTE COUNTS COME OUT OF THE CONTAINER, not out of this file. They were
# two hardcoded constants fitted to the DLX4 gate container, and session 28's
# re-encode went red on both of them for the right reason -- the container had
# changed and the expectation had not. A gate whose expected value is a
# literal tests the literal.
EXPECT_B=$(python3 -c "
import sys; sys.path.insert(0, 'tools/encoder')
from dlx import DLX
print(sum(DLX('$DLX').record_lengths()))")
grep -aq "REAL TRANSPORT: 120 READ(10)s by the 68000, $EXPECT_B B into the ring" \
tmp/p4b_check.log || {
echo "FAIL: the 68000 did not fetch all 120 records, or did not fetch"
echo " $EXPECT_B B of them. A short record is a desync, not a shortfall."
exit 1; }
# DLX5 MAKES THESE THE SAME NUMBER, and that identity IS the finding (59.4,
# and 58.3 option C): a sector-aligned container has no covering-sector read,
# so the disc moves exactly the records and nothing else. Under DLX4 they
# differed by 1.34% and both were gated so neither could drift into the other;
# under DLX5 the gate is that they are EQUAL. If a windowed read ever came
# back -- a container that was not aligned, or a layout that lost the
# alignment -- the disc figure would exceed the ring figure and this goes red.
grep -aq "SECTOR OVERHEAD: $EXPECT_B B off the disc for $EXPECT_B B of record = 0.00%" \
tmp/p4b_check.log || {
echo "FAIL: the disc no longer moves EXACTLY the records. On a sector-"
echo " aligned container (DLX5) there is no covering-sector read at"
echo " all, so these two counts must be the same $EXPECT_B B. If they"
echo " differ, either the container lost its alignment or scsi.i is"
echo " windowing again -- and a DMA channel cannot window (59.4)."
exit 1; }
grep -aq "ring: 18 wraps" tmp/p4b_check.log || {
echo "FAIL: the placement policy tiled this container differently with a"
echo " real transport behind it than with a modelled one. ring.i is"
echo " not supposed to be able to tell them apart."; exit 1; }
grep -aq "^OK" tmp/p4b_check.log || {
echo "FAIL: the pass off the SCSI volume was not pixel-exact."
tail -4 tmp/p4b_check.log; exit 1; }
DLX_PACE=2 DLX_RINGOWN=1 DLX_QDEPTH=2 DLX_ITER=2 DLX_XFER=scsi \
DLX_SECONDS=240 bash tools/bench/pace_run.sh 256 0 \
> tmp/p4b_seek_check.log 2>&1 || {
echo "FAIL: the seek pass off the SCSI volume did not complete."
tail -12 tmp/p4b_seek_check.log; exit 1; }
grep -aE "SEEK PASS|IS VACUOUS" tmp/p4b_seek_check.log | sed "s/^ *//;s/^/ /"
grep -aq "SEEK PASS 2" tmp/p4b_seek_check.log || {
echo "FAIL: no real seek -- the second pass never threw its ring away, so"
echo " ring_seek's quiet-wait was never asked to retire an outstanding"
echo " transfer and this gated nothing."; exit 1; }
grep -aq "^OK" tmp/p4b_seek_check.log || {
echo "FAIL: the decode after a seek off the SCSI volume was not pixel-exact."
tail -4 tmp/p4b_seek_check.log; exit 1; }
else
echo " SKIPPED: no chdman (ships with mame-tools) -- cannot build the volume"
fi
echo "--- session 27: the DMAC drives the data phase, and holds the bus (FINDINGS 59) ---"
# ROADMAP P4a, the last item before M2. The two stages above have the CPU moving
# every byte itself, at the 87.28 clocks per delivered byte FINDINGS 58.2
# measured -- 391.8% of a 12 fps frame. This one hands the DATA IN phase to the
# HD63450 and gates on the thing 57.3 said would be hard to show: that the DMAC,
# and not the CPU, is driving it.
#
# IT IS GATED WITHOUT LOOKING AT $EA0015, and that is the design. With the
# DMAC's OWN asserted -- which it is at idle here -- MAME cannot distinguish a
# CPU-driven byte at that address from a DMAC-driven one, so watching it proves
# nothing. What is gated instead is THE CPU'S OWN PROGRESS:
# * the same 2,048 B off the disc three ways -- PIO, held, stealing -- all
# three byte-exact against the host's copy, so the configuration is being
# compared against a delivery that works and not against nothing;
# * MTC sampled by the INSTRUCTION AFTER the one that starts the channel: zero
# in the held configuration (the whole transfer happened between two
# instructions, because the 68000 did not execute in between) and the full
# count in the stealing one;
# * the CPU's own trip count round its wait loop: 1 against hundreds. A
# counter that CANNOT come out different is 58.3's vacuous "UNDERRUNS: 0/120"
# again, so the run asserts the contrast and not just the held value;
# * the channel's own CSR/CER/MTC/MAR, which must say it moved every byte
# without error;
# * THE PALETTE REGISTERS AT $E82000 (session 30, ROADMAP K1): the same
# transfer aimed at the palette, byte-exact into 256 register words read
# back by the 68000; the SAME transfer aimed 20 KB away leaving the palette
# as the CPU poisoned it, which is what attributes the first run to the
# channel's MAR; and ONE array-chained start crossing from the registers
# into GVRAM, which is the shape of a whole frame -- a palette entry and
# 192 row entries, started once. What this does NOT settle is the board:
# MAME models the palette as a generic palette_device over memory_array,
# whose write16 is a plain COMBINE_DATA, so it cannot tell a register file
# that takes byte writes from one that does not (FINDINGS 62.4);
# * and a WINDOWED read through the channel REFUSED. 117 of 120 records start
# part way into a sector (58.3); a channel writes a contiguous run and cannot
# drop the bytes in front of one, so it would write the neighbouring records
# into the ring with no bounds check to catch it (49.2). The refusal is what
# makes "sector-aligned container" a precondition the transport states.
#
# NOT GATED ON RATE, and it cannot be: MAME's DMAC runs on wall-clock attotimes
# (42.5) and models a held bus by HALTING the CPU rather than by charging it
# cycles per operand. `W` is untouched. tools/analysis/28_autorequest_cost.py
# prices what this configuration costs, from the datasheet and an explicit rate.
# Skipped rather than failed when chdman is absent.
if command -v chdman > /dev/null; then
bash tools/bench/dma_run.sh "$DLX" > tmp/dma_gate.log 2>&1 || {
echo "FAIL: the DMAC did not drive the SCSI data phase."
tail -16 tmp/dma_gate.log; exit 1; }
grep -aE "BYTES OK|MTC one instruction|trips round|REFUSED|PALETTE|ONE START" \
tmp/dma_gate.log \
| sed 's/^ *//;s/^/ /'
else
echo " SKIPPED: no chdman (ships with mame-tools) -- cannot build the volume"
fi
echo "--- session 24: the scene graph, and the gap between branch points (FINDINGS 56) ---"
# The arcade scene graph is not in this repo and is not redistributable from
# here. tools/import/scenegraph.py is the ONE file in the tree that knows the
# outside projects exist; it writes tmp/scenegraph.json in this project's own
# DLXSCENE1 schema and everything downstream reads only that.
# What is gated is the IMPORT, not the numbers: 516 sequences and 906 input
# windows, and the four timing helpers still being the formulas the importer
# evaluates. Skipped when the checkout is absent.
DIRKSIMPLE=${DLX_DIRKSIMPLE:-tmp/scenegraph/DirkSimple}
if [ -f "$DIRKSIMPLE/data/games/lair/game.lua" ]; then
DLX_DIRKSIMPLE="$DIRKSIMPLE" python3 tools/import/scenegraph.py \
-o tmp/scenegraph.json > tmp/scenegraph_import.log 2>&1 \
|| { cat tmp/scenegraph_import.log; exit 1; }
sed "s/^/ /" tmp/scenegraph_import.log
grep -q "516 sequences, 906 input windows" tmp/scenegraph_import.log || {
echo "FAIL: the scene graph did not import to 516/906 -- upstream changed,"
echo " or the parser silently dropped branches."; exit 1; }
python3 tools/analysis/25_scene_graph.py --kbps 488 --ring 256 \
> tmp/scenegraph_check.log 2>&1 \
|| { tail -20 tmp/scenegraph_check.log; exit 1; }
grep -aE "^ WORST |^ ZERO-PLAY|^ BRANCH STRUCTURE" tmp/scenegraph_check.log
else
echo " SKIPPED: no DirkSimple checkout at $DIRKSIMPLE"
echo " (git clone --depth 1 https://github.com/icculus/DirkSimple)"
fi
echo "--- session 29: the packed paint, and what it does to the codec (FINDINGS 61) ---"
# tools/bench/blit.s gained V8/V9/V10 -- the packed full-frame blit, and the two
# ways a 4x4 BLOCK decoder could survive the packed layout. 47.6.1 had filed the
# packed paint's `movem` shape as an ASSUMPTION since session 16; this measures
# it, in the same run as V1/V3/V4 so it is quoted against numbers that have not
# moved since session 9.
#
# WHAT IS GATED IS STRUCTURAL, not numeric, for the reason the load stage gives:
# MAME samples these on a 1/56.69 s clock and no cost model in the tree depends
# on their exact value. What DOES depend on them is the ORDER, and the order is
# the whole of FINDINGS 61:
# V8 < V1 packing halves the full-frame literal paint
# V9 > V4 packing makes a BLOCK decoder DEARER, not cheaper
# V10 < V4 unless the blocks are paired, which costs SKIPs instead
# A tree where any of those flipped has a different answer to 44.7 and should
# say so out loud rather than let 29_packed_player.py narrate the old one.
python3 tools/bench/prep_frame.py tmp/fr_00020 tmp/frame256.bin 0 --reserve-black
rm -f tmp/blit_v8.log
( cd tmp && SDL_VIDEODRIVER=dummy stdbuf -oL timeout -k 5 300 mame x68000 \
-bios ipl10 -ramsize 2M -video soft -window -sound none -nothrottle -plugins \
-autoboot_script ../tools/bench/blit.lua -seconds_to_run 120 \
> blit_v8.log 2>&1 )
grep -aq "summary (instruction cycles only" tmp/blit_v8.log || {
echo "FAIL: the blit timing run produced no summary -- it did not finish."
tail -8 tmp/blit_v8.log; exit 1; }
python3 - <<'EOF' || exit 1
import re, sys
v = {}
for line in open("tmp/blit_v8.log", errors="replace"):
m = re.search(r"V(\d+)\s+(\d+) cyc", line)
if m: v[int(m.group(1))] = int(m.group(2))
need = (1, 2, 3, 4, 8, 9, 10)
missing = [n for n in need if n not in v]
if missing: sys.exit(f"FAIL: blit.lua reported no V{missing} -- run incomplete.")
for a, op, b, why in ((8, "<", 1, "packing did not halve the literal paint"),
(9, ">", 4, "packed BLOCK order came out CHEAPER than "
"unpacked -- 61.3's conclusion is inverted"),
(10, "<", 4, "pairing the blocks did not buy back the "
"movem shape")):
ok = v[a] < v[b] if op == "<" else v[a] > v[b]
if not ok:
sys.exit(f"FAIL: V{a} {v[a]:,} is not {op} V{b} {v[b]:,} -- {why}.")
print(f" V1 {v[1]:,} / V8 PACKED {v[8]:,} = {100*v[8]/v[1]:.0f}% -- "
f"and V3, the unpacked WRITE-ONLY floor, is {v[3]:,}")
print(f" V4 {v[4]:,} / V9 packed-block {v[9]:,} = {100*v[9]/v[4]:.0f}% -- "
f"packing costs a BLOCK decoder {100*v[9]/v[4]-100:.0f}%")
print(f" V10 paired blocks {v[10]:,} = {100*v[10]/v[4]:.0f}% of V4, and pairing "
f"is paid for in SKIPs")
EOF
python3 tools/analysis/29_packed_player.py "$DLX" > tmp/packed_player.log 2>&1 \
|| { tail -20 tmp/packed_player.log; exit 1; }
grep -aE "SKIP block PAIRS|free / DMAC->GVRAM / PACKED|^ CODEC, gate" \
tmp/packed_player.log
echo "--- session 31: the PACKED container, and the picture re-derived (FINDINGS 63) ---"
# ROADMAP K2. The container is REBUILT every run rather than reused when
# present, the way the codec's gate container is: a packed encode is 3 seconds
# because there is no k-means in it, so there is no reason to let a stale file
# stand between the encoder and the gate.
# The window's audio has to exist before the container can carry it, so the
# extraction that used to live in session 33's stage moves up here. Same seconds
# as the frames, and that is not a convenience: an audio stream that is not the
# same seconds as the picture is not this project's audio.
[ -f tmp/au_singe.raw ] || python3 tools/encoder/extract_audio.py 00223 tmp/au_singe.raw 15625 539.4 10.0
python3 tools/encoder/pack.py tmp/fr_singe tmp/packed_singe.dlxp \
--nframes "$NF" --audio tmp/au_singe.raw > tmp/pack_encode.log 2>&1 \
|| { cat tmp/pack_encode.log; exit 1; }
grep -aE "^ (record|wire|DLXP2|the four axes|lump payload)" tmp/pack_encode.log
# ...and the SILENT control beside it, which is what says the interleave moved no
# picture byte. It is the same encode with one flag off, and a packed encode is
# four seconds because there is no k-means in it, so the control is cheap enough
# to build every run rather than reason about.
python3 tools/encoder/pack.py tmp/fr_singe tmp/packed_singe_silent.dlxp \
--nframes "$NF" > tmp/pack_encode_silent.log 2>&1 \
|| { cat tmp/pack_encode_silent.log; exit 1; }
# WHAT IS GATED. Four format invariants that a DMA channel cannot check for
# itself -- it copies bytes and has no opinion about them (FINDINGS 62) -- and
# the three quality claims FINDINGS 61.9 rests the whole packed branch on:
# round-trip, sector geometry, index 0 unused, palette words agree
# packed > the shipping codec / > the codec's CEILING / > a SCENE-palette control
# A tree where any of the last three flipped has a different answer to ROADMAP K
# and should say so out loud rather than let the branch keep building.
python3 tools/analysis/30_packed_container.py tmp/packed_singe.dlxp \
--codec "$DLX" > tmp/packed_container.log 2>&1 \
|| { cat tmp/packed_container.log; exit 1; }
sed -n '/^PSNR/,$p' tmp/packed_container.log | grep -aE "RGB888|CODEC|CEILING|CONTROL|PER-FRAME|packed vs|worth|costs"
# And the container's OWN BYTES through px68k's real gvram.c, with the harness
# computing no interleave -- the only test that can catch an encoder whose byte
# order is wrong, because the container round-trips against its own inverse
# either way. Same skip-not-fail rule as the C68K stage: px68k is not in here.
if [ -f "$PX68K/x68k/gvram.c" ]; then
make -s -C tools/bench/gvpack PX68K="$PX68K"
# Frame 0 carries the two negative controls; the other two are there because
# one frame rendering does not say the container's 120th record is placed
# right, and record placement is arithmetic this format has no index to check.
python3 tools/bench/gvpack/verify_dlxp.py tmp/packed_singe.dlxp 0 --controls || exit 1
for f in $((NF / 2)) $((NF - 1)); do
# NOT piped into head: `set -e` reads a pipeline's status from its LAST
# command, so a piped verifier that failed would be reported by head's zero.
python3 tools/bench/gvpack/verify_dlxp.py tmp/packed_singe.dlxp "$f" \
> "tmp/dlxp_f$f.log" 2>&1 || { cat "tmp/dlxp_f$f.log"; exit 1; }
head -1 "tmp/dlxp_f$f.log"
done
else
echo " SKIPPED: no px68k at $PX68K -- the container's bytes were not rendered"
fi
echo "--- session 32: the PACKED PLAYER, end to end off the disc (FINDINGS 64) ---"
# ROADMAP K3. src/player/packed.s brings up its own display, builds its own
# 193-entry DMA chain, keeps its own frame clock off V-DISP and fetches every
# record itself with READ(10) off a CZ-6BS1. This script writes no picture byte
# and no palette entry.
#
# WHY EVERY FRAME IS COMPARED AND THE CODEC'S GATE COMPARES ONE. The codec is
# temporally recursive -- a SKIP block is a claim about the previous frame -- so
# its last frame audits all 120. A packed frame is a LITERAL and frame 119 says
# nothing about frame 60. The simplification that deleted the ring also deleted
# the gate's free lunch.
#
# AND WHY IT IS PACED AT HALF RATE. The write window has to be OPEN for the
# whole transfer and buffer mode blanks the graphics layer, so at the
# container's own 12 fps there is no instant at which a complete frame is
# displayable and there is nothing to snapshot (FINDINGS 64.2 -- that is the
# session's finding, not a rig limitation being worked around). Half rate opens
# a display interval without changing one byte of the transfer.
#
# GATE ONLY: runs 2-4 of packed_run.sh measure the apparatus rather than gate
# it, and they are three more MAME jobs for numbers that cannot change unless
# MAME does. tools/bench/packed_run.sh with no DLX_PK_GATE_ONLY runs all four.
if [ -f "$HOME/mame/roms/x68000.zip" ] || [ -d "$HOME/mame/roms/x68000" ]; then
DLX_PK_GATE_ONLY=1 DLX_PK_NFR="$NF" bash tools/bench/packed_run.sh \
tmp/packed_singe.dlxp > tmp/packed_gate_stage.log 2>&1 \
|| { cat tmp/packed_gate_stage.log; exit 1; }
# SPLIT AT RUN 5, because the audio run below is in the same log and its
# verifier emits OK lines too -- one grep over the whole file would print
# session 36's result under session 31's heading.
sed -n '1,/--- 5. THE AUDIO/p' tmp/packed_gate_stage.log | \
grep -aE "^ (FLAG|array|chain|frame clock|late frames|WRITE WINDOW)|^OK "
else
echo " SKIPPED: no x68000 romset -- the player was not run"
fi
echo "--- session 36: THE CONTAINER'S OWN AUDIO, OUT OF THE CHIP (FINDINGS 68) ---"
# ROADMAP P6c. Every piece of this existed before this stage did and none of it
# was joined up: the container has carried the lumps since 67, the transport is
# the IPL ROM's channel-3 configuration from 66, and what was missing was the
# lump buffer and the remainder accumulator in a player. This is the run where
# a byte of the container reaches the speaker.
#
# WHY THE GATE IS A WAV AND NOT A COUNTER. src/player/packed.s reports lumps
# armed, lumps fetched, payload bytes and starves, and EVERY ONE OF THOSE CAN BE
# RIGHT WHILE THE SOUND IS WRONG -- which is not hypothetical, it is the bug
# this session shipped: the refill ran one lump too far ahead and overwrote the
# buffer channel 3 was reading out of, and the player's account of it was
# 11 of 11 armed, 11 fetched, 78,125 B, no starve. Nothing parses a packed
# container (FINDINGS 67.4), so a wrong byte is not an error, it is a sound.
# tools/bench/verify_packed_audio.py accounts for all 78,125 bytes against
# MAME's own capture, one delivered byte at a time.
#
# The run is part of packed_run.sh's gate half, so DLX_PK_GATE_ONLY takes it.
if [ -f tmp/packed_gate_stage.log ] && \
grep -aq -- "--- 5. THE AUDIO" tmp/packed_gate_stage.log; then
sed -n '/--- 5. THE AUDIO/,/--- 7. THE SEEK/p' tmp/packed_gate_stage.log | \
grep -aE "^(OK|FAIL) |^ AUDIO:|^ {5}(nibbles per|worst|payload|-> |the player)"
else
echo " SKIPPED: no x68000 romset, or the container is silent"
fi
echo "--- session 33: AUDIO -- the encoder, and what it does to the wire (FINDINGS 65) ---"
# ROADMAP P6, everything in it except the bus half session 20 closed. The audio
# is the SAME WINDOW as the frames -- 00223 from 539.4 s for 10 s -- because an
# audio stream that is not the same seconds as the picture is not this project's
# audio, and a gate that lets the two drift apart would never say so.
# tmp/au_singe.raw was extracted by session 31's stage, which needs it to build
# the container.
# There is NO ffmpeg encoder for this format -- adpcm_ima_oki is decode-only --
# so the encoder cannot be checked against a reference. What is checked is that
# the decoder our encoder runs in its own loop IS ffmpeg's, sample for sample.
# An encoder that agrees with its own wrong decoder is the failure this catches.
python3 tools/bench/verify_adpcm.py tmp/au_singe.raw || exit 1
# And the container arithmetic. The interesting line is the padding: a packed
# record has no index BY DESIGN, so audio has to ride a fixed cadence, and the
# obvious cadence throws away a third of every audio sector.
python3 tools/analysis/32_audio_wire.py tmp/packed_singe.dlxp \
> tmp/audio_wire.log 2>&1 || { cat tmp/audio_wire.log; exit 1; }
grep -aE "^ ( 1| 11| 81) |THE FLOOR|F=1 |F=11|is ZERO|SOUND IS WHAT" tmp/audio_wire.log
echo "--- session 34: THE CHIP'S OWN DECODER, off the machine (FINDINGS 66) ---"
# ROADMAP P6a. 68000 code programs HD63450 channel 3 exactly as the IPL ROM
# programs it and feeds the MSM6258 a designed nibble stream at the chip's own
# pace; ONE of sixteen candidate decoder models reproduces MAME's capture
# sample-exact, and every axis has a negative control. The encoder disagreed
# with the chip on ALL FOUR axes, and the largest of them is not the delta
# formula 65 named -- it is the NIBBLE ORDER, at -25.7 dB.
bash tools/bench/adpcm_run.sh > tmp/adpcm_gate.log 2>&1 || {
cat tmp/adpcm_gate.log; exit 1; }
grep -aE "^(OK|FAIL) |^ (feed|nibbles|delta|clamp|accumulator)" tmp/adpcm_gate.log \
| sed 's/^/ /'
# And the bill, on the same ten seconds every other audio figure is quoted on.
python3 tools/analysis/33_adpcm_model.py tmp/au_singe.raw > tmp/adpcm_model.log 2>&1 \
|| { cat tmp/adpcm_model.log; exit 1; }
grep -aE "wrong only here|played on the chip|decoded on the encoder|headroom left" \
tmp/adpcm_model.log
echo "--- session 35: DLXP2 -- a packed container with sound in it (FINDINGS 67) ---"
# ROADMAP P6b. 65.3 did the arithmetic and wrote no byte; 66 measured which of
# sixteen decoder models the chip runs. This is the container both produce, and
# it needs a gate of its own because NOTHING PARSES A PACKED CONTAINER: a lump
# one sector out does not fail, it paints 512 B of audio and plays 512 B of
# picture, and a gate that only looked for errors would pass it.
#
# The picture side of it is gated twice over: here against a SILENT control
# built from the same frames, and above by the packed player itself, which now
# carries the `(i//F)*A` term and still gets all 120 frames pixel-exact off a
# real volume.
python3 tools/analysis/34_packed_audio.py tmp/packed_singe.dlxp \
> tmp/packed_audio.log 2>&1 || { cat tmp/packed_audio.log; exit 1; }
grep -aE "^ (OK|FAIL) |^ (order|variant|bits|init) |min of play ->" \
tmp/packed_audio.log
echo "--- session 37: THE AUDIO LEVEL, measured off the whole disc (FINDINGS 69) ---"
# ROADMAP P6, the item 66.3 reopened and two sessions deferred. The chip clamps
# its accumulator at 10 bits INSIDE the recursion, and the ten seconds every
# audio figure in this tree is quoted on peak at 435 of 511 -- which fits, and
# fits BY ACCIDENT, because that window is a -13.4 dBFS passage.
#
# So the level is measured against the loudest thing the game can play, which
# means every stream of the unique scene footage (00000-00201, FINDINGS 32.1)
# through extract_audio.py's own chain. The gate asserts the disc's peak and
# the census behind it; a different pressing is a legitimate reason for it to
# go red, a different ffmpeg downmix is not.
#
# ~18 s, and it needs the Blu-ray mounted like every other stage here.
python3 tools/analysis/35_audio_level.py --gate > tmp/audio_level.log 2>&1 \
|| { cat tmp/audio_level.log; exit 1; }
grep -aE "DISC PEAK|LOUDEST PASSAGE|THE CLAMP|LEVEL GATE|NO AUDIO TRACK|^ 1\.0000|^ 0\.5" \
tmp/audio_level.log
echo "--- session 39: THE SEEK, on the machine, with sound across it (FINDINGS 71) ---"
# Run 7 of packed_run.sh, split out of the same log for the same reason session
# 36's stage is split out of it: three verifiers in one file all emit OK lines,
# and one grep over the whole thing prints this session's result under session
# 31's heading.
if [ -f tmp/packed_gate_stage.log ] && \
grep -aq -- "--- 7. THE SEEK" tmp/packed_gate_stage.log; then
sed -n '/--- 7. THE SEEK/,$p' tmp/packed_gate_stage.log | \
grep -aE "^(OK|FAIL) |^ {4}-- the chip|^ {5}(the chip was|error against|-> )|DC +-?[0-9.]+ +AC"
else
echo " SKIPPED: no x68000 romset, or the container is silent"
fi
echo "--- session 39: the audio seek path, run across a real branch (FINDINGS 71) ---"
# ROADMAP P6d, and it is the item session 38's handoff put first: 70.3 named
# what was missing -- "src/player/packed.s starts PG_AK/PG_AKF at lump 0 and has
# no audio seek path at all" -- and priced its absence at a mean 416.5 ms of
# silence over the arcade's 409 within-container seek targets.
#
# The MACHINE half of this is run 7 of tools/bench/packed_run.sh, above: two
# passes over the container with the second starting at frame 37, which is four
# frames into lump 3 and therefore NOT on a group boundary, so the byte offset
# inside the lump is load-bearing. It runs in both chip configurations.
#
# THIS STAGE IS THE HOST HALF, and what it adds is the census the one branch
# point on the machine cannot give. The chip's accumulator is an integrator with
# NO LEAKAGE TERM, so the state error a branch creates is a DC offset that never
# decays -- and the only fix that makes a branch free is in the ENCODER, not the
# player. Both are priced here, and section 1 checks the host's arithmetic for
# the machine's own branch point against what MAME's capture measured.
python3 tools/analysis/37_audio_seek.py > tmp/audio_seek.log 2>&1 \
|| { cat tmp/audio_seek.log; exit 1; }
grep -aE "^(OK|FAIL) |RE-PLAYED BRANCH|\|DC\| against|step index the encoder|frame 37, the branch|reset every|never \(shipped\)|frames \(the cadence\)|1 frame |ONLY FIX" \
tmp/audio_seek.log
echo "--- session 38: the refill climb with a second consumer, through a real branch point (FINDINGS 70) ---"
# ROADMAP P6, the oldest item in it: 65.6 and 67.6 both recorded that the slack
# table existed and that 51.3's climb had never met a branch point with audio on
# the wire. This is that run. It needs the scene graph, so it skips with the
# session-24 stage when there is no checkout.
#
# WHAT IS GATED IS STRUCTURAL, and deliberately not the milliseconds: the
# silences move with the scene table and with the one-container-per-scene
# assumption the tool prints in its own section 5. What must not move is the
# ORDER and the SIGNS -- audio never shortens a climb, the shipped cadence
# strands most within-container branch points off a group boundary, F=1 strands
# none, and the lump read that removes the silence is an order of magnitude
# cheaper than the silence. A tree where any of those flipped has a different
# answer to the cadence pick.
if [ -f "$DIRKSIMPLE/data/games/lair/game.lua" ]; then
python3 tools/analysis/36_branch_audio.py --gate \
> tmp/branch_audio.log 2>&1 || { cat tmp/branch_audio.log; exit 1; }
grep -aE "^ (OK|FAIL) |SECOND CONSUMER IS|BRANCH-AUDIO GATE|^ mean |^ free " \
tmp/branch_audio.log
else
echo " SKIPPED: no DirkSimple checkout at $DIRKSIMPLE"
fi
echo "ALL GREEN"
tools/bench/clock.lua
+240
View File
@@ -0,0 +1,240 @@
-- Drive src/player/clockgate.s: measure the 68000's own FRAME CLOCK.
-- ROADMAP P3.
--
-- Two things are being measured and they need different instruments.
--
-- THE RATE AND THE CADENCE are counted, not timed. The clock's tick is a
-- V-DISP interrupt, and MAME's Lua sees the machine once per screen frame --
-- which is once per V-DISP. So the host's sampling granularity is exactly the
-- clock's own granularity, and the cadence comes out as integers: how many
-- refreshes each frame tick waited. There is no timing error to argue about
-- in a count of 4s and 5s.
--
-- THE COST IS TIMED BY THE 68000, because the host cannot. 1/55.46 s of host
-- granularity is 18 ms and the interrupt costs microseconds. So the 68000 runs
-- a one-instruction loop for a window of thousands of refreshes and the host
-- reads the iteration count at both ends; the interrupt cost falls out of the
-- difference between a run with the clock armed and one without. See the head
-- of src/player/clockgate.s for the arithmetic. This script emits the raw
-- counts; tools/bench/clock_cost.py does the subtraction, so that the two runs
-- it needs can be separate MAME invocations.
--
-- MEASUREMENT SCOPE. This is MAME 0.277's emulated X68000, not real hardware.
-- What is being priced is the interrupt sequence of MAME's cycle-accurate
-- M68000 core (src/devices/cpu/m68000, the `M68000` device x68k.cpp:1133 asks
-- for) against zero-wait-state RAM. Real DRAM adds wait states to the six bus
-- cycles of the exception and the four of the handler alike, so this is a LOWER
-- BOUND in the same way every other 68000 figure in this project is.
--
-- Env:
-- DLX_CLK_ON 1 = arm the frame clock, 0 = leave it off (the calibration
-- run). REQUIRED -- the two runs are not interchangeable and a
-- default would let one be reported as the other.
-- DLX_CLK_FPS frame rate to ask clk_init for (default 12)
-- DLX_CLK_WIN measurement window, in raster frames (default 3000 = 54.1 s)
-- DLX_CLK_OUT where to write the raw counts (default tmp/clock_run.txt)
M = manager.machine
SP = M.devices[":maincpu"].spaces["program"]
local function findfile(n)
for _,p in ipairs{"../tools/bench/"..n, "tools/bench/"..n, n} do
local f = io.open(p,"rb"); if f then f:close(); return p end
end
error(n.." not found")
end
local MODE = loadfile(findfile("crtc_mode.lua"))()
local CGFLAG, CGON, CGCNT = 0x18070, 0x18074, 0x18078
local CLK_PACE = 0x18034
local CLK_ACC, CLK_INCR = 0x18060, 0x18062
local CLK_VDISP, CLK_FPS = 0x18064, 0x18068
local CLK_ERR = 0x1806C
local CPUHZ = 10000000
local ONS = os.getenv("DLX_CLK_ON")
local FPS = tonumber(os.getenv("DLX_CLK_FPS") or "") or 12
local WIN = tonumber(os.getenv("DLX_CLK_WIN") or "") or 3000
local OUT = os.getenv("DLX_CLK_OUT") or "clock_run.txt"
local function P(s) print("[CLK] "..s) end
if ONS ~= "0" and ONS ~= "1" then
P("DLX_CLK_ON must be 0 (calibration, clock off) or 1 (clock armed). The "
.."cost figure is the DIFFERENCE between the two runs, so neither is "
.."meaningful alone and neither gets to be the default.")
M:exit()
return
end
local ON = (ONS == "1")
local code do local f=io.open("clockgate.bin","rb"); code=f:read("a"); f:close() end
local function T() local t=M.time; return t.seconds + t.attoseconds/1e18 end
-- Settling frames between the gate reporting `running` and the window opening.
-- The CPU may still be inside clk_init when the host first sees CGFLAG=1, and
-- the first V-DISP edge after arming lands wherever the raster happens to be.
-- Two frames puts the window entirely inside the steady state.
local SETTLE = 2
local st, n = "boot", 0
local f_ready, f0, f1 = nil, nil, nil
local c0, c1, v0, v1, p0, p1, t0, t1
-- Cadence: refreshes between consecutive frame ticks. Recorded as a histogram
-- and as the raw first few, because the interesting claim is not the mean (the
-- divider makes that exact by construction) but that the SPREAD is only ever
-- the two values either side of fps*VTOTAL/HFREQ.
local last_pace, last_pace_f, cad, seen_tick = nil, nil, {}, false
SUB = emu.add_machine_frame_notifier(function()
local ok, err = pcall(function()
n = n + 1
if st == "boot" then
if T() < 3.0 then return end
MODE.apply(SP)
for i = 1, #code do SP:write_u8(0x10000+i-1, string.byte(code,i)) end
SP:write_u32(CGFLAG, 0)
SP:write_u32(CGON, ON and 1 or 0)
SP:write_u32(CLK_FPS, FPS)
local cpu = M.devices[":maincpu"]
cpu.state["SR"].value = 0x2700 -- clk_init lowers it to $2500 itself
cpu.state["SP"].value = 0x8000
cpu.state["PC"].value = 0x10000
P(string.format("clockgate.bin=%d B, clock %s, asking for %d fps, "
.."window %d raster frames", #code,
ON and "ARMED" or "OFF (calibration run)", FPS, WIN))
st = "wait"; return
end
if st == "wait" then
local fl = SP:read_u32(CGFLAG)
if fl == 0xEE then
local e = SP:read_u32(CLK_ERR)
P("clk_init REFUSED: CLK_ERR="..e..(e == 1 and
" (CRTC is not in a 31.5 kHz mode, so HFREQ=31500 would be wrong)" or
e == 2 and " (fps*VTOTAL does not fit the 16-bit accumulator)" or ""))
M:exit(); return
end
if fl ~= 1 then
if T() > 60 then P("TIMEOUT: the gate never started"); M:exit() end
return
end
f_ready = n; st = "settle"; return
end
if st == "settle" then
if n < f_ready + SETTLE then return end
f0, t0 = n, T()
c0 = SP:read_u32(CGCNT)
v0 = SP:read_u32(CLK_VDISP)
p0 = SP:read_u32(CLK_PACE)
last_pace, last_pace_f = p0, n
if ON then
P(string.format("armed: incr=%d (fps*VTOTAL), acc=%d, first tick "
.."pending", SP:read_u16(CLK_INCR),
SP:read_u16(CLK_ACC)))
end
st = "run"; return
end
if st == "run" then
if ON then
local pc = SP:read_u32(CLK_PACE)
if pc ~= last_pace then
-- The FIRST change is dropped. Its interval runs from the window
-- opening rather than from a tick, so it measures where the window
-- happened to start and would show up as a spurious short bucket.
if seen_tick then
-- More than one tick in a single refresh would mean fps above the
-- raster rate; give it its own bucket rather than averaging it in.
local gap = n - last_pace_f
if pc - last_pace > 1 then gap = 0 end
cad[gap] = (cad[gap] or 0) + 1
end
seen_tick = true
last_pace, last_pace_f = pc, n
end
end
if n < f0 + WIN then return end
f1, t1 = n, T()
c1 = SP:read_u32(CGCNT)
v1 = SP:read_u32(CLK_VDISP)
p1 = SP:read_u32(CLK_PACE)
st = "done"
local frames = f1 - f0
local secs = t1 - t0
local clocks = secs * CPUHZ
local iters = c1 - c0
local ints = v1 - v0
local ticks = p1 - p0
P(string.format("window: %d raster frames, %.6f s emulated -> %.0f "
.."68000 clocks", frames, secs, clocks))
-- THE INSTRUMENT IS 2.22% FAST AND IT IS WORTH SAYING SO EVERY RUN.
-- The CRTC registers describe a 31,500 lines/s raster of VTOTAL lines.
-- MAME does not run it at that rate: x68k_crtc.cpp refresh_mode()
-- computes the frame period as (scr.max_x * scr.max_y) dots with
-- scr.max_x = m_htotal - 8, one character cell short and an INCLUSIVE
-- rectangle bound used as a count. So the emulated raster is fast by
-- htotal/(htotal-8) -- 368/360 in this mode -- and every rate derived
-- from it here is fast by the same factor. The divider under test is
-- built on the registers, so its HARDWARE rate is the asked-for one and
-- what this rig can check is that it tracks whatever raster it is given.
local vtotal = SP:read_u16(0xE80008) + 1
local htotal = (SP:read_u16(0xE80000) + 1) * 8
local hw_hz = 31500 / vtotal
local skew = htotal / (htotal - 8)
P(string.format(" raster period %.4f ms = %.4f Hz", 1000*secs/frames,
frames/secs))
P(string.format(" the CRTC registers describe 31500/%d = %.4f Hz; "
.."MAME is fast by htotal/(htotal-8) = %d/%d = %.4f",
vtotal, hw_hz, htotal, htotal-8, skew))
P(string.format(" loop iterations %d", iters))
if ON then
P(string.format(" V-DISP interrupts %d, frame ticks %d", ints,
ticks))
-- The self-check that makes the rest of it worth reading: the interrupt
-- count and the host's screen-frame count are supposed to be the SAME
-- clock seen from two sides. If they disagree by more than the one
-- edge the window boundaries can straddle, the tick is not the raster.
if math.abs(ints - frames) > 1 then
P(string.format("FAIL: %d V-DISP interrupts over %d raster frames. "
.."The tick is not coming from the raster.", ints,
frames))
M:exit(); return
end
-- Two numbers, and confusing them is the whole trap. The measured rate
-- is against MAME's fast raster; dividing the skew out gives the rate
-- the same code produces on a machine whose raster matches its own
-- registers, which is the number the player is judged on.
local meas = ticks/secs
P(string.format(" measured rate %.6f fps against MAME's raster "
.."(%+.0f ppm vs the asked %d)", meas,
1e6*(meas/FPS - 1), FPS))
P(string.format(" de-skewed %.6f fps -> %+.1f ppm from %d, "
.."which is the tick quantisation of %d ticks and not "
.."drift", meas/skew, 1e6*(meas/skew/FPS - 1), FPS,
ticks))
local ks = {}
for k in pairs(cad) do ks[#ks+1] = k end
table.sort(ks)
local s = ""
for _,k in ipairs(ks) do
s = s .. string.format("%d:%d ", k, cad[k])
end
P(" cadence, refreshes per frame tick: "..s)
end
local fh = io.open(OUT, "w")
fh:write(string.format("on %d\nfps %d\nframes %d\nsecs %.15g\n"
.."clocks %.15g\niters %d\nints %d\nticks %d\n"
.."vtotal %d\nhtotal %d\nhw_hz %.15g\nskew %.15g\n",
ON and 1 or 0, FPS, frames, secs, clocks, iters,
ints, ticks, vtotal, htotal, hw_hz, skew))
for k, v in pairs(cad) do fh:write(string.format("cad %d %d\n", k, v)) end
fh:close()
P("counts -> "..OUT)
P("done")
M:exit(); return
end
end)
if not ok then print("[CLK] LUA ERROR: "..tostring(err)); M:exit() end
end)
tools/bench/clock_cost.py
+132
View File
@@ -0,0 +1,132 @@
"""What the 68000's own frame clock costs, out of the two clock.lua runs.
ROADMAP P3. Usage: clock_cost.py <off-run.txt> <on-run.txt>
THE SUBTRACTION. Both runs execute the same one-instruction loop over a window
of the same number of raster frames, so the window is the same number of 68000
clocks in both. With the clock off, every clock in the window went into loop
iterations:
L = clocks / iters_off clocks per iteration
With it armed, the interrupts took some of them:
H = (clocks - iters_on * L) / ints clocks per V-DISP interrupt
L is CALIBRATED rather than looked up. That is the point: this project's cost
model (tools/analysis/buscost.py) says a 68000 bus cycle is 4 clocks and an
instruction costs 4 * (instruction words + data accesses), and the whole reason
to measure is to avoid scoring the clock against the table the table is meant to
be checked by. L falling on a whole number of clocks is therefore a RESULT, not
an assumption, and it is reported as one.
WHAT THE FIGURE IS PER FRAME. Not H -- the interrupt fires once per refresh and
a frame is several refreshes. On the hardware raster that is 31500/VTOTAL over
fps interrupts per frame, and the de-skewed rate is the one to use: MAME's
raster is fast by htotal/(htotal-8) (see tools/bench/clock.lua), and charging
the player the emulator's extra interrupts would overstate the cost by that
same 2.2%.
"""
import sys
def read(path):
d, cad = {}, {}
for line in open(path):
f = line.split()
if f[0] == "cad":
cad[int(f[1])] = int(f[2])
else:
d[f[0]] = float(f[1])
d["cad"] = cad
return d
def main(off_path, on_path):
off, on = read(off_path), read(on_path)
if off["on"] != 0 or on["on"] != 1:
sys.exit("FAIL: expected the calibration run first and the armed run "
"second; got on=%d then on=%d" % (off["on"], on["on"]))
for k in ("frames", "clocks", "fps", "vtotal"):
if off[k] != on[k]:
sys.exit("FAIL: the two runs do not share a window: %s is %g in "
"the calibration run and %g in the armed one"
% (k, off[k], on[k]))
clocks = off["clocks"]
L = clocks / off["iters"]
ints = on["ints"]
H = (clocks - on["iters"] * L) / ints
# The self-check that licenses the subtraction: the interrupt count must be
# the raster frame count. clock.lua already fails on this, restated here
# because this file is also read on its own.
if abs(ints - on["frames"]) > 1:
sys.exit("FAIL: %d interrupts over %g raster frames -- not the raster"
% (ints, on["frames"]))
fps, skew = on["fps"], on["skew"]
hw_hz = on["hw_hz"]
per_frame_ints = hw_hz / fps
per_frame = H * per_frame_ints
FRAME_CLK = 10e6 / fps
print(" calibration: %.6f clocks per loop iteration over %d iterations"
% (L, off["iters"]))
print(" (%s a whole number of clocks -- the loop is one "
"`addq.l #1,abs.l` at 7 bus cycles plus a `bra.s`)"
% ("lands on" if abs(L - round(L)) < 1e-3 else "does NOT land on"))
print(" INTERRUPT: %.2f clocks per V-DISP, measured over %d of them"
% (H, ints))
print(" PER FRAME: %.2f interrupts x %.2f = %.0f clocks = %.4f%% of a "
"%g fps frame" % (per_frame_ints, H, per_frame,
100 * per_frame / FRAME_CLK, fps))
print(" (%.4f refreshes per frame on the HARDWARE raster of "
"31500/%d = %.4f Hz, not on MAME's, which is %.4fx fast)"
% (per_frame_ints, on["vtotal"], hw_hz, skew))
# THE DRIFT GATE, and it is stated in TICKS rather than in ppm on purpose.
# A remainder-keeping divider emits floor() or ceil() of the exact tick
# count over any window and never accumulates -- so the only honest
# tolerance is one tick, and any ppm figure is that one tick divided by
# however long the window happened to be. Quoting ppm would let a longer
# window advertise a tighter clock for no reason.
want = on["frames"] * fps * on["vtotal"] / 31500.0
ticks = on["ticks"]
print(" DRIFT: %d ticks over %d refreshes; exact is %.4f, so the "
"error is %+.4f ticks" % (ticks, on["frames"], want, ticks - want))
if abs(ticks - want) > 1.0:
sys.exit("FAIL: %d ticks where %.4f were due -- off by %.2f, which is "
"more than the one tick a remainder can hold back. The "
"divider is accumulating drift." % (ticks, want, ticks - want))
cad = on["cad"]
tot = sum(cad.values())
if tot:
# Refreshes per frame is 31500 / (fps * VTOTAL) exactly -- the divider's
# own ratio, upside down. A remainder-keeping divider can only ever
# emit the two whole numbers either side of it, so anything else in the
# histogram is a bug in the divider and not a rounding taste.
rpf = 31500.0 / (fps * on["vtotal"])
lo, hi = int(rpf), int(rpf) + 1
print(" CADENCE: %s (%d intervals; %.4f refreshes per frame, so "
"only %d and %d are possible)"
% (", ".join("%dx%d (%.1f%%)" % (k, v, 100.0 * v / tot)
for k, v in sorted(cad.items())), tot, rpf, lo, hi))
for k in cad:
if k not in (lo, hi):
sys.exit("FAIL: a frame tick waited %d refreshes, which a "
"remainder-keeping divider cannot produce" % k)
# The mix is forced too: lo*a + hi*b = refreshes, a + b = ticks.
b = tot * rpf - lo * tot
print(" expected %d:%d split %.1f%% / %.1f%%, got "
"%.1f%% / %.1f%%"
% (lo, hi, 100 * (tot - b) / tot, 100 * b / tot,
100.0 * cad.get(lo, 0) / tot, 100.0 * cad.get(hi, 0) / tot))
return 0
if __name__ == "__main__":
if len(sys.argv) != 3:
sys.exit(__doc__)
sys.exit(main(sys.argv[1], sys.argv[2]))
tools/bench/clock_run.sh
Executable
+36
View File
@@ -0,0 +1,36 @@
#!/bin/bash
# One frame-clock run: the 68000 derives its own 12 fps tick from the raster
# (ROADMAP P3, FINDINGS 54).
#
# tools/bench/clock_run.sh [window-in-raster-frames] [fps]
#
# TWO MAME INVOCATIONS, and they are not interchangeable. The first leaves the
# clock off and calibrates the cost of the gate's own loop; the second arms it.
# The interrupt cost is the difference, so a run that reported only the second
# would be reporting a number it cannot compute. See src/player/clockgate.s.
#
# Only MAME can run this: the frame clock is an MFP interrupt driven by the
# CRTC's V-DISP output, and tools/bench/c68k has neither device. That is why
# this stage has no second-core half, unlike load_run.sh.
set -e
cd "$(dirname "$0")/../.."
WIN=${1:-3000}
FPS=${2:-12}
tools/vasm/vasmm68k_mot -Fbin -o tmp/clockgate.bin src/player/clockgate.s > /dev/null
for ON in 0 1; do
# stdbuf -oL: without it a long MAME run is unobservable until it exits, and
# a run that is merely finishing looks exactly like one that is wedged (34.1).
( cd tmp && DLX_CLK_ON=$ON DLX_CLK_FPS=$FPS DLX_CLK_WIN=$WIN \
DLX_CLK_OUT=clock_$ON.txt SDL_VIDEODRIVER=dummy stdbuf -oL \
timeout -k 5 600 mame x68000 -bios ipl10 -ramsize 2M -video soft -window \
-sound none -nothrottle -plugins -autoboot_script ../tools/bench/clock.lua \
-seconds_to_run 240 > clock_$ON.log 2>&1 )
# A run that never reached the counts must fail as that, not as bad arithmetic.
grep -q "^\[CLK\] done" tmp/clock_$ON.log || {
echo "FAIL: the clock rig did not finish run ON=$ON -- no completion marker."
tail -8 tmp/clock_$ON.log; exit 1; }
done
grep -a "^\[CLK\]" tmp/clock_1.log | sed -n '/window:/,/cadence/p' | sed 's/\[CLK\] / /'
python3 tools/bench/clock_cost.py tmp/clock_0.txt tmp/clock_1.txt
tools/bench/crtc_mode.lua
+20
View File
@@ -23,6 +23,26 @@
-- Total blanking time is identical to the 768 mode (112 dots @ 11.592MHz =
-- 336 dots @ 34.776MHz = 9.66us), which is what a real monitor needs.
--
-- THE EMULATOR DOES NOT RUN THE RASTER THESE REGISTERS DESCRIBE, and every
-- rig in this tree samples the machine at ITS rate, not at the hardware's.
-- x68k_crtc.cpp refresh_mode() builds the frame period as
--
-- (scr.max_x * scr.max_y) dots / dotclock, scr.max_x = m_htotal - 8
--
-- which is one character cell short AND uses an inclusive rectangle bound as a
-- count. So MAME's refresh is fast by htotal/(htotal-8) = 368/360 = 1.02222:
-- 56.6901 Hz where the registers say 55.4577. MEASURED, not read off the
-- source alone -- tools/bench/clock.lua reports both every run, and they agree
-- to six digits (FINDINGS 54.5).
--
-- It matters in exactly two places and is harmless in the rest. Anything timed
-- by counting host frames has 1/56.69 s of granularity, not 1/55.46; and
-- anything PACED by the raster runs 2.22% fast under MAME. It does NOT touch
-- 68000 cycle figures: the CPU clock is 40 MHz/4 and has nothing to do with the
-- screen. Do not "correct" the 55.4577 below to match a measurement -- it is
-- the hardware's, derived from the dot clocks above, and it is what
-- src/player/clock.i builds its divider on.
--
-- VERTICAL registers are NOT halved. The CRTC still generates a 568-line
-- 31.5kHz raster (31500/568 = 55.46 Hz); "256 lines" is a graphics-layer
-- double-scan (draw_gfx() halves gfxrect, x68k_v.cpp:401). Halving them would
tools/bench/decode.lua
+6 -1
View File
@@ -93,7 +93,7 @@ end
-- The plan: one sequential correctness pass, then the cost anchors, then a
-- full pass timed. Iteration counts target ~4 emulated seconds each so the
-- 1/55.46 s timing granularity costs under 0.5%.
-- 1/56.69 s timing granularity (crtc_mode.lua) costs under 0.5%.
-- DLX_VERIFY_ONLY=1 drops the cost anchors and runs only the correctness pass,
-- so tools/bench/check.sh can gate the decoder without paying for ~2 minutes of
-- timing runs that would make the green light sensitive to host load anyway.
@@ -157,6 +157,11 @@ SUB = emu.add_machine_frame_notifier(function()
M.video:snapshot()
P("snapshot taken after the sequential pass -- last frame, 68000-decoded")
step = step + 1
-- DLX_VERIFY_ONLY leaves nothing after the correctness pass, and this
-- used to walk off the end of PLAN and raise a Lua error AFTER the
-- snapshot was already on disk -- harmless to check.sh, and exactly the
-- kind of thing that gets mistaken for a decoder failure later.
if not PLAN[step] then st = "finish"; return end
launch(PLAN[step].off, PLAN[step].nfr, PLAN[step].iter)
st, t0 = "running", nil; return
end
tools/bench/dlxload.py
+60
View File
@@ -0,0 +1,60 @@
"""Load-time transforms every src/player/ front-end's loader has to do.
Split out of prep_dlx.py in session 18 so that prep_dlx.py (the preloaded-stream
rig) and prep_stream.py (the ring-buffer streaming rig, FINDINGS 49) share ONE
copy of them. Two copies would drift, and the drift would be silent: both rigs
would still decode, and only the colours or the codebook scaling would be
subtly wrong in one of them.
The split is a no-op by construction -- tools/bench/check.sh asserts prep_dlx.py
still emits a byte-identical blob for the gate container.
Neither transform is part of the per-frame cost being measured. The 68000 would
do both once at load time; charging them to the inner loop would flatter or damn
it for no reason.
"""
import numpy as np
def expand_codebooks(d):
"""CB1/CB4 to one WORD per pixel, so the inner loop movems them straight out.
The high byte of every GVRAM word write is discarded by the hardware, so it
is left zero and never has to be cleared. Word-per-pixel form is also what
makes index scaling a shift rather than a multiply: lsl.w #5 and lsl.w #3.
"""
cb1 = np.zeros((d.k1, 16, 2), np.uint8); cb1[:, :, 1] = d.cb1.reshape(d.k1, 16)
cb4 = np.zeros((d.k4, 4, 2), np.uint8); cb4[:, :, 1] = d.cb4.reshape(d.k4, 4)
return cb1, cb4
def pack_palette(d):
"""24-bit palette -> GGGGGRRRRRBBBBBI, shared LSB chosen PER ENTRY.
Choosing I per entry by minimum squared error rather than fixing it is worth
1.96 dB (FINDINGS 23.3). Identical maths to tools/bench/verify_frame256.py,
which is the point: the verifier and the loader must agree or a colour bug
reads as a decoder bug.
Returns (palette bytes 256x2 big-endian, index of the darkest entry, and the
RGB888 the hardware actually RENDERS from those words). The encoder does not
reserve a black entry in the CODEC container (docs/STATUS.md, encoder gaps),
so the letterbox gets the closest thing to black the palette has; the PACKED
container does reserve one (tools/encoder/dlxp.py, index 255).
`d` is a DLX container OR a bare (256,3) uint8 palette. The packed path has
no codebooks and so no DLX object to carry a palette on, and this had to stay
the ONE copy of the GRB555+I maths -- the verifier, the loader and now the
packed encoder all have to agree or a colour bug reads as a decoder bug.
"""
pal = (d if isinstance(d, np.ndarray) else d.pal).astype(int)
p6 = lambda v: ((v << 2) | (v >> 4)) & 0xFF
f = pal >> 3
render = lambda I: p6((f << 1) | I[:, None])
I = (((render(np.ones(256, int)) - pal) ** 2).sum(1)
< ((render(np.zeros(256, int)) - pal) ** 2).sum(1)).astype(int)
words = (f[:, 1] << 11) | (f[:, 0] << 6) | (f[:, 2] << 1) | I
palb = np.zeros((256, 2), np.uint8)
palb[:, 0], palb[:, 1] = words >> 8, words & 0xFF
dark = int(((render(I).astype(int)) ** 2).sum(1).argmin())
return palb, dark, render(I)
tools/bench/dma.lua
+312
View File
@@ -0,0 +1,312 @@
-- Drive src/player/dmagate.s: does the HD63450 drive the SCSI data phase, and
-- does it HOLD THE BUS? (ROADMAP P4a)
--
-- THE APPARATUS is tools/bench/scsi_run.sh's, unchanged and stated again
-- because it is two substitutions deep: `x68000 -exp1 cz6bs1` (the board 42.5
-- says to benchmark, never x68ksupr, whose internal SCSI is PIO-only in MAME),
-- and a ZERO-FILLED scsiexrom.bin on a private rompath, which is honest only
-- because the player drives the SPC registers directly and never executes a
-- byte of that ROM.
--
-- WHAT THIS RIG DOES NOT DO, and it is the point of the whole design: it never
-- looks at $EA0015. 57.3 showed that address cannot answer the question --
-- with the DMAC's OWN asserted MAME cannot tell a CPU-driven byte there from a
-- DMAC-driven one. What separates the two configurations below is whether the
-- 68000 EXECUTED ANYTHING while the bytes were arriving, which is a fact about
-- the CPU and is read out of the DMAC's own registers plus a counter the
-- machine incremented itself.
--
-- AND IT IS NOT A RATE. MAME's DMAC is configured in wall-clock attotimes
-- (42.5); its burst mode halts the CPU outright rather than charging it cycles
-- per operand. `W` is untouched here and still wants a board.
local M = manager.machine
local SP = M.devices[":maincpu"].spaces["program"]
local function P(s) print("[DMA] "..s) end
local function T() local t=M.time; return t.seconds + t.attoseconds/1e18 end
local DGFLAG, DGREC, DGREC_SZ = 0x18600, 0x18800, 32
local DGWIN, DGWERR, DGR20, DGR20N, DGR20C = 0x18700, 0x18704, 0x18708, 0x1870C, 0x18710
local DGR20P = 0x18714
local CHROW, CHN, CHBASE = 256, 8, 0xC10000
local R20OF -- filled in after the mailbox addresses are known
local GV = 0xC00000
local DGLBA, DGBLK = 1000, 4
-- the palette runs (ROADMAP K1). PS7/PS8/PS9 are the SNAPSHOTS dmagate.s takes
-- by reading $E82000 back with the 68000 after each run; the registers
-- themselves hold only the last of the three by the time the host looks.
local PAL, PALN, PALB, POIS = 0xE82000, 256, 512, 0xA500
local PS7, PS8, PS9 = 0x1A000, 0x1A200, 0x1A400
local CHROW2, CH2BASE, CHN2ROWS = 256, 0xC14000, 6
local DST = {0x20000, 0x24000, 0x28000, 0xC08000, 0xC0C000, 0xC10000,
PAL, 0x2C000, CH2BASE}
local NAME = {"PIO (the path FINDINGS 58 measured)",
"DMA, BUS HELD (DCR $00 burst, OCR $81 max rate)",
"DMA, STEALING (DCR $80 cycle steal, OCR $80 limited)",
"DMA -> GVRAM (bus held, R20 bit 11 = BUFFER MODE) [47.6.2]",
"DMA -> GVRAM (the SAME, bit 11 CLEAR -- NEGATIVE CONTROL)",
"DMA -> GVRAM (ARRAY CHAINED, 8 rows at the 1024 B line stride)",
"DMA -> PALETTE (bus held, 512 B into $E82000) [K1, 61.9]",
"DMA -> RAM (the SAME read aimed elsewhere -- NEGATIVE CONTROL:"
.." the palette must still read poison)",
"DMA -> PALETTE + SIX ROWS (ONE array-chained start across two"
.." kinds of destination)"}
local SHORT = {"pio", "held", "steal", "gvram", "masked", "chain",
"pal", "palctl", "palchain"}
local LENOF = {[6]=PALB, [7]=PALB} -- everything else is DGBLK*512
local ERRNAME = {[0]="OK", "SELECTION TIMEOUT -- no target answered",
"UNEXPECTED PHASE", "POLL TIMEOUT -- a phase never arrived",
"NON-ZERO SCSI STATUS",
"WINDOWED READ REFUSED -- a channel cannot drop bytes"}
R20OF = {[3]=DGR20, [4]=DGR20N, [5]=DGR20C, [8]=DGR20P}
local DISK = os.getenv("DLX_SCSI_IMG") or "dlxdisk.img"
local code do local f=io.open("dmagate.bin","rb"); code=f:read("a"); f:close() end
-- the disc's own bytes, once, for all three comparisons
local want do
local f = io.open(DISK, "rb")
if f then f:seek("set", DGLBA*512); want = f:read(DGBLK*512); f:close() end
end
local st = "boot"
SUB = emu.add_machine_frame_notifier(function()
local ok, err = pcall(function()
if st == "boot" then
if T() < 3.0 then return end
for i = 1, #code do SP:write_u8(0x10000+i-1, string.byte(code,i)) end
SP:write_u32(DGFLAG, 0)
local cpu = M.devices[":maincpu"]
cpu.state["SR"].value = 0x2700
cpu.state["SP"].value = 0x8000
cpu.state["PC"].value = 0x10000
P(string.format("dmagate.bin=%d B loaded at $10000; reading LBA %d, %d B, "
.."three ways, then once more into GVRAM",
#code, DGLBA, DGBLK*512))
st = "wait"; return
end
if st == "wait" then
if SP:read_u32(DGFLAG) ~= 1 then
if T() > 60 then P("TIMEOUT: the gate never finished"); P("done"); M:exit() end
return
end
if not want then P("no "..DISK.." to check against"); P("done"); M:exit(); return end
for i = 0, 8 do
local LEN = LENOF[i] or DGBLK*512
local b = DGREC + i*DGREC_SZ
local rc = SP:read_u32(b)
local e = SP:read_u32(b+4)
local mtc0 = SP:read_u32(b+8)
local spin = SP:read_u32(b+12)
local csr = SP:read_u32(b+16)
local cer = SP:read_u32(b+20)
local mtcf = SP:read_u32(b+24)
local marf = SP:read_u32(b+28)
P(NAME[i+1])
if rc ~= 0 then
P(string.format(" FAILED: err=%d (%s)", e, ERRNAME[e] or "?"))
else
local bad, first = 0, nil
-- The GVRAM run is read back a WORD at a time and split by hand.
-- SP:read_u8 on $C00000 goes through gvram_r, which in buffer mode
-- returns the whole word; asking for one byte of it would hand back
-- whichever half MAME's address space happens to hand over, and the
-- question here is precisely WHICH HALF each disc byte landed in.
-- Even disc byte -> high half (page 1), odd -> low half (page 0),
-- because the 68000 is big-endian and an even address is the MS byte.
local pg1, pg0, bad_hi, bad_lo = 0, 0, 0, 0
for k = 1, LEN do
local got
-- `a` is set for every destination that has to be read a WORD at a
-- time and split by hand -- GVRAM in buffer mode, and the palette
-- snapshots, whose words are what the 68000 read back out of
-- $E82000. Where it stays nil the destination is plain RAM.
local off, a
if i == 5 then
-- The chained run's destination is not linear: byte k of the
-- transfer is byte k%256 of row k//256, and the rows are a full
-- 1024 B line stride apart. If the channel had ignored the array
-- and run contiguously, every byte past the first row would be
-- in the wrong place and this comparison would say so.
off = (k-1) % CHROW
a = CHBASE + ((k-1) // CHROW) * 1024 + (off & ~1)
elseif i == 6 then
off = (k-1) % 2
a = PS7 + ((k-1) & ~1)
elseif i == 8 then
-- ONE transfer across two kinds of destination: the first sector
-- is the palette, the rest is six picture rows at the line
-- stride. The split is the array's, and this walks it the same
-- way the channel was told to.
if k <= PALB then
off = (k-1) % 2
a = PS9 + ((k-1) & ~1)
else
local idx = k - PALB - 1
off = idx % 2
a = CH2BASE + (idx // CHROW2) * 1024 + ((idx % CHROW2) & ~1)
end
elseif i >= 3 and i ~= 7 then
off = (k-1) % 2
a = DST[i+1] + ((k-1) & ~1)
end
if a then
local w = SP:read_u16(a)
if (off % 2) == 0 then got = (w >> 8) & 0xff; pg1 = pg1 + 1
else got = w & 0xff; pg0 = pg0 + 1 end
else
got = SP:read_u8(DST[i+1]+k-1)
end
if got ~= string.byte(want, k) then
bad = bad + 1; first = first or (k-1)
if ((k-1) % 2) == 0 then bad_hi = bad_hi + 1
else bad_lo = bad_lo + 1 end
end
end
if R20OF[i] then
P(string.format(" R20 during the run = $%04X (bit 11 %s); %d bytes "
.."read back out of the HIGH half of a destination "
.."word and %d out of the LOW half",
SP:read_u32(R20OF[i]),
((SP:read_u32(R20OF[i]) & 0x0800) ~= 0)
and "SET" or "CLEAR",
pg1, pg0))
end
-- THE PALETTE RUNS' OWN VACUITY CHECK. Run 7's destination was
-- poisoned by the 68000 first, so "it matches the disc" cannot be
-- satisfied by a channel that did nothing -- but only if the poison
-- and the disc actually differ everywhere they are compared. That is
-- a property of THIS record and is counted rather than assumed.
if i == 6 then
local diff = 0
for j = 0, PALN-1 do
local w = (POIS | j) & 0xffff
if ((w >> 8) & 0xff) ~= string.byte(want, 2*j+1) then diff = diff + 1 end
if (w & 0xff) ~= string.byte(want, 2*j+2) then diff = diff + 1 end
end
P(string.format(" PALETTE POISON IS A DISCRIMINATOR: %d of %d "
.."positions differ from the disc's bytes -- a "
.."channel that wrote nothing could not have passed "
.."in those.", diff, PALB))
P(string.format(" %d bytes read back out of the HIGH half of a "
.."palette word (G and the top of R) and %d out of "
.."the LOW half", pg1, pg0))
end
if bad == 0 then
P(string.format(" BYTES OK: %d B from LBA %d match %s byte for byte "
.."[%s]", LEN, DGLBA, DISK, SHORT[i+1]))
if i == 3 then
P(" A CHANNEL FILLS THE PACKED LAYOUT: every disc byte landed in "
.."its own half of a GVRAM word, with the CPU halted -- so a "
.."stream interleaved (right<<8)|left goes from disc to screen "
.."with no CPU in the loop (47.6.2, first half).")
end
if i == 5 then
P(string.format(" THE CHANNEL WALKED THE ARRAY ITSELF: %d rows of "
.."%d B landed at a %d B line stride from ONE start, CPU halted "
.."throughout. A frame is %d such entries; the CPU does not "
.."restart the channel per row.", CHN, CHROW, 1024, 192))
end
if i == 4 then
P(" CONTROL DID NOT FAIL: the masked write path delivered every "
.."byte too, so the run above is not evidence about R20 bit 11.")
end
if i == 6 then
P(" A CHANNEL WRITES THE PALETTE REGISTERS: 512 B off the disc "
.."became 256 palette words, read back OUT OF $E82000 by the "
.."68000 itself, with the CPU halted for the transfer. Each "
.."disc byte landed in its own half of a register word, so a "
.."per-frame palette needs no CPU (61.9, ROADMAP K1).")
end
if i == 8 then
P(string.format(" ONE START PAINTED THE PALETTE AND %d ROWS: a "
.."single array-chained transfer crossed from device registers "
.."at $%06X into GVRAM at $%06X, %d B in %d entries, CPU halted "
.."throughout. A frame is that shape with %d row entries "
.."instead of %d.", CHN2ROWS, PAL, CH2BASE, LEN, CHN2ROWS+1,
192, CHN2ROWS))
end
elseif i == 6 or i == 8 then
P(string.format(" PALETTE WRONG [%s]: %d of %d differ, first at "
.."+%d -- %d at EVEN offsets (the HIGH half of a "
.."word), %d at ODD.",
SHORT[i+1], bad, LEN, first, bad_hi, bad_lo))
elseif i == 4 then
-- THE CLAIM IS NOT "half the bytes differ". In masked 256-colour
-- mode gvram_w takes `data & 0x00ff` and ignores mem_mask, so a byte
-- written to an EVEN address is never stored and the high half keeps
-- whatever it held; some of those stale halves match the disc by
-- coincidence, and this record is full of pad, so a lot of them do.
-- The mechanism's signature is WHERE the damage is, not how much:
-- every ODD byte must survive and only EVEN ones may be lost.
P(string.format(" BYTES LOST [masked]: %d of %d differ (first at "
.."+%d) -- %d at EVEN offsets, %d at ODD.",
bad, LEN, first, bad_hi, bad_lo))
if bad_lo == 0 and bad_hi > 0 then
P(string.format(" EXACTLY THE MECHANISM: all %d survivors of the "
.."high half are stale GVRAM that happens to match "
.."(this record is mostly pad); not one of the %d "
.."ODD bytes was harmed. Bit 11 is what carried the "
.."even ones in the run above.", LEN//2 - bad_hi, LEN//2))
end
else
P(string.format(" BYTES WRONG [%s]: %d of %d differ, first at +%d",
SHORT[i+1], bad, LEN, first))
end
end
if i == 7 then
-- THE ATTRIBUTION CONTROL'S SECOND CLAIM, and the one that makes run
-- 7 mean something: the same transfer aimed 20 KB away leaves the
-- palette exactly as the 68000 poisoned it. If this comes back with
-- the disc's bytes in it, something other than the channel's MAR
-- decides what reaches $E82000 and run 7 measured that instead.
local stale, first_s = 0, nil
for j = 0, PALN-1 do
if SP:read_u16(PS8 + 2*j) ~= ((POIS | j) & 0xffff) then
stale = stale + 1; first_s = first_s or j
end
end
if stale == 0 then
P(string.format(" PALETTE UNTOUCHED BY THE CONTROL: %d of %d words "
.."still read the poison the 68000 wrote, so the "
.."bytes in run 7 got there because the channel's "
.."MAR pointed at $%06X.", PALN, PALN, PAL))
else
P(string.format(" CONTROL DID NOT FAIL [palctl]: %d of %d palette "
.."words are no longer poison (first at entry %d) "
.."-- the palette changed without a channel aimed "
.."at it.", stale, PALN, first_s))
end
end
if i > 0 then
-- THE DISCRIMINATOR. MTC as the instruction after START saw it, and
-- the number of times the CPU went round its own wait loop.
P(string.format(" MTC one instruction after START: %d of %d -> the "
.."CPU %s while the transfer ran [%s]",
mtc0, LEN,
(mtc0 == 0) and "NEVER EXECUTED" or "kept executing",
SHORT[i+1]))
P(string.format(" CPU trips round the wait loop: %d [%s]", spin, SHORT[i+1]))
P(string.format(" channel: CSR=$%02X (%s%s%s) CER=$%02X MTC=%d "
.."MAR=$%06X (+%d) [%s]",
csr,
((csr & 0x80) ~= 0) and "COC " or "",
((csr & 0x10) ~= 0) and "ERR " or "",
((csr & 0x08) ~= 0) and "ACT" or "idle",
cer, mtcf, marf, marf - DST[i+1], SHORT[i+1]))
end
end
-- The refusal. Expected to fail, and the run is only green if it did.
local w, we = SP:read_u32(DGWIN), SP:read_u32(DGWERR)
if w == 0xFFFFFFFF and we == 5 then
P("WINDOWED DMA READ REFUSED, as it must be: a channel writes a "
.."contiguous run and cannot drop the 300 B in front of the record "
.."(58.3). P4a's precondition is a SECTOR-ALIGNED container.")
else
P(string.format("WINDOW NOT REFUSED: rc=%d err=%d -- the transport would "
.."have written the neighbours' bytes into the ring.", w, we))
end
P("done"); M:exit(); return
end
end)
if not ok then P("LUA ERROR: "..tostring(err)); P("done"); M:exit() end
end)
tools/bench/dma_run.sh
Executable
+154
View File
@@ -0,0 +1,154 @@
#!/bin/bash
# One HD63450 data-phase run: does the DMAC drive the SCSI data phase, and does
# it HOLD THE BUS? (ROADMAP P4a, the last item before M2.)
#
# tools/bench/dma_run.sh [container.dlx]
#
# The apparatus is tools/bench/scsi_run.sh's -- `x68000 -exp1 cz6bs1` and a
# zero-filled scsiexrom.bin on a private rompath -- and the volume is
# tools/bench/mkvol.sh's, the same bytes the host-file ring rig reads.
#
# WHAT A GREEN RUN MEANS: the same 2,048 B came off the disc three ways -- PIO,
# the channel with the bus held, the channel stealing cycles -- all three
# byte-exact against the host's copy; and in the held configuration THE WHOLE
# TRANSFER HAPPENED BETWEEN TWO INSTRUCTIONS, which is what holding the bus
# means and is not a claim about $EA0015 (57.3).
#
# WHAT IT DOES NOT MEAN: anything about `W`. MAME's DMAC runs on wall-clock
# attotimes (42.5) and models a held bus by HALTING the CPU rather than by
# charging it cycles per operand. This settles which configuration works.
set -e
cd "$(dirname "$0")/../.."
DLX=${1:-tmp/rc_fr_singe_scsi_span.dlx}
bash tools/bench/mkvol.sh "$DLX"
tools/vasm/vasmm68k_mot -Fbin -o tmp/dmagate.bin src/player/dmagate.s > /dev/null
# What the player will program, decoded out of the same constants it programs.
python3 tools/analysis/27_dmac_config.py
# stdbuf -oL: without it a long MAME run is unobservable until it exits, and a
# run that is merely finishing looks exactly like one that is wedged (34.1).
( cd tmp && SDL_VIDEODRIVER=dummy stdbuf -oL timeout -k 5 300 \
mame x68000 -bios ipl10 -exp1 cz6bs1 \
-rompath "$HOME/mame/roms;./p4roms" -hard dlxdisk.chd \
-ramsize 2M -video soft -window -sound none -nothrottle -plugins \
-autoboot_script ../tools/bench/dma.lua \
-seconds_to_run 90 > dma_run.log 2>&1 )
grep -aq "^\[DMA\] done" tmp/dma_run.log || {
echo "FAIL: the DMA gate did not finish -- no completion marker."
tail -8 tmp/dma_run.log; exit 1; }
grep -a "^\[DMA\]" tmp/dma_run.log | sed 's/^\[DMA\] / /'
# THE ASSERTIONS. Printing a result and gating on it are different things.
fail() { echo "FAIL: $1"; exit 1; }
grep -aq "BYTES OK: 2048 B from LBA 1000 .*\[pio\]" tmp/dma_run.log || \
fail "the PIO reference read did not match -- nothing below is about the DMAC."
grep -aq "BYTES OK: 2048 B from LBA 1000 .*\[held\]" tmp/dma_run.log || \
fail "the bus-held DMA read did not deliver the disc's bytes."
grep -aq "BYTES OK: 2048 B from LBA 1000 .*\[steal\]" tmp/dma_run.log || \
fail "the cycle-stealing DMA read did not deliver the disc's bytes."
grep -aq "MTC one instruction after START: 0 of 2048 .*NEVER EXECUTED .*\[held\]" \
tmp/dma_run.log || \
fail "the bus was NOT held: the CPU executed while the channel ran, so this is
not the configuration ROADMAP P4a asks for. That MTC is the whole of the
evidence that does not come from watching \$EA0015 (57.3)."
grep -aq "CPU trips round the wait loop: 1 \[held\]" tmp/dma_run.log || \
fail "the held configuration's CPU went round its wait loop more than once --
it was running, so the bus was not held for the whole transfer."
# A NEGATIVE ASSERTION IS WRITTEN AS AN `if`, not as `grep ... && fail`: under
# `set -e` a failing grep in an AND-list takes the whole script's exit status
# with it, so the run would report the failure it was looking for as a pass.
SPIN=$(sed -n 's/.*CPU trips round the wait loop: \([0-9]*\) \[steal\].*/\1/p' \
tmp/dma_run.log)
[ -n "$SPIN" ] && [ "$SPIN" -ge 100 ] || \
fail "the cycle-stealing configuration did not leave the CPU running (spin
= ${SPIN:-none}) -- the two configurations are meant to DIFFER in exactly
that, and a contrast of one against one is not a contrast."
if grep -aq "MTC one instruction after START: 0 of 2048 .*\[steal\]" tmp/dma_run.log
then
fail "the cycle-stealing configuration also finished between two instructions,
so the comparison has no contrast in it and the discriminator is measuring
something other than bus ownership."
fi
grep -aq "COC .*CER=\$00 MTC=0 .*(+2048) \[held\]" tmp/dma_run.log || \
fail "the held channel did not report a clean completion of every byte."
grep -aq "COC .*CER=\$00 MTC=0 .*(+2048) \[steal\]" tmp/dma_run.log || \
fail "the stealing channel did not report a clean completion of every byte."
# ---- the GVRAM run and its control (47.6.2). A channel that writes GVRAM in
# buffer mode is the decoder-free packed player's entire per-frame path, and a
# run with no control is 58.3's vacuous "UNDERRUNS: 0/120" again -- the IPL
# leaves R20 = $0B16, bit 11 ALREADY SET, so the first cut of this test could
# not have failed.
grep -aq "BYTES OK: 2048 B from LBA 1000 .*\[gvram\]" tmp/dma_run.log || \
fail "the channel did not fill GVRAM in buffer mode -- a device->GVRAM
transfer is the whole of the decoder-free packed player's frame."
grep -aq "R20 during the run = \$0916 (bit 11 SET)" tmp/dma_run.log || \
fail "the GVRAM run did not run in buffer mode with a KNOWN R20."
grep -aq "R20 during the run = \$0116 (bit 11 CLEAR)" tmp/dma_run.log || \
fail "the negative control did not run with bit 11 clear."
if grep -aq "BYTES OK: 2048 B from LBA 1000 .*\[masked\]" tmp/dma_run.log
then
fail "the MASKED control delivered every byte, so the run above is not a
measurement of R20 bit 11 -- it is a measurement of nothing."
fi
grep -aq "EXACTLY THE MECHANISM" tmp/dma_run.log || \
fail "the masked control lost bytes at ODD offsets too, or lost none at all.
The claim is not a COUNT -- stale GVRAM matches the disc by coincidence
wherever the record is pad -- it is a PLACE: gvram_w's 256-colour arm
drops what the channel wrote to EVEN addresses and stores what it wrote
to odd ones. Damage anywhere else is a different mechanism."
grep -aq "BYTES OK: 2048 B from LBA 1000 .*\[chain\]" tmp/dma_run.log || \
fail "the array-chained run did not put the bytes at the row bases its array
named. A picture row is 256 B of a 1024 B line stride, so a frame is 192
destinations; if the channel cannot walk them the CPU has to restart it
per row and the decoder-free path costs a per-row front end."
grep -aq "THE CHANNEL WALKED THE ARRAY ITSELF" tmp/dma_run.log || \
fail "the chained run did not report walking its own array."
# ---- THE PALETTE (ROADMAP K1, FINDINGS 61.9). If the registers at $E82000 take
# a byte-wide DMA the way GVRAM does in buffer mode, a per-frame palette is a
# 193rd array-chain entry and ONE channel start paints a whole frame; if they do
# not, the CPU writes 256 words a frame and the architecture still stands. The
# run is poisoned first and controlled twice -- once by aiming the same transfer
# elsewhere, once by counting how many of the 512 positions the poison and the
# disc actually differ in.
grep -aq "BYTES OK: 512 B from LBA 1000 .*\[pal\]" tmp/dma_run.log || \
fail "the channel did not write the palette registers at \$E82000 -- so a
per-frame palette costs the CPU 256 word writes and cannot ride the
frame's array chain (61.9). That is a RESULT, not a broken run: check the
PALETTE WRONG line above for whether the bytes were dropped or misplaced."
DIFF=$(sed -n 's/.*PALETTE POISON IS A DISCRIMINATOR: \([0-9]*\) of 512.*/\1/p' \
tmp/dma_run.log)
[ -n "$DIFF" ] && [ "$DIFF" -ge 500 ] || \
fail "the poison and the disc's bytes agree in ${DIFF:-?} of 512 positions, so
the palette run could have passed without a channel writing anything --
this is run 4's could-not-fail trap in a new place. Change DGPOIS."
grep -aq "BYTES OK: 512 B from LBA 1000 .*\[palctl\]" tmp/dma_run.log || \
fail "the ATTRIBUTION control's read did not land in RAM, so its palette claim
is about a transfer that did not happen."
grep -aq "PALETTE UNTOUCHED BY THE CONTROL: 256 of 256 words" tmp/dma_run.log || \
fail "the palette changed during a transfer aimed 20 KB away from it. Then
what reached \$E82000 in the run above was not decided by the channel's
MAR, and that run measured something else."
if grep -aq "CONTROL DID NOT FAIL \[palctl\]" tmp/dma_run.log
then
fail "the control reported its own failure -- see the line above it."
fi
grep -aq "BYTES OK: 2048 B from LBA 1000 .*\[palchain\]" tmp/dma_run.log || \
fail "ONE array-chained start could not cross from the palette registers into
GVRAM. A frame is one palette entry and 192 row entries; if the two kinds
of destination cannot share a chain, the CPU is back in the video path
once a frame to start the second half of it."
grep -aq "ONE START PAINTED THE PALETTE AND 6 ROWS" tmp/dma_run.log || \
fail "the palette+rows run did not report the crossing it exists to show."
grep -aq "WINDOWED DMA READ REFUSED" tmp/dma_run.log || \
fail "a WINDOWED read through the channel was not refused. 117 of 120 records
start part way into a sector (58.3), and a channel cannot drop the bytes
in front of one -- so it would write the neighbouring records into the
ring, over data the decoder has not finished with, with no bounds check
to catch it (49.2)."
exit 0
tools/bench/gvpack/Makefile
+9
View File
@@ -0,0 +1,9 @@
PX68K ?= $(HOME)/src/px68k
CFLAGS = -O2 -fno-strict-aliasing -Wall -Wno-unused-result \
-I$(PX68K)/m68000 -I$(PX68K)/x11 -I$(PX68K)/win32api -I$(PX68K)/x68k
gvpack: harness.c $(PX68K)/x68k/gvram.c
$(CC) $(CFLAGS) -o $@ harness.c $(PX68K)/x68k/gvram.c
clean:
rm -f gvpack
tools/bench/gvpack/gvpack
Executable
BIN
View File
Binary file not shown.
tools/bench/gvpack/harness.c
+178
View File
@@ -0,0 +1,178 @@
/* Headless harness for px68k's GVRAM write and display model.
*
* Tests FINDINGS 46.6 -- the packed 1.0 byte/pixel layout -- on a SECOND
* emulator, the way tools/bench/c68k does for the CPU core. It links px68k's
* real x68k/gvram.c: the address decode, the CRTC R20 bit-11 buffer-mode write
* path, the page-byte selection, the scroll wrap and the index-0 transparency
* test are all px68k's own code, not a reimplementation.
*
* What IS glue here, and is declared as such: the ~12 lines of page-ordering
* from x11/windraw.c's 256-colour case (which page is drawn opaque and which
* transparent, as a function of the video controller's priority register).
* windraw.c is SDL-bound and cannot be linked headless, so that dispatch is
* mirrored. It is quoted verbatim in pick_order() so the mirroring is
* auditable.
*
* GrphPal is set to the IDENTITY, so what lands in Grp_LineBuf is the 8-bit
* palette INDEX rather than a host pixel. That keeps the harness out of
* px68k's host-format colour conversion, and it is faithful: px68k's
* transparency test is on the index (`if (v != 0x00)`), before the lookup.
*/
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include "common.h"
#include "gvram.h"
/* --- the globals gvram.c expects from the rest of px68k -------------------- */
BYTE CRTC_Regs[48];
WORD CRTC_FastClrMask;
DWORD GrphScrollX[4], GrphScrollY[4];
WORD GrphPal[256];
BYTE TextDirtyLine[1024];
DWORD TextDotX, TextDotY;
DWORD VLINE;
BYTE Pal_Regs[1024];
WORD Pal16[65536];
WORD Ibit, Pal_HalfMask, Pal_Ix2;
extern BYTE GVRAM[0x80000];
extern WORD Grp_LineBuf[1024];
#define W 256
#define H 256
/* 68000 word write: two byte writes, high byte first, as the bus does. */
static void wr16(DWORD adr, WORD v)
{
GVRAM_Write(adr, (BYTE)(v >> 8));
GVRAM_Write(adr + 1, (BYTE)(v & 0xff));
}
static void set_r20(WORD r20) /* CRTC R20 = byte pair 0x28/0x29 */
{
CRTC_Regs[0x28] = (BYTE)(r20 >> 8);
CRTC_Regs[0x29] = (BYTE)(r20 & 0xff);
}
/* Mirrors x11/windraw.c, 256-colour case:
*
* if ( (VCReg1[1]&3) <= ((VCReg1[1]>>4)&3) ) {
* ... Grp_DrawLine8(1, 1); opaq = 0;
* ... Grp_DrawLine8(0, opaq);
* } else {
* ... Grp_DrawLine8(0, 1); opaq = 0;
* ... Grp_DrawLine8(1, opaq);
* }
*
* i.e. the first page drawn is OPAQUE (the bottom) and the second is drawn
* with opaq=0 (the transparent top).
*/
static void draw_line(BYTE vcreg1_lo)
{
int bottom = ((vcreg1_lo & 3) <= ((vcreg1_lo >> 4) & 3)) ? 1 : 0;
Grp_DrawLine8(bottom, 1);
Grp_DrawLine8(bottom ^ 1, 0);
}
int main(int argc, char **argv)
{
const char *blob = argc > 1 ? argv[1] : "tmp/frame256p.bin";
const char *out = argc > 2 ? argv[2] : "tmp/gvpack_px68k.raw";
int packed = !(argc > 3 && !strcmp(argv[3], "--unpacked"));
BYTE vc1 = (BYTE)(argc > 4 ? strtol(argv[4], NULL, 0) : 0x02);
/* --nobuffer: run the packed layout WITHOUT CRTC R20 bit 11, to show the
* bit is load-bearing here and not decoration. */
int buffer = !(argc > 5 && !strcmp(argv[5], "--nobuffer"));
int scroll = !(argc > 5 && !strcmp(argv[5], "--noscroll"));
/* --keepbuffer: leave R20 bit 11 SET while drawing. MAME blanks the
* graphics layer in that state; does px68k? */
int keepbuf = (argc > 5 && !strcmp(argv[5], "--keepbuffer"));
FILE *f = fopen(blob, "rb");
if (!f) { perror(blob); return 2; }
static BYTE d[8 + 768 + 256 * 256];
size_t n = fread(d, 1, sizeof d, f);
fclose(f);
int iw = (d[4] << 8) | d[5], ih = (d[6] << 8) | d[7];
if (n < (size_t)(8 + 768 + iw * ih)) { fprintf(stderr, "short blob\n"); return 2; }
const BYTE *pix = d + 8 + 768;
/* 'DLXQ' -- the blob is PRE-INTERLEAVED: `pix` is already the bytes a DLXP1
* record carries, in GVRAM order. The ordinary 'DLXR' path computes the
* interleave here, which tests the LAYOUT; this path tests the CONTAINER,
* by writing its bytes verbatim and asking px68k's own gvram.c what they
* display as. The two agreeing is the claim ROADMAP K2 has to make: the
* encoder's byte order is the one 47.2 verified as a picture. */
int prepacked = (d[3] == 'Q');
if (prepacked && !packed) {
fprintf(stderr, "a pre-interleaved blob has no unpacked form\n");
return 2;
}
int yoff = (H - ih) / 2;
const BYTE BLACK = 255;
#define PIX(y, x) ((y) < yoff || (y) >= yoff + ih ? BLACK : pix[((y) - yoff) * iw + (x)])
memset(GVRAM, 0, sizeof GVRAM);
for (int i = 0; i < 256; i++) GrphPal[i] = (WORD)i; /* identity */
TextDotX = W; TextDotY = H;
/* 256x256, 256 colours -- the same R20 tools/bench/crtc_mode.lua applies */
const WORD R20_DISPLAY = 0x0110;
set_r20(R20_DISPLAY);
/* page 0 -> scroll sets 0,1; page 1 -> scroll sets 2,3 */
GrphScrollX[0] = GrphScrollX[1] = 0;
GrphScrollY[0] = GrphScrollY[1] = 0;
GrphScrollX[2] = GrphScrollX[3] = (packed && scroll) ? 384 : 0;
GrphScrollY[2] = GrphScrollY[3] = 0;
if (packed) {
if (buffer) set_r20(R20_DISPLAY | 0x0800); /* buffer mode: unmasked */
for (int y = 0; y < H; y++) {
DWORD base = 0xC00000 + y * 1024;
for (int i = 128; i < 512; i++) wr16(base + i * 2, 0);
for (int i = 0; i < 128; i++) {
WORD w;
if (prepacked) {
/* The letterbox rows are STATIC SETUP and are not in a
* record (dlxp.py), so they are supplied here, the way a
* player's scene setup supplies them: both halves BLACK. */
if (y < yoff || y >= yoff + ih)
w = (WORD)((BLACK << 8) | BLACK);
else {
const BYTE *row = pix + (y - yoff) * iw;
w = (WORD)((row[i * 2] << 8) | row[i * 2 + 1]);
}
} else {
w = (WORD)((PIX(y, i + 128) << 8) | PIX(y, i));
}
wr16(base + i * 2, w);
}
}
if (!keepbuf) set_r20(R20_DISPLAY); /* back to display */
} else {
/* the ordinary 2.0 B/pixel path, for a control */
for (int y = 0; y < H; y++) {
DWORD base = 0xC00000 + y * 1024;
for (int x = 0; x < W; x++) wr16(base + x * 2, PIX(y, x));
}
}
FILE *o = fopen(out, "wb");
if (!o) { perror(out); return 2; }
for (int y = 0; y < H; y++) {
VLINE = (DWORD)y;
memset(Grp_LineBuf, 0, sizeof Grp_LineBuf);
draw_line(vc1);
static BYTE row[W];
for (int x = 0; x < W; x++) row[x] = (BYTE)(Grp_LineBuf[x] & 0xff);
fwrite(row, 1, W, o);
}
fclose(o);
fprintf(stderr, "[GVPACK] px68k model: %s, vcreg1=%02X, bottom page=%d -> %s\n",
packed ? (buffer ? "PACKED 1.0 B/px" : "PACKED but bit11 OFF")
: "unpacked 2.0 B/px", vc1,
((vc1 & 3) <= ((vc1 >> 4) & 3)) ? 1 : 0, out);
return 0;
}
tools/bench/gvpack/verify_dlxp.py
+87
View File
@@ -0,0 +1,87 @@
#!/usr/bin/env python3
"""The PACKED CONTAINER's own bytes, through px68k's real GVRAM code.
python3 tools/bench/gvpack/verify_dlxp.py [packed.dlxp] [frame] [--controls]
`verify_gvpack.py` checks the LAYOUT: it hands the harness a picture and lets
the harness compute the interleave, so what it proves is that FINDINGS 47.2's
scheme renders. This checks the CONTAINER: it writes a DLXP1 record's bytes
into GVRAM VERBATIM -- no interleave computed anywhere in the harness -- and
asks px68k what they display as. That is the only way to test a format whose
whole design is that nothing parses it (dlxp.py): if the encoder's byte order
were wrong, every check upstream of the display would still pass, because the
container round-trips against its own inverse.
It is the same second-emulator argument tools/bench/c68k makes for cycles: the
address decode, the R20 bit-11 write path, the page-byte selection, the scroll
wrap and the index-0 transparency test are px68k's own `x68k/gvram.c`.
Two negative controls, because a test that cannot fail proves nothing, and both
are mechanisms this container depends on rather than decoration:
--nobuffer R20 bit 11 CLEAR -- the high byte of every word is masked away,
so page 1 (columns 128..255) never gets written
--noscroll page 1 unscrolled -- its storage sits under the wrong columns
"""
import os, struct, subprocess, sys
sys.path.insert(0, "tools/encoder")
import numpy as np
from dlxp import DLXP
args = [x for x in sys.argv[1:] if not x.startswith("--")]
path = args[0] if args else "tmp/packed_singe.dlxp"
frame = int(args[1]) if len(args) > 1 else 0
controls = "--controls" in sys.argv
d = DLXP(path)
_, pic = d._split(frame)
blob = b"DLXQ" + struct.pack(">HH", d.W, d.H) + b"\0" * 768 + pic
open("tmp/dlxp_gvpack.bin", "wb").write(blob)
BIN = "tools/bench/gvpack/gvpack"
if not os.path.exists(BIN):
sys.exit(f"{BIN} not built -- make -C tools/bench/gvpack PX68K=...")
def run(extra=None):
cmd = [BIN, "tmp/dlxp_gvpack.bin", "tmp/dlxp_gvpack.raw", "--packed", "0x02"]
if extra:
cmd.append(extra)
subprocess.run(cmd, check=True, stderr=subprocess.DEVNULL)
g = np.frombuffer(open("tmp/dlxp_gvpack.raw", "rb").read(), np.uint8)
return g.reshape(256, 256)
want = d.indices(frame)
yoff = (256 - d.H) // 2
g = run()
act = g[yoff:yoff + d.H]
fail = []
if not np.array_equal(act, want):
bad = act != want
fail.append(f"{bad.sum()} px differ (left half {bad[:, :128].sum()}, "
f"right half {bad[:, 128:].sum()})")
bars = np.concatenate([g[:yoff], g[yoff + d.H:]])
if bars.size and (bars != 255).any():
fail.append(f"letterbox is not index 255: {(bars != 255).sum()} px")
if (act == 0).any():
fail.append(f"index 0 reached the screen: {(act == 0).sum()} px")
if controls and not fail:
for flag, why in (("--nobuffer", "R20 bit 11 clear"),
("--noscroll", "page 1 unscrolled")):
c = run(flag)[yoff:yoff + d.H]
n = int((c != want).sum())
print(f" control {flag:<11s} ({why}): {n:,} px differ"
+ ("" if n else " <-- IT DID NOT FAIL"))
if not n:
fail.append(f"control {flag} passed -- the test cannot fail on it")
for x in fail:
print("FAIL " + x)
if fail:
sys.exit(1)
print(f"OK {os.path.basename(path)} frame {frame}: px68k's own gvram.c renders "
f"the container's {d.pic_bytes:,} bytes index-exact over {d.W}x{d.H},")
print(f" letterbox on the reserved black, and the transparency key never "
f"reaches the screen. The harness computed no interleave.")
tools/bench/gvpack/verify_gvpack.py
+53
View File
@@ -0,0 +1,53 @@
#!/usr/bin/env python3
"""Check px68k's render of the packed layout against the same reference MAME is
judged on (tools/bench/verify_frame256.py, criterion 3 and 4).
python3 tools/bench/gvpack/verify_gvpack.py <raw> [blob]
<raw> is 256x256 palette INDICES straight out of px68k's Grp_DrawLine8.
"""
import struct, sys
import numpy as np
raw = sys.argv[1] if len(sys.argv) > 1 else "tmp/gvpack_px68k.raw"
blob = sys.argv[2] if len(sys.argv) > 2 else "tmp/frame256p.bin"
g = np.frombuffer(open(raw, "rb").read(), np.uint8).reshape(256, 256)
d = open(blob, "rb").read()
W, H = struct.unpack(">HH", d[4:8])
pal = np.frombuffer(d[8:8+768], np.uint8).reshape(256, 3).astype(int)
idx = np.frombuffer(d[8+768:8+768+W*H], np.uint8).reshape(H, W)
yoff = (256 - H) // 2
act = g[yoff:yoff+H]
fail = []
if not np.array_equal(act, idx):
bad = act != idx
fail.append(f"active area index-exact: {bad.sum()} px differ "
f"(left half {bad[:, :128].sum()}, right half {bad[:, 128:].sum()})")
bars = np.concatenate([g[:yoff], g[yoff+H:]])
if bars.size and (bars != 255).any():
fail.append(f"letterbox not the reserved black index 255: "
f"{(bars != 255).sum()} px")
if (act == 0).any():
fail.append(f"index 0 appeared in the picture: {(act == 0).sum()} px "
f"-- it is the transparency key and must stay unused")
for x in fail:
print("FAIL " + x)
if fail:
sys.exit(1)
p6 = lambda v: ((v << 2) | (v >> 4)) & 0xFF
f = pal >> 3
render = lambda I: p6((f << 1) | I[:, None])
I = (((render(np.ones(256, int)) - pal) ** 2).sum(1)
< ((render(np.zeros(256, int)) - pal) ** 2).sum(1)).astype(int)
mse = ((render(I)[act].astype(int) - pal[idx]) ** 2).mean()
print(f"OK {raw}: px68k renders the packed layout index-exact over {W}x{H}, "
f"letterbox on the reserved black")
print(f" palette ceiling vs 24-bit palettised source: "
f"{10*np.log10(255**2/mse):.2f} dB")
tools/bench/load.lua
+178
View File
@@ -0,0 +1,178 @@
-- Time and verify src/player/load.i on the emulated 68000 (ROADMAP P1+P2).
--
-- Two questions, one run, exactly as decode.lua asks them of the decoder:
-- 1. CORRECTNESS. Does the 68000 produce, out of the RAW container header,
-- byte for byte what tools/bench/dlxload.py produces host-side? The
-- expanded codebooks are read back out of RAM and the palette out of the
-- PALETTE REGISTERS -- not out of a RAM shadow, because "the words reached
-- $E82000" is the claim being tested. tools/bench/verify_load.py does the
-- comparison against dlxload.py, so the ground truth stays in one place.
-- 2. COST. How long does it take, split into the codebook expansion and the
-- palette pack, and what is that as a fraction of a 12 fps frame -- the
-- only unit this project prices anything in.
--
-- Nothing here is pre-chewed: the blob pushed into RAM is the first 5,920 bytes
-- of the container as they come off the disc. That is the whole point of the
-- exercise, and it is also, not incidentally, exactly the read a player has to
-- complete at a scene change before it can draw a single frame.
--
-- MEASUREMENT SCOPE, unchanged from decode.lua: MAME's memory carries no wait
-- states, so these are pure 68000 instruction cycles -- a LOWER BOUND on real
-- hardware. Interrupts are masked (SR=$2700). The host clock has 1/56.69 s
-- granularity and the job takes milliseconds, so each configuration is repeated
-- LITER times and divided; repeating is honest because do_load is not
-- temporally recursive -- every pass rewrites what the last one wrote, from the
-- same source bytes.
M = manager.machine
SP = M.devices[":maincpu"].spaces["program"]
local function findfile(n)
for _,p in ipairs{"../tools/bench/"..n, "tools/bench/"..n, n} do
local f = io.open(p,"rb"); if f then f:close(); return p end
end
error(n.." not found")
end
local MODE = loadfile(findfile("crtc_mode.lua"))()
local META = loadfile("load_meta.lua")()
local LFLAG, LHDR, LDARK = 0x18040, 0x18044, 0x18048
local LK1, LK4, LMODE, LITER = 0x1804C, 0x18050, 0x18054, 0x18058
local CB1, CB4, RAW = 0x20000, 0x22000, 0x30000
local GPAL = 0xE82000
local CPUHZ = 10000000 -- x68k.cpp:1133, 40_MHz_XTAL/4
local FPS = 12
local FRAME12 = CPUHZ / FPS
local ITER = tonumber(os.getenv("DLX_LOAD_ITER") or "40")
local code do local f=io.open("loadgate.bin","rb"); code=f:read("a"); f:close() end
local data do local f=io.open("load_data.bin","rb"); data=f:read("a"); f:close() end
local function T() local t=M.time; return t.seconds + t.attoseconds/1e18 end
local function P(s) print("[LOD] "..s) end
local function push(addr, s, from, len)
local i, n = from, len
while n >= 4 do
SP:write_u32(addr, (string.unpack(">I4", s, i)))
addr, i, n = addr+4, i+4, n-4
end
while n > 0 do
SP:write_u8(addr, string.byte(s,i)); addr, i, n = addr+1, i+1, n-1
end
end
-- Poison every destination before each run. Without this a stage that wrote
-- NOTHING would still compare equal to the previous stage's output, and the
-- palette-only run would "pass" the codebook check for free.
--
-- The three scratch tables are poisoned only before a run that CLAIMS to build
-- them (mode bit 2). They are scene-independent, so the palette-entry stage is
-- entitled to find them already there -- that is the whole point of measuring
-- it separately -- but a stage that says it builds them must be shown to.
local P6TAB, TABEND = 0x19000, 0x19340
local function poison(mode)
for a = CB1, CB1 + META.cb1_len - 2, 2 do SP:write_u16(a, 0xDEAD) end
for a = CB4, CB4 + META.cb4_len - 2, 2 do SP:write_u16(a, 0xDEAD) end
for c = 0, 255 do SP:write_u16(GPAL + c*2, 0xDEAD) end
SP:write_u32(LDARK, 0xFFFFFFFF)
if mode & 4 ~= 0 then
for a = P6TAB, TABEND - 2, 2 do SP:write_u16(a, 0xDEAD) end
end
end
local function setup()
MODE.apply(SP)
push(RAW, data, 1, META.raw_len)
for i = 1, #code do SP:write_u8(0x10000+i-1, string.byte(code,i)) end
P(string.format("loaded loadgate.bin=%d B, raw container header %d B at 0x%X",
#code, META.raw_len, RAW))
end
local function launch(mode, iter)
poison(mode)
SP:write_u32(LFLAG, 0)
SP:write_u32(LHDR, RAW)
SP:write_u32(LMODE, mode)
SP:write_u32(LITER, iter)
local cpu = M.devices[":maincpu"]
cpu.state["SR"].value = 0x2700 -- supervisor, ALL interrupts masked
cpu.state["SP"].value = 0x8000
cpu.state["PC"].value = 0x10000
end
-- Written after the mode-3 run, and only after it: it is the output of ONE
-- do_load call over the whole header, which is what the player does.
local function dump()
local out = io.open("load_out.bin", "wb")
for a = CB1, CB1 + META.cb1_len - 1 do out:write(string.char(SP:read_u8(a))) end
for a = CB4, CB4 + META.cb4_len - 1 do out:write(string.char(SP:read_u8(a))) end
for c = 0, 255 do out:write(string.pack(">I2", SP:read_u16(GPAL + c*2) & 0xFFFF)) end
out:close()
P(string.format("dumped %d B of 68000 output to tmp/load_out.bin",
META.cb1_len + META.cb4_len + 512))
P(string.format("DARK=%d (host-side dlxload.py says %d), K1=%d K4=%d",
SP:read_u32(LDARK), META.dark, SP:read_u32(LK1), SP:read_u32(LK4)))
end
-- Order matters: the scratch tables are built by the first stage and the
-- palette-entry stage runs on them, which is exactly how a player would be
-- arranged. The two stages that stand for real player events -- boot, and a
-- scene change -- come last, and the dump the verifier checks is taken from the
-- BOOT one, so the path that is proved correct is the one that builds
-- everything from nothing.
local PLAN = {
{name="scratch tables only (boot, once)", mode=4, iter=ITER},
{name="codebook expansion only (P1)", mode=1, iter=ITER},
{name="palette entries only (P2)", mode=2, iter=ITER},
{name="BOOT: tables + codebooks + palette", mode=7, iter=ITER, dump=true},
{name="SCENE CHANGE: codebooks + palette", mode=3, iter=ITER},
}
local step, st, t0 = 0, "boot", nil
local results = {}
SUB = emu.add_machine_frame_notifier(function()
local ok, err = pcall(function()
local t = T()
if st == "boot" then
if t < 3.0 then return end
setup(); step = 1; launch(PLAN[1].mode, PLAN[1].iter)
st, t0 = "running", nil; return
end
if st == "running" then
local fl = SP:read_u32(LFLAG)
if fl == 1 and not t0 then t0 = t; return end
if fl == 0xEE then
P("BAD HEADER -- load.i did not find the 'DLX3' magic at LHDR")
M:exit(); return
end
if fl == 0xFF then
local p = PLAN[step]
local dt = t - (t0 or t)
local cyc = dt * CPUHZ / p.iter
results[#results+1] = {name=p.name, cyc=cyc}
P(string.format("%s: %d passes in %.4f s -> %.0f cycles = %.1f%% of a "
.."%dfps frame (%.2f ms)", p.name, p.iter, dt, cyc,
100*cyc/FRAME12, FPS, 1000*cyc/CPUHZ))
if p.dump then dump() end
step = step + 1
if PLAN[step] then launch(PLAN[step].mode, PLAN[step].iter); st, t0 = "running", nil
else st = "finish" end
return
end
if t > 400 then P("TIMEOUT flag="..string.format("%08X",fl)); M:exit() end
return
end
if st == "finish" then
P("---- summary (instruction cycles only; real RAM adds wait states) ----")
for _,r in ipairs(results) do
P(string.format(" %-44s %8.0f cyc %5.1f%% of a frame %6.2f ms",
r.name, r.cyc, 100*r.cyc/FRAME12, 1000*r.cyc/CPUHZ))
end
P("done")
M:exit()
end
end)
if not ok then print("[LOD] LUA ERROR: "..tostring(err)); M:exit() end
end)
tools/bench/load_run.sh
Executable
+54
View File
@@ -0,0 +1,54 @@
#!/bin/bash
# One load-time transform run: the 68000 builds its own codebooks and palette
# out of the RAW container header, on both CPU cores (ROADMAP P1+P2, FINDINGS
# 53).
#
# tools/bench/load_run.sh [container]
#
# Both instruments run the same loadgate.bin over the same header bytes:
# * MAME, which is the only one of the two with real PALETTE REGISTERS -- the
# packed words are read back out of $E82000, not out of a RAM shadow, so
# "the words reached the hardware" is part of what passes.
# * px68k's C68K, which is exact to the cycle and counts BUS cycles, and is a
# second opinion on the cost from a separately written cycle table.
# Both outputs are compared byte-for-byte against tools/bench/dlxload.py, which
# stays the reference: this code replaces where those transforms RUN, not what
# they produce.
set -e
cd "$(dirname "$0")/../.."
DLX=${1:-tmp/rc_fr_singe_scsi_span.dlx}
PX68K=${PX68K:-$HOME/src/px68k}
ITER=${DLX_LOAD_ITER:-40}
tools/vasm/vasmm68k_mot -Fbin -o tmp/loadgate.bin src/player/loadgate.s > /dev/null
python3 tools/bench/prep_load.py "$DLX" > tmp/prep_load.log
cat tmp/prep_load.log
# stdbuf -oL: without it a long MAME run is unobservable until it exits, and a
# run that is merely finishing looks exactly like one that is wedged (34.1).
( cd tmp && DLX_LOAD_ITER=$ITER SDL_VIDEODRIVER=dummy stdbuf -oL timeout -k 5 180 \
mame x68000 -bios ipl10 -ramsize 2M -video soft -window -sound none \
-nothrottle -plugins -autoboot_script ../tools/bench/load.lua \
-seconds_to_run 30 > load_check.log 2>&1 )
# A run that never reached the dump must fail as that, not as a byte mismatch.
grep -q "^\[LOD\] done" tmp/load_check.log || {
echo "FAIL: the load rig did not finish -- no completion marker."
tail -6 tmp/load_check.log; exit 1; }
grep -a "^\[LOD\]" tmp/load_check.log | sed -n '/summary/,$p' | sed 's/\[LOD\] / /'
python3 tools/bench/verify_load.py "$DLX"
if [ -f "$PX68K/m68000/c68k.c" ]; then
make -s -C tools/bench/c68k PX68K="$PX68K" 2>/dev/null
for M in 4 1 2 7 3; do
tools/bench/c68k/c68k_bench --code tmp/loadgate.bin --loadraw tmp/load_data.bin \
--loadmode $M --loaditer 1 --cb1 8192 --cb4 2048 \
$([ $M = 3 ] && echo "--loaddump tmp/load_c68k.bin") 2>&1 >/dev/null \
| grep -av arena | sed 's/\[C68K\] / /'
done
# The second core's bytes are held to the same standard as the first's.
cmp -s tmp/load_c68k.bin tmp/load_out.bin || {
echo "FAIL: the two CPU cores produced DIFFERENT load-time output."; exit 1; }
echo " OK both CPU cores produced the same $(stat -c%s tmp/load_out.bin) B"
else
echo " SKIPPED: no px68k at $PX68K (set PX68K= to point at a checkout)"
fi
tools/bench/mkvol.sh
Executable
+76
View File
@@ -0,0 +1,76 @@
#!/bin/bash
# Build the SCSI VOLUME the P4 rigs read, and the blank card ROM MAME needs to
# instantiate the card. Sourced-by-calling from tools/bench/scsi_run.sh and
# tools/bench/pace_run.sh so there is ONE copy of the layout.
#
# tools/bench/mkvol.sh [container.dlx | container.dlxp]
#
# TWO CONTAINERS, ONE VOLUME BUILDER. A DLX volume is tools/bench/prep_stream.py's
# disk image -- the codec's records, laid down from sector 0. A DLXP volume
# (ROADMAP K2/K3) needs no preparation at all: the container is ALREADY a
# sector-aligned image of itself -- a 512 B header, then fixed 97-sector records
# -- so the file IS the volume and copying it is the whole build. That is not a
# convenience, it is the format's central claim (tools/encoder/dlxp.py) arriving
# at the disc, and a builder that transformed it on the way would be hiding the
# claim rather than testing it.
#
# The two get DIFFERENT CHD NAMES. Alternating between the packed rig and the
# codec rig would otherwise rebuild the volume on every run, and -- much worse --
# a stale CHD under the name the other rig expected would serve one container's
# bytes to the other's gate, which reads as a decode failure and is not one.
#
# ONE COPY, ON PURPOSE. The volume is tmp/stream_disk.bin -- byte for byte the
# file the host-file ring rig reads -- laid out as 512 B sectors. If two scripts
# each built it, a difference between the SCSI rig and the modelled-transport rig
# could be a difference in what they were reading, and the whole value of running
# both is that it cannot be. This tree has already paid twice for a transform
# with two copies of itself (FINDINGS 49.7.5, and check.sh's dlxload note).
#
# THE BLANK BOOT ROM is the substitution session 25 argued for and it is
# unchanged: MAME refuses to instantiate the CZ-6BS1 without an 8 KB
# `scsiexrom.bin` (CRC 7be488de) that the player never executes, so a zero-filled
# placeholder goes on a SEPARATE rompath and the user's romset is untouched.
# MAME prints WRONG CHECKSUMS, as it should. DO NOT reuse this rompath for
# anything that boots from the card or calls SCSI IOCS -- those DO execute it.
set -e
cd "$(dirname "$0")/../.."
DLX=${1:-tmp/rc_fr_singe_scsi_span.dlx}
case "$DLX" in
*.dlxp) SRC="$DLX"
IMG=tmp/dlxpdisk.img; CHD=tmp/dlxpdisk.chd ;;
*) SRC=tmp/stream_disk.bin
IMG=tmp/dlxdisk.img; CHD=tmp/dlxdisk.chd
[ -f "$SRC" ] || python3 tools/bench/prep_stream.py "$DLX" > /dev/null ;;
esac
if [ ! -f "$CHD" ] || [ "$SRC" -nt "$CHD" ]; then
SRC="$SRC" IMG="$IMG" python3 - <<'PY'
import os
src, img = os.environ["SRC"], os.environ["IMG"]
d = open(src, "rb").read()
n = (len(d) + 511) // 512
open(img, "wb").write(d + b"\0" * (n * 512 - len(d)))
print(f" disc image: {len(d)} B of records -> {n} sectors")
PY
rm -f "$CHD"
# -c none IS LOAD-BEARING, and it was found by a gate rather than by taste.
# Session 28, on the DLX5 volume: with the default (lzma/zlib/huff/flac) MAME
# 0.277 served the CHD FILE'S OWN BYTES as sector data -- the destination
# buffer after READ(10) at LBA 0 was byte-for-byte the first 4,096 bytes of
# dlxdisk.chd, starting "MComprHD" -- while `chdman verify` reported both SHA1s
# correct. Uncompressed, the identical image reads byte-exact. The trigger is
# the image's CONTENT: the same 8,768-sector length that works for the DLX4
# volume fails for the DLX5 one, a conventional 16x63 geometry fails too, and
# `-c zlib` alone fails as well. The MAME-side cause is NOT diagnosed; what is
# measured is that compression decides it and uncompressed is sound.
# Costs 4.5 MB in tmp/ against 1.6 MB. DO NOT restore compression to save the
# disc space: the failure is SILENT at the transport layer -- every READ(10)
# reports success and returns the wrong bytes -- and only the byte comparison
# in tools/bench/scsi.lua catches it.
chdman createhd -i "$IMG" -o "$CHD" -ss 512 -c none > /dev/null 2>&1
fi
mkdir -p tmp/p4roms/x68k_cz6bs1
[ -f tmp/p4roms/x68k_cz6bs1/scsiexrom.bin ] || \
head -c 8192 /dev/zero > tmp/p4roms/x68k_cz6bs1/scsiexrom.bin
tools/bench/pace_run.sh
Executable
+120
View File
@@ -0,0 +1,120 @@
#!/bin/bash
# One paced ring-buffer run (STATUS item 4, FINDINGS 49.7.2).
#
# tools/bench/check.sh runs the ring pass FREE-RUNNING, which is right for what
# it gates -- wrap correctness at a fixed ring size, delivery removed as a
# variable by an unlimited pipe. It cannot answer the buffering question,
# because a free-running decoder never lets the ring back up.
#
# This runs the same rig with the decoder held to the container's frame rate,
# so the ring fills and FR_HEAD-FR_TAIL means "frames the decoder could still
# draw with the pipe dead". Every run is verified PIXEL-EXACT: a paced decode
# that drops a pixel is not a slack measurement, it is a bug.
#
# tools/bench/pace_run.sh <ring_kb> <kbps> [cut_at_tick] [cut_frames]
#
# kbps 0 = unlimited pipe. There is no default rate anywhere in this tree
# (FINDINGS 50) and there is none here either.
#
# DLX_RINGOWN=1 hands the RING to the 68000 as well (ROADMAP P5,
# src/player/ring.i): this script's Lua stops placing records and becomes a
# transport that answers one request at a time. DLX_PREFILL_FR is then the
# prefill policy, in whole records. It needs a DLX4 container, because the
# machine cannot learn a record's length by walking a stream it has not fetched.
#
# DLX_ITER=2 runs the scene TWICE, which under DLX_RINGOWN means a real seek
# between the passes: the channel goes quiet, the ring is declared empty and the
# whole accumulated lookahead is thrown away and rebuilt from the prefill. It
# needs DLX_PACE=2, because rebasing the frame clock across a pass is the
# machine's to do and a host-written tick would carry on counting.
#
# DLX_XFER=scsi replaces the MODELLED transport with a real one (ROADMAP P4b,
# src/player/xfer.i): the machine gets a CZ-6BS1 and the same volume the SCSI
# gate reads, this script stops moving bytes altogether, and every record is
# fetched by the 68000 with READ(10). It needs DLX_RINGOWN=1 -- the mailbox it
# answers is ring.i's -- and it FORBIDS a modelled rate, because there is no
# longer anything for one to model.
#
# DLX_PACE selects WHO KEEPS THE TIME: 1 (default) is the host writing the tick,
# 2 is the 68000 writing it off the CRTC's V-DISP (ROADMAP P3, FINDINGS 54).
# Everything else about the run is identical, which is the whole point -- the
# pace gate in src/player/stream.s cannot tell them apart, so a difference in
# the result is a difference in the CLOCK and not in the rig.
set -e
cd "$(dirname "$0")/../.."
RING=${1:?ring KB}; KBPS=${2:?pipe KB/s, or 0 for unlimited}
CUT_AT=$3; CUT_FR=${4:-1}
DLX=${DLX:-tmp/rc_fr_singe_scsi_span.dlx}
PACE=${DLX_PACE:-1}
OWN=${DLX_RINGOWN:-0}
ITERS=${DLX_ITER:-1}
XFER=${DLX_XFER:-model}
# EMULATED seconds the run is allowed. A pass that is cut short compares a
# half-drawn screen and reads as a wrap bug, so this is raised deliberately
# rather than left to a timeout: a DLX_XFER=scsi pass costs ~46 s of emulated
# time against the modelled transport's ~7, because the CPU moves every byte
# itself (FINDINGS 58.2), and two of them do not fit in 90.
SECS=${DLX_SECONDS:-90}
if [ "$XFER" = scsi ]; then
[ "$OWN" = 1 ] || { echo "DLX_XFER=scsi needs DLX_RINGOWN=1: the transport in"
echo "src/player/xfer.i answers src/player/ring.i's mailbox, and with the"
echo "host owning the ring there is no mailbox to answer."; exit 2; }
# A rate is not merely ignored here, it is REFUSED. The bytes now arrive on
# the emulated machine's own time, and a run labelled "488 KB/s" that did not
# deliver at 488 KB/s is exactly the kind of number this project has twice
# paid for. There is no rate in a DLX_XFER=scsi run, and the log says so.
[ "$KBPS" = 0 ] || { echo "DLX_XFER=scsi takes kbps 0. The transport is real,"
echo "so nothing here delivers at a modelled rate -- and MAME's device"
echo "models are functional, not transfer-timing accurate, so the rate it"
echo "DOES deliver at is not a measurement either (docs/BENCHMARK.md)."
exit 2; }
command -v chdman > /dev/null || { echo "DLX_XFER=scsi needs chdman (ships"
echo "with mame-tools) to build the volume."; exit 2; }
bash tools/bench/mkvol.sh "$DLX"
fi
if [ "$OWN" = 1 ] && [ "$ITERS" != 1 ] && [ "$PACE" != 2 ]; then
echo "DLX_ITER>1 needs DLX_PACE=2: the frame clock is rebased per pass by"
echo "src/player/stream.s, and a host-written tick would go on counting"
echo "through the seek and open every slot of the second pass at once."
exit 2
fi
TAG="r${RING}_k${KBPS}${CUT_AT:+_cut${CUT_AT}x${CUT_FR}}"
# The default tag is left ALONE when the host keeps the time: tools/bench/
# pace_sweep.sh reads tmp/pace_r<ring>_k<kbps>.log by name, and renaming the
# host-paced logs would break a sweep that has nothing to do with this option.
if [ "$PACE" != 1 ]; then TAG="${TAG}_p$PACE"; fi
if [ "$OWN" = 1 ]; then TAG="${TAG}_own"; fi
if [ "$ITERS" != 1 ]; then TAG="${TAG}_x$ITERS"; fi
if [ "$XFER" != model ]; then TAG="${TAG}_$XFER"; fi
MAMEX=()
if [ "$XFER" = scsi ]; then
MAMEX=(-exp1 cz6bs1 -rompath "$HOME/mame/roms;./p4roms" -hard dlxdisk.chd)
fi
tools/vasm/vasmm68k_mot -Fbin -o tmp/stream.bin src/player/stream.s > /dev/null
[ -f tmp/stream_disk.bin ] || python3 tools/bench/prep_stream.py "$DLX" > tmp/prep_stream.log
mkdir -p "tmp/snap_pace_$TAG"; rm -f "tmp/snap_pace_$TAG/x68000"/*.png
# `env` rather than an assignment prefix: an empty ${CUT_AT:+...} in the middle
# of a prefix is not an assignment token, so bash takes the next word as the
# command and the run dies with "SDL_VIDEODRIVER=dummy: command not found".
CUTENV=(); [ -n "$CUT_AT" ] && CUTENV=(DLX_CUT_AT="$CUT_AT" DLX_CUT_FR="$CUT_FR")
( cd tmp && env DLX_PACE=$PACE DLX_RING_KB=$RING DLX_STREAM_KBPS=$KBPS \
DLX_RINGOWN=$OWN DLX_ITER=$ITERS DLX_XFER=$XFER \
${DLX_PREFILL_FR:+DLX_PREFILL_FR=$DLX_PREFILL_FR} \
"${CUTENV[@]}" DLX_SLACK_CSV="slack_$TAG.csv" \
SDL_VIDEODRIVER=dummy stdbuf -oL timeout -k 5 900 \
mame x68000 -bios ipl10 "${MAMEX[@]}" -ramsize 2M -video soft -window \
-sound none \
-nothrottle -plugins -autoboot_script ../tools/bench/stream.lua \
-snapshot_directory "./snap_pace_$TAG" -snapview native \
-seconds_to_run $SECS \
> "pace_$TAG.log" 2>&1 )
# The completion marker is not optional: a run killed mid-decode compares a
# half-drawn screen and reads as a wrap bug rather than as a truncated run.
grep -q "snapshot taken" "tmp/pace_$TAG.log" || {
echo "FAIL($TAG): no snapshot marker -- the pass did not complete."
tail -6 "tmp/pace_$TAG.log"; exit 1; }
echo "=== $TAG"
grep -aE "decoder (SELF-PACED|PACED|FREE)|FRAME CLOCK|ring: |UNDERRUNS|NO IDLE|SEEK SLACK|RING-BOUND|RATE-BOUND|BUILD TIME|PIPE CUT|DEADLINE|REQUIRED|MACHINE-OWNED|PREFILL:|CHANNEL IDLE|MISPLACED|SEEK PASS|REAL TRANSPORT|SECTOR OVERHEAD|TRANSPORT FAILED|IS VACUOUS" \
"tmp/pace_$TAG.log" | sed "s/\[STR\] / /"
python3 tools/bench/verify_decode.py "$DLX" --snap "tmp/snap_pace_$TAG" | tail -2
tools/bench/pace_sweep.sh
Executable
+36
View File
@@ -0,0 +1,36 @@
#!/bin/bash
# Ring x pipe grid for the PACED rig (STATUS item 4, FINDINGS 51).
#
# tools/bench/pace_sweep.sh "<ring KB list>" "<KB/s list>"
#
# Every cell is a full 120-frame decode on the emulated 68000, pixel-verified.
# There is no default rate list: FINDINGS 50 removed the delivery constant from
# this tree and a sweep that invented one back would be the same mistake with
# more rows. `0` means an unlimited pipe, which measures the RING's ceiling with
# delivery removed as a variable -- an upper bound, not a prediction.
set -e
cd "$(dirname "$0")/../.."
RINGS=${1:?ring KB list, quoted}
RATES=${2:?pipe KB/s list, quoted, 0 = unlimited}
printf "%6s %9s %9s %9s %8s %9s %s\n" ring kbps ceiling build_s mean underruns bound
for R in $RINGS; do for K in $RATES; do
L=tmp/pace_r${R}_k${K}.log
bash tools/bench/pace_run.sh "$R" "$K" > /dev/null 2>&1 || { \
printf "%6s %9s FAILED (see %s)\n" "$R" "$K" "$L"; continue; }
python3 - "$L" "$R" "$K" <<'PY'
import re, sys
log, ring, kbps = sys.argv[1], sys.argv[2], sys.argv[3]
t = open(log, errors="replace").read()
def g(p, d="?"):
m = re.search(p, t)
return m.group(1) if m else d
ceil_ = g(r"SEEK SLACK: ceiling (\d+) frames")
build = g(r"BUILD TIME: (\d+) ticks")
mean = g(r"mean ([\d.]+) over the window")
under = g(r"UNDERRUNS: (\d+)/")
bound = "ring" if "RING-BOUND" in t else ("rate" if "RATE-BOUND" in t else "?")
fps = 12.0
print("%6s %9s %9s %9.2f %8s %9s %s" % (
ring, kbps, ceil_, (int(build)/fps if build.isdigit() else -1), mean, under, bound))
PY
done; done
tools/bench/packed.lua
+509
View File
@@ -0,0 +1,509 @@
-- Drive src/player/packed.s: THE DECODER-FREE PACKED PLAYER, END TO END,
-- OFF A REAL VOLUME. ROADMAP K3.
--
-- WHAT THIS SCRIPT DOES NOT DO IS THE POINT OF IT. tools/bench/stream.lua
-- pushes expanded codebooks and a packed palette into RAM, plays a transport at
-- a modelled byte rate, and writes the frame tick. This one pushes 2,898 bytes
-- of 68000 code and eleven mailbox words, and then READS. It moves no picture
-- byte, models no rate, sets no CRTC register and writes no palette entry: the
-- machine brings up its own display, builds its own chain, keeps its own clock
-- off V-DISP and fetches every record itself with READ(10) off a CZ-6BS1.
--
-- SO THE GATE IS NOT THE ONE THE CODEC USES, AND IT HAD TO CHANGE.
-- tools/bench/verify_decode.py checks ONE frame -- the last -- and that audits
-- the whole run because the codec is temporally recursive: a SKIP block is a
-- claim about the previous frame still being on screen, so the final frame is
-- only right if all 120 were. A packed frame is a LITERAL. Frame 119 being
-- pixel-exact says nothing whatever about frame 60. This script therefore
-- snapshots EVERY frame and tools/bench/verify_packed.py compares all of them;
-- the simplification that deleted the ring also deleted the gate's free lunch.
--
-- WHEN A SNAPSHOT IS TAKEN, and why not on the frame it changed. PG_SHOWN is
-- bumped by the 68000 after it clears R20 bit 11, so a change means "a complete
-- frame is now displayable". But MAME's screen bitmap for the host frame in
-- progress was drawn partly before that instant, so snapshotting immediately
-- would sample the write window -- which BLANKS the graphics layer -- for part
-- of the picture. A 12 fps frame lasts 4 or 5 host refreshes at 56.69 Hz, so
-- waiting SNAP_DELAY whole host frames is safely inside the slot and safely
-- after the window closed.
--
-- Env:
-- DLX_PK_HELD 1 = the channel HOLDS THE BUS (burst, max rate), 0 = it
-- steals cycles. Not two speeds of one thing: 59.3 showed an
-- auto-requested channel is charged by TIME, so held is the
-- 68000 stopped for as long as the record takes to arrive.
-- Default 1.
-- DLX_PK_PACE 1 = the machine holds itself to the container's fps off
-- V-DISP (default). 0 free-runs, which tests the CHAIN with
-- the clock out of the way.
-- DLX_PK_ITER passes over the scene (default 1). >1 exercises the SEEK.
-- DLX_PK_SEEK the frame passes after the first start at (default 0, i.e.
-- a replay). On the video path the seek IS arithmetic and
-- nothing else; on the audio path it is a second read at a
-- separate LBA, because a DLXP2 group puts lump k in FRONT of
-- its records and a branch lands `f mod F` frames into it
-- (FINDINGS 70.3). Setting this to a frame that is NOT a
-- multiple of the cadence is the point: 36 of the arcade's
-- 409 within-container seek targets land on a boundary and
-- 373 do not.
-- DLX_PK_NFR play only the first N frames (default: all of them)
-- DLX_PK_FPS pace at this rate instead of the container's. NOT a
-- cosmetic knob and not a way to make a number look better:
-- under MAME the emulated transport takes about a whole 12 fps
-- slot to deliver a 49,664 B record, and the write window has
-- to be OPEN for all of it -- so at 12 fps there is no instant
-- at which a complete frame is displayable and nothing can be
-- snapshotted. Pacing slower opens a display interval without
-- changing one byte of the transfer, which is what lets the
-- PIXEL-EXACTNESS of all 120 frames be gated separately from
-- the RATE the emulated transport happens to run at. The two
-- are different questions and this is the knob that separates
-- them.
-- DLX_PK_CSV write the per-frame arrival series here
M = manager.machine
SP = M.devices[":maincpu"].spaces["program"]
local function findfile(n)
for _,p in ipairs{"../tools/bench/"..n, "tools/bench/"..n, n} do
local f = io.open(p,"rb"); if f then f:close(); return p end
end
error(n.." not found")
end
local META = loadfile("packed_meta.lua")()
-- src/player/packed.s. Inputs first, then outputs; the split is the file's.
local PG_FLAG, PG_NFR, PG_FPS, PG_LBA0 = 0x18900, 0x18904, 0x18908, 0x1890C
local PG_RECS, PG_PALL, PG_HELD = 0x18910, 0x18914, 0x18918
local PG_PACEON, PG_ITER = 0x1891C, 0x18920
local PG_CADF, PG_CADA, PG_AUDON = 0x18924, 0x18928, 0x1892C
local PG_SHOWN, PG_ERR, PG_ERRAT = 0x18930, 0x18934, 0x18938
local PG_LATE, PG_LATE1, PG_LATEM = 0x1893C, 0x18940, 0x18944
local PG_VDISP, PG_VD0, PG_TSPIN = 0x18948, 0x1894C, 0x18950
local PG_GSPIN, PG_LOSTV, PG_ARRN = 0x18954, 0x18958, 0x1895C
-- ROADMAP P6c: the audio path's inputs and its own account of what it did.
local PG_AFPS, PG_AHZ, PG_ALBA0 = 0x18960, 0x18964, 0x18968
local PG_NLUMP, PG_ABYTES, PG_APRE = 0x1896C, 0x18970, 0x18974
local PG_SEEKF, PG_ARST = 0x18978, 0x1897C
local PG_APOS, PG_ASKIP = 0x189C8, 0x189CC
local PG_ASKN, PG_ASKB, PG_TSEQ = 0x189D0, 0x189D4, 0x189DC
local PG_AARM, PG_AFET, PG_ABYT = 0x18980, 0x18984, 0x18988
local PG_ADRY, PG_ASEAM, PG_ASRV = 0x1898C, 0x18990, 0x18994
local PG_ACSR, PG_ACER, PG_ALATE = 0x18998, 0x1899C, 0x189A0
local PG_AMTC0, PG_AK, PG_AKF, PG_AACC = 0x189A4, 0x189B0, 0x189B4, 0x189B8
local PG_AFERR, PG_AFERA = 0x189A8, 0x189AC
local PG_ARR = 0x1B000
-- src/player/clock.i and src/player/scsi.i, read for diagnosis only.
local CLK_PACE, CLK_VDISP, CLK_ERR = 0x18034, 0x18064, 0x1806C
local SC_ERR = 0x18200
local CRTC_R20 = 0xE80028
local HELD = (os.getenv("DLX_PK_HELD") or "1") == "1"
local PACED = (os.getenv("DLX_PK_PACE") or "1") == "1"
local ITERS = tonumber(os.getenv("DLX_PK_ITER") or "") or 1
local SEEKF = tonumber(os.getenv("DLX_PK_SEEK") or "") or 0
-- ROADMAP P6d. 1 = STOP and re-PLAY the chip at a branch. Not a tidiness
-- knob: the chip's accumulator has no leak, so the two settings are a large
-- decaying error against a small permanent one (FINDINGS 71.3).
local ARST = (os.getenv("DLX_PK_ARST") or "0") == "1"
local NFR = tonumber(os.getenv("DLX_PK_NFR") or "") or META.nframes
local FPS = tonumber(os.getenv("DLX_PK_FPS") or "") or META.fps
local CSV = os.getenv("DLX_PK_CSV")
-- ROADMAP P6c. OFF by default and it is not a convenience: a run with the chip
-- silent is the CONTROL this one is read against, and every gate that existed
-- before session 36 is that control. DLX_PK_APRE is the lumps fetched before
-- frame 0 -- see src/player/packed.s on why the answer is not 1.
local AUDIO = (os.getenv("DLX_PK_AUD") or "0") == "1"
local APRE = tonumber(os.getenv("DLX_PK_APRE") or "") or 2
local AJSON = os.getenv("DLX_PK_AJSON")
local SNAP_DELAY = 2
local SCERRNAME = {[0]="OK", "SELECTION TIMEOUT -- no target answered",
"UNEXPECTED PHASE", "POLL TIMEOUT -- a phase never arrived",
"NON-ZERO SCSI STATUS",
"WINDOWED READ REFUSED -- a channel cannot drop bytes"}
local code do local f=assert(io.open("packed.bin","rb")); code=f:read("a"); f:close() end
local function T() local t=M.time; return t.seconds + t.attoseconds/1e18 end
local function P(s) print("[PK] "..s) end
local function setup()
for i = 1, #code do SP:write_u8(0x10000+i-1, string.byte(code,i)) end
SP:write_u32(PG_FLAG, 0)
SP:write_u32(PG_NFR, NFR)
SP:write_u32(PG_FPS, FPS)
SP:write_u32(PG_LBA0, META.lba0)
SP:write_u32(PG_RECS, META.rec_sectors)
SP:write_u32(PG_PALL, META.palette_last)
SP:write_u32(PG_HELD, HELD and 1 or 0)
SP:write_u32(PG_PACEON, PACED and 1 or 0)
SP:write_u32(PG_ITER, ITERS)
SP:write_u32(PG_SEEKF, SEEKF)
SP:write_u32(PG_ARST, ARST and 1 or 0)
-- DLXP2's cadence. Zero for a silent container, and the 68000 branches on the
-- zero: a player told the wrong cadence does not fail, it reads an audio lump
-- as a record and paints it.
SP:write_u32(PG_CADF, META.cad_f or 0)
SP:write_u32(PG_CADA, META.cad_a or 0)
SP:write_u32(PG_SHOWN, 0)
-- P6c. AUDON is separate from the cadence on purpose: the LBA arithmetic has
-- to skip the lumps whether or not a chip is being fed, and a run that skips
-- them without playing them is the control this one is measured against.
SP:write_u32(PG_AUDON, (AUDIO and (META.has_audio or 0) == 1) and 1 or 0)
SP:write_u32(PG_AFPS, META.fps) -- the CONTAINER's, NOT the pace
SP:write_u32(PG_AHZ, META.aud_hz or 0)
SP:write_u32(PG_ALBA0, META.lba_aud or 0)
SP:write_u32(PG_NLUMP, META.n_lumps or 0)
SP:write_u32(PG_ABYTES, META.aud_bytes or 0)
SP:write_u32(PG_APRE, APRE)
if ITERS > 1 then
local grp = (META.cad_f or 0) > 0 and (SEEKF % META.cad_f) or 0
P(string.format("SEEK: %d passes, and passes 2..%d start at FRAME %d%s",
ITERS, ITERS, SEEKF,
(META.cad_f or 0) == 0 and " (silent container)"
or string.format(" -- lump %d, %d frame(s) into its group "
.."of %d, so the audio needs a second read and a byte "
.."offset (FINDINGS 70.3)%s",
SEEKF // META.cad_f, grp, META.cad_f,
grp == 0 and " -- ON a group boundary, the free case"
or "")))
P(ARST and " the chip is STOPPED and re-PLAYED at the branch: its "
.."accumulator goes to the container's own init and its step "
.."index to 0"
or " the chip PLAYS THROUGH the branch: it keeps the predictor "
.."state the previous scene's audio left it in")
end
P(string.format("packed.bin=%d B, %dx%d %d fps, %d of %d frames, %d passes",
#code, META.W, META.H, META.fps, NFR, META.nframes, ITERS))
local cad = ""
if (META.cad_f or 0) > 0 then
cad = string.format(" + (i//%d)*%d", META.cad_f, META.cad_a)
end
P(string.format("record %d B = %d sectors at LBA %d + i*%d%s, palette %s",
META.rec_bytes, META.rec_sectors, META.lba0,
META.rec_sectors, cad,
META.palette_last == 1 and "LAST" or "FIRST"))
if (META.cad_f or 0) > 0 then
P(string.format("DLXP2: %d B of ADPCM at %d Hz rides in %d lumps of %d sectors, "
.."one in front of every %d records -- the third term above is the "
.."whole cost of it on the video path",
META.aud_bytes, META.aud_hz, META.n_lumps, META.cad_a, META.cad_f))
end
if AUDIO and (META.has_audio or 0) == 1 then
P(string.format("AUDIO ON: lump k at LBA %d + k*%d, payload 11*%d/24 B a "
.."group -- the PAYLOAD and not the %d B lump (FINDINGS "
.."67.2). Decoder from the header: %s/%s, %d-bit clamp, "
.."accumulator %d at PLAY. Prefill %d lumps of %d.",
META.lba_aud, META.cad_f*META.rec_sectors + META.cad_a,
META.aud_hz, META.cad_a*512, META.aud_variant,
META.aud_order, META.aud_bits, META.aud_init,
APRE, META.n_lumps))
elseif (META.has_audio or 0) == 1 then
P("audio present in the container and NOT played -- this is the silent "
.."control (DLX_PK_AUD=1 plays it)")
end
P(string.format("channel: %s, %s",
HELD and "BUS HELD (burst, max rate)" or "CYCLE STEALING",
PACED and ("SELF-PACED at "..FPS.." fps off V-DISP"
..(FPS ~= META.fps and (" -- NOT the container's "
..META.fps..", see DLX_PK_FPS") or ""))
or "FREE-RUNNING (tests the chain, not the clock)"))
P("this script writes NO picture byte, NO palette entry and NO CRTC register: "
.."the machine brings up its own display and fetches its own records.")
end
local function launch()
local cpu = M.devices[":maincpu"]
cpu.state["SR"].value = 0x2700 -- supervisor, all interrupts masked;
cpu.state["SP"].value = 0x8000 -- clk_init lowers it to $2500 itself
cpu.state["PC"].value = 0x10000
end
local st, t0 = "boot", nil
local shown, pending, snaps = 0, nil, 0
local arrive, hostfr, missed = {}, 0, 0
-- WHICH FRAME EACH SNAPSHOT IS. MAME numbers snapshots 0000, 0001, ... in the
-- order they were taken, and a frame that could not be sampled leaves no gap in
-- that sequence -- so the file name is NOT the frame index and a verifier that
-- assumed it was would compare frame 61 against record 60 and report a codec
-- bug that is really a bookkeeping one. This is the map, written out for it.
local snapfr = {}
local r20seen = {}
SUB = emu.add_machine_frame_notifier(function()
local ok, err = pcall(function()
local t = T()
if st == "boot" then
if t < 3.0 then return end
setup(); launch(); t0 = t; st = "running"; return
end
if st ~= "running" then return end
hostfr = hostfr + 1
-- WHAT THE SCREEN MODE WAS, sampled every host frame. R20 bit 11 blanks
-- the graphics layer, so this is the only way to see the shutter the player
-- is running: the fraction of host frames that found the window OPEN is the
-- fraction of the scene the display spent dark, and it is a MEASUREMENT of
-- 47.4's cost under MAME rather than a restatement of the prior.
local r20 = SP:read_u16(CRTC_R20)
r20seen[#r20seen+1] = ((r20 >> 11) & 1)
local s = SP:read_u32(PG_SHOWN)
if s > shown then
-- Only the LAST change matters if several landed in one host frame; that
-- cannot happen at 12 fps on a 56.69 Hz raster, and if it ever does the
-- gate below catches it as a missing snapshot rather than a wrong one.
arrive[#arrive+1] = {n = s, t = t - t0}
shown = s
pending = SNAP_DELAY
end
if pending then
pending = pending - 1
if pending <= 0 then
pending = nil
if (SP:read_u16(CRTC_R20) >> 11) & 1 == 0 then
M.video:snapshot(); snaps = snaps + 1
snapfr[#snapfr+1] = shown - 1
else
-- The window was open again when the delay expired: the frame we
-- meant to sample is being overwritten. COUNTED, NOT PRINTED -- when
-- the transfer is longer than the slot EVERY frame misses, and 119
-- identical lines bury the four numbers the run exists to report.
-- The count is reported once at the end and the gate reads it there.
missed = missed + 1
end
end
end
local flag = SP:read_u32(PG_FLAG)
if flag ~= 1 and pending then
-- THE LAST FRAME IS STILL PENDING. packed.s spins in pg_hold with the
-- window CLOSED once the scene is over, so the delay can simply run out;
-- returning here rather than reporting is what stops the final frame
-- being the one frame the gate never sees.
return
end
if flag ~= 1 then
st = "done"
local wall = t - t0
P(string.format("FLAG=$%02X after %.3f s, %d frames shown, %d snapshots, "
.."%d frames NOT SAMPLED (the write window had reopened "
.."-- the transfer is longer than the display interval)",
flag, wall, shown, snaps, missed))
local err = SP:read_u32(PG_ERR)
if err ~= 0 then
P(string.format("TRANSPORT FAILED on frame %d: %s",
SP:read_u32(PG_ERRAT), SCERRNAME[err] or ("code "..err)))
end
P(string.format("array: the 68000 built %d entries (the container wants "
.."%d)", SP:read_u32(PG_ARRN), META.entries))
-- The first entry, read back out of the machine's own RAM. It is the one
-- place palette-first and palette-last are visible as a FACT rather than
-- as a flag the rig passed in and the rig read back.
P(string.format("chain[0] = MAR $%06X MTC %d ; chain[1] = MAR $%06X MTC %d",
SP:read_u32(PG_ARR), SP:read_u16(PG_ARR+4),
SP:read_u32(PG_ARR+6), SP:read_u16(PG_ARR+10)))
if PACED then
local vd = SP:read_u32(PG_VDISP) - SP:read_u32(PG_VD0)
-- THE CLOCK, AGAINST THE RASTER THAT DROVE IT. CLK_VDISP counts the
-- edges the 68000's ISR SAW. hostfr counts the frames MAME actually
-- drew. A held channel halts the CPU, and the MFP's pending bit is one
-- bit, so an edge that falls inside a transfer long enough to span two
-- of them is an edge the machine can never count. Nothing in this
-- project has ever run a transfer and a clock at once, so nothing could
-- have seen this before.
P(string.format("frame clock: PACE=%d ticks, V-DISP edges SEEN=%d, "
.."host frames drawn=%d -> %d edges LOST (%.1f%%)",
SP:read_u32(CLK_PACE), vd, hostfr, hostfr - vd,
hostfr > 0 and (hostfr-vd)*100/hostfr or 0))
local nlate = SP:read_u32(PG_LATE)
P(string.format("late frames (tick already past at the gate): %d%s",
nlate, nlate > 0 and string.format(", first %d, worst "
.."%d ticks", SP:read_u32(PG_LATE1),
SP:read_u32(PG_LATEM)) or ""))
-- AND WHY `late = 0` IS NOT `on time`. The gate compares the frame
-- index against PACE, and PACE is advanced by the ISR that the held
-- channel stops the CPU from running. A clock that loses edges loses
-- them from BOTH sides of the comparison, so a player whose own clock
-- has halved still reports every frame early. The LOST figure above is
-- the only thing in this run that can contradict it, and it comes from
-- the host's raster count rather than from the machine.
if hostfr - vd > 0 then
P(string.format(" ...and %d of those ticks were never "
.."issued, so `late=%d` is measured against a clock "
.."running at %.1f%% of the raster. The player "
.."believes it is at %d fps and the screen is at "
.."%.2f.", hostfr - vd, nlate, vd*100/hostfr, FPS,
FPS * vd / hostfr))
end
end
P(string.format("CPU: %d trips round the TRANSFER wait in total, %d round "
.."the PACE gate", SP:read_u32(PG_TSPIN),
SP:read_u32(PG_GSPIN)))
if AUDIO and (META.has_audio or 0) == 1 then
local armed, fet = SP:read_u32(PG_AARM), SP:read_u32(PG_AFET)
local byt, dry = SP:read_u32(PG_ABYT), SP:read_u32(PG_ADRY)
local seam, srv = SP:read_u32(PG_ASEAM), SP:read_u32(PG_ASRV)
local late, acc = SP:read_u32(PG_ALATE), SP:read_u32(PG_AACC)
P(string.format("AUDIO: %d of %d lumps armed, %d fetched, %d B of "
.."payload handed to the chip (the stream is %d B)",
armed, META.n_lumps, fet, byt, META.aud_bytes))
-- THE PADDING, CHARGED. A player that fed the chip the whole lump
-- would have handed it n_lumps*A*512 B; the difference is the drift
-- FINDINGS 67.2 priced at 1.25 s over the game, and printing both
-- numbers is the only way the accumulator is visible from outside.
-- FULL groups only. The last lump is short when the scene's frame
-- count is not a multiple of F, and averaging that in reports the
-- scene's TAIL as though it were the cadence -- which is a different
-- number from the drift and looks like a worse one. The percentage
-- lives in verify_packed_audio.py, which knows each lump's payload.
local nfull = META.n_lumps
if NFR % META.cad_f ~= 0 then nfull = nfull - 1 end
local lumpb = nfull * META.cad_a * 512
P(string.format(" %d whole groups: %d B of lump space for the "
.."payload the accumulator asked for. The whole-lump "
.."player feeds the chip that space, and the excess is "
.."DRIFT and not waste (67.2). Accumulator left at "
.."%d/%d.", nfull, lumpb, acc, 2*META.fps))
P(string.format(" service: %d calls, %d found the channel "
.."counted out, %d of those had NO lump ready (a "
.."STARVE -- the chip replays its last byte)",
srv, seam, dry))
P(string.format(" re-arms with MTC still non-zero: %d (bytes "
.."fetched and never played; 0 is the correct value)",
late))
-- THE SEEK'S OWN ACCOUNT. PG_ASKN is the second reads and PG_ASKB the
-- bytes they skipped at the head of a lump: a seek path with no offset
-- term would report the first and zero for the second, and would be
-- indistinguishable from a correct one on any target that happened to
-- land on a group boundary.
local skn, skb = SP:read_u32(PG_ASKN), SP:read_u32(PG_ASKB)
if skn > 0 then
P(string.format(" SEEK: %d audio seek(s), %d B skipped into "
.."the head of a lump. Stream position ended at %d B "
.."and the chip was handed %d -- they differ BY the "
.."skip, which is the whole reason they are two cells "
.."(FINDINGS 71).", skn, skb, SP:read_u32(PG_APOS),
byt))
end
local ferr = SP:read_u32(PG_AFERR)
if ferr ~= 0 then
P(string.format(" A LUMP FETCH FAILED on lump %d: %s -- the "
.."picture is unaffected and the sound is gone, which "
.."is why this has its own error word",
SP:read_u32(PG_AFERA), SCERRNAME[ferr] or ("code "..ferr)))
end
P(string.format(" channel 3 at the end: CSR=$%02X CER=$%02X, "
.."MTC one instruction after the first START = %d",
SP:read_u32(PG_ACSR), SP:read_u32(PG_ACER),
SP:read_u32(PG_AMTC0)))
-- WHAT HOLDING THE BUS COSTS A SECOND CONSUMER, and it is this line.
-- Stealing, pg_aserv runs from inside dma.i's transfer wait as well as
-- twice a frame; held, the 68000 is HALTED for the whole transfer and
-- the two frame-loop calls are all it gets. The ratio is the audio's
-- half of FINDINGS 64.3.
P(string.format(" -> %.1f service calls per frame shown. %s",
shown > 0 and srv/shown or 0,
HELD and ("BUS HELD: the 68000 is halted for the whole "
.."transfer, so DM_HOOK never runs and this is the "
.."two frame-loop calls and nothing else.")
or ("CYCLE STEALING: DM_HOOK ran from inside the "
.."transfer wait, which is where a 68000 driving this "
.."video path has any time at all.")))
if AJSON then
local f = io.open(AJSON, "w")
f:write(string.format('{"armed":%d,"fetched":%d,"bytes":%d,'
..'"starve":%d,"seam":%d,"serv":%d,"late":%d,"acc":%d,'
..'"csr":%d,"cer":%d,"held":%s,"fps":%d,"shown":%d,'
..'"seekn":%d,"seekb":%d,"pos":%d,"seekf":%d,"iters":%d,'
..'"arst":%s}\n',
armed, fet, byt, dry, seam, srv, late, acc,
SP:read_u32(PG_ACSR), SP:read_u32(PG_ACER),
HELD and "true" or "false", FPS, shown,
SP:read_u32(PG_ASKN), SP:read_u32(PG_ASKB),
SP:read_u32(PG_APOS), SEEKF, ITERS,
ARST and "true" or "false"))
f:close()
P("audio counters -> "..AJSON)
end
end
local open = 0
for _,v in ipairs(r20seen) do open = open + v end
P(string.format("WRITE WINDOW OPEN on %d of %d host frames (%.1f%%) -- "
.."buffer mode blanks the graphics layer, so that is the "
.."share of the scene the display spent DARK under MAME",
open, #r20seen, #r20seen > 0 and open*100/#r20seen or 0))
if #arrive >= 2 then
local dts, first, last = {}, arrive[1].t, arrive[#arrive].t
for i = 2, #arrive do dts[#dts+1] = arrive[i].t - arrive[i-1].t end
table.sort(dts)
-- CADENCE, AND THE GRANULARITY IT IS MEASURED AT. PG_SHOWN is sampled
-- once per host frame, so a single inter-frame figure is quantised to
-- 1/56.69 s = 17.6 ms and the min/median/max below are multiples of it.
-- The MEAN over the whole run is not: the quantisation error is bounded
-- by one host frame at each END, so over n-1 intervals it is 35 ms/(n-1)
-- -- 0.30 ms a frame over 120. Read the mean; the spread is the
-- sampler's, not the player's.
local mean = (last-first)/(#arrive-1)
P(string.format("cadence: %d frames in %.3f s = %.3f fps, mean "
.."%.2f ms/frame (+/- %.2f ms, the sampler's); "
.."inter-frame min %.1f median %.1f max %.1f ms",
#arrive, last-first, (#arrive-1)/(last-first),
mean*1000, 35.3/(#arrive-1),
dts[1]*1000, dts[math.ceil(#dts/2)]*1000,
dts[#dts]*1000))
-- THE TRANSPORT'S TIME IS ONLY READABLE OFF A FREE-RUNNING RUN.
-- Paced, the mean inter-frame IS THE PACE PERIOD: the player waits for
-- its tick and the record's delivery hides inside the slot, so dividing
-- the record by that mean reports the clock rather than the transport
-- and reports it as a rate. The first cut of this script printed
-- "297.4 KB/s" off a 6 fps gate run, which is the pace and not the
-- disc. Free-running there is no gate and the loop is transfer-bound,
-- so the mean is the transfer.
if not PACED then
P(string.format(" -> FREE-RUNNING, so the mean IS the "
.."transport: a %d B record lands in %.2f ms, i.e. "
.."%.1f KB/s and %.1f%% of a %d fps slot. MAME's "
.."device models carry no transfer timing "
.."(docs/BENCHMARK.md, 42.5), so this is a property "
.."of the APPARATUS -- it is not W and it is not a "
.."measurement of any medium.",
META.rec_bytes, mean*1000,
META.rec_bytes/mean/1024,
mean*META.fps*100, META.fps))
else
-- What a paced run CAN say about the transfer, and it says it from
-- the display rather than from the clock: the window is open for
-- exactly as long as the record takes, so the open fraction times the
-- slot is the transfer time, sampled at the host's frame rate.
P(string.format(" -> PACED, so this mean is the PACE and "
.."NOT the transport. What the run does bound is the "
.."transfer: the window was open %.1f%% of a "
.."%.2f ms slot = %.1f ms a record.",
open*100/#r20seen, mean*1000,
open/#r20seen*mean*1000))
end
end
if CSV then
local f = io.open(CSV, "w")
f:write("frame,t_s\n")
for _,a in ipairs(arrive) do f:write(string.format("%d,%.6f\n", a.n-1, a.t)) end
f:close()
P("arrivals -> "..CSV)
end
do
local f = assert(io.open("packed_snaps.csv", "w"))
f:write("snapshot,frame\n")
for i, fr in ipairs(snapfr) do
f:write(string.format("%04d,%d\n", i-1, fr))
end
f:close()
P(string.format("%d snapshots -> tmp/packed_snaps.csv", #snapfr))
end
P("done")
M:exit()
end
end)
if not ok then print("[PK] LUA ERROR: "..tostring(err)); M:exit() end
end)
tools/bench/packed_run.sh
Executable
+290
View File
@@ -0,0 +1,290 @@
#!/bin/bash
# THE PACKED PLAYER, END TO END, OFF A REAL VOLUME. ROADMAP K3.
#
# tools/bench/packed_run.sh [container.dlxp]
#
# Four runs of src/player/packed.s, and each answers a different question. They
# are separate runs because the questions interfere: the write window has to be
# OPEN for the whole transfer and buffer mode blanks the graphics layer, so at
# the container's own 12 fps there is no instant at which a complete frame is
# displayable and the pixel gate has nothing to sample. Pacing slower opens a
# display interval without changing one byte of the transfer. Reporting the
# rate off the gate run instead would have been the flattering shortcut, and it
# reports the PACE rather than the disc -- the first cut of packed.lua did
# exactly that and printed 297 KB/s off a 6 fps run.
#
# 1. GATE, stealing, paced at half rate: 120 records, 120 snapshots, every one
# compared. A packed frame is a LITERAL, so unlike the codec's gate the last
# frame audits nothing and all 120 have to be checked (verify_packed.py).
# 2. RATE, stealing, FREE-RUNNING: the loop is transfer-bound, so the mean
# inter-frame IS the emulated transport's time for a record.
# 3. RATE, held, FREE-RUNNING: the same, with the bus held.
# 4. CLOCK, held, paced at the container's fps: what holding the bus does to a
# frame clock built on counting V-DISP interrupts.
#
# DLX_PK_GATE_ONLY=1 runs 1 alone. That is what tools/bench/check.sh takes: the
# green light's job is to catch a regression in the PLAYER, and runs 2-4 measure
# the apparatus rather than gate it -- three more MAME jobs for numbers that
# cannot change unless MAME does.
#
# THE APPARATUS is tools/bench/dma_run.sh's -- `x68000 -exp1 cz6bs1` and a
# zero-filled scsiexrom.bin on a private rompath -- and the volume is
# tools/bench/mkvol.sh's, which for a DLXP container is the container itself.
#
# WHAT NO RUN HERE MEASURES: `W`, and any rate a real medium would deliver.
# MAME's device models carry no transfer timing (docs/BENCHMARK.md, 42.5). What
# is measured is the SHAPE -- one channel start, 193 destinations, 120 times,
# on a clock the machine keeps itself, with every frame pixel-exact.
set -e
cd "$(dirname "$0")/../.."
DLXP=${1:-tmp/packed_singe.dlxp}
NFR=${DLX_PK_NFR:-120}
GATE_FPS=${DLX_PK_GATE_FPS:-6}
bash tools/bench/mkvol.sh "$DLXP"
python3 tools/bench/prep_packed.py "$DLXP"
# WHICH ORDER THIS CONTAINER USES, read out of the container rather than
# assumed. FINDINGS 62.5/63.4 priced palette-first and palette-last at -12.8 dB
# for one paint apiece and could not choose between them, so the format records
# it (dlxp.py flags bit 1) and BOTH have to pass this gate. The chain assertion
# below is the only place the difference is visible from outside the machine,
# and hard-coding either order there would turn "K3 ran both" into "K3 ran one
# and the other could not have failed".
PALLAST=$(sed -n 's/.*palette_last = \([01]\),.*/\1/p' tmp/packed_meta.lua)
if [ "$PALLAST" = "1" ]; then
CHAIN0='chain\[0\] = MAR \$C08000 MTC 256'
ORDER="palette LAST -- the 193rd entry"
else
CHAIN0='chain\[0\] = MAR \$E82000 MTC 512 ; chain\[1\] = MAR \$C08000 MTC 256'
ORDER="palette FIRST -- entry 0, then 192 rows"
fi
echo " container order: $ORDER"
tools/vasm/vasmm68k_mot -Fbin -o tmp/packed.bin src/player/packed.s > /dev/null
# One run. $1 names the log, the rest are environment.
run() {
local tag=$1; shift
rm -rf "tmp/snap_packed_$tag"; mkdir -p "tmp/snap_packed_$tag"
# stdbuf -oL: without it a long MAME run is unobservable until it exits, and a
# run that is merely finishing looks exactly like one that is wedged (34.1).
( cd tmp && env SDL_VIDEODRIVER=dummy "$@" stdbuf -oL timeout -k 5 900 \
mame x68000 -bios ipl10 -exp1 cz6bs1 \
-rompath "$HOME/mame/roms;./p4roms" -hard dlxpdisk.chd \
-ramsize 2M -video soft -window $SOUNDARGS -nothrottle -plugins \
-autoboot_script ../tools/bench/packed.lua \
-snapshot_directory "./snap_packed_$tag" -snapview native \
-seconds_to_run "$SECS" > "packed_$tag.log" 2>&1 )
grep -aq "^\[PK\] done" "tmp/packed_$tag.log" || {
echo "FAIL: the $tag run did not finish -- no completion marker."
tail -12 "tmp/packed_$tag.log"; exit 1; }
grep -a "^\[PK\]" "tmp/packed_$tag.log" | sed 's/^\[PK\] / /'
}
fail() { echo "FAIL: $1"; exit 1; }
# -sound none for every run that is not about sound, which is all of them until
# run 5. 15,625 is not a preference there: it is the chip's own stream rate
# (8 MHz / 512), and equal rates are what keep MAME's resampler from filtering
# the thing being measured (FINDINGS 66, adpcm_run.sh).
SOUNDARGS="-sound none"
echo "--- 1. THE GATE: $NFR records, paced at $GATE_FPS fps, channel stealing ---"
SECS=$(( NFR / GATE_FPS + 25 ))
run gate DLX_PK_HELD=0 DLX_PK_PACE=1 DLX_PK_FPS=$GATE_FPS DLX_PK_NFR=$NFR
cp tmp/packed_snaps.csv tmp/packed_snaps_gate.csv
# THE ASSERTIONS. Printing a result and gating on it are different things.
grep -aq "^\[PK\] FLAG=\$FF" tmp/packed_gate.log || \
fail "the player did not reach the end of the scene. FLAG=\$E1 is a CRTC mode
the frame clock cannot divide, \$E2 is a transport failure -- and the
TRANSPORT FAILED line above names which."
grep -aq "array: the 68000 built 193 entries (the container wants 193)" \
tmp/packed_gate.log || \
fail "the 68000 built a chain of the wrong length. One entry short delivers a
picture with its last row missing, which looks like a decode bug and is a
layout bug; the container's geometry and the player's arithmetic are two
independent statements of one number and they have to agree."
grep -aq "$CHAIN0" tmp/packed_gate.log || \
fail "the chain does not have the shape this container asks for ($ORDER).
The crossing from the palette registers into GVRAM IS the packed frame
(FINDINGS 62) -- a palette entry and 192 row entries, one start, the CPU
halted throughout -- and an array built the other way round from the
record feeding it does not fail: it paints 192 rows of picture into the
palette registers and 512 B of palette across the top of the screen."
grep -aq "late frames (tick already past at the gate): 0$" tmp/packed_gate.log || \
fail "a frame missed its slot in the GATE run, which is paced at half rate on
purpose. That is not a rate result -- it means the transfer did not fit in
a slot twice as long as the container's, and the pixel comparison below is
then sampling frames the player was still overwriting."
grep -aq "0 frames NOT SAMPLED" tmp/packed_gate.log || \
fail "the gate run could not sample every frame: the write window reopened
before the snapshot on at least one. At half the container's rate the
transfer must fit inside the display interval with room to spare, and if
it does not the comparison below is checking frames the player was still
overwriting."
python3 tools/bench/verify_packed.py "$DLXP" --snap tmp/snap_packed_gate \
--map tmp/packed_snaps_gate.csv --min-frames "$NFR"
# ---------------------------------------------------------------------------
# 5. THE AUDIO. ROADMAP P6c: the container's own bytes, out of channel 3,
# beside the video channel. Paced at the CONTAINER's rate rather than the
# gate's half rate, because the audio was cut at 12 fps and a 6 fps run would
# starve the chip for half of every group -- the picture can be slowed down and
# a crystal cannot. Nothing is snapshotted; the instrument is the WAV, and
# tools/bench/verify_packed_audio.py accounts for every byte of the stream in
# it. The tag is `aud` and not `audio` because tmp/packed_audio.log is
# tools/analysis/34_packed_audio.py's, in check.sh.
#
# CYCLE STEALING, and that is a result rather than a setting -- run 6 below is
# the same run with the bus held and it is a CONTRAST, not a gate.
AUDIO_ON=$(sed -n 's/.*has_audio = \([01]\),.*/\1/p' tmp/packed_meta.lua)
CFPS=$(sed -n 's/^ fps = \([0-9]*\),.*/\1/p' tmp/packed_meta.lua)
if [ "$AUDIO_ON" = "1" ]; then
echo
echo "--- 5. THE AUDIO: the container's own lumps, out of channel 3, while"
echo " the video channel is on the same bus (ROADMAP P6c) ---"
SECS=$(( NFR / 8 + 30 ))
SOUNDARGS="-samplerate 15625 -wavwrite packed_aud.wav"
run aud DLX_PK_HELD=0 DLX_PK_PACE=1 DLX_PK_FPS="$CFPS" DLX_PK_NFR=$NFR \
DLX_PK_AUD=1 DLX_PK_AJSON=packed_aud.json
SOUNDARGS="-sound none"
grep -aq "^\[PK\] FLAG=\$FF" tmp/packed_aud.log || \
fail "the audio run did not reach the end of the scene."
python3 tools/bench/verify_packed_audio.py "$DLXP" tmp/packed_aud.wav \
tmp/packed_aud.json || \
fail "the chip did not play the container. The counters above can all be
right while this fails -- nothing parses a packed container, so a lump
fetched into a buffer that is still being read is not an error, it is a
sound (FINDINGS 67.4, and it is the bug session 36 shipped and caught)."
fi
# ---------------------------------------------------------------------------
# 7. THE SEEK, WITH SOUND ON IT. FINDINGS 70.3 asked for this and named what
# was missing: "src/player/packed.s starts PG_AK/PG_AKF at lump 0 and has no
# audio seek path at all". This is that path, run.
#
# THE FRAME IS CHOSEN NOT TO BE A MULTIPLE OF THE CADENCE, and that is the whole
# design of the run. A DLXP2 group is `lump k, then F records`, so a branch
# that lands on a group boundary needs no offset and no second read -- and 36 of
# the arcade's 409 within-container seek targets do land on one. A run that
# picked one of those would exercise the arithmetic that was already there and
# report success. DLX_PK_SEEK defaults below to a frame `f mod F != 0`, so the
# byte offset inside the lump is load-bearing: get it wrong and the chip is fed
# a stream that starts up to F frames early, which is not an error, it is a
# rate, and only verify_packed_audio.py's spliced walk can see it.
#
# AND THE SECOND RESULT IS ONE NO COUNTER CAN REACH: the chip's predictor does
# not seek. Every byte can arrive, in order, exactly -- and the samples still
# be wrong, because the encoder chose them for a state a continuous play would
# have been in. The verifier measures that against its own control.
if [ "$AUDIO_ON" = "1" ] && [ "${DLX_PK_NOSEEK:-0}" != "1" ]; then
SEEKF=${DLX_PK_SEEK:-37}
CADF=$(sed -n 's/^ cad_f = \([0-9]*\),.*/\1/p' tmp/packed_meta.lua)
[ $((SEEKF % CADF)) -ne 0 ] || \
fail "the seek frame $SEEKF is a multiple of the cadence $CADF, so it lands
ON a group boundary -- the one case that needs no byte offset and would
pass with the offset arithmetic deleted (FINDINGS 70.3)."
echo
echo "--- 7. THE SEEK: two passes, the second starting at frame $SEEKF --"
echo " lump $((SEEKF / CADF)), $((SEEKF % CADF)) frame(s) into its group"
echo " of $CADF (FINDINGS 70.3/71) ---"
TOTFR=$(( NFR + NFR - SEEKF ))
SECS=$(( TOTFR / 8 + 35 ))
# BOTH CONFIGURATIONS, and they are not two speeds of one thing. The chip's
# accumulator is an integrator with no leak, so what a branch costs is set by
# the state it lands in: play THROUGH and the chip keeps whatever the previous
# scene left it in; STOP and re-PLAY and it goes to the container's own `init`
# with the step index at 0. Neither is zero and they are 5.5x apart, so the
# run measures both and FINDINGS 71.3 chooses.
for M in 0 1; do
TAG=seek; [ "$M" = 1 ] && TAG=seek_rst
echo " -- the chip $([ "$M" = 1 ] && echo 'STOPPED and re-PLAYED' \
|| echo 'PLAYING THROUGH') the branch"
SOUNDARGS="-samplerate 15625 -wavwrite packed_$TAG.wav"
run $TAG DLX_PK_HELD=0 DLX_PK_PACE=1 DLX_PK_FPS="$CFPS" DLX_PK_NFR=$NFR \
DLX_PK_AUD=1 DLX_PK_AJSON=packed_$TAG.json \
DLX_PK_ITER=2 DLX_PK_SEEK=$SEEKF DLX_PK_ARST=$M
SOUNDARGS="-sound none"
grep -aq "^\[PK\] FLAG=\$FF" "tmp/packed_$TAG.log" || \
fail "the $TAG run did not reach the end of the second pass."
python3 tools/bench/verify_packed_audio.py "$DLXP" "tmp/packed_$TAG.wav" \
"tmp/packed_$TAG.json" --seek "$SEEKF" --iters 2 || \
fail "the player did not play the container ACROSS A BRANCH ($TAG).
Nothing here parses anything (FINDINGS 67.4): a seek that fetched the
wrong lump plays 7,168 B of the wrong part of the scene, and one that
dropped the byte offset plays the right lump from up to $((CADF-1))
frames too early -- neither is an error and both are a sound."
done
fi
if [ "${DLX_PK_GATE_ONLY:-0}" = "1" ]; then exit 0; fi
echo
echo "--- 2/3. THE RATE: free-running, both channel configurations ---"
SECS=$(( NFR / 8 + 25 ))
run free_steal DLX_PK_HELD=0 DLX_PK_PACE=0 DLX_PK_NFR=$NFR
run free_held DLX_PK_HELD=1 DLX_PK_PACE=0 DLX_PK_NFR=$NFR
for t in free_steal free_held; do
grep -aq "FREE-RUNNING, so the mean IS the transport" "tmp/packed_$t.log" || \
fail "the $t run did not report a transport time, so there is no rate here
to read -- and a rate read off a PACED run is the pace."
done
echo
echo "--- 4. THE CLOCK: held, paced at the container's own rate ---"
SECS=$(( NFR / 12 + 25 ))
run held_paced DLX_PK_HELD=1 DLX_PK_PACE=1 DLX_PK_NFR=$NFR
# THE FINDING THIS RUN EXISTS FOR, asserted rather than admired. A held channel
# halts the 68000, and the frame clock is an INTERRUPT off V-DISP whose pending
# bit is ONE BIT -- so every edge that falls inside a transfer spanning two of
# them is an edge the machine can never count. If this ever comes back at zero,
# either the transfer got short enough to fit between two rasters or the held
# configuration stopped halting the CPU, and both change what the run means.
LOST=$(sed -n 's/.*-> \([0-9]*\) edges LOST.*/\1/p' tmp/packed_held_paced.log | head -1)
[ -n "$LOST" ] && [ "$LOST" -gt 0 ] || \
fail "the held run lost no V-DISP edges (${LOST:-none}). Either the bus is no
longer being held for the transfer, or the transfer now fits between two
rasters -- and the comparison with the stealing run below is then a
comparison of two configurations that do the same thing."
LOSTS=$(sed -n 's/.*-> \([0-9]*\) edges LOST.*/\1/p' tmp/packed_gate.log | head -1)
# ---------------------------------------------------------------------------
# 6. THE SAME AUDIO WITH THE BUS HELD, and this is the interaction ROADMAP P6c
# said neither half had met. A burst channel HALTS the 68000 for the whole
# 88 ms record, so the audio service cannot run during it -- src/player/dma.i's
# DM_HOOK is never reached and the chip is looked at twice a frame instead of
# two thousand times. The bytes are the same bytes either way; what changes is
# WHEN the next lump is armed, and the chip has no starvation state: what it
# does in between is replay the byte pair it is holding.
#
# NOT A GATE. Both configurations play the container byte for byte and the
# verifier passes on both; the difference is entirely in the seams, and a seam
# is a design cost rather than a correctness one.
if [ "$AUDIO_ON" = "1" ]; then
echo
echo "--- 6. THE AUDIO AGAIN, WITH THE BUS HELD (the contrast, not a gate) ---"
SECS=$(( NFR / 6 + 30 ))
SOUNDARGS="-samplerate 15625 -wavwrite packed_aud_held.wav"
run aud_held DLX_PK_HELD=1 DLX_PK_PACE=1 DLX_PK_FPS="$CFPS" DLX_PK_NFR=$NFR \
DLX_PK_AUD=1 DLX_PK_AJSON=packed_aud_held.json
SOUNDARGS="-sound none"
python3 tools/bench/verify_packed_audio.py "$DLXP" tmp/packed_aud_held.wav \
tmp/packed_aud_held.json || fail "the held run did not play the
container. The bytes are not what holding the bus was expected to cost."
echo
echo " THE SEAM, STEALING AGAINST HELD -- audio does not merely cost clocks:"
for t in aud aud_held; do
printf ' %-9s ' "$t"
python3 tools/bench/verify_packed_audio.py "$DLXP" "tmp/packed_$t.wav" \
| sed -n 's/^ worst \(.*\)$/\1/p' | head -1
done
echo " Stealing, the 68000 sees the channel from inside dma.i's transfer"
echo " wait. Held, it is halted for the whole record and cannot look at all."
fi
echo
echo " V-DISP edges lost: $LOST held at 12 fps, $LOSTS stealing at $GATE_FPS fps."
echo " A player keeps a clock, reads a stick and feeds ADPCM. Which of the two"
echo " configurations can do any of that is a DESIGN question, and it is the"
echo " one this run answers; neither figure is W."
exit 0
tools/bench/prep_adpcm.py
+141
View File
@@ -0,0 +1,141 @@
#!/usr/bin/env python3
"""Build the nibble stream that asks the MSM6258 which decoder it is.
ROADMAP P6a.
WHAT HAS TO BE DISCRIMINATED, and it is four things rather than the one
FINDINGS 65 named:
1. DELTA FORMULA -- 'shift' (ffmpeg's adpcm_ima_oki) against 'terms' (the
datasheet's per-term truncation). Worth 25 dB (65.2).
2. NIBBLE ORDER -- which half of a byte handed to the data register is played
FIRST. 65.1 measured 'high' AGAINST FFMPEG, which is a fact about the VOX
file convention and not about a chip's data register.
3. THE CLAMP -- the accumulator saturates somewhere, and where is inside
the recursion, so it is not an output scaling that can be undone.
4. THE INITIAL ACCUMULATOR at the instant of PLAY.
The stream is in three parts and each part exists for a reason:
PROLOGUE, 16 zero nibbles. Nibble 0 moves the step index DOWN, so it stays
pinned at 0 and the delta is a constant +2 under every candidate. That makes
the prologue a RAMP that both formulas agree on, which is what absorbs the one
thing this rig cannot control: how many times the chip consumes byte 0 before
the channel delivers byte 1. The verifier reads that count off the capture
instead of assuming it.
SEGMENT A, a quiet sine, encoded by tools/encoder/adpcm.py itself. Amplitude
300 keeps it clear of even the 10-bit clamp, so A discriminates the FORMULA
and the ORDER without the clamp confounding either. Using the shipping
encoder rather than a hand-written pattern is deliberate: the nibbles the chip
is asked about are the kind of nibbles it will be sent.
SEGMENT B, loud bursts. It exists ONLY to cross the 10-bit clamp, which
segment A is built never to reach, and it is last because a clamp is
irreversible state and everything after it would be measuring segment B.
"""
import json, math, os, sys
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "encoder"))
import adpcm
PRO_NIB = 16 # prologue nibbles (byte 0 = $00, so a repeat costs nothing)
A_SAMPLES = 1500 # segment A, one nibble each
A_AMP = 300 # clear of the 10-bit clamp at 511 with room for the ramp
A_HZ = 61.0 # ~256 samples a cycle at 15,625 Hz: many step indices
RATE = 15625.0
BUF = 0x30000 # where the harness pushes the bytes
OUT_BIN = "tmp/adpcm_data.bin"
OUT_META = "tmp/adpcm_meta.lua"
OUT_SEQ = "tmp/adpcm_seq.json"
# THE TRIGGER, and it is here because the first cut of this file did not have
# one and measured ONE differing sample in 1,676. The two formulas are
# IDENTICAL whenever the step value is a multiple of 8:
#
# terms - shift = b2*floor(r/2) + b3*floor(r/4) - floor((4*b2+2*b3+1)*r/8)
#
# with r = step mod 8 and (b1,b2,b3) the nibble's low three bits. It is zero
# for r = 0, and the step table STARTS at 16. A quiet signal never moves the
# step index off its floor, so a probe made of quiet nibbles asks the chip a
# question that has the same answer either way.
#
# nibble 4 at step 16: delta 18 under both, and it moves the index to 2
# nibble 3 at step 19: shift 16, terms 15 <- the two states part company
#
# After that they never rejoin, because the delta is added to a running
# predictor -- so ONE two-nibble trigger converts the rest of the stream into
# discriminating evidence. That is the same recursion 65.2 priced at 25 dB,
# used deliberately instead of suffered.
TRIGGER = [4, 3]
def segment_a():
"""The trigger, then a sine encoded by the shipping encoder. The model the
sine is encoded under does not matter for discrimination -- once the trigger
has parted the two states, any nibble stream keeps them apart -- so the
defaults are used and the choice is recorded rather than tuned. Using the
shipping encoder rather than a hand-written pattern is the point: the
nibbles the chip is asked about are the kind of nibbles it will be sent."""
sig = [int(round(A_AMP * math.sin(2 * math.pi * A_HZ * i / RATE)))
for i in range(A_SAMPLES)]
return TRIGGER + list(adpcm.encode(sig, "shift"))
def segment_b():
"""Loud, and alternating in sign so the step index does not simply pin: 40
up, 40 down, twice. Under a 10-bit accumulator this saturates; under a
12-bit one it does not, and that difference is the whole point of it."""
return ([7] * 40 + [15] * 40) * 2
def main():
core = segment_a() + segment_b()
nibs = [0] * PRO_NIB + core
data = adpcm.pack(nibs, "high") # HIGH first: the encoder's convention,
# which is one of the things on trial
os.makedirs("tmp", exist_ok=True)
open(OUT_BIN, "wb").write(data)
# HOW MUCH DISCRIMINATING POWER IS IN IT, counted rather than asserted. A
# probe that cannot separate two candidates reports a match against both and
# a gate that did not count this would call that a result.
ref = adpcm.decode(nibs, "shift", init=-2, bits=10)
axes = {}
for name, kw in (("formula", dict(variant="terms")),
("order", dict(order="low")),
("clamp", dict(bits=12)),
("init", dict(init=0))):
order = kw.pop("order", "high")
n2 = ([0] * PRO_NIB
+ list(adpcm.unpack(data, len(nibs), order))[PRO_NIB:]) \
if order != "high" else nibs
n2 = list(adpcm.unpack(data, len(nibs), order))
alt = adpcm.decode(n2, kw.get("variant", "shift"),
init=kw.get("init", -2), bits=kw.get("bits", 10))
d = sum(1 for a, b in zip(ref, alt) if a != b)
axes[name] = d
seq = {"nibbles": nibs, "core": core, "pro": PRO_NIB,
"bytes": len(data), "buf": BUF, "axes": axes,
"a_samples": A_SAMPLES, "a_amp": A_AMP, "a_hz": A_HZ}
json.dump(seq, open(OUT_SEQ, "w"))
with open(OUT_META, "w") as f:
f.write("return {\n")
f.write(f" buf = 0x{BUF:X},\n")
f.write(f" nbytes = {len(data)},\n")
f.write(f" nnibs = {len(nibs)},\n")
f.write("}\n")
print(f" probe stream: {len(nibs)} nibbles = {len(data)} B "
f"= {len(nibs)/RATE*1000:.1f} ms at 15,625 Hz")
print(f" prologue {PRO_NIB} zero nibbles, segment A {len(segment_a())} "
f"(sine {A_AMP} @ {A_HZ} Hz), segment B {len(segment_b())} (loud)")
print(" DISCRIMINATING POWER -- samples that change when ONE axis is "
"flipped away from MAME's own model:")
for k, v in axes.items():
print(f" {k:8s} {v:5d} of {len(ref)}")
if __name__ == "__main__":
main()
tools/bench/prep_dlx.py
+152 -30
View File
@@ -28,32 +28,112 @@ import sys, os, argparse
sys.path.insert(0, "tools/encoder")
import numpy as np
from dlx import DLX
sys.path.insert(0, "tools/bench")
import dlxload as DL
import spans as SP
# The harness loads the WHOLE container into emulated RAM at STREAM=0x30000 and
# the target is a stock 2 MB machine, so there is a hard ceiling on how much of
# a stream can be verified in one pass. The shipping player streams from disk
# into a ring buffer and has no such limit; this is a property of the test rig.
# A `scsi` window overruns it -- 2.84 MB of stream ends at 0x2E591C, 940 KB past
# the 0x200000 top of RAM -- so the frame list is truncated to what fits and the
# truncation is announced. Verifying a prefix is still a real test: SKIP blocks
# make every frame a claim about the one before it.
STREAM_BASE = 0x30000
RAM_TOP = 0x200000
MARGIN = 0x8000 # stack, flags, codebooks live below STREAM_BASE
ap = argparse.ArgumentParser()
ap.add_argument("container")
ap.add_argument("--out", default="tmp/decode")
ap.add_argument("--ram", type=lambda v: int(v, 0), default=RAM_TOP,
help="top of emulated RAM (default 0x200000, a stock 2 MB machine)")
ap.add_argument("--all-frames", action="store_true",
help="do NOT truncate to what fits in RAM (the loader will write "
"past the top of memory and the decoder will read garbage)")
a = ap.parse_args()
d = DLX(a.container)
if d.idx_bytes != 1:
sys.exit("2-byte codebook indices: decode.s assumes 1 (k<=256)")
# decode.s reads a u16 span count out of every frame record (FINDINGS 41), so a
# DLX2 container is not merely span-less to it -- the first two bytes of the
# block payload would be read as a count and the frame would decode as garbage.
# Fail here rather than there.
if not d.has_spans:
sys.exit(f"{a.container} is DLX{d.version}: src/player/decode.s expects the "
f"DLX3 span section. Re-encode (tools/encoder/encode.py emits DLX3 "
f"by default) or pass --spans off and use an older decoder.")
# --- codebooks, expanded to one WORD per pixel (high byte is discarded by
# gvram_w, so it is left zero and never has to be cleared)
cb1 = np.zeros((d.k1, 16, 2), np.uint8); cb1[:, :, 1] = d.cb1.reshape(d.k1, 16)
cb4 = np.zeros((d.k4, 4, 2), np.uint8); cb4[:, :, 1] = d.cb4.reshape(d.k4, 4)
# --- codebooks and palette. Both transforms live in tools/bench/dlxload.py so
# that prep_stream.py's ring-buffer rig shares one copy of them rather than
# keeping a second that could drift silently (FINDINGS 49).
cb1, cb4 = DL.expand_codebooks(d)
palb, dark, rendered = DL.pack_palette(d)
def build_synth(d):
"""The synthetic timing frames, as record bodies.
No real frame is all one mode, but the mix is exactly what the "76.6% x
non-SKIP fraction" model of FINDINGS 24.5 assumes away: it prices every
non-SKIP block as one V1-style burst. These price the modes separately,
which is the only way to see which one is expensive.
Every record carries the DLX3 span section, empty or not -- decode.s reads a
u16 count out of all of them, and a synthetic frame that omitted it would
desync the bitstream exactly where the harness is least likely to look.
The last two are the mode the block loop cannot express: a frame that is ALL
SPAN, its mode header entirely SKIP. Two run lengths, because a span costs
per-span plus per-pixel and one length cannot separate them --
all-SPAN-64 full-row runs, the floor of the mode (154 clocks/block,
FINDINGS 30.4)
all-SPAN-4 4-block runs, the break-even against V1 (FINDINGS 40.1)
They price v7 INSIDE decode.s against the constants tools/bench/span.sh
fitted in blit.s. Agreement cross-checks both; disagreement means the
decoder's span pass is not the sequence that was measured.
"""
out = {}
empty = SP.serialise([])
for name, mo, per in (("all-SKIP", 0, 0), ("all-V1", 1, 1),
("all-V4", 2, 4), ("all-RAW", 3, 16)):
out[name] = (bytes([mo * 0x55] * d.mode_bytes) + empty
+ bytes(d.nb * per))
# MIXED-SKIP frames. Every other synthetic frame here is a pure population,
# which is exactly why none of them prices the commonest block in a real
# container: a SKIP that shares its header byte with a coded block, and so
# cannot take the all-SKIP fast path. vq_hybrid's C_SKIP_MIXED has never
# been measured -- it was derived -- and a spanned container is made mostly
# of them, because a spanned block reads SKIP. FINDINGS 41.5.
#
# Two mixes per coded mode, because one equation cannot separate the SKIP
# cost from the cost of the block it shares a group with.
#
# THE HEADER BYTES ROTATE, and that is not decoration. decode.s reaches a
# block's 2 mode bits with `lsr.b #6/#4/#2` and no shift at all for the last
# one, so a block costs 52/48/44/34 clocks of dispatch depending on WHERE in
# its header byte it sits. A fixed byte like 0x01 puts every SKIP at the
# three expensive positions and every V1 at the free one, and solving two
# such equations returns a number that describes no real frame. Cycling the
# byte through the four rotations puts each mode at each position equally,
# which is what a real mode map does.
for nm, bys, per in (("mix-3SKIP-V1", (0x01, 0x04, 0x10, 0x40), 1),
("mix-1SKIP-3V1", (0x54, 0x51, 0x45, 0x15), 1),
("mix-3SKIP-RAW", (0x03, 0x0C, 0x30, 0xC0), 16),
("mix-1SKIP-3RAW", (0xFC, 0xF3, 0xCF, 0x3F), 16)):
hdr = bytes(bys[i % 4] for i in range(d.mode_bytes))
ncoded = sum(bin(b).count("1") and
sum(1 for k in range(4) if (b >> (2 * k)) & 3) for b in hdr[:1])
ncoded = sum(sum(1 for k in range(4) if (b >> (2 * k)) & 3) for b in hdr)
out[nm] = hdr + empty + bytes(ncoded * per)
pat = np.tile(np.arange(d.W, dtype=np.uint8), (d.H, 1))
for name, blocks in (("all-SPAN-64", d.W // 4), ("all-SPAN-4", 4)):
sp = [(y, x, pat[y, x:x + blocks * 4])
for y in range(d.H) for x in range(0, d.W, blocks * 4)]
out[name] = bytes(d.mode_bytes) + SP.serialise(sp)
return out
# --- palette words, I chosen per entry (identical maths to verify_frame256.py)
pal = d.pal.astype(int)
p6 = lambda v: ((v << 2) | (v >> 4)) & 0xFF
f = pal >> 3
render = lambda I: p6((f << 1) | I[:, None])
I = (((render(np.ones(256, int)) - pal) ** 2).sum(1)
< ((render(np.zeros(256, int)) - pal) ** 2).sum(1)).astype(int)
words = (f[:, 1] << 11) | (f[:, 0] << 6) | (f[:, 2] << 1) | I
palb = np.zeros((256, 2), np.uint8)
palb[:, 0], palb[:, 1] = words >> 8, words & 0xFF
dark = int(((render(I).astype(int)) ** 2).sum(1).argmin())
# --- frame stream: [u32 len][modes][payload] per frame, each record start
# rounded up to a 4-byte boundary.
@@ -65,28 +145,56 @@ dark = int(((render(I).astype(int)) ** 2).sum(1).argmin())
# so this loader realigns it; the encoder should carry the padding itself
# (FINDINGS 28.3). It costs at most 3 bytes per frame -- 36 B/s at 12fps,
# against a 110 KB/s budget.
budget = a.ram - STREAM_BASE - MARGIN
stream, rec_off, pad = bytearray(), [], 0
dropped = 0
# The synthetic timing frames are built FIRST, so their size comes out of the
# RAM budget rather than being appended past it. It used to be appended: the
# stream ran 26 KB beyond the top of a 2 MB machine, which was survivable only
# because the modes it overran are data-independent -- their cost is in the
# mode header, and reading junk payload costs the same as reading pixels. A
# span is not: its two jump DISPLACEMENTS come out of the stream, so an
# out-of-RAM span record jumps into open bus. FINDINGS 41.4.
SYNTH = build_synth(d)
budget -= sum(4 + len(b) + 3 for b in SYNTH.values())
for (o, n) in d.frames:
while len(stream) % 4:
stream += b"\0"; pad += 1
if not a.all_frames and len(stream) + 4 + n > budget:
dropped = d.nframes - len(rec_off)
break
rec_off.append(len(stream))
stream += n.to_bytes(4, "big") + d.raw[o:o + n]
NFRAMES = len(rec_off)
if dropped:
print(f" TRUNCATED: {NFRAMES}/{d.nframes} frames fit in RAM "
f"(stream budget {budget:,} B at 0x{STREAM_BASE:X} under a "
f"{a.ram/1024/1024:.0f} MB machine); {dropped} frames dropped.\n"
f" This is the TEST RIG's limit, not the player's -- the player "
f"streams into a ring buffer.")
# Synthetic single-mode frames. No real frame is all one mode, but the mix is
# exactly what the "76.6% x non-SKIP fraction" model of FINDINGS 24.5 assumes
# away: it prices every non-SKIP block as one V1-style burst. These four price
# the modes separately, which is the only way to see which one is expensive.
# Append the synthetic frames the budget above already reserved.
synth = {}
for name, mo, per in (("all-SKIP", 0, 0), ("all-V1", 1, 1),
("all-V4", 2, 4), ("all-RAW", 3, 16)):
for name, body in SYNTH.items():
while len(stream) % 4:
stream += b"\0"; pad += 1
synth[name] = len(stream)
hdr = bytes([mo * 0x55] * d.mode_bytes)
stream += (d.mode_bytes + d.nb * per).to_bytes(4, "big") + hdr + bytes(d.nb * per)
stream += len(body).to_bytes(4, "big") + body
assert STREAM_BASE + len(stream) <= a.ram, (
f"stream ends at 0x{STREAM_BASE+len(stream):X}, past the 0x{a.ram:X} top "
f"of RAM -- the budget arithmetic above is wrong")
# --- timing anchors: the distribution, not its mean (FINDINGS 25.6's lesson)
ns = np.array([100 * (d.modes(i) != 0).mean() for i in range(d.nframes)])
#
# A spanned block reads SKIP here, so this fraction is the BLOCK-LOOP workload
# and no longer the frame's whole cost: the span section is the rest of it. The
# anchors still pick out the extremes of the block loop, which is what they are
# for, but a frame's total decode time now has two terms.
ns = np.array([100 * (d.modes(i) != 0).mean() for i in range(NFRAMES)])
nsp = np.array([len(d.spans(i)[0]) for i in range(NFRAMES)])
spx = np.array([sum(len(p) for _, _, p in d.spans(i)[0]) for i in range(NFRAMES)])
order = np.argsort(ns)
pick = {
"min non-SKIP %.1f%%" % ns[order[0]]: int(order[0]),
@@ -95,16 +203,18 @@ pick = {
"max non-SKIP %.1f%%" % ns[order[-1]]: int(order[-1]),
}
anchors = [(n, rec_off[i], float(ns[i])) for n, i in pick.items()]
for name in ("all-SKIP", "all-V1", "all-V4", "all-RAW"):
for name in ("all-SKIP", "all-V1", "all-V4", "all-RAW",
"all-SPAN-64", "all-SPAN-4", "mix-3SKIP-V1", "mix-1SKIP-3V1",
"mix-3SKIP-RAW", "mix-1SKIP-3RAW"):
anchors.append((f"synthetic {name}", synth[name],
0.0 if name == "all-SKIP" else 100.0))
0.0 if name.startswith(("all-SKIP", "all-SPAN")) else 100.0))
blob = cb1.tobytes() + cb4.tobytes() + palb.tobytes() + bytes(stream)
open(a.out + "_data.bin", "wb").write(blob)
with open(a.out + "_meta.lua", "w") as fh:
fh.write("-- generated by tools/bench/prep_dlx.py -- do not edit\nreturn {\n")
fh.write(f" W={d.W}, H={d.H}, fps={d.fps}, nframes={d.nframes},\n")
fh.write(f" W={d.W}, H={d.H}, fps={d.fps}, nframes={NFRAMES},\n")
fh.write(f" k1={d.k1}, k4={d.k4}, dark={dark},\n")
fh.write(f" cb1_len={cb1.nbytes}, cb4_len={cb4.nbytes}, pal_len={palb.nbytes},\n")
fh.write(f" stream_len={len(stream)},\n")
@@ -118,6 +228,18 @@ print(f" cb1 {cb1.nbytes} B + cb4 {cb4.nbytes} B expanded, palette {palb.nbytes
f"stream {len(stream)} B -> {a.out}_data.bin ({len(blob)} B)")
print(f" non-SKIP blocks/frame: median {np.median(ns):.1f}% "
f"p90 {np.percentile(ns,90):.1f}% max {ns.max():.1f}%")
print(f" darkest palette entry: index {dark} -> {tuple(render(I)[dark])}")
print(f" 4-byte record alignment cost {pad} B over {d.nframes} frames "
f"({pad / d.nframes:.2f} B/frame = {pad / d.nframes * d.fps:.0f} B/s)")
print(f" v7 spans/frame: median {np.median(nsp):.0f} max {nsp.max()} "
f"({int((nsp>0).sum())}/{NFRAMES} frames); pixels painted by one: "
f"median {100*np.median(spx)/(d.W*d.H):.1f}% "
f"max {100*spx.max()/(d.W*d.H):.1f}% of the picture")
print(f" darkest palette entry: index {dark} -> {tuple(rendered[dark])}")
# A DLX2 container already carries this padding (FINDINGS 28.3 closed, session
# 9), so the realignment above re-derives bytes that were already there and the
# loader is doing no work. On a DLX1 container it is load-bearing: 94 of 120
# record starts land on odd addresses, and each one is an address error.
src_bad = sum(1 for (o, _) in d.frames[:NFRAMES] if (o - 4) % 4)
print(f" 4-byte record alignment cost {pad} B over {NFRAMES} frames "
f"({pad / NFRAMES:.2f} B/frame = {pad / NFRAMES * d.fps:.0f} B/s)")
print(f" source container is DLX{d.version}: {src_bad}/{NFRAMES} record starts "
f"unaligned" + (" -- this loader is what makes it decodable"
if src_bad else " -- the container carries its own padding"))
tools/bench/prep_frame.py
+14 -3
View File
@@ -15,16 +15,27 @@ argv = [a for a in sys.argv[1:] if not a.startswith("--")]
# to 0 displays palette entry 0, and a free mediancut palette puts a real image
# colour there. Costs one of 256 entries; measured quality cost is negligible.
RESERVE = "--reserve-black" in sys.argv
# --pack-transparent: the layout FINDINGS 46.6 needs. The packed scheme puts
# the TOP graphics page's index 0 to work as a transparency key, so index 0 must
# never appear in the picture -- and black therefore cannot live there. So:
# quantise to 254, place them at 1..254, put black at 255, leave 0 UNUSED.
# Costs two of 256 entries against --reserve-black's one.
PACKT = "--pack-transparent" in sys.argv
src, out = argv[0], argv[1]
f = sorted(glob.glob(f"{src}/*.png"))[int(argv[2]) if len(argv) > 2 else 0]
im = Image.open(f).convert("RGB")
W, H = im.size
n = 255 if RESERVE else 256
n = 254 if PACKT else (255 if RESERVE else 256)
q = im.quantize(colors=n, method=Image.MEDIANCUT, dither=Image.NONE)
pal = np.array(q.getpalette()[:n*3], dtype=np.uint8).reshape(n, 3)
idx = np.asarray(q, dtype=np.uint8)
if RESERVE:
if PACKT:
# 0 unused (transparency key), 1..254 picture, 255 black
pal = np.vstack([np.zeros((1, 3), np.uint8), pal, np.zeros((1, 3), np.uint8)])
idx = idx + 1
assert idx.min() >= 1 and idx.max() <= 254, "index 0/255 must stay free"
elif RESERVE:
pal = np.vstack([np.zeros((1, 3), np.uint8), pal]) # index 0 = black
idx = idx + 1
@@ -37,4 +48,4 @@ with open(out, "wb") as fh:
# reference PNG of exactly what the X68000 should display
Image.fromarray(pal[idx]).save(out.replace(".bin", "_ref.png"))
print(f"src={f} {W}x{H} colors={len(np.unique(idx))}"
f"{' (idx 0 reserved black)' if RESERVE else ''} -> {out}")
f"{' (idx 0 unused/transparent, 255 black)' if PACKT else (' (idx 0 reserved black)' if RESERVE else '')} -> {out}")
tools/bench/prep_load.py
+59
View File
@@ -0,0 +1,59 @@
#!/usr/bin/env python3
"""Lay out the LOAD-TIME test: raw container header in, expected results out.
python3 tools/bench/prep_load.py <in.dlx> [--out tmp/load]
src/player/load.i does on the 68000 what tools/bench/dlxload.py has been doing
host-side since session 1: expand the two codebooks to word-per-pixel form and
pack the 24-bit palette into GGGGGRRRRRBBBBBI with the shared LSB chosen per
entry (ROADMAP P1 and P2). This writes both halves of that comparison.
<out>_data.bin the container's HEADER REGION, byte for byte as it comes
off the disc: magic, geometry, the three section offsets,
the 768-byte palette, CB1 and CB4. Nothing is pre-chewed --
that is the entire point. It ends where the frame stream
begins, so it is also exactly what a player would have to
read before it could draw anything.
<out>_expect.bin what dlxload.py says the 68000 must produce: expanded CB1,
expanded CB4, then 256 big-endian palette words.
<out>_meta.lua sizes, k1/k4, and the expected darkest-entry index.
The expectation is generated by the SAME module the two decode rigs load
through, so this cannot pass by agreeing with a second copy of the maths.
"""
import sys, argparse
sys.path.insert(0, "tools/encoder")
sys.path.insert(0, "tools/bench")
from dlx import DLX
import dlxload as DL
ap = argparse.ArgumentParser()
ap.add_argument("container")
ap.add_argument("--out", default="tmp/load")
a = ap.parse_args()
d = DLX(a.container)
if d.version < 3:
sys.exit(f"{a.container} is DLX{d.version}: load.i wants DLX3 or DLX4")
if d.idx_bytes != 1:
sys.exit("2-byte codebook indices: load.i expands one source byte per pixel")
off_frm = int.from_bytes(d.raw[28:32], "big")
raw = d.raw[:off_frm]
cb1, cb4 = DL.expand_codebooks(d)
palb, dark, _ = DL.pack_palette(d)
open(a.out + "_data.bin", "wb").write(raw)
open(a.out + "_expect.bin", "wb").write(cb1.tobytes() + cb4.tobytes() + palb.tobytes())
with open(a.out + "_meta.lua", "w") as fh:
fh.write("-- generated by tools/bench/prep_load.py -- do not edit\nreturn {\n")
fh.write(f" k1={d.k1}, k4={d.k4}, dark={dark},\n")
fh.write(f" raw_len={len(raw)}, cb1_len={cb1.nbytes}, cb4_len={cb4.nbytes},\n")
fh.write(f" pal_len={palb.nbytes},\n}}\n")
print(f"{a.container}: k1={d.k1} k4={d.k4}, header region {len(raw)} B "
f"(pal 768 + cb1 {d.k1*16} + cb4 {d.k4*4} + 32)")
print(f" the 68000 must produce {cb1.nbytes} + {cb4.nbytes} B of expanded "
f"codebook and {palb.nbytes} B of palette, darkest entry {dark}")
tools/bench/prep_packed.py
+79
View File
@@ -0,0 +1,79 @@
#!/usr/bin/env python3
"""Everything tools/bench/packed.lua needs to know about a DLXP container.
python3 tools/bench/prep_packed.py <in.dlxp> -> tmp/packed_meta.lua
THERE IS NO BLOB TO PREPARE, and that is the whole difference from
`prep_stream.py`. The codec's rig has to hand the machine expanded codebooks, a
packed palette and a record index, because a DLX record cannot be found or drawn
without them; `prep_dlx.py` and `prep_stream.py` exist for that and FINDINGS
49.7.5 records what it cost to have two copies of one of those transforms. A
packed container carries no such thing: record `i` is at sector 1 + i*97 by
geometry and its bytes are already in the order GVRAM wants them (dlxp.py). So
this file emits METADATA ONLY -- six numbers the rig would otherwise have to
hard-code, every one of them read out of the container's own header.
The volume is the container itself; tools/bench/mkvol.sh copies it.
"""
import os, sys
sys.path.insert(0, os.path.join(os.path.dirname(os.path.abspath(__file__)),
"..", "encoder"))
from dlxp import DLXP, SECTOR
if len(sys.argv) != 2:
sys.exit(__doc__)
d = DLXP(sys.argv[1])
# The record's sector count, and the array chain's entry count, DERIVED here and
# asserted by the 68000 (PG_ARRN). Two independent statements of one geometry
# is the only way a container and a player can be caught disagreeing about it --
# a chain one entry short delivers a picture with its last row missing, which
# looks like a decode bug and is a layout bug.
recs = d.rec_bytes // SECTOR
rows = d.H
entries = rows + (1 if d.has_palette else 0)
out = "tmp/packed_meta.lua"
with open(out, "w") as fh:
fh.write("-- generated by tools/bench/prep_packed.py; do not edit\n")
fh.write("return {\n")
for k, v in [("W", d.W), ("H", d.H), ("fps", d.fps), ("nframes", d.nframes),
("rec_bytes", d.rec_bytes), ("rec_sectors", recs),
("pal_bytes", d.pal_bytes), ("pic_bytes", d.pic_bytes),
("lba0", d.off_frm // SECTOR),
("palette_last", int(d.palette_last)),
("has_palette", int(d.has_palette)),
# DLXP2. Zero in a silent container, and the player branches on
# the zero rather than being built two ways.
("cad_f", d.cad_f), ("cad_a", d.cad_a),
("has_audio", int(d.has_audio)),
("aud_bytes", d.aud_bytes), ("aud_hz", d.aud_hz),
("n_lumps", d.n_lumps),
# ROADMAP P6c. lba_aud is off_aud/512 and is a HEADER field
# rather than the constant 1 it happens to equal: a shipping
# volume has a filesystem in front of the stream and the
# player adds PG_LBA0's base to neither of them by accident.
("lba_aud", d.off_aud // SECTOR if d.has_audio else 0),
# the four axes, so the rig can print what the container says
# it was encoded for and the verifier can decode with it
("aud_variant", '"%s"' % d.decoder()["variant"]
if d.has_audio else '""'),
("aud_order", '"%s"' % d.decoder()["order"]
if d.has_audio else '""'),
("aud_bits", d.aud_bits), ("aud_init", d.aud_init),
("entries", entries)]:
fh.write(f" {k} = {v},\n")
fh.write("}\n")
print(f"{sys.argv[1]}: DLXP{d.version} {d.W}x{d.H} {d.fps}fps {d.nframes} frames"
+ (f", AUDIO F={d.cad_f} A={d.cad_a}" if d.has_audio else ", silent"))
print(f" record {d.rec_bytes:,} B = {recs} sectors, palette "
f"{'LAST' if d.palette_last else 'FIRST'}, {d.pal_bytes} B")
cad = (f" + (i//{d.cad_f})*{d.cad_a}" if d.has_audio else "")
print(f" record i is at LBA {d.off_frm // SECTOR} + i*{recs}{cad} -- ARITHMETIC. "
f"There is no index in this container and none can be needed.")
print(f" the chain the 68000 must build: {entries} entries "
f"({rows} rows{' + 1 palette' if d.has_palette else ''})")
print(f" wire {d.video_kbps():.1f}"
+ (f" + {d.audio_kbps():.2f} = {d.kbps():.1f}" if d.has_audio else "")
+ f" KB/s, FIXED by geometry -> {out}")
tools/bench/prep_spans.py
+170
View File
@@ -0,0 +1,170 @@
#!/usr/bin/env python3
"""Generate V5 span streams for tools/bench/span.lua (FINDINGS 29.5 item 1).
FINDINGS 29 prices a new decoder mode -- a row-linear run of word-expanded
literal pixels, movem.l'd straight from the stream buffer into GVRAM -- at
`4 * (50 + 4L * 9.08)` cycles for a run of L blocks. Both halves of that are
extrapolations: the 50-cycle per-span overhead is hand-derived, and the 9.08
cycles/pixel was measured (FINDINGS 24 V1) at FULL ROW WIDTH with 12-register
bursts, which a short span cannot match. This script builds the stimulus that
replaces both numbers with measured ones.
One stream per span length. Every stream covers the SAME 192x256 picture
completely, so all of them draw an identical, verifiable frame and differ only
in how many spans it is cut into -- which is what lets span.lua regress
cycles = A * spans + B * pixels
across the set and read the per-span overhead off directly.
Two stream formats, both big-endian, both drawing the same frame.
v5 -- a decoder handed (x, npix) that works out the copy itself:
per row, 192 rows in order:
u16 nspans
nspans * { u16 x, u16 npix, npix * u16 pixel }
v6 -- the same spans with that arithmetic moved here, where it is free:
u16 nspans (whole frame; there is no row structure)
nspans * { u32 absolute GVRAM address, u16 jump displacement,
units * 48 bytes of pixels }
Span lengths are multiples of 24 pixels (one chain unit) and the last span
in a row may overrun the visible 256 by up to 23 pixels, which is free: the
line stride is 1024 bytes and only the first 512 are displayed. The jump
displacement selects an entry point into the decoder's unrolled copy chain.
v7 -- v6 plus a second, FINER chain for the tail (FINDINGS 39.4). Padding to
v6's 24-pixel quantum wastes ~11 pixels on an average span, and FINDINGS
39.3 attributes 86% of the DMAC array-chain's advantage over v6 to it. A
v7 span is 24*c + 2*f pixels, so the quantum is 2 and a run of 4x4 blocks
(always a multiple of 4 pixels) pads to NOTHING:
u16 nspans
nspans * { u32 absolute GVRAM address, u16 coarse displacement,
c * 48 bytes of pixels,
u16 fine displacement, f * 4 bytes of pixels }
The fine displacement is in the STREAM rather than the record because that
is what lets the decoder keep all 12 payload registers: the coarse chain
falls out into a `move.w (a0)+,d0 / jmp` with d0 dead and a0 pointing at
it. Costed here as an 8-byte record, since it is 2 more bytes a span.
Pixels are word-expanded with the palette index in the low byte; the high byte
is whatever we put there because gvram_w masks it off (x68k_crtc.cpp:501).
"""
import struct, sys
import numpy as np
SRC = sys.argv[1] if len(sys.argv) > 1 else "tmp/frame256.bin"
OUT = sys.argv[2] if len(sys.argv) > 2 else "tmp/spans.bin"
META = OUT.replace(".bin", "_meta.lua")
d = open(SRC, "rb").read()
assert d[:4] == b"DLXR", SRC
W, H = struct.unpack(">HH", d[4:8])
idx = np.frombuffer(d[8+768:8+768+W*H], np.uint8).reshape(H, W)
assert (W, H) == (256, 192), f"{W}x{H}: span bench assumes the 256x192 picture"
# (span length in pixels, x of the first span). 4 px = one 4x4 block wide, the
# case the whole FINDINGS 29 argument turns on; 256 = one span per row, the
# case closest to the V1 measurement it extrapolates from. 16u starts at an
# odd x so its bursts run at addr mod 4 == 2: a claim about the 68000's 16-bit
# bus that costs nothing to test and would be embarrassing to assume.
CONFIGS = [(4, 0), (8, 0), (12, 0), (16, 0), (16, 1), (20, 0), (24, 0),
(32, 0), (48, 0), (64, 0), (128, 0), (256, 0)]
# v6 geometry, and it must match blit.s: 12 registers per movem = 48 bytes =
# 24 pixels per chain unit, 11 units in the chain.
UNITPX, UNITSZ, UNITS = 24, 12, 11
# v7 geometry, and it must match blit.s: coarse unit as v6, fine unit is one
# `move.l (a0)+,(a2)+` = 2 bytes of code = 2 pixels, 11 of them (22 px > 24).
FINEPX, FINESZ, FINES = 2, 2, 11
GVRAM, YOFF, STRIDE = 0xC00000, 32, 1024
# v7 span lengths, in pixels. Multiples of 4 (a real span is a run of 4x4
# blocks), chosen so the fine remainder P mod 24 takes every value a real span
# can: 0, 4, 8, 12, 16, 20. 4/8/12/16/20 are pure-fine, 24/48/72/120/240 are
# pure-coarse, the rest mix -- which is what makes the three-term fit
# cycles = A*spans + Bc*coarse_px + Bf*fine_px identifiable.
V7CONFIGS = [4, 8, 12, 16, 20, 24, 28, 44, 48, 72, 100, 120, 256]
blob, metas = bytearray(), []
for P, x0 in CONFIGS:
off = len(blob)
nspans = npix = 0
for y in range(H):
cuts = []
x = 0
if x0: # a short leading span to shift the phase
cuts.append((0, x0)); x = x0
while x < W:
n = min(P, W - x)
cuts.append((x, n)); x += n
blob += struct.pack(">H", len(cuts))
for x, n in cuts:
blob += struct.pack(">HH", x, n)
blob += idx[y, x:x+n].astype(">u2").tobytes()
nspans += 1; npix += n
metas.append(dict(name=f"{P}{'u' if x0 else ''}", p=P, x0=x0, off=off,
len=len(blob)-off, nspans=nspans, npix=npix,
cpx=npix, fpx=0, var=5))
# v6: one config per chain depth, so the fit sees spans from 24 to 264 pixels.
for units in range(1, UNITS+1):
P = units * UNITPX
off = len(blob)
nspans = npix = 0
rows = []
for y in range(H):
x = 0
while x < W:
rows.append((y, x)); x += P
blob += struct.pack(">H", len(rows))
for y, x in rows:
blob += struct.pack(">IH", GVRAM + (YOFF+y)*STRIDE + x*2,
(UNITS-units)*UNITSZ)
# Pad the last span of a row past the visible width; the overrun lands
# in the undisplayed half of the line.
px = np.concatenate([idx[y, x:x+P], np.zeros(max(0, x+P-W), np.uint8)])
blob += px.astype(">u2").tobytes()
nspans += 1; npix += P
metas.append(dict(name=f"{P}", p=P, x0=0, off=off, len=len(blob)-off,
nspans=nspans, npix=npix, cpx=npix, fpx=0, var=6))
# v7: same tiling, but the span is cut at a 2-pixel quantum instead of 24.
for P in V7CONFIGS:
units, fine = divmod(P, UNITPX)
assert fine % FINEPX == 0 and fine // FINEPX <= FINES, P
assert units <= UNITS, P
off = len(blob)
nspans = npix = 0
rows = []
for y in range(H):
x = 0
while x < W:
rows.append((y, x)); x += P
blob += struct.pack(">H", len(rows))
for y, x in rows:
assert x + P <= STRIDE // 2, (P, x) # the overrun must stay on the line
blob += struct.pack(">IH", GVRAM + (YOFF+y)*STRIDE + x*2,
(UNITS-units)*UNITSZ)
px = np.concatenate([idx[y, x:x+P], np.zeros(max(0, x+P-W), np.uint8)])
blob += px[:units*UNITPX].astype(">u2").tobytes()
blob += struct.pack(">H", (FINES - fine//FINEPX)*FINESZ)
blob += px[units*UNITPX:].astype(">u2").tobytes()
nspans += 1; npix += P
metas.append(dict(name=f"{P}", p=P, x0=0, off=off, len=len(blob)-off,
nspans=nspans, npix=npix,
cpx=nspans*units*UNITPX, fpx=nspans*fine, var=7))
open(OUT, "wb").write(blob)
with open(META, "w") as f:
f.write("-- generated by tools/bench/prep_spans.py -- do not edit\nreturn {\n")
f.write(f" W={W}, H={H}, total={len(blob)},\n configs = {{\n")
for m in metas:
f.write(" {{var={var}, name=\"{name}\", p={p}, x0={x0}, off={off},"
" len={len}, nspans={nspans}, npix={npix}, cpx={cpx},"
" fpx={fpx}}},\n".format(**m))
f.write(" },\n}\n")
print(f"{SRC} {W}x{H} -> {OUT} {len(blob)} B, {len(metas)} configs")
for m in metas:
print(f" v{m['var']} span {m['name']:>4} px: {m['nspans']:6d} spans, "
f"{m['npix']:6d} px, {m['len']:7d} B "
f"(+{100*m['len']/(2*W*H)-100:.1f}% over bare pixels)")
tools/bench/prep_stream.py
+125
View File
@@ -0,0 +1,125 @@
#!/usr/bin/env python3
"""Lay a DLX3 container out as a DISK for the ring-buffer rig (FINDINGS 49).
python3 tools/bench/prep_stream.py <in.dlx> [--out tmp/stream]
prep_dlx.py's output is one blob that tools/bench/decode.lua pushes into
emulated RAM in its entirety. That is what makes its rig RAM-bound -- a `scsi`
window is 5,261,814 B of stream and needs a 6 MB machine to hold it (FINDINGS
45) -- and, much more importantly, it is nothing like the shipping player, which
never holds a window at once.
This writes three files instead:
<out>_cb.bin codebooks + palette. ~10 KB, loaded into RAM once, exactly as
before: these are LOAD-TIME costs and not per-frame ones.
<out>_disk.bin the frame records, `[u32 len][body]` each padded up to 4, laid
end to end. tools/bench/stream.lua reads this from the HOST
filesystem and feeds it into a bounded ring, so the emulated
machine's RAM stops bounding how much of a window can be
tested. A stock 2 MB machine can now run all 120 frames.
<out>_idx.bin the DLX4 RECORD INDEX, nframes u16 big-endian, straight out
of the container's scene header. This is what the 68000
producer reads (src/player/ring.i); it is not derived here,
because deriving it is precisely what a player cannot do.
<out>_meta.lua geometry, and the record index.
THE RECORD INDEX IS NOT A CONVENIENCE. src/player/stream.s takes each frame's
base address from a descriptor the producer wrote, rather than deriving it from
where the last frame ended, because under the `aligned` wrap policy the next
record may be at the ring's base instead of just after its predecessor. The
producer therefore has to know record boundaries before it places them -- which
is what an index is. A branching laserdisc game needs one anyway to seek to a
branch point, so the policy that costs no clocks (tools/analysis/19_ring_stream.py)
reuses a structure the player cannot avoid.
The 4-byte record padding is the same one decode.s needs and DLX3 already
carries: `move.l (a0)+,d0` on an odd address is an ADDRESS ERROR on a 68000, not
a slow read. FINDINGS 28.3.
NO SYNTHETIC TIMING FRAMES. prep_dlx.py appends ten of them to price the block
modes separately; this rig measures delivery, not decode, and its per-frame cost
anchors are prep_dlx.py's job. Mixing them in would put frames on the wire that
no encoder emits and no rate controller sized.
"""
import sys, os, argparse
sys.path.insert(0, "tools/encoder")
sys.path.insert(0, "tools/bench")
import numpy as np
from dlx import DLX
import dlxload as DL
ap = argparse.ArgumentParser()
ap.add_argument("container")
ap.add_argument("--out", default="tmp/stream")
a = ap.parse_args()
d = DLX(a.container)
if d.idx_bytes != 1:
sys.exit("2-byte codebook indices: src/player/ assumes 1 (k<=256)")
if not d.has_spans:
sys.exit(f"{a.container} is DLX{d.version}: src/player/stream.s expects the "
f"DLX3 span section (see prep_dlx.py for why a DLX2 container "
f"decodes as garbage rather than merely losing its spans).")
cb1, cb4 = DL.expand_codebooks(d)
palb, dark, rendered = DL.pack_palette(d)
open(a.out + "_cb.bin", "wb").write(cb1.tobytes() + cb4.tobytes() + palb.tobytes())
disk, index = bytearray(), []
for (o, n) in d.frames:
start = len(disk)
disk += n.to_bytes(4, "big") + d.raw[o:o + n]
# The container's own alignment rule, not this script's copy of it: DLX5
# pads to 512 so a DMA channel can read whole sectors into the ring, DLX4
# to 4 so `move.l (a0)+` does not take an address error (28.3).
while len(disk) % d.rec_align:
disk += b"\0"
index.append((start, len(disk) - start))
open(a.out + "_disk.bin", "wb").write(bytes(disk))
# THE CONTAINER'S OWN INDEX, and it is checked against this layout rather than
# regenerated from it. src/player/ring.i walks the disk with a running sum of
# these lengths and never reads a record's length word before fetching it, so a
# container index that disagreed with the disk image by one byte would place
# every later record at the wrong address -- and the block loop reads without a
# bounds check (49.2), so the symptom would be wrong pixels, not a fault.
if d.has_index:
want = [ln // 4 for _, ln in index]
if d.index != want:
bad = next(i for i in range(len(want)) if d.index[i] != want[i])
sys.exit(f"{a.container}: the DLX4 index disagrees with this disk "
f"layout at record {bad}: {d.index[bad]} vs {want[bad]} "
f"longwords")
open(a.out + "_idx.bin", "wb").write(
b"".join(q.to_bytes(2, "big") for q in d.index))
else:
if os.path.exists(a.out + "_idx.bin"):
os.remove(a.out + "_idx.bin") # a stale index is worse than none
rec = np.array([n for _, n in index])
with open(a.out + "_meta.lua", "w") as fh:
fh.write("-- generated by tools/bench/prep_stream.py -- do not edit\nreturn {\n")
fh.write(f" W={d.W}, H={d.H}, fps={d.fps}, nframes={d.nframes}, dark={dark},\n")
fh.write(f" cb1_len={cb1.nbytes}, cb4_len={cb4.nbytes}, pal_len={palb.nbytes},\n")
fh.write(f" disk_len={len(disk)}, maxrec={int(rec.max())},\n")
fh.write(f" dlx_version={d.version}, "
f"has_index={'true' if d.has_index else 'false'},\n")
fh.write(f" padrec={{{','.join(str(ln) for _, ln in index)}}},\n")
fh.write(" index={\n")
for off, ln in index:
fh.write(f" {{off={off}, len={ln}}},\n")
fh.write(" },\n}\n")
print(f"{a.container}: {d.nframes} frames, {d.W}x{d.H}")
print(f" codebooks+palette {cb1.nbytes + cb4.nbytes + palb.nbytes:,} B -> "
f"{a.out}_cb.bin")
print(f" disk image {len(disk):,} B -> {a.out}_disk.bin "
f"(records: min {rec.min():,} median {int(np.median(rec)):,} "
f"max {rec.max():,})")
print(f" wire rate {rec.mean()*d.fps/1024:.1f} KB/s video at {d.fps} fps")
if d.has_index:
print(f" DLX4 record index: {2*d.nframes:,} B of scene header, and it "
f"agrees with the disk image on all {d.nframes} records")
print(f" A ring must hold one whole record contiguously: >= {rec.max():,} B "
f"({rec.max()/1024:.1f} KB) before any policy or prefill.")

Some files were not shown because too many files have changed in this diff Show More