drwily/Dragon-s-Lair-X68k
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Pace the ring, then read the DMAC config out of the IPL ROM: audio is cheap and the disk is not

Browse Source 
Two sessions that were never separated in the working tree, so they land as one
commit. check.sh ALL GREEN before and after both.

SESSION 19 -- the ring rig gets a frame clock (FINDINGS 51).

src/player/stream.s had no frame clock: it asked for record i the instant it
finished i-1, outran any finite pipe, and never let the ring back up. The 49.1
sweep passing at 48 KB was therefore a wrap-correctness result and nothing else.
PACE/PACEON ($18034/$18038) hold the decoder to 12 fps, so FR_HEAD-FR_TAIL
finally means what it reads as: whole frames the decoder could still draw with
delivery stopped dead. PACEON=0 free-runs and is what the wrap gate still uses,
so every figure in 49 is unmoved.

Paced, on the gate container: 64 KB holds 2 frames, 256 KB holds 7-8, 512 KB
holds 14-15, all pixel-exact. Tolerance is ceiling-1, measured by cutting the
pipe: 256 KB buys 500 ms of dead pipe, not 583.

SLACK IS ACCUMULATED, NOT OWNED. It is built out of pipe-wire and a seek spends
all of it. At 488 KB/s a 256 KB ring needs 4.83 s of play to reach its ceiling
from empty; 512 KB needs 8.42 s to reach 14. A bigger ring raises the ceiling
AND lengthens the climb, so a branch point does not ask "is the buffer big
enough" but "has there been enough play since the last one" -- and Dragon's
Lair's decision points are seconds apart. The rig now also says WHICH resource
is binding: at 460 KB/s every ring from 192 KB to 512 KB is rate-bound at
ceiling 4 and never fills, so larger rings are dead RAM in that scene.
20_seek_slack.py is the same model rewritten in Python from record sizes,
sharing no code with the Lua producer: 35/35 ceilings inside its bracket.

SESSION 20 -- the DMAC configuration was in the IPL ROM the whole time
(FINDINGS 52).

ROADMAP's "do this first" was to put the ADPCM stream on the bus. That needs a
clocks-per-byte figure for the audio channel, and 11_cpu_budget.py was charging
audio the DISK's rate -- 5 clk/B, its own help text calling it "single-address,
bus held". Audio was being charged the favourable end of B3, a 242 KB/s open
question.

It never had to be a guess. The IPL ROM programs all four HD63450 channels
itself and MAME boots the rig with it, so 21_iplrom_dmac.py reads the
configuration out of the image and decodes the MC68450 fields. Eight
(address, expected bytes, meaning) sites; a mismatch or an unknown revision
exits non-zero. In check.sh, no emulator, milliseconds.

ch3 DCR=$80, OCR=$32: dual address, 8-bit port, cycle steal WITHOUT hold,
REQG=10 external request. The DMAC arbitrates once per byte with no burst to
amortise the 5..8 + 2 over, so an audio byte is 16..19 clocks, not 5 -- the old
debit was 3.2x..3.8x small. And on the bus it is still nothing: 651 B/frame is
1.25%..1.48% of a frame, about 4% of what the decoder leaves. P6's bus risk
does not materialise. The unit worry was worth checking and nearly right: 15.6
kHz is 8 MHz/512 = 15,625 samples/s, two 4-bit samples to a byte = 7,812.5 B/s
exactly, and AUDIO_KBPS=7.8 is that in decimal kB while the tool multiplied by
1024.

THE DISK CHANNEL IS PROGRAMMED IDENTICALLY. ch1 (SASI) is DCR=$80 too, and so
is ch0. That is 16..19 clocks per delivered byte, where 42.4 brackets W at 5..12
and 42.5 has W=8 already missing 47/120 frames. The only worked example of a
disk DMA configuration on this machine sits above the entire bracket, and at
that price nothing fits at any container size. It is not scsiexrom.bin so B3
stays open -- what changed is that a cheap configuration is now the thing that
has to be SHOWN. W <= 12 is a requirement on the player's DMAC programming, not
a range the hardware hands us, and it is now the largest open number in the
project, ahead of the rate.

An unforced cross-check fell out: 15_bus_occupancy.py's new W sweep puts W=8 at
105.7% of the frame, agreeing with 42.5's 47/120, from mode histograms and bus
clocks respectively, two models sharing no code.

Also: ADPCM outranks the disk at the arbiter (CPR 1 against 2), so an audio byte
never waits and a video byte does -- relevant to 51's smooth-rate delivery model.

README MEDIA.

stream.lua gains DLX_SNAP_EVERY=1 (needs DLX_PACE, off by default, on no path
check.sh takes) and tools/media/make_readme_media.py turns the PNGs into
docs/img/. The stills and both clips are MAME's own screen pixels.

Building it turned up something worth recording. 116 of 119 captured frames are
pixel-exact against dlx.py; three are TORN -- frame n on top, frame n-1 below
the tear line -- because MAME captured the screen while the block loop was
partway down it. decode.s writes straight to the displayed page (one display
path, 28.1), so a real player tears the same way, and this is the first time
that consequence has been visible rather than argued. The script ASSERTS the
tear and refuses to build otherwise, rather than trimming three frames and
reporting "every frame I kept is exact". Second correction the capture forced:
the snapshot fires before frame n is decoded, so the obvious reading is that it
holds frame n-1 -- it does not, because MAME renders the screen at the end of
the machine frame, by which time the 68000 has finished frame n.

11_cpu_budget.py's "validated to within 1 pt" line is also corrected: the model
reads 2..10 pt HIGH and by more as the frame gets harder, which was already true
before either session.

src/player/decode.s is unchanged; decode.bin is still 1,296 B at the same MD5.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
This commit is contained in:
prosolis
2026-08-24 18:14:01 -07:00
parent b49bbdc939
commit 2f9f5cc995
49 changed files with 6276 additions and 375 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tools/analysis/08_peaks.py
+1 -1
View File
@@ -1,4 +1,4 @@
"""At a 488 KB/s (4 Mbps) ceiling and ~52% mean utilisation, the mean is not the
"""At any fixed delivery ceiling and ~52% mean utilisation, the mean is not the
risk -- the peaks are. Measure per-frame peak-to-mean, then check whether the
leaky-bucket rate controller actually holds the ceiling."""
import sys; sys.path.insert(0,'tools/encoder')
tools/analysis/11_cpu_budget.py
+51 -9
View File
@@ -24,7 +24,13 @@ import numpy as np
from dlx import DLX
import vq_hybrid as H
import ratectl as RC
RC_AUDIO_BPS = RC.AUDIO_KBPS * 1024
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import buscost as B
# The audio byte rate is now DERIVED, not restated: 15.6 kHz mono MSM6258V is
# 15,625 4-bit samples/s, two to a byte. RC.AUDIO_KBPS's 7.8 is that figure in
# DECIMAL kB, and was being multiplied by 1024 here -- a 2.4% overstatement,
# harmless, but it hid which unit the constant was in.
RC_AUDIO_BPS = B.ADPCM_BYTES_PER_S
# Machine clocks, confirmed from MAME 0.277 src/mame/sharp/x68k.cpp:1133/1194/
# 1200 -- not recalled. x68000 and x68ksupr are BOTH 40_MHz_XTAL/4 = 10 MHz;
@@ -58,8 +64,26 @@ ap.add_argument("--dma-clocks-per-word", type=float, default=8.0,
help="HD63450 cycle-steal. ESTIMATE from FINDINGS 5, NEVER "
"MEASURED, and the most load-bearing unmeasured number "
"in the project (FINDINGS 35.3)")
ap.add_argument("--dma-clocks-per-byte", type=float, default=5.0,
help="what the SCSI DMA costs per DELIVERED BYTE. The MB89352 "
"is an 8-bit port, so the DMAC pays per byte and the "
"per-word denominator of FINDINGS 5/39.7 was half the "
"real debit (FINDINGS 43). 5 = single-address, bus held, "
"no drive wait; 9 = dual-address")
ap.add_argument("--pio-clocks-per-byte", type=float, default=12.0,
help="hand-derived floor for a 68000 register-to-RAM copy")
# Audio is NOT the disk, and charging it the disk's rate was charging it the
# favourable side of an open question. tools/analysis/21_iplrom_dmac.py reads
# the IPL ROM's own HD63450 setup: channel 3 is dual address, 8-bit port, cycle
# steal WITHOUT hold, external request -- one full arbitration per byte, no
# burst to amortise it over. 16 is the datasheet best case, 19 the worst.
ap.add_argument("--adpcm-clocks-per-byte", type=float,
default=B.ADPCM_CLK_BYTE_BEST,
help="what an ADPCM byte costs. READ OUT OF THE IPL ROM's DMAC "
"configuration (21_iplrom_dmac.py), not assumed: dual "
"address + per-byte arbitration = 16 best, 19 worst. The "
"audio stream always DMAs, whatever --io says about the "
"disk")
a = ap.parse_args()
CPUHZ = CLOCKS[a.machine] * 1e6
FPS = a.fps
@@ -72,13 +96,15 @@ d = DLX(a.container)
# --- what the transfer costs, from the container's own byte rate
vid_bps = sum(n + 4 for (_, n) in d.frames) / d.nframes * d.fps
io_bps = vid_bps + RC_AUDIO_BPS
aud_cycles_per_s = RC_AUDIO_BPS * a.adpcm_clocks_per_byte
if a.io == "dma":
io_cycles_per_s = (io_bps / 2) * a.dma_clocks_per_word
io_cycles_per_s = vid_bps * a.dma_clocks_per_byte + aud_cycles_per_s
elif a.io == "pio":
io_cycles_per_s = io_bps * a.pio_clocks_per_byte
io_cycles_per_s = vid_bps * a.pio_clocks_per_byte + aud_cycles_per_s
else:
io_cycles_per_s = 0.0
io_pct = 100 * io_cycles_per_s / CPUHZ
aud_pct = 100 * aud_cycles_per_s / CPUHZ
FRAME_NET = FRAME * (1 - io_pct / 100)
modes = [d.modes(f) for f in range(d.nframes)]
@@ -91,9 +117,20 @@ print(f"budget: {a.machine} @ {CLOCKS[a.machine]:.2f} MHz, {FPS:g} fps "
f"-> {FRAME:,.0f} cycles/frame")
print(f" I/O ({a.io}): {io_bps/1024:.1f} KB/s costs {io_pct:.1f}% of the CPU "
f"-> {FRAME_NET:,.0f} cycles/frame left for decoding")
if a.io != "none":
print(f" video {vid_bps/1024:6.1f} KB/s x "
f"{(a.dma_clocks_per_byte if a.io=='dma' else a.pio_clocks_per_byte):g}"
f" clk/B = {io_pct-aud_pct:5.2f}% "
f"(W: still open, ROADMAP B3 / FINDINGS 42.4)\n"
f" audio {RC_AUDIO_BPS/1024:6.2f} KB/s x {a.adpcm_clocks_per_byte:g}"
f" clk/B = {aud_pct:5.2f}% "
f"(SETTLED: read out of the IPL ROM, FINDINGS 52)")
if a.io == "dma":
print(f" {a.dma_clocks_per_word:g} clocks/word is an ESTIMATE (FINDINGS 5), "
f"never measured -- see FINDINGS 35.3")
print(f" {a.dma_clocks_per_byte:g} clocks/BYTE, the MC68450 datasheet "
f"floor for an 8-bit port (FINDINGS 43).\n It is not measured on "
f"hardware; what IS settled is that the per-word denominator this\n"
f" used before session 14 was physically impossible -- 2.5 "
f"clocks/byte is below\n the 68000's 4-clock minimum bus cycle.")
elif a.io == "none":
print(" WARNING: --io none scores the decoder as if the disk were free. "
"That is the\n premise FINDINGS 35 overturned; every 'N frames miss' "
@@ -114,15 +151,20 @@ TIMED_FRAMES = (("min non-SKIP", 15.4, 31.5), ("median", 48.1, 73.8),
("p90", 82.5, 116.4), ("max non-SKIP", 100.0, 135.8))
if (os.path.abspath(a.container) == os.path.abspath(TIMED)
and a.machine == "stock" and a.fps == 12):
print("model vs the frames actually timed on the 68000:")
print("model vs the frames actually timed on the 68000 "
"(the model reads HIGH, and by more\n as the frame gets harder -- "
"so a 'does not fit' from it is the safe direction):")
for label, frac, meas in TIMED_FRAMES:
i = int(np.argmin(abs(ns - frac)))
print(f" {label:<14} non-SKIP {ns[i]:5.1f}% model {pct[i]:6.1f}% "
f"measured {meas:5.1f}% error {pct[i]-meas:+.1f} pt")
else:
print(f"(no 68000 timings for this container/machine -- the model was "
f"validated to\n within 1 pt on {TIMED} at stock/12fps;\n"
f" run tools/bench/decode.lua to time another container)")
print(f"(no 68000 timings for this container/machine. The model is "
f"validated against four\n frames timed on the 68000, and only on "
f"{TIMED}\n at stock/12fps -- run it on that container to see the "
f"errors, which are a few points\n CONSERVATIVE and grow with the "
f"non-SKIP fraction. Run tools/bench/decode.lua to\n time another "
f"container.)")
print(f"\nper-frame cost, % of a {FPS:g}fps frame budget:")
print(f" measured-cost model: median {np.median(pct):5.1f} "
tools/analysis/12_span_tradeoff.py
+4 -4
View File
@@ -1,9 +1,9 @@
#!/usr/bin/env python3
"""What does spending the idle bus bandwidth buy back in CPU cycles?
python3 tools/analysis/12_span_tradeoff.py [container.dlx] [--bus 488]
python3 tools/analysis/12_span_tradeoff.py [container.dlx] --bus <KB/s>
FINDINGS 28 leaves the decoder CPU-bound at 110 KB/s on a 488 KB/s pipe. Every
FINDINGS 28 leaves the decoder CPU-bound at 110 KB/s on a much wider pipe. Every
codec decision was made when bytes were scarce, so each one trades cycles to
save them -- and the cheapest thing a 68000 can be handed is the most expensive
thing to store: word-expanded pixels in row-linear runs.
@@ -50,8 +50,8 @@ def span_px(npix): # a span is a whole number of units
ap = argparse.ArgumentParser()
ap.add_argument("container", nargs="?",
default="tmp/rc_fr_singe_sasi_rcprofile.dlx")
ap.add_argument("--bus", type=float, default=488.0,
help="sustained KB/s the pipe delivers (FINDINGS 21)")
ap.add_argument("--bus", type=float, required=True,
help="REQUIRED. There is no default: the delivery rate is a property of the medium and this project has never measured it. FINDINGS 42.1 -- the figure this tool used to default to was a user-supplied '4 Mbps' with no provenance, was a tenth of SCSI-1's asynchronous rating, and was never a bus measurement at all. A default let every table in FINDINGS 30-49 be scored against it without anyone restating it. Pass one explicitly.")
ap.add_argument("--fps", type=float, default=12.0)
a = ap.parse_args()
if not os.path.exists(a.container):
tools/analysis/14_dmac_chain.py
+20 -9
View File
@@ -1,7 +1,7 @@
#!/usr/bin/env python3
"""Would letting the HD63450 paint the spans beat letting the 68000 do it?
python3 tools/analysis/14_dmac_chain.py [container.dlx] [--bus 488]
python3 tools/analysis/14_dmac_chain.py [container.dlx] --bus <KB/s>
[--dma-px-bus 2] [--disk-bus-byte 1]
FINDINGS 29.6 called this the one lever that could move the CPU budget without
@@ -11,7 +11,7 @@ prices the two against each other, and the answer turns on a resource neither
section costed: the 68000's own LOCAL BUS.
FINDINGS 29's "the bus has 4x the headroom the CPU has" is about the SCSI pipe,
110 KB/s of 488. That is a different bus. The 68000's memory bus runs one 4-clock
110 KB/s of the delivery pipe. That is a different bus. The 68000's memory bus runs one 4-clock
cycle at a time and carries instruction prefetch as well as data, and
tools/analysis/15_bus_occupancy.py measures the decoder using 86.7% of it.
@@ -65,17 +65,28 @@ SPAN_BYTES_PX, SPAN_HDR = 2, 6
ap = argparse.ArgumentParser()
ap.add_argument("container", nargs="?", default="tmp/rc_fr_singe_scsi_cpufit.dlx")
ap.add_argument("--bus", type=float, default=488.0, help="SCSI pipe, KB/s")
ap.add_argument("--bus", type=float, required=True,
help="REQUIRED. There is no default: the delivery rate is a property of the medium and this project has never measured it. FINDINGS 42.1 -- the figure this tool used to default to was a user-supplied '4 Mbps' with no provenance, was a tenth of SCSI-1's asynchronous rating, and was never a bus measurement at all. A default let every table in FINDINGS 30-49 be scored against it without anyone restating it. Pass one explicitly.")
ap.add_argument("--fps", type=float, default=12.0)
ap.add_argument("--dma-px-clk", type=float, default=B.DMA_PX_CLK,
help="clocks the DMAC spends per pixel, dual-address word "
"between two 16-bit ports. 9 is the DATASHEET figure "
"(MC68450 Fig 4-25 sheet 4).")
ap.add_argument("--disk-clk-word", type=float, default=8.0,
help="clocks the SCSI DMA steals per word. The datasheet "
"brackets it at 5 (DMAC holds the bus) to 12 (arbitrates "
"per word); FINDINGS 5's estimate of 8 is the midpoint.")
ap.add_argument("--disk-clk-byte", type=float, default=5.0,
help="clocks the SCSI DMA steals per BYTE delivered. The SPC is "
"an 8-bit port, so the DMAC pays per byte, not per word "
"(FINDINGS 43). 5, the default, is the OPTIMISTIC end and "
"what ratectl encodes against: single-address, bus held, no "
"drive wait (Fig 4-25 sheet 2). 9 is dual-address, which is "
"what MAME models and what applies if the board does not "
"drive DACK. Score both.")
ap.add_argument("--disk-clk-word", type=float, default=None,
help="DEPRECATED denominator of FINDINGS 39.7/42, kept so the "
"old tables reproduce: sets --disk-clk-byte to half this")
a = ap.parse_args()
if a.disk_clk_word is not None:
a.disk_clk_byte = a.disk_clk_word / 2.0
if not os.path.exists(a.container):
sys.exit(f"missing {a.container}")
@@ -154,7 +165,7 @@ def score(design):
for k, c in BLK_C.items():
cpu += (mm == k).sum() * c
pref, data = B.block_bus(m, spanned)
disk = byt / 2.0 * a.disk_clk_word
disk = byt * a.disk_clk_byte
# additive: CPU work, then span painting, then the disk stealing the bus
out.append((cpu + span_clk + disk, (pref + data) * B.BUS_CLK, byt,
spanned.sum()))
@@ -192,7 +203,7 @@ row("frames missing", lambda v: f"{v}/{d.nframes}",
row("blocks spanned/frame", lambda v: f"{v:,.0f}", lambda r: r[3].mean())
print(f"\n ADDITIVE: frame = CPU + span painting + disk DMA. The 68000 has no"
f"\n cache and a two-word prefetch queue, so it stalls the moment another"
f"\n master takes the bus. Disk debited at {a.disk_clk_word:g} clocks/word.")
f"\n master takes the bus. Disk debited at {a.disk_clk_byte:g} clocks/byte.")
# What is left of the case, isolated.
v6m = int((res["v6 span"][0] > FRAME_CYC).sum())
tools/analysis/15_bus_occupancy.py
+62
View File
@@ -140,3 +140,65 @@ print(f"\nprefetch is {100*pref_t.sum()/tot.sum():.0f}% of the decoder's bus tra
print(f"A DMAC painting spans at 8 clocks (2 bus cycles) per pixel could use at\n"
f"most {free.mean()/2:,.0f} pixels' worth of the mean frame's spare slots "
f"-- against {d.nb*16:,} pixels\nin a whole screen.")
# ---------------------------------------------------------------------------
# THE OTHER TWO MASTERS. Everything above is the 68000's own traffic, and it
# was the whole of this tool until session 20. The frame also has to carry the
# bitstream in off the disk and a byte of ADPCM out to $E92003 every 128 us,
# and neither has ever appeared in a bus figure -- FINDINGS 35's lesson, which
# was about the CLOCK budget, had never been applied to the BUS one.
#
# The DMAC does not overlap with the CPU (buscost.DMA_OVERLAPS = False): the
# 68000 has no cache and a two-word prefetch queue that empties at once, so a
# stolen bus cycle is a stopped CPU. The three demands therefore ADD.
#
# Audio's per-byte figure is SETTLED, not bracketed by taste:
# tools/analysis/21_iplrom_dmac.py reads the IPL ROM's own HD63450 setup and
# finds channel 3 dual-address, 8-bit port, cycle steal without hold, external
# request -- one arbitration per byte, no burst. Video's is NOT settled: it is
# ROADMAP B3 / FINDINGS 42.4-42.6's W, so it is swept rather than picked.
print("\n" + "=" * 72)
print("THE OTHER TWO MASTERS -- what the DMAC takes out of the same frame\n")
FPS = d.fps
CPUHZ = 10e6 # stock X68000, MAME 0.277 x68k.cpp:1133
FRAME_CLK = CPUHZ / FPS
vid_bpf = sum(n + 4 for (_, n) in d.frames[:NF]) / NF # DLX2 record padding
aud_bpf = B.ADPCM_BYTES_PER_S / FPS
a_lo = aud_bpf * B.ADPCM_CLK_BYTE_BEST
a_hi = aud_bpf * B.ADPCM_CLK_BYTE_WORST
cpu_clk = cyc_t.mean() if cyc_t.any() else float("nan")
print(f"frame period at {FPS:g} fps on a 10 MHz 68000: {FRAME_CLK:,.0f} clocks")
if cyc_t.any():
print(f" decoder, MEASURED (C68K) {cpu_clk:>10,.0f} clk "
f"{100*cpu_clk/FRAME_CLK:5.1f}% worst frame "
f"{100*cyc_t.max()/FRAME_CLK:.1f}%")
print(f" audio DMA, {aud_bpf:,.1f} B/frame {a_lo:>10,.0f} clk "
f"{100*a_lo/FRAME_CLK:5.2f}% .. {a_hi:,.0f} clk "
f"({100*a_hi/FRAME_CLK:.2f}%)")
print(f" {B.ADPCM_CLK_BYTE_BEST}..{B.ADPCM_CLK_BYTE_WORST} clk/byte, "
f"from the ROM's own DCR/OCR (21_iplrom_dmac.py). NOT a guess, and\n"
f" not the disk's rate: audio arbitrates for the bus once per byte "
f"and cannot burst.")
print(f"\n video DMA, {vid_bpf:,.0f} B/frame, swept over W -- ROADMAP B3 is "
f"still open:")
print(f" {'W (clk/byte)':<16}{'clk/frame':>12}{'% of frame':>12} "
f"{'CPU+audio+video':>18}")
for W, note in ((5.0, "single address, bus held (11_cpu_budget.py default)"),
(8.0, "FINDINGS 5's long-standing per-word ESTIMATE"),
(12.0, "single address, arbitrated per byte"),
(16.0, "what the ROM programs for SASI (best case)"),
(19.0, "what the ROM programs for SASI (worst case)")):
v = vid_bpf * W
tot_clk = (cpu_clk if cyc_t.any() else 0) + a_lo + v
print(f" {W:<16.0f}{v:>12,.0f}{100*v/FRAME_CLK:>11.1f}% "
f"{100*tot_clk/FRAME_CLK:>17.1f}% {note}")
print(f"\n (the last column adds the MEASURED mean decode and the BEST-CASE "
f"audio, so it is\n the optimistic end of every row. 100% is the frame "
f"deadline at {FPS:g} fps.)")
print(f"""
Audio is {100*a_lo/FRAME_CLK:.2f}%..{100*a_hi/FRAME_CLK:.2f}% of the frame and video is {vid_bpf*5/FRAME_CLK*100:.0f}%..{vid_bpf*19/FRAME_CLK*100:.0f}%. The unpriced audio
stream was never the risk P6 called it -- ON THE BUS. What the same reading of
the ROM found is that the DISK's per-byte cost has a worked example on this
machine, it is 16..19 clocks, and at that price this design does not fit at any
container size. W is the number to attack, and it is a PLAYER decision.""")
tools/analysis/16_span_roundtrip.py
+4 -3
View File
@@ -2,7 +2,7 @@
"""GATE for the DLX3 span container: does the reference decoder reproduce the
encoder's own reconstruction, from the emitted bytes?
python3 tools/analysis/16_span_roundtrip.py [frames_dir] [--kbps 488]
python3 tools/analysis/16_span_roundtrip.py [frames_dir] --kbps <KB/s>
Exits non-zero if any frame differs by a single pixel.
@@ -23,7 +23,7 @@ generated artefact, not from an assumption. So the thresholds below are
asserted, not printed.
The `--kbps` default is the BUS rate, not the `scsi` profile's 280: spans are
bought with bytes, and 14_dmac_chain.py scores them against the 488 KB/s pipe.
bought with bytes, and 14_dmac_chain.py scores them against the delivery pipe.
At the profile rate the lam search has already spent the allowance and there is
nothing left to buy a span with -- which is a real finding about the encoder
(FINDINGS 41.2), not a reason for the gate to test nothing.
@@ -36,7 +36,8 @@ from dlx import DLX
ap = argparse.ArgumentParser()
ap.add_argument("frames_dir", nargs="?", default="tmp/fr_singe")
ap.add_argument("--kbps", type=float, default=488.0)
ap.add_argument("--kbps", type=float, required=True,
help="REQUIRED. There is no default: the delivery rate is a property of the medium and this project has never measured it. FINDINGS 42.1 -- the figure this tool used to default to was a user-supplied '4 Mbps' with no provenance, was a tenth of SCSI-1's asynchronous rating, and was never a bus measurement at all. A default let every table in FINDINGS 30-49 be scored against it without anyone restating it. Pass one explicitly.")
ap.add_argument("--out", default="tmp/s12_roundtrip")
ap.add_argument("--cache", default=None)
a = ap.parse_args()
tools/analysis/17_span_delivered.py
+19 -7
View File
@@ -1,7 +1,7 @@
#!/usr/bin/env python3
"""What do the spans the ENCODER actually emitted cost, and what do they buy?
python3 tools/analysis/17_span_delivered.py a.dlx [b.dlx ...] [--bus 488]
python3 tools/analysis/17_span_delivered.py a.dlx [b.dlx ...] --bus <KB/s>
Every span figure before this one -- FINDINGS 29 through 40, and
tools/analysis/12 and 14 -- was scored by SIMULATING span selection over mode
@@ -40,12 +40,24 @@ AUDIO_KBPS = 7.8
ap = argparse.ArgumentParser()
ap.add_argument("containers", nargs="+")
ap.add_argument("--bus", type=float, default=488.0, help="SCSI pipe, KB/s")
ap.add_argument("--bus", type=float, required=True,
help="REQUIRED. There is no default: the delivery rate is a property of the medium and this project has never measured it. FINDINGS 42.1 -- the figure this tool used to default to was a user-supplied '4 Mbps' with no provenance, was a tenth of SCSI-1's asynchronous rating, and was never a bus measurement at all. A default let every table in FINDINGS 30-49 be scored against it without anyone restating it. Pass one explicitly.")
ap.add_argument("--fps", type=float, default=12.0)
ap.add_argument("--disk-clk-word", type=float, default=8.0,
help="clocks the SCSI DMA steals per word (FINDINGS 39.7 "
"brackets it at 5..12; 8 is the midpoint)")
ap.add_argument("--disk-clk-byte", type=float, default=5.0,
help="clocks the SCSI DMA steals per BYTE delivered. The SPC is "
"an 8-bit port, so the DMAC pays per byte, not per word "
"(FINDINGS 43). 5, the default, is the OPTIMISTIC end and "
"what ratectl encodes against: single-address, bus held, no "
"drive wait (Fig 4-25 sheet 2). 9 is dual-address, which is "
"what MAME models and what applies if the board does not "
"drive DACK. Score both.")
ap.add_argument("--disk-clk-word", type=float, default=None,
help="DEPRECATED denominator of FINDINGS 39.7/42, kept so the "
"old tables reproduce: sets --disk-clk-byte to half this")
a = ap.parse_args()
if a.disk_clk_word is not None:
a.disk_clk_byte = a.disk_clk_word / 2.0
def score(path):
@@ -57,7 +69,7 @@ def score(path):
_, n = d.frames[f]
blk = H.cycles(mode)
spc = sum(SP.clocks(len(p)) for _, _, p in sp)
disk = n / 2.0 * a.disk_clk_word
disk = n * a.disk_clk_byte
rows.append((blk, spc, disk, n, len(sp),
sum(len(p) for _, _, p in sp)))
return d, np.array(rows).T
@@ -81,7 +93,7 @@ for path in a.containers:
f"{int((tot > FRAME_CYC).sum()):>6}/{d.nframes:<3}")
print(f"\n ADDITIVE: frame = block decode + span painting + disk DMA, the model"
f"\n of 14_dmac_chain.py. Disk debited at {a.disk_clk_word:g} clocks/word "
f"\n of 14_dmac_chain.py. Disk debited at {a.disk_clk_byte:g} clocks/byte "
f"over the\n container's own byte count; CPU budget {FRAME_CYC:,.0f} "
f"clocks at {a.fps:g} fps.")
tools/analysis/18_text_plane_16col.py
+88
View File
@@ -0,0 +1,88 @@
#!/usr/bin/env python3
"""What 256 -> 16 colours actually costs, on real frames.
python3 tools/analysis/18_text_plane_16col.py [frames_dir]
FINDINGS 46.3 opened a lead and could not price it: the X68000 text plane is
4bpp planar -- 0.5 bytes/pixel against the graphics planes' 2.0 -- so a LITERAL
uncompressed 16-colour frame is 288.0 KB/s against the shipping compressed
256-colour container's 496.7 KB/s. 42% cheaper on the wire, with no decoder.
The whole lead turns on one number nobody had computed: the quality cost of 16
colours. This computes it, and it is deliberately generous to the 16-colour
side on every axis where the hardware allows it:
* PER-FRAME palettes are legitimate here. The text palette is 16 entries and
reloading it is 16 words a frame -- nothing, against a 833,333-clock budget.
The 256-colour path cannot do this: its palette is shared scene-wide
(vq.scene_palette) because the codec's codebooks are indices INTO it.
* DITHERING is free here, and only here. The tree does not dither (vq.py:32,
"cel art is flat") because dither destroys the inter-frame coherence SKIP
blocks and v7 spans are built on. A literal frame has no codec to wreck, so
Floyd-Steinberg is available to this path at zero runtime cost.
Both are measured, so the comparison cannot be accused of hobbling the option it
is testing. Reported against the 256-colour scene-palette ceiling (the tree's
existing "palette ceiling" figure) and against the shipping container's PSNR.
"""
import sys, os
sys.path.insert(0, "tools/encoder")
import numpy as np
from PIL import Image
import vq as VQ
FRAMES = sys.argv[1] if len(sys.argv) > 1 else "tmp/fr_singe"
SHIPPED_PSNR = 29.19 # docs/STATUS.md, --spans all, c=5, 496.7 KB/s
rgb = VQ.load_frames(FRAMES)
H, W = rgb[0].shape[:2]
n = len(rgb)
print(f"{FRAMES}: {n} frames, {W}x{H}")
print()
def recon_scene(colors, dither):
"""One palette for the whole scene -- what the 256 path is forced to do."""
d = Image.FLOYDSTEINBERG if dither else Image.NONE
samp = np.concatenate([r.reshape(-1, 3) for r in rgb[::3]])
ref = Image.fromarray(samp.reshape(-1, 1, 3)).quantize(
colors=colors, method=Image.MEDIANCUT, dither=Image.NONE)
pal = np.array(ref.getpalette()[:colors * 3], np.uint8).reshape(-1, 3)
return [pal[np.asarray(Image.fromarray(r).quantize(palette=ref, dither=d),
np.uint8)] for r in rgb]
def recon_perframe(colors, dither):
"""A fresh palette every frame -- what the text plane can afford."""
d = Image.FLOYDSTEINBERG if dither else Image.NONE
out = []
for r in rgb:
q = Image.fromarray(r).quantize(colors=colors, method=Image.MEDIANCUT,
dither=d)
pal = np.array(q.getpalette()[:colors * 3], np.uint8).reshape(-1, 3)
out.append(pal[np.asarray(q, np.uint8)])
return out
def report(name, recon):
per = np.array([VQ.psnr(a, b) for a, b in zip(rgb, recon)])
print(f" {name:<42s} {per.mean():6.2f} dB "
f"(min {per.min():5.2f} max {per.max():5.2f})")
return per.mean()
print("PSNR vs the 24-bit source, mean over frames:")
c256 = report("256 colours, scene palette [the tree's]", recon_scene(256, False))
report("256 colours, per-frame palette", recon_perframe(256, False))
print()
s16 = report("16 colours, scene palette", recon_scene(16, False))
p16 = report("16 colours, per-frame palette", recon_perframe(16, False))
p16d = report("16 colours, per-frame + FS dither", recon_perframe(16, True))
print()
print(f" the 16-colour ceiling is the best of those: {max(s16, p16, p16d):.2f} dB")
print(f" cost of 256 -> 16, at each side's best: "
f"{c256 - max(s16, p16, p16d):.2f} dB")
print()
print(f" for scale, the shipping container delivers {SHIPPED_PSNR:.2f} dB "
f"at 496.7 KB/s")
print(f" a 16-colour literal would deliver "
f"{max(s16, p16, p16d):.2f} dB at 288.0 KB/s")
delta = max(s16, p16, p16d) - SHIPPED_PSNR
print(f" so the text-plane path is {abs(delta):.2f} dB "
f"{'BETTER' if delta > 0 else 'WORSE'} at 58% of the bitrate")
tools/analysis/19_ring_stream.py
+275
View File
@@ -0,0 +1,275 @@
"""Ring-buffer streaming simulation, against the CONTIGUITY constraint (STATUS 3/4).
09_buffer_sim.py asked one question -- does cumulative supply ever fall behind
cumulative demand -- and answered it in BYTES. FINDINGS 21 got "zero required
prefill" out of it at 110 and 280 KB/s. That test is necessary and not
sufficient, and the missing half is the whole of STATUS item 3:
src/player/decode.s reads a frame record with a MONOTONICALLY INCREASING a0
and no bounds check anywhere. `move.l (a0)+,d0` for the length, `lea
MODEB(a0),a0` for the span section, eleven unrolled `movem.l (a0)+` chains,
`move.b (a0)+` per block index. Nothing in it can survive an address that
wraps mid-record. So the buffer does not merely need ENOUGH BYTES resident
by the deadline -- it needs the WHOLE NEXT RECORD resident and CONTIGUOUS.
Having enough bytes and having them contiguous are different conditions, and a
byte-counting simulation cannot tell them apart. This one models the ring's
addresses, not just its occupancy.
THREE WRAP POLICIES, and the point of the tool is that they are not equivalent:
split the writer wraps mid-record; the reader cannot. Requires a SHADOW of
the ring's first MAXREC bytes mirrored past its end, so any record
start can be read linearly for MAXREC bytes. Every byte landing in
that first MAXREC is written twice. Costs 68000 CLOCKS, forever, at a
rate set by MAXREC/ring -- and those clocks come out of the same
budget the decoder is already spending 77.0% of (FINDINGS 45).
aligned the writer refuses to start a record it cannot finish before the end
of the ring; it leaves a hole and restarts at 0. Costs RAM (the mean
hole) and nothing else -- no copy, no per-byte work. Needs a frame
INDEX so the fill side knows record boundaries, which a branching
laserdisc game needs anyway to seek to a branch point.
none the decoder handles the wrap itself. Priced here only to show what it
would cost: a bounds test in the block loop is inside the sequence
FINDINGS 30.4/40 fitted, so it does not cost a branch -- it costs
every span and per-block constant in the tree being re-measured.
Not simulated; see the note printed at the end.
DEADLINE MODEL, and it is the conservative one: record i must be wholly
resident when frame i's decode BEGINS. The decoder in fact reads a record
progressively over ~77% of a frame time, so a byte arriving mid-frame would in
practice be in time -- but that is a race between the DMAC's fill address and
a0, and this tool refuses to certify a design on a race it cannot see.
Fill is quantised to 512-byte SCSI blocks: a partial sector is not resident.
python3 tools/analysis/19_ring_stream.py [container ...] --kbps R [--ring KB]
`--kbps` is REQUIRED and has no default -- see the argument's help text.
"""
import sys, os, argparse
sys.path.insert(0, "tools/encoder")
import numpy as np
from dlx import DLX
import ratectl as RC
SECTOR = 512
# 5 clocks/byte for a 68000 `move.l (a0)+,(a1)+` copy: 20 clocks moves 4 bytes
# on a 16-bit bus (2 read + 2 write bus cycles at 4 clocks, plus the fetch it
# shares with the loop). Deliberately the OPTIMISTIC figure -- a movem-shaped
# copy is what the shadow would really use, and it is the same 5.0.
COPY_CLK_PER_BYTE = 5.0
CPUHZ = 10_000_000
def records(path):
"""Padded record sizes, exactly as the 68000 walks them.
prep_dlx.py rounds each record START up to 4 (FINDINGS 28.3), so the bytes
the ring must hold per frame are the padded ones, not the payload.
"""
d = DLX(path)
rec = np.array([4 + n + (-(4 + n) % 4) for _, n in d.frames], np.int64)
return d, rec
def simulate(rec, fill_per_frame, ring, policy, maxrec):
"""Address-level ring simulation. Returns a dict of results.
The ring is modelled as a write cursor and a read cursor over `ring` bytes.
Supply arrives at `fill_per_frame` bytes per frame time, sector-quantised.
Record i is due at the start of frame i.
"""
n = len(rec)
resident = 0.0 # bytes fully arrived and not yet consumed
carry = 0.0 # sub-sector remainder of the fill
wcur = 0 # write cursor within the ring
holes = [] # bytes wasted per wrap, `aligned` policy
shadow_bytes = 0 # bytes double-written, `split` policy
occ = []
prefill = 0.0
late = []
free = ring
# Required prefill is solved rather than searched: run once with an infinite
# head start to find the worst deficit, exactly as 09_buffer_sim does, then
# assert the ring can hold it.
deficit = np.maximum.accumulate(np.cumsum(rec - fill_per_frame))
prefill = float(max(0.0, deficit.max()))
for i, r in enumerate(rec):
# --- supply for this frame time, sector-quantised
avail = carry + fill_per_frame
sectors = int(avail // SECTOR)
got = sectors * SECTOR
carry = avail - got
# --- placement: does this frame's arriving data cross the ring end?
if policy == "aligned":
# The writer will not start a record it cannot finish. Charge the
# hole when the NEXT record would not fit in the tail.
if wcur + r > ring:
holes.append(ring - wcur)
wcur = 0
wcur += r
else: # split
end = wcur + r
if end > ring:
wcur = end - ring
# every byte that landed in the first MAXREC of the ring is
# mirrored into the shadow
shadow_bytes += min(wcur, maxrec)
else:
wcur = end
if wcur <= maxrec:
shadow_bytes += r
elif wcur - r < maxrec:
shadow_bytes += maxrec - (wcur - r)
resident += got
if resident + 1e-9 < r:
late.append((i, float(r - resident)))
resident -= r
occ.append(resident)
hole_mean = float(np.mean(holes)) if holes else 0.0
usable = ring - hole_mean if policy == "aligned" else ring
copy_clk = shadow_bytes * COPY_CLK_PER_BYTE / max(1, n)
return dict(prefill=prefill, late=late, occ=np.array(occ),
holes=holes, hole_mean=hole_mean, usable=usable,
shadow_bytes=shadow_bytes, copy_clk_per_frame=copy_clk,
wraps=len(holes) if policy == "aligned" else None)
def main():
ap = argparse.ArgumentParser()
ap.add_argument("containers", nargs="*",
default=["tmp/s14_d5_all1500.dlx",
"tmp/rc_fr_singe_scsi_span.dlx"])
ap.add_argument("--kbps", type=float, required=True,
help="delivered pipe, KB/s. REQUIRED, and deliberately has "
"no default: the delivery rate is a property of the "
"medium and this project has never measured it. The "
"figure that used to sit here was a user-supplied "
"'4 Mbps' with no provenance and was never a bus "
"measurement (FINDINGS 42.1); leaving it as a default "
"let table after table be scored against it without "
"anyone restating what it was.")
ap.add_argument("--ring", type=float, default=256.0,
help="ring size in KB (default 256, FINDINGS 21's sizing)")
a = ap.parse_args()
FPS = 12
print(f"ring {a.ring:.0f} KB sector {SECTOR} B "
f"audio {RC.AUDIO_KBPS} KB/s debited from the pipe\n")
for path in a.containers:
if not os.path.exists(path):
print(f"{path}: MISSING -- skipped\n"); continue
d, rec = records(path)
maxrec = int(rec.max())
ring = int(a.ring * 1024)
wire = rec.mean() * FPS / 1024 + RC.AUDIO_KBPS
print(f"=== {path}")
print(f" {d.nframes} frames @ {d.fps}fps, record bytes "
f"min {rec.min():,} median {int(np.median(rec)):,} max {maxrec:,}")
print(f" wire demand {wire:.1f} KB/s "
f"(video {rec.mean()*FPS/1024:.1f} + audio {RC.AUDIO_KBPS}), "
f"including the u32 length and the 4-byte record pad")
# A required prefill is only a startup cost if the window's MEAN demand
# is under the pipe. If the mean is over, the deficit grows for as long
# as the scene runs and the prefill this window reports is just how far
# it got in 120 frames -- no ring size fixes that, and quoting a KB
# figure for it would be the most flattering possible way to state a
# sustained overrun. FINDINGS 21's "zero prefill" never had to make
# this distinction because it ran far under the pipe it assumed.
if wire > a.kbps:
over = wire - a.kbps
print(f" !! SUSTAINED OVERRUN at the {a.kbps:.0f} KB/s pipe: "
f"demand exceeds supply by {over:.1f} KB/s on the MEAN, not "
f"on a burst.")
print(f" The deficit grows {over*1024/FPS:,.0f} B per frame "
f"for as long as the scene runs -- {over*1024*120/FPS/1024:.0f} "
f"KB over this 120-frame window, {over*60:.0f} KB per minute "
f"of play. Prefill below is where it got in 120 frames, NOT a "
f"startup cost that fixes it.")
if maxrec > ring:
print(f" !! MAXREC {maxrec:,} > ring {ring:,}: no policy works. "
f"decode.s needs one whole record contiguous.\n")
continue
# --- the requirement on the medium, which is the useful output, and
# the reason this tool takes no default rate. There is no measured
# pipe figure to score against (42.1), and the intent is to measure
# a BlueSCSI directly -- so the tool reports the THRESHOLD to
# measure against. The sweep is anchored to the container's own
# wire demand rather than to a list of fixed rates, so it stays
# meaningful for any container and privileges no constant.
print(f" {'pipe KB/s':>10} {'vs wire':>8} {'prefill KB':>11} "
f"{'records':>8} {'seek slack':>11}")
for mult in (0.90, 0.95, 1.00, 1.02, 1.05, 1.10, 1.25, 1.50, 2.00):
kbps = wire * mult
fill = (kbps - RC.AUDIO_KBPS) * 1024 / FPS
r = simulate(rec, fill, ring, "aligned", maxrec)
pf = r["prefill"]
# Branch-point seek slack, STATICALLY: with the ring FULL, how many
# frame times can the fill be zero before the next record is not
# resident? It is an upper bound and it assumes the premise that
# FINDINGS 51.3 took apart -- the ring is NOT full at a branch
# point, it is empty, and refilling it takes seconds of play. For
# the measured figure use tools/analysis/20_seek_slack.py, or the
# rig itself (tools/bench/pace_run.sh). Kept here as the ceiling
# this container's record sizes allow, which is what the rest of
# this row is about.
slack = (r["usable"] - maxrec) / rec.mean()
flag = ""
if pf + maxrec > r["usable"]:
flag = " <- does not fit the ring"
print(f" {kbps:>10.1f} {mult:>7.2f}x {pf/1024:>11.1f} "
f"{pf/rec.mean():>8.2f} {slack:>8.1f} fr{flag}")
# smallest pipe needing zero prefill, to 0.1 KB/s
lo, hi = wire, wire + 400
for _ in range(40):
mid = (lo + hi) / 2
f = (mid - RC.AUDIO_KBPS) * 1024 / FPS
if simulate(rec, f, ring, "aligned", maxrec)["prefill"] > 0:
lo = mid
else:
hi = mid
print(f" ZERO-PREFILL PIPE: {hi:.1f} KB/s "
f"({hi - wire:+.1f} KB/s over the wire demand, "
f"{100*hi/wire - 100:+.1f}%)")
print(f" ^ this is the number to measure a medium against. It is a "
f"REQUIREMENT, not a verdict.")
# --- the policy trade, at the default pipe
fill = (a.kbps - RC.AUDIO_KBPS) * 1024 / FPS
print(f" wrap policy, at pipe {a.kbps:.0f} KB/s:")
for policy in ("aligned", "split"):
r = simulate(rec, fill, ring, policy, maxrec)
if policy == "aligned":
print(f" aligned wraps {r['wraps']:3} mean hole "
f"{r['hole_mean']/1024:6.1f} KB usable ring "
f"{r['usable']/1024:6.1f} KB "
f"({100*r['usable']/ring:.1f}%) CPU cost 0")
else:
pct = 100 * r["copy_clk_per_frame"] / (CPUHZ / FPS)
print(f" split shadow {r['shadow_bytes']/1024:8.1f} KB "
f"= {r['copy_clk_per_frame']:8.0f} clk/frame = "
f"{pct:.2f}% of the frame budget, forever RAM cost 0")
print()
print("The `none` policy -- decoder wraps its own reads -- is not simulated.")
print("It has no RAM or copy cost and it is still the expensive one: the")
print("bounds test lands inside the exact instruction sequences FINDINGS")
print("30.4 and 40 fitted, so it does not cost a branch, it costs every span")
print("and per-block constant in the tree being re-measured. FINDINGS 28.3.")
if __name__ == "__main__":
main()
tools/analysis/20_seek_slack.py
+146
View File
@@ -0,0 +1,146 @@
"""Seek slack: how long a branch point can stop delivery (STATUS 4, FINDINGS 51).
19_ring_stream.py asks whether a container ARRIVES in time, and prints one
"seek slack" column derived statically as (usable ring - maxrec)/mean record.
That is a capacity estimate and it quietly assumes the ring is full when the
seek happens. It is not, and the difference is the whole finding:
A ring's slack is ACCUMULATED, not owned. It is built out of the surplus
between the pipe and the wire demand, at (pipe - wire) bytes per second, and
a seek spends all of it. How long a branch point can stall is a property of
the ring; how soon the NEXT branch point can be afforded is a property of the
surplus, and a bigger ring makes that one WORSE.
This is the paced-rig model (tools/bench/stream.lua with DLX_PACE=1) written
independently, and it exists to be compared against it, not to replace it. The
rig drives a real 68000 through a real ring and is the measurement; this is the
cheap sweep that says where to point it. Where they disagree, the rig wins.
python3 tools/analysis/20_seek_slack.py [container ...] --kbps R [R ...]
[--ring KB [KB ...]]
`--kbps` is REQUIRED and takes no default, for the reason FINDINGS 50 gives.
"""
import sys, os, argparse
sys.path.insert(0, "tools/encoder")
import numpy as np
from dlx import DLX
import ratectl as RC
SECTOR = 512
def records(path):
d = DLX(path)
rec = np.array([4 + n + (-(4 + n) % 4) for _, n in d.frames], np.int64)
return d, rec
def paced_sim(rec, ring, fill_per_frame, ticks_per_frame=8):
"""Paced-decoder ring sim. Returns TWO per-tick lookahead series.
THE ANSWER IS BRACKETED TO ONE RECORD AND IS NOT SHARPER THAN THAT. At
these rates the pipe delivers almost exactly one record per frame slot, so
"how many records are resident at slot i" depends on whether you look before
or after that slot's delivery -- and the two answers differ by one, every
time. Sampled after, this agreed with the rig's ceiling in 33 of 35 cells;
sampled before, it was exactly one record lower in 33 of 35. Neither is
wrong. Picking the one that matched would have been fitting the model to
the measurement and then reporting the agreement as a cross-check, so both
are returned and the caller prints the range. The rig sits at the top of it.
The producer is `aligned` (19_ring_stream.py): it will not start a record it
cannot finish before the end of the ring, and it will not place one over
bytes the decoder still owns. The decoder consumes exactly one record per
frame time and releases it whole.
Sub-stepping matters. Delivery and consumption interleave inside a frame
time on the rig -- the producer runs on MAME's machine-frame notifier, ~5x
per 12fps slot -- and a model that delivers a whole frame's bytes at once
can place a record into space the decoder has not released yet, or refuse
one it has. Eight sub-steps is well past the point the answer stops moving.
"""
n = len(rec)
live = [] # [idx, off, len] still owned by the decoder
wcur, nsent, credit = 0, 0, 0.0
lo, hi, ring_ref, rate_ref = [], [], 0, 0
def overlaps(off, ln):
return any(off < r[1] + r[2] and r[1] < off + ln for r in live)
for i in range(n):
if nsent < n:
lo.append(sum(1 for r in live if r[0] >= i))
for _ in range(ticks_per_frame):
credit += fill_per_frame / ticks_per_frame
while nsent < n:
r = int(rec[nsent])
if credit < r:
rate_ref += 1
break
w, hole = wcur, 0
if w + r > ring:
w, hole = 0, ring - wcur
if overlaps(w, r):
ring_ref += 1
break
# sector quantisation: a partial sector is not resident
credit -= r
live.append([nsent, w, r])
wcur, nsent = w + r, nsent + 1
if nsent < n:
hi.append(sum(1 for r in live if r[0] >= i))
# the decoder consumed record i during the slot and releases it whole
live = [r for r in live if r[0] > i]
return np.array(lo), np.array(hi), ring_ref, rate_ref
def main():
ap = argparse.ArgumentParser()
ap.add_argument("containers", nargs="*",
default=["tmp/rc_fr_singe_scsi_span.dlx"])
ap.add_argument("--kbps", type=float, nargs="+", required=True,
help="delivered pipe rates, KB/s. REQUIRED, no default "
"(FINDINGS 50): this project has never measured the "
"delivery pipe and a default is how the last unmeasured "
"one stayed load-bearing for five sessions.")
ap.add_argument("--ring", type=float, nargs="+",
default=[64, 96, 128, 192, 256, 384, 512])
a = ap.parse_args()
FPS = 12
for path in a.containers:
if not os.path.exists(path):
print(f"{path}: MISSING -- skipped\n"); continue
d, rec = records(path)
wire = rec.mean() * FPS / 1024 + RC.AUDIO_KBPS
print(f"=== {path}: {d.nframes} frames @ {d.fps}fps, mean record "
f"{rec.mean()/1024:.1f} KB, wire {wire:.1f} KB/s")
print(f"{'ring KB':>8} {'pipe':>8} {'ceiling':>9} {'build s':>8} "
f"{'mean':>11} bound")
for ring_kb in a.ring:
ring = int(ring_kb * 1024)
if rec.max() > ring:
print(f"{ring_kb:>8.0f} maxrec {rec.max():,} does not fit")
continue
for kbps in a.kbps:
fill = ((kbps - RC.AUDIO_KBPS) * 1024 / FPS) if kbps > 0 else 1e12
lo, hi, ring_ref, rate_ref = paced_sim(rec, ring, fill)
c_lo, c_hi = int(lo.max()), int(hi.max())
build = int(np.argmax(hi >= c_hi)) if len(hi) else -1
print(f"{ring_kb:>8.0f} {kbps:>8.0f} "
f"{f'{c_lo}-{c_hi}':>9} {build/FPS:>8.2f} "
f"{f'{lo.mean():.1f}-{hi.mean():.1f}':>11} "
f"{'ring' if ring_ref else 'rate'}")
# The surplus model, stated so it can be checked against the sweep
# above rather than believed: slack accrues at (pipe - wire) and a
# full ring holds `ceiling` records, so a branch point costs about
# ceiling*mean_record/(pipe - wire) seconds of play to earn back.
print()
print("Slack is accumulated, not owned. A bigger ring raises the ceiling AND")
print("lengthens the climb to it: the surplus (pipe - wire) is what fills it,")
print("and that is set by the encoder and the medium, not by the buffer.")
if __name__ == "__main__":
main()
tools/analysis/21_iplrom_dmac.py
+255
View File
@@ -0,0 +1,255 @@
#!/usr/bin/env python3
"""What the X68000's own ROM programs into the DMAC -- read out of the bytes.
python3 tools/analysis/21_iplrom_dmac.py [iplrom.dat]
FINDINGS 48.4 / ROADMAP B3 left the single-address vs dual-address question
open for the disk, priced it at 242 KB/s and 0.69 dB, and blocked it on
sourcing `scsiexrom.bin` so its DMAC init could be disassembled. The same
question was open for AUDIO and nobody had asked it: ROADMAP P6 budgets ADPCM
at 7.8 KB/s and `11_cpu_budget.py` charges those bytes the DISK's per-byte
rate, which is a guess about a channel whose configuration was never read.
It does not have to be a guess. **The IPL ROM is on this machine** -- MAME runs
the player rig with `-bios ipl10` -- and it programs all four HD63450 channels
itself. This script reads the configuration straight out of the ROM image and
decodes the MC68450 register fields, so every claim below is a byte at a named
address rather than a recollection about a chip.
It is a GATE, not a report: each piece of evidence is (address, expected bytes,
what it means), and a mismatch exits non-zero. If a different ROM revision is
pointed at it, it says so instead of quietly decoding something else.
SOURCED for the field layouts: MC68450 Direct Memory Access Controller,
Motorola, Jul 1989 (bitsavers) -- the same document FINDINGS 39 already cites
for the transfer timings in tools/analysis/buscost.py.
NOTE THE LAYER: this is the ROM's own choice of configuration, read from the
shipping image. It is not a measurement of a running machine, and it is not
proof that a different configuration is impossible -- our player programs these
registers itself. It is evidence about what Sharp's engineers could get the
board to do, from the vendor, for these exact devices.
"""
import sys, os, argparse, hashlib
BASE = 0xFE0000 # where the IPL ROM is mapped (and its 0xFF0000 alias)
# The image this was decoded against. A different revision is a different
# machine's answer, so it is named rather than assumed.
KNOWN = {
"7fd4caabac1d9169e289f0f7bbf71d8e":
"IPL 1.0 (MAME x68000 -bios ipl10), 131,072 B",
}
# --- MC68450 register map, by offset inside a channel's 0x40 block ----------
REG = {0x00: "CSR", 0x01: "CER", 0x04: "DCR", 0x05: "OCR", 0x06: "SCR",
0x07: "CCR", 0x0A: "MTC", 0x0C: "MAR", 0x14: "DAR", 0x1A: "BTC",
0x1C: "BAR", 0x25: "NIV", 0x27: "EIV", 0x29: "MFC", 0x2D: "CPR",
0x31: "DFC", 0x39: "BFC"}
XRM = {0: "burst",
1: "UNDEFINED",
2: "cycle steal WITHOUT hold (bus released between operands)",
3: "cycle steal with hold"}
DTYP = {0: "68000-compatible, EXPLICITLY addressed -> DUAL ADDRESS",
1: "6800-compatible, EXPLICITLY addressed -> DUAL ADDRESS",
2: "device with ACK, implicitly addressed -> SINGLE ADDRESS",
3: "device with ACK and RDY, implicit -> SINGLE ADDRESS"}
DPS = {0: "8-bit port", 1: "16-bit port"}
PCL = {0: "status input", 1: "status input with interrupt",
2: "start pulse", 3: "abort input"}
SIZE = {0: "byte", 1: "word", 2: "long word", 3: "byte, unpacked"}
CHAIN= {0: "none", 1: "UNDEFINED", 2: "array", 3: "linked array"}
REQG = {0: "auto-request at limited rate", 1: "auto-request at max rate",
2: "EXTERNAL request (one operand per device request)",
3: "auto-request first operand, external thereafter"}
def dcr(v):
return [f"XRM = {v>>6&3:02b} {XRM[v>>6&3]}",
f"DTYP = {v>>4&3:02b} {DTYP[v>>4&3]}",
f"DPS = {v>>3&1:b} {DPS[v>>3&1]}",
f"PCL = {v&3:02b} {PCL[v&3]}"]
def ocr(v):
return [f"DIR = {v>>7&1:b} " +
("device -> memory (read)" if v & 0x80 else "memory -> device (write)"),
f"SIZE = {v>>4&3:02b} {SIZE[v>>4&3]}",
f"CHAIN= {v>>2&3:02b} {CHAIN[v>>2&3]}",
f"REQG = {v&3:02b} {REQG[v&3]}"]
def scr(v):
m = {0: "no count", 1: "increment", 2: "decrement", 3: "UNDEFINED"}
return [f"MAC = {v>>2&3:02b} memory address {m[v>>2&3]}",
f"DAC = {v&3:02b} device address {m[v&3]}"]
# --- the evidence ----------------------------------------------------------
# (address, expected bytes, one-line description). Every register value quoted
# anywhere below comes out of one of these; nothing is typed in twice.
EV = [
(0xFF0BEA, "49f900e84080197c00080004197c0005",
"boot: lea $E84080,a4 (ch2) ; DCR=$08 ; SCR=$05..."),
(0xFF0C2E, "49f900e840c0197c00800004197c00040006197c00050029197c0001002d"
"197c00050031197c00050039297c00e92003",
"boot: lea $E840C0,a4 (ch3, ADPCM) ; DCR=$80 SCR=$04 MFC=$05 CPR=$01 "
"DFC=$05 BFC=$05 DAR=$E92003"),
(0xFF0D8E, "0480060429052d0031054480460469056d027105",
"boot: the ch0/ch1 init TABLE, ten (offset,value) pairs, written by the "
"loop at $FF0CD8"),
(0xFF0CE4, "217c00e940030014217c00e960010054",
"boot: DAR ch0 = $E94003 (FDC data) ; DAR ch1 = $E96001 (SASI data)"),
(0xFF9A82, "13fc003200e840c5610a13fc000200e920014e75",
"IOCS ADPCM PLAY: OCR(ch3) = $32 ; then command $02 to $E92001"),
(0xFF9A5E, "13fc00b200e840c5612e13fc000400e920014e75",
"IOCS ADPCM RECORD: OCR(ch3) = $B2 ; then command $04 to $E92001"),
(0xFF9A96, "13fc00ff00e840c023c900e840cc33c200e840ca",
"IOCS ADPCM arm: CSR=$FF ; MAR = a1 ; MTC = d2 (DCR/SCR untouched)"),
(0xFF9944, "13fc00ff00e8404013fc00b200e84045601013fc00ff00e8404013fc003200"
"e8404523c900e8404c33c300e8404a13fc008000e840474e75",
"IOCS SASI: OCR(ch1) = $B2 read / $32 write ; MAR ; MTC ; CCR = $80"),
]
ap = argparse.ArgumentParser()
ap.add_argument("rom", nargs="?",
default=os.path.expanduser("~/mame/roms/iplrom.dat"))
a = ap.parse_args()
if not os.path.exists(a.rom):
sys.exit(f"missing {a.rom} -- point this at the IPL ROM MAME boots the rig "
f"with (-bios ipl10).")
d = open(a.rom, "rb").read()
md5 = hashlib.md5(d).hexdigest()
print(f"{a.rom}: {len(d):,} B, md5 {md5}")
if md5 in KNOWN:
print(f" {KNOWN[md5]}\n")
else:
sys.exit(f"\nUNKNOWN ROM. Every field decoded below was read out of\n"
f" {list(KNOWN.values())[0]}\n"
f"and a different revision is a different machine's answer, not a "
f"detail. Add its\nmd5 to KNOWN only after re-reading the sites -- "
f"the addresses are revision-specific.")
print("EVIDENCE -- each line is bytes at an address, not a recollection")
bad = 0
for addr, hx, what in EV:
want = bytes.fromhex(hx)
got = d[addr - BASE: addr - BASE + len(want)]
ok = got == want
bad += not ok
print(f" {'OK ' if ok else 'FAIL'} ${addr:06X} {what}")
if not ok:
print(f" expected {want.hex()}\n got {got.hex()}")
if bad:
sys.exit(f"\nFAIL: {bad} evidence site(s) do not hold. The decode below "
"would be about\nsome other code, so it is not printed.")
# The ch0/ch1 table, decoded from the bytes rather than restated.
tbl = d[0xFF0D8E - BASE: 0xFF0D8E - BASE + 20]
init = {}
for i in range(0, len(tbl), 2):
off, val = tbl[i], tbl[i + 1]
init[(off >> 6, off & 0x3F)] = val
init[(2, 0x04)] = 0x08 # from the inline moves at $FF0BEA
init[(2, 0x06)] = 0x05
init[(2, 0x2D)] = 0x03
init[(3, 0x04)] = 0x80 # ...and at $FF0C2E
init[(3, 0x06)] = 0x04
init[(3, 0x2D)] = 0x01
DEV = {0: ("FDC", "$E94003"), 1: ("SASI", "$E96001"),
2: ("IOCS _DMAMOVE (general purpose)", "set per call"),
3: ("ADPCM MSM6258V", "$E92003")}
print("\nWHAT THE ROM PROGRAMS, per channel")
for ch in range(4):
name, dar = DEV[ch]
print(f"\n ch{ch} base $E840{ch*0x40:02X} {name} DAR = {dar}")
v = init[(ch, 0x04)]
print(f" DCR = ${v:02X}")
for line in dcr(v):
print(f" {line}")
v = init[(ch, 0x06)]
print(f" SCR = ${v:02X} " + " ; ".join(scr(v)))
print(f" CPR = ${init[(ch,0x2D)]:02X} channel priority "
f"({init[(ch,0x2D)]}, 0 = highest)")
print("\nAND THE PER-TRANSFER OCR, written every time a transfer is armed")
for label, ch, v in (("ADPCM playback", 3, 0x32), ("ADPCM record", 3, 0xB2),
("SASI write", 1, 0x32), ("SASI read", 1, 0xB2)):
print(f"\n {label:<15} ch{ch} OCR = ${v:02X}")
for line in ocr(v):
print(f" {line}")
print(f"""
WHAT THIS SETTLES
1. AUDIO IS DUAL ADDRESS, AND IT CANNOT HOLD THE BUS. ch3 DCR = $80: DTYP =
00, explicitly addressed, so every ADPCM byte is a MEMORY READ FOLLOWED BY A
DEVICE WRITE -- not the single-address 5 clocks the disk debit is written in.
XRM = 10 is cycle steal WITHOUT hold and OCR REQG = 10 is external request,
so the DMAC arbitrates for the bus ONCE PER BYTE and gives it straight back.
There is no burst to amortise the arbitration over.
2. THE PORT IS 8 BITS AND THE OPERAND IS A BYTE. DCR DPS = 0, OCR SIZE = 11.
One MSM6258V byte is two 4-bit samples, so 15.6 kHz is 7,812.5 BYTES/s and
7,812.5 DMA REQUESTS/s -- the request count does not halve the way a 16-bit
port's would. That is the FINDINGS 43 unit trap, in the other stream.
3. THE DISK CHANNEL IS PROGRAMMED IDENTICALLY, AND THAT IS THE BIGGER NEWS.
ch1 (SASI, DAR = $E96001) gets DCR = $80 and OCR = $B2 -- dual address,
8-bit port, cycle steal WITHOUT hold, external request. Byte by byte, with a
full arbitration each time, exactly like the audio. ch0 (FDC) too. Sharp
programs every explicitly-addressed 8-bit device on this board the same way.
This is not scsiexrom.bin and it does not close ROADMAP B3 -- a different
ROM drives a different SPC. But it is the same vendor, the same DMAC and the
same class of device, and it lands on the EXPENSIVE side of B3's 242 KB/s.
4. AND IT IS OUTSIDE THE BRACKET THE PROJECT HAS BEEN COSTING P4 IN.
FINDINGS 42.4-42.6 brackets W, the clocks stolen per delivered byte, at
5..12, and reports that W <= 6 fits 0/120 frames while W = 8 misses 47/120.
The ROM's own disk configuration costs 16..19. It is still true that the
player programs these registers itself and the choice is ours (42.6) -- but
the only worked example on the machine sits ABOVE the whole bracket, and
nothing in this tree has yet shown that a cheaper configuration is reachable
for an explicitly-addressed port. Treat W <= 12 as a REQUIREMENT ON THE
PLAYER'S DMAC PROGRAMMING, not as a range the hardware hands us.
5. AUDIO OUTRANKS THE DISK AT THE ARBITER. CPR: FDC 0, ADPCM 1, SASI 2,
_DMAMOVE 3, lower being higher priority. When both channels want the bus in
the same slot, the ROM's arrangement serves ADPCM first. An audio byte is
never the thing that waits; a video byte is.
""")
# --- what it costs ---------------------------------------------------------
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import buscost as B
ADPCM_HZ = 15625.0 # 8 MHz MSM6258V clock / 512
ADPCM_BPS = ADPCM_HZ / 2 # 4-bit samples, two to a byte
FPS, CPUHZ = 12.0, 10e6
lo = B.DMA_DUAL_BYTE_CLK + B.DMA_FRONT_CLK + B.DMA_BACK_CLK
hi = B.DMA_DUAL_BYTE_CLK + B.DMA_FRONT_CLK_WORST + B.DMA_BACK_CLK
bpf = ADPCM_BPS / FPS
print(f"WHAT IT COSTS, at the configuration above\n"
f" 15.6 kHz mono = {ADPCM_HZ:,.0f} samples/s = {ADPCM_BPS:,.1f} B/s "
f"= {ADPCM_BPS/1024:.2f} KiB/s\n"
f" (ratectl.AUDIO_KBPS is 7.8, which is this figure in DECIMAL kB; "
f"as KiB it is {ADPCM_BPS/1024:.2f})\n"
f" dual-address byte transfer {B.DMA_DUAL_BYTE_CLK} clk "
f"(read {B.DMA_READ_CLK} + write {B.DMA_WRITE_CLK}, Fig 4-25 sheet 4 note 2)\n"
f" + arbitration, EVERY byte {B.DMA_FRONT_CLK}..{B.DMA_FRONT_CLK_WORST}"
f" front + {B.DMA_BACK_CLK} back (sect 4.5.2.1/4.5.2.2)\n"
f" = {lo}..{hi} clocks per audio byte\n\n"
f" per frame at {FPS:g} fps: {bpf:,.1f} B costs {bpf*lo:,.0f}..{bpf*hi:,.0f} "
f"clocks of {CPUHZ/FPS:,.0f}\n"
f" = {100*bpf*lo/(CPUHZ/FPS):.2f}%..{100*bpf*hi/(CPUHZ/FPS):.2f}% of the "
f"frame, stolen from the 68000\n\n"
f" 11_cpu_budget.py charges audio --dma-clocks-per-byte, default 5, "
f"described\n as 'single-address, bus held, no drive wait'. The ROM says "
f"audio is neither\n single-address nor able to hold the bus, so that "
f"debit is {lo/5:.1f}x..{hi/5:.1f}x too small.\n"
f" In absolute terms it is small -- but it is small IN THE RESOURCE THE "
f"PROJECT IS\n SHORT OF, and it was being taken from the wrong side of "
f"an open question.")
tools/analysis/buscost.py
+25
View File
@@ -63,12 +63,37 @@ DMA_CHAIN_CLK = 36
# Sect 4.5.2.1 front-end overhead 5 clocks best case, 8 worst; 4.5.2.2
# back-end 2 clocks best. Once per period of bus ownership, not per span.
DMA_FRONT_CLK, DMA_BACK_CLK = 5, 2
DMA_FRONT_CLK_WORST = 8
# Fig 4-25 note 2 again, split out because the ADPCM channel needs the halves
# apart: a DMAC READ is 4 clocks and a WRITE is 5, on either bus width. A
# dual-address BYTE transfer is therefore one 4 and one 5.
DMA_READ_CLK, DMA_WRITE_CLK = 4, 5
DMA_DUAL_BYTE_CLK = DMA_READ_CLK + DMA_WRITE_CLK
# Fig 4-25 sheet 3, SINGLE ADDRESS: W/B READ 4 clocks, W/B WRITE 5 clocks.
# A device->memory disk transfer is one memory WRITE = 5 clocks if the DMAC
# holds the bus, or 5 + front + back = 12 if it arbitrates per word.
# FINDINGS 5's long-standing 8 clk/word ESTIMATE sits inside that range.
DMA_DISK_CLK_WORD_HELD, DMA_DISK_CLK_WORD_ARB = 5, 12
# --- the ADPCM stream, as the IPL ROM actually programs it -----------------
# READ OUT OF THE ROM, not recalled: tools/analysis/21_iplrom_dmac.py decodes
# the HD63450 registers Sharp's own IPL 1.0 writes, and gates on the bytes still
# being there. Channel 3, DCR = $80, OCR = $32 for playback:
#
# DTYP = 00 explicitly addressed -> DUAL ADDRESS (memory read, device write)
# DPS = 0 8-bit port -> one byte per operand
# XRM = 10 cycle steal WITHOUT hold, and REQG = 10 external request
# -> the DMAC arbitrates ONCE PER BYTE. No burst to amortise over.
#
# So an audio byte costs the dual-address transfer PLUS a full arbitration,
# every time -- unlike a disk record, which can at least be argued to hold the
# bus for a run of bytes. This is the number the audio side of the I/O debit
# should be denominated in; DISK_CLK_BYTE is not it.
ADPCM_SAMPLE_HZ = 15625.0 # MSM6258V, 8 MHz clock / 512 (the 15.6 kHz mode)
ADPCM_BYTES_PER_S = ADPCM_SAMPLE_HZ / 2 # 4-bit samples, two to a byte
ADPCM_CLK_BYTE_BEST = DMA_DUAL_BYTE_CLK + DMA_FRONT_CLK + DMA_BACK_CLK # 16
ADPCM_CLK_BYTE_WORST = DMA_DUAL_BYTE_CLK + DMA_FRONT_CLK_WORST + DMA_BACK_CLK # 19
# The 68000 cannot execute while another master owns the bus: no cache, and a
# two-word prefetch queue that empties immediately. So DMA time is ADDITIVE to
# CPU time, not overlapped -- which is what FINDINGS 35's flat debit assumed