prosolis
1be428c270
Two sessions, unrecorded until now, committed together because their edits share files and cannot be split cleanly after the fact. Session 28 (FINDINGS 60): the container is DLX5 -- every record sector-aligned, 120/120 starting on a boundary where 3/120 did, +0.48% on the wire and zero clocks -- and the ring's release rounds to RECALN so no pad is stranded. Two encoder levers measured and refused: `--spans all` buys +0.19 dB for +67% of the wire, and joint span/lam selection emits byte-identical containers because `lam` never leaves its floor on any of 120 frames. Session 29 (FINDINGS 61): the packed full-frame blit is 27.3% of a 12 fps frame, a channel fills GVRAM in buffer mode off the disc with the CPU halted, and it walks the 1,024 B line stride itself through array chaining. At the 9 clk/B dual-address floor the codec is 110.4% of a frame and a decoder-free packed literal player is 55.2%, at +4.89 dB -- 2.75 dB past a ceiling the codec's scene-wide palette cannot cross. Encoder work is parked; the codec is kept and not built on. check.sh is ALL GREEN before and after, plus one new stage that gates the ORDER of the measured paint costs rather than their values. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
276 lines
13 KiB
Python
276 lines
13 KiB
Python
"""Ring-buffer streaming simulation, against the CONTIGUITY constraint (STATUS 3/4).
|
|
|
|
09_buffer_sim.py asked one question -- does cumulative supply ever fall behind
|
|
cumulative demand -- and answered it in BYTES. FINDINGS 21 got "zero required
|
|
prefill" out of it at 110 and 280 KB/s. That test is necessary and not
|
|
sufficient, and the missing half is the whole of STATUS item 3:
|
|
|
|
src/player/decode.s reads a frame record with a MONOTONICALLY INCREASING a0
|
|
and no bounds check anywhere. `move.l (a0)+,d0` for the length, `lea
|
|
MODEB(a0),a0` for the span section, eleven unrolled `movem.l (a0)+` chains,
|
|
`move.b (a0)+` per block index. Nothing in it can survive an address that
|
|
wraps mid-record. So the buffer does not merely need ENOUGH BYTES resident
|
|
by the deadline -- it needs the WHOLE NEXT RECORD resident and CONTIGUOUS.
|
|
|
|
Having enough bytes and having them contiguous are different conditions, and a
|
|
byte-counting simulation cannot tell them apart. This one models the ring's
|
|
addresses, not just its occupancy.
|
|
|
|
THREE WRAP POLICIES, and the point of the tool is that they are not equivalent:
|
|
|
|
split the writer wraps mid-record; the reader cannot. Requires a SHADOW of
|
|
the ring's first MAXREC bytes mirrored past its end, so any record
|
|
start can be read linearly for MAXREC bytes. Every byte landing in
|
|
that first MAXREC is written twice. Costs 68000 CLOCKS, forever, at a
|
|
rate set by MAXREC/ring -- and those clocks come out of the same
|
|
budget the decoder is already spending 77.0% of (FINDINGS 45).
|
|
|
|
aligned the writer refuses to start a record it cannot finish before the end
|
|
of the ring; it leaves a hole and restarts at 0. Costs RAM (the mean
|
|
hole) and nothing else -- no copy, no per-byte work. Needs a frame
|
|
INDEX so the fill side knows record boundaries, which a branching
|
|
laserdisc game needs anyway to seek to a branch point.
|
|
|
|
none the decoder handles the wrap itself. Priced here only to show what it
|
|
would cost: a bounds test in the block loop is inside the sequence
|
|
FINDINGS 30.4/40 fitted, so it does not cost a branch -- it costs
|
|
every span and per-block constant in the tree being re-measured.
|
|
Not simulated; see the note printed at the end.
|
|
|
|
DEADLINE MODEL, and it is the conservative one: record i must be wholly
|
|
resident when frame i's decode BEGINS. The decoder in fact reads a record
|
|
progressively over ~77% of a frame time, so a byte arriving mid-frame would in
|
|
practice be in time -- but that is a race between the DMAC's fill address and
|
|
a0, and this tool refuses to certify a design on a race it cannot see.
|
|
|
|
Fill is quantised to 512-byte SCSI blocks: a partial sector is not resident.
|
|
|
|
python3 tools/analysis/19_ring_stream.py [container ...] --kbps R [--ring KB]
|
|
|
|
`--kbps` is REQUIRED and has no default -- see the argument's help text.
|
|
"""
|
|
import sys, os, argparse
|
|
sys.path.insert(0, "tools/encoder")
|
|
import numpy as np
|
|
from dlx import DLX
|
|
import ratectl as RC
|
|
|
|
SECTOR = 512
|
|
# 5 clocks/byte for a 68000 `move.l (a0)+,(a1)+` copy: 20 clocks moves 4 bytes
|
|
# on a 16-bit bus (2 read + 2 write bus cycles at 4 clocks, plus the fetch it
|
|
# shares with the loop). Deliberately the OPTIMISTIC figure -- a movem-shaped
|
|
# copy is what the shadow would really use, and it is the same 5.0.
|
|
COPY_CLK_PER_BYTE = 5.0
|
|
CPUHZ = 10_000_000
|
|
|
|
|
|
def records(path):
|
|
"""Padded record sizes, exactly as the 68000 walks them.
|
|
|
|
prep_dlx.py rounds each record START up to 4 (FINDINGS 28.3), so the bytes
|
|
the ring must hold per frame are the padded ones, not the payload.
|
|
"""
|
|
d = DLX(path)
|
|
rec = np.array(d.record_lengths(), np.int64)
|
|
return d, rec
|
|
|
|
|
|
def simulate(rec, fill_per_frame, ring, policy, maxrec):
|
|
"""Address-level ring simulation. Returns a dict of results.
|
|
|
|
The ring is modelled as a write cursor and a read cursor over `ring` bytes.
|
|
Supply arrives at `fill_per_frame` bytes per frame time, sector-quantised.
|
|
Record i is due at the start of frame i.
|
|
"""
|
|
n = len(rec)
|
|
resident = 0.0 # bytes fully arrived and not yet consumed
|
|
carry = 0.0 # sub-sector remainder of the fill
|
|
wcur = 0 # write cursor within the ring
|
|
holes = [] # bytes wasted per wrap, `aligned` policy
|
|
shadow_bytes = 0 # bytes double-written, `split` policy
|
|
occ = []
|
|
prefill = 0.0
|
|
late = []
|
|
free = ring
|
|
|
|
# Required prefill is solved rather than searched: run once with an infinite
|
|
# head start to find the worst deficit, exactly as 09_buffer_sim does, then
|
|
# assert the ring can hold it.
|
|
deficit = np.maximum.accumulate(np.cumsum(rec - fill_per_frame))
|
|
prefill = float(max(0.0, deficit.max()))
|
|
|
|
for i, r in enumerate(rec):
|
|
# --- supply for this frame time, sector-quantised
|
|
avail = carry + fill_per_frame
|
|
sectors = int(avail // SECTOR)
|
|
got = sectors * SECTOR
|
|
carry = avail - got
|
|
|
|
# --- placement: does this frame's arriving data cross the ring end?
|
|
if policy == "aligned":
|
|
# The writer will not start a record it cannot finish. Charge the
|
|
# hole when the NEXT record would not fit in the tail.
|
|
if wcur + r > ring:
|
|
holes.append(ring - wcur)
|
|
wcur = 0
|
|
wcur += r
|
|
else: # split
|
|
end = wcur + r
|
|
if end > ring:
|
|
wcur = end - ring
|
|
# every byte that landed in the first MAXREC of the ring is
|
|
# mirrored into the shadow
|
|
shadow_bytes += min(wcur, maxrec)
|
|
else:
|
|
wcur = end
|
|
if wcur <= maxrec:
|
|
shadow_bytes += r
|
|
elif wcur - r < maxrec:
|
|
shadow_bytes += maxrec - (wcur - r)
|
|
|
|
resident += got
|
|
if resident + 1e-9 < r:
|
|
late.append((i, float(r - resident)))
|
|
resident -= r
|
|
occ.append(resident)
|
|
|
|
hole_mean = float(np.mean(holes)) if holes else 0.0
|
|
usable = ring - hole_mean if policy == "aligned" else ring
|
|
copy_clk = shadow_bytes * COPY_CLK_PER_BYTE / max(1, n)
|
|
return dict(prefill=prefill, late=late, occ=np.array(occ),
|
|
holes=holes, hole_mean=hole_mean, usable=usable,
|
|
shadow_bytes=shadow_bytes, copy_clk_per_frame=copy_clk,
|
|
wraps=len(holes) if policy == "aligned" else None)
|
|
|
|
|
|
def main():
|
|
ap = argparse.ArgumentParser()
|
|
ap.add_argument("containers", nargs="*",
|
|
default=["tmp/s14_d5_all1500.dlx",
|
|
"tmp/rc_fr_singe_scsi_span.dlx"])
|
|
ap.add_argument("--kbps", type=float, required=True,
|
|
help="delivered pipe, KB/s. REQUIRED, and deliberately has "
|
|
"no default: the delivery rate is a property of the "
|
|
"medium and this project has never measured it. The "
|
|
"figure that used to sit here was a user-supplied "
|
|
"'4 Mbps' with no provenance and was never a bus "
|
|
"measurement (FINDINGS 42.1); leaving it as a default "
|
|
"let table after table be scored against it without "
|
|
"anyone restating what it was.")
|
|
ap.add_argument("--ring", type=float, default=256.0,
|
|
help="ring size in KB (default 256, FINDINGS 21's sizing)")
|
|
a = ap.parse_args()
|
|
|
|
FPS = 12
|
|
print(f"ring {a.ring:.0f} KB sector {SECTOR} B "
|
|
f"audio {RC.AUDIO_KBPS} KB/s debited from the pipe\n")
|
|
|
|
for path in a.containers:
|
|
if not os.path.exists(path):
|
|
print(f"{path}: MISSING -- skipped\n"); continue
|
|
d, rec = records(path)
|
|
maxrec = int(rec.max())
|
|
ring = int(a.ring * 1024)
|
|
wire = rec.mean() * FPS / 1024 + RC.AUDIO_KBPS
|
|
|
|
print(f"=== {path}")
|
|
print(f" {d.nframes} frames @ {d.fps}fps, record bytes "
|
|
f"min {rec.min():,} median {int(np.median(rec)):,} max {maxrec:,}")
|
|
print(f" wire demand {wire:.1f} KB/s "
|
|
f"(video {rec.mean()*FPS/1024:.1f} + audio {RC.AUDIO_KBPS}), "
|
|
f"including the u32 length and the 4-byte record pad")
|
|
|
|
# A required prefill is only a startup cost if the window's MEAN demand
|
|
# is under the pipe. If the mean is over, the deficit grows for as long
|
|
# as the scene runs and the prefill this window reports is just how far
|
|
# it got in 120 frames -- no ring size fixes that, and quoting a KB
|
|
# figure for it would be the most flattering possible way to state a
|
|
# sustained overrun. FINDINGS 21's "zero prefill" never had to make
|
|
# this distinction because it ran far under the pipe it assumed.
|
|
if wire > a.kbps:
|
|
over = wire - a.kbps
|
|
print(f" !! SUSTAINED OVERRUN at the {a.kbps:.0f} KB/s pipe: "
|
|
f"demand exceeds supply by {over:.1f} KB/s on the MEAN, not "
|
|
f"on a burst.")
|
|
print(f" The deficit grows {over*1024/FPS:,.0f} B per frame "
|
|
f"for as long as the scene runs -- {over*1024*120/FPS/1024:.0f} "
|
|
f"KB over this 120-frame window, {over*60:.0f} KB per minute "
|
|
f"of play. Prefill below is where it got in 120 frames, NOT a "
|
|
f"startup cost that fixes it.")
|
|
|
|
if maxrec > ring:
|
|
print(f" !! MAXREC {maxrec:,} > ring {ring:,}: no policy works. "
|
|
f"decode.s needs one whole record contiguous.\n")
|
|
continue
|
|
|
|
# --- the requirement on the medium, which is the useful output, and
|
|
# the reason this tool takes no default rate. There is no measured
|
|
# pipe figure to score against (42.1), and the intent is to measure
|
|
# a BlueSCSI directly -- so the tool reports the THRESHOLD to
|
|
# measure against. The sweep is anchored to the container's own
|
|
# wire demand rather than to a list of fixed rates, so it stays
|
|
# meaningful for any container and privileges no constant.
|
|
print(f" {'pipe KB/s':>10} {'vs wire':>8} {'prefill KB':>11} "
|
|
f"{'records':>8} {'seek slack':>11}")
|
|
for mult in (0.90, 0.95, 1.00, 1.02, 1.05, 1.10, 1.25, 1.50, 2.00):
|
|
kbps = wire * mult
|
|
fill = (kbps - RC.AUDIO_KBPS) * 1024 / FPS
|
|
r = simulate(rec, fill, ring, "aligned", maxrec)
|
|
pf = r["prefill"]
|
|
# Branch-point seek slack, STATICALLY: with the ring FULL, how many
|
|
# frame times can the fill be zero before the next record is not
|
|
# resident? It is an upper bound and it assumes the premise that
|
|
# FINDINGS 51.3 took apart -- the ring is NOT full at a branch
|
|
# point, it is empty, and refilling it takes seconds of play. For
|
|
# the measured figure use tools/analysis/20_seek_slack.py, or the
|
|
# rig itself (tools/bench/pace_run.sh). Kept here as the ceiling
|
|
# this container's record sizes allow, which is what the rest of
|
|
# this row is about.
|
|
slack = (r["usable"] - maxrec) / rec.mean()
|
|
flag = ""
|
|
if pf + maxrec > r["usable"]:
|
|
flag = " <- does not fit the ring"
|
|
print(f" {kbps:>10.1f} {mult:>7.2f}x {pf/1024:>11.1f} "
|
|
f"{pf/rec.mean():>8.2f} {slack:>8.1f} fr{flag}")
|
|
# smallest pipe needing zero prefill, to 0.1 KB/s
|
|
lo, hi = wire, wire + 400
|
|
for _ in range(40):
|
|
mid = (lo + hi) / 2
|
|
f = (mid - RC.AUDIO_KBPS) * 1024 / FPS
|
|
if simulate(rec, f, ring, "aligned", maxrec)["prefill"] > 0:
|
|
lo = mid
|
|
else:
|
|
hi = mid
|
|
print(f" ZERO-PREFILL PIPE: {hi:.1f} KB/s "
|
|
f"({hi - wire:+.1f} KB/s over the wire demand, "
|
|
f"{100*hi/wire - 100:+.1f}%)")
|
|
print(f" ^ this is the number to measure a medium against. It is a "
|
|
f"REQUIREMENT, not a verdict.")
|
|
|
|
# --- the policy trade, at the default pipe
|
|
fill = (a.kbps - RC.AUDIO_KBPS) * 1024 / FPS
|
|
print(f" wrap policy, at pipe {a.kbps:.0f} KB/s:")
|
|
for policy in ("aligned", "split"):
|
|
r = simulate(rec, fill, ring, policy, maxrec)
|
|
if policy == "aligned":
|
|
print(f" aligned wraps {r['wraps']:3} mean hole "
|
|
f"{r['hole_mean']/1024:6.1f} KB usable ring "
|
|
f"{r['usable']/1024:6.1f} KB "
|
|
f"({100*r['usable']/ring:.1f}%) CPU cost 0")
|
|
else:
|
|
pct = 100 * r["copy_clk_per_frame"] / (CPUHZ / FPS)
|
|
print(f" split shadow {r['shadow_bytes']/1024:8.1f} KB "
|
|
f"= {r['copy_clk_per_frame']:8.0f} clk/frame = "
|
|
f"{pct:.2f}% of the frame budget, forever RAM cost 0")
|
|
print()
|
|
|
|
print("The `none` policy -- decoder wraps its own reads -- is not simulated.")
|
|
print("It has no RAM or copy cost and it is still the expensive one: the")
|
|
print("bounds test lands inside the exact instruction sequences FINDINGS")
|
|
print("30.4 and 40 fitted, so it does not cost a branch, it costs every span")
|
|
print("and per-block constant in the tree being re-measured. FINDINGS 28.3.")
|
|
|
|
|
|
if __name__ == "__main__":
|
|
main()
|