Pace the ring, then read the DMAC config out of the IPL ROM: audio is cheap and the disk is not
Two sessions that were never separated in the working tree, so they land as one commit. check.sh ALL GREEN before and after both. SESSION 19 -- the ring rig gets a frame clock (FINDINGS 51). src/player/stream.s had no frame clock: it asked for record i the instant it finished i-1, outran any finite pipe, and never let the ring back up. The 49.1 sweep passing at 48 KB was therefore a wrap-correctness result and nothing else. PACE/PACEON ($18034/$18038) hold the decoder to 12 fps, so FR_HEAD-FR_TAIL finally means what it reads as: whole frames the decoder could still draw with delivery stopped dead. PACEON=0 free-runs and is what the wrap gate still uses, so every figure in 49 is unmoved. Paced, on the gate container: 64 KB holds 2 frames, 256 KB holds 7-8, 512 KB holds 14-15, all pixel-exact. Tolerance is ceiling-1, measured by cutting the pipe: 256 KB buys 500 ms of dead pipe, not 583. SLACK IS ACCUMULATED, NOT OWNED. It is built out of pipe-wire and a seek spends all of it. At 488 KB/s a 256 KB ring needs 4.83 s of play to reach its ceiling from empty; 512 KB needs 8.42 s to reach 14. A bigger ring raises the ceiling AND lengthens the climb, so a branch point does not ask "is the buffer big enough" but "has there been enough play since the last one" -- and Dragon's Lair's decision points are seconds apart. The rig now also says WHICH resource is binding: at 460 KB/s every ring from 192 KB to 512 KB is rate-bound at ceiling 4 and never fills, so larger rings are dead RAM in that scene. 20_seek_slack.py is the same model rewritten in Python from record sizes, sharing no code with the Lua producer: 35/35 ceilings inside its bracket. SESSION 20 -- the DMAC configuration was in the IPL ROM the whole time (FINDINGS 52). ROADMAP's "do this first" was to put the ADPCM stream on the bus. That needs a clocks-per-byte figure for the audio channel, and 11_cpu_budget.py was charging audio the DISK's rate -- 5 clk/B, its own help text calling it "single-address, bus held". Audio was being charged the favourable end of B3, a 242 KB/s open question. It never had to be a guess. The IPL ROM programs all four HD63450 channels itself and MAME boots the rig with it, so 21_iplrom_dmac.py reads the configuration out of the image and decodes the MC68450 fields. Eight (address, expected bytes, meaning) sites; a mismatch or an unknown revision exits non-zero. In check.sh, no emulator, milliseconds. ch3 DCR=$80, OCR=$32: dual address, 8-bit port, cycle steal WITHOUT hold, REQG=10 external request. The DMAC arbitrates once per byte with no burst to amortise the 5..8 + 2 over, so an audio byte is 16..19 clocks, not 5 -- the old debit was 3.2x..3.8x small. And on the bus it is still nothing: 651 B/frame is 1.25%..1.48% of a frame, about 4% of what the decoder leaves. P6's bus risk does not materialise. The unit worry was worth checking and nearly right: 15.6 kHz is 8 MHz/512 = 15,625 samples/s, two 4-bit samples to a byte = 7,812.5 B/s exactly, and AUDIO_KBPS=7.8 is that in decimal kB while the tool multiplied by 1024. THE DISK CHANNEL IS PROGRAMMED IDENTICALLY. ch1 (SASI) is DCR=$80 too, and so is ch0. That is 16..19 clocks per delivered byte, where 42.4 brackets W at 5..12 and 42.5 has W=8 already missing 47/120 frames. The only worked example of a disk DMA configuration on this machine sits above the entire bracket, and at that price nothing fits at any container size. It is not scsiexrom.bin so B3 stays open -- what changed is that a cheap configuration is now the thing that has to be SHOWN. W <= 12 is a requirement on the player's DMAC programming, not a range the hardware hands us, and it is now the largest open number in the project, ahead of the rate. An unforced cross-check fell out: 15_bus_occupancy.py's new W sweep puts W=8 at 105.7% of the frame, agreeing with 42.5's 47/120, from mode histograms and bus clocks respectively, two models sharing no code. Also: ADPCM outranks the disk at the arbiter (CPR 1 against 2), so an audio byte never waits and a video byte does -- relevant to 51's smooth-rate delivery model. README MEDIA. stream.lua gains DLX_SNAP_EVERY=1 (needs DLX_PACE, off by default, on no path check.sh takes) and tools/media/make_readme_media.py turns the PNGs into docs/img/. The stills and both clips are MAME's own screen pixels. Building it turned up something worth recording. 116 of 119 captured frames are pixel-exact against dlx.py; three are TORN -- frame n on top, frame n-1 below the tear line -- because MAME captured the screen while the block loop was partway down it. decode.s writes straight to the displayed page (one display path, 28.1), so a real player tears the same way, and this is the first time that consequence has been visible rather than argued. The script ASSERTS the tear and refuses to build otherwise, rather than trimming three frames and reporting "every frame I kept is exact". Second correction the capture forced: the snapshot fires before frame n is decoded, so the obvious reading is that it holds frame n-1 -- it does not, because MAME renders the screen at the end of the machine frame, by which time the 68000 has finished frame n. 11_cpu_budget.py's "validated to within 1 pt" line is also corrected: the model reads 2..10 pt HIGH and by more as the frame gets harder, which was already true before either session. src/player/decode.s is unchanged; decode.bin is still 1,296 B at the same MD5. Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
This commit is contained in:
prosolis
@@ -6,6 +6,56 @@ This is fundamentally a **video codec problem**, not a game-logic problem: the
|
||||
game logic is a scene table with branching input windows; the difficulty is
|
||||
pushing ~22 minutes of Don Bluth animation through a 10MHz 68000.
|
||||
|
||||
---
|
||||
|
||||
## What it looks like
|
||||
|
||||
|
||||
|
||||
Left, the Blu-ray frame cropped to 256x192. Right, **the same frame as the
|
||||
emulated 68000 actually drew it** — 256 colours out of the X68000's 65536, one
|
||||
16-colour-per-4x4-block codebook, decoded by `src/player/decode.s` from the
|
||||
container. Not a re-render: these are the pixels MAME had on screen, extracted
|
||||
from its own snapshot. 2x nearest-neighbour, no filtering.
|
||||
|
||||
**The player, running.** 119 frames out of a **256 KB ring buffer on an emulated
|
||||
stock 2 MB X68000**, paced to a 12 fps frame clock, streamed from a host file at
|
||||
488 KB/s — `src/player/stream.s`, no Lua in the decode path. Source on the left,
|
||||
the machine's screen on the right.
|
||||
|
||||
<video src="docs/img/player.webm" controls muted loop width="100%"></video>
|
||||
|
||||
[`docs/img/player.webm`](docs/img/player.webm) — 119 frames, 12 fps, VP9
|
||||
|
||||
116 of those 119 frames are **pixel-exact** against `tools/encoder/dlx.py`'s
|
||||
reference reconstruction. The other three are **torn** — the top of the picture
|
||||
is frame *n* and the bottom still holds frame *n-1*, because MAME captured the
|
||||
screen while the block loop was partway down it. That is not a rig artefact:
|
||||
`decode.s` writes straight to the displayed page (one display path, FINDINGS
|
||||
28.1 — the dual-path plan is kept runnable as a counterexample precisely because
|
||||
it corrupts frames), so a real player tears the same way.
|
||||
`tools/media/make_readme_media.py` asserts the tear rather than trimming it —
|
||||
every differing pixel has to come from the previous frame, or it refuses to
|
||||
build.
|
||||
|
||||
**What the decoder is actually doing.** The same window with the block-mode map
|
||||
beside it: **black = SKIP** (costs nothing, draws nothing — the previous frame
|
||||
stands), **blue = V1** (one codebook index for a whole 4x4 block), **amber = V4**
|
||||
(four indices), **red = RAW** (sixteen bytes verbatim). The mode mix is what
|
||||
every cost table in `docs/FINDINGS.md` is really about — V4 costs 1.5x V1, and
|
||||
since session 8 the mode decision is charged both bytes *and* cycles, which is
|
||||
why a byte-rich profile buys its way out to RAW instead of V4.
|
||||
|
||||
<video src="docs/img/modes.webm" controls muted loop width="100%"></video>
|
||||
|
||||
[`docs/img/modes.webm`](docs/img/modes.webm) — the same 119 frames with the mode map
|
||||
|
||||
**Name the layer:** everything above is **emulated** (MAME 0.277 `x68000`,
|
||||
`-bios ipl10`, stock 10 MHz / 2 MB), cross-checked frame-for-frame on a second
|
||||
CPU core (px68k's C68K). Nothing in this project has run on real hardware yet.
|
||||
|
||||
---
|
||||
|
||||
**And the binding resource is the 68000's local BUS, not its clock.** The
|
||||
decoder occupies 86.7% of it once instruction prefetch is counted, and 52 of the
|
||||
53 frames that miss the 12fps budget miss it on the bus, not the CPU
|
||||
@@ -15,11 +65,60 @@ The **literal span with a fine tail** (v7) is now IN the player: `decode.s`
|
||||
paints it, pixel-exact under both CPU cores, and it costs inside the decoder
|
||||
what `blit.s` said it would to 0.2% (FINDINGS 41).
|
||||
|
||||
**It only pays if the stream is allowed to run near the pipe.** Spans buy the
|
||||
68000's deadline with bytes, and at the 280 KB/s profile the mode decision has
|
||||
already spent them: 77/120 frames over budget against 86 without spans. Given
|
||||
the full 488 KB/s pipe it is **34/120, and 0.36 dB better** — so the next
|
||||
decision is a rate point, not an optimisation (FINDINGS 41.2, docs/STATUS.md).
|
||||
**The delivery path is built and tested too** (FINDINGS 49). `src/player/stream.s`
|
||||
decodes the whole 120-frame window **out of a 256 KB ring on a stock 2 MB
|
||||
machine**, final frame pixel-exact, with the container in a host file rather than
|
||||
preloaded into RAM. The constraint turned out to be **contiguity, not byte
|
||||
count** — the block loop reads with a monotonically increasing `a0` and no bounds
|
||||
check, so the ring needs the whole next record resident *and contiguous*, which
|
||||
is a condition no byte-counting buffer simulation can see.
|
||||
|
||||
**And building it caught a live defect**, then cost the project a constant.
|
||||
The shipping candidate is 496.7 KB/s; the pipe figure the design had been
|
||||
simulated against since session 2 was smaller, and nothing in the tree was
|
||||
comparing the two — the rate controller binds on clocks and has no pipe term at
|
||||
all, while the buffer sizing kept standing on a constant the design had stopped
|
||||
enforcing.
|
||||
|
||||
**So the pipe constant is retired (session 18, USER DECISION).** It was never a
|
||||
bus measurement — user-supplied, no provenance, 10% of SCSI-1's asynchronous
|
||||
rating (FINDINGS 42.1). It is gone as a default from every analysis tool and
|
||||
from `stream.lua`; `--bus` / `--kbps` / `DLX_STREAM_KBPS` are now **required
|
||||
arguments**, so no table can be scored against a rate its own output does not
|
||||
state. **There is no working delivery figure, and that is the honest state.**
|
||||
|
||||
What replaces it is a requirement rather than a constant:
|
||||
`tools/analysis/19_ring_stream.py` reports the **zero-prefill pipe**, the rate a
|
||||
medium must clear for a container to need no prefill. For the candidate that is
|
||||
**513.2 KB/s** — a hardware acceptance test to measure a BlueSCSI against.
|
||||
|
||||
**Bytes are not free, and the number that said they were was in the wrong
|
||||
unit.** Session 13 found the pipe figure the design was built against was never
|
||||
a bus figure (SCSI-1 is 1.5 MB/s asynchronous) and concluded the span pass
|
||||
saturates at ~837 KB/s, 0/120 frames over budget. Session 14 found the disk
|
||||
debit behind that was charged **per word of stream to a byte-wide port** — the
|
||||
MB89352 is an 8-bit SPC, so the DMAC pays per BYTE, and the debit is 2x every
|
||||
table since FINDINGS 5. No 68000 bus cycle is shorter than four clocks, so the
|
||||
old figure was below a physical floor.
|
||||
|
||||
**What survives, re-encoded honestly: 496.7 KB/s at 29.19 dB, 1 frame of 120
|
||||
over the 12fps budget** — and that one is frame 0, the intra frame, late on
|
||||
purpose. The remaining lever is not ours: **whether the CZ-6BS1 wires the SPC's
|
||||
DACK to the bus's `#EXACK`**, which decides 5 clocks/byte against 9, and with
|
||||
it 242 KB/s and 0.69 dB. Read FINDINGS 43 before quoting any rate figure.
|
||||
|
||||
**And session 20 read the answer the machine already had.** The X68000's IPL ROM
|
||||
programs all four HD63450 channels itself, and MAME boots the rig with it, so
|
||||
`tools/analysis/21_iplrom_dmac.py` decodes the configuration straight out of the
|
||||
image and gates on the bytes still being there. The audio channel is
|
||||
dual-address, 8-bit port, cycle steal **without hold**, one external request per
|
||||
byte: **16..19 clocks a byte, not 5** — which prices the ADPCM stream at
|
||||
1.25%..1.48% of a frame and closes ROADMAP's "do this first" item. The disk
|
||||
channel is programmed **identically**. That is 16..19 clocks per delivered byte,
|
||||
above the whole 5..12 bracket the project costs the transport in, and at that
|
||||
price nothing fits. It is SASI and not the MB89352, so it does not settle the
|
||||
question — but **a cheap configuration is now the thing that has to be shown,
|
||||
not the thing assumed.** FINDINGS 52.
|
||||
|
||||
**Green-light check:** `./tools/bench/check.sh` (~3 min, needs the Blu-ray
|
||||
mounted) re-runs both display regression tests, the rate-control drift test, the
|
||||
@@ -33,6 +132,9 @@ display-path coherency counterexample and a 120-frame 68000 decode, then prints
|
||||
- **`docs/STATUS.md`** — current state, working setup, blockers, next steps.
|
||||
**Start here.** It also lists what has been explicitly abandoned, so old ideas
|
||||
do not get re-proposed.
|
||||
- **`docs/ROADMAP.md`** — the remaining work to a completion target, and which
|
||||
milestone that target is. Read it with STATUS, not instead of it: STATUS holds
|
||||
the measurements, ROADMAP holds the shape and goes stale first.
|
||||
- **`docs/BENCHMARK.md`** — how to measure the storage subsystem, and why a
|
||||
bandwidth figure out of MAME would be meaningless.
|
||||
- **`docs/HARDWARE.md`** — X68000 GVRAM/CRTC reference.
|
||||
@@ -70,6 +172,17 @@ tools/analysis/ measurement scripts, numbered in the order they were written
|
||||
emitted too few spans to have tested anything. 17 prices the
|
||||
spans the encoder ACTUALLY emitted, with no selection model,
|
||||
which is what 12 and 14 could only simulate.
|
||||
18 measures what a 16-colour text-plane literal would cost in
|
||||
dB, and closes that direction (FINDINGS 46.3).
|
||||
19 is the RING-BUFFER simulation, and it supersedes 09_buffer_
|
||||
sim.py's question rather than repeating it: it models the ring's
|
||||
ADDRESSES, because src/player/ needs each record contiguous and
|
||||
not merely resident. It reports the ZERO-PREFILL PIPE -- the
|
||||
rate a medium must clear for a container to need no prefill --
|
||||
and warns explicitly when demand exceeds supply on the MEAN,
|
||||
where a "required prefill" figure would flatter a sustained
|
||||
overrun. Its wrap count and hole size match tools/bench/
|
||||
stream.lua's, measured on a real 68000, to the digit.
|
||||
buscost.py is the shared bus-cycle table both import; the
|
||||
per-BLOCK constants live in tools/encoder/vq_hybrid.py and are
|
||||
imported, never copied (session 12 corrected one of them).
|
||||
@@ -90,6 +203,12 @@ tools/bench/ MAME Lua injection harness + 68000 benchmark sources.
|
||||
and R20 — do not write CRTC values anywhere else.
|
||||
`prep_dlx.py`/`decode.lua`/`verify_decode.py` load, time and
|
||||
verify `src/player/decode.s`; the verify pass is in check.sh.
|
||||
`prep_stream.py`/`stream.lua` do the same for `stream.s`, but
|
||||
lay the container out as a DISK in a host file and feed it
|
||||
through a bounded ring at a modelled pipe rate -- so the rig is
|
||||
no longer bounded by the emulated machine's RAM, and a stock
|
||||
2 MB machine runs the whole window. `dlxload.py` holds the
|
||||
codebook/palette load-time maths both preps share.
|
||||
tools/bench/c68k/ headless px68k C68K harness -- a SECOND emulator for every
|
||||
68000 cycle figure (FINDINGS 37). Links only px68k's CPU core:
|
||||
no SDL, no ROMs, no emulated machine. `make PX68K=~/src/px68k`
|
||||
@@ -107,7 +226,13 @@ tools/encoder/ hybrid VQ encoder + DLX3 container writer (working).
|
||||
DLX2 4-byte-aligns every frame record: an odd `move.l` is an
|
||||
ADDRESS ERROR on a 68000, not a slow read (FINDINGS 28.3).
|
||||
dlx.py is the reference DECODER -- ground truth for the 68000.
|
||||
src/player/ decode.s: the 68000 DLX3 decoder. Pixel-exact under MAME and
|
||||
src/player/ decode.s: the 68000 DLX3 decoder, PRELOADED-stream front-end.
|
||||
stream.s: the same decoder behind a bounded RING (FINDINGS 49).
|
||||
Both include frame.i (the block loop and span chain) and geom.i
|
||||
(the constants) so there is exactly ONE copy of the bytes every
|
||||
cycle constant in FINDINGS 24/30/40/41 is fitted to. check.sh
|
||||
asserts decode.s still assembles to the same 1,296 bytes.
|
||||
decode.s: the 68000 DLX3 decoder. Pixel-exact under MAME and
|
||||
px68k's C68K core, blocks and v7 literal spans both. The span
|
||||
pass is blit.s v7 verbatim -- the same instruction sequence the
|
||||
66.0/9.143/9.978 fit was measured on, so do not tidy it.
|
||||
@@ -140,6 +265,18 @@ The profile bitrate is a **ceiling**: lam is bisected per frame under a leaky
|
||||
bucket, so the profile's `lam` is a quality floor rather than a setting
|
||||
(`--fixed-lam` opts out).
|
||||
|
||||
**But at `--spans all` none of that binds.** A 32-frame bucket emits the same
|
||||
container byte for byte as an 8-frame one, and `lam` never leaves its floor on
|
||||
any frame of the reference window: the rate is set by the span pass and by `mu`,
|
||||
not by `--kbps` or the bucket (FINDINGS 44.3). Two known unit inconsistencies on
|
||||
that side are implemented and default OFF because they measure as a wash --
|
||||
`--joint-decide` (the per-block lagrangian prices a byte at `lam + mu*c` rather
|
||||
than `lam`) and `--joint-bucket` (the bucket may not lend clocks it cannot
|
||||
repay). FINDINGS 44.
|
||||
|
||||
An encode is ~95% k-means; a 120-frame window is ~29 s, of which ~22 s is
|
||||
training the two codebooks (FINDINGS 44.5).
|
||||
|
||||
There are **two** ceilings, on two different axes. The second is the 68000's
|
||||
decode budget: `mu` is bisected per frame against 833,333 cycles so the frame
|
||||
also *decodes* in time, which takes the worst sustained window from 37 frames
|
||||
|
||||
+23
-5
@@ -53,13 +53,31 @@ was used. **Do not record KB/s and treat it as a hardware figure.**
|
||||
Already partly in FINDINGS 5. Bounds worth tightening from datasheets:
|
||||
|
||||
- 68000 bus cycle: 4 clocks @ 10MHz, 16-bit => **5 MB/s** absolute ceiling
|
||||
- HD63450 single-address DMA, ~8 clocks/word => **~2.5 MB/s** practical ceiling
|
||||
- HD63450 single-address DMA, **5 clocks/BYTE** => **2.0 MB/s** practical ceiling
|
||||
(dual-address is 9 clocks/byte => 1.11 MB/s). CORRECTED session 14: this line
|
||||
read "~8 clocks/word => ~2.5 MB/s", which charged a byte-wide SPC per word.
|
||||
FINDINGS 43.
|
||||
- SCSI-1 asynchronous REQ/ACK handshake per byte, plus MB89352 FIFO depth
|
||||
=> the real limiter, and the number we do not have from a primary source
|
||||
|
||||
The user's working figure is **4 Mbps = 488 KB/s**, which sits sensibly between
|
||||
the derived DMA ceiling and observed period-drive rates. **Provenance not yet
|
||||
recorded — worth pinning down, because every profile now hangs off it.**
|
||||
**RETIRED, session 18 (USER DECISION).** This document used to name a working
|
||||
figure of "4 Mbps" here and note that every profile hung off it. It was never a
|
||||
bus measurement — user-supplied, no provenance, and 10% of SCSI-1's asynchronous
|
||||
rating (FINDINGS 42.1). It has been removed as a default from every analysis
|
||||
tool and from `tools/bench/stream.lua`; the tools now REQUIRE an explicit rate,
|
||||
so nothing can be scored against a figure the scorer never restates.
|
||||
|
||||
**There is no working delivery figure. That is the honest state, and it is the
|
||||
point:** the rate is a property of the medium, the medium is a BlueSCSI, and it
|
||||
has not been measured. `tools/analysis/19_ring_stream.py` reports the
|
||||
**zero-prefill pipe** — the rate a medium must clear for a given container to
|
||||
need no prefill at all — which is the threshold a measurement should be taken
|
||||
against. For the session-14 candidate that is **513.2 KB/s** (FINDINGS 49.5).
|
||||
|
||||
One place still carries the old number: `GATE_SPAN_KBPS` in
|
||||
`tools/bench/check.sh`, because the gate container was *encoded* with it and
|
||||
every per-block and span constant in FINDINGS 41/43/45/49 is fitted to that
|
||||
container. It is a container recipe, not a claim about any medium.
|
||||
|
||||
### The coupling nobody had counted
|
||||
Cycle-stealing DMA is not free DMA. At ~8 clocks per 16-bit word:
|
||||
@@ -69,7 +87,7 @@ Cycle-stealing DMA is not free DMA. At ~8 clocks per 16-bit word:
|
||||
| 110 KB/s | 4.5% | 42.8% |
|
||||
| 250 KB/s | 10.2% | 48.5% |
|
||||
| 450 KB/s | 18.4% | 56.7% |
|
||||
| 488 KB/s | 20.0% | 58.3% |
|
||||
| ~490 KB/s | 20.0% | 58.3% |
|
||||
|
||||
FINDINGS 5 concluded that because transfers are DMA, "streaming costs
|
||||
essentially no CPU". **That is wrong.** It costs up to a fifth of the machine at
|
||||
|
||||
+1767
-4
File diff suppressed because it is too large
Load Diff
+244
@@ -0,0 +1,244 @@
|
||||
# Roadmap — remaining work to a completion target
|
||||
|
||||
Written end of session 19 (2026-08-24), against a tree that is ALL GREEN.
|
||||
|
||||
**THE COMPLETION TARGET IS M3, THE VERTICAL SLICE** (USER DECISION): one scene
|
||||
tree — a decision point, two outcomes, a death clip — with audio, streaming from
|
||||
a real SCSI volume on a stock 2 MB machine, playable. That is the point at which
|
||||
every layer of this design has been shown to work at once. M4 is listed because
|
||||
it is real work, but past M3 it is content grinding rather than open questions.
|
||||
|
||||
`docs/STATUS.md` remains the session-by-session record and the handoff. This file
|
||||
is the shape of what is left; where the two disagree about what is done, STATUS
|
||||
is the one with the measurements and this one is the one that goes stale. Both
|
||||
were wrong about two encoder gaps until this file was written — see "What was
|
||||
already done" below.
|
||||
|
||||
---
|
||||
|
||||
## Status of the four resources
|
||||
|
||||
The project's own framing, restated because every item below is priced in one of
|
||||
these units:
|
||||
|
||||
| resource | state |
|
||||
|---|---|
|
||||
| **68000 local bus** | the binding one. Decoder occupies 86.7%; 52 of 53 missed frames miss on the bus, not the clock (FINDINGS 38). |
|
||||
| **68000 clocks** | measured, and the rate controller binds on them. |
|
||||
| **Delivery rate** | **no working figure, deliberately** (FINDINGS 50, USER DECISION). Every tool REQUIRES an explicit rate. |
|
||||
| **Seek time** | **no figure at all, and never had one.** 51.3/51.4 made it matter. |
|
||||
| **W, clocks stolen per delivered byte** | bracketed 5..12 (42.4); the IPL ROM's own disk channel is **16..19** (52.5). **The largest open number in the project.** |
|
||||
|
||||
---
|
||||
|
||||
## What was already done, and was still on the list
|
||||
|
||||
Found while inventorying for this file. Both had been closed in code for several
|
||||
sessions and were still listed as open gaps in `docs/STATUS.md`:
|
||||
|
||||
- **4-byte record padding.** `DLX2`, `encode.py:139-156`, inside rate-control
|
||||
accounting, reported per frame and per second.
|
||||
- **CPU cost in the mode decision.** `vq_hybrid.py:218`, priced against measured
|
||||
per-mode cycles with the exact clustered SKIP rule.
|
||||
|
||||
Both entries are now struck in STATUS. **The lesson is procedural: a gap list
|
||||
that is only ever appended to manufactures phantom work.** Anything crossed off
|
||||
below should be crossed off in STATUS in the same sitting.
|
||||
|
||||
---
|
||||
|
||||
## Blocked on hardware this tree does not have
|
||||
|
||||
None of these block M2 or M3 software work, because session 18 forced every rate
|
||||
to be an explicit argument. They set constants, and two of them decide how much
|
||||
headroom the finished player has.
|
||||
|
||||
**B1. Measure the BlueSCSI — throughput AND seek time.**
|
||||
Throughput has an acceptance test already derived from real record sizes:
|
||||
**513.2 KB/s** for the session-14 candidate, **451.4 KB/s** for the gate
|
||||
container (`19_ring_stream.py`, FINDINGS 49.5). Seek time has nothing.
|
||||
51.3/51.4 is why the second half matters: slack is *accumulated* out of
|
||||
`pipe - wire`, so what a branch point costs is set by the rate and the time since
|
||||
the last branch, not by the ring size. At 460 KB/s every ring from 192 KB to
|
||||
512 KB is rate-bound and never fills. **Do not substitute a guess** — run at
|
||||
several explicit rates and report the sensitivity. That is exactly how the
|
||||
retired pipe constant survived five sessions after 42.1 called it folklore.
|
||||
|
||||
**B2. Does buffer mode blank the display?** `probe_bit11_blank.lua` is written
|
||||
and settles it in minutes on a real board. FINDINGS 48 shifted the prior toward
|
||||
MAME and toward "unusable" — **do not pre-build on 1.0 B/pixel**. Same sitting:
|
||||
the priority register `0xE82500` at `0x0000` (47.3).
|
||||
|
||||
**B3. Single-address vs dual-address DMA.** 242 KB/s and 0.69 dB. Needs
|
||||
`scsiexrom.bin` (8 KB, CRC `7be488de`) sourced, then its DMAC init disassembled
|
||||
for DCR's DTYP: `10`/`11` = single (5.0 clk/B), `00`/`01` = dual (9.0).
|
||||
FINDINGS 48.4. Not on this machine (checked, session 18).
|
||||
**This is also P4's input** — the handshake the player drives is the same
|
||||
question from the software side.
|
||||
|
||||
> **Session 20 moved the prior hard, and it moved the wrong way (FINDINGS 52.5).**
|
||||
> The IPL ROM *is* on this machine, and `tools/analysis/21_iplrom_dmac.py` reads
|
||||
> its HD63450 setup: the on-board disk channel (ch1, SASI) is `DCR = $80` —
|
||||
> **dual address, 8-bit port, cycle steal WITHOUT hold**, with `REQG = 10`
|
||||
> external request, i.e. a full bus arbitration per byte. That is **16..19
|
||||
> clocks per delivered byte**, above the whole 5..12 bracket 42.4 costs P4 in.
|
||||
> Same vendor, same DMAC, same class of 8-bit port — but it is *not*
|
||||
> `scsiexrom.bin`, so B3 stays open. What it changes is that a cheap
|
||||
> configuration is now the thing that has to be **shown**, not assumed.
|
||||
|
||||
---
|
||||
|
||||
## M2 — a player, as opposed to a decoder
|
||||
|
||||
`decode.s` draws pixel-exact frames from RAM Lua pre-loaded; `stream.s` decodes
|
||||
out of a bounded ring fed by a host file on a paced clock. Neither is a player.
|
||||
|
||||
**Exit criterion: boots from a real SCSI volume on a stock 2 MB X68000, plays
|
||||
one scene at 12 fps from disc, no host-file pipe, no Lua in the loop. Silent.**
|
||||
|
||||
**P1. Codebook expansion on the 68000.** `dlxload.py:19` expands CB1 to 32 B per
|
||||
entry and CB4 to 8 B, host-side, because at the time it was a load-time cost that
|
||||
would have flattered or damned the inner loop. The player must do it: **8 KB +
|
||||
2 KB per scene**. Note where that lands — *at a scene change, when the ring is
|
||||
empty because of the seek*. It compounds with 51.3 and should be priced against
|
||||
the refill climb, not treated as free setup.
|
||||
|
||||
**P2. Palette packing on the 68000.** The encoder still emits RGB888; the X68000
|
||||
word packing is Lua-side. Whatever writes real palette words must pick `I` per
|
||||
entry by minimum squared error (**1.96 dB**, FINDINGS 23.3) and reserve index 0
|
||||
as black with `I = 0` (23.4).
|
||||
|
||||
**P3. A real frame clock.** `stream.s` has `PACE`/`PACEON` (`$18034`/`$18038`)
|
||||
but the 12 fps tick comes from the Lua producer. Needs MFP timer or VBL. Keep
|
||||
`PACEON=0` free-run working — the wrap gate uses it and every FINDINGS 49 figure
|
||||
depends on it.
|
||||
|
||||
**P4. Real transport.** Drive the MB89352 instead of a host file. The `W`
|
||||
handshake — clocks stolen per delivered byte, bracketed 5..12 by MC68450 Fig
|
||||
4-25 — is listed in "Decisions locked" as UNDECIDED and as the thing that
|
||||
decides the project: `W<=6` fits 0/120 frames, `W=8` misses 47/120. It is a
|
||||
property of how the player drives the SPC, **so it is ours to choose, not to
|
||||
receive** (FINDINGS 42.4-42.6). B3 informs it.
|
||||
|
||||
**Session 20 promoted this to the project's biggest open number.** FINDINGS 52.5
|
||||
found the only worked example of a disk DMA configuration on this machine — the
|
||||
IPL ROM's own — sitting at 16..19 clk/B, outside the bracket entirely, where the
|
||||
whole design fails at any container size (`15_bus_occupancy.py` sweeps it).
|
||||
`W <= 12` is now a **requirement on the player's DMAC programming**, not a range
|
||||
the hardware hands us, and demonstrating a configuration that meets it is P4's
|
||||
first job rather than its last.
|
||||
|
||||
**P5. Seek and branch.** Per-record index (the `aligned` producer needs one
|
||||
anyway, 49.3), prefill policy, and the accumulated-slack rule from 51.3 made
|
||||
explicit in the player rather than implied by the rig.
|
||||
|
||||
**P7. Boot.** The player as an executable loading from the SCSI volume.
|
||||
|
||||
---
|
||||
|
||||
## M3 — the vertical slice, and the completion target
|
||||
|
||||
**Exit criterion: one decision point, two outcomes, a death clip, with audio,
|
||||
playing from disc on stock hardware.**
|
||||
|
||||
**P6. Audio — and it is the largest unpriced risk left in the project.**
|
||||
MSM6258 ADPCM, 15.6 kHz mono, **7.8 KB/s**. That figure is in `ratectl.py`'s
|
||||
budget and nowhere else: not extracted, not encoded, not interleaved into the
|
||||
container, and **never priced on the bus**. Two reasons to treat it as a risk
|
||||
rather than a task:
|
||||
|
||||
1. A second DMA consumer attacks **the bus** — the resource this project already
|
||||
established is the binding one, at 86.7% occupied. Clock headroom says
|
||||
nothing about whether it fits.
|
||||
2. 7.8 KB/s is a *byte* figure. The last time a byte/word unit error went
|
||||
unexamined in a delivery budget it cost the project a 2x error in every table
|
||||
since FINDINGS 5 (session 14, the MB89352 being an 8-bit SPC).
|
||||
|
||||
~~**Price it before writing it**: add the ADPCM DMA stream to `15_bus.py` and see
|
||||
what it does to the 86.7%.~~ **DONE, session 20 — FINDINGS 52.** It is in
|
||||
`15_bus_occupancy.py` and the answer is **1.25%..1.48% of the frame**, about 4%
|
||||
of what the decoder leaves. The per-byte cost is no longer a guess borrowed from
|
||||
the disk: `tools/analysis/21_iplrom_dmac.py` reads the IPL ROM's own HD63450
|
||||
configuration and finds ch3 dual-address, 8-bit port, cycle steal without hold,
|
||||
external request — **16..19 clocks per byte**, where `11_cpu_budget.py` had been
|
||||
charging audio the disk's 5. Both worries above resolve:
|
||||
|
||||
1. **The bus concern does not materialise.** A second DMA consumer at 7.8 kB/s
|
||||
is not what a bus at 88% occupancy is short of.
|
||||
2. **The unit was checked and is nearly right.** 15.6 kHz = 8 MHz ÷ 512 =
|
||||
15,625 samples/s, 4 bits each, two to a byte = **7,812.5 B/s exactly**. The
|
||||
7.8 was decimal kB being multiplied by 1024; 2.4% high, now derived from the
|
||||
sample rate in `buscost.ADPCM_BYTES_PER_S`.
|
||||
|
||||
**What is still open in P6 is everything except the bus:** extraction, encode,
|
||||
container interleave, and what a second stream does to `wire` — and therefore to
|
||||
`pipe - wire`, and therefore to 51.3's refill climb. That last one is the
|
||||
interaction to price next, and it is E2's question with a second consumer in it.
|
||||
|
||||
**E6. Container v2** — audio interleave, per-record index, scene table. Depends
|
||||
on P6's answer and on P5's index.
|
||||
|
||||
**G1. Import the scene graph — early, because it is a measurement input.**
|
||||
SNES project `data/events/` (MIT, cleared) diffed against DirkSimple (zlib),
|
||||
which transcribed the same data independently, to catch transcription errors
|
||||
before anything reaches 68000 tables. **Neither is on this box** — both need
|
||||
fetching.
|
||||
|
||||
The reason to pull this ahead of the game logic that consumes it: 51.3 says
|
||||
4.83 s of play to refill a 256 KB ring at 488 KB/s, and Dragon's Lair's decision
|
||||
points are seconds apart. **Nothing in this tree can currently say what the worst
|
||||
gap between consecutive decision points is** — only the scene table knows, and
|
||||
until it is imported, whether this design survives a back-to-back branch is an
|
||||
open question nobody is able to ask.
|
||||
|
||||
---
|
||||
|
||||
## M4 — the whole game
|
||||
|
||||
Listed for completeness; past M3 these are scope, not risk.
|
||||
|
||||
- **C1. Full-disc survey**, 22.8 minutes. Classify **content / menu / bonus** —
|
||||
not menu vs content: the two largest streams are bonus material and look like
|
||||
content by size, duration and bitrate alike (25.1). Run
|
||||
`07_motion_survey.py` per stream first for a hot-window shortlist.
|
||||
**Gated by E4.**
|
||||
- **E4. `H.build` k-means**, 51 s of a 55 s run, once per scene. The thing to
|
||||
attack before C1, and not anything in the per-frame path (27.6).
|
||||
- **E2. `--spans all` as default.** Still a recommendation, not a measurement
|
||||
(43.6.1), and the only loaded lever on the encoder's byte side (44.3). **It
|
||||
spends every profitable byte, which raises `wire`, which shrinks `pipe - wire`,
|
||||
which lengthens the refill climb after every branch.** That interaction is not
|
||||
priced, and M3 is where it becomes measurable.
|
||||
- **E3. Re-derive span selection jointly with `lam`** (39.3).
|
||||
- **C2. Framing** — crop vs squash vs wide (FINDINGS 12). Needs an eyeball
|
||||
against arcade reference, not a measurement. Cheap; blocks only final encodes.
|
||||
- **C3. Disk image packaging**, ~1.09 GiB at the candidate rate.
|
||||
- **G2/G3.** Branching, input windows, death clips, attract mode; playtest.
|
||||
|
||||
---
|
||||
|
||||
## Dependency summary
|
||||
|
||||
```
|
||||
B1 seek+rate ─┐
|
||||
B3 DTYP ──────┴─> P4 transport ─┐
|
||||
├─> M2 ─> M3 (COMPLETION TARGET) ─> M4
|
||||
P1 P2 P3 P5 P7 ─────────────────┘ ^
|
||||
│
|
||||
P6 (bus cost DONE, 52) ──────────────────┤
|
||||
G1 scene graph (fetch, do early) ─────────┘
|
||||
B2 blanking ─> (page 1; do not pre-build on it)
|
||||
```
|
||||
|
||||
## Standing rules that apply to all of it
|
||||
|
||||
- **Green light first and last.** `./tools/bench/check.sh`, ALL GREEN, before and
|
||||
after. **Never two MAME jobs at once** — session 18 did it, two `decode.lua`
|
||||
runs shared a log file, and it produced a 0-byte log and 15 wasted minutes.
|
||||
- **Name the layer.** Emulated, or real hardware. Every progress claim.
|
||||
- **Label measured / estimated / folklore.** A rate with no provenance is
|
||||
folklore even when it is plausible, and this project has already paid for that
|
||||
twice.
|
||||
- **No new default constants.** Rates stay explicit arguments. If a measurement
|
||||
is not available, report the sensitivity across several rates rather than
|
||||
picking one.
|
||||
+909
-62
File diff suppressed because it is too large
Load Diff
Binary file not shown.
|
After Width: | Height: | Size: 64 KiB |
Binary file not shown.
Binary file not shown.
Binary file not shown.
|
After Width: | Height: | Size: 182 KiB |
+2
-194
@@ -88,18 +88,8 @@ FPTR = $18010 ; -> first frame record
|
||||
SCR_N = $18014 ; frames remaining this pass
|
||||
SCR_END = $18018 ; expected end of the current payload
|
||||
|
||||
CB1 = $20000 ; expanded 4x4 codebook
|
||||
CB4 = $22000 ; expanded 2x2 codebook
|
||||
include "src/player/geom.i"
|
||||
|
||||
DST0 = $C08000 ; GVRAM + 32*1024 (first picture row)
|
||||
DSTE = $C38000 ; GVRAM + 224*1024 (one past last)
|
||||
BROW = 4096 ; bytes per block row (4 picture rows)
|
||||
ROWLEN = 512 ; bytes per block row of blocks (64 * 8)
|
||||
MODEB = 768 ; packed mode header, 3072 blocks * 2 bits
|
||||
SPCU = 12 ; bytes of code per COARSE span unit (24 px)
|
||||
SPCN = 11 ; coarse units: 11*24 = 264 px >= one row
|
||||
SPFU = 2 ; bytes of code per FINE span unit (2 px)
|
||||
SPFN = 11 ; fine units: 11*2 = 22 px > one coarse unit
|
||||
|
||||
org $10000
|
||||
start:
|
||||
@@ -130,186 +120,4 @@ hold: bra.s hold
|
||||
desync: move.l #$EE,FLAG.l
|
||||
bra.s hold
|
||||
|
||||
; ---------------------------------------------------------------- one block
|
||||
; \1 = right-shift needed to bring this block's 2 mode bits to bits 1-0.
|
||||
BLOCK macro
|
||||
move.b (a1),d0
|
||||
ifne \1
|
||||
lsr.b #\1,d0
|
||||
endc
|
||||
and.w #3,d0
|
||||
beq .sk\@ ; 00 SKIP -- the median block
|
||||
subq.w #1,d0
|
||||
beq .v1\@ ; 01 V1
|
||||
subq.w #1,d0
|
||||
bne .rw\@ ; 11 RAW, else 10 V4
|
||||
|
||||
; -- V4: four 2x2 codewords, sub-block order TL TR BL BR (vq_hybrid.paint)
|
||||
moveq #0,d0
|
||||
move.b (a0)+,d0
|
||||
lsl.w #3,d0
|
||||
movem.l (a3,d0.w),d0-d1
|
||||
move.l d0,(a4)
|
||||
move.l d1,1024(a4)
|
||||
moveq #0,d0
|
||||
move.b (a0)+,d0
|
||||
lsl.w #3,d0
|
||||
movem.l (a3,d0.w),d0-d1
|
||||
move.l d0,4(a4)
|
||||
move.l d1,1028(a4)
|
||||
moveq #0,d0
|
||||
move.b (a0)+,d0
|
||||
lsl.w #3,d0
|
||||
movem.l (a3,d0.w),d0-d1
|
||||
move.l d0,2048(a4)
|
||||
move.l d1,3072(a4)
|
||||
moveq #0,d0
|
||||
move.b (a0)+,d0
|
||||
lsl.w #3,d0
|
||||
movem.l (a3,d0.w),d0-d1
|
||||
move.l d0,2052(a4)
|
||||
move.l d1,3076(a4)
|
||||
bra .sk\@
|
||||
|
||||
; -- V1: one 4x4 codeword, 32 bytes, straight out of the expanded codebook
|
||||
.v1\@:
|
||||
moveq #0,d0
|
||||
move.b (a0)+,d0
|
||||
lsl.w #5,d0
|
||||
movem.l (a2,d0.w),d0-d7 ; EA is resolved before the load
|
||||
movem.l d0-d1,(a4)
|
||||
movem.l d2-d3,1024(a4)
|
||||
movem.l d4-d5,2048(a4)
|
||||
movem.l d6-d7,3072(a4)
|
||||
bra .sk\@
|
||||
|
||||
; -- RAW: 16 literal palette indices. Two indices are assembled into one long
|
||||
; via swap, so each pair of pixels costs one write instead of two; the high
|
||||
; byte of each word is left as zero because the hardware discards it anyway.
|
||||
.rw\@:
|
||||
RAWPAIR 0
|
||||
RAWPAIR 4
|
||||
RAWPAIR 1024
|
||||
RAWPAIR 1028
|
||||
RAWPAIR 2048
|
||||
RAWPAIR 2052
|
||||
RAWPAIR 3072
|
||||
RAWPAIR 3076
|
||||
.sk\@:
|
||||
addq.l #8,a4
|
||||
endm
|
||||
|
||||
RAWPAIR macro
|
||||
moveq #0,d0
|
||||
move.b (a0)+,d0
|
||||
swap d0
|
||||
move.b (a0)+,d0
|
||||
move.l d0,\1(a4)
|
||||
endm
|
||||
|
||||
; ------------------------------------------------------- the span section
|
||||
; in: a0 = span section, a1 = mode header (preserved across the call)
|
||||
; out: a0 = one past the section, i.e. the block payload
|
||||
;
|
||||
; This is tools/bench/blit.s v7 verbatim, and deliberately so: the 66.0 clocks
|
||||
; per span + 9.143 per coarse pixel + 9.978 per fine pixel of FINDINGS 40 were
|
||||
; measured on exactly this instruction sequence, over thirteen span lengths, and
|
||||
; a "tidier" rewrite here would silently invalidate every span figure in
|
||||
; FINDINGS 39/40 and in tools/analysis/14_dmac_chain.py.
|
||||
;
|
||||
; The fine chain is entered by FALLING OUT of the coarse one, so a span with no
|
||||
; coarse units enters at v7cx with d0 already reloaded -- which is why the
|
||||
; coarse displacement for c=0 is SPCN*SPCU, one past the last coarse unit,
|
||||
; rather than a special case.
|
||||
paint_spans:
|
||||
move.w (a0)+,d7 ; spans in this frame
|
||||
subq.w #1,d7
|
||||
bmi spnone ; a frame may legitimately have none (the
|
||||
; chain is far past a short branch)
|
||||
move.l a1,-(sp) ; a1 is a payload register below
|
||||
spspan: move.l (a0)+,a2 ; absolute GVRAM destination
|
||||
move.w (a0)+,d0 ; (SPCN - coarse) * SPCU
|
||||
jmp spch(pc,d0.w)
|
||||
spch:
|
||||
movem.l (a0)+,d0-d6/a1/a3-a6
|
||||
movem.l d0-d6/a1/a3-a6,(a2)
|
||||
lea 48(a2),a2
|
||||
movem.l (a0)+,d0-d6/a1/a3-a6
|
||||
movem.l d0-d6/a1/a3-a6,(a2)
|
||||
lea 48(a2),a2
|
||||
movem.l (a0)+,d0-d6/a1/a3-a6
|
||||
movem.l d0-d6/a1/a3-a6,(a2)
|
||||
lea 48(a2),a2
|
||||
movem.l (a0)+,d0-d6/a1/a3-a6
|
||||
movem.l d0-d6/a1/a3-a6,(a2)
|
||||
lea 48(a2),a2
|
||||
movem.l (a0)+,d0-d6/a1/a3-a6
|
||||
movem.l d0-d6/a1/a3-a6,(a2)
|
||||
lea 48(a2),a2
|
||||
movem.l (a0)+,d0-d6/a1/a3-a6
|
||||
movem.l d0-d6/a1/a3-a6,(a2)
|
||||
lea 48(a2),a2
|
||||
movem.l (a0)+,d0-d6/a1/a3-a6
|
||||
movem.l d0-d6/a1/a3-a6,(a2)
|
||||
lea 48(a2),a2
|
||||
movem.l (a0)+,d0-d6/a1/a3-a6
|
||||
movem.l d0-d6/a1/a3-a6,(a2)
|
||||
lea 48(a2),a2
|
||||
movem.l (a0)+,d0-d6/a1/a3-a6
|
||||
movem.l d0-d6/a1/a3-a6,(a2)
|
||||
lea 48(a2),a2
|
||||
movem.l (a0)+,d0-d6/a1/a3-a6
|
||||
movem.l d0-d6/a1/a3-a6,(a2)
|
||||
lea 48(a2),a2
|
||||
movem.l (a0)+,d0-d6/a1/a3-a6
|
||||
movem.l d0-d6/a1/a3-a6,(a2)
|
||||
lea 48(a2),a2
|
||||
spcx: move.w (a0)+,d0 ; (SPFN - fine) * SPFU, from mid-stream
|
||||
jmp spfh(pc,d0.w)
|
||||
spfh:
|
||||
move.l (a0)+,(a2)+
|
||||
move.l (a0)+,(a2)+
|
||||
move.l (a0)+,(a2)+
|
||||
move.l (a0)+,(a2)+
|
||||
move.l (a0)+,(a2)+
|
||||
move.l (a0)+,(a2)+
|
||||
move.l (a0)+,(a2)+
|
||||
move.l (a0)+,(a2)+
|
||||
move.l (a0)+,(a2)+
|
||||
move.l (a0)+,(a2)+
|
||||
move.l (a0)+,(a2)+
|
||||
dbra d7,spspan
|
||||
move.l (sp)+,a1
|
||||
spnone: rts
|
||||
|
||||
; ------------------------------------------------------------- one frame
|
||||
; in: a0 = payload, a1 = packed mode header
|
||||
; out: a0 = one past the last payload byte consumed
|
||||
decode_frame:
|
||||
lea CB1,a2
|
||||
lea CB4,a3
|
||||
lea DST0,a6
|
||||
rowloop:
|
||||
move.l a6,a4
|
||||
lea ROWLEN(a6),a5
|
||||
byteloop:
|
||||
tst.b (a1) ; four SKIPs in one test
|
||||
beq allskip
|
||||
BLOCK 6
|
||||
BLOCK 4
|
||||
BLOCK 2
|
||||
BLOCK 0
|
||||
addq.l #1,a1
|
||||
cmpa.l a5,a4
|
||||
bne byteloop
|
||||
bra rowdone
|
||||
allskip:
|
||||
addq.l #1,a1
|
||||
lea 32(a4),a4
|
||||
cmpa.l a5,a4
|
||||
bne byteloop
|
||||
rowdone:
|
||||
lea BROW(a6),a6
|
||||
cmpa.l #DSTE,a6
|
||||
bne rowloop
|
||||
rts
|
||||
include "src/player/frame.i"
|
||||
@@ -0,0 +1,183 @@
|
||||
; ---------------------------------------------------------------- one block
|
||||
; \1 = right-shift needed to bring this block's 2 mode bits to bits 1-0.
|
||||
BLOCK macro
|
||||
move.b (a1),d0
|
||||
ifne \1
|
||||
lsr.b #\1,d0
|
||||
endc
|
||||
and.w #3,d0
|
||||
beq .sk\@ ; 00 SKIP -- the median block
|
||||
subq.w #1,d0
|
||||
beq .v1\@ ; 01 V1
|
||||
subq.w #1,d0
|
||||
bne .rw\@ ; 11 RAW, else 10 V4
|
||||
|
||||
; -- V4: four 2x2 codewords, sub-block order TL TR BL BR (vq_hybrid.paint)
|
||||
moveq #0,d0
|
||||
move.b (a0)+,d0
|
||||
lsl.w #3,d0
|
||||
movem.l (a3,d0.w),d0-d1
|
||||
move.l d0,(a4)
|
||||
move.l d1,1024(a4)
|
||||
moveq #0,d0
|
||||
move.b (a0)+,d0
|
||||
lsl.w #3,d0
|
||||
movem.l (a3,d0.w),d0-d1
|
||||
move.l d0,4(a4)
|
||||
move.l d1,1028(a4)
|
||||
moveq #0,d0
|
||||
move.b (a0)+,d0
|
||||
lsl.w #3,d0
|
||||
movem.l (a3,d0.w),d0-d1
|
||||
move.l d0,2048(a4)
|
||||
move.l d1,3072(a4)
|
||||
moveq #0,d0
|
||||
move.b (a0)+,d0
|
||||
lsl.w #3,d0
|
||||
movem.l (a3,d0.w),d0-d1
|
||||
move.l d0,2052(a4)
|
||||
move.l d1,3076(a4)
|
||||
bra .sk\@
|
||||
|
||||
; -- V1: one 4x4 codeword, 32 bytes, straight out of the expanded codebook
|
||||
.v1\@:
|
||||
moveq #0,d0
|
||||
move.b (a0)+,d0
|
||||
lsl.w #5,d0
|
||||
movem.l (a2,d0.w),d0-d7 ; EA is resolved before the load
|
||||
movem.l d0-d1,(a4)
|
||||
movem.l d2-d3,1024(a4)
|
||||
movem.l d4-d5,2048(a4)
|
||||
movem.l d6-d7,3072(a4)
|
||||
bra .sk\@
|
||||
|
||||
; -- RAW: 16 literal palette indices. Two indices are assembled into one long
|
||||
; via swap, so each pair of pixels costs one write instead of two; the high
|
||||
; byte of each word is left as zero because the hardware discards it anyway.
|
||||
.rw\@:
|
||||
RAWPAIR 0
|
||||
RAWPAIR 4
|
||||
RAWPAIR 1024
|
||||
RAWPAIR 1028
|
||||
RAWPAIR 2048
|
||||
RAWPAIR 2052
|
||||
RAWPAIR 3072
|
||||
RAWPAIR 3076
|
||||
.sk\@:
|
||||
addq.l #8,a4
|
||||
endm
|
||||
|
||||
RAWPAIR macro
|
||||
moveq #0,d0
|
||||
move.b (a0)+,d0
|
||||
swap d0
|
||||
move.b (a0)+,d0
|
||||
move.l d0,\1(a4)
|
||||
endm
|
||||
|
||||
; ------------------------------------------------------- the span section
|
||||
; in: a0 = span section, a1 = mode header (preserved across the call)
|
||||
; out: a0 = one past the section, i.e. the block payload
|
||||
;
|
||||
; This is tools/bench/blit.s v7 verbatim, and deliberately so: the 66.0 clocks
|
||||
; per span + 9.143 per coarse pixel + 9.978 per fine pixel of FINDINGS 40 were
|
||||
; measured on exactly this instruction sequence, over thirteen span lengths, and
|
||||
; a "tidier" rewrite here would silently invalidate every span figure in
|
||||
; FINDINGS 39/40 and in tools/analysis/14_dmac_chain.py.
|
||||
;
|
||||
; The fine chain is entered by FALLING OUT of the coarse one, so a span with no
|
||||
; coarse units enters at v7cx with d0 already reloaded -- which is why the
|
||||
; coarse displacement for c=0 is SPCN*SPCU, one past the last coarse unit,
|
||||
; rather than a special case.
|
||||
paint_spans:
|
||||
move.w (a0)+,d7 ; spans in this frame
|
||||
subq.w #1,d7
|
||||
bmi spnone ; a frame may legitimately have none (the
|
||||
; chain is far past a short branch)
|
||||
move.l a1,-(sp) ; a1 is a payload register below
|
||||
spspan: move.l (a0)+,a2 ; absolute GVRAM destination
|
||||
move.w (a0)+,d0 ; (SPCN - coarse) * SPCU
|
||||
jmp spch(pc,d0.w)
|
||||
spch:
|
||||
movem.l (a0)+,d0-d6/a1/a3-a6
|
||||
movem.l d0-d6/a1/a3-a6,(a2)
|
||||
lea 48(a2),a2
|
||||
movem.l (a0)+,d0-d6/a1/a3-a6
|
||||
movem.l d0-d6/a1/a3-a6,(a2)
|
||||
lea 48(a2),a2
|
||||
movem.l (a0)+,d0-d6/a1/a3-a6
|
||||
movem.l d0-d6/a1/a3-a6,(a2)
|
||||
lea 48(a2),a2
|
||||
movem.l (a0)+,d0-d6/a1/a3-a6
|
||||
movem.l d0-d6/a1/a3-a6,(a2)
|
||||
lea 48(a2),a2
|
||||
movem.l (a0)+,d0-d6/a1/a3-a6
|
||||
movem.l d0-d6/a1/a3-a6,(a2)
|
||||
lea 48(a2),a2
|
||||
movem.l (a0)+,d0-d6/a1/a3-a6
|
||||
movem.l d0-d6/a1/a3-a6,(a2)
|
||||
lea 48(a2),a2
|
||||
movem.l (a0)+,d0-d6/a1/a3-a6
|
||||
movem.l d0-d6/a1/a3-a6,(a2)
|
||||
lea 48(a2),a2
|
||||
movem.l (a0)+,d0-d6/a1/a3-a6
|
||||
movem.l d0-d6/a1/a3-a6,(a2)
|
||||
lea 48(a2),a2
|
||||
movem.l (a0)+,d0-d6/a1/a3-a6
|
||||
movem.l d0-d6/a1/a3-a6,(a2)
|
||||
lea 48(a2),a2
|
||||
movem.l (a0)+,d0-d6/a1/a3-a6
|
||||
movem.l d0-d6/a1/a3-a6,(a2)
|
||||
lea 48(a2),a2
|
||||
movem.l (a0)+,d0-d6/a1/a3-a6
|
||||
movem.l d0-d6/a1/a3-a6,(a2)
|
||||
lea 48(a2),a2
|
||||
spcx: move.w (a0)+,d0 ; (SPFN - fine) * SPFU, from mid-stream
|
||||
jmp spfh(pc,d0.w)
|
||||
spfh:
|
||||
move.l (a0)+,(a2)+
|
||||
move.l (a0)+,(a2)+
|
||||
move.l (a0)+,(a2)+
|
||||
move.l (a0)+,(a2)+
|
||||
move.l (a0)+,(a2)+
|
||||
move.l (a0)+,(a2)+
|
||||
move.l (a0)+,(a2)+
|
||||
move.l (a0)+,(a2)+
|
||||
move.l (a0)+,(a2)+
|
||||
move.l (a0)+,(a2)+
|
||||
move.l (a0)+,(a2)+
|
||||
dbra d7,spspan
|
||||
move.l (sp)+,a1
|
||||
spnone: rts
|
||||
|
||||
; ------------------------------------------------------------- one frame
|
||||
; in: a0 = payload, a1 = packed mode header
|
||||
; out: a0 = one past the last payload byte consumed
|
||||
decode_frame:
|
||||
lea CB1,a2
|
||||
lea CB4,a3
|
||||
lea DST0,a6
|
||||
rowloop:
|
||||
move.l a6,a4
|
||||
lea ROWLEN(a6),a5
|
||||
byteloop:
|
||||
tst.b (a1) ; four SKIPs in one test
|
||||
beq allskip
|
||||
BLOCK 6
|
||||
BLOCK 4
|
||||
BLOCK 2
|
||||
BLOCK 0
|
||||
addq.l #1,a1
|
||||
cmpa.l a5,a4
|
||||
bne byteloop
|
||||
bra rowdone
|
||||
allskip:
|
||||
addq.l #1,a1
|
||||
lea 32(a4),a4
|
||||
cmpa.l a5,a4
|
||||
bne byteloop
|
||||
rowdone:
|
||||
lea BROW(a6),a6
|
||||
cmpa.l #DSTE,a6
|
||||
bne rowloop
|
||||
rts
|
||||
@@ -0,0 +1,27 @@
|
||||
; Geometry and codebook constants shared by every front-end in src/player/.
|
||||
;
|
||||
; Split out of decode.s in session 18 so that decode.s (the preloaded-stream
|
||||
; rig, gated by tools/bench/check.sh) and stream.s (the ring-buffer streaming
|
||||
; rig, FINDINGS 49) assemble from LITERALLY THE SAME BYTES for the block loop
|
||||
; and the span chain. Those bytes are not incidental: the 66.0 clocks/span,
|
||||
; 9.143 clocks/coarse pixel and 9.978 clocks/fine pixel of FINDINGS 40, and
|
||||
; every per-block constant in FINDINGS 24/30/41, are fitted to this exact
|
||||
; instruction sequence. Two hand-maintained copies of it would drift, and the
|
||||
; drift would be invisible -- both would still decode correctly, and only the
|
||||
; cost model would be wrong.
|
||||
;
|
||||
; The split is a no-op by construction: tools/bench/check.sh asserts that
|
||||
; decode.s still assembles to the same 1,296 bytes it did before it.
|
||||
|
||||
CB1 = $20000 ; expanded 4x4 codebook
|
||||
CB4 = $22000 ; expanded 2x2 codebook
|
||||
|
||||
DST0 = $C08000 ; GVRAM + 32*1024 (first picture row)
|
||||
DSTE = $C38000 ; GVRAM + 224*1024 (one past last)
|
||||
BROW = 4096 ; bytes per block row (4 picture rows)
|
||||
ROWLEN = 512 ; bytes per block row of blocks (64 * 8)
|
||||
MODEB = 768 ; packed mode header, 3072 blocks * 2 bits
|
||||
SPCU = 12 ; bytes of code per COARSE span unit (24 px)
|
||||
SPCN = 11 ; coarse units: 11*24 = 264 px >= one row
|
||||
SPFU = 2 ; bytes of code per FINE span unit (2 px)
|
||||
SPFN = 11 ; fine units: 11*2 = 22 px > one coarse unit
|
||||
@@ -0,0 +1,184 @@
|
||||
; DLX3 frame decoder, RING-BUFFER front-end -- STATUS item 3, FINDINGS 49.
|
||||
;
|
||||
; src/player/decode.s decodes a stream that is ALREADY WHOLLY IN RAM: the rig
|
||||
; preloads 5,261,814 bytes at $30000 and walks a0 forward through all of it.
|
||||
; That gate proves the decoder is pixel-exact over a 120-frame window
|
||||
; (FINDINGS 45) and says NOTHING about how the bytes got there. The shipping
|
||||
; player never holds a window at once; it streams from a SCSI disk into a ring
|
||||
; a fraction of the size. Nothing in this tree has ever tested that path.
|
||||
;
|
||||
; WHAT IS ACTUALLY HARD ABOUT IT. The block loop and the span chain read the
|
||||
; stream with a monotonically increasing a0 and no bounds check anywhere --
|
||||
; `move.l (a0)+,d0`, `lea MODEB(a0),a0`, eleven unrolled `movem.l (a0)+`, a
|
||||
; `move.b (a0)+` per block index. None of it can survive an address that wraps
|
||||
; mid-record. So the ring does not merely need ENOUGH BYTES resident by the
|
||||
; deadline -- 09_buffer_sim.py's question, and FINDINGS 21's answer -- it needs
|
||||
; the WHOLE NEXT RECORD resident and CONTIGUOUS.
|
||||
;
|
||||
; THE WRAP POLICY IS `aligned`, and it was chosen on measurement, not taste
|
||||
; (tools/analysis/19_ring_stream.py). The producer refuses to start a record it
|
||||
; cannot finish before the end of the ring: it leaves a hole and restarts at 0.
|
||||
;
|
||||
; aligned costs RAM -- a mean hole of 23.4 KB in a 256 KB ring, 9.1% of it --
|
||||
; and ZERO CPU.
|
||||
; split lets records wrap and mirrors the ring's first MAXREC bytes into a
|
||||
; shadow past its end, so any record start reads linearly. Costs
|
||||
; zero RAM and 46,394 clocks/frame of memcpy -- 5.57% of the frame
|
||||
; budget, forever.
|
||||
;
|
||||
; Both figures are for s14_d5_all1500, the shipping candidate, in a 256 KB ring;
|
||||
; they scale with record size, so they are per container, not universal. The
|
||||
; lighter gate container makes it 5.7% of the ring against 3.64% of the budget --
|
||||
; same direction, same verdict.
|
||||
;
|
||||
; The decoder already spends 77.0% of the budget on the mean frame and 91.1% at
|
||||
; p90 (FINDINGS 45). 5.57% more puts p90 at 96.7%. RAM is the resource this
|
||||
; machine has 2 MB of and clocks are the one it has none of, so the trade is not
|
||||
; close. `aligned` also needs a per-record INDEX on the fill side, which a
|
||||
; BRANCHING laserdisc game needs anyway to seek to a branch point -- so the
|
||||
; policy that costs no clocks also reuses a structure the player cannot avoid.
|
||||
;
|
||||
; The third option -- teach the block loop to wrap its own reads -- is the
|
||||
; expensive one and not because of the branch. A bounds test lands INSIDE the
|
||||
; instruction sequences FINDINGS 30.4 and 40 fitted their constants to, so it
|
||||
; does not cost a compare, it costs every span and per-block figure in the tree
|
||||
; being re-measured.
|
||||
;
|
||||
; THE PRODUCER IS OUTSIDE THIS FILE. Here it is tools/bench/stream.lua playing
|
||||
; a SCSI disk at a modelled byte rate; in the player it is the MB89352 and a
|
||||
; DMAC channel. The handshake is deliberately the same either way:
|
||||
;
|
||||
; producer -> FR_HEAD count of records made wholly resident (monotonic)
|
||||
; DESC[] ring of record base addresses, DESCN entries
|
||||
; decoder -> FR_TAIL count of records consumed (monotonic)
|
||||
; RD_PTR one past the last byte read; everything below is free
|
||||
;
|
||||
; Two monotonic counters and a released-to pointer -- no lock, no shared cursor,
|
||||
; single reader and single writer, so it is correct on a 68000 with no atomics
|
||||
; provided each side only ever writes its own words. That is why FR_TAIL is
|
||||
; the decoder's and FR_HEAD is the producer's rather than one shared index.
|
||||
;
|
||||
; STALLS ARE COUNTED, NOT HIDDEN. A frame whose record is not resident when the
|
||||
; decoder wants it spins in `waitrec`, and STALLS counts the FRAMES that had to
|
||||
; wait at all (not the polls). A rig that silently absorbed an underrun would
|
||||
; report a pixel-exact decode of a stream that arrived late, which is precisely
|
||||
; the failure this front-end exists to make visible. The spin is bounded:
|
||||
; SPINMAX polls without progress sets FLAG=$E1, so a wedged producer fails as a
|
||||
; wedged producer instead of as a MAME timeout with no diagnosis (FINDINGS 34.1).
|
||||
|
||||
FLAG = $18000 ; 0 idle / 1 running / $FF done / $EE desync
|
||||
; / $E1 producer stalled out
|
||||
ITER = $18008 ; outer repeat count, written by Lua
|
||||
NFR = $1800C ; frames per pass
|
||||
SCR_N = $18014 ; frames remaining this pass
|
||||
SCR_END = $18018 ; expected end of the current payload
|
||||
RD_PTR = $18020 ; decoder -> producer: released up to here
|
||||
FR_HEAD = $18024 ; producer -> decoder: records resident
|
||||
FR_TAIL = $18028 ; decoder -> producer: records consumed
|
||||
STALLS = $1802C ; frames that had to wait for their record
|
||||
SPINS = $18030 ; total poll iterations spent waiting
|
||||
PACE = $18034 ; producer -> decoder: frame ticks elapsed since
|
||||
; release. Frame i may not START before tick i.
|
||||
PACEON = $18038 ; 1 = obey PACE. 0 leaves the loop free-running,
|
||||
; byte for byte the loop FINDINGS 49 measured.
|
||||
DESC = $18100 ; DESCN x u32, record base addresses
|
||||
|
||||
DESCN = 64 ; power of two; the index is masked, not compared
|
||||
DESCM = (DESCN-1)*4 ; mask for a BYTE offset into DESC
|
||||
|
||||
SPINMAX = 2000000 ; polls with no progress before giving up
|
||||
|
||||
include "src/player/geom.i"
|
||||
|
||||
org $10000
|
||||
start:
|
||||
move.l #1,FLAG.l ; timer starts here
|
||||
outer:
|
||||
move.l NFR.l,SCR_N.l
|
||||
clr.l FR_TAIL.l
|
||||
clr.l STALLS.l
|
||||
clr.l SPINS.l
|
||||
frameloop:
|
||||
; ---- PACE GATE (FINDINGS 49.7.2, and it is the whole point of this session).
|
||||
; Free-running, this loop asks for record i the instant it finishes record i-1,
|
||||
; so it outruns any finite pipe, the ring NEVER backs up, and the producer's
|
||||
; overlap test never refuses a placement. A ring-size sweep under those
|
||||
; conditions tests WRAP CORRECTNESS at each size and nothing about BUFFERING:
|
||||
; 48 KB passes while holding one record. A shipping player does not do this --
|
||||
; it draws frame i, waits for its slot, and spends the rest of the frame time
|
||||
; idle while the disk fills the ring behind it.
|
||||
;
|
||||
; So the rig gets a frame clock. PACE is bumped by the producer (in the player,
|
||||
; vblank or an MFP timer) and frame i is forbidden to start before tick i. With
|
||||
; the decoder held to 12 fps the ring fills, the producer starts hitting its own
|
||||
; overlap test, and FR_HEAD-FR_TAIL becomes what it claims to be: the number of
|
||||
; whole frames the decoder could run on if delivery stopped dead -- which is the
|
||||
; branch-point seek question stated in frames.
|
||||
;
|
||||
; It also makes STALLS mean something. Free-running, a stall is EARLINESS
|
||||
; (49.6); paced, a frame that has to wait for its record is a real underrun.
|
||||
tst.l PACEON.l
|
||||
beq.s nopace
|
||||
pacewait:
|
||||
move.l PACE.l,d0
|
||||
cmp.l FR_TAIL.l,d0 ; d0 - FR_TAIL; carry = tick not reached
|
||||
bcs.s pacewait
|
||||
nopace:
|
||||
; ---- wait until the producer has made this record wholly resident.
|
||||
; d1 counts polls for this frame; a nonzero d1 on exit means the frame stalled.
|
||||
moveq #0,d1
|
||||
move.l FR_TAIL.l,d2
|
||||
waitrec:
|
||||
move.l FR_HEAD.l,d0
|
||||
cmp.l d2,d0
|
||||
bhi.s gotrec ; HEAD > TAIL: at least one record ready
|
||||
addq.l #1,d1
|
||||
cmp.l #SPINMAX,d1
|
||||
bcs.s waitrec
|
||||
move.l #$E1,FLAG.l ; producer never delivered
|
||||
bra hold
|
||||
gotrec:
|
||||
tst.l d1
|
||||
beq.s nostall
|
||||
addq.l #1,STALLS.l
|
||||
add.l d1,SPINS.l
|
||||
nostall:
|
||||
; ---- pop the descriptor. DESCN is a power of two, so the wrap is an and.
|
||||
move.l d2,d0
|
||||
lsl.l #2,d0
|
||||
and.w #DESCM,d0
|
||||
lea DESC,a1 ; DESC is absolute; (d0.w) needs a base
|
||||
move.l (a1,d0.w),a0 ; a1 is reloaded from a0 two lines below
|
||||
|
||||
; ---- from here to the release, byte for byte what decode.s does. a0 is
|
||||
; inside the ring rather than inside a preloaded blob, and the block loop
|
||||
; cannot tell the difference -- which is the whole claim being tested.
|
||||
move.l (a0)+,d0 ; u32 payload length, big-endian
|
||||
lea 0(a0,d0.l),a1
|
||||
move.l a1,SCR_END.l ; where the payload must end
|
||||
move.l a0,a1 ; a1 = packed mode header
|
||||
lea MODEB(a0),a0 ; a0 = span section
|
||||
bsr paint_spans ; -> a0 = block payload, a1 preserved
|
||||
bsr decode_frame
|
||||
cmpa.l SCR_END.l,a0 ; bitstream desync is silent otherwise
|
||||
bne desync
|
||||
|
||||
; ---- release. Round up to 4 the same way decode.s does: the producer lays
|
||||
; records on 4-byte boundaries, so the byte one past this record's padded
|
||||
; end is the first byte the producer may reuse.
|
||||
move.l a0,d0
|
||||
addq.l #3,d0
|
||||
and.b #$FC,d0
|
||||
move.l d0,RD_PTR.l
|
||||
addq.l #1,FR_TAIL.l
|
||||
|
||||
subq.l #1,SCR_N.l
|
||||
bne frameloop
|
||||
subq.l #1,ITER.l
|
||||
bne outer
|
||||
move.l #$FF,FLAG.l ; timer stops here
|
||||
hold: bra.s hold
|
||||
desync: move.l #$EE,FLAG.l
|
||||
bra.s hold
|
||||
|
||||
include "src/player/frame.i"
|
||||
@@ -1,4 +1,4 @@
|
||||
"""At a 488 KB/s (4 Mbps) ceiling and ~52% mean utilisation, the mean is not the
|
||||
"""At any fixed delivery ceiling and ~52% mean utilisation, the mean is not the
|
||||
risk -- the peaks are. Measure per-frame peak-to-mean, then check whether the
|
||||
leaky-bucket rate controller actually holds the ceiling."""
|
||||
import sys; sys.path.insert(0,'tools/encoder')
|
||||
|
||||
@@ -24,7 +24,13 @@ import numpy as np
|
||||
from dlx import DLX
|
||||
import vq_hybrid as H
|
||||
import ratectl as RC
|
||||
RC_AUDIO_BPS = RC.AUDIO_KBPS * 1024
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
import buscost as B
|
||||
# The audio byte rate is now DERIVED, not restated: 15.6 kHz mono MSM6258V is
|
||||
# 15,625 4-bit samples/s, two to a byte. RC.AUDIO_KBPS's 7.8 is that figure in
|
||||
# DECIMAL kB, and was being multiplied by 1024 here -- a 2.4% overstatement,
|
||||
# harmless, but it hid which unit the constant was in.
|
||||
RC_AUDIO_BPS = B.ADPCM_BYTES_PER_S
|
||||
|
||||
# Machine clocks, confirmed from MAME 0.277 src/mame/sharp/x68k.cpp:1133/1194/
|
||||
# 1200 -- not recalled. x68000 and x68ksupr are BOTH 40_MHz_XTAL/4 = 10 MHz;
|
||||
@@ -58,8 +64,26 @@ ap.add_argument("--dma-clocks-per-word", type=float, default=8.0,
|
||||
help="HD63450 cycle-steal. ESTIMATE from FINDINGS 5, NEVER "
|
||||
"MEASURED, and the most load-bearing unmeasured number "
|
||||
"in the project (FINDINGS 35.3)")
|
||||
ap.add_argument("--dma-clocks-per-byte", type=float, default=5.0,
|
||||
help="what the SCSI DMA costs per DELIVERED BYTE. The MB89352 "
|
||||
"is an 8-bit port, so the DMAC pays per byte and the "
|
||||
"per-word denominator of FINDINGS 5/39.7 was half the "
|
||||
"real debit (FINDINGS 43). 5 = single-address, bus held, "
|
||||
"no drive wait; 9 = dual-address")
|
||||
ap.add_argument("--pio-clocks-per-byte", type=float, default=12.0,
|
||||
help="hand-derived floor for a 68000 register-to-RAM copy")
|
||||
# Audio is NOT the disk, and charging it the disk's rate was charging it the
|
||||
# favourable side of an open question. tools/analysis/21_iplrom_dmac.py reads
|
||||
# the IPL ROM's own HD63450 setup: channel 3 is dual address, 8-bit port, cycle
|
||||
# steal WITHOUT hold, external request -- one full arbitration per byte, no
|
||||
# burst to amortise it over. 16 is the datasheet best case, 19 the worst.
|
||||
ap.add_argument("--adpcm-clocks-per-byte", type=float,
|
||||
default=B.ADPCM_CLK_BYTE_BEST,
|
||||
help="what an ADPCM byte costs. READ OUT OF THE IPL ROM's DMAC "
|
||||
"configuration (21_iplrom_dmac.py), not assumed: dual "
|
||||
"address + per-byte arbitration = 16 best, 19 worst. The "
|
||||
"audio stream always DMAs, whatever --io says about the "
|
||||
"disk")
|
||||
a = ap.parse_args()
|
||||
CPUHZ = CLOCKS[a.machine] * 1e6
|
||||
FPS = a.fps
|
||||
@@ -72,13 +96,15 @@ d = DLX(a.container)
|
||||
# --- what the transfer costs, from the container's own byte rate
|
||||
vid_bps = sum(n + 4 for (_, n) in d.frames) / d.nframes * d.fps
|
||||
io_bps = vid_bps + RC_AUDIO_BPS
|
||||
aud_cycles_per_s = RC_AUDIO_BPS * a.adpcm_clocks_per_byte
|
||||
if a.io == "dma":
|
||||
io_cycles_per_s = (io_bps / 2) * a.dma_clocks_per_word
|
||||
io_cycles_per_s = vid_bps * a.dma_clocks_per_byte + aud_cycles_per_s
|
||||
elif a.io == "pio":
|
||||
io_cycles_per_s = io_bps * a.pio_clocks_per_byte
|
||||
io_cycles_per_s = vid_bps * a.pio_clocks_per_byte + aud_cycles_per_s
|
||||
else:
|
||||
io_cycles_per_s = 0.0
|
||||
io_pct = 100 * io_cycles_per_s / CPUHZ
|
||||
aud_pct = 100 * aud_cycles_per_s / CPUHZ
|
||||
FRAME_NET = FRAME * (1 - io_pct / 100)
|
||||
|
||||
modes = [d.modes(f) for f in range(d.nframes)]
|
||||
@@ -91,9 +117,20 @@ print(f"budget: {a.machine} @ {CLOCKS[a.machine]:.2f} MHz, {FPS:g} fps "
|
||||
f"-> {FRAME:,.0f} cycles/frame")
|
||||
print(f" I/O ({a.io}): {io_bps/1024:.1f} KB/s costs {io_pct:.1f}% of the CPU "
|
||||
f"-> {FRAME_NET:,.0f} cycles/frame left for decoding")
|
||||
if a.io != "none":
|
||||
print(f" video {vid_bps/1024:6.1f} KB/s x "
|
||||
f"{(a.dma_clocks_per_byte if a.io=='dma' else a.pio_clocks_per_byte):g}"
|
||||
f" clk/B = {io_pct-aud_pct:5.2f}% "
|
||||
f"(W: still open, ROADMAP B3 / FINDINGS 42.4)\n"
|
||||
f" audio {RC_AUDIO_BPS/1024:6.2f} KB/s x {a.adpcm_clocks_per_byte:g}"
|
||||
f" clk/B = {aud_pct:5.2f}% "
|
||||
f"(SETTLED: read out of the IPL ROM, FINDINGS 52)")
|
||||
if a.io == "dma":
|
||||
print(f" {a.dma_clocks_per_word:g} clocks/word is an ESTIMATE (FINDINGS 5), "
|
||||
f"never measured -- see FINDINGS 35.3")
|
||||
print(f" {a.dma_clocks_per_byte:g} clocks/BYTE, the MC68450 datasheet "
|
||||
f"floor for an 8-bit port (FINDINGS 43).\n It is not measured on "
|
||||
f"hardware; what IS settled is that the per-word denominator this\n"
|
||||
f" used before session 14 was physically impossible -- 2.5 "
|
||||
f"clocks/byte is below\n the 68000's 4-clock minimum bus cycle.")
|
||||
elif a.io == "none":
|
||||
print(" WARNING: --io none scores the decoder as if the disk were free. "
|
||||
"That is the\n premise FINDINGS 35 overturned; every 'N frames miss' "
|
||||
@@ -114,15 +151,20 @@ TIMED_FRAMES = (("min non-SKIP", 15.4, 31.5), ("median", 48.1, 73.8),
|
||||
("p90", 82.5, 116.4), ("max non-SKIP", 100.0, 135.8))
|
||||
if (os.path.abspath(a.container) == os.path.abspath(TIMED)
|
||||
and a.machine == "stock" and a.fps == 12):
|
||||
print("model vs the frames actually timed on the 68000:")
|
||||
print("model vs the frames actually timed on the 68000 "
|
||||
"(the model reads HIGH, and by more\n as the frame gets harder -- "
|
||||
"so a 'does not fit' from it is the safe direction):")
|
||||
for label, frac, meas in TIMED_FRAMES:
|
||||
i = int(np.argmin(abs(ns - frac)))
|
||||
print(f" {label:<14} non-SKIP {ns[i]:5.1f}% model {pct[i]:6.1f}% "
|
||||
f"measured {meas:5.1f}% error {pct[i]-meas:+.1f} pt")
|
||||
else:
|
||||
print(f"(no 68000 timings for this container/machine -- the model was "
|
||||
f"validated to\n within 1 pt on {TIMED} at stock/12fps;\n"
|
||||
f" run tools/bench/decode.lua to time another container)")
|
||||
print(f"(no 68000 timings for this container/machine. The model is "
|
||||
f"validated against four\n frames timed on the 68000, and only on "
|
||||
f"{TIMED}\n at stock/12fps -- run it on that container to see the "
|
||||
f"errors, which are a few points\n CONSERVATIVE and grow with the "
|
||||
f"non-SKIP fraction. Run tools/bench/decode.lua to\n time another "
|
||||
f"container.)")
|
||||
|
||||
print(f"\nper-frame cost, % of a {FPS:g}fps frame budget:")
|
||||
print(f" measured-cost model: median {np.median(pct):5.1f} "
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
#!/usr/bin/env python3
|
||||
"""What does spending the idle bus bandwidth buy back in CPU cycles?
|
||||
|
||||
python3 tools/analysis/12_span_tradeoff.py [container.dlx] [--bus 488]
|
||||
python3 tools/analysis/12_span_tradeoff.py [container.dlx] --bus <KB/s>
|
||||
|
||||
FINDINGS 28 leaves the decoder CPU-bound at 110 KB/s on a 488 KB/s pipe. Every
|
||||
FINDINGS 28 leaves the decoder CPU-bound at 110 KB/s on a much wider pipe. Every
|
||||
codec decision was made when bytes were scarce, so each one trades cycles to
|
||||
save them -- and the cheapest thing a 68000 can be handed is the most expensive
|
||||
thing to store: word-expanded pixels in row-linear runs.
|
||||
@@ -50,8 +50,8 @@ def span_px(npix): # a span is a whole number of units
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("container", nargs="?",
|
||||
default="tmp/rc_fr_singe_sasi_rcprofile.dlx")
|
||||
ap.add_argument("--bus", type=float, default=488.0,
|
||||
help="sustained KB/s the pipe delivers (FINDINGS 21)")
|
||||
ap.add_argument("--bus", type=float, required=True,
|
||||
help="REQUIRED. There is no default: the delivery rate is a property of the medium and this project has never measured it. FINDINGS 42.1 -- the figure this tool used to default to was a user-supplied '4 Mbps' with no provenance, was a tenth of SCSI-1's asynchronous rating, and was never a bus measurement at all. A default let every table in FINDINGS 30-49 be scored against it without anyone restating it. Pass one explicitly.")
|
||||
ap.add_argument("--fps", type=float, default=12.0)
|
||||
a = ap.parse_args()
|
||||
if not os.path.exists(a.container):
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Would letting the HD63450 paint the spans beat letting the 68000 do it?
|
||||
|
||||
python3 tools/analysis/14_dmac_chain.py [container.dlx] [--bus 488]
|
||||
python3 tools/analysis/14_dmac_chain.py [container.dlx] --bus <KB/s>
|
||||
[--dma-px-bus 2] [--disk-bus-byte 1]
|
||||
|
||||
FINDINGS 29.6 called this the one lever that could move the CPU budget without
|
||||
@@ -11,7 +11,7 @@ prices the two against each other, and the answer turns on a resource neither
|
||||
section costed: the 68000's own LOCAL BUS.
|
||||
|
||||
FINDINGS 29's "the bus has 4x the headroom the CPU has" is about the SCSI pipe,
|
||||
110 KB/s of 488. That is a different bus. The 68000's memory bus runs one 4-clock
|
||||
110 KB/s of the delivery pipe. That is a different bus. The 68000's memory bus runs one 4-clock
|
||||
cycle at a time and carries instruction prefetch as well as data, and
|
||||
tools/analysis/15_bus_occupancy.py measures the decoder using 86.7% of it.
|
||||
|
||||
@@ -65,17 +65,28 @@ SPAN_BYTES_PX, SPAN_HDR = 2, 6
|
||||
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("container", nargs="?", default="tmp/rc_fr_singe_scsi_cpufit.dlx")
|
||||
ap.add_argument("--bus", type=float, default=488.0, help="SCSI pipe, KB/s")
|
||||
ap.add_argument("--bus", type=float, required=True,
|
||||
help="REQUIRED. There is no default: the delivery rate is a property of the medium and this project has never measured it. FINDINGS 42.1 -- the figure this tool used to default to was a user-supplied '4 Mbps' with no provenance, was a tenth of SCSI-1's asynchronous rating, and was never a bus measurement at all. A default let every table in FINDINGS 30-49 be scored against it without anyone restating it. Pass one explicitly.")
|
||||
ap.add_argument("--fps", type=float, default=12.0)
|
||||
ap.add_argument("--dma-px-clk", type=float, default=B.DMA_PX_CLK,
|
||||
help="clocks the DMAC spends per pixel, dual-address word "
|
||||
"between two 16-bit ports. 9 is the DATASHEET figure "
|
||||
"(MC68450 Fig 4-25 sheet 4).")
|
||||
ap.add_argument("--disk-clk-word", type=float, default=8.0,
|
||||
help="clocks the SCSI DMA steals per word. The datasheet "
|
||||
"brackets it at 5 (DMAC holds the bus) to 12 (arbitrates "
|
||||
"per word); FINDINGS 5's estimate of 8 is the midpoint.")
|
||||
ap.add_argument("--disk-clk-byte", type=float, default=5.0,
|
||||
help="clocks the SCSI DMA steals per BYTE delivered. The SPC is "
|
||||
"an 8-bit port, so the DMAC pays per byte, not per word "
|
||||
"(FINDINGS 43). 5, the default, is the OPTIMISTIC end and "
|
||||
"what ratectl encodes against: single-address, bus held, no "
|
||||
"drive wait (Fig 4-25 sheet 2). 9 is dual-address, which is "
|
||||
"what MAME models and what applies if the board does not "
|
||||
"drive DACK. Score both.")
|
||||
ap.add_argument("--disk-clk-word", type=float, default=None,
|
||||
help="DEPRECATED denominator of FINDINGS 39.7/42, kept so the "
|
||||
"old tables reproduce: sets --disk-clk-byte to half this")
|
||||
a = ap.parse_args()
|
||||
if a.disk_clk_word is not None:
|
||||
a.disk_clk_byte = a.disk_clk_word / 2.0
|
||||
|
||||
if not os.path.exists(a.container):
|
||||
sys.exit(f"missing {a.container}")
|
||||
|
||||
@@ -154,7 +165,7 @@ def score(design):
|
||||
for k, c in BLK_C.items():
|
||||
cpu += (mm == k).sum() * c
|
||||
pref, data = B.block_bus(m, spanned)
|
||||
disk = byt / 2.0 * a.disk_clk_word
|
||||
disk = byt * a.disk_clk_byte
|
||||
# additive: CPU work, then span painting, then the disk stealing the bus
|
||||
out.append((cpu + span_clk + disk, (pref + data) * B.BUS_CLK, byt,
|
||||
spanned.sum()))
|
||||
@@ -192,7 +203,7 @@ row("frames missing", lambda v: f"{v}/{d.nframes}",
|
||||
row("blocks spanned/frame", lambda v: f"{v:,.0f}", lambda r: r[3].mean())
|
||||
print(f"\n ADDITIVE: frame = CPU + span painting + disk DMA. The 68000 has no"
|
||||
f"\n cache and a two-word prefetch queue, so it stalls the moment another"
|
||||
f"\n master takes the bus. Disk debited at {a.disk_clk_word:g} clocks/word.")
|
||||
f"\n master takes the bus. Disk debited at {a.disk_clk_byte:g} clocks/byte.")
|
||||
|
||||
# What is left of the case, isolated.
|
||||
v6m = int((res["v6 span"][0] > FRAME_CYC).sum())
|
||||
|
||||
@@ -140,3 +140,65 @@ print(f"\nprefetch is {100*pref_t.sum()/tot.sum():.0f}% of the decoder's bus tra
|
||||
print(f"A DMAC painting spans at 8 clocks (2 bus cycles) per pixel could use at\n"
|
||||
f"most {free.mean()/2:,.0f} pixels' worth of the mean frame's spare slots "
|
||||
f"-- against {d.nb*16:,} pixels\nin a whole screen.")
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# THE OTHER TWO MASTERS. Everything above is the 68000's own traffic, and it
|
||||
# was the whole of this tool until session 20. The frame also has to carry the
|
||||
# bitstream in off the disk and a byte of ADPCM out to $E92003 every 128 us,
|
||||
# and neither has ever appeared in a bus figure -- FINDINGS 35's lesson, which
|
||||
# was about the CLOCK budget, had never been applied to the BUS one.
|
||||
#
|
||||
# The DMAC does not overlap with the CPU (buscost.DMA_OVERLAPS = False): the
|
||||
# 68000 has no cache and a two-word prefetch queue that empties at once, so a
|
||||
# stolen bus cycle is a stopped CPU. The three demands therefore ADD.
|
||||
#
|
||||
# Audio's per-byte figure is SETTLED, not bracketed by taste:
|
||||
# tools/analysis/21_iplrom_dmac.py reads the IPL ROM's own HD63450 setup and
|
||||
# finds channel 3 dual-address, 8-bit port, cycle steal without hold, external
|
||||
# request -- one arbitration per byte, no burst. Video's is NOT settled: it is
|
||||
# ROADMAP B3 / FINDINGS 42.4-42.6's W, so it is swept rather than picked.
|
||||
print("\n" + "=" * 72)
|
||||
print("THE OTHER TWO MASTERS -- what the DMAC takes out of the same frame\n")
|
||||
FPS = d.fps
|
||||
CPUHZ = 10e6 # stock X68000, MAME 0.277 x68k.cpp:1133
|
||||
FRAME_CLK = CPUHZ / FPS
|
||||
vid_bpf = sum(n + 4 for (_, n) in d.frames[:NF]) / NF # DLX2 record padding
|
||||
aud_bpf = B.ADPCM_BYTES_PER_S / FPS
|
||||
a_lo = aud_bpf * B.ADPCM_CLK_BYTE_BEST
|
||||
a_hi = aud_bpf * B.ADPCM_CLK_BYTE_WORST
|
||||
cpu_clk = cyc_t.mean() if cyc_t.any() else float("nan")
|
||||
|
||||
print(f"frame period at {FPS:g} fps on a 10 MHz 68000: {FRAME_CLK:,.0f} clocks")
|
||||
if cyc_t.any():
|
||||
print(f" decoder, MEASURED (C68K) {cpu_clk:>10,.0f} clk "
|
||||
f"{100*cpu_clk/FRAME_CLK:5.1f}% worst frame "
|
||||
f"{100*cyc_t.max()/FRAME_CLK:.1f}%")
|
||||
print(f" audio DMA, {aud_bpf:,.1f} B/frame {a_lo:>10,.0f} clk "
|
||||
f"{100*a_lo/FRAME_CLK:5.2f}% .. {a_hi:,.0f} clk "
|
||||
f"({100*a_hi/FRAME_CLK:.2f}%)")
|
||||
print(f" {B.ADPCM_CLK_BYTE_BEST}..{B.ADPCM_CLK_BYTE_WORST} clk/byte, "
|
||||
f"from the ROM's own DCR/OCR (21_iplrom_dmac.py). NOT a guess, and\n"
|
||||
f" not the disk's rate: audio arbitrates for the bus once per byte "
|
||||
f"and cannot burst.")
|
||||
print(f"\n video DMA, {vid_bpf:,.0f} B/frame, swept over W -- ROADMAP B3 is "
|
||||
f"still open:")
|
||||
print(f" {'W (clk/byte)':<16}{'clk/frame':>12}{'% of frame':>12} "
|
||||
f"{'CPU+audio+video':>18}")
|
||||
for W, note in ((5.0, "single address, bus held (11_cpu_budget.py default)"),
|
||||
(8.0, "FINDINGS 5's long-standing per-word ESTIMATE"),
|
||||
(12.0, "single address, arbitrated per byte"),
|
||||
(16.0, "what the ROM programs for SASI (best case)"),
|
||||
(19.0, "what the ROM programs for SASI (worst case)")):
|
||||
v = vid_bpf * W
|
||||
tot_clk = (cpu_clk if cyc_t.any() else 0) + a_lo + v
|
||||
print(f" {W:<16.0f}{v:>12,.0f}{100*v/FRAME_CLK:>11.1f}% "
|
||||
f"{100*tot_clk/FRAME_CLK:>17.1f}% {note}")
|
||||
print(f"\n (the last column adds the MEASURED mean decode and the BEST-CASE "
|
||||
f"audio, so it is\n the optimistic end of every row. 100% is the frame "
|
||||
f"deadline at {FPS:g} fps.)")
|
||||
print(f"""
|
||||
Audio is {100*a_lo/FRAME_CLK:.2f}%..{100*a_hi/FRAME_CLK:.2f}% of the frame and video is {vid_bpf*5/FRAME_CLK*100:.0f}%..{vid_bpf*19/FRAME_CLK*100:.0f}%. The unpriced audio
|
||||
stream was never the risk P6 called it -- ON THE BUS. What the same reading of
|
||||
the ROM found is that the DISK's per-byte cost has a worked example on this
|
||||
machine, it is 16..19 clocks, and at that price this design does not fit at any
|
||||
container size. W is the number to attack, and it is a PLAYER decision.""")
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
"""GATE for the DLX3 span container: does the reference decoder reproduce the
|
||||
encoder's own reconstruction, from the emitted bytes?
|
||||
|
||||
python3 tools/analysis/16_span_roundtrip.py [frames_dir] [--kbps 488]
|
||||
python3 tools/analysis/16_span_roundtrip.py [frames_dir] --kbps <KB/s>
|
||||
|
||||
Exits non-zero if any frame differs by a single pixel.
|
||||
|
||||
@@ -23,7 +23,7 @@ generated artefact, not from an assumption. So the thresholds below are
|
||||
asserted, not printed.
|
||||
|
||||
The `--kbps` default is the BUS rate, not the `scsi` profile's 280: spans are
|
||||
bought with bytes, and 14_dmac_chain.py scores them against the 488 KB/s pipe.
|
||||
bought with bytes, and 14_dmac_chain.py scores them against the delivery pipe.
|
||||
At the profile rate the lam search has already spent the allowance and there is
|
||||
nothing left to buy a span with -- which is a real finding about the encoder
|
||||
(FINDINGS 41.2), not a reason for the gate to test nothing.
|
||||
@@ -36,7 +36,8 @@ from dlx import DLX
|
||||
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("frames_dir", nargs="?", default="tmp/fr_singe")
|
||||
ap.add_argument("--kbps", type=float, default=488.0)
|
||||
ap.add_argument("--kbps", type=float, required=True,
|
||||
help="REQUIRED. There is no default: the delivery rate is a property of the medium and this project has never measured it. FINDINGS 42.1 -- the figure this tool used to default to was a user-supplied '4 Mbps' with no provenance, was a tenth of SCSI-1's asynchronous rating, and was never a bus measurement at all. A default let every table in FINDINGS 30-49 be scored against it without anyone restating it. Pass one explicitly.")
|
||||
ap.add_argument("--out", default="tmp/s12_roundtrip")
|
||||
ap.add_argument("--cache", default=None)
|
||||
a = ap.parse_args()
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
#!/usr/bin/env python3
|
||||
"""What do the spans the ENCODER actually emitted cost, and what do they buy?
|
||||
|
||||
python3 tools/analysis/17_span_delivered.py a.dlx [b.dlx ...] [--bus 488]
|
||||
python3 tools/analysis/17_span_delivered.py a.dlx [b.dlx ...] --bus <KB/s>
|
||||
|
||||
Every span figure before this one -- FINDINGS 29 through 40, and
|
||||
tools/analysis/12 and 14 -- was scored by SIMULATING span selection over mode
|
||||
@@ -40,12 +40,24 @@ AUDIO_KBPS = 7.8
|
||||
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("containers", nargs="+")
|
||||
ap.add_argument("--bus", type=float, default=488.0, help="SCSI pipe, KB/s")
|
||||
ap.add_argument("--bus", type=float, required=True,
|
||||
help="REQUIRED. There is no default: the delivery rate is a property of the medium and this project has never measured it. FINDINGS 42.1 -- the figure this tool used to default to was a user-supplied '4 Mbps' with no provenance, was a tenth of SCSI-1's asynchronous rating, and was never a bus measurement at all. A default let every table in FINDINGS 30-49 be scored against it without anyone restating it. Pass one explicitly.")
|
||||
ap.add_argument("--fps", type=float, default=12.0)
|
||||
ap.add_argument("--disk-clk-word", type=float, default=8.0,
|
||||
help="clocks the SCSI DMA steals per word (FINDINGS 39.7 "
|
||||
"brackets it at 5..12; 8 is the midpoint)")
|
||||
ap.add_argument("--disk-clk-byte", type=float, default=5.0,
|
||||
help="clocks the SCSI DMA steals per BYTE delivered. The SPC is "
|
||||
"an 8-bit port, so the DMAC pays per byte, not per word "
|
||||
"(FINDINGS 43). 5, the default, is the OPTIMISTIC end and "
|
||||
"what ratectl encodes against: single-address, bus held, no "
|
||||
"drive wait (Fig 4-25 sheet 2). 9 is dual-address, which is "
|
||||
"what MAME models and what applies if the board does not "
|
||||
"drive DACK. Score both.")
|
||||
ap.add_argument("--disk-clk-word", type=float, default=None,
|
||||
help="DEPRECATED denominator of FINDINGS 39.7/42, kept so the "
|
||||
"old tables reproduce: sets --disk-clk-byte to half this")
|
||||
a = ap.parse_args()
|
||||
if a.disk_clk_word is not None:
|
||||
a.disk_clk_byte = a.disk_clk_word / 2.0
|
||||
|
||||
|
||||
|
||||
def score(path):
|
||||
@@ -57,7 +69,7 @@ def score(path):
|
||||
_, n = d.frames[f]
|
||||
blk = H.cycles(mode)
|
||||
spc = sum(SP.clocks(len(p)) for _, _, p in sp)
|
||||
disk = n / 2.0 * a.disk_clk_word
|
||||
disk = n * a.disk_clk_byte
|
||||
rows.append((blk, spc, disk, n, len(sp),
|
||||
sum(len(p) for _, _, p in sp)))
|
||||
return d, np.array(rows).T
|
||||
@@ -81,7 +93,7 @@ for path in a.containers:
|
||||
f"{int((tot > FRAME_CYC).sum()):>6}/{d.nframes:<3}")
|
||||
|
||||
print(f"\n ADDITIVE: frame = block decode + span painting + disk DMA, the model"
|
||||
f"\n of 14_dmac_chain.py. Disk debited at {a.disk_clk_word:g} clocks/word "
|
||||
f"\n of 14_dmac_chain.py. Disk debited at {a.disk_clk_byte:g} clocks/byte "
|
||||
f"over the\n container's own byte count; CPU budget {FRAME_CYC:,.0f} "
|
||||
f"clocks at {a.fps:g} fps.")
|
||||
|
||||
|
||||
@@ -0,0 +1,88 @@
|
||||
#!/usr/bin/env python3
|
||||
"""What 256 -> 16 colours actually costs, on real frames.
|
||||
|
||||
python3 tools/analysis/18_text_plane_16col.py [frames_dir]
|
||||
|
||||
FINDINGS 46.3 opened a lead and could not price it: the X68000 text plane is
|
||||
4bpp planar -- 0.5 bytes/pixel against the graphics planes' 2.0 -- so a LITERAL
|
||||
uncompressed 16-colour frame is 288.0 KB/s against the shipping compressed
|
||||
256-colour container's 496.7 KB/s. 42% cheaper on the wire, with no decoder.
|
||||
|
||||
The whole lead turns on one number nobody had computed: the quality cost of 16
|
||||
colours. This computes it, and it is deliberately generous to the 16-colour
|
||||
side on every axis where the hardware allows it:
|
||||
|
||||
* PER-FRAME palettes are legitimate here. The text palette is 16 entries and
|
||||
reloading it is 16 words a frame -- nothing, against a 833,333-clock budget.
|
||||
The 256-colour path cannot do this: its palette is shared scene-wide
|
||||
(vq.scene_palette) because the codec's codebooks are indices INTO it.
|
||||
* DITHERING is free here, and only here. The tree does not dither (vq.py:32,
|
||||
"cel art is flat") because dither destroys the inter-frame coherence SKIP
|
||||
blocks and v7 spans are built on. A literal frame has no codec to wreck, so
|
||||
Floyd-Steinberg is available to this path at zero runtime cost.
|
||||
|
||||
Both are measured, so the comparison cannot be accused of hobbling the option it
|
||||
is testing. Reported against the 256-colour scene-palette ceiling (the tree's
|
||||
existing "palette ceiling" figure) and against the shipping container's PSNR.
|
||||
"""
|
||||
import sys, os
|
||||
sys.path.insert(0, "tools/encoder")
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
import vq as VQ
|
||||
|
||||
FRAMES = sys.argv[1] if len(sys.argv) > 1 else "tmp/fr_singe"
|
||||
SHIPPED_PSNR = 29.19 # docs/STATUS.md, --spans all, c=5, 496.7 KB/s
|
||||
|
||||
rgb = VQ.load_frames(FRAMES)
|
||||
H, W = rgb[0].shape[:2]
|
||||
n = len(rgb)
|
||||
print(f"{FRAMES}: {n} frames, {W}x{H}")
|
||||
print()
|
||||
|
||||
def recon_scene(colors, dither):
|
||||
"""One palette for the whole scene -- what the 256 path is forced to do."""
|
||||
d = Image.FLOYDSTEINBERG if dither else Image.NONE
|
||||
samp = np.concatenate([r.reshape(-1, 3) for r in rgb[::3]])
|
||||
ref = Image.fromarray(samp.reshape(-1, 1, 3)).quantize(
|
||||
colors=colors, method=Image.MEDIANCUT, dither=Image.NONE)
|
||||
pal = np.array(ref.getpalette()[:colors * 3], np.uint8).reshape(-1, 3)
|
||||
return [pal[np.asarray(Image.fromarray(r).quantize(palette=ref, dither=d),
|
||||
np.uint8)] for r in rgb]
|
||||
|
||||
def recon_perframe(colors, dither):
|
||||
"""A fresh palette every frame -- what the text plane can afford."""
|
||||
d = Image.FLOYDSTEINBERG if dither else Image.NONE
|
||||
out = []
|
||||
for r in rgb:
|
||||
q = Image.fromarray(r).quantize(colors=colors, method=Image.MEDIANCUT,
|
||||
dither=d)
|
||||
pal = np.array(q.getpalette()[:colors * 3], np.uint8).reshape(-1, 3)
|
||||
out.append(pal[np.asarray(q, np.uint8)])
|
||||
return out
|
||||
|
||||
def report(name, recon):
|
||||
per = np.array([VQ.psnr(a, b) for a, b in zip(rgb, recon)])
|
||||
print(f" {name:<42s} {per.mean():6.2f} dB "
|
||||
f"(min {per.min():5.2f} max {per.max():5.2f})")
|
||||
return per.mean()
|
||||
|
||||
print("PSNR vs the 24-bit source, mean over frames:")
|
||||
c256 = report("256 colours, scene palette [the tree's]", recon_scene(256, False))
|
||||
report("256 colours, per-frame palette", recon_perframe(256, False))
|
||||
print()
|
||||
s16 = report("16 colours, scene palette", recon_scene(16, False))
|
||||
p16 = report("16 colours, per-frame palette", recon_perframe(16, False))
|
||||
p16d = report("16 colours, per-frame + FS dither", recon_perframe(16, True))
|
||||
print()
|
||||
print(f" the 16-colour ceiling is the best of those: {max(s16, p16, p16d):.2f} dB")
|
||||
print(f" cost of 256 -> 16, at each side's best: "
|
||||
f"{c256 - max(s16, p16, p16d):.2f} dB")
|
||||
print()
|
||||
print(f" for scale, the shipping container delivers {SHIPPED_PSNR:.2f} dB "
|
||||
f"at 496.7 KB/s")
|
||||
print(f" a 16-colour literal would deliver "
|
||||
f"{max(s16, p16, p16d):.2f} dB at 288.0 KB/s")
|
||||
delta = max(s16, p16, p16d) - SHIPPED_PSNR
|
||||
print(f" so the text-plane path is {abs(delta):.2f} dB "
|
||||
f"{'BETTER' if delta > 0 else 'WORSE'} at 58% of the bitrate")
|
||||
@@ -0,0 +1,275 @@
|
||||
"""Ring-buffer streaming simulation, against the CONTIGUITY constraint (STATUS 3/4).
|
||||
|
||||
09_buffer_sim.py asked one question -- does cumulative supply ever fall behind
|
||||
cumulative demand -- and answered it in BYTES. FINDINGS 21 got "zero required
|
||||
prefill" out of it at 110 and 280 KB/s. That test is necessary and not
|
||||
sufficient, and the missing half is the whole of STATUS item 3:
|
||||
|
||||
src/player/decode.s reads a frame record with a MONOTONICALLY INCREASING a0
|
||||
and no bounds check anywhere. `move.l (a0)+,d0` for the length, `lea
|
||||
MODEB(a0),a0` for the span section, eleven unrolled `movem.l (a0)+` chains,
|
||||
`move.b (a0)+` per block index. Nothing in it can survive an address that
|
||||
wraps mid-record. So the buffer does not merely need ENOUGH BYTES resident
|
||||
by the deadline -- it needs the WHOLE NEXT RECORD resident and CONTIGUOUS.
|
||||
|
||||
Having enough bytes and having them contiguous are different conditions, and a
|
||||
byte-counting simulation cannot tell them apart. This one models the ring's
|
||||
addresses, not just its occupancy.
|
||||
|
||||
THREE WRAP POLICIES, and the point of the tool is that they are not equivalent:
|
||||
|
||||
split the writer wraps mid-record; the reader cannot. Requires a SHADOW of
|
||||
the ring's first MAXREC bytes mirrored past its end, so any record
|
||||
start can be read linearly for MAXREC bytes. Every byte landing in
|
||||
that first MAXREC is written twice. Costs 68000 CLOCKS, forever, at a
|
||||
rate set by MAXREC/ring -- and those clocks come out of the same
|
||||
budget the decoder is already spending 77.0% of (FINDINGS 45).
|
||||
|
||||
aligned the writer refuses to start a record it cannot finish before the end
|
||||
of the ring; it leaves a hole and restarts at 0. Costs RAM (the mean
|
||||
hole) and nothing else -- no copy, no per-byte work. Needs a frame
|
||||
INDEX so the fill side knows record boundaries, which a branching
|
||||
laserdisc game needs anyway to seek to a branch point.
|
||||
|
||||
none the decoder handles the wrap itself. Priced here only to show what it
|
||||
would cost: a bounds test in the block loop is inside the sequence
|
||||
FINDINGS 30.4/40 fitted, so it does not cost a branch -- it costs
|
||||
every span and per-block constant in the tree being re-measured.
|
||||
Not simulated; see the note printed at the end.
|
||||
|
||||
DEADLINE MODEL, and it is the conservative one: record i must be wholly
|
||||
resident when frame i's decode BEGINS. The decoder in fact reads a record
|
||||
progressively over ~77% of a frame time, so a byte arriving mid-frame would in
|
||||
practice be in time -- but that is a race between the DMAC's fill address and
|
||||
a0, and this tool refuses to certify a design on a race it cannot see.
|
||||
|
||||
Fill is quantised to 512-byte SCSI blocks: a partial sector is not resident.
|
||||
|
||||
python3 tools/analysis/19_ring_stream.py [container ...] --kbps R [--ring KB]
|
||||
|
||||
`--kbps` is REQUIRED and has no default -- see the argument's help text.
|
||||
"""
|
||||
import sys, os, argparse
|
||||
sys.path.insert(0, "tools/encoder")
|
||||
import numpy as np
|
||||
from dlx import DLX
|
||||
import ratectl as RC
|
||||
|
||||
SECTOR = 512
|
||||
# 5 clocks/byte for a 68000 `move.l (a0)+,(a1)+` copy: 20 clocks moves 4 bytes
|
||||
# on a 16-bit bus (2 read + 2 write bus cycles at 4 clocks, plus the fetch it
|
||||
# shares with the loop). Deliberately the OPTIMISTIC figure -- a movem-shaped
|
||||
# copy is what the shadow would really use, and it is the same 5.0.
|
||||
COPY_CLK_PER_BYTE = 5.0
|
||||
CPUHZ = 10_000_000
|
||||
|
||||
|
||||
def records(path):
|
||||
"""Padded record sizes, exactly as the 68000 walks them.
|
||||
|
||||
prep_dlx.py rounds each record START up to 4 (FINDINGS 28.3), so the bytes
|
||||
the ring must hold per frame are the padded ones, not the payload.
|
||||
"""
|
||||
d = DLX(path)
|
||||
rec = np.array([4 + n + (-(4 + n) % 4) for _, n in d.frames], np.int64)
|
||||
return d, rec
|
||||
|
||||
|
||||
def simulate(rec, fill_per_frame, ring, policy, maxrec):
|
||||
"""Address-level ring simulation. Returns a dict of results.
|
||||
|
||||
The ring is modelled as a write cursor and a read cursor over `ring` bytes.
|
||||
Supply arrives at `fill_per_frame` bytes per frame time, sector-quantised.
|
||||
Record i is due at the start of frame i.
|
||||
"""
|
||||
n = len(rec)
|
||||
resident = 0.0 # bytes fully arrived and not yet consumed
|
||||
carry = 0.0 # sub-sector remainder of the fill
|
||||
wcur = 0 # write cursor within the ring
|
||||
holes = [] # bytes wasted per wrap, `aligned` policy
|
||||
shadow_bytes = 0 # bytes double-written, `split` policy
|
||||
occ = []
|
||||
prefill = 0.0
|
||||
late = []
|
||||
free = ring
|
||||
|
||||
# Required prefill is solved rather than searched: run once with an infinite
|
||||
# head start to find the worst deficit, exactly as 09_buffer_sim does, then
|
||||
# assert the ring can hold it.
|
||||
deficit = np.maximum.accumulate(np.cumsum(rec - fill_per_frame))
|
||||
prefill = float(max(0.0, deficit.max()))
|
||||
|
||||
for i, r in enumerate(rec):
|
||||
# --- supply for this frame time, sector-quantised
|
||||
avail = carry + fill_per_frame
|
||||
sectors = int(avail // SECTOR)
|
||||
got = sectors * SECTOR
|
||||
carry = avail - got
|
||||
|
||||
# --- placement: does this frame's arriving data cross the ring end?
|
||||
if policy == "aligned":
|
||||
# The writer will not start a record it cannot finish. Charge the
|
||||
# hole when the NEXT record would not fit in the tail.
|
||||
if wcur + r > ring:
|
||||
holes.append(ring - wcur)
|
||||
wcur = 0
|
||||
wcur += r
|
||||
else: # split
|
||||
end = wcur + r
|
||||
if end > ring:
|
||||
wcur = end - ring
|
||||
# every byte that landed in the first MAXREC of the ring is
|
||||
# mirrored into the shadow
|
||||
shadow_bytes += min(wcur, maxrec)
|
||||
else:
|
||||
wcur = end
|
||||
if wcur <= maxrec:
|
||||
shadow_bytes += r
|
||||
elif wcur - r < maxrec:
|
||||
shadow_bytes += maxrec - (wcur - r)
|
||||
|
||||
resident += got
|
||||
if resident + 1e-9 < r:
|
||||
late.append((i, float(r - resident)))
|
||||
resident -= r
|
||||
occ.append(resident)
|
||||
|
||||
hole_mean = float(np.mean(holes)) if holes else 0.0
|
||||
usable = ring - hole_mean if policy == "aligned" else ring
|
||||
copy_clk = shadow_bytes * COPY_CLK_PER_BYTE / max(1, n)
|
||||
return dict(prefill=prefill, late=late, occ=np.array(occ),
|
||||
holes=holes, hole_mean=hole_mean, usable=usable,
|
||||
shadow_bytes=shadow_bytes, copy_clk_per_frame=copy_clk,
|
||||
wraps=len(holes) if policy == "aligned" else None)
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("containers", nargs="*",
|
||||
default=["tmp/s14_d5_all1500.dlx",
|
||||
"tmp/rc_fr_singe_scsi_span.dlx"])
|
||||
ap.add_argument("--kbps", type=float, required=True,
|
||||
help="delivered pipe, KB/s. REQUIRED, and deliberately has "
|
||||
"no default: the delivery rate is a property of the "
|
||||
"medium and this project has never measured it. The "
|
||||
"figure that used to sit here was a user-supplied "
|
||||
"'4 Mbps' with no provenance and was never a bus "
|
||||
"measurement (FINDINGS 42.1); leaving it as a default "
|
||||
"let table after table be scored against it without "
|
||||
"anyone restating what it was.")
|
||||
ap.add_argument("--ring", type=float, default=256.0,
|
||||
help="ring size in KB (default 256, FINDINGS 21's sizing)")
|
||||
a = ap.parse_args()
|
||||
|
||||
FPS = 12
|
||||
print(f"ring {a.ring:.0f} KB sector {SECTOR} B "
|
||||
f"audio {RC.AUDIO_KBPS} KB/s debited from the pipe\n")
|
||||
|
||||
for path in a.containers:
|
||||
if not os.path.exists(path):
|
||||
print(f"{path}: MISSING -- skipped\n"); continue
|
||||
d, rec = records(path)
|
||||
maxrec = int(rec.max())
|
||||
ring = int(a.ring * 1024)
|
||||
wire = rec.mean() * FPS / 1024 + RC.AUDIO_KBPS
|
||||
|
||||
print(f"=== {path}")
|
||||
print(f" {d.nframes} frames @ {d.fps}fps, record bytes "
|
||||
f"min {rec.min():,} median {int(np.median(rec)):,} max {maxrec:,}")
|
||||
print(f" wire demand {wire:.1f} KB/s "
|
||||
f"(video {rec.mean()*FPS/1024:.1f} + audio {RC.AUDIO_KBPS}), "
|
||||
f"including the u32 length and the 4-byte record pad")
|
||||
|
||||
# A required prefill is only a startup cost if the window's MEAN demand
|
||||
# is under the pipe. If the mean is over, the deficit grows for as long
|
||||
# as the scene runs and the prefill this window reports is just how far
|
||||
# it got in 120 frames -- no ring size fixes that, and quoting a KB
|
||||
# figure for it would be the most flattering possible way to state a
|
||||
# sustained overrun. FINDINGS 21's "zero prefill" never had to make
|
||||
# this distinction because it ran far under the pipe it assumed.
|
||||
if wire > a.kbps:
|
||||
over = wire - a.kbps
|
||||
print(f" !! SUSTAINED OVERRUN at the {a.kbps:.0f} KB/s pipe: "
|
||||
f"demand exceeds supply by {over:.1f} KB/s on the MEAN, not "
|
||||
f"on a burst.")
|
||||
print(f" The deficit grows {over*1024/FPS:,.0f} B per frame "
|
||||
f"for as long as the scene runs -- {over*1024*120/FPS/1024:.0f} "
|
||||
f"KB over this 120-frame window, {over*60:.0f} KB per minute "
|
||||
f"of play. Prefill below is where it got in 120 frames, NOT a "
|
||||
f"startup cost that fixes it.")
|
||||
|
||||
if maxrec > ring:
|
||||
print(f" !! MAXREC {maxrec:,} > ring {ring:,}: no policy works. "
|
||||
f"decode.s needs one whole record contiguous.\n")
|
||||
continue
|
||||
|
||||
# --- the requirement on the medium, which is the useful output, and
|
||||
# the reason this tool takes no default rate. There is no measured
|
||||
# pipe figure to score against (42.1), and the intent is to measure
|
||||
# a BlueSCSI directly -- so the tool reports the THRESHOLD to
|
||||
# measure against. The sweep is anchored to the container's own
|
||||
# wire demand rather than to a list of fixed rates, so it stays
|
||||
# meaningful for any container and privileges no constant.
|
||||
print(f" {'pipe KB/s':>10} {'vs wire':>8} {'prefill KB':>11} "
|
||||
f"{'records':>8} {'seek slack':>11}")
|
||||
for mult in (0.90, 0.95, 1.00, 1.02, 1.05, 1.10, 1.25, 1.50, 2.00):
|
||||
kbps = wire * mult
|
||||
fill = (kbps - RC.AUDIO_KBPS) * 1024 / FPS
|
||||
r = simulate(rec, fill, ring, "aligned", maxrec)
|
||||
pf = r["prefill"]
|
||||
# Branch-point seek slack, STATICALLY: with the ring FULL, how many
|
||||
# frame times can the fill be zero before the next record is not
|
||||
# resident? It is an upper bound and it assumes the premise that
|
||||
# FINDINGS 51.3 took apart -- the ring is NOT full at a branch
|
||||
# point, it is empty, and refilling it takes seconds of play. For
|
||||
# the measured figure use tools/analysis/20_seek_slack.py, or the
|
||||
# rig itself (tools/bench/pace_run.sh). Kept here as the ceiling
|
||||
# this container's record sizes allow, which is what the rest of
|
||||
# this row is about.
|
||||
slack = (r["usable"] - maxrec) / rec.mean()
|
||||
flag = ""
|
||||
if pf + maxrec > r["usable"]:
|
||||
flag = " <- does not fit the ring"
|
||||
print(f" {kbps:>10.1f} {mult:>7.2f}x {pf/1024:>11.1f} "
|
||||
f"{pf/rec.mean():>8.2f} {slack:>8.1f} fr{flag}")
|
||||
# smallest pipe needing zero prefill, to 0.1 KB/s
|
||||
lo, hi = wire, wire + 400
|
||||
for _ in range(40):
|
||||
mid = (lo + hi) / 2
|
||||
f = (mid - RC.AUDIO_KBPS) * 1024 / FPS
|
||||
if simulate(rec, f, ring, "aligned", maxrec)["prefill"] > 0:
|
||||
lo = mid
|
||||
else:
|
||||
hi = mid
|
||||
print(f" ZERO-PREFILL PIPE: {hi:.1f} KB/s "
|
||||
f"({hi - wire:+.1f} KB/s over the wire demand, "
|
||||
f"{100*hi/wire - 100:+.1f}%)")
|
||||
print(f" ^ this is the number to measure a medium against. It is a "
|
||||
f"REQUIREMENT, not a verdict.")
|
||||
|
||||
# --- the policy trade, at the default pipe
|
||||
fill = (a.kbps - RC.AUDIO_KBPS) * 1024 / FPS
|
||||
print(f" wrap policy, at pipe {a.kbps:.0f} KB/s:")
|
||||
for policy in ("aligned", "split"):
|
||||
r = simulate(rec, fill, ring, policy, maxrec)
|
||||
if policy == "aligned":
|
||||
print(f" aligned wraps {r['wraps']:3} mean hole "
|
||||
f"{r['hole_mean']/1024:6.1f} KB usable ring "
|
||||
f"{r['usable']/1024:6.1f} KB "
|
||||
f"({100*r['usable']/ring:.1f}%) CPU cost 0")
|
||||
else:
|
||||
pct = 100 * r["copy_clk_per_frame"] / (CPUHZ / FPS)
|
||||
print(f" split shadow {r['shadow_bytes']/1024:8.1f} KB "
|
||||
f"= {r['copy_clk_per_frame']:8.0f} clk/frame = "
|
||||
f"{pct:.2f}% of the frame budget, forever RAM cost 0")
|
||||
print()
|
||||
|
||||
print("The `none` policy -- decoder wraps its own reads -- is not simulated.")
|
||||
print("It has no RAM or copy cost and it is still the expensive one: the")
|
||||
print("bounds test lands inside the exact instruction sequences FINDINGS")
|
||||
print("30.4 and 40 fitted, so it does not cost a branch, it costs every span")
|
||||
print("and per-block constant in the tree being re-measured. FINDINGS 28.3.")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,146 @@
|
||||
"""Seek slack: how long a branch point can stop delivery (STATUS 4, FINDINGS 51).
|
||||
|
||||
19_ring_stream.py asks whether a container ARRIVES in time, and prints one
|
||||
"seek slack" column derived statically as (usable ring - maxrec)/mean record.
|
||||
That is a capacity estimate and it quietly assumes the ring is full when the
|
||||
seek happens. It is not, and the difference is the whole finding:
|
||||
|
||||
A ring's slack is ACCUMULATED, not owned. It is built out of the surplus
|
||||
between the pipe and the wire demand, at (pipe - wire) bytes per second, and
|
||||
a seek spends all of it. How long a branch point can stall is a property of
|
||||
the ring; how soon the NEXT branch point can be afforded is a property of the
|
||||
surplus, and a bigger ring makes that one WORSE.
|
||||
|
||||
This is the paced-rig model (tools/bench/stream.lua with DLX_PACE=1) written
|
||||
independently, and it exists to be compared against it, not to replace it. The
|
||||
rig drives a real 68000 through a real ring and is the measurement; this is the
|
||||
cheap sweep that says where to point it. Where they disagree, the rig wins.
|
||||
|
||||
python3 tools/analysis/20_seek_slack.py [container ...] --kbps R [R ...]
|
||||
[--ring KB [KB ...]]
|
||||
|
||||
`--kbps` is REQUIRED and takes no default, for the reason FINDINGS 50 gives.
|
||||
"""
|
||||
import sys, os, argparse
|
||||
sys.path.insert(0, "tools/encoder")
|
||||
import numpy as np
|
||||
from dlx import DLX
|
||||
import ratectl as RC
|
||||
|
||||
SECTOR = 512
|
||||
|
||||
|
||||
def records(path):
|
||||
d = DLX(path)
|
||||
rec = np.array([4 + n + (-(4 + n) % 4) for _, n in d.frames], np.int64)
|
||||
return d, rec
|
||||
|
||||
|
||||
def paced_sim(rec, ring, fill_per_frame, ticks_per_frame=8):
|
||||
"""Paced-decoder ring sim. Returns TWO per-tick lookahead series.
|
||||
|
||||
THE ANSWER IS BRACKETED TO ONE RECORD AND IS NOT SHARPER THAN THAT. At
|
||||
these rates the pipe delivers almost exactly one record per frame slot, so
|
||||
"how many records are resident at slot i" depends on whether you look before
|
||||
or after that slot's delivery -- and the two answers differ by one, every
|
||||
time. Sampled after, this agreed with the rig's ceiling in 33 of 35 cells;
|
||||
sampled before, it was exactly one record lower in 33 of 35. Neither is
|
||||
wrong. Picking the one that matched would have been fitting the model to
|
||||
the measurement and then reporting the agreement as a cross-check, so both
|
||||
are returned and the caller prints the range. The rig sits at the top of it.
|
||||
|
||||
The producer is `aligned` (19_ring_stream.py): it will not start a record it
|
||||
cannot finish before the end of the ring, and it will not place one over
|
||||
bytes the decoder still owns. The decoder consumes exactly one record per
|
||||
frame time and releases it whole.
|
||||
|
||||
Sub-stepping matters. Delivery and consumption interleave inside a frame
|
||||
time on the rig -- the producer runs on MAME's machine-frame notifier, ~5x
|
||||
per 12fps slot -- and a model that delivers a whole frame's bytes at once
|
||||
can place a record into space the decoder has not released yet, or refuse
|
||||
one it has. Eight sub-steps is well past the point the answer stops moving.
|
||||
"""
|
||||
n = len(rec)
|
||||
live = [] # [idx, off, len] still owned by the decoder
|
||||
wcur, nsent, credit = 0, 0, 0.0
|
||||
lo, hi, ring_ref, rate_ref = [], [], 0, 0
|
||||
|
||||
def overlaps(off, ln):
|
||||
return any(off < r[1] + r[2] and r[1] < off + ln for r in live)
|
||||
|
||||
for i in range(n):
|
||||
if nsent < n:
|
||||
lo.append(sum(1 for r in live if r[0] >= i))
|
||||
for _ in range(ticks_per_frame):
|
||||
credit += fill_per_frame / ticks_per_frame
|
||||
while nsent < n:
|
||||
r = int(rec[nsent])
|
||||
if credit < r:
|
||||
rate_ref += 1
|
||||
break
|
||||
w, hole = wcur, 0
|
||||
if w + r > ring:
|
||||
w, hole = 0, ring - wcur
|
||||
if overlaps(w, r):
|
||||
ring_ref += 1
|
||||
break
|
||||
# sector quantisation: a partial sector is not resident
|
||||
credit -= r
|
||||
live.append([nsent, w, r])
|
||||
wcur, nsent = w + r, nsent + 1
|
||||
if nsent < n:
|
||||
hi.append(sum(1 for r in live if r[0] >= i))
|
||||
# the decoder consumed record i during the slot and releases it whole
|
||||
live = [r for r in live if r[0] > i]
|
||||
return np.array(lo), np.array(hi), ring_ref, rate_ref
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("containers", nargs="*",
|
||||
default=["tmp/rc_fr_singe_scsi_span.dlx"])
|
||||
ap.add_argument("--kbps", type=float, nargs="+", required=True,
|
||||
help="delivered pipe rates, KB/s. REQUIRED, no default "
|
||||
"(FINDINGS 50): this project has never measured the "
|
||||
"delivery pipe and a default is how the last unmeasured "
|
||||
"one stayed load-bearing for five sessions.")
|
||||
ap.add_argument("--ring", type=float, nargs="+",
|
||||
default=[64, 96, 128, 192, 256, 384, 512])
|
||||
a = ap.parse_args()
|
||||
FPS = 12
|
||||
|
||||
for path in a.containers:
|
||||
if not os.path.exists(path):
|
||||
print(f"{path}: MISSING -- skipped\n"); continue
|
||||
d, rec = records(path)
|
||||
wire = rec.mean() * FPS / 1024 + RC.AUDIO_KBPS
|
||||
print(f"=== {path}: {d.nframes} frames @ {d.fps}fps, mean record "
|
||||
f"{rec.mean()/1024:.1f} KB, wire {wire:.1f} KB/s")
|
||||
print(f"{'ring KB':>8} {'pipe':>8} {'ceiling':>9} {'build s':>8} "
|
||||
f"{'mean':>11} bound")
|
||||
for ring_kb in a.ring:
|
||||
ring = int(ring_kb * 1024)
|
||||
if rec.max() > ring:
|
||||
print(f"{ring_kb:>8.0f} maxrec {rec.max():,} does not fit")
|
||||
continue
|
||||
for kbps in a.kbps:
|
||||
fill = ((kbps - RC.AUDIO_KBPS) * 1024 / FPS) if kbps > 0 else 1e12
|
||||
lo, hi, ring_ref, rate_ref = paced_sim(rec, ring, fill)
|
||||
c_lo, c_hi = int(lo.max()), int(hi.max())
|
||||
build = int(np.argmax(hi >= c_hi)) if len(hi) else -1
|
||||
print(f"{ring_kb:>8.0f} {kbps:>8.0f} "
|
||||
f"{f'{c_lo}-{c_hi}':>9} {build/FPS:>8.2f} "
|
||||
f"{f'{lo.mean():.1f}-{hi.mean():.1f}':>11} "
|
||||
f"{'ring' if ring_ref else 'rate'}")
|
||||
# The surplus model, stated so it can be checked against the sweep
|
||||
# above rather than believed: slack accrues at (pipe - wire) and a
|
||||
# full ring holds `ceiling` records, so a branch point costs about
|
||||
# ceiling*mean_record/(pipe - wire) seconds of play to earn back.
|
||||
print()
|
||||
print("Slack is accumulated, not owned. A bigger ring raises the ceiling AND")
|
||||
print("lengthens the climb to it: the surplus (pipe - wire) is what fills it,")
|
||||
print("and that is set by the encoder and the medium, not by the buffer.")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,255 @@
|
||||
#!/usr/bin/env python3
|
||||
"""What the X68000's own ROM programs into the DMAC -- read out of the bytes.
|
||||
|
||||
python3 tools/analysis/21_iplrom_dmac.py [iplrom.dat]
|
||||
|
||||
FINDINGS 48.4 / ROADMAP B3 left the single-address vs dual-address question
|
||||
open for the disk, priced it at 242 KB/s and 0.69 dB, and blocked it on
|
||||
sourcing `scsiexrom.bin` so its DMAC init could be disassembled. The same
|
||||
question was open for AUDIO and nobody had asked it: ROADMAP P6 budgets ADPCM
|
||||
at 7.8 KB/s and `11_cpu_budget.py` charges those bytes the DISK's per-byte
|
||||
rate, which is a guess about a channel whose configuration was never read.
|
||||
|
||||
It does not have to be a guess. **The IPL ROM is on this machine** -- MAME runs
|
||||
the player rig with `-bios ipl10` -- and it programs all four HD63450 channels
|
||||
itself. This script reads the configuration straight out of the ROM image and
|
||||
decodes the MC68450 register fields, so every claim below is a byte at a named
|
||||
address rather than a recollection about a chip.
|
||||
|
||||
It is a GATE, not a report: each piece of evidence is (address, expected bytes,
|
||||
what it means), and a mismatch exits non-zero. If a different ROM revision is
|
||||
pointed at it, it says so instead of quietly decoding something else.
|
||||
|
||||
SOURCED for the field layouts: MC68450 Direct Memory Access Controller,
|
||||
Motorola, Jul 1989 (bitsavers) -- the same document FINDINGS 39 already cites
|
||||
for the transfer timings in tools/analysis/buscost.py.
|
||||
|
||||
NOTE THE LAYER: this is the ROM's own choice of configuration, read from the
|
||||
shipping image. It is not a measurement of a running machine, and it is not
|
||||
proof that a different configuration is impossible -- our player programs these
|
||||
registers itself. It is evidence about what Sharp's engineers could get the
|
||||
board to do, from the vendor, for these exact devices.
|
||||
"""
|
||||
import sys, os, argparse, hashlib
|
||||
|
||||
BASE = 0xFE0000 # where the IPL ROM is mapped (and its 0xFF0000 alias)
|
||||
|
||||
# The image this was decoded against. A different revision is a different
|
||||
# machine's answer, so it is named rather than assumed.
|
||||
KNOWN = {
|
||||
"7fd4caabac1d9169e289f0f7bbf71d8e":
|
||||
"IPL 1.0 (MAME x68000 -bios ipl10), 131,072 B",
|
||||
}
|
||||
|
||||
# --- MC68450 register map, by offset inside a channel's 0x40 block ----------
|
||||
REG = {0x00: "CSR", 0x01: "CER", 0x04: "DCR", 0x05: "OCR", 0x06: "SCR",
|
||||
0x07: "CCR", 0x0A: "MTC", 0x0C: "MAR", 0x14: "DAR", 0x1A: "BTC",
|
||||
0x1C: "BAR", 0x25: "NIV", 0x27: "EIV", 0x29: "MFC", 0x2D: "CPR",
|
||||
0x31: "DFC", 0x39: "BFC"}
|
||||
|
||||
XRM = {0: "burst",
|
||||
1: "UNDEFINED",
|
||||
2: "cycle steal WITHOUT hold (bus released between operands)",
|
||||
3: "cycle steal with hold"}
|
||||
DTYP = {0: "68000-compatible, EXPLICITLY addressed -> DUAL ADDRESS",
|
||||
1: "6800-compatible, EXPLICITLY addressed -> DUAL ADDRESS",
|
||||
2: "device with ACK, implicitly addressed -> SINGLE ADDRESS",
|
||||
3: "device with ACK and RDY, implicit -> SINGLE ADDRESS"}
|
||||
DPS = {0: "8-bit port", 1: "16-bit port"}
|
||||
PCL = {0: "status input", 1: "status input with interrupt",
|
||||
2: "start pulse", 3: "abort input"}
|
||||
SIZE = {0: "byte", 1: "word", 2: "long word", 3: "byte, unpacked"}
|
||||
CHAIN= {0: "none", 1: "UNDEFINED", 2: "array", 3: "linked array"}
|
||||
REQG = {0: "auto-request at limited rate", 1: "auto-request at max rate",
|
||||
2: "EXTERNAL request (one operand per device request)",
|
||||
3: "auto-request first operand, external thereafter"}
|
||||
|
||||
|
||||
def dcr(v):
|
||||
return [f"XRM = {v>>6&3:02b} {XRM[v>>6&3]}",
|
||||
f"DTYP = {v>>4&3:02b} {DTYP[v>>4&3]}",
|
||||
f"DPS = {v>>3&1:b} {DPS[v>>3&1]}",
|
||||
f"PCL = {v&3:02b} {PCL[v&3]}"]
|
||||
|
||||
|
||||
def ocr(v):
|
||||
return [f"DIR = {v>>7&1:b} " +
|
||||
("device -> memory (read)" if v & 0x80 else "memory -> device (write)"),
|
||||
f"SIZE = {v>>4&3:02b} {SIZE[v>>4&3]}",
|
||||
f"CHAIN= {v>>2&3:02b} {CHAIN[v>>2&3]}",
|
||||
f"REQG = {v&3:02b} {REQG[v&3]}"]
|
||||
|
||||
|
||||
def scr(v):
|
||||
m = {0: "no count", 1: "increment", 2: "decrement", 3: "UNDEFINED"}
|
||||
return [f"MAC = {v>>2&3:02b} memory address {m[v>>2&3]}",
|
||||
f"DAC = {v&3:02b} device address {m[v&3]}"]
|
||||
|
||||
|
||||
# --- the evidence ----------------------------------------------------------
|
||||
# (address, expected bytes, one-line description). Every register value quoted
|
||||
# anywhere below comes out of one of these; nothing is typed in twice.
|
||||
EV = [
|
||||
(0xFF0BEA, "49f900e84080197c00080004197c0005",
|
||||
"boot: lea $E84080,a4 (ch2) ; DCR=$08 ; SCR=$05..."),
|
||||
(0xFF0C2E, "49f900e840c0197c00800004197c00040006197c00050029197c0001002d"
|
||||
"197c00050031197c00050039297c00e92003",
|
||||
"boot: lea $E840C0,a4 (ch3, ADPCM) ; DCR=$80 SCR=$04 MFC=$05 CPR=$01 "
|
||||
"DFC=$05 BFC=$05 DAR=$E92003"),
|
||||
(0xFF0D8E, "0480060429052d0031054480460469056d027105",
|
||||
"boot: the ch0/ch1 init TABLE, ten (offset,value) pairs, written by the "
|
||||
"loop at $FF0CD8"),
|
||||
(0xFF0CE4, "217c00e940030014217c00e960010054",
|
||||
"boot: DAR ch0 = $E94003 (FDC data) ; DAR ch1 = $E96001 (SASI data)"),
|
||||
(0xFF9A82, "13fc003200e840c5610a13fc000200e920014e75",
|
||||
"IOCS ADPCM PLAY: OCR(ch3) = $32 ; then command $02 to $E92001"),
|
||||
(0xFF9A5E, "13fc00b200e840c5612e13fc000400e920014e75",
|
||||
"IOCS ADPCM RECORD: OCR(ch3) = $B2 ; then command $04 to $E92001"),
|
||||
(0xFF9A96, "13fc00ff00e840c023c900e840cc33c200e840ca",
|
||||
"IOCS ADPCM arm: CSR=$FF ; MAR = a1 ; MTC = d2 (DCR/SCR untouched)"),
|
||||
(0xFF9944, "13fc00ff00e8404013fc00b200e84045601013fc00ff00e8404013fc003200"
|
||||
"e8404523c900e8404c33c300e8404a13fc008000e840474e75",
|
||||
"IOCS SASI: OCR(ch1) = $B2 read / $32 write ; MAR ; MTC ; CCR = $80"),
|
||||
]
|
||||
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("rom", nargs="?",
|
||||
default=os.path.expanduser("~/mame/roms/iplrom.dat"))
|
||||
a = ap.parse_args()
|
||||
if not os.path.exists(a.rom):
|
||||
sys.exit(f"missing {a.rom} -- point this at the IPL ROM MAME boots the rig "
|
||||
f"with (-bios ipl10).")
|
||||
d = open(a.rom, "rb").read()
|
||||
md5 = hashlib.md5(d).hexdigest()
|
||||
print(f"{a.rom}: {len(d):,} B, md5 {md5}")
|
||||
if md5 in KNOWN:
|
||||
print(f" {KNOWN[md5]}\n")
|
||||
else:
|
||||
sys.exit(f"\nUNKNOWN ROM. Every field decoded below was read out of\n"
|
||||
f" {list(KNOWN.values())[0]}\n"
|
||||
f"and a different revision is a different machine's answer, not a "
|
||||
f"detail. Add its\nmd5 to KNOWN only after re-reading the sites -- "
|
||||
f"the addresses are revision-specific.")
|
||||
|
||||
print("EVIDENCE -- each line is bytes at an address, not a recollection")
|
||||
bad = 0
|
||||
for addr, hx, what in EV:
|
||||
want = bytes.fromhex(hx)
|
||||
got = d[addr - BASE: addr - BASE + len(want)]
|
||||
ok = got == want
|
||||
bad += not ok
|
||||
print(f" {'OK ' if ok else 'FAIL'} ${addr:06X} {what}")
|
||||
if not ok:
|
||||
print(f" expected {want.hex()}\n got {got.hex()}")
|
||||
if bad:
|
||||
sys.exit(f"\nFAIL: {bad} evidence site(s) do not hold. The decode below "
|
||||
"would be about\nsome other code, so it is not printed.")
|
||||
|
||||
# The ch0/ch1 table, decoded from the bytes rather than restated.
|
||||
tbl = d[0xFF0D8E - BASE: 0xFF0D8E - BASE + 20]
|
||||
init = {}
|
||||
for i in range(0, len(tbl), 2):
|
||||
off, val = tbl[i], tbl[i + 1]
|
||||
init[(off >> 6, off & 0x3F)] = val
|
||||
init[(2, 0x04)] = 0x08 # from the inline moves at $FF0BEA
|
||||
init[(2, 0x06)] = 0x05
|
||||
init[(2, 0x2D)] = 0x03
|
||||
init[(3, 0x04)] = 0x80 # ...and at $FF0C2E
|
||||
init[(3, 0x06)] = 0x04
|
||||
init[(3, 0x2D)] = 0x01
|
||||
|
||||
DEV = {0: ("FDC", "$E94003"), 1: ("SASI", "$E96001"),
|
||||
2: ("IOCS _DMAMOVE (general purpose)", "set per call"),
|
||||
3: ("ADPCM MSM6258V", "$E92003")}
|
||||
print("\nWHAT THE ROM PROGRAMS, per channel")
|
||||
for ch in range(4):
|
||||
name, dar = DEV[ch]
|
||||
print(f"\n ch{ch} base $E840{ch*0x40:02X} {name} DAR = {dar}")
|
||||
v = init[(ch, 0x04)]
|
||||
print(f" DCR = ${v:02X}")
|
||||
for line in dcr(v):
|
||||
print(f" {line}")
|
||||
v = init[(ch, 0x06)]
|
||||
print(f" SCR = ${v:02X} " + " ; ".join(scr(v)))
|
||||
print(f" CPR = ${init[(ch,0x2D)]:02X} channel priority "
|
||||
f"({init[(ch,0x2D)]}, 0 = highest)")
|
||||
|
||||
print("\nAND THE PER-TRANSFER OCR, written every time a transfer is armed")
|
||||
for label, ch, v in (("ADPCM playback", 3, 0x32), ("ADPCM record", 3, 0xB2),
|
||||
("SASI write", 1, 0x32), ("SASI read", 1, 0xB2)):
|
||||
print(f"\n {label:<15} ch{ch} OCR = ${v:02X}")
|
||||
for line in ocr(v):
|
||||
print(f" {line}")
|
||||
|
||||
print(f"""
|
||||
WHAT THIS SETTLES
|
||||
|
||||
1. AUDIO IS DUAL ADDRESS, AND IT CANNOT HOLD THE BUS. ch3 DCR = $80: DTYP =
|
||||
00, explicitly addressed, so every ADPCM byte is a MEMORY READ FOLLOWED BY A
|
||||
DEVICE WRITE -- not the single-address 5 clocks the disk debit is written in.
|
||||
XRM = 10 is cycle steal WITHOUT hold and OCR REQG = 10 is external request,
|
||||
so the DMAC arbitrates for the bus ONCE PER BYTE and gives it straight back.
|
||||
There is no burst to amortise the arbitration over.
|
||||
|
||||
2. THE PORT IS 8 BITS AND THE OPERAND IS A BYTE. DCR DPS = 0, OCR SIZE = 11.
|
||||
One MSM6258V byte is two 4-bit samples, so 15.6 kHz is 7,812.5 BYTES/s and
|
||||
7,812.5 DMA REQUESTS/s -- the request count does not halve the way a 16-bit
|
||||
port's would. That is the FINDINGS 43 unit trap, in the other stream.
|
||||
|
||||
3. THE DISK CHANNEL IS PROGRAMMED IDENTICALLY, AND THAT IS THE BIGGER NEWS.
|
||||
ch1 (SASI, DAR = $E96001) gets DCR = $80 and OCR = $B2 -- dual address,
|
||||
8-bit port, cycle steal WITHOUT hold, external request. Byte by byte, with a
|
||||
full arbitration each time, exactly like the audio. ch0 (FDC) too. Sharp
|
||||
programs every explicitly-addressed 8-bit device on this board the same way.
|
||||
|
||||
This is not scsiexrom.bin and it does not close ROADMAP B3 -- a different
|
||||
ROM drives a different SPC. But it is the same vendor, the same DMAC and the
|
||||
same class of device, and it lands on the EXPENSIVE side of B3's 242 KB/s.
|
||||
|
||||
4. AND IT IS OUTSIDE THE BRACKET THE PROJECT HAS BEEN COSTING P4 IN.
|
||||
FINDINGS 42.4-42.6 brackets W, the clocks stolen per delivered byte, at
|
||||
5..12, and reports that W <= 6 fits 0/120 frames while W = 8 misses 47/120.
|
||||
The ROM's own disk configuration costs 16..19. It is still true that the
|
||||
player programs these registers itself and the choice is ours (42.6) -- but
|
||||
the only worked example on the machine sits ABOVE the whole bracket, and
|
||||
nothing in this tree has yet shown that a cheaper configuration is reachable
|
||||
for an explicitly-addressed port. Treat W <= 12 as a REQUIREMENT ON THE
|
||||
PLAYER'S DMAC PROGRAMMING, not as a range the hardware hands us.
|
||||
|
||||
5. AUDIO OUTRANKS THE DISK AT THE ARBITER. CPR: FDC 0, ADPCM 1, SASI 2,
|
||||
_DMAMOVE 3, lower being higher priority. When both channels want the bus in
|
||||
the same slot, the ROM's arrangement serves ADPCM first. An audio byte is
|
||||
never the thing that waits; a video byte is.
|
||||
""")
|
||||
|
||||
# --- what it costs ---------------------------------------------------------
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
import buscost as B
|
||||
|
||||
ADPCM_HZ = 15625.0 # 8 MHz MSM6258V clock / 512
|
||||
ADPCM_BPS = ADPCM_HZ / 2 # 4-bit samples, two to a byte
|
||||
FPS, CPUHZ = 12.0, 10e6
|
||||
lo = B.DMA_DUAL_BYTE_CLK + B.DMA_FRONT_CLK + B.DMA_BACK_CLK
|
||||
hi = B.DMA_DUAL_BYTE_CLK + B.DMA_FRONT_CLK_WORST + B.DMA_BACK_CLK
|
||||
bpf = ADPCM_BPS / FPS
|
||||
print(f"WHAT IT COSTS, at the configuration above\n"
|
||||
f" 15.6 kHz mono = {ADPCM_HZ:,.0f} samples/s = {ADPCM_BPS:,.1f} B/s "
|
||||
f"= {ADPCM_BPS/1024:.2f} KiB/s\n"
|
||||
f" (ratectl.AUDIO_KBPS is 7.8, which is this figure in DECIMAL kB; "
|
||||
f"as KiB it is {ADPCM_BPS/1024:.2f})\n"
|
||||
f" dual-address byte transfer {B.DMA_DUAL_BYTE_CLK} clk "
|
||||
f"(read {B.DMA_READ_CLK} + write {B.DMA_WRITE_CLK}, Fig 4-25 sheet 4 note 2)\n"
|
||||
f" + arbitration, EVERY byte {B.DMA_FRONT_CLK}..{B.DMA_FRONT_CLK_WORST}"
|
||||
f" front + {B.DMA_BACK_CLK} back (sect 4.5.2.1/4.5.2.2)\n"
|
||||
f" = {lo}..{hi} clocks per audio byte\n\n"
|
||||
f" per frame at {FPS:g} fps: {bpf:,.1f} B costs {bpf*lo:,.0f}..{bpf*hi:,.0f} "
|
||||
f"clocks of {CPUHZ/FPS:,.0f}\n"
|
||||
f" = {100*bpf*lo/(CPUHZ/FPS):.2f}%..{100*bpf*hi/(CPUHZ/FPS):.2f}% of the "
|
||||
f"frame, stolen from the 68000\n\n"
|
||||
f" 11_cpu_budget.py charges audio --dma-clocks-per-byte, default 5, "
|
||||
f"described\n as 'single-address, bus held, no drive wait'. The ROM says "
|
||||
f"audio is neither\n single-address nor able to hold the bus, so that "
|
||||
f"debit is {lo/5:.1f}x..{hi/5:.1f}x too small.\n"
|
||||
f" In absolute terms it is small -- but it is small IN THE RESOURCE THE "
|
||||
f"PROJECT IS\n SHORT OF, and it was being taken from the wrong side of "
|
||||
f"an open question.")
|
||||
@@ -63,12 +63,37 @@ DMA_CHAIN_CLK = 36
|
||||
# Sect 4.5.2.1 front-end overhead 5 clocks best case, 8 worst; 4.5.2.2
|
||||
# back-end 2 clocks best. Once per period of bus ownership, not per span.
|
||||
DMA_FRONT_CLK, DMA_BACK_CLK = 5, 2
|
||||
DMA_FRONT_CLK_WORST = 8
|
||||
# Fig 4-25 note 2 again, split out because the ADPCM channel needs the halves
|
||||
# apart: a DMAC READ is 4 clocks and a WRITE is 5, on either bus width. A
|
||||
# dual-address BYTE transfer is therefore one 4 and one 5.
|
||||
DMA_READ_CLK, DMA_WRITE_CLK = 4, 5
|
||||
DMA_DUAL_BYTE_CLK = DMA_READ_CLK + DMA_WRITE_CLK
|
||||
# Fig 4-25 sheet 3, SINGLE ADDRESS: W/B READ 4 clocks, W/B WRITE 5 clocks.
|
||||
# A device->memory disk transfer is one memory WRITE = 5 clocks if the DMAC
|
||||
# holds the bus, or 5 + front + back = 12 if it arbitrates per word.
|
||||
# FINDINGS 5's long-standing 8 clk/word ESTIMATE sits inside that range.
|
||||
DMA_DISK_CLK_WORD_HELD, DMA_DISK_CLK_WORD_ARB = 5, 12
|
||||
|
||||
# --- the ADPCM stream, as the IPL ROM actually programs it -----------------
|
||||
# READ OUT OF THE ROM, not recalled: tools/analysis/21_iplrom_dmac.py decodes
|
||||
# the HD63450 registers Sharp's own IPL 1.0 writes, and gates on the bytes still
|
||||
# being there. Channel 3, DCR = $80, OCR = $32 for playback:
|
||||
#
|
||||
# DTYP = 00 explicitly addressed -> DUAL ADDRESS (memory read, device write)
|
||||
# DPS = 0 8-bit port -> one byte per operand
|
||||
# XRM = 10 cycle steal WITHOUT hold, and REQG = 10 external request
|
||||
# -> the DMAC arbitrates ONCE PER BYTE. No burst to amortise over.
|
||||
#
|
||||
# So an audio byte costs the dual-address transfer PLUS a full arbitration,
|
||||
# every time -- unlike a disk record, which can at least be argued to hold the
|
||||
# bus for a run of bytes. This is the number the audio side of the I/O debit
|
||||
# should be denominated in; DISK_CLK_BYTE is not it.
|
||||
ADPCM_SAMPLE_HZ = 15625.0 # MSM6258V, 8 MHz clock / 512 (the 15.6 kHz mode)
|
||||
ADPCM_BYTES_PER_S = ADPCM_SAMPLE_HZ / 2 # 4-bit samples, two to a byte
|
||||
ADPCM_CLK_BYTE_BEST = DMA_DUAL_BYTE_CLK + DMA_FRONT_CLK + DMA_BACK_CLK # 16
|
||||
ADPCM_CLK_BYTE_WORST = DMA_DUAL_BYTE_CLK + DMA_FRONT_CLK_WORST + DMA_BACK_CLK # 19
|
||||
|
||||
# The 68000 cannot execute while another master owns the bus: no cache, and a
|
||||
# two-word prefetch queue that empties immediately. So DMA time is ADDITIVE to
|
||||
# CPU time, not overlapped -- which is what FINDINGS 35's flat debit assumed
|
||||
|
||||
+142
-8
@@ -10,6 +10,25 @@ cd "$(dirname "$0")/../.."
|
||||
python3 tools/encoder/extract.py 00020 tmp/fr_00020 12 crop
|
||||
mkdir -p tmp/snap_verify tmp/snap256
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# THE ONE PLACE THE RETIRED PIPE FIGURE STILL LIVES. Session 18 removed it as
|
||||
# a default from every analysis tool and from tools/bench/stream.lua, because it
|
||||
# was never a bus measurement -- a user-supplied "4 Mbps" with no provenance,
|
||||
# 10% of SCSI-1's asynchronous rating (FINDINGS 42.1) -- and a default let table
|
||||
# after table be scored against it without anyone restating what it was.
|
||||
#
|
||||
# It survives HERE and only here because the gate container was ENCODED with it,
|
||||
# and every per-block and span constant in FINDINGS 41/43/45/49 is fitted to that
|
||||
# container. Changing this number is not an edit, it is a re-encode plus a
|
||||
# re-measurement of all of them.
|
||||
#
|
||||
# It is a CONTAINER RECIPE, not a claim about any medium. Do not read a delivery
|
||||
# rate out of it, do not copy it into a tool, and do not add a default anywhere
|
||||
# that would resurrect it. When the pipe is finally measured, this becomes an
|
||||
# ordinary encoder setting and the comment goes.
|
||||
GATE_SPAN_KBPS=488
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
run() { # run <script> <snapdir>
|
||||
rm -f "tmp/$2/x68000"/*.png
|
||||
( cd tmp && SDL_VIDEODRIVER=dummy timeout -k 5 120 mame x68000 -bios ipl10 \
|
||||
@@ -46,7 +65,10 @@ echo "--- session 12: the DLX3 span container round-trips (FINDINGS 41) ---"
|
||||
# header and is painted by the span section instead -- so this encodes, WRITES
|
||||
# the container, reads it back with the reference decoder and compares. It also
|
||||
# asserts that it emitted enough spans to have tested anything.
|
||||
python3 tools/analysis/16_span_roundtrip.py > tmp/span_roundtrip.log 2>&1 \
|
||||
# --kbps is required now (session 18): the tool has no default rate, so the gate
|
||||
# has to say which one it is testing at. Same recipe constant as the container.
|
||||
python3 tools/analysis/16_span_roundtrip.py --kbps $GATE_SPAN_KBPS \
|
||||
> tmp/span_roundtrip.log 2>&1 \
|
||||
|| { cat tmp/span_roundtrip.log; exit 1; }
|
||||
tail -4 tmp/span_roundtrip.log
|
||||
|
||||
@@ -69,16 +91,31 @@ echo "--- session 7: 68000 decoder is pixel-exact (FINDINGS 28) ---"
|
||||
# right if all 120 were.
|
||||
# The gate container is the HEAVIEST stream the encoder emits: the scsi mode
|
||||
# decision (the only profile left after session 9 dropped sasi on capacity,
|
||||
# FINDINGS 32) with the span pass drawing on the full 488 KB/s pipe, so every
|
||||
# frame carries a span table and all four block modes are still exercised.
|
||||
# FINDINGS 32) with the span pass drawing on a byte ceiling wide enough that
|
||||
# every frame carries a span table and all four block modes are still exercised.
|
||||
# That ceiling is GATE_SPAN_KBPS above -- a recipe, not a delivery rate.
|
||||
# Spans are the newest and least-proven path in decode.s; gating on a container
|
||||
# where they are rare would be gating on the old decoder. FINDINGS 41.
|
||||
DLX=tmp/rc_fr_singe_scsi_span.dlx
|
||||
[ -f "$DLX" ] || python3 tools/encoder/encode.py tmp/fr_singe "$DLX" --profile scsi \
|
||||
--kbps 280 --span-kbps 488 --spans all
|
||||
python3 tools/bench/prep_dlx.py "$DLX" > tmp/prep_dlx.log
|
||||
# The rig loads the whole stream into a 2 MB machine, so a scsi window does not
|
||||
# fit and prep_dlx truncates it. Verify against exactly the prefix it emitted.
|
||||
--kbps 280 --span-kbps $GATE_SPAN_KBPS --spans all
|
||||
# RIG_RAM is the EMULATED MACHINE's memory, and it is not a claim about the
|
||||
# target. The rig preloads the whole container into RAM at 0x30000; the shipping
|
||||
# player streams from disk into a ring buffer and never holds a window at once,
|
||||
# so preloading is unlike the player at ANY size. At the 2 MB of a stock machine
|
||||
# this gate covered 37 of 120 frames (FINDINGS 44.6.4) -- the span-heavy
|
||||
# container is 5,261,814 B of stream, ending at 0x534BF6. 6 MB covers all 120.
|
||||
#
|
||||
# Raising it is licensed by measurement, not by convenience: at 2M and 6M the
|
||||
# five synthetic anchors come out BIT-IDENTICAL (40,729 / 921,187 / 1,376,881 /
|
||||
# 1,229,883 / 506,533 cycles) despite sitting at different addresses in the two
|
||||
# layouts, so MAME's cycle model does not depend on ramsize over this range.
|
||||
# FINDINGS 45. What is still NOT tested, at either size, is the streaming path.
|
||||
RIG_RAM=${RIG_RAM:-6}
|
||||
python3 tools/bench/prep_dlx.py "$DLX" --ram $((RIG_RAM * 0x100000)) > tmp/prep_dlx.log
|
||||
# Verify against exactly the frame list prep_dlx emitted. It no longer truncates
|
||||
# at the default RIG_RAM, but the guard stays: lower RIG_RAM, or a heavier
|
||||
# container, brings truncation straight back and it must stay announced.
|
||||
NF=$(sed -n 's/.*nframes=\([0-9]*\),.*/\1/p' tmp/decode_meta.lua)
|
||||
grep -a "TRUNCATED" tmp/prep_dlx.log || true
|
||||
tools/vasm/vasmm68k_mot -Fbin -o tmp/decode.bin src/player/decode.s > /dev/null
|
||||
@@ -93,7 +130,7 @@ rm -f tmp/snap_decode/x68000/*.png
|
||||
# compared a partially drawn screen and reported 49,005 differing pixels, which
|
||||
# reads as a decoder bug and is not one.
|
||||
( cd tmp && DLX_VERIFY_ONLY=1 SDL_VIDEODRIVER=dummy stdbuf -oL timeout -k 5 300 mame x68000 \
|
||||
-bios ipl10 -ramsize 2M -video soft -window -sound none -nothrottle -plugins \
|
||||
-bios ipl10 -ramsize ${RIG_RAM}M -video soft -window -sound none -nothrottle -plugins \
|
||||
-autoboot_script ../tools/bench/decode.lua \
|
||||
-snapshot_directory ./snap_decode -snapview native -seconds_to_run 60 \
|
||||
> decode_check.log 2>&1 )
|
||||
@@ -130,4 +167,101 @@ else
|
||||
echo " SKIPPED: no px68k at $PX68K (set PX68K= to point at a checkout)"
|
||||
fi
|
||||
|
||||
echo "--- session 20: the DMAC config, read out of the IPL ROM (FINDINGS 52) ---"
|
||||
# The audio and disk per-byte debits are no longer a recollection about the
|
||||
# HD63450: they are bytes at named addresses in the ROM MAME boots this rig
|
||||
# with. This gate re-reads them. It is cheap, it needs no emulator, and if a
|
||||
# different ROM revision is ever pointed at it, it says so rather than decoding
|
||||
# some other code and reporting a number.
|
||||
# Skipped rather than failed when the ROM is not where MAME keeps it: that is a
|
||||
# path outside this repo.
|
||||
IPLROM=${IPLROM:-$HOME/mame/roms/iplrom.dat}
|
||||
if [ -f "$IPLROM" ]; then
|
||||
python3 tools/analysis/21_iplrom_dmac.py "$IPLROM" > tmp/iplrom_dmac.log 2>&1 \
|
||||
|| { cat tmp/iplrom_dmac.log; exit 1; }
|
||||
grep -ac "^ OK " tmp/iplrom_dmac.log | xargs printf " %s evidence sites hold; "
|
||||
sed -n 's/^ = \(.*clocks per audio byte\)/audio is \1/p' tmp/iplrom_dmac.log
|
||||
else
|
||||
echo " SKIPPED: no IPL ROM at $IPLROM (set IPLROM= to point at it)"
|
||||
fi
|
||||
|
||||
echo "--- session 18: the shared-body split is a no-op (FINDINGS 49.7.5) ---"
|
||||
# src/player/decode.s and src/player/stream.s assemble from ONE copy of the block
|
||||
# loop and the span chain (src/player/frame.i) so that the two front-ends cannot
|
||||
# drift apart. The drift would be silent -- both would still decode correctly,
|
||||
# and only the cost model would be wrong, because the 66.0 clocks/span, 9.143
|
||||
# clocks/coarse pixel and every per-block constant in FINDINGS 24/30/40/41 are
|
||||
# fitted to those exact bytes. So the split is asserted to be a no-op rather than
|
||||
# assumed to be one.
|
||||
DECODE_MD5=7a7a06f8c6d097ee0041bca4aefa3eb2 # decode.bin before the split, 1296 B
|
||||
GOT=$(md5sum tmp/decode.bin | cut -d" " -f1)
|
||||
[ "$GOT" = "$DECODE_MD5" ] || {
|
||||
echo "FAIL: decode.bin is $GOT, expected $DECODE_MD5 ($(stat -c%s tmp/decode.bin) B)."
|
||||
echo " The block loop or the span chain changed. That is allowed -- but"
|
||||
echo " every cycle constant in FINDINGS 24/30/40/41 is fitted to the old"
|
||||
echo " bytes, so re-measure them and move this hash, do not just move it."
|
||||
exit 1; }
|
||||
echo " decode.bin unchanged at $(stat -c%s tmp/decode.bin) B ($DECODE_MD5)"
|
||||
# Same argument for the loader maths, which prep_dlx.py and prep_stream.py now
|
||||
# share via tools/bench/dlxload.py: a second copy of the palette packing would
|
||||
# drift and the symptom would be wrong colours in one rig only.
|
||||
python3 tools/bench/prep_dlx.py "$DLX" --ram $((RIG_RAM * 0x100000)) --out tmp/_pdchk > /dev/null
|
||||
cmp -s tmp/_pdchk_data.bin tmp/decode_data.bin || {
|
||||
echo "FAIL: prep_dlx.py is not reproducible"; exit 1; }
|
||||
echo " prep_dlx.py blob reproducible ($(stat -c%s tmp/decode_data.bin) B)"
|
||||
rm -f tmp/_pdchk_data.bin tmp/_pdchk_meta.lua
|
||||
|
||||
echo "--- session 18: 120 frames through a bounded RING (FINDINGS 49) ---"
|
||||
# The gate above preloads the whole container into RAM and proves the DECODER.
|
||||
# This proves the DELIVERY path: the same 120 frames decoded out of a 256 KB
|
||||
# ring on a STOCK 2 MB machine, with the container in a host file. The block
|
||||
# loop reads with a monotonically increasing a0 and no bounds check, so a record
|
||||
# placed wrongly by the wrap policy corrupts pixels rather than faulting -- which
|
||||
# is why this is gated on the same pixel-exact comparison and not on a checksum.
|
||||
tools/vasm/vasmm68k_mot -Fbin -o tmp/stream.bin src/player/stream.s > /dev/null
|
||||
python3 tools/bench/prep_stream.py "$DLX" > tmp/prep_stream.log
|
||||
mkdir -p tmp/snap_stream
|
||||
rm -f tmp/snap_stream/x68000/*.png
|
||||
( cd tmp && DLX_STREAM_KBPS=0 SDL_VIDEODRIVER=dummy stdbuf -oL timeout -k 5 600 \
|
||||
mame x68000 -bios ipl10 -ramsize 2M -video soft -window -sound none \
|
||||
-nothrottle -plugins -autoboot_script ../tools/bench/stream.lua \
|
||||
-snapshot_directory ./snap_stream -snapview native -seconds_to_run 90 \
|
||||
> stream_check.log 2>&1 )
|
||||
# Same truncation trap as the decode stage: without this, a run that exited
|
||||
# mid-decode is compared against a half-drawn screen and reads as a wrap bug.
|
||||
grep -q "snapshot taken" tmp/stream_check.log || {
|
||||
echo "FAIL: the ring-buffer pass did not complete -- no snapshot marker."
|
||||
tail -5 tmp/stream_check.log; exit 1; }
|
||||
grep -a "ring: \|DEADLINE" tmp/stream_check.log | sed "s/\[STR\] / /"
|
||||
python3 tools/bench/verify_decode.py "$DLX" --snap tmp/snap_stream
|
||||
|
||||
echo "--- session 19: the PACED ring, and what a branch point costs (FINDINGS 51) ---"
|
||||
# The stage above runs the ring FREE-RUNNING, which is right for what it gates:
|
||||
# an unlimited pipe removes delivery as a variable and leaves the wrap policy
|
||||
# alone under test. It cannot see buffering, because a decoder that never waits
|
||||
# never lets the ring back up -- 49.7.2, and it is why 48 KB passed while
|
||||
# holding one record. This runs the same 120 frames with the decoder held to
|
||||
# 12 fps, which is the only configuration in which FR_HEAD-FR_TAIL means what
|
||||
# it is read to mean.
|
||||
#
|
||||
# Gated on: pixel-exact, zero UNDERRUNS, and a ceiling that has not moved. The
|
||||
# ceiling is a property of THIS container in a 256 KB ring; it is asserted
|
||||
# rather than printed because a change in it is a change in how much a branch
|
||||
# point can afford, and that should not slip through as a line in a log.
|
||||
bash tools/bench/pace_run.sh 256 0 > tmp/pace_check.log 2>&1 || {
|
||||
echo "FAIL: the paced ring pass did not complete."; tail -8 tmp/pace_check.log
|
||||
exit 1; }
|
||||
grep -aE "SEEK SLACK|UNDERRUNS" tmp/pace_check.log
|
||||
grep -q "UNDERRUNS: 0/120" tmp/pace_check.log || {
|
||||
echo "FAIL: the paced decoder underran -- a frame's slot arrived before its"
|
||||
echo " record did. Free-running this is earliness (49.6); paced it is not."
|
||||
exit 1; }
|
||||
grep -q "ceiling 8 frames" tmp/pace_check.log || {
|
||||
echo "FAIL: the 256 KB seek-slack ceiling is no longer 8 frames (FINDINGS 51)."
|
||||
echo " Re-run tools/bench/pace_sweep.sh and re-derive 51 before editing"
|
||||
echo " this number -- it is what a branch point can spend."
|
||||
exit 1; }
|
||||
grep -q "^OK" tmp/pace_check.log || { echo "FAIL: paced pass not pixel-exact";
|
||||
tail -4 tmp/pace_check.log; exit 1; }
|
||||
|
||||
echo "ALL GREEN"
|
||||
|
||||
@@ -0,0 +1,53 @@
|
||||
"""Load-time transforms every src/player/ front-end's loader has to do.
|
||||
|
||||
Split out of prep_dlx.py in session 18 so that prep_dlx.py (the preloaded-stream
|
||||
rig) and prep_stream.py (the ring-buffer streaming rig, FINDINGS 49) share ONE
|
||||
copy of them. Two copies would drift, and the drift would be silent: both rigs
|
||||
would still decode, and only the colours or the codebook scaling would be
|
||||
subtly wrong in one of them.
|
||||
|
||||
The split is a no-op by construction -- tools/bench/check.sh asserts prep_dlx.py
|
||||
still emits a byte-identical blob for the gate container.
|
||||
|
||||
Neither transform is part of the per-frame cost being measured. The 68000 would
|
||||
do both once at load time; charging them to the inner loop would flatter or damn
|
||||
it for no reason.
|
||||
"""
|
||||
import numpy as np
|
||||
|
||||
|
||||
def expand_codebooks(d):
|
||||
"""CB1/CB4 to one WORD per pixel, so the inner loop movems them straight out.
|
||||
|
||||
The high byte of every GVRAM word write is discarded by the hardware, so it
|
||||
is left zero and never has to be cleared. Word-per-pixel form is also what
|
||||
makes index scaling a shift rather than a multiply: lsl.w #5 and lsl.w #3.
|
||||
"""
|
||||
cb1 = np.zeros((d.k1, 16, 2), np.uint8); cb1[:, :, 1] = d.cb1.reshape(d.k1, 16)
|
||||
cb4 = np.zeros((d.k4, 4, 2), np.uint8); cb4[:, :, 1] = d.cb4.reshape(d.k4, 4)
|
||||
return cb1, cb4
|
||||
|
||||
|
||||
def pack_palette(d):
|
||||
"""24-bit palette -> GGGGGRRRRRBBBBBI, shared LSB chosen PER ENTRY.
|
||||
|
||||
Choosing I per entry by minimum squared error rather than fixing it is worth
|
||||
1.96 dB (FINDINGS 23.3). Identical maths to tools/bench/verify_frame256.py,
|
||||
which is the point: the verifier and the loader must agree or a colour bug
|
||||
reads as a decoder bug.
|
||||
|
||||
Returns (palette bytes 256x2 big-endian, index of the darkest entry). The
|
||||
encoder does not yet reserve a black entry (docs/STATUS.md, encoder gaps),
|
||||
so the letterbox gets the closest thing to black the palette has.
|
||||
"""
|
||||
pal = d.pal.astype(int)
|
||||
p6 = lambda v: ((v << 2) | (v >> 4)) & 0xFF
|
||||
f = pal >> 3
|
||||
render = lambda I: p6((f << 1) | I[:, None])
|
||||
I = (((render(np.ones(256, int)) - pal) ** 2).sum(1)
|
||||
< ((render(np.zeros(256, int)) - pal) ** 2).sum(1)).astype(int)
|
||||
words = (f[:, 1] << 11) | (f[:, 0] << 6) | (f[:, 2] << 1) | I
|
||||
palb = np.zeros((256, 2), np.uint8)
|
||||
palb[:, 0], palb[:, 1] = words >> 8, words & 0xFF
|
||||
dark = int(((render(I).astype(int)) ** 2).sum(1).argmin())
|
||||
return palb, dark, render(I)
|
||||
@@ -0,0 +1,9 @@
|
||||
PX68K ?= $(HOME)/src/px68k
|
||||
CFLAGS = -O2 -fno-strict-aliasing -Wall -Wno-unused-result \
|
||||
-I$(PX68K)/m68000 -I$(PX68K)/x11 -I$(PX68K)/win32api -I$(PX68K)/x68k
|
||||
|
||||
gvpack: harness.c $(PX68K)/x68k/gvram.c
|
||||
$(CC) $(CFLAGS) -o $@ harness.c $(PX68K)/x68k/gvram.c
|
||||
|
||||
clean:
|
||||
rm -f gvpack
|
||||
Executable
BIN
Binary file not shown.
@@ -0,0 +1,152 @@
|
||||
/* Headless harness for px68k's GVRAM write and display model.
|
||||
*
|
||||
* Tests FINDINGS 46.6 -- the packed 1.0 byte/pixel layout -- on a SECOND
|
||||
* emulator, the way tools/bench/c68k does for the CPU core. It links px68k's
|
||||
* real x68k/gvram.c: the address decode, the CRTC R20 bit-11 buffer-mode write
|
||||
* path, the page-byte selection, the scroll wrap and the index-0 transparency
|
||||
* test are all px68k's own code, not a reimplementation.
|
||||
*
|
||||
* What IS glue here, and is declared as such: the ~12 lines of page-ordering
|
||||
* from x11/windraw.c's 256-colour case (which page is drawn opaque and which
|
||||
* transparent, as a function of the video controller's priority register).
|
||||
* windraw.c is SDL-bound and cannot be linked headless, so that dispatch is
|
||||
* mirrored. It is quoted verbatim in pick_order() so the mirroring is
|
||||
* auditable.
|
||||
*
|
||||
* GrphPal is set to the IDENTITY, so what lands in Grp_LineBuf is the 8-bit
|
||||
* palette INDEX rather than a host pixel. That keeps the harness out of
|
||||
* px68k's host-format colour conversion, and it is faithful: px68k's
|
||||
* transparency test is on the index (`if (v != 0x00)`), before the lookup.
|
||||
*/
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
#include "common.h"
|
||||
#include "gvram.h"
|
||||
|
||||
/* --- the globals gvram.c expects from the rest of px68k -------------------- */
|
||||
BYTE CRTC_Regs[48];
|
||||
WORD CRTC_FastClrMask;
|
||||
DWORD GrphScrollX[4], GrphScrollY[4];
|
||||
WORD GrphPal[256];
|
||||
BYTE TextDirtyLine[1024];
|
||||
DWORD TextDotX, TextDotY;
|
||||
DWORD VLINE;
|
||||
BYTE Pal_Regs[1024];
|
||||
WORD Pal16[65536];
|
||||
WORD Ibit, Pal_HalfMask, Pal_Ix2;
|
||||
|
||||
extern BYTE GVRAM[0x80000];
|
||||
extern WORD Grp_LineBuf[1024];
|
||||
|
||||
#define W 256
|
||||
#define H 256
|
||||
|
||||
/* 68000 word write: two byte writes, high byte first, as the bus does. */
|
||||
static void wr16(DWORD adr, WORD v)
|
||||
{
|
||||
GVRAM_Write(adr, (BYTE)(v >> 8));
|
||||
GVRAM_Write(adr + 1, (BYTE)(v & 0xff));
|
||||
}
|
||||
|
||||
static void set_r20(WORD r20) /* CRTC R20 = byte pair 0x28/0x29 */
|
||||
{
|
||||
CRTC_Regs[0x28] = (BYTE)(r20 >> 8);
|
||||
CRTC_Regs[0x29] = (BYTE)(r20 & 0xff);
|
||||
}
|
||||
|
||||
/* Mirrors x11/windraw.c, 256-colour case:
|
||||
*
|
||||
* if ( (VCReg1[1]&3) <= ((VCReg1[1]>>4)&3) ) {
|
||||
* ... Grp_DrawLine8(1, 1); opaq = 0;
|
||||
* ... Grp_DrawLine8(0, opaq);
|
||||
* } else {
|
||||
* ... Grp_DrawLine8(0, 1); opaq = 0;
|
||||
* ... Grp_DrawLine8(1, opaq);
|
||||
* }
|
||||
*
|
||||
* i.e. the first page drawn is OPAQUE (the bottom) and the second is drawn
|
||||
* with opaq=0 (the transparent top).
|
||||
*/
|
||||
static void draw_line(BYTE vcreg1_lo)
|
||||
{
|
||||
int bottom = ((vcreg1_lo & 3) <= ((vcreg1_lo >> 4) & 3)) ? 1 : 0;
|
||||
Grp_DrawLine8(bottom, 1);
|
||||
Grp_DrawLine8(bottom ^ 1, 0);
|
||||
}
|
||||
|
||||
int main(int argc, char **argv)
|
||||
{
|
||||
const char *blob = argc > 1 ? argv[1] : "tmp/frame256p.bin";
|
||||
const char *out = argc > 2 ? argv[2] : "tmp/gvpack_px68k.raw";
|
||||
int packed = !(argc > 3 && !strcmp(argv[3], "--unpacked"));
|
||||
BYTE vc1 = (BYTE)(argc > 4 ? strtol(argv[4], NULL, 0) : 0x02);
|
||||
/* --nobuffer: run the packed layout WITHOUT CRTC R20 bit 11, to show the
|
||||
* bit is load-bearing here and not decoration. */
|
||||
int buffer = !(argc > 5 && !strcmp(argv[5], "--nobuffer"));
|
||||
int scroll = !(argc > 5 && !strcmp(argv[5], "--noscroll"));
|
||||
/* --keepbuffer: leave R20 bit 11 SET while drawing. MAME blanks the
|
||||
* graphics layer in that state; does px68k? */
|
||||
int keepbuf = (argc > 5 && !strcmp(argv[5], "--keepbuffer"));
|
||||
|
||||
FILE *f = fopen(blob, "rb");
|
||||
if (!f) { perror(blob); return 2; }
|
||||
static BYTE d[8 + 768 + 256 * 256];
|
||||
size_t n = fread(d, 1, sizeof d, f);
|
||||
fclose(f);
|
||||
int iw = (d[4] << 8) | d[5], ih = (d[6] << 8) | d[7];
|
||||
if (n < (size_t)(8 + 768 + iw * ih)) { fprintf(stderr, "short blob\n"); return 2; }
|
||||
const BYTE *pix = d + 8 + 768;
|
||||
int yoff = (H - ih) / 2;
|
||||
const BYTE BLACK = 255;
|
||||
|
||||
#define PIX(y, x) ((y) < yoff || (y) >= yoff + ih ? BLACK : pix[((y) - yoff) * iw + (x)])
|
||||
|
||||
memset(GVRAM, 0, sizeof GVRAM);
|
||||
for (int i = 0; i < 256; i++) GrphPal[i] = (WORD)i; /* identity */
|
||||
TextDotX = W; TextDotY = H;
|
||||
|
||||
/* 256x256, 256 colours -- the same R20 tools/bench/crtc_mode.lua applies */
|
||||
const WORD R20_DISPLAY = 0x0110;
|
||||
set_r20(R20_DISPLAY);
|
||||
|
||||
/* page 0 -> scroll sets 0,1; page 1 -> scroll sets 2,3 */
|
||||
GrphScrollX[0] = GrphScrollX[1] = 0;
|
||||
GrphScrollY[0] = GrphScrollY[1] = 0;
|
||||
GrphScrollX[2] = GrphScrollX[3] = (packed && scroll) ? 384 : 0;
|
||||
GrphScrollY[2] = GrphScrollY[3] = 0;
|
||||
|
||||
if (packed) {
|
||||
if (buffer) set_r20(R20_DISPLAY | 0x0800); /* buffer mode: unmasked */
|
||||
for (int y = 0; y < H; y++) {
|
||||
DWORD base = 0xC00000 + y * 1024;
|
||||
for (int i = 128; i < 512; i++) wr16(base + i * 2, 0);
|
||||
for (int i = 0; i < 128; i++)
|
||||
wr16(base + i * 2, (WORD)((PIX(y, i + 128) << 8) | PIX(y, i)));
|
||||
}
|
||||
if (!keepbuf) set_r20(R20_DISPLAY); /* back to display */
|
||||
} else {
|
||||
/* the ordinary 2.0 B/pixel path, for a control */
|
||||
for (int y = 0; y < H; y++) {
|
||||
DWORD base = 0xC00000 + y * 1024;
|
||||
for (int x = 0; x < W; x++) wr16(base + x * 2, PIX(y, x));
|
||||
}
|
||||
}
|
||||
|
||||
FILE *o = fopen(out, "wb");
|
||||
if (!o) { perror(out); return 2; }
|
||||
for (int y = 0; y < H; y++) {
|
||||
VLINE = (DWORD)y;
|
||||
memset(Grp_LineBuf, 0, sizeof Grp_LineBuf);
|
||||
draw_line(vc1);
|
||||
static BYTE row[W];
|
||||
for (int x = 0; x < W; x++) row[x] = (BYTE)(Grp_LineBuf[x] & 0xff);
|
||||
fwrite(row, 1, W, o);
|
||||
}
|
||||
fclose(o);
|
||||
fprintf(stderr, "[GVPACK] px68k model: %s, vcreg1=%02X, bottom page=%d -> %s\n",
|
||||
packed ? (buffer ? "PACKED 1.0 B/px" : "PACKED but bit11 OFF")
|
||||
: "unpacked 2.0 B/px", vc1,
|
||||
((vc1 & 3) <= ((vc1 >> 4) & 3)) ? 1 : 0, out);
|
||||
return 0;
|
||||
}
|
||||
@@ -0,0 +1,53 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Check px68k's render of the packed layout against the same reference MAME is
|
||||
judged on (tools/bench/verify_frame256.py, criterion 3 and 4).
|
||||
|
||||
python3 tools/bench/gvpack/verify_gvpack.py <raw> [blob]
|
||||
|
||||
<raw> is 256x256 palette INDICES straight out of px68k's Grp_DrawLine8.
|
||||
"""
|
||||
import struct, sys
|
||||
import numpy as np
|
||||
|
||||
raw = sys.argv[1] if len(sys.argv) > 1 else "tmp/gvpack_px68k.raw"
|
||||
blob = sys.argv[2] if len(sys.argv) > 2 else "tmp/frame256p.bin"
|
||||
|
||||
g = np.frombuffer(open(raw, "rb").read(), np.uint8).reshape(256, 256)
|
||||
d = open(blob, "rb").read()
|
||||
W, H = struct.unpack(">HH", d[4:8])
|
||||
pal = np.frombuffer(d[8:8+768], np.uint8).reshape(256, 3).astype(int)
|
||||
idx = np.frombuffer(d[8+768:8+768+W*H], np.uint8).reshape(H, W)
|
||||
|
||||
yoff = (256 - H) // 2
|
||||
act = g[yoff:yoff+H]
|
||||
|
||||
fail = []
|
||||
if not np.array_equal(act, idx):
|
||||
bad = act != idx
|
||||
fail.append(f"active area index-exact: {bad.sum()} px differ "
|
||||
f"(left half {bad[:, :128].sum()}, right half {bad[:, 128:].sum()})")
|
||||
|
||||
bars = np.concatenate([g[:yoff], g[yoff+H:]])
|
||||
if bars.size and (bars != 255).any():
|
||||
fail.append(f"letterbox not the reserved black index 255: "
|
||||
f"{(bars != 255).sum()} px")
|
||||
|
||||
if (act == 0).any():
|
||||
fail.append(f"index 0 appeared in the picture: {(act == 0).sum()} px "
|
||||
f"-- it is the transparency key and must stay unused")
|
||||
|
||||
for x in fail:
|
||||
print("FAIL " + x)
|
||||
if fail:
|
||||
sys.exit(1)
|
||||
|
||||
p6 = lambda v: ((v << 2) | (v >> 4)) & 0xFF
|
||||
f = pal >> 3
|
||||
render = lambda I: p6((f << 1) | I[:, None])
|
||||
I = (((render(np.ones(256, int)) - pal) ** 2).sum(1)
|
||||
< ((render(np.zeros(256, int)) - pal) ** 2).sum(1)).astype(int)
|
||||
mse = ((render(I)[act].astype(int) - pal[idx]) ** 2).mean()
|
||||
print(f"OK {raw}: px68k renders the packed layout index-exact over {W}x{H}, "
|
||||
f"letterbox on the reserved black")
|
||||
print(f" palette ceiling vs 24-bit palettised source: "
|
||||
f"{10*np.log10(255**2/mse):.2f} dB")
|
||||
Executable
+47
@@ -0,0 +1,47 @@
|
||||
#!/bin/bash
|
||||
# One paced ring-buffer run (STATUS item 4, FINDINGS 49.7.2).
|
||||
#
|
||||
# tools/bench/check.sh runs the ring pass FREE-RUNNING, which is right for what
|
||||
# it gates -- wrap correctness at a fixed ring size, delivery removed as a
|
||||
# variable by an unlimited pipe. It cannot answer the buffering question,
|
||||
# because a free-running decoder never lets the ring back up.
|
||||
#
|
||||
# This runs the same rig with the decoder held to the container's frame rate,
|
||||
# so the ring fills and FR_HEAD-FR_TAIL means "frames the decoder could still
|
||||
# draw with the pipe dead". Every run is verified PIXEL-EXACT: a paced decode
|
||||
# that drops a pixel is not a slack measurement, it is a bug.
|
||||
#
|
||||
# tools/bench/pace_run.sh <ring_kb> <kbps> [cut_at_tick] [cut_frames]
|
||||
#
|
||||
# kbps 0 = unlimited pipe. There is no default rate anywhere in this tree
|
||||
# (FINDINGS 50) and there is none here either.
|
||||
set -e
|
||||
cd "$(dirname "$0")/../.."
|
||||
RING=${1:?ring KB}; KBPS=${2:?pipe KB/s, or 0 for unlimited}
|
||||
CUT_AT=$3; CUT_FR=${4:-1}
|
||||
DLX=${DLX:-tmp/rc_fr_singe_scsi_span.dlx}
|
||||
TAG="r${RING}_k${KBPS}${CUT_AT:+_cut${CUT_AT}x${CUT_FR}}"
|
||||
|
||||
tools/vasm/vasmm68k_mot -Fbin -o tmp/stream.bin src/player/stream.s > /dev/null
|
||||
[ -f tmp/stream_disk.bin ] || python3 tools/bench/prep_stream.py "$DLX" > tmp/prep_stream.log
|
||||
mkdir -p "tmp/snap_pace_$TAG"; rm -f "tmp/snap_pace_$TAG/x68000"/*.png
|
||||
# `env` rather than an assignment prefix: an empty ${CUT_AT:+...} in the middle
|
||||
# of a prefix is not an assignment token, so bash takes the next word as the
|
||||
# command and the run dies with "SDL_VIDEODRIVER=dummy: command not found".
|
||||
CUTENV=(); [ -n "$CUT_AT" ] && CUTENV=(DLX_CUT_AT="$CUT_AT" DLX_CUT_FR="$CUT_FR")
|
||||
( cd tmp && env DLX_PACE=1 DLX_RING_KB=$RING DLX_STREAM_KBPS=$KBPS \
|
||||
"${CUTENV[@]}" DLX_SLACK_CSV="slack_$TAG.csv" \
|
||||
SDL_VIDEODRIVER=dummy stdbuf -oL timeout -k 5 900 \
|
||||
mame x68000 -bios ipl10 -ramsize 2M -video soft -window -sound none \
|
||||
-nothrottle -plugins -autoboot_script ../tools/bench/stream.lua \
|
||||
-snapshot_directory "./snap_pace_$TAG" -snapview native -seconds_to_run 90 \
|
||||
> "pace_$TAG.log" 2>&1 )
|
||||
# The completion marker is not optional: a run killed mid-decode compares a
|
||||
# half-drawn screen and reads as a wrap bug rather than as a truncated run.
|
||||
grep -q "snapshot taken" "tmp/pace_$TAG.log" || {
|
||||
echo "FAIL($TAG): no snapshot marker -- the pass did not complete."
|
||||
tail -6 "tmp/pace_$TAG.log"; exit 1; }
|
||||
echo "=== $TAG"
|
||||
grep -aE "decoder (PACED|FREE)|ring: |UNDERRUNS|SEEK SLACK|RING-BOUND|RATE-BOUND|BUILD TIME|PIPE CUT|DEADLINE|REQUIRED" \
|
||||
"tmp/pace_$TAG.log" | sed "s/\[STR\] / /"
|
||||
python3 tools/bench/verify_decode.py "$DLX" --snap "tmp/snap_pace_$TAG" | tail -2
|
||||
Executable
+36
@@ -0,0 +1,36 @@
|
||||
#!/bin/bash
|
||||
# Ring x pipe grid for the PACED rig (STATUS item 4, FINDINGS 51).
|
||||
#
|
||||
# tools/bench/pace_sweep.sh "<ring KB list>" "<KB/s list>"
|
||||
#
|
||||
# Every cell is a full 120-frame decode on the emulated 68000, pixel-verified.
|
||||
# There is no default rate list: FINDINGS 50 removed the delivery constant from
|
||||
# this tree and a sweep that invented one back would be the same mistake with
|
||||
# more rows. `0` means an unlimited pipe, which measures the RING's ceiling with
|
||||
# delivery removed as a variable -- an upper bound, not a prediction.
|
||||
set -e
|
||||
cd "$(dirname "$0")/../.."
|
||||
RINGS=${1:?ring KB list, quoted}
|
||||
RATES=${2:?pipe KB/s list, quoted, 0 = unlimited}
|
||||
printf "%6s %9s %9s %9s %8s %9s %s\n" ring kbps ceiling build_s mean underruns bound
|
||||
for R in $RINGS; do for K in $RATES; do
|
||||
L=tmp/pace_r${R}_k${K}.log
|
||||
bash tools/bench/pace_run.sh "$R" "$K" > /dev/null 2>&1 || { \
|
||||
printf "%6s %9s FAILED (see %s)\n" "$R" "$K" "$L"; continue; }
|
||||
python3 - "$L" "$R" "$K" <<'PY'
|
||||
import re, sys
|
||||
log, ring, kbps = sys.argv[1], sys.argv[2], sys.argv[3]
|
||||
t = open(log, errors="replace").read()
|
||||
def g(p, d="?"):
|
||||
m = re.search(p, t)
|
||||
return m.group(1) if m else d
|
||||
ceil_ = g(r"SEEK SLACK: ceiling (\d+) frames")
|
||||
build = g(r"BUILD TIME: (\d+) ticks")
|
||||
mean = g(r"mean ([\d.]+) over the window")
|
||||
under = g(r"UNDERRUNS: (\d+)/")
|
||||
bound = "ring" if "RING-BOUND" in t else ("rate" if "RATE-BOUND" in t else "?")
|
||||
fps = 12.0
|
||||
print("%6s %9s %9s %9.2f %8s %9s %s" % (
|
||||
ring, kbps, ceil_, (int(build)/fps if build.isdigit() else -1), mean, under, bound))
|
||||
PY
|
||||
done; done
|
||||
+8
-17
@@ -28,6 +28,8 @@ import sys, os, argparse
|
||||
sys.path.insert(0, "tools/encoder")
|
||||
import numpy as np
|
||||
from dlx import DLX
|
||||
sys.path.insert(0, "tools/bench")
|
||||
import dlxload as DL
|
||||
import spans as SP
|
||||
|
||||
# The harness loads the WHOLE container into emulated RAM at STREAM=0x30000 and
|
||||
@@ -64,22 +66,11 @@ if not d.has_spans:
|
||||
f"DLX3 span section. Re-encode (tools/encoder/encode.py emits DLX3 "
|
||||
f"by default) or pass --spans off and use an older decoder.")
|
||||
|
||||
# --- codebooks, expanded to one WORD per pixel (high byte is discarded by
|
||||
# gvram_w, so it is left zero and never has to be cleared)
|
||||
cb1 = np.zeros((d.k1, 16, 2), np.uint8); cb1[:, :, 1] = d.cb1.reshape(d.k1, 16)
|
||||
cb4 = np.zeros((d.k4, 4, 2), np.uint8); cb4[:, :, 1] = d.cb4.reshape(d.k4, 4)
|
||||
|
||||
# --- palette words, I chosen per entry (identical maths to verify_frame256.py)
|
||||
pal = d.pal.astype(int)
|
||||
p6 = lambda v: ((v << 2) | (v >> 4)) & 0xFF
|
||||
f = pal >> 3
|
||||
render = lambda I: p6((f << 1) | I[:, None])
|
||||
I = (((render(np.ones(256, int)) - pal) ** 2).sum(1)
|
||||
< ((render(np.zeros(256, int)) - pal) ** 2).sum(1)).astype(int)
|
||||
words = (f[:, 1] << 11) | (f[:, 0] << 6) | (f[:, 2] << 1) | I
|
||||
palb = np.zeros((256, 2), np.uint8)
|
||||
palb[:, 0], palb[:, 1] = words >> 8, words & 0xFF
|
||||
dark = int(((render(I).astype(int)) ** 2).sum(1).argmin())
|
||||
# --- codebooks and palette. Both transforms live in tools/bench/dlxload.py so
|
||||
# that prep_stream.py's ring-buffer rig shares one copy of them rather than
|
||||
# keeping a second that could drift silently (FINDINGS 49).
|
||||
cb1, cb4 = DL.expand_codebooks(d)
|
||||
palb, dark, rendered = DL.pack_palette(d)
|
||||
|
||||
def build_synth(d):
|
||||
"""The synthetic timing frames, as record bodies.
|
||||
@@ -241,7 +232,7 @@ print(f" v7 spans/frame: median {np.median(nsp):.0f} max {nsp.max()} "
|
||||
f"({int((nsp>0).sum())}/{NFRAMES} frames); pixels painted by one: "
|
||||
f"median {100*np.median(spx)/(d.W*d.H):.1f}% "
|
||||
f"max {100*spx.max()/(d.W*d.H):.1f}% of the picture")
|
||||
print(f" darkest palette entry: index {dark} -> {tuple(render(I)[dark])}")
|
||||
print(f" darkest palette entry: index {dark} -> {tuple(rendered[dark])}")
|
||||
# A DLX2 container already carries this padding (FINDINGS 28.3 closed, session
|
||||
# 9), so the realignment above re-derives bytes that were already there and the
|
||||
# loader is doing no work. On a DLX1 container it is load-bearing: 94 of 120
|
||||
|
||||
@@ -15,16 +15,27 @@ argv = [a for a in sys.argv[1:] if not a.startswith("--")]
|
||||
# to 0 displays palette entry 0, and a free mediancut palette puts a real image
|
||||
# colour there. Costs one of 256 entries; measured quality cost is negligible.
|
||||
RESERVE = "--reserve-black" in sys.argv
|
||||
# --pack-transparent: the layout FINDINGS 46.6 needs. The packed scheme puts
|
||||
# the TOP graphics page's index 0 to work as a transparency key, so index 0 must
|
||||
# never appear in the picture -- and black therefore cannot live there. So:
|
||||
# quantise to 254, place them at 1..254, put black at 255, leave 0 UNUSED.
|
||||
# Costs two of 256 entries against --reserve-black's one.
|
||||
PACKT = "--pack-transparent" in sys.argv
|
||||
src, out = argv[0], argv[1]
|
||||
f = sorted(glob.glob(f"{src}/*.png"))[int(argv[2]) if len(argv) > 2 else 0]
|
||||
im = Image.open(f).convert("RGB")
|
||||
W, H = im.size
|
||||
|
||||
n = 255 if RESERVE else 256
|
||||
n = 254 if PACKT else (255 if RESERVE else 256)
|
||||
q = im.quantize(colors=n, method=Image.MEDIANCUT, dither=Image.NONE)
|
||||
pal = np.array(q.getpalette()[:n*3], dtype=np.uint8).reshape(n, 3)
|
||||
idx = np.asarray(q, dtype=np.uint8)
|
||||
if RESERVE:
|
||||
if PACKT:
|
||||
# 0 unused (transparency key), 1..254 picture, 255 black
|
||||
pal = np.vstack([np.zeros((1, 3), np.uint8), pal, np.zeros((1, 3), np.uint8)])
|
||||
idx = idx + 1
|
||||
assert idx.min() >= 1 and idx.max() <= 254, "index 0/255 must stay free"
|
||||
elif RESERVE:
|
||||
pal = np.vstack([np.zeros((1, 3), np.uint8), pal]) # index 0 = black
|
||||
idx = idx + 1
|
||||
|
||||
@@ -37,4 +48,4 @@ with open(out, "wb") as fh:
|
||||
# reference PNG of exactly what the X68000 should display
|
||||
Image.fromarray(pal[idx]).save(out.replace(".bin", "_ref.png"))
|
||||
print(f"src={f} {W}x{H} colors={len(np.unique(idx))}"
|
||||
f"{' (idx 0 reserved black)' if RESERVE else ''} -> {out}")
|
||||
f"{' (idx 0 unused/transparent, 255 black)' if PACKT else (' (idx 0 reserved black)' if RESERVE else '')} -> {out}")
|
||||
|
||||
@@ -0,0 +1,93 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Lay a DLX3 container out as a DISK for the ring-buffer rig (FINDINGS 49).
|
||||
|
||||
python3 tools/bench/prep_stream.py <in.dlx> [--out tmp/stream]
|
||||
|
||||
prep_dlx.py's output is one blob that tools/bench/decode.lua pushes into
|
||||
emulated RAM in its entirety. That is what makes its rig RAM-bound -- a `scsi`
|
||||
window is 5,261,814 B of stream and needs a 6 MB machine to hold it (FINDINGS
|
||||
45) -- and, much more importantly, it is nothing like the shipping player, which
|
||||
never holds a window at once.
|
||||
|
||||
This writes three files instead:
|
||||
|
||||
<out>_cb.bin codebooks + palette. ~10 KB, loaded into RAM once, exactly as
|
||||
before: these are LOAD-TIME costs and not per-frame ones.
|
||||
<out>_disk.bin the frame records, `[u32 len][body]` each padded up to 4, laid
|
||||
end to end. tools/bench/stream.lua reads this from the HOST
|
||||
filesystem and feeds it into a bounded ring, so the emulated
|
||||
machine's RAM stops bounding how much of a window can be
|
||||
tested. A stock 2 MB machine can now run all 120 frames.
|
||||
<out>_meta.lua geometry, and the record index.
|
||||
|
||||
THE RECORD INDEX IS NOT A CONVENIENCE. src/player/stream.s takes each frame's
|
||||
base address from a descriptor the producer wrote, rather than deriving it from
|
||||
where the last frame ended, because under the `aligned` wrap policy the next
|
||||
record may be at the ring's base instead of just after its predecessor. The
|
||||
producer therefore has to know record boundaries before it places them -- which
|
||||
is what an index is. A branching laserdisc game needs one anyway to seek to a
|
||||
branch point, so the policy that costs no clocks (tools/analysis/19_ring_stream.py)
|
||||
reuses a structure the player cannot avoid.
|
||||
|
||||
The 4-byte record padding is the same one decode.s needs and DLX3 already
|
||||
carries: `move.l (a0)+,d0` on an odd address is an ADDRESS ERROR on a 68000, not
|
||||
a slow read. FINDINGS 28.3.
|
||||
|
||||
NO SYNTHETIC TIMING FRAMES. prep_dlx.py appends ten of them to price the block
|
||||
modes separately; this rig measures delivery, not decode, and its per-frame cost
|
||||
anchors are prep_dlx.py's job. Mixing them in would put frames on the wire that
|
||||
no encoder emits and no rate controller sized.
|
||||
"""
|
||||
import sys, os, argparse
|
||||
sys.path.insert(0, "tools/encoder")
|
||||
sys.path.insert(0, "tools/bench")
|
||||
import numpy as np
|
||||
from dlx import DLX
|
||||
import dlxload as DL
|
||||
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("container")
|
||||
ap.add_argument("--out", default="tmp/stream")
|
||||
a = ap.parse_args()
|
||||
|
||||
d = DLX(a.container)
|
||||
if d.idx_bytes != 1:
|
||||
sys.exit("2-byte codebook indices: src/player/ assumes 1 (k<=256)")
|
||||
if not d.has_spans:
|
||||
sys.exit(f"{a.container} is DLX{d.version}: src/player/stream.s expects the "
|
||||
f"DLX3 span section (see prep_dlx.py for why a DLX2 container "
|
||||
f"decodes as garbage rather than merely losing its spans).")
|
||||
|
||||
cb1, cb4 = DL.expand_codebooks(d)
|
||||
palb, dark, rendered = DL.pack_palette(d)
|
||||
open(a.out + "_cb.bin", "wb").write(cb1.tobytes() + cb4.tobytes() + palb.tobytes())
|
||||
|
||||
disk, index = bytearray(), []
|
||||
for (o, n) in d.frames:
|
||||
start = len(disk)
|
||||
disk += n.to_bytes(4, "big") + d.raw[o:o + n]
|
||||
while len(disk) % 4:
|
||||
disk += b"\0"
|
||||
index.append((start, len(disk) - start))
|
||||
open(a.out + "_disk.bin", "wb").write(bytes(disk))
|
||||
|
||||
rec = np.array([n for _, n in index])
|
||||
with open(a.out + "_meta.lua", "w") as fh:
|
||||
fh.write("-- generated by tools/bench/prep_stream.py -- do not edit\nreturn {\n")
|
||||
fh.write(f" W={d.W}, H={d.H}, fps={d.fps}, nframes={d.nframes}, dark={dark},\n")
|
||||
fh.write(f" cb1_len={cb1.nbytes}, cb4_len={cb4.nbytes}, pal_len={palb.nbytes},\n")
|
||||
fh.write(f" disk_len={len(disk)}, maxrec={int(rec.max())},\n")
|
||||
fh.write(" index={\n")
|
||||
for off, ln in index:
|
||||
fh.write(f" {{off={off}, len={ln}}},\n")
|
||||
fh.write(" },\n}\n")
|
||||
|
||||
print(f"{a.container}: {d.nframes} frames, {d.W}x{d.H}")
|
||||
print(f" codebooks+palette {cb1.nbytes + cb4.nbytes + palb.nbytes:,} B -> "
|
||||
f"{a.out}_cb.bin")
|
||||
print(f" disk image {len(disk):,} B -> {a.out}_disk.bin "
|
||||
f"(records: min {rec.min():,} median {int(np.median(rec)):,} "
|
||||
f"max {rec.max():,})")
|
||||
print(f" wire rate {rec.mean()*d.fps/1024:.1f} KB/s video at {d.fps} fps")
|
||||
print(f" A ring must hold one whole record contiguously: >= {rec.max():,} B "
|
||||
f"({rec.max()/1024:.1f} KB) before any policy or prefill.")
|
||||
@@ -0,0 +1,85 @@
|
||||
-- FINDINGS 47: does CRTC R20 bit 11 ("G-VRAM set to buffer") blank the display?
|
||||
--
|
||||
-- Byte-for-byte tools/bench/show_frame256.lua -- the KNOWN-GOOD 256-colour test
|
||||
-- that check.sh gates on -- with exactly one line added: R20 bit 11 is set after
|
||||
-- MODE.apply. Everything else, including the ordinary one-word-per-pixel
|
||||
-- picture, is unchanged, so anything but a correct frame is caused by that bit.
|
||||
--
|
||||
-- MEASURED on MAME: the screen goes FULLY BLACK (max channel 0). Buffer mode is
|
||||
-- a write window, not a display mode.
|
||||
-- MEASURED on px68k (tools/bench/gvpack --keepbuffer): it does NOT blank.
|
||||
-- The two emulators disagree, and that disagreement is what FINDINGS 47.4 is
|
||||
-- about -- it decides whether the packed path is usable for continuous video.
|
||||
-- Same as show_frame.lua, but sets a REAL 256x256 CRTC mode instead of
|
||||
-- borrowing the IPL's 768x512 text timing. Proves the mode table in
|
||||
-- crtc_mode.lua and removes the x=512 wrap of FINDINGS 22.5.
|
||||
M=manager.machine; SP=M.devices[":maincpu"].spaces["program"]; SUB=nil
|
||||
|
||||
local function load_mode()
|
||||
for _,p in ipairs{"../tools/bench/crtc_mode.lua","tools/bench/crtc_mode.lua","crtc_mode.lua"} do
|
||||
local f=loadfile(p); if f then return f() end
|
||||
end
|
||||
error("crtc_mode.lua not found")
|
||||
end
|
||||
local MODE = load_mode()
|
||||
|
||||
local GVRAM, GPAL = 0xC00000, 0xE82000
|
||||
|
||||
local f=io.open("frame256.bin","rb"); local d=f:read("a"); f:close()
|
||||
local function B(i) return string.byte(d,i) end
|
||||
local W,H = B(5)*256+B(6), B(7)*256+B(8)
|
||||
local PAL0, PIX0 = 9, 9+256*3
|
||||
local YOFF = (MODE.height - H) // 2 -- letterbox 192 rows inside 256
|
||||
|
||||
-- GGGGGRRRRRBBBBBI, confirmed from x68k_v.cpp. The LSB "I" is SHARED by all
|
||||
-- three channels: each renders as pal6bit((field<<1)|I). Hardcoding I=1 (as
|
||||
-- show_frame.lua does) makes true black unreachable -- pal6bit(1) = 4 -- so I
|
||||
-- is chosen per entry to minimise summed squared error over R,G,B.
|
||||
local function pal6(v) return ((v<<2)|(v>>4)) & 0xff end
|
||||
local function pack(r,g,b)
|
||||
local f = {r>>3, g>>3, b>>3}
|
||||
local best, bestI = nil, 1
|
||||
for I=0,1 do
|
||||
local e=0
|
||||
for c=1,3 do
|
||||
local want = ({r,g,b})[c]
|
||||
local d = pal6((f[c]<<1)|I) - want
|
||||
e = e + d*d
|
||||
end
|
||||
if best==nil or e<best then best,bestI = e,I end
|
||||
end
|
||||
return (f[2]<<11)|(f[1]<<6)|(f[3]<<1)|bestI
|
||||
end
|
||||
local function T() local t=M.time; return t.seconds+t.attoseconds/1e18 end
|
||||
local st,tp="wait",nil
|
||||
|
||||
SUB = emu.add_machine_frame_notifier(function()
|
||||
local t=T()
|
||||
if st=="wait" then
|
||||
if t<3.0 then return end
|
||||
MODE.apply(SP)
|
||||
SP:write_u16(0xE80000+20*2, MODE.r20 | 0x0800) -- BUFFER MODE
|
||||
-- clear the letterbox rows: GVRAM holds IPL leftovers, not zeros
|
||||
for y=0,MODE.height-1 do
|
||||
if y<YOFF or y>=YOFF+H then
|
||||
local base=GVRAM+y*1024
|
||||
for x=0,MODE.width-1 do SP:write_u16(base+x*2,0) end
|
||||
end
|
||||
end
|
||||
for c=0,255 do
|
||||
local o=PAL0+c*3
|
||||
SP:write_u16(GPAL+c*2, pack(B(o),B(o+1),B(o+2)))
|
||||
end
|
||||
for y=0,H-1 do
|
||||
local row,base = PIX0+y*W, GVRAM+(y+YOFF)*1024
|
||||
for x=0,W-1 do SP:write_u16(base+x*2, B(row+x)) end
|
||||
end
|
||||
print(string.format("[256] R00-R08 %d %d %d %d %d %d %d %d %d R20=%04X yoff=%d t=%.3f",
|
||||
SP:read_u16(0xE80000),SP:read_u16(0xE80002),SP:read_u16(0xE80004),SP:read_u16(0xE80006),
|
||||
SP:read_u16(0xE80008),SP:read_u16(0xE8000A),SP:read_u16(0xE8000C),SP:read_u16(0xE8000E),
|
||||
SP:read_u16(0xE80010),SP:read_u16(0xE80028), YOFF, t))
|
||||
st,tp="painted",t
|
||||
elseif st=="painted" and t>tp+0.30 then
|
||||
M.video:snapshot(); print("[256] snapshot"); st="done"; M:exit()
|
||||
end
|
||||
end)
|
||||
@@ -0,0 +1,36 @@
|
||||
-- Diagnostic for FINDINGS 46.6: separate the WRITE path from the DISPLAY path.
|
||||
M=manager.machine; SP=M.devices[":maincpu"].spaces["program"]; SUB=nil
|
||||
local function load_mode()
|
||||
for _,p in ipairs{"../tools/bench/crtc_mode.lua","tools/bench/crtc_mode.lua"} do
|
||||
local f=loadfile(p); if f then return f() end end
|
||||
error("no crtc_mode.lua") end
|
||||
local MODE = load_mode()
|
||||
local CRTC, GV = 0xE80000, 0xC00000
|
||||
local function T() local t=M.time; return t.seconds+t.attoseconds/1e18 end
|
||||
local st="wait"
|
||||
SUB = emu.add_machine_frame_notifier(function()
|
||||
if st~="wait" then return end
|
||||
if T()<3.0 then return end
|
||||
MODE.apply(SP)
|
||||
local function trial(name, r20)
|
||||
SP:write_u16(CRTC+20*2, r20)
|
||||
-- clear both bytes of word 0 via the two aliases, masked mode
|
||||
SP:write_u16(CRTC+20*2, MODE.r20)
|
||||
SP:write_u16(GV, 0); SP:write_u16(GV+0x80000, 0)
|
||||
SP:write_u16(CRTC+20*2, r20)
|
||||
SP:write_u16(GV, 0xAB5C) -- the write under test
|
||||
local raw = SP:read_u16(GV)
|
||||
SP:write_u16(CRTC+20*2, MODE.r20) -- back to masked to read pages
|
||||
local p0 = SP:read_u16(GV) -- page 0 alias -> low byte
|
||||
local p1 = SP:read_u16(GV+0x80000) -- page 1 alias -> high byte
|
||||
SP:write_u16(CRTC+20*2, r20)
|
||||
print(string.format("[PROBE] %-22s R20=%04X wrote AB5C raw=%04X page0=%02X page1=%02X",
|
||||
name, r20, raw, p0 & 0xff, p1 & 0xff))
|
||||
end
|
||||
trial("masked (bit11=0)", MODE.r20)
|
||||
trial("buffer (bit11=1)", MODE.r20 | 0x0800)
|
||||
-- what does the video controller look like after MODE.apply?
|
||||
print(string.format("[PROBE] VC R0=%04X R1=%04X R2=%04X",
|
||||
SP:read_u16(0xE82400), SP:read_u16(0xE82500), SP:read_u16(0xE82600)))
|
||||
st="done"; M:exit()
|
||||
end)
|
||||
@@ -0,0 +1,52 @@
|
||||
-- Does graphics PAGE 1 display at all in 256-colour mode, and under what
|
||||
-- priority? page0 <- 0 everywhere, page1 <- 0xC0 (white) everywhere.
|
||||
-- If page 0 composites on top TRANSPARENTLY, the screen should be white.
|
||||
M=manager.machine; SP=M.devices[":maincpu"].spaces["program"]; SUB=nil
|
||||
local function load_mode()
|
||||
for _,p in ipairs{"../tools/bench/crtc_mode.lua","tools/bench/crtc_mode.lua"} do
|
||||
local f=loadfile(p); if f then return f() end end
|
||||
error("no crtc_mode.lua") end
|
||||
local MODE = load_mode()
|
||||
local CRTC, GV, GPAL = 0xE80000, 0xC00000, 0xE82000
|
||||
local function T() local t=M.time; return t.seconds+t.attoseconds/1e18 end
|
||||
local st,tp,n = "wait",nil,0
|
||||
|
||||
-- (label, VCReg1, VCReg2, page1 scrollX)
|
||||
local TRIALS = {
|
||||
{"vc1=0000 scroll384", 0x0000, 0x001F, 384},
|
||||
{"vc1=0002 scroll384", 0x0002, 0x001F, 384},
|
||||
{"vc1=06E4 scroll384", 0x06E4, 0x001F, 384},
|
||||
{"vc1=0000 scroll0", 0x0000, 0x001F, 0},
|
||||
{"vc1=0002 scroll0", 0x0002, 0x001F, 0},
|
||||
{"vc1=0000 vc2=00FF", 0x0000, 0x00FF, 0},
|
||||
}
|
||||
|
||||
SUB = emu.add_machine_frame_notifier(function()
|
||||
local t=T()
|
||||
if st=="wait" then
|
||||
if t<3.0 then return end
|
||||
MODE.apply(SP)
|
||||
SP:write_u16(GPAL+0*2, 0x0000) -- index 0 = black
|
||||
SP:write_u16(GPAL+0xC0*2, 0xFFFF) -- index C0 = white
|
||||
-- buffer mode: one pass sets page0=0x00 and page1=0xC0 for every word
|
||||
SP:write_u16(CRTC+20*2, MODE.r20 | 0x0800)
|
||||
for y=0,255 do
|
||||
local base=GV+y*1024
|
||||
for x=0,255 do SP:write_u16(base+x*2, 0xC000) end
|
||||
end
|
||||
SP:write_u16(CRTC+20*2, MODE.r20)
|
||||
st,tp="run",t
|
||||
elseif st=="run" and t>tp+0.30 then
|
||||
n = n + 1
|
||||
if n > #TRIALS then print("[P1] done"); M:exit(); st="done"; return end
|
||||
local tr = TRIALS[n]
|
||||
SP:write_u16(0xE82500, tr[2])
|
||||
SP:write_u16(0xE82600, tr[3])
|
||||
SP:write_u16(CRTC+16*2, tr[4]); SP:write_u16(CRTC+18*2, tr[4])
|
||||
print(string.format("[P1] trial %d: %s", n, tr[1]))
|
||||
tp = t
|
||||
st = "snap"
|
||||
elseif st=="snap" and t>tp+0.20 then
|
||||
M.video:snapshot(); st="run"; tp=t
|
||||
end
|
||||
end)
|
||||
@@ -0,0 +1,141 @@
|
||||
-- FINDINGS 46.6: does the PACKED layout display correctly?
|
||||
--
|
||||
-- The claim under test is that a 256-colour frame can be delivered at 1.0 byte
|
||||
-- per pixel instead of 2.0, by writing FULL 16-bit words into GVRAM and letting
|
||||
-- the two 256-colour pages show different halves of the screen:
|
||||
--
|
||||
-- * CRTC R20 bit 11 ("G-VRAM set to buffer") stops the write path masking the
|
||||
-- CPU's high byte away, so one word write lands TWO picture bytes.
|
||||
-- MAME x68k_crtc.cpp gvram_w; px68k GVRAM_Write's CRTC_Regs[0x28]&8.
|
||||
-- * word value = (page1 << 8) | page0 -- page 0 is the LOW byte, page 1 the
|
||||
-- HIGH byte (gvram_w writes `data & 0x00ff` for page 0 and
|
||||
-- `(data & 0x00ff) << 8` for page 1).
|
||||
-- * page 0 is the OPAQUE bottom layer, unscrolled: it carries screen columns
|
||||
-- 0..127 from the low bytes of words 0..127.
|
||||
-- * page 1 is the TRANSPARENT top layer, X-scrolled by 384 (= -128 mod 512).
|
||||
-- Column c fetches page1[(c + 384) & 511]:
|
||||
-- c = 128..255 -> storage 0..127 -> the high bytes of words 0..127,
|
||||
-- which carry the right half.
|
||||
-- c = 0..127 -> storage 384..511 -> zeroed, so transparent, so the
|
||||
-- opaque page 0 shows through.
|
||||
-- * so words 0..127 of each row hold the WHOLE row: 128 words = 256 bytes for
|
||||
-- 256 pixels. 1.0 byte/pixel against 2.0.
|
||||
--
|
||||
-- Which page is on top is set by video controller R1 (0xE82500). MEASURED:
|
||||
-- 0x0000 puts page 0 on top (its zeros then cover page 1 and the right half is
|
||||
-- black -- this was the first failure); 0x0002 puts page 1 on top, which is
|
||||
-- what this needs.
|
||||
--
|
||||
-- The transparency is why the blob is built with --pack-transparent: the top
|
||||
-- page's index 0 is the key, so index 0 must never appear in the picture --
|
||||
-- black lives at 255 instead.
|
||||
--
|
||||
-- PASS = the snapshot is pixel-identical to what the ordinary unpacked
|
||||
-- 256-colour path produces, which tools/bench/verify_frame256.py already checks.
|
||||
M=manager.machine; SP=M.devices[":maincpu"].spaces["program"]; SUB=nil
|
||||
|
||||
local function load_mode()
|
||||
for _,p in ipairs{"../tools/bench/crtc_mode.lua","tools/bench/crtc_mode.lua","crtc_mode.lua"} do
|
||||
local f=loadfile(p); if f then return f() end
|
||||
end
|
||||
error("crtc_mode.lua not found")
|
||||
end
|
||||
local MODE = load_mode()
|
||||
|
||||
local GVRAM, GPAL = 0xC00000, 0xE82000
|
||||
local CRTC = 0xE80000
|
||||
|
||||
local f=io.open("frame256p.bin","rb"); local d=f:read("a"); f:close()
|
||||
local function B(i) return string.byte(d,i) end
|
||||
local W,H = B(5)*256+B(6), B(7)*256+B(8)
|
||||
local PAL0, PIX0 = 9, 9+256*3
|
||||
local YOFF = (MODE.height - H) // 2
|
||||
local BLACK = 255 -- letterbox index, NOT 0
|
||||
|
||||
local function pal6(v) return ((v<<2)|(v>>4)) & 0xff end
|
||||
local function pack(r,g,b)
|
||||
local fl = {r>>3, g>>3, b>>3}
|
||||
local best, bestI = nil, 1
|
||||
for I=0,1 do
|
||||
local e=0
|
||||
for c=1,3 do
|
||||
local want = ({r,g,b})[c]
|
||||
local dd = pal6((fl[c]<<1)|I) - want
|
||||
e = e + dd*dd
|
||||
end
|
||||
if best==nil or e<best then best,bestI = e,I end
|
||||
end
|
||||
return (fl[2]<<11)|(fl[1]<<6)|(fl[3]<<1)|bestI
|
||||
end
|
||||
|
||||
-- screen index at (y,x) over the full 256x256, letterbox included
|
||||
local function pix(y,x)
|
||||
if y < YOFF or y >= YOFF+H then return BLACK end
|
||||
return B(PIX0 + (y-YOFF)*W + x)
|
||||
end
|
||||
|
||||
local function T() local t=M.time; return t.seconds+t.attoseconds/1e18 end
|
||||
local st,tp="wait",nil
|
||||
|
||||
SUB = emu.add_machine_frame_notifier(function()
|
||||
local t=T()
|
||||
if st=="wait" then
|
||||
if t<3.0 then return end
|
||||
MODE.apply(SP)
|
||||
|
||||
-- R20 with bit 11 SET: unmasked full-word writes into GVRAM.
|
||||
local R20 = MODE.r20 | 0x0800
|
||||
SP:write_u16(CRTC + 20*2, R20)
|
||||
|
||||
-- Graphic scroll. A 256-colour page is assembled from TWO nibble planes
|
||||
-- with independent scroll registers (px68k Grp_DrawLine8 reads scroll sets
|
||||
-- page*2 and page*2+1), so BOTH of a page's registers must agree or the
|
||||
-- page tears between its low and high nibble.
|
||||
-- R12/R13 = set 0 X/Y, R14/R15 = set 1 -> page 0
|
||||
-- R16/R17 = set 2 X/Y, R18/R19 = set 3 -> page 1
|
||||
SP:write_u16(CRTC + 12*2, 0); SP:write_u16(CRTC + 13*2, 0)
|
||||
SP:write_u16(CRTC + 14*2, 0); SP:write_u16(CRTC + 15*2, 0)
|
||||
SP:write_u16(CRTC + 16*2, 384); SP:write_u16(CRTC + 17*2, 0)
|
||||
SP:write_u16(CRTC + 18*2, 384); SP:write_u16(CRTC + 19*2, 0)
|
||||
|
||||
-- Priority: page 1 ON TOP of page 0, index 0 transparent. Measured on
|
||||
-- MAME (tools/bench/probe_page1.lua): 0x0000 -> page 0 on top, screen right
|
||||
-- half black; 0x0002 -> page 1 on top, right half correct.
|
||||
SP:write_u16(0xE82500, 0x0002)
|
||||
|
||||
for c=0,255 do
|
||||
local o=PAL0+c*3
|
||||
SP:write_u16(GPAL+c*2, pack(B(o),B(o+1),B(o+2)))
|
||||
end
|
||||
|
||||
-- The whole 256x256 screen, packed. Words 0..127 carry columns i (page 0,
|
||||
-- low byte) and i+128 (page 1, high byte).
|
||||
--
|
||||
-- Words 128..511 are zeroed ONCE and never touched per frame: what matters
|
||||
-- there is page 1's storage at 384..511, which the +384 scroll puts under
|
||||
-- screen columns 0..127 and which must read 0 so the opaque page 0 shows
|
||||
-- through. This is static setup, not part of the 1.0 B/pixel payload.
|
||||
for y=0,MODE.height-1 do
|
||||
local base = GVRAM + y*1024
|
||||
for i=128,511 do
|
||||
SP:write_u16(base + i*2, 0)
|
||||
end
|
||||
for i=0,127 do
|
||||
SP:write_u16(base + i*2, (pix(y, i+128) << 8) | pix(y, i))
|
||||
end
|
||||
end
|
||||
|
||||
-- Buffer mode BLANKS the graphics layer (measured: the screen is black
|
||||
-- while bit 11 is set), so it is a write window, not a display mode.
|
||||
-- Clear it now that the packed words are in and let the display read them.
|
||||
SP:write_u16(CRTC + 20*2, MODE.r20)
|
||||
|
||||
print(string.format("[PACK] R20=%04X (bit11=%d) scrollX p0=%d p1=%d "
|
||||
.."wrote %d words/row for %d px/row yoff=%d",
|
||||
SP:read_u16(CRTC+20*2), (SP:read_u16(CRTC+20*2)>>11)&1,
|
||||
SP:read_u16(CRTC+12*2), SP:read_u16(CRTC+16*2), 128, 256, YOFF))
|
||||
st,tp="painted",t
|
||||
elseif st=="painted" and t>tp+0.30 then
|
||||
M.video:snapshot(); print("[PACK] snapshot"); st="done"; M:exit()
|
||||
end
|
||||
end)
|
||||
@@ -0,0 +1,487 @@
|
||||
-- Drive src/player/stream.s: decode a whole window through a BOUNDED RING,
|
||||
-- with a modelled SCSI pipe as the producer. STATUS item 3, FINDINGS 49.
|
||||
--
|
||||
-- tools/bench/decode.lua preloads the entire container into emulated RAM and
|
||||
-- lets the 68000 walk a0 through all of it. That gate is pixel-exact over 120
|
||||
-- frames (FINDINGS 45) and tests nothing about delivery -- 45.4.1 says so in as
|
||||
-- many words. It is also RAM-bound for a reason that has nothing to do with the
|
||||
-- player: 5,261,814 B of stream needs a 6 MB machine.
|
||||
--
|
||||
-- Here the container lives in a HOST file (tools/bench/prep_stream.py's disk
|
||||
-- image) and this script plays the part of the MB89352 plus a DMAC channel:
|
||||
-- it delivers bytes at a modelled rate into a ring of DLX_RING_KB, and the
|
||||
-- 68000 decodes out of that ring and nothing else. The emulated machine holds
|
||||
-- ~256 KB of stream instead of 5 MB, so a STOCK 2 MB machine runs the whole
|
||||
-- window -- the RAM ceiling of FINDINGS 44.6.4/45 is a property of the old rig
|
||||
-- and this one does not have it.
|
||||
--
|
||||
-- WHAT IS BEING TESTED, precisely: that the block loop and the span chain --
|
||||
-- which read with a monotonically increasing a0 and no bounds check anywhere --
|
||||
-- stay pixel-exact when a0 is inside a ring one twentieth the size of the
|
||||
-- stream, and when the address it is handed jumps backwards to the ring base
|
||||
-- roughly every sixth frame. A green run is not "the decoder still works"; it
|
||||
-- is "the wrap policy in src/player/stream.s does not corrupt a single pixel of
|
||||
-- a temporally recursive 120-frame decode".
|
||||
--
|
||||
-- THE WRAP POLICY IS `aligned` (tools/analysis/19_ring_stream.py): never start a
|
||||
-- record that will not fit before the end of the ring; leave the hole and
|
||||
-- restart at the base. The alternative, letting records wrap and mirroring the
|
||||
-- ring head into a shadow, costs 5.57% of the frame budget forever against this
|
||||
-- one's 9.1% of a buffer, and the decoder is already at 91.1% of budget at p90.
|
||||
-- (Those two are s14_d5_all1500's; the gate container this rig usually runs
|
||||
-- makes it 3.64% of the budget against 5.7% of the ring. Per container.)
|
||||
--
|
||||
-- MEASUREMENT SCOPE, unchanged from decode.lua: MAME's gvram_w/gvram_r carry no
|
||||
-- timing, so decode times here are pure 68000 instruction cycles against
|
||||
-- zero-wait-state memory -- a LOWER BOUND. The pipe, likewise, is a MODEL: a
|
||||
-- constant byte rate on the emulated clock, not a simulation of the MB89352.
|
||||
-- What it is honest about is ARRIVAL ORDER and RESIDENCY, which is what the
|
||||
-- ring exists to manage; it says nothing about the clocks the DMAC steals from
|
||||
-- the 68000 while it does it. That debit is FINDINGS 43.2's and is not modelled
|
||||
-- here -- so a zero-stall result from this rig means "the bytes were in time",
|
||||
-- NOT "the frame fits".
|
||||
--
|
||||
-- Env:
|
||||
-- DLX_RING_KB ring size in KB (default 256, FINDINGS 21's)
|
||||
-- DLX_STREAM_KBPS modelled pipe, KB/s REQUIRED -- no default.
|
||||
-- 0 = unlimited, which isolates the WRAP question from the
|
||||
-- DELIVERY one and is what the green light uses.
|
||||
-- There is deliberately no default rate: this project has
|
||||
-- never measured the delivery pipe, and the figure that used
|
||||
-- to be defaulted to here was a user-supplied "4 Mbps" with
|
||||
-- no provenance that was never a bus measurement at all
|
||||
-- (FINDINGS 42.1). A default is how a folklore number ends up
|
||||
-- silently underneath a table nobody restates it in.
|
||||
-- DLX_PREFILL_KB bytes to deliver before releasing the CPU (default 0)
|
||||
-- DLX_PACE 1 = hold the decoder to META.fps (default 0 = free-run)
|
||||
-- DLX_CUT_AT frame tick at which the pipe stops dead (a seek). Needs
|
||||
-- DLX_PACE; unset = no cut.
|
||||
-- DLX_CUT_FR how many frame times the cut lasts (default 1)
|
||||
-- DLX_SLACK_CSV write the per-tick lookahead series to this path
|
||||
-- DLX_SNAP_EVERY 1 = snapshot every frame tick (needs DLX_PACE). For
|
||||
-- recording the player; not used by check.sh.
|
||||
--
|
||||
-- PACING, AND WHY THE UNPACED RIG COULD NOT ANSWER THE BUFFERING QUESTION
|
||||
-- (FINDINGS 49.7.2). Free-running, src/player/stream.s asks for record i the
|
||||
-- instant it finishes record i-1. It therefore outruns any finite pipe, the
|
||||
-- ring never backs up, `overlaps` never refuses a placement, and every ring size
|
||||
-- down to 48 KB passes while holding ONE record. That sweep tests wrap
|
||||
-- correctness, which is real, and says nothing about buffering, which is what a
|
||||
-- branch point needs. With DLX_PACE=1 the producer supplies a frame clock and
|
||||
-- the decoder may not start frame i before tick i, so the ring fills to its own
|
||||
-- capacity and FR_HEAD-FR_TAIL becomes the honest number: whole frames the
|
||||
-- decoder could run on with delivery stopped dead. DLX_CUT_AT/DLX_CUT_FR then
|
||||
-- stop it dead and check the answer against a real underrun.
|
||||
|
||||
M = manager.machine
|
||||
SP = M.devices[":maincpu"].spaces["program"]
|
||||
|
||||
local function findfile(n)
|
||||
for _,p in ipairs{"../tools/bench/"..n, "tools/bench/"..n, n} do
|
||||
local f = io.open(p,"rb"); if f then f:close(); return p end
|
||||
end
|
||||
error(n.." not found")
|
||||
end
|
||||
local MODE = loadfile(findfile("crtc_mode.lua"))()
|
||||
local META = loadfile("stream_meta.lua")()
|
||||
|
||||
local FLAG, ITER, NFR = 0x18000, 0x18008, 0x1800C
|
||||
local RD_PTR, FR_HEAD, FR_TAIL = 0x18020, 0x18024, 0x18028
|
||||
local STALLS, SPINS, DESC = 0x1802C, 0x18030, 0x18100
|
||||
local PACE, PACEON = 0x18034, 0x18038
|
||||
local DESCN = 64
|
||||
local CB1, CB4 = 0x20000, 0x22000
|
||||
local RING = 0x40000
|
||||
local GVRAM, GPAL = 0xC00000, 0xE82000
|
||||
local CPUHZ = 10000000
|
||||
local FRAME12 = CPUHZ / META.fps
|
||||
local AUDIO_KBPS = 7.8 -- ratectl.AUDIO_KBPS; the pipe carries it too
|
||||
|
||||
local RING_KB = tonumber(os.getenv("DLX_RING_KB") or "") or 256
|
||||
local KBPS = tonumber(os.getenv("DLX_STREAM_KBPS") or "")
|
||||
if KBPS == nil then
|
||||
print("[STR] DLX_STREAM_KBPS is not set and has no default. Set it to the "
|
||||
.."delivery rate you want to model, or to 0 for an unlimited pipe "
|
||||
.."(which is what tools/bench/check.sh uses -- it gates the WRAP "
|
||||
.."policy, and an unlimited pipe removes delivery as a variable).")
|
||||
manager.machine:exit()
|
||||
return
|
||||
end
|
||||
local PREFILL = (tonumber(os.getenv("DLX_PREFILL_KB") or "") or 0) * 1024
|
||||
local PACED = (os.getenv("DLX_PACE") == "1")
|
||||
local CUT_AT = tonumber(os.getenv("DLX_CUT_AT") or "")
|
||||
local CUT_FR = tonumber(os.getenv("DLX_CUT_FR") or "") or 1
|
||||
local SLACK_CSV = os.getenv("DLX_SLACK_CSV")
|
||||
-- One snapshot per frame tick instead of one at the end of the run. This is a
|
||||
-- DOCUMENTATION artefact -- 120 PNGs of a paced player, for a recording -- and
|
||||
-- it is deliberately not on any path tools/bench/check.sh takes. Needs
|
||||
-- DLX_PACE: snapshotting a free-running decoder would sample the screen at
|
||||
-- whatever rate the 68000 happened to finish frames at, which is not a frame
|
||||
-- rate and would misrepresent the player as faster than it is.
|
||||
local SNAP_EVERY = (os.getenv("DLX_SNAP_EVERY") == "1")
|
||||
local RINGSZ = RING_KB * 1024
|
||||
-- Video's share of the pipe. Debiting audio is not optional bookkeeping: the
|
||||
-- ADPCM stream comes off the same disk and out of the same budget (FINDINGS 33).
|
||||
local BPS = (KBPS > 0) and (KBPS - AUDIO_KBPS) * 1024 or math.huge
|
||||
|
||||
local code do local f=io.open("stream.bin","rb"); code=f:read("a"); f:close() end
|
||||
local cb do local f=io.open("stream_cb.bin","rb"); cb=f:read("a"); f:close() end
|
||||
local DISK = assert(io.open("stream_disk.bin","rb"))
|
||||
|
||||
local YOFF = (MODE.height - META.H) // 2
|
||||
local function T() local t=M.time; return t.seconds + t.attoseconds/1e18 end
|
||||
local function P(s) print("[STR] "..s) end
|
||||
|
||||
local function push(addr, s, from, len)
|
||||
local i, n = from, len
|
||||
while n >= 4 do
|
||||
SP:write_u32(addr, (string.unpack(">I4", s, i)))
|
||||
addr, i, n = addr+4, i+4, n-4
|
||||
end
|
||||
while n > 0 do
|
||||
SP:write_u8(addr, string.byte(s,i)); addr, i, n = addr+1, i+1, n-1
|
||||
end
|
||||
end
|
||||
|
||||
-- ------------------------------------------------------------ the producer
|
||||
-- Ring occupancy is tracked as an explicit list of records still owned by the
|
||||
-- decoder, rather than as a modular write-minus-read distance. Under `aligned`
|
||||
-- the ring is not a simple modulus -- a wrap leaves a HOLE of arbitrary size --
|
||||
-- so a distance would have to carry the holes as a correction term and would be
|
||||
-- the easiest thing in this file to get quietly wrong. Six-ish live records is
|
||||
-- a short list; an O(n) overlap test on it is exact and obviously exact.
|
||||
-- arrival[i] = emulated time at which record i became RESIDENT. This, not the
|
||||
-- decoder's spin counter, is what answers the delivery question. stream.s has
|
||||
-- no frame clock: it asks for the next record the instant it finishes the last
|
||||
-- one, so it outruns any finite pipe and "stalled" is what a decoder that is
|
||||
-- merely EARLY looks like. A shipping player waits for vblank at 12 fps and
|
||||
-- spends that same time idle. So the honest test is not "did the decoder ever
|
||||
-- wait" but "was record i resident by its 12 fps deadline", which is a question
|
||||
-- about arrival times alone and does not need the decoder paced.
|
||||
local arrival = {}
|
||||
local live, wcur, nsent = {}, 0, 0
|
||||
local n_rate, n_ring = 0, 0
|
||||
local holes, hole_bytes, credit = 0, 0, 0
|
||||
local last_t, pf_t0, delivered = nil, nil, 0
|
||||
|
||||
local function overlaps(off, len)
|
||||
for _,r in ipairs(live) do
|
||||
if off < r.off + r.len and r.off < off + len then return true end
|
||||
end
|
||||
return false
|
||||
end
|
||||
|
||||
local function reap()
|
||||
local tail = SP:read_u32(FR_TAIL)
|
||||
local i = 1
|
||||
while i <= #live do
|
||||
if live[i].idx < tail then
|
||||
-- RD_PTR is the decoder's byte-granular release, and nothing here needs
|
||||
-- it -- this producer has the index and works in whole records. Checking
|
||||
-- it makes it a cross-check instead of a field nothing reads: a real
|
||||
-- producer without an index (a DMAC chasing the CPU) has only this.
|
||||
--
|
||||
-- It holds for the LAST record retired in a pass and not for the others.
|
||||
-- RD_PTR is a single released-to pointer, so it names the end of record
|
||||
-- tail-1; if several records were consumed since the previous reap -- and
|
||||
-- under a paced decoder with a stopped pipe, several is normal -- the
|
||||
-- earlier ones are long overwritten by it. Asserting per record made the
|
||||
-- check a test of how often reap happened to run.
|
||||
if live[i].idx == tail - 1 then
|
||||
local rp = SP:read_u32(RD_PTR)
|
||||
local want = RING + live[i].off + live[i].len
|
||||
if rp ~= want then
|
||||
P(string.format("RD_PTR MISMATCH after frame %d: decoder released "
|
||||
.."%08X, record ends %08X", live[i].idx, rp, want))
|
||||
M:exit()
|
||||
end
|
||||
end
|
||||
table.remove(live, i)
|
||||
else
|
||||
i = i + 1
|
||||
end
|
||||
end
|
||||
end
|
||||
|
||||
-- Cut window: the pipe stops dead, as it does across a seek. Credit is FROZEN
|
||||
-- rather than left to accumulate -- a drive that is repositioning is not
|
||||
-- banking bytes it will burst on arrival, and letting credit build would hand
|
||||
-- the ring back everything the cut took the moment it ended, which is the one
|
||||
-- way to make a seek look free.
|
||||
local cut_t0, cut_t1, cut_done = nil, nil, false
|
||||
|
||||
local function produce(now)
|
||||
local dt = now - (last_t or now); last_t = now
|
||||
local cut = (cut_t0 and now >= cut_t0 and now < cut_t1)
|
||||
-- A seek stops DELIVERY. It does not stop the decoder, which goes on draining
|
||||
-- the ring and releasing bytes behind itself, so reap() runs either way --
|
||||
-- skipping it would have the ring look full for the whole cut and hide the
|
||||
-- one thing the cut is for.
|
||||
if not cut then credit = credit + dt * BPS end
|
||||
reap()
|
||||
if cut then return end
|
||||
while nsent < META.nframes do
|
||||
local rec = META.index[nsent + 1]
|
||||
-- WHICH RESOURCE REFUSED, counted separately. A producer that stops
|
||||
-- because it has no credit is RATE-bound and a bigger ring buys nothing; one
|
||||
-- that stops because the decoder still owns the bytes is RING-bound and a
|
||||
-- faster pipe buys nothing. The two look identical from the decoder's side
|
||||
-- -- both are simply "no new record" -- and they have opposite fixes.
|
||||
if credit < rec.len then n_rate = n_rate + 1; break end
|
||||
-- `aligned`: refuse to start a record that will not finish inside the ring.
|
||||
-- The hole is charged only once the record is actually PLACED. Charging it
|
||||
-- at the point the wrap is decided counts one hole per retry while the
|
||||
-- decoder still owns the ring base -- which is every tick of a fast pipe --
|
||||
-- and reported 105 wraps over 120 records where there are 18.
|
||||
local w, hole = wcur, 0
|
||||
if w + rec.len > RINGSZ then w, hole = 0, RINGSZ - wcur end
|
||||
if overlaps(w, rec.len) then n_ring = n_ring + 1; break end -- decoder owns them
|
||||
if hole > 0 then holes = holes + 1; hole_bytes = hole_bytes + hole end
|
||||
DISK:seek("set", rec.off)
|
||||
push(RING + w, DISK:read(rec.len), 1, rec.len)
|
||||
-- The descriptor MUST be visible before the count that advertises it. Here
|
||||
-- the CPU is stopped while this runs so the order is academic; on hardware
|
||||
-- it is not, and stream.s reads them in the opposite order for that reason.
|
||||
SP:write_u32(DESC + (nsent % DESCN) * 4, RING + w)
|
||||
live[#live+1] = {idx=nsent, off=w, len=rec.len}
|
||||
wcur, nsent = w + rec.len, nsent + 1
|
||||
credit, delivered = credit - rec.len, delivered + rec.len
|
||||
SP:write_u32(FR_HEAD, nsent)
|
||||
arrival[nsent] = now -- record nsent-1 is now resident
|
||||
end
|
||||
end
|
||||
|
||||
-- ------------------------------------------------------------------ setup
|
||||
local function setup()
|
||||
MODE.apply(SP)
|
||||
push(CB1, cb, 1, META.cb1_len)
|
||||
push(CB4, cb, 1 + META.cb1_len, META.cb4_len)
|
||||
local palo = 1 + META.cb1_len + META.cb4_len
|
||||
for c = 0, 255 do
|
||||
SP:write_u16(GPAL + c*2, (string.unpack(">I2", cb, palo + c*2)))
|
||||
end
|
||||
for y = 0, MODE.height-1 do
|
||||
local base, v = GVRAM + y*1024, 0
|
||||
if y < YOFF or y >= YOFF+META.H then v = META.dark end
|
||||
for x = 0, MODE.width-1 do SP:write_u16(base + x*2, v) end
|
||||
end
|
||||
for i = 1, #code do SP:write_u8(0x10000+i-1, string.byte(code,i)) end
|
||||
SP:write_u32(FLAG, 0); SP:write_u32(FR_HEAD, 0); SP:write_u32(FR_TAIL, 0)
|
||||
SP:write_u32(ITER, 1); SP:write_u32(NFR, META.nframes)
|
||||
SP:write_u32(PACE, 0); SP:write_u32(PACEON, PACED and 1 or 0)
|
||||
P(string.format("stream.bin=%d B, codebooks %d+%d B, disk %d B, %d frames",
|
||||
#code, META.cb1_len, META.cb4_len, META.disk_len, META.nframes))
|
||||
P(string.format("decoder %s%s", PACED and ("PACED at "..META.fps.." fps")
|
||||
or "FREE-RUNNING (tests wrap, not buffering -- 49.7.2)",
|
||||
CUT_AT and string.format(", pipe cut at tick %d for %.2f fr",
|
||||
CUT_AT, CUT_FR) or ""))
|
||||
P(string.format("ring %d KB at %06X, pipe %s, prefill %d KB, maxrec %d B",
|
||||
RING_KB, RING, (KBPS > 0) and (KBPS.." KB/s") or "unlimited",
|
||||
PREFILL // 1024, META.maxrec))
|
||||
if META.maxrec > RINGSZ then
|
||||
P("RING TOO SMALL: one record does not fit. stream.s needs a whole record "
|
||||
.."contiguous."); M:exit()
|
||||
end
|
||||
end
|
||||
|
||||
local function launch()
|
||||
local cpu = M.devices[":maincpu"]
|
||||
cpu.state["SR"].value = 0x2700 -- supervisor, ALL interrupts masked
|
||||
cpu.state["SP"].value = 0x8000
|
||||
cpu.state["PC"].value = 0x10000
|
||||
end
|
||||
|
||||
local st, t0, t_rel = "boot", nil, 0
|
||||
local pace, min_ahead, min_at = -1, math.huge, -1
|
||||
local sum_ahead, n_ahead = 0, 0
|
||||
local slack_series = {}
|
||||
|
||||
SUB = emu.add_machine_frame_notifier(function()
|
||||
local ok, err = pcall(function()
|
||||
local t = T()
|
||||
if st == "boot" then
|
||||
if t < 3.0 then return end
|
||||
setup(); last_t, pf_t0 = t, t; st = "prefill"; return
|
||||
end
|
||||
if st == "prefill" then
|
||||
produce(t)
|
||||
if delivered >= PREFILL then
|
||||
P(string.format("prefill done: %.1f KB in %.3f s, releasing the CPU",
|
||||
delivered/1024, t - pf_t0))
|
||||
launch(); t_rel = t; st = "running"
|
||||
end
|
||||
return
|
||||
end
|
||||
if st == "running" then
|
||||
if PACED then
|
||||
local tick = math.floor((t - t_rel) * META.fps)
|
||||
if tick > pace then
|
||||
pace = tick
|
||||
SP:write_u32(PACE, pace)
|
||||
-- Taken BEFORE this tick's frame is decoded, so snapshot n is the
|
||||
-- finished picture of frame n-1. tick 0 is skipped: nothing has been
|
||||
-- drawn yet and it would record a black screen as a decoded frame.
|
||||
if SNAP_EVERY and tick > 0 then M.video:snapshot() end
|
||||
-- Sampled AT the tick, before this frame is decoded: FR_TAIL is the
|
||||
-- count of frames already done, FR_HEAD the count resident, so the
|
||||
-- difference is exactly how many further frames the decoder could
|
||||
-- draw if the pipe went silent at this instant. Whole records, not
|
||||
-- bytes -- the contiguity constraint of 49.2 means a partial record
|
||||
-- buys nothing.
|
||||
local ahead = SP:read_u32(FR_HEAD) - SP:read_u32(FR_TAIL)
|
||||
-- Only sampled while the producer still HAS records to place. Once
|
||||
-- it has sent the last one the lookahead drains to zero for reasons
|
||||
-- that are about the window ending, not about the ring's capacity or
|
||||
-- the pipe's rate -- and the minimum over the whole run would then
|
||||
-- always be the drain tail, which is the one part of it that tells
|
||||
-- you nothing.
|
||||
if nsent < META.nframes then
|
||||
if ahead < min_ahead then min_ahead, min_at = ahead, tick end
|
||||
sum_ahead, n_ahead = sum_ahead + ahead, n_ahead + 1
|
||||
slack_series[#slack_series+1] = {tick, ahead, n_ring, n_rate}
|
||||
end
|
||||
if CUT_AT and tick >= CUT_AT and not cut_t0 then
|
||||
cut_t0, cut_t1 = t, t + CUT_FR / META.fps
|
||||
P(string.format("PIPE CUT at tick %d for %.2f frame times "
|
||||
.."(%.1f ms), with %d frames resident ahead",
|
||||
tick, CUT_FR, 1000*CUT_FR/META.fps, ahead))
|
||||
end
|
||||
end
|
||||
end
|
||||
produce(t)
|
||||
if cut_t0 and not cut_done and t >= cut_t1 then cut_done = true end
|
||||
local fl = SP:read_u32(FLAG)
|
||||
if fl == 1 and not t0 then t0 = t; return end
|
||||
if fl == 0xEE then
|
||||
P("BITSTREAM DESYNC -- the decoder consumed the wrong number of bytes.")
|
||||
P(" Under a ring that is the whole point: it means a record was placed "
|
||||
.."or described wrongly, not that the codec changed.")
|
||||
M:exit(); return
|
||||
end
|
||||
if fl == 0xE1 then
|
||||
P("PRODUCER STALLED OUT -- stream.s spun SPINMAX times with no new "
|
||||
.."record. Delivered "..nsent.."/"..META.nframes..".")
|
||||
M:exit(); return
|
||||
end
|
||||
if fl == 0xFF then
|
||||
local dt = t - (t0 or t)
|
||||
local stalls = SP:read_u32(STALLS)
|
||||
local spins = SP:read_u32(SPINS)
|
||||
if PACED then
|
||||
-- Paced, the wall clock measures the PACE, not the decode: the loop
|
||||
-- spends whatever is left of each slot spinning in `pacewait`. The
|
||||
-- decode cost of this container is decode.lua's and FINDINGS 45's;
|
||||
-- printing a cycles/frame here would just report 1/fps back.
|
||||
P(string.format("decoded %d frames in %.4f s emulated at a %d fps "
|
||||
.."pace (%.2f s nominal) -- the clock here measures "
|
||||
.."the PACE, not the decode",
|
||||
META.nframes, dt, META.fps, META.nframes/META.fps))
|
||||
else
|
||||
P(string.format("decoded %d frames in %.4f s emulated -> %.0f cycles/"
|
||||
.."frame = %.1f%% of a %dfps frame",
|
||||
META.nframes, dt, dt*CPUHZ/META.nframes,
|
||||
100*(dt*CPUHZ/META.nframes)/FRAME12, META.fps))
|
||||
end
|
||||
P(string.format("ring: %d wraps, %d B of hole (mean %.1f KB, %.1f%% of "
|
||||
.."the ring)", holes, hole_bytes,
|
||||
holes > 0 and hole_bytes/holes/1024 or 0,
|
||||
100*(holes > 0 and hole_bytes/holes or 0)/RINGSZ))
|
||||
-- The decoder's own spin counter, kept for what it is: evidence that
|
||||
-- stream.s outran the pipe, NOT evidence of an underrun. See the note
|
||||
-- on `arrival` above.
|
||||
if PACED then
|
||||
-- Paced, a stall is an UNDERRUN: the frame's slot arrived and its
|
||||
-- record had not. Free-running it is earliness (49.6) and means the
|
||||
-- opposite thing, so the two are never printed in the same words.
|
||||
P(string.format("UNDERRUNS: %d/%d frames waited past their %d fps slot "
|
||||
.."(%d polls)", stalls, META.nframes, META.fps, spins))
|
||||
-- WHAT THE MINIMUM OVER A RUN IS, AND IS NOT. At release the ring
|
||||
-- holds only what the prefill put there, so the early ticks report a
|
||||
-- buffer that has not been built yet, not a ring or a pipe that
|
||||
-- cannot build it. A seek empties the ring the same way, so that
|
||||
-- transient is the branch-point case rather than an artefact to be
|
||||
-- trimmed -- but it has to be told apart from the CEILING, which is
|
||||
-- what the ring is worth once it is full.
|
||||
--
|
||||
-- Which resource stopped the producer says which is which, and only
|
||||
-- RING refusals count: a rate refusal fires on nearly every tick
|
||||
-- (credit accrues continuously and records are placed whole), so it
|
||||
-- marks nothing. A ring refusal means the ring actually filled.
|
||||
local ceiling, ceil_at = 0, -1
|
||||
for _,e in ipairs(slack_series) do
|
||||
if e[2] > ceiling then ceiling, ceil_at = e[2], e[1] end
|
||||
end
|
||||
local ss_min, ss_at = math.huge, -1
|
||||
for _,e in ipairs(slack_series) do
|
||||
if e[3] > 0 and e[2] < ss_min then ss_min, ss_at = e[2], e[1] end
|
||||
end
|
||||
P(string.format("SEEK SLACK: ceiling %d frames (%.0f ms), first "
|
||||
.."reached at tick %d; mean %.1f over the window",
|
||||
ceiling, 1000*ceiling/META.fps, ceil_at,
|
||||
sum_ahead/math.max(1,n_ahead)))
|
||||
if n_ring > 0 then
|
||||
P(string.format(" RING-BOUND: the ring filled (%d refusals). Once "
|
||||
.."full it survives delivery stopped dead for %d "
|
||||
.."frames = %.0f ms; min after first fill %d "
|
||||
.."(tick %d)", n_ring, ceiling,
|
||||
1000*ceiling/META.fps, ss_min, ss_at))
|
||||
else
|
||||
P(string.format(" RATE-BOUND: the ring NEVER filled in %d frames. "
|
||||
.."A bigger ring buys nothing at this pipe; the "
|
||||
.."buffer is still accumulating when the window "
|
||||
.."ends.", META.nframes))
|
||||
end
|
||||
P(string.format(" BUILD TIME: %d ticks = %.2f s of play to reach the "
|
||||
.."ceiling from empty -- which is what a branch point "
|
||||
.."costs before the NEXT seek is affordable",
|
||||
ceil_at, ceil_at/META.fps))
|
||||
if SLACK_CSV then
|
||||
local fh = io.open(SLACK_CSV, "w")
|
||||
fh:write("tick,ahead,ring_refusals,rate_refusals\n")
|
||||
for _,e in ipairs(slack_series) do
|
||||
fh:write(string.format("%d,%d,%d,%d\n", e[1], e[2], e[3], e[4]))
|
||||
end
|
||||
fh:close()
|
||||
P("slack series -> "..SLACK_CSV)
|
||||
end
|
||||
else
|
||||
P(string.format("decoder waited on %d/%d frames (%d polls) -- it is "
|
||||
.."free-running, so this is earliness, not underrun",
|
||||
stalls, META.nframes, spins))
|
||||
end
|
||||
|
||||
-- The delivery result. Deadline for record i is release + i/fps.
|
||||
local misses, worst, prefill_s = 0, 0.0, 0.0
|
||||
for i = 0, META.nframes-1 do
|
||||
local a = arrival[i+1]
|
||||
if a then
|
||||
local late = a - (t_rel + i / META.fps)
|
||||
if late > 0 then
|
||||
misses = misses + 1
|
||||
if late > worst then worst = late end
|
||||
end
|
||||
if late > prefill_s then prefill_s = late end
|
||||
end
|
||||
end
|
||||
P(string.format("DEADLINE at %d fps: %d/%d records late, worst by "
|
||||
.."%.1f ms (%.2f frame times)", META.fps, misses,
|
||||
META.nframes, worst*1000, worst*META.fps))
|
||||
-- The smallest start delay that makes every deadline: FINDINGS 21's
|
||||
-- "required prefill", measured on the emulated clock through the real
|
||||
-- ring rather than simulated from record sizes.
|
||||
P(string.format("REQUIRED PREFILL: %.1f ms = %.2f frame times = %.1f KB "
|
||||
.."at %s", prefill_s*1000, prefill_s*META.fps,
|
||||
(KBPS > 0) and (prefill_s * BPS / 1024) or 0.0,
|
||||
(KBPS > 0) and (KBPS.." KB/s") or "an unlimited pipe"))
|
||||
M.video:snapshot()
|
||||
P("snapshot taken after the sequential pass -- last frame, decoded "
|
||||
.."entirely out of a "..RING_KB.." KB ring")
|
||||
st = "snapped"; return
|
||||
end
|
||||
if t > 900 then P("TIMEOUT flag="..string.format("%08X",fl)); M:exit() end
|
||||
return
|
||||
end
|
||||
if st == "snapped" then M:exit(); return end
|
||||
end)
|
||||
if not ok then print("[STR] LUA ERROR: "..tostring(err)); M:exit() end
|
||||
end)
|
||||
@@ -19,8 +19,9 @@ import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
snap = sys.argv[1] if len(sys.argv) > 1 else "tmp/snap256/x68000/0000.png"
|
||||
blob = sys.argv[2] if len(sys.argv) > 2 else "tmp/frame256.bin"
|
||||
s = np.asarray(Image.open(snap).convert("RGB")).astype(int)
|
||||
d = open("tmp/frame256.bin", "rb").read()
|
||||
d = open(blob, "rb").read()
|
||||
W, H = struct.unpack(">HH", d[4:8])
|
||||
pal = np.frombuffer(d[8:8+768], np.uint8).reshape(256, 3).astype(int)
|
||||
idx = np.frombuffer(d[8+768:8+768+W*H], np.uint8).reshape(H, W)
|
||||
|
||||
+47
-6
@@ -174,22 +174,44 @@ def main():
|
||||
help="override the profile's bitrate CEILING. The profile "
|
||||
"is a rate point on a delivery medium; this is for "
|
||||
"asking what the codec does at another one -- e.g. "
|
||||
"the 488 KB/s bus figure the span analyses of "
|
||||
"FINDINGS 30/40 are scored against.")
|
||||
"a measured delivery rate. Do not reach for the "
|
||||
"retired 4 Mbps figure; FINDINGS 42.1.")
|
||||
ap.add_argument("--span-kbps", type=float, default=None,
|
||||
help="byte ceiling the SPAN pass may draw on, if it differs "
|
||||
"from the profile's. The profile is a quality rate "
|
||||
"point; the pipe is hardware. Bytes between the two "
|
||||
"buy a better picture if spent on lam and the 68000's "
|
||||
"deadline if spent on spans -- and nothing at all if "
|
||||
"left unspent (FINDINGS 41.2). 488 is the bus figure "
|
||||
"tools/analysis/14_dmac_chain.py scores against.")
|
||||
"left unspent (FINDINGS 41.2). Pass the pipe rate "
|
||||
"tools/analysis/14_dmac_chain.py is scored against.")
|
||||
ap.add_argument("--spans", choices=("off", "need", "all"), default="need",
|
||||
help="v7 literal spans (FINDINGS 40). `need` (default) "
|
||||
"spends container bytes on spans only where a frame "
|
||||
"misses the 68000's decode deadline; `all` spends "
|
||||
"every profitable byte, which is the model "
|
||||
"14_dmac_chain.py scores; `off` emits DLX2.")
|
||||
ap.add_argument("--disk-clk-byte", type=float, default=None,
|
||||
help="clocks the SCSI DMA steals per DELIVERED BYTE, "
|
||||
"charged against the same frame budget the decoder "
|
||||
"spends (FINDINGS 43). Default 5.0, the "
|
||||
"single-address floor; 9.0 is dual-address; 0 "
|
||||
"restores the pre-43 encoder, which priced a byte at "
|
||||
"nothing and reported deadlines it could not meet.")
|
||||
ap.add_argument("--joint-decide", action="store_true",
|
||||
help="let the per-block lagrangian see the disk debit too, "
|
||||
"pricing a payload byte at lam+mu*c instead of lam. "
|
||||
"The default is OFF because it MEASURES as a wash: "
|
||||
"same 1/120, 0.02 dB worse, and it trades 6,058 "
|
||||
"clocks of disk for 17,207 of block decode "
|
||||
"(FINDINGS 44)")
|
||||
ap.add_argument("--joint-bucket", action="store_true",
|
||||
help="cap what the leaky bucket may lend a frame at what "
|
||||
"its clock budget can still absorb, since a borrowed "
|
||||
"byte is DISK_CLK_BYTE borrowed clocks and there is "
|
||||
"no double buffer to repay them from (FINDINGS 43.5). "
|
||||
"Default OFF: measured, it is worth one frame of 120 "
|
||||
"at --spans need and a 2%% regression at --spans all "
|
||||
"(FINDINGS 44)")
|
||||
ap.add_argument("--no-cpu-fit", action="store_true",
|
||||
help="drop the per-frame 68000 decode ceiling (session 7 "
|
||||
"behaviour: 31%% of frames on hard content do not fit)")
|
||||
@@ -214,6 +236,10 @@ def main():
|
||||
# The CPU ceiling is hardware, not taste: without it 31%% of frames on the
|
||||
# worst sustained window do not decode in time on a stock 68000, and with
|
||||
# it that is one frame -- the intra frame -- for 0.26 dB. FINDINGS 31.
|
||||
if a.disk_clk_byte is not None:
|
||||
RC.DISK_CLK_BYTE = a.disk_clk_byte
|
||||
RC.JOINT_DECIDE = a.joint_decide
|
||||
RC.JOINT_BUCKET = a.joint_bucket
|
||||
cyc_budget = None if a.no_cpu_fit else RC.FRAME_CYCLES
|
||||
span_mode = None if (a.spans == "off" or not rc) else a.spans
|
||||
|
||||
@@ -225,6 +251,13 @@ def main():
|
||||
print(f" CPU ceiling: " + (f"mu bisected per frame against "
|
||||
f"{RC.FRAME_CYCLES:,.0f} cycles (12fps, stock 68000)"
|
||||
if cyc_budget else "OFF (--no-cpu-fit)"))
|
||||
print(f" disk debit: {RC.DISK_CLK_BYTE:g} clocks per delivered byte, "
|
||||
f"charged INSIDE that ceiling (FINDINGS 43), and "
|
||||
+ ("SEEN by the mode decision at lam+mu*c (--joint-decide)"
|
||||
if RC.JOINT_DECIDE else "not seen by the mode decision, "
|
||||
"which measures as the better container (FINDINGS 44)")
|
||||
if RC.DISK_CLK_BYTE else
|
||||
" disk debit: 0 -- bytes priced at nothing (pre-FINDINGS-43)")
|
||||
else:
|
||||
print(f" target {prof['kbps']} KB/s, FIXED lam={lam} (no rate control)")
|
||||
print(f" k1={k1} k4={k4}, {_IDX_BYTES}-byte indices")
|
||||
@@ -305,11 +338,19 @@ def main():
|
||||
# moved into the span section rather than removed.
|
||||
cyc = (np.asarray(enc["cycles"]) if "cycles" in enc
|
||||
else np.array([H.cycles(mm) for mm in enc["modes"]]))
|
||||
pct = 100 * cyc / RC.FRAME_CYCLES
|
||||
# The frame's real cost is decode PLUS the bus the SCSI DMA steals to
|
||||
# deliver it. Reporting only `cyc` is what let session 13 print 0/120 for
|
||||
# a container no machine could have played (FINDINGS 43).
|
||||
disk = np.asarray(enc["sizes"], float) * RC.DISK_CLK_BYTE
|
||||
pct = 100 * (cyc + disk) / RC.FRAME_CYCLES
|
||||
miss = int((pct > 100).sum())
|
||||
print(f" non-SKIP blocks/frame: median {np.median(ns):.1f}% "
|
||||
f"p90 {np.percentile(ns, 90):.1f}% max {ns.max():.1f}%")
|
||||
print(f" decode cost: median {np.median(pct):.1f}% "
|
||||
dpct = 100 * disk / RC.FRAME_CYCLES
|
||||
print(f" disk debit: median {np.median(dpct):.1f}% "
|
||||
f"p90 {np.percentile(dpct, 90):.1f}% max {dpct.max():.1f}% "
|
||||
f"of the frame budget, at {RC.DISK_CLK_BYTE:g} clocks/byte")
|
||||
print(f" decode+disk: median {np.median(pct):.1f}% "
|
||||
f"p90 {np.percentile(pct, 90):.1f}% max {pct.max():.1f}% "
|
||||
f"of a {a.fps}fps frame")
|
||||
print(f" frames that do NOT decode in time: {miss}/{len(pct)} "
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Derive quality profiles FROM a measured bandwidth, instead of guessing lam.
|
||||
|
||||
python3 tools/encoder/profile_gen.py --bw-kbps 488 --name scsi
|
||||
python3 tools/encoder/profile_gen.py --bw-kbps <KB/s> --name scsi
|
||||
python3 tools/encoder/profile_gen.py --bw-mbps 4 # same thing
|
||||
|
||||
Session 2 set the profile bitrates by eye off the rate-distortion knee, which
|
||||
@@ -18,7 +18,7 @@ Three things eat the pipe before video gets any:
|
||||
passes with ZERO required prefill. Peak sizing is kept
|
||||
behind --size-for-peak only as a pessimistic bound.
|
||||
3. DMA CYCLE-STEAL -- the HD63450 steals ~8 clocks per 16-bit word from the
|
||||
68000. At 488 KB/s that is 20% of the CPU, on top of the
|
||||
68000. At ~500 KB/s that is ~20% of the CPU, on top of the
|
||||
blit. Bandwidth and CPU are NOT independent budgets.
|
||||
FINDINGS 5 said streaming "costs essentially no CPU";
|
||||
that is wrong -- cycle-stealing DMA is not free DMA.
|
||||
|
||||
+73
-10
@@ -110,6 +110,58 @@ MU_FLOOR = 1e-4 # bisection is geometric, so lo must be > 0
|
||||
# a frame that misses its deadline is simply late. FINDINGS 28.
|
||||
FRAME_CYCLES = 10_000_000 / 12.0
|
||||
|
||||
# What one delivered byte costs the 68000, in clocks of that same budget. The
|
||||
# SCSI DMA does not overlap with the CPU -- it steals the bus (FINDINGS 38.3) --
|
||||
# and the MB89352 is an 8-bit port, so the DMAC pays PER BYTE and not per word
|
||||
# (FINDINGS 43). 5.0 is the floor: single-address device-to-memory, bus held,
|
||||
# zero drive wait (MC68450 Fig 4-25 sheet 2). Dual-address costs 9.
|
||||
# 0.0 restores the pre-43 encoder, which priced a byte at nothing.
|
||||
DISK_CLK_BYTE = 5.0
|
||||
|
||||
# Does the MODE DECISION see that debit, or only the fit test above it?
|
||||
# FINDINGS 43.6 charged the disk inside the rate controller's ceiling but left
|
||||
# vq_hybrid.decide() ranking modes with a free byte, so `mu` still bought
|
||||
# cycles by moving V4 -> RAW: 47.8 cycles saved for 12 extra payload bytes,
|
||||
# which at any c >= 4 clocks/byte is a net LOSS on the very budget mu is
|
||||
# enforcing. True makes `decide` price a byte at `lam + mu*DISK_CLK_BYTE`.
|
||||
#
|
||||
# IT IS OFF, AND THAT IS A MEASUREMENT, NOT AN OVERSIGHT (FINDINGS 44). The
|
||||
# inconsistency is real and the fix is a wash: on the 120-frame Singe window at
|
||||
# c=5 it delivers 482.5 KB/s / 29.17 dB against 496.7 / 29.19, the same 1/120
|
||||
# frames over budget, the same 112.9% worst frame, and 5,389 MORE clocks in the
|
||||
# mean frame -- it buys 6,058 clocks of disk with 17,207 clocks of block
|
||||
# decode. Scored at c=4 and c=9 as well, it is marginally the worse container
|
||||
# at every price. `--joint-decide` turns it on.
|
||||
JOINT_DECIDE = False
|
||||
|
||||
# The leaky bucket lets a quiet frame bank bytes for a busy one, because the
|
||||
# player's ring buffer can hold them. True when a byte was only a byte. A byte
|
||||
# is now also DISK_CLK_BYTE clocks of the frame's decode budget, and FINDINGS 28
|
||||
# established there is no double buffer to decode ahead into -- so a frame that
|
||||
# borrows bytes from the bucket borrows clocks it cannot repay (FINDINGS 43.5).
|
||||
# True caps the borrow at what the clock budget can still absorb; the bucket
|
||||
# then smooths only what is left after the disk is paid. It never caps BELOW
|
||||
# the frame's own un-banked byte budget: that is rate control's job, not the
|
||||
# clock budget's.
|
||||
#
|
||||
# IT IS OFF, AND THAT IS A MEASUREMENT (FINDINGS 44). At `--spans all`, the
|
||||
# mode this project now recommends, the byte-side controller is INERT: a
|
||||
# 32-frame bucket and an 8-frame bucket emit the same container byte for byte,
|
||||
# and lam never leaves its floor on any of 120 frames. The cap is worth one
|
||||
# frame of 120 at `--spans need` (2/120 -> 1/120, for +26 KB/s) and is a 2%
|
||||
# regression at `all` (506.4 KB/s and 89.6% median frame against 496.7 and
|
||||
# 87.9%), because capping the block payload only moves those bytes into the
|
||||
# span section, which draws on its own flat pipe. Extending the cap to the
|
||||
# span section too was measured and is much worse: 273.7 KB/s, 28.88 dB, and
|
||||
# still 1/120 -- it starves the pass that was buying the deadline.
|
||||
# `--joint-bucket` turns it on.
|
||||
JOINT_BUCKET = False
|
||||
|
||||
|
||||
def _byte_clk():
|
||||
"""The debit the mode decision is allowed to see (0 = the old decision)."""
|
||||
return DISK_CLK_BYTE if JOINT_DECIDE else 0.0
|
||||
|
||||
AUDIO_KBPS = 7.8 # MSM6258 ADPCM 15.6kHz mono -- comes out of the same budget
|
||||
|
||||
|
||||
@@ -129,17 +181,18 @@ def _search_lam(ctx, allow, lam_lo, lam_hi, iters=12, mu=0.0):
|
||||
not fit: that frame is emitted over budget on purpose. Past the FINDINGS 15
|
||||
cliff a frame is not rate-controlled, it is destroyed, so a visible overrun
|
||||
is the better failure (FINDINGS 26.2)."""
|
||||
mode, sz = H.decide(ctx, lam_lo, mu)
|
||||
bc = _byte_clk()
|
||||
mode, sz = H.decide(ctx, lam_lo, mu, bc)
|
||||
if sz <= allow:
|
||||
return lam_lo, mode, sz, False
|
||||
mode_hi, sz_hi = H.decide(ctx, lam_hi, mu)
|
||||
mode_hi, sz_hi = H.decide(ctx, lam_hi, mu, bc)
|
||||
if sz_hi > allow:
|
||||
return lam_hi, mode_hi, sz_hi, True
|
||||
lo, hi = lam_lo, lam_hi # lo does not fit, hi does
|
||||
best = (lam_hi, mode_hi, sz_hi)
|
||||
for _ in range(iters):
|
||||
mid = float(np.sqrt(lo * hi))
|
||||
mode_m, sz_m = H.decide(ctx, mid, mu)
|
||||
mode_m, sz_m = H.decide(ctx, mid, mu, bc)
|
||||
if sz_m <= allow:
|
||||
hi = mid; best = (mid, mode_m, sz_m)
|
||||
else:
|
||||
@@ -172,12 +225,12 @@ def _search_mu(ctx, allow, lam_lo, lam_hi, cyc_budget, iters=10):
|
||||
Returns (mu, lam, mode, size, cyc, over_bytes, over_cycles)."""
|
||||
lam, mode, sz, ovr = _search_lam(ctx, allow, lam_lo, lam_hi, mu=0.0)
|
||||
cyc = H.cycles(mode)
|
||||
if cyc <= cyc_budget:
|
||||
if cyc + DISK_CLK_BYTE * sz <= cyc_budget:
|
||||
return 0.0, lam, mode, sz, cyc, ovr, False
|
||||
|
||||
lam_h, mode_h, sz_h, ovr_h = _search_lam(ctx, allow, lam_lo, lam_hi, mu=MU_CLIFF)
|
||||
cyc_h = H.cycles(mode_h)
|
||||
if cyc_h > cyc_budget: # cannot fit even frozen: emit late
|
||||
if cyc_h + DISK_CLK_BYTE * sz_h > cyc_budget: # cannot fit even frozen
|
||||
return MU_CLIFF, lam_h, mode_h, sz_h, cyc_h, ovr_h, True
|
||||
|
||||
lo, hi = MU_FLOOR, MU_CLIFF # lo overruns, hi fits
|
||||
@@ -186,7 +239,7 @@ def _search_mu(ctx, allow, lam_lo, lam_hi, cyc_budget, iters=10):
|
||||
mid = float(np.sqrt(lo * hi))
|
||||
lam_m, mode_m, sz_m, ovr_m = _search_lam(ctx, allow, lam_lo, lam_hi, mu=mid)
|
||||
cyc_m = H.cycles(mode_m)
|
||||
if cyc_m <= cyc_budget:
|
||||
if cyc_m + DISK_CLK_BYTE * sz_m <= cyc_budget:
|
||||
hi = mid; best = (mid, lam_m, mode_m, sz_m, cyc_m, ovr_m)
|
||||
else:
|
||||
lo = mid
|
||||
@@ -226,7 +279,7 @@ def _fit_spans(m, ctx, mode, sz, room, cyc_budget, span_mode, ib):
|
||||
return mode, sz, H.cycles(mode), None
|
||||
sel = SP.select(mode, src, m["nbx"], m["nby"], room,
|
||||
need_clocks=(None if span_mode == "all" else cyc_budget),
|
||||
idx_bytes=ib)
|
||||
idx_bytes=ib, disk_clk_byte=DISK_CLK_BYTE, base_bytes=sz)
|
||||
if not sel["spans"]:
|
||||
return mode, sz, H.cycles(mode), None
|
||||
nmode = sel["mode"]
|
||||
@@ -292,16 +345,26 @@ def encode_rate_controlled(m, target_kbps, fps=12, bucket_frames=8,
|
||||
# and a pipe cannot be saved up. None means "the same allowance the lam
|
||||
# search had", which is what leaves spans nothing to buy with at a rate
|
||||
# point the block coder has already spent (FINDINGS 41.2).
|
||||
hard = None
|
||||
if (JOINT_BUCKET and cycle_budget is not None and DISK_CLK_BYTE > 0):
|
||||
# What can this frame's clock budget still absorb, once its own
|
||||
# block decode is paid? Priced at the mode map the UN-banked budget
|
||||
# buys, so the cap is a property of the frame's content rather than
|
||||
# of how full the bucket happens to be.
|
||||
_, mode0, _, _ = _search_lam(ctx, budget, lam_lo, lam_hi)
|
||||
hard = (cycle_budget - H.cycles(mode0)) / DISK_CLK_BYTE
|
||||
allow = min(allow, max(budget, hard))
|
||||
span_allow = allow if span_budget is None else span_budget
|
||||
sel = None
|
||||
lam, mode, sz, ovr = _search_lam(ctx, allow, lam_lo, lam_hi)
|
||||
mu, cyc, late = 0.0, H.cycles(mode), False
|
||||
if span_mode and (span_mode == "all"
|
||||
or (cycle_budget is not None and cyc > cycle_budget)):
|
||||
or (cycle_budget is not None
|
||||
and cyc + DISK_CLK_BYTE * sz > cycle_budget)):
|
||||
mode_pre = mode
|
||||
mode, sz, cyc, sel = _fit_spans(m, ctx, mode, sz, span_allow,
|
||||
cycle_budget, span_mode, ib)
|
||||
if cycle_budget is not None and cyc > cycle_budget:
|
||||
if cycle_budget is not None and cyc + DISK_CLK_BYTE * sz > cycle_budget:
|
||||
# The byte allowance could not buy the frame's deadline, so fall
|
||||
# back to the controller that pays in picture -- and then offer
|
||||
# spans the bytes the smaller mode map just freed.
|
||||
@@ -311,7 +374,7 @@ def encode_rate_controlled(m, target_kbps, fps=12, bucket_frames=8,
|
||||
mode_pre = mode
|
||||
mode, sz, cyc, sel = _fit_spans(m, ctx, mode, sz, span_allow,
|
||||
cycle_budget, span_mode, ib)
|
||||
late = cyc > cycle_budget
|
||||
late = cyc + DISK_CLK_BYTE * sz > cycle_budget
|
||||
# Paint from the mode map as it was BEFORE spanning. A spanned run's
|
||||
# blocks read SKIP in the emitted header, but SKIP means "hold the
|
||||
# previous reconstruction" and on the first frame there is none -- and
|
||||
|
||||
+25
-9
@@ -117,20 +117,31 @@ BLK_BYT = {1: 1, 2: 4, 3: 16}
|
||||
|
||||
|
||||
def select(mode, src_idx, nbx, nby, byte_room, need_clocks=None,
|
||||
idx_bytes=1):
|
||||
idx_bytes=1, disk_clk_byte=0.0, base_bytes=0):
|
||||
"""Choose which runs to paint as spans.
|
||||
|
||||
`mode` 1-D mode map, modified nowhere (a new one is returned)
|
||||
`src_idx` (H, W) palettised source -- what the spans will carry
|
||||
`byte_room` container bytes the frame may still spend
|
||||
`need_clocks` stop as soon as the frame's decode cost is at or below this;
|
||||
`need_clocks` stop as soon as the frame's cost is at or below this;
|
||||
None spends every profitable byte instead (the model
|
||||
tools/analysis/14_dmac_chain.py scores).
|
||||
`disk_clk_byte` what a delivered byte costs the 68000 in clocks, because
|
||||
the SCSI DMA steals the bus from it (FINDINGS 43). 0.0 is
|
||||
the pre-43 behaviour: bytes are free and a span is judged on
|
||||
decode clocks alone. At the real value a span's two wire
|
||||
bytes per pixel are the dominant term and the ranking
|
||||
inverts on short runs.
|
||||
`base_bytes` the frame's byte count before any span is added, so the
|
||||
running total this loop compares against `need_clocks` can
|
||||
include the disk debit the frame is already carrying.
|
||||
|
||||
Ranked by clocks saved per byte spent, which is the same greedy 12 and 14
|
||||
use. Selection is deliberately conservative in two ways and the reported
|
||||
figures are exact rather than greedy: a run is only offered if the span
|
||||
beats the blocks it replaces on cycles ALONE, and the saving credited here
|
||||
use. A run is only offered if the span beats the blocks it replaces on
|
||||
TOTAL clocks -- decode plus the disk debit of the bytes it adds -- which at
|
||||
`disk_clk_byte`=0 reduces to the cycles-alone test this used before
|
||||
FINDINGS 43. Selection is otherwise deliberately conservative and the
|
||||
reported figures are exact rather than greedy: the saving credited here
|
||||
ignores the extra all-SKIP header bytes spanning tends to create. The
|
||||
caller recomputes the frame's real cost from the returned mode map.
|
||||
|
||||
@@ -146,10 +157,13 @@ def select(mode, src_idx, nbx, nby, byte_room, need_clocks=None,
|
||||
cur_b = sum(BLK_BYT[int(b)] * (idx_bytes if int(b) != 3 else 1)
|
||||
for b in m2[by][i:j])
|
||||
sc = run_clocks(L) + L * C_SKIP_MIXED # the dispatch still happens
|
||||
if sc >= cur_c:
|
||||
continue
|
||||
db = run_bytes(L) - cur_b
|
||||
cand.append(((cur_c - sc) / max(db, 1), cur_c - sc, db, by, i, j))
|
||||
# TOTAL saving: decode clocks won, less the clocks the extra bytes cost
|
||||
# on the way in. With disk_clk_byte=0 this is (cur_c - sc) exactly.
|
||||
net = (cur_c - sc) - disk_clk_byte * db
|
||||
if net <= 0:
|
||||
continue
|
||||
cand.append((net / max(db, 1), cur_c - sc, db, by, i, j))
|
||||
cand.sort(key=lambda s: -s[0])
|
||||
|
||||
# `need_clocks` is measured against the frame as it stands, so the loop
|
||||
@@ -161,7 +175,9 @@ def select(mode, src_idx, nbx, nby, byte_room, need_clocks=None,
|
||||
total_b, total_c = 0.0, 0.0
|
||||
chosen = []
|
||||
for _, dc, db, by, i, j in cand:
|
||||
if need_clocks is not None and H.cycles(cur) + total_c <= need_clocks:
|
||||
if (need_clocks is not None
|
||||
and H.cycles(cur) + total_c
|
||||
+ disk_clk_byte * (base_bytes + total_b) <= need_clocks):
|
||||
break
|
||||
if total_b + db > byte_room:
|
||||
continue
|
||||
|
||||
+45
-6
@@ -15,6 +15,7 @@ entries MUST be legal palette indices -- both quantisation losses compose.
|
||||
No sklearn on this box; k-means is hand-rolled (chunked, numpy).
|
||||
"""
|
||||
import numpy as np, glob, os, sys
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from PIL import Image
|
||||
|
||||
BW = BH = 4 # block size
|
||||
@@ -83,14 +84,52 @@ def kmeans(X, k, iters=24, seed=0):
|
||||
return C, assign(X, C)
|
||||
|
||||
|
||||
def assign(X, C, chunk=8192):
|
||||
"""Nearest centroid, chunked to bound memory."""
|
||||
# k-means assignment is ~95% of an encode's wall clock (the rest of the encoder,
|
||||
# rate control included, is about a second for a 120-frame window), so it is
|
||||
# worth the three details below. All three are EXACT: the labels are unchanged
|
||||
# bit for bit, which is what lets the containers this encoder emits stay
|
||||
# byte-identical across the change.
|
||||
_ASSIGN_CHUNK = 2048 # measured: see the note in assign()
|
||||
_ASSIGN_THREADS = min(8, (os.cpu_count() or 1))
|
||||
|
||||
|
||||
def _assign_range(X, CT, Cn, out, lo, hi, chunk):
|
||||
for i in range(lo, hi, chunk):
|
||||
j = min(i + chunk, hi)
|
||||
d = Cn[None, :] - 2.0 * (X[i:j] @ CT) # + |x|^2, constant per row
|
||||
out[i:j] = np.argmin(d, axis=1)
|
||||
|
||||
|
||||
def assign(X, C, chunk=_ASSIGN_CHUNK, threads=None):
|
||||
"""Nearest centroid, chunked to bound memory.
|
||||
|
||||
Three things make this 5.9x faster than the obvious version, measured on
|
||||
the 120-frame Singe window (1,474,560 2x2 blocks against k=256), and none
|
||||
of them changes a label:
|
||||
|
||||
* `C.T` is a VIEW, and a non-contiguous right-hand operand makes BLAS
|
||||
copy it per chunk: 1.83s -> 1.02s just from materialising it once.
|
||||
* chunk=2048, not 8192. The temporary is (chunk, k) float32 and the win
|
||||
is cache residency, not memory: 8192 is 1.01s, 32768 is 2.80s.
|
||||
* the chunk loop is embarrassingly parallel and numpy releases the GIL in
|
||||
both the matmul and the argmin, so a plain thread pool scales it:
|
||||
1.02 -> 0.31s on 8 threads. Partitioning by row cannot change an
|
||||
argmin, so the labels are identical to the serial ones -- asserted in
|
||||
tools/analysis/09_ratectl_drift.py by the fact that every container
|
||||
this encoder emits still hashes the same.
|
||||
"""
|
||||
Cn = (C ** 2).sum(1)
|
||||
CT = np.ascontiguousarray(C.T)
|
||||
out = np.empty(X.shape[0], dtype=np.int32)
|
||||
for i in range(0, X.shape[0], chunk):
|
||||
x = X[i:i + chunk]
|
||||
d = Cn[None, :] - 2.0 * (x @ C.T) # + |x|^2, constant per row
|
||||
out[i:i + chunk] = np.argmin(d, axis=1)
|
||||
n = X.shape[0]
|
||||
nt = _ASSIGN_THREADS if threads is None else threads
|
||||
if nt <= 1 or n < 4 * chunk:
|
||||
_assign_range(X, CT, Cn, out, 0, n, chunk)
|
||||
return out
|
||||
bnd = [(n * i) // nt for i in range(nt + 1)]
|
||||
with ThreadPoolExecutor(nt) as ex:
|
||||
list(ex.map(lambda ab: _assign_range(X, CT, Cn, out, ab[0], ab[1], chunk),
|
||||
zip(bnd[:-1], bnd[1:])))
|
||||
return out
|
||||
|
||||
|
||||
|
||||
@@ -215,24 +215,38 @@ def frame_ctx(m, f, prev, idx_bytes=None):
|
||||
idx_bytes=default_idx_bytes(m) if idx_bytes is None else idx_bytes)
|
||||
|
||||
|
||||
def decide(ctx, lam, mu=0.0):
|
||||
def decide(ctx, lam, mu=0.0, byte_clk=0.0):
|
||||
"""Lagrangian mode decision at one lam and one mu. Returns (mode, bytes).
|
||||
|
||||
Minimises `distortion + lam*bytes + mu*cycles` per block. `mu=0` is the
|
||||
byte-only decision every session before 8 made; the machine's binding
|
||||
budget is cycles, and bytes and cycles do not rank the modes the same way
|
||||
(V4 is 4x V1 in bytes, 1.49x in cycles; RAW is dearer than V4 in bytes and
|
||||
CHEAPER in cycles, so mu inverts that preference -- FINDINGS 28.8).
|
||||
Minimises `distortion + lam*bytes + mu*(decode cycles + byte_clk*bytes)`
|
||||
per block. `mu=0` is the byte-only decision every session before 8 made.
|
||||
|
||||
`byte_clk` is what a DELIVERED byte costs the 68000 in clocks of the same
|
||||
budget mu is bisected against -- ratectl.DISK_CLK_BYTE, 5.0 on the
|
||||
single-address row of FINDINGS 43.2. It defaults to 0, which reproduces
|
||||
sessions 8-14 exactly, and it is the last place in the encoder where a byte
|
||||
was still free: FINDINGS 43.6 put the disk debit in the rate controller's
|
||||
FIT TEST, but the decision underneath it still ranked modes as though the
|
||||
16 bytes of a RAW block cost nothing to deliver.
|
||||
|
||||
THAT INVERTS FINDINGS 28.8. RAW is 400.4 cycles against V4's 448.2, so
|
||||
with a free byte, raising mu buys cycles by moving V4 -> RAW. Priced, a RAW
|
||||
block costs `400.4 + 16c` and a V4 block `448.2 + 4c`, which cross at
|
||||
**c = 3.98 clocks/byte** -- and 43.1's floor argument (a 68000 bus cycle is
|
||||
four clocks and the SPC hands over one byte per cycle) says c >= 4 on any
|
||||
real machine. So on hardware mu's escape hatch was never there: it was
|
||||
spending 12 clocks of bus to save 47.8 of CPU.
|
||||
|
||||
Cheap by design: no painting, no image-sized work. A search calls this a
|
||||
dozen times per lam step and paints once."""
|
||||
ib = ctx["idx_bytes"]
|
||||
s = ctx["sym"]
|
||||
mc = mu * MODE_CYCLES
|
||||
lb = lam + mu * byte_clk # what one payload byte costs, both budgets
|
||||
cost = np.stack([ctx["eS"] + mc[0],
|
||||
s["e1"] + lam * (1.0 * ib) + mc[1],
|
||||
s["e4"] + lam * (4.0 * ib) + mc[2],
|
||||
np.full(ctx["nb"], lam * RAW_BYTES + mc[3])])
|
||||
s["e1"] + lb * (1.0 * ib) + mc[1],
|
||||
s["e4"] + lb * (4.0 * ib) + mc[2],
|
||||
np.full(ctx["nb"], lb * RAW_BYTES + mc[3])])
|
||||
mode = np.argmin(cost, axis=0).astype(np.uint8)
|
||||
return mode, frame_bytes(mode, ctx["nb"], ib)
|
||||
|
||||
|
||||
@@ -0,0 +1,219 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Build the README's stills and clips out of a real emulated run.
|
||||
|
||||
tools/bench/pace_run.sh ... # or the DLX_SNAP_EVERY=1 run below
|
||||
python3 tools/media/make_readme_media.py <container.dlx> [--snap tmp/snap_rec]
|
||||
|
||||
Everything this writes comes from PNGs MAME wrote while `src/player/stream.s`
|
||||
decoded out of a 256 KB ring on an emulated stock X68000. Nothing is redrawn by
|
||||
the reference decoder and nothing is upscaled with interpolation -- the pixels
|
||||
in `docs/img/` are the pixels that were on the emulated screen.
|
||||
|
||||
THE MAPPING IS ASSERTED, NOT ASSUMED. `stream.lua` snapshots at the frame tick,
|
||||
BEFORE that tick's frame is decoded, so snapshot n holds frame n-1 -- and that
|
||||
is an off-by-one waiting to put the wrong caption under a picture. This script
|
||||
finds the offset by comparing against `tools/encoder/dlx.py`'s reconstruction
|
||||
and refuses to write anything unless every frame matches exactly at one offset.
|
||||
A README that illustrates a pixel-exact decoder with an approximate picture
|
||||
would be a small lie about the one property the project keeps testing.
|
||||
|
||||
Writes:
|
||||
docs/img/decoded-frame.png one frame, 3x nearest
|
||||
docs/img/source-vs-decoded.png Blu-ray source | 68000 output, same frame
|
||||
docs/img/player.webm the 120-frame window, side by side, 12 fps
|
||||
docs/img/modes.webm the same window with the block-mode map
|
||||
"""
|
||||
import argparse, os, subprocess, sys, shutil
|
||||
sys.path.insert(0, "tools/encoder")
|
||||
import numpy as np
|
||||
from PIL import Image, ImageDraw
|
||||
from dlx import DLX
|
||||
|
||||
SNAP_H, SNAP_W = 512, 256 # MAME -snapview native, double-scanned
|
||||
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("container", nargs="?", default="tmp/rc_fr_singe_scsi_span.dlx")
|
||||
ap.add_argument("--snap", default="tmp/snap_rec")
|
||||
ap.add_argument("--src", default="tmp/fr_singe", help="the extracted frames")
|
||||
ap.add_argument("--out", default="docs/img")
|
||||
ap.add_argument("--still", type=int, default=96, help="which frame for the stills")
|
||||
ap.add_argument("--fps", type=float, default=12.0)
|
||||
a = ap.parse_args()
|
||||
if not shutil.which("ffmpeg"):
|
||||
sys.exit("ffmpeg not found -- needed for the webm")
|
||||
os.makedirs(a.out, exist_ok=True)
|
||||
|
||||
d = DLX(a.container)
|
||||
|
||||
# --- the reference reconstruction, in the palette the machine actually shows.
|
||||
# Same derivation verify_decode.py uses: the X68000 word is GGGGGRRRRRBBBBBI, so
|
||||
# a channel is 5 bits plus a shared intensity, and I is chosen per entry by
|
||||
# minimum squared error against the RGB888 the encoder emitted (FINDINGS 23.3).
|
||||
pal = d.pal.astype(int)
|
||||
p6 = lambda v: ((v << 2) | (v >> 4)) & 0xFF
|
||||
fl = pal >> 3
|
||||
render = lambda I: p6((fl << 1) | I[:, None])
|
||||
I = (((render(np.ones(256, int)) - pal) ** 2).sum(1)
|
||||
< ((render(np.zeros(256, int)) - pal) ** 2).sum(1)).astype(int)
|
||||
LUT = render(I).astype(np.uint8)
|
||||
|
||||
canvas = np.zeros((d.H, d.W), np.uint8)
|
||||
ref, modes = [], []
|
||||
for f in range(d.nframes):
|
||||
d.paint(canvas, f)
|
||||
ref.append(LUT[canvas].copy())
|
||||
modes.append(d.modes(f).reshape(d.nby, d.nbx).copy())
|
||||
|
||||
|
||||
def picture(png):
|
||||
"""The 256x192 picture out of one MAME native snapshot."""
|
||||
s = np.asarray(Image.open(png).convert("RGB"))
|
||||
if s.shape[:2] != (SNAP_H, SNAP_W):
|
||||
sys.exit(f"{png}: expected {SNAP_W}x{SNAP_H}, got {s.shape[1]}x{s.shape[0]}")
|
||||
g = s[0::2] # undo the double scan
|
||||
y = (g.shape[0] - d.H) // 2 # the picture is centred
|
||||
return g[y:y + d.H]
|
||||
|
||||
|
||||
snaps = sorted(f"{a.snap}/x68000/{n}" for n in os.listdir(f"{a.snap}/x68000")
|
||||
if n.endswith(".png"))
|
||||
if not snaps:
|
||||
sys.exit(f"no snapshots in {a.snap}/x68000 -- run stream.lua with "
|
||||
f"DLX_PACE=1 DLX_SNAP_EVERY=1")
|
||||
shots = [picture(p) for p in snaps]
|
||||
|
||||
# --- match every snapshot to a frame, and classify what does not match -----
|
||||
# The naive expectation is wrong in a way worth recording: stream.lua fires the
|
||||
# snapshot at the tick, BEFORE frame n is decoded, but MAME renders the screen
|
||||
# at the END of the machine frame -- by which time the 68000 has finished frame
|
||||
# n (it needs 69% of a 12 fps slot). So snapshot n IS frame n. That is a
|
||||
# statement about when a screen bitmap is captured, not about the decoder, and
|
||||
# it is the kind of thing to check rather than reason about.
|
||||
#
|
||||
# A handful of snapshots are TORN: the top of the picture is frame n and the
|
||||
# bottom still holds frame n-1, because the capture landed while the block loop
|
||||
# was partway down the screen. That is not a decoder fault and it is not an
|
||||
# artefact of the rig either -- decode.s writes straight to the displayed page
|
||||
# (FINDINGS 28.1, one display path, no flip), so a real player tears the same
|
||||
# way. They are KEPT, with the tear asserted: every differing pixel must equal
|
||||
# the previous frame, or this is something else and the script stops.
|
||||
frames, torn = {}, []
|
||||
snap0 = shots[0]
|
||||
for n in range(1, len(shots)):
|
||||
j = n
|
||||
if j >= d.nframes: # pace_run.sh's own end-of-run snapshot
|
||||
continue
|
||||
sh = shots[n]
|
||||
if np.array_equal(sh, ref[j]):
|
||||
frames[j] = sh
|
||||
continue
|
||||
diff = (sh != ref[j]).any(2)
|
||||
if not np.array_equal(sh[diff], ref[j - 1][diff]):
|
||||
sys.exit(f"snapshot {n} is neither frame {j} nor a tear against frame "
|
||||
f"{j-1}:\n {diff.sum():,} pixels differ and they do not come "
|
||||
f"from the previous frame.\nThat is a decoder fault or a "
|
||||
f"changed snapshot geometry, not something to align around.")
|
||||
rows = np.where(diff.any(1))[0]
|
||||
torn.append((j, int(rows.min()), int(rows.max())))
|
||||
frames[j] = sh
|
||||
|
||||
exact = len(frames) - len(torn)
|
||||
print(f"{len(snaps)} snapshots -> frames 1..{max(frames)} of {d.nframes}.")
|
||||
print(f" {exact} PIXEL-EXACT against tools/encoder/dlx.py")
|
||||
if torn:
|
||||
print(f" {len(torn)} torn by the capture, each verified to be frame n on "
|
||||
f"top of frame n-1:")
|
||||
for j, r0, r1 in torn:
|
||||
print(f" frame {j}: rows {r0}..{r1} of {d.H} still hold frame {j-1}")
|
||||
print(f" dropped snapshot 0 (frame 0 caught part-drawn, nothing behind it) "
|
||||
f"and the\n end-of-run duplicate.")
|
||||
|
||||
src = {}
|
||||
for j in frames:
|
||||
p = f"{a.src}/f{j+1:04d}.png"
|
||||
if os.path.exists(p):
|
||||
src[j] = np.asarray(Image.open(p).convert("RGB"))
|
||||
|
||||
Z = 2 # nearest-neighbour zoom
|
||||
BAR = 22 # caption strip height
|
||||
|
||||
|
||||
def up(img, z=Z):
|
||||
return np.repeat(np.repeat(img, z, 0), z, 1)
|
||||
|
||||
|
||||
def captioned(img, text, z=Z):
|
||||
w = img.shape[1] * z
|
||||
out = Image.new("RGB", (w, img.shape[0] * z + BAR), (16, 16, 18))
|
||||
out.paste(Image.fromarray(up(img, z)), (0, BAR))
|
||||
ImageDraw.Draw(out).text((6, 6), text, fill=(190, 190, 196))
|
||||
return out
|
||||
|
||||
|
||||
def side_by_side(j):
|
||||
left = captioned(src[j], "Blu-ray source, cropped 256x192")
|
||||
right = captioned(frames[j], "68000 output (emulated X68000, 256 colours)")
|
||||
out = Image.new("RGB", (left.width + right.width + 8, left.height), (16, 16, 18))
|
||||
out.paste(left, (0, 0)); out.paste(right, (left.width + 8, 0))
|
||||
return out
|
||||
|
||||
|
||||
# --- stills ----------------------------------------------------------------
|
||||
still = a.still if a.still in frames else sorted(frames)[len(frames) // 2]
|
||||
Image.fromarray(up(frames[still], 3)).save(f"{a.out}/decoded-frame.png")
|
||||
print(f" {a.out}/decoded-frame.png frame {still}, 3x nearest")
|
||||
if still in src:
|
||||
side_by_side(still).save(f"{a.out}/source-vs-decoded.png")
|
||||
print(f" {a.out}/source-vs-decoded.png frame {still}")
|
||||
|
||||
|
||||
# --- the block-mode map ----------------------------------------------------
|
||||
# The decoder's whole cost model is per mode, so the map is the picture the
|
||||
# FINDINGS tables are really about. Colours are the four modes, not a heat map.
|
||||
MODE_RGB = np.array([[16, 16, 18], # SKIP -- costs nothing, draws nothing
|
||||
[60, 130, 220], # V1 -- one index for a 4x4 block
|
||||
[235, 175, 60], # V4 -- four indices
|
||||
[225, 70, 70]], # RAW -- sixteen bytes verbatim
|
||||
np.uint8)
|
||||
NAMES = ("SKIP", "V1", "V4", "RAW")
|
||||
|
||||
|
||||
def mode_panel(j):
|
||||
m = MODE_RGB[modes[j]]
|
||||
m = np.repeat(np.repeat(m, 4, 0), 4, 1) # a block is 4x4 pixels
|
||||
return m[:d.H, :d.W]
|
||||
|
||||
|
||||
def mode_pair(j):
|
||||
left = captioned(frames[j], "68000 output")
|
||||
counts = np.bincount(modes[j].ravel(), minlength=4)
|
||||
lab = " ".join(f"{n} {100*c/counts.sum():.0f}%"
|
||||
for n, c in zip(NAMES, counts))
|
||||
right = captioned(mode_panel(j), "block modes: " + lab)
|
||||
out = Image.new("RGB", (left.width + right.width + 8, left.height), (16, 16, 18))
|
||||
out.paste(left, (0, 0)); out.paste(right, (left.width + 8, 0))
|
||||
return out
|
||||
|
||||
|
||||
# --- clips -----------------------------------------------------------------
|
||||
def webm(name, maker, keys):
|
||||
tmpd = f"tmp/_media_{name}"
|
||||
os.makedirs(tmpd, exist_ok=True)
|
||||
for n, j in enumerate(keys):
|
||||
maker(j).save(f"{tmpd}/{n:04d}.png")
|
||||
outp = f"{a.out}/{name}.webm"
|
||||
# VP9, near-lossless: this is 256x192 pixel art scaled by integers, and a
|
||||
# codec that smooths a 4x4 block boundary would be editorialising about the
|
||||
# one thing the picture is evidence of.
|
||||
cmd = ["ffmpeg", "-y", "-loglevel", "error", "-framerate", str(a.fps),
|
||||
"-i", f"{tmpd}/%04d.png", "-c:v", "libvpx-vp9", "-crf", "12",
|
||||
"-b:v", "0", "-pix_fmt", "yuv444p", "-row-mt", "1", outp]
|
||||
subprocess.run(cmd, check=True)
|
||||
shutil.rmtree(tmpd)
|
||||
print(f" {outp} {len(keys)} frames @ {a.fps:g} fps, "
|
||||
f"{os.path.getsize(outp)/1024:.0f} KB")
|
||||
|
||||
|
||||
keys = sorted(k for k in frames if k in src)
|
||||
webm("player", side_by_side, keys)
|
||||
webm("modes", mode_pair, sorted(frames))
|
||||
Reference in New Issue
Block a user