drwily/Dragon-s-Lair-X68k
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Import the scene graph, and find the worst gap between two branch points is zero

Browse Source 
ROADMAP G1, scheduled early because it is a measurement input, and it paid for
that immediately. 51.3 established that a ring's lookahead is accumulated out of
pipe - wire and that a seek spends all of it, so what a branch point costs is
set by the time since the last one. 55.5 rehearsed a seek on the machine and
said out loud that it could not ask the question, because nothing here knew
where the branch points are.

They are everywhere, and 5.4% of them are free of charge to the player and not
to us. Over 612 distinct transitions into a seek, taking the earliest instant
each input window opens: worst 0.000 s, p10 0.950, median 3.473, best 82.497.
33 open on the first frame of a clip the disc SEEKED to -- press right as
flaming_ropes.enter_room appears and you are in fall_to_death, two seeks with no
play between them. 51.2's slack rule can therefore be answered NO by the
content rather than by the buffer, and no amount of ring is a defence.

It does not break the design. A branch on an empty ring costs the 2-record
prefill, 149.7 ms at 488 KB/s, not the climb. What it removes is margin: at that
rate in a 256 KB ring, 76% of this game's branch points arrive before the ring
has refilled, and a 512 KB ring makes that 90%, because doubling the ceiling
does not touch the surplus. The ring is not the lever; the surplus is.

CORRECTION to FINDINGS 16: there is only one transcription. The SNES chapter
set says in its own README that it is derived from DirkSimple, so the planned
diff of two independent sources catches conversion errors only. Run anyway:
durations agree 388/505 within a frame, branch structure 470/505, and of the 35
differences 16 are renames and 18 of the other 19 are that port dropping the
arcade's diagonals. Zero transcription discrepancies, and none were findable.

Two constraints on the input layer come free: the arcade needs eight directions,
and the shortest input window is 98 ms against 54.4's 72.13/90.16 ms frame slot,
so input cannot be polled on the frame tick.

The coupling to outside source is contained to one file (USER DECISION).
tools/import/scenegraph.py is the only code here that knows those projects
exist -- their paths, table names, timing formulas, constants -- and it writes
DLXSCENE1, this project's own schema, into gitignored tmp/ with the sources'
licences inside it. tools/analysis/25_scene_graph.py reads only that. Nothing is
vendored and nothing outside-derived is committed. The split was made after the
measurement and the whole output was re-run byte for byte to show it moved no
number.

Both import gates are negative-tested: deleting one sequence upstream fails the
516/906 count, and closing the table early fails the constructor-end check,
which replaced one that was vacuous.

No 68000 code ran or changed; decode.bin is still 1,296 B at the same MD5.
check.sh gains an import stage that skips when there is no checkout. ALL GREEN
before and after.

Claude-Session: https://claude.ai/code/session_01194oWYW8DQXK1SZ2DnChW6
This commit is contained in:
prosolis
2026-08-24 22:11:04 -07:00
parent 2676f3b835
commit 00232bb22b
7 changed files with 1187 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+34 -1
View File
@@ -126,6 +126,28 @@ at 488 KB/s a one-deep request queue gives away **6.8% of the pipe and underruns
59 of 120 frames**, a two-deep one gives away 3.4% and underruns none — on a 59 of 120 frames**, a two-deep one gives away 3.4% and underruns none — on a
container whose whole surplus over the wire is 8.7% (FINDINGS 55). container whose whole surplus over the wire is 8.7% (FINDINGS 55).
**The scene graph is in, and the worst gap between two decision points is
zero.** `tools/import/scenegraph.py` imports the arcade scene graph — 40 scenes,
516 sequences, 906 input windows — and 5.4% of the game's 612 branch transitions
open an input window on the first frame of a clip the disc *seeked to*, so two
seeks can fall back to back with no play between them. A rule of the form "has
there been enough play since the last branch" can therefore be answered no by
the **content**, not by the buffer. It does not break the design: a branch on an
empty ring costs the 2-record prefill, **149.7 ms at 488 KB/s**, not the climb.
What it removes is margin — at that rate in a 256 KB ring, **76% of this game's
branch points arrive before the ring has refilled**, and a 512 KB ring makes it
90%, because doubling the ceiling does not touch `pipe - wire` (FINDINGS 56).
**Nothing outside-derived is committed here.** The scene graph is not
redistributable from this tree; it is regenerated from a reader's own clones
into gitignored `tmp/`, and `tools/import/scenegraph.py` is the single file in
the repo coupled to those projects — everything downstream reads `DLXSCENE1`,
this project's own schema, with the sources' attribution carried in it.
DirkSimple is zlib (Ryan C. Gordon); the SNES chapter set is MIT (Chad
Doebelin) and, by its own README, *derived* from DirkSimple rather than an
independent transcription, which struck a cross-check this project had planned
on for eight sessions.
**Current encode:** 496.7 KB/s at 29.19 dB, 1 frame of 120 over the 12fps **Current encode:** 496.7 KB/s at 29.19 dB, 1 frame of 120 over the 12fps
budget, and that one is frame 0, the intra frame, late on purpose. budget, and that one is frame 0, the intra frame, late on purpose.
@@ -133,7 +155,8 @@ budget, and that one is frame 0, the intra frame, late on purpose.
mounted) re-runs both display regression tests, the rate-control drift gate, the mounted) re-runs both display regression tests, the rate-control drift gate, the
display-path coherency counterexample, a 120-frame 68000 decode on two CPU display-path coherency counterexample, a 120-frame 68000 decode on two CPU
cores, the ring and paced-ring passes, the DMAC configuration gate and the cores, the ring and paced-ring passes, the DMAC configuration gate and the
load-time transforms on both cores, then prints `ALL GREEN`. load-time transforms on both cores, then imports and gates the scene graph
when a DirkSimple checkout is present, then prints `ALL GREEN`.
## Reproducing this ## Reproducing this
@@ -316,6 +339,16 @@ tools/bench/c68k/ headless px68k C68K harness, a SECOND emulator for every
which MAME cannot report. The Makefile's -no-pie and the which MAME cannot report. The Makefile's -no-pie and the
harness's MAP_32BIT arena are load-bearing: C68K truncates harness's MAP_32BIT arena are load-bearing: C68K truncates
host pointers to 32 bits. host pointers to 32 bits.
25 imports nothing itself: it reads the DLXSCENE1 scene
table and reports the worst gap between two decision points,
what the input layer has to survive, and what both cost in
51.3's accumulated slack across explicit rates.
tools/import/ the ONLY code in this tree coupled to somebody else's source.
scenegraph.py reads a DirkSimple checkout (and optionally the
SNES chapter XMLs) and writes tmp/scenegraph.json in this
project's own DLXSCENE1 schema, with the sources' licences and
attribution inside it. Nothing is vendored and the output is
gitignored derived data.
tools/media/ builds docs/img/ from a paced recording run tools/media/ builds docs/img/ from a paced recording run
tools/vasm/ vasm m68k assembler, binary plus source tarball tools/vasm/ vasm m68k assembler, binary plus source tarball
tools/encoder/ hybrid VQ encoder and DLX3 container writer. tools/encoder/ hybrid VQ encoder and DLX3 container writer.
docs/FINDINGS.md
+212
View File
@@ -5017,3 +5017,215 @@ and are exact, so 49.6's table is unaffected.
two-instruction test at the top of the pace wait that routes a self-filled two-instruction test at the top of the pace wait that routes a self-filled
run into a polling wait loop. The legacy wait loops are byte for byte the ones run into a polling wait loop. The legacy wait loops are byte for byte the ones
FINDINGS 51 measured and a host-filled run executes none of the new code. FINDINGS 51 measured and a host-filled run executes none of the new code.
---
# Findings — session 24 (2026-08-24)
## 56. The scene graph is in, and the worst gap between two decision points is zero (session 24)
ROADMAP G1, and it was scheduled early *because it is a measurement input*.
FINDINGS 51.3 established that a ring's lookahead is accumulated out of
`pipe - wire` and that a seek spends all of it, so what a branch point costs is
set by the rate and by **the time since the last branch**. 55.5 rehearsed a seek
on the machine and then said out loud that it could not ask the question that
matters, because nothing in this tree knew where the branch points *are*.
Now it does, in two files with a hard line between them (USER DECISION,
session 24):
- **`tools/import/scenegraph.py` is the only file in this tree that knows
anything about somebody else's source.** Their file layout, table names,
timing formulas and magic constants are wired into it and nowhere else. It
writes **`DLXSCENE1`**, this project's own schema — a clip's length, its exits
and the earliest instant each of them can fire — into gitignored `tmp/`.
- **`tools/analysis/25_scene_graph.py` reads only DLXSCENE1** and could not name
an outside project if it wanted to.
Nothing is vendored and nothing outside-derived is committed. That line is worth
drawing before the game-logic layer exists rather than after: when one of those
projects moves, exactly one file in this tree breaks, and when a scene table is
eventually committed rather than regenerated, there is one place the attribution
already lives. Restructuring changed no number in this section — the split was
made after the measurement and the whole output was re-run byte for byte.
### 56.1 What was imported, and what gates it
**40 scenes, 516 sequences, 906 input windows**, plus `scene_manager.rows` — the
13x3 scene order the arcade walks. **282 of the 516 sequences (54.7%) are
entered by a seek**; the rest play on from wherever the disc already is, which
is the distinction the whole finding rests on.
The source table is Lua, and there is no Lua interpreter on this box, so the
importer contains a **deliberately small Lua-subset parser**: table constructors,
literals, bare identifiers, the file's four timing helpers and `+`/`-` between
them. Anything else is a parse error rather than a silent skip. Three gates
stand behind that choice:
1. the four timing helpers are **matched against the text that defines them** —
`frame / 23.976`, the `- 6297.0` ROM offset, `noseek` returning `-1`,
`(seconds * 1000) + ms`. If upstream changes a formula, the tool stops
rather than keeps evaluating the old one.
2. the parse must reach **516 sequences and 906 input windows** exactly. A
parser that quietly dropped a branch would produce a *smaller* graph and a
*longer* worst gap, i.e. it would fail in the flattering direction.
3. `check.sh` runs the import and then the analysis, and **skips rather than
fails** when there is no checkout, like every other outside-this-repo input
in the tree.
### 56.2 CORRECTION to FINDINGS 16: there is only one transcription
> 16 cleared two permissively licensed transcriptions and planned to **diff them
> against each other to catch transcription errors**. That plan does not work.
The SNES project's own `data/events/README.md` states its 516 chapter XMLs are
"**derived from DirkSimple game data**". They are a *conversion* of the same
transcription, not a second one. The diff below is still worth running — it
catches conversion errors — but it **cannot** catch a transcription error,
because there is nothing independent to compare against. DirkSimple is the
single source, and its own provenance is the arcade ROM's data table.
That correction was cheap to make and it was one sentence of a README away from
never being made at all. It belongs with FINDINGS 42.1: a plausible source with
no provenance is folklore, and so is a plausible cross-check.
### 56.3 THE MEASUREMENT: the worst gap is zero, and 5.4% of branches are
Chaining play across non-seeking sequences and taking the **earliest** moment an
input window opens (the least play a clip can deliver before the branch it leads
to), over 612 distinct transitions into a seek, excluding attract mode:
| | seconds of play |
|---|---:|
| worst | **0.000** |
| p10 | 0.950 |
| p25 | 1.966 |
| median | 3.473 |
| p75 | 5.800 |
| p90 | 9.548 |
| best | 82.497 |
**33 of the 612 (5.4%) are zero**: an input window that opens at t=0 of a clip
the disc *seeked to*, so two seeks can fall back to back with no play between
them at all. `flaming_ropes.enter_room -> fall_to_death` is one — press right on
the frame the clip starts and the player dies immediately.
This is not an edge case to be designed around; it is the game. **A rule of the
form "has there been enough play since the last branch" — 51.2's slack rule,
`ring_may_seek` in `src/player/ring.i` — can be answered NO by the content,
not by the buffer**, and no amount of ring is a defence.
**203 of the 612 end the scene**, which in this design is also a container
change and needs 6,164 header bytes before frame 0 (53, 55.1). The worst
scene-change gap is **0.541 s** and the median is 2.501 s.
### 56.4 The median branch point arrives before the ring has refilled
51.3's climb, against the game's own gap distribution. Gate container
(`rc_fr_singe_scsi_span.dlx`, mean record 36.5 KB, wire 446.1 KB/s); rates are
explicit arguments and every one of them is a sensitivity, not a claim:
| ring KB | pipe KB/s | ceiling | climb s | branch points under the climb |
|---:|---:|---:|---:|---:|
| 256 | 451.4 | 3 | 20.83 | 601/612 (98%) |
| 256 | 488.0 | 7 | 6.11 | 468/612 (76%) |
| 256 | 513.2 | 7 | 3.81 | 370/612 (60%) |
| 256 | 600.0 | 7 | 1.66 | 129/612 (21%) |
| 512 | 488.0 | 11 | 9.60 | 551/612 (90%) |
| 512 | 513.2 | 14 | 7.63 | 512/612 (84%) |
| 512 | 600.0 | 14 | 3.32 | 301/612 (49%) |
Two things fall out, and the second is the one that costs something.
1. **At every rate this tree has considered, most branch points arrive with less
lookahead than the one before them.** At 488 KB/s in a 256 KB ring — the
configuration `check.sh` gates — that is 76%.
2. **A bigger ring makes this metric worse, and now content says so too.** 51.3
derived it from the surplus alone; here the same rate goes from 76% under the
climb at 256 KB to 90% at 512 KB, because doubling the ring doubles the
ceiling without touching `pipe - wire`. **The ring is not the lever. The
surplus is.**
### 56.5 What a branch actually costs when the gap bought nothing
The climb is what a player needs to *tolerate* the next branch. What it *pays*
at one is the prefill, because the ring is empty after a seek and 55.4's shipped
policy releases the decoder at 2 records:
| pipe KB/s | 2-record prefill | as a scene change (+6,164 B) |
|---:|---:|---:|
| 451.4 | 161.8 ms (1.94 frame slots) | 175.2 ms (2.10) |
| 488.0 | 149.7 ms (1.80) | 162.0 ms (1.94) |
So a zero-play branch is **not** a failure: it costs about two frame slots of
black, every time. What it removes is margin. A player that branches at p10
(0.950 s) has spent its whole lookahead and rebuilt almost none of it, and the
next slow record has nothing behind it. **The finding is not "this breaks", it
is "this design runs permanently at minimum lookahead, and the arcade content is
what puts it there."** 22_scene_load.py prices the scene-change case properly,
clocks included; the mechanical seek is still unmodelled (B1) and is charged on
top of all of it.
### 56.6 What the cross-check IS worth, now that it is not a cross-check
Run anyway, against 518 SNES chapter XMLs, with the 40 scene abbreviations
matched to DirkSimple scenes **from the data** — the abbreviation's letters must
be a subsequence of the scene name, ranked by sequence-name overlap, with a
reversed-scene rule for the thirteen mirrored scenes. (Overlap alone mapped
`snkr` to `black_knight`, because two scenes full of `seqN` names look alike.)
- **Start times do not compare at all.** The SNES XMLs are on their own
extraction's timeline; DirkSimple is on arcade ROM frames minus 6,297 ms. The
difference is not a constant and **does not even keep its sign** (spread
-8.0 s .. +6.4 s). Reported as a spread rather than as a check, because a
check it is not.
- **Durations do compare** — they are offset-invariant. 388/505 agree within one
laserdisc frame (76.8%), median difference 0 ms.
- **Branch structure compares**: 470/505 chapters (93.1%) carry the identical
set of (input -> target) edges. Of the 35 that differ, **16 are renames**
(`captured_by_ghouls` -> `ghoul_capture`) and **18 of the remaining 19 are the
SNES conversion dropping the arcade's diagonals**. The 19th is
`intr_castle_exterior`, where the SNES *added* a skip.
**Zero transcription discrepancies were found, and none could have been.** What
the diff produced is one useful fact about our own input layer, below.
### 56.7 Two constraints on the input layer, from the same import
1. **The arcade uses eight directions plus action and start.** Counted over the
906 windows: left 233, right 217, up 209, down 152, action 72, **upleft 13,
upright 4, downleft 3, downright 1**, start 2. The diagonals are 21 windows
out of 906 — rare enough to be dropped by a port that had to (the SNES one
did) and **not** droppable by one aiming at the arcade.
2. **The shortest input window is 98 ms.** Median 950 ms (11.4 frame slots), p10
393 ms, but the floor is 98 ms — and by 54.4 a 12 fps frame slot is 72.13 ms
or 90.16 ms, never 83.33. **A 98 ms window is one or two frames wide.**
Polling input on the frame tick is therefore marginal by construction: the
input layer has to run off something faster than the frame clock, and
`src/player/clock.i` already owns an MFP interrupt at 181.35 clocks a
V-DISP (54.2) that is 8.6x faster and costs 0.1% of the budget.
### 56.8 What this does NOT establish
1. **No rate here is measured.** Every column is a sensitivity across explicit
rates (FINDINGS 50). B1 is still open and the mechanical seek still has no
figure at all.
2. **The gap model is a lower bound by construction.** It takes the earliest
instant an input window opens, so it is what an *expert* player can force,
not what a typical one produces. That is the right bound for a buffer
design and the wrong one for describing play.
3. **It is the arcade's graph, not this port's.** The 612 transitions assume
the port reproduces every branch. Nothing has been mapped onto our 224
Blu-ray streams yet — the SNES project's 516 chapters are finer-grained than
our streams, and that mapping is still C1's problem.
4. **Nothing ran on the 68000 this session.** This is host-side analysis of an
imported table. `decode.s`, `stream.s`, `ring.i`, `clock.i` and `load.i` are
untouched, `decode.bin` is still 1,296 B at the same MD5, and the green light
was ALL GREEN before and after.
5. **The scene graph is not vendored, and the coupling is contained.** Neither
repo ships here and neither is redistributable from this tree; both are
permissive (DirkSimple zlib, Ryan C. Gordon; SNES project MIT, Chad
Doebelin) and both are cloned by the reader. The `sources` block of every
generated table carries the attribution. **`tmp/scenegraph.json` is
generated, gitignored and derived data**: committing it, or any table built
from it, is redistribution and the attribution has to travel with it.
docs/ROADMAP.md
+25 -12
View File
@@ -3,6 +3,8 @@
Written end of session 19 (2026-08-24), against a tree that is ALL GREEN. Written end of session 19 (2026-08-24), against a tree that is ALL GREEN.
Amended end of session 21: P1 done, P2 half done (FINDINGS 53). Amended end of session 21: P1 done, P2 half done (FINDINGS 53).
Amended end of session 22: P3 done (FINDINGS 54). Amended end of session 22: P3 done (FINDINGS 54).
Amended end of session 23: P5 done (FINDINGS 55).
Amended end of session 24: G1 done (FINDINGS 56).
**THE COMPLETION TARGET IS M3, THE VERTICAL SLICE** (USER DECISION): one scene **THE COMPLETION TARGET IS M3, THE VERTICAL SLICE** (USER DECISION): one scene
tree — a decision point, two outcomes, a death clip — with audio, streaming from tree — a decision point, two outcomes, a death clip — with audio, streaming from
@@ -256,18 +258,29 @@ interaction to price next, and it is E2's question with a second consumer in it.
**E6. Container v2** — audio interleave, per-record index, scene table. Depends **E6. Container v2** — audio interleave, per-record index, scene table. Depends
on P6's answer and on P5's index. on P6's answer and on P5's index.
**G1. Import the scene graph — early, because it is a measurement input.** ~~**G1. Import the scene graph — early, because it is a measurement input.**~~
SNES project `data/events/` (MIT, cleared) diffed against DirkSimple (zlib), **DONE, session 24 — FINDINGS 56.** It was pulled ahead for exactly the reason
which transcribed the same data independently, to catch transcription errors given, and it paid: **the worst gap between two consecutive decision points is
before anything reaches 68000 tables. **Neither is on this box** — both need zero**, and 5.4% of the game's 612 branch transitions are. Two seeks can fall
fetching. back to back with no play between them, so 51.2's slack rule can be answered NO
by the content rather than by the buffer.
The reason to pull this ahead of the game logic that consumes it: 51.3 says It does not break the design — a branch on an empty ring costs the prefill
4.83 s of play to refill a 256 KB ring at 488 KB/s, and Dragon's Lair's decision (149.7 ms, 1.80 frame slots at 488 KB/s), not the climb — but it removes the
points are seconds apart. **Nothing in this tree can currently say what the worst margin: at 488 KB/s in a 256 KB ring, **76% of this game's branch points arrive
gap between consecutive decision points is** — only the scene table knows, and before the ring has refilled**, and a 512 KB ring makes that 90%. **The ring is
until it is imported, whether this design survives a back-to-back branch is an not the lever; the surplus is.**
open question nobody is able to ask.
Two constraints on the input layer came with it: the arcade needs **eight
directions**, and the shortest input window is **98 ms** against a 72.13/90.16 ms
frame slot, so input cannot be polled on the frame tick (56.7).
**The cross-check plan was wrong and is struck.** The SNES chapters are
*derived* from DirkSimple, by their own README, so there is one transcription and
not two; the diff catches conversion errors only (56.2). **Nothing is vendored:**
`tools/import/scenegraph.py` is the one file coupled to those projects and it
writes this project's own `DLXSCENE1` schema into gitignored `tmp/`
(USER DECISION, session 24).
--- ---
@@ -304,7 +317,7 @@ B3 DTYP ──────┴─> P4 transport ─┐
P1 P2(half) P3 P5 P7 ───────────┘ ^ P1 P2(half) P3 P5 P7 ───────────┘ ^
│ │
P6 (bus cost DONE, 52) ──────────────────┤ P6 (bus cost DONE, 52) ──────────────────┤
G1 scene graph (fetch, do early) ─────────┘ G1 scene graph (DONE, 56) ───────────────┘
B2 blanking ─> (page 1; do not pre-build on it) B2 blanking ─> (page 1; do not pre-build on it)
``` ```
docs/STATUS.md
+91
View File
@@ -1,3 +1,94 @@
# Status & next-session handoff — end of session 24 (2026-08-24)
## Session 24: the scene graph is in, and the worst gap between two decision points is zero
**Green light first and last: `./tools/bench/check.sh` was ALL GREEN before any
of this and ALL GREEN after**, plus a new import stage.
**ROADMAP G1 is DONE. FINDINGS 56.** G1 was scheduled early because it is a
*measurement input*, and it paid for that immediately.
**1. THE ANSWER: the worst gap is zero.** Over 612 distinct transitions into a
seek, taking the earliest instant each input window opens and chaining play
across sequences the disc plays through without seeking:
| | worst | p10 | median | p75 | p90 | best |
|---|---:|---:|---:|---:|---:|---:|
| seconds of play between two seeks | **0.000** | 0.950 | 3.473 | 5.800 | 9.548 | 82.497 |
**33 of the 612 (5.4%) are zero** — an input window that opens at t=0 of a clip
the disc seeked to, so two seeks can fall back to back with no play at all
(`flaming_ropes.enter_room -> fall_to_death`: press right on the first frame and
you die). **51.2's slack rule can be answered NO by the content, not by the
buffer**, and no amount of ring is a defence. 56.3.
**2. Most branch points arrive before the ring has refilled, at every rate this
tree has considered.** 51.3's climb against the game's own gaps, gate container:
| ring KB | pipe KB/s | ceiling | climb s | branch points under the climb |
|---:|---:|---:|---:|---:|
| 256 | 451.4 | 3 | 20.83 | 601/612 (98%) |
| 256 | 488.0 | 7 | 6.11 | **468/612 (76%)** |
| 256 | 513.2 | 7 | 3.81 | 370/612 (60%) |
| 256 | 600.0 | 7 | 1.66 | 129/612 (21%) |
| 512 | 488.0 | 11 | 9.60 | **551/612 (90%)** |
**A bigger ring makes this worse and now content says so too**: same rate, 76%
at 256 KB and 90% at 512 KB, because doubling the ceiling does not touch
`pipe - wire`. **The ring is not the lever; the surplus is.** 56.4.
**3. It does not break — it removes margin.** A branch on an empty ring costs
the prefill, not the climb: **149.7 ms (1.80 frame slots) at 488 KB/s**, 162.0 ms
if it is a scene change carrying the 6,164-byte header. So the finding is not
"this fails", it is **"this design runs permanently at minimum lookahead, and
the arcade content is what puts it there"**. The mechanical seek is still
unmodelled (B1) and is charged on top. 56.5.
**4. CORRECTION to FINDINGS 16: there is only one transcription.** 16 cleared
two permissively licensed sources and planned to diff them "to catch
transcription errors". The SNES project's own `data/events/README.md` says its
chapters are "derived from DirkSimple game data" — a second *copy*, not a second
transcription. The diff runs anyway and catches conversion errors: durations
agree 388/505 within one frame, branch structure 470/505, and of the 35
differences **16 are renames and 18 of the other 19 are the SNES conversion
dropping the arcade's diagonals**. Zero transcription discrepancies were found
and none could have been. 56.2, 56.6.
**5. Two constraints on the input layer, free with the import.** The arcade uses
**eight directions plus action and start** (diagonals are 21 of 906 windows —
droppable by a port that must, not by one aiming at the arcade), and the
**shortest input window is 98 ms** against a frame slot of 72.13 or 90.16 ms
(54.4). **Input cannot be polled on the frame tick**; `clock.i`'s V-DISP
interrupt already runs 8.6x faster at 0.1% of the budget. 56.7.
**6. The coupling to outside source is contained to one file (USER DECISION).**
`tools/import/scenegraph.py` is the only file in this tree that knows those
projects exist — their paths, table names, timing formulas, constants — and it
writes **`DLXSCENE1`**, our own schema, into gitignored `tmp/`.
`tools/analysis/25_scene_graph.py` reads only that. Nothing is vendored, nothing
outside-derived is committed, and the generated table carries its own `sources`
attribution block. The split was made after the measurement and changed no
number in it.
**New in the tree:** `tools/import/` (new directory, one file),
`tools/analysis/25_scene_graph.py`, and a `check.sh` stage that imports, gates
on 516 sequences / 906 input windows, and runs the analysis — skipped when there
is no checkout, like the px68k and IPL ROM stages.
**No 68000 code ran and none changed.** `decode.bin` is still 1,296 B at the
same MD5.
**Next:** **P4** (drive the MB89352, settle `W`) still decides the project and
still needs hardware or a MAME that models the SPC. What 56 changes about it:
the transport now has a *content* requirement as well as a rate one — it has to
survive a branch with an empty ring at zero notice, 5.4% of the time. **P2's
remaining half** (reserve index 0 as black) is unchanged and still bundled with
the two other re-encode-class questions from 55: the delivered-rate rate point
and 54.4's short slot. All three are still one re-encode plus one
re-measurement, and still want deciding together.
---
# Status & next-session handoff — end of session 23 (2026-08-24) # Status & next-session handoff — end of session 23 (2026-08-24)
## Session 23: the 68000 fills its own ring, and the player's request loop turns out to cost more than the medium does ## Session 23: the 68000 fills its own ring, and the player's request loop turns out to cost more than the medium does
tools/analysis/25_scene_graph.py
+363
View File
@@ -0,0 +1,363 @@
"""The worst gap between two decision points, out of the scene graph (G1).
FINDINGS 51.3 measured that a ring's lookahead is ACCUMULATED out of
`pipe - wire` and that a seek spends all of it, so what a branch point costs is
set by the rate and by the time since the last branch. 55.5 rehearsed a seek on
the machine and could not ask the question that matters, because nothing in this
tree knew where the branch points ARE:
what is the WORST gap, in seconds of play, between two consecutive
decision points, and does the refill climb survive it?
Only the arcade scene graph knows. This answers it in the currency 51.3
established.
python3 tools/import/scenegraph.py # writes tmp/scenegraph.json
python3 tools/analysis/25_scene_graph.py --kbps R [R ...] [--ring KB [KB ...]]
`--kbps` is REQUIRED and takes no default, for the reason FINDINGS 50 gives.
THIS FILE KNOWS NOTHING ABOUT WHERE THE TABLE CAME FROM, deliberately. It reads
`DLXSCENE1`, which is this project's own schema; `tools/import/scenegraph.py` is
the single file in the tree that knows anything about the outside projects the
table is built from, and it carries their attribution. Nothing is vendored.
"""
import sys, os, json, argparse, importlib.util
sys.path.insert(0, "tools/encoder")
TABLE = os.environ.get("DLX_SCENEGRAPH", "tmp/scenegraph.json")
LD_FPS = 23.976 # the medium's frame rate; one frame is the comparison floor
# ------------------------------------------------------------------- graph
class Node:
"""One (scene, sequence): a clip, its exits, and whether entering it seeks."""
def __init__(self, scene, name, seq):
self.scene, self.name, self.seq = scene, name, seq
self.start = seq["start_ms"] # ms, or -1 for no seek
self.seeks = self.start >= 0
self.timeout_ms = seq["timeout_ms"]
self.exits = [(seq["timeout_ms"], "timeout", seq["timeout_next"])]
for a in seq["actions"]:
# The player may press as early as `from_ms`, so that is the least
# play this clip can deliver before the branch it leads to.
self.exits.append((a["from_ms"], "action", a["next"]))
@property
def key(self):
return f"{self.scene}.{self.name}"
def build_graph(scenes):
nodes = {}
for scene, seqs in scenes.items():
for name, seq in seqs.items():
n = Node(scene, name, seq)
nodes[n.key] = n
return nodes
def worst_gap(nodes):
"""Least play time, in ms, between one seek and the next.
A seek is entering a sequence whose start is >= 0; a negative start means
the disc keeps playing, so play ACCUMULATES across such sequences and the
gap is a shortest path over them. Bellman-Ford rather than Dijkstra
because a zero-length timeout is common (`start_alive` chains) and the
graph has cycles; all weights are non-negative so it terminates.
A null exit ends the scene: the player moves to another scene entirely,
which is a seek AND a container change (FINDINGS 53), so it counts as a
seek and is flagged.
"""
INF = float("inf")
dist = {k: (0.0 if n.seeks else INF) for k, n in nodes.items()}
for _ in range(len(nodes) + 1):
changed = False
for n in nodes.values():
if dist[n.key] == INF:
continue
for elapsed, kind, tgt in n.exits:
if tgt is None:
continue
tk = f"{n.scene}.{tgt}"
if tk not in nodes:
continue
d = dist[n.key] + max(0.0, float(elapsed))
if not nodes[tk].seeks and d < dist[tk] - 1e-9:
dist[tk], changed = d, True
if not changed:
break
best = {}
for n in nodes.values():
if dist[n.key] == INF:
continue
for elapsed, kind, tgt in n.exits:
tk = f"{n.scene}.{tgt}" if tgt is not None else None
ends_scene = tgt is None
if ends_scene or (tk in nodes and nodes[tk].seeks):
g = dist[n.key] + max(0.0, float(elapsed))
# One entry per (source, destination): several input windows can
# lead to the same clip and only the earliest of them binds.
k = (n.key, tgt)
if k not in best or g < best[k][0]:
best[k] = (g, n.key, tgt if tgt else "<end of scene>",
kind, ends_scene, n.scene)
return sorted(best.values()), dist
# ------------------------------------------------------ 51.3's currency
def slack_model(gap_s, kbps, wire_kbps, mean_rec_b):
"""Records of lookahead accrued in `gap_s` seconds of play at `kbps`, and
the seconds one record of lookahead costs.
This is 51.3's surplus model and nothing more: slack accrues at
(pipe - wire) bytes per second. The paced rig is the measurement; this
says whether the gap is even in the right order of magnitude.
"""
surplus = (kbps - wire_kbps) * 1024.0
if surplus <= 0:
return 0.0, float("inf")
return surplus * gap_s / mean_rec_b, mean_rec_b / surplus
def main():
ap = argparse.ArgumentParser()
ap.add_argument("--kbps", type=float, nargs="+", required=True,
help="delivered pipe rates, KB/s. REQUIRED, no default "
"(FINDINGS 50).")
ap.add_argument("--table", default=TABLE, help="DLXSCENE1 scene table")
ap.add_argument("--container", default="tmp/rc_fr_singe_scsi_span.dlx",
help="container the wire demand and mean record come from")
ap.add_argument("--ring", type=float, nargs="+", default=[256, 512])
ap.add_argument("--top", type=int, default=12)
a = ap.parse_args()
if not os.path.exists(a.table):
print(f"no scene table at {a.table} -- run:\n"
f" python3 tools/import/scenegraph.py")
return 2
doc = json.load(open(a.table))
if doc.get("format") != "DLXSCENE1":
print(f"{a.table}: not a DLXSCENE1 table")
return 2
nodes = build_graph(doc["scenes"])
c = doc["counts"]
nseek = sum(1 for n in nodes.values() if n.seeks)
print(f"=== the scene graph ({a.table})")
for s in doc["sources"]:
print(f" from {s['name']} ({s['licence']}, {s['holder']}): {s['role']}")
print(f" scenes {c['scenes']}, sequences {c['sequences']}, "
f"input windows {c['windows']}")
print(f" sequences entered by a SEEK: {nseek}/{c['sequences']} "
f"({100*nseek/c['sequences']:.1f}%); the rest play on from where the "
f"disc is")
print(f" scene order: {len(doc['rows'])} rows x {len(doc['rows'][0])}")
# Gates. A parser that quietly dropped a branch would produce a SMALLER
# graph and a LONGER worst gap -- it would fail in the flattering direction.
assert c["sequences"] == 516, f"expected 516 sequences, got {c['sequences']}"
assert c["windows"] == 906, f"expected 906 input windows, got {c['windows']}"
# ---- the measurement
gaps, dist = worst_gap(nodes)
play = [g for g in gaps if g[5] != "attract_mode"]
zero = [g for g in play if g[0] <= 1e-9]
print()
print("=== the worst gap between two consecutive decision points")
print(" A 'gap' is the LEAST play time the disc delivers between one seek")
print(" and the next: the earliest an input window opens, chained across")
print(" sequences the disc plays through without seeking. One entry per")
print(" (source, destination); attract mode is excluded and reported")
print(" separately, because nothing branches there under a 12 fps budget.")
print(f" {'gap s':>7} from -> to")
for g, src, tgt, kind, ends, scene in play[:a.top]:
print(f" {g/1000:>7.3f} {src} -{kind}-> {tgt}"
+ (" [SCENE CHANGE]" if ends else ""))
sc_gaps = [g for g in play if g[4]]
print(f" ... {len(play)} distinct transitions into a seek "
f"({len(gaps)-len(play)} more in attract mode)")
worst = play[0][0] / 1000.0
med = play[len(play) // 2][0] / 1000.0
print(f" WORST {worst:.3f} s, median {med:.3f} s, "
f"best {play[-1][0]/1000:.3f} s")
print(f" SCENE CHANGES specifically ({len(sc_gaps)} of them, and each also")
print(f" needs a header before its frame 0, FINDINGS 53/55.1): worst "
f"{sc_gaps[0][0]/1000:.3f} s, median "
f"{sc_gaps[len(sc_gaps)//2][0]/1000:.3f} s")
print(f" ZERO-PLAY BRANCHES: {len(zero)} of {len(play)} "
f"({100*len(zero)/len(play):.1f}%) open an input window at t=0 of a")
print(" clip the disc SEEKED to, so two seeks can fall back to back with no")
print(" play between them at all. A rule of the form 'has there been")
print(" enough play since the last branch' (51.2's ring_may_seek) can be")
print(" answered NO by the content, not by the buffer.")
# ---- what the input layer has to survive, from the same table
inputs, windows = {}, []
for n in nodes.values():
for x in n.seq["actions"]:
inputs[x["input"]] = inputs.get(x["input"], 0) + 1
windows.append(x["to_ms"] - x["from_ms"])
windows.sort()
print()
print("=== what the input layer has to survive")
print(" " + ", ".join(f"{k} {v}" for k, v in
sorted(inputs.items(), key=lambda x: -x[1])))
diag = sum(v for k, v in inputs.items()
if k in ("upleft", "upright", "downleft", "downright"))
print(f" diagonals are {diag} windows of {len(windows)}: rare enough for a "
f"port to drop\n and not droppable by one aiming at the arcade")
print(f" window length: shortest {windows[0]:.0f} ms "
f"({windows[0]/(1000/12):.2f} frame slots at 12 fps), p10 "
f"{windows[len(windows)//10]:.0f} ms, median "
f"{windows[len(windows)//2]:.0f} ms")
print(" the floor is one to two frames wide (54.4: a slot is 72.13 or")
print(" 90.16 ms, never 83.33), so input cannot be polled on the frame tick")
# ---- 51.3's currency
if os.path.exists(a.container):
spec = importlib.util.spec_from_file_location(
"seek_slack", "tools/analysis/20_seek_slack.py")
m = importlib.util.module_from_spec(spec)
spec.loader.exec_module(m)
import ratectl as RC
d, rec = m.records(a.container)
mean_rec = float(rec.mean())
wire = mean_rec * 12 / 1024 + RC.AUDIO_KBPS
gs = [g[0] / 1000.0 for g in play]
print()
print(f"=== what that gap buys, at explicit rates "
f"({os.path.basename(a.container)}: mean record "
f"{mean_rec/1024:.1f} KB, wire {wire:.1f} KB/s)")
print(f" {'ring KB':>8} {'pipe':>7} {'ceiling':>8} {'climb s':>8} "
f"{'median gap':>11} {'accrued':>8} {'under climb':>12}")
for ring_kb in a.ring:
ring = int(ring_kb * 1024)
for kbps in a.kbps:
fill = (kbps - RC.AUDIO_KBPS) * 1024 / 12
lo, hi, ring_ref, rate_ref = m.paced_sim(rec, ring, fill)
ceiling = int(hi.max())
accrued, per_rec = slack_model(med, kbps, wire, mean_rec)
climb = ceiling * per_rec
under = sum(1 for g in gs if g < climb)
print(f" {ring_kb:>8.0f} {kbps:>7.1f} {ceiling:>8} "
f"{climb:>8.2f} {med:>11.3f} {accrued:>8.2f} "
f"{f'{under}/{len(gs)}':>12} {100*under/len(gs):.0f}%")
# What a branch costs when the gap before it bought nothing. The ring
# is empty after a seek and the decoder is released at the prefill depth
# (55.4's shipped policy is 2 records), so this is the stall the player
# eats every time -- not the climb to the ceiling, which is what it
# needs in order to TOLERATE the next one. A scene change additionally
# needs its header before frame 0; 22_scene_load.py prices that case
# properly, clocks included.
PREFILL_REC, HDR_B = 2, 6164
print()
print(f" A branch taken on an empty ring, at the shipped prefill of "
f"{PREFILL_REC} records:")
for kbps in a.kbps:
b = PREFILL_REC * mean_rec
ms = b / (kbps * 1024) * 1000
hms = (b + HDR_B) / (kbps * 1024) * 1000
print(f" {kbps:>7.1f} KB/s: {ms:>7.1f} ms "
f"({ms/(1000/12):.2f} frame slots), and {hms:>7.1f} ms "
f"({hms/(1000/12):.2f}) if it is a scene change carrying "
f"{HDR_B:,} header bytes")
print()
print(" 'climb s' is 51.3's: seconds of play to refill from empty to")
print(" the ceiling. 'under climb' is how many of this game's own")
print(" branch points arrive sooner than that, i.e. are reached with")
print(" LESS lookahead than the one before them. The worst gap is")
print(f" {worst:.3f} s and buys nothing at any rate in this table.")
else:
print(f"\n{a.container}: MISSING -- rate half skipped")
# ---- the cross-check, and what it is worth
print()
print("=== the second table, and why it is not a second transcription")
cross, meta = doc.get("crosscheck"), doc.get("crosscheck_meta")
if not cross:
print(" none in this table -- the import ran without it")
return 0
print(f" {meta['chapters']} chapters, {meta['scenes_mapped']} scenes")
print(f" PROVENANCE: {meta['provenance']}")
print(" FINDINGS 16 planned to diff two INDEPENDENT transcriptions to")
print(" catch transcription errors. There is only one transcription.")
print(" This diff catches CONVERSION errors and nothing more.")
FRAME_MS = 1000.0 / LD_FPS
offs, durs, same, diff, missing = [], [], 0, 0, 0
renames, inputs_differ, examples = 0, [], []
for scene, seqs in sorted(cross.items()):
for seq, ch in sorted(seqs.items()):
n = nodes.get(f"{scene}.{seq}")
if n is None:
missing += 1
continue
if n.seeks:
offs.append(ch["start_ms"] - n.start)
durs.append(((ch["end_ms"] - ch["start_ms"]) - n.timeout_ms,
ch["chapter"]))
ce = sorted((x["input"], x["next"]) for x in ch["actions"])
de = sorted((x["input"], x["next"]) for x in n.seq["actions"])
if ce == de:
same += 1
continue
diff += 1
ci = sorted(i for i, _ in ce)
di = sorted(i for i, _ in de)
if ci != di:
inputs_differ.append((ch["chapter"], ci, di))
else:
renames += 1
if len(examples) < 3:
examples.append((ch["chapter"], ce, de))
if offs:
offs.sort()
print(f" START TIMES are on different timelines and do not compare: "
f"{len(offs)} seeking chapters,")
print(f" offset spread {min(offs)/1000:,.1f} s .. "
f"{max(offs)/1000:,.1f} s, median {offs[len(offs)//2]/1000:,.1f} s"
f" -- not a constant, and not even one sign.")
if durs:
dd = sorted(x for x, _ in durs)
agree = sum(1 for x in dd if abs(x) <= FRAME_MS)
print(f" DURATIONS compare (offset-invariant): {agree}/{len(dd)} "
f"within one frame of the medium ({100*agree/len(dd):.1f}%), "
f"median {dd[len(dd)//2]:,.0f} ms")
for x, ch in sorted(durs, key=lambda x: -abs(x[0]))[:3]:
print(f" widest {ch}: {x/1000:+.3f} s")
if same + diff:
print(f" BRANCH STRUCTURE compares: {same}/{same+diff} chapters have "
f"the identical set of (input -> target) edges "
f"({100*same/(same+diff):.1f}%)")
if diff:
print(f" {renames} of the {diff} differ only in what a target "
f"sequence is NAMED")
DIAG = ("upleft", "upright", "downleft", "downright")
lost = [ch for ch, ci, di in inputs_differ
if set(di) - set(ci) and all(x in DIAG for x in set(di) - set(ci))]
print(f" {len(inputs_differ)} differ in the INPUT SET, and "
f"{len(lost)} of those are the other table dropping the arcade's")
print(" DIAGONALS: a controller decision, not a transcription "
"difference.")
for ch, ci, di in inputs_differ:
if ch not in lost:
print(f" the remaining one: {ch}, {ci} vs {di}")
for ch, ce, de in examples:
print(f" e.g. {ch}\n cross {ce}\n graph {de}")
if missing:
print(f" {missing} chapters have no sequence in the graph "
f"(the other project added them)")
return 0
if __name__ == "__main__":
sys.exit(main())
tools/bench/check.sh
+26
View File
@@ -406,4 +406,30 @@ grep -q "^OK" tmp/ringseek_check.log || {
echo "FAIL: the decode after the seek was not pixel-exact." echo "FAIL: the decode after the seek was not pixel-exact."
tail -4 tmp/ringseek_check.log; exit 1; } tail -4 tmp/ringseek_check.log; exit 1; }
echo "--- session 24: the scene graph, and the gap between branch points (FINDINGS 56) ---"
# The arcade scene graph is not in this repo and is not redistributable from
# here. tools/import/scenegraph.py is the ONE file in the tree that knows the
# outside projects exist; it writes tmp/scenegraph.json in this project's own
# DLXSCENE1 schema and everything downstream reads only that.
# What is gated is the IMPORT, not the numbers: 516 sequences and 906 input
# windows, and the four timing helpers still being the formulas the importer
# evaluates. Skipped when the checkout is absent.
DIRKSIMPLE=${DLX_DIRKSIMPLE:-tmp/scenegraph/DirkSimple}
if [ -f "$DIRKSIMPLE/data/games/lair/game.lua" ]; then
DLX_DIRKSIMPLE="$DIRKSIMPLE" python3 tools/import/scenegraph.py \
-o tmp/scenegraph.json > tmp/scenegraph_import.log 2>&1 \
|| { cat tmp/scenegraph_import.log; exit 1; }
sed "s/^/ /" tmp/scenegraph_import.log
grep -q "516 sequences, 906 input windows" tmp/scenegraph_import.log || {
echo "FAIL: the scene graph did not import to 516/906 -- upstream changed,"
echo " or the parser silently dropped branches."; exit 1; }
python3 tools/analysis/25_scene_graph.py --kbps 488 --ring 256 \
> tmp/scenegraph_check.log 2>&1 \
|| { tail -20 tmp/scenegraph_check.log; exit 1; }
grep -aE "^ WORST |^ ZERO-PLAY|^ BRANCH STRUCTURE" tmp/scenegraph_check.log
else
echo " SKIPPED: no DirkSimple checkout at $DIRKSIMPLE"
echo " (git clone --depth 1 https://github.com/icculus/DirkSimple)"
fi
echo "ALL GREEN" echo "ALL GREEN"
tools/import/scenegraph.py
+436
View File
@@ -0,0 +1,436 @@
"""The ONE file in this tree that knows anything about somebody else's source.
Everything downstream -- `tools/analysis/25_scene_graph.py` and whatever the
game-logic layer eventually becomes -- reads the neutral table this writes and
knows nothing about where it came from. That is deliberate. Two outside
projects are read here, neither is vendored, neither ships in this repo, and
their file layouts, table names, timing formulas and magic constants are wired
into THIS file and nowhere else. When one of them moves, one file breaks.
python3 tools/import/scenegraph.py [-o tmp/scenegraph.json]
Sources, both permissively licensed and both cloned by the reader:
icculus/DirkSimple -- zlib, Ryan C. Gordon. `data/games/lair/game.lua` holds
the arcade scene graph, transcribed from the arcade ROM's own data table.
THE source; the measurement is made on this.
DLX_DIRKSIMPLE=/path (default tmp/scenegraph/DirkSimple)
astrobleem/SNES-SuperDragonsLairArcade -- MIT, Chad Doebelin.
`data/events/*.xml` holds 516 chapter definitions. Read as a CROSS-CHECK
only, and a weak one: their own README states the chapters are "derived
from DirkSimple game data", so this is a second COPY, not a second
transcription (FINDINGS 56.2). Optional; skipped when absent.
DLX_SNESLAIR=/path (default tmp/scenegraph/SNES-SuperDragonsLairArcade)
WHAT COMES OUT is `DLXSCENE1`, our own schema, in one JSON file:
{"format": "DLXSCENE1",
"sources": [{"name", "url", "licence", "holder", "role"} ...],
"scenes": {scene: {sequence: {
"start_ms": float, absolute position on the medium, or -1 for
"keep playing from where the disc already is",
"timeout_ms": float, how long the clip runs unbranched,
"kills", "single_frame": bool,
"timeout_next": sequence name or null,
"actions": [{"input", "from_ms", "to_ms", "next"}]}}},
"rows": [[scene, scene, scene] ...], the arcade's 13x3 scene order
"counts": {"scenes", "sequences", "windows"},
"crosscheck": {scene: {sequence: {"start_ms", "end_ms", "actions": [...],
"chapter"}}} or null}
Nothing in the schema is a copy of either project's structure: it is what this
project's own analysis needs, which is a clip's length, its exits and the
earliest instant each of them can fire.
If a scene table is ever COMMITTED to this repo rather than regenerated into
gitignored tmp/, it becomes redistribution of derived data and the attribution
in "sources" has to travel with it. FINDINGS 16, 56.5.
"""
import sys, os, re, json, argparse
import xml.etree.ElementTree as ET
DIRK = os.environ.get("DLX_DIRKSIMPLE", "tmp/scenegraph/DirkSimple")
SNES = os.environ.get("DLX_SNESLAIR", "tmp/scenegraph/SNES-SuperDragonsLairArcade")
# DirkSimple's own constants, from data/games/lair/game.lua. Restated here
# because this file evaluates its table; both are gated below against the text
# of the file they came from, so they cannot go stale silently.
LD_FPS = 23.976 # laserdisc_frame_to_ms
ROM_OFFSET_MS = 6297.0 # time_laserdisc_frame's "magic millisecond offset"
SOURCES = [
{"name": "icculus/DirkSimple",
"url": "https://github.com/icculus/DirkSimple",
"licence": "zlib", "holder": "Ryan C. Gordon",
"role": "the scene graph itself, transcribed from the arcade ROM"},
{"name": "astrobleem/SNES-SuperDragonsLairArcade",
"url": "https://github.com/astrobleem/SNES-SuperDragonsLairArcade",
"licence": "MIT", "holder": "Chad Doebelin",
"role": "cross-check only; itself derived from DirkSimple (FINDINGS 56.2)"},
]
# ---------------------------------------------------------------- Lua subset
class LuaParser:
"""Just enough Lua to read game.lua's `scenes` table constructor.
Not a Lua interpreter and not trying to be: the grammar is table
constructors, string/number/boolean/nil literals, bare identifiers (used
only for function references like `interrupt=game_over_complete`), calls to
four known helpers, and + / - between them. Anything else is a parse error
rather than a silent skip, and the caller asserts the whole table was
consumed, so a change upstream shows up as a failure and not as a smaller
scene graph.
"""
def __init__(self, text, helpers):
self.s, self.i, self.helpers = text, 0, helpers
def error(self, msg):
line = self.s.count("\n", 0, self.i) + 1
raise SyntaxError(f"{msg} at line {line}: {self.s[self.i:self.i+60]!r}")
def ws(self):
while self.i < len(self.s):
c = self.s[self.i]
if c in " \t\r\n":
self.i += 1
elif self.s.startswith("--", self.i):
self.i = self.s.find("\n", self.i)
if self.i < 0:
self.i = len(self.s)
else:
return
def take(self, lit):
self.ws()
if self.s.startswith(lit, self.i):
self.i += len(lit)
return True
return False
def expect(self, lit):
if not self.take(lit):
self.error(f"expected {lit!r}")
def name(self):
self.ws()
m = re.compile(r"[A-Za-z_][A-Za-z0-9_]*").match(self.s, self.i)
if not m:
return None
self.i = m.end()
return m.group(0)
def value(self):
self.ws()
if self.take("{"):
return self.table()
m = re.compile(r"-?\d+(\.\d+)?").match(self.s, self.i)
if m:
self.i = m.end()
v = float(m.group(0))
return self.arith(int(v) if v.is_integer() else v)
if self.s[self.i] in "\"'":
q = self.s[self.i]
j = self.s.index(q, self.i + 1)
v = self.s[self.i + 1:j]
self.i = j + 1
return v
n = self.name()
if n is None:
self.error("expected a value")
if n == "nil":
return None
if n in ("true", "false"):
return n == "true"
if self.take("("): # a call
args = []
if not self.take(")"):
while True:
args.append(self.value())
if self.take(")"):
break
self.expect(",")
if n not in self.helpers:
self.error(f"unknown helper {n!r}")
return self.arith(self.helpers[n](*args))
return Symbol(n) # a function reference
def arith(self, left):
"""+ and - between numbers. Present in the source and load-bearing:
e.g. `time_laserdisc_frame(1823) - laserdisc_frame_to_ms(2)`."""
while True:
self.ws()
if self.i < len(self.s) and self.s[self.i] in "+-":
op = self.s[self.i]
self.i += 1
right = self.value()
left = left + right if op == "+" else left - right
else:
return left
def table(self):
d, arr = {}, []
while True:
self.ws()
if self.take("}"):
break
save = self.i
k = self.name()
if k is not None and self.take("="):
d[k] = self.value()
else:
self.i = save
arr.append(self.value())
if not (self.take(",") or self.take(";")):
self.expect("}")
break
if d and arr:
self.error("mixed array/hash table")
return arr if arr or not d else d
class Symbol(str):
"""A bare Lua identifier used as a value (a function reference)."""
def load_dirksimple(path):
"""Parse lair/game.lua's `scenes` and `scene_manager` tables.
Gates the four timing helpers against the text that defines them, so this
tool cannot keep evaluating a formula upstream has changed.
"""
src = open(os.path.join(path, "data/games/lair/game.lua"), encoding="utf-8").read()
def gate(pat, what):
if not re.search(pat, src):
raise AssertionError(f"DirkSimple's {what} is not what this tool "
f"evaluates any more (looked for {pat!r})")
gate(r"frame\s*/\s*23\.976", "laserdisc_frame_to_ms")
gate(r"-\s*6297\.0", "time_laserdisc_frame's ROM offset")
gate(r"time_laserdisc_noseek.*?\n\s*return -1", "time_laserdisc_noseek")
gate(r"\(seconds \* 1000\) \+ ms", "time_to_ms")
helpers = {
"laserdisc_frame_to_ms": lambda f: (f / LD_FPS) * 1000.0,
"time_laserdisc_frame": lambda f: (f / LD_FPS) * 1000.0 - ROM_OFFSET_MS,
"time_laserdisc_noseek": lambda: -1,
# time_to_ms takes (seconds, ms); the source calls it with a third
# argument exactly once, which Lua discards.
"time_to_ms": lambda s, ms=0, *_: s * 1000 + ms,
}
out = {}
for var in ("scenes", "scene_manager"):
m = re.search(rf"^{var} = \{{", src, re.M)
if not m:
raise AssertionError(f"{var} table not found in game.lua")
p = LuaParser(src, helpers)
p.i = m.end()
out[var] = p.table()
# The parser must have consumed the WHOLE constructor: table() returns
# with the cursor just past the matching close brace, and in this file
# every top-level table ends on a `}` in column 0. A parser that
# stopped early would return a smaller graph, which is the direction
# that flatters the measurement, so this is checked rather than assumed.
if src[p.i - 1] != "}":
raise AssertionError(f"{var}: parser did not end on a close brace")
if src.rfind("\n", 0, p.i) != p.i - 2:
raise AssertionError(f"{var}: the constructor ended mid-line at "
f"{p.i}, so the parse stopped early")
return out["scenes"], out["scene_manager"]
# ------------------------------------------------- the SNES cross-check
def load_snes(path):
"""Parse the SNES project's chapter XMLs into {chapter: (start_ms, end_ms,
[(input, from_ms, to_ms, target)])}, plus the provenance line."""
ev = os.path.join(path, "data/events")
readme = os.path.join(ev, "README.md")
prov = None
if os.path.exists(readme):
txt = open(readme, encoding="utf-8").read()
if "derived from DirkSimple" in txt:
prov = "derived from DirkSimple game data (data/events/README.md)"
def ms(e):
return (int(e.get("min", 0)) * 60000 + int(e.get("second", 0)) * 1000
+ int(e.get("ms", 0)))
chapters = {}
for f in sorted(os.listdir(ev)):
if not f.endswith(".xml"):
continue
root = ET.parse(os.path.join(ev, f)).getroot()
tl = root.find("timeline")
s = tl.find("timestart")
e = tl.find("timeend")
acts = []
for evt in root.findall("./events/event"):
if evt.get("type") != "direction":
continue
t = evt.find("timeline")
p = evt.find("./params/str[@key='type']")
r = evt.find("./result/playchapter")
acts.append((p.get("value") if p is not None else "?",
ms(t.find("timestart")),
ms(t.find("timeend")) if t.find("timeend") is not None else -1,
r.get("name") if r is not None else None))
chapters[f[:-4]] = (ms(s) if s is not None else -1,
ms(e) if e is not None else -1, acts)
return chapters, prov
def map_abbrevs(chapters, scenes):
"""Match each SNES scene abbreviation to a DirkSimple scene from the DATA,
not from a hand-written table, so a wrong match is visible rather than
assumed. Three signals, in order:
1. the abbreviation's letters must be a SUBSEQUENCE of the scene name
(`snkr` is snake_room, and cannot be black_knight however similar
their `seqN` names are -- overlap alone got that one wrong);
2. the Jaccard overlap of the two sequence-name sets;
3. the reversed-scene rule: Dragon's Lair mirrors thirteen scenes, and
where BOTH `x` and `xr` exist as abbreviations, `xr` is the
`_reversed` scene and `x` is not. Their sequence names are identical,
so nothing else separates them.
Assignment is 1:1 and greedy on that ranking.
"""
def subseq(a, b):
it = iter(b)
return all(c in it for c in a)
by_abbr = {}
for ch in chapters:
abbr, _, seq = ch.partition("_")
by_abbr.setdefault(abbr, set()).add(seq)
pairs = []
for abbr, seqs in by_abbr.items():
mirror = abbr.endswith("r") and abbr[:-1] in by_abbr
for scene, s in scenes.items():
if not subseq(abbr, scene.replace("_", "")):
continue
union = len(seqs | set(s))
j = len(seqs & set(s)) / union if union else 0.0
rev = scene.endswith("_reversed")
pairs.append((j, 1 if rev == mirror else 0, -len(scene),
abbr, scene, len(seqs)))
pairs.sort(reverse=True)
mapping, used = {}, set()
for j, _, _, abbr, scene, n in pairs:
if abbr in mapping or scene in used:
continue
mapping[abbr] = (scene, j, n)
used.add(scene)
unmatched = [(a, None, 0.0, len(s)) for a, s in by_abbr.items()
if a not in mapping]
return mapping, unmatched
# ------------------------------------------------------------------- emit
def to_neutral(scenes, mgr):
"""DirkSimple's tables -> DLXSCENE1. The one place their shape is read."""
out = {}
for scene, seqs in scenes.items():
s = out.setdefault(scene, {})
for name, seq in seqs.items():
t = seq["timeout"]
s[name] = {
"start_ms": float(seq["start_time"]),
"timeout_ms": float(t["when"]),
"timeout_next": t.get("nextsequence"),
"kills": bool(seq.get("kills_player")),
"single_frame": bool(seq.get("is_single_frame")),
"actions": [{"input": a["input"],
"from_ms": float(a["from"]),
"to_ms": float(a["to"]),
"next": a.get("nextsequence")}
for a in seq.get("actions", [])],
}
rows = [list(r) for r in mgr["rows"]]
return out, rows
def crosscheck_neutral(chapters, scenes):
"""The SNES chapters, keyed the way OUR table is keyed.
Their `abbr_sequence` naming is resolved to a scene here, from the data
(see map_abbrevs), so nothing downstream has to know an abbreviation
exists. Chapters the mapping cannot place are dropped and counted.
"""
mapping, unmatched = map_abbrevs(chapters, scenes)
out, dropped = {}, 0
for ch, (s_ms, e_ms, acts) in chapters.items():
abbr, _, seq = ch.partition("_")
if abbr not in mapping:
dropped += 1
continue
scene = mapping[abbr][0]
pre = abbr + "_"
out.setdefault(scene, {})[seq] = {
"chapter": ch,
"start_ms": float(s_ms),
"end_ms": float(e_ms),
"actions": [{"input": i, "from_ms": float(f), "to_ms": float(t2),
"next": (n[len(pre):] if n and n.startswith(pre) else n)}
for i, f, t2, n in acts],
}
weak = sorted((round(j, 3), ab, sc) for ab, (sc, j, _n) in mapping.items()
if j < 1.0)
return out, {"chapters": len(chapters), "dropped": dropped,
"scenes_mapped": len(mapping),
"weakest_mappings": weak[:6]}
def main():
ap = argparse.ArgumentParser()
ap.add_argument("-o", "--out", default="tmp/scenegraph.json")
a = ap.parse_args()
if not os.path.isdir(DIRK):
sys.exit(f"no DirkSimple checkout at {DIRK} -- set DLX_DIRKSIMPLE, or\n"
f" git clone --depth 1 https://github.com/icculus/DirkSimple")
scenes, mgr = load_dirksimple(DIRK)
neutral, rows = to_neutral(scenes, mgr)
nseq = sum(len(s) for s in neutral.values())
nwin = sum(len(q["actions"]) for s in neutral.values() for q in s.values())
cross, cross_meta = None, None
if os.path.isdir(SNES):
chapters, prov = load_snes(SNES)
cross, cross_meta = crosscheck_neutral(chapters, scenes)
cross_meta["provenance"] = prov
# 56.2: this is the sentence the whole "second source" claim turned on.
# If upstream ever removes it, say so rather than silently promoting a
# copy back to an independent transcription.
if not prov:
cross_meta["provenance"] = ("UNSTATED -- data/events/README.md no "
"longer says where the chapters came "
"from; re-check before trusting this "
"as anything.")
doc = {"format": "DLXSCENE1", "sources": SOURCES,
"scenes": neutral, "rows": rows,
"counts": {"scenes": len(neutral), "sequences": nseq,
"windows": nwin},
"crosscheck": cross, "crosscheck_meta": cross_meta}
os.makedirs(os.path.dirname(a.out) or ".", exist_ok=True)
with open(a.out, "w") as f:
json.dump(doc, f, indent=1, sort_keys=True)
print(f"{a.out}: {len(neutral)} scenes, {nseq} sequences, {nwin} input "
f"windows, {len(rows)} rows"
+ (f", cross-check {cross_meta['chapters']} chapters"
if cross_meta else ", no cross-check (SNES tree absent)"))
return 0
if __name__ == "__main__":
sys.exit(main())