Record the Spanish rebuild and how to do the next one

dict.db now holds all five languages on the VPS. The deploy notes carry the
recipe, the two traps (SUBTLEX-US fails harmlessly; check the other languages'
counts are unchanged before shipping) and the measured coverage, so the next
rebuild is not archaeology.

Claude-Session: https://claude.ai/code/session_016y6gyuHkQXPiEuW8RGQyua
This commit is contained in:
prosolis
2026-07-27 10:53:22 -07:00
parent 74bf600593
commit 4de83d0da5
2 changed files with 34 additions and 5 deletions
+29 -4
View File
@@ -335,10 +335,35 @@ use by a real writer. `lexicon.Set.For` is where that decision lives — one
For pt-PT and French the same measurement reads 62%, which is why they use
DreamDict: there is no alternative source for them at all.
**The deployed `dict.db` is from 2026-04-04 and has no Spanish data**, because
DreamDict grew Spanish support after it was built. A Spanish-pair writer gets
empty glosses (and English definitions) until it is rebuilt — do that before
the Spanish pair ships.
### Rebuilding it
Rebuilt 2026-07-27 to add Spanish (the previous file predated DreamDict's
Spanish support). The recipe, since it will be needed again:
```bash
# on millenia, from a clean checkout of dreamdict main
./scripts/download-dict-data.sh ~/dreamdict/data # idempotent; skips what's there
go run ./cmd/dictimport --data ~/dreamdict/data --db ./dict.db --clean
```
~6 minutes on 32 cores; the data directory is ~7 GB and mostly already
downloaded. **Build to a new path, never over a file in use** — then verify by
hash on both ends before swapping.
Two things worth knowing before trusting a rebuild:
- The SUBTLEX-US download fails (the source moved behind a manual export). It
does not matter: the loader falls back to `SUBTLEX-US.txt`, which is present,
and English "frequency" is mostly SCOWL's commonness bucket anyway —
1000/800/600/…/50, refined by SUBTLEX for only ~1,600 words. That is why the
word-difficulty chip reads `difficulty`, not `frequency`.
- Check the *other* languages' counts are unchanged before shipping. The 2026-07
rebuild came out byte-identical for en/fr/pt-PT/zh, which is what says it
added a language rather than quietly shifting the rest.
Gloss coverage of the 2,000 commonest English words, after the rebuild:
**es 68.6%**, fr 63.1%, pt-PT 62.1%, zh 53.2%. The startup line reports actual
per-language row counts, so a database missing a language says so.
---