Record the Spanish rebuild and how to do the next one
dict.db now holds all five languages on the VPS. The deploy notes carry the recipe, the two traps (SUBTLEX-US fails harmlessly; check the other languages' counts are unchanged before shipping) and the measured coverage, so the next rebuild is not archaeology. Claude-Session: https://claude.ai/code/session_016y6gyuHkQXPiEuW8RGQyua
This commit is contained in:
+29
-4
@@ -335,10 +335,35 @@ use by a real writer. `lexicon.Set.For` is where that decision lives — one
|
||||
For pt-PT and French the same measurement reads 62%, which is why they use
|
||||
DreamDict: there is no alternative source for them at all.
|
||||
|
||||
**The deployed `dict.db` is from 2026-04-04 and has no Spanish data**, because
|
||||
DreamDict grew Spanish support after it was built. A Spanish-pair writer gets
|
||||
empty glosses (and English definitions) until it is rebuilt — do that before
|
||||
the Spanish pair ships.
|
||||
### Rebuilding it
|
||||
|
||||
Rebuilt 2026-07-27 to add Spanish (the previous file predated DreamDict's
|
||||
Spanish support). The recipe, since it will be needed again:
|
||||
|
||||
```bash
|
||||
# on millenia, from a clean checkout of dreamdict main
|
||||
./scripts/download-dict-data.sh ~/dreamdict/data # idempotent; skips what's there
|
||||
go run ./cmd/dictimport --data ~/dreamdict/data --db ./dict.db --clean
|
||||
```
|
||||
|
||||
~6 minutes on 32 cores; the data directory is ~7 GB and mostly already
|
||||
downloaded. **Build to a new path, never over a file in use** — then verify by
|
||||
hash on both ends before swapping.
|
||||
|
||||
Two things worth knowing before trusting a rebuild:
|
||||
|
||||
- The SUBTLEX-US download fails (the source moved behind a manual export). It
|
||||
does not matter: the loader falls back to `SUBTLEX-US.txt`, which is present,
|
||||
and English "frequency" is mostly SCOWL's commonness bucket anyway —
|
||||
1000/800/600/…/50, refined by SUBTLEX for only ~1,600 words. That is why the
|
||||
word-difficulty chip reads `difficulty`, not `frequency`.
|
||||
- Check the *other* languages' counts are unchanged before shipping. The 2026-07
|
||||
rebuild came out byte-identical for en/fr/pt-PT/zh, which is what says it
|
||||
added a language rather than quietly shifting the rest.
|
||||
|
||||
Gloss coverage of the 2,000 commonest English words, after the rebuild:
|
||||
**es 68.6%**, fr 63.1%, pt-PT 62.1%, zh 53.2%. The startup line reports actual
|
||||
per-language row counts, so a database missing a language says so.
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user