Phase 20: the dictionary stops being English and Chinese only
Word lookups now come from DreamDict's dict.db for every pair but Chinese — opened read-only beside petal.db, no service, nothing over the VPN, because a hover gloss has to answer in milliseconds. `Provider` is the two questions the popover and the tooltip already asked, so the embedded *Lexicon satisfies it with no changes at all; Set.For(lang) is the single place the choice between them is made. The prerequisite in the dreamdict repo turned out to be two things, not one: the module path was unfetchable *and* the query layer sat in internal/, which no other module may import whatever the module is called. Both fixed upstream. The plan's central assumption did not survive the data. It mapped Gloss ← Translate(word, "en", L1) one-to-one; against the real 452 MB database that table answers for 17% of the 2,000 commonest English words into pt-PT. Wiktionary's translation sections are thin in that direction — "ephemeral", "think" and "quickly" have no en→pt-PT row at all. Shared WordNet synsets answer for 61%, so DreamDict gained Equivalents() and Petal glosses through it. Ordering those was wrong in an instructive way too: sorting by frequency glosses "think" as lembrar, "remember", because lembrar is the commoner Portuguese word even though pensar shares six of think's synsets to lembrar's one. Counting sense agreement first asks the right question. The same measurement is why zh stays on ECDICT: DreamDict reaches a Chinese gloss for 53% of those words, ECDICT for nearly all of them. The plan said converge only if quality holds. It didn't, so nothing converged. Two decisions about failure worth keeping. A missing dict.db is not an error — a laptop checkout has never had one — but a present-and-never-imported one is, because that is a half-finished deploy. And a pt-PT writer with no dictionary falls back to the embedded datasets with the gloss suppressed, keeping definitions, synonyms and phonetics rather than blanking the popover: an empty field reads as "not found", the wrong language reads as broken. The new fields surface as an etymology line and a three-band chip. Three, not five: the difficulty score separates "everyday" from "you'll have to explain this" but cannot rank obfuscate against serendipity, and a finer scale would be a confident-looking lie. An unscored word gets no chip. Writing the tests found two bugs first — trimEtymology sliced by byte, which would have emitted invalid UTF-8 for exactly the Greek and Latin etymologies the feature exists for, and its ellipsis path overran its own cap. go build/vet/test, tsc, vite, vitest 96/96 clean; live smoke against the real dict.db with one instance flipped from zh to pt-PT mid-run. Not deployed: go.mod still replaces github.com/prosolis/dreamdict with ../dreamdict, so the Docker build needs the two upstream commits pushed and the replace dropped. The deployed dict.db also predates DreamDict's Spanish data. Claude-Session: https://claude.ai/code/session_016y6gyuHkQXPiEuW8RGQyua
This commit is contained in:
@@ -284,6 +284,64 @@ a migration took; the presence of the row itself says nothing.
|
||||
|
||||
---
|
||||
|
||||
## 4b. The dictionary (`dict.db`)
|
||||
|
||||
Word lookups for the French, European Portuguese and Spanish pairs come from
|
||||
[DreamDict](https://github.com/prosolis/dreamdict)'s built database, which Petal
|
||||
opens **read-only** beside `petal.db`. Petal imports DreamDict's `dictionary`
|
||||
package directly — there is no DreamDict service to run and nothing to reach
|
||||
over the VPN, which matters because a hover gloss must answer in milliseconds.
|
||||
|
||||
`dict.db` is **optional**. With no file at `DICT_PATH` Petal logs
|
||||
|
||||
```
|
||||
dictionary: no dict.db at /data/dict.db — English/Chinese only
|
||||
```
|
||||
|
||||
and serves lookups from the datasets compiled into the binary. The Chinese pair
|
||||
is unaffected either way — it stays on ECDICT (see below) — and a non-Chinese
|
||||
writer still gets English definitions, synonyms and pronunciation, losing only
|
||||
the translation. **A dictionary that failed to deploy costs the gloss, not the
|
||||
popover.** A file that is present but was never imported is a different matter
|
||||
and is logged as an error.
|
||||
|
||||
### Installing it
|
||||
|
||||
The database is built by DreamDict's own import CLI from ~6 GB of source data;
|
||||
it is not built on the VPS. Copy the built file into the data volume:
|
||||
|
||||
```bash
|
||||
# on the machine holding a built dict.db (millenia: ~/dreamdict/data/dict.db)
|
||||
scp ~/dreamdict/data/dict.db reala@100.64.0.1:/home/reala/petal/data/dict.db
|
||||
# on parodia
|
||||
chown "$(id -u):$(id -g)" /home/reala/petal/data/dict.db
|
||||
docker compose restart petal # the handle is opened once, at startup
|
||||
```
|
||||
|
||||
Expect ~450 MB. It sits inside the LUKS volume with everything else (§6). The
|
||||
backups name `petal.db` explicitly rather than sweeping the data directory
|
||||
(§5), so `dict.db` stays out of them — which is the right outcome and worth
|
||||
keeping: it is rebuildable from public data and would otherwise dominate every
|
||||
nightly snapshot. Petal never writes to it.
|
||||
|
||||
### Why Chinese doesn't use it
|
||||
|
||||
The zh pair stays on the embedded ECDICT gloss, deliberately. Measured on the
|
||||
deployed database, DreamDict reaches a Chinese gloss for 53% of the 2,000
|
||||
commonest English words; ECDICT covers essentially all of them and is in daily
|
||||
use by a real writer. `lexicon.Set.For` is where that decision lives — one
|
||||
`switch`, changed the day a comparison on her actual lookups says otherwise.
|
||||
|
||||
For pt-PT and French the same measurement reads 62%, which is why they use
|
||||
DreamDict: there is no alternative source for them at all.
|
||||
|
||||
**The deployed `dict.db` is from 2026-04-04 and has no Spanish data**, because
|
||||
DreamDict grew Spanish support after it was built. A Spanish-pair writer gets
|
||||
empty glosses (and English definitions) until it is rebuilt — do that before
|
||||
the Spanish pair ships.
|
||||
|
||||
---
|
||||
|
||||
## 5. Backups
|
||||
|
||||
### On the VPS — folded into `parodia-backup`
|
||||
|
||||
Reference in New Issue
Block a user