Plan: DreamDict answers the Portuguese dictionary question
OPEN #6 assumed the pt-PT gloss was gated on finding a dataset of ECDICT's quality. It isn't — dreamdict already covers en, fr, pt-PT and zh, and its API maps almost 1:1 onto lexicon.Result. Replaces the data question with an integration one (OPEN #6a): HTTP service, build-time extraction into Petal's embedded gz format, or importing the dictionary package and opening dict.db read-only. Argues for the third — same CGO-free sqlite driver Petal already depends on, no runtime service, and it deletes ~11.6 MB of embedded data plus the ECDICT build scripts. Adds the migration caution that matters most: the zh path is in daily use, so wire pt-PT/fr first (nothing to regress) and leave zh on ECDICT until CC-CEDICT gloss quality has been compared on her real lookups.
This commit is contained in:
+90
-20
@@ -210,7 +210,7 @@ dataset identical for everyone.
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 6. Phase D — per-user language
|
## 6. Phase D — per-user language (and DreamDict)
|
||||||
|
|
||||||
Tracked here because it lands on the same `users` row and shouldn't be designed
|
Tracked here because it lands on the same `users` row and shouldn't be designed
|
||||||
twice.
|
twice.
|
||||||
@@ -219,26 +219,89 @@ twice.
|
|||||||
language — the one glosses and explanations are written in. Mandarin ships today;
|
language — the one glosses and explanations are written in. Mandarin ships today;
|
||||||
European Portuguese (pt-PT, explicitly not pt-BR) is wanted; French is possible.
|
European Portuguese (pt-PT, explicitly not pt-BR) is wanted; French is possible.
|
||||||
|
|
||||||
That makes native language a `users` column and turns these into per-user
|
### OPEN #6 is answered: DreamDict
|
||||||
lookups:
|
|
||||||
|
|
||||||
| Piece | Today | Notes |
|
The original worry here was data sourcing — Petal's gloss comes from ECDICT
|
||||||
|---|---|---|
|
(English↔Chinese), and a pt-PT equivalent of comparable quality and license
|
||||||
| Gloss / lexicon data | ECDICT, English↔Chinese | pt-PT needs its own source — **this is the hard part**, not the code |
|
looked like the blocker.
|
||||||
| LLM prompt copy | Mandarin-first bilingual | `internal/llm/prompts.go` |
|
|
||||||
| Companion tips | bilingual `tips.ts` | per-language copy |
|
|
||||||
| TTS L1 voice | Piper zh_CN-huayan | Piper has pt-PT voices |
|
|
||||||
| Font stacks | CJK fallbacks | not needed for Latin-script L1 |
|
|
||||||
|
|
||||||
English-side machinery (nspell en-US, the IPA/phonetic dataset, the EN Piper
|
`~/git/dreamdict` already solves it, and more completely than expected. It
|
||||||
voice) is unaffected and stays shared.
|
supports **en, fr, pt-PT, and zh** (~136k/56k/136k/121k words), and its shape maps
|
||||||
|
almost 1:1 onto `lexicon.Result`:
|
||||||
|
|
||||||
**OPEN #6:** the gloss dataset is the blocker, not the plumbing. Is there an
|
| Petal field | DreamDict |
|
||||||
open English↔Portuguese dictionary of ECDICT's quality and license? If not, this
|
|---|---|
|
||||||
phase is gated on data sourcing and the code work is comparatively trivial. Worth
|
| `Gloss` | `Translate(word, "en", L1)` |
|
||||||
answering before scheduling it.
|
| `Phonetic` | pronunciation (CMU + IPA for en, Wiktionary IPA elsewhere) |
|
||||||
|
| `Definitions` | `Define(word, lang)` — curated sources ranked above Wiktionary |
|
||||||
|
| `Synonyms` | `Synonyms(word, lang)` |
|
||||||
|
|
||||||
---
|
It also carries data Petal has no equivalent for and could use: `Antonyms`,
|
||||||
|
`Frequency`, `Difficulty`, and `Etymology`.
|
||||||
|
|
||||||
|
So Phase D stops being gated on data and becomes an integration decision.
|
||||||
|
|
||||||
|
### OPEN #6a (new): how to integrate
|
||||||
|
|
||||||
|
**Option 1 — HTTP client.** Petal calls DreamDict on localhost:7777, exactly the
|
||||||
|
pattern already used for Piper TTS (including graceful degradation when it's
|
||||||
|
down).
|
||||||
|
- *For:* zero coupling, DreamDict updates independently, all endpoints available.
|
||||||
|
- *Against:* a second service Petal now depends on at runtime, and the gloss is a
|
||||||
|
350ms hover tooltip where "the dictionary service is down" is a visible
|
||||||
|
regression from today's always-there embedded data.
|
||||||
|
|
||||||
|
**Option 2 — build-time extraction.** A script (sibling to the existing
|
||||||
|
`scripts/build_gloss.py`) generates Petal's embedded `.json.gz` datasets per
|
||||||
|
language from DreamDict's `dict.db`.
|
||||||
|
- *For:* preserves the embedded/offline property exactly; no runtime dependency;
|
||||||
|
no architectural change at all.
|
||||||
|
- *Against:* every language multiplies the binary (the four current gz files are
|
||||||
|
already ~11.6 MB); updating the dictionary means rebuilding and redeploying
|
||||||
|
Petal; the richer fields are lost unless separately extracted.
|
||||||
|
|
||||||
|
**Option 3 — import the package, open `dict.db` read-only.** DreamDict's
|
||||||
|
`internal/dictionary` is a plain library with `NewReadOnly(dbPath)`, and its only
|
||||||
|
dependency is `modernc.org/sqlite` — the same CGO-free driver Petal already uses.
|
||||||
|
Petal opens `dict.db` as a second read-only handle beside `petal.db`.
|
||||||
|
- *For:* no service, no HTTP, no new dependency, lookups stay local-file fast,
|
||||||
|
all four languages at once, and it deletes ~11.6 MB of embedded gz plus the
|
||||||
|
ECDICT build scripts. One dictionary, maintained once, shared with GogoBee.
|
||||||
|
- *Against:* Petal stops being a self-contained binary in the "just run it" sense
|
||||||
|
— `dict.db` has to be deployed alongside. In practice Petal already ships a
|
||||||
|
data directory (`petal.db`, images, TTS cache), so this is a smaller loss than
|
||||||
|
it first sounds.
|
||||||
|
|
||||||
|
**My recommendation: Option 3.** It is the only one that gets all four languages,
|
||||||
|
keeps lookups offline and instant, and *removes* code rather than adding a
|
||||||
|
subsystem. Option 1's runtime dependency buys flexibility Petal doesn't need for
|
||||||
|
a dictionary that changes a few times a year.
|
||||||
|
|
||||||
|
**Prerequisite:** DreamDict's module path is currently `module dreamdict`, which
|
||||||
|
isn't fetchable. Importing it needs the module renamed to something like
|
||||||
|
`gitea.parodia.dev/drwily/dreamdict` (or a local `replace` directive for
|
||||||
|
development). Small, but it must happen first.
|
||||||
|
|
||||||
|
### Migration caution
|
||||||
|
|
||||||
|
Whichever option wins, the zh path is **currently working and in daily use**. The
|
||||||
|
gloss quality difference between ECDICT and CC-CEDICT is unknown and matters more
|
||||||
|
than the architecture.
|
||||||
|
|
||||||
|
Proposal: introduce DreamDict behind Petal's existing lexicon interface as a
|
||||||
|
*provider*, wire pt-PT and fr to it first (nothing to regress — they don't exist
|
||||||
|
yet), and keep zh on ECDICT until the two have been compared on real lookups from
|
||||||
|
her actual documents. Converge only if quality holds. This also de-risks the whole
|
||||||
|
change: if DreamDict turns out to be a poor fit, only the unshipped languages are
|
||||||
|
affected.
|
||||||
|
|
||||||
|
### Still per-user regardless
|
||||||
|
|
||||||
|
Native language becomes a `users` column, and these become per-user lookups:
|
||||||
|
LLM prompt copy (`internal/llm/prompts.go`, currently Mandarin-first), companion
|
||||||
|
tips (`tips.ts`), the L1 Piper voice (Piper has pt-PT voices), and the CJK font
|
||||||
|
stacks (not needed for Latin-script L1). English-side machinery — nspell en-US,
|
||||||
|
the phonetic dataset, the EN voice — is unaffected and stays shared.
|
||||||
|
|
||||||
## 7. Suggested sequence
|
## 7. Suggested sequence
|
||||||
|
|
||||||
@@ -250,7 +313,10 @@ answering before scheduling it.
|
|||||||
4. Image store table + migration (same phase, per OPEN #5).
|
4. Image store table + migration (same phase, per OPEN #5).
|
||||||
5. Provision the second real account; migrate `local`'s data.
|
5. Provision the second real account; migrate `local`'s data.
|
||||||
6. `localStorage` namespacing.
|
6. `localStorage` namespacing.
|
||||||
7. Per-user language, gated on the dataset question.
|
7. Per-user language. **No longer gated on data** — DreamDict covers all four
|
||||||
|
languages. Sequence within it: rename DreamDict's module path → wire it in as
|
||||||
|
a lexicon provider → pt-PT/fr first → compare zh quality → converge if it
|
||||||
|
holds.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -279,7 +345,11 @@ answering before scheduling it.
|
|||||||
4. Migration script vs. admin endpoint for moving `local`'s data? (OPEN #4)
|
4. Migration script vs. admin endpoint for moving `local`'s data? (OPEN #4)
|
||||||
5. Fix the image store alongside auth, or ship auth with it as a known
|
5. Fix the image store alongside auth, or ship auth with it as a known
|
||||||
limitation? (OPEN #5)
|
limitation? (OPEN #5)
|
||||||
6. Is there a usable open English↔European-Portuguese dictionary dataset?
|
6. ~~Is there a usable open English↔European-Portuguese dictionary dataset?~~
|
||||||
(OPEN #6 — gates Phase D entirely)
|
**Answered: DreamDict**, which covers en/fr/pt-PT/zh. The live question is now
|
||||||
|
*how* to integrate it — HTTP service, build-time extraction, or importing the
|
||||||
|
package and opening `dict.db` read-only. (OPEN #6a; I recommend the third)
|
||||||
|
7. Does replacing the zh gloss (ECDICT → CC-CEDICT) risk regressing a feature in
|
||||||
|
daily use, and should zh stay on ECDICT until the two are compared?
|
||||||
|
|
||||||
Anything above that reads as settled but shouldn't be is also fair game.
|
Anything above that reads as settled but shouldn't be is also fair game.
|
||||||
|
|||||||
Reference in New Issue
Block a user