Phase 20: the dictionary stops being English and Chinese only

Word lookups now come from DreamDict's dict.db for every pair but Chinese —
opened read-only beside petal.db, no service, nothing over the VPN, because a
hover gloss has to answer in milliseconds.

`Provider` is the two questions the popover and the tooltip already asked, so
the embedded *Lexicon satisfies it with no changes at all; Set.For(lang) is the
single place the choice between them is made. The prerequisite in the dreamdict
repo turned out to be two things, not one: the module path was unfetchable
*and* the query layer sat in internal/, which no other module may import
whatever the module is called. Both fixed upstream.

The plan's central assumption did not survive the data. It mapped
Gloss ← Translate(word, "en", L1) one-to-one; against the real 452 MB database
that table answers for 17% of the 2,000 commonest English words into pt-PT.
Wiktionary's translation sections are thin in that direction — "ephemeral",
"think" and "quickly" have no en→pt-PT row at all. Shared WordNet synsets
answer for 61%, so DreamDict gained Equivalents() and Petal glosses through it.
Ordering those was wrong in an instructive way too: sorting by frequency
glosses "think" as lembrar, "remember", because lembrar is the commoner
Portuguese word even though pensar shares six of think's synsets to lembrar's
one. Counting sense agreement first asks the right question.

The same measurement is why zh stays on ECDICT: DreamDict reaches a Chinese
gloss for 53% of those words, ECDICT for nearly all of them. The plan said
converge only if quality holds. It didn't, so nothing converged.

Two decisions about failure worth keeping. A missing dict.db is not an error —
a laptop checkout has never had one — but a present-and-never-imported one is,
because that is a half-finished deploy. And a pt-PT writer with no dictionary
falls back to the embedded datasets with the gloss suppressed, keeping
definitions, synonyms and phonetics rather than blanking the popover: an empty
field reads as "not found", the wrong language reads as broken.

The new fields surface as an etymology line and a three-band chip. Three, not
five: the difficulty score separates "everyday" from "you'll have to explain
this" but cannot rank obfuscate against serendipity, and a finer scale would be
a confident-looking lie. An unscored word gets no chip.

Writing the tests found two bugs first — trimEtymology sliced by byte, which
would have emitted invalid UTF-8 for exactly the Greek and Latin etymologies
the feature exists for, and its ellipsis path overran its own cap.

go build/vet/test, tsc, vite, vitest 96/96 clean; live smoke against the real
dict.db with one instance flipped from zh to pt-PT mid-run.

Not deployed: go.mod still replaces github.com/prosolis/dreamdict with
../dreamdict, so the Docker build needs the two upstream commits pushed and the
replace dropped. The deployed dict.db also predates DreamDict's Spanish data.

Claude-Session: https://claude.ai/code/session_016y6gyuHkQXPiEuW8RGQyua

web/src/components/Editor/WordCard.tsx

@@ -1,6 +1,7 @@
 import type { WordInfo } from '../../api/client'
 import { speak, speechSupported } from '../../audio/speech'
 import { usePack } from '../../i18n'
+import { wordBand } from './wordband'
 
 // WordCard is the right-click popover for any word: its dictionary definition(s)
 // on top and tappable synonym pills below. Clicking a synonym replaces the word
@@ -26,6 +27,9 @@ export function WordCard({ word, info, loading, saved, onToggleSave, style, onRe
   const synonyms = info?.synonyms ?? []
   const gloss = info?.gloss ?? ''
   const phonetic = info?.phonetic ?? ''
+  const etymology = info?.etymology ?? ''
+  // Null whenever the dictionary has no opinion — the chip then doesn't render.
+  const band = info ? wordBand(info.frequency ?? 0, info.difficulty ?? -1) : null
   const empty = !loading && !gloss && definitions.length === 0 && synonyms.length === 0
 
   return (
@@ -85,12 +89,27 @@ export function WordCard({ word, info, loading, saved, onToggleSave, style, onRe
         </div>
       </div>
 
-      {/* How to say it — the pronunciation aid for an English learner, paired
-          with the 🔊 button above. */}
-      {phonetic && (
-        <p className="mt-1.5 text-sm" style={{ color: 'var(--color-muted)' }}>
-          /{phonetic}/
-        </p>
+      {/* How to say it, and how hard it is. The pronunciation aid pairs with the
+          🔊 button above; the band answers the question a learner actually has
+          when she has found a word she likes — "can I use this?". Both are
+          quiet, muted lines: information she can take or leave, never a verdict
+          on her writing. */}
+      {(phonetic || band) && (
+        <div className="mt-1.5 flex items-center gap-2 text-sm">
+          {phonetic && <span style={{ color: 'var(--color-muted)' }}>/{phonetic}/</span>}
+          {band && (
+            <span
+              className="rounded-full px-2 py-0.5 text-xs font-semibold"
+              title={t.editor.wordBands[band].en}
+              style={{
+                background: 'var(--color-surface-alt)',
+                color: band === 'advanced' ? 'var(--color-accent-hover)' : 'var(--color-muted)',
+              }}
+            >
+              {t.editor.wordBands[band].native}
+            </span>
+          )}
+        </div>
       )}
 
       {/* Chinese gloss first — it's what the Mandarin-speaking writer reaches for. */}
@@ -165,6 +184,21 @@ export function WordCard({ word, info, loading, saved, onToggleSave, style, onRe
         </div>
       )}
 
+      {/* Where the word came from. Last, and in small muted type, because it is
+          the one thing here that is interesting rather than useful — and for a
+          writer whose own language shares Latin roots with English, "efémero"
+          sitting under "ephemeral" is how a word stops needing to be memorised. */}
+      {etymology && (
+        <div className="mt-3">
+          <p className="mb-1 text-xs font-bold" style={{ color: 'var(--color-muted)' }}>
+            {t.editor.origin}
+          </p>
+          <p className="text-xs leading-snug" style={{ color: 'var(--color-muted)' }}>
+            {etymology}
+          </p>
+        </div>
+      )}
+
       {empty && (
         <p className="mt-3 leading-snug" style={{ color: 'var(--color-muted)' }}>
           {t.editor.nothingFound}

web/src/components/Editor/wordband.test.ts

@@ -0,0 +1,41 @@
+import { describe, expect, it } from 'vitest'
+import { wordBand } from './wordband'
+
+describe('wordBand', () => {
+  it('says nothing when the dictionary knows nothing', () => {
+    // The embedded datasets carry no scores at all, and this is the common
+    // case for a zh-pair writer. An unknown word must produce no chip rather
+    // than a default one — "standard" would be an invention.
+    expect(wordBand(0, -1)).toBeNull()
+  })
+
+  it('bands the words a writer actually looks up', () => {
+    // Real scores from the deployed dict.db. These are the sanity checks that
+    // would catch a threshold drifting away from the data.
+    expect(wordBand(1000, 0.304)).toBe('simple') // cat
+    expect(wordBand(1000, 0.326)).toBe('simple') // house, write
+    expect(wordBand(1000, 0.37)).toBe('simple') // beautiful
+    expect(wordBand(600, 0.462)).toBe('standard') // ephemeral
+    expect(wordBand(400, 0.53)).toBe('standard') // serendipity
+    expect(wordBand(50, 0.8)).toBe('advanced') // antidisestablishmentarianism
+  })
+
+  it('prefers difficulty over frequency when both are known', () => {
+    // Difficulty is the finer signal — 206 distinct values against frequency's
+    // handful of buckets — so a hard word with a high frequency reads as hard.
+    expect(wordBand(1000, 0.9)).toBe('advanced')
+    expect(wordBand(2, 0.2)).toBe('simple')
+  })
+
+  it('falls back to frequency for a word with no difficulty score', () => {
+    expect(wordBand(1000, -1)).toBe('simple')
+    expect(wordBand(500, -1)).toBe('standard')
+    expect(wordBand(10, -1)).toBe('advanced')
+  })
+
+  it('treats a difficulty of exactly zero as a score, not as missing', () => {
+    // 0.0 is the easiest word there is. The API sends -1 for unknown precisely
+    // so this case survives; a falsy check here would throw it away.
+    expect(wordBand(0, 0)).toBe('simple')
+  })
+})

web/src/components/Editor/wordband.ts

@@ -0,0 +1,53 @@
+// How hard is this word?
+//
+// DreamDict scores every headword for difficulty (0.0 easiest → 1.0 hardest,
+// computed from frequency, length and syllable count) and for raw frequency.
+// Neither number means anything to a writer — 0.462 is not an answer to "should
+// I use this word?" — so the popover shows a band instead.
+//
+// Like gradeBand in the status bar, this returns the band's *name*. The wording
+// belongs to the writer's langpack.
+
+export type WordBand = 'simple' | 'standard' | 'advanced'
+
+// The cuts come from the real dictionary rather than from round numbers. Across
+// 136k English headwords the difficulty scores bunch between 0.45 and 0.60 —
+// that band is the fifth percentile to the eightieth — because a dictionary is
+// mostly words nobody uses. The words a writer actually reaches for sit well
+// below it: the, cat, house, write and beautiful all score under 0.40.
+//
+// Three bands, not five. The score is a heuristic over word length and corpus
+// counts, and it is good enough to separate "everyday" from "you will need to
+// explain this" — not good enough to rank obfuscate against serendipity. A
+// finer scale would be a confident-looking lie.
+const SIMPLE_MAX = 0.42
+const STANDARD_MAX = 0.58
+
+// Frequency is the fallback for a word DreamDict carries with no difficulty
+// score. It is heavily bucketed in the data (400/600/800/1000 hold most of the
+// mass), so these compare against buckets, not a continuum.
+const FREQ_SIMPLE_MIN = 800
+const FREQ_STANDARD_MIN = 400
+
+/**
+ * wordBand picks the band for a looked-up word, or null when the dictionary
+ * knows nothing about how hard it is — the embedded datasets never do, and
+ * neither does DreamDict for a word it has no score for. Null means the chip
+ * doesn't render at all: silence is better than a guess dressed as a fact.
+ *
+ * `difficulty` is -1 when unknown and `frequency` is 0 when unknown, matching
+ * what the API sends.
+ */
+export function wordBand(frequency: number, difficulty: number): WordBand | null {
+  if (difficulty >= 0) {
+    if (difficulty <= SIMPLE_MAX) return 'simple'
+    if (difficulty <= STANDARD_MAX) return 'standard'
+    return 'advanced'
+  }
+  if (frequency > 0) {
+    if (frequency >= FREQ_SIMPLE_MIN) return 'simple'
+    if (frequency >= FREQ_STANDARD_MIN) return 'standard'
+    return 'advanced'
+  }
+  return null
+}