Petal learns French, and the pack that shipped was misspelling itself
Phase 24, the fr half: langpack, Hunspell dictionary, Piper voice, and the lexicon coverage that turned out to have been measured already (63.1%, better than pt-PT's 62.1%). No migration; not deployed. The plan recorded that build_ptpt_dictionary.py "generalizes" to French. It did not. It handled single-character flags and plain PFX/SFX and stopped on everything else, and fr.aff uses four of the things it stopped on. FLAG long is the dangerous one: French flags are two characters, so the old reader's set(flagstr) yields a bag of unrelated letters and expands every entry through the wrong paradigm without ever erroring. Plus continuation flags (French really does affix an affixed form), NEEDAFFIX on 68,075 of 84,140 stems, and FULLSTRIP. Renamed build_hunspell_dictionary.py with a per-language profile, asserting that CIRCUMFIX and FORBIDDENWORD are still unused rather than assuming it — and it rebuilds pt-PT byte-identical to the shipped asset, which is the only thing that makes "generalized" a claim rather than a hope. Elision was decided by building both halves and measuring. Keeping l'arbre and its thirty-three siblings: 3,159,832 forms, 8.25 MB gzipped. Dropping them: 473,326 and 1.19 MB. They are not new words, but the tokenizer keeps internal apostrophes, so they genuinely would have been underlined — so they moved out of the dictionary into withElision, which splits at a known clitic and still requires the remainder to be a word (l'zzzz stays flagged). Real nspell: 369 ms and 74 MB, against pt-PT's 842 ms and 139 MB, on the larger language. Where the regional trap lives is the mirror image of Portuguese's: every fr_* Piper voice is fr_FR and Debian's fr_FR/fr_CA/fr_BE dictionaries are one shared word list, so nothing can be quietly wrong about the country and the whole decision sits in the copy. What French has instead is the 1990 reform, packaged three ways; comprehensive ships, because Petal never corrects her French and coût and cout are both correct. Then the interim review pass, at the user's suggestion and explicitly "for now": four models read each Latin pack independently, and only findings at least two of them reached on their own were applied — five per pack. It earned its keep on the pack that was already live. pt-PT was carrying pre-Acordo spellings (adjectivos, actualmente) in a file whose own header commits to post-Acordo, plus Brazilian decepção, because the Phase 21 greps checked for Brazilian vocabulary and never checked the pack against its own spelling policy. That grep now exists and was confirmed to fail on the old text before being kept. Where reviewers agreed a line was wrong but split on the fix, the wording is mine and the reasoning is in BUILD_PLAN rather than averaged away. Still owed, and both packs now say so precisely: a quorum of models agreeing is agreement, not authority. No native speaker has read either pack, and none of this has been seen in a browser. go build/vet/test clean, tsc, vite build, vitest 190/190. Claude-Session: https://claude.ai/code/session_016y6gyuHkQXPiEuW8RGQyua
This commit is contained in:
prosolis
@@ -0,0 +1,432 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Build one of Petal's browser spelling dictionaries from a Hunspell one.
|
||||
|
||||
Why this script exists at all
|
||||
----------------------------
|
||||
English is vendored the obvious way: `dictionary-en`'s `en.aff` + `en.dic` go
|
||||
into web/public/dictionaries/en and nspell reads them in the browser. The plan
|
||||
for Phase 21 said "Hunspell pt-PT vendored like en-US", and that turns out not to
|
||||
work, for a measured reason.
|
||||
|
||||
nspell expands affixes **eagerly at load time** — it materialises every surface
|
||||
form into a hash the moment you construct it. English gets away with this: ~50k
|
||||
stems and a small rule set. European Portuguese does not. `pt_PT.aff` carries
|
||||
1,340 affix rules (the full verb paradigm: six persons x a dozen tenses, plus
|
||||
diminutives, plus productive prefixes) over 44,257 stems. Measured on this
|
||||
machine, nspell needed ~340 MB of heap for the first 12,000 entries alone and had
|
||||
not returned after three minutes on the whole file; extrapolated, it wants well
|
||||
over a gigabyte. That is not something to hand a browser, still less a tablet.
|
||||
French is larger again: 5,600 affix rules over 84,140 stems.
|
||||
|
||||
So the expansion happens **here**, once, at build time, and the browser gets a
|
||||
flat word list it can load with no affix machinery at all. The runtime code path
|
||||
is then *identical* to English — same nspell, same interface — which is the real
|
||||
prize. The aff shipped alongside keeps only the suggestion-shaping directives
|
||||
(TRY/KEY/REP/MAP), so corrections still know that "cao" wants "ção" and that a
|
||||
missing acute accent is a near miss.
|
||||
|
||||
What Phase 24 had to add
|
||||
------------------------
|
||||
The build plan recorded that the pt-PT version of this script "generalizes" to
|
||||
French. It did not. It handled single-character flags and plain PFX/SFX and
|
||||
stopped on anything else — the right call, because `fr.aff` uses four of the
|
||||
things it stopped on, and getting any of them wrong changes which words are
|
||||
accepted:
|
||||
|
||||
* **`FLAG long`** — French flags are *two characters* (`S.`, `L'`, `Um`). The
|
||||
pt-PT reader took `set(flagstr)`, one flag per character, which on a French
|
||||
entry yields a bag of unrelated single letters: every entry would have been
|
||||
expanded through the wrong paradigm. This is the one that fails silently.
|
||||
* **Continuation flags** — pt-PT's affixes append plain text, so that script
|
||||
dropped anything after a `/` and asserted the drop was safe. French really
|
||||
does affix an affixed form: `PFX Um 0 0/S.` says the prefixed form then takes
|
||||
the plural suffix, and the elision prefixes arrive the same way from the
|
||||
other side (`SFX ... ait/n'q'l'm't's'`).
|
||||
* **`NEEDAFFIX`** — French marks thousands of stems "not a word on its own"
|
||||
(`Allemagne/S.()`), the bare form arriving instead through a zero-append
|
||||
rule. Ignoring the flag accepts stems the real dictionary rejects.
|
||||
* **`FULLSTRIP`** — a rule may strip the whole stem.
|
||||
|
||||
`CIRCUMFIX` and `FORBIDDENWORD` are *declared* in `fr.aff` and used by nothing,
|
||||
which this script asserts rather than assumes: an upstream release that started
|
||||
using either would otherwise change what is accepted without changing this file.
|
||||
`KEEPCASE` and `NOSUGGEST` are honoured by being ignored on purpose — they shape
|
||||
casing and suggestions, not membership, and a NOSUGGEST word is still a word.
|
||||
|
||||
Elision is handled at lookup, not here — and that is the size decision
|
||||
----------------------------------------------------------------------
|
||||
Most of French's affix machinery by volume is elision: `l'`, `d'`, `qu'`, `j'`,
|
||||
`n'`, `s'`, `jusqu'`, `puisqu'`. Hunspell treats `l'arbre` as one word, so a
|
||||
faithful expansion carries much of the language thirty-four times over — and
|
||||
Petal's tokenizer keeps internal apostrophes, so `l'arbre` really does arrive at
|
||||
the dictionary as one token and really would be underlined if it were absent.
|
||||
|
||||
Both halves were built and measured. Keeping the elided forms: **3,159,832 forms,
|
||||
8.25 MB gzipped**, ~45 MB of text for nspell to hash on a tablet. Dropping them:
|
||||
**473,326 forms, 1.19 MB gzipped**. The elided seven-eighths are not new words —
|
||||
they are thirteen little words glued to words already in the list — so the third
|
||||
option is the one taken: rules whose append carries an apostrophe are skipped
|
||||
here (the count is printed), and `withElision` in `useSpellChecker.ts` splits a
|
||||
token at a *known clitic* and checks the remainder. `l'arbre` costs one extra
|
||||
lookup instead of seven megabytes, and `zzz'arbre` is still flagged because
|
||||
`zzz` is not one of the thirteen.
|
||||
|
||||
Stems that carry an apostrophe of their own — `aujourd'hui`, `quelqu'un`,
|
||||
`presqu'île`, `prud'homme` — are dictionary entries rather than affixed forms,
|
||||
so they are kept verbatim and matched directly. `entr'aide` and `grand'mère` are
|
||||
absent for the same reason they are absent from Dicollecte: modern French spells
|
||||
them `entraide` and `grand-mère`.
|
||||
|
||||
Choosing the source
|
||||
-------------------
|
||||
Both languages have a trap here, and they are different traps.
|
||||
|
||||
**pt-PT: the wrong country.** npm's `dictionary-pt` is not European Portuguese.
|
||||
Both it and `dictionary-pt-br` package VERO ("Verificador Ortográfico Livre",
|
||||
Brasil), so vendoring the obvious npm name would have shipped Brazilian spellings
|
||||
under a pt-PT label — the pt-BR drift SUGGESTIONS.md §3 warns about, arriving
|
||||
through the packaging rather than through the model. The authentic dictionary is
|
||||
the Projecto Natura one (Universidade do Minho) that LibreOffice ships and Debian
|
||||
packages as `hunspell-pt-pt`; its aff declares `LANG pt_PT`.
|
||||
|
||||
**fr: the wrong side of an argument the French have not settled.** The regional
|
||||
question turns out to be a non-question — Debian's `fr_FR`, `fr_CA`, `fr_BE`,
|
||||
`fr_CH`, `fr_LU` and `fr_MC` are all symlinks to one `fr.dic`, so unlike pt there
|
||||
is no country here to get wrong. What there is instead is the 1990 spelling
|
||||
reform, packaged three ways: `hunspell-fr-classical` (traditional), `-revised`
|
||||
(reform only) and `-comprehensive` (both). Petal ships **comprehensive**, because
|
||||
Petal never corrects her French — the only thing this dictionary can do is
|
||||
underline something. *coût* and *cout* are both correct French, taught in
|
||||
different decades to different people, and a writing companion has no business
|
||||
underlining one of them to take a side. The `fr` MUST_ACCEPT list is written to
|
||||
*prove* which package was used: classical rejects `cout`, revised rejects `coût`,
|
||||
and only comprehensive accepts both.
|
||||
|
||||
Licensing: pt-PT is GPL-2 or LGPL-2.1 or MPL-1.1, (c) José João de Almeida, Rui
|
||||
Vilela, Alberto Simões. fr is MPL-2.0, (c) 2007-2018 the Dicollecte contributors
|
||||
(grammalecte.net). The upstream copyright file is vendored beside each output.
|
||||
|
||||
Usage
|
||||
-----
|
||||
apt-get download hunspell-fr-comprehensive # or hunspell-pt-pt
|
||||
dpkg-deb -x hunspell-fr-comprehensive_*.deb src
|
||||
python3 scripts/build_hunspell_dictionary.py fr \\
|
||||
src/usr/share/hunspell/fr.aff \\
|
||||
src/usr/share/hunspell/fr.dic \\
|
||||
web/public/dictionaries/fr
|
||||
"""
|
||||
import gzip
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
import unicodedata
|
||||
|
||||
# Directives worth keeping in the shipped aff. These shape *suggestions*, not
|
||||
# membership: TRY orders the alphabet the corrector tries, KEY knows which keys
|
||||
# are adjacent, REP holds the language's own confusions (cao/ção, ss/ç), and MAP
|
||||
# says an accented vowel and its bare form are the same letter for scoring —
|
||||
# which is most of what an ESL writer gets wrong in either language.
|
||||
KEEP_DIRECTIVES = ("SET", "TRY", "KEY", "REP", "MAP", "WORDCHARS")
|
||||
|
||||
# Directives that would change which words are *accepted* and that this expander
|
||||
# does not implement. If a future upstream release starts using one, the output
|
||||
# would silently disagree with the real dictionary, so the build stops instead.
|
||||
UNSUPPORTED = (
|
||||
"COMPOUNDFLAG", "COMPOUNDMIN", "COMPOUNDRULE", "COMPOUNDBEGIN",
|
||||
"ONLYINCOMPOUND", "PSEUDOROOT", "AF", "AM",
|
||||
)
|
||||
|
||||
APOSTROPHES = "'’"
|
||||
|
||||
|
||||
class Aff:
|
||||
"""The parts of an .aff file that decide which words exist."""
|
||||
|
||||
def __init__(self):
|
||||
self.pfx = {} # flag -> [(strip, append, condition, continuation)]
|
||||
self.sfx = {}
|
||||
self.cross = {} # flag -> bool
|
||||
self.flag_kind = "char"
|
||||
self.needaffix = None
|
||||
self.circumfix = None
|
||||
self.forbidden = None
|
||||
self.dropped_apostrophe_rules = 0
|
||||
|
||||
|
||||
def parse_flags(raw, kind):
|
||||
"""Split a flag string into flags, per the aff's FLAG declaration."""
|
||||
raw = raw.strip()
|
||||
if not raw:
|
||||
return set()
|
||||
if kind == "long":
|
||||
return {raw[i:i + 2] for i in range(0, len(raw) - 1, 2)}
|
||||
if kind == "num":
|
||||
return {f for f in raw.split(",") if f}
|
||||
return set(raw)
|
||||
|
||||
|
||||
def parse_aff(path):
|
||||
with open(path, encoding="utf-8") as fh:
|
||||
lines = fh.read().splitlines()
|
||||
|
||||
aff = Aff()
|
||||
|
||||
# FLAG has to be known before anything containing a flag is read, and it can
|
||||
# sit anywhere in the file. So: header pass first, rules second.
|
||||
for line in lines:
|
||||
parts = line.split()
|
||||
if not parts:
|
||||
continue
|
||||
head = parts[0]
|
||||
if head in UNSUPPORTED:
|
||||
raise SystemExit(
|
||||
f"{path}: unsupported directive {head!r} — this expander handles "
|
||||
"PFX/SFX affixation with continuation flags, and honouring "
|
||||
f"{head} would change which words are accepted. Extend the "
|
||||
"script before shipping."
|
||||
)
|
||||
if len(parts) < 2:
|
||||
continue
|
||||
if head == "FLAG":
|
||||
aff.flag_kind = parts[1]
|
||||
if aff.flag_kind not in ("long", "num", "UTF-8"):
|
||||
raise SystemExit(f"{path}: unknown FLAG type {aff.flag_kind!r}")
|
||||
elif head == "NEEDAFFIX":
|
||||
aff.needaffix = parts[1]
|
||||
elif head == "CIRCUMFIX":
|
||||
aff.circumfix = parts[1]
|
||||
elif head == "FORBIDDENWORD":
|
||||
aff.forbidden = parts[1]
|
||||
|
||||
i = 0
|
||||
while i < len(lines):
|
||||
parts = lines[i].split()
|
||||
if parts and parts[0] in ("PFX", "SFX"):
|
||||
kind, flag, cross_flag, count = parts[0], parts[1], parts[2], int(parts[3])
|
||||
table = aff.pfx if kind == "PFX" else aff.sfx
|
||||
aff.cross[flag] = cross_flag == "Y"
|
||||
rules = table.setdefault(flag, [])
|
||||
for j in range(1, count + 1):
|
||||
p = lines[i + j].split()
|
||||
strip = "" if p[2] == "0" else p[2]
|
||||
append, _, cont_raw = p[3].partition("/")
|
||||
if append == "0":
|
||||
append = ""
|
||||
# Elision. See the header: these forms are `l'` and its twelve
|
||||
# siblings glued to words already in the list, they multiply the
|
||||
# download by seven, and `withElision` reconstructs them at
|
||||
# lookup for the cost of one extra hash probe.
|
||||
if any(a in append for a in APOSTROPHES):
|
||||
aff.dropped_apostrophe_rules += 1
|
||||
continue
|
||||
cont = parse_flags(cont_raw, aff.flag_kind)
|
||||
if aff.circumfix and aff.circumfix in cont:
|
||||
raise SystemExit(
|
||||
f"{path}: CIRCUMFIX is used by a {kind} {flag} rule. It "
|
||||
"was declared-but-unused when this expander was written "
|
||||
"and is not implemented; honouring it would change which "
|
||||
"words are accepted."
|
||||
)
|
||||
cond = p[4] if len(p) > 4 else "."
|
||||
anchored = ("^" + cond) if kind == "PFX" else (cond + "$")
|
||||
rules.append((strip, append, re.compile(anchored), cont))
|
||||
i += count + 1
|
||||
continue
|
||||
i += 1
|
||||
return aff
|
||||
|
||||
|
||||
def apply_suffix(word, rules):
|
||||
"""Every (form, continuation flags) a suffix table yields for `word`."""
|
||||
out = []
|
||||
for strip, append, cond, cont in rules:
|
||||
if strip and not word.endswith(strip):
|
||||
continue
|
||||
if not cond.search(word):
|
||||
continue
|
||||
stem = word[: len(word) - len(strip)] if strip else word
|
||||
out.append((stem + append, cont))
|
||||
return out
|
||||
|
||||
|
||||
def apply_prefix(word, rules):
|
||||
out = []
|
||||
for strip, append, cond, cont in rules:
|
||||
if strip and not word.startswith(strip):
|
||||
continue
|
||||
if not cond.search(word):
|
||||
continue
|
||||
stem = word[len(strip):] if strip else word
|
||||
out.append((append + stem, cont))
|
||||
return out
|
||||
|
||||
|
||||
def expand_entry(word, flags, aff, out):
|
||||
"""Add every surface form of one dictionary entry to `out`.
|
||||
|
||||
Hunspell's model without compounding: a form is the stem plus at most one
|
||||
prefix and at most one suffix. A flag reaches an affix either from the stem's
|
||||
own flags or from the continuation flags of the affix applied on the other
|
||||
side; when both sides apply, both rules must be declared cross-product.
|
||||
|
||||
NEEDAFFIX is why the bare form is not simply added: the flag says *this* form
|
||||
is not a word, only whatever can be built from it — and it arrives both on
|
||||
stems and on continuations.
|
||||
"""
|
||||
def is_word(carried):
|
||||
return not (aff.needaffix and aff.needaffix in carried)
|
||||
|
||||
if is_word(flags):
|
||||
out.add(word)
|
||||
|
||||
suffixed = [] # (form, flag, continuation flags)
|
||||
for f in flags:
|
||||
if f in aff.sfx:
|
||||
for form, cont in apply_suffix(word, aff.sfx[f]):
|
||||
suffixed.append((form, f, cont))
|
||||
if is_word(cont):
|
||||
out.add(form)
|
||||
|
||||
prefixed = []
|
||||
for f in flags:
|
||||
if f in aff.pfx:
|
||||
for form, cont in apply_prefix(word, aff.pfx[f]):
|
||||
prefixed.append((form, f, cont))
|
||||
if is_word(cont):
|
||||
out.add(form)
|
||||
|
||||
# Prefix then suffix. The suffix flag may come from the stem or from the
|
||||
# prefix's own continuation (`PFX Um 0 0/S.`), and the suffix condition is
|
||||
# matched against the whole prefixed word, which is what hunspell does.
|
||||
for form, pf, pcont in prefixed:
|
||||
if not aff.cross.get(pf):
|
||||
continue
|
||||
for f in flags | pcont:
|
||||
if f in aff.sfx and aff.cross.get(f):
|
||||
for full, _ in apply_suffix(form, aff.sfx[f]):
|
||||
out.add(full)
|
||||
|
||||
# Suffix then prefix — the same pair reached from the other side, which is
|
||||
# how the elision prefixes arrive in French. Only the flags the suffix hands
|
||||
# forward are new here; the stem's own were covered above.
|
||||
for form, sf, scont in suffixed:
|
||||
if not aff.cross.get(sf):
|
||||
continue
|
||||
for f in scont:
|
||||
if f in aff.pfx and aff.cross.get(f):
|
||||
for full, _ in apply_prefix(form, aff.pfx[f]):
|
||||
out.add(full)
|
||||
|
||||
|
||||
def expand(aff_path, dic_path):
|
||||
aff = parse_aff(aff_path)
|
||||
forms = set()
|
||||
needaffix_stems = 0
|
||||
with open(dic_path, encoding="utf-8") as fh:
|
||||
fh.readline() # leading entry count, not a word
|
||||
for raw in fh:
|
||||
# Morphological fields (po:nom is:fem) follow the entry, separated by
|
||||
# a tab in pt-PT and by a space in fr.
|
||||
entry = raw.strip().split("\t")[0].split(" ")[0]
|
||||
if not entry:
|
||||
continue
|
||||
word, _, flagstr = entry.partition("/")
|
||||
word = word.strip()
|
||||
if not word:
|
||||
continue
|
||||
flags = parse_flags(flagstr, aff.flag_kind)
|
||||
if aff.forbidden and aff.forbidden in flags:
|
||||
raise SystemExit(
|
||||
f"{dic_path}: FORBIDDENWORD is in use ({word!r}). It was "
|
||||
"declared-but-unused when this expander was written; the "
|
||||
"forms it removes would be wrongly accepted."
|
||||
)
|
||||
if aff.needaffix and aff.needaffix in flags:
|
||||
needaffix_stems += 1
|
||||
expand_entry(word, flags, aff, forms)
|
||||
|
||||
# NFC, because the aff's own ICONV table normalises decomposed accents on the
|
||||
# way in and the browser hands nspell whatever the keyboard produced.
|
||||
forms = {unicodedata.normalize("NFC", f) for f in forms}
|
||||
return forms, aff, needaffix_stems
|
||||
|
||||
|
||||
def shipped_aff(aff_path):
|
||||
keep = []
|
||||
for line in open(aff_path, encoding="utf-8").read().splitlines():
|
||||
head = line.split()[0] if line.split() else ""
|
||||
if head in KEEP_DIRECTIVES:
|
||||
keep.append(line)
|
||||
return "\n".join(keep) + "\n"
|
||||
|
||||
|
||||
# Words the built list must accept, and must reject, before it is written. Each
|
||||
# set is chosen to fail loudly on the *specific* wrong source that language has a
|
||||
# packaged, plausible way of reaching — not to spot-check spelling in general.
|
||||
PROFILES = {
|
||||
# The pt-PT/pt-BR fault lines: post-Acordo spellings, the European lexicon,
|
||||
# and the first-person-plural preterite accent that only pt-PT writes.
|
||||
"pt-PT": {
|
||||
"accept": ("receção", "húmido", "telemóvel", "autocarro", "comboio",
|
||||
"ótimo", "pensámos", "escrevêssemos", "jardim"),
|
||||
"reject": ("recepção", "úmido", "ônibus", "óptimo"),
|
||||
"wrong": "this does not look like European Portuguese",
|
||||
},
|
||||
# Which of the three 1990-reform packagings this is. `coût`/`cout` and
|
||||
# `paraître`/`paraitre` are each accepted by exactly one of classical and
|
||||
# revised, so a build accepting all four is comprehensive and one that drops
|
||||
# any of them is not. `Allemagne` is a NEEDAFFIX stem reachable only through
|
||||
# a zero-append rule and `km` only through a prefix continuation, so between
|
||||
# them they also check that this expander honoured the two features the
|
||||
# pt-PT one refused.
|
||||
"fr": {
|
||||
"accept": ("coût", "cout", "paraître", "paraitre", "nénuphar", "nénufar",
|
||||
"oignon", "ognon", "événement", "évènement", "jardin",
|
||||
"Allemagne", "écrivissions", "km"),
|
||||
"reject": ("jardinn", "écrivaitz", "xyzzyque"),
|
||||
"wrong": "this does not look like the comprehensive French dictionary",
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def main(lang, aff_path, dic_path, out_dir):
|
||||
profile = PROFILES.get(lang)
|
||||
if profile is None:
|
||||
raise SystemExit(f"no profile for {lang!r}; known: {', '.join(PROFILES)}")
|
||||
|
||||
forms, aff, needaffix_stems = expand(aff_path, dic_path)
|
||||
|
||||
missing = [w for w in profile["accept"] if w not in forms]
|
||||
present = [w for w in profile["reject"] if w in forms]
|
||||
if missing or present:
|
||||
raise SystemExit(
|
||||
f"{profile['wrong']}: missing {missing}, unexpectedly present {present}"
|
||||
)
|
||||
|
||||
os.makedirs(out_dir, exist_ok=True)
|
||||
ordered = sorted(forms)
|
||||
body = f"{len(ordered)}\n" + "\n".join(ordered) + "\n"
|
||||
|
||||
dic_out = os.path.join(out_dir, f"{lang}.dic.gz")
|
||||
# mtime=0 so rebuilding identical input produces an identical file — a
|
||||
# vendored asset that changes on every build is noise in the diff.
|
||||
with gzip.GzipFile(dic_out, "wb", compresslevel=9, mtime=0) as fh:
|
||||
fh.write(body.encode("utf-8"))
|
||||
|
||||
aff_out = os.path.join(out_dir, f"{lang}.aff")
|
||||
with open(aff_out, "w", encoding="utf-8") as fh:
|
||||
fh.write(shipped_aff(aff_path))
|
||||
|
||||
print(f"{len(ordered)} forms -> {dic_out} "
|
||||
f"({os.path.getsize(dic_out) / 1e6:.2f} MB gzipped); "
|
||||
f"{needaffix_stems} NEEDAFFIX stems, "
|
||||
f"{aff.dropped_apostrophe_rules} elision rules skipped", file=sys.stderr)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
if len(sys.argv) != 5:
|
||||
raise SystemExit(
|
||||
"usage: build_hunspell_dictionary.py <lang> <aff> <dic> <out-dir>\n"
|
||||
f" lang is one of: {', '.join(PROFILES)}"
|
||||
)
|
||||
main(*sys.argv[1:5])
|
||||
Reference in New Issue
Block a user