Consolidate word filtering into shared isJunkWord helper

Replace copy-pasted hasLetter/containsDigit/containsSpace/containsNonLatin
checks across 9 loaders with a single isJunkWord() call. SCOWL and CEDICT
keep their own filtering for valid reasons (SCOWL rejects hyphens/apostrophes,
CEDICT handles non-Latin Chinese characters).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
prosolis
2026-04-03 20:01:02 -07:00
parent aff583dc28
commit 28ec11ba4e
10 changed files with 15 additions and 10 deletions

View File

@@ -158,3 +158,8 @@ func hasLetter(s string) bool {
}
return false
}
// isJunkWord returns true if the word should be rejected from the dictionary.
func isJunkWord(word string) bool {
return word == "" || !hasLetter(word) || containsDigit(word) || containsSpace(word) || containsNonLatin(word)
}