prosolis 2045cc2ada Add loader tests and fix header detection and punctuation filtering ...

Tests: 23 new tests across cetempublico, affix, and wiktionary loaders
covering column order detection, ISO-8859-1 encoding, junk filtering,
frequency scaling, affix expansion, and sense-level synonym extraction.

Fixes: header detection now checks either field for numeric value (was
only checking last field, breaking count-first format). Added hasLetter
filter to reject pure punctuation entries like "." and ",".

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

2026-04-03 09:15:13 -07:00

7.0 KiB

Raw Permalink Blame History

View Raw