`pair_lang` had always been answering a second question nobody asked: it says which two languages, and every surface built on it assumed English was the one being learned. That is why hanzi is never tokenized, never spell-checked, never glossed — correct for a Mandarin native practising English, backwards for an English native practising Mandarin. `users.direction` (migration 0016) separates the two questions; a `zh-learner` pair code would have been cheaper and would have made two directions of one pair look like two unrelated languages to every query. Segmentation is what replaces `wordAt` where there are no spaces: a shortest-path walk over log-probabilities, 232 ms and 14 MB for 188,522 words. The browser gets the word list because segmentation runs on hover; the server keeps the whole dictionary. Their coverage gates come out opposite on purpose — the client list is frequency-gated because the segmentation is measurably identical without the tail, and the dictionary is gated by nothing, because its only power is to explain and the word a learner stops on is the rare one. The 错别字 pack is 24 confusable pairs behind two mechanical gates. One admits a pair only if the wrong form is not a dictionary word and the right form is, which is why it refuses 自已 for 自己 — a real error whose wrong form is a headword. The other asks the segmenter whether the two characters already belong to two different words, without which 自己经常, 睡觉的时候 and 不知到底 would all be corrupted silently into text still made of real characters. Not deployed (this carries a migration), not seen in a browser, and no account has ever been in the learner direction. The IME composition guards were in scope and are not done — see BUILD_PLAN Phase 26.
156 lines
7.1 KiB
Go
156 lines
7.1 KiB
Go
package db
|
|
|
|
import "time"
|
|
|
|
// User is an account. Its ID is the OIDC subject for anyone who signed in, or
|
|
// LocalUserID for the pre-auth single user (and for local development, where
|
|
// StaticResolver still hands out that id).
|
|
type User struct {
|
|
ID string `json:"id"`
|
|
Email string `json:"email"`
|
|
DisplayName string `json:"display_name"`
|
|
CreatedAt time.Time `json:"created_at"`
|
|
|
|
// PairLang is the X in this writer's (English + X) language pair — "zh"
|
|
// today, "pt-PT"/"fr"/"es" once the langpacks land. It selects the UI copy
|
|
// and dictionary set, not the language they may type in.
|
|
PairLang string `json:"pair_lang"`
|
|
|
|
// Direction says which half of the pair is being *learned*. Every pair until
|
|
// now assumed one answer: the writer is native in X and practising English,
|
|
// so hanzi is never tokenized and English is what gets underlined. Turn it
|
|
// around — a native English speaker learning Chinese — and the same pair
|
|
// wants the opposite of nearly every default.
|
|
//
|
|
// It is a separate column from PairLang rather than a second pair code
|
|
// ("zh-learner") because it is a genuinely separate question: the pair says
|
|
// *which two languages*, this says *which way round*. Keeping them apart is
|
|
// what lets fr, es and pt-PT inherit the learner direction later without a
|
|
// second langpack each.
|
|
Direction string `json:"direction"`
|
|
}
|
|
|
|
// Document is a single piece of writing. `Content` is the Tiptap JSON document
|
|
// (source of truth for the editor); `ContentText` is the flattened plain text
|
|
// kept in sync on every save and fed to the LLM.
|
|
type Document struct {
|
|
ID string `json:"id"`
|
|
UserID string `json:"user_id"`
|
|
Title string `json:"title"`
|
|
Content string `json:"content"` // Tiptap JSON
|
|
ContentText string `json:"content_text"` // plain text for the LLM
|
|
Tone string `json:"tone"` // target writing tone; steers LLM advice
|
|
WordCount int `json:"word_count"`
|
|
CreatedAt time.Time `json:"created_at"`
|
|
UpdatedAt time.Time `json:"updated_at"`
|
|
|
|
// PreserveHistory opts this document out of auto-snapshot pruning so its
|
|
// full writing trail survives as authorship evidence (see the passport).
|
|
PreserveHistory bool `json:"preserve_history"`
|
|
}
|
|
|
|
// DocumentVersion is a point-in-time snapshot of a document's body, captured so
|
|
// a writer can recover from a bad edit or an unwanted change. `Content` mirrors
|
|
// the document's Tiptap JSON at snapshot time; `Kind` records why it was taken
|
|
// (see the kind constants). List responses omit the heavy Content/ContentText
|
|
// fields (the `omitempty`-friendly zero strings) and load them only on preview
|
|
// or restore.
|
|
type DocumentVersion struct {
|
|
ID string `json:"id"`
|
|
DocID string `json:"doc_id"`
|
|
Title string `json:"title"`
|
|
Content string `json:"content,omitempty"` // Tiptap JSON; omitted in list view
|
|
ContentText string `json:"content_text,omitempty"` // plain text; omitted in list view
|
|
WordCount int `json:"word_count"`
|
|
Kind string `json:"kind"` // auto | manual | pre_restore
|
|
CreatedAt time.Time `json:"created_at"`
|
|
|
|
// ContentHash chains this snapshot to the previous one (PrevHash), so a
|
|
// history that was edited or thinned after the fact fails verification.
|
|
// Both are empty for snapshots taken before the chain existed. Omitted from
|
|
// list responses; the passport loads them explicitly.
|
|
ContentHash string `json:"content_hash,omitempty"`
|
|
PrevHash string `json:"prev_hash,omitempty"`
|
|
}
|
|
|
|
// Document version kinds, mirrored from the schema CHECK constraint.
|
|
const (
|
|
VersionKindAuto = "auto" // throttled background snapshot on save
|
|
VersionKindManual = "manual" // explicit "save a restore point"
|
|
VersionKindPreRestore = "pre_restore" // safety copy taken just before a restore
|
|
)
|
|
|
|
// Tag is a user-scoped label for organizing documents. `Color` is a palette key
|
|
// (rose, mint, peach, lavender, sky, honey) the frontend maps to a CSS color;
|
|
// storing the key (not a hex value) keeps tags in step with the design tokens.
|
|
// `DocCount` is populated only by the tag-list endpoint (how many documents wear
|
|
// the tag); it's omitted from per-document tag lists.
|
|
type Tag struct {
|
|
ID string `json:"id"`
|
|
Name string `json:"name"`
|
|
Color string `json:"color"`
|
|
DocCount int `json:"doc_count,omitempty"`
|
|
}
|
|
|
|
// Tag color palette keys, mirrored on the frontend. Kept small and aligned with
|
|
// the existing design tokens; unknown values fall back to rose client-side.
|
|
const (
|
|
TagColorRose = "rose"
|
|
TagColorMint = "mint"
|
|
TagColorPeach = "peach"
|
|
TagColorLavender = "lavender"
|
|
TagColorSky = "sky"
|
|
TagColorHoney = "honey"
|
|
)
|
|
|
|
// Suggestion is a single LLM-proposed edit anchored to a span of the document.
|
|
//
|
|
// FromPos/ToPos are plaintext offsets into ContentText for server-side use only;
|
|
// the frontend re-anchors by matching the `Original` string in ProseMirror
|
|
// coordinates at render time (spec Note #6). `Replacement` is empty for `voice`
|
|
// flags — those are awareness-only, with no correction to apply.
|
|
type Suggestion struct {
|
|
ID string `json:"id"`
|
|
DocID string `json:"doc_id"`
|
|
FromPos int `json:"from_pos"`
|
|
ToPos int `json:"to_pos"`
|
|
Original string `json:"original"`
|
|
Replacement string `json:"replacement"`
|
|
Explanation string `json:"explanation"`
|
|
Type string `json:"type"` // grammar | phrasing | idiom | clarity | translate | voice | collocation
|
|
Status string `json:"status"` // pending | accepted | rejected
|
|
// Source names the engine that proposed the edit, not its family: an offline
|
|
// rule and the model can both propose a collocation, and the writer is never
|
|
// told which one spoke. It exists so each pass can replace its own rows.
|
|
Source string `json:"source"` // llm | local
|
|
CreatedAt time.Time `json:"created_at"`
|
|
}
|
|
|
|
// Suggestion type and status values, mirrored from the schema CHECK constraints.
|
|
const (
|
|
SuggestionTypeGrammar = "grammar"
|
|
SuggestionTypePhrasing = "phrasing"
|
|
SuggestionTypeIdiom = "idiom"
|
|
SuggestionTypeClarity = "clarity"
|
|
// A span she wrote in her own language, rendered into English. Not a
|
|
// correction — nothing was wrong with it — which is why it is its own type
|
|
// rather than a clarity fix: the card is the pair model's flagship moment
|
|
// (SUGGESTIONS §1), and labelling it "Clarity" reads as a tidy-up of her
|
|
// first language. The model isn't asked for this label; it is derived from the
|
|
// span itself (see suggestions/language.go), so it can't drift.
|
|
SuggestionTypeTranslate = "translate"
|
|
SuggestionTypeVoice = "voice"
|
|
SuggestionTypeCollocation = "collocation"
|
|
SuggestionTypeMechanics = "mechanics" // deterministic rule-based pass (no LLM)
|
|
|
|
// Who proposed it. The offline rule pack ('local') runs on every edit inside
|
|
// the browser and survives a VPN-down box; the model ('llm') adds the long
|
|
// tail when it is reachable.
|
|
SuggestionSourceLLM = "llm"
|
|
SuggestionSourceLocal = "local"
|
|
|
|
SuggestionStatusPending = "pending"
|
|
SuggestionStatusAccepted = "accepted"
|
|
SuggestionStatusRejected = "rejected"
|
|
)
|