prosolis 8bd2509bc2 Checkpoint: raise output cap to 4096 + salvage truncated JSON ...

The grammar checkpoint capped num_predict at 1024, but qwen3.5:9b ignores
the prompt's "up to 5 issues" and emits ~17-20 suggestions (~2000 tokens)
on a 300+ word doc. The output hit the cap mid-array (done_reason=length),
the JSON never closed, and ParseCheckpoint found no parseable object -> a
502 in ~20s on every long doc (1024 tok @ ~50 tok/s, not a network
timeout). The repeated failures made the writing-assist helper look
permanently asleep.

Fix:
- Raise the cap to checkpointMaxTokens=4096. It is a ceiling, not a
  target -- the model stops at its JSON close, so shorter docs are
  unaffected; only genuinely long outputs use the headroom.
- Make ParseCheckpoint salvage the completed {...} suggestion objects
  from a truncated array (refactor extractJSONObject onto a shared
  firstBalancedObject scanner), so an over-long doc degrades to partial
  feedback instead of a hard 502.

Verified live on millenia: 'The Missing Key' (382 words) now returns 200
with 17 suggestions in ~38s, previously 502 every time.

Claude-Session: https://claude.ai/code/session_016Yr6jELuRc7hyzYLccQKZd

2026-06-26 14:16:07 -07:00

3.8 KiB

Raw Blame History

View Raw