llm/ollama: disable thinking so reasoning models return content
Qwen 3.5 — the spec's recommended model — is a reasoning model. With thinking on, Ollama streams its chain-of-thought into a separate `thinking` field and hits num_predict before emitting any answer into `content`, so Complete() got an empty string and the checkpoint failed with "no JSON object in model output". Sending `"think": false` on every /api/chat request fixes it; non-thinking models (qwen2.5) ignore the flag. Validated end-to-end on deployment hardware (Ollama, qwen3.5:9b): the grammar checkpoint now caught all five ESL errors in a 3-sentence sample with correct JSON and string-anchoring, ~8.5s warm. Claude-Session: https://claude.ai/code/session_016Yr6jELuRc7hyzYLccQKZd
This commit is contained in:
@@ -16,10 +16,17 @@ type OllamaClient struct {
|
||||
|
||||
// ollamaRequest mirrors Ollama's /api/chat body. Sampling parameters live under
|
||||
// "options" (see the cheatsheet in the spec).
|
||||
//
|
||||
// Think is sent false unconditionally: reasoning models (e.g. qwen3.5) otherwise
|
||||
// stream their chain-of-thought into a separate "thinking" field and can exhaust
|
||||
// num_predict before emitting any answer in "content" — leaving us an empty
|
||||
// response. We want direct output (structured JSON for the checkpoint, concise
|
||||
// prose for chat), not reasoning. Non-thinking models simply ignore the flag.
|
||||
type ollamaRequest struct {
|
||||
Model string `json:"model"`
|
||||
Messages []Message `json:"messages"`
|
||||
Stream bool `json:"stream"`
|
||||
Think bool `json:"think"`
|
||||
Options ollamaOptions `json:"options"`
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user