Checkpoint: auto-retry after a failed pass; release rate-limit slot on failure
A paste fires exactly one grammar checkpoint, and a failed one never retried until the next keystroke — stranding the writer on "Petal's helper is resting" after a paste. Long docs make it worse: their 15-25s checks have a wide window to catch a transient 502 from the shared Ollama (co-tenant apps load other models and evict the 9B). A failed pass also burned the per-document rate-limit slot, so a retry within 30s hit the throttle path and got an empty set back. - llm.RateLimiter.Release rolls back a slot when its pass fails; Allow now returns the recorded timestamp so Release only frees its own slot. - suggestions.runPass releases the slot on LLM failure before returning 502. - useCheckpoint auto-retries a failed checkpoint with backoff (3/12/35s), keeping the breathing dot up and only flagging "resting" once retries exhaust. Claude-Session: https://claude.ai/code/session_016Yr6jELuRc7hyzYLccQKZd
This commit is contained in:
@@ -102,7 +102,8 @@ func (h *Handler) runPass(w http.ResponseWriter, r *http.Request, limiter *llm.R
|
||||
return
|
||||
}
|
||||
|
||||
if ok, _ := limiter.Allow(docID); !ok {
|
||||
ok, _, slotAt := limiter.Allow(docID)
|
||||
if !ok {
|
||||
// Throttled: return the existing pending set unchanged rather than an
|
||||
// error, so the frontend keeps showing current suggestions.
|
||||
existing, err := h.fetchPending(docID)
|
||||
@@ -116,6 +117,10 @@ func (h *Handler) runPass(w http.ResponseWriter, r *http.Request, limiter *llm.R
|
||||
|
||||
raw, err := run(r.Context(), h.Client, contentText, tone)
|
||||
if err != nil {
|
||||
// Allow ran before the model call, so a failed pass would otherwise hold
|
||||
// the per-document slot for the full interval — stranding the frontend's
|
||||
// auto-retry on the throttle path. Release it so a retry can re-run.
|
||||
limiter.Release(docID, slotAt)
|
||||
errorJSON(w, http.StatusBadGateway, "llm pass failed: "+err.Error())
|
||||
return
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user