Phase 4: Ask Petal SSE chat
Conversational follow-up on a suggestion, streamed token-by-token. Backend (interface-only; handlers never touch a concrete LLM client): - internal/llm/chat.go: StreamAskPetal with conversational sampling (max_tokens 512, temp 0.7, rep 1.15, top_p 0.92, stop "\n\n\n"), reusing AskPetalSystemPrompt + TrimHistory. - internal/suggestions/chat.go: POST /api/suggestions/:id/chat. One user-scoped join loads the suggestion + parent content_text; surroundingParagraph extracts the \n\n-bounded paragraph at from_pos (whole-doc fallback when unlocated) and injects it server-side. Streams event: token / event: done SSE frames with JSON-encoded data so token newlines can't break framing; real http.Flusher per chunk. LLM-unreachable -> 502 before SSE headers; unknown suggestion -> 404. Frontend: - streamSuggestionChat: fetch + ReadableStream SSE parser (not EventSource, needs POST), abortable. - AskPetal.tsx: whole conversation in component state (no persistence, cleared on close), Petal's first bubble pre-seeded with the explanation, rose/lavender bubbles, CJK font stack on the bubbles only (Note #17), streaming caret. - SuggestionCard "Ask Petal" pill pins the card open while chatting (hover-close suppressed, click-away closes) and widens it to 340px. Tests: chat_test.go covers streamed-text concat + done event, server-side context injection on the system message, sampling params, 404, and surroundingParagraph. go build/vet/test clean, tsc clean, vite build OK. Live SSE smoke-tested against a fake streaming vLLM: tokens flushed individually through the chi middleware stack, done terminator, 502 on LLM-down, 404 on unknown suggestion. Claude-Session: https://claude.ai/code/session_016Yr6jELuRc7hyzYLccQKZd
This commit is contained in:
29
internal/llm/chat.go
Normal file
29
internal/llm/chat.go
Normal file
@@ -0,0 +1,29 @@
|
||||
package llm
|
||||
|
||||
import "context"
|
||||
|
||||
// AskPetalMessages assembles the message array for one Ask Petal turn: the
|
||||
// suggestion-context system prompt followed by the (length-capped) client-side
|
||||
// conversation history. The backend is stateless, so the full history rides on
|
||||
// every request (spec: no server-side sessions).
|
||||
func AskPetalMessages(systemPrompt string, history []Message) []Message {
|
||||
msgs := make([]Message, 0, len(history)+1)
|
||||
msgs = append(msgs, Message{Role: "system", Content: systemPrompt})
|
||||
msgs = append(msgs, TrimHistory(history)...)
|
||||
return msgs
|
||||
}
|
||||
|
||||
// StreamAskPetal opens an SSE-friendly token stream for an Ask Petal reply using
|
||||
// the conversational sampling parameters from the spec. The caller forwards the
|
||||
// returned chunks to the browser; the channel closes when generation ends.
|
||||
func StreamAskPetal(ctx context.Context, client LLMClient, systemPrompt string, history []Message) (<-chan string, error) {
|
||||
return client.Stream(ctx, CompletionRequest{
|
||||
Messages: AskPetalMessages(systemPrompt, history),
|
||||
MaxTokens: 512,
|
||||
Temperature: 0.7,
|
||||
RepetitionPenalty: 1.15,
|
||||
TopP: 0.92,
|
||||
Stop: []string{"\n\n\n"},
|
||||
Stream: true,
|
||||
})
|
||||
}
|
||||
Reference in New Issue
Block a user