3c5f3ecb96
Conversational follow-up on a suggestion, streamed token-by-token. Backend (interface-only; handlers never touch a concrete LLM client): - internal/llm/chat.go: StreamAskPetal with conversational sampling (max_tokens 512, temp 0.7, rep 1.15, top_p 0.92, stop "\n\n\n"), reusing AskPetalSystemPrompt + TrimHistory. - internal/suggestions/chat.go: POST /api/suggestions/:id/chat. One user-scoped join loads the suggestion + parent content_text; surroundingParagraph extracts the \n\n-bounded paragraph at from_pos (whole-doc fallback when unlocated) and injects it server-side. Streams event: token / event: done SSE frames with JSON-encoded data so token newlines can't break framing; real http.Flusher per chunk. LLM-unreachable -> 502 before SSE headers; unknown suggestion -> 404. Frontend: - streamSuggestionChat: fetch + ReadableStream SSE parser (not EventSource, needs POST), abortable. - AskPetal.tsx: whole conversation in component state (no persistence, cleared on close), Petal's first bubble pre-seeded with the explanation, rose/lavender bubbles, CJK font stack on the bubbles only (Note #17), streaming caret. - SuggestionCard "Ask Petal" pill pins the card open while chatting (hover-close suppressed, click-away closes) and widens it to 340px. Tests: chat_test.go covers streamed-text concat + done event, server-side context injection on the system message, sampling params, 404, and surroundingParagraph. go build/vet/test clean, tsc clean, vite build OK. Live SSE smoke-tested against a fake streaming vLLM: tokens flushed individually through the chi middleware stack, done terminator, 502 on LLM-down, 404 on unknown suggestion. Claude-Session: https://claude.ai/code/session_016Yr6jELuRc7hyzYLccQKZd
30 lines
1.2 KiB
Go
30 lines
1.2 KiB
Go
package llm
|
|
|
|
import "context"
|
|
|
|
// AskPetalMessages assembles the message array for one Ask Petal turn: the
|
|
// suggestion-context system prompt followed by the (length-capped) client-side
|
|
// conversation history. The backend is stateless, so the full history rides on
|
|
// every request (spec: no server-side sessions).
|
|
func AskPetalMessages(systemPrompt string, history []Message) []Message {
|
|
msgs := make([]Message, 0, len(history)+1)
|
|
msgs = append(msgs, Message{Role: "system", Content: systemPrompt})
|
|
msgs = append(msgs, TrimHistory(history)...)
|
|
return msgs
|
|
}
|
|
|
|
// StreamAskPetal opens an SSE-friendly token stream for an Ask Petal reply using
|
|
// the conversational sampling parameters from the spec. The caller forwards the
|
|
// returned chunks to the browser; the channel closes when generation ends.
|
|
func StreamAskPetal(ctx context.Context, client LLMClient, systemPrompt string, history []Message) (<-chan string, error) {
|
|
return client.Stream(ctx, CompletionRequest{
|
|
Messages: AskPetalMessages(systemPrompt, history),
|
|
MaxTokens: 512,
|
|
Temperature: 0.7,
|
|
RepetitionPenalty: 1.15,
|
|
TopP: 0.92,
|
|
Stop: []string{"\n\n\n"},
|
|
Stream: true,
|
|
})
|
|
}
|