Phase 4: Ask Petal SSE chat
Conversational follow-up on a suggestion, streamed token-by-token. Backend (interface-only; handlers never touch a concrete LLM client): - internal/llm/chat.go: StreamAskPetal with conversational sampling (max_tokens 512, temp 0.7, rep 1.15, top_p 0.92, stop "\n\n\n"), reusing AskPetalSystemPrompt + TrimHistory. - internal/suggestions/chat.go: POST /api/suggestions/:id/chat. One user-scoped join loads the suggestion + parent content_text; surroundingParagraph extracts the \n\n-bounded paragraph at from_pos (whole-doc fallback when unlocated) and injects it server-side. Streams event: token / event: done SSE frames with JSON-encoded data so token newlines can't break framing; real http.Flusher per chunk. LLM-unreachable -> 502 before SSE headers; unknown suggestion -> 404. Frontend: - streamSuggestionChat: fetch + ReadableStream SSE parser (not EventSource, needs POST), abortable. - AskPetal.tsx: whole conversation in component state (no persistence, cleared on close), Petal's first bubble pre-seeded with the explanation, rose/lavender bubbles, CJK font stack on the bubbles only (Note #17), streaming caret. - SuggestionCard "Ask Petal" pill pins the card open while chatting (hover-close suppressed, click-away closes) and widens it to 340px. Tests: chat_test.go covers streamed-text concat + done event, server-side context injection on the system message, sampling params, 404, and surroundingParagraph. go build/vet/test clean, tsc clean, vite build OK. Live SSE smoke-tested against a fake streaming vLLM: tokens flushed individually through the chi middleware stack, done terminator, 502 on LLM-down, 404 on unknown suggestion. Claude-Session: https://claude.ai/code/session_016Yr6jELuRc7hyzYLccQKZd
This commit is contained in:
@@ -78,3 +78,64 @@ export const api = {
|
||||
dismissSuggestion: (id: string) =>
|
||||
req<void>(`/suggestions/${id}/dismiss`, { method: 'POST' }),
|
||||
}
|
||||
|
||||
// One turn in an Ask Petal conversation. History lives only in the component —
|
||||
// the server is stateless and re-injects the suggestion context every request.
|
||||
export interface ChatMessage {
|
||||
role: 'user' | 'assistant'
|
||||
content: string
|
||||
}
|
||||
|
||||
// streamSuggestionChat POSTs the conversation to the SSE chat endpoint and
|
||||
// invokes onToken for each text chunk as it arrives. It uses fetch + a
|
||||
// ReadableStream reader (not EventSource, which can't POST) and resolves when
|
||||
// the stream ends. Abort via the optional signal to cancel mid-response.
|
||||
export async function streamSuggestionChat(
|
||||
suggestionId: string,
|
||||
messages: ChatMessage[],
|
||||
onToken: (text: string) => void,
|
||||
signal?: AbortSignal,
|
||||
): Promise<void> {
|
||||
const res = await fetch(`/api/suggestions/${suggestionId}/chat`, {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ messages }),
|
||||
signal,
|
||||
})
|
||||
if (!res.ok || !res.body) {
|
||||
const detail = await res.text().catch(() => '')
|
||||
throw new Error(`${res.status} ${res.statusText}${detail ? `: ${detail}` : ''}`)
|
||||
}
|
||||
|
||||
const reader = res.body.getReader()
|
||||
const decoder = new TextDecoder()
|
||||
let buf = ''
|
||||
for (;;) {
|
||||
const { done, value } = await reader.read()
|
||||
if (done) break
|
||||
buf += decoder.decode(value, { stream: true })
|
||||
// SSE events are separated by a blank line. Process every complete one and
|
||||
// keep the trailing partial in the buffer.
|
||||
let sep: number
|
||||
while ((sep = buf.indexOf('\n\n')) >= 0) {
|
||||
const event = parseSSE(buf.slice(0, sep))
|
||||
buf = buf.slice(sep + 2)
|
||||
if (event.name === 'done') return
|
||||
if (event.name === 'token' && event.data) {
|
||||
const text = (JSON.parse(event.data) as { text: string }).text
|
||||
if (text) onToken(text)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// parseSSE pulls the event name and data payload out of one raw SSE frame.
|
||||
function parseSSE(frame: string): { name: string; data: string } {
|
||||
let name = 'message'
|
||||
const dataLines: string[] = []
|
||||
for (const line of frame.split('\n')) {
|
||||
if (line.startsWith('event:')) name = line.slice(6).trim()
|
||||
else if (line.startsWith('data:')) dataLines.push(line.slice(5).trim())
|
||||
}
|
||||
return { name, data: dataLines.join('\n') }
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user