在大模型应用中,如何实现流式输出 (Streaming / SSE)?请写出关键代码。
📋 面试问题
在大模型应用中,如何实现流式输出 (Streaming / SSE)?请写出关键代码。
✅ 期望回答
改善用户体验——首token出现越快,用户等待感越小(TTFT从5s变为0.3s)
Python FastAPI 实现(SSE - Server-Sent Events):from fastapi import FastAPI
from fastapi.responses import StreamingResponse
from openai import AsyncOpenAI
import asyncio, json
app = FastAPI()
client = AsyncOpenAI()
async def generate_stream(prompt: str):
stream = await client.chat.completions.create(
model='gpt-4',
messages=[{'role': 'user', 'content': prompt}],
stream=True
)
async for chunk in stream:
if chunk.choices[0].delta.content:
data = json.dumps({
'delta': chunk.choices[0].delta.content,
'finish_reason': chunk.choices[0].finish_reason
})
yield f'data: {data}\n\n'
yield 'data: [DONE]\n\n'
@app.post('/chat/stream')
async def chat_stream(prompt: str):
return StreamingResponse(
generate_stream(prompt),
media_type='text/event-stream'
)
前端(React/JS)消费SSE:
const response = await fetch('/chat/stream', { method: 'POST', body: formData });
const reader = response.body.getReader();
const decoder = new TextDecoder();
while (true) {
const { done, value } = await reader.read();
if (done) break;
const chunk = decoder.decode(value);
updateUIRender(chunk);
}
WebSocket场景:适合需要双向通信的实时对话系统