# Streaming

Stream assistant responses token-by-token into kai-chat or kai-response-stream using a ReadableStream, an AsyncIterable, or chunk-by-chunk mutation.

Display each token as it arrives — no buffering, no flash of finished content. AI/UI supports two approaches: **mutate `messages` in place** (the standard pattern with `kai-chat`) or **feed an `AsyncIterable<string>`** directly to `kai-response-stream`.

## Stream into kai-chat

`kai-chat` streams by appending an empty assistant message, then replacing its `parts` as tokens arrive. Set `loading = true` during the request so the input is disabled and the loading state shows; set it back to `false` when the stream ends.

```js

const chat = document.querySelector('kai-chat');

chat.addEventListener('kai-submit', async (e) => {
  const text = e.detail.value.trim();
  if (!text) return;

  // Append the user message
  const history = [
    ...chat.messages,
    { id: crypto.randomUUID(), role: 'user', parts: [{ type: 'text', text }] },
  ];
  chat.messages = history;
  chat.loading = true;

  // createAssistantStream appends the in-flight assistant message and folds each
  // delta onto its parts, always through a new array reference.
  const stream = createAssistantStream((update) => {
    chat.messages = update(chat.messages);
  });

  try {
    // Point at your own backend in production. Never expose an API key in the
    // browser. toOpenAIMessages encodes the thread for the OpenAI wire, tool
    // calls and their results included.
    const res = await fetch('/api/chat', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({ messages: toOpenAIMessages(history) }),
    });

    // readOpenAIStream parses the SSE: keep-alive comments, multi-line frames,
    // codepoints split across a socket boundary, tool calls, reasoning.
    await readOpenAIStream(res, stream);
  } finally {
    stream.done();
    chat.loading = false;
  }
});
```

Both imports come from the package: `createAssistantStream` owns the message, `readOpenAIStream` fills it. For the tool loop, `readAnthropicStream`, custom formats and the error model, see the [wire adapter recipe](/guides/recipes/wire-adapter/).

> **caution:** 
Never send your LLM provider key to the browser. Point `fetch` at your own backend endpoint that authenticates the user and proxies the request.

### OpenRouter example

If you are proxying an [OpenRouter](https://openrouter.ai) request on your server, the body sent upstream looks like this:

```json
{
  "model": "anthropic/claude-sonnet-4",
  "stream": true,
  "messages": [{ "role": "user", "content": "Hello" }]
}
```

The SSE response follows the OpenAI streaming format (`data: {...}` lines, terminated with `data: [DONE]`), so `readOpenAIStream` reads it as-is. Forward `upstream.body` from your route untouched.

This recipe is the *mechanic*. For connecting a specific provider, gateway, agent framework, or harness — and switching models from the chat header — see [Integrations](/integrations/overview/).

## Stream with kai-response-stream

`kai-response-stream` is a lower-level web component for streaming plain text or markdown outside a full chat thread. Pass either a complete string (renders with the reveal animation) or an **`AsyncIterable<string>`** (streams tokens live).

| Prop | Type | Default | Notes |
|------|------|---------|-------|
| `text` | `string \| AsyncIterable<string>` | `''` | Assign it in JavaScript — an async iterable can't be an HTML attribute. |
| `mode` | `'typewriter' \| 'fade'` | `'typewriter'` | Reveal animation. |
| `speed` | `number` | `20` | Characters/segments per tick. |
| `as` | `string` | — | Element tag to render into. |

**Event:** `kai-complete` — fires when streaming finishes.

### Feeding an AsyncIterable

This element wants plain text deltas, not message parts, so reach past the readers for `sseJson`, the same SSE decoder they use, exported on its own.

```js

await customElements.whenDefined('kai-response-stream');

const el = document.querySelector('kai-response-stream');

// sseJson yields one decoded frame per SSE event and stops at [DONE].
async function* tokenStream(res) {
  for await (const frame of sseJson(res.body)) {
    const delta = frame.choices?.[0]?.delta?.content;
    if (delta) yield delta;
  }
}

const res = await fetch('/api/chat', { method: 'POST', body: '...' });
el.text = tokenStream(res); // pass the AsyncIterable directly
```

### Passing a finished string

Assigning a complete string runs the reveal animation without a live connection — useful for replaying or previewing a cached response.

```js
el.text = 'The assistant response rendered with the typewriter effect.';
```

## Choosing an approach

| Scenario | Use |
|----------|-----|
| Full chat UI with history | `kai-chat` + mutate `messages` |
| Isolated response widget | `kai-response-stream` |
| Replay a saved response | `kai-response-stream` with a string |
| Render finished markdown | `kai-markdown` with a `content` string |

`kai-markdown` accepts only a static `content` string and renders it immediately — use it when the full text is available, not for live streaming.

## Without the adapter

If you would rather not add the import, here is the whole thing by hand. It handles text and nothing else: no tool calls, no reasoning, no citations. It also treats every `data:` line as a complete frame, and the SSE spec joins the several `data:` lines of one frame into a single payload, so parsing them separately drops that frame entirely. And it replaces the message's `parts` wholesale, which deletes any reasoning or tool panel already on the message it is streaming into. `readOpenAIStream` gets all three right, which is why the rest of this page uses it.

```js
// Seed the assistant placeholder yourself, since nothing else will.
const assistantId = crypto.randomUUID();
chat.messages = [...chat.messages, { id: assistantId, role: 'assistant', parts: [] }];

const res = await fetch('/api/chat', { method: 'POST', body: '...' });

const reader = res.body.getReader();
const decoder = new TextDecoder();
let buffer = '';
let answer = '';

while (true) {
  const { value, done } = await reader.read();
  if (done) break;
  buffer += decoder.decode(value, { stream: true });

  const lines = buffer.split('\n');
  buffer = lines.pop(); // carry the incomplete line forward

  for (const line of lines) {
    const s = line.trim();
    if (!s.startsWith('data:')) continue;
    const payload = s.slice(5).trim();
    if (payload === '[DONE]') continue;
    try {
      const delta = JSON.parse(payload).choices?.[0]?.delta?.content;
      if (!delta) continue;
      answer += delta;
      // A new array AND a new message object, or the row will not re-render.
      chat.messages = chat.messages.map((m) =>
        m.id === assistantId ? { ...m, parts: [{ type: 'text', text: answer }] } : m
      );
    } catch {
      // Ignore non-JSON keep-alive lines
    }
  }
}
```