> ## Documentation Index
> Fetch the complete documentation index at: https://wiki.llm.sg/llms.txt
> Use this file to discover all available pages before exploring further.

# Stream responses and manage conversations

> Read incremental text from compatible model responses and send relevant message history to continue a conversation through the llm.sg API endpoint.

Use streaming to display text as it arrives, and send relevant message history to give a llm.sg model context for the next turn.

> [!NOTE]
>
> These Python examples assume the OpenAI-compatible chat format and a supporting model. Streaming and conversation behavior still require validation with an authorized key.

## Stream text

Complete the [quickstart](https://wiki.llm.sg/en/docs/quickstart.md) setup, then set `stream=True` and read incremental text from each chunk.

```python theme={null}
import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["LLM_SG_API_KEY"],
    base_url="https://api.llm.sg/v1",
)
stream = client.chat.completions.create(
    model=os.environ["LLM_SG_MODEL"],
    messages=[{"role": "user", "content": "Explain model routing briefly."}],
    stream=True,
)
try:
    for chunk in stream:
        if chunk.choices:
            content = chunk.choices[0].delta.content
            if content:
                print(content, end="", flush=True)
finally:
    stream.close()
print()
```

Some chunks contain no text. Handle connection failures during iteration and distinguish an interrupted stream from a completed answer. Streaming changes when content appears; it does not guarantee shorter total generation time.

## Continue a conversation

For the compatible chat format, send the relevant user and assistant messages in each request. Your application manages the history it chooses to send.

```python theme={null}
import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["LLM_SG_API_KEY"],
    base_url="https://api.llm.sg/v1",
)
model = os.environ["LLM_SG_MODEL"]
messages = [{"role": "user", "content": "What is an API?"}]
first = client.chat.completions.create(model=model, messages=messages)
answer = first.choices[0].message.content
if not answer:
    raise RuntimeError("This example requires a text response")
print(answer)

messages.append({"role": "assistant", "content": answer})
messages.append({"role": "user", "content": "Give a restaurant analogy."})
second = client.chat.completions.create(model=model, messages=messages)
print(second.choices[0].message.content)
```

History adds input tokens. Keep relevant recent messages or summarize older content, and leave enough context capacity for the answer.

Check image input, tool calling, reasoning, and structured output support separately before using them. A model's tool request does not execute the tool; your application usually performs that action and sends the result back.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.