Build a Streaming Chat with connic WebSocket Connectors

A connic WebSocket connector connects a chat app to a deployed agent over one persistent connection. The browser opens that connection itself, each message starts an agent run, and the reply streams back as events that carry the message's ID. This tutorial builds the client and the agent configuration: per-user authentication, streamed replies and progress events, conversation history that survives a reconnect, and the cases where output guardrails, approvals, or errors change how a reply arrives.

Create the connector

The connector belongs to the agent that answers the chat, here support-chat. In the dashboard, open the agent, click Add inbound connector in the Connector Flow, choose Create New Connector, and select WebSocket. connic creates the URL and shows it in the connector's detail drawer. The connector can also be declared in code, so its settings deploy together with the agent:

type: websocket
config:
  require_auth: false            # each message carries a user token instead
  progress_events: true          # model, tool, and approval events
  session_timeout: 1800          # close a connection after 30 idle minutes
  max_messages_per_session: 500
agents:
  - support-chat

The next deploy creates the connector. Streaming is on by default. require_auth: false turns off Require Authentication, so the browser can connect without the connector secret, and progress_events adds the status events shown below. session_timeout closes a connection after that many idle seconds (60 to 86,400, default 3,600), and max_messages_per_session caps the messages one connection can send (1 to 10,000, default 100). Both limits close the connection and leave the conversation's history in place. The connectors in code reference covers per-environment overrides.

Authenticate every message

Without Require Authentication, anyone who knows the URL can open a connection, so the agent checks who sent each message. The browser adds the signed-in user's token to every message, and the agent's middleware verifies it before the run continues. The token belongs in every message because each message is a separate run. The connector secret stays out of the page, where any visitor could read it, and each run knows the user it serves:

from typing import Any

from connic import StopProcessing


def user_from_token(token: str | None) -> str | None:
    """Check the token with your sign-in provider and return the user ID."""
    raise NotImplementedError


async def before(content: dict[str, Any], context: dict[str, Any]) -> dict[str, Any]:
    payload = context["payload"]
    user_id = user_from_token(payload.get("token"))
    if user_id is None:
        raise StopProcessing("Please sign in again.")
    context["user_id"] = user_id
    context["conversation_key"] = f"{user_id}:{payload['conversation_id']}"
    files = [part for part in content["parts"] if "text" not in part]
    content["parts"] = [{"text": payload["message"]}, *files]
    return content

user_from_token stands for the check that the app's sign-in provider offers. When it fails, StopProcessing completes the run with its message as the reply, so a user with an expired session reads "Please sign in again." in the chat. The middleware also runs when a run resumes after an approval or starts after waiting in a queue, with the token from the original message, so tokens should stay valid for longer than the approval timeout. It replaces the model's input with the message text, so the token and the conversation ID never reach the agent's model. The recorded run input, run context, and session state still contain the payload. New environments already redact fields named token in those records. Older environments can add *.token to their sensitive data redaction setting. A backend that calls the connector without a browser can keep Require Authentication on and send the connector secret instead. If Require Authentication is still on when a browser connects, connic waits for that secret and closes the connection after 30 seconds with an AUTH_TIMEOUT error.

Send messages and read the reply events

When the connection opens, connic sends a connected event that lists the linked agents and the connection limits. Each chat message is a message event with a payload object. The id is optional, but connic copies it into the acknowledgment, the stream events, and most errors about that message, and the client matches replies by it. Errors about the connection, and a few others such as MESSAGE_LIMIT, carry no ID. The payload holds the text, the token, and a conversation_id for the history set up in the next section:

{
  "type": "message",
  "id": "m-1",
  "payload": {
    "conversation_id": "conv-blender",
    "message": "My blender arrived with a cracked jar.",
    "token": "<session token>"
  }
}

connic acknowledges the message and streams the reply:

{ "type": "ack", "id": "m-1", "message_number": 1 }
{ "type": "stream_start", "id": "m-1", "agent": "support-chat" }
{ "type": "thinking", "id": "m-1", "agent": "support-chat" }
{ "type": "stream_chunk", "id": "m-1", "agent": "support-chat", "chunk": "Sorry to hear that. " }
{ "type": "stream_chunk", "id": "m-1", "agent": "support-chat", "chunk": "Would you like a replacement jar or a refund?" }
{
  "type": "stream_end",
  "id": "m-1",
  "agent": "support-chat",
  "full_response": "Sorry to hear that. Would you like a replacement jar or a refund?",
  "token_usage": { "input_tokens": 212, "output_tokens": 18, "total_tokens": 230 }
}

stream_start opens the reply, which is the moment to show a typing indicator, and thinking follows once the model starts. stream_chunk events carry text as the model writes it. stream_end closes the reply with full_response, the run's final output, and token_usage. full_response is the text to store. Middleware that rewrites the answer can make it differ from the joined chunks, so chunks work as a preview that stream_end replaces. With streaming turned off on the connector, a single response event takes the place of the stream events, and connic sends no progress events.

With streaming on, progress events report what the agent does before and between text chunks. thinking marks the start of each model call, tool_call and tool_result name a tool when the agent calls it and when it returns, and approval_pending and approval_resolved frame an approval pause. A reply that looks up an order:

{ "type": "thinking", "id": "m-5", "agent": "support-chat" }
{ "type": "tool_call", "id": "m-5", "agent": "support-chat", "tool": "lookup_order" }
{ "type": "tool_result", "id": "m-5", "agent": "support-chat", "tool": "lookup_order" }
{ "type": "thinking", "id": "m-5", "agent": "support-chat" }
{ "type": "stream_chunk", "id": "m-5", "agent": "support-chat", "chunk": "Your order shipped on Monday." }

The events carry only the tool's name, never its arguments or result, and the client picks the label, for example "Looking up your order" for lookup_order. Because the browser receives them directly, the tool names are visible in its developer tools.

A minimal browser client. getToken returns the signed-in user's current token from the app's sign-in library, so a refreshed token goes out with the next message:

const MAX_RETRY_DELAY_MS = 30_000;

export function createChatConnection({ url, getToken, onPreview, onReply, onFailure, onStatus }) {
  let socket;
  let ready = false;
  let stopped = false;
  let attempt = 0;
  const inFlight = new Map(); // message id -> preview text

  function connect() {
    if (stopped) return;
    socket = new WebSocket(url);
    socket.onmessage = (event) => handle(JSON.parse(event.data));
    socket.onclose = () => {
      ready = false;
      for (const id of inFlight.keys()) onFailure(id, "interrupted");
      inFlight.clear();
      if (stopped) return;
      setTimeout(connect, Math.min(1000 * 2 ** attempt++, MAX_RETRY_DELAY_MS));
    };
  }

  function handle(event) {
    switch (event.type) {
      case "connected":
        ready = true;
        attempt = 0;
        break;
      case "stream_chunk": {
        const preview = (inFlight.get(event.id) ?? "") + event.chunk;
        inFlight.set(event.id, preview);
        onPreview(event.id, preview);
        break;
      }
      case "stream_end":
        inFlight.delete(event.id);
        if (event.error) onFailure(event.id, event.error);
        else onReply(event.id, event.full_response);
        break;
      case "thinking":
      case "tool_call":
      case "tool_result":
      case "approval_pending":
      case "approval_resolved":
        onStatus(event.id, event); // the UI turns event.type and event.tool into a label
        break;
      case "error":
        if (!event.id) break; // connection errors carry no id; onclose reports open messages
        inFlight.delete(event.id);
        onFailure(event.id, event.code);
        break;
      // ack, stream_start and ping need no handling here
    }
  }

  connect();

  return {
    close() {
      stopped = true;
      socket.close();
    },
    async send(conversationId, text) {
      const token = await getToken();
      if (!ready) return null; // the UI keeps the text and shows "reconnecting"
      const id = crypto.randomUUID();
      inFlight.set(id, "");
      socket.send(JSON.stringify({
        type: "message",
        id,
        payload: { conversation_id: conversationId, message: text, token },
      }));
      return id;
    },
  };
}

Each browser tab opens its own connection. Within one connection, messages are processed one at a time, in the order received, and a message sent while a reply is running waits for it. The UI calls close() when the chat closes; otherwise the client reconnects after every close, including the idle timeout. Runs from all connections share the agent's max_concurrent_runs, which defaults to 1, so with the default one user's reply waits for another's. The agent file in the next section raises it to 10, which needs the developer plan or higher. The subscription plan caps the value, and a deploy that exceeds the cap fails. If several agents are linked to the connector, each one answers in turn with its own stream_start to stream_end sequence and its name in agent. The client above assumes one agent.

Keep the conversation across reconnects

Every message starts a separate agent run. The agent sees earlier messages only through a persistent session, and session.key decides which messages share one. Here it reads conversation_key, which the middleware builds from the verified user ID and the conversation_id:

version: "1.0"

name: support-chat
type: llm
model: connic/gpt-5.6-terra
description: "Answers customer questions in the in-app chat"
system_prompt: |
  You are the support assistant in the Acme app. Answer in a few sentences.

max_concurrent_runs: 10

session:
  key: context.conversation_key
  ttl: 604800   # history expires after 7 days without a message

concurrency:
  key: conversation_id
  on_conflict: queue

Because the key contains the user ID, a user who sends someone else's conversation_id gets a separate, empty session rather than that conversation's history. One connection can carry several conversations, and a new connection continues a conversation as soon as it sends the same conversation_id with a valid token. A run whose key is missing fails. Here, one user has two conversations open on the same connection when it drops. Each reply shows its full_response:

After the reconnect, the agent still knows that the blender conversation is about the cracked jar, and the shipping conversation contains only its own messages. Long conversations can outgrow the model's context window. With context compression enabled on the agent, connic shortens the history when that happens.

Connection limits and history expiry

The connector settings limit the connection. The agent's ttl limits the conversation:

After either of the first two, the client reconnects and continues the same conversation. Without ttl, history never expires.

Reconnects and interrupted replies

connic does not send earlier messages to a new connection. The chat UI keeps its own transcript for display, while the session holds the history the model reads. A reply that is still running when the connection drops does not reach the new connection either. Its run finishes anyway, and the answer becomes part of the history. The client above reports such replies as interrupted; the UI can show that and let the user ask again.

The concurrency block in the agent file covers this case. Runs from different connections are not serialized, and when two runs in one session overlap, the run that writes second fails. With the concurrency rule on conversation_id, a message sent right after a reconnect, or from a second browser tab, waits until the earlier run in that conversation has finished.

Handle guardrails, approvals, and failures

Every reply uses the same event sequence. Guardrails, approvals, and failures change when its text arrives and whether it ends with full_response or an error.

Output guardrails send the reply in one piece

Output guardrails check the complete reply before the client sees any of it. connic therefore does not stream partial text for an agent with output rules. This rule blocks replies that contain card numbers, US social security numbers, or API keys, and sends the customer a fixed sentence instead:

guardrails:
  output:
    - type: pii_leakage
      mode: block
      config:
        rejection_message: "I can't share that here. Please contact support by email."

The same question, with and without the rule:

The customer waits longer for the first text, but the format does not change. A client that makes no assumption about the number of chunks handles both cases without changes. Progress events still arrive during the wait, because they contain no reply text. When the rule blocks, the rejection_message replaces the reply and the run completes, so the client shows it like any other answer. With fail_run: true in the rule's config, the run fails instead and stream_end carries an error. Input guardrails run before the agent and do not change streaming.

Approvals pause the reply

A tool that needs approval stops the run until a team member approves or rejects the call. The connection stays busy with that reply, and later messages on the connection queue behind it. approval_pending names the tool when the run pauses, which gives the UI its waiting state, and approval_resolved reports the decision (approved, rejected, or timeout) once it is made. Without progress events, nothing arrives during the wait. The waiting time does not count toward the run timeout, and approval.timeout defaults to 3,600 seconds, a long time for a customer looking at a chat window. A short timeout (the minimum is 30 seconds) and on_rejection: continue keep the conversation moving:

tools:
  - orders.lookup_order
  - orders.cancel_order

approval:
  tools:
    - orders.cancel_order
  timeout: 120
  on_rejection: continue
  message: "A customer asked the chat assistant to cancel an order."

With on_rejection: continue, a rejection or an expired timeout returns a rejection message to the model as the tool result, and the agent tells the customer what happened. With the default, fail, the run fails and the reply ends with an error. After the decision, the rest of the reply streams, and full_response contains the text from before and after the pause. Human-in-the-loop approvals covers conditional gates and notification channels.

Failures end the reply with an error

Every message ends with stream_end (response with streaming off) or with an error event, unless the connection drops first. A failed, cancelled, or timed-out run produces stream_end with an error field instead of full_response. That text is written for the team, so the UI shows its own message to the customer and the team finds the details in the run's trace. A rejected token is not a failure: its StopProcessing message arrives as a normal reply. error events report the other common problems with a code:

On an idle connection, connic also sends ping events, which the client can ignore. The WebSocket connector reference covers setup, settings, and the message format.