Skip to content

Streaming responses (SSE) ​

A route normally answers once, with one JSON body, when its flow ends. Some answers can't wait for that: AI tokens as they are generated, progress of a long job, live updates. Server-sent events (SSE) let a route keep the connection open and push events to the client while the work happens. The work can happen anywhere: in the route itself, in a background workflow, in a custom block or in a plugin block.

This page walks through the whole setup.

How it fits together ​

  1. The client calls your route.
  2. The route calls sse.connect(id). From that moment the reply is an event stream, not a JSON body. You get back a handle, a short string that points at this stream.
  3. You pass the handle to whatever produces the data, such as a workflow.
  4. That producer calls sse.send(handle, data) for each event and sse.close(handle) when it is done.
  5. Each event reaches the client, even when the producer runs on a different server from the one that holds the connection.

Events travel through a broker: a NATS or Redis integration of your own. That is how an event sent on one server reaches the server that holds the client's connection.

1. Pick a broker ​

Open the route, then Route settings → Advanced → Streaming → SSE broker, and choose one of your project's integrations:

BrokerIntegration
NATSA message queue integration of the NATS kind
RedisA key-value store integration of the Redis kind

Only NATS and Redis integrations are listed. There is no separate "SSE enabled" switch: a route stays a normal JSON route until its code calls sse.connect. Picking a broker only makes streaming possible.

One broker per route

Every route that streams needs its own broker setting, but many routes can share the same integration. Streams of different projects never mix, even on a shared broker.

2. Connect in the route ​

Call sse.connect(id) in a JS Runner block on the route. The id is yours to choose: a conversation id, a job id, a user id. It names the stream.

js
const handle = sse.connect(getQueryParam("conversation"));
return { handle };
  • connect returns the handle, in the form <integrationId>:<id>, for example int_8f2a:conv_123.
  • sse.connectionId is the same handle, readable anywhere later in this request.
  • connect works only on a route. In a workflow or a job it throws.

Check who is asking first

The route decides who may connect to an id, not the client. Calling connect with an id takes that stream over from whoever held it before. If ids are easy to guess and the route has no auth check, anyone can take over someone else's stream and read what is sent to it. Check the caller (for example with JWT auth) and that the id belongs to them before calling connect.

3. Keep the stream open: Hold connection ​

When the route's flow ends, the stream ends too, unless you hold it. Turn on Hold connection on the route's Response block:

  • The Response block's body is sent as the first event. A null body sends no event.
  • The flow ends as usual, but the connection stays open for later events.

Without Hold connection, everything you want to send must be sent before the flow ends.

4. Hand the handle to the producer ​

Pass the handle like any other value. To stream from a background workflow, start it with a Trigger Workflow block and put the handle in its data:

FieldValue
Workflowanswer-question
Data{ "handle": "js: return sse.connectionId;", "question": "js: return getRequestBody().question;" }

The workflow reads it as input.handle. The handle carries everything needed to find the stream, so the workflow never needs to know which route it came from.

5. Send events ​

js
await sse.send(input.handle, "Hello");
await sse.send(input.handle, { progress: 40 }, { event: "progress", eventId: "4" });
ArgumentWhat it does
handleThe handle from connect
dataText is sent as it is. Anything else is sent as JSON. Text with several lines becomes one data: line per line, and the client puts them back together
eventOptional. The event name the client listens for. Without it, the client sees a plain message
eventIdOptional. The event's id: field

sse.send, sse.close and sse.signal work in routes, workflows and custom blocks, on any server. Plugin blocks use the same calls as ctx.sse.

sse.send returns true, or false when no client is connected or the broker failed. It doesn't throw for either, so a user refreshing the page never fails your run. Keep sending, and the client gets events again once it reconnects:

js
const delivered = await sse.send(input.handle, token);
if (!delivered) unsent.push(token); // optional: keep what the client missed

The answer is a best guess. Fluxify learns that a client left while your code is sending, so the first send, or one after 10 minutes without sending, returns true even if nobody is there.

6. Stop when nobody is listening ​

sse.signal(handle) returns an AbortSignal that aborts when:

  • the client disconnected,
  • the stream was closed, or
  • another connection took the id over.

Check it in a loop, or hand it to anything that accepts a signal, so the work stops when nobody is reading:

js
const signal = sse.signal(input.handle);
for (const step of steps) {
	if (signal.aborted) break;
	await sse.send(input.handle, await runStep(step));
}

This matters most for AI calls: a model that keeps generating for a closed tab still costs tokens. See Stream an AI response.

If the broker can't be reached, the signal aborts too, because Fluxify can no longer tell whether the client is still there.

7. Close ​

js
await sse.close(input.handle);

The client's connection ends. Calling close twice is safe, and it works from any server.

Let the user stop the work ​

sse.close ends the stream, not the workflow. To give the user a Stop button, use a Run ID and a second route:

RouteBlocks
POST /chatCheck the caller → sse.connect(id) → Trigger Workflow, Now, with Run ID set to id and the handle in its data
POST /chat/stopCheck the caller → Trigger Workflow, Stop a run, with Run ID set to id → sse.close(handle)

Inside the workflow, abortSignal aborts on the stop. Pass it to fetch. AI, database, key-value, queue and httpClient calls stop by themselves.

Don't stop the agent from sse.signal

sse.signal also fires on a takeover, and a page refresh is a takeover. A run that ends on it dies every time the user refreshes the tab. Use it only for work that isn't worth keeping once this connection is gone. For work that should survive a refresh, use the stop route.

A stop has limits. Most importantly, code that never waits keeps running. Read what a stop cannot do.

Long-running workflows and agents ​

An agent can run for a long time. Four settings decide how long it gets:

SettingDefaultWhat to know
The workflow's timeout5 minutes, up to 1 hourRaise it on the workflow for a long agent. Past it, the run fails and follows the normal retry rules. Like a stop, it only takes effect at I/O, the next block or the next loop iteration
The route's timeout30 secondsIt does not end a held stream, so it doesn't limit the agent
SSE_LIVENESS_TIMEOUT60 secondsUsually the first limit you hit. An agent silent for longer loses its stream, so send a progress event, or raise the setting
Trigger Workflow concurrency (Project settings → General)10 runs at onceFurther runs wait, and runs don't start strictly in order unless it is set to 1

Reconnecting ​

To reconnect, the client calls the route again with the same id. The new connection takes the id over, and any older connection to it (another tab, a stale socket) is closed. There is nothing else to set up.

Events in the gap are lost

Events sent while no client is connected are not stored. A client that reconnects gets only the events sent after it is back. Replaying missed events (Last-Event-ID) is not supported.

A worker restart also drops its open streams. Clients reconnect the same way.

A browser client ​

The browser's built-in EventSource reads the stream:

js
const source = new EventSource(`/chat?conversation=${encodeURIComponent(conversationId)}`);

source.onmessage = (e) => {
	output.textContent += e.data; // events sent without a name
};
source.addEventListener("progress", (e) => {
	showProgress(JSON.parse(e.data).progress);
});
source.addEventListener("done", () => source.close());
source.addEventListener("error", (e) => {
	if (e.data) console.error(JSON.parse(e.data).message); // an error event from Fluxify
});

Two things to know about EventSource:

  • It reconnects by itself a few seconds after the stream ends. Since reconnecting means calling the route again, send a last event (such as done) and call source.close() when you get it.
  • It can't send custom headers. Authenticate with a cookie, or read the stream with fetch and response.body.getReader() when you need an Authorization header.

Limits ​

LimitValueWhat happens
No message for too long60 seconds by defaultThe client gets an error event, {"message":"SSE stream timed out"}, and the stream closes. Every send resets the clock. The server setting is SSE_LIVENESS_TIMEOUT
Keep-aliveEvery 15 seconds, or half the timeout if that is shorterA : ping comment line. Clients ignore it. It keeps proxies from cutting the connection, but does not reset the timeout above
Client not readingAbout 1 MB of waiting eventsThe client is disconnected
Async repliesNot supportedA route that answers 202 right away can't stream
One stream per requestCalling connect twice in one request throws

The route's own timeout does not end a held stream. The timeout above does.

Errors ​

ErrorCause
SSE needs a broker integration in route settingsconnect ran on a route with no SSE broker. Pick one in route settings
sse.connect can only be used in a routeconnect ran in a workflow or a job
sse.connect needs an idThe id was empty or not text
SSE cannot be used with async repliesThe route answers asynchronously (202)
This request already opened an SSE streamconnect ran twice in the same request
not an SSE connection handlesend, close or signal got something that is not a handle. Pass the handle from connect unchanged

If the flow throws an error after connect, the client gets an error event with the message, and the stream closes. The same happens if the broker fails.

Test suites

Test suites don't collect events yet. A route that calls connect fails there with the "needs a broker" error. Try streaming routes with curl -N or a browser instead.

Released under the Apache License 2.0. Enterprise features are under the Fluxify Enterprise Edition License.