-
Notifications
You must be signed in to change notification settings - Fork 65
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Proposal: HTTP Server Sent Events Support #6687
Proposal: HTTP Server Sent Events Support #6687
Comments
@MohamedSabthar I have the following concerns,
|
@TharmiganK Yes, you're correct – we can't send the status code twice, I've updated the proposal by removing them. Ideally, the server should specify an event type indicating event completion. Upon receiving such an event type, the client should close the stream, preventing unnecessary reconnection attempts. And for the second comment, yes, we can introduce a getter and setter in the |
Is it normal to support only SSE in GET requests? Seems like libraries like fastapi supports this in POST eps as well. |
SSE are traditionally initiated with GET requests, and the Browser EventSource API doesn't seem to allow POST requests. While the standard doesn't explicitly say POST cannot be used, it also doesn't specify any details on how to handle reconnection attempts with POST requests. Ideally, POST requests are intended for one-time data submissions, and resending the same POST data on every reconnection would be inefficient. However, I can see a few use cases where SSE could be used with POST:
We could enable POST requests by mirroring the API changes mentioned in the proposal for GET, but without reconnection attempts. @shafreenAnfar, @TharmiganK what are your thoughts on this? |
I thought that these servers sends a redirect response for the Lets say we do not have retry, and then if the server close connection/ if there are any network failures, there will be a client error when we consume the stream. In the case of OpenAPI and anthropic, they sent a specific event to close the connection. So the we can manually call |
@MohamedSabthar Since we are adding the SSE support in the General HTTP client, we need to review the client level configurations to see whether they have any impact on the SSE implementation. For example, |
As discussed in the offline chat with @shafreenAnfar , we won't be supporting the SSE retry mechanism using the last event ID and last retry values from the latest events. When the stream ends with an error on the client-side due to network errors, the retry mechanism is only intended for idempotent GET requests, and implementing this is not an 80% use case. However, the user can reinitialize the client and manually retry to obtain the new stream if the previous stream ends with an error. |
With the design and code review with the team (@shafreenAnfar, @TharmiganK, @xlight05, and @AzeemMuzammil), we have agreed on the following changes:
public type SseEvent record {|
string event?;
string id?;
string data?;
int 'retry?;
string comment?;
|}; The reason for this change is that the serialization and deserialization of the
Users can override this behavior by capturing the error in the stream and returning a custom event. |
Summary
The Ballerina
http
package offers service and client interfaces to produce and consume HTTP services. However, neither the service nor the client currently provides an interface to work with Server-Sent Events (SSE). This proposal aims to add SSE support to thehttp
package.Goals
http:Service
.http:Client
.Motivation
Implementing SSE support in the Ballerina http package would significantly enhance its capabilities, offering benefits such as real-time data streaming and reliable event delivery. In the current trend, many foundation model providers exclusively use SSEs for their streaming APIs, making this an essential feature to support.
Description
Server-Sent Events (SSE) is a standardized technology for pushing real-time updates from a server to a client (unidirectional). At a high level, SSE works by keeping an open connection between the server and the client, allowing the server to send updates whenever they are available. The connection is kept alive using ‘Connection: keep-alive’ headers to ensure it stays open. If the connection drops, the client automatically tries to reconnect and resumes receiving events from the point of interruption. This makes SSE a reliable choice for applications needing live updates. The following sections describe the API changes to the existing
http:Client
andhttp:Service
to incorporate SSE, along with high-level implementation details.The
SseEvent
TypeThis type will be introduced in the http package. Which represents a Server Sent Event format.
API Changes related to
http:Service
Changes in the Resource Function Signature
The
stream<http:SseEvent, error?>
type will be allowed as the return type inget
resource methods ofhttp:Service
.Users can write similar code as follows to send SSE:
Changes in the
http:Caller
andhttp:ResponseMessage
The
http:ResponseMessage
union type will be modified to accommodatestream<http:SseEvent, error?>
, which changes thehttp:Caller
respond
method signature.With this proposed change, users will be able to write similar code as follows:
Processing Streams and Sending Events
The following pseudocode describes how to process the stream and send events over the wire when a stream is returned/responded from the resource methods.
stream
from the resource method.next()
method of thestream
and obtain the result:http:SseEvent
:nil
:error
:API Changes in http:Client
Changes to the
get
Resource Function SignatureThe
get
resource function signatures will be modified to supportstream<http:SseEvent, error?>
as it’s return type.With the above changes, users will be able to consume the Server Sent Events as a Ballerina stream. The following is an example of user written code.
http:Client Behavior on Consuming SSE
This section describes the client behaviors when consuming an event stream with the proposed API:
If the underlying Netty client receives a response without headers containing the following key-value pairs:
Content-Type: 'text/event-stream'
Connection: 'keep-alive'
Then, an
http:ClientError
is returned at the API level instead of returning a stream.If the underlying Netty client receives an HTTP 204 response from the server during event transmission, a
nil
value is returned from the stream, causing the stream to end. Further reconnection attempts to the server will not be made.If the underlying Netty client encounters network failures or receives HTTP 500 responses, it attempts to reconnect to the server based on values (
retry
andid
fields) consumed from the previousSseEvent
. Upon reconnecting, the client automatically sets theLast-Event-ID
header if an event ID was previously received from the server. At the API level, no errors are thrown if the client successfully reconnects to the server; the stream continues without errors. Otherwise, an error is returned from the stream, ending the stream and closing the connection.The
http:Client
will attempt to reconnect to the server based on the retry configuration provided via the existing client configurationhttp:RetryConfig
. If the number of retry attempts (RetryConfig.count
) is exhausted, the stream ends with an error. If the interval value ofhttp:RetryConfig
is set to-1
, then the retry interval value received from the previousSseEvent
will be used when reconnecting; otherwise, the interval configuration provided viaRetryConfig
is used.If the
stream.close()
function is manually called by the user, the underlying connection will be closed.Consuming Events and Producing Streams
This section explains high-level implementation details on how
http:Client
produces streams from consumed events.For each HTTP connection related to Server Sent Events created by the
http:Client
, a stream generator object may need to be created and maintained internally. This object should contain a blocking queue to store events and maintain a reference to the underlying connection. The object'snext
function will consume events from the blocking queue, while theclose
method closes the underlying HTTP connection. Furthermore, this object will track theid
andretry
values obtained from the latestSseEvent
, which are referenced during client reconnection in case of failure.API Changes in http:Response class
The following getters and setters will be introduced in the
http:Response
class.The existing
setPayload
method signature will be modified to accommodatestream<SseEvent, error?>
.With these proposed changes, the client and service can directly consume or produce the stream from the
http:Response
object.References
The text was updated successfully, but these errors were encountered: