Summary
Currently, the server-side session persistence introduced in #87 only applies to non-streaming (complete) responses. Streaming responses bypass session persistence entirely, meaning that conversations conducted over the streaming path are not stored or manageable by the server.
Problem
When a client uses the streaming mode of the Responses API, the response chunks are forwarded directly without being accumulated and saved to the session store. This creates an inconsistency: non-streaming sessions are persisted and can be referenced via previous_response_id, but streaming sessions are not.
Proposed Solution
Implement a stream accumulator that:
- Collects streamed response chunks as they arrive from the upstream provider.
- Reconstructs the full assistant message(s) from the accumulated chunks.
- Saves the resulting conversation (request messages + accumulated assistant response) into the session store after the stream completes, in the same way non-streaming responses are handled.
This will ensure consistency between streaming and non-streaming paths for server-side session management.
References
/cc @bzp2010
Summary
Currently, the server-side session persistence introduced in #87 only applies to non-streaming (complete) responses. Streaming responses bypass session persistence entirely, meaning that conversations conducted over the streaming path are not stored or manageable by the server.
Problem
When a client uses the streaming mode of the Responses API, the response chunks are forwarded directly without being accumulated and saved to the session store. This creates an inconsistency: non-streaming sessions are persisted and can be referenced via
previous_response_id, but streaming sessions are not.Proposed Solution
Implement a stream accumulator that:
This will ensure consistency between streaming and non-streaming paths for server-side session management.
References
/cc @bzp2010