Summary
POST /turn runs a full agent turn but returns a single JSON blob after the
turn completes — no incremental output. Any interactive / desktop-harness-style
UX (watching tokens arrive, watching the agent call tools) is impossible today.
This issue tracks adding a streaming response mode to /turn so callers can
observe assistant-text deltas and tool-call events live, while keeping the
existing one-shot JSON contract intact.
Current behavior
handleTurn reads the body, calls runTurn, and emits one JSON response
{ sessionId, response, ... } — packages/knative-server/src/server.ts:66
(route wired at :338, response written at :91).
runTurn runs await session.prompt(prompt) to completion and only then
extracts the final assistant message text —
harness/src/run-turn.ts:361 and following (function at :296).
- Net: the client blocks for the whole turn and sees nothing until
end_turn.
The agent itself is already a real tool-using agent in this path — runTurn
registers k8sSandboxExtension (harness/src/run-turn.ts:323), so tool calls
run in sandbox-0. Streaming would surface those tool calls as they happen.
Proposed approach
- Content-negotiate on
/turn: when the request sends
Accept: text/event-stream, respond with SSE; otherwise behave exactly as
today. (Keeps a single public noun; a separate POST /turn/stream route is an
acceptable alternative if negotiation is awkward.)
- Relay the Pi agent session's turn events as SSE frames:
- assistant-text deltas,
- tool-use start / result events,
- a terminal frame carrying
{ sessionId, stopReason } so the client can
resume on the same session.
- Thread a callback / async-iterator out of
runTurn (today it only returns the
final TurnResult) so the HTTP layer can forward events without runTurn
knowing about HTTP.
- Back-compat is a hard requirement: the non-streaming JSON path must be
byte-for-byte unchanged, and session persistence / resume must be identical.
Scope / acceptance criteria
Open questions / spike
- Exact Pi event surface: which events does the
session from
createAgentSession emit for text deltas and tool_use? A short spike against
pi-fork should confirm before design lock.
- SSE vs chunked JSON-lines — SSE is the interoperable default; confirm no proxy
(Kourier/Knative activator) buffers it in a way that defeats streaming.
- Knative
timeoutSeconds still bounds a streamed turn (same as sync today) —
document the ceiling; long-running work belongs in the async path instead.
Out of scope
- Backgrounded / async free-form prompts — tracked separately (companion issue:
async free-form prompt dispatch). Async is "fire and poll"; this issue is
"watch live". The two are orthogonal.
- Auth / credential changes.
References
packages/knative-server/src/server.ts:66 (handleTurn), :91, :338
harness/src/run-turn.ts:296 (runTurn), :323 (extensions), :361 (session.prompt)
Summary
POST /turnruns a full agent turn but returns a single JSON blob after theturn completes — no incremental output. Any interactive / desktop-harness-style
UX (watching tokens arrive, watching the agent call tools) is impossible today.
This issue tracks adding a streaming response mode to
/turnso callers canobserve assistant-text deltas and tool-call events live, while keeping the
existing one-shot JSON contract intact.
Current behavior
handleTurnreads the body, callsrunTurn, and emits one JSON response{ sessionId, response, ... }—packages/knative-server/src/server.ts:66(route wired at
:338, response written at:91).runTurnrunsawait session.prompt(prompt)to completion and only thenextracts the final assistant message text —
harness/src/run-turn.ts:361and following (function at:296).end_turn.The agent itself is already a real tool-using agent in this path —
runTurnregisters
k8sSandboxExtension(harness/src/run-turn.ts:323), so tool callsrun in
sandbox-0. Streaming would surface those tool calls as they happen.Proposed approach
/turn: when the request sendsAccept: text/event-stream, respond with SSE; otherwise behave exactly astoday. (Keeps a single public noun; a separate
POST /turn/streamroute is anacceptable alternative if negotiation is awkward.)
{ sessionId, stopReason }so the client canresume on the same session.
runTurn(today it only returns thefinal
TurnResult) so the HTTP layer can forward events withoutrunTurnknowing about HTTP.
byte-for-byte unchanged, and session persistence / resume must be identical.
Scope / acceptance criteria
Accept: text/event-stream(or/turn/stream) yields incremental SSEframes: text deltas + tool-call events + a terminal
{sessionId,stopReason}./turn(no streaming Accept) is unchanged.curl -Nexample + a smoke assertion demonstrating deltas.Open questions / spike
sessionfromcreateAgentSessionemit for text deltas and tool_use? A short spike againstpi-forkshould confirm before design lock.(Kourier/Knative activator) buffers it in a way that defeats streaming.
timeoutSecondsstill bounds a streamed turn (same as sync today) —document the ceiling; long-running work belongs in the async path instead.
Out of scope
async free-form prompt dispatch). Async is "fire and poll"; this issue is
"watch live". The two are orthogonal.
References
packages/knative-server/src/server.ts:66(handleTurn),:91,:338harness/src/run-turn.ts:296(runTurn),:323(extensions),:361(session.prompt)