feat: route each model through an API it advertises - #269
Open
graehl wants to merge 2 commits into
Open
Conversation
Preserve Copilot model endpoint and reasoning metadata, then route each model only through an API it advertises. Pass native Anthropic Messages requests through unchanged and translate Responses-only models so Claude Code can use Terra with tools, signed reasoning, streaming, and explicit effort. Keep Chat Completions as the legacy fallback when older catalogs omit endpoint metadata. Reject known unsupported endpoint sets instead of advertising a false capability. Update production HTTP dependencies to patched releases after rejecting wholesale adoption of the much larger third-party fork.
The ya claude gateway saw degraded structure from both translated
paths: deltas aimed at already-stopped content blocks, zeroed usage,
invisible web searches, and prior-turn thinking replayed as visible
assistant text (which teaches models to emit pseudo-thinking prose).
Chat Completions -> Anthropic stream:
- Keep tool_use blocks open until the stream ends so interleaved
parallel tool-call deltas can never target a stopped block; text
after a tool call opens a fresh block. Anthropic consumers address
deltas by block index, so multiple open blocks are safe.
- Capture usage from any chunk, including the trailing usage-only
chunk (empty choices) that was previously dropped, and defer
message_delta/message_stop to a new finalizeAnthropicStream() the
handler calls once the upstream stream is exhausted.
Responses -> Anthropic stream:
- Text-block identity now includes content_index; a second content
part in one message item gets its own block instead of feeding a
block that output_text.done already stopped.
- Blocks carry a closed flag; late deltas to a closed block are
dropped rather than emitted as protocol violations.
- Reasoning summary parts are separated with a blank line, matching
the non-stream join.
Web search (both Responses paths): web_search_call output items now
translate to server_tool_use + web_search_tool_result blocks, with
usage.server_tool_use.web_search_requests, instead of vanishing.
Requests with a web_search tool add the
web_search_call.action.sources include so result URLs come back
(verified live against Copilot: sources arrive as {type:"url",url}
entries, no titles, hence the title-falls-back-to-url mapping).
These blocks intentionally do not round-trip: request translation
already drops them since Copilot rejects them as input.
Chat Completions request: assistant thinking blocks are dropped
instead of flattened into visible text, mirroring how Anthropic
ignores prior-turn thinking.
Verified live on port 4779: gemini-3.6-flash parallel tool calls
(two tool_use blocks, correct args, non-zero usage), gpt-5.6-luna
web search (structured blocks, sources populated, protocol-clean
SSE), claude-haiku-4.5 passthrough unaffected.
Author
|
Pushed 7e92143 fixing several stream-translation defects found while exercising this branch against a live gateway:
Verified live against Copilot (parallel tool calls, web search, Claude passthrough) plus new regression tests for each defect. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Copilot's model catalog declares per-model
supported_endpoints, but copilot-api currently translates everything through Chat Completions. This routes each model through an API it actually advertises:/v1/messagespassthrough. Anthropic-API clients (e.g. Claude Code) get full fidelity — thinking blocks, signed reasoning, exact tool_use semantics — instead of a lossy Anthropic→OpenAI→Anthropic round-trip.gpt-5.6-terradon't support Chat Completions at all, so they previously couldn't be served to Anthropic-API clients. A new Anthropic↔Responses adapter covers tools, streaming, signed reasoning, and explicit reasoning-effort mapping./v1/models; a model whose advertised endpoint set isn't supported is rejected up front rather than failing confusingly at request time. Catalogs without endpoint metadata keep the existing Chat Completions fallback.bun testpasses. This has been in daily production use since late July as the gateway behind yepanywhere's Claude Gateway provider, serving real Claude Code sessions —claude-opus-5via the native passthrough andgpt-5.6-terravia the Responses adapter, with tools, streaming, and effort selection exercised end to end.Independent of #268 (no overlapping files).
🤖 Generated by Fable