Repository navigation
Add ZeroGPU connector - #9
Conversation
OpenAI-compatible inference on ZeroGPU's open-weight models and specialized small language models (ZLMs), forwarding only POST /v1/chat/completions and POST /v1/responses. The API has no authenticated read-only endpoint (GET /v1/models is public), so the probe is a minimal chat completion on our cheapest ZLM; the Billing notice discloses the charge.
|
@nishithatanukunuri This is currently marked as draft but so far it looks good. I can take another look if you end up making changes before marking it ready. |
ZeroGPU now has an authenticated, read-only key identity endpoint, so the probe no longer runs a paid chat completion. GET /v1/me is free, does not touch the balance or free trial calls, and rejects revoked keys immediately. It is added to the allowed routes, and the Billing and Scope notices are updated to match.
|
Thanks @aezell! I made one change since your look, and it's now ready for review. The probe now uses a new GET /v1/me endpoint on our API instead of a chat completion. It returns the key, project and organization the key belongs to. It's free and read-only: it runs no inference, doesn't check the balance and records no usage. It also reads the key's row on every call, so a revoked key fails right away. GET /v1/me is added to the allowed routes, and the Billing and Scope notices are updated to say the key check is free. The probe passes against production with a valid key, and a wrong key is classified as invalid_api_key. The fixture is re-recorded with placeholder IDs. GET /v1/me is documented in connectors/zerogpu/README.md. The docs link in the description goes live on docs.zerogpu.ai when the connector ships. Happy to change anything else. |
|
@nishithatanukunuri The CI is misbehaving. I'm going to try closing and reopening the PR. Wanted to warn you if you saw notifications. |
owner: vendor
Adds
connectors/zerogpu, which lets a Sprite run OpenAI-compatible inference on ZeroGPU's open-weight models and specialized small language models (ZLMs) for generation, classification and extraction, without holding the ZeroGPU API key. Submitted by ZeroGPU.Routes
POST /v1/chat/completionsPOST /v1/responsesGET /v1/me(probe)connectors/zerogpu/README.mddocument the endpoint)Model IDs used in the examples: https://docs.zerogpu.ai/docs/model-catalog
Both routes support SSE (
"stream": true). Left out: model listing, embeddings, moderations, audio, batches, files, API-key management, billing and account routes.Credentials
x-api-key.x-project-idheader, so it is not forwarded.Probe
GET /v1/mereturns the key, project and organization the key belongs to ({"object": "api_key", "id", "project_id", "organization_id"}). We added this authenticated read-only endpoint for the connector, becauseGET /v1/modelsis public and can't tell a good key from a bad one.Cache-Control: no-store.403with the plain-text bodyInvalid credentials, hence theerrorsrule. A missing key gets401.object=api_key, non-emptyid,project_idandorganization_id, and an absenterror.forward_response_headers:x-request-id, which is on every response and which support uses to trace a request, andretry-after, which comes with429s.Icons
Both icons come from ZeroGPU's own logo, the dark favicon our dashboard serves at https://platform.zerogpu.ai/assets/favicon_dark.svg. ZeroGPU owns the mark and submits it for this listing.
icon.svgis the logo (black disc, green ring and halves) without the square favicon background, cropped to the circle.mark.svgis the ring and halves alone, with every fill set tocurrentColor.Validation
node scripts/check-connectors.mjs: 14 connector(s) ok (Node 22)npx --yes ajv-cli@5 validate --spec=draft2020 --strict=false -s v1/schema.json -d connectors/zerogpu/connector.json: validnode scripts/probe.mjs zerogpu --key-env ZEROGPU_API_KEYwith a valid key:HTTP 403→FAIL invalid_api_key. A revoked key also gets403(checked on our staging and dev environments).fixtures/zerogpu/probe_200.json. Credential headers are removed, and the three IDs in the body are replaced with placeholder UUIDs.Sprites-API-Gatewayuser agent.404model_not_found.