Skip to content

Endpoint(workers=(0, N)) deploys workersStandby: 1, so an app documented as scale-to-zero always holds a worker warm #364

Description

@kbennett2000

Versions: runpod-flash 1.19.0, runpod 1.12.0, runpodctl
2.9.0-c094cac, Python 3.13.

What happens. workers= is documented as (min, max). Deploying with
workers=(0, 1) produces workersMin: 0 and also workersStandby: 1
the API field behind what the console calls an active worker. The endpoint
holds one worker warm continuously and never reaches zero workers.

Reproduce.

api = Endpoint(name="hello-flash", gpu=GpuGroup.AMPERE_16,
               workers=(0, 1), idle_timeout=60, dependencies=[])

@api.post("/predict")
async def predict(data: dict):
    return {"ok": True}
$ flash deploy --app hello-flash
$ curl -s https://rest.runpod.io/v1/endpoints -H "Authorization: Bearer $KEY"
{ "id": "...", "workersMin": 0, "workersMax": 1,
  "workersStandby": 1,          <-- not requested
  "idleTimeout": 60 }

Observed on two independent deploys of the same unchanged app. Polling
GET /v2/<id>/health for four minutes with zero jobs ever queued shows the
worker oscillating between running and idle/ready, never scaling to zero,
well past the 60-second idle timeout.

This is not a billing complaint. Measured over 11 m 13 s with no requests,
and again over 2 h 59 m, clientBalance did not move at ten decimal places —
where a billed 16 GB worker at $0.58/hr would have accrued $0.108 and $1.74
respectively. The standby worker did not bill. The report is that the
deployed configuration does not match the requested one
, and that the
difference is invisible.

It is hard to see and hard to undo.

  • runpodctl serverless list and serverless get <id> return workersMax and
    omit both workersMin and workersStandby.
  • runpodctl serverless update has --workers-min and --workers-max and no
    standby flag.
  • PATCH /v1/endpoints/<id> with {"workersStandby": 0} returns
    400 {"error":"Extra input keys provided in request body","problems":["key provided in request body which is not in input schema: 'workersStandby'"]}.
    The v1 REST API reports the field and will not accept it.

The only lever found was deleting the endpoint.

Suggestions.

  1. Have workers=(min, max) set workersStandby to min, so the deployed
    configuration matches the requested one.
  2. If a standby worker is intentional, document it next to workers= on the
    Endpoint reference and expose it as a parameter that can be set to 0.
  3. Separately: include workersMin and workersStandby in
    runpodctl serverless get, and accept workersStandby on the REST update
    route. A setting that can be read but not written is difficult to correct.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions