A list of GpuTypes in gpu= does not constrain placement; a single GpuType does

[kbennett2000]

Versions: runpod-flash 1.19.0, runpod 1.12.0, runpodctl 2.9.0-c094cac, Python 3.13.

What happens. A list of GpuTypes in gpu= is stored and read back correctly, and then does not constrain where workers are placed. A single GpuType does constrain it.

Reproduce.

1. Deploy a client-mode endpoint asking for two specific cards:

Endpoint(
    name="scriptorium-imagegen",
    image="ghcr.io/<owner>/<image>:<tag>",
    gpu=[GpuType.NVIDIA_RTX_A5000, GpuType.NVIDIA_GEFORCE_RTX_3090],
    workers=(0, 1),
    idle_timeout=60,
)

2. Read the endpoint back. The request was stored exactly as sent:

"gpuTypeIds": ["NVIDIA RTX A5000", "NVIDIA GeForce RTX 3090"]

3. Send requests and ask the container which card it is on. All 7 of 7 ran on:

cuda:0 NVIDIA RTX PRO 6000 Blackwell Server Edition MIG 1g.24gb : cudaMallocAsync

That is neither card in the list. It is in the same 24 GB VRAM tier and on the same price line, so this looks like tier-level substitution — but the API accepted a card-level request.

4. Control: the same image and app with a single GpuType:

gpu=GpuType.NVIDIA_GEFORCE_RTX_4090

ran 7 of 7 requests on cuda:0 NVIDIA GeForce RTX 4090 : cudaMallocAsync. The single pin was honoured.

Endpoint ids for both passes, in case they are still in your logs: ugculdhag081uh (the two-card list, placed on the Blackwell MIG slice) and h4rz8tmjkq35fu (the single 4090 pin, honoured).

Two problems.

1. The list form appears to be advisory while the single form is binding, and nothing distinguishes them. The Flash API reference presents gpu= as accepting "a GpuGroup, a GpuType, or a list of either" as though the three were the same kind of constraint.

2. Nothing in the API reports which card a worker actually ran on. The substitution is invisible from the outside: gpuTypeIds keeps reporting what was requested. This was only detectable because the container asks ComfyUI's /system_stats for devices[0].name and returns it in the response body. NVIDIA_VISIBLE_DEVICES is not usable for this — on a serverless worker it returned the literal string void.

Why it matters beyond curiosity. The two cards in this measurement differ by 2.8× in render time for identical work, and they are on the same price tier. A workload that asks for specific cards and silently gets a third has no way to know its per-unit cost model is wrong.

Suggestion. Document the difference between the list and single forms, and expose the placed GPU type on the worker or the job record.