DRA: pin topology-aware containers to their claims

[bart0sh]

Pinning containers to their claims

Seventh PR of the DRA series, on top of #821. Two commits: the
topology-aware policy pins a container holding claims to the claims' CPUs, and
the test42-dra-devices e2e test checks the pinning. Nothing in pkg/resmgr
changes.

What a claiming container runs on

A container holding claims runs on the CPUs of its claims. A container with no
CPU request of its own, the usual case, runs on its claims only. A container
with a CPU request also gets its normal allocation: exclusive CPUs, and shared
CPUs if it has a shared portion. A container with reserved CPUs, such as one
in kube-system, gets its reserved CPUs as well if it has a CPU request.

Hiding hyperthreads applies to the exclusive CPUs only. The claimed CPUs are
pinned as they are, since they are exactly what the claim asked for. The
claimed CPUs must also follow a container's strict topology hints. The
scheduler picks the claim's CPUs and the policy cannot move them, so a claim
that fails a strict hint fails the allocation.

Where a claiming container goes

A claiming container with normal CPUs gets its grant from the pool of the
claimed CPUs, the same pool the claim grant itself came from, rather than from
the pool scoring would pick. So its memory comes from the claims' NUMA node,
and so do its own exclusive and shared CPUs.

A container with reserved CPUs stays in the root pool, like other reserved
containers, and its memory comes from there. The claim's pool usually has no
reserved CPUs, so the container would get normal CPUs there instead.

The CPU shares follow the rule for mixed allocations: they come from the
shared or reserved portion if there is one, otherwise from all the CPUs the
container runs on.

A besteffort container holding a claim is also not counted as a zero-request
shared container of its pool. Otherwise it would hold back a milli-CPU of
shared capacity there, and the last CPUs of its node could not be claimed.

A container's claims are fixed for its lifetime: the kubelet prepares them
before the container is created and unprepares them only after all of the
pod's containers have stopped, so no running container ever holds a released
claim. updateSharedAllocations leaves a claiming container with no shared
portion alone. One with a shared portion is updated like any other container,
and keeps its claimed CPUs. Containers with cpuPreserve keep the cpuset the
runtime gave them, as before.

How a container is linked to its claims

The policy reads the DRA_CPUSET_<claim UID> variable #821 puts in the
container's environment. A container holds a claim if it has the variable for
a claim grant the policy holds, and its value names that grant's CPUs. The
value is compared as a cpuset, so 0-1 and 1,0 name the same CPUs. A
variable for an unknown claim, or naming other CPUs, is ignored. No cache or
interface change is needed: the policy looks the variables up with the
existing GetEnv.

The claims are looked up once, when the container is allocated, and kept on
its grant. Placement, pinning, CPU shares and the shared-pool checks all read
them from there, so pool scoring does not look up every claim for every
container.

This is a temporary approach. The pod spec can set the variable too, so a
pod which knows a claim's UID and CPUs can run on them without holding the
claim. A container's CDI devices name its claims and are set by the runtime
alone, so they are the right link. But not every maintained runtime release
reports them to NRI yet: containerd does from v2.3, CRI-O does not yet. We
switch to CDI device names once every maintained release of both runtimes
reports them.

What is not here

These are known limitations, each lifted by a later PR in the series:

• Claims do not survive a restart. They are saved with the other grants,
  but the policy resets its saved state on start, so after the plugin
  restarts their CPUs are free again and can be granted twice.
• The NodeResourceTopology CR goes stale after a claim, as nothing
  refreshes it when claimed CPUs leave the free supply.
• No DeviceClass ships yet, so a claim needs one written by the user.
  The e2e test creates its own.

Verification

• go test -race ./cmd/plugins/topology-aware/... ./pkg/resmgr/... passes,
  and golangci-lint finds no issues.
• The new policy test covers claiming containers that are besteffort, shared,
  exclusive, exclusive and shared, reserved, and reserved with no CPU request,
  a container holding two claims, a variable naming other CPUs, one naming an
  unknown claim, and a claiming container keeping its claim while another
  claim comes and goes. For a claiming container it also checks the pool, the
  memory zone, the shared or reserved portion and the CPU shares. Each of
  these changes fails it: dropping the claimed CPUs from applyGrant, pinning
  a claiming container without its own CPUs, skipping every claiming container
  in updateSharedAllocations, placing a reserved container in the claim's
  pool, and zeroing the container's own CPU request in newRequest.
• Another test places a besteffort container holding a claim and then claims
  the rest of its node. It fails if the container is counted as a zero-request
  shared container. A third test checks the claimed CPUs against a strict
  topology hint, with an aligned and a misaligned claim.
• A claim variable naming the claim's CPUs in another form still links the
  container. A container holding a claim stays on it across a reconfiguration.
  Comparing the value as a string, or not carrying the claims onto the grant
  or through a reconfiguration, fails the tests.
• test42-dra-devices passes on n4c16 with Kubernetes 1.37, containerd
  v2.4.1, and the DRAConsumableCapacity and DRANodeAllocatableResources
  gates. Every claiming container with no CPU request runs on exactly its
  claim's CPUs and on memory of its claim's node: 12-14 on node 3, 4-5 and 6-7
  on node 1, then 4-5 for the third claim once the first is released. A claim
  on the node the besteffort container runs on, node 0 in this run, gets 0-1.
  Two containers sharing one claim both run on its CPUs, 0-1, with node 0's
  memory. The besteffort container still runs on no claimed CPU and gets
  released CPUs back. A pod with a 100m CPU request of its own and a claim on
  node 2 runs on CPUs 8-11, all CPUs of node 2: the claimed CPUs plus the
  shared ones, with node 2's memory.

[Copilot]

Copilot review overview

🟢 Approval recommended

The implementation matches the stated behavior and includes focused unit and end-to-end coverage.

Review effort: Balanced
Findings: None

What changed in this PR

Pins topology-aware containers to DRA-claimed CPUs while preserving exclusive allocations.

Changes:

• Detects claims through CDI-provided environment variables.
• Excludes claiming containers from shared CPU updates.
• Adds unit and end-to-end coverage for pinning and overcommit behavior.

File	Description
cmd/​plugins/​topology-aware/​policy/​dra.go	Resolves container claims to CPU sets.
cmd/​plugins/​topology-aware/​policy/​pools.go	Applies claimed CPU pinning.
cmd/​plugins/​topology-aware/​policy/​dra_test.go	Tests pinning and overcommit handling.
test/​e2e/​.../​code.var.sh	Verifies claim CPU placement end to end.
test/​e2e/​.../​dra-pod.yaml.in	Supports multiple claim-sharing containers.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Copilot review overview

🟡 Changes recommended

Releasing the final claim does not restore the container’s original fractional CPU allocation or accounting.

Review effort: Balanced
Findings: 1

Open (1)

• Preserve shared CPU request after the final claim is released · New

[Copilot]

Copilot review overview

🟡 Changes recommended

Strict topology validation does not account for claimed CPUs in the runnable cpuset.

Review effort: Balanced
Findings: 1

Open (1)

• Strict topology validation omits claimed CPUs · New

Resolved since last review (1)

• Preserve shared CPU request after the final claim is released

[klihub]

Hmm, why do the claimed CPUs replace the shared set ? Is that the best/most natural choice ? At least in the original proto the claimed CPUs were added to the container-exclusive set.

So basically the proto semantics were:

1. Claimed CPUs are always treated as exclusively allocated ones from the containers point of view
2. However claimed CPUs are shared between multiple containers referencing the same claim.
3. If a container both claims CPUs and asks for them using the normal CPU request/limit combo, then the effective CPU set of the container is the union of a traditional CPU allocation and the claimed CPUs.

@bart0sh @askervin @kad Is there a reason why we couldn't go with those ? Either some technical limitation or some other reason why different semantics make more sense ?

[askervin]

I don't see any reason for changing the proto semantics. I implemented their balloons-equivalent semantics in #828.

Unlike in the initial balloons implementation, here a container could benefit from getting the information on which CPUs belong to shared claim(s). I'd assume that a container having a shared CPU claim and CPUs of its own might have different uses for those two CPU sets.

[bart0sh]

No technical reason. I followed the community DRA CPU driver, where a container runs only on its claimed CPUs. But that driver has no shared pool, so your model definitely fits better.

So I would change the code this way:

• A container with no CPU request (the usual case) runs only on its claimed CPUs, as now.
• A container that also has a CPU request gets its normal allocation as well: exclusive + claimed CPUs, plus shared CPUs if it has a shared portion.
• A container with reserved CPUs gets reserved + claimed CPUs.

Does this sound correct?

[bart0sh]

@klihub Does this look like a correct fix: https://github.com/containers/nri-plugins/compare/3b8593f36ec927993decfca838dda405cdd98a62..97309a507f33c031d2c74c5c307a2534d08cca7d ?

[Copilot]

🟡 Changes recommended

Reserved and fractional CPU handling contradicts the documented claim-placement behavior.

2 open findings

• Claimed CPUs do not override reserved pool placement · New
• Fractional claim holders incorrectly retain shared or reserved CPUs · New

1 resolved since last review

• Strict topology validation omits claimed CPUs

🧠 Review effort: Balanced

---

Give feedback about Copilot approvals in this survey to enter a drawing for a $150 gift card.

[Copilot]

🟢 Approval recommended

The implementation is consistent across allocation paths and is supported by focused unit and end-to-end coverage.

0 open findings

2 resolved since last review

• Claimed CPUs do not override reserved pool placement
• Fractional claim holders incorrectly retain shared or reserved CPUs

🧠 Review effort: Balanced

---

Give feedback about Copilot approvals in this survey to enter a drawing for a $150 gift card.