fix(inference): serialise the deferred weight move across threads

[tuncaybahadir]

Summary

The model's weights move from CPU to the accelerator on the first predict(), inference() or export() call (_move_model_context_to_device). That move takes no lock. When several threads share one model (for example one camera channel per thread) and reach that first call together, they run overlapping in-place nn.Module.to() calls on the same module.

Repro on a cold segmentation model: four threads call predict() at once. There is no exception and no warning, but 13 of the 573 state_dict tensors come out corrupted (values around 1e24–1e28), and the model then returns no detections on every frame. The sequential baseline returns 23–25 detections per frame on the same inputs. I did not trace the byte-level mechanism, only the outcome.

Changes

• _move_model_context_to_device takes a module-level lock for the move and re-checks the device under it, so later callers see the move as done instead of repeating it.
• The torch.inference_mode(False) guard around the move is unchanged.
• Adds a CHANGELOG.md entry.

Verification

• New test TestMoveModelContextConcurrency::test_concurrent_first_calls_move_exactly_once: four threads race on a cold stand-in module whose .to() yields for 0.2 s. It failed before the fix (assert 4 == 1) and passes after.
• pytest src/ tests/ scripts/ -n 3 -m "not gpu and not coco17 and not integration and not xla and not tpu" ...: 6531 passed, 128 skipped.
• pre-commit run --all-files: passed.
• Not verified: the GPU test suite (no GPU in the CPU-only environment I used for the suite). The corrupted-weights repro above was run on an RTX 4070 Ti SUPER before writing the fix.

[codecov]

Codecov Report

❌ Patch coverage is 93.54839% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 90%. Comparing base (d502116) to head (fd156da).

Additional details and impacted files

@@           Coverage Diff           @@
##           develop   #1564   +/-   ##
=======================================
  Coverage       90%     90%           
=======================================
  Files          136     136           
  Lines        18070   18088   +18     
=======================================
+ Hits         16219   16236   +17     
- Misses        1851    1852    +1     

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Copilot]

Copilot review overview

🟡 Changes recommended

The pre-lock device check can still let inference proceed while later parameters are moving.

Review effort: Balanced
Findings: 1 · 2

Open (3)

• Lock inference until all parameters finish moving · New
• Add required explanation comment above lock constant · New
• Add required Examples doctest for delayed move helper · New

What changed in this PR

Serializes deferred model device moves to prevent concurrent weight corruption.

Changes:

• Adds locking and device re-checking around lazy device placement.
• Adds a concurrency regression test.
• Documents the fix in the changelog.

File	Description
src/​rfdetr/​detr.py	Adds device-move synchronization.
tests/​inference/​test_model_device_move.py	Tests concurrent initial moves.
CHANGELOG.md	Records the concurrency fix.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[tuncaybahadir]

Addressed the Copilot review in 66e24f2:

• Lock bypass (high): confirmed — nn.Module.to() rewrites parameters one by one, so a lock-free first_param.device != target check could see the first parameter already moved and let inference start on a half-moved model. The device check now runs under _DEVICE_MOVE_LOCK, so callers arriving mid-move wait for it. The regression test now models this (the first parameter reports the target immediately, completed flips only after the whole move) and asserts no thread returns before completion; it failed on the previous commit (assert not True) and passes now.
• #: constant comment and the Examples doctest on the test helper added as requested.

pre-commit clean; tests/inference 397 passed on CPU.

[tuncaybahadir]

GPU check of the fix (RTX 4070 Ti SUPER, torch 2.14 + CUDA 13.0, RFDETRSegLarge with the pretrained rf-detr-seg-large weights, one 1920x1080 frame, threshold 0.3). A cold model, four threads calling predict() at once behind a barrier, compared with a sequential single-thread run of a second instance of the same model:

	detections per thread	corrupted state_dict tensors
sequential reference	18	—
this branch (66e24f2), 4 threads	18, 18, 18, 18	0 / 573

Without the lock (same script on a branch that does not contain it), four threads left 11 of 573 tensors corrupted and every thread returned 0 detections. This is one run on one frame; I have not run the repo's GPU test suite.