Repository navigation
[Python] Fix max_models_per_worker_hint not being enforced in KeyedModelHandler - #40500
Open
schizophrenicmaniac wants to merge 1 commit into
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #40500 +/- ##
============================================
+ Coverage 56.16% 56.23% +0.06%
Complexity 2288 2288
============================================
Files 1124 1124
Lines 178336 178362 +26
Branches 1489 1489
============================================
+ Hits 100170 100300 +130
+ Misses 75645 75541 -104
Partials 2521 2521
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Contributor
|
Assigning reviewers: R: @shunping for label python. Note: If you would like to opt out of this review, comment Available commands:
The PR bot will only process comments in the main thread (not review comments). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The
max_models_per_worker_hintlimit for a KeyedModelHandler with several model handlers was never actually enforced.run_inferencecreated a newthreading.Lock()on every call, so acquiring it always succeeded. Each deserialized copy of the handler (one per DoFn instance, plus new copies when bundle processors are recreated) still had the hint set, so every copy added it to the shared_ModelHandlerManageragain. In the repro from the issue, five handler copies raised the limit to 5 and all three models stayed loaded with a hint of 1.This change makes
_ModelHandlerManager.increment_max_modelstake an optionalprocess_idand count each process only once.KeyedModelHandler.run_inferencepassesos.getpid(). The pid is computed on the caller's side because the manager is proxied from a separate MultiProcessShared process. Calls without a process id work as before.I added a unit test for the manager and one based on the repro in the issue (five pickled handler copies sharing one manager, hint of 1). The repro test fails on master and passes with this change. I also added a CHANGES.md entry.
Fixes #40468