Skip to content

fix: retain executor owner handles across restarts - #492

Open
my-vegetable-has-exploded wants to merge 3 commits into
ray-project:masterfrom
my-vegetable-has-exploded:fix/reuse-executor-owner-handle-upstream
Open

my-vegetable-has-exploded wants to merge 3 commits into
ray-project:masterfrom
my-vegetable-has-exploded:fix/reuse-executor-owner-handle-upstream

Conversation

@my-vegetable-has-exploded

Copy link
Copy Markdown
Contributor

What does this PR do?

Reuse the actor slot's original owner handle when an executor restarts, while assigning a new Spark executor ID. Keep rejecting restart requests for removed or unknown slots.

Why is this change needed?

The restart handler currently calls Ray.getActor and replaces the stored owner handle with a named-lookup handle. This can make the original owner handle eligible for garbage collection while the executor is still needed. Reusing the existing handle preserves its lifetime and avoids an unnecessary lookup.

Related Issue

None.

Type of Change

  • Bug fix
  • New feature
  • Documentation update
  • Test update
  • Refactoring
  • Build / CI / dependency change
  • Other

How was this tested?

With JDK 17 and the default Spark 4.0.0 / Scala 2.13 configuration:

cd core
mvn clean package -DskipTests
mvn -pl raydp-main -am verify -Dscalastyle.includeTestSourceDirectory=true

All 12 core/shim modules build successfully. All 17 tests pass, including three new RPC regression tests for consecutive restarts, removed slots, and unknown slots. Java and Scala style checks pass. The new tests fail on the original handler because it requires a named lookup; they verify handle identity and recovery bookkeeping without starting Ray.

Checklist

  • I have added or updated tests if needed
  • I have updated documentation if needed (no user-facing documentation changes needed)
  • I have run relevant tests locally
  • I have checked that this change is backward compatible

epsilonwang added 2 commits October 16, 2026 09:30
Signed-off-by: epsilonwang <epsilonwang@didiglobal.com>
Signed-off-by: epsilonwang <epsilonwang@didiglobal.com>
@pang-wu

pang-wu commented Nov 2, 2026

Copy link
Copy Markdown
Collaborator

@my-vegetable-has-exploded can you rebase from master?

@my-vegetable-has-exploded

Copy link
Copy Markdown
Contributor Author

@my-vegetable-has-exploded can you rebase from master?

done!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants