Repository navigation
Conversation
Contributor
Author
|
ci failure appears unrelated |
Member
yes maybe a startup race in containerd FIFO handling. Created #7301 |
crazy-max
reviewed
Oct 19, 2026
Zaczero
force-pushed
the
fix-exec-stream-failure
branch
from
October 20, 2026 00:18
fb3e1c9 to
afeb757
Compare
Process receivers only observe their caller context and deregistration, so Start and Wait can hang after the shared ExecProcess receiver exits. Transport errors cancel the forwarder context without notifying them; EOF and cancellation leave even that context live. Cancel the forwarder on every receive error and let process receivers return its cause. Preserve normal deregistration and Release semantics, and use the existing output-handler cleanup to unblock pending stdin. Exercise EOF and three gRPC status errors over a real stream with two processes starting or waiting, plus successful completion before stream termination. Check that Release still reports only failures other than EOF and cancellation. Signed-off-by: Kamil Monicz <kamil@monicz.dev>
Zaczero
force-pushed
the
fix-exec-stream-failure
branch
from
October 20, 2026 00:24
afeb757 to
3c23c1a
Compare
Keep sending stdin EOF during cleanup so the gateway can finish its Done handshake, but ignore send failures when the input copy was interrupted by closing the pipe. Preserve send errors when the input reaches EOF naturally. Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
Member
|
While checking the completed-process cases with stdin left open, I reproduced a case where the cleanup EOF send replaces a successful process result with A loopback gRPC test also reproduced this when the server sent The additiional commit keeps sending EOF because the gateway needs it before sending I also added coverage for stream termination between $ CGO_ENABLED=1 go test -mod=vendor -race ./frontend/gateway/grpcclient -count=100 -timeout=120s
ok github.com/moby/buildkit/frontend/gateway/grpcclient 15.551s |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The gateway client's
Container.StartandContainerProcess.Waitcan hang indefinitely when the sharedExecProcessstream ends while their caller contexts remain live. The receive loop stops delivering messages, but the per-process receivers only observe their caller's context and deregistration. Transport errors cancel the forwarder's errgroup context without reaching those receivers; EOF and gRPC cancellation return nil without canceling it.Cancel the forwarder's context with the receive error on every stream termination, including EOF and cancellation, and have process receivers observe that context. Return the cancellation cause directly from the mailbox, so Start and Wait propagate it through their existing error paths. Keep the existing Release behavior for EOF and cancellation, and use the existing output-handler cleanup to unblock pending stdin.
The regression uses a real gRPC server on a loopback listener. It terminates the stream with EOF, Unavailable, Canceled and DeadlineExceeded, both before Started and while waiting for completion, with two processes sharing the stream and stdin left open. It checks that both calls return the transport failure while their caller context is still live. Separate completed-process cases verify that Exit and Done followed by stream termination still produce successful waits. Each case also checks that
Releasestill returns nil for EOF and cancellation and the transport error otherwise.Validation:
go test -race ./frontend/gateway/grpcclient -count=100 -timeout=120spasses, including the four completed-process cases.golangci-lint run ./frontend/gateway/grpcclient/...