Repository navigation
Make proc_open() pipes overlapped named pipes on Windows when a provider asks - #5
Closed
EdmondDantes wants to merge 1 commit into
Closed
EdmondDantes wants to merge 1 commit into
EdmondDantes wants to merge 1 commit into
Conversation
…der asks An anonymous pipe takes no overlapped I/O, so a coroutine reading a proc_open() pipe blocked the thread, and stream_select() on pipes only polled. With php_io_overlapped_pipes set (by a provider at MINIT), the parent's end of each 'pipe' descriptor is a named pipe created with FILE_FLAG_OVERLAPPED, FILE_FLAG_FIRST_PIPE_INSTANCE and 64 KB buffers, and the child's end is a synchronous CreateFile() handle, as libuv's uv_spawn() makes them. The parent's end opens in binary mode, since its reads and writes go around the CRT. Without the setting nothing changes; STDIN and exec pipes stay as they are. Reads and writes of such a pipe (PHP_IO_OP_F_PIPE) go to the provider. Without one, or when it answers Unsupported, the thread waits on an overlapped request whose event has the low-order bit set, so no packet is queued; a write to a pipe whose reader closed fails with EPIPE. stream_set_blocking(false) is kept on the stream, since the handle has no non-blocking mode: a read then returns only the bytes the pipe holds. stream_select() asks the provider for read readiness with a zero-byte overlapped read; a pipe in the write set is always writable, and one in the except set is answered as php_select() answers it. When another process holds a passed pipe, a member is a descriptor no socket poll takes, or no provider can wait for a pipe, php_select() takes the call (for the time left). php_select() judges every pipe by the bytes it holds, since an overlapped pipe's handle is signaled whatever it holds. A cast fails as a concurrent access while another call has an op on the stream, and waits for an orphaned one. Right before CreateProcessW(), every queue of the thread lets go of each pipe passed to the child (queue op release(); the Ring calls ior_release_handle()), so the child's own overlapped I/O does not post to the parent's port. A pipe with an op in flight fails proc_open() as a concurrent access; where ior cannot release the handle (an op of its own still on it, a system before Windows 8.1), it drops the child's packets instead. After CreateProcessW() the pipe is marked handed out and the parent's later calls on it skip the provider.
Owner
|
I don't see why this should be extension specific and only enabled on MINIT. I think it should follow the same logic as for overlapped files where we use overlapped flag (and other bits) if the provider is enabled. Also it should be abstracted to php_io_ interface which means that there should be something like php_io_pipe that will take care of creation. In other words, please limit the modifications to proc_open.c |
Owner
|
I merged it in php@6c25ec4 and did follow up refactorings: php@665acf8 to address the comment above and php@4e58da8 to make it a bit nicer. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
On Windows
proc_open()makes its pipes withCreatePipe(), anonymous pipes that take no overlapped I/O. A provider cannot perform a read or write on them, so a coroutine reading a child's stdout blocks the whole thread, andstream_select()on pipes only polls. This commit lets the extension that installs the provider ask for overlapped pipes, made as libuv'suv_spawn()makes them. Nothing changes unless it asks.What changes
php_io_overlapped_pipes(main/php_io_hooks.h), set at MINIT by the extension that installs the provider. With it set, the parent's end of each'pipe'descriptor is a named pipe created withFILE_FLAG_OVERLAPPED | FILE_FLAG_FIRST_PIPE_INSTANCEand 64 KB buffers, and the child's end a synchronousCreateFile(). The parent's end opens in binary mode, since its reads and writes go around the CRT. STDIN andexecpipes stay as they are.PHP_IO_OP_F_PIPE; the flag's comment says what the provider must report (data, the writer's close, EPIPE). Without a provider, or when it answers Unsupported, the thread waits on an overlapped request whose event has the low-order bit set, so no packet goes to anyone's completion port.stream_select(): a pipe in the read set asks the provider for readiness (a POLL op with the flag, which the Ring answers with a zero-byte overlapped read); a pipe in the write set is always writable; the except set is answered asphp_select()answers it.stream_cast(), a pipe given to anotherproc_open()) while another call has an op on the stream fails with "Concurrent access to a stream"; an orphaned op is waited for first.release()andphp_io_queues_release(). Right beforeCreateProcessW(), every queue of the thread lets go of each overlapped pipe passed to the child; the Ring callsior_release_handle()(ior, IOCP), so the child's own overlapped I/O on the pipe does not post to the parent's port. A pipe with an op in flight failsproc_open()with "Concurrent access to a stream", as the cast does. Where ior cannot release (an op it still holds, Windows before 8.1) its packet filter drops the child's packets.CreateProcessW()the pipe is marked handed out (PHP_STREAM_OPTION_OVERLAPPED_PIPE, HAND_OUT): the parent's later reads and writes on it skip the provider and wait on the thread.Needs ior with
ior_release_handle(): libior/ior@main...true-async:ior:release-handle (the PR text is inior-drop-foreign-packets-pr.md).Tests
No new phpt here: only an extension can set
php_io_overlapped_pipesat MINIT, so the core test run never makes such a pipe. TrueAsync's Windows suite covers it (provider tests for pipe reads, writes, select, cast and hand-out,proc_open()from coroutines).Run on Windows 11, MSVC, ZTS, this commit merged into
io_hooks_poc566a683 with iorrelease-handle: PHP'sproc_open*,proc_*andext/standard/tests/streams/hookswith no failures, with TrueAsync loaded and without; TrueAsync's Windows suite 1225 passed, 0 failed, on Debug_TS and Release_TS. Linux (io_uring): PHP's streams, proc, pcntl and fibers tests, 743 passed, 0 failed.