Skip to content

Make proc_open() pipes overlapped named pipes on Windows when a provider asks - #5

Closed
EdmondDantes wants to merge 1 commit into
bukka:io_hooks_pocfrom
true-async:io-hooks-overlapped-pipes
Closed

EdmondDantes wants to merge 1 commit into
bukka:io_hooks_pocfrom
true-async:io-hooks-overlapped-pipes

Conversation

@EdmondDantes

Copy link
Copy Markdown

On Windows proc_open() makes its pipes with CreatePipe(), anonymous pipes that take no overlapped I/O. A provider cannot perform a read or write on them, so a coroutine reading a child's stdout blocks the whole thread, and stream_select() on pipes only polls. This commit lets the extension that installs the provider ask for overlapped pipes, made as libuv's uv_spawn() makes them. Nothing changes unless it asks.

What changes

  • php_io_overlapped_pipes (main/php_io_hooks.h), set at MINIT by the extension that installs the provider. With it set, the parent's end of each 'pipe' descriptor is a named pipe created with FILE_FLAG_OVERLAPPED | FILE_FLAG_FIRST_PIPE_INSTANCE and 64 KB buffers, and the child's end a synchronous CreateFile(). The parent's end opens in binary mode, since its reads and writes go around the CRT. STDIN and exec pipes stay as they are.
  • Reads and writes of such a pipe go to the provider as READ and WRITE ops with a new flag, PHP_IO_OP_F_PIPE; the flag's comment says what the provider must report (data, the writer's close, EPIPE). Without a provider, or when it answers Unsupported, the thread waits on an overlapped request whose event has the low-order bit set, so no packet goes to anyone's completion port.
  • stream_select(): a pipe in the read set asks the provider for readiness (a POLL op with the flag, which the Ring answers with a zero-byte overlapped read); a pipe in the write set is always writable; the except set is answered as php_select() answers it.
  • A cast (stream_cast(), a pipe given to another proc_open()) while another call has an op on the stream fails with "Concurrent access to a stream"; an orphaned op is waited for first.
  • New queue op release() and php_io_queues_release(). Right before CreateProcessW(), every queue of the thread lets go of each overlapped pipe passed to the child; the Ring calls ior_release_handle() (ior, IOCP), so the child's own overlapped I/O on the pipe does not post to the parent's port. A pipe with an op in flight fails proc_open() with "Concurrent access to a stream", as the cast does. Where ior cannot release (an op it still holds, Windows before 8.1) its packet filter drops the child's packets.
  • After CreateProcessW() the pipe is marked handed out (PHP_STREAM_OPTION_OVERLAPPED_PIPE, HAND_OUT): the parent's later reads and writes on it skip the provider and wait on the thread.
    Needs ior with ior_release_handle(): libior/ior@main...true-async:ior:release-handle (the PR text is in ior-drop-foreign-packets-pr.md).

Tests

No new phpt here: only an extension can set php_io_overlapped_pipes at MINIT, so the core test run never makes such a pipe. TrueAsync's Windows suite covers it (provider tests for pipe reads, writes, select, cast and hand-out, proc_open() from coroutines).

Run on Windows 11, MSVC, ZTS, this commit merged into io_hooks_poc 566a683 with ior release-handle: PHP's proc_open*, proc_* and ext/standard/tests/streams/hooks with no failures, with TrueAsync loaded and without; TrueAsync's Windows suite 1225 passed, 0 failed, on Debug_TS and Release_TS. Linux (io_uring): PHP's streams, proc, pcntl and fibers tests, 743 passed, 0 failed.

…der asks

An anonymous pipe takes no overlapped I/O, so a coroutine reading a
proc_open() pipe blocked the thread, and stream_select() on pipes only
polled. With php_io_overlapped_pipes set (by a provider at MINIT), the
parent's end of each 'pipe' descriptor is a named pipe created with
FILE_FLAG_OVERLAPPED, FILE_FLAG_FIRST_PIPE_INSTANCE and 64 KB buffers,
and the child's end is a synchronous CreateFile() handle, as libuv's
uv_spawn() makes them. The parent's end opens in binary mode, since its
reads and writes go around the CRT. Without the setting nothing changes;
STDIN and exec pipes stay as they are.

Reads and writes of such a pipe (PHP_IO_OP_F_PIPE) go to the provider.
Without one, or when it answers Unsupported, the thread waits on an
overlapped request whose event has the low-order bit set, so no packet
is queued; a write to a pipe whose reader closed fails with EPIPE.
stream_set_blocking(false) is kept on the stream, since the handle has
no non-blocking mode: a read then returns only the bytes the pipe holds.

stream_select() asks the provider for read readiness with a zero-byte
overlapped read; a pipe in the write set is always writable, and one in
the except set is answered as php_select() answers it. When another
process holds a passed pipe, a member is a descriptor no socket poll
takes, or no provider can wait for a pipe, php_select() takes the call
(for the time left). php_select() judges every pipe by the bytes it
holds, since an overlapped pipe's handle is signaled whatever it holds.
A cast fails as a concurrent access while another call has an op on the
stream, and waits for an orphaned one.

Right before CreateProcessW(), every queue of the thread lets go of each
pipe passed to the child (queue op release(); the Ring calls
ior_release_handle()), so the child's own overlapped I/O does not post
to the parent's port. A pipe with an op in flight fails proc_open() as a
concurrent access; where ior cannot release the handle (an op of its own
still on it, a system before Windows 8.1), it drops the child's packets
instead. After CreateProcessW() the pipe is marked handed out and the
parent's later calls on it skip the provider.
@EdmondDantes
EdmondDantes requested a review from bukka as a code owner October 9, 2026 07:47
@bukka

bukka commented Oct 9, 2026

Copy link
Copy Markdown
Owner

I don't see why this should be extension specific and only enabled on MINIT. I think it should follow the same logic as for overlapped files where we use overlapped flag (and other bits) if the provider is enabled.

Also it should be abstracted to php_io_ interface which means that there should be something like php_io_pipe that will take care of creation. In other words, please limit the modifications to proc_open.c

@bukka

bukka commented Oct 10, 2026

Copy link
Copy Markdown
Owner

I merged it in php@6c25ec4 and did follow up refactorings: php@665acf8 to address the comment above and php@4e58da8 to make it a bit nicer.

@bukka bukka closed this Oct 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants