Repository navigation
Conversation
robhogan
added a commit
that referenced
this pull request
Oct 25, 2026
Without Watchman, `Watcher` starts its backends before the crawl and buffers their events, so that a change made during the crawl isn't missed: https://github.com/react/metro/blob/56efab302200568c919ac1b3c28dacf8d84fc692/packages/metro-file-map/src/Watcher.js#L118-L127 For `FallbackWatcher` that means walking every root to watch each directory, and then the node crawler walks the same tree again. This lets the crawl do both. Just before the crawler lists a directory it passes it to the watcher, which watches it - the order the watcher's own walk uses, so a change still can't slip between the two. Directories the crawl skips as ignored are passed on too, and the watcher walks those itself unless it ignores them as well (eg `node_modules` without `retainAllFiles`). The hand-off is an internal second parameter of the built-in node crawler, so there's no API change. A `crawlerFactory` crawler, a scoped recrawl and a root added with `addRoot` keep the watcher's own walk. The longer-term shape for this is a backend that owns both crawling and watching, which this would move into. ## Benchmark This takes another 25% off watch-mode startup on Linux and Windows. With the rest of this stack, watch-mode startup without Watchman goes from 1,894ms to 508ms (-73%) on Linux, and from 2,580ms to 539ms (-79%) on Windows. On macOS, which uses `NativeWatcher`, PRs 2-5 measure the same as the first PR (124ms). | | `main` | #2107 | This PR | vs `main` | vs #2107 | |---|---|---|---|---|---| | Linux: Watch-mode start, warm cache | 1,894ms | 685ms | 508ms | -73.2% (-73.4% to -72.9%) | -25.9% (-26.4% to -25.4%) | | Windows: Watch-mode start, warm cache | 2,580ms | 721ms | 539ms | -79.1% (-79.3% to -78.3%) | -25.2% (-27.3% to -22.3%) | | Linux: Watch-mode start, cold cache | 1,876ms | 652ms | 472ms | -74.8% (-75.2% to -74.5%) | -27.6% (-28.5% to -26.7%) | | Windows: Watch-mode start, cold cache | 2,567ms | 695ms | 516ms | -79.9% (-80.2% to -79.8%) | -25.7% (-28.2% to -24.9%) | | Linux: Heap after a watch-mode start | 31.8MB | 28.3MB | 28.3MB | -11.1% (-11.1% to -11.0%) | -0.0% (-0.0% to -0.0%) | | Windows: Heap after a watch-mode start | 31.9MB | 28.4MB | 28.4MB | -11.1% (-11.1% to -11.1%) | -0.0% (-0.0% to -0.0%) | | Linux: RSS after a watch-mode start | 197.4MB | 176.3MB | 170.9MB | -13.4% (-15.0% to -12.0%) | -3.1% (-4.6% to -1.6%) | | Windows: RSS after a watch-mode start | 205.7MB | 182.6MB | 184.6MB | -10.3% (-10.7% to -9.4%) | +1.1% (+0.9% to +1.4%) | ## Changelog ``` - **[Performance]**: metro-file-map: Set up file watching during the crawl instead of after it, on Linux and Windows without Watchman ``` ## Test plan New `Watcher-test.js` cases: the crawl's directories reach a `FallbackWatcher`, and a recrawl or a `crawlerFactory` crawler leaves it to walk. New `watch-integration-test.js` case for a directory the crawl skips but the watcher doesn't. Benchmark as in #2104.
This was referenced Oct 25, 2026
robhogan
added a commit
that referenced
this pull request
Oct 25, 2026
`NativeWatcher.startWatching` resolves as soon as `fs.watch` returns, but on macOS that can be before any changes are reported. libuv only signals another thread to (re)create the FSEvents stream, and the stream reports only changes made after it starts: https://github.com/libuv/libuv/blob/2b4b918d3381100854250c89d5159d4206daafb7/src/unix/fsevents.c#L362-L372 So a change made straight after startup can be silently missed, and since #2054 started non-Watchman backends before the crawl, so can one made during the crawl. On a slow host the gap is long enough to hit regularly - it's the remaining flake in the Native watcher integration tests, where the suite's first event never arrives. 14 of 297 macOS test jobs since #1974 hit it, across all three Node versions. This has `Watcher` confirm each root's watch before relying on it. Once every backend has started, it writes a health check file into each root and waits for it to be reported, as `checkHealth()` already does for the project root. If a watch can't be confirmed within 10s, Metro warns and carries on. Roots added later with `addRoot` are confirmed the same way. A backend opts in by returning `false` from a new static `isLiveOnStart()`, which only `NativeWatcher` does - Watchman subscribes from the crawl's clock, and `FallbackWatcher`'s watches are live as soon as `fs.watch` returns. It's done in `Watcher`, after all the backends have started, because `NativeWatcher`s for multiple roots share one FSEvents stream, which restarts as each root is added. This adds ~10ms to a watch-mode start on macOS without Watchman (145ms -> 155ms on our benchmark app). Changelog: ``` - **[Fix]**: Fix changes made immediately after startup sometimes being missed on macOS ``` Test plan: New `Watcher-test.js` cases cover confirming each root before the crawl, warning and carrying on when a watch can't be confirmed, confirming a root added later, and leaving backends that are live on start alone. The backend integration tests now wait for a cookie of their own, as `Watcher` does. To reproduce the race, I patched `NativeWatcher` to drop all events for its first 300ms. A `FileMap` in watch mode with two roots, writing a file to each as soon as `build()` resolves, sees neither change without this PR (3/3 runs) and both with it (3/3). The `metro-file-map` suite passes on macOS, Linux and Windows. Benchmark (AI-driven): the harness from #2104, 8 interleaved rounds on an M4 Pro Mac mini, n=32 warm watch-mode starts per variant (+6.7%, 95% CI +5.2% to +8.0%).
robhogan
added a commit
that referenced
this pull request
Oct 25, 2026
Without Watchman, `Watcher` starts its backends before the crawl, so that nothing changed during the crawl is missed: https://github.com/react/metro/blob/56efab302200568c919ac1b3c28dacf8d84fc692/packages/metro-file-map/src/Watcher.js#L118-L127 For `FallbackWatcher` that means walking the tree once to watch it and again to crawl it. This lets the crawl do both: the crawler hands each directory to the watcher just before listing it, the order the watcher's own walk uses. Directories the crawl skips as ignored are handed over too, for the watcher to walk unless it ignores them as well (eg `node_modules` without `retainAllFiles`). This is internal to the built-in node crawler. A `crawlerFactory` crawler, a recrawl and a root added with `addRoot` keep the watcher's own walk. ## Benchmark This takes another 28-30% off watch-mode startup on Linux and Windows. With the rest of this stack, watch-mode startup without Watchman goes from 1,896ms to 510ms (-73%) on Linux, and from 2,554ms to 522ms (-80%) on Windows. On macOS, which uses `NativeWatcher`, PRs 2-5 measure within noise of #2104. | | `main` | #2107 | This PR | vs `main` | vs #2107 | |---|---|---|---|---|---| | Linux: Watch-mode start, warm cache | 1,896ms | 707ms | 510ms | -73.1% (-73.4% to -72.8%) | -27.9% (-28.6% to -27.2%) | | Windows: Watch-mode start, warm cache | 2,554ms | 721ms | 522ms | -79.6% (-79.7% to -79.4%) | -27.7% (-28.0% to -27.2%) | | Linux: Watch-mode start, cold cache | 1,882ms | 699ms | 491ms | -73.9% (-74.5% to -73.5%) | -29.7% (-31.1% to -28.5%) | | Windows: Watch-mode start, cold cache | 2,547ms | 709ms | 507ms | -80.1% (-80.2% to -80.0%) | -28.5% (-28.9% to -27.9%) | | Linux: Heap after a watch-mode start | 31.8MB | 28.3MB | 28.3MB | -11.1% (-11.1% to -11.1%) | -0.1% (-0.1% to -0.1%) | | Windows: Heap after a watch-mode start | 31.9MB | 28.4MB | 28.4MB | -11.1% (-11.1% to -11.1%) | -0.1% (-0.1% to -0.1%) | | Linux: RSS after a watch-mode start | 197.6MB | 175.2MB | 174.3MB | -11.8% (-12.3% to -11.3%) | -0.5% (-0.9% to +0.0%) | | Windows: RSS after a watch-mode start | 206.1MB | 182.9MB | 183.9MB | -10.7% (-11.3% to -10.3%) | +0.6% (+0.4% to +0.8%) | ## Changelog ``` - **[Performance]**: metro-file-map: Set up file watching during the crawl instead of after it, on Linux and Windows without Watchman ``` ## Test plan New unit tests, and a `watch-integration-test.js` case for a directory the crawl skips but the watcher doesn't. Benchmark as in #2104.
Without Watchman, the node crawler `lstat`s every file on every start, so that `getDifference` can tell what changed while Metro wasn't running: https://github.com/react/metro/blob/56efab302200568c919ac1b3c28dacf8d84fc692/packages/metro-file-map/src/crawlers/node/index.js#L83-L96 Most of those files are never read, and an mtime only matters for a file with something derived from it (a SHA-1 or plugin data) - that's all a change can invalidate. This takes the idea in @kitten's #1686 and makes it hold across restarts. A regular file is crawled without a stat, and its mtime is recorded when data is first derived from it, from a stat taken before it's read. A file with no mtime then has nothing that could be stale, so a start only stats the files that had an mtime before. - A recrawl while watching still stats every file, because consumers like `PackageCache` hold contents read outside the file map. - Symlinks are still read eagerly - every lookup through a link needs its target. - `linkStats()` gives a null `modifiedTime` and `size` for a file that hasn't been stat'd. - `CACHE_BREAKER` is bumped, or an existing cache would keep every file stat'd. ## Benchmark Without Watchman, this takes 40-45% off a cold or warm start on Linux and Windows (Linux warm: 649ms -> 371ms), 9-15% off on macOS, and 4-7% off RSS. Watch mode gains less on Linux and Windows, where setting up the watcher dominates - the rest of this stack deals with that. The cost is a stat when a file is first read: 10-20% more on a first SHA-1 pass over the bundle's files (8-47ms here), once per file per cache. | | `main` | This PR | vs `main` | |---|---|---|---| | macOS: Cold start | 156ms | 132ms | -15.4% (-23.3% to -8.7%) | | Linux: Cold start | 644ms | 360ms | -44.0% (-45.7% to -43.3%) | | Windows: Cold start | 369ms | 208ms | -43.7% (-46.6% to -42.7%) | | macOS: Warm start | 142ms | 129ms | -9.4% (-10.7% to -8.3%) | | Linux: Warm start | 649ms | 371ms | -42.8% (-43.3% to -42.5%) | | Windows: Warm start | 357ms | 225ms | -37.2% (-37.7% to -36.7%) | | macOS: Watch-mode start, warm cache | 143ms | 130ms | -9.6% (-10.6% to -8.7%) | | Linux: Watch-mode start, warm cache | 1,896ms | 1,617ms | -14.7% (-15.9% to -14.0%) | | Windows: Watch-mode start, warm cache | 2,554ms | 2,429ms | -4.9% (-5.4% to -4.3%) | | macOS: First SHA-1 of the bundle's 3,125 files | 75ms | 83ms | +10.4% (+8.6% to +11.8%) | | Linux: First SHA-1 of the bundle's 3,125 files | 110ms | 131ms | +19.5% (+17.2% to +20.5%) | | Windows: First SHA-1 of the bundle's 3,125 files | 270ms | 317ms | +17.2% (+13.7% to +19.5%) | | macOS: RSS after a warm start | 142.6MB | 132.1MB | -7.4% (-7.9% to -6.0%) | | Linux: RSS after a warm start | 175.1MB | 164.1MB | -6.3% (-6.9% to -6.0%) | | Windows: RSS after a warm start | 149.6MB | 144.0MB | -3.8% (-4.1% to -3.3%) | ## Changelog ``` - **[Performance]**: metro-file-map: Start faster without Watchman, by not stat-ing files whose contents Metro hasn't read ``` ## Test plan New unit tests, and a `stats-between-starts-test.js` that edits files between two runs of FileMap over a real directory and cache. SHA-1s, eager and lazy plugin data and a retargeted symlink are recomputed, and an unread file is stat'd only once it's read. These include the cases #1686 gets wrong. Benchmark app (AI-driven harness, used throughout this stack): a React Native 0.87.1 app with 23 common dependencies, 34k files in the file map and 3.1k modules in its iOS dev bundle, started through Metro's `DependencyGraph` with `resolver.useWatchman: false`. Each round runs, per variant: 2 cold starts, a watch-mode session that computes the SHA-1s of the bundle's files (as transforming them does) and saves them to the cache, 3 warm starts and 3 watch-mode starts from that cache, and 2 watch-mode cold starts. 10 interleaved rounds per OS, alternating the variant order, then medians and bootstrap 95% CIs. Heap is measured after a forced GC. Machines: Mac mini M4 Pro (macOS, APFS, Node 26.11.1), Mac mini i5-4278U (Linux Mint 22.3, ext4, Node 24.21.0) and a Ryzen 7 7730U laptop (Windows 11, NTFS, Defender on, Node 24.21.0).
robhogan
added a commit
that referenced
this pull request
Oct 26, 2026
Without Watchman, `Watcher` starts its backends before the crawl, so that nothing changed during the crawl is missed: https://github.com/react/metro/blob/56efab302200568c919ac1b3c28dacf8d84fc692/packages/metro-file-map/src/Watcher.js#L118-L127 For `FallbackWatcher` that means walking the tree once to watch it and again to crawl it. This lets the crawl do both: the crawler hands each directory to the watcher just before listing it, the order the watcher's own walk uses. Directories the crawl skips as ignored are handed over too, for the watcher to walk unless it ignores them as well (eg `node_modules` without `retainAllFiles`). This is internal to the built-in node crawler. A `crawlerFactory` crawler, a recrawl and a root added with `addRoot` keep the watcher's own walk. ## Benchmark This takes another 28-30% off watch-mode startup on Linux and Windows. With the rest of this stack, watch-mode startup without Watchman goes from 1,896ms to 510ms (-73%) on Linux, and from 2,554ms to 522ms (-80%) on Windows. On macOS, which uses `NativeWatcher`, PRs 2-5 measure within noise of #2104. | | `main` | #2107 | This PR | vs `main` | vs #2107 | |---|---|---|---|---|---| | Linux: Watch-mode start, warm cache | 1,896ms | 707ms | 510ms | -73.1% (-73.4% to -72.8%) | -27.9% (-28.6% to -27.2%) | | Windows: Watch-mode start, warm cache | 2,554ms | 721ms | 522ms | -79.6% (-79.7% to -79.4%) | -27.7% (-28.0% to -27.2%) | | Linux: Watch-mode start, cold cache | 1,882ms | 699ms | 491ms | -73.9% (-74.5% to -73.5%) | -29.7% (-31.1% to -28.5%) | | Windows: Watch-mode start, cold cache | 2,547ms | 709ms | 507ms | -80.1% (-80.2% to -80.0%) | -28.5% (-28.9% to -27.9%) | | Linux: Heap after a watch-mode start | 31.8MB | 28.3MB | 28.3MB | -11.1% (-11.1% to -11.1%) | -0.1% (-0.1% to -0.1%) | | Windows: Heap after a watch-mode start | 31.9MB | 28.4MB | 28.4MB | -11.1% (-11.1% to -11.1%) | -0.1% (-0.1% to -0.1%) | | Linux: RSS after a watch-mode start | 197.6MB | 175.2MB | 174.3MB | -11.8% (-12.3% to -11.3%) | -0.5% (-0.9% to +0.0%) | | Windows: RSS after a watch-mode start | 206.1MB | 182.9MB | 183.9MB | -10.7% (-11.3% to -10.3%) | +0.6% (+0.4% to +0.8%) | ## Changelog ``` - **[Performance]**: metro-file-map: Set up file watching during the crawl instead of after it, on Linux and Windows without Watchman ``` ## Test plan New unit tests, and a `watch-integration-test.js` case for a directory the crawl skips but the watcher doesn't. Benchmark as in #2104.
robhogan
added a commit
that referenced
this pull request
Oct 26, 2026
Without Watchman, `Watcher` starts its backends before the crawl, so that nothing changed during the crawl is missed: https://github.com/react/metro/blob/56efab302200568c919ac1b3c28dacf8d84fc692/packages/metro-file-map/src/Watcher.js#L118-L127 For `FallbackWatcher` that means walking the tree once to watch it and again to crawl it. This lets the crawl do both: the crawler hands each directory to the watcher just before listing it, the order the watcher's own walk uses. Directories the crawl skips as ignored are handed over too, for the watcher to walk unless it ignores them as well (eg `node_modules` without `retainAllFiles`). This is internal to the built-in node crawler. A `crawlerFactory` crawler, a recrawl and a root added with `addRoot` keep the watcher's own walk. ## Benchmark This takes another 28-30% off watch-mode startup on Linux and Windows. With the rest of this stack, watch-mode startup without Watchman goes from 1,896ms to 510ms (-73%) on Linux, and from 2,554ms to 522ms (-80%) on Windows. On macOS, which uses `NativeWatcher`, PRs 2-5 measure within noise of #2104. | | `main` | #2107 | This PR | vs `main` | vs #2107 | |---|---|---|---|---|---| | Linux: Watch-mode start, warm cache | 1,896ms | 707ms | 510ms | -73.1% (-73.4% to -72.8%) | -27.9% (-28.6% to -27.2%) | | Windows: Watch-mode start, warm cache | 2,554ms | 721ms | 522ms | -79.6% (-79.7% to -79.4%) | -27.7% (-28.0% to -27.2%) | | Linux: Watch-mode start, cold cache | 1,882ms | 699ms | 491ms | -73.9% (-74.5% to -73.5%) | -29.7% (-31.1% to -28.5%) | | Windows: Watch-mode start, cold cache | 2,547ms | 709ms | 507ms | -80.1% (-80.2% to -80.0%) | -28.5% (-28.9% to -27.9%) | | Linux: Heap after a watch-mode start | 31.8MB | 28.3MB | 28.3MB | -11.1% (-11.1% to -11.1%) | -0.1% (-0.1% to -0.1%) | | Windows: Heap after a watch-mode start | 31.9MB | 28.4MB | 28.4MB | -11.1% (-11.1% to -11.1%) | -0.1% (-0.1% to -0.1%) | | Linux: RSS after a watch-mode start | 197.6MB | 175.2MB | 174.3MB | -11.8% (-12.3% to -11.3%) | -0.5% (-0.9% to +0.0%) | | Windows: RSS after a watch-mode start | 206.1MB | 182.9MB | 183.9MB | -10.7% (-11.3% to -10.3%) | +0.6% (+0.4% to +0.8%) | ## Changelog ``` - **[Performance]**: metro-file-map: Set up file watching during the crawl instead of after it, on Linux and Windows without Watchman ``` ## Test plan New unit tests, and a `watch-integration-test.js` case for a directory the crawl skips but the watcher doesn't. Benchmark as in #2104.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Without Watchman, the node crawler
lstats every file on every start, so thatgetDifferencecan tell what changed while Metro wasn't running:metro/packages/metro-file-map/src/crawlers/node/index.js
Lines 83 to 96 in 56efab3
Most of those files are never read, and an mtime only matters for a file with something derived from it (a SHA-1 or plugin data) - that's all a change can invalidate.
This takes the idea in @kitten's #1686 and makes it hold across restarts. A regular file is crawled without a stat, and its mtime is recorded when data is first derived from it, from a stat taken before it's read. A file with no mtime then has nothing that could be stale, so a start only stats the files that had an mtime before.
PackageCachehold contents read outside the file map.linkStats()gives a nullmodifiedTimeandsizefor a file that hasn't been stat'd.CACHE_BREAKERis bumped, or an existing cache would keep every file stat'd.Benchmark
Without Watchman, this takes 40-45% off a cold or warm start on Linux and Windows (Linux warm: 649ms -> 371ms), 9-15% off on macOS, and 4-7% off RSS. Watch mode gains less on Linux and Windows, where setting up the watcher dominates - the rest of this stack deals with that. The cost is a stat when a file is first read: 10-20% more on a first SHA-1 pass over the bundle's files (8-47ms here), once per file per cache.
mainmainChangelog
Test plan
New unit tests, and a
stats-between-starts-test.jsthat edits files between two runs of FileMap over a real directory and cache. SHA-1s, eager and lazy plugin data and a retargeted symlink are recomputed, and an unread file is stat'd only once it's read. These include the cases #1686 gets wrong.Benchmark app (AI-driven harness, used throughout this stack): a React Native 0.87.1 app with 23 common dependencies, 34k files in the file map and 3.1k modules in its iOS dev bundle, started through Metro's
DependencyGraphwithresolver.useWatchman: false. Each round runs, per variant: 2 cold starts, a watch-mode session that computes the SHA-1s of the bundle's files (as transforming them does) and saves them to the cache, 3 warm starts and 3 watch-mode starts from that cache, and 2 watch-mode cold starts. 10 interleaved rounds per OS, alternating the variant order, then medians and bootstrap 95% CIs. Heap is measured after a forced GC. Machines: Mac mini M4 Pro (macOS, APFS, Node 26.11.1), Mac mini i5-4278U (Linux Mint 22.3, ext4, Node 24.21.0) and a Ryzen 7 7730U laptop (Windows 11, NTFS, Defender on, Node 24.21.0).