Skip to content

feat(task-graph): load a package's config only when a run reads its tasks - #944

Merged
wan9chi merged 2 commits into
claude/lazy-loading-task-graph-6fbaeefrom
claude/lazy-load-task-configs
Nov 5, 2026
Merged

wan9chi merged 2 commits into
claude/lazy-loading-task-graph-6fbaeefrom
claude/lazy-load-task-configs

Conversation

@wan9chi

@wan9chi wan9chi commented Nov 5, 2026 •

Copy link
Copy Markdown
Member

Motivation

vp run loads every package's vite.config.* before it plans, so a run in one package waits for all of them, and a broken config in any package fails every run in the workspace. With every task read going through the lazy graph (#943), configs can load only when a run reaches their tasks.

Changes

  • IndexedTaskGraph::load no longer loads every task and resolves every dependsOn up front. It loads the package list and the root config; another package's config loads when a run first reads its tasks, through the task query, a dependsOn or a nested vp run.
  • Checks that covered the whole workspace now run when a package's config loads: cache or enablePrePostScripts outside the root config, a task named like a package.json script, a dependsOn naming a missing task, and cache settings at the top level of a task. That last error used to list every such task in the workspace; it now lists the loading package's, as the snapshot of top_level_cache_fields_workspace_error shows.
  • vp run without a task lists every task, so it still loads every config.

E2E fixture unreached_package_config: packages/broken sets cache, which only the root config may set, or, in the second case, has a vite-task.json that isn't valid JSON. The first commit records today's output: every run fails with that package's error, including vt run -t build in packages/app, which only runs @test/app and @test/lib. In the second, that run succeeds, while vt run -r build and a dependsOn on @test/broken#build still fail.

🤖 Generated with Claude Code


Note

Medium Risk
Changes when config validation runs and which errors surface, which could hide issues until a package is first reached in a run.

Overview
vp run no longer loads every package's vite.config.* up front. IndexedTaskGraph::load now loads the package list and workspace root config only; each other package's config loads the first time a run reads that package's tasks (task selection, -r, dependsOn, or listing all tasks).

Runs scoped to one package ignore broken or invalid configs in packages the graph never touches, so startup is faster and unrelated packages can't block the run. When a run does reach a package, config errors (invalid JSON, cache outside the root, misplaced cache fields, etc.) still fail at load time as before. vp run without a task still enumerates every task, so it still loads every config.

Workspace-wide aggregation of top-level cache field errors is removed: those messages now list only tasks from the package being loaded. E2E fixture unreached_package_config documents the new behavior, and the changelog is updated.

Reviewed by Cursor Bugbot for commit 3a205d0. Bugbot is set up for automated code reviews on this repo. Configure here.

@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

fspy benchmark

linux

dynamic/launch             change  +0.24%  [ -5.27% ..  +6.20%]  overhead   +59.23%
dynamic/access             change  +1.24%  [ -2.79% ..  +6.19%]  overhead   +21.51%
dynamic/access-relative    change  -0.27%  [ -5.10% ..  +5.25%]  overhead   +70.67%
dynamic/access-contended   change  -2.79%  [-16.59% ..  +6.43%]  overhead    +3.54%
static/launch              change  +0.02%  [ -5.81% ..  +5.36%]  overhead  +198.40%
static/access              change  +0.40%  [ -5.89% ..  +7.85%]  overhead +1346.15%
static/access-relative     change  -0.02%  [ -8.92% ..  +9.47%]  overhead +2007.73%
static/access-contended    change  -0.86%  [ -6.84% ..  +2.64%]  overhead +1559.27%

macos

dynamic/launch             change  +0.10%  [ -4.07% ..  +4.40%]  overhead  +247.89%
dynamic/access             change  +0.34%  [ -3.74% ..  +3.08%]  overhead    +5.49%
dynamic/access-relative    change  -0.42%  [ -5.23% ..  +6.61%]  overhead  +255.77%
dynamic/access-contended   change  +1.95%  [ -7.01% ..  +8.96%]  overhead    +2.47%

windows

dynamic/launch             change  -0.23%  [ -2.78% ..  +3.21%]  overhead   +27.36%
dynamic/access             change  +0.00%  [ -1.34% ..  +1.14%]  overhead    +0.96%
dynamic/access-relative    change  -0.18%  [ -1.08% ..  +0.94%]  overhead    +0.90%
dynamic/access-contended   change  -0.18%  [ -1.09% ..  +1.59%]  overhead    +1.25%

@wan9chi
wan9chi force-pushed the claude/lazy-load-task-configs branch 2 times, most recently from 65950ea to cc5065b Compare November 5, 2026 12:21
wan9chi and others added 2 commits November 5, 2026 20:41
Add e2e cases where `packages/broken` sets `cache`, which only the root
config may set, or has a `vite-task.json` that isn't valid JSON. Every
run fails with that package's error, including `vt run -t build` in
another package that never needs it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…asks

`IndexedTaskGraph::load` no longer reads every package's config and
resolves every `dependsOn` up front; it loads the package list and the
root config, and the lazy graph loads the rest as runs read them. A run
in one package no longer loads, or fails on, configs of packages it
never reaches. Checks that ran over the whole workspace, such as `cache`
outside the root config, now run when that package's config loads, so
the error for misplaced top-level cache fields lists only that
package's tasks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@wan9chi
wan9chi force-pushed the claude/lazy-load-task-configs branch from cc5065b to 3a205d0 Compare November 5, 2026 12:43
@wan9chi
wan9chi merged commit 650d697 into main Nov 5, 2026
23 checks passed
@wan9chi
wan9chi deleted the claude/lazy-load-task-configs branch November 5, 2026 14:12
wan9chi added a commit that referenced this pull request Nov 5, 2026
## Motivation

`vp run` loads every package's `vite.config.*` before it plans anything,
even when the run touches one package. To load configs only when a run
needs them (#944), every reader of the task graph has to go through
methods that can load what they read. Today the planner, the task
selector and the session read petgraph nodes and edges directly, and
call `load_task_graph()` first so that everything is loaded.

This PR moves every reader onto such methods without changing what
loads: everything still loads up front.

## Changes

- `vt_graph::lazy_graph::LazyGraph`: a generic graph that loads a
partition's nodes the first time any of them is read, and resolves a
node's edges the first time they're read. Resolving edges can read
nodes, loading their partitions, but not other nodes' edges, so a
`dependsOn` cycle across packages can't make two loads wait on each
other. Nodes live in an append-only `elsa::FrozenMap`, so reads return
plain references.
- `TaskLoader` holds the per-package code that was in
`IndexedTaskGraph::load`: config loading, its checks, and `dependsOn`
resolution. The root config loads with the package list, since its
settings apply to the whole workspace.
- `IndexedTaskGraph` exposes only reads: `task`, `tasks`, `depends_on`,
`query_tasks`, `get_script_hook` and `list_tasks`. Tasks are keyed by
`TaskId` (package index and task name) instead of petgraph node indices,
and a query returns a `BTreeMap` of the tasks it selected.
- The planner, the task selector and the session use those reads.
`TaskGraphLoader`, `Session::ensure_task_graph_loaded`, `TaskNodeIndex`
and the petgraph `TaskGraph` type are removed, and
`Session::task_graph()` returns the graph. vite-plus's
`bench/benches/workspace_load.rs` calls `ensure_task_graph_loaded`,
which becomes `task_graph()`.

`IndexedTaskGraph::load` still reads every config and resolves every
`dependsOn`, through the same reads, so output and errors don't change
and no snapshot changes. One difference: when several packages have
config errors, the first one reported can differ. Before, every config
was read and checked before any task was created; now each package's
config and tasks load together, the root's first, then the others in
package order. The error that lists tasks setting cache fields at the
top level still covers every package; plan fixture
`top_level_cache_fields_workspace_error` checks that with the root and a
package.

`LazyGraph` has unit tests for what each read loads, a cycle across
partitions, a retry after a failed load, and duplicate edges.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Medium Risk**
> Touches core task-graph loading, querying, and planning with
widespread async API changes, though behavior is intended to stay the
same aside from multi-error ordering.
> 
> **Overview**
> Refactors the workspace task graph so **all reads go through async
APIs** built on a new **`LazyGraph`**, instead of a fully built
`petgraph` and `TaskGraphLoader`.
> 
> **`LazyGraph`** loads graph partitions on first access (nodes per
package, edges on first `dependsOn` read), with stable node refs via
**`elsa::FrozenMap`**. **`TaskLoader`** holds the per-package
config/script loading and dependency resolution that used to live in
**`IndexedTaskGraph::load`**.
> 
> **`IndexedTaskGraph`** is now keyed by **`TaskId`** (not
**`TaskNodeIndex`**). Callers use **`task`**, **`tasks`**,
**`depends_on`**, **`query_tasks`**, **`list_tasks`**, and
**`get_script_hook`**. Query results are a **`BTreeMap`** of
**`QueriedTask`** instead of a petgraph of indices.
> 
> **Session** caches the graph in **`tokio::sync::OnceCell`**
(`**task_graph()`** replaces **`ensure_task_graph_loaded`**).
**`vt_plan`** takes a loaded **`IndexedTaskGraph`** directly and plans
from **`TaskNode`** references.
> 
> **`IndexedTaskGraph::load`** still runs **`load_all`** so every
package and **`dependsOn`** is validated up front (same outcomes;
**which config error surfaces first** can change when several packages
fail). **`TopLevelCacheFieldsCollector::absorb`** aggregates misplaced
cache fields across packages; a new plan snapshot fixture covers root +
package errors together.
> 
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
8584d82. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant