Skip to content

refactor(task-graph): read tasks through a lazily loaded graph - #943

Merged
wan9chi merged 2 commits into
mainfrom
claude/lazy-loading-task-graph-6fbaee
Nov 5, 2026
Merged

wan9chi merged 2 commits into
mainfrom
claude/lazy-loading-task-graph-6fbaee

Conversation

@wan9chi

@wan9chi wan9chi commented Nov 5, 2026 •

Copy link
Copy Markdown
Member

Motivation

vp run loads every package's vite.config.* before it plans anything, even when the run touches one package. To load configs only when a run needs them (#944), every reader of the task graph has to go through methods that can load what they read. Today the planner, the task selector and the session read petgraph nodes and edges directly, and call load_task_graph() first so that everything is loaded.

This PR moves every reader onto such methods without changing what loads: everything still loads up front.

Changes

  • vt_graph::lazy_graph::LazyGraph: a generic graph that loads a partition's nodes the first time any of them is read, and resolves a node's edges the first time they're read. Resolving edges can read nodes, loading their partitions, but not other nodes' edges, so a dependsOn cycle across packages can't make two loads wait on each other. Nodes live in an append-only elsa::FrozenMap, so reads return plain references.
  • TaskLoader holds the per-package code that was in IndexedTaskGraph::load: config loading, its checks, and dependsOn resolution. The root config loads with the package list, since its settings apply to the whole workspace.
  • IndexedTaskGraph exposes only reads: task, tasks, depends_on, query_tasks, get_script_hook and list_tasks. Tasks are keyed by TaskId (package index and task name) instead of petgraph node indices, and a query returns a BTreeMap of the tasks it selected.
  • The planner, the task selector and the session use those reads. TaskGraphLoader, Session::ensure_task_graph_loaded, TaskNodeIndex and the petgraph TaskGraph type are removed, and Session::task_graph() returns the graph. vite-plus's bench/benches/workspace_load.rs calls ensure_task_graph_loaded, which becomes task_graph().

IndexedTaskGraph::load still reads every config and resolves every dependsOn, through the same reads, so output and errors don't change and no snapshot changes. One difference: when several packages have config errors, the first one reported can differ. Before, every config was read and checked before any task was created; now each package's config and tasks load together, the root's first, then the others in package order. The error that lists tasks setting cache fields at the top level still covers every package; plan fixture top_level_cache_fields_workspace_error checks that with the root and a package.

LazyGraph has unit tests for what each read loads, a cycle across partitions, a retry after a failed load, and duplicate edges.

🤖 Generated with Claude Code


Note

Medium Risk
Touches core task-graph loading, querying, and planning with widespread async API changes, though behavior is intended to stay the same aside from multi-error ordering.

Overview
Refactors the workspace task graph so all reads go through async APIs built on a new LazyGraph, instead of a fully built petgraph and TaskGraphLoader.

LazyGraph loads graph partitions on first access (nodes per package, edges on first dependsOn read), with stable node refs via elsa::FrozenMap. TaskLoader holds the per-package config/script loading and dependency resolution that used to live in IndexedTaskGraph::load.

IndexedTaskGraph is now keyed by TaskId (not TaskNodeIndex). Callers use task, tasks, depends_on, query_tasks, list_tasks, and get_script_hook. Query results are a BTreeMap of QueriedTask instead of a petgraph of indices.

Session caches the graph in tokio::sync::OnceCell (**task_graph()** replaces ensure_task_graph_loaded). vt_plan takes a loaded IndexedTaskGraph directly and plans from TaskNode references.

IndexedTaskGraph::load still runs load_all so every package and dependsOn is validated up front (same outcomes; which config error surfaces first can change when several packages fail). TopLevelCacheFieldsCollector::absorb aggregates misplaced cache fields across packages; a new plan snapshot fixture covers root + package errors together.

Reviewed by Cursor Bugbot for commit 8584d82. Bugbot is set up for automated code reviews on this repo. Configure here.

@socket-security

socket-security Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Addedcargo/​elsa@​1.11.210010093100100

View full report

@wan9chi
wan9chi added this pull request to stack #945 November 5, 2026 10:53
@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

fspy benchmark

linux

dynamic/launch             change  -0.33%  [ -3.62% ..  +3.71%]  overhead   +57.02%
dynamic/access             change  +0.86%  [ -3.03% ..  +6.65%]  overhead   +19.91%
dynamic/access-relative    change  -0.58%  [ -3.42% ..  +1.80%]  overhead   +71.63%
dynamic/access-contended   change  +1.54%  [ -6.70% .. +10.25%]  overhead    +9.45%
static/launch              change  +0.86%  [ -4.28% ..  +5.18%]  overhead  +202.22%
static/access              change  +0.41%  [ -0.97% ..  +2.04%]  overhead +1479.74%
static/access-relative     change  +0.34%  [ -1.68% ..  +2.85%]  overhead +1971.84%
static/access-contended    change  -0.22%  [ -1.53% ..  +1.52%]  overhead +1832.25%

macos

dynamic/launch             change  -0.81%  [ -3.77% ..  +3.22%]  overhead  +244.39%
dynamic/access             change  +0.34%  [-10.69% ..  +3.42%]  overhead    +5.94%
dynamic/access-relative    change  -0.34%  [ -8.23% ..  +5.88%]  overhead  +258.59%
dynamic/access-contended   change  -2.91%  [-49.31% ..  +3.27%]  overhead    -0.73%

windows

dynamic/launch             change  +0.37%  [ -6.25% ..  +7.97%]  overhead   +25.21%
dynamic/access             change  +0.00%  [ -2.00% ..  +2.55%]  overhead    +1.69%
dynamic/access-relative    change  +0.34%  [ -2.99% ..  +5.48%]  overhead    +2.12%
dynamic/access-contended   change  -0.68%  [-13.21% .. +11.36%]  overhead    -0.52%

Comment thread crates/vt_graph/src/lib.rs Resolved

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 424f37e. Configure here.

Comment thread crates/vt_graph/src/lib.rs Resolved
wan9chi and others added 2 commits November 5, 2026 20:40
Tasks now live in `LazyGraph`, a generic graph that loads one package's
tasks the first time any of them is read and resolves a task's
`dependsOn` the first time it's read. `IndexedTaskGraph` exposes only
reads (`task`, `tasks`, `depends_on`, `query_tasks`, ...) built on it, so
the planner, the task selector and the session no longer see petgraph or
call a load step: the `TaskGraphLoader` trait and
`Session::ensure_task_graph_loaded` are gone, and `Session::task_graph`
returns the graph.

`IndexedTaskGraph::load` still reads every config and resolves every
`dependsOn` up front, so errors and output are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…her packages'

The root package loads before the rest, so when its tasks set cache
fields at the top level, `load` returned that error before visiting the
other packages, and the error listed only the root's tasks. Pass it to
`load_all`, which adds the other packages' tasks, as before this stack.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@wan9chi
wan9chi force-pushed the claude/lazy-loading-task-graph-6fbaee branch from 619ae94 to 8584d82 Compare November 5, 2026 12:43
@wan9chi
wan9chi merged commit 264449a into main Nov 5, 2026
21 of 22 checks passed
@wan9chi
wan9chi deleted the claude/lazy-loading-task-graph-6fbaee branch November 5, 2026 14:12
wan9chi added a commit that referenced this pull request Nov 5, 2026
…asks (#944)

## Motivation

`vp run` loads every package's `vite.config.*` before it plans, so a run
in one package waits for all of them, and a broken config in any package
fails every run in the workspace. With every task read going through the
lazy graph (#943), configs can load only when a run reaches their tasks.

## Changes

- `IndexedTaskGraph::load` no longer loads every task and resolves every
`dependsOn` up front. It loads the package list and the root config;
another package's config loads when a run first reads its tasks, through
the task query, a `dependsOn` or a nested `vp run`.
- Checks that covered the whole workspace now run when a package's
config loads: `cache` or `enablePrePostScripts` outside the root config,
a task named like a `package.json` script, a `dependsOn` naming a
missing task, and cache settings at the top level of a task. That last
error used to list every such task in the workspace; it now lists the
loading package's, as the snapshot of
`top_level_cache_fields_workspace_error` shows.
- `vp run` without a task lists every task, so it still loads every
config.

E2E fixture `unreached_package_config`: `packages/broken` sets `cache`,
which only the root config may set, or, in the second case, has a
`vite-task.json` that isn't valid JSON. The first commit records today's
output: every run fails with that package's error, including `vt run -t
build` in `packages/app`, which only runs `@test/app` and `@test/lib`.
In the second, that run succeeds, while `vt run -r build` and a
`dependsOn` on `@test/broken#build` still fail.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Medium Risk**
> Changes when config validation runs and which errors surface, which
could hide issues until a package is first reached in a run.
> 
> **Overview**
> **`vp run` no longer loads every package's `vite.config.*` up front.**
`IndexedTaskGraph::load` now loads the package list and workspace root
config only; each other package's config loads the first time a run
reads that package's tasks (task selection, `-r`, `dependsOn`, or
listing all tasks).
> 
> Runs scoped to one package **ignore broken or invalid configs in
packages the graph never touches**, so startup is faster and unrelated
packages can't block the run. When a run **does** reach a package,
config errors (invalid JSON, `cache` outside the root, misplaced cache
fields, etc.) still fail at load time as before. **`vp run` without a
task** still enumerates every task, so it still loads every config.
> 
> Workspace-wide aggregation of top-level cache field errors is removed:
those messages now list only tasks from the package being loaded. E2E
fixture **`unreached_package_config`** documents the new behavior, and
the changelog is updated.
> 
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
3a205d0. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
wan9chi added a commit to voidzero-dev/vite-plus that referenced this pull request Nov 5, 2026
Point the vite-task git dependencies at 650d697, which loads a
package's `vite.config.*` only when a run reaches its tasks
(voidzero-dev/vite-task#943, #944), so the preview workflow produces a
build to measure against the 0b37399 baseline. This exists only to
generate a preview build and is not meant to be merged.

`Session::ensure_task_graph_loaded` is now `Session::task_graph`, which
takes `&self`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant