Autonomous task executor that reads a spec file, decomposes it into ordered steps via LLM, and executes each step through ACP sessions with test verification, retries, and progress checkpointing.
TaskRunner is a product-layer superset of the workflow run substrate. It keeps ownership of planning, approval gates, retries, replanning, test verification, git/worktree coordination, persistence, pause/resume, and cleanup. The workflow service supplies the common run identity, event history, source/provenance, and saved-definition invocation used by all workflow-like execution. It does not replace or reinterpret TaskRunner execution.
Supports multiple concurrent tasks, interactive tool approval, per-step session isolation with full memory injection, git-coordinated step commits and reverts, independent review via actual diffs, cycle detection, disk persistence across restarts, activity-aware stall detection, and semaphore-bounded parallel execution to prevent resource exhaustion.
Each task captures execution_context before asynchronous planning: stable
member/store identity, template, retention mode and application provenance.
Async entrypoints offload persistent session capture with asyncio.to_thread
before further planning or request-body awaits; a supplied carrier bypasses capture.
They first snapshot live retention restrictions through the shared workflow
admission helper, so parent closure during capture cannot loosen the child's mode.
The task's own record carries it through steps, review, retry and restart.
Changing or closing the parent chat cannot select another member. Each child
receives that same context before provider allocation. Explicit member selection
uses the existing member's store; a missing member never falls back to Global.
Ordinary tasks without a member retain their existing V1 memory behavior.
Persistent tasks store one complete task record. Incognito and temporary tasks keep their bodies, steps, outputs and execution state in memory only: checkpoints, progress files, task history and failure-lesson extraction do not retain them. Temporary tasks also skip memory preparation and automatic lessons. Restricted mode cannot be weakened by a retry or child. If reading the task registry fails, the runner reports incomplete recovery and refuses to overwrite unknown records.
Internal HTTP callers inherit their authenticated execution. Request labels and arbitrary database paths cannot override it. Normal source-file checks, tool allow/deny rules, application permissions and owner-only aggregate controls remain independent. Cancellation resolves a canonical task ID once and operates on that same task. Member scope itself is not a confidentiality permission.
The task runner is split into an orchestrator, whose TaskRunner binds its members from the taskrunner_parts/ owners (§ TaskRunner owner modules), plus 4 focused helper modules under src/kiro_crew/:
taskrunner.py (orchestrator: the TaskRunner class)
├── taskrunner_parts/ (TaskRunner member bodies, bound back by name)
├── task_models.py (data models + constants)
├── task_planner.py (LLM decomposition + task parsing + parallel grouping)
├── task_executor.py (task execution + retries + tests + self-review)
└── task_reporter.py (status + notifications + progress checkpoints + resume context)
| Module | Class/Functions | Responsibility |
|---|---|---|
task_models.py |
TaskStatus, Task, WorkingMemory, Project, constants |
Shared data types and configuration constants |
task_planner.py |
decompose(), parse_tasks(), normalize_cross_group_deps(), group_parallel_tasks(), plan_to_chat_context(), update_plan_tasks(), auto_name() |
LLM spec decomposition, task parsing, dependency normalization, parallel grouping, plan-to-chat formatting |
task_executor.py |
execute_task(), build_task_prompt(), self_review(), run_tests(), check_context() |
Task execution with retry/recovery budgets, prompt building, context compaction, test running, self-review |
task_reporter.py |
NotifyCallback, notify(), build_status(), save_progress(), load_checkpoint(), build_resume_context(), format_completion_summary() |
Notification contract, status reporting, TASK_PROGRESS.md checkpointing, resume context |
taskrunner.py |
TaskRunner |
Orchestrator — the class, its constructor and properties; its run lifecycle, _try_replan, watchdog and run persistence (_persist_runs/_load_runs) are bound from the taskrunner_parts/ owners below; delegates decomposition/execution/reporting to the helper modules |
TaskRunner's own behaviour lives in the taskrunner_parts/ package, one
responsibility per module. A member's body is a module-level function there whose
first parameter is self (annotated TaskRunner for type checkers only), and the
class binds it by name under one comment header per owner (_execute_tasks = _run_steps._execute_tasks, staticmethod(...) for a static method). Every member
stays an attribute of the one class, so a patch of a method on the class or on an
instance reaches every caller, a TaskRunner.__new__ build works, and the run state
stays on the instance; each owner's docstring names the instance fields its members
write and read. A bound member's __qualname__ and __module__ name its owner
(nothing reads them); a class an owner defines (WorkflowInitializing,
WorkflowRunPublisher, _MonotonicAge) keeps kiro_crew.taskrunner as its
__module__, so error chains and tracebacks name the path callers import it from.
Owners log under the kiro_crew.taskrunner logger, never import taskrunner.py at
run time, and import only owners listed before them in _OWNERS. taskrunner.py
keeps every project import it had and every name the tests patch through it (stdlib
imports that only moved code uses live in its owners) and serves the module-level
names it defined from their owners; a write through it reaches every owner binding the
same object (kiro_crew._facade_mirror). test/test_taskrunner_bindings.py pins the
member inventory, the binding and the reach.
| TaskRunner owner | Owns |
|---|---|
taskrunner_parts/run_inputs.py |
what a run is started from: the workspace_dir gate, the spec read gate (_read_spec_text, _read_spec_prefix), audited YAML decomposition and the unattended sources that may not fall back from it, and the live taskrunner.* config each run entry adopts (_refresh_from_config, _clamp_parallel_steps) |
taskrunner_parts/run_scope.py |
what belongs to one run: its session namespace, execution identity and history key, the per-run auto-approve grant, and teardown of its sessions and runtime |
taskrunner_parts/workflow_port.py |
the shared workflow-history port: WorkflowRunPublisher, WorkflowInitializing, the attach/defer gate, and begin, plan, rebind, finalize and delete publication |
taskrunner_parts/taskq_bridge.py |
the durable task queue: attaching admission, adopting orphaned rows, the run's container row and each step's admission |
taskrunner_parts/run_registry.py |
the runs.json registry: ordered snapshots, restore and crash recovery, the TASK_PROGRESS.md checkpoints and the status view |
taskrunner_parts/run_reporting.py |
notifications routed to the originating conversation, each step's history line, and failure lessons |
taskrunner_parts/run_steps.py |
running the plan within the run's bounds: groups and the parallel cap, step admission and publication, replanning, the token budget, the run-age clock (_MonotonicAge) and the watchdog |
taskrunner_parts/plan_mode.py |
plan mode: decomposition, building and editing a planned project, saved-definition starts and the plan views |
taskrunner_parts/run_lifecycle.py |
every way a run starts or stops: start admission and the concurrent-run cap, run, start_background, the resumable-workspace guard, cancel, pause and delete; execute_plan and retry_from_task stay TaskRunner methods in taskrunner.py (mypy narrowing, below), and a new start or stop entry point still goes here |
taskrunner.py keeps the TaskRunner class with its constructor, properties and
_RUNS_FILE, the Step / StepStatus / TaskRun aliases and the re-exports, and two
run drivers as methods: execute_plan and retry_from_task read, in their background
closures, the run their guard narrowed, and mypy keeps that narrowing only in a method
checked after the class's bindings (an owner function is checked before them). New
TaskRunner behaviour goes to the owner whose row names its responsibility, bound in
the class under that owner's header; new module-level code goes to that owner, never
to taskrunner.py.
The gateway attaches its singleton WorkflowService and TaskRunner only after
complete workflow recovery. Both server entrypoints launch this recovery as a
tracked background task after the listener binds and its credentials are published,
never from on_startup or an awaited pre-bind step. The existing async factory
retains complete run restoration, off-loop disk reads and eviction, and
owning-loop handle hydration. Authenticated workflow routes and TaskRunner mutation
requests receive HTTP 503 until recovery and both attachments succeed; TaskRunner
status and cancel remain available. The canonical defer_workflow_attachment()
gate also rejects non-HTTP mutations. Initialization failure unpublishes both ports
and keeps this gate closed, with code workflow_initialization_failed and a message
to restart the gateway; only pending recovery asks callers to retry.
Shutdown closes admission, cancels and drains initialization, and forbids late
publication even if the factory returns after cancellation.
The dependency is optional so CLI, tests, and headless callers outside the gateway
retain the existing TaskRunner behavior when no workflow service is present.
An async dashboard host first calls defer_workflow_attachment() on the shared
runner before yielding during startup. While deferred, plan, run,
start_background, execute_plan, and retry_from_task reject new admission;
delete_run, update_plan, and update_task also reject while deferred so a
persisted task cannot be changed without propagating to its restored workflow.
The shared check raises WorkflowInitializing (a RuntimeError subtype).
Dashboard start, plan, retry, execute, delete and update handlers catch that type
around the actual operation and return an initializing 503 with the stable code
workflow_initializing; unrelated runtime errors retain their existing handling. Chat-to-plan retains an early check before
allocating its own placeholder or directory and catches the same typed exception.
The projects API's delete alias applies the same typed 503/code mapping around
delete_run, while preserving its existing not-found and source behavior.
Status and cancel
remain available. These checks run before creating or mutating run state, including
calls from messaging channels
that retain the gateway's runner reference. Attachment releases that gate;
explicit attachment of None releases standalone fallback. Gateway failure cleanup
immediately defers again without yielding, so a failed gateway never opens
that fallback. Cancellation or slow I/O does not release the gate. Standalone and headless
callers that never defer attachment retain their existing admission behavior.
Workflow publication is best-effort once attachment has settled: an absent or
failed publication service cannot fail standalone planning or execution. Pending
initialization is not that fallback: admitting even a fresh run would permanently
omit its workflow identity because attachment does not backfill already-started
runs. A retryable refusal preserves that evidence without a new reconciliation
mechanism. Every host lifecycle checkpoint — registration, source,
rebind, phase/step events, pause, terminal state, and deletion — awaits the workflow
service's off-loop durable mirror, so a maximum-size YAML plan cannot block the
gateway event loop while its shared run record is written. Registration also awaits
old terminal-run eviction off-loop, using the same per-run write/delete fence as
checkpoints. Repeated cancellation drains this work before removing an unreturned
registration. This changes only the workflow mirror; TaskRunner still acknowledges
its ordinary task snapshot before finalizing the linked workflow.
The chat-to-plan dashboard route registers its placeholder project with this
port before applying steps, and the dashboard delete route delegates to
TaskRunner.delete_run, so those established entrypoints cannot leave an
unlinked or orphaned common run.
Each Project persists workflow_run_id plus optional saved-definition
provenance (workflow_id, workflow_slug, workflow_revision). Planning
registers one host-driven workflow run, publishes the exact canonical plan YAML,
and pauses that same run while the project awaits execution. execute_plan,
retry, and restart recovery rebind the existing run rather than allocating a
second identity. A terminal project marks the linked workflow run terminal;
deleting the project removes the linked workflow record. The common terminal
transition occurs only after TaskRunner has written its durable state and
completed git/worktree finalization, so the shared view cannot report completion
ahead of the product-layer owner.
Task execution emits common workflow lifecycle and agent-step events around the existing _execute_tasks path. Step result summaries use the workflow event contract's bounded summary field; complete TaskRunner results remain in TaskRunner storage. Cancellation binds the workflow handle to the actual TaskRunner asyncio task. The workflow handle disables chat completion injection because TaskRunner retains its existing reporting and notification path, preventing duplicate completion messages.
Direct run() setup performs workflow registration, task binding, and the initial
TaskRunner registry write inside the same lifecycle try block as execution. A
cancellation at any of those awaits therefore reaches the established cleanup and
terminal-projection path; neither the TaskRunner project nor its shared workflow run
can remain running after its driver task exits.
Planning treats workflow publication plus the first TaskRunner registry write as one ownership handoff. If cancellation or persistence failure occurs before that handoff commits, TaskRunner removes the in-memory placeholder, its owned plan directory, and the linked workflow run. A workflow identity therefore cannot survive as active when the corresponding project was never returned or durably registered.
Retry and recovery preserve the linked workflow identity when it remains available.
If eviction or an incompatible restored record requires a replacement, TaskRunner
durably writes the replacement workflow_run_id before rebinding or publishing more
progress. A gateway crash therefore cannot leave the project pointing at the rejected
identity while the replacement survives as an orphaned workflow run.
Background admission uses the same ownership rule: its placeholder is durably written before the execution task is registered, and rollback deletes the linked workflow run before removing the placeholder. Cancellation at that persistence await cannot leave an active workflow with no TaskRunner task capable of driving it.
The placeholder's spec_content is a bounded prefix of the spec (_read_spec_prefix,
4000 characters, on a worker thread). The spec path passes hooks.validate_file_path
first, but that judges the NAME, so the prefix is read through
hooks.safe_read_file_bytes_nolink rather than by re-opening the validated name: the
gate opens first (refusing a link at the final component), fstats that one descriptor
and refuses st_nlink > 1, a non-regular inode, and a sensitive or out-of-root real
path, then reads the same descriptor — so a hardlink alias of a protected file planted
under an innocent spec name yields no bytes. within_root is the spec's own directory,
which is what carries the guarantee onto Windows, where O_NOFOLLOW does not exist.
The byte cap is 4 * max_chars with truncation allowed (a UTF-8 code point is at most
four bytes), the decode is strict — an incomplete sequence at EOF raises like any other malformed UTF-8 and the caller maps it to the empty prefix — except that a full-length result, the one case the byte cap itself may have cut mid code point, holds the dangling tail back (every complete character before such a cut already lies past max_chars), and newlines
are normalized as the text-mode read they replace did. Every refusal, like every read
error, becomes an empty prefix, so admission is no oracle for whether a path is
protected.
The prefix is not the only spec read the gate covers. Every read of a task spec goes
through the one gated reader, _read_spec_text(path, max_chars): the bounded
4000-character placeholder prefix above, and the whole-spec reads in TaskRunner.run()
and TaskRunner.plan(source="file"). The whole-spec reads pass no byte cap of their
own and are bounded by the gate's own hooks.MAX_FILE_BYTES; a spec past that cap
raises hooks.FileTooLargeError rather than coming back as a silent prefix. Where the
placeholder prefix maps a refusal to an empty prefix, a refusal of a whole-spec read
raises PermissionError, so the run fails rather than proceeding on a spec the gate
withheld — the alternative is the aliased target's bytes in the LLM prompt, the
persisted run and the review context. The two reads share one function so the gate
cannot hold at one and lapse at the other, and in both within_root is derived from the
canonical path (hooks.validate_file_path), not from the spelling the caller hands over,
so a ~ or a relative path names the same directory the descriptor lands in.
Saved definitions whose immutable format is task-plan are invoked through
TaskRunner.start_workflow_definition. The saved YAML is parsed exactly; it is
not re-decomposed by an LLM. The resulting project then follows the normal
TaskRunner execution pipeline, including requires_approval and
force_approval. Free-form /workflow input is recorded as the project's
original input for run context and provenance; it does not mutate the saved plan.
If execution admission rejects the invocation, TaskRunner deletes the newly planned
project and its linked workflow run, then returns the admission error to the workflow
caller so chat can complete normally without exposing an orphaned run.
task_models ← task_planner
task_models ← task_executor (+ task_planner for parallel grouping)
task_models ← task_reporter (+ task_planner for parallel grouping)
task_models ← taskrunner_parts (+ all above modules)
taskrunner_parts ← taskrunner (an owner never imports taskrunner at run time)
The domain model was renamed Step → Task, StepStatus → TaskStatus, and
TaskRun → Project. taskrunner.py re-exports the real symbols from
task_models and also defines back-compat aliases so existing imports keep working:
from kiro_crew.task_models import Task, TaskStatus, WorkingMemory, Project # noqa: F401
from kiro_crew.task_reporter import NotifyCallback # noqa: F401
# ── Backward-compat re-exports ──
Step = Task
StepStatus = TaskStatus
TaskRun = Projecttaskrunner.py therefore re-exports Task, TaskStatus, WorkingMemory,
Project and NotifyCallback (which task_reporter owns) and keeps the
Step / StepStatus / TaskRun aliases, so an import from kiro_crew.taskrunner
needs no change.
class TaskRunner:
def __init__(
self,
sessions: SessionManager,
context_builder: ContextBuilder | None = None,
on_notify: NotifyCallback | None = None,
auto_test: bool = True,
auto_commit: bool = False,
work_dir: Path | None = None,
conversation_log: ConversationLog | None = None,
consolidator: HistoryConsolidator | None = None,
lesson_store: LessonStore | None = None,
fresh: bool = False,
global_timeout: float = 0.0,
token_budget: int = DEFAULT_TOKEN_BUDGET,
on_approval: Callable[[Task], Awaitable[bool]] | None = None,
max_parallel_steps: int | None = None, # None/0 -> memory-sized host ceiling
workspace_dir: str = "",
workflow_service: WorkflowRunPublisher | None = None,
test_cmd: list[str] | None = None, # taskrunner.test_command; see Test verification
require_verification: bool = False, # taskrunner.require_verification
) -> None: ...
# Delegates to module-level functions: task_planner.decompose(),
# task_executor.execute_task()/self_review(), task_reporter.build_status()
def attach_workflow_service(self, service: WorkflowRunPublisher | None) -> None
async def plan(self, input_text: str = "", source: str = "text", spec_path: str = "", agent: str = "", workspace_dir: str = "", workflow_name: str = "", workflow_id: str = "", workflow_slug: str = "", workflow_revision: int = 0, workflow_source: str = "", session_key: str = "", execution_context: ExecutionContext | None = None, start_priority: StartPriority = BACKGROUND) -> Project
async def run(self, spec_path: str | Path, task_id: str = "", name: str = "", source: str = "", workspace_dir: str = "", auto_approve: bool = False, input_content: str | None = None) -> Project
async def start_background(self, spec_path: str | Path, agent: str = "", name: str = "", source: str = "", workspace_dir: str = "", auto_approve: bool = False, *, session_key: str = "", execution_context: ExecutionContext | None = None, input_content: str | None = None) -> str
def cancel(self, task_id: str | None = None, *, exact: bool = False) -> None # None = cancel all
def pause(self, task_id: str) -> None
def status(self) -> dict
@property
def running(self) -> bool
@property
def current_run(self) -> TaskRun | None
# Mutation APIs await fsync-backed persistence off the event loop.
async def update_plan(task_id: str, tasks: list[dict]) -> TaskRun
async def update_task(task_id: str, index: int, updates: dict) -> dict
async def execute_plan(task_id: str, agent: str = "", fresh: bool = False, workspace_dir: str = "", auto_approve: bool = False) -> str
async def retry_from_task(task_id: str, from_task: int, agent: str = "") -> str
async def delete_run(task_id: str) -> bool
# Internal but accessed by handlers for read-only projection
_runs: dict[str, TaskRun]
async def _apersist_runs() -> None
_persist_runs() -> None # synchronous compatibility/testing helper onlyTaskRun.source tracks where a task was started from. The dashboard Tasks page
filters runs by source (dashboard/handlers/taskrunner.py, visible_sources) to avoid showing cron-triggered background tasks:
visible_sources = {"text", "spec", "file", "chat", "dashboard", "mcp", "yaml"}| Entry Point | Source Value | Visible on Tasks Page |
|---|---|---|
| Dashboard UI | "dashboard" |
✅ |
Chat task run <spec> / /task run <spec> |
"chat" |
✅ |
MCP task_run tool |
"mcp" |
✅ |
CLI kirocrew run |
"" (runner.run without source, in a process-local runner) |
❌ (not the gateway's runner) |
plan() API |
"text", "spec", "file" |
✅ |
| Cron job | must pass source="cron" |
❌ (filtered out) |
The status response derives its top-level running flag from the runs that remain
visible after the source filter. Hidden cron work cannot make an otherwise idle
result set report that one of its returned projects is running. Caller admission
and visibility filtering remain unchanged.
run() picks the decomposer from the spec's suffix, not from the caller. A spec whose
path ends in .yaml/.yml is decomposed deterministically by decompose_yaml for
every source, so a cron- or MCP-started workflow spec produces the same task DAG as
the same file started from the dashboard. Inline YAML submitted with source="yaml" is
likewise decomposed deterministically. Any other spec is decomposed by the LLM.
Deny-by-default governs the invalid case, and it is keyed on whether anyone is
watching. When an unattended run's .yaml/.yml spec is not workflow-shaped —
source in _UNATTENDED_SOURCES = {"cron", "mcp"} — the run fails and is never
retried through the LLM decomposer. Every other source — including chat,
dashboard, file/CLI, and an unsourced run — falls back to the LLM decomposer because
it is not in the unattended set. Testing the two explicit unattended values rather than
source truthiness preserves the existing attended paths. The SEL decompose_yaml
error event is recorded either way.
"Not workflow-shaped" includes a document YAML cannot parse at all. decompose_yaml
raises ValueError for every rejected spec, its own shape checks and a yaml.YAMLError
out of safe_load alike, and this gate catches (ValueError, KeyError) — so a syntax error,
the most ordinary way a hand-written spec is wrong, takes the same branch as a semantic
one instead of failing an attended run that would otherwise have been given the LLM
fallback.
Every decompose_yaml audit event carries the run's provenance — source and
spec_name in its metadata, and the source as its caller identity (dashboard for
unsourced runs), so a denial is attributed to the surface that started the run.
Named TaskStatus/Task/Project live in task_models.py; StepStatus/Step/TaskRun
remain as back-compat aliases exported from taskrunner.py. The excerpt below shows the
execution-critical fields, not the complete dataclass constructors; the source also
carries execution identity, source/UI metadata, estimates, timestamps, and git state.
class TaskStatus(Enum):
PENDING, IN_PROGRESS, REVIEWING, PASSED, FAILED, SKIPPED, CANCELLED
@dataclass
class Task:
index: int
title: str
description: str
status: TaskStatus = PENDING
attempts: int = 0
error: str = ""
result: str = "" # updated during streaming (partial results visible)
verification: str = "" # VerificationStatus value; "" until a verdict exists (see Test verification)
resume_hint: str = "" # "resume, do not restart" after a death that may have run the prompt
requires_approval: bool = False
force_approval: bool = False # blocks even in YOLO mode
depends_on: list[int] = field(default_factory=list)
@dataclass
class Project:
spec_path: str
spec_content: str
tasks: list[Task]
started_at: float
finished_at: float
status: str # pending/planning/planned/running/pausing/cancelling/paused/completed/failed/cancelled
current_task: int
error: str
tokens_used: int
replan_count: int
memory: WorkingMemory
task_id: str
work_dir: str
last_task_time: float # tracks activity for watchdog
branch_name: str # git branch for task (e.g. kirocrew/task/{task_id})
base_branch: str # original branch before task started
commit_hashes: list[str] # per-step commit SHAs
worktree_path: str # git worktree path (empty for a non-git folder)
repo_root: str # original repo root (for worktree cleanup)
auto_approve: bool = False # per-run trust: auto-approve tool permission requests
# (deny-lists + force_approval gates still apply)
workflow_run_id: str = "" # shared workflow-run identity
workflow_id: str = "" # exact saved-definition provenance
workflow_slug: str = ""
workflow_revision: int = 0
derived_from_workflow_id: str = "" # saved ancestor after an edit or replan
derived_from_revision: int = 0_runs: dict[str, TaskRun]— keyed by task_id_tasks: dict[str, asyncio.Task]— background asyncio tasksstart_background()accepts optionalagentparam, returns a collision-resistant task ID ({spec_stem}_{time_ns})_start_lockserializes concurrency admission, completed-run pruning, ID allocation, durable planning-placeholder persistence, and_tasksregistration- Step, review and decompose sessions are opened through
sessions.open_task_session(..., agent=...)so the task runs with the specified agent;get_or_create()is only the lesson-extraction fallback when there is no runtime key - Each step gets its own session:
taskrunner:{task_id}:task{N}(fresh per step, reset after) - Each task gets its own work dir:
{work_dir}/{spec_stem}/ cancel(task_id)cancels specific task;cancel()cancels all- Completed cron runs are pruned on new start; other completed runs retain bounded history.
_taskscleaned infinallyblock (no leaks)start_background()andexecute_plan()enforce_MAX_CONCURRENT_TASKSbefore changing run state, so rejected admission cannot leave a partially started run. With a task admission attached (below) the guard is skipped: the lane meters the steps, so an over-limit run queues instead of being refused.- Replanned steps also reset sessions after execution (no leaks in
_try_replan)
attach_task_admission(admission) routes the runner through the shared task
store (docs/system-specs/modules/taskq.md, § Runner adapters). None (the
default, and every test that does not attach one) keeps the legacy behaviour.
Incognito and temporary executions use the admission's existing in-memory path.
They share the same live lane and memory-pressure checks with persistent steps,
but create no run container or step row, event, dependency-coordinator entry, or
terminal retry write. Their handles release the lane normally on completion or
cancellation. Restricted executions are not resumed from durable queue rows;
restart recovery has no restricted task body to replay.
For persistent executions with an admission attached:
| Unit | Row | Lifecycle |
|---|---|---|
| a run | taskrunner:<task_id> (kind=taskrunner_step, params={task_id, name, spec_path, steps, safe_retry}) |
_taskq_begin_run (an async def: accept_async + claim_only_async + running_async, every store touch on the writer thread) accepts + claims it with no lane slot -- the run executes nothing itself -- right before _execute_tasks, so safe_retry = bool(run.branch_name) is known. A refused starting → running mark on this row is answered by the row's STATE, not by the refusal (below); _taskq_end_run (also async def) settles it from the final status: completed → done, failed → failed, paused / cancelled → cancelled (an operator decision ends this execution; a resume is a NEW row, so the adopter never restarts what the operator stopped) |
| a step | taskrunner:<task_id>:task<N> (child of the run row, params={task_id, index, title, safe_retry}, scope_ref={auto_approve, agent}) |
_taskq_admit_step accepts it through accept_async (write-before-ack; a refused write fails the step closed, it never runs off the record) and await admission.admit(...) -- memory-pressure DEFER, a lane slot by effective cap, claim under a lease -- BEFORE the step opens a session; _execute_single_task marks it running and settles it done / failed / cancelled — once the step is RUNNING. That mark is a FENCE, not a notification: admit committed starting one statement earlier, so a refusal is a newer owner or a store outage, and the step body does not run under it (the row is failed and the step returns False) because a row left starting reaches no WAITING state and could not persist the first dependency or input wait the turn needed. A cancel that lands earlier, while admit is still waiting for a lane slot (a run cancel, a pause, the global timeout), is settled cancelled by the ADMISSION instead: _execute_single_task catches only RunnerTaskCancelled there, and a queued row left behind would never be claimed again because taskrunner_step has no dispatcher (taskq.md § Runner adapters). The normal arms await (done_async / fail_async / cancel_async); the except CancelledError / except BaseException arms use the SYNCHRONOUS write, because an await there can be interrupted before the write is submitted and a dropped terminal write leaves the row active for the next boot's reconciler |
A step re-run after a failure (retry, replan, resume) gets ~N suffixed ids so the
earlier outcome stays on the record. The lane is system for a run whose
source is cron or hook, else the session the run was started from
(_run_session_keys).
The run row's slotlessness is what keeps this parent/child pair off the lane's self-block fence: a step is a DESCENDANT of the run row, and the lane refuses a descendant whose own ancestry holds every slot (taskq.md § A descendant is never parked behind its own ancestor). A container row that took a slot would be exactly that ancestor, so its steps would be refused instead of queued at cap 1.
The container row's running mark is read for the row's STATE, never for the
refusal itself (_taskq_begin_run → _taskq_row_state), and that is the one
place this row is deliberately unlike a step row. A step fails closed on the same
False (the cell above, and workflows/agent_pool.py on the workflow path)
because it holds a lane slot and must reach a WAITING state, which starting
cannot; this row holds no slot and enters no wait, and TRANSITIONS[STARTING]
carries every active terminal, so _taskq_end_run's write still commits from
starting -- measured: a database is locked on that one write leaves the row
starting for the whole run, the run completes, the terminal write commits
done, and the next boot's reconcile examines nothing. What the lost mark costs
is the {phase, steps} progress marker. Failing the run over it would also be
incoherent with the same function's other arms, which run the plan with NO
container row at all when the accept or the claim does not commit. A TERMINAL
state is the case that does end the start (RunnerTaskCancelled, no handle kept,
so the run's exit path writes nothing over the outcome): the unfenced
unknown_side_effect a second incarnation's reconcile_on_boot writes for an
active row it does not lease can land in exactly that window, and a plan must
never execute under a row that already has an outcome. An unreadable state reads
the same as an absent one -- a read that could not be taken is never why an
accepted run is refused. Pinned by
test_a_run_whose_container_row_already_ended_does_not_start and
test_a_lost_run_row_mark_keeps_the_run_and_still_settles_the_row.
Inside a step (task_executor.execute_task(..., taskq=handle)):
- Stop reason.
EVENT_COMPLETEgoes throughclassify_stop_reason; a non-success raises_TurnNotCompletedinside the attempt.stalled/recoveringconsult the recovery ladder's L3 rung (admission.decide_recovery, ordefault_ladder()when no ladder is attached): aretrydecision writes the rowrunning → recoveringwithnext_run_at = now + delay, releases the lane slot, waits_recovery_delay(delay)(a module seam), thenreclaims the row under a NEW generation -- the interrupted turn's late writes are fenced asstale_result-- and re-runs the turn with a prompt that names the stall.execute_taskis a coroutine on the gateway loop, so every handle write it makes goes through an*_asyncseam (recovering_async,running_async) and every answer read throughrecorded_answer_async/consume_answer_async: the DB half runs on the store's writer thread, the slot release and the in-memory bookkeeping stay on the loop. An exhausted rung ends the step FAILED withtask.result(the partial) kept andtask.errorsaying so.cancelledand a non-retryablefailedend the step FAILED at once, partial kept; a retryableerror:goes through the ordinary bounded retry ladder. Without ataskqhandle the stall goes through that same ladder immediately (no delay), which is whattest_subagent_stop_reason_consistency.pypins. - A stall and a logic failure spend DIFFERENT budgets, because they are
different in kind. A logic retry is a fresh attempt at work that WAS
attempted and came back wrong; an in-place stall recovery is the same attempt
continuing -- the backend went away, the work has not finished being attempted
once.
MAX_RETRIEStherefore bounds the logic failures only: each approved stall recovery raises the loop's ceiling by its own turn (attempt < MAX_RETRIES + stop_recoveries) rather than spending one, so two ordinary failures can no longer consume the budget a later stall needs, and three stalls can no longer take the retries an ordinary failure after them needs.attemptstays the TURN ordinal, which is what keeps the re-run's prompt naming the stall and continuing from the partial instead of re-running bare. - The ceiling on in-place re-runs is the step's own, not the ladder's. The
ladder's L3 count is per-unit and DECAYS after
cooldown_secs, so a stall once per cooldown is approved for ever;STOP_RECOVERY_MAX_RETRIES-- the same in-place budget the chat slot's_tool_stall_retriesand the sub-agent's_stop_recovery_usedspend -- bounds one step's re-runs whatever the ladder forgets. Both bounds apply: the ladder refuses first in one incident, the ceiling holds when its count has decayed. - Every durable write on the recovery path is a PRECONDITION, and its refusal
is an in-band step outcome. A refused
recoveringwrite leaves the rowrunning, which the re-claim would find under another owner and RAISE on -- unwinding an accepted run over one step -- so the step ends FAILED with its partial instead. Same for areclaimthat raises (RunnerTaskCancelledfrom an operator cancel during the backoff,RunnerAdmissionRefusedfrom the store) and for a refusedrunningmark after the re-claim:startingreaches no WAITING state, so a turn re-run under one could not persist the first dependency or input wait it needed, and the mark may have been fenced by a newer owner. Pinned bytest_task_executor_stall_recovery.py, one case per refusal. - A refused ADMISSION is the same kind of outcome: a failed step, in band.
_execute_single_taskcatchesRunnerAdmissionRefusedbesideRunnerTaskCancelledand returns False, so the step is FAILED with the refusal named and the run reaches_try_replan— the runner's whole answer to a step that did not land. Every refusal that gets there is covered by the one arm: the accept the store would not take, aclaimthat hit an outage, the fencedstartingwrite, andRunnerLaneSelfBlocked(aRunnerAdmissionRefusedsubclass) when the lane is held end to end by the asking row's own ancestors. It has to be caught HERE and not in one branch of_execute_tasks: the parallel branch'sgather(return_exceptions=True)absorbs a raise while the sequential branch has no such net, so a refusal there would unwind the accepted run past the replan — and even in the parallel branch the absorbed exception left the steppendingwith no error to show.task.errorhas ONE writer for this outcome (_taskq_admit_stepre-raises without writing it), so an operator never reads the message the other one overwrote. Pinned for both branches bytest_taskrunner_taskq.py::test_a_refused_admission_fails_the_step_and_reaches_replan. - Dependency signal. An exception
taskq.dependency.classify_exceptionrecognises (or one carryingdependency_signal) parks the row inwaiting_dependency(admission.yield_dependency), slot released, the session resident; the coordinator's wake (oradmission.tick()atretry_at) re-admits it through capacity and re-runs the turn without spending an attempt. Terminal signals (auth, permanent parameter error) and more thanDEFAULT_MAX_ATTEMPTSwaits fail the step. - Input wait. The runner adapter's
waiting_input/answer_inputpair (taskq/adapters/runner.py) parks a row with its lane slot released and wakes it with the operator's answer; a step re-dispatched after a crash replays a persisted, not-yet-consumed answer (admission.recorded_answer_async) under## Operator inputand marks it consumed (consume_answer_async) only once the step has durably completed. No TaskRunner caller enterswaiting_input: the executor never parks a step in this wait on its own. Never auto-answered.
The boot reconciler has no adapter for taskrunner_step, so it only drops the
dead lease and stamps awaiting_adapter; legacy import rows arrive
recovering. attach_task_admission schedules one adopt_task_rows() sweep
(a concurrent explicit call joins it rather than adopting twice), which runs
taskq.adapters.runner.adopt_orphaned_rows. The sweep is armed only when the
admission ALREADY has a store: the gateway attaches once while the manager's
store may still be opening off the loop (so both consumers can refuse typed
from the moment they serve) and again at the store-ready boundary, and the
store getter is live — an attach that armed a sweep with no store would adopt
on the loop pass after the store landed, concurrently with the sweep the second
attach arms, and two sweeps over one set of rows can settle a row the other has
already handed back to its run:
| Row | Verdict |
|---|---|
run row, is_safe_retry (params.safe_retry, i.e. the run had a git worktree, or class none / idempotent_key) |
recovering, then execute_plan(task_id) -- only steps that did not PASS re-run (runs.json / progress.md is the checkpoint) |
| run row, not safe (legacy import, no worktree) | unknown_side_effect; the run stays paused, a "Run not auto-resumed" notice tells the operator to review and resume by hand |
run row, safe, that model.RECOVERY_MAX_ATTEMPTS (3) earlier gateway lives already resumed |
failed (adopted: lost_owner_limit, listed in AdoptReport.lost_owner_limit); the run stays paused, a "Run not auto-resumed" notice says the limit counts every gateway stop, a routine restart as much as a crash, and the operator resumes by hand (which opens a new ~N row) |
| step row, safe | failed ("interrupted by a gateway restart; re-run on resume") -- the resume creates a fresh ~N row for it |
| step row, not safe | unknown_side_effect |
any row still queued (accepted, never claimed: the crash landed while it waited for a lane slot) whose params.accepted_by is a DEAD incarnation |
cancelled ("never started") -- nothing ran under it, and a resume re-accepts a new row, so leaving it claimable would only park it forever: taskrunner_step has no dispatcher |
still leased by this incarnation, or queued under THIS incarnation (its owner is parked in a live admit) |
skipped |
Pinned by test/test_taskrunner_taskq.py and test/test_taskq_runner_adapter_durability.py.
Tasks can be paused and resumed without losing progress:
pause(task_id)— setsrun.status = "pausing"and cancels the asyncio task gracefully; the_execute()finallyblock promotes"pausing"→"paused"after session cleanup- Resume is not a dedicated method — call
execute_plan(task_id, agent="", fresh=False)to restart a run whose status is"planned","paused","cancelled", or"failed". It resets incomplete (non-passed/non-skipped) tasks toPENDINGand re-runs from there (withfresh=True, resets all tasks) - Paused status visible in dashboard UI as distinct color/icon
- API:
POST /api/taskrunner/{task_id}/pause,POST /api/taskrunner/{task_id}/execute(resume/restart) — there is no/resumeroute
On gateway restart (_load_runs), a run left in an active state is settled so no run appears active without a backing asyncio task:
| Persisted status | Becomes |
|---|---|
running / pausing with tasks |
paused; an IN_PROGRESS task returns to PENDING with its attempt refunded |
running / pausing with no tasks |
failed (crashed before decomposition completed) |
planning |
failed (re-plan to continue) |
cancelling |
cancelled; open tasks (IN_PROGRESS, PENDING, REVIEWING) become CANCELLED |
A run recovered from running, pausing or cancelling also has auto_approve cleared and its auto-approve scope deactivated, so the user re-affirms trust on resume.
- Prevents zombie tasks that appear running but have no backing asyncio task
- User can resume manually from dashboard
- Persisted via
runs.json— status survives restart - With a task admission attached, a git-coordinated run is resumed automatically from its checkpoint by the adoption sweep (§ Durable task queue); one without a worktree stays paused for the user
task_executor.execute_single_task() evaluates task-level approval before agent execution.
- With an
on_approvalcallback, eitherrequires_approvalorforce_approvalprompts the owning surface; a denial pauses the project for editing. - Without that callback,
requires_approvallogs a warning and continues, whileforce_approvalfails closed and prevents replanning around the gate. cli_server.pyconstructs the standalonekirocrew run TASK.mdrunner without an approval callback. Useforce_approval, notrequires_approval, for an action that must not execute unattended.- The dashboard supplies the callback and renders Approve/Deny controls in the project detail view.
Parallel groups are throttled to prevent resource exhaustion from simultaneous kiro-cli cold starts. Each kiro-cli cold start spawns MCP server child processes, so concurrent tasks multiply startup pressure.
Every resolved task in a parallel group is dispatched at once and an
asyncio.Semaphore caps how many run simultaneously, so a slot freed by a
finished task is refilled immediately (TaskRunner._execute_tasks, taskrunner_parts/run_steps.py):
max_parallel_steps = self._max_parallel_steps # bound ONCE per execution
sem = asyncio.Semaphore(max_parallel_steps)
async def _run_bounded(t: Task) -> bool:
async with sem:
return await self._execute_single_task(run, t, history_key, session_key=...)
results = await asyncio.gather(
*(_run_bounded(t) for t in resolved),
return_exceptions=True,
)The limit is self._max_parallel_steps, computed in __init__ as
min(taskrunner.max_parallel_steps, compute_memory_sized_parallel_cap(cfg)) and
re-derived at every run entry (see Live config below):
compute_memory_sized_parallel_capis the host-safe ceiling: available memory and the learned/configured per-agent memory cost size the result, thenagent.subagent_auto_maxcaps it, never below 3 (3 when memory cannot be read), so the auto value is never 0. It keeps the memory arithmetic the subagent cap dropped (compute_max_subagentsis now the baresubagent_auto_maxceiling, because memory bounds subagents per start) because no per-start memory floor prices a TaskRunner step. CPU is deliberately not a sizing term; the adaptive controller reacts to failing work instead.- A positive
taskrunner.max_parallel_stepsmay only lower it (intentional throttling for cost / rate limits).0or unset means "use the ceiling". - An explicit knob value can therefore never raise concurrency above the
host-safe maximum. A test that asserts a specific concurrency must pin
compute_memory_sized_parallel_cap, or it measures the runner's hardware rather than the knob — a small CI runner computes 3.
The gateway constructs one TaskRunner from these two fields, but neither is
boot-only. _refresh_from_config() runs at the entry of run(), plan() and
execute_plan(): it reads live.snapshot() (the watcher's last applied config,
a plain attribute read) and re-applies the clamp above to the parallel cap and
_resolve_workspace_dir (the same sensitive-path validation the constructor
runs) to the workspace. So a write from any writer takes effect on the NEXT run
without a restart. Before the watcher has primed there is no snapshot and the
refresh is skipped -- a load() there would parse and validate the file on the
event loop these entry points run on -- so the constructor's values stand until
the first run after priming. Three rules keep this predictable:
- A running execution keeps its values.
_execute_tasksbinds the cap into a local before its first group and every group of that run uses it; the work dir is bound intorun.work_dirat entry. A reload adopted by a later run's entry never resizes or re-targets a run already in flight. - Only a MOVED field is adopted. The runner records the
taskrunner.*values config held at construction; a field whose value differs from that baseline is taken from config, a field that is unchanged keeps the constructor's argument. An embedder or test that passes an explicitmax_parallel_steps=2orworkspace_dir=...is therefore not overridden by aconfig.jsonthat never mentioned them, and writing a field back to its construction-time value restores the constructor's target exactly. - A rejected
workspace_dirkeeps the current target. The validator'sValueError(sensitive / credential path, already SEL-audited) is logged at WARNING and the previous work dir stays in force.
Per-task sessions (taskrunner:{task_id}:task{N}) are reset in a finally
block after the gather, so sessions are cleaned up even if CancelledError
interrupts it.
There is no per-index stagger delay or os.getloadavg() load guard — the
semaphore and the host-safe ceiling are the only throttling mechanisms.
Persistent, non-cron runs are snapshotted throughout their lifecycle to
{work_dir}/runs.json and loaded by __init__. Admission, plan/task edits,
progress, terminal transitions, and deletion all await the fsync-backed snapshot
path. See Task snapshot persistence for the full
record, privacy exclusions, failure handling, and ordering contract.
A plan's default work directory is provisional until the plan is accepted. A failed attempt removes that taskrunner-owned directory; an explicit caller workspace is never removed.
| Path | Entry Point | Behavior |
|---|---|---|
| CLI | kirocrew run TASK.md |
Blocking, stdout progress, --no-test flag (each step reports verification skipped) |
| Slack | task run <spec>, task run status, task run cancel (alias project run), and the /task run … slash form |
Keyword interception (messaging/commands.py:task_command_reply); a bare run <path> is not matched |
| Dashboard | REST API + Tasks UI panel | See API Endpoints below |
| Method | Path | Purpose |
|---|---|---|
| GET | /api/taskrunner |
Status with all runs, each with its task_details |
| POST | /api/taskrunner |
Start from file path or inline (__inline__: prefix) |
| POST | /api/taskrunner/cancel |
Cancel specific ({task_id} in body) or all |
| POST | /api/taskrunner/plan |
Decompose input into a planned project |
| POST | /api/taskrunner/plan/cancel |
Cancel planning |
| POST | /api/taskrunner/from-chat |
Create or update a plan from chat-provided steps |
| DELETE | /api/taskrunner/{task_id} |
Delete finished run from memory + disk |
| PATCH | /api/taskrunner/{task_id}/name |
Rename a project (name must be a string, else 400; trimmed; blank is 400 here and no change on the PUT /api/projects/{id} alias) |
| PATCH | /api/taskrunner/{task_id}/tasks/{index} |
Edit a pending task |
| PUT | /api/taskrunner/{task_id}/plan |
Replace the planned task list |
| POST | /api/taskrunner/{task_id}/retry |
Retry from step N ({from_step} in body) |
| POST | /api/taskrunner/{task_id}/pause |
Pause a running project |
| POST | /api/taskrunner/{task_id}/execute |
Execute or resume a planned project |
| POST | /api/taskrunner/{task_id}/to-chat |
Open task results in a chat slot for manual review: task-review-<token>, linked to the taskrunner:<task_id>:chat:<token> session minted for it and owned by the app the task ran for, which passes the per-slot checkpoint on it (App Kit platform §13); export and rewind still refuse an app any slot with a linked session, this one included. The first POST for a completion appends the run summary (Plan: <task_id> for a planned run, # Task Review otherwise) and starts an agent turn on it. A completion is one run attempt in one state, keyed on the task id, the run's start time, its status and the branch (plan or review). A status the run settles out of during cleanup (cancelling, pausing) counts as the one it settles on, and the finish time is left out because the runner stamps it again after announcing the completion. An in-memory map on the dashboard state records the slot each one opened. While that slot is open, a repeat POST returns it with no second summary and no new turn; a concurrent POST waits for the first one's slot. Executing, resuming or retrying a run stamps a new start time and opens a new slot, as does the run reaching its outcome after a POST while it was going, and a POST after the slot was closed or the gateway restarted |
| GET | /api/taskrunner/{task_id}/plan-context |
Return plan text for chat pre-fill |
| GET | /api/taskrunner/{task_id}/plan.yaml |
Export the plan as YAML |
| POST | /api/taskrunner/refine |
Start background user-input → task-spec refinement; progress arrives via refine WS events or GET polling |
| GET | /api/taskrunner/refine |
Refine status |
| POST | /api/taskrunner/refine/cancel |
Cancel refine |
| POST | /api/taskrunner/refine/answer |
Legacy answer hook; current refine never enters a question wait, so ordinary calls return 409 |
| POST | /api/reveal |
Reveal a path in the host file manager for a direct-local caller; remote or launcher-less cases return a clipboard-copy fallback |
The mutating task-runner routes pass _require_taskrunner_owner: a dashboard-user
request (app == "") must be the dashboard owner (require_owner_dashboard_request,
403 owner_only); an internal-secret loopback caller (the task_run MCP tool) and an
app token keep the controls that already govern them (the app is confined to its
manifest's declared paths). POST /api/taskrunner with a non-inline spec path
screens it through _validate_spec_path (hooks.validate_file_path, off the event
loop): a traversal, a missing file or a UNC/laundered link answers
400 invalid_spec_path, and a sensitive credential path answers 403 access_denied.
Starting is refused while the projects app (the Task Runner page) is
switched off in Library: POST /api/taskrunner answers 409 app_disabled
("Task Runner is disabled. Enable it in Library to start tasks.") before any
inline spec is written, the task_run MCP tool relays the same refusal, and
the chat command task run <spec> replies with that sentence. Only an
explicit enabled: false install record counts as disabled
(apps.manager.app_disabled); a host with no record runs tasks.
A partial example; each run also carries name, auto_approve,
auto_approve_remaining_secs, original_input, source, spec_content,
lessons_learned, commits and groups, and each task detail carries
depends_on, requires_approval, force_approval and its timestamps
(task_reporter.build_status).
{
"running": true,
"runs": [{
"task_id": "my-task_1771822344",
"running": true,
"status": "running",
"spec": "/path/to/spec.md",
"spec_name": "my-task",
"started_at": 1771822344.0,
"finished_at": 0,
"tasks": 3,
"current_task": 2,
"completed": 1,
"failed": 0,
"skipped": 0,
"error": "",
"tokens_used": 5000,
"replan_count": 0,
"task_details": [{
"index": 1, "title": "Create handler", "description": "...",
"status": "passed", "verification": "not_configured", "error": "", "result": "...(bounded)...", "attempts": 1
}],
"work_dir": "/path/to/work/dir",
"branch_name": "kirocrew/task/my-task_1771822344"
}]
}task_models.py owns retry, recovery, replan, total-task, timeout, token-budget, progress-file, and session-prefix constants; TaskRunner owns the persistence filename (_RUNS_FILE); the concurrent-run fallback (_MAX_CONCURRENT_TASKS) lives in taskrunner_parts/run_lifecycle.py and the parallel-step fallback (_MAX_PARALLEL_TASKS) in taskrunner_parts/run_inputs.py, both still importable from kiro_crew.taskrunner. task_executor.execute_task(), TaskRunner._try_replan(), TaskRunner._watchdog_loop(), and task_executor.run_tests() consume those bounds. test_scenarios_v2_logic.py pins retry exhaustion and test_taskrunner_planning.py::test_replan_blocked_by_step_limit pins the total-task boundary, so documentation names the enforcement seams instead of copying tunables.
The global timeout and the watchdog's stall bounds compare a run's AGE, not two wall-clock readings. started_at and last_task_time stay wall-clock for display and runs.json; _MonotonicAge keeps a monotonic reading of each. The one for started_at is taken where the run is stamped (a new run, a restart, a retry) and shared by _execute_tasks and _watchdog_loop, so the executor and the watchdog cannot disagree, and a wall-clock step neither times out or stall-cancels a healthy run nor spares one past its bound. The age is awake time, like the steps' own asyncio deadlines: time.monotonic() stops while the host is suspended on Linux and macOS, so a suspend there is neither timeout nor stall time, and it keeps counting on Windows. test_taskrunner_run_steps.py::TestGlobalTimeoutClock and TestWatchdog::test_watchdog_measures_stalls_on_the_monotonic_clock pin the step cases; a suspend on Linux or macOS reads as their forward step, the wall clock moving while the monotonic clock does not.
Execution and verification are separate outcomes. TaskStatus.PASSED says a
step's work completed; Task.verification says whether the operator's test
command proved it, as a VerificationStatus value (task_models.py):
| Verdict | When |
|---|---|
passed |
the configured command ran and exited 0. The only verified verdict |
failed |
it exited non-zero (a failing test, pytest collecting no tests, a timeout) |
skipped |
tests are turned off (--no-test), or the command could not start (a missing program or working directory) |
not_configured |
taskrunner.test_command is empty |
"" |
no verdict yet. Task.__setattr__ (task_models.py) clears the verdict whenever a step's status is set to pending or in_progress. That one rule covers every reset (retry_from_task, a resume or restart through execute_plan, a pause, crash recovery, an approval denial) and each attempt's start, so a pending or running step never reports an earlier verdict, in memory, in runs.json or in the status payload. A step that finished keeps its verdict while it waits for review |
The command comes only from taskrunner.test_command in config.json: an argv
list, run without a shell through the same sandboxed, process-group-reaped
spawn as before (task_executor.run_tests). Nothing infers a command from the
project, and an in-sandbox agent cannot write one, because config.json is a
read-only sandbox leaf (security.md). Both keys are restart=True: the gateway
and kirocrew run pass them to TaskRunner(test_cmd=..., require_verification=...)
at construction. Where the agent sandbox is off, config.json is writable by the
agent, as it is for every other setting; the key is read only at startup. A
malformed list loads as empty, never trimmed, since dropping one element would run
a different command; a list with an empty-string element is malformed too, and
loads as empty without a warning.
The exit status alone is the verdict. run_tests keeps only the tail of a
failing log, so output text never decides it. A runner that exits 0 without
running a test (pytest with every test skipped, for example) therefore reads
passed; choose a command that fails on zero tests. A failed verdict takes the
existing bounded retry path (MAX_RETRIES, loop detection).
run_tests returns None instead of a success boolean when the command could
not start, the same on every platform. Before the spawn it checks that the work
directory is a directory and, on POSIX, that argv[0] resolves the way the child
resolves it: a bare name on the child's PATH with the spawn's own resolver
(sandbox._resolve_spawn_target), a path from the work directory. On POSIX the
program is exec'd in the child, by the spawn shim or by a sandbox wrapper, so a
missing one would otherwise be only an exit status (the shim's 127) and read
failed. On Windows the parent's CreateProcess resolves the program and raises
FileNotFoundError, which maps to None too. An exec that fails in the child
after the check (a script whose #! interpreter is missing, for example) still
exits non-zero and reads failed.
With taskrunner.require_verification off (the default), a skipped or
not_configured step still passes as before and reports its verdict. With it
on, only passed lets a step pass. A step whose verifier did not run fails on
that attempt, without a retry. With tests off or no command, no agent turn can
change that. A command that could not start is a fault in the configured command
or its environment, not in the step's work. A path-style program resolves from
the work directory, so a turn could create it, but the verdict would then come
from a program the agent wrote. _execute_tasks fails the run before its first step when no command
will run (an empty command, or --no-test). Required mode judges steps as they
run: a step that passed before the key was turned on (stored verdict "") is kept
by a resume and is not re-verified.
runs.json keeps each step's verdict across a restart, /api/taskrunner
returns it in task_details, and format_completion_summary counts only
passed as verified. Pinned by test/test_taskrunner_verification.py.
Notifications with a run are prefixed with [run.name], falling back to the spec path stem, via _notify(title, body, run=run). The table lists the main events; it is not exhaustive (approval, denial and context-compression notices also exist).
| Event | Title | Body |
|---|---|---|
| Task started | 🚀 Task started | Spec name |
| Plan ready | 📋 Plan ready | Step list |
| Task passed | ✅ Task N/M | Title + bounded result preview |
| Task failed | ❌ Task N/M failed | Title + error |
| Task completed | ✅ Task completed | Steps passed/failed, elapsed, tokens, work dir, full step list |
| Task error | ❌ Task error | Exception message |
| Stall warning | Minutes since last activity | |
| Session reset | 🔧 Watchdog: cancelling stalled task | Minutes + resetting |
| Process died | 💀 Task N: process died | Recovery count |
| Lesson learned | 📝 Lesson learned | Rule text |
| Replan started | 🔄 Re-planning (attempt N/2) | Failed step title + error |
| Revised plan | 📋 Revised plan | New step count + titles |
| Possible loop | Same error repeated Nx | |
| Token budget | 💰 Token budget exceeded | Usage vs budget |
| Branch ready | 🌿 Branch: name |
Shown in completion summary |
start_background(..., session_key=) records the conversation the run was
started FROM in TaskRunner._run_session_keys (task_id → key, in memory only —
a persisted channel key would outlive the binding it names and send a restart's
first notice into a conversation that may no longer resolve). _notify resolves
it from run.task_id and hands it to task_reporter.notify, which forwards it
to the sink. It is dropped when the run is pruned or deleted; a notification with
no run attached carries no key.
The sink is what decides where a notice goes, and the one notice a run cannot
proceed without is an approval request. The gateway's _task_notify therefore
tries the governed cross-surface channel ladder first (_deliver_channel_reply,
see slack-gateway) and keeps the owner Slack DM as the
fallback, so a Telegram-only operator is told about an approval their task is
waiting on.
task_reporter.NotifyCallback is Union[SessionAwareNotify, LegacyNotify], not
one widened signature:
SessionAwareNotify—(title, body, task_id="", *, session_key="");LegacyNotify—Callable[[str, str, str], Awaitable[None]], the shape of the CLI's printer and of three-argument test doubles.
No single signature is satisfied by both, so the union is what keeps mypy
checking the arity of each. notify() widens the CALL only when there is a
conversation to carry AND _accepts_session_key(callback) confirms the sink
takes the keyword; otherwise it makes the exact three-argument call a
legacy sink is written against. The probe is not paranoia:
notify() swallows sink failures at debug level, so an unconditional keyword
handed to a legacy sink would silently stop that sink's notifications with
nothing logged above debug, and a TypeError retry cannot tell an arity
mismatch from one raised inside the sink's own body.
A task rooted in an existing git repository runs on an isolated branch via
git_coord.py; a non-git folder runs in place without initializing a repository:
- Existing repo:
git worktree addcreates an isolated working directory; the user's checkout stays untouched. - Non-git folder:
git_enabledbecomes false; no repository, branch, commit, revert, or git-diff review is created. - Per-step commits (git runs):
git add -A && git commitafter each passed step. - Revert on failure (git runs):
git reset --hard HEAD~1when review fails, before retry. - State summary:
git log --oneline+git diff --statis injected when git coordination is active. - Review diff: the step's diff (
git_coord.get_step_diff, capped at 8000 characters) feeds the independent review session; only a non-git run or a step whose commit failed uses the generic review prompt. - Finalize: an owned worktree is cleaned up when the run exits.
- Resume recovery: a retry — and a restart of a run that already had a worktree — validates the saved worktree's path, repository, and branch before dispatching more steps. A lost worktree is recreated from its original repository and existing task branch only when a surviving Git pointer positively identifies the stale checkout; an arbitrary replacement directory is left untouched and the retry fails closed. The identified checkout is renamed aside before it is deleted, so the delete can only ever destroy the tree that was validated -- never whatever happens to occupy the path afterwards. Ownership is proved a second time against the renamed tree, because the first proof and the rename are separate moments: anything that arrived in between is put back and the retry fails closed. A set-aside tree that cannot be deleted is logged and left on disk beside the recreated worktree rather than failing the retry.
Git discovery or first-run worktree initialization failure is non-fatal — the
task continues without git coordination. A resumed run (retry or restart of a run
that already had a worktree) whose worktree is lost and cannot be recreated
fails closed instead of continuing without git isolation. This governs
fresh=True restarts too: fresh resets task results, not git identity, so a
fresh restart of a run that already owns a task branch resumes that branch
under the same validate-or-fail-closed contract rather than minting a new
worktree — recovery keeps every prior step commit reachable, and a run whose
branch is truly gone fails closed; the escape is planning the task again from
scratch. A restart is also refused while the previous run's background task is
still finishing, because the terminal status is persisted before the old
finalizer removes the worktree.
task_executor.execute_task() tracks consecutive identical errors:
- 2nd identical error →
⚠️ warning notification - 3rd identical error → step FAILED with "Loop detected" message
- Different error resets the counter
AcpProcessDied(process crash) does NOT count — crashes don't pollute the error tracker
Applies to both exception errors and test failure outputs.
Identity is _error_fingerprint, not raw string equality. A fingerprint is the
FAILED / ERROR summary lines when the text is a test run (the run_tests
tail truncation makes the head of the output the unstable part), otherwise the
first _ERROR_FINGERPRINT_LINES lines. Only genuinely volatile forms are
masked -- hex addresses, durations, port N / pid N, and timestamps -- and
whitespace is collapsed.
Bare digit runs are identity and are never masked. error variant 1 and
error variant 2 are successive cases of the same parametrized run, and
assert 3 == 0 and assert 1 == 0 are different assertions; collapsing them
would report a steadily-advancing agent as a loop and overwrite the real error
with "Loop detected" on the third attempt. Pinned by
test/test_task_error_fingerprint.py::TestErrorFingerprint and by
test/test_scenarios_v2_logic.py::TestScenarioCycleDetection::test_different_errors_no_cycle.
Bracketed pytest parametrize ids (test_x[1s], test_x[30m]) are identity, not
noise, and are left unmasked so a volatile pattern cannot collapse the
successive cases of a pytest -x run that steps through time-unit ids. Only a
[...] ATTACHED to an identifier is a node id; one masking pass
(_FINGERPRINT_MASK_RE) alternates the node-id span against the volatile
patterns and returns a matched node-id span verbatim while masking every other
match. A bracketed log prefix that stands on its own ([2026-09-29T05:00:00Z] ..., [pid 9912], [120ms]) is NOT a node id, so its volatile contents are
still masked and an error whose only moving part is such a prefix still compares
equal across retries.
Test-run identity is the pytest short-summary (FAILED / ERROR) lines PLUS
each failure's location -- the E assertion lines and the
<path>:<line>: <ExcType> traceback lines, all of which run_tests keeps in
the tail. The location is folded in because on a non-tty pytest truncates each
short-summary line after - to the terminal width: without the location, a
test that the agent is converging on -- it fixes one assertion and the same test
then fails at the next -- would share one truncated summary line across attempts
and be mis-read as a loop. A different failing line or assertion now yields a
different fingerprint, while a genuinely stuck test (same location, same
assertion, only a volatile value moving) still collapses to one.
Fingerprints are comparison-only; task.error always keeps the raw text.
task_executor.build_task_prompt() assembles context for each step (async):
- Role prompt — autonomous execution agent identity + git branch awareness
- Git context (git runs) —
git_coord.get_state_summary()(log + diff stat) - Working memory (whenever non-empty, git or not) —
run.memory.summary() - Full Execution Plan (follow strictly) — every task by group, passed ones marked done and the current one marked, with dependencies and a short description
- Original Context (
run.original_input) or else Original Spec (run.spec_content), first 2000 characters - Current Task (N/M) — title and description
- Previous Attempt Failed (attempt > 1 with an error) — the previous error
- Resume (do not restart) (whenever
resume_hintis set, at any attempt) — the instruction to inspect current repository and session state first - Instructions — the closing rules, git or non-git variant
resume_hint is set when a step's AcpProcessDied may have left its prompt
run: the death carries ambiguous_delivery (a stdin stall the live child may
still read past), or the attempt had already produced output or a tool call. A
mid-stream compaction after output sets it too, since it can end in a session
reset that restates the step. A verbatim retry would run its tools again, and the death keeps the attempt
number, so the hint cannot ride on the attempt > 1 guard. It is cleared ONLY
when an attempt completes normally. It survives every give-up, a Resume or
retry of the failed run (which restarts the step at attempt 1 in the same work
dir), and the automatic replan, whose prompt is told the failed step may
already have run. Keeping it costs one inspection; dropping it risks repeated
side effects.
Independent review using separate session (taskrunner:{task_id}:review):
- Step set to
REVIEWINGstatus before review starts (visible in UI as 🔍) - Only set to
PASSEDafter review succeeds - Reads the step's actual diff (
git_coord.get_step_diff, first 8000 characters), not the LLM's self-report - Separate session = no bias from having written the code
- The generic review prompt is used only for a non-git run, or a git run whose commit failed
- Fails closed. A review that raises, or a git run whose step left no diff (unless its commit failed), sets
task.errorto start with_REVIEW_UNVERIFIEDand FAILS the step: no revert and no re-run, because a re-run would repeat any push or CR the step made and pass without a review - Only a real
{"ok": false}verdict → revert commit → re-run the step once → re-commit on success
task_executor.execute_task() hands every permission request of a step turn to
tool_permission.settle with the step's ladder (_step_permission_policy, rebuilt
when a mode switch changes whose spec hooks gate the turn). In order:
- The agent spec's PreToolUse hooks (with the Hooks page's), on a backend that never receives them, run first and fail closed: a deny, a hook that errors, or hooks that cannot be read block the call (
metadata.reason=spec_hook_deny). Then the hook rules: a deny remains a denial, and an explicit hook auto-approval remains eligible. A hook auto-approval for a shell command is honoured only aftername_grant.refusal_for_event(event)confirms each program name in the command still resolves to the program it appears to name; a refusal downgrades to the run's trust grant, then the interactive prompt (or the headless deny-by-default), and is audited asoutcome=auto_approve_declinedwithreason=name_grant. - The run's own trust grant (below).
- The mid-stream context check, which runs even for a request a hook or the run's trust already granted: past
_MID_STREAM_COMPACT_PCTthe request is rejected bare and the turn re-run after compaction. - When nothing grants the request,
on_tool_approvaldecides it if the runner has a callback; otherwise the headless path rejects the tool withheadless_no_authorization.
A mid-step agent switch (EVENT_AGENT_SWITCHED) rebuilds the ladder for the new agent's spec hooks and calls spec_hooks.refuse_stale_switch: when the switched-to agent auto-approves a capability a PreToolUse hook covers, those calls would never reach the permission request, so the turn is cancelled (StaleProjectionError) and the next claim resets the stale session.
The SEL row is written before any wire I/O for a refusal, and a refusal row that cannot be written is raised before the wire rather than delivered unaudited, as is an exception from the interactive handler; an approval is audited after the wire answered (approved, or rejected_transport_floor when the transport floor turned it into a rejection). A HOST refusal steers the in-band deny notice before the reject; the interactive handler's no and the compaction reject stay bare. The ladder's stages, the shared adapters and this surface's SEL rows (TaskrunnerRows) live in tool_permission.py; what reads or moves the run's own state (its trust grant _RunTrust, its log lines, the watchdog's activity stamp) stays in task_executor.py.
Project.auto_approve is a per-run trust-intent flag (default False); the live authorization is the scoped SafetyOverride grant described below. The intent is opt-in
at execute time via the dashboard (auto_approve in the execute/start request
body) and threaded through execute_plan(), run(), and start_background().
- Default off → current interactive behavior (tool permission requests
prompt via
on_tool_approval, or deny-by-default when headless). - On → the run's tool permission requests are auto-approved WITHOUT the
interactive prompt, and the SEL tool-invocation audit records the approval
with reason
run_auto_approve(vshook_auto_approvefor an explicit hook trust).
Two guardrails remain intact for a trusted run:
- Hook deny-lists / sensitive-path blocks are evaluated BEFORE the
auto-approve check, so a
TOOL_DENYstill rejects the tool. force_approvalis a separate task-level path at the top ofexecute_single_task()and fails closed when no approval handler exists.requires_approvalonly prompts when that handler exists; without it, the task continues after a warning.
The mid-stream context-overflow check still runs before final approval.
Both dashboard launch handlers (POST /api/taskrunner and
POST /api/taskrunner/{task_id}/execute) route the requested auto_approve
through the shared async _gate_auto_approve() provenance gate before honoring
it. Per-run trust is a human-at-the-dashboard decision, so a grant is honored
ONLY for a dashboard-context request (request["app"] == ""); an app/proxy
caller cannot mint trust even while claiming source: "dashboard".
The grant decision is SEL-audited fail-closed. The audit is written
critical=True (a synchronous, raise-on-failure write) but offloaded via
asyncio.to_thread so the synchronous flush does not block the gateway event
loop while the await still surfaces a write failure. The write is contained in
the gate itself (not per-endpoint), so if the grant cannot be persisted to the
SEL trail it is downgraded to denied — an un-auditable grant is never
honored — and no unsanitized exception escapes as an HTTP 500 (CWE-755). This
invariant holds for every current and future launch caller.
Hardening measures scope the trust tightly. It is not the global SafetyOverride
singleton (which would leak trust to every session), but the authoritative grant
IS held by SafetyOverride — as a task-scoped grant — so per-run trust is
audited and expires through the same primitive the backend-security-controls
rule mandates, with no independent approval state living on the run:
- SafetyOverride scoped grant (audited, TTL-bounded, slide-renewed) — enabling
auto_approvecallssafety_override().activate_scoped("taskrunner:{task_id}:autoapprove", source="dashboard"), which fail-closed audits the activation to the SEL before committing and stamps the dashboard-window TTL.is_scope_active(scope)authorizes every approval; when the grant lapses or is absent after restart, the run intent is revoked and the tool falls through to interactive approval or denial. Each auto-approved tool call slides the grant within its hard ceiling, so an idle run lapses.scope_remaining_secs()feedsbuild_status;Project.auto_approveis only persisted UI intent, whileTaskRunner._grant_run_trust(run, enabled)owns both intent and grant so they cannot diverge, and_release_run_runtime()revokes the grant at teardown.activate_scopedcan refuse — anapproval_modesdeny ofyolo, or an SEL audit write that fails — andrun.auto_approveis set from the activation result, so a requested trust grant can end with the flag off. - Deny-by-default parsing — the API reads
auto_approveasbody.get(...) is True, so only a literal JSONtrueenables trust; truthy non-booleans ("false","0",[],{}) do NOT. - Provenance gated at the boundary (label-based, shared by every launch endpoint)
— a single
_gate_auto_approve()helper is applied by BOTHapi_taskrunner_startANDapi_taskrunner_execute_plan(and any future launch surface), so the gate can't drift between routes. It honorsauto_approveonly when the request is not app/proxy-embedded (request["app"] == "", set bytoken_auth_middlewarefor the dashboard itself) — blocking an embedded app/proxy from minting trust even while claimingsource: "dashboard"— and, onstart(which carries a source claim), only when the caller EXPLICITLY declaredsource == "dashboard"(checked on the raw claimed value, so an omitted/unknown source cannot inherit trust via coercion). The decision is SEL-audited (auto_approve_grantwith endpoint + claimed-vs-resolved source +request["app"]). Residual: a raw token-holder is indistinguishable from the dashboard UI (the gateway's trust model is "token == user"), so this remains a declared-label gate; a sub-principal auth model would be a platform-level follow-up. - Reset on crash-recovery + affirmative re-grant on resume — a run recovered from
an active state (
running/pausing/cancelling) on gateway restart hasauto_approveforcedFalseand its scoped grant deactivated in_load_runs(); and because a grant is torn down at run teardown, the dashboard toggle re-syncs from the live grant (auto_approve_remaining_secs > 0), not stale persisted intent — so resuming a paused/planned run shows the toggle UNCHECKED and requires an affirmative re-grant rather than a click on a pre-checked box.
Per-run trust is reachable only through the dashboard launch endpoints' _gate_auto_approve() check. cli_server.py does not request it when it constructs the standalone runner, so kirocrew run TASK.md cannot turn on run-scoped tool approval. With no on_tool_approval callback, the step's permission ladder rejects every tool request that lacks explicit hook approval; test_taskrunner_autoapprove.py::test_headless_no_authorization_rejects pins this fail-closed posture.
This tool-authorization default does not convert requires_approval into an unattended task gate: execute_single_task() continues a requires_approval task when no on_approval callback exists. A spec that needs an attended task boundary uses force_approval; the standalone CLI then stops as failed rather than proceeding.
Activity-aware stall detection. Tracks run.last_task_time which is bumped on:
- Every text chunk during LLM streaming
- Every tool approval (auto or interactive)
- Step/approval gate entry
- AcpProcessDied recovery
Only fires when there is truly ZERO activity for the stall period.
- Sustained inactivity first emits a warning notification, then resets the session and enters
AcpProcessDiedrecovery. - Resets the current step session:
taskrunner:{task_id}:task{current_task} - Stall flag cleared on recovery (can fire again if retry also stalls)
last_task_timereset after recovery (fresh window for retry)- Watchdog cancelled in
finallyblock when task finishes - Cannot delete or cancel a task — only resets ACP session
- Each step:
taskrunner:{task_id}:task{N}— fresh session per step, reset after completion (owned bytask_executor.py) - Decomposition:
taskrunner:{task_id}:decompose(throwaway, reset in finally) (owned bytask_planner.py)- Returns
{"steps": [...]}(or{"tasks": [...]}) or a bare JSON array of steps
- Returns
- Self-review:
taskrunner:{task_id}:review(separate session, reset in finally) (owned bytask_executor.py) - Context compaction between steps routes through
SessionManager.compact_if_needed(key), preserving the gateway's deduplication, cooldown, turn-semaphore exclusion, and skills reinjection. A"busy"decline is retried later with no directprovider.compact()fallback. Its shared post-check uses the attempt's immediate effect verdict (_POST_COMPACT_RESET_PCT) and awaits a reset before the next step cold-starts; deferred readings only damp later growth, while the mid-stream overflow guard covers the interim. - The mid-stream overflow handler compacts in place and resets the session if that does not complete. On a backend
messaging.commands.compact_unsupported_backend(client)names, it skips the/compactcall and its wait and resets the session straight away, since the wait could only time out.
Every step gets is_new=True on its first message, which triggers full ContextBuilder injection: user preferences, active projects, recent history, semantic memory, lessons, episodic memory queried by the step prompt, and triggered skills. The budget matches a normal chat session.
The Compose mode's Refine into Spec action uses a single-shot LLM call to rewrite the user's rough natural language input into a structured task specification. No tools, no file reading, no clarifying questions — just a fast spec rewrite.
- User describes task in natural language
- LLM rewrites it into a structured spec (Goal / Requirements / Acceptance Criteria)
- Spec appears in the editable textarea — the user can edit it, then start it with the ordinary Run control
No tools allowed during refine — all tool calls are rejected. The refiner's only job is to produce a better-written spec from the user's input.
WS events: refine type with {status, text, error} fields. A refine whose turn the
backend never finished (stop reason timeout or stale_recover) ends with status error
and an error that names the stop reason, with the partial text still sent beside it, so it
cannot pass for a finished spec.
The page uses a left run rail plus a detail/compose area. The rail defaults to 260px, is resizable from 220–460px, and can collapse to a 48px strip.
- Run rail: always present; compact run cards show status, name, progress, live auto-approve state, and cancel/delete controls. The top action starts a new task. Runs are listed newest first by
started_at, sorted on the client (listedRunsinwebsite/src/pages/ProjectsPage.tsx); the API keeps the registry's insertion order. Equal stamps put the later-inserted run first. A run is stamped when it starts, when a plan is executed, and on Restart (retry_from_task), so a restarted run moves to the top. A plan that has not run yet hasstarted_at0 and sits at the bottom. - Compose area (no run selected): Compose, From Spec, and From YAML modes share
AgentSelectorand an optional workspace override. - Compose mode: task text, Refine into Spec, Plan, per-run auto-approve, and Run controls; planning has a cancellable
PlanningBanner. - From Spec / From YAML: editable text plus file upload, Run, and Plan. YAML mode states that a workflow-shaped document bypasses the LLM decomposer.
- Project detail (
ProjectDetailPage): Idea/Tasks tab bar.- Idea tab: read-only spec content + Edit in Chat.
- Tasks tab: DAG/Phased view toggle with
DagViewandPhasedView.
- Run actions: planned runs offer Execute/Chat/Discard; running runs offer Pause/Cancel; paused runs offer Resume; terminal runs offer the applicable Chat/Restart/Schedule actions.
- Updates:
push_refresh("taskrunner")on notifications plus 3s polling.
runs.json is the ordinary JSON list of retained task records, including each
run's captured execution_context, input, results and diagnostic state. Each
task entry carries its resume_hint (see Step Prompt Context), restored on load
so it survives a gateway restart; the /api/taskrunner status payload does not
expose it. Each task entry also carries its verification verdict (see Test
verification), which the status payload does return. There is
no private sidecar, hidden directory or separate public projection. Incognito and
Temporary runs are omitted according to their captured mode; cron runs are also
excluded from this registry. Restored records reuse their captured identity,
rather than reconstructing membership from a task name or the current selection.
The event loop snapshots its owned registry; a worker writes it with
atomic_write(..., fsync=True). A sequence number and write lock prevent a delayed
older worker from overwriting a newer successful snapshot. An older worker behind
a newer failed attempt raises instead of claiming success. Write failures raise a
sanitized TaskSnapshotError; a later successful snapshot can retry the operation.
Async persistence drains its owned worker before propagating cancellation,
including repeated cancellation. Admission, edits, deletion and completion await
acknowledged writes. The synchronous compatibility helper alone is best-effort.
Admission failures roll back their in-memory registration; failed deletion restores the entry so another delete can retry. Retry resolves its bound history before resetting results and reserves the canonical run ID against concurrent starts. A failed write or cancellation restores task fields and run state in place; the selected agent changes only after acknowledgment. Handoff follows without another await, and the reservation is released on failure or handoff. Terminal-write failures still stop watchdogs and release background bookkeeping, but do not publish a successful workflow terminal or remove its recovery worktree. A failed completion write cannot send a completed notification. Done observers retrieve finalization exceptions and log their type; awaiters still receive the original exception, and cancellation is not logged as failure.
Restore reads this same registry. Unreadable storage leaves the file untouched
and fences subsequent snapshot writes until restart after recovery. Invalid JSON
is preserved as .corrupt when renaming succeeds. An invalid snapshot shape or
malformed execution context refuses restore and fences writes; it is not hydrated
through a hidden row or downgraded to Global memory. An obsolete
{"task_id": ..., "private_payload": true} reference, which 0.7.0-insider.1 to .5
wrote for a member task whose payload lived in the hidden
memory_stores/.task-runs/ sidecar, refuses the row, never the runner: a row of
exactly that shape is never hydrated or run and is left out of the restored runs;
the other rows restore, and one warning names the left-out task ids and says their
payloads were not read, so those tasks cannot resume and must be re-created. Restore
writes nothing for them and does not fence snapshot writes on them, so the next
snapshot rewrites the registry without those rows; a restart that finds the same
rows again restores the same way and warns again. Any other private_payload row
still refuses restore. Other per-record construction failures leave later valid rows
restorable while marking recovery incomplete and fencing writes. A legacy record
with no execution-context field retains the ordinary Global compatibility path;
an explicitly malformed field does not take that fallback. Existing crash recovery
resets interrupted task states and withdraws active runs' auto-approve grants.
save_progress writes TASK_PROGRESS.md beside a spec only for persistent runs
with a spec path. Ad-hoc saved plans have no spec file and create no project-visible
progress file. Progress writes are best-effort; load_checkpoint returns no
checkpoint when it cannot read one and otherwise recognizes completed task titles.
These writes are not a transaction across task snapshots, workflow-run files,
project checkpoints, workspaces or external effects. A process exit after commit
but before reply can leave an unacknowledged task. Cancellation after a successful
write can likewise leave disk ahead of an in-memory rollback until a later persist.
Durability inherits atomic_write's fsync/platform limits and does not provide
cross-process writer coordination or repair deleted files.
Persistent tasks keep their execution context and diagnostic state in the ordinary task record; there is no separate hidden member snapshot or protected-session diagnostic scope. Incognito and Temporary tasks skip durable task persistence according to the captured mode. Ordinary credential redaction, authorization and provider-retention controls remain independent requirements.