Add workflow engine with catalog system

[Copilot]

Adds a workflow subsystem to Specify CLI for defining multi-step, resumable automation workflows in YAML. The engine dispatches commands to CLI-based integrations, evaluates control flow, and pauses at human review gates. Ships with a catalog system mirroring the existing extension/preset catalog pattern.

Workflow engine core (src/specify_cli/workflows/)

• base.py — StepBase abstract class, StepContext, StepResult, status enums
• __init__.py — STEP_REGISTRY with auto-discovery, mirrors INTEGRATION_REGISTRY
• expressions.py — Sandboxed Jinja2-subset evaluator (variable interpolation, comparisons, boolean logic, filters like default, join, map, contains)
• engine.py — WorkflowDefinition (YAML loader), validate_workflow(), WorkflowEngine (sequential executor with recursive nested step support), RunState (JSON state persistence for resume)
• catalog.py — WorkflowCatalog (multi-catalog stack: env var → project → user → built-in), WorkflowRegistry (installed workflow tracking)

10 built-in step types (workflows/steps/)

command, prompt, shell, gate, if (then/else), switch, while, do-while, fan-out, fan-in — each a StepBase subclass in its own subpackage, same extensibility model as integrations.

CLI commands

specify workflow run|resume|status|list|add|remove|search|info plus specify workflow catalog list|add|remove.

Catalog files

workflows/catalog.json, workflows/catalog.community.json, and an example workflows/speckit/workflow.yml (full SDD cycle).

Example workflow definition

schema_version: "1.0"
workflow:
  id: "sdd-pipeline"
  name: "SDD Pipeline"
  version: "1.0.0"
  integration: claude

inputs:
  feature_name:
    type: string
    required: true

steps:
  - id: specify
    command: speckit.specify
    input:
      args: "{{ inputs.feature_name }}"

  - id: review-spec
    type: gate
    message: "Review the generated spec."
    options: [approve, reject]
    on_reject: abort

  - id: check-scope
    type: if
    condition: "{{ inputs.scope == 'full' }}"
    then:
      - id: full-plan
        command: speckit.plan
        integration: gemini
        model: gemini-2.5-pro
    else:
      - id: quick-plan
        command: speckit.plan

Tests

122 new tests covering step registry, expression engine, all step types, workflow validation, engine execution (including branching/gates/shell), state persistence, catalog resolution, and registry operations. Full suite passes (1362 total).

[Copilot]

Pull request overview

Adds a new workflow subsystem to Specify CLI that can load YAML workflow definitions, evaluate simple template expressions, execute step types, persist run state for resuming, and discover workflows via a catalog/registry model (similar to extensions/presets).

Changes:

• Introduces specify_cli.workflows core modules (base types, expression evaluator, engine + run-state persistence, catalog + registry).
• Adds 9 built-in step types (command/shell/gate + control-flow primitives) with auto-registration.
• Adds specify workflow ... CLI commands plus initial workflow catalog JSONs and a sample speckit workflow, with a large new test suite.

Show a summary per file

File	Description
workflows/speckit/workflow.yml	Adds an example “Full SDD Cycle” workflow definition.
workflows/catalog.json	Adds default workflow catalog stub (empty).
workflows/catalog.community.json	Adds community workflow catalog stub (empty).
tests/test_workflows.py	Adds comprehensive tests for workflow engine components and step types.
src/specify_cli/workflows/base.py	Defines workflow step/run base types (StepBase, StepContext, StepResult, enums).
src/specify_cli/workflows/init.py	Implements global step registry + registers built-in step implementations.
src/specify_cli/workflows/expressions.py	Adds a sandboxed, Jinja-like expression/template evaluator.
src/specify_cli/workflows/engine.py	Adds workflow YAML loader/validator, executor, and run-state persistence/resume.
src/specify_cli/workflows/catalog.py	Adds catalog stack resolution, URL validation, caching, search, and installed registry.
src/specify_cli/workflows/steps/init.py	Declares step auto-discovery package.
src/specify_cli/workflows/steps/command/init.py	Adds the command step type.
src/specify_cli/workflows/steps/shell/init.py	Adds the shell step type.
src/specify_cli/workflows/steps/gate/init.py	Adds the gate step type.
src/specify_cli/workflows/steps/if_then/init.py	Adds the if step type.
src/specify_cli/workflows/steps/switch/init.py	Adds the switch step type.
src/specify_cli/workflows/steps/while_loop/init.py	Adds the while step type.
src/specify_cli/workflows/steps/do_while/init.py	Adds the do-while step type.
src/specify_cli/workflows/steps/fan_out/init.py	Adds the fan-out step type.
src/specify_cli/workflows/steps/fan_in/init.py	Adds the fan-in step type.
src/specify_cli/init.py	Adds specify workflow CLI commands (run/resume/status/list/add/remove/search/info + catalog list/add/remove).

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comments suppressed due to low confidence (1)

src/specify_cli/workflows/engine.py:535

• _execute_steps() treats result.next_steps as a one-time nested sequence. That means loop steps (while, do-while) and fan-out/fan-in currently can't implement their advertised semantics (re-evaluating conditions, enforcing max_iterations, running per-item dispatch, joining). If these step types are intended to work, the engine needs explicit handling for them here (or the step contract needs to change so the step implementation can drive iteration/execution).

            # Execute nested steps (from control flow)
            if result.next_steps:
                self._execute_steps(result.next_steps, context, state, registry)
                if state.status in (
                    RunStatus.PAUSED,
                    RunStatus.FAILED,
                    RunStatus.ABORTED,
                ):
                    return

• Files reviewed: 20/20 changed files
• Comments generated: 6

[Copilot]

Pull request overview

Introduces a new workflow subsystem to Specify CLI that can load YAML-defined workflows, execute step graphs with state persistence (run/resume/status), and manage workflow catalogs/installation—along with integration updates to support non-interactive CLI dispatch.

Changes:

• Added specify_cli.workflows engine core (definition parsing, validation, expression evaluation, run state persistence, catalog/registry).
• Added built-in workflow step types and corresponding tests.
• Added specify workflow ... CLI commands plus integration enhancements (build_exec_args / dispatch support for Codex + Copilot).

Show a summary per file

File	Description
workflows/speckit/workflow.yml	Adds an example built-in workflow definition for a full SDD cycle.
workflows/catalog.json	Adds an official workflow catalog skeleton file.
workflows/catalog.community.json	Adds a community workflow catalog skeleton file.
workflows/README.md	Adds end-user documentation for workflows, step types, catalogs, and resume.
workflows/PUBLISHING.md	Adds workflow publishing guidance and catalog submission format.
workflows/ARCHITECTURE.md	Documents workflow engine internals, step registry, and catalog resolution.
tests/test_workflows.py	Adds a large test suite covering workflow parsing/validation/execution, steps, catalog, and registry.
src/specify_cli/workflows/base.py	Introduces core workflow base types (StepBase, StepContext, StepResult, status enums).
src/specify_cli/workflows/init.py	Implements STEP_REGISTRY and registers built-in step implementations.
src/specify_cli/workflows/expressions.py	Implements the template/expression evaluator used by workflow YAML.
src/specify_cli/workflows/engine.py	Implements workflow definition loading, validation, execution, resume, and run state persistence.
src/specify_cli/workflows/catalog.py	Implements workflow catalog stacking, caching, searching, and installed workflow registry.
src/specify_cli/workflows/steps/init.py	Adds steps package marker for built-in step types.
src/specify_cli/workflows/steps/command/init.py	Adds command step type for dispatching Spec Kit commands via integration CLIs.
src/specify_cli/workflows/steps/shell/init.py	Adds shell step type for running local shell commands.
src/specify_cli/workflows/steps/gate/init.py	Adds gate step type for interactive/pause-based human review checkpoints.
src/specify_cli/workflows/steps/if_then/init.py	Adds if branching step type producing nested step lists.
src/specify_cli/workflows/steps/switch/init.py	Adds switch branching step type based on an evaluated expression.
src/specify_cli/workflows/steps/while_loop/init.py	Adds while loop step type returning nested steps conditionally.
src/specify_cli/workflows/steps/do_while/init.py	Adds do-while loop step type intended to repeat nested steps.
src/specify_cli/workflows/steps/fan_out/init.py	Adds fan-out step type intended for parallel dispatch over a collection.
src/specify_cli/workflows/steps/fan_in/init.py	Adds fan-in step type for aggregating results from prior steps.
src/specify_cli/workflows/steps/prompt/init.py	Adds prompt step type for free-form prompts to integration CLIs.
src/specify_cli/init.py	Adds the specify workflow CLI command group (run/resume/status/list/add/remove/search/info + catalog ops).
src/specify_cli/integrations/base.py	Adds integration primitives for CLI execution and command dispatch (build_exec_args, dispatch_command, invocation helpers).
src/specify_cli/integrations/codex/init.py	Adds Codex CLI exec argument builder for non-interactive dispatch.
src/specify_cli/integrations/copilot/init.py	Adds GitHub Copilot CLI dispatch support (agent-based invocation + exec args).

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

• Files reviewed: 27/27 changed files
• Comments generated: 12

[Copilot]

Pull request overview

Adds a new workflow subsystem to Specify CLI, enabling YAML-defined, resumable multi-step automation with control-flow steps, an expression engine, state persistence, and a workflow catalog/registry—integrated into the CLI and backed by a large new test suite.

Changes:

• Introduces specify_cli.workflows (engine, step registry, step implementations, expressions, run state persistence).
• Adds workflow catalog + registry support (multi-source catalog resolution, caching, install/remove/search/info).
• Adds new specify workflow ... CLI commands and expands integration dispatch support (build_exec_args / dispatch_command).

Show a summary per file

File	Description
workflows/speckit/workflow.yml	Adds an example “Full SDD Cycle” workflow definition.
workflows/catalog.json	Adds the built-in “official” workflow catalog scaffold.
workflows/catalog.community.json	Adds the built-in “community” workflow catalog scaffold.
workflows/README.md	Documents workflow concepts, step types, expressions, and catalog management.
workflows/PUBLISHING.md	Adds community publishing guidance for workflows.
workflows/ARCHITECTURE.md	Documents workflow engine internals and module structure.
tests/test_workflows.py	Adds extensive tests for workflows subsystem behavior.
src/specify_cli/workflows/init.py	Defines and populates STEP_REGISTRY for built-in step types.
src/specify_cli/workflows/base.py	Adds core workflow types: context, results, and status enums.
src/specify_cli/workflows/expressions.py	Implements a sandboxed Jinja-like expression evaluator.
src/specify_cli/workflows/engine.py	Implements workflow loading, validation, execution, resume, and state persistence.
src/specify_cli/workflows/catalog.py	Implements catalog stack resolution, caching, search, and installed workflow registry.
src/specify_cli/workflows/steps/init.py	Adds step package marker for built-in step types.
src/specify_cli/workflows/steps/command/init.py	Implements command step dispatching via integrations.
src/specify_cli/workflows/steps/prompt/init.py	Implements prompt step dispatching.
src/specify_cli/workflows/steps/shell/init.py	Implements shell step execution via local subprocess.
src/specify_cli/workflows/steps/gate/init.py	Implements interactive/non-interactive human gate behavior.
src/specify_cli/workflows/steps/if_then/init.py	Implements if branching step.
src/specify_cli/workflows/steps/switch/init.py	Implements switch branching step.
src/specify_cli/workflows/steps/while_loop/init.py	Implements while looping step.
src/specify_cli/workflows/steps/do_while/init.py	Implements do-while looping step.
src/specify_cli/workflows/steps/fan_out/init.py	Implements fan-out step metadata (engine expands execution).
src/specify_cli/workflows/steps/fan_in/init.py	Implements fan-in aggregation step.
src/specify_cli/integrations/base.py	Adds CLI dispatch primitives (build_exec_args, dispatch_command, skills invocation changes).
src/specify_cli/integrations/copilot/init.py	Adds Copilot CLI dispatch support and agent-based invocation.
src/specify_cli/integrations/codex/init.py	Adds Codex CLI dispatch args builder.
src/specify_cli/init.py	Adds specify workflow ... and specify workflow catalog ... CLI commands.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comments suppressed due to low confidence (2)

src/specify_cli/workflows/engine.py:571

• Nested step execution uses step_offset=-1, which prevents updating RunState.current_step_index while running nested steps. This makes the persisted run state ambiguous (index points at the parent step, while current_step_id points at a nested step), and interacts badly with resume logic that relies on current_step_index. If nested steps are intended to be resumable, the state model likely needs to track nested execution position explicitly.

            # Execute nested steps (from control flow)
            if result.next_steps:
                self._execute_steps(
                    result.next_steps, context, state, registry,
                    step_offset=-1,
                )

src/specify_cli/workflows/catalog.py:413

• In search(), tags = wf_data.get("tags", []) is assumed to be a list; if a catalog provides tags as null or a string, [t.lower() for t in tags] will raise or behave unexpectedly. Normalize tags to a list (or skip invalid tags) before filtering.

                    continue
            if tag:
                tags = wf_data.get("tags", [])
                if tag.lower() not in [t.lower() for t in tags]:
                    continue

• Files reviewed: 27/27 changed files
• Comments generated: 9

[Copilot]

Pull request overview

Introduces a new workflow subsystem to Specify CLI, enabling multi-step, resumable automation workflows defined in YAML, plus a workflow catalog/registry model and new specify workflow ... CLI commands.

Changes:

• Added workflow engine core (definition loading/validation, execution, state persistence, expression evaluation) and 10 built-in step types.
• Added workflow catalog + registry support (multi-catalog resolution, caching, search, install/remove) and corresponding CLI commands.
• Added documentation, example workflow(s), and a large new test suite for workflow functionality.

Show a summary per file

File	Description
workflows/speckit/workflow.yml	Adds an example “Full SDD Cycle” workflow definition.
workflows/catalog.json	Adds the default (official) workflow catalog file scaffold.
workflows/catalog.community.json	Adds the community workflow catalog file scaffold.
workflows/README.md	Adds end-user documentation for workflows, step types, inputs, state/resume, and catalogs.
workflows/PUBLISHING.md	Adds workflow publishing guidelines and catalog submission conventions.
workflows/ARCHITECTURE.md	Documents workflow engine internals (execution model, registry, expressions, catalogs).
tests/test_workflows.py	Adds extensive unit/integration tests for workflows, steps, expressions, catalogs, and persistence.
src/specify_cli/workflows/base.py	Defines workflow base types: context/result/status enums and step interface.
src/specify_cli/workflows/init.py	Implements global step registry and registers built-in step types.
src/specify_cli/workflows/expressions.py	Implements sandboxed, Jinja-like expression evaluation and filters.
src/specify_cli/workflows/engine.py	Implements workflow execution, validation, state persistence, and resume.
src/specify_cli/workflows/catalog.py	Implements catalog stack resolution, caching, search, and installed-workflow registry.
src/specify_cli/workflows/steps/init.py	Step package marker for built-in step discovery.
src/specify_cli/workflows/steps/command/init.py	Adds command step type that dispatches Spec Kit commands via integration CLIs.
src/specify_cli/workflows/steps/prompt/init.py	Adds prompt step type for ad-hoc integration prompts.
src/specify_cli/workflows/steps/shell/init.py	Adds shell step type for running local shell commands.
src/specify_cli/workflows/steps/gate/init.py	Adds interactive gate step type for human review checkpoints.
src/specify_cli/workflows/steps/if_then/init.py	Adds conditional branching step type.
src/specify_cli/workflows/steps/switch/init.py	Adds switch/multi-branch dispatch step type.
src/specify_cli/workflows/steps/while_loop/init.py	Adds while-loop step type.
src/specify_cli/workflows/steps/do_while/init.py	Adds do-while loop step type.
src/specify_cli/workflows/steps/fan_out/init.py	Adds fan-out step type (engine-driven per-item execution).
src/specify_cli/workflows/steps/fan_in/init.py	Adds fan-in aggregation step type.
src/specify_cli/integrations/base.py	Adds a CLI dispatch interface (build_exec_args / dispatch_command) for integrations.
src/specify_cli/integrations/copilot/init.py	Extends Copilot integration with CLI execution + agent dispatch support.
src/specify_cli/integrations/codex/init.py	Adds CLI execution argument builder for Codex integration.
src/specify_cli/init.py	Adds specify workflow and specify workflow catalog command groups and subcommands.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

• Files reviewed: 27/27 changed files
• Comments generated: 8

[Copilot]

Pull request overview

Adds a new workflow subsystem to specify CLI, enabling YAML-defined, multi-step automation (with resumable run state), plus a workflow catalog/registry and new CLI commands for running and managing workflows.

Changes:

• Introduces workflow core modules (definition loading/validation, expression evaluation, execution engine with persisted RunState, and catalog/registry support).
• Adds 10 built-in step types (command/prompt/shell/gate + control-flow constructs) and a built-in “speckit” workflow YAML example.
• Extends integration dispatch capabilities for non-interactive CLI execution and wires new specify workflow ... CLI commands; adds a comprehensive test suite.

Show a summary per file

File	Description
workflows/speckit/workflow.yml	Adds a built-in example workflow (“Full SDD Cycle”) using command + gate steps.
workflows/catalog.json	Introduces the official workflow catalog scaffold.
workflows/catalog.community.json	Introduces the community workflow catalog scaffold.
workflows/README.md	Documents workflow usage, step types, expressions, state/resume, and catalogs.
workflows/PUBLISHING.md	Adds workflow publishing guidance and catalog contribution rules.
workflows/ARCHITECTURE.md	Describes workflow engine execution model, state, step registry, expressions, and catalog resolution.
tests/test_workflows.py	Adds end-to-end and unit test coverage for workflow engine, steps, expressions, catalog, and registry.
src/specify_cli/workflows/init.py	Implements the global STEP_REGISTRY and registers built-in step implementations.
src/specify_cli/workflows/base.py	Adds workflow base types: StepBase, StepContext, StepResult, and status enums.
src/specify_cli/workflows/expressions.py	Adds a sandboxed Jinja-like expression evaluator used in YAML templates.
src/specify_cli/workflows/engine.py	Implements workflow parsing/validation, execution, nested-step handling, run persistence, and resume.
src/specify_cli/workflows/catalog.py	Implements workflow catalog stack resolution, caching, searching, and installed workflow registry.
src/specify_cli/workflows/steps/init.py	Adds steps package marker for built-in step type discovery.
src/specify_cli/workflows/steps/command/init.py	Implements command step dispatching to integration CLIs.
src/specify_cli/workflows/steps/prompt/init.py	Implements prompt step for sending arbitrary prompts to integration CLIs.
src/specify_cli/workflows/steps/shell/init.py	Implements shell step for running local shell commands.
src/specify_cli/workflows/steps/gate/init.py	Implements gate step for interactive/non-interactive review checkpoints.
src/specify_cli/workflows/steps/if_then/init.py	Implements if conditional branching step.
src/specify_cli/workflows/steps/switch/init.py	Implements switch multi-branch dispatch step.
src/specify_cli/workflows/steps/while_loop/init.py	Implements while loop step returning nested steps when condition holds.
src/specify_cli/workflows/steps/do_while/init.py	Implements do-while loop step (body runs at least once).
src/specify_cli/workflows/steps/fan_out/init.py	Implements fan-out step (item expansion metadata for engine execution).
src/specify_cli/workflows/steps/fan_in/init.py	Implements fan-in step for aggregating outputs from referenced steps.
src/specify_cli/integrations/base.py	Adds integration CLI dispatch primitives (build_exec_args, dispatch_command, invocation helpers).
src/specify_cli/integrations/copilot/init.py	Adds Copilot CLI dispatch support (copilot -p ..., agent selection) and updates docstring.
src/specify_cli/integrations/codex/init.py	Adds Codex CLI build_exec_args implementation (codex exec ...).
src/specify_cli/init.py	Wires new specify workflow ... and specify workflow catalog ... CLI commands.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

• Files reviewed: 27/27 changed files
• Comments generated: 6