Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions NOTICE
Original file line number Diff line number Diff line change
Expand Up @@ -69,3 +69,8 @@ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

Bounded workspace adoption, aggregate verification and recoverable configuration transactions
Copyright 2026 FINOS
Distributed under the repository Apache-2.0 license. No new third-party code or
license restrictions are introduced by workspace migration.
4 changes: 3 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,9 @@ and `morphir config set/unset` provide typed, scoped TOML previews and writes,
source-preserving changes, optimistic conflicts and explicit recovery ownership.
Shared [classic project migration](docs/content/docs/configuration/project-migration.md)
and `morphir project migrate` provide loss-checked previews and guarded TOML
adoption while retaining classic bytes. Workspace migration remains planned. See the
adoption while retaining classic bytes. Shared [workspace migration](docs/content/docs/configuration/workspace-migration.md)
adds bounded inventory, verified membership and recoverable batch application through
`morphir workspace migrate [directory]`. See the
[reader profile](src/Morphir.Configuration.IO/README.md) for syntax and provenance.

## SDK libraries
Expand Down
2 changes: 1 addition & 1 deletion docs/content/docs/cli/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ These exit codes are designed for CI/CD integration and scripting.

The [configuration guide]({{< relref "/guides/configuration" >}}) distinguishes the compatibility TOML cache loader from the additive shared configuration and project system.

The [morphir config read commands]({{< relref "/docs/cli/config" >}}) inspect effective/scoped values, source reports and redacted explanations. JSON output follows the published schema; success, missing keys and usage/loading failures use exit codes 0/1/2. Shared `config set/unset` editing and [project migrate]({{< relref "/docs/cli/project-migrate" >}}) are available; workspace migration remains planned. Start with the [workspace configuration walkthrough]({{< relref "/docs/getting-started/configuration" >}}).
The [morphir config read commands]({{< relref "/docs/cli/config" >}}) inspect effective/scoped values, source reports and redacted explanations. JSON output follows the published schema; success, missing keys and usage/loading failures use exit codes 0/1/2. Shared `config set/unset` editing, [project migrate]({{< relref "/docs/cli/project-migrate" >}}) and bounded [workspace migrate]({{< relref "/docs/cli/workspace-migrate" >}}) are available. Start with the [workspace configuration walkthrough]({{< relref "/docs/getting-started/configuration" >}}).

## Troubleshooting

Expand Down
2 changes: 1 addition & 1 deletion docs/content/docs/cli/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -240,7 +240,7 @@ Native filesystem access keeps the existing no-follow policy. On macOS, the conv

## Project migration

Use [project migrate]({{< relref "/docs/cli/project-migrate" >}}) to adopt one classic project through loss-checked shared planning and guarded application. Workspace migration remains planned. Editing does not remove classic manifests, perform migration equivalence checks or adopt workspace membership automatically.
Use [project migrate]({{< relref "/docs/cli/project-migrate" >}}) to adopt one classic project, or [workspace migrate]({{< relref "/docs/cli/workspace-migrate" >}}) for a bounded repository. Both use loss-checked shared planning and guarded application. Editing does not remove classic manifests, perform migration equivalence checks or adopt workspace membership automatically.

## Related documentation

Expand Down
41 changes: 40 additions & 1 deletion docs/content/docs/cli/troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -155,7 +155,7 @@ Message: Malformed JSON: 'i' is an invalid start of a value. LineNumber: 6

## Configuration

The shared libraries and `morphir config get/list/show/path/explain` provide multi-format resolution, bounded discovery and inspection. Configuration editing uses shared plan/apply services; classic project migration is available and workspace migration remains planned. Existing artifact/cache tooling retains TOML cache settings and Git-first discovery. See the [guide]({{< relref "/guides/configuration" >}}) for the library APIs and the [command reference]({{< relref "/docs/cli/config" >}}) for read/write options and recovery semantics.
The shared libraries and `morphir config get/list/show/path/explain` provide multi-format resolution, bounded discovery and inspection. Configuration editing, classic project migration and bounded workspace migration use shared plan/apply services. Existing artifact/cache tooling retains TOML cache settings and Git-first discovery. See the [guide]({{< relref "/guides/configuration" >}}) for the library APIs and the [command reference]({{< relref "/docs/cli/config" >}}) for read/write options and recovery semantics.

### Wrong workspace or missing project

Expand Down Expand Up @@ -582,3 +582,42 @@ write failures. A `rolled-back` outcome restored the original target; a
`recovery-required` outcome names owned journal, stage and backup artifacts.
Preserve them and later user edits until manual recovery is resolved. Migration
retains the original classic source and never edits workspace membership.


## Workspace migration blocks or needs recovery

Run `morphir workspace migrate <directory> --dry-run --json` and inspect
`inventory`, `projects`, `diagnostics`, and `verified`. The selected directory is
the boundary even if it has no workspace. A child invocation does not adopt its
parent or siblings. Excluded candidates, default skipped directories, links,
and nested workspace boundaries appear in the inventory. Include/exclude controls
are root relative and repeatable; existing workspace exclusions stay effective.

One unsupported classic field or conflicting TOML value blocks the entire batch.
Resolve the reported project blocker and create a fresh preview. Migration retains
classic JSON bytes, uses the single-project loss ledger, and excludes ambient
personal/machine/profile/environment settings from equivalence checks. An adoption
effect for shared output roots does not permit source-directory or setting drift.
Generated artifacts remain where they are.

A conflict means input, target, listing, or topology evidence changed after
planning. Replan after reviewing that change. Do not weaken filesystem boundary or
link guards. On macOS use physical `/private/tmp` rather than its linked aliases.

`recovery-required` is unfinished work. Preserve all listed journals, stages,
backups, and user-modified targets. Overlapping configuration edits, project
migration, and workspace migration remain blocked until explicit recovery resolves
the journal. `uncertainDirectories` identifies directories whose creation receipt
was interrupted. Do not delete them even if they are empty. Explicit recovery can
preserve reviewed uncertain directories and reports them in `preservedDirectories`.
Inspect the shared transaction recovery report before choosing a recovery action. It identifies restored and unresolved paths; unresolved user
changes are preserved. See [workspace API and recovery]({{< relref
"/docs/configuration/workspace-migration" >}}) for the supported API. There is no
claim of cross-file crash atomicity, and silently deleting artifacts is not
recovery.

Every JSON result, including parse or host-initialization failure, follows the
[workspace result schema](https://github.com/finos/morphir-dotnet/blob/main/docs/spec/schemas/workspace-migration-result.schema.json).
Exit 0 means applicable preview, success, or valid-workspace no-op. Exit 2 means
blocked, invalid, failed, or recovery required. Diagnostics go to stderr and stdout
contains one JSON document when `--json` is supplied.
190 changes: 190 additions & 0 deletions docs/content/docs/cli/workspace-migrate.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,190 @@
---
title: "workspace migrate command"
linkTitle: "workspace migrate"
weight: 51
description: "Preview and apply bounded workspace adoption with per-project loss checks and recoverable writes"
---

# workspace migrate

`morphir workspace migrate` adopts eligible projects within one selected directory.
It uses the reusable inventory, aggregate planner/verifier, and transaction APIs.
The same operation is available to embedded hosts without invoking this command.

## Synopsis

```text
morphir workspace migrate [<directory>]
[--include <pattern>]... [--exclude <pattern>]...
[--dry-run] [--json]
```

`<directory>` defaults to the current directory. It must name an existing directory.
That directory bounds inventory, discovery, and all proposed destinations, including
when it contains no workspace declaration. Migration does not search ancestors for
projects to adopt. Select the intended repository root explicitly when invoking
from a member or source directory.

## Arguments and options

| Control | Meaning |
| --- | --- |
| `<directory>` | Bounded root; omitted means current directory |
| `--include <pattern>` | Include candidate project directories matching a root-relative pattern; repeatable |
| `--exclude <pattern>` | Exclude candidate project directories; repeatable |
| `--dry-run` | Return the complete verified plan and write nothing |
| `--json` | Write one redacted JSON result to stdout; diagnostics remain on stderr |
| `--help` | Show usage without discovery or migration |

Patterns are relative to the selected root. `*` and `?` match within a path
segment; `**` matches across directory segments. Absolute patterns and `..`
segments are invalid. Exclusions prune matching subtrees. Shell-quote wildcards
so the shell does not expand them before the CLI receives them. Prefix a literal
leading-dash pattern with `./`, for example `--include './--experimental'`, so it
cannot be mistaken for a missing value followed by an option. Existing `workspace.exclude`
settings remain effective alongside the supplied exclusions. Explicit filters do
not override nested workspace, link, version-control, dependency, or build-output
boundaries. Invalid controls block before application.

```bash
morphir workspace migrate . --dry-run
morphir workspace migrate ./repo --dry-run --json
morphir workspace migrate . --include 'packages/*' --include 'services/*' \
--exclude 'packages/experimental-*' --exclude 'services/retired' --dry-run --json
```

## Inventory and membership

Inventory reports project candidates, excluded paths, skipped links, nested declared
workspace boundaries, and already migrated projects in deterministic path order.
It remains bounded even without an existing workspace. It skips generated output,
version-control, and dependency directories and does not follow discovered links.

The plan uses explicit root-relative paths for new members. Existing membership,
exclusions, settings, and valid `workspace.default_member` remain in place. Already
migrated members are retained without rewriting their primaries. A root project
keeps its identity. A root without a project becomes a virtual workspace. Duplicate
project names produce diagnostics while their distinct paths remain separate.

For example, a new virtual workspace containing `packages/orders` and
`packages/payments` receives:

```toml
[workspace]
members = ["packages/orders", "packages/payments"]
```

Each classic project receives its own supported TOML primary. Hidden primary
layouts keep the owning project directory as the source-path anchor.

## Conversion and verification

Every selected classic project uses the [single-project migration
contract]({{< relref "/docs/cli/project-migrate" >}}), including the complete raw
loss ledger, unsupported-field blockers, conflict checks, and materialized legacy
defaults. A blocked member blocks the aggregate before any destination replacement.

Migration compares each classic source and its TOML conversion inside the same
proposed base workspace context. It excludes converted classic sources from the
candidate and ambient personal, machine, environment, explicit CLI, and activated
profile overlays from both sides. Declared profile controls are still checked.
The comparison preserves typed values and source-path meaning and checks the
resulting workspace discovery, membership, and selection.

Adoption effects appear separately in the plan: ownership, membership, default
selection, and shared output roots. Standalone projects may acquire a shared
`<workspace>/.morphir/out` output context. That effect cannot excuse unrelated
setting or source-path changes. Migration moves no generated artifacts and keeps
every classic source byte-identical.

## Output

Human output names the outcome, inventory entries, member paths, project edits,
workspace edits, adoption effects, and verification. Diagnostics go to stderr.

JSON follows the [workspace migration result
schema](https://github.com/finos/morphir-dotnet/blob/main/docs/spec/schemas/workspace-migration-result.schema.json).
Its fields include:

| Field | Content |
| --- | --- |
| `operation`, `outcome`, `exitCode` | Operation identity, result state, and exit code |
| `rootPath`, `rootIsProject` | Bounded root and root versus virtual identity |
| `changed`, `dryRun`, `verified` | Planned changes, preview mode, and aggregate verification |
| `inventory` | Path, relative path, kind, reason, and project name |
| `projects` | Per-project redacted ledger, edits, conflicts, and independent verification |
| `members`, `workspaceTarget`, `workspaceEdits` | Reviewed membership and workspace primary edits |
| `adoptionEffects` | Intentional ownership, membership, default-selection, and output-root effects |
| `diagnostics` | Structured warnings and blockers also sent to stderr |
| `recoveryArtifacts`, `restoredPaths`, `unresolvedPaths` | Retained owned evidence and recovery state |
| `uncertainDirectories`, `preservedDirectories` | Directory paths whose ownership is uncertain, or that explicit recovery was instructed to preserve |
| `valueTypes` | Typed encoding metadata for nonfinite and temporal values in workspace edits |

Each nested project report has its own `valueTypes` array. Its JSON pointers are
relative to that project report. Sensitive settings and opaque secret references
are redacted throughout the reports; reference targets are never resolved.

```bash
morphir workspace migrate . --dry-run --json > workspace-preview.json
morphir workspace migrate . --json > workspace-result.json
morphir workspace migrate . --json # unchanged after successful adoption
```

## Exit codes

| Code | Meaning |
| --- | --- |
| `0` | Applicable preview, successful application, or valid existing-workspace no-op |
| `2` | Invalid input, blocked conversion, failed verification, stale evidence, failed application, or recovery required |

An undeclared directory with no eligible projects reports no adoption plan and
exits 2. A valid existing workspace requiring no conversions succeeds unchanged.
Equivalent repeats preserve target bytes and timestamps.

## Application and recovery

Application validates and stages the entire aggregate before any destination is
replaced. It coordinates overlapping cooperating writers, rechecks source,
target, listing, and topology preconditions, and records durable recovery evidence.
A later replacement failure triggers rollback attempts. The result identifies
restored and unresolved paths.

Per-file atomic replacement does not make the batch crash-atomic. Incomplete
rollback or interrupted state retains journals, stages, and backups and produces
`recovery-required`. A read-only preview also reports pending recovery with exit
2 and retains the journal metadata. Overlapping configuration editing and
project/workspace migration writes remain blocked until explicit recovery resolves
that state.
Successful application and proven complete rollback remove only owned transaction
artifacts. User changes and unresolved evidence are preserved. If interruption
occurs after directory creation but before its ownership receipt returns,
`uncertainDirectories` names those paths explicitly. Recovery does not claim or
delete them. The shared recovery API can explicitly preserve reviewed uncertain
directories and reports them in `preservedDirectories`.

Use the shared [inspection and recovery APIs]({{< relref
"/docs/configuration/workspace-migration" >}}). There is no automatic cleanup of
uncertain recovery state. Do not delete a journal or backup to force a repeat
migration through. Classic sources are never removed by migration or recovery.

## Troubleshooting and automation

For a blocker, inspect `projects[*].ledger`, `projects[*].conflicts`, and
`diagnostics` before changing source or target settings. A fresh preview is needed
after source, target, relevant listing, or topology changes. A shared-output-root
adoption effect does not permit a different source directory.

CI should inspect the exit code and retain a failed JSON result as an artifact:

```bash
morphir workspace migrate "$REPOSITORY" --dry-run --json > workspace-plan.json
```

A preview still requires independent review before choosing to apply it. No
release, deployment, source deletion, or generated-output publication is part of
this operation. Migration is owned by `project` and `workspace`; there is no
`config migrate` alias.

See the [walkthrough]({{< relref "/docs/getting-started/configuration#adopt-a-repository-of-classic-projects" >}}),
[workspace overview]({{< relref "/docs/cli/workspace" >}}),
and [troubleshooting]({{< relref "/docs/cli/troubleshooting#workspace-migration-blocks-or-needs-recovery" >}}).
8 changes: 5 additions & 3 deletions docs/content/docs/cli/workspace.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,12 @@
title: "Workspace migration"
linkTitle: "Workspace"
weight: 50
description: "Planned bounded adoption of classic projects into a Morphir workspace"
description: "Bounded adoption of classic projects into a Morphir workspace"
---

# Workspace migration

Status: planned. `morphir workspace migrate` is not yet implemented in .NET. It will coordinate the shared [project migrator]({{< relref "/docs/cli/project" >}}) and workspace membership planning. Other hosts will use the same library services.
`morphir workspace migrate` coordinates the shared [project migrator]({{< relref "/docs/cli/project" >}}), bounded inventory, aggregate verification, and recoverable application. Other hosts use the same library services. The [complete command reference]({{< relref "/docs/cli/workspace-migrate" >}}) describes the JSON contract and recovery outcomes.

Use it when a repository contains several classic projects, including when the repository does not yet declare a Morphir workspace. For one project without workspace membership changes, use `morphir project migrate`.

Expand Down Expand Up @@ -53,7 +53,9 @@ Check the resulting workspace as a whole against those declared effects. Do not

Validate and stage the entire plan before replacing any destination. Recheck relevant source and target preconditions, coordinate overlapping writers, and journal replacements so a commit failure can attempt rollback.

Multiple file replacements are not crash-atomic. Report restored and unresolved paths explicitly if rollback fails. Retain recovery data and block overlapping writes until recovery is resolved. Successful application and complete rollback clean their temporary artifacts. Every classic `morphir.json` remains unchanged.
Multiple file replacements are not crash-atomic. Report restored and unresolved paths explicitly if rollback fails. Retain recovery data and block overlapping writes until recovery is resolved.
A preview encountering pending recovery reports `recovery-required` with exit 2
and the retained metadata without modifying it. Successful application and complete rollback clean their temporary artifacts. Every classic `morphir.json` remains unchanged.

An equivalent repeat reports unchanged files without rewriting them. A valid existing workspace needing no further conversion is a successful no-op. An undeclared root with no eligible projects reports that no adoption can be planned.

Expand Down
Loading
Loading