Skip to content

Move the config-manager file splitting and merging into frodo-lib (file sets) #1365

Open

Description

@vscheuber

Summary

Move the splitting and merging of configuration for the config-manager commands from the CLI into frodo-lib, so that the library turns configuration into file sets and back and never touches the file system, and the CLI keeps the interface and the file system adapters. The rule is the proposed core record "The library turns configuration into file sets and back, and never touches the file system" (core 0024, in both repositories: #1366 and rockcarver/frodo-lib#1115). It completes the direction that CLI records 0023, config-manager has its own layer outside ops and 0024, config-manager logic lives in the CLI left unscheduled.

Library twin: rockcarver/frodo-lib#1114. Related: #1264 (the plan for the command group), #1355, #1362 (the key set rule is a pure module that moves unchanged).

Schedule (maintainer decision, 2026-11-16)

The move is finished before 5.0.0 final is released. Prereleases of 5.0.0 continue meanwhile. A move changes no byte of what a command writes or reads, so users see no change.

Terms

Agreed:

Term Meaning
file set A list of files, each with a path and content. As data only: it is never the file system itself. In code the type is called VirtualFileSet. What a pull returns and a push receives.
file set reader (FileSetReader) What a push asks for files: it lists a directory and reads a file. The consumer supplies it.
pull, push The verbs of the tenant-facing functions in a type's ops module (pullJourney, pullJourneys, pushJourney, pushJourneys): pull reads from the tenant and returns a file set, push takes a file set from a reader and writes it to the tenant. They are also the CLI's words, so config-manager pull journeys calls pullJourneys. Needs a library record that supersedes library record 0006, CRUDQ naming for facade methods and teaches the capability registry the two prefixes (pull as export, read-only; push as import, a write).
export, import Keep their meaning: the JSON export of the library, used by the heritage commands (exportJourneys, importJourneys).
split, merge The pure functions with no tenant, no state and no file system: splitJourney(s) turns a journey export into a file set, mergeJourney(s) turns a file set back. Exported from the type's ops module, not on the facade.
configuration type, or just type One type of object that config-manager manages and that has one owner (core 0022, a configuration type has one owner in every layer): a journey, a script, a trusted JWT issuer. In code ConfigType, never a bare Type.
supertype, subtype Where AM has the structure, one type is a subtype of another. Agent is a supertype and each agent type (OAuth2 client, CIMD profile, trusted JWT issuer, web agent, gateway agent, and so on) is a subtype of it. The supertype's module owns every agent type that has no owner of its own; a subtype with an owner of its own is skipped by the supertype. Journeys, scripts and themes have no subtypes. Replaces "generic" and "dedicated" (oauth2-agents against oauth-issuers and the other dedicated commands).
owner The module that reads and writes a type (core 0022): the subtype's own module, or the supertype's when it has none. Replaces "a kind of its own".

To settle before the first library pull request (each gets a decision round with the maintainer, nothing is built before):

  1. The registry: its name and what each owner declares (key, scope, static or dynamic, deployments, order).
  2. The names of the file entry type and of the CLI's adapters (the one that writes a file set to the file system and the one that reads it back; proposed names: fsFileSetWriter and fsFileSetReader).
  3. How a file set says which folders it owns, so that --clean knows what to remove (journeys remove their nodes folder today).
  4. Whether the CLI's existing "kind", "generic" and "dedicated" names (ConfigManagerAgentKind, DedicatedAgentKinds, pullAgentKind and the texts of records 0053 and 0054, one writer and the plain shape) are renamed to "type", "supertype" and "subtype" in a rename-only pull request in phase 1.
  5. The names of the callbacks and the result of a push.

Phases

Each phase ends with a library release, a pin and the CLI pull request that uses it. A type is done when its library function exists with a test that pins its file set byte for byte against what the CLI wrote before, the CLI command uses it, and the CLI's own copy is deleted in the same pull request.

Phase What Done when
0 Core record, library record and terms The core record is accepted in both repositories, a library record supersedes library record 0006 (pull and push, and the two registry prefixes), and the terms above are settled
1 Untangle in the CLI Every legacy module keeps its pure split and merge apart from file reads and writes, with unit tests; names renamed if item 4 says so. No behaviour change, the existing e2e snapshots hold
2 Foundations in the library The file set and FileSetReader types, the registry, the generic engine, overrides, placeholders, callbacks; the CLI's file system adapters
3 Pilot: the agent supertype and its subtypes The agent supertype (which owns remote consent agents, OAuth2 clients and every agent type that has no owner of its own) and the subtypes with an owner of their own: trusted JWT issuers, CIMD profiles, software publishers. The pure modules AgentFileShape and AgentKeySets move unchanged
4 The simple types The types of the 25 simple legacy modules: access config, audit, cookie domains, CORS, CSP, email provider and templates, endpoints, KBA, locales, password policy, remote servers, schedules, terms and conditions, UI config, internal roles, connector definitions and mappings, managed and service objects, organisation privileges, secret mappings, telemetry, authentication
5 The transform-heavy types Journeys, SAML, secrets, variables, scripts, authorization policies, services, IGA workflows, themes, custom nodes, applications (the folder per application of record 0042)
6 The aggregate ConfigOps (the full export and import) and the file-set versions of pull all, push all-static and delete iterate over the registry that the owners fill in; the CLI's own table of types, static and dynamic configuration and deployments is removed
7 Commands and cleanup Every config-manager command uses the library; src/configManagerOps/ is gone; the CLI's records 0023 and 0024 are superseded

What each type checks before it moves

A pull is a split of the type's export and a push is a merge followed by its import (core 0024), so a type's export and import must offer what the pull and push need. On 2026-11-17 about 9 of the 46 CLI modules call an export function of the library; the others call read functions, so the export of a type often has to learn something first:

  • Data shape: whether the export returns what the layout needs (decoded scripts, no base64 or string arrays, no ESV placeholders where the layout wants the value) or whether the split decodes it.
  • Dependencies: the export can select which dependencies it reads. A journey pull writes the journey, its inner journeys and its scripts, not themes, SAML providers or email templates (those belong to their own types).
  • Per-object results: a result callback for each object, so that one failure does not stop the others and the summary and --allow-failures (decision record 0040, a command that runs many operations runs them all) work.
  • Push behaviour: options of the import for what the CLI does today: skip what is identical, name an attribute the server refuses, create then update.
  • Requests: the pull makes the requests of today's pull (a list read where the pull uses one), checked by a test, so the recordings still replay.
  • Functions that write files today: exportScriptToFiles, exportMappingToDirectory, exportCircleOfTrustToFile and their import counterparts are reworked into the file-set pattern.

Risks

  • A type whose export reads differently from today's pull changes the requests of its e2e replays. Those recordings are made again against a development tenant, type by type, in the phase of the type.
  • The options of export and import grow. A new option is added only when a pull or push needs it, and the heritage commands may use it.

Inventory

On 2026-11-16, at main after #1362: the legacy layer src/configManagerOps/ has 42 modules and about 8,900 lines; the newer types in src/ops/ (the shared agent module, issuers, CIMD profiles, software publishers, applications, and the two pure modules) have about 3,300 lines; the commands are about 120 files. CLI record 0024 sized the whole move as large when it counted 5,200 lines of operations on 2026-10-24.

Not in this issue

New command names or options (those are #1264), and any change to the files a command writes.

Activity

  1. added
    enhancementNew feature or request
    no-auto-processingConcierge never touches this item; remove the label to hand it over
    on Nov 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestno-auto-processingConcierge never touches this item; remove the label to hand it over

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions