Skip to content

test: record the e2e tests of the CIMD profile kind and write down the live check (cli#1252) - #1361

Merged
vscheuber merged 2 commits into
mainfrom
test/oauth-cimd-profiles-e2e
Nov 17, 2026
Merged

vscheuber merged 2 commits into
mainfrom
test/oauth-cimd-profiles-e2e

Conversation

@vscheuber

Copy link
Copy Markdown
Contributor

Part of #1252.

Test-only. The recorded e2e tests of config-manager pull, push and delete oauth-cimd-profiles, their fixture and snapshots, and the live check of the kind on a Cloud development tenant (realm alpha, throw-away profiles zz-live-cimd and zz-e2e-cimd, both deleted again; the listing of CIMD profiles is empty before and after).

What is added

  • test/e2e/config-manager-{push,pull,delete}-cimd-profiles.e2e.test.js, recorded in that order against zz-e2e-cimd, with 7 snapshots (read before committing) and the cassettes under test/e2e/mocks/config-manager_*/{push,pull,delete}_*/oauth-cimd-profiles_*/. Recordings are made against the real tenant and were not edited.
  • The fixture test/e2e/exports/fr-config-manager/cloud/realms/alpha/realm-config/agents/CIMDProfile/zz-e2e-cimd.json, from a real pull (plain shape, no _rev, one *.example.com pattern).
  • A sentence in the comments of config-manager-pull-all.e2e.test.js (both blocks) that names the oauth-cimd-profiles step: the replays are not re-recorded here (Re-record the config-manager pull all and pull all-static recordings: five steps fail on replay #1102), and the step fails in them because the old recordings predate it.

What the live check settled (the full table is at the end)

  • One request per realm: a pull makes exactly one GET .../CIMDProfile?_queryFilter=true and no per-profile read, for all realms and with -n (the recorded pull cassette holds the same single list request). The files are plain (config-manager: write agent files without the { inherited, value } wrapper (60 percent fewer lines in the committed samples) #1355, record 0054): no wrapper members, no _rev.
  • Round trip: pull, delete, push the pulled file, pull again, push again, pull again: the three pulled files are byte-identical.
  • Create is a PUT that answers 201 and a second PUT that answers 200; an existing profile is updated by one PUT. A server refusal (Data validation failed for the attribute, Tree Name) fails that profile only, with the server's words, and exits 1 (0 with --allow-failures). A missing placeholder variable fails naming it and sends nothing. The delete guard, --dry-run and oauth2-agents skips behave as specified.

Deviations from the issue text (nothing in the code needs to change for them)

  • frodo agent export -i <profile> does not know the profile (the library keeps types with their own module out of it); the comparison was made with frodo oauth cimd export, which gives the same values plus a meta block.
  • A push of a named new profile makes no list read; CIMD profiles have no _rev; delete has --dry-run only; pull all-static has no -C.
  • Not run: checklist item 9 second half and item 15 (they would write outside realm alpha); item 16 is not testable on Cloud. Not exercised live: a complete profile with unset members (the live profile is sparse).

Verification

  • npm run check, npx tsc --noEmit, npm run test:guidance (693 tests), the whole test/e2e/config-manager directory in replay (114 suites, 255 tests, 5369 snapshots, no obsolete snapshot) and the whole suite pass (1100 suites, 10798 tests, 10286 snapshots; 10 suites and 149 tests are skipped, as on main).
  • Mutation check on the replay: changing the path segment CIMD_PROFILE_AGENT_TYPE to 'CIMDProfileX' makes the recorded push and pull tests fail (2 of 3); restored, all 3 and 7 snapshots pass.
  • The new cassettes and the fixture contain no token, cookie, key or password (scanned). They name the development tenant host and the service-account id, like the existing recordings; the service-account lookup response carries the same description (with a maintainer e-mail address) as 515 recordings that are already committed, so no new data, but the recording filter does not remove it.

The issue stays open for a maintainer to read the outcomes, write the confirmed facts back into #1247 to #1251 and close it.

Live checklist outcomes (cli#1252, A)

Where the issue says "wrapped form", the result is the plain form since #1358.

# Result
1 OK. pull oauth-cimd-profiles -D out in empty alpha: exit 0, No CIMD profiles found., no file written. With -n zz-live-cimd: exit 1, CIMD profile 'zz-live-cimd' not found in realm 'alpha'.
2 OK, created through the kind's own push from a hand-written plain file (_id and one pattern https://zz-live.example.com/*). Requests: the type check (POST .../agents?_action=getAllTypes), PUT .../CIMDProfile/zz-live-cimd = 201 Created, a second PUT = 200. DEVIATION from the plan of item 5a ("one list read, one import, one second PUT"): a push of a named new profile makes NO list read.
3 OK. Pull of all realms: exactly ONE request for the type, GET .../CIMDProfile?_queryFilter=true, no per-profile read (cli#1358). The file realms/alpha/realm-config/agents/CIMDProfile/zz-live-cimd.json is plain (0 wrapper members, no _rev, 13 lines). pull -n zz-live-cimd into a second directory: diff -r identical (also one list request). DEVIATION: frodo agent export -i zz-live-cimd answers alpha realm agent 'zz-live-cimd' not found (exit 1): the library keeps agent types that have their own module out of the generic agent export. The comparison was made with frodo oauth cimd export -i zz-live-cimd: the object (minus _rev, which is absent anyway) equals the pulled file exactly; the export adds a meta block (exportDate, exportTool, exportToolVersion, exportedBy, origin) that the config-manager file does not have.
4 OK. Overrides file oauth-cimd-profiles.json in the plain form of the help example (cimdConfig.cimdClientIdUrlPatterns: ["${ZZ_LIVE_PATTERN}"]) with a profile that holds two patterns: the pull writes the placeholder as the first pattern and the second pattern stays visible (element-wise merge). The old wrapped form of the same override ({ "inherited": false, "value": [...] }, written for the first 5.0.0 prereleases) still works and gives an identical file (record 0054).
5 OK. ZZ_LIVE_PATTERN=https://zz-live-2.example.com/* ... push oauth-cimd-profiles -n zz-live-cimd with the variable in the PROCESS environment (cli#1341): exit 0; an existing profile is updated by ONE PUT (200); read back: first pattern changed, second unchanged. Push of the unchanged file again: exit 0, nothing changes. DEVIATION: CIMD profiles have no _rev (the export shows none), so "the profile's _rev does not change" cannot be checked; content read back is identical.
5a OK, with the cli#1355 round trip. Pull (file A), delete by name, push file A to the now missing name: PUT 201 Created then PUT 200; pull again (file B): diff -r A B BYTE-IDENTICAL; push file A again, pull (file C): diff -r A C BYTE-IDENTICAL. Every pull made one list request. The stored profile is sparse (only what the file holds): the second PUT stores exactly the file.
6 OK. A file that carries "_rev": "-123456789": exit 0, no 400 Invalid attribute specified.
7 OK. A tree with zz-e2e-cimd (advancedOAuth2ClientConfig.treeName = a journey that does not exist) and zz-live-cimd (a changed pattern): the bad one fails with the server's words Data validation failed for the attribute, Tree Name, the good one is imported (read back), summary 1 of 2 CIMD profiles failed: - alpha/zz-e2e-cimd: Data validation failed for the attribute, Tree Name, exit 1; with --allow-failures: same output, exit 0. The failed profile was not created.
8 OK. Without ZZ_LIVE_PATTERN set: exit 1, No value found for placeholder "ZZ_LIVE_PATTERN", 1 of 1 CIMD profile failed; zero PUT or POST requests for the type (--curlirize log).
9 PARTLY. Push to a realm that has no profile of that name creates it (201): shown in items 2 and 5a. NOT RUN: "push to another realm with the same file is independent": it needs a write in bravo; this session wrote only in alpha.
10 OK. pull all -C cfg -D all1 (30 s, 623 files): CIMDProfile/zz-live-cimd.json is in the tree once. pull all-static -D all3 (576 files): no CIMD file, and no agents folder at all. (pull all-static has no -C option.) pull all twice: 622 of 623 files identical, including the CIMD file; the one difference is a file TrustedJwtIssuer/zz-live-issuer.json that appears only in the second tree: another recording session created that throw-away issuer in between (not mine, not touched). The local tenant trees were deleted afterwards.
11 OK. pull oauth2-agents -f with a CIMDProfile entry (with overrides): Skipping 1 CIMDProfile entry of realm 'alpha' (zz-live-cimd): CIMD profiles are pulled by "frodo config-manager pull oauth-cimd-profiles". Its overrides are not applied here..., exit 0, no file, the config file unchanged, no agent request at all. push oauth2-agents with a tree that holds only the CIMDProfile folder: Skipping the 1 CIMDProfile file(s) of realm 'alpha': they are pushed by "frodo config-manager push oauth-cimd-profiles"., exit 0, no write request, no request for the type.
12 OK. Fresh empty directory: pull oauth-cimd-profiles (1 file) and then pull applications -D the same directory: exit 0, no refusal.
13 OK. delete oauth-cimd-profiles --dry-run: Dry run: Deleting CIMD profile: alpha/zz-live-cimd, exit 0, nothing deleted (the option is long-only; the issue's -d is not an option). Without a terminal and without -y: Refusing to delete all CIMD profiles in the selected realms without confirmation: there is no interactive terminal to ask..., exit 1, the profile still exists. -y (run only after a fresh listing showed that alpha held nothing but zz-live-cimd): Deleted CIMD profile: alpha/zz-live-cimd, exit 0, alpha empty. -n zz-live-cimd again: exit 1, CIMD profile 'zz-live-cimd' not found in realm 'alpha'.
14 OK. pull oauth-cimd-profiles for realm /: The root realm of an Identity Cloud tenant has no CIMD profiles., exit 0, nothing written; with -n x: same message, exit 1.
15 NOT RUN as a test (observation only). Realm bravo offers the type (a pull prints No CIMD profiles found., exit 0). No realm without the OAuth 2.0 provider is known; creating in one would be a write outside alpha.
16 Not testable on Cloud (the type is present); covered by mocks (cli#1253).

…e live check

The push, pull and delete of oauth-cimd-profiles are recorded against a throw-away profile
on a development tenant, with the fixture, the snapshots and a sentence in the pull all
test that names the new step. The pull makes one list request per realm and writes the
plain shape; a pulled file pushed to a deleted profile and pulled again is byte-identical.
Part of #1252.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@vscheuber vscheuber added the no-auto-processing Concierge never touches this item; remove the label to hand it over label Nov 16, 2026
The conflict was in comments of test/e2e/config-manager-pull-all.e2e.test.js only. The comments
on main (frodo-cli#1359) name all three newer steps (CIMD profiles, issuers, software publishers)
and are the more complete text, so the branch takes them. Nothing else changes.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@vscheuber
vscheuber merged commit 8a6b4d8 into main Nov 17, 2026
15 checks passed
@vscheuber
vscheuber deleted the test/oauth-cimd-profiles-e2e branch November 17, 2026 00:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no-auto-processing Concierge never touches this item; remove the label to hand it over

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant