Repository navigation
test: record the e2e tests of the CIMD profile kind and write down the live check (cli#1252) - #1361
Merged
Merged
Conversation
…e live check The push, pull and delete of oauth-cimd-profiles are recorded against a throw-away profile on a development tenant, with the fixture, the snapshots and a sentence in the pull all test that names the new step. The pull makes one list request per realm and writes the plain shape; a pulled file pushed to a deleted profile and pulled again is byte-identical. Part of #1252. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The conflict was in comments of test/e2e/config-manager-pull-all.e2e.test.js only. The comments on main (frodo-cli#1359) name all three newer steps (CIMD profiles, issuers, software publishers) and are the more complete text, so the branch takes them. Nothing else changes. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #1252.
Test-only. The recorded e2e tests of
config-manager pull,pushanddelete oauth-cimd-profiles, their fixture and snapshots, and the live check of the kind on a Cloud development tenant (realmalpha, throw-away profileszz-live-cimdandzz-e2e-cimd, both deleted again; the listing of CIMD profiles is empty before and after).What is added
test/e2e/config-manager-{push,pull,delete}-cimd-profiles.e2e.test.js, recorded in that order againstzz-e2e-cimd, with 7 snapshots (read before committing) and the cassettes undertest/e2e/mocks/config-manager_*/{push,pull,delete}_*/oauth-cimd-profiles_*/. Recordings are made against the real tenant and were not edited.test/e2e/exports/fr-config-manager/cloud/realms/alpha/realm-config/agents/CIMDProfile/zz-e2e-cimd.json, from a real pull (plain shape, no_rev, one*.example.compattern).config-manager-pull-all.e2e.test.js(both blocks) that names theoauth-cimd-profilesstep: the replays are not re-recorded here (Re-record the config-manager pull all and pull all-static recordings: five steps fail on replay #1102), and the step fails in them because the old recordings predate it.What the live check settled (the full table is at the end)
GET .../CIMDProfile?_queryFilter=trueand no per-profile read, for all realms and with-n(the recorded pull cassette holds the same single list request). The files are plain (config-manager: write agent files without the { inherited, value } wrapper (60 percent fewer lines in the committed samples) #1355, record 0054): no wrapper members, no_rev.PUTthat answers201and a secondPUTthat answers200; an existing profile is updated by onePUT. A server refusal (Data validation failed for the attribute, Tree Name) fails that profile only, with the server's words, and exits 1 (0 with--allow-failures). A missing placeholder variable fails naming it and sends nothing. The delete guard,--dry-runandoauth2-agentsskips behave as specified.Deviations from the issue text (nothing in the code needs to change for them)
frodo agent export -i <profile>does not know the profile (the library keeps types with their own module out of it); the comparison was made withfrodo oauth cimd export, which gives the same values plus ametablock._rev;deletehas--dry-runonly;pull all-statichas no-C.alpha); item 16 is not testable on Cloud. Not exercised live: a complete profile with unset members (the live profile is sparse).Verification
npm run check,npx tsc --noEmit,npm run test:guidance(693 tests), the wholetest/e2e/config-managerdirectory in replay (114 suites, 255 tests, 5369 snapshots, no obsolete snapshot) and the whole suite pass (1100 suites, 10798 tests, 10286 snapshots; 10 suites and 149 tests are skipped, as onmain).CIMD_PROFILE_AGENT_TYPEto'CIMDProfileX'makes the recorded push and pull tests fail (2 of 3); restored, all 3 and 7 snapshots pass.The issue stays open for a maintainer to read the outcomes, write the confirmed facts back into #1247 to #1251 and close it.
Live checklist outcomes (cli#1252, A)
Where the issue says "wrapped form", the result is the plain form since #1358.
pull oauth-cimd-profiles -D outin empty alpha: exit 0,No CIMD profiles found., no file written. With-n zz-live-cimd: exit 1,CIMD profile 'zz-live-cimd' not found in realm 'alpha'._idand one patternhttps://zz-live.example.com/*). Requests: the type check (POST .../agents?_action=getAllTypes),PUT .../CIMDProfile/zz-live-cimd=201 Created, a secondPUT=200. DEVIATION from the plan of item 5a ("one list read, one import, one second PUT"): a push of a named new profile makes NO list read.GET .../CIMDProfile?_queryFilter=true, no per-profile read (cli#1358). The filerealms/alpha/realm-config/agents/CIMDProfile/zz-live-cimd.jsonis plain (0 wrapper members, no_rev, 13 lines).pull -n zz-live-cimdinto a second directory:diff -ridentical (also one list request). DEVIATION:frodo agent export -i zz-live-cimdanswersalpha realm agent 'zz-live-cimd' not found(exit 1): the library keeps agent types that have their own module out of the generic agent export. The comparison was made withfrodo oauth cimd export -i zz-live-cimd: the object (minus_rev, which is absent anyway) equals the pulled file exactly; the export adds ametablock (exportDate,exportTool,exportToolVersion,exportedBy,origin) that the config-manager file does not have.oauth-cimd-profiles.jsonin the plain form of the help example (cimdConfig.cimdClientIdUrlPatterns: ["${ZZ_LIVE_PATTERN}"]) with a profile that holds two patterns: the pull writes the placeholder as the first pattern and the second pattern stays visible (element-wise merge). The old wrapped form of the same override ({ "inherited": false, "value": [...] }, written for the first 5.0.0 prereleases) still works and gives an identical file (record 0054).ZZ_LIVE_PATTERN=https://zz-live-2.example.com/* ... push oauth-cimd-profiles -n zz-live-cimdwith the variable in the PROCESS environment (cli#1341): exit 0; an existing profile is updated by ONEPUT(200); read back: first pattern changed, second unchanged. Push of the unchanged file again: exit 0, nothing changes. DEVIATION: CIMD profiles have no_rev(the export shows none), so "the profile's_revdoes not change" cannot be checked; content read back is identical.PUT201 CreatedthenPUT200; pull again (file B):diff -r A BBYTE-IDENTICAL; push file A again, pull (file C):diff -r A CBYTE-IDENTICAL. Every pull made one list request. The stored profile is sparse (only what the file holds): the secondPUTstores exactly the file."_rev": "-123456789": exit 0, no400 Invalid attribute specified.zz-e2e-cimd(advancedOAuth2ClientConfig.treeName= a journey that does not exist) andzz-live-cimd(a changed pattern): the bad one fails with the server's wordsData validation failed for the attribute, Tree Name, the good one is imported (read back), summary1 of 2 CIMD profiles failed: - alpha/zz-e2e-cimd: Data validation failed for the attribute, Tree Name, exit 1; with--allow-failures: same output, exit 0. The failed profile was not created.ZZ_LIVE_PATTERNset: exit 1,No value found for placeholder "ZZ_LIVE_PATTERN",1 of 1 CIMD profile failed; zero PUT or POST requests for the type (--curlirizelog).201): shown in items 2 and 5a. NOT RUN: "push to another realm with the same file is independent": it needs a write inbravo; this session wrote only inalpha.pull all -C cfg -D all1(30 s, 623 files):CIMDProfile/zz-live-cimd.jsonis in the tree once.pull all-static -D all3(576 files): no CIMD file, and noagentsfolder at all. (pull all-statichas no-Coption.)pull alltwice: 622 of 623 files identical, including the CIMD file; the one difference is a fileTrustedJwtIssuer/zz-live-issuer.jsonthat appears only in the second tree: another recording session created that throw-away issuer in between (not mine, not touched). The local tenant trees were deleted afterwards.pull oauth2-agents -fwith aCIMDProfileentry (with overrides):Skipping 1 CIMDProfile entry of realm 'alpha' (zz-live-cimd): CIMD profiles are pulled by "frodo config-manager pull oauth-cimd-profiles". Its overrides are not applied here..., exit 0, no file, the config file unchanged, no agent request at all.push oauth2-agentswith a tree that holds only theCIMDProfilefolder:Skipping the 1 CIMDProfile file(s) of realm 'alpha': they are pushed by "frodo config-manager push oauth-cimd-profiles"., exit 0, no write request, no request for the type.pull oauth-cimd-profiles(1 file) and thenpull applications -Dthe same directory: exit 0, no refusal.delete oauth-cimd-profiles --dry-run:Dry run: Deleting CIMD profile: alpha/zz-live-cimd, exit 0, nothing deleted (the option is long-only; the issue's-dis not an option). Without a terminal and without-y:Refusing to delete all CIMD profiles in the selected realms without confirmation: there is no interactive terminal to ask..., exit 1, the profile still exists.-y(run only after a fresh listing showed that alpha held nothing butzz-live-cimd):Deleted CIMD profile: alpha/zz-live-cimd, exit 0, alpha empty.-n zz-live-cimdagain: exit 1,CIMD profile 'zz-live-cimd' not found in realm 'alpha'.pull oauth-cimd-profilesfor realm/:The root realm of an Identity Cloud tenant has no CIMD profiles., exit 0, nothing written; with-n x: same message, exit 1.bravooffers the type (a pull printsNo CIMD profiles found., exit 0). No realm without the OAuth 2.0 provider is known; creating in one would be a write outsidealpha.