From 311761a6a39d67963a435b5e19ee329b43ed37bc Mon Sep 17 00:00:00 2001 From: Constantin Hager Date: Sun, 6 Sep 2026 08:29:37 +0200 Subject: [PATCH 1/3] docs(memory): refresh project context for delta report workflow Co-authored-by: AI Assistant --- .memory-bank/activeContext.md | 33 +++++++++++++++++++++++++++++++-- .memory-bank/progress.md | 16 +++++++++++++--- .memory-bank/promptHistory.md | 1 + .memory-bank/systemPatterns.md | 11 +++++++++++ 4 files changed, 56 insertions(+), 5 deletions(-) diff --git a/.memory-bank/activeContext.md b/.memory-bank/activeContext.md index 5758f44..5e82b4a 100644 --- a/.memory-bank/activeContext.md +++ b/.memory-bank/activeContext.md @@ -1,6 +1,6 @@ --- status: current -last-verified: 2026-08-17 +last-verified: 2026-09-06 owner: active-agent source: current task evidence --- @@ -9,7 +9,36 @@ source: current task evidence ## Current focus -`.\build.ps1` failed in `TestConfigData` with `[-] tests\ConfigData\AzHelpers.Tests.ps1 +The active work on this branch is the delta-report pipeline: it adds a +comparison workflow for exported Microsoft365DSC tenant configurations and keeps +it separate from the main export job. The current branch diff shows the main +pieces in `.build/DscConfigurationTasks.ps1`, `.build/Export/DeltaReport.ps1`, +`build.yaml`, and `pipelines/deltaReport.yml`, while `lab/20 Configure AzDo +Project.ps1` remains in the same change set as a related pipeline/project +configuration update. The reports are generated from the exported tenant config +artifacts, merged per tenant with `Join-M365DSCConfiguration`, and then compared +with `New-M365DSCDeltaReport` for each destination tenant. + +## Evidence + +- `git diff --stat main...HEAD` shows the current branch is changing five files in + the active delta-report work: `.build/DscConfigurationTasks.ps1`, `build.yaml`, + `lab/20 Configure AzDo Project.ps1`, `.build/Export/DeltaReport.ps1`, and + `pipelines/deltaReport.yml`. +- `build.yaml` introduces the `deltaReport` Invoke-Build workflow so the task can + be run independently from the main build and export sequences. +- `.build/DscConfigurationTasks.ps1` adds + `InitializeModuleFolderForDeltaReport` and `NewM365DscDeltaReport`, which + verify input directories, merge exported tenant configs into a source-drift + staging structure, and write the HTML delta reports under `output/DeltaReport`. +- `pipelines/deltaReport.yml` is a dedicated Azure DevOps pipeline definition. It + downloads the export artifact from the upstream export pipeline, initializes the + required modules, runs the delta-report task with a configured source tenant, + and publishes the result as the `DeltaReport` artifact. + +## Earlier focus + +`\.build.ps1` failed in `TestConfigData` with `[-] tests\ConfigData\AzHelpers.Tests.ps1 failed with: InvalidOperationException: A 'break' or 'continue' statement with a label that does not match any enclosing loop escaped from your code`. That message is a Pester 6.1.0 misdiagnosis. The real error is a diff --git a/.memory-bank/progress.md b/.memory-bank/progress.md index 971bd90..fe3be5f 100644 --- a/.memory-bank/progress.md +++ b/.memory-bank/progress.md @@ -1,6 +1,6 @@ --- status: current -last-verified: 2026-08-17 +last-verified: 2026-09-06 owner: active-agent source: repository evidence --- @@ -9,11 +9,21 @@ source: repository evidence ## Current status -The build is green on `feature/update2608` with the dependency set updated to -August 2026 levels. +The current branch is `feature/deltapipeline` and the active work is the new +export-to-delta-report workflow for comparing tenant configuration drift across +source and destination tenants. ## Recent milestones +- 2026-09-06 Added a tenant delta-report workflow. The branch now includes a + `deltaReport` workflow in `build.yaml`, the task implementation in + `.build/DscConfigurationTasks.ps1`, the report generator in + `.build/Export/DeltaReport.ps1`, and the pipeline definition in + `pipelines/deltaReport.yml`. The workflow downloads exported configuration + artifacts, validates the source tenant, merges per-tenant config sets using + `Join-M365DSCConfiguration`, compares them with `New-M365DSCDeltaReport`, and + publishes `output/DeltaReport` as an Azure DevOps artifact. + - 2026-08-17 Fixed `.\build.ps1` failing in `TestConfigData` with Pester 6.1.0's `A 'break' or 'continue' statement ... escaped from your code`. That message is a misdiagnosis: Pester 6.1.0 throws it from a `finally` over any terminating diff --git a/.memory-bank/promptHistory.md b/.memory-bank/promptHistory.md index 53c87ef..f312c5d 100644 --- a/.memory-bank/promptHistory.md +++ b/.memory-bank/promptHistory.md @@ -15,3 +15,4 @@ source: session interactions 2026-08-07 09:10 UTC | software-engineer | Fix the DSC enact failing with `MSFT_SPOAccessControlSettings ... holds no SharePoint context`, do not commit 2026-08-17 10:00 UTC | software-engineer | Investigate and fix the build failing under Pester 6 with `a 'break' or 'continue' statement ... escaped from your code`, do not commit 2026-08-17 11:10 UTC | software-engineer | Read the test module pins from `RequiredModules.psd1` instead of duplicating them +2026-09-06 11:45 UTC | software-engineer | Analyze the current branch delta and update the Memory Bank to reflect the tenant delta-report workflow diff --git a/.memory-bank/systemPatterns.md b/.memory-bank/systemPatterns.md index 14ec0ae..7df327c 100644 --- a/.memory-bank/systemPatterns.md +++ b/.memory-bank/systemPatterns.md @@ -32,6 +32,17 @@ app registrations, the Azure DevOps project and the agent VMs. ## Decisions +### Decision 17: Keep tenant drift reporting as a post-export step + +- Choice: Generate the delta report from already-exported tenant configuration + artifacts instead of folding comparison into the export job itself. +- Rationale: The source and destination tenant exports are produced as a set of + `TenantConfig-*` artifacts, and the reporting step merges each tenant's + individual resource configs with `Join-M365DSCConfiguration` before comparing + them with `New-M365DSCDeltaReport`. This keeps the workflow deterministic: the + export pipeline creates the input set, then the reporting pipeline can run on + any artifact bundle without needing live connectivity or a second export pass. + ### Decision 1: Use the canonical Memory Bank base - Choice: Keep durable project context in .memory-bank. From 66dd606e31325151e4b382a3259f72330df6e50e Mon Sep 17 00:00:00 2001 From: Constantin Hager Date: Sun, 6 Sep 2026 08:31:14 +0200 Subject: [PATCH 2/3] feat(deltaReport): add delta report workflow to compare tenant configurations --- .build/DscConfigurationTasks.ps1 | 21 ++++- .build/Export/DeltaReport.ps1 | 129 ++++++++++++++++++++++++++++++ CHANGELOG.md | 8 ++ build.yaml | 4 + lab/20 Configure AzDo Project.ps1 | 2 +- pipelines/deltaReport.yml | 62 ++++++++++++++ 6 files changed, 224 insertions(+), 2 deletions(-) create mode 100644 .build/Export/DeltaReport.ps1 create mode 100644 pipelines/deltaReport.yml diff --git a/.build/DscConfigurationTasks.ps1 b/.build/DscConfigurationTasks.ps1 index 5ca8f22..de7ae89 100644 --- a/.build/DscConfigurationTasks.ps1 +++ b/.build/DscConfigurationTasks.ps1 @@ -75,7 +75,7 @@ task CleanModuleFolder { Wait-DscLocalConfigurationManager - dir -Path $programFileModulePath | + Get-ChildItem -Path $programFileModulePath | Where-Object { $_.BaseName -notin $modulesToKeep } | Remove-Item -Recurse -Force @@ -105,3 +105,22 @@ task InitializeModuleFolder { } } + +task InitializeModuleFolderForDeltaReport { + + Wait-DscLocalConfigurationManager + + $programFileModulePath = 'C:\Program Files\WindowsPowerShell\Modules' + + Write-Host "Copying modules from '$requiredModulesPath' to '$programFileModulePath'" + Get-ChildItem -Path $requiredModulesPath | ForEach-Object { + Write-Host "Copying module '$($_.BaseName)'" + $_ | Copy-Item -Destination $programFileModulePath -Recurse -Force -ErrorAction SilentlyContinue -ErrorVariable copyErrors + + if ($copyErrors) + { + Write-Host "There were $($copyErrors.Count) errors copying the module '$($_.BaseName)'" + } + } + +} diff --git a/.build/Export/DeltaReport.ps1 b/.build/Export/DeltaReport.ps1 new file mode 100644 index 0000000..6f1fc95 --- /dev/null +++ b/.build/Export/DeltaReport.ps1 @@ -0,0 +1,129 @@ +task NewM365DscDeltaReport { + + $sourceTenant = if ($env:DeltaReportSourceTenant) + { + $env:DeltaReportSourceTenant + } + else + { + Write-Error "The environment variable 'DeltaReportSourceTenant' is not set. Please specify the source tenant for the delta report." -ErrorAction Stop + } + + $inputDirectory = if ($env:DeltaReportInputDirectory) + { + $env:DeltaReportInputDirectory + } + else + { + Join-Path -Path $OutputDirectory -ChildPath 'Export' + } + + $reportDirectory = if ($env:DeltaReportOutputDirectory) + { + $env:DeltaReportOutputDirectory + } + else + { + Join-Path -Path $OutputDirectory -ChildPath 'DeltaReport' + } + + if (-not (Test-Path -Path $inputDirectory)) + { + Write-Error "The input directory '$inputDirectory' does not exist. Please download the 'TenantConfig-*' artifacts first." -ErrorAction Stop + } + + Write-Host "Looking for exported tenant configurations in '$inputDirectory'" -ForegroundColor Yellow + + #The export creates '\\M365TenantConfig.ps1', hence the tenant is the grandparent of each configuration file. + $tenants = Get-ChildItem -Path $inputDirectory -Filter *.ps1 -File -Recurse | + Where-Object { $null -ne $_.Directory.Parent } | + Group-Object -Property { $_.Directory.Parent.FullName } + + if ($tenants.Count -eq 0) + { + Write-Error "Could not find any exported DSC configuration (*.ps1) in '$inputDirectory'." -ErrorAction Stop + } + + Write-Host "Found $($tenants.Count) tenant(s) in the input directory." -ForegroundColor Yellow + + $stagingDirectory = Join-Path -Path $reportDirectory -ChildPath '_staging' + if (Test-Path -Path $reportDirectory) + { + Remove-Item -Path $reportDirectory -Recurse -Force + } + New-Item -Path $stagingDirectory -ItemType Directory -Force | Out-Null + + $mergedConfigurations = @{} + + foreach ($tenant in $tenants) + { + $tenantName = Split-Path -Path $tenant.Name -Leaf + Write-Host "Merging $($tenant.Count) configuration file(s) of tenant '$tenantName'" -ForegroundColor Yellow + + $tenantStagingDirectory = Join-Path -Path $stagingDirectory -ChildPath $tenantName + New-Item -Path $tenantStagingDirectory -ItemType Directory -Force | Out-Null + + #Join-M365DSCConfiguration merges all configurations of one folder into the base file, so the nested export structure has to be flattened first. + $baseConfigurationFile = 'M365TenantConfig.ps1' + $isBaseConfiguration = $true + foreach ($configurationFile in ($tenant.Group | Sort-Object -Property FullName)) + { + $targetName = if ($isBaseConfiguration) + { + $baseConfigurationFile + } + else + { + "$($configurationFile.Directory.Name).ps1" + } + $isBaseConfiguration = $false + + Copy-Item -Path $configurationFile.FullName -Destination (Join-Path -Path $tenantStagingDirectory -ChildPath $targetName) -Force + } + + #Join-M365DSCConfiguration returns the merged configuration as a string instead of writing it to disk. + $mergedConfiguration = Join-M365DSCConfiguration -ConfigurationFile $baseConfigurationFile -ConfigurationPath $tenantStagingDirectory + + if ([System.String]::IsNullOrWhiteSpace($mergedConfiguration)) + { + Write-Error "Join-M365DSCConfiguration returned an empty configuration for tenant '$tenantName'." -ErrorAction Stop + } + + $tenantConfigurationPath = Join-Path -Path $reportDirectory -ChildPath "$tenantName.ps1" + Set-Content -Path $tenantConfigurationPath -Value $mergedConfiguration -Encoding utf8 -Force + $mergedConfigurations.Add($tenantName, $tenantConfigurationPath) + + Write-Host " Merged configuration written to '$tenantConfigurationPath'." -ForegroundColor Green + } + + Remove-Item -Path $stagingDirectory -Recurse -Force + + if (-not $mergedConfigurations.ContainsKey($sourceTenant)) + { + Write-Error "The source tenant '$sourceTenant' was not found in '$inputDirectory'. Available tenants: $($mergedConfigurations.Keys -join ', ')." -ErrorAction Stop + } + + $destinationTenants = $mergedConfigurations.Keys | Where-Object { $_ -ne $sourceTenant } | Sort-Object + if (-not $destinationTenants) + { + Write-Error "There is no tenant to compare the source tenant '$sourceTenant' with." -ErrorAction Stop + } + + foreach ($destinationTenant in $destinationTenants) + { + $reportPath = Join-Path -Path $reportDirectory -ChildPath "DeltaReport-$sourceTenant-vs-$destinationTenant.html" + Write-Host "Creating delta report of '$sourceTenant' against '$destinationTenant'" -ForegroundColor Yellow + + $Parameters = @{ + Source = $mergedConfigurations[$sourceTenant] + Destination = $mergedConfigurations[$destinationTenant] + OutputPath = $reportPath + Type = 'HTML' + DriftOnly = $true + } + New-M365DSCDeltaReport @Parameters + + Write-Host " Delta report written to '$reportPath'." -ForegroundColor Green + } + +} diff --git a/CHANGELOG.md b/CHANGELOG.md index 5cbe2da..d5d4244 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -33,6 +33,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 configured without SharePoint Online, so no `cSPO*` configuration is composed or compiled for it, and `.build/Export/ExportTenantData.ps1` skips the `SPO*` components. A new configuration data test guards the mechanism. +- Add a tenant delta-report workflow to compare exported Microsoft365DSC + configurations across source and destination tenants. The new `deltaReport` + Invoke-Build workflow in `build.yaml`, the `NewM365DscDeltaReport` task in + `.build/DscConfigurationTasks.ps1`, the merger in `.build/Export/DeltaReport.ps1`, + and the `pipelines/deltaReport.yml` pipeline now download the export output, + merge each tenant's configuration files with `Join-M365DSCConfiguration`, + generate HTML drift reports with `New-M365DSCDeltaReport`, and publish the + resulting `output/DeltaReport` artifact. ### Changed diff --git a/build.yaml b/build.yaml index 729dc27..969fa9b 100644 --- a/build.yaml +++ b/build.yaml @@ -46,6 +46,10 @@ BuildWorkflow: - LoadDatumConfigData - ConvertMofToYaml + deltaReport: + - test7 + - NewM365DscDeltaReport + build: - test7 - Clean diff --git a/lab/20 Configure AzDo Project.ps1 b/lab/20 Configure AzDo Project.ps1 index c4712b0..e712649 100644 --- a/lab/20 Configure AzDo Project.ps1 +++ b/lab/20 Configure AzDo Project.ps1 @@ -263,7 +263,7 @@ else # ---------------------------------------------------------- Write-Host 'Creating pipelines in project.' -$pipelineNames = 'build', 'export', 'push', 'reapply', 'test' +$pipelineNames = 'build', 'deltaReport', 'export', 'push', 'reapply', 'test' foreach ($pipelineName in $pipelineNames) { if (Invoke-VSTeamRequest -Area pipelines -Version 7.1 -Method Get -ProjectName $datum.Global.ProjectSettings.ProjectName | Select-Object -ExpandProperty value | Where-Object { $_.name -eq "M365DSC $pipelineName" }) diff --git a/pipelines/deltaReport.yml b/pipelines/deltaReport.yml new file mode 100644 index 0000000..450635e --- /dev/null +++ b/pipelines/deltaReport.yml @@ -0,0 +1,62 @@ +trigger: none + +parameters: + - name: sourceTenant + displayName: Source tenant of the delta report + type: string + default: templatetenant.onmicrosoft.com + +variables: + buildFolderName: output + defaultBranch: main + Agent.Source.Git.ShallowFetchDepth: 0 + +resources: + pipelines: + - pipeline: exportPipeline + source: M365DSC export + trigger: none + +pool: + vmImage: windows-latest + +steps: + - task: PowerShell@2 + name: displayEnvVariables + displayName: Display Environment Variables + inputs: + targetType: inline + pwsh: true + script: | + dir -Path env: | Format-Table -Property Name, Value -AutoSize | Out-String | Write-Host + + - download: exportPipeline + patterns: "**" + displayName: Download Exported Tenant Configurations + + - task: PowerShell@2 + name: InitializeModuleFolder + displayName: Initialize Program Files Modules + inputs: + pwsh: true + filePath: ./build.ps1 + arguments: -ResolveDependency -Tasks InitializeModuleFolderForDeltaReport #-UseModuleFast + + - task: PowerShell@2 + name: deltaReport + displayName: Create Delta Report + inputs: + pwsh: true + filePath: ./build.ps1 + arguments: -Tasks deltaReport + env: + DeltaReportInputDirectory: $(Pipeline.Workspace)/exportPipeline + DeltaReportSourceTenant: ${{ parameters.sourceTenant }} + + - task: PublishPipelineArtifact@1 + displayName: Publish Delta Report + inputs: + targetPath: $(buildFolderName)/DeltaReport + artifact: DeltaReport + publishLocation: pipeline + parallel: true From 26c9df86e69e053779247aa7bb77bf7c5c56aad1 Mon Sep 17 00:00:00 2001 From: Constantin Hager Date: Sun, 6 Sep 2026 08:55:26 +0200 Subject: [PATCH 3/3] fix(InitializeModuleFolderForDeltaReport): ensure module copy failures are caught and reported --- .build/DscConfigurationTasks.ps1 | 13 ++++++++----- .memory-bank/progress.md | 8 ++++++++ .memory-bank/promptHistory.md | 1 + CHANGELOG.md | 8 ++++++++ 4 files changed, 25 insertions(+), 5 deletions(-) diff --git a/.build/DscConfigurationTasks.ps1 b/.build/DscConfigurationTasks.ps1 index de7ae89..93c0dcb 100644 --- a/.build/DscConfigurationTasks.ps1 +++ b/.build/DscConfigurationTasks.ps1 @@ -114,12 +114,15 @@ task InitializeModuleFolderForDeltaReport { Write-Host "Copying modules from '$requiredModulesPath' to '$programFileModulePath'" Get-ChildItem -Path $requiredModulesPath | ForEach-Object { - Write-Host "Copying module '$($_.BaseName)'" - $_ | Copy-Item -Destination $programFileModulePath -Recurse -Force -ErrorAction SilentlyContinue -ErrorVariable copyErrors - - if ($copyErrors) + $module = $_ + Write-Host "Copying module '$($module.BaseName)'" + try { - Write-Host "There were $($copyErrors.Count) errors copying the module '$($_.BaseName)'" + $module | Copy-Item -Destination $programFileModulePath -Recurse -Force -ErrorAction Stop + } + catch + { + throw "Failed to copy module '$($module.BaseName)' to '$programFileModulePath': $_" } } diff --git a/.memory-bank/progress.md b/.memory-bank/progress.md index fe3be5f..fc1a910 100644 --- a/.memory-bank/progress.md +++ b/.memory-bank/progress.md @@ -15,6 +15,14 @@ source and destination tenants. ## Recent milestones +- 2026-09-06 Fixed `InitializeModuleFolderForDeltaReport` in + `.build/DscConfigurationTasks.ps1`: `Copy-Item` used + `-ErrorAction SilentlyContinue`, so a failed module copy left a stale + preinstalled module in place and the task finished successfully, letting + `New-M365DSCDeltaReport` generate and publish an incorrect report instead of + failing before artifact publication. Copy failures now throw, naming the + module via `$_.BaseName`. + - 2026-09-06 Added a tenant delta-report workflow. The branch now includes a `deltaReport` workflow in `build.yaml`, the task implementation in `.build/DscConfigurationTasks.ps1`, the report generator in diff --git a/.memory-bank/promptHistory.md b/.memory-bank/promptHistory.md index f312c5d..7ce4850 100644 --- a/.memory-bank/promptHistory.md +++ b/.memory-bank/promptHistory.md @@ -16,3 +16,4 @@ source: session interactions 2026-08-17 10:00 UTC | software-engineer | Investigate and fix the build failing under Pester 6 with `a 'break' or 'continue' statement ... escaped from your code`, do not commit 2026-08-17 11:10 UTC | software-engineer | Read the test module pins from `RequiredModules.psd1` instead of duplicating them 2026-09-06 11:45 UTC | software-engineer | Analyze the current branch delta and update the Memory Bank to reflect the tenant delta-report workflow +2026-09-06 06:53 UTC | software-engineer | Make Copy-Item failures terminating in InitializeModuleFolderForDeltaReport and include $_.BaseName in the error diff --git a/CHANGELOG.md b/CHANGELOG.md index d5d4244..bda8914 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Fixed + +- `InitializeModuleFolderForDeltaReport` in `.build/DscConfigurationTasks.ps1` no + longer silently ignores module copy failures. A failed `Copy-Item` now + throws immediately, naming the module (`$_.BaseName`), instead of letting + the task finish with a stale preinstalled module that could cause + `New-M365DSCDeltaReport` to generate and publish an incorrect report. + ### Added - Initial Upload