Reset legacy test container DB in bulk instead of per-app uninstall loop - #9836
Reset legacy test container DB in bulk instead of per-app uninstall loop#9836Alexander Holstrup (aholstrup1) wants to merge 11 commits into
Conversation
Replace the per-app uninstall + schema-clean work in NewBcContainer.ps1 with a single Clean-BcContainerDatabase -useNewDatabase call, which drops and recreates an empty application database in one atomic operation instead of running Sync-NAVApp -mode Clean over ~150 pre-installed apps (~19 min per legacy job). The existing removal loop is kept but now runs after the reset, so it only iterates the few apps -useNewDatabase leaves (System Application, which BCApps rebuilds from source) and completes in seconds. Copilot-Session: 5871c8be-e8c1-4fb7-a0e4-b4a9a1df6f0f
Clean-BcContainerDatabase -useNewDatabase recreates the application database, which drops the developer license that ships pre-loaded in the BC artifact database. BCApps CI supplies no license file, so every test job failed with "Container must be started with a developer license". Re-import the .bclicense shipped inside the downloaded BC artifact (found under $bcContainerHelperConfig.bcartifactsCacheFolder) after the reset, then restart the container so the service tier picks it up. Copilot-Session: 5871c8be-e8c1-4fb7-a0e4-b4a9a1df6f0f
Clean-BcContainerDatabase -useNewDatabase throws "Container must be started with a developer license" at line 63 when no license.* file exists in the container's my-folder (Extensions\<container>\my). BCApps CI never passes a licenseFile to New-BcContainer, so that file was absent and every test job failed inside the reset - before the previous post-reset re-import could run. Copy the artifact's .bclicense (present in the BcContainerHelper artifacts cache after New-BcContainer) into the my-folder before calling Clean-BcContainerDatabase. The reset then passes its license check and re-imports the license into the fresh database itself (Clean-BcContainerDatabase line 181), so the separate post-reset re-import and restart are no longer needed and have been removed. Copilot-Session: 5871c8be-e8c1-4fb7-a0e4-b4a9a1df6f0f
Validated the -useNewDatabase database reset locally against a real BC
29.0.52913.0 container: reproduced the "must be started with a developer license"
failure (no license.* in the my-folder) and confirmed that seeding the artifact
license before the reset makes Clean-BcContainerDatabase succeed (~4 min, System
Application published) and leaves only System Application published.
Resolve the license via the artifact's manifest.json ("licenseFile") for the
exact artifactUrl the container was built from, instead of a recursive
*.bclicense search. On a machine with several cached artifact versions the
recursive search could pick a stale version or Master.bclicense; the manifest
lookup is version-pinned and selects the correct Cronus license.
Copilot-Session: 5871c8be-e8c1-4fb7-a0e4-b4a9a1df6f0f
Replace the ~19 min per-app uninstall/unpublish loop in NewBcContainer.ps1 with a
single bulk application-database drop, validated locally at ~3.6 min on a real BC
29.0.52913.0 multitenant container.
Every host-side UnInstall/Unpublish call is a separate in-container session (~8s of
overhead each), so unpublishing ~150 pre-installed apps one by one costs ~19 min
regardless of schema handling. Dropping and recreating the application database
removes all apps in one operation instead.
New build/scripts/ResetBcContainerDatabase.psm1 exposes
Reset-BcContainerApplicationDatabase(-ContainerName, -Credential, -LicenseFile):
- extracts the System Application from the container's own published copy (survives
BCApps' platform override, which strips it from C:\Applications),
- drops the application + tenant databases and creates a fresh empty application
database in one in-container session,
- re-imports the developer license (dropped with the old database),
- re-publishes the System Application (needed for the SUPER permission set, since
BCApps runs with UsePermissionSetsFromExtensions=true),
- creates a SUPER user,
- restores multitenancy with an empty 'default' tenant.
NewBcContainer.ps1 resolves the license from the artifact manifest, runs the reset,
then the existing loop removes the single remaining System Application so AL-Go
publishes the repository-built version.
Copilot-Session: 5871c8be-e8c1-4fb7-a0e4-b4a9a1df6f0f
The bulk container DB reset left the default tenant mounted read-only against the application database (AllowAppDatabaseWrite=False), so demo-data generation failed to insert Media records in all test-running jobs. Pass -allowAppDatabaseWrite on the default mount. Also force Windows PowerShell 5.1 (-usePwsh \False) for the multitenant-switch scriptblock; its Invoke-Sqlcmd and SMO-based Export-/Copy-NavDatabase calls fail to load the SQL SMO assemblies under the container's pwsh7. Both validated locally on a multitenant BC 28 container. Copilot-Session: 5871c8be-e8c1-4fb7-a0e4-b4a9a1df6f0f
…gacy-container-db-clean
Copilot-Session: 5871c8be-e8c1-4fb7-a0e4-b4a9a1df6f0f
Extract Export-SystemApplicationFromContainer, Clear-BcApplicationDatabase, and Restore-BcMultitenancy so the orchestrator reads as its sequence of phases and each in-container block can be exercised on its own. No behavior change. Copilot-Session: 5871c8be-e8c1-4fb7-a0e4-b4a9a1df6f0f
Locally verified the blocks still fail under the container's pwsh7 (New-NAVApplicationDatabase / Invoke-Sqlcmd hit a Microsoft.Extensions.Logging assembly-load conflict on BC 28), even though Copy-NavDatabase is no longer SMO-based. Comments now describe the real cause. Copilot-Session: 5871c8be-e8c1-4fb7-a0e4-b4a9a1df6f0f
|
In Clear-BcApplicationDatabase, Suggested fix (apply manually — could not be anchored as a one-click suggestion): $dbproperties = Invoke-Sqlcmd -ServerInstance $databaseServerInstance -Query "SELECT [applicationfamily] FROM [$databaseName].[dbo].[`$ndo`$dbproperty]"
if (-not $dbproperties -or -not $dbproperties.applicationfamily) {
throw "Could not read applicationfamily from database '$databaseName'; aborting reset to avoid losing this platform-required value."
}Line mapping was unavailable, so this was posted as an issue comment. 👍 useful · ❤️ especially valuable · 👎 wrong - reply with why · AL review agent v1.32.4 |
|
Reset-BcContainerApplicationDatabase runs Clear-BcApplicationDatabase (stop service tier, drop app+tenant+default DBs, flip to single-tenant, recreate 'tenant', restart) followed by license import, System Application publish, user creation, and Restore-BcMultitenancy (switch back to multitenant, mount 'default'), with no try/finally or cleanup path. If any step in this multi-stage, stateful sequence throws (e.g. Import-BcContainerLicense, Publish-BcContainerApp, or a step inside the container script blocks), the container is left half-reset — service tier possibly stopped, Line mapping was unavailable, so this was posted as an issue comment. 👍 useful · ❤️ especially valuable · 👎 wrong - reply with why · AL review agent v1.32.4 |
If the applicationfamily SELECT returns no row, the recreated database would be silently written with an empty applicationfamily - the exact value the read exists to preserve. Throw instead so the failure is loud. Addresses PR review feedback. Copilot-Session: 5871c8be-e8c1-4fb7-a0e4-b4a9a1df6f0f
What & why
Legacy test jobs spend ~19-24 min of every container setup uninstalling and unpublishing the ~150 apps pre-installed in the BC image, one at a time, before AL-Go publishes the repo-built apps. The bottleneck is not schema work but per-app container-session overhead (~8s each round-trip), so it cannot be optimized by tuning the loop.
This replaces that loop with a single bulk operation: drop and recreate the application database, which removes every app at once, then rebuild only what AL-Go needs to take over.
Approach
New
build/scripts/ResetBcContainerDatabase.psm1exposesReset-BcContainerApplicationDatabase, which:UsePermissionSetsFromExtensions=true), and creates a SUPER user,defaulttenant.build/scripts/NewBcContainer.ps1resolves the license deterministically from the artifact manifest and calls the reset. The existing removal loop stays but now runs over the single remaining System App.Linked work
Fixes AB#646519
How I validated this
What I tested and the outcome
AllowAppDatabaseWrite=True.-allowAppDatabaseWrite(otherwise demo-data generation cannot insert Media records), and the in-container database scriptblocks must run under Windows PowerShell 5.1 (-usePwsh $false) because the NAV management cmdlets /Invoke-Sqlcmdhit .NET assembly-load conflicts under the container's pwsh7.Note: the remaining known-flaky test failures in this run are pre-existing and are tolerated on real PR builds (the Test Tolerance feature only applies when
GITHUB_EVENT_NAME == 'pull_request', not on theworkflow_dispatchrun used above).Risk & compatibility
build/scripts); no product code changes.manifest.json). The reset throws early if the System App is not published or the license is missing, so a misconfigured container fails fast rather than silently.