Fix clean validation workflow failing due to az CLI azure-firewall extension change

[Copilot]

What is being addressed

The Clean Validation Environments workflow was consistently failing due to a checksum verification error triggered by the experimental az config command group. The az config set calls used to configure dynamic extension installation were themselves failing with a checksum mismatch before the azure-firewall extension could be installed.

How is this addressed

• Replaced az config set extension.use_dynamic_install=yes_without_prompt and az config set extension.dynamic_install_allow_preview=true with the AZURE_EXTENSION_USE_DYNAMIC_INSTALL=yes_without_prompt environment variable, avoiding the experimental az config command entirely
• Kept explicit installation of the azure-firewall extension via az extension add --name azure-firewall --allow-preview true --upgrade
• Retained all az network firewall CLI commands for firewall management
• Updated CHANGELOG.md

[rudolphjacksonm]

@copilot I've run the workflow against this branch and it progressed and then hit another azure cli issue:

WARNING: Command group 'config' is experimental and under development. Reference and support levels: https://aka.ms/CLI_refstatus
WARNING: Command group 'config' is experimental and under development. Reference and support levels: https://aka.ms/CLI_refstatus
ERROR: The checksum of the extension does not match the expected value. Use --debug for more information.

[Copilot]

@copilot I've run the workflow against this branch and it progressed and then hit another azure cli issue:

WARNING: Command group 'config' is experimental and under development. Reference and su...

The az extension add is also hitting the same CDN checksum issue. Updated the fix in commit to eliminate the azure-firewall extension dependency entirely by replacing all az network firewall commands with az rest direct ARM REST API calls — no extension installation needed at all.

[rudolphjacksonm]

@copilot I've run the workflow against this branch and it progressed and then hit another azure cli issue:

WARNING: Command group 'config' is experimental and under development. Reference and su...

The az extension add is also hitting the same CDN checksum issue. Updated the fix in commit to eliminate the azure-firewall extension dependency entirely by replacing all az network firewall commands with az rest direct ARM REST API calls — no extension installation needed at all.

Don't use direct API calls. The firewall extension works. The next error refers to az config being an experimental command group and the checksum of this not matching from what I can tell.

[Copilot]

@copilot I've run the workflow against this branch and it progressed and then hit another azure cli issue:

WARNING: Command group 'config' is experimental and under development. Refe...

Reverted to az network firewall commands and fixed the root cause. The az config command group is experimental and was itself triggering the checksum error. Replaced both az config set lines with the AZURE_EXTENSION_USE_DYNAMIC_INSTALL=yes_without_prompt environment variable — no az config calls needed. The az extension add --name azure-firewall --allow-preview true --upgrade line is kept to install the extension.

[rudolphjacksonm]

@copilot determined this was a transient issue due to the way this workflow is configured to pull the latest Azure CLI package at runtime on the host. We should pin this workflow like we do the others, something like this after the checkout block and before Azure Login:

- name: Install specific Azure CLI version
        run: |
          AZ_DIST=$(lsb_release -cs)
          AZ_VER=2.81.0

          apt-cache policy azure-cli
          sudo apt-get install azure-cli="${AZ_VER}-1~${AZ_DIST}" --allow-downgrades
          az version

Revert all the other changes around sub groups/extensions/etc.

[Copilot]

@copilot determined this was a transient issue due to the way this workflow is configured to pull the latest Azure CLI package at runtime on the host. We should pin this workflow like we do the others...

Done. Added the Azure CLI version pinning step (2.81.0) to clean_validation_envs.yml after the checkout block and before Azure Login, matching the pattern used in lets_encrypt.yml. All changes to control_tre.sh have been reverted.