learntocloud · madebygps · May 27, 2026 · May 27, 2026
diff --git a/.github/skills/ctf-testing/SKILL.md b/.github/skills/ctf-testing/SKILL.md
@@ -1,30 +1,32 @@
 ---
 name: ctf-testing
-description: Deploy and test Linux CTF challenges across cloud providers (AWS, Azure, GCP). Use when testing CTF setup, validating challenges work correctly, running the full test suite, verifying services survive VM reboots, or after creating new challenges.
+description: Deploy and test Linux CTF challenges across cloud providers (AWS, Azure, GCP). Use when running basic tests without reboot or full tests with reboot for CTF setup, challenge validation, and release confidence.
 ---
 
 # CTF Challenge Testing
 
-This skill deploys CTF infrastructure to cloud providers and validates all challenges work correctly.
-
-## Decision Tree: Which Provider?
-
-```
-What to test? → Testing new challenge locally first?
-├─ Yes → Use Azure (fastest deploy, ~3 min)
-│
-└─ No → Full validation before release?
-    ├─ Quick check → Pick one: aws | azure | gcp
-    └─ Full release validation → Use "all" + --with-reboot
-```
+This skill deploys CTF infrastructure to cloud providers and validates that learners can complete the lab.
 
 ## When to Use
 
-- Testing changes to `ctf_setup.sh`
+- Testing changes to `ctf_setup.sh`, `setup/`, or `verify/`
 - Validating challenge setup across AWS, Azure, or GCP
-- Running the full test suite before releases
+- Running the full contributor-mode test suite before releases
 - Verifying services survive VM reboots
 
+## Trigger Examples
+
+Use this skill when the user asks things like:
+
+- "Run a basic test on Azure"
+- "Run a basic test on GCP"
+- "Run a basic test on AWS"
+- "Run a full test on Azure"
+- "Run a full test on GCP"
+- "Run a full test on AWS"
+- "Run a basic test on all providers"
+- "Run a full test on all providers"
+
 ## Prerequisites
 
 1. **terraform** (>= 1.0)
@@ -35,107 +37,59 @@ What to test? → Testing new challenge locally first?
    - Azure: `az account show`
    - GCP: `gcloud auth list --filter=status:ACTIVE`
 
-## Running Tests
-
-The scripts are black boxes - run with `--help` or use these commands:
-
-```bash
-./.github/skills/ctf-testing/deploy_and_test.sh <provider> [--with-reboot]
-```
-
-| Command | Description |
-|---------|-------------|
-| `deploy_and_test.sh aws` | Test AWS only (~15 min) |
-| `deploy_and_test.sh azure` | Test Azure only (~15 min) |
-| `deploy_and_test.sh gcp` | Test GCP only (~15 min) |
-| `deploy_and_test.sh all` | Test all providers (~45 min) |
-| `deploy_and_test.sh aws --with-reboot` | Test with reboot verification (~20 min) |
-
-## What Gets Tested
-
-The test script simulates a real user journey:
-
-1. **Verify command sanity check** - Confirms `verify` command works
-2. **Challenge solving** - All 18 challenges discovered and solved using hints
-3. **Verification token** - Token generation and format validation
-4. **Export certificate** - Certificate generation with correct metadata
-
-**With `--with-reboot`:**
-5. **Service resilience** - All systemd services restart after reboot
-6. **Progress persistence** - Completed challenges survive reboot
-
-## Common Pitfalls
-
-❌ **Don't** run tests without checking cloud CLI authentication first
-✅ **Do** verify with `aws sts get-caller-identity` / `az account show` / `gcloud auth list`
-
-❌ **Don't** forget to check for leftover resources after a failed run
-✅ **Do** run the cleanup verification commands in "Post-Test Cleanup" section
-
-❌ **Don't** run `all` for quick iteration - it takes 45+ minutes
-✅ **Do** pick one provider (Azure is fastest) for development, `all` for releases
-
-## Features
-
-- **Timestamped logging** - All output includes `[HH:MM:SS]` timestamps
-- **Graceful interrupt handling** - Ctrl+C triggers cleanup of deployed infrastructure
-- **Proper VM wait logic** - Uses cloud-native waits instead of arbitrary sleeps
-- **IP validation** - Verifies retrieved IPs are valid before attempting SSH
+## Agent Workflow
 
-## Expected Results
-
-A successful run shows **~25 tests passing**, followed by a summary:
-- 1 verify sanity check
-- 18 challenge solutions
-- 4 verification token tests
-- 2 export certificate tests
-
-**With `--with-reboot`:** Additional 6 service checks + 1 progress persistence check.
-
-Summary line: `RESULT: PASS (<providers>)` or `RESULT: FAIL (<providers>)`
-
-## Troubleshooting
-
-### Setup not completing
-- Check `/var/log/setup_complete` exists on VM
-- Review cloud-init logs: `/var/log/cloud-init-output.log`
-
-### Service not running
-- Check status: `systemctl status <service-name>`
-- Check logs: `journalctl -u <service-name>`
-
-### Port not accessible externally
-- Verify firewall rules in Terraform allow the port
-- Check security group/NSG in cloud console
-
-### SSH connection fails
-- Wait longer for VM setup (~3-5 minutes after IP available)
-- Verify security group allows port 22
-
-## Post-Test Cleanup
-
-Always verify resources were destroyed to avoid unexpected charges:
-
-**AWS:**
-```bash
-aws ec2 describe-instances --filters "Name=tag:Name,Values=CTF*" "Name=instance-state-name,Values=running,pending,stopping,stopped" --query 'Reservations[*].Instances[*].[InstanceId,State.Name]' --output table
-aws ec2 describe-vpcs --filters "Name=tag:Name,Values=CTF*" --query 'Vpcs[*].VpcId' --output table
-```
-
-**Azure:**
-```bash
-az group list --query "[?starts_with(name, 'ctf')].name" --output table
-```
-
-**GCP:**
-```bash
-gcloud compute instances list --filter="name~'ctf'" --format="table(name,zone,status)"
-```
-
-If resources remain, manually destroy:
-```bash
-cd <provider> && terraform destroy -auto-approve
-```
+1. **Determine the mode and provider.**
+   - Basic test means no reboot. Use this when the user asks for a basic test or does not mention reboot.
+   - Full test means reboot. Use this when the user asks for a full test or asks to verify reboot behavior.
+   - Provider must be `aws`, `azure`, `gcp`, or `all`. Azure is usually the fastest single-provider test.
+2. **Check provider authentication before deploying.**
+   - AWS: `aws sts get-caller-identity`
+   - Azure: `az account show`
+   - GCP: `gcloud auth list --filter=status:ACTIVE`
+3. **Run the requested test from the repository root.**
+
+   | User request | Command | What gets tested |
+   | --- | --- | --- |
+   | Basic test on AWS | `./.github/skills/ctf-testing/deploy_and_test.sh aws` | Verify command sanity checks, all 18 challenges, export certificate, token format, username, challenge count, and frozen completion time |
+   | Basic test on Azure | `./.github/skills/ctf-testing/deploy_and_test.sh azure` | Verify command sanity checks, all 18 challenges, export certificate, token format, username, challenge count, and frozen completion time |
+   | Basic test on GCP | `./.github/skills/ctf-testing/deploy_and_test.sh gcp` | Verify command sanity checks, all 18 challenges, export certificate, token format, username, challenge count, and frozen completion time |
+   | Basic test on all providers | `./.github/skills/ctf-testing/deploy_and_test.sh all` | Same basic checks across AWS, Azure, and GCP |
+   | Full test on AWS | `./.github/skills/ctf-testing/deploy_and_test.sh aws --with-reboot` | Basic checks plus reboot, required service checks, and progress persistence |
+   | Full test on Azure | `./.github/skills/ctf-testing/deploy_and_test.sh azure --with-reboot` | Basic checks plus reboot, required service checks, and progress persistence |
+   | Full test on GCP | `./.github/skills/ctf-testing/deploy_and_test.sh gcp --with-reboot` | Basic checks plus reboot, required service checks, and progress persistence |
+   | Full test on all providers | `./.github/skills/ctf-testing/deploy_and_test.sh all --with-reboot` | Same full checks across AWS, Azure, and GCP |
+
+   The deployment script uses contributor mode, so Terraform uploads the local setup package and runs the code from your working tree. It does not test GitHub Release assets.
+4. **Treat cleanup as part of the test, not an optional follow-up.**
+   - Verify no test resources remain after the script exits.
+   - If resources remain, run provider-specific cleanup or `terraform destroy` from the provider directory.
+   - If cleanup still fails, report the remaining resources clearly.
+5. **Report the result plainly.**
+   - Include provider, mode, pass/fail result, cleanup status, and blocker if any.
+   - A successful basic run shows about 27 tests passing.
+   - A successful full run includes a second pass after reboot with 5 service checks and 1 progress persistence check.
+   - Summary line: `RESULT: PASS (<providers>)` or `RESULT: FAIL (<providers>)`.
+
+## Failure Handling
+
+When a run fails, identify the first failing layer:
+
+| Failure point | What to collect |
+| --- | --- |
+| Terraform failed | Provider name and Terraform error |
+| SSH failed | VM IP and SSH wait output |
+| Setup failed | `/var/lib/linux-ctfs/setup.failed`, `/var/log/ctf_setup.log`, and `/var/log/cloud-init-output.log` |
+| Challenge failed | Challenge number and failing test output |
+| Reboot failed | Failed service name and `journalctl -u <service-name>` output |
+| Cleanup failed | Remaining cloud resources |
+
+## CTF Safety Rules
+
+- Do not create committed solution files.
+- Keep solution commands only in `.github/skills/ctf-testing/test_ctf_challenges.sh`.
+- Do not add flags or challenge solutions to learner-facing docs.
+- Do not make challenges easier while fixing tests.
 
 ## Scripts
 

diff --git a/.github/skills/ctf-testing/deploy_and_test.sh b/.github/skills/ctf-testing/deploy_and_test.sh
@@ -353,6 +353,41 @@ _wait_for_ssh() {
     return 1
 }
 
+# Wait for setup markers before running challenge tests
+# Arguments:
+#   $1 - IP address of the VM
+# Returns:
+#   0 on success, 1 on timeout
+_wait_for_setup() {
+    local ip="$1"
+    local attempt=1
+
+    _log INFO "Waiting for CTF setup to finish..."
+
+    while [[ ${attempt} -le ${MAX_SSH_ATTEMPTS} ]]; do
+        # shellcheck disable=SC2086
+        if _sshpass_cmd ssh ${SSH_OPTS} "${SSH_USER}@${ip}" \
+            "test -f /var/lib/linux-ctfs/setup.done || test -f /var/lib/cloud/instance/ctf-setup.done || test -f /var/log/setup_complete" &>/dev/null; then
+            _log OK "CTF setup is complete"
+            return 0
+        fi
+
+        # shellcheck disable=SC2086
+        if _sshpass_cmd ssh ${SSH_OPTS} "${SSH_USER}@${ip}" \
+            "test -f /var/lib/linux-ctfs/setup.failed" &>/dev/null; then
+            _log ERROR "CTF setup reported failure. Check /var/log/ctf_setup.log on the VM."
+            return 1
+        fi
+
+        echo "  Attempt ${attempt}/${MAX_SSH_ATTEMPTS} - waiting..."
+        sleep "${SSH_RETRY_INTERVAL}"
+        ((attempt++))
+    done
+
+    _log ERROR "CTF setup timed out"
+    return 1
+}
+
 # Reboot/restart a VM and return the new IP address
 # Arguments:
 #   $1 - Cloud provider name (aws, azure, gcp)
@@ -538,6 +573,15 @@ _test_provider() {
         return 1
     fi
 
+    # Wait for setup markers
+    if ! _wait_for_setup "${ip}"; then
+        _log ERROR "Setup did not complete for ${provider}"
+        CLEANUP_ON_EXIT=false
+        _terraform_destroy "${provider}"
+        CURRENT_PROVIDER=""
+        return 1
+    fi
+
     # Run tests
     local test_exit_code=0
     _run_tests "${provider}" "${ip}" || test_exit_code=$?

diff --git a/.github/skills/ctf-testing/test_ctf_challenges.sh b/.github/skills/ctf-testing/test_ctf_challenges.sh
@@ -147,7 +147,7 @@ if [[ -f "${REBOOT_MARKER}" ]]; then
         EXPECTED=$(cat "$PROGRESS_SNAPSHOT")
         ACTUAL=$(sort -u /var/ctf/completed_challenges 2>/dev/null | wc -l)
         if [ "$ACTUAL" -ge "$EXPECTED" ]; then
-            pass "Progress persisted after reboot ($ACTUAL challenges)"
+            _pass "Progress persisted after reboot ($ACTUAL challenges)"
         else
             _fail "Progress lost after reboot (expected ${EXPECTED}, got ${ACTUAL})"
         fi
@@ -356,19 +356,20 @@ else
     FLAGS[8]=""
 fi
 
-# Challenge 9: DNS Configuration
-# Hint: "DNS settings are stored in /etc/resolv.conf"
-echo "Challenge 9: DNS Configuration"
-if [[ -r /etc/resolv.conf ]]; then
-    FLAG_9=$(grep -ao 'CTF{[^}]*}' /etc/resolv.conf 2>/dev/null | head -1) || true
+# Challenge 9: DNS Inspection
+# Hint: "Inspect systemd-resolved configuration safely"
+echo "Challenge 9: DNS Inspection"
+DNS_DROP_IN="/etc/systemd/resolved.conf.d/ctf-dns.conf"
+if [[ -r "${DNS_DROP_IN}" ]]; then
+    FLAG_9=$(grep -ao 'CTF{[^}]*}' "${DNS_DROP_IN}" 2>/dev/null | head -1) || true
     if [[ -n "${FLAG_9}" ]]; then
         _verify_flag 9 "${FLAG_9}" "Solved challenge 9" "Challenge 9: Found flag but verify rejected it - SETUP BUG"
     else
-        _fail "Challenge 9: resolv.conf has no CTF flag - SETUP BUG"
+        _fail "Challenge 9: DNS drop-in has no CTF flag - SETUP BUG"
         FLAGS[9]=""
     fi
 else
-    _fail "Challenge 9: /etc/resolv.conf not readable - SETUP BUG"
+    _fail "Challenge 9: DNS drop-in not readable - SETUP BUG"
     FLAGS[9]=""
 fi
 

diff --git a/.github/workflows/release-setup.yml b/.github/workflows/release-setup.yml
@@ -0,0 +1,56 @@
+name: Release setup package
+
+on:
+  release:
+    types: [published]
+  workflow_dispatch:
+    inputs:
+      tag:
+        description: "Release tag to upload assets to"
+        required: true
+
+permissions:
+  contents: write
+
+jobs:
+  package-setup:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Resolve release tag
+        id: release
+        run: |
+          if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
+            echo "tag=${{ inputs.tag }}" >> "$GITHUB_OUTPUT"
+          else
+            echo "tag=${{ github.event.release.tag_name }}" >> "$GITHUB_OUTPUT"
+          fi
+
+      - name: Check out release code
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ steps.release.outputs.tag }}
+
+      - name: Build deterministic setup archive
+        run: |
+          set -euo pipefail
+          asset="linux-ctfs-setup.tar.gz"
+          tar \
+            --sort=name \
+            --owner=0 \
+            --group=0 \
+            --numeric-owner \
+            --mtime="UTC 1970-01-01" \
+            -cf - ctf_setup.sh setup verify | gzip -n > "$asset"
+          sha256sum "$asset" > "$asset.sha256"
+
+      - name: Upload release assets
+        env:
+          GH_TOKEN: ${{ github.token }}
+          TAG: ${{ steps.release.outputs.tag }}
+        run: |
+          set -euo pipefail
+          gh release upload "$TAG" \
+            "linux-ctfs-setup.tar.gz" \
+            "linux-ctfs-setup.tar.gz.sha256" \
+            --clobber
diff --git a/.gitignore b/.gitignore
@@ -7,6 +7,11 @@
 *.swp
 *.swo
 
+# Python
+__pycache__/
+*.py[cod]
+.venv/
+
 # Terraform
 *.tfstate
 *.tfstate.*