Add MiniStack CORS proxy and AWS Glue auto-log-group setup

README.md

@@ -100,17 +100,39 @@ When you access the **hosted app** at [app.openarchflow.cloud](https://app.opena
 
 **Getting started:**
 
-**Option 1: OpenArchFlow running locally** (`localhost:3000`):
+**Option 1: OpenArchFlow running locally** (`localhost:3000`) — Basic MiniStack:
 ```bash
 docker run -p 4566:4566 ministackorg/ministack
 ```
 Then open `localhost:3000`, configure MiniStack endpoint as `http://localhost:4566`, and deploy.
 
-**Option 2: Using the hosted app** ([app.openarchflow.cloud](https://app.openarchflow.cloud)) with local MiniStack:
+**Option 2: OpenArchFlow running locally** (`localhost:3000`) — With Glue support and persistent S3:
+```bash
+docker run -d -p 4566:4566 \
+  -v /var/run/docker.sock:/var/run/docker.sock \
+  -e GLUE_DOCKER_IMAGE=ghcr.io/dmux/openarchflow/ministack_glue_libs_4.0.0_image_01:latest \
+  -e S3_PERSIST=1 \
+  ministackorg/ministack:full
+```
+Or use Docker Compose:
+```bash
+docker compose -f docker-compose.ministack-aws-glue-full.yml up -d
+```
+The compose file automatically pulls the required Glue image before starting MiniStack.
+
+**Option 3: Using the hosted app** ([app.openarchflow.cloud](https://app.openarchflow.cloud)) with local MiniStack and CORS proxy:
+
+*Basic setup (recommended for simple testing):*
 ```bash
 docker compose -f docker-compose.ministack.yml up -d
 ```
-This starts MiniStack **and** a local nginx sidecar that adds CORS headers, listening on `http://localhost:4567`. Then:
+
+*With Glue support (for AWS Glue testing):*
+```bash
+docker compose -f docker-compose.ministack-aws-glue-full.yml up -d
+```
+
+Both options start MiniStack **and** a local nginx sidecar that adds CORS headers, listening on `http://localhost:4567`. Then:
 1. Open [app.openarchflow.cloud](https://app.openarchflow.cloud)
 2. Click the **Rocket** icon → **Configure MiniStack**
 3. Set **Endpoint URL** to `http://localhost:4567` (note: port `4567`, not `4566`)
@@ -164,6 +186,26 @@ The proxy (port 4567) forwards all requests to MiniStack (port 4566) and adds `A
 
 **Why is this needed?** When you access the hosted app (`app.openarchflow.cloud`) from your browser, the browser enforces CORS policy. Your browser can reach `localhost:4566` on your machine, but only if the response includes appropriate CORS headers. The proxy adds these headers, enabling secure cross-origin access.
 
+#### Troubleshooting: AWS Glue image not loading
+
+**Error:** `[glue] Glue: image ghcr.io/dmux/openarchflow/ministack_glue_libs_4.0.0_image_01:latest not available — stubbing job`
+
+**Solution:** The docker-compose file automatically handles image pulling:
+
+```bash
+docker compose -f docker-compose.ministack-aws-glue-full.yml up -d
+```
+
+The compose configuration:
+- Has a `glue-image-puller` service that pulls the Glue image before MiniStack starts
+- Uses `pull_policy: always` to ensure the latest image is used
+- MiniStack waits for image pull to complete before starting
+
+If you still see the error, verify:
+1. Docker daemon is running
+2. You have internet access to ghcr.io (GitHub Container Registry)
+3. Try manually: `docker pull ghcr.io/dmux/openarchflow/ministack_glue_libs_4.0.0_image_01:latest`
+
 ### 🏗️ Terraform IaC Generation (NEW)
 
 - **HCL Code Generation**: Export your AWS architecture diagram directly as production-ready HashiCorp Terraform — `main.tf`, `variables.tf`, and `outputs.tf` generated from your nodes and edges.
@@ -322,9 +364,10 @@ pnpm start
 ### 6. Deploy to Local AWS (MiniStack)
 
 1. Start MiniStack (see **MiniStack Local Deploy** section above for setup options)
-   - Basic: `docker run -p 4566:4566 ministackorg/ministack`
-   - With Glue support: `docker run -p 4566:4566 -v /var/run/docker.sock:/var/run/docker.sock -e GLUE_DOCKER_IMAGE=ghcr.io/dmux/openarchflow/ministack_glue_libs_4.0.0_image_01:latest -e S3_PERSIST=1 ministackorg/ministack:full`
-   - With CORS proxy (hosted app): `docker compose -f docker-compose.ministack.yml up -d`
+   - **Basic** (no Glue): `docker run -p 4566:4566 ministackorg/ministack`
+   - **With Glue & persistent S3** (Docker Compose): `docker compose -f docker-compose.ministack-aws-glue-full.yml up -d`
+   - **With CORS proxy** (for hosted app): `docker compose -f docker-compose.ministack.yml up -d`
+   - **With Glue + CORS proxy** (hosted app + Glue): `docker compose -f docker-compose.ministack-aws-glue-full.yml up -d` then use `http://localhost:4567`
 2. In the app, click the **🚀 Rocket** icon in the toolbar
 3. In the MiniStack Connection dialog:
    - Set **Endpoint URL** to:

docker-compose.ministack-aws-glue-full.yml

@@ -0,0 +1,33 @@
+version: "3.8"
+
+services:
+  glue-image-puller:
+    image: docker:latest
+    volumes:
+      - /var/run/docker.sock:/var/run/docker.sock
+    command: pull ghcr.io/dmux/openarchflow/ministack_glue_libs_4.0.0_image_01:latest
+    restart: "no"
+
+  ministack:
+    image: ministackorg/ministack:full
+    pull_policy: always
+    ports:
+      - "4566:4566"
+    volumes:
+      - /var/run/docker.sock:/var/run/docker.sock
+    environment:
+      GLUE_DOCKER_IMAGE: ghcr.io/dmux/openarchflow/ministack_glue_libs_4.0.0_image_01:latest
+      S3_PERSIST: "1"
+    depends_on:
+      glue-image-puller:
+        condition: service_completed_successfully
+    restart: unless-stopped
+
+  ministack-cors-proxy:
+    build:
+      context: ./docker/ministack-cors-proxy
+    ports:
+      - "4567:4566"
+    depends_on:
+      - ministack
+    restart: unless-stopped

docker-compose.ministack.yml

@@ -2,7 +2,7 @@ version: "3.8"
 
 services:
   ministack:
-    image: ministackorg/ministack
+    image: ministackorg/ministack:full
     ports:
       - "4566:4566"
     restart: unless-stopped

docker/ministack-cors-proxy/nginx.conf

@@ -33,7 +33,6 @@ server {
         # per the Fetch spec, so the browser sends an OPTIONS preflight first.
         # MiniStack never needs to see these — answer them here directly.
         if ($request_method = OPTIONS) {
-            add_header Access-Control-Allow-Origin "*" always;
             add_header Access-Control-Allow-Methods "GET,POST,PUT,PATCH,DELETE,HEAD,OPTIONS" always;
             # Reflect whatever headers the SDK says it's about to send, rather
             # than hard-coding an exhaustive SigV4 header allowlist.

package.json

@@ -25,53 +25,53 @@
     "signaling": "node scripts/signaling.mjs"
   },
   "dependencies": {
-    "@aws-sdk/client-api-gateway": "^3.1085.0",
-    "@aws-sdk/client-athena": "^3.1085.0",
-    "@aws-sdk/client-bedrock": "^3.1085.0",
-    "@aws-sdk/client-bedrock-runtime": "^3.1085.0",
-    "@aws-sdk/client-cloudfront": "^3.1085.0",
-    "@aws-sdk/client-cloudwatch-logs": "^3.1085.0",
-    "@aws-sdk/client-cognito-identity-provider": "^3.1085.0",
-    "@aws-sdk/client-dynamodb": "^3.1085.0",
-    "@aws-sdk/client-ec2": "^3.1085.0",
-    "@aws-sdk/client-ecs": "^3.1085.0",
-    "@aws-sdk/client-eks": "^3.1085.0",
-    "@aws-sdk/client-elastic-load-balancing-v2": "^3.1085.0",
-    "@aws-sdk/client-elasticache": "^3.1085.0",
-    "@aws-sdk/client-eventbridge": "^3.1085.0",
-    "@aws-sdk/client-glue": "^3.1085.0",
-    "@aws-sdk/client-iam": "^3.1085.0",
-    "@aws-sdk/client-kinesis": "^3.1085.0",
-    "@aws-sdk/client-kms": "^3.1085.0",
-    "@aws-sdk/client-lambda": "^3.1085.0",
-    "@aws-sdk/client-pricing": "^3.1085.0",
-    "@aws-sdk/client-rds": "^3.1085.0",
-    "@aws-sdk/client-route-53": "^3.1085.0",
-    "@aws-sdk/client-s3": "^3.1085.0",
-    "@aws-sdk/client-secrets-manager": "^3.1085.0",
-    "@aws-sdk/client-sfn": "^3.1085.0",
-    "@aws-sdk/client-sns": "^3.1085.0",
-    "@aws-sdk/client-sqs": "^3.1085.0",
-    "@aws-sdk/client-ssm": "^3.1085.0",
-    "@aws-sdk/client-sso": "^3.1085.0",
-    "@aws-sdk/client-sso-oidc": "^3.1085.0",
+    "@aws-sdk/client-api-gateway": "^3.1094.0",
+    "@aws-sdk/client-athena": "^3.1094.0",
+    "@aws-sdk/client-bedrock": "^3.1094.0",
+    "@aws-sdk/client-bedrock-runtime": "^3.1094.0",
+    "@aws-sdk/client-cloudfront": "^3.1094.0",
+    "@aws-sdk/client-cloudwatch-logs": "^3.1094.0",
+    "@aws-sdk/client-cognito-identity-provider": "^3.1094.0",
+    "@aws-sdk/client-dynamodb": "^3.1094.0",
+    "@aws-sdk/client-ec2": "^3.1094.0",
+    "@aws-sdk/client-ecs": "^3.1094.0",
+    "@aws-sdk/client-eks": "^3.1094.0",
+    "@aws-sdk/client-elastic-load-balancing-v2": "^3.1094.0",
+    "@aws-sdk/client-elasticache": "^3.1094.0",
+    "@aws-sdk/client-eventbridge": "^3.1094.0",
+    "@aws-sdk/client-glue": "^3.1094.0",
+    "@aws-sdk/client-iam": "^3.1094.0",
+    "@aws-sdk/client-kinesis": "^3.1094.0",
+    "@aws-sdk/client-kms": "^3.1094.0",
+    "@aws-sdk/client-lambda": "^3.1094.0",
+    "@aws-sdk/client-pricing": "^3.1094.0",
+    "@aws-sdk/client-rds": "^3.1094.0",
+    "@aws-sdk/client-route-53": "^3.1094.0",
+    "@aws-sdk/client-s3": "^3.1094.0",
+    "@aws-sdk/client-secrets-manager": "^3.1094.0",
+    "@aws-sdk/client-sfn": "^3.1094.0",
+    "@aws-sdk/client-sns": "^3.1094.0",
+    "@aws-sdk/client-sqs": "^3.1094.0",
+    "@aws-sdk/client-ssm": "^3.1094.0",
+    "@aws-sdk/client-sso": "^3.1094.0",
+    "@aws-sdk/client-sso-oidc": "^3.1094.0",
     "@ducanh2912/next-pwa": "^10.2.9",
     "@google/generative-ai": "^0.24.1",
     "@mlc-ai/web-llm": "^0.2.84",
     "@modelcontextprotocol/sdk": "^1.29.0",
     "@monaco-editor/react": "^4.7.0",
-    "@radix-ui/react-alert-dialog": "^1.1.19",
-    "@radix-ui/react-dialog": "^1.1.19",
-    "@radix-ui/react-dropdown-menu": "^2.1.20",
-    "@radix-ui/react-label": "^2.1.11",
-    "@radix-ui/react-popover": "^1.1.19",
-    "@radix-ui/react-progress": "^1.1.12",
-    "@radix-ui/react-scroll-area": "^1.2.14",
-    "@radix-ui/react-separator": "^1.1.11",
-    "@radix-ui/react-slider": "^1.4.3",
-    "@radix-ui/react-slot": "^1.3.0",
-    "@radix-ui/react-switch": "^1.3.3",
-    "@radix-ui/react-tooltip": "^1.2.12",
+    "@radix-ui/react-alert-dialog": "^1.1.21",
+    "@radix-ui/react-dialog": "^1.1.21",
+    "@radix-ui/react-dropdown-menu": "^2.1.22",
+    "@radix-ui/react-label": "^2.1.13",
+    "@radix-ui/react-popover": "^1.1.21",
+    "@radix-ui/react-progress": "^1.1.14",
+    "@radix-ui/react-scroll-area": "^1.2.16",
+    "@radix-ui/react-separator": "^1.1.13",
+    "@radix-ui/react-slider": "^1.4.5",
+    "@radix-ui/react-slot": "^1.3.1",
+    "@radix-ui/react-switch": "^1.3.5",
+    "@radix-ui/react-tooltip": "^1.2.14",
     "@upstash/ratelimit": "^2.0.8",
     "@upstash/redis": "^1.38.0",
     "@vercel/analytics": "^1.6.1",
@@ -88,10 +88,10 @@
     "html2canvas": "^1.4.1",
     "jspdf": "^4.2.1",
     "lucide-react": "^0.577.0",
-    "next": "^16.2.10",
+    "next": "^16.2.11",
     "next-themes": "^0.4.6",
-    "react": "^19.2.7",
-    "react-dom": "^19.2.7",
+    "react": "^19.2.8",
+    "react-dom": "^19.2.8",
     "react-icons": "^5.7.0",
     "react-markdown": "^10.1.0",
     "react-syntax-highlighter": "^16.1.1",
@@ -113,12 +113,12 @@
     "@types/react": "^19.2.17",
     "@types/react-dom": "^19.2.3",
     "@types/react-syntax-highlighter": "^15.5.13",
-    "@typescript-eslint/eslint-plugin": "^8.63.0",
-    "@typescript-eslint/parser": "^8.63.0",
-    "autoprefixer": "^10.5.2",
+    "@typescript-eslint/eslint-plugin": "^8.65.0",
+    "@typescript-eslint/parser": "^8.65.0",
+    "autoprefixer": "^10.5.4",
     "eslint": "^9.39.5",
-    "eslint-config-next": "^16.2.10",
-    "postcss": "^8.5.17",
+    "eslint-config-next": "^16.2.11",
+    "postcss": "^8.5.22",
     "tailwindcss": "^3.4.19",
     "typescript": "^5.9.3"
   }