fix(helm): correct chart docs, examples, and dead config across the board (#5907)

* fix(helm): correct chart docs, examples, and dead config across the board

Audit-driven accuracy pass over the chart's entire documentation surface,
verified by rendering every example against the templates:

- migrations run as an init container on the app pod, not a Job — fix the
  README component list, troubleshooting commands, and sim-helm skill refs;
  drop the dead migrations-job NetworkPolicy ingress rule
- referenced-but-never-created resources: document the GKE ManagedCertificate
  creation (values-gcp), comment out the key-file Secret mount that stuck all
  pods in ContainerCreating (values-gcp), enable certManager for the postgres
  TLS issuerRef (values-production), add the cert-manager cluster-issuer
  annotation nginx needs (values-azure)
- values-external-db: networkPolicy.egress is a list, not a map (the map
  rendered an invalid manifest); fill schema-failing placeholder host/username
- realtime >1 replica requires REDIS_URL (Socket.IO Redis adapter) — default
  examples to 1 replica with the scaling note, and warn where autoscaling
  HPAs override replicaCount
- pod anti-affinity selectors matched nothing (simstudio vs sim name label)
- kubernetes.mdx: install commands were missing required CRON_SECRET and
  postgresql password (failed at template time), wrong deployment name in
  port-forward, stale version requirements, unsupported key-remapping claim
- remove unimplemented app.secrets.existingSecret.keys from values + schema;
  fix README PDB default, cronjob list, /metrics caveat, NOTES secret count,
  Azure-only StorageClass in generic examples, dead SOCKET_SERVER_URL and
  GOOGLE_CLOUD_* env, ESO apiVersion mismatch, and skill-reference drift
- bump chart to 1.0.1

* fix(helm): review round 1 — scoped example egress, in-tab secret note, copilot Job wording

- external-db example egress scopes to a placeholder database CIDR instead of
  to: [] (which allowed every destination on 5432, defeating the isolation
  the example teaches)
- kubernetes.mdx cloud tabs state explicitly that they reuse the variables
  generated in the Installation block
- Copilot migrations really do run as a Helm-hook Job — restore Job wording
  there (only the app migrations are an init container)

* fix(helm): template-sweep fixes — telemetry validity, ESO rollout checksums, dead passwordKey knob

- telemetry: memory_limiter gets the required check_interval (collector
  failed startup validation whenever telemetry.enabled=true); the jaeger
  exporter was removed from collector-contrib in v0.86 — export to Jaeger
  via its native OTLP endpoint instead (otlp/jaeger, default port 4317)
- app/realtime rollout checksums now hash the ExternalSecret manifest too,
  mirroring the copilot pattern — with ESO enabled the inline Secret renders
  empty, so remoteRefs changes never rolled the pods
- remove the unimplemented existingSecret.passwordKey knob (values, schema,
  README, dead helpers): nothing consumed it, and a non-default value
  silently produced a DATABASE_URL with an unexpandable placeholder; secrets
  must use the standard POSTGRES_PASSWORD / EXTERNAL_DB_PASSWORD keys
- drop the orphaned sim.migrations.labels helper (its only consumer was the
  dead NetworkPolicy rule removed earlier)
- helm test pod image resolves through sim.image so global.imageRegistry
  mirroring applies; NetworkPolicy realtime-ingress comment reflects actual
  traffic direction; smoke unittest suite loads the newly referenced
  external-secret template

* chore(helm): bump chart to 1.1.0 with upgrade notes

Removing (inert) documented values keys and changing the rollout-checksum
inputs is a values-surface change — per SemVer chart conventions that is
more than a patch. Adds an Upgrading section documenting the one-time pod
roll, the removed no-op keys, and the Jaeger-over-OTLP change.

* fix(helm): review round — external-db NP opt-in with real-CIDR-first flow, prod Jaeger OTLP endpoint

- external-db example ships networkPolicy disabled so a verbatim install
  always reaches the database; the scoped egress rule stays as the documented
  opt-in (set your CIDR first, then enable)
- values-production still pointed telemetry.jaeger at the legacy 14250
  collector port — now Jaeger's OTLP gRPC endpoint to match the otlp/jaeger
  exporter

* feat(helm): autoscaling.realtime.enabled toggle so examples can scale the app without unsafe realtime replicas

Cursor correctly flagged that comment-level warnings didn't stop a verbatim
production/external-db install from running the realtime HPA at minReplicas 2
without REDIS_URL (silent cross-pod event loss). Adds an opt-out toggle
(default true — existing deployments unchanged): the realtime HPA renders only
when autoscaling.realtime.enabled, and the realtime Deployment keeps
spec.replicas under its control when the HPA is excluded. The three
autoscaling examples set it false with the Redis rationale; README and
upgrade notes document the toggle.

* fix(helm): review round — whitelabeled realtime HPA opt-out, external-db isolation on with required CIDR in install flow

- values-whitelabeled now actually sets autoscaling.realtime.enabled: false
  (the earlier batch aborted before reaching this file — Cursor caught it)
- external-db keeps networkPolicy enabled (no isolation regression); the DB
  egress CIDR is marked REQUIRED and wired into both documented install
  commands via --set, so the copy-paste flow sets the real subnet in the same
  breath as the DB host

* fix(helm): use a syntactically valid example CIDR in the external-db install commands

An unreplaced <YOUR_DB_CIDR> literal fails Kubernetes CIDR validation and
aborts the install; every sibling placeholder in the same command (host,
username) installs fine and simply doesn't connect until replaced. The CIDR
now behaves the same way: valid example value (10.20.0.0/24), explicitly
marked as the operator's database subnet in both commands and the values
comment.

* fix(helm): remove text after continuation backslashes in external-db install commands

A trailing comment after the line-continuation backslash (and equally a
comment line spliced mid-command) breaks the shell command when copied —
the remaining --set overrides run as separate commands and required-value
validation fails. Both documented commands now reconstruct to bash-clean
multi-line invocations (verified with bash -n); the CIDR guidance lives in
the networkPolicy section comment.

* fix(docs): cloud-tab installs are alternatives via helm upgrade --install

Following Installation and then a cloud tab ran helm install twice for the
same release and failed on the second. The tabs now state they replace the
generic install and use helm upgrade --install, which is idempotent and also
converts an existing generic install to the cloud values.

* fix(docs): honest conversion caveat for cloud-tab upgrades over an existing install

The cloud values rename the bundled Postgres database to simstudio, which
Postgres only applies at first initialization — an in-place conversion of a
generic install would point DATABASE_URL at a nonexistent database. Document
the two safe paths: keep the original name via --set, or uninstall + delete
PVCs and install fresh.

* fix(helm): external-db header install command declares its secrets and includes CRON_SECRET

The primary documented command failed the chart's required-value validation
(missing app.env.CRON_SECRET with cronjobs default-on) and referenced an
undeclared DB_PASSWORD. It now shows the export lines for every variable it
uses and sets CRON_SECRET; both commands verified with bash -n.

* chore(helm): restore values.schema.json formatting — surgical deletions only

The earlier programmatic edit reformatted the whole file (~390 lines of
whitespace churn hiding the 12 real deleted lines). Re-applied the removal
of the dead keys/passwordKey properties as text-level deletions preserving
the original style.

* fix(helm+docs): explicit secret exports in external-db header; conversion must reuse original secrets

- all five export lines are written out (three were only named in a trailing
  comment, so a verbatim copy passed empty required values)
- the cloud-conversion caveat now leads with reusing the original secret
  values (helm get values) — a regenerated ENCRYPTION_KEY makes previously
  encrypted credentials undecryptable

Author: waleedlatif1

apps/docs/content/docs/en/platform/self-hosting/kubernetes.mdx

@@ -9,8 +9,8 @@ import { FAQ } from '@/components/ui/faq'
 
 ## Prerequisites
 
-- Kubernetes 1.19+
-- Helm 3.0+
+- Kubernetes 1.25+
+- Helm 3.8+
 - PV provisioner support
 
 ## Installation
@@ -23,47 +23,59 @@ git clone https://github.com/simstudioai/sim.git && cd sim
 BETTER_AUTH_SECRET=$(openssl rand -hex 32)
 ENCRYPTION_KEY=$(openssl rand -hex 32)
 INTERNAL_API_SECRET=$(openssl rand -hex 32)
+CRON_SECRET=$(openssl rand -hex 32)
+POSTGRES_PASSWORD=$(openssl rand -hex 24)
 
 # Install
 helm install sim ./helm/sim \
   --set app.env.BETTER_AUTH_SECRET="$BETTER_AUTH_SECRET" \
   --set app.env.ENCRYPTION_KEY="$ENCRYPTION_KEY" \
   --set app.env.INTERNAL_API_SECRET="$INTERNAL_API_SECRET" \
+  --set app.env.CRON_SECRET="$CRON_SECRET" \
+  --set postgresql.auth.password="$POSTGRES_PASSWORD" \
   --namespace simstudio --create-namespace
 ```
 
 ## Cloud-Specific Values
 
+These are cloud-tuned **alternatives** to the generic install above — pick one path, don't run both. The commands reuse the `$BETTER_AUTH_SECRET`, `$ENCRYPTION_KEY`, `$INTERNAL_API_SECRET`, `$CRON_SECRET`, and `$POSTGRES_PASSWORD` variables generated in [Installation](#installation) above, so run that block's `openssl` lines first in the same shell. They use `helm upgrade --install`, so they work whether or not a release exists yet. Two caveats when converting an existing generic install rather than starting fresh: (1) **reuse the original secret values** — recover them with `helm get values sim -n simstudio` if your shell no longer has them; supplying a newly generated `ENCRYPTION_KEY` makes every previously encrypted credential (OAuth tokens, provider keys, environment variables) undecryptable. (2) The cloud values rename the bundled PostgreSQL database to `simstudio`, but Postgres only applies that setting on first initialization — add `--set postgresql.auth.database=sim` to keep your existing database. If you'd rather start clean, `helm uninstall sim -n simstudio`, delete its PVCs, and run the cloud command fresh.
+
 <Tabs items={['AWS EKS', 'Azure AKS', 'GCP GKE']}>
   <Tab value="AWS EKS">
 ```bash
-helm install sim ./helm/sim \
+helm upgrade --install sim ./helm/sim \
   --values ./helm/sim/examples/values-aws.yaml \
   --set app.env.BETTER_AUTH_SECRET="$BETTER_AUTH_SECRET" \
   --set app.env.ENCRYPTION_KEY="$ENCRYPTION_KEY" \
   --set app.env.INTERNAL_API_SECRET="$INTERNAL_API_SECRET" \
+  --set app.env.CRON_SECRET="$CRON_SECRET" \
+  --set postgresql.auth.password="$POSTGRES_PASSWORD" \
   --set app.env.NEXT_PUBLIC_APP_URL="https://sim.yourdomain.com" \
   --namespace simstudio --create-namespace
 ```
   </Tab>
   <Tab value="Azure AKS">
 ```bash
-helm install sim ./helm/sim \
+helm upgrade --install sim ./helm/sim \
   --values ./helm/sim/examples/values-azure.yaml \
   --set app.env.BETTER_AUTH_SECRET="$BETTER_AUTH_SECRET" \
   --set app.env.ENCRYPTION_KEY="$ENCRYPTION_KEY" \
   --set app.env.INTERNAL_API_SECRET="$INTERNAL_API_SECRET" \
+  --set app.env.CRON_SECRET="$CRON_SECRET" \
+  --set postgresql.auth.password="$POSTGRES_PASSWORD" \
   --set app.env.NEXT_PUBLIC_APP_URL="https://sim.yourdomain.com" \
   --namespace simstudio --create-namespace
 ```
   </Tab>
   <Tab value="GCP GKE">
 ```bash
-helm install sim ./helm/sim \
+helm upgrade --install sim ./helm/sim \
   --values ./helm/sim/examples/values-gcp.yaml \
   --set app.env.BETTER_AUTH_SECRET="$BETTER_AUTH_SECRET" \
   --set app.env.ENCRYPTION_KEY="$ENCRYPTION_KEY" \
   --set app.env.INTERNAL_API_SECRET="$INTERNAL_API_SECRET" \
+  --set app.env.CRON_SECRET="$CRON_SECRET" \
+  --set postgresql.auth.password="$POSTGRES_PASSWORD" \
   --set app.env.NEXT_PUBLIC_APP_URL="https://sim.yourdomain.com" \
   --namespace simstudio --create-namespace
 ```
@@ -115,7 +127,7 @@ externalDatabase:
 
 ```bash
 # Port forward for local access
-kubectl port-forward deployment/sim-sim-app 3000:3000 -n simstudio
+kubectl port-forward deployment/sim-app 3000:3000 -n simstudio
 
 # View logs
 kubectl logs -l app.kubernetes.io/component=app -n simstudio --tail=100
@@ -130,7 +142,7 @@ helm uninstall sim --namespace simstudio
 <FAQ items={[
   { question: "What are the default resource limits for the Sim app pod?", answer: "The Helm chart defaults to 8 Gi memory limit / 4 Gi request and 2000m CPU limit / 1000m request per app pod. The pod runs as non-root (UID 1001) with fsGroup 1001 for security." },
   { question: "Can I use an external database instead of the bundled PostgreSQL?", answer: "Yes. Set postgresql.enabled to false and configure the externalDatabase section with your host, port, username, password, database name, and sslMode. The external database must have the pgvector extension installed." },
-  { question: "How do I manage secrets securely in Kubernetes?", answer: "The Helm chart supports pre-existing Kubernetes secrets via app.secrets.existingSecret. Set enabled to true and provide the secret name. This integrates with External Secrets Operator, HashiCorp Vault, Azure Key Vault, and similar tools. Key mappings can be customized if your secret uses different key names." },
+  { question: "How do I manage secrets securely in Kubernetes?", answer: "The Helm chart supports pre-existing Kubernetes secrets via app.secrets.existingSecret. Set enabled to true and provide the secret name. This integrates with External Secrets Operator, HashiCorp Vault, Azure Key Vault, and similar tools. The secret must use the standard key names (BETTER_AUTH_SECRET, ENCRYPTION_KEY, INTERNAL_API_SECRET, CRON_SECRET, ...) — it is consumed wholesale, so key remapping is not supported." },
   { question: "Can I scale the app to multiple replicas?", answer: "Yes. Set app.replicaCount in your values.yaml. The chart supports standard Kubernetes scaling. Ensure your database can handle the additional connections from multiple replicas." },
   { question: "Are there cloud-specific example configurations?", answer: "Yes. The Helm chart includes example values files for AWS EKS (values-aws.yaml), Azure AKS (values-azure.yaml), GCP GKE (values-gcp.yaml), as well as files for production, development, external databases, external secrets, and whitelabeled deployments." },
   { question: "What ingress options are supported?", answer: "The chart supports ingress with configurable className (e.g., nginx), TLS termination, and separate host configuration for the app. Enable it with ingress.enabled: true and set your host under ingress.app.host." },

helm/sim/.claude/skills/sim-helm/SKILL.md

@@ -121,7 +121,7 @@ These are non-negotiable. Violating any of these has burned users in the past.
 2. **Never set `image.tag: latest`.** The chart defaults to `Chart.AppVersion` for a reason — reproducible rollouts. If the user pinned `latest`, push back.
 3. **Never edit chart templates to work around a `fail` statement.** The validation exists because a misconfiguration would otherwise surface as a runtime CrashLoopBackOff with cryptic env errors.
 4. **Never drop `automountServiceAccountToken: false`** unless the workload genuinely needs in-cluster API access (Sim's app/realtime/postgres pods do not).
-5. **Never `kubectl delete sts` without `--cascade=orphan`** on a live Postgres. It deletes the pods and PVCs.
+5. **Never `kubectl delete sts` without `--cascade=orphan`** on a live Postgres. It deletes the pods (PVCs survive, but the database goes down immediately).
 6. **Never tell a user "the chart works on your cluster" without `helm lint` + `helm template` against their values.** Static reading is not validation.
 7. **Always confirm before `helm uninstall` in a shared namespace.** PVCs survive but other namespace resources may not.
 
@@ -136,7 +136,7 @@ kubectl --namespace <ns> get pods,events --sort-by='.lastTimestamp'
 kubectl --namespace <ns> logs deploy/sim-app --tail=200
 kubectl --namespace <ns> logs deploy/sim-realtime --tail=200
 kubectl --namespace <ns> logs sts/sim-postgresql --tail=200
-kubectl --namespace <ns> logs job/sim-migrations --tail=200 2>/dev/null
+kubectl --namespace <ns> logs deploy/sim-app -c migrations --tail=200 2>/dev/null
 kubectl --namespace <ns> describe pod -l app.kubernetes.io/name=sim
 ```
 

helm/sim/.claude/skills/sim-helm/references/secrets.md

@@ -24,7 +24,7 @@ export POSTGRES_PASSWORD=$(openssl rand -base64 24 | tr -d '/+=')  # if using ch
 | `INTERNAL_API_SECRET` | Shared auth between `sim-app` ↔ `sim-realtime` pods | 32 bytes = 64 hex chars | Both deployments must roll together — temporary realtime errors during the rollout |
 | `CRON_SECRET` | Authenticates scheduled CronJob pods to the app | 32 bytes = 64 hex chars | Rotating just needs `helm upgrade`; next cron run uses the new value |
 | `API_ENCRYPTION_KEY` (optional) | Encrypts user-stored API keys (OpenAI tokens, etc.) at rest in Postgres | **Exactly 64 hex chars** (the app rejects other lengths) | Without it, keys are stored plain. Once set, never rotate without a migration |
-| `POSTGRES_PASSWORD` (chart-bundled Postgres only) | Postgres superuser password | Any length ≥ 12 chars matching `^[a-zA-Z0-9._-]+$` | Requires Postgres pod restart + app rollout |
+| `POSTGRES_PASSWORD` (chart-bundled Postgres only) | Postgres superuser password | ≥ 8 chars (schema-enforced) matching `^[a-zA-Z0-9._-]+$`; 32-byte hex recommended | Requires Postgres pod restart + app rollout |
 
 The `^[a-zA-Z0-9._-]+$` constraint on the Postgres password exists because the chart embeds the password into `DATABASE_URL` without URL-encoding. The `tr -d '/+='` strips the three problematic characters from `openssl rand -base64` output. The chart enforces this regex at template time.
 

helm/sim/.claude/skills/sim-helm/references/troubleshooting.md

@@ -11,7 +11,7 @@ kubectl --namespace $NS describe pod -l app.kubernetes.io/instance=sim
 kubectl --namespace $NS logs deploy/sim-app --tail=200
 kubectl --namespace $NS logs deploy/sim-realtime --tail=200
 kubectl --namespace $NS logs sts/sim-postgresql --tail=200
-kubectl --namespace $NS logs job/sim-migrations --tail=200 2>/dev/null || true
+kubectl --namespace $NS logs deploy/sim-app -c migrations --tail=200 2>/dev/null || true
 ```
 
 ---
@@ -63,9 +63,9 @@ Match the error:
 |---|---|---|
 | `Invalid env: ... NEXT_PUBLIC_APP_URL: Invalid url` | URL field set to empty string or invalid format | Set `app.env.NEXT_PUBLIC_APP_URL` to a valid URL — `https://sim.example.com` in prod, `http://localhost:3000` in dev |
 | `getaddrinfo ENOTFOUND ... -postgresql` / `connect ECONNREFUSED` | App can't reach Postgres | Check `kubectl get pod -l app.kubernetes.io/name=postgresql` is `Running`; check `postgresql.auth.password` matches the password in the Secret |
-| `password authentication failed for user "sim"` | Postgres password rotated but app pod wasn't restarted, OR password contains URL-unsafe chars | `kubectl rollout restart deploy/sim-app -n sim`; regenerate password with `openssl rand -base64 24 \| tr -d '/+='` |
+| `password authentication failed for user "postgres"` | Postgres password rotated but app pod wasn't restarted, OR password contains URL-unsafe chars | `kubectl rollout restart deploy/sim-app -n sim`; regenerate password with `openssl rand -base64 24 \| tr -d '/+='` |
 | `BETTER_AUTH_SECRET is missing` / `INTERNAL_API_SECRET is required` | Required env var not present in the Secret | Verify with `kubectl get secret sim-app-secrets -o jsonpath='{.data}' \| jq 'keys'`; if missing, fix your secret strategy |
-| `Migration failed` or app starts before migration | Migration Job hasn't completed | `kubectl logs job/sim-migrations -n sim`; rerun with `kubectl delete job/sim-migrations && helm upgrade ...` |
+| `Migration failed` | Migrations init container on the app pod failed | `kubectl logs deploy/sim-app -c migrations -n sim`; rerun with `kubectl rollout restart deploy/sim-app -n sim` (migrations run before each app pod starts, so the app can never start ahead of them) |
 
 ---
 
@@ -112,7 +112,7 @@ kubectl describe ingress -n sim
 | No `ADDRESS` in `kubectl get ingress` | Ingress controller not installed — install `ingress-nginx`, AWS LBC, GCP LB controller, etc. |
 | `ingressClassName` doesn't match installed controller | `kubectl get ingressclass` to list installed classes, set `ingress.className` to match |
 | Address is set but DNS resolves to wrong IP | `dig <your-host>` — point DNS at the ingress controller's external IP / LoadBalancer / CNAME |
-| TLS cert errors | If using cert-manager, check `kubectl describe certificate -n sim`; verify `ingress.tls.issuerRef` |
+| TLS cert errors | If using cert-manager, check `kubectl describe certificate -n sim`; verify the `cert-manager.io/cluster-issuer` annotation on the ingress |
 | `503 Service Unavailable` | Ingress routing is fine but app pod isn't `Ready` — go back to the diagnostic block |
 
 ---
@@ -153,7 +153,7 @@ Bump the relevant resource limit. Defaults:
 |---|---|---|
 | `app` | `1000m` CPU / `4Gi` memory | `2000m` CPU / `8Gi` memory |
 | `realtime` | `250m` CPU / `512Mi` memory | `500m` CPU / `1Gi` memory |
-| `postgresql` | `250m` CPU / `512Mi` memory | `1000m` CPU / `2Gi` memory |
+| `postgresql` | `500m` CPU / `1Gi` memory | `2Gi` memory (no CPU limit) |
 
 Override in values:
 
@@ -172,7 +172,7 @@ app:
 
 ```bash
 helm version
-kubectl version --short
+kubectl version
 helm get values sim -n sim --revision $(helm history sim -n sim | tail -1 | awk '{print $1}')
 kubectl get all,pvc,ingress,externalsecret -n sim -o wide
 kubectl describe pods -n sim -l app.kubernetes.io/instance=sim | head -200

helm/sim/.claude/skills/sim-helm/references/values-model.md

@@ -27,7 +27,7 @@ The Sim chart splits configuration across **four** layers. Understanding which l
                                   ▼
 ┌───────────────────────────────────────────────────────────────────────────┐
 │ Layer 3: chart-computed (inline env: on the Deployment)                    │
-│   → DATABASE_URL, SOCKET_SERVER_URL, OLLAMA_URL                           │
+│   → DATABASE_URL, SOCKET_SERVER_URL, OLLAMA_URL, PII_URL                           │
 │   → Derived from postgresql.* / externalDatabase.* / service.* values     │
 │   → CANNOT be overridden via app.env — chart filters them out             │
 └───────────────────────────────────────────────────────────────────────────┘
@@ -63,7 +63,7 @@ The exhaustive list of keys per layer lives in `helm/sim/values.yaml`. Read the
 | Rate limits | 2 (envDefaults) | `RATE_LIMIT_WINDOW_MS`, `RATE_LIMIT_FREE_SYNC`, etc. |
 | Execution timeouts | 2 (envDefaults) | `EXECUTION_TIMEOUT_FREE`, `EXECUTION_TIMEOUT_PRO`, etc. |
 | IVM pool / quotas | 2 (envDefaults) | `IVM_POOL_SIZE`, `IVM_MAX_CONCURRENT`, `IVM_MAX_PER_WORKER`, etc. |
-| Connection strings | 3 (chart-computed) | `DATABASE_URL`, `SOCKET_SERVER_URL`, `OLLAMA_URL` |
+| Connection strings | 4 (chart-computed) | `DATABASE_URL`, `SOCKET_SERVER_URL`, `OLLAMA_URL`, `PII_URL` |
 | Custom downward API / configMapKeyRef | 4 (extraEnvVars) | anything that needs `valueFrom:` |
 
 ## Common authoring patterns

helm/sim/Chart.yaml

@@ -2,7 +2,7 @@ apiVersion: v2
 name: sim
 description: A Helm chart for Sim - the open-source AI workspace where teams build, deploy, and manage AI agents
 type: application
-version: 1.0.0
+version: 1.1.0
 appVersion: "0.6.73"
 kubeVersion: ">=1.25.0-0"
 home: https://sim.ai