Skip to content

AGENTS.md

Latest commit

 

History

History
292 lines (232 loc) · 16.3 KB

AGENTS.md

File metadata and controls

292 lines (232 loc) · 16.3 KB
Raw

Repository Agent Instructions

This file provides guidance to coding agents working with this repository.

This repository holds OpenShift cluster manifests, component build manifests and CI workflow configuration for OpenShift component repositories for both OKD and OCP.

Documentation

Additional documentation and references for working with this repository:

  • OpenShift CI Documentation - Main documentation site for OpenShift CI infrastructure
  • CI Operator Reference - Comprehensive reference for CI operator configuration specification and step registry
  • README.md - Repository overview and contribution entry points
  • CONTRIBUTING.md - Contribution requirements
  • clusters/README.md - CI cluster manifest conventions
  • docs/dptp-triage-sop/ - Operational triage procedures

Repository Structure

  • ci-operator/config/ - CI configuration files defining builds and tests for component repositories
  • ci-operator/jobs/ - Generated Prow job configurations (auto-generated, rarely edited manually)
  • ci-operator/step-registry/ - Reusable test steps, chains, and workflows for multi-stage jobs
  • ci-operator/templates/ - Legacy black-box test workflows (deprecated, use step-registry instead)
  • clusters/ - Active manifests for CI control planes, build clusters, hosted management clusters, and their GitOps configuration
  • core-services/ - Shared and legacy service configuration, including Prow, release-controller, cluster pools, and secret management
  • services/ - Legacy additional-service area; verify the current deployment path before adding content
  • hack/ - Repository automation, generators, validators, bootstrap tools, and operational scripts
  • docs/ - Operational runbooks and design or implementation plans
  • projects/ - Experimental, legacy, or non-critical service manifests
  • tools/ - Container image build manifests for tooling
  • .agents/skills/ - Codex-compatible repository skills
  • .claude/ - Claude Code commands, rules, skills, and settings

Agent Commands and Skills

This repository includes harness integrations for common workflows:

/step-finder - Component Discovery

Search the step-registry (4,400+ reusable CI components) to find existing steps, workflows, and chains before creating new ones:

/step-finder aws upgrade workflow          # Find AWS upgrade workflows
/step-finder install operator step yes     # Find operator installation steps with usage examples

Always use /step-finder before creating new step-registry components to avoid duplication.

/test-repo-files - Repo File URL Validation

Validate that core-services/release-controller/_repos/ocp-*.repo files have correct mirror2 and CDN URLs after editing:

/test-repo-files X.Y               # Test all repo files for a release; replace X.Y
/test-repo-files ocp-X.Y-rhel9     # Test a specific repo file
/test-repo-files                   # Test all modified repo files vs main

See .claude/SLASH_COMMANDS.md for detailed documentation of the Claude Code commands. Codex-compatible skills live in .agents/skills/.

Common Development Commands

Configuration Updates

After modifying CI configuration files, regenerate downstream artifacts:

make update

This runs: make jobs, make ci-operator-config, make prow-config, make registry-metadata, make release-controllers, make boskos-config

CI Job Configuration

When modifying CI jobs in ci-operator/config/:

make update

This validates config, generates Prow job configs, and sanitizes job definitions.

Validation

make check              # Run repository-wide service and configuration checks
make check-core         # Validate core-services
make check-services     # Validate services
make checkconfig        # Validate Prow configuration

New Repository Setup

make new-repo

Interactive tool to configure CI workflow for a new component repository. Automatically runs make update afterwards.

Step Registry Validation

make validate-step-registry

Validates step registry definitions for correctness.

Release Controllers

make release-controllers

Regenerates release controller configurations.

Repository Automation (hack/)

The hack/ directory contains shared implementation used by Make targets and CI jobs, as well as targeted operational utilities.

  • Prefer an existing Make target or hack/ utility over reimplementing the same validation or generation logic.
  • Inspect a script before running it. Some scripts mutate cluster or external state and require credentials or an explicit target.
  • Keep generated-file changes coupled with their source changes and invoke the narrowest applicable generator or validation target.
  • Common examples include repository validation (hack/validate-*.sh and hack/validate-*.py), release-controller generation (hack/generators/release-controllers/), secret/bootstrap tooling, and Prow job utilities.

Cluster Manifests and GitOps

  • clusters/app.ci/ contains manifests for the Prow control-plane cluster.
  • clusters/core-ci/ contains manifests for the core CI cluster.
  • clusters/build-clusters/ contains per-cluster and shared build-farm configuration.
  • clusters/hosted-mgmt/ and clusters/hosted-mgmt2/ contain hosted-management cluster configuration.
  • clusters/gitops/apps/ defines Argo CD ApplicationSets that select and apply many of these directories. Changes under selected paths may deploy automatically after merge.
  • Before editing cluster manifests, read the nearest README.md and OWNERS, then inspect the matching ApplicationSet to understand the target cluster, sync, pruning, and shared-directory behavior.

Architecture

CI Operator Configuration Flow

  1. Developers edit YAML files in ci-operator/config/<org>/<repo>/
  2. Each config file defines:
    • build_root: Base image for builds
    • images: Container images to build
    • tests: Test definitions (can reference step-registry workflows)
    • promotion: Where to push successful builds
    • releases: OpenShift releases to test against
  3. Running make update generates Prow job configs in ci-operator/jobs/
  4. Prow triggers jobs based on PR/merge/periodic events
  5. Jobs execute ci-operator with the config, which orchestrates builds and tests

Step Registry System

The step-registry contains reusable CI components:

  • Steps (.yaml + -commands.sh): Atomic tasks (e.g., openshift-e2e-test)
  • Chains (-chain.yaml): Ordered sequences of steps
  • Workflows (-workflow.yaml): Complete test scenarios with pre/test/post phases
  • References (-ref.yaml): Step definitions with metadata

Steps are referenced in ci-operator/config/ test definitions using multi-stage syntax. The registry allows sharing common operations (cluster provisioning, testing, cleanup) across many repositories.

Configuration Generation Pipeline

  1. ci-operator/config/ is the source of truth (manually edited)
  2. ci-operator-prowgen generates ci-operator/jobs/ from config files
  3. determinize-ci-operator normalizes config file formatting
  4. determinize-prow-config normalizes Prow configuration
  5. All generation uses containerized tools (pulled from quay.io/openshift/ci-public)

File Naming Conventions

  • ci-operator/config/: <org>-<repo>-<branch>.yaml
  • ci-operator/config/: <org>-<repo>-<branch>__periodics.yaml (variant periodic config)
  • ci-operator/jobs/: <org>-<repo>-<org>-<repo>-<branch>-<jobtype>.yaml
  • step-registry/: Component name prefixed (e.g., openshift-e2e-test-ref.yaml)
  • core-services/: admin_*.yaml for admin resources, _*.yaml excluded from application

Variant Periodic Configuration Pattern

Periodic tests can be separated from main configuration into dedicated __periodics.yaml files:

  • Maintains release specific periodic configuration that is consumable by CI analytical tooling
  • Example: openshift-cluster-authentication-operator-release-4.21__periodics.yaml
  • Contains only periodic tests with interval: or cron: scheduling
  • zz_generated_metadata contains data extracted from the __periodics.yaml configuration during make update
  • Generated jobs go to separate -periodics.yaml files in ci-operator/jobs/

OpenShift Release Periodic Jobs (Nightly & CI)

The main OpenShift periodic jobs that test nightly and CI payload streams are defined in ci-operator/config/openshift/release/. These are the top-level e2e and upgrade jobs that gate release payloads, distinct from per-component periodic jobs.

Files follow the naming convention openshift-release-main__<stream>-<version>.yaml:

  • Nightly stream: openshift-release-main__nightly-<version>.yaml — e2e tests against nightly payloads
  • CI stream: openshift-release-main__ci-<version>.yaml — e2e tests against CI payloads
  • Upgrade jobs: openshift-release-main__nightly-<version>-upgrade-from-stable-<prev>.yaml — upgrade tests from prior stable releases
  • Other variants: OKD SCOS (__okd-scos-<version>.yaml), CNV (__cnv-nightly-<version>.yaml)

The release controller configurations that define acceptance gates for these streams live in core-services/release-controller/_releases/ (for example, release-ocp-<version>.json for nightly and release-ocp-<version>-ci.json for CI).

Key Workflows

Modifying CI for a Repository

  1. Edit config in ci-operator/config/<org>/<repo>/
  2. Run make update to generate zz_generated_metadata and Prow jobs
  3. Run make checkconfig to validate
  4. Commit both config and generated job files

Adding a New Test Step

  1. Create files in ci-operator/step-registry/<component>/:
    • <name>-ref.yaml (step metadata)
    • <name>-commands.sh (script to execute)
  2. Run make registry-metadata to update metadata
  3. Reference the step in workflow chains or directly in configs

Creating a New Workflow

  1. Define individual steps if needed
  2. Create a workflow file: <name>-workflow.yaml
  3. Define pre, test, and post phases using existing steps/chains
  4. Run make validate-step-registry to check correctness

Creating Variant Periodic Configurations

CI analytical tooling requires release specific configuration. Branch fast-forwarding from main to the active release duplicates configuration and job execution for periodics defined in the default configuration. For jobs producing test data that will be processed by CI analytical tooling, periodic jobs should be configured separately from the main config:

  1. Create ci-operator/config/<org>/<repo>/<org>-<repo>-<branch>__periodics.yaml
  2. Include base_images, build_root, images, promotion, releases sections (copy from main config)
  3. Add only tests: entries with interval: or cron: (periodic scheduling)
  4. Run make update to generate Prow jobs in separate -periodics.yaml file
  5. This pattern creates release specific config that produces data consumable by CI analysis tooling

Security Best Practices for Step Registry Scripts

When writing step registry command scripts (*-commands.sh), protect sensitive information from leaking into CI logs:

Avoid Accidental Disclosure:

  • Never echo or print passwords, tokens, API keys, cluster URLs, or kubeconfig contents
  • Be cautious with set -x (debug tracing) - it logs all executed commands with their arguments
  • Variable expansions in traced commands will expose their values in logs

Protecting Sensitive Operations: When handling credentials or sensitive data, temporarily disable tracing:

# Save current tracing state
[[ $- == *x* ]] && WAS_TRACING=true || WAS_TRACING=false
set +x  # Disable tracing for sensitive operations

# Handle passwords, tokens, or URLs here
secret=$(oc get secret --template='{{.data.password}}' | base64 -d)
echo "$secret" > "${SHARED_DIR}/password"

# Restore previous tracing state
$WAS_TRACING && set -x

Best Practices:

  • Default to set -euo pipefail (without -x) in scripts
  • Only enable -x when actively debugging
  • Add clear comments when disabling tracing: # Disable tracing due to password handling
  • Keep the tracing-disabled scope minimal (only around sensitive operations)
  • Use ${SHARED_DIR} for sharing data between steps instead of echoing to logs

OpenShift Release Versioning

Release Branches

OpenShift uses major/minor release lines. Historical and active 4.x and 5.x configurations coexist in this repository, so determine the target version from the files being changed rather than from a hard-coded list here.

  • Branch names: release-<major>.<minor> (or main/master)
  • Config files: One per branch per repository (e.g., openshift-oauth-server-release-<major>.<minor>.yaml)
  • CI Analysis: Release-specific configurations are necessary for CI analytical tooling

Config Brancher Tool

The config-brancher tool automates creating CI configs for new releases:

config-brancher --config-dir ./ci-operator/config --current-release X.Y --future-release X.Z --confirm

Replace X.Y and X.Z with the applicable release lines. The source content is always the current development branch, such as main or master. --current-release identifies the release currently targeted by that branch; --future-release asks the tool to maintain an additional future-release config. The tool copies and adjusts versions in the managed release configs.

Use --skip-periodics to avoid branching periodic jobs (they're often managed separately in __periodics.yaml files). Repos not relying on release specific CI analysis typically only have periodics set up for master and only want to run jobs for master code. The code in both managed release branches is identical so there is no point in having copies of these periodics; they build and test the same code. That's why --skip-periodics exists. When teams rely on release specific CI analysis then they should only exist in release specific __periodics.yaml CI configurations so the data they produce is tied to a specific release. In this case we do not want the jobs to run against the main or master configurations and we cannot put them in the default release branch CI configurations because that's owned by config-brancher enforcing that the release branch config is derived from main config.

Python Environment Setup

If Python scripts fail (e.g., generate-release-controllers.py), set up a virtual environment:

python3 -m venv venv/           # First time only
source venv/bin/activate
python3 -m pip install pyyaml
# Run your Python commands...
deactivate                      # When done

Container Engine

By default, podman is used. Override with:

export CONTAINER_ENGINE=docker

Agent Harness Configuration

Claude Code slash commands in .claude/commands/ must include frontmatter:

---
description: Brief description of what the command does
args: "[arg1] [arg2]"
allowed-tools: Read, Edit, Bash(make release-controllers), AskUserQuestion
---

Shared skill implementations live in .agents/skills/<skill-name>/SKILL.md. For Claude Code compatibility, each .claude/skills/<skill-name> entry is a relative symlink to the corresponding directory under .agents/skills/. Edit the canonical .agents/skills/ content rather than replacing a symlink with a copied directory. Keep Claude-specific commands, rules, scripts, and settings under .claude/, while keeping shared repository guidance in this file.

Important Notes

  • Never manually edit files in ci-operator/jobs/ - always edit ci-operator/config/ and regenerate
  • New test workflows should use step-registry, not templates (templates are legacy)
  • Deployment behavior varies across clusters/, core-services/, and services/; do not assume a directory is applied to api.ci or by a particular mechanism without checking its current README, Prow configuration, and GitOps ApplicationSet
  • Many paths under clusters/ are GitOps sources and may be applied automatically after merge; verify the target and ownership before editing
  • ConfigMaps are updated by the config-updater Prow plugin only where configured
  • The repository uses containerized tooling for most operations (check Makefile for specific images)
  • Config files are deterministically formatted - manual formatting will be overwritten by generation tools
  • Metadata like zz_generated_metadata is created by make update and should not be manually edited