Skip to content

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

Terraform Provider for SELECT - Development

This repository contains the source code for the SELECT Terraform Provider. This provider is mostly auto-generated from SELECT's public OpenAPI specification, enabling developers to manage SELECT platform resources through Terraform.

For usage documentation and examples, visit the official Terraform Registry documentation.

About SELECT

SELECT helps organizations optimize their Snowflake usage and costs. This Terraform provider enables Infrastructure as Code management of SELECT platform resources like usage groups and usage group sets.

Architecture Overview

This provider is built using:

Code Generation Workflow

The provider code is generated from SELECT's v2 OpenAPI specification:

  1. Fetch OpenAPI Spec: Downloads the latest spec
  2. Generate Schema: tfplugingen-openapi converts it to Terraform schema definitions
  3. Patch Schema: tools/specpatch fills in what tfplugingen-openapi cannot produce — descriptions it drops for nullable fields, sensitive attributes, and plan modifiers
  4. Generate Code: tfplugingen-framework creates the final provider code
  5. Manual Customization: generator_config.v2.yml and generator_overrides.v2.yml allow for customizations and overrides

This ensures the provider stays in sync with SELECT's API automatically.

Resources

Resource API
select_usage_group_set v2
select_usage_group v2
select_snowflake_account v2
select_databricks_connection v2
select_bigquery_connection v2
select_aws_connection v2
select_team v2
select_team_member v2
select_team_role v2
select_user_role v2
select_default_role v2
select_sso_group v2

Data Sources

Data Source API
select_team v2
select_users v2

To manage users and roles, see the user admin guide (docs/guides/user-admin.md). Its example is examples/guides/user_admin/main.tf.

Each v2 resource needs an API key carrying that resource's own scopes: snowflake_accounts:read and snowflake_accounts:write for Snowflake accounts, databricks_connections:read and databricks_connections:write for Databricks connections, bigquery_connections:read and bigquery_connections:write for BigQuery connections, aws_accounts:read and aws_accounts:write for AWS connections.

Development Requirements

Required Tools

The following tools are automatically installed during the build process:

# These are installed automatically by the Makefile
go install github.com/hashicorp/terraform-plugin-codegen-openapi/cmd/tfplugingen-openapi@latest
go install github.com/hashicorp/terraform-plugin-codegen-framework/cmd/tfplugingen-framework@latest

Quick Start

# Clone the repository
git clone https://github.com/get-select/terraform-provider-select
cd terraform-provider-select

# Generate provider code from OpenAPI spec and build
make reset

# Set up local development
make setup-dev-overrides

Development Commands

Command Description
make help Show all available commands
make codegen Download OpenAPI spec and generate provider code
make build Build the provider binary
make install Install provider locally for testing
make setup-dev-overrides Configure Terraform to use local provider
make test Run provider tests
make clean Remove all generated files and build artifacts
make reset Full reset: clean, regenerate, and install

Development Workflow

  1. Initial Setup:

    make reset                    # Generate code and build
    make setup-dev-overrides     # Configure local development overrides
  2. Making Changes:

    # After modifying generator_config.v2.yml or when API updates:
    make reset
    
    # For code-only changes (if manually editing generated code):
    make build install

Types from the API spec are added to internal/provider/. Some boilerplate is then needed to connect those types to the Api in a way that terraform understands. There is one of these per resource in internal/.

  1. Testing:

Testign requires and API key and organization generated from teh SELECT backend you wish to test against. You can generate an API key in Settings -> API Keys and you can find your organization id in Settings -> Profile.

# Set required environment variables
export TF_VAR_select_api_key="your-api-key"
export TF_VAR_select_organization_id="your-org-id"

# Run tests
make test

Configuration Files

generator_config.v2.yml

Configures the code generation process:

  • Resource mappings (API endpoints to Terraform resources)
  • Attribute aliases and descriptions
  • Ignored fields that don't map well to Terraform

Note that tfplugingen-openapi only honours description in an attribute override — its Override struct has no other fields, and unknown YAML keys are dropped without an error.

generator_overrides.v2.yml

Everything about the v2 schema that generator_config.v2.yml cannot express, applied to the generated code spec by tools/specpatch: the resource's description, which OpenAPI schemas to recover dropped attribute descriptions from, and per-attribute plan modifiers.

Attribute sensitivity is deliberately absent from this file. specpatch derives it from the OpenAPI document's x-terraform-sensitive markers, and refuses to generate a write-only property that is not marked, so a new secret cannot reach plan output unmasked.

example.terraformrc

Template for Terraform development overrides that allows using the locally built provider instead of downloading from the registry.

Project Structure

terraform-provider-select/
├── internal/                    # Hand-written provider core
│   ├── provider.go             # Provider configuration and setup
│   ├── api.go                  # HTTP client and API utilities
│   ├── usage_group_resource.go # Custom resource implementations
│   ├── v2_api.go               # Conventions shared by every v2 resource
│   ├── snowflake_account_resource.go # Snowflake account resource (v2 API)
│   ├── snowflake_account_api.go      # Its request payloads and conversions
│   ├── databricks_connection_resource.go # Databricks connection resource (v2 API)
│   ├── databricks_connection_api.go      # Its request payloads and conversions
│   ├── bigquery_connection_resource.go   # BigQuery connection resource (v2 API)
│   ├── bigquery_connection_api.go        # Its request payloads and conversions
│   ├── aws_connection_resource.go        # AWS connection resource (v2 API)
│   ├── aws_connection_api.go             # Its request payloads and conversions
│   └── provider/               # Generated code (git-ignored)
├── tools/specpatch/            # Post-processor for the generated v2 code spec
├── tests/                      # Provider tests
├── docs/                       # Generated documentation
├── examples/                   # Usage examples
├── generator_config.v2.yml     # v2 code generation configuration
├── generator_overrides.v2.yml  # v2 schema details codegen cannot express
├── Makefile                    # Development commands
└── main.go                     # Provider entry point

OpenAPI Dependency

Important: This provider is entirely dependent on SELECT's OpenAPI specification. It is:

  • Hosted at: https://api.select.dev/v2/openapi.json
  • Auto-fetched: Every make codegen downloads the latest
  • Single Source of Truth: Changes to the API automatically reflect in the provider

If that endpoint is unavailable, the code generation will fail. For offline development, you can work with a previously downloaded openapi.v2.json and run make codegen-go, which skips the download.

Testing

The provider includes comprehensive tests that validate functionality against the live SELECT API:

# Go unit tests, which need no API access
make test-go

# Go unit tests plus the Terraform provider tests
make test

# Run specific test files  
cd tests && terraform test provider.tftest.hcl

# Run specific test cases
cd tests && terraform test provider.tftest.hcl -filter=create_usage_group_set

# Snowflake account tests, which manage a real connection
make test-snowflake

# Databricks connection tests, which manage a real connection
make test-databricks

# BigQuery connection tests, which manage a real connection
make test-bigquery

# AWS connection tests, which manage a real connection
make test-aws

Note: The Terraform tests require valid SELECT API credentials and will create/modify real resources.

make test-snowflake, make test-databricks, make test-bigquery and make test-aws are separate from make test because adding a connection makes SELECT validate the configuration against Snowflake, Databricks, BigQuery or AWS for real: they need credentials that work, and the TF_VAR_snowflake_* / TF_VAR_databricks_* / TF_VAR_bigquery_* / TF_VAR_aws_* variables listed above each target in the Makefile.

Documentation Generation

Some documentation is semi auto-generated from the provider schema. Field descriptions are pulled from the Open API spec and should be changed in the backends endpoint metadata. Other parts of the documentation are maintained in templates/

make docs

This creates documentation in the docs/ directory that matches the format used in the Terraform Registry.

You can use this tool to preview a document file before releasing it https://registry.terraform.io/tools/doc-preview

Release Process

Releases are automated via GitHub Actions when tags are pushed, you can also create a new release via the github UI by creating a release with a new tag that objects the Semantic Versioning format, for example: v1.2.3.

  1. Code is generated from the latest OpenAPI spec
  2. Provider is built for multiple platforms
  3. Binaries are signed and uploaded to GitHub releases
  4. Terraform Registry is automatically updated

Contributing

  1. Fork the repository
  2. Create a feature branch from main
  3. Make your changes to generator_config.v2.yml or core provider files
  4. Test your changes with make test
  5. Submit a pull request

Common Development Tasks

Adding a new resource:

  1. Update generator_config.v2.yml with the new resource configuration
  2. Run make reset to regenerate code
  3. Add any custom logic in internal/
  4. Add tests in tests/

Modifying existing resources:

  1. Update the relevant section in generator_config.v2.yml
  2. Run make reset
  3. Test changes with make test

Troubleshooting

"Provider not found" errors:

make setup-dev-overrides  # Ensure dev overrides are configured

Code generation failures:

# Check internet connection and try again
make clean && make codegen

Test failures:

# Ensure credentials are set
export TF_VAR_select_api_key="your-key"
export TF_VAR_select_organization_id="your-org-id"

# Clean test state
make test-clean && make test

Links

License

This project is licensed under the Mozilla Public License 2.0 - see the LICENSE file for details.

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages