Redis backed Caching of Internal Embeddings

[ajtiwari07]

1. Embeddings-Specific Cache Configuration

• New EmbeddingsCacheOptions class for embeddings-specific cache settings
• Configurable TTL (Time-To-Live) with default of 24 hours (optimized for deterministic embeddings)
• Independent cache configuration from global runtime cache
• Support for enabling/disabling caching at the embeddings level

2. Level 2 (L2) Distributed Redis Cache Support

• New EmbeddingsCacheLevel2Options class for Redis-specific configuration
• Dedicated Redis connection for embeddings cache
• Independent from global runtime L2 cache (separate connection string)
• FusionCache integration with Redis backplane for distributed cache invalidation
• Graceful fallback to L1-only if L2 is not configured or fails

3. Enhanced Configuration Schema

• Updated JSON schema with new cache section under embeddings
• Support for environment variable substitution in Redis connection strings
• Clear documentation and examples in schema
• Validation for required fields when L2 is enabled

4. Dedicated FusionCache Instance for Embeddings

• Separate FusionCache instance named "EmbeddingsCache"
• Independent cache lifecycle from global DAB cache
• Optimized entry options for embedding workloads
• Error resilience with exception suppression for distributed operations

5. Comprehensive Test Coverage

• Unit tests for EmbeddingsCacheOptions class (205 lines)
• Unit tests for EmbeddingsCacheLevel2Options class (64 lines)
• Serialization tests for cache options (233 lines)
• Runtime cache L2 options tests (241 lines)
• Redis connection startup tests (138 lines)

  "$schema": "https://github.com/Azure/data-api-builder/releases/latest/download/dab.draft.schema.json",
  "data-source": {
    "database-type": "mssql",
    "connection-string": "@env('DATABASE_CONNECTION_STRING')"
  },
  "runtime": {
    "embeddings": {
      "enabled": true,
      "provider": "azure-openai",
      "base-url": "https://my-openai.openai.azure.com",
      "api-key": "@env('AZURE_OPENAI_API_KEY')",
      "model": "text-embedding-ada-002",
      "dimensions": 1536,
      "cache": {
        "enabled": true,
        "ttl-hours": 72,
        "level-2": {
          "enabled": true,
          "connection-string": "@env('EMBEDDINGS_REDIS_CONNECTION_STRING')"
        }
      }
    }
  }```

[Copilot]

Pull request overview

This PR extends the embeddings feature set (“phase2”) by adding embeddings-specific cache configuration (L1 in-memory + optional L2 Redis), wiring a dedicated FusionCache instance for embeddings in the Service startup path, and updating config serialization/schema and tests to cover the new options and related behaviors.

Changes:

• Add runtime.embeddings.cache configuration surface (object model + JSON converters + JSON schema).
• Configure a named FusionCache instance (EmbeddingsCache) and optionally attach Redis distributed cache/backplane for embeddings.
• Add/adjust unit tests and update configuration snapshots to reflect the new runtime config shape.

Reviewed changes

Copilot reviewed 22 out of 23 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
src/Service/Startup.cs	Adds embeddings cache setup and overrides embedding service registration to use a named FusionCache (and optional Redis).
src/Service/HealthCheck/HealthCheckHelper.cs	Adds fields intended to track incoming role/token for health checks.
src/Service/Controllers/EmbeddingController.cs	Minor formatting change (blank line).
src/Service.Tests/UnitTests/StartupRedisConnectionTests.cs	New unit tests for Redis Entra-auth detection logic in Startup.
src/Service.Tests/UnitTests/EmbeddingsCacheOptionsTests.cs	New unit tests for embeddings cache option defaults/behavior.
src/Service.Tests/UnitTests/EmbeddingsCacheOptionsSerializationTests.cs	New unit tests validating (de)serialization of embeddings cache options with runtime config serializer options.
src/Service.Tests/UnitTests/EmbeddingsCacheLevel2OptionsTests.cs	New unit tests for embeddings L2 cache option record/config behavior.
src/Service.Tests/UnitTests/EmbeddingControllerTests.cs	Extends controller tests around service availability; currently introduces duplicate local declarations.
src/Service.Tests/UnitTests/ConfigValidationUnitTests.cs	Adds additional embeddings validation tests; currently duplicates existing test methods in the same class.
src/Service.Tests/Snapshots/ConfigurationTests.TestReadingRuntimeConfigForPostgreSql.verified.txt	Updates expected runtime-config snapshot output to match new serialization/config shape.
src/Service.Tests/Snapshots/ConfigurationTests.TestReadingRuntimeConfigForMySql.verified.txt	Updates expected runtime-config snapshot output to match new serialization/config shape.
src/Service.Tests/Snapshots/ConfigurationTests.TestReadingRuntimeConfigForCosmos.verified.txt	Updates expected runtime-config snapshot output to match new serialization/config shape.
src/Service.Tests/Configuration/ConfigurationTests.cs	Fixes JSON request body formatting in an existing strict-mode test.
src/Core/Services/Embeddings/EmbeddingTelemetryHelper.cs	Adds OpenTelemetry namespace import for tracing-related extensions.
src/Config/RuntimeConfigLoader.cs	Registers the new EmbeddingsCacheOptionsConverterFactory in serializer options.
src/Config/ObjectModel/Embeddings/EmbeddingsOptions.cs	Adds cache property to embeddings runtime options and helper booleans for cache enablement.
src/Config/ObjectModel/Embeddings/EmbeddingsCacheOptions.cs	Introduces embeddings cache options model (enabled/ttl/level2 + “user provided TTL” semantics).
src/Config/ObjectModel/Embeddings/EmbeddingsCacheLevel2Options.cs	Introduces embeddings L2 cache options model (Redis connection string).
src/Config/Converters/EmbeddingsOptionsConverterFactory.cs	Extends embeddings options converter to read/write the new cache section.
src/Config/Converters/EmbeddingsCacheOptionsConverterFactory.cs	New custom converter to control embeddings cache options serialization (avoid writing default TTL unless explicitly provided).
src/Cli/Commands/ConfigureOptions.cs	Adds CLI flags for embeddings runtime configuration; currently contains duplicated option/property blocks.
schemas/dab.draft.schema.json	Adds schema definition for runtime.embeddings.cache (including L2 settings).

Comments suppressed due to low confidence (1)

src/Cli/Commands/ConfigureOptions.cs:474

• Duplicate option properties were introduced for embeddings (e.g., RuntimeEmbeddingsEnabled/Provider/BaseUrl/etc. are declared twice). This will not compile due to duplicate member definitions and duplicate [Option] attributes; remove the duplicated block and keep a single set of embeddings options (including endpoint.path/chunking options as needed).

        [Option("runtime.embeddings.enabled", Required = false, HelpText = "Enable/disable the embedding service. Default: true")]
        public CliBool? RuntimeEmbeddingsEnabled { get; }

        [Option("runtime.embeddings.provider", Required = false, HelpText = "Configure embedding provider type. Allowed values: azure-openai, openai.")]
        public EmbeddingProviderType? RuntimeEmbeddingsProvider { get; }

[ajtiwari07]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 6 pipeline(s).

[ajtiwari07]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 6 pipeline(s).

[ajtiwari07]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 6 pipeline(s).

[souvikghosh04]

Posting some comments which needs to be fixed. Will have another review after that.