Skip to content

feat(api): add event subject list endpoint#4497

Open
tothandras wants to merge 2 commits into
mainfrom
feat/api-event-subjects
Open

feat(api): add event subject list endpoint#4497
tothandras wants to merge 2 commits into
mainfrom
feat/api-event-subjects

Conversation

@tothandras

@tothandras tothandras commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Summary by CodeRabbit

  • New Features

    • Added event-subjects listing endpoint with cursor-based pagination and alphabetical ordering.
    • Added subject filtering by key (contains) and by customer attribution (true/false).
    • Client libraries and SDKs updated to expose subject-listing and related types.
  • Documentation

    • Updated README operations table to include the new subjects listing entry.
  • Tests

    • Added end-to-end and unit tests covering listing, pagination, filters, and attribution behavior.
  • Chores

    • Optional server-side projection for faster distinct subject lookups (disabled by default).

@tothandras tothandras added the release-note/feature Release note: Exciting New Features label Jun 9, 2026
@coderabbitai

coderabbitai Bot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 2efc3b32-abc6-4df5-95b3-e260726aae98

📥 Commits

Reviewing files that changed from the base of the PR and between 3bf0c38 and 72ee5e2.

📒 Files selected for processing (7)
  • e2e/config.yaml
  • e2e/events_subjects_v3_test.go
  • openmeter/meterevent/adapter/subject.go
  • openmeter/meterevent/adapter/subject_test.go
  • openmeter/streaming/clickhouse/connector.go
  • openmeter/streaming/clickhouse/subject_query.go
  • openmeter/streaming/clickhouse/subject_query_test.go
🚧 Files skipped from review as they are similar to previous changes (6)
  • e2e/events_subjects_v3_test.go
  • openmeter/streaming/clickhouse/subject_query_test.go
  • openmeter/meterevent/adapter/subject_test.go
  • openmeter/streaming/clickhouse/connector.go
  • openmeter/meterevent/adapter/subject.go
  • openmeter/streaming/clickhouse/subject_query.go

📝 Walkthrough

Walkthrough

This PR adds /openmeter/events/subjects: TypeSpec contracts, generated JS client and docs, Go HTTP handler and server route, domain/service types and validation, streaming/ClickHouse v2 listing + optional projection, adapter scan/attribution logic, config/wiring, and unit + e2e tests.

Changes

Event Subjects Listing API

Layer / File(s) Summary
TypeSpec API contract and schema
api/spec/packages/aip/src/events/subject.tsp, api/spec/packages/aip/src/events/operations.tsp, api/spec/packages/aip/src/events/index.tsp, api/spec/packages/aip/src/konnect.tsp, api/spec/packages/aip/src/openmeter.tsp
Defines Events.Subject model with key, ListSubjectsParamsFilter with key and attributed, EventsSubjectsOperations with list-event-subjects, and routes at /openmeter/events/subjects.
TypeScript client types and functions
api/spec/packages/aip-client-javascript/src/models/schemas.ts, api/spec/packages/aip-client-javascript/src/models/types.ts, api/spec/packages/aip-client-javascript/src/models/operations/events.ts, api/spec/packages/aip-client-javascript/src/funcs/events.ts, api/spec/packages/aip-client-javascript/src/sdk/events.ts, api/spec/packages/aip-client-javascript/src/index.ts, api/spec/packages/aip-client-javascript/README.md
Adds Zod schemas and TS types (EventSubject, ListSubjectsParamsFilter, SubjectPaginatedResponse), listEventSubjects client function, SDK Events.listSubjects wrapper, and README table update.
HTTP handler and server routing
api/v3/handlers/events/list_subjects.go, api/v3/handlers/events/handler.go, api/v3/server/routes.go
Implements ListEventSubjects handler: parses/validates cursor pagination and filters, calls metereventService.ListSubjects, maps results to cursor pagination response, and wires server route.
Subject domain type and service interface
openmeter/meterevent/subject.go, openmeter/meterevent/subject_test.go, openmeter/meterevent/service.go
Adds Subject type with Cursor(), ListSubjectsParams with validation, and Service.ListSubjects interface method.
Streaming connector API
openmeter/streaming/connector.go, openmeter/streaming/testutils/streaming.go, openmeter/server/server_test.go
Adds ListSubjectsV2 connector method and ListSubjectsV2Params with validation; test mock/stub implementations added.
ClickHouse implementation
openmeter/streaming/clickhouse/connector.go, openmeter/streaming/clickhouse/subject_query.go, openmeter/streaming/clickhouse/subject_query_test.go
Adds CreateSubjectsProjection config flag and projection DDL, ListSubjectsV2 implementation, listSubjectsV2 SQL builder using GROUP BY for projection compatibility, and SQL-generation tests.
Meterevent adapter implementation
openmeter/meterevent/adapter/subject.go, openmeter/meterevent/adapter/subject_test.go
Implements ListSubjects adapter: iterative scanning via ListSubjectsV2, optional attributed-key resolution via customerService.ListCustomers, page trimming, NextCursor logic, and scan-round cap; includes unit tests with fakes.
Configuration and wiring
app/config/aggregation.go, app/common/streaming.go, openmeter/streaming/retry/retry.go
Adds CreateSubjectsProjection flag (default false), forwards it into ClickHouse connector config, and wraps ListSubjectsV2 with retry logic.
End-to-end testing and helpers
e2e/events_subjects_v3_test.go, e2e/v3helpers_test.go, e2e/config.yaml
Adds TestEventSubjectsV3 e2e test that ingests events and validates subject listing, filtering, and pagination; adds IngestEvents and ListEventSubjects test helpers and e2e config enabling projection in CI config.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Suggested labels

kind/feature

Suggested reviewers

  • turip
  • chrisgacsal
  • gergely-kurucz-konghq
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 15.38% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'feat(api): add event subject list endpoint' clearly and accurately summarizes the main change—adding a new endpoint for listing event subjects across the entire changeset.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/api-event-subjects

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@tothandras tothandras force-pushed the feat/api-event-subjects branch from 9937a40 to 1f3c37e Compare June 10, 2026 10:23
@tothandras tothandras force-pushed the feat/api-event-subjects branch from 1f3c37e to 449cad9 Compare June 11, 2026 19:52
@tothandras tothandras marked this pull request as ready for review June 11, 2026 19:52
@tothandras tothandras requested a review from a team as a code owner June 11, 2026 19:52

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
openmeter/meterevent/adapter/subject.go (1)

104-137: ⚡ Quick win

Two separate customer queries for attribution resolution.

The implementation makes two ListCustomers calls (lines 114-123) to resolve attributed subjects: one searching by UsageAttributionSubjectKey and another by customer Key. This is necessary because these are different search fields, and the customer service likely doesn't support OR queries across these fields.

While this works correctly, if the customer service API is ever extended to support OR queries or multi-field searches, combining these into a single call could reduce overhead, especially when many subjects need attribution checks.

For now, this is fine—just something to keep in mind for future optimization if attribution checks become a hot path.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@openmeter/meterevent/adapter/subject.go` around lines 104 - 137, The function
attributedSubjectKeys currently issues two ListCustomers calls (via
a.customerService.ListCustomers) to match keys against
UsageAttributionSubjectKey and Key; add a concise TODO comment in
attributedSubjectKeys noting that these separate queries are intentional but
should be combined into a single OR/multi-field query if/when the customer
service gains support for OR searches (or add a feature-flagged path to call a
combined search method such as ListCustomersWithOr if implemented), referencing
the existing fields UsageAttributionSubjectKey and Key and the ListCustomers
call so future maintainers can find and replace the two-query approach with a
single call.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@openmeter/meterevent/adapter/subject.go`:
- Around line 104-137: The function attributedSubjectKeys currently issues two
ListCustomers calls (via a.customerService.ListCustomers) to match keys against
UsageAttributionSubjectKey and Key; add a concise TODO comment in
attributedSubjectKeys noting that these separate queries are intentional but
should be combined into a single OR/multi-field query if/when the customer
service gains support for OR searches (or add a feature-flagged path to call a
combined search method such as ListCustomersWithOr if implemented), referencing
the existing fields UsageAttributionSubjectKey and Key and the ListCustomers
call so future maintainers can find and replace the two-query approach with a
single call.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 7777cf3c-0234-47c6-9ead-9cb9ea1a5d74

📥 Commits

Reviewing files that changed from the base of the PR and between 50c8999 and 449cad9.

⛔ Files ignored due to path filters (1)
  • api/v3/openapi.yaml is excluded by !**/openapi.yaml
📒 Files selected for processing (32)
  • api/spec/packages/aip-client-javascript/README.md
  • api/spec/packages/aip-client-javascript/src/funcs/events.ts
  • api/spec/packages/aip-client-javascript/src/index.ts
  • api/spec/packages/aip-client-javascript/src/models/operations/events.ts
  • api/spec/packages/aip-client-javascript/src/models/schemas.ts
  • api/spec/packages/aip-client-javascript/src/models/types.ts
  • api/spec/packages/aip-client-javascript/src/sdk/events.ts
  • api/spec/packages/aip/src/events/index.tsp
  • api/spec/packages/aip/src/events/operations.tsp
  • api/spec/packages/aip/src/events/subject.tsp
  • api/spec/packages/aip/src/konnect.tsp
  • api/spec/packages/aip/src/openmeter.tsp
  • api/v3/api.gen.go
  • api/v3/handlers/events/handler.go
  • api/v3/handlers/events/list_subjects.go
  • api/v3/server/routes.go
  • app/common/streaming.go
  • app/config/aggregation.go
  • e2e/events_subjects_v3_test.go
  • e2e/v3helpers_test.go
  • openmeter/meterevent/adapter/subject.go
  • openmeter/meterevent/adapter/subject_test.go
  • openmeter/meterevent/service.go
  • openmeter/meterevent/subject.go
  • openmeter/meterevent/subject_test.go
  • openmeter/server/server_test.go
  • openmeter/streaming/clickhouse/connector.go
  • openmeter/streaming/clickhouse/subject_query.go
  • openmeter/streaming/clickhouse/subject_query_test.go
  • openmeter/streaming/connector.go
  • openmeter/streaming/retry/retry.go
  • openmeter/streaming/testutils/streaming.go

@greptile-apps

greptile-apps Bot commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds a new GET /events/subjects endpoint that lists the distinct subject keys of ingested metering events, with cursor-based keyset pagination, alphabetical ordering, a filter[key][contains] string filter, and an attributed boolean filter that post-filters subjects by whether they are linked to a customer.

  • The adapter's ListSubjects iterates ClickHouse batches in a scan loop (capped at 10 rounds) and resolves customer attribution via two ListCustomers calls per batch when the attributed filter is active; the scanLimit is widened to MaximumLimit (100) for attributed requests to fill pages in fewer rounds.
  • An optional prj_namespace_subject ClickHouse aggregate projection (disabled by default) is introduced to speed up the GROUP BY namespace, subject query on large tables.
  • The JS/TS client is updated with the new listEventSubjects function and matching types.

Confidence Score: 5/5

The change is safe to merge. The new endpoint is marked x-internal and x-unstable, the projection is opt-in and off by default, and the scan-loop logic, cursor pagination, and attribution resolution are all correctly implemented.

No correctness bugs were found. The multi-batch scan loop correctly handles all edge cases: empty batches, exact full pages, over-full pages, and the scan-round cap. Attribution resolution via two ListCustomers lookups per batch is correct, and the cursor propagation through both early-return paths and the cap path is sound.

No files require special attention beyond the minor suggestions on openmeter/streaming/connector.go and openmeter/streaming/clickhouse/subject_query.go.

Important Files Changed

Filename Overview
openmeter/meterevent/adapter/subject.go New file — implements the multi-batch scan loop with keyset cursor, attribution resolution, and scan-round cap; logic is correct and well-commented.
openmeter/streaming/clickhouse/subject_query.go Adds listSubjectsV2Query and projection DDL; SETTINGS clause is correct but uses non-sorted map iteration for query settings, consistent with the existing meter_query pattern.
openmeter/streaming/connector.go Adds ListSubjectsV2Params with validation; lower-bound Limit check is present but upper-bound (MaximumLimit) check is absent, unlike the service-layer counterpart.
api/v3/handlers/events/list_subjects.go New handler for GET /events/subjects; page parsing, cursor decoding, filter conversion, and response marshalling are all correct.
openmeter/meterevent/subject.go New file — defines Subject, cursor helper, and ListSubjectsParams with complete validation; clean and straightforward.
openmeter/streaming/clickhouse/connector.go Adds ListSubjectsV2 and optional subjects projection creation; wiring from config is correct and projection is opt-in (disabled by default).
openmeter/meterevent/adapter/subject_test.go Comprehensive unit tests including multi-batch scan, attribution by customer key, scan-round-cap, and cursor continuation; both attribution paths exercised.
e2e/events_subjects_v3_test.go New e2e test covering pagination, attributed/unattributed filtering, key filter, and invalid param rejection.
app/config/aggregation.go Adds CreateSubjectsProjection config field with default false; well-documented with the manual backfill caveat.

Sequence Diagram

sequenceDiagram
    participant Client
    participant Handler as HTTP Handler
    participant Adapter as adapter.ListSubjects
    participant CH as ClickHouse ListSubjectsV2
    participant CS as CustomerService ListCustomers

    Client->>Handler: "GET /events/subjects?filter[attributed][eq]=true&page[size]=5"
    Handler->>Adapter: ListSubjects(params)

    loop scan rounds up to 10
        Adapter->>CH: "ListSubjectsV2(namespace, key, cursor, limit=100)"
        CH-->>Adapter: batch of subject keys
        Adapter->>CS: ListCustomers(UsageAttributionSubjectKey IN keys)
        CS-->>Adapter: customers
        Adapter->>CS: ListCustomers(Key IN keys)
        CS-->>Adapter: customers
        Note over Adapter: filter batch by attribution
        alt result full
            Note over Adapter: trim, set NextCursor, return
        else batch exhausted
            Note over Adapter: return without NextCursor
        else advance cursor
            Note over Adapter: cursor = last key in batch
        end
    end

    Note over Adapter: scan cap hit, set NextCursor=cursor
    Adapter-->>Handler: pagination.Result[Subject]
    Handler-->>Client: 200 SubjectPaginatedResponse
Loading

Fix All in Claude Code

Reviews (3): Last reviewed commit: "refactor: WIP" | Re-trigger Greptile

Comment thread openmeter/meterevent/adapter/subject.go
Comment thread openmeter/meterevent/adapter/subject_test.go
Comment thread openmeter/meterevent/adapter/subject.go

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@openmeter/meterevent/adapter/subject.go`:
- Around line 128-131: The accumulator currently only adds attribution values
from c.GetUsageAttribution().GetValues(), missing the customer's own key; update
the loop over customerList.Items (variable c) to also insert the customer's key
into the attributed set (e.g., attributed[c.GetKey()] = struct{}{} or the
appropriate getter used in this type), guarding for empty/nil keys if needed so
the customer's own key match path is preserved for filtering.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: c094f8c6-a5aa-4659-9ca7-3cba8a9eb910

📥 Commits

Reviewing files that changed from the base of the PR and between 449cad9 and 3bf0c38.

⛔ Files ignored due to path filters (1)
  • api/v3/openapi.yaml is excluded by !**/openapi.yaml
📒 Files selected for processing (32)
  • api/spec/packages/aip-client-javascript/README.md
  • api/spec/packages/aip-client-javascript/src/funcs/events.ts
  • api/spec/packages/aip-client-javascript/src/index.ts
  • api/spec/packages/aip-client-javascript/src/models/operations/events.ts
  • api/spec/packages/aip-client-javascript/src/models/schemas.ts
  • api/spec/packages/aip-client-javascript/src/models/types.ts
  • api/spec/packages/aip-client-javascript/src/sdk/events.ts
  • api/spec/packages/aip/src/events/index.tsp
  • api/spec/packages/aip/src/events/operations.tsp
  • api/spec/packages/aip/src/events/subject.tsp
  • api/spec/packages/aip/src/konnect.tsp
  • api/spec/packages/aip/src/openmeter.tsp
  • api/v3/api.gen.go
  • api/v3/handlers/events/handler.go
  • api/v3/handlers/events/list_subjects.go
  • api/v3/server/routes.go
  • app/common/streaming.go
  • app/config/aggregation.go
  • e2e/events_subjects_v3_test.go
  • e2e/v3helpers_test.go
  • openmeter/meterevent/adapter/subject.go
  • openmeter/meterevent/adapter/subject_test.go
  • openmeter/meterevent/service.go
  • openmeter/meterevent/subject.go
  • openmeter/meterevent/subject_test.go
  • openmeter/server/server_test.go
  • openmeter/streaming/clickhouse/connector.go
  • openmeter/streaming/clickhouse/subject_query.go
  • openmeter/streaming/clickhouse/subject_query_test.go
  • openmeter/streaming/connector.go
  • openmeter/streaming/retry/retry.go
  • openmeter/streaming/testutils/streaming.go
✅ Files skipped from review due to trivial changes (2)
  • api/spec/packages/aip/src/konnect.tsp
  • api/spec/packages/aip-client-javascript/README.md
🚧 Files skipped from review as they are similar to previous changes (27)
  • api/spec/packages/aip/src/openmeter.tsp
  • api/spec/packages/aip/src/events/index.tsp
  • openmeter/streaming/testutils/streaming.go
  • api/spec/packages/aip-client-javascript/src/sdk/events.ts
  • openmeter/server/server_test.go
  • api/v3/handlers/events/handler.go
  • app/config/aggregation.go
  • app/common/streaming.go
  • api/v3/server/routes.go
  • e2e/events_subjects_v3_test.go
  • openmeter/streaming/retry/retry.go
  • api/spec/packages/aip-client-javascript/src/funcs/events.ts
  • api/spec/packages/aip-client-javascript/src/index.ts
  • openmeter/meterevent/service.go
  • api/spec/packages/aip/src/events/subject.tsp
  • e2e/v3helpers_test.go
  • openmeter/streaming/connector.go
  • api/spec/packages/aip/src/events/operations.tsp
  • api/spec/packages/aip-client-javascript/src/models/operations/events.ts
  • api/v3/handlers/events/list_subjects.go
  • openmeter/streaming/clickhouse/subject_query_test.go
  • openmeter/meterevent/adapter/subject_test.go
  • openmeter/streaming/clickhouse/connector.go
  • openmeter/streaming/clickhouse/subject_query.go
  • openmeter/meterevent/subject.go
  • api/spec/packages/aip-client-javascript/src/models/schemas.ts
  • api/spec/packages/aip-client-javascript/src/models/types.ts

Comment on lines +128 to +131
for _, c := range customerList.Items {
for _, value := range c.GetUsageAttribution().GetValues() {
attributed[value] = struct{}{}
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Include customer-key hits in the attributed set.

On Line 128 onward, the accumulator only adds GetUsageAttribution().GetValues(). That misses the “customer’s own key” match path documented above the function, so attributed filtering can drop valid subjects.

💡 Suggested fix
 		for _, c := range customerList.Items {
+			if c.Key != "" {
+				attributed[c.Key] = struct{}{}
+			}
 			for _, value := range c.GetUsageAttribution().GetValues() {
 				attributed[value] = struct{}{}
 			}
 		}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@openmeter/meterevent/adapter/subject.go` around lines 128 - 131, The
accumulator currently only adds attribution values from
c.GetUsageAttribution().GetValues(), missing the customer's own key; update the
loop over customerList.Items (variable c) to also insert the customer's key into
the attributed set (e.g., attributed[c.GetKey()] = struct{}{} or the appropriate
getter used in this type), guarding for empty/nil keys if needed so the
customer's own key match path is preserved for filtering.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release-note/feature Release note: Exciting New Features

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant