Skip to content

fix(mcp): resolve attribute scope and explain empty results - #1227

Merged
Makisuo merged 8 commits into
fix/mcp-metric-countersfrom
fix/mcp-attribute-scope-matching
Oct 3, 2026
Merged

Makisuo merged 8 commits into
fix/mcp-metric-countersfrom
fix/mcp-attribute-scope-matching

Conversation

@Makisuo

@Makisuo Makisuo commented Oct 3, 2026 •

Copy link
Copy Markdown
Collaborator

What

  • query_data resolves attribute_key against span then resource attributes (or attribute_scope), for filters and group-by. Resource keys like deployment.environment, vcs.ref.head.revision, k8s.pod.name returned empty or zeros before.
  • Fixed group_by: resource_attribute returning service names on metrics, and the attribute filter being dropped when grouping by the same key.
  • Traces accept group_by: resource_attribute; logs get attribute filters; environments applies to logs and metrics.
  • search_traces: environment filter, and rows carry deployment.environment and service.version.
  • Empty results suggest close matches for service, environment, attribute key and span name (query_data, find_errors, search_logs, search_traces, list_error_issues).
  • query_data retries a span name as a case-insensitive substring before giving up, matching search_traces.
  • Counts are rounded and labeled sample-weighted estimates; percentiles are labeled unweighted (weighting them would make rollup-backed paths disagree).
  • explore_attributes honors service_name for key listings (sampled from the service's spans).
  • create_dashboard accepts query_data group_by spellings.

Tests

query-engine metrics, attribute-keys, traces, pipe-dispatch, catalog(+baseline); apps/ai filter-suggestions, query-data, create-dashboard (new), registry(.contract), query-spec-tokens; apps/ai typecheck.

…and service-scoped keys

Traces breakdowns and timeseries can now group by a ResourceAttributes key
(filters.groupByResourceAttributeKey). Metric breakdowns apply the service,
environment and datapoint attribute filters they used to drop, so filtering
and grouping on the same label narrows the result. A metric attribute filter
without a value means exists, not equals the empty string.

span/resource attribute key and value discovery honors service_name by
sampling that service's raw spans (the hourly rollup has no ServiceName).
Root trace rows carry deployment environment and service.version, and
span_search accepts a deployment_env filter.
query_data resolves attribute_key against span (or log/metric label) keys,
then resource keys, so deployment.environment or k8s.pod.name filter and
group instead of returning nothing; attribute_scope overrides. Traces accept
group_by=resource_attribute, metrics stop collapsing it to service, and a
value filter on the grouped key is kept. Logs take attribute and environment
filters, metrics take environments.

span_name falls back to a case-insensitive substring when nothing matches
exactly. An empty result carries did-you-mean hints for the service,
environment, attribute key and span name (shared helper). Trace counts are
rounded and labelled as sample-weighted estimates; durations are labelled as
unweighted.
…mpty results

search_traces takes an environment filter, matches a resource attribute
(k8s.pod.name=...) when attribute_key resolves to one, and its root rows
carry deployment.environment and service.version. find_errors, search_logs,
search_traces and list_error_issues add hints on an empty result when the
service, environment or attribute key does not exist, naming close matches.
explore_attributes notes that service-scoped key counts come from a sample.
Simple widgets rejected group_by=service with "Valid: service.name, span.name"
while query_data uses service/span_name. A small alias map translates the
query_data tokens to the builder ones when valid for the widget's source.
@coderabbitai

coderabbitai Bot commented Oct 3, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 15 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 404dd4b2-d9d5-42f9-8128-7dc7792c06d9
📥 Commits

Reviewing files that changed from the base of the PR and between 21ee643 and 6bbb4ea.

📒 Files selected for processing (35)
  • apps/ai/src/mcp/lib/attribute-scope.ts
  • apps/ai/src/mcp/lib/empty-result-hints.ts
  • apps/ai/src/mcp/lib/filter-suggestions.test.ts
  • apps/ai/src/mcp/lib/filter-suggestions.ts
  • apps/ai/src/mcp/lib/format-query-result.ts
  • apps/ai/src/mcp/tools/create-dashboard.test.ts
  • apps/ai/src/mcp/tools/create-dashboard.ts
  • apps/ai/src/mcp/tools/explore-attributes.ts
  • apps/ai/src/mcp/tools/find-errors.ts
  • apps/ai/src/mcp/tools/list-error-issues.ts
  • apps/ai/src/mcp/tools/query-data.test.ts
  • apps/ai/src/mcp/tools/query-data.ts
  • apps/ai/src/mcp/tools/search-logs.ts
  • apps/ai/src/mcp/tools/search-traces.ts
  • packages/domain/src/mcp-outputs/errors.ts
  • packages/domain/src/mcp-outputs/issues.ts
  • packages/domain/src/mcp-outputs/services.ts
  • packages/domain/src/mcp-outputs/traces.ts
  • packages/domain/src/query-engine.ts
  • packages/domain/src/tinybird/endpoints.ts
  • packages/query-engine/src/__sql_baseline__/catalog.sql
  • packages/query-engine/src/benchmark/builders.ts
  • packages/query-engine/src/ch/index.ts
  • packages/query-engine/src/ch/pipe-dispatch.test.ts
  • packages/query-engine/src/ch/pipe-dispatch.ts
  • packages/query-engine/src/ch/queries/attribute-keys.test.ts
  • packages/query-engine/src/ch/queries/attribute-keys.ts
  • packages/query-engine/src/ch/queries/metrics.test.ts
  • packages/query-engine/src/ch/queries/traces.test.ts
  • packages/query-engine/src/ch/queries/traces.ts
  • packages/query-engine/src/observability/row-mappers.test.ts
  • packages/query-engine/src/observability/row-mappers.ts
  • packages/query-engine/src/observability/search-traces.ts
  • packages/query-engine/src/observability/types.ts
  • packages/query-engine/src/runtime/query-engine.ts
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@maple-review-bot

maple-review-bot Bot commented Oct 3, 2026 •

Copy link
Copy Markdown

Maple review

🔴 Confidence 2/5 · risky as written
The other MCP tool diffs, the tail of the runtime diff and the warehouse fetch layer went unread, so hint wiring and client spans are unverified.
quality 90/100 · 1 warning · tests partial · risk medium · 0/2 new units observable

Warning

This review ended early; what follows is what it established.

Makes query_data resolve an attribute key's span/log/label versus resource scope automatically (new resolveAttributeScope, pickAttributeScope), adds resource-attribute group-by to trace queries and to buildRawSpec, adds a deployment-environment filter to search_traces, exposes root deployment.environment/service.version, and adds did-you-mean emptyHints to the empty results of several tools.

  • resolveAttributeScope picks the span or resource map for query_data attribute keys
  • Trace queries group by a resource key through groupByResourceAttributeKey
  • Empty results carry emptyHints from missingFilterHints close-match lookups
  • search_traces filters by environment; root spans expose deployment env and service.version

Findings

🟠 Warning · F1 · Capped environment list is reported to the agent as complete

correctness · apps/ai/src/mcp/lib/filter-suggestions.ts:117

facets() builds environments from the same services_facets response that caps each facet at 50 rows (empty-result-hints.ts:8, :27), yet the environment hint passes complete: true hard-coded here. An environment that exists in the window but sits outside the top 50 is therefore reported as "environment "x" was not seen in this window", and the agent discards a filter that has data. Services already carry servicesComplete for exactly this reason: derive the same flag for environments and pass it instead of true.

Add `environmentsComplete` to `KnownValues`, set it from `environments.length < FACET_CAP` in `facets()`, and pass `known.environmentsComplete ?? true` here (mirroring the services branch on line 106).
🤖 Prompt to fix this finding with an AI agent
Findings from an automated review of commit 78a61126f1e76dbb2ea3e29cd49b4b3088e39e81. Verify each one against the current code before changing anything, fix only those that still apply, and keep each fix to the lines it names.

---

F1 · Warning · correctness · apps/ai/src/mcp/lib/filter-suggestions.ts:117
Capped environment list is reported to the agent as complete
`facets()` builds `environments` from the same `services_facets` response that caps each facet at 50 rows (`empty-result-hints.ts:8`, `:27`), yet the environment hint passes `complete: true` hard-coded here. An environment that exists in the window but sits outside the top 50 is therefore reported as "environment \"x\" was not seen in this window", and the agent discards a filter that has data. Services already carry `servicesComplete` for exactly this reason: derive the same flag for environments and pass it instead of `true`.
Suggested fix: Add `environmentsComplete` to `KnownValues`, set it from `environments.length < FACET_CAP` in `facets()`, and pass `known.environmentsComplete ?? true` here (mirroring the services branch on line 106).
What was checked
  • Resource group-by reaches the SQL: buildGroupNameExpr/buildBreakdownGroupExpr take the new key and the runtime validator accepts it (ch/queries/traces.ts:184, runtime/query-engine.ts:525)
  • Grouping an attribute without a value still adds no exists filter, and span/product-event paths keep attributeFilters (query-data.ts:149, decode asserted in query-data.test.ts:89)
  • New service-scoped key queries stay bounded and org-scoped (orgId/serviceName predicates plus SERVICE_SCOPED_SPAN_SAMPLE, queries/attribute-keys.ts:194–217)
Observability coverage: 0 of 2 changes observable
Change Kind Observable Evidence
Empty-result hint lookups: services_facets, trace attribute keys, span-name breakdown outbound no Only Effect.withSpan("McpTool.emptyResultHints") (empty-result-hints.ts:86); the warehouse fetches themselves are unread, so no Client/peer.service could be confirmed
Attribute-scope key lookups on every query_data call with an attribute_key outbound no Effect.fn("McpTool.resolveAttributeScope") wraps two exploreAttributeKeys/metric_attribute_keys fetches (attribute-scope.ts:42, 63); fetch-layer instrumentation unread
Files not reviewed (6)

The review ended before it read these diffs, so nothing above vouches for them.

  • apps/ai/src/mcp/tools/create-dashboard.test.ts
  • packages/query-engine/src/ch/pipe-dispatch.test.ts
  • packages/query-engine/src/ch/queries/attribute-keys.test.ts
  • packages/query-engine/src/ch/queries/metrics.test.ts
  • packages/query-engine/src/ch/queries/traces.test.ts
  • packages/query-engine/src/observability/row-mappers.test.ts

78a6112 · Updated on every push. Reply "won't fix" to dismiss a finding, or mention @maple-review-bot to ask about one.

@Makisuo
Makisuo added this pull request to stack #1234 October 3, 2026 22:23
@Makisuo Makisuo changed the title fix/mcp attribute scope matching fix(mcp): resolve attribute scope and explain empty results Oct 3, 2026

@maple-review-bot maple-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 inline note from Maple's review. The score and summary are in the review comment above.

Comment thread apps/ai/src/mcp/lib/filter-suggestions.ts Outdated
services_facets caps environments at 50 rows like services, so absence from
that list no longer produces a "not seen in this window" hint. The
create_dashboard group_by alias table is a Map so it keeps its known entries
instead of widening to an open dictionary.
@maple-review-bot

maple-review-bot Bot commented Oct 3, 2026 •

Copy link
Copy Markdown

Maple review

🟡 Confidence 3/5 · needs attention
Empty-result lookups are untested and the query-engine/domain halves of the diff went unread before the pass ended.
quality 98/100 · 1 note · tests partial · risk medium · 0/2 new units observable

Warning

This review ended early; what follows is what it established.

Adds "did you mean" empty-result hints across the MCP tools, auto-resolves attribute scope, accepts query_data group_by spellings in create_dashboard, and changes query-engine group-by/filter handling for resource attributes. The MCP surface is reasonable, but the hint lookups hide their own failures.

  • emptyResultHints runs facets, attribute-key and span-name lookups on an empty result
  • resolveAttributeScope picks span vs resource from the known keys
  • normalizeGroupBy accepts query_data group_by spellings in create_dashboard
  • Query-engine honors resource-attribute group-by and metric/service scoping

Findings

🔵 Note · F2 · soft hides every hint-lookup failure with no signal

observability · STAT-02 · apps/ai/src/mcp/lib/empty-result-hints.ts:78

Effect.orElseSucceed turns a failed services_facets, attribute-key or span-name lookup into an empty KnownValues, so when the warehouse query breaks the agent still sees a bare "No data found." and operators get nothing: no log, no span event, and the McpTool.emptyResultHints span ends Unset. Wire one Effect.tapError(Effect.logWarning) (or a span event) before the fallback so the swallowed failure is visible.

Add a log or span event that records the error before `orElseSucceed` discards it, e.g. `effect.pipe(Effect.tapError((e) => Effect.logWarning("empty-result hint lookup failed", e)), Effect.orElseSucceed((): KnownValues => ({})))`.
🤖 Prompt to fix this finding with an AI agent
Findings from an automated review of commit 6ecdd214423adca08b572f109efa77029c31413f. Verify each one against the current code before changing anything, fix only those that still apply, and keep each fix to the lines it names.

---

F2 · Note · observability · STAT-02 · apps/ai/src/mcp/lib/empty-result-hints.ts:78
`soft` hides every hint-lookup failure with no signal
`Effect.orElseSucceed` turns a failed `services_facets`, attribute-key or span-name lookup into an empty `KnownValues`, so when the warehouse query breaks the agent still sees a bare "No data found." and operators get nothing: no log, no span event, and the `McpTool.emptyResultHints` span ends Unset. Wire one `Effect.tapError(Effect.logWarning)` (or a span event) before the fallback so the swallowed failure is visible.
Suggested fix: Add a log or span event that records the error before `orElseSucceed` discards it, e.g. `effect.pipe(Effect.tapError((e) => Effect.logWarning("empty-result hint lookup failed", e)), Effect.orElseSucceed((): KnownValues => ({})))`.
What was checked
  • closestMatches scoring on stg/Production/maple-alerting matches its tests (read the algorithm at filter-suggestions.ts:33-60)
  • Facet cap: services.length < FACET_CAP and environments.length < FACET_CAP both come from the 50-row-per-branch services_facets query (services.ts:915-960), so a truncated list yields no absence…
  • hintFor takes the first hint path only against a complete list, and attribute/span-name hints pass complete: false
Observability coverage: 0 of 2 changes observable
Change Kind Observable Evidence
emptyResultHints warehouse lookups (services_facets, attribute keys, span-name breakdown) outbound database query no Effect.withSpan("McpTool.emptyResultHints") at empty-result-hints.ts:92; the individual lookups carry no Client span attributes of their own
emptyResultHints failure fallback error path no soft() = Effect.orElseSucceed at empty-result-hints.ts:78 turns any lookup failure into {} with no log or span event

6ecdd21 · Updated on every push. Reply "won't fix" to dismiss a finding, or mention @maple-review-bot to ask about one. Check ids refer to Maple's instrumentation audit.

@maple-review-bot

maple-review-bot Bot commented Oct 3, 2026 •

Copy link
Copy Markdown

Note

A newer push replaced f8eb861 before its review finished. The latest commit is reviewed in a new comment.

@maple-review-bot

maple-review-bot Bot commented Oct 3, 2026 •

Copy link
Copy Markdown

Note

A newer push replaced 4993eba before its review finished. The latest commit is reviewed in a new comment.

@maple-review-bot

maple-review-bot Bot commented Oct 3, 2026 •

Copy link
Copy Markdown

Maple review

🟡 Confidence 3/5 · needs attention
quality 98/100 · 1 note · tests partial · risk medium · 3/3 new units observable

Warning

This review ended early; what follows is what it established.

Resolves attribute_key to span or resource scope across query_data/search_traces, teaches traces and metrics a resource-attribute group-by, adds an environment filter and service-scoped wrappers to explore_attributes, and adds "did you mean" hints on empty results. The new pipe params are all consumed downstream. One test file, packages/query-engine/src/ch/queries/metrics.test.ts, was not read.

  • normalizeGroupBy accepts query_data's group-by spellings in create-dashboard
  • search_traces gains an environment filter and resource-attribute key resolution
  • serviceScopedAttributeKeysQuery/serviceScopedAttributeValuesQuery back explore_attributes with a service
  • Empty results return emptyHints with close matches for service, environment and attribute key

Still open from earlier reviews

What was checked
  • resource_filter_key/resource_filter_value, deployment_env and resource_attribute_filters are consumed at pipe-dispatch.ts:223, :231, :829
  • explore-attributes.ts:100,141 already forwards service_name, so the service-scoped pipe branches are reachable
  • errors_by_type consumers read deployment_envs (pipe-dispatch.ts:486), matching the find-errors hint params
Observability coverage: 3 of 3 changes observable
Change Kind Observable Evidence
query_data attribute-scope resolution outbound warehouse query yes Effect.fn span in attribute-scope.ts; queries via exploreAttributeKeys/WarehouseExecutor
emptyResultHints lookups outbound warehouse query yes Effect.withSpan("McpTool.emptyResultHints") in empty-result-hints.ts
search_traces environment filter inbound param + warehouse query yes deployment_env consumed at pipe-dispatch.ts:223 and :829
Files not reviewed (1)

The review ended before it read these diffs, so nothing above vouches for them.

  • packages/query-engine/src/ch/queries/metrics.test.ts

6bbb4ea · Updated on every push. Reply "won't fix" to dismiss a finding, or mention @maple-review-bot to ask about one.

@Makisuo
Makisuo merged commit 5c93c91 into main Oct 3, 2026
42 checks passed
@Makisuo
Makisuo deleted the fix/mcp-attribute-scope-matching branch October 3, 2026 23:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant