Skip to content

feat(analytics): consolidate agent run, token, and cost observability - #469

Merged
esafwan merged 9 commits into
developfrom
consolidation/agent-run-analytics
Jul 26, 2026
Merged

feat(analytics): consolidate agent run, token, and cost observability#469
esafwan merged 9 commits into
developfrom
consolidation/agent-run-analytics

Conversation

@esafwan

@esafwan esafwan commented Jul 26, 2026

Copy link
Copy Markdown
Contributor

Summary

Consolidates the complete execution analytics and usage-observability work from #445 into one reviewable PR.

Included from #445

  • Persist compatibility-safe usage snapshots and cost provenance on terminal Agent Run writes.
  • Add immutable hourly and daily Agent Run Analytics Rollup records.
  • Refresh rollups every five minutes with a correction window.
  • Add portable raw-run and rollup indexes through a post-model-sync migration.
  • Add System Manager aggregate analytics APIs.
  • Add an Executions-page KPI dashboard backed by scheduled rollups.
  • Add prompt-cache tracking and cache economics indicators.
  • Add context composition metrics and the Agent Run context bar.
  • Preserve the original analytics screenshots and test modules.

Design boundaries

  • The frontend reads precomputed rollup buckets and does not aggregate raw runs in the browser.
  • Existing cached_tokens and cost compatibility fields remain intact.
  • The authoritative usage snapshot is null-aware and retains cost provenance.
  • Global rollups are initially System Manager scoped because ownership filtering is not yet safe.

Validation

  • Python compilation passed for all changed backend modules.
  • git diff --check passed.
  • The source PR reported yarn typecheck, production yarn build, migration, analytics tests, and live hourly/daily rollup/API verification.
  • Frontend checks should be rerun by CI for this consolidation branch.

Source PR

Follow-up

  • Preserve the detailed cache/cost analytics rationale and live-probe notes in the CacheAnalytics track documentation.

esafwan added 9 commits July 26, 2026 19:58
- Apply cache_control to conversation history blocks
- Track cache miss/creation tokens
- Add UI indicator for silent cache degradation
- Add Cached Tokens column to Executions list view
Adds segment_tokens (system/tools/knowledge/history/message token
composition) and prefix_breakpoints (per-breakpoint cache prefix hashes)
into the existing usage_snapshot field, plus a compute_run_metrics helper
producing cache read share, effective input multiplier, wasted-writes
tracking, prefix stability, and counterfactual savings. Adds a
get_run_context_metrics API and a reusable ContextBar component wired
into the Agent Run detail page's new Context card.
@esafwan
esafwan merged commit e850d87 into develop Jul 26, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant