Skip to content

Export OTel metrics to distributed telemetry over UDS (#4554) - #4554

Open
thedavekwon wants to merge 4 commits into
meta-pytorch:mainfrom
thedavekwon:export-D113848481
Open

Export OTel metrics to distributed telemetry over UDS (#4554)#4554
thedavekwon wants to merge 4 commits into
meta-pytorch:mainfrom
thedavekwon:export-D113848481

Conversation

@thedavekwon

@thedavekwon thedavekwon commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Summary:

  • Register UdsMetricExporter as an independent PeriodicReader on the process-global SdkMeterProvider, alongside Scuba in Meta builds and OTLP in OSS builds.
  • Encode OTel gauges, sums, and explicit histograms into the aggregation-specific metric_gauges, metric_sums, and metric_histograms tables while preserving exact f64, i64, and u64 values plus OTel metadata.
  • Route each non-empty metric buffer through the lazily activated process-global UnixSocketSink as its own nonblocking Arrow frame with existing dropped-frame accounting.
  • Cover typed row encoding, all three Unix-socket table routes, and distributed SQL queries against metric_sums.

Differential Revision: D113848481

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Meta Open Source bot. label Jul 30, 2026
@meta-codesync

meta-codesync Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

@thedavekwon has exported this pull request. If you are a Meta employee, you can view the originating Diff in D113848481.

)

Summary:

Remove the private `cleanup_callbacks` fixture and its call sites because isolated tests mutate process-global telemetry registration in child processes while fixture teardown ran in the parent. Isolate the remaining public trace test so every test that installs global telemetry state uses subprocess isolation.

Differential Revision: D113848487
Summary:

Have Scuba and OTLP expose `PushMetricExporter` sinks while `otel::init_metrics` owns periodic-reader construction, export interval configuration, backend-provided resource attachment, provider construction, and process-global installation. Retain the first `SdkMeterProvider` in a `OnceLock` so repeated or concurrent initialization cannot replace the provider that existing instruments use. Keep resource identity policy in the existing backend configuration rather than introducing hard-coded host or process attributes.

Differential Revision: D113848483
)

Summary:

Add the bounded-retention `metrics` record-batch schema with separate `f64`, `i64`, and `u64` columns, point and resource attributes, sum temporality and monotonicity, and raw explicit-histogram fields. Register the table with distributed telemetry before ingest starts, cover schema round-tripping and retention, and keep the Python binding documentation aligned with the expanded registration API.

Differential Revision: D113848482
)

Summary:

- Register `UdsMetricExporter` as an independent `PeriodicReader` on the process-global `SdkMeterProvider`, alongside Scuba in Meta builds and OTLP in OSS builds.
- Encode OTel gauges, sums, and explicit histograms into the aggregation-specific `metric_gauges`, `metric_sums`, and `metric_histograms` tables while preserving exact `f64`, `i64`, and `u64` values plus OTel metadata.
- Route each non-empty metric buffer through the lazily activated process-global `UnixSocketSink` as its own nonblocking Arrow frame with existing dropped-frame accounting.
- Cover typed row encoding, all three Unix-socket table routes, and distributed SQL queries against `metric_sums`.

Differential Revision: D113848481
@meta-codesync meta-codesync Bot changed the title Export OTel metrics to distributed telemetry over UDS Export OTel metrics to distributed telemetry over UDS (#4554) Jul 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot. meta-exported

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant