Skip to content

Document distributed OTel metric tables (#4555) - #4555

Open
thedavekwon wants to merge 5 commits into
meta-pytorch:mainfrom
thedavekwon:export-D114248851
Open

Document distributed OTel metric tables (#4555)#4555
thedavekwon wants to merge 5 commits into
meta-pytorch:mainfrom
thedavekwon:export-D114248851

Conversation

@thedavekwon

@thedavekwon thedavekwon commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Summary:

Explain how the process-global OTel metric provider feeds both the distributed telemetry UDS exporter and external exporters. Document the aggregation-specific metric tables, their typed columns, a delta-counter SQL query, and metric retention. Update the external OTel example to clarify that instrumentation is shared across exporters.

Differential Revision: D114248851

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Meta Open Source bot. label Jul 30, 2026
@meta-codesync

meta-codesync Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

@thedavekwon has exported this pull request. If you are a Meta employee, you can view the originating Diff in D114248851.

)

Summary:

Remove the private `cleanup_callbacks` fixture and its call sites because isolated tests mutate process-global telemetry registration in child processes while fixture teardown ran in the parent. Isolate the remaining public trace test so every test that installs global telemetry state uses subprocess isolation.

Differential Revision: D113848487
Summary:

Have Scuba and OTLP expose `PushMetricExporter` sinks while `otel::init_metrics` owns periodic-reader construction, export interval configuration, backend-provided resource attachment, provider construction, and process-global installation. Retain the first `SdkMeterProvider` in a `OnceLock` so repeated or concurrent initialization cannot replace the provider that existing instruments use. Keep resource identity policy in the existing backend configuration rather than introducing hard-coded host or process attributes.

Differential Revision: D113848483
)

Summary:

Add the bounded-retention `metrics` record-batch schema with separate `f64`, `i64`, and `u64` columns, point and resource attributes, sum temporality and monotonicity, and raw explicit-histogram fields. Register the table with distributed telemetry before ingest starts, cover schema round-tripping and retention, and keep the Python binding documentation aligned with the expanded registration API.

Differential Revision: D113848482
)

Summary:

- Register `UdsMetricExporter` as an independent `PeriodicReader` on the process-global `SdkMeterProvider`, alongside Scuba in Meta builds and OTLP in OSS builds.
- Encode OTel gauges, sums, and explicit histograms into the aggregation-specific `metric_gauges`, `metric_sums`, and `metric_histograms` tables while preserving exact `f64`, `i64`, and `u64` values plus OTel metadata.
- Route each non-empty metric buffer through the lazily activated process-global `UnixSocketSink` as its own nonblocking Arrow frame with existing dropped-frame accounting.
- Cover typed row encoding, all three Unix-socket table routes, and distributed SQL queries against `metric_sums`.

Differential Revision: D113848481
Summary:

Explain how the process-global OTel metric provider feeds both the distributed telemetry UDS exporter and external exporters. Document the aggregation-specific metric tables, their typed columns, a delta-counter SQL query, and metric retention. Update the external OTel example to clarify that instrumentation is shared across exporters.

Differential Revision: D114248851
@meta-codesync meta-codesync Bot changed the title Document distributed OTel metric tables Document distributed OTel metric tables (#4555) Jul 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot. meta-exported

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant