Skip to content

[Feature]: Define DCGM diagnostic telemetry and GPU health tiers #281

Description

@chokevin

User problem

TauGrid operators and researchers need detailed DCGM telemetry for GPU troubleshooting and experiment correlation, including per-link NVLink counters, raw NVLink CRC counters, and power/thermal throttling residency. These fields are valuable, but they are not uniformly available through the AKS-managed DCGM exporter and do not independently indicate unhealthy hardware. Treating them as universal required health fields can create unsupported configuration, excessive metric cardinality, or false unhealthy classifications.

Desired outcome

Establish and document a deliberate GPU metric model that clearly separates:

  • Baseline health fields reliably available from the AKS-managed exporter.
  • Extended health fields requiring an explicitly configured exporter and suitable health semantics.
  • Deep diagnostic and performance telemetry used for dashboards, experiment correlation, and troubleshooting without automatically affecting GPU health.

The resulting contract should define field availability, metric types, cardinality, retention, coverage behavior, GPU-generation applicability, and evidence-based alert thresholds.

Proposed approach

Review the GPU monitoring chart, collector rules, dashboards, and exporter integration end to end. Specifically:

  • Inventory AKS-managed, NVIDIA default, and TauGrid-requested DCGM fields.
  • Evaluate per-link NVLink CRC/data/replay metrics across supported GPU generations and topologies.
  • Determine how power and thermal violation-duration counters should be visualized and correlated with workloads.
  • Separate telemetry collection from health classification and fail-closed metric coverage.
  • Consider optional collector profiles such as baseline, extended-health, and deep-diagnostics rather than one universal CSV.
  • Define model-specific rate or threshold policies before promoting diagnostic counters into alerts.
  • Document how a separately managed exporter can consume optional TauGrid CSV profiles without modifying the AKS-managed exporter.
  • Add focused validation for rendered profiles, metric types, and rule/profile compatibility.

Any implementation should preserve default AKS compatibility and avoid deploying overlapping GPU drivers, device plugins, or host engines.

Alternatives considered

  • Include every available DCGM field in the default CSV. This risks unsupported fields, high cardinality, and inconsistent behavior across GPU generations.
  • Remove the diagnostic fields entirely. This loses useful evidence for troubleshooting and performance analysis.
  • Treat every counter increment as a health failure. This is not appropriate for CRC rates or normal power/thermal throttling behavior without hardware- and workload-specific thresholds.

Confirmation

  • I searched existing issues and roadmap items for this request.
  • This request does not include confidential or security-sensitive information.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions