Add Sonda integration chapter for synthetic network telemetry testing

[davidban77]

Context

Sonda is a synthetic telemetry generator that can replace Containerlab + Telegraf for testing observability and automation pipelines without real (or containerized) network devices. This issue proposes a new chapter that demonstrates Sonda integration with the lab's existing pipeline.

Why

The webinar-completed chapter runs a production-grade automation pipeline:

Containerlab (srl1, srl2) → Telegraf (gNMI/SNMP) → Prometheus → Alertmanager → Webhook → Prefect

Sonda can replace the first two hops — pushing synthetic metrics directly to Prometheus with the same schema Telegraf produces. This enables:

• Testing automation workflows without network devices — no Containerlab, no Telegraf, just Sonda + the automation stack
• Scripted failure scenarios — precise control over BGP session flaps, interface state transitions, and timing
• Faster iteration — no waiting for device boot or gNMI subscription setup
• CI-friendly testing — the entire pipeline becomes testable in GitHub Actions

Analysis: How Sonda Maps to the Lab

Metric Schema Mapping

The lab uses Telegraf-produced metrics with a specific naming/label schema. Sonda needs to match this exactly so alert rules fire unchanged.

Lab Metric	Labels	Sonda Generator	Notes
bgp_admin_state	device, peer_address, afi_safi_name, name	constant (value: 1 = enable)	Typically static during tests
bgp_oper_state	device, peer_address, afi_safi_name, name	sequence (1=up, 2=down, 3=idle, etc.)	Key metric for BgpSessionNotUp alert
bgp_received_routes / sent_routes / active_routes	same	sequence or sawtooth	Route count changes during session flap
interface_oper_state	device, ifName, ifAlias	sequence (up/down)	Interface failure scenarios
interface_in_octets / out_octets	device, ifName, ifAlias	sawtooth (counter ramp)	Traffic counters
device_cpu_percent / device_memory_percent	device	sine	System health

Alert Rules That Fire

The existing prometheus/rules/alerting_rules.yml includes:

- alert: BgpSessionNotUp
  expr: |
    (bgp_admin_state{afi_safi_name="ipv4-unicast", name="default"} == 1)
    and on (device, peer_address, afi_safi_name, name)
    (bgp_oper_state{afi_safi_name="ipv4-unicast", name="default"} != 1)
  for: 30s

Sonda can trigger this by pushing bgp_admin_state=1 (constant) and bgp_oper_state transitioning from 1 to 2 (sequence generator). The Prefect flow receives the alert unchanged.

Prefect Flow Compatibility

The student_flow.py alert_receiver flow extracts device, peer_address, and afi_safi from alert labels and routes to quarantine_bgp_flow(). As long as Sonda's metrics carry the correct labels, the entire Prefect pipeline (evidence collection, policy evaluation, quarantine) runs without modification.

The SoT gate in Nautobot checks if the peer is "intended" — the Sonda scenario's peer_address label needs to match an entry in lab_vars.yml's observability_intent.bgp.intended_peers.

Integration Architecture

Sonda (replaces Containerlab + Telegraf)
  |
  |-- push metrics (http_push or remote_write) -->  Prometheus
  |                                                    |
  |                                              Alert rules evaluate
  |                                                    |
  |                                              Alertmanager
  |                                                    |
  |                                              Webhook (FastAPI :9997)
  |                                                    |
  |                                              Prefect flow: alert_receiver
  |                                                    |
  |                                              Evidence + Policy + Quarantine

Proposed Chapter Content

A new chapter (e.g., chapters/sonda-integration/) containing:

1. docker-compose.yml — stripped-down stack: Sonda + Prometheus + Alertmanager + Webhook + Prefect (no Containerlab, no Telegraf, no Nautobot for simplicity or optionally included)
2. Sonda scenario files — YAML configs producing metrics matching the lab's Telegraf schema (bgp_oper_state, bgp_admin_state, interface counters, etc.)
3. Alert rules — reuse existing alerting_rules.yml (or a subset for BgpSessionNotUp)
4. Walkthrough — step-by-step: start stack, run Sonda scenarios, watch alerts fire, verify Prefect flow triggers quarantine
5. Flap testing — use Sonda's sequence generator to script rapid/slow BGP flaps and validate Prefect handles them correctly
6. CI example — GitHub Actions workflow that spins up the stack, runs Sonda, asserts the Prefect flow executed

References

• Sonda docs: Network Device Telemetry
• Sonda docs: Network Automation Testing (PR: docs: add network automation testing guide (#96) davidban77/sonda#141)
• Sonda docs: Alerting Pipeline
• Lab chapter reference: chapters/webinar-completed/