Pratik Mankawde e7f9685e0e fix(telemetry): poll Prometheus for late-populating metrics in validation ...

The workload validator queried each expected metric once, immediately after
a fixed post-workload propagation wait. Several beast::insight metrics
(ledger-age and peer-finder gauges, overlay-traffic and rpc-request counters)
only populate after the node validates ledgers and sustains peer traffic,
then travel a 1s OTLP export + 15s Prometheus scrape before they are
queryable. On a slower CI runner that pipeline can settle after the wait
ends, so the single query raced and reported "0 series", failing 12 checks
that pass locally with the same config and binary.

Poll each metric on the /api/v1/series endpoint until it appears or a 45s
window (two scrape cycles) elapses. Present metrics still return on the first
query with no added delay; a genuinely-absent metric still fails after the
timeout. Makes the check robust to runner speed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

2026-07-08 12:02:42 +01:00

..

telemetry

fix(telemetry): poll Prometheus for late-populating metrics in validation

2026-07-08 12:02:42 +01:00