feat(telemetry): correct ledger-close-time derivation on dashboards

The "ledger close time" was mis-derived and partly un-queryable:

- Build vs Close Duration derived close time from the consensus.ledger_close
  span, which only wraps the onClose() prologue (~0.8ms live) — not the close.
  Repoint the close series to consensus.round (full round, live P95 ~4.8s).
- The network close-time value (close_time) lived only as a span attribute,
  un-queryable in Prometheus and unfit as a spanmetrics label (monotonic
  timestamp -> unbounded cardinality). Expose it as last_close_time on the
  existing server_info observable gauge (native gauge, no new instrument).
- Add a "Ledger Close Interval & Age" panel to ledger-operations and
  node-health: interval = 1/rate(ledgers_closed_total) (counter-based,
  scrape-independent); age = time() - (last_close_time + epoch_offset).

A gauge delta is deliberately NOT used for the interval — a timestamp gauge's
delta aliases to the scrape period, not the close cadence (verified live).
Guardrail comments in both collector configs record why close_time must never
become a spanmetrics dimension. Docs (09-reference) and the operator runbook
updated to match.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Pratik Mankawde
2026-07-22 12:23:21 +01:00
parent 1f98f2c46a
commit f52673b6c1
7 changed files with 230 additions and 44 deletions

View File

@@ -848,6 +848,22 @@ MetricsRegistry::registerServerInfoGauge()
"last_close_converge_time_ms",
static_cast<int64_t>(consensusInfo["previous_mseconds"].asUInt()));
}
// Network close time of the last closed ledger, as NetClock
// seconds since the Ripple epoch (2000-01-01). Unlike a span
// timestamp, a gauge value survives as a queryable time series,
// so dashboards can show last-close age (staleness) via
// now - value. The close interval comes from the
// ledgers_closed_total counter, not a delta of this gauge
// (a timestamp gauge's delta aliases to the scrape period).
// Skip until a ledger has closed.
if (auto const closed = app.getLedgerMaster().getClosedLedger())
{
observe(
"last_close_time",
static_cast<int64_t>(
closed->header().closeTime.time_since_epoch().count()));
}
}
catch (...) // NOLINT(bugprone-empty-catch)
{