diff --git a/.codecov.yml b/.codecov.yml index 3d9d2734e8..1fbab1ea32 100644 --- a/.codecov.yml +++ b/.codecov.yml @@ -39,5 +39,3 @@ ignore: # Telemetry modules — conditionally compiled behind XRPL_ENABLE_TELEMETRY, # which is not enabled in coverage builds. - "src/xrpld/telemetry/" - - "src/libxrpl/beast/insight/OTelCollector.cpp" - - "include/xrpl/beast/insight/OTelCollector.h" diff --git a/OpenTelemetryPlan/09-data-collection-reference.md b/OpenTelemetryPlan/09-data-collection-reference.md index fe39dd6ba3..4199cecc01 100644 --- a/OpenTelemetryPlan/09-data-collection-reference.md +++ b/OpenTelemetryPlan/09-data-collection-reference.md @@ -42,7 +42,7 @@ graph LR BP -->|"OTLP/gRPC :4317"| D SM -->|"span_calls_total
span_duration_ms
(6 dimension labels)"| E - R2 -->|"xrpld_* gauges
xrpld_* counters
xrpld_* summaries"| E + R2 -->|"rippled_* gauges
rippled_* counters
rippled_* summaries"| E E -->|"Prometheus
data source"| F D -->|"Tempo
data source"| F @@ -400,56 +400,57 @@ These are system-level metrics emitted by xrpld's `beast::insight` framework via [insight] server=statsd address=127.0.0.1:8125 -prefix=xrpld +prefix=rippled ``` +> **Note**: The `prefix` value is user-configurable — all metric names in the tables below assume `prefix=rippled` (matching the integration test and Grafana dashboards). If you change the prefix, replace `rippled_` with `{your_prefix}_` in all PromQL queries. + ### 2.1 Gauges -| Prometheus Metric | Source File | Description | Typical Range | -| ------------------------------------------------- | --------------------- | ----------------------------------------- | ------------------------------- | -| `xrpld_LedgerMaster_Validated_Ledger_Age` | LedgerMaster.h | Seconds since last validated ledger | 0–10 (healthy), >30 (stale) | -| `xrpld_LedgerMaster_Published_Ledger_Age` | LedgerMaster.h | Seconds since last published ledger | 0–10 (healthy) | -| `xrpld_State_Accounting_Disconnected_duration` | NetworkOPs.cpp | Cumulative seconds in Disconnected state | Monotonic | -| `xrpld_State_Accounting_Connected_duration` | NetworkOPs.cpp | Cumulative seconds in Connected state | Monotonic | -| `xrpld_State_Accounting_Syncing_duration` | NetworkOPs.cpp | Cumulative seconds in Syncing state | Monotonic | -| `xrpld_State_Accounting_Tracking_duration` | NetworkOPs.cpp | Cumulative seconds in Tracking state | Monotonic | -| `xrpld_State_Accounting_Full_duration` | NetworkOPs.cpp | Cumulative seconds in Full state | Monotonic (should dominate) | -| `xrpld_State_Accounting_Disconnected_transitions` | NetworkOPs.cpp | Count of transitions to Disconnected | Low | -| `xrpld_State_Accounting_Connected_transitions` | NetworkOPs.cpp | Count of transitions to Connected | Low | -| `xrpld_State_Accounting_Syncing_transitions` | NetworkOPs.cpp | Count of transitions to Syncing | Low | -| `xrpld_State_Accounting_Tracking_transitions` | NetworkOPs.cpp | Count of transitions to Tracking | Low | -| `xrpld_State_Accounting_Full_transitions` | NetworkOPs.cpp | Count of transitions to Full | Low (should be 1 after startup) | -| `xrpld_Peer_Finder_Active_Inbound_Peers` | PeerfinderManager.cpp | Active inbound peer connections | 0–85 | -| `xrpld_Peer_Finder_Active_Outbound_Peers` | PeerfinderManager.cpp | Active outbound peer connections | 10–21 | -| `xrpld_Overlay_Peer_Disconnects` | OverlayImpl.cpp | Cumulative peer disconnection count | Low growth | -| `xrpld_Overlay_Peer_Disconnects_Charges` | OverlayImpl.cpp | Disconnects due to resource limit charges | Low growth (subset of above) | -| `xrpld_job_count` | JobQueue.cpp | Current job queue depth | 0–100 (healthy) | +| Prometheus Metric | Source File | Description | Typical Range | +| --------------------------------------------------- | --------------------- | ---------------------------------------- | ------------------------------- | +| `rippled_LedgerMaster_Validated_Ledger_Age` | LedgerMaster.h | Seconds since last validated ledger | 0–10 (healthy), >30 (stale) | +| `rippled_LedgerMaster_Published_Ledger_Age` | LedgerMaster.h | Seconds since last published ledger | 0–10 (healthy) | +| `rippled_State_Accounting_Disconnected_duration` | NetworkOPs.cpp | Cumulative seconds in Disconnected state | Monotonic | +| `rippled_State_Accounting_Connected_duration` | NetworkOPs.cpp | Cumulative seconds in Connected state | Monotonic | +| `rippled_State_Accounting_Syncing_duration` | NetworkOPs.cpp | Cumulative seconds in Syncing state | Monotonic | +| `rippled_State_Accounting_Tracking_duration` | NetworkOPs.cpp | Cumulative seconds in Tracking state | Monotonic | +| `rippled_State_Accounting_Full_duration` | NetworkOPs.cpp | Cumulative seconds in Full state | Monotonic (should dominate) | +| `rippled_State_Accounting_Disconnected_transitions` | NetworkOPs.cpp | Count of transitions to Disconnected | Low | +| `rippled_State_Accounting_Connected_transitions` | NetworkOPs.cpp | Count of transitions to Connected | Low | +| `rippled_State_Accounting_Syncing_transitions` | NetworkOPs.cpp | Count of transitions to Syncing | Low | +| `rippled_State_Accounting_Tracking_transitions` | NetworkOPs.cpp | Count of transitions to Tracking | Low | +| `rippled_State_Accounting_Full_transitions` | NetworkOPs.cpp | Count of transitions to Full | Low (should be 1 after startup) | +| `rippled_Peer_Finder_Active_Inbound_Peers` | PeerfinderManager.cpp | Active inbound peer connections | 0–85 | +| `rippled_Peer_Finder_Active_Outbound_Peers` | PeerfinderManager.cpp | Active outbound peer connections | 10–21 | +| `rippled_Overlay_Peer_Disconnects` | OverlayImpl.cpp | Cumulative peer disconnection count | Low growth | +| `rippled_job_count` | JobQueue.cpp | Current job queue depth | 0–100 (healthy) | **Grafana dashboard**: _Node Health (StatsD)_ (`xrpld-statsd-node-health`) ### 2.2 Counters -| Prometheus Metric | Source File | Description | -| ------------------------------- | ------------------ | --------------------------------------------- | -| `xrpld_rpc_requests` | ServerHandler.cpp | Total RPC requests received | -| `xrpld_ledger_fetches` | InboundLedgers.cpp | Inbound ledger fetch attempts | -| `xrpld_ledger_history_mismatch` | LedgerHistory.cpp | Ledger hash mismatches detected | -| `xrpld_warn` | Logic.h | Resource manager warnings issued | -| `xrpld_drop` | Logic.h | Resource manager drops (connections rejected) | +| Prometheus Metric | Source File | Description | +| --------------------------------- | ------------------ | --------------------------------------------- | +| `rippled_rpc_requests` | ServerHandler.cpp | Total RPC requests received | +| `rippled_ledger_fetches` | InboundLedgers.cpp | Inbound ledger fetch attempts | +| `rippled_ledger_history_mismatch` | LedgerHistory.cpp | Ledger hash mismatches detected | +| `rippled_warn` | Logic.h | Resource manager warnings issued | +| `rippled_drop` | Logic.h | Resource manager drops (connections rejected) | -**Note**: `xrpld_warn` and `xrpld_drop` use non-standard StatsD meter type (`|m`). The OTel StatsD receiver only recognizes `|c`, `|g`, `|ms`, `|h`, `|s` — these metrics may be silently dropped. See Known Issues below. +**Note**: `rippled_warn` and `rippled_drop` use non-standard StatsD meter type (`|m`). The OTel StatsD receiver only recognizes `|c`, `|g`, `|ms`, `|h`, `|s` — these metrics may be silently dropped. See Known Issues below. **Grafana dashboard**: _RPC & Pathfinding (StatsD)_ (`xrpld-statsd-rpc`) ### 2.3 Histograms (from StatsD timers) -| Prometheus Metric | Source File | Unit | Description | -| --------------------- | ----------------- | ----- | ------------------------------ | -| `xrpld_rpc_time` | ServerHandler.cpp | ms | RPC response time distribution | -| `xrpld_rpc_size` | ServerHandler.cpp | bytes | RPC response size distribution | -| `xrpld_ios_latency` | Application.cpp | ms | I/O service loop latency | -| `xrpld_pathfind_fast` | PathRequests.h | ms | Fast pathfinding duration | -| `xrpld_pathfind_full` | PathRequests.h | ms | Full pathfinding duration | +| Prometheus Metric | Source File | Unit | Description | +| ----------------------- | ----------------- | ----- | ------------------------------ | +| `rippled_rpc_time` | ServerHandler.cpp | ms | RPC response time distribution | +| `rippled_rpc_size` | ServerHandler.cpp | bytes | RPC response size distribution | +| `rippled_ios_latency` | Application.cpp | ms | I/O service loop latency | +| `rippled_pathfind_fast` | PathRequests.h | ms | Fast pathfinding duration | +| `rippled_pathfind_full` | PathRequests.h | ms | Full pathfinding duration | Quantiles collected: 0th, 50th, 90th, 95th, 99th, 100th percentile. @@ -459,10 +460,10 @@ Quantiles collected: 0th, 50th, 90th, 95th, 99th, 100th percentile. For each of the 45+ overlay traffic categories (defined in `TrafficCount.h`), four gauges are emitted: -- `xrpld_{category}_Bytes_In` -- `xrpld_{category}_Bytes_Out` -- `xrpld_{category}_Messages_In` -- `xrpld_{category}_Messages_Out` +- `rippled_{category}_Bytes_In` +- `rippled_{category}_Bytes_Out` +- `rippled_{category}_Messages_In` +- `rippled_{category}_Messages_Out` **Key categories**: @@ -614,19 +615,19 @@ sum by (xrpl_peer_proposal_trusted) (rate(traces_span_metrics_calls_total{span_n ```promql # Validated ledger age (should be < 10s) -xrpld_LedgerMaster_Validated_Ledger_Age +rippled_LedgerMaster_Validated_Ledger_Age # Active peer count -xrpld_Peer_Finder_Active_Inbound_Peers + xrpld_Peer_Finder_Active_Outbound_Peers +rippled_Peer_Finder_Active_Inbound_Peers + rippled_Peer_Finder_Active_Outbound_Peers # RPC response time p95 -histogram_quantile(0.95, xrpld_rpc_time_bucket) +histogram_quantile(0.95, rippled_rpc_time_bucket) # Total network bytes in (rate) -rate(xrpld_total_Bytes_In[5m]) +rate(rippled_total_Bytes_In[5m]) # Operating mode (should be "Full" after startup) -xrpld_State_Accounting_Full_duration +rippled_State_Accounting_Full_duration ``` --- @@ -655,8 +656,8 @@ All span names and attributes are defined as compile-time constants in colocated | Issue | Impact | Status | | ------------------------------------------------------------------ | ------------------------------------------------ | -------------------------------------------------------------------- | | `warn` and `drop` metrics use non-standard StatsD `\|m` meter type | Metrics silently dropped by OTel StatsD receiver | Phase 6 Task 6.1 — needs `\|m` → `\|c` change in StatsDCollector.cpp | -| `xrpld_job_count` may not emit in standalone mode | Missing from Prometheus in some test configs | Requires active job queue activity | -| `xrpld_rpc_requests` depends on `[insight]` config | Zero series if StatsD not configured | Requires `[insight] server=statsd` in xrpld.cfg | +| `rippled_job_count` may not emit in standalone mode | Missing from Prometheus in some test configs | Requires active job queue activity | +| `rippled_rpc_requests` depends on `[insight]` config | Zero series if StatsD not configured | Requires `[insight] server=statsd` in xrpld.cfg | | Peer tracing disabled by default | No `peer.*` spans unless `trace_peer=1` | Intentional — high volume on mainnet | --- @@ -688,7 +689,7 @@ enabled=1 [insight] server=statsd address=127.0.0.1:8125 -prefix=xrpld +prefix=rippled ``` ### Production Setup @@ -705,7 +706,7 @@ max_queue_size=4096 [insight] server=statsd address=otel-collector:8125 -prefix=xrpld +prefix=rippled ``` ### Trace Category Toggle diff --git a/docker/telemetry/otel-collector-config.yaml b/docker/telemetry/otel-collector-config.yaml index 297f673560..bfe782ffd5 100644 --- a/docker/telemetry/otel-collector-config.yaml +++ b/docker/telemetry/otel-collector-config.yaml @@ -2,11 +2,15 @@ # # Pipelines: # traces: OTLP receiver -> batch processor -> debug + Tempo + spanmetrics -# metrics: spanmetrics connector -> Prometheus exporter +# metrics: StatsD receiver + spanmetrics connector -> Prometheus exporter # # xrpld sends traces via OTLP/HTTP to port 4318. The collector batches # them, forwards to Tempo, and derives RED metrics via the spanmetrics # connector, which Prometheus scrapes on port 8889. +# +# xrpld's beast::insight framework sends StatsD UDP metrics to port 8125. +# The StatsD receiver aggregates them and exports to Prometheus alongside +# the span-derived metrics. receivers: otlp: @@ -15,6 +19,20 @@ receivers: endpoint: 0.0.0.0:4317 http: endpoint: 0.0.0.0:4318 + statsd: + endpoint: "0.0.0.0:8125" + aggregation_interval: 15s + enable_metric_type: true + is_monotonic_counter: true + timer_histogram_mapping: + - statsd_type: "timing" + observer_type: "summary" + summary: + percentiles: [0, 50, 90, 95, 99, 100] + - statsd_type: "histogram" + observer_type: "summary" + summary: + percentiles: [0, 50, 90, 95, 99, 100] processors: batch: @@ -62,5 +80,5 @@ service: processors: [batch] exporters: [debug, otlp/tempo, spanmetrics] metrics: - receivers: [spanmetrics] + receivers: [spanmetrics, statsd] exporters: [prometheus] diff --git a/docs/telemetry-runbook.md b/docs/telemetry-runbook.md index e8bdccd1e1..f15b4bdf80 100644 --- a/docs/telemetry-runbook.md +++ b/docs/telemetry-runbook.md @@ -207,7 +207,7 @@ Add to `xrpld.cfg`: [insight] server=statsd address=127.0.0.1:8125 -prefix=xrpld +prefix=rippled ``` The OTel Collector receives these via a `statsd` receiver on UDP port 8125 and exports them to Prometheus alongside spanmetrics. @@ -216,38 +216,38 @@ The OTel Collector receives these via a `statsd` receiver on UDP port 8125 and e #### Gauges -| Prometheus Metric | Source | Description | -| ------------------------------------------- | ------------------------- | -------------------------------------------------------------------------- | -| `xrpld_LedgerMaster_Validated_Ledger_Age` | LedgerMaster.h:373 | Age of validated ledger (seconds) | -| `xrpld_LedgerMaster_Published_Ledger_Age` | LedgerMaster.h:374 | Age of published ledger (seconds) | -| `xrpld_State_Accounting_{Mode}_duration` | NetworkOPs.cpp:774 | Time in each operating mode (Disconnected/Connected/Syncing/Tracking/Full) | -| `xrpld_State_Accounting_{Mode}_transitions` | NetworkOPs.cpp:780 | Transition count per mode | -| `xrpld_Peer_Finder_Active_Inbound_Peers` | PeerfinderManager.cpp:214 | Active inbound peer connections | -| `xrpld_Peer_Finder_Active_Outbound_Peers` | PeerfinderManager.cpp:215 | Active outbound peer connections | -| `xrpld_Overlay_Peer_Disconnects` | OverlayImpl.h:557 | Peer disconnect count | -| `xrpld_job_count` | JobQueue.cpp:26 | Current job queue depth | -| `xrpld_{category}_Bytes_In/Out` | OverlayImpl.h:535 | Overlay traffic bytes per category (57 categories) | -| `xrpld_{category}_Messages_In/Out` | OverlayImpl.h:535 | Overlay traffic messages per category | +| Prometheus Metric | Source | Description | +| --------------------------------------------- | ------------------------- | -------------------------------------------------------------------------- | +| `rippled_LedgerMaster_Validated_Ledger_Age` | LedgerMaster.h:373 | Age of validated ledger (seconds) | +| `rippled_LedgerMaster_Published_Ledger_Age` | LedgerMaster.h:374 | Age of published ledger (seconds) | +| `rippled_State_Accounting_{Mode}_duration` | NetworkOPs.cpp:774 | Time in each operating mode (Disconnected/Connected/Syncing/Tracking/Full) | +| `rippled_State_Accounting_{Mode}_transitions` | NetworkOPs.cpp:780 | Transition count per mode | +| `rippled_Peer_Finder_Active_Inbound_Peers` | PeerfinderManager.cpp:214 | Active inbound peer connections | +| `rippled_Peer_Finder_Active_Outbound_Peers` | PeerfinderManager.cpp:215 | Active outbound peer connections | +| `rippled_Overlay_Peer_Disconnects` | OverlayImpl.h:557 | Peer disconnect count | +| `rippled_job_count` | JobQueue.cpp:26 | Current job queue depth | +| `rippled_{category}_Bytes_In/Out` | OverlayImpl.h:535 | Overlay traffic bytes per category (57 categories) | +| `rippled_{category}_Messages_In/Out` | OverlayImpl.h:535 | Overlay traffic messages per category | #### Counters -| Prometheus Metric | Source | Description | -| ------------------------------- | --------------------- | ------------------------------ | -| `xrpld_rpc_requests` | ServerHandler.cpp:108 | Total RPC request count | -| `xrpld_ledger_fetches` | InboundLedgers.cpp:44 | Ledger fetch request count | -| `xrpld_ledger_history_mismatch` | LedgerHistory.cpp:16 | Ledger hash mismatch count | -| `xrpld_warn` | Logic.h:33 | Resource manager warning count | -| `xrpld_drop` | Logic.h:34 | Resource manager drop count | +| Prometheus Metric | Source | Description | +| --------------------------------- | --------------------- | ------------------------------ | +| `rippled_rpc_requests` | ServerHandler.cpp:108 | Total RPC request count | +| `rippled_ledger_fetches` | InboundLedgers.cpp:44 | Ledger fetch request count | +| `rippled_ledger_history_mismatch` | LedgerHistory.cpp:16 | Ledger hash mismatch count | +| `rippled_warn` | Logic.h:33 | Resource manager warning count | +| `rippled_drop` | Logic.h:34 | Resource manager drop count | #### Histograms (from StatsD timers) -| Prometheus Metric | Source | Description | -| --------------------- | --------------------- | ------------------------------ | -| `xrpld_rpc_time` | ServerHandler.cpp:110 | RPC response time (ms) | -| `xrpld_rpc_size` | ServerHandler.cpp:109 | RPC response size (bytes) | -| `xrpld_ios_latency` | Application.cpp:438 | I/O service loop latency (ms) | -| `xrpld_pathfind_fast` | PathRequests.h:23 | Fast pathfinding duration (ms) | -| `xrpld_pathfind_full` | PathRequests.h:24 | Full pathfinding duration (ms) | +| Prometheus Metric | Source | Description | +| ----------------------- | --------------------- | ------------------------------ | +| `rippled_rpc_time` | ServerHandler.cpp:110 | RPC response time (ms) | +| `rippled_rpc_size` | ServerHandler.cpp:109 | RPC response size (bytes) | +| `rippled_ios_latency` | Application.cpp:438 | I/O service loop latency (ms) | +| `rippled_pathfind_fast` | PathRequests.h:23 | Fast pathfinding duration (ms) | +| `rippled_pathfind_full` | PathRequests.h:24 | Full pathfinding duration (ms) | ## Grafana Dashboards @@ -320,42 +320,42 @@ Requires `trace_peer=1` in the `[telemetry]` config section. ### Node Health — StatsD (`xrpld-statsd-node-health`) -| Panel | Type | PromQL | Labels Used | -| -------------------------- | ---------- | ---------------------------------------------------- | ----------- | -| Validated Ledger Age | stat | `xrpld_LedgerMaster_Validated_Ledger_Age` | — | -| Published Ledger Age | stat | `xrpld_LedgerMaster_Published_Ledger_Age` | — | -| Operating Mode Duration | timeseries | `xrpld_State_Accounting_*_duration` | — | -| Operating Mode Transitions | timeseries | `xrpld_State_Accounting_*_transitions` | — | -| I/O Latency | timeseries | `histogram_quantile(0.95, xrpld_ios_latency_bucket)` | — | -| Job Queue Depth | timeseries | `xrpld_job_count` | — | -| Ledger Fetch Rate | stat | `rate(xrpld_ledger_fetches[5m])` | — | -| Ledger History Mismatches | stat | `rate(xrpld_ledger_history_mismatch[5m])` | — | +| Panel | Type | PromQL | Labels Used | +| -------------------------- | ---------- | ------------------------------------------------------ | ----------- | +| Validated Ledger Age | stat | `rippled_LedgerMaster_Validated_Ledger_Age` | — | +| Published Ledger Age | stat | `rippled_LedgerMaster_Published_Ledger_Age` | — | +| Operating Mode Duration | timeseries | `rippled_State_Accounting_*_duration` | — | +| Operating Mode Transitions | timeseries | `rippled_State_Accounting_*_transitions` | — | +| I/O Latency | timeseries | `histogram_quantile(0.95, rippled_ios_latency_bucket)` | — | +| Job Queue Depth | timeseries | `rippled_job_count` | — | +| Ledger Fetch Rate | stat | `rate(rippled_ledger_fetches[5m])` | — | +| Ledger History Mismatches | stat | `rate(rippled_ledger_history_mismatch[5m])` | — | ### Network Traffic — StatsD (`xrpld-statsd-network`) -| Panel | Type | PromQL | Labels Used | -| ---------------------- | ---------- | ------------------------------------ | ----------- | -| Active Peers | timeseries | `xrpld_Peer_Finder_Active_*_Peers` | — | -| Peer Disconnects | timeseries | `xrpld_Overlay_Peer_Disconnects` | — | -| Total Network Bytes | timeseries | `xrpld_total_Bytes_In/Out` | — | -| Total Network Messages | timeseries | `xrpld_total_Messages_In/Out` | — | -| Transaction Traffic | timeseries | `xrpld_transactions_Messages_In/Out` | — | -| Proposal Traffic | timeseries | `xrpld_proposals_Messages_In/Out` | — | -| Validation Traffic | timeseries | `xrpld_validations_Messages_In/Out` | — | -| Traffic by Category | bargauge | `topk(10, xrpld_*_Bytes_In)` | — | +| Panel | Type | PromQL | Labels Used | +| ---------------------- | ---------- | -------------------------------------- | ----------- | +| Active Peers | timeseries | `rippled_Peer_Finder_Active_*_Peers` | — | +| Peer Disconnects | timeseries | `rippled_Overlay_Peer_Disconnects` | — | +| Total Network Bytes | timeseries | `rippled_total_Bytes_In/Out` | — | +| Total Network Messages | timeseries | `rippled_total_Messages_In/Out` | — | +| Transaction Traffic | timeseries | `rippled_transactions_Messages_In/Out` | — | +| Proposal Traffic | timeseries | `rippled_proposals_Messages_In/Out` | — | +| Validation Traffic | timeseries | `rippled_validations_Messages_In/Out` | — | +| Traffic by Category | bargauge | `topk(10, rippled_*_Bytes_In)` | — | ### RPC & Pathfinding — StatsD (`xrpld-statsd-rpc`) -| Panel | Type | PromQL | Labels Used | -| ------------------------- | ---------- | ------------------------------------------------------ | ----------- | -| RPC Request Rate | stat | `rate(xrpld_rpc_requests[5m])` | — | -| RPC Response Time | timeseries | `histogram_quantile(0.95, xrpld_rpc_time_bucket)` | — | -| RPC Response Size | timeseries | `histogram_quantile(0.95, xrpld_rpc_size_bucket)` | — | -| RPC Response Time Heatmap | heatmap | `xrpld_rpc_time_bucket` | — | -| Pathfinding Fast Duration | timeseries | `histogram_quantile(0.95, xrpld_pathfind_fast_bucket)` | — | -| Pathfinding Full Duration | timeseries | `histogram_quantile(0.95, xrpld_pathfind_full_bucket)` | — | -| Resource Warnings Rate | stat | `rate(xrpld_warn[5m])` | — | -| Resource Drops Rate | stat | `rate(xrpld_drop[5m])` | — | +| Panel | Type | PromQL | Labels Used | +| ------------------------- | ---------- | -------------------------------------------------------- | ----------- | +| RPC Request Rate | stat | `rate(rippled_rpc_requests[5m])` | — | +| RPC Response Time | timeseries | `histogram_quantile(0.95, rippled_rpc_time_bucket)` | — | +| RPC Response Size | timeseries | `histogram_quantile(0.95, rippled_rpc_size_bucket)` | — | +| RPC Response Time Heatmap | heatmap | `rippled_rpc_time_bucket` | — | +| Pathfinding Fast Duration | timeseries | `histogram_quantile(0.95, rippled_pathfind_fast_bucket)` | — | +| Pathfinding Full Duration | timeseries | `histogram_quantile(0.95, rippled_pathfind_full_bucket)` | — | +| Resource Warnings Rate | stat | `rate(rippled_warn[5m])` | — | +| Resource Drops Rate | stat | `rate(rippled_drop[5m])` | — | ### Span → Metric → Dashboard Summary