diff --git a/.cspell.config.yaml b/.cspell.config.yaml index b94bca489f..854992e9b8 100644 --- a/.cspell.config.yaml +++ b/.cspell.config.yaml @@ -135,6 +135,7 @@ words: - godexsoft - gpgcheck - gpgkey + - havetxset - hotwallet - hicpp - htpasswd diff --git a/OpenTelemetryPlan/09-data-collection-reference.md b/OpenTelemetryPlan/09-data-collection-reference.md index 41f2a506ca..27a5f3a19e 100644 --- a/OpenTelemetryPlan/09-data-collection-reference.md +++ b/OpenTelemetryPlan/09-data-collection-reference.md @@ -41,7 +41,7 @@ graph LR BP -->|"OTLP/gRPC :4317"| D SM -->|"span_calls_total
span_duration_ms
(6 dimension labels)"| E - R1 -->|"xrpld_* gauges
xrpld_* counters
xrpld_* histograms"| E + R1 -->|"* gauges
* counters
* histograms"| E E -->|"Prometheus
data source"| F D -->|"Tempo
data source"| F @@ -418,12 +418,12 @@ the parent `ledger.build` carries `ledger_seq` and the close-time attributes. The OTel Collector's SpanMetrics connector automatically generates RED (Rate, Errors, Duration) metrics from every span. No custom metrics code in xrpld is needed. -| Prometheus Metric | Type | Description | -| -------------------------------------------------- | --------- | ------------------------------------------------------------------------------ | -| `traces_span_metrics_calls_total` | Counter | Total span invocations | -| `traces_span_metrics_duration_milliseconds_bucket` | Histogram | Latency distribution (buckets: 1, 5, 10, 25, 50, 100, 250, 500, 1000, 5000 ms) | -| `traces_span_metrics_duration_milliseconds_count` | Histogram | Observation count | -| `traces_span_metrics_duration_milliseconds_sum` | Histogram | Cumulative latency | +| Prometheus Metric | Type | Description | +| ----------------------------------- | --------- | ------------------------------------------------------------------------------ | +| `span_calls_total` | Counter | Total span invocations | +| `span_duration_milliseconds_bucket` | Histogram | Latency distribution (buckets: 1, 5, 10, 25, 50, 100, 250, 500, 1000, 5000 ms) | +| `span_duration_milliseconds_count` | Histogram | Observation count | +| `span_duration_milliseconds_sum` | Histogram | Cumulative latency | **Standard labels on every metric**: `span_name`, `status_code`, `service_name`, `span_kind` @@ -466,7 +466,7 @@ _Transaction Overview_ dashboard charts rate, p95 latency, and failure rate by s > the retained traces, whereas native StatsD/meter metrics do not sample. > Account for any collector-side tail sampling when reading absolute stage rates. -**Where to query**: Prometheus → `traces_span_metrics_calls_total{span_name="rpc.command.server_info"}` +**Where to query**: Prometheus → `span_calls_total{span_name="rpc.command.server_info"}` --- @@ -499,51 +499,51 @@ prefix=xrpld ### 2.1 Gauges -| Prometheus Metric | Source File | Description | Typical Range | -| ------------------------------------------------- | --------------------- | ----------------------------------------- | ------------------------------- | -| `xrpld_LedgerMaster_Validated_Ledger_Age` | LedgerMaster.h | Seconds since last validated ledger | 0–10 (healthy), >30 (stale) | -| `xrpld_LedgerMaster_Published_Ledger_Age` | LedgerMaster.h | Seconds since last published ledger | 0–10 (healthy) | -| `xrpld_State_Accounting_Disconnected_duration` | NetworkOPs.cpp | Cumulative seconds in Disconnected state | Monotonic | -| `xrpld_State_Accounting_Connected_duration` | NetworkOPs.cpp | Cumulative seconds in Connected state | Monotonic | -| `xrpld_State_Accounting_Syncing_duration` | NetworkOPs.cpp | Cumulative seconds in Syncing state | Monotonic | -| `xrpld_State_Accounting_Tracking_duration` | NetworkOPs.cpp | Cumulative seconds in Tracking state | Monotonic | -| `xrpld_State_Accounting_Full_duration` | NetworkOPs.cpp | Cumulative seconds in Full state | Monotonic (should dominate) | -| `xrpld_State_Accounting_Disconnected_transitions` | NetworkOPs.cpp | Count of transitions to Disconnected | Low | -| `xrpld_State_Accounting_Connected_transitions` | NetworkOPs.cpp | Count of transitions to Connected | Low | -| `xrpld_State_Accounting_Syncing_transitions` | NetworkOPs.cpp | Count of transitions to Syncing | Low | -| `xrpld_State_Accounting_Tracking_transitions` | NetworkOPs.cpp | Count of transitions to Tracking | Low | -| `xrpld_State_Accounting_Full_transitions` | NetworkOPs.cpp | Count of transitions to Full | Low (should be 1 after startup) | -| `xrpld_Peer_Finder_Active_Inbound_Peers` | PeerfinderManager.cpp | Active inbound peer connections | 0–85 | -| `xrpld_Peer_Finder_Active_Outbound_Peers` | PeerfinderManager.cpp | Active outbound peer connections | 10–21 | -| `xrpld_Overlay_Peer_Disconnects` | OverlayImpl.cpp | Cumulative peer disconnection count | Low growth | -| `xrpld_Overlay_Peer_Disconnects_Charges` | OverlayImpl.cpp | Disconnects due to resource limit charges | Low growth (subset of above) | -| `xrpld_job_count` | JobQueue.cpp | Current job queue depth | 0–100 (healthy) | +| Prometheus Metric | Source File | Description | Typical Range | +| ------------------------------------------- | --------------------- | ----------------------------------------- | ------------------------------- | +| `ledgermaster_validated_ledger_age` | LedgerMaster.h | Seconds since last validated ledger | 0–10 (healthy), >30 (stale) | +| `ledgermaster_published_ledger_age` | LedgerMaster.h | Seconds since last published ledger | 0–10 (healthy) | +| `state_accounting_disconnected_duration` | NetworkOPs.cpp | Cumulative seconds in Disconnected state | Monotonic | +| `state_accounting_connected_duration` | NetworkOPs.cpp | Cumulative seconds in Connected state | Monotonic | +| `state_accounting_syncing_duration` | NetworkOPs.cpp | Cumulative seconds in Syncing state | Monotonic | +| `state_accounting_tracking_duration` | NetworkOPs.cpp | Cumulative seconds in Tracking state | Monotonic | +| `state_accounting_full_duration` | NetworkOPs.cpp | Cumulative seconds in Full state | Monotonic (should dominate) | +| `state_accounting_disconnected_transitions` | NetworkOPs.cpp | Count of transitions to Disconnected | Low | +| `state_accounting_connected_transitions` | NetworkOPs.cpp | Count of transitions to Connected | Low | +| `state_accounting_syncing_transitions` | NetworkOPs.cpp | Count of transitions to Syncing | Low | +| `state_accounting_tracking_transitions` | NetworkOPs.cpp | Count of transitions to Tracking | Low | +| `state_accounting_full_transitions` | NetworkOPs.cpp | Count of transitions to Full | Low (should be 1 after startup) | +| `peer_finder_active_inbound_peers` | PeerfinderManager.cpp | Active inbound peer connections | 0–85 | +| `peer_finder_active_outbound_peers` | PeerfinderManager.cpp | Active outbound peer connections | 10–21 | +| `overlay_peer_disconnects` | OverlayImpl.cpp | Cumulative peer disconnection count | Low growth | +| `overlay_peer_disconnects_charges` | OverlayImpl.cpp | Disconnects due to resource limit charges | Low growth (subset of above) | +| `job_count` | JobQueue.cpp | Current job queue depth | 0–100 (healthy) | **Grafana dashboard**: _Node Health_ (`node-health`) ### 2.2 Counters -| Prometheus Metric | Source File | Description | -| ------------------------------- | ------------------ | --------------------------------------------- | -| `xrpld_rpc_requests` | ServerHandler.cpp | Total RPC requests received | -| `xrpld_ledger_fetches` | InboundLedgers.cpp | Inbound ledger fetch attempts | -| `xrpld_ledger_history_mismatch` | LedgerHistory.cpp | Ledger hash mismatches detected | -| `xrpld_warn` | Logic.h | Resource manager warnings issued | -| `xrpld_drop` | Logic.h | Resource manager drops (connections rejected) | +| Prometheus Metric | Source File | Description | +| ------------------------- | ------------------ | --------------------------------------------- | +| `rpc_requests` | ServerHandler.cpp | Total RPC requests received | +| `ledger_fetches` | InboundLedgers.cpp | Inbound ledger fetch attempts | +| `ledger_history_mismatch` | LedgerHistory.cpp | Ledger hash mismatches detected | +| `warn` | Logic.h | Resource manager warnings issued | +| `drop` | Logic.h | Resource manager drops (connections rejected) | -**Note**: With `server=otel`, `xrpld_warn` and `xrpld_drop` are properly exported as OTel Counter instruments. The previous StatsD `|m` type limitation no longer applies. +**Note**: With `server=otel`, `warn` and `drop` are properly exported as OTel Counter instruments. The previous StatsD `|m` type limitation no longer applies. **Grafana dashboard**: _RPC & Pathfinding_ (`rpc-pathfinding`) ### 2.3 Histograms (Event timers) -| Prometheus Metric | Source File | Unit | Description | -| --------------------- | ----------------- | ----- | ------------------------------ | -| `xrpld_rpc_time` | ServerHandler.cpp | ms | RPC response time distribution | -| `xrpld_rpc_size` | ServerHandler.cpp | bytes | RPC response size distribution | -| `xrpld_ios_latency` | Application.cpp | ms | I/O service loop latency | -| `xrpld_pathfind_fast` | PathRequests.h | ms | Fast pathfinding duration | -| `xrpld_pathfind_full` | PathRequests.h | ms | Full pathfinding duration | +| Prometheus Metric | Source File | Unit | Description | +| ----------------- | ----------------- | ----- | ------------------------------ | +| `rpc_time` | ServerHandler.cpp | ms | RPC response time distribution | +| `rpc_size` | ServerHandler.cpp | bytes | RPC response size distribution | +| `ios_latency` | Application.cpp | ms | I/O service loop latency | +| `pathfind_fast` | PathRequests.h | ms | Fast pathfinding duration | +| `pathfind_full` | PathRequests.h | ms | Full pathfinding duration | Quantiles collected: 0th, 50th, 90th, 95th, 99th, 100th percentile. @@ -687,38 +687,38 @@ ledger.store (persist to DB) ```promql # RPC request rate by command (last 5 minutes) -sum by (command) (rate(traces_span_metrics_calls_total{span_name=~"rpc.command.*"}[5m])) +sum by (command) (rate(span_calls_total{span_name=~"rpc.command.*"}[5m])) # RPC p95 latency by command -histogram_quantile(0.95, sum by (le, command) (rate(traces_span_metrics_duration_milliseconds_bucket{span_name=~"rpc.command.*"}[5m]))) +histogram_quantile(0.95, sum by (le, command) (rate(span_duration_milliseconds_bucket{span_name=~"rpc.command.*"}[5m]))) # Consensus round duration p95 -histogram_quantile(0.95, sum by (le) (rate(traces_span_metrics_duration_milliseconds_bucket{span_name="consensus.round"}[5m]))) +histogram_quantile(0.95, sum by (le) (rate(span_duration_milliseconds_bucket{span_name="consensus.round"}[5m]))) # Transaction processing rate (local vs relay) -sum by (local) (rate(traces_span_metrics_calls_total{span_name="tx.process"}[5m])) +sum by (local) (rate(span_calls_total{span_name="tx.process"}[5m])) # Trusted vs untrusted proposal rate -sum by (proposal_trusted) (rate(traces_span_metrics_calls_total{span_name="peer.proposal.receive"}[5m])) +sum by (proposal_trusted) (rate(span_calls_total{span_name="peer.proposal.receive"}[5m])) ``` ### StatsD Metrics ```promql # Validated ledger age (should be < 10s) -xrpld_LedgerMaster_Validated_Ledger_Age +ledgermaster_validated_ledger_age # Active peer count -xrpld_Peer_Finder_Active_Inbound_Peers + xrpld_Peer_Finder_Active_Outbound_Peers +peer_finder_active_inbound_peers + peer_finder_active_outbound_peers # RPC response time p95 -histogram_quantile(0.95, xrpld_rpc_time_bucket) +histogram_quantile(0.95, rpc_time_bucket) # Total network bytes in (rate) -rate(xrpld_total_Bytes_In[5m]) +rate(total_bytes_in[5m]) # Operating mode (should be "Full" after startup) -xrpld_State_Accounting_Full_duration +state_accounting_full_duration ``` --- @@ -801,8 +801,8 @@ count_over_time({job="xrpld"} |= "trace_id=" [5m]) | Issue | Impact | Status | | ------------------------------------------------------------------ | ------------------------------------------------ | -------------------------------------------------------------------- | | `warn` and `drop` metrics use non-standard StatsD `\|m` meter type | Metrics silently dropped by OTel StatsD receiver | Phase 6 Task 6.1 — needs `\|m` → `\|c` change in StatsDCollector.cpp | -| `xrpld_job_count` may not emit in standalone mode | Missing from Prometheus in some test configs | Requires active job queue activity | -| `xrpld_rpc_requests` depends on `[insight]` config | Zero series if StatsD not configured | Requires `[insight] server=statsd` in xrpld.cfg | +| `job_count` may not emit in standalone mode | Missing from Prometheus in some test configs | Requires active job queue activity | +| `rpc_requests` depends on `[insight]` config | Zero series if StatsD not configured | Requires `[insight] server=statsd` in xrpld.cfg | | Peer tracing enabled by default | `peer.*` spans emit unless `trace_peer=0` | High volume — set `trace_peer=0` to opt out on busy mainnet nodes | --- diff --git a/docker/telemetry/grafana/dashboards/consensus-health.json b/docker/telemetry/grafana/dashboards/consensus-health.json index 8ef89ade27..71acd6c43a 100644 --- a/docker/telemetry/grafana/dashboards/consensus-health.json +++ b/docker/telemetry/grafana/dashboards/consensus-health.json @@ -29,14 +29,14 @@ "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile(0.95, sum by (le, service_instance_id) (rate(traces_span_metrics_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", consensus_mode=~\"$consensus_mode\", span_name=\"consensus.accept\"}[5m])))", + "expr": "histogram_quantile(0.95, sum by (le, service_instance_id) (rate(span_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", consensus_mode=~\"$consensus_mode\", span_name=\"consensus.accept\"}[5m])))", "legendFormat": "P95 Round Duration [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile(0.50, sum by (le, service_instance_id) (rate(traces_span_metrics_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", consensus_mode=~\"$consensus_mode\", span_name=\"consensus.accept\"}[5m])))", + "expr": "histogram_quantile(0.50, sum by (le, service_instance_id) (rate(span_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", consensus_mode=~\"$consensus_mode\", span_name=\"consensus.accept\"}[5m])))", "legendFormat": "P50 Round Duration [{{service_instance_id}}]" } ], @@ -75,7 +75,7 @@ "datasource": { "type": "prometheus" }, - "expr": "sum by (service_instance_id) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", consensus_mode=~\"$consensus_mode\", span_name=\"consensus.proposal.send\"}[5m]))", + "expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", consensus_mode=~\"$consensus_mode\", span_name=\"consensus.proposal.send\"}[5m]))", "legendFormat": "Proposals / Sec [{{service_instance_id}}]" } ], @@ -114,7 +114,7 @@ "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile(0.95, sum by (le, service_instance_id) (rate(traces_span_metrics_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", consensus_mode=~\"$consensus_mode\", span_name=\"consensus.ledger_close\"}[5m])))", + "expr": "histogram_quantile(0.95, sum by (le, service_instance_id) (rate(span_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", consensus_mode=~\"$consensus_mode\", span_name=\"consensus.ledger_close\"}[5m])))", "legendFormat": "P95 Close Duration [{{service_instance_id}}]" } ], @@ -153,7 +153,7 @@ "datasource": { "type": "prometheus" }, - "expr": "sum by (service_instance_id) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", consensus_mode=~\"$consensus_mode\", span_name=\"consensus.validation.send\"}[5m]))", + "expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", consensus_mode=~\"$consensus_mode\", span_name=\"consensus.validation.send\"}[5m]))", "legendFormat": "Validations / Sec [{{service_instance_id}}]" } ], @@ -179,14 +179,14 @@ "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile(0.95, sum by (le, service_instance_id) (rate(traces_span_metrics_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", consensus_mode=~\"$consensus_mode\", span_name=\"consensus.accept.apply\"}[5m])))", + "expr": "histogram_quantile(0.95, sum by (le, service_instance_id) (rate(span_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", consensus_mode=~\"$consensus_mode\", span_name=\"consensus.accept.apply\"}[5m])))", "legendFormat": "P95 Apply Duration [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile(0.50, sum by (le, service_instance_id) (rate(traces_span_metrics_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", consensus_mode=~\"$consensus_mode\", span_name=\"consensus.accept.apply\"}[5m])))", + "expr": "histogram_quantile(0.50, sum by (le, service_instance_id) (rate(span_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", consensus_mode=~\"$consensus_mode\", span_name=\"consensus.accept.apply\"}[5m])))", "legendFormat": "P50 Apply Duration [{{service_instance_id}}]" } ], @@ -219,7 +219,7 @@ "datasource": { "type": "prometheus" }, - "expr": "sum by (close_time_correct, service_instance_id) (rate(traces_span_metrics_calls_total{span_name=\"consensus.accept.apply\", consensus_mode=~\"$consensus_mode\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]))", + "expr": "sum by (close_time_correct, service_instance_id) (rate(span_calls_total{span_name=\"consensus.accept.apply\", consensus_mode=~\"$consensus_mode\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]))", "legendFormat": "Close Time Correct={{close_time_correct}} [{{service_instance_id}}]" } ], @@ -258,7 +258,7 @@ "datasource": { "type": "prometheus" }, - "expr": "sum by (consensus_mode, service_instance_id) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", consensus_mode=~\"$consensus_mode\", span_name=\"consensus.ledger_close\"}[5m]))", + "expr": "sum by (consensus_mode, service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", consensus_mode=~\"$consensus_mode\", span_name=\"consensus.ledger_close\"}[5m]))", "legendFormat": "{{consensus_mode}} [{{service_instance_id}}]" } ], @@ -297,14 +297,14 @@ "datasource": { "type": "prometheus" }, - "expr": "sum by (service_instance_id) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.accept\"}[5m]))", + "expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.accept\"}[5m]))", "legendFormat": "Accepts / Sec [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "sum by (service_instance_id) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.ledger_close\"}[5m]))", + "expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.ledger_close\"}[5m]))", "legendFormat": "Closes / Sec [{{service_instance_id}}]" } ], @@ -343,14 +343,14 @@ "datasource": { "type": "prometheus" }, - "expr": "sum by (service_instance_id) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.validation.send\"}[5m]))", + "expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.validation.send\"}[5m]))", "legendFormat": "Validations / Sec [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "sum by (service_instance_id) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.ledger_close\"}[5m]))", + "expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.ledger_close\"}[5m]))", "legendFormat": "Closes / Sec [{{service_instance_id}}]" } ], @@ -393,7 +393,7 @@ "datasource": { "type": "prometheus" }, - "expr": "sum(increase(traces_span_metrics_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.accept\"}[5m])) by (le)", + "expr": "sum(increase(span_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.accept\"}[5m])) by (le)", "legendFormat": "{{le}}", "format": "heatmap" } @@ -686,7 +686,7 @@ "datasource": { "type": "prometheus" }, - "expr": "sum by (consensus_state) (increase(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.accept\", consensus_state!=\"\"}[5m]))", + "expr": "sum by (consensus_state) (increase(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.accept\", consensus_state!=\"\"}[5m]))", "legendFormat": "{{consensus_state}}" } ], @@ -718,14 +718,14 @@ "datasource": { "type": "prometheus" }, - "expr": "sum by (service_instance_id) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.accept\", consensus_state=\"moved_on\"}[5m]))", + "expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.accept\", consensus_state=\"moved_on\"}[5m]))", "legendFormat": "moved_on [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "sum by (service_instance_id) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.accept\", consensus_state=\"expired\"}[5m]))", + "expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.accept\", consensus_state=\"expired\"}[5m]))", "legendFormat": "expired [{{service_instance_id}}]" } ], @@ -764,14 +764,14 @@ "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile(0.95, sum by (le, service_instance_id) (rate(traces_span_metrics_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", consensus_mode=~\"$consensus_mode\", span_name=\"consensus.round\"}[5m])))", + "expr": "histogram_quantile(0.95, sum by (le, service_instance_id) (rate(span_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", consensus_mode=~\"$consensus_mode\", span_name=\"consensus.round\"}[5m])))", "legendFormat": "P95 Round [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile(0.50, sum by (le, service_instance_id) (rate(traces_span_metrics_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", consensus_mode=~\"$consensus_mode\", span_name=\"consensus.round\"}[5m])))", + "expr": "histogram_quantile(0.50, sum by (le, service_instance_id) (rate(span_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", consensus_mode=~\"$consensus_mode\", span_name=\"consensus.round\"}[5m])))", "legendFormat": "P50 Round [{{service_instance_id}}]" } ], @@ -810,14 +810,14 @@ "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile(0.95, sum by (le, service_instance_id) (rate(traces_span_metrics_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.phase.open\"}[5m])))", + "expr": "histogram_quantile(0.95, sum by (le, service_instance_id) (rate(span_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.phase.open\"}[5m])))", "legendFormat": "P95 Open Phase [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile(0.95, sum by (le, service_instance_id) (rate(traces_span_metrics_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.establish\"}[5m])))", + "expr": "histogram_quantile(0.95, sum by (le, service_instance_id) (rate(span_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.establish\"}[5m])))", "legendFormat": "P95 Establish Phase [{{service_instance_id}}]" } ], @@ -856,14 +856,14 @@ "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile(0.95, sum by (le, service_instance_id) (rate(traces_span_metrics_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.update_positions\"}[5m])))", + "expr": "histogram_quantile(0.95, sum by (le, service_instance_id) (rate(span_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.update_positions\"}[5m])))", "legendFormat": "P95 Update [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile(0.5, sum by (le, service_instance_id) (rate(traces_span_metrics_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.update_positions\"}[5m])))", + "expr": "histogram_quantile(0.5, sum by (le, service_instance_id) (rate(span_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.update_positions\"}[5m])))", "legendFormat": "P50 Update [{{service_instance_id}}]" } ], @@ -902,14 +902,14 @@ "datasource": { "type": "prometheus" }, - "expr": "sum by (service_instance_id) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.check\", consensus_stalled=\"true\"}[5m]))", + "expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.check\", consensus_stalled=\"true\"}[5m]))", "legendFormat": "Stalled [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "sum by (service_instance_id) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.check\", consensus_stalled=\"false\"}[5m]))", + "expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.check\", consensus_stalled=\"false\"}[5m]))", "legendFormat": "Not Stalled [{{service_instance_id}}]" } ], @@ -948,7 +948,7 @@ "datasource": { "type": "prometheus" }, - "expr": "sum by (mode_new, service_instance_id) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.mode_change\"}[5m]))", + "expr": "sum by (mode_new, service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.mode_change\"}[5m]))", "legendFormat": "{{mode_new}} [{{service_instance_id}}]" } ], @@ -1036,7 +1036,7 @@ "label": "Node", "description": "Filter by rippled node (service.instance.id — e.g. Node-1)", "type": "query", - "query": "label_values(traces_span_metrics_calls_total, service_instance_id)", + "query": "label_values(span_calls_total, service_instance_id)", "datasource": { "type": "prometheus", "uid": "prometheus" @@ -1056,7 +1056,7 @@ "label": "Consensus Mode", "description": "Filter by consensus mode (Proposing, Observing, Wrong Ledger, Switched Ledger)", "type": "query", - "query": "label_values(traces_span_metrics_calls_total{span_name=\"consensus.ledger_close\"}, consensus_mode)", + "query": "label_values(span_calls_total{span_name=\"consensus.ledger_close\"}, consensus_mode)", "datasource": { "type": "prometheus", "uid": "prometheus" diff --git a/docker/telemetry/grafana/dashboards/ledger-data-sync.json b/docker/telemetry/grafana/dashboards/ledger-data-sync.json index 8fd39fca3e..c162f99857 100644 --- a/docker/telemetry/grafana/dashboards/ledger-data-sync.json +++ b/docker/telemetry/grafana/dashboards/ledger-data-sync.json @@ -30,56 +30,56 @@ "datasource": { "type": "prometheus" }, - "expr": "xrpld_ledger_data_get_Bytes_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "ledger_data_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Ledger Data Get [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_ledger_data_share_Bytes_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "ledger_data_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Ledger Data Share [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_ledger_data_Transaction_Set_candidate_get_Bytes_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "ledger_data_transaction_set_candidate_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "TX Set Candidate Get [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_ledger_data_Transaction_Set_candidate_share_Bytes_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "ledger_data_transaction_set_candidate_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "TX Set Candidate Share [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_ledger_data_Transaction_Node_get_Bytes_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "ledger_data_transaction_node_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "TX Node Get [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_ledger_data_Transaction_Node_share_Bytes_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "ledger_data_transaction_node_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "TX Node Share [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_ledger_data_Account_State_Node_get_Bytes_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "ledger_data_account_state_node_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Account State Node Get [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_ledger_data_Account_State_Node_share_Bytes_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "ledger_data_account_state_node_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Account State Node Share [{{service_instance_id}}]" } ], @@ -118,56 +118,56 @@ "datasource": { "type": "prometheus" }, - "expr": "xrpld_ledger_share_Bytes_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "ledger_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Ledger Share In [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_ledger_get_Bytes_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "ledger_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Ledger Get In [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_ledger_Transaction_Set_candidate_share_Bytes_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "ledger_transaction_set_candidate_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "TX Set Candidate Share [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_ledger_Transaction_Set_candidate_get_Bytes_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "ledger_transaction_set_candidate_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "TX Set Candidate Get [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_ledger_Transaction_node_share_Bytes_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "ledger_transaction_node_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "TX Node Share [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_ledger_Transaction_node_get_Bytes_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "ledger_transaction_node_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "TX Node Get [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_ledger_Account_State_node_share_Bytes_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "ledger_account_state_node_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Account State Share [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_ledger_Account_State_node_get_Bytes_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "ledger_account_state_node_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Account State Get [{{service_instance_id}}]" } ], @@ -206,56 +206,56 @@ "datasource": { "type": "prometheus" }, - "expr": "xrpld_getobject_Ledger_get_Bytes_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "getobject_ledger_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Ledger Get [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_getobject_Ledger_share_Bytes_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "getobject_ledger_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Ledger Share [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_getobject_Transaction_get_Bytes_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "getobject_transaction_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Transaction Get [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_getobject_Transaction_share_Bytes_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "getobject_transaction_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Transaction Share [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_getobject_Transaction_node_get_Bytes_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "getobject_transaction_node_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "TX Node Get [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_getobject_Transaction_node_share_Bytes_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "getobject_transaction_node_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "TX Node Share [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_getobject_Account_State_node_get_Bytes_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "getobject_account_state_node_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Account State Get [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_getobject_Account_State_node_share_Bytes_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "getobject_account_state_node_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Account State Share [{{service_instance_id}}]" } ], @@ -294,49 +294,49 @@ "datasource": { "type": "prometheus" }, - "expr": "xrpld_getobject_CAS_get_Bytes_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "getobject_cas_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "CAS Get [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_getobject_CAS_share_Bytes_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "getobject_cas_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "CAS Share [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_getobject_Fetch_Pack_share_Bytes_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "getobject_fetch_pack_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Fetch Pack Share [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_getobject_Fetch_Pack_get_Bytes_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "getobject_fetch_pack_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Fetch Pack Get [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_getobject_Transactions_get_Bytes_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "getobject_transactions_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Transactions Get [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_getobject_get_Bytes_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "getobject_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Aggregate Get [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_getobject_share_Bytes_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "getobject_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Aggregate Share [{{service_instance_id}}]" } ], @@ -375,49 +375,49 @@ "datasource": { "type": "prometheus" }, - "expr": "xrpld_getobject_Ledger_get_Messages_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "getobject_ledger_get_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Ledger Get [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_getobject_Transaction_get_Messages_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "getobject_transaction_get_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Transaction Get [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_getobject_Transaction_node_get_Messages_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "getobject_transaction_node_get_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "TX Node Get [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_getobject_Account_State_node_get_Messages_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "getobject_account_state_node_get_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Account State Get [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_getobject_CAS_get_Messages_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "getobject_cas_get_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "CAS Get [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_getobject_Fetch_Pack_get_Messages_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "getobject_fetch_pack_get_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Fetch Pack Get [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_getobject_Transactions_get_Messages_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "getobject_transactions_get_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Transactions Get [{{service_instance_id}}]" } ], @@ -463,7 +463,7 @@ "datasource": { "type": "prometheus" }, - "expr": "topk(20, {service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", __name__=~\"xrpld_.*_Bytes_In\", __name__!~\"xrpld_total_.*\"})", + "expr": "topk(20, {service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", __name__=~\".*_bytes_in\", __name__!~\"total_.*\"})", "legendFormat": "{{__name__}} [{{service_instance_id}}]" } ], @@ -561,7 +561,7 @@ "label": "Node", "description": "Filter by rippled node (service.instance.id)", "type": "query", - "query": "label_values(xrpld_ledger_data_get_Bytes_In, service_instance_id)", + "query": "label_values(ledger_data_get_bytes_in, service_instance_id)", "datasource": { "type": "prometheus", "uid": "prometheus" diff --git a/docker/telemetry/grafana/dashboards/ledger-operations.json b/docker/telemetry/grafana/dashboards/ledger-operations.json index cb980bca7a..92f0db5f09 100644 --- a/docker/telemetry/grafana/dashboards/ledger-operations.json +++ b/docker/telemetry/grafana/dashboards/ledger-operations.json @@ -29,7 +29,7 @@ "datasource": { "type": "prometheus" }, - "expr": "sum by (service_instance_id) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"ledger.build\"}[5m]))", + "expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"ledger.build\"}[5m]))", "legendFormat": "Builds / Sec [{{service_instance_id}}]" } ], @@ -61,14 +61,14 @@ "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile(0.95, sum by (le, service_instance_id) (rate(traces_span_metrics_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"ledger.build\"}[5m])))", + "expr": "histogram_quantile(0.95, sum by (le, service_instance_id) (rate(span_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"ledger.build\"}[5m])))", "legendFormat": "P95 Build Duration [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile(0.50, sum by (le, service_instance_id) (rate(traces_span_metrics_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"ledger.build\"}[5m])))", + "expr": "histogram_quantile(0.50, sum by (le, service_instance_id) (rate(span_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"ledger.build\"}[5m])))", "legendFormat": "P50 Build Duration [{{service_instance_id}}]" } ], @@ -107,7 +107,7 @@ "datasource": { "type": "prometheus" }, - "expr": "sum by (service_instance_id) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"ledger.validate\"}[5m]))", + "expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"ledger.validate\"}[5m]))", "legendFormat": "Validations / Sec [{{service_instance_id}}]" } ], @@ -142,7 +142,7 @@ "datasource": { "type": "prometheus" }, - "expr": "sum(increase(traces_span_metrics_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"ledger.build\"}[5m])) by (le)", + "expr": "sum(increase(span_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"ledger.build\"}[5m])) by (le)", "legendFormat": "{{le}}", "format": "heatmap" } @@ -175,14 +175,14 @@ "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile(0.95, sum by (le, service_instance_id) (rate(traces_span_metrics_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"tx.apply\"}[5m])))", + "expr": "histogram_quantile(0.95, sum by (le, service_instance_id) (rate(span_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"tx.apply\"}[5m])))", "legendFormat": "P95 tx.apply [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile(0.50, sum by (le, service_instance_id) (rate(traces_span_metrics_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"tx.apply\"}[5m])))", + "expr": "histogram_quantile(0.50, sum by (le, service_instance_id) (rate(span_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"tx.apply\"}[5m])))", "legendFormat": "P50 tx.apply [{{service_instance_id}}]" } ], @@ -221,7 +221,7 @@ "datasource": { "type": "prometheus" }, - "expr": "sum by (service_instance_id) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"tx.apply\"}[5m]))", + "expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"tx.apply\"}[5m]))", "legendFormat": "tx.apply / Sec [{{service_instance_id}}]" } ], @@ -260,7 +260,7 @@ "datasource": { "type": "prometheus" }, - "expr": "sum by (service_instance_id) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"ledger.store\"}[5m]))", + "expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"ledger.store\"}[5m]))", "legendFormat": "Stores / Sec [{{service_instance_id}}]" } ], @@ -292,14 +292,14 @@ "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile(0.95, sum by (le, service_instance_id) (rate(traces_span_metrics_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"ledger.build\"}[5m])))", + "expr": "histogram_quantile(0.95, sum by (le, service_instance_id) (rate(span_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"ledger.build\"}[5m])))", "legendFormat": "P95 ledger.build [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile(0.95, sum by (le, service_instance_id) (rate(traces_span_metrics_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.ledger_close\"}[5m])))", + "expr": "histogram_quantile(0.95, sum by (le, service_instance_id) (rate(span_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.ledger_close\"}[5m])))", "legendFormat": "P95 consensus.ledger_close [{{service_instance_id}}]" } ], @@ -387,7 +387,7 @@ "label": "Node", "description": "Filter by rippled node (service.instance.id — e.g. Node-1)", "type": "query", - "query": "label_values(traces_span_metrics_calls_total, service_instance_id)", + "query": "label_values(span_calls_total, service_instance_id)", "datasource": { "type": "prometheus", "uid": "prometheus" diff --git a/docker/telemetry/grafana/dashboards/network-traffic.json b/docker/telemetry/grafana/dashboards/network-traffic.json index 89d0d6e63c..2d4e22b595 100644 --- a/docker/telemetry/grafana/dashboards/network-traffic.json +++ b/docker/telemetry/grafana/dashboards/network-traffic.json @@ -30,14 +30,14 @@ "datasource": { "type": "prometheus" }, - "expr": "xrpld_Peer_Finder_Active_Inbound_Peers{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "peer_finder_active_inbound_peers{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Inbound Peers [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_Peer_Finder_Active_Outbound_Peers{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "peer_finder_active_outbound_peers{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Outbound Peers [{{service_instance_id}}]" } ], @@ -76,7 +76,7 @@ "datasource": { "type": "prometheus" }, - "expr": "xrpld_Overlay_Peer_Disconnects{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "overlay_peer_disconnects{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Disconnects [{{service_instance_id}}]" } ], @@ -115,14 +115,14 @@ "datasource": { "type": "prometheus" }, - "expr": "xrpld_total_Bytes_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "total_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Bytes In [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_total_Bytes_Out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "total_bytes_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Bytes Out [{{service_instance_id}}]" } ], @@ -161,14 +161,14 @@ "datasource": { "type": "prometheus" }, - "expr": "xrpld_total_Messages_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "total_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Messages In [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_total_Messages_Out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "total_messages_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Messages Out [{{service_instance_id}}]" } ], @@ -207,21 +207,21 @@ "datasource": { "type": "prometheus" }, - "expr": "xrpld_transactions_Messages_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "transactions_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "TX Messages In [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_transactions_Messages_Out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "transactions_messages_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "TX Messages Out [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_transactions_duplicate_Messages_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "transactions_duplicate_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "TX Duplicate In [{{service_instance_id}}]" } ], @@ -260,28 +260,28 @@ "datasource": { "type": "prometheus" }, - "expr": "xrpld_proposals_Messages_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "proposals_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Proposals In [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_proposals_Messages_Out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "proposals_messages_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Proposals Out [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_proposals_untrusted_Messages_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "proposals_untrusted_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Untrusted In [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_proposals_duplicate_Messages_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "proposals_duplicate_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Duplicate In [{{service_instance_id}}]" } ], @@ -320,28 +320,28 @@ "datasource": { "type": "prometheus" }, - "expr": "xrpld_validations_Messages_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "validations_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Validations In [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_validations_Messages_Out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "validations_messages_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Validations Out [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_validations_untrusted_Messages_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "validations_untrusted_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Untrusted In [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_validations_duplicate_Messages_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "validations_duplicate_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Duplicate In [{{service_instance_id}}]" } ], @@ -380,7 +380,7 @@ "datasource": { "type": "prometheus" }, - "expr": "topk(10, {service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", __name__=~\"xrpld_.*_Bytes_In\", __name__!~\"xrpld_total_.*\"})", + "expr": "topk(10, {service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", __name__=~\".*_bytes_in\", __name__!~\"total_.*\"})", "legendFormat": "{{__name__}} [{{service_instance_id}}]" } ], @@ -392,7 +392,7 @@ { "matcher": { "id": "byName", - "options": "xrpld_transactions_Bytes_In" + "options": "transactions_bytes_in" }, "properties": [ { @@ -404,7 +404,7 @@ { "matcher": { "id": "byName", - "options": "xrpld_proposals_Bytes_In" + "options": "proposals_bytes_in" }, "properties": [ { @@ -416,7 +416,7 @@ { "matcher": { "id": "byName", - "options": "xrpld_validations_Bytes_In" + "options": "validations_bytes_in" }, "properties": [ { @@ -428,7 +428,7 @@ { "matcher": { "id": "byName", - "options": "xrpld_overhead_Bytes_In" + "options": "overhead_bytes_in" }, "properties": [ { @@ -440,7 +440,7 @@ { "matcher": { "id": "byName", - "options": "xrpld_overhead_overlay_Bytes_In" + "options": "overhead_overlay_bytes_in" }, "properties": [ { @@ -452,7 +452,7 @@ { "matcher": { "id": "byName", - "options": "xrpld_ping_Bytes_In" + "options": "ping_bytes_in" }, "properties": [ { @@ -464,7 +464,7 @@ { "matcher": { "id": "byName", - "options": "xrpld_status_Bytes_In" + "options": "status_bytes_in" }, "properties": [ { @@ -476,7 +476,7 @@ { "matcher": { "id": "byName", - "options": "xrpld_getObject_Bytes_In" + "options": "getobject_bytes_in" }, "properties": [ { @@ -488,7 +488,7 @@ { "matcher": { "id": "byName", - "options": "xrpld_haveTxSet_Bytes_In" + "options": "havetxset_bytes_in" }, "properties": [ { @@ -500,7 +500,7 @@ { "matcher": { "id": "byName", - "options": "xrpld_ledgerData_Bytes_In" + "options": "ledgerdata_bytes_in" }, "properties": [ { @@ -512,7 +512,7 @@ { "matcher": { "id": "byName", - "options": "xrpld_ledger_share_Bytes_In" + "options": "ledger_share_bytes_in" }, "properties": [ { @@ -524,7 +524,7 @@ { "matcher": { "id": "byName", - "options": "xrpld_ledger_data_get_Bytes_In" + "options": "ledger_data_get_bytes_in" }, "properties": [ { @@ -536,7 +536,7 @@ { "matcher": { "id": "byName", - "options": "xrpld_ledger_data_share_Bytes_In" + "options": "ledger_data_share_bytes_in" }, "properties": [ { @@ -548,7 +548,7 @@ { "matcher": { "id": "byName", - "options": "xrpld_ledger_data_Account_State_Node_get_Bytes_In" + "options": "ledger_data_account_state_node_get_bytes_in" }, "properties": [ { @@ -560,7 +560,7 @@ { "matcher": { "id": "byName", - "options": "xrpld_ledger_data_Account_State_Node_share_Bytes_In" + "options": "ledger_data_account_state_node_share_bytes_in" }, "properties": [ { @@ -572,7 +572,7 @@ { "matcher": { "id": "byName", - "options": "xrpld_ledger_data_Transaction_Node_get_Bytes_In" + "options": "ledger_data_transaction_node_get_bytes_in" }, "properties": [ { @@ -584,7 +584,7 @@ { "matcher": { "id": "byName", - "options": "xrpld_ledger_data_Transaction_Node_share_Bytes_In" + "options": "ledger_data_transaction_node_share_bytes_in" }, "properties": [ { @@ -596,7 +596,7 @@ { "matcher": { "id": "byName", - "options": "xrpld_ledger_data_Transaction_Set_candidate_get_Bytes_In" + "options": "ledger_data_transaction_set_candidate_get_bytes_in" }, "properties": [ { @@ -608,7 +608,7 @@ { "matcher": { "id": "byName", - "options": "xrpld_ledger_Account_State_node_share_Bytes_In" + "options": "ledger_account_state_node_share_bytes_in" }, "properties": [ { @@ -620,7 +620,7 @@ { "matcher": { "id": "byName", - "options": "xrpld_ledger_Transaction_Set_candidate_share_Bytes_In" + "options": "ledger_transaction_set_candidate_share_bytes_in" }, "properties": [ { @@ -632,7 +632,7 @@ { "matcher": { "id": "byName", - "options": "xrpld_ledger_Transaction_node_share_Bytes_In" + "options": "ledger_transaction_node_share_bytes_in" }, "properties": [ { @@ -644,7 +644,7 @@ { "matcher": { "id": "byName", - "options": "xrpld_set_get_Bytes_In" + "options": "set_get_bytes_in" }, "properties": [ { @@ -677,42 +677,42 @@ "datasource": { "type": "prometheus" }, - "expr": "rate(xrpld_transactions_duplicate_Bytes_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])", + "expr": "rate(transactions_duplicate_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])", "legendFormat": "TX Duplicate In" }, { "datasource": { "type": "prometheus" }, - "expr": "rate(xrpld_transactions_duplicate_Bytes_Out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])", + "expr": "rate(transactions_duplicate_bytes_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])", "legendFormat": "TX Duplicate Out" }, { "datasource": { "type": "prometheus" }, - "expr": "rate(xrpld_proposals_duplicate_Bytes_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])", + "expr": "rate(proposals_duplicate_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])", "legendFormat": "Proposals Duplicate In" }, { "datasource": { "type": "prometheus" }, - "expr": "rate(xrpld_proposals_duplicate_Bytes_Out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])", + "expr": "rate(proposals_duplicate_bytes_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])", "legendFormat": "Proposals Duplicate Out" }, { "datasource": { "type": "prometheus" }, - "expr": "rate(xrpld_validations_duplicate_Bytes_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])", + "expr": "rate(validations_duplicate_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])", "legendFormat": "Validations Duplicate In" }, { "datasource": { "type": "prometheus" }, - "expr": "rate(xrpld_validations_duplicate_Bytes_Out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])", + "expr": "rate(validations_duplicate_bytes_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])", "legendFormat": "Validations Duplicate Out" } ], @@ -751,7 +751,7 @@ "datasource": { "type": "prometheus" }, - "expr": "topk(15, rate({__name__=~\"xrpld_.*_Bytes_In\", __name__!~\"xrpld_total_.*\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m]))", + "expr": "topk(15, rate({__name__=~\".*_bytes_in\", __name__!~\"total_.*\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m]))", "legendFormat": "{{__name__}}" } ], @@ -839,7 +839,7 @@ "label": "Node", "description": "Filter by rippled node (service.instance.id)", "type": "query", - "query": "label_values(xrpld_Peer_Finder_Active_Inbound_Peers, service_instance_id)", + "query": "label_values(peer_finder_active_inbound_peers, service_instance_id)", "datasource": { "type": "prometheus", "uid": "prometheus" diff --git a/docker/telemetry/grafana/dashboards/node-health.json b/docker/telemetry/grafana/dashboards/node-health.json index 8b887a0d68..1e64425157 100644 --- a/docker/telemetry/grafana/dashboards/node-health.json +++ b/docker/telemetry/grafana/dashboards/node-health.json @@ -30,7 +30,7 @@ "datasource": { "type": "prometheus" }, - "expr": "xrpld_LedgerMaster_Validated_Ledger_Age{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "ledgermaster_validated_ledger_age{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Validated Age [{{service_instance_id}}]" } ], @@ -78,7 +78,7 @@ "datasource": { "type": "prometheus" }, - "expr": "xrpld_LedgerMaster_Published_Ledger_Age{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "ledgermaster_published_ledger_age{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Published Age [{{service_instance_id}}]" } ], @@ -126,35 +126,35 @@ "datasource": { "type": "prometheus" }, - "expr": "xrpld_State_Accounting_Full_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "state_accounting_full_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Full [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_State_Accounting_Tracking_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "state_accounting_tracking_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Tracking [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_State_Accounting_Syncing_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "state_accounting_syncing_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Syncing [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_State_Accounting_Connected_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "state_accounting_connected_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Connected [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_State_Accounting_Disconnected_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "state_accounting_disconnected_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Disconnected [{{service_instance_id}}]" } ], @@ -193,35 +193,35 @@ "datasource": { "type": "prometheus" }, - "expr": "xrpld_State_Accounting_Full_transitions{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "state_accounting_full_transitions{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Full [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_State_Accounting_Tracking_transitions{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "state_accounting_tracking_transitions{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Tracking [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_State_Accounting_Syncing_transitions{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "state_accounting_syncing_transitions{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Syncing [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_State_Accounting_Connected_transitions{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "state_accounting_connected_transitions{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Connected [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_State_Accounting_Disconnected_transitions{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "state_accounting_disconnected_transitions{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Disconnected [{{service_instance_id}}]" } ], @@ -260,14 +260,14 @@ "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile(0.95, sum by (le, service_instance_id) (rate(xrpld_ios_latency_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", + "expr": "histogram_quantile(0.95, sum by (le, service_instance_id) (rate(ios_latency_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", "legendFormat": "P95 I/O Latency [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile(0.50, sum by (le, service_instance_id) (rate(xrpld_ios_latency_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", + "expr": "histogram_quantile(0.50, sum by (le, service_instance_id) (rate(ios_latency_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", "legendFormat": "P50 I/O Latency [{{service_instance_id}}]" } ], @@ -306,7 +306,7 @@ "datasource": { "type": "prometheus" }, - "expr": "xrpld_job_count{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "job_count{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Job Queue Depth [{{service_instance_id}}]" } ], @@ -345,7 +345,7 @@ "datasource": { "type": "prometheus" }, - "expr": "rate(xrpld_ledger_fetches_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])", + "expr": "rate(ledger_fetches_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])", "legendFormat": "Fetches / Sec [{{service_instance_id}}]" } ], @@ -377,7 +377,7 @@ "datasource": { "type": "prometheus" }, - "expr": "rate(xrpld_ledger_history_mismatch_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])", + "expr": "rate(ledger_history_mismatch_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])", "legendFormat": "Mismatches / Sec [{{service_instance_id}}]" } ], @@ -433,77 +433,77 @@ "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile($quantile, sum by (le, service_instance_id) (rate(xrpld_job_running_duration_us_bucket{job_type=\"acceptLedger\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", + "expr": "histogram_quantile($quantile, sum by (le, service_instance_id) (rate(job_running_us_bucket{job_type=\"acceptLedger\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", "legendFormat": "Accept Ledger [{{quantile}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile($quantile, sum by (le, service_instance_id) (rate(xrpld_job_running_duration_us_bucket{job_type=\"advanceLedger\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", + "expr": "histogram_quantile($quantile, sum by (le, service_instance_id) (rate(job_running_us_bucket{job_type=\"advanceLedger\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", "legendFormat": "Advance Ledger [{{quantile}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile($quantile, sum by (le, service_instance_id) (rate(xrpld_job_running_duration_us_bucket{job_type=\"transaction\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", + "expr": "histogram_quantile($quantile, sum by (le, service_instance_id) (rate(job_running_us_bucket{job_type=\"transaction\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", "legendFormat": "Transaction [{{quantile}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile($quantile, sum by (le, service_instance_id) (rate(xrpld_job_running_duration_us_bucket{job_type=\"writeObjects\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", + "expr": "histogram_quantile($quantile, sum by (le, service_instance_id) (rate(job_running_us_bucket{job_type=\"writeObjects\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", "legendFormat": "Write Objects [{{quantile}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile($quantile, sum by (le, service_instance_id) (rate(xrpld_job_running_duration_us_bucket{job_type=\"heartbeat\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", + "expr": "histogram_quantile($quantile, sum by (le, service_instance_id) (rate(job_running_us_bucket{job_type=\"heartbeat\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", "legendFormat": "Heartbeat [{{quantile}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile($quantile, sum by (le, service_instance_id) (rate(xrpld_job_running_duration_us_bucket{job_type=\"sweep\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", + "expr": "histogram_quantile($quantile, sum by (le, service_instance_id) (rate(job_running_us_bucket{job_type=\"sweep\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", "legendFormat": "Sweep [{{quantile}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile($quantile, sum by (le, service_instance_id) (rate(xrpld_job_running_duration_us_bucket{job_type=\"trustedValidation\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", + "expr": "histogram_quantile($quantile, sum by (le, service_instance_id) (rate(job_running_us_bucket{job_type=\"trustedValidation\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", "legendFormat": "Trusted Validation [{{quantile}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile($quantile, sum by (le, service_instance_id) (rate(xrpld_job_running_duration_us_bucket{job_type=\"trustedProposal\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", + "expr": "histogram_quantile($quantile, sum by (le, service_instance_id) (rate(job_running_us_bucket{job_type=\"trustedProposal\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", "legendFormat": "Trusted Proposal [{{quantile}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile($quantile, sum by (le, service_instance_id) (rate(xrpld_job_running_duration_us_bucket{job_type=\"publishNewLedger\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", + "expr": "histogram_quantile($quantile, sum by (le, service_instance_id) (rate(job_running_us_bucket{job_type=\"publishNewLedger\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", "legendFormat": "Publish New Ledger [{{quantile}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile($quantile, sum by (le, service_instance_id) (rate(xrpld_job_running_duration_us_bucket{job_type=\"clientRPC\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", + "expr": "histogram_quantile($quantile, sum by (le, service_instance_id) (rate(job_running_us_bucket{job_type=\"clientRPC\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", "legendFormat": "Client RPC [{{quantile}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile($quantile, sum by (le, service_instance_id) (rate(xrpld_job_running_duration_us_bucket{job_type=\"ledgerData\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", + "expr": "histogram_quantile($quantile, sum by (le, service_instance_id) (rate(job_running_us_bucket{job_type=\"ledgerData\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", "legendFormat": "Ledger Data [{{quantile}}]" } ], @@ -542,77 +542,77 @@ "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile($quantile, sum by (le, service_instance_id) (rate(xrpld_job_queued_duration_us_bucket{job_type=\"acceptLedger\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", + "expr": "histogram_quantile($quantile, sum by (le, service_instance_id) (rate(job_queued_us_bucket{job_type=\"acceptLedger\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", "legendFormat": "Accept Ledger [{{quantile}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile($quantile, sum by (le, service_instance_id) (rate(xrpld_job_queued_duration_us_bucket{job_type=\"advanceLedger\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", + "expr": "histogram_quantile($quantile, sum by (le, service_instance_id) (rate(job_queued_us_bucket{job_type=\"advanceLedger\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", "legendFormat": "Advance Ledger [{{quantile}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile($quantile, sum by (le, service_instance_id) (rate(xrpld_job_queued_duration_us_bucket{job_type=\"transaction\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", + "expr": "histogram_quantile($quantile, sum by (le, service_instance_id) (rate(job_queued_us_bucket{job_type=\"transaction\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", "legendFormat": "Transaction [{{quantile}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile($quantile, sum by (le, service_instance_id) (rate(xrpld_job_queued_duration_us_bucket{job_type=\"writeObjects\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", + "expr": "histogram_quantile($quantile, sum by (le, service_instance_id) (rate(job_queued_us_bucket{job_type=\"writeObjects\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", "legendFormat": "Write Objects [{{quantile}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile($quantile, sum by (le, service_instance_id) (rate(xrpld_job_queued_duration_us_bucket{job_type=\"heartbeat\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", + "expr": "histogram_quantile($quantile, sum by (le, service_instance_id) (rate(job_queued_us_bucket{job_type=\"heartbeat\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", "legendFormat": "Heartbeat [{{quantile}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile($quantile, sum by (le, service_instance_id) (rate(xrpld_job_queued_duration_us_bucket{job_type=\"sweep\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", + "expr": "histogram_quantile($quantile, sum by (le, service_instance_id) (rate(job_queued_us_bucket{job_type=\"sweep\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", "legendFormat": "Sweep [{{quantile}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile($quantile, sum by (le, service_instance_id) (rate(xrpld_job_queued_duration_us_bucket{job_type=\"trustedValidation\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", + "expr": "histogram_quantile($quantile, sum by (le, service_instance_id) (rate(job_queued_us_bucket{job_type=\"trustedValidation\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", "legendFormat": "Trusted Validation [{{quantile}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile($quantile, sum by (le, service_instance_id) (rate(xrpld_job_queued_duration_us_bucket{job_type=\"trustedProposal\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", + "expr": "histogram_quantile($quantile, sum by (le, service_instance_id) (rate(job_queued_us_bucket{job_type=\"trustedProposal\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", "legendFormat": "Trusted Proposal [{{quantile}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile($quantile, sum by (le, service_instance_id) (rate(xrpld_job_queued_duration_us_bucket{job_type=\"publishNewLedger\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", + "expr": "histogram_quantile($quantile, sum by (le, service_instance_id) (rate(job_queued_us_bucket{job_type=\"publishNewLedger\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", "legendFormat": "Publish New Ledger [{{quantile}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile($quantile, sum by (le, service_instance_id) (rate(xrpld_job_queued_duration_us_bucket{job_type=\"clientRPC\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", + "expr": "histogram_quantile($quantile, sum by (le, service_instance_id) (rate(job_queued_us_bucket{job_type=\"clientRPC\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", "legendFormat": "Client RPC [{{quantile}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile($quantile, sum by (le, service_instance_id) (rate(xrpld_job_queued_duration_us_bucket{job_type=\"ledgerData\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", + "expr": "histogram_quantile($quantile, sum by (le, service_instance_id) (rate(job_queued_us_bucket{job_type=\"ledgerData\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", "legendFormat": "Ledger Data [{{quantile}}]" } ], @@ -651,7 +651,7 @@ "datasource": { "type": "prometheus" }, - "expr": "xrpld_Node_family_full_below_cache_size{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "node_family_full_below_cache_size{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "FullBelowCache Size" } ], @@ -690,7 +690,7 @@ "datasource": { "type": "prometheus" }, - "expr": "xrpld_Node_family_full_below_cache_hit_rate{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "node_family_full_below_cache_hit_rate{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Hit Rate" } ], @@ -740,7 +740,7 @@ "datasource": { "type": "prometheus" }, - "expr": "xrpld_LedgerMaster_Published_Ledger_Age{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"} - xrpld_LedgerMaster_Validated_Ledger_Age{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "ledgermaster_published_ledger_age{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"} - ledgermaster_validated_ledger_age{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Publish Gap" } ], @@ -788,14 +788,14 @@ "datasource": { "type": "prometheus" }, - "expr": "rate(xrpld_State_Accounting_Full_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m]) / 1000000", + "expr": "rate(state_accounting_full_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m]) / 1000000", "legendFormat": "Full Mode Rate" }, { "datasource": { "type": "prometheus" }, - "expr": "rate(xrpld_State_Accounting_Tracking_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m]) / 1000000", + "expr": "rate(state_accounting_tracking_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m]) / 1000000", "legendFormat": "Tracking Mode Rate" } ], @@ -834,7 +834,7 @@ "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile($quantile, sum by (le, job_type) (rate(xrpld_job_running_duration_us_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", + "expr": "histogram_quantile($quantile, sum by (le, job_type) (rate(job_running_us_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", "legendFormat": "{{job_type}}" } ], @@ -873,7 +873,7 @@ "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile($quantile, sum by (le, job_type) (rate(xrpld_job_queued_duration_us_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", + "expr": "histogram_quantile($quantile, sum by (le, job_type) (rate(job_queued_us_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", "legendFormat": "{{job_type}}" } ], @@ -961,7 +961,7 @@ "label": "Node", "description": "Filter by rippled node (service.instance.id)", "type": "query", - "query": "label_values(xrpld_LedgerMaster_Validated_Ledger_Age, service_instance_id)", + "query": "label_values(ledgermaster_validated_ledger_age, service_instance_id)", "datasource": { "type": "prometheus", "uid": "prometheus" diff --git a/docker/telemetry/grafana/dashboards/overlay-traffic-detail.json b/docker/telemetry/grafana/dashboards/overlay-traffic-detail.json index d0c2cbe662..0333f10357 100644 --- a/docker/telemetry/grafana/dashboards/overlay-traffic-detail.json +++ b/docker/telemetry/grafana/dashboards/overlay-traffic-detail.json @@ -30,42 +30,42 @@ "datasource": { "type": "prometheus" }, - "expr": "xrpld_squelch_Messages_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "squelch_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Squelch In [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_squelch_Messages_Out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "squelch_messages_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Squelch Out [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_squelch_suppressed_Messages_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "squelch_suppressed_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Suppressed In [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_squelch_suppressed_Messages_Out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "squelch_suppressed_messages_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Suppressed Out [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_squelch_ignored_Messages_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "squelch_ignored_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Ignored In [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_squelch_ignored_Messages_Out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "squelch_ignored_messages_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Ignored Out [{{service_instance_id}}]" } ], @@ -104,42 +104,42 @@ "datasource": { "type": "prometheus" }, - "expr": "xrpld_overhead_Bytes_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "overhead_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Base Overhead In [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_overhead_Bytes_Out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "overhead_bytes_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Base Overhead Out [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_overhead_cluster_Bytes_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "overhead_cluster_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Cluster In [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_overhead_cluster_Bytes_Out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "overhead_cluster_bytes_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Cluster Out [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_overhead_manifest_Bytes_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "overhead_manifest_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Manifest In [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_overhead_manifest_Bytes_Out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "overhead_manifest_bytes_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Manifest Out [{{service_instance_id}}]" } ], @@ -178,28 +178,28 @@ "datasource": { "type": "prometheus" }, - "expr": "xrpld_validator_lists_Bytes_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "validator_lists_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Bytes In [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_validator_lists_Bytes_Out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "validator_lists_bytes_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Bytes Out [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_validator_lists_Messages_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "validator_lists_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Messages In [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_validator_lists_Messages_Out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "validator_lists_messages_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Messages Out [{{service_instance_id}}]" } ], @@ -255,28 +255,28 @@ "datasource": { "type": "prometheus" }, - "expr": "xrpld_set_get_Bytes_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "set_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Set Get In [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_set_get_Bytes_Out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "set_get_bytes_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Set Get Out [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_set_share_Bytes_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "set_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Set Share In [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_set_share_Bytes_Out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "set_share_bytes_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Set Share Out [{{service_instance_id}}]" } ], @@ -315,28 +315,28 @@ "datasource": { "type": "prometheus" }, - "expr": "xrpld_have_transactions_Messages_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "have_transactions_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Have TX In [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_have_transactions_Messages_Out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "have_transactions_messages_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Have TX Out [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_requested_transactions_Messages_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "requested_transactions_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Requested TX In [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_requested_transactions_Messages_Out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "requested_transactions_messages_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Requested TX Out [{{service_instance_id}}]" } ], @@ -375,28 +375,28 @@ "datasource": { "type": "prometheus" }, - "expr": "xrpld_unknown_Bytes_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "unknown_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Unknown Bytes In [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_unknown_Bytes_Out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "unknown_bytes_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Unknown Bytes Out [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_unknown_Messages_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "unknown_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Unknown Messages In [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_unknown_Messages_Out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "unknown_messages_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Unknown Messages Out [{{service_instance_id}}]" } ], @@ -452,28 +452,28 @@ "datasource": { "type": "prometheus" }, - "expr": "xrpld_proof_path_request_Bytes_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "proof_path_request_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Request Bytes In [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_proof_path_request_Bytes_Out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "proof_path_request_bytes_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Request Bytes Out [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_proof_path_response_Bytes_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "proof_path_response_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Response Bytes In [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_proof_path_response_Bytes_Out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "proof_path_response_bytes_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Response Bytes Out [{{service_instance_id}}]" } ], @@ -512,28 +512,28 @@ "datasource": { "type": "prometheus" }, - "expr": "xrpld_replay_delta_request_Bytes_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "replay_delta_request_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Request Bytes In [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_replay_delta_request_Bytes_Out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "replay_delta_request_bytes_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Request Bytes Out [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_replay_delta_response_Bytes_In{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "replay_delta_response_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Response Bytes In [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "xrpld_replay_delta_response_Bytes_Out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", + "expr": "replay_delta_response_bytes_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}", "legendFormat": "Response Bytes Out [{{service_instance_id}}]" } ], @@ -621,7 +621,7 @@ "label": "Node", "description": "Filter by rippled node (service.instance.id)", "type": "query", - "query": "label_values(xrpld_squelch_Messages_In, service_instance_id)", + "query": "label_values(squelch_messages_in, service_instance_id)", "datasource": { "type": "prometheus", "uid": "prometheus" diff --git a/docker/telemetry/grafana/dashboards/peer-network.json b/docker/telemetry/grafana/dashboards/peer-network.json index f223b48349..2760bf9fc2 100644 --- a/docker/telemetry/grafana/dashboards/peer-network.json +++ b/docker/telemetry/grafana/dashboards/peer-network.json @@ -30,7 +30,7 @@ "datasource": { "type": "prometheus" }, - "expr": "sum by (service_instance_id) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"peer.proposal.receive\"}[5m]))", + "expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"peer.proposal.receive\"}[5m]))", "legendFormat": "Proposals Received / Sec [{{service_instance_id}}]" } ], @@ -69,7 +69,7 @@ "datasource": { "type": "prometheus" }, - "expr": "sum by (service_instance_id) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"peer.validation.receive\"}[5m]))", + "expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"peer.validation.receive\"}[5m]))", "legendFormat": "Validations Received / Sec [{{service_instance_id}}]" } ], @@ -108,7 +108,7 @@ "datasource": { "type": "prometheus" }, - "expr": "sum by (proposal_trusted, service_instance_id) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", proposal_trusted=~\"$proposal_trusted\", span_name=\"peer.proposal.receive\"}[5m]))", + "expr": "sum by (proposal_trusted, service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", proposal_trusted=~\"$proposal_trusted\", span_name=\"peer.proposal.receive\"}[5m]))", "legendFormat": "Trusted = {{proposal_trusted}} [{{service_instance_id}}]" } ], @@ -140,7 +140,7 @@ "datasource": { "type": "prometheus" }, - "expr": "sum by (validation_trusted, service_instance_id) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", validation_trusted=~\"$validation_trusted\", span_name=\"peer.validation.receive\"}[5m]))", + "expr": "sum by (validation_trusted, service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", validation_trusted=~\"$validation_trusted\", span_name=\"peer.validation.receive\"}[5m]))", "legendFormat": "Trusted = {{validation_trusted}} [{{service_instance_id}}]" } ], @@ -221,7 +221,7 @@ "label": "Node", "description": "Filter by rippled node (service.instance.id — e.g. Node-1)", "type": "query", - "query": "label_values(traces_span_metrics_calls_total, service_instance_id)", + "query": "label_values(span_calls_total, service_instance_id)", "datasource": { "type": "prometheus", "uid": "prometheus" @@ -241,7 +241,7 @@ "label": "Proposal Trusted", "description": "Filter by proposal trust status (true = from trusted validator)", "type": "query", - "query": "label_values(traces_span_metrics_calls_total{span_name=\"peer.proposal.receive\"}, proposal_trusted)", + "query": "label_values(span_calls_total{span_name=\"peer.proposal.receive\"}, proposal_trusted)", "datasource": { "type": "prometheus", "uid": "prometheus" @@ -261,7 +261,7 @@ "label": "Validation Trusted", "description": "Filter by validation trust status (true = from trusted validator)", "type": "query", - "query": "label_values(traces_span_metrics_calls_total{span_name=\"peer.validation.receive\"}, validation_trusted)", + "query": "label_values(span_calls_total{span_name=\"peer.validation.receive\"}, validation_trusted)", "datasource": { "type": "prometheus", "uid": "prometheus" diff --git a/docker/telemetry/grafana/dashboards/rpc-pathfinding.json b/docker/telemetry/grafana/dashboards/rpc-pathfinding.json index fa65f39c82..90a3f29c64 100644 --- a/docker/telemetry/grafana/dashboards/rpc-pathfinding.json +++ b/docker/telemetry/grafana/dashboards/rpc-pathfinding.json @@ -30,7 +30,7 @@ "datasource": { "type": "prometheus" }, - "expr": "rate(xrpld_rpc_requests_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])", + "expr": "rate(rpc_requests_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])", "legendFormat": "Requests / Sec [{{service_instance_id}}]" } ], @@ -62,14 +62,14 @@ "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile(0.95, sum by (le, service_instance_id) (rate(xrpld_rpc_time_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", + "expr": "histogram_quantile(0.95, sum by (le, service_instance_id) (rate(rpc_time_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", "legendFormat": "P95 Response Time [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile(0.5, sum by (le, service_instance_id) (rate(xrpld_rpc_time_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", + "expr": "histogram_quantile(0.5, sum by (le, service_instance_id) (rate(rpc_time_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", "legendFormat": "P50 Response Time [{{service_instance_id}}]" } ], @@ -108,14 +108,14 @@ "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile(0.95, sum by (le, service_instance_id) (rate(xrpld_rpc_size_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", + "expr": "histogram_quantile(0.95, sum by (le, service_instance_id) (rate(rpc_size_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", "legendFormat": "P95 Response Size [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile(0.5, sum by (le, service_instance_id) (rate(xrpld_rpc_size_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", + "expr": "histogram_quantile(0.5, sum by (le, service_instance_id) (rate(rpc_size_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", "legendFormat": "P50 Response Size [{{service_instance_id}}]" } ], @@ -154,28 +154,28 @@ "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile(0.5, sum by (le, service_instance_id) (rate(xrpld_rpc_time_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", + "expr": "histogram_quantile(0.5, sum by (le, service_instance_id) (rate(rpc_time_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", "legendFormat": "P50 [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile(0.9, sum by (le, service_instance_id) (rate(xrpld_rpc_time_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", + "expr": "histogram_quantile(0.9, sum by (le, service_instance_id) (rate(rpc_time_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", "legendFormat": "P90 [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile(0.95, sum by (le, service_instance_id) (rate(xrpld_rpc_time_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", + "expr": "histogram_quantile(0.95, sum by (le, service_instance_id) (rate(rpc_time_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", "legendFormat": "P95 [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile(0.99, sum by (le, service_instance_id) (rate(xrpld_rpc_time_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", + "expr": "histogram_quantile(0.99, sum by (le, service_instance_id) (rate(rpc_time_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", "legendFormat": "P99 [{{service_instance_id}}]" } ], @@ -214,14 +214,14 @@ "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile(0.95, sum by (le, service_instance_id) (rate(xrpld_pathfind_fast_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", + "expr": "histogram_quantile(0.95, sum by (le, service_instance_id) (rate(pathfind_fast_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", "legendFormat": "P95 Fast Pathfind [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile(0.5, sum by (le, service_instance_id) (rate(xrpld_pathfind_fast_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", + "expr": "histogram_quantile(0.5, sum by (le, service_instance_id) (rate(pathfind_fast_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", "legendFormat": "P50 Fast Pathfind [{{service_instance_id}}]" } ], @@ -260,14 +260,14 @@ "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile(0.95, sum by (le, service_instance_id) (rate(xrpld_pathfind_full_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", + "expr": "histogram_quantile(0.95, sum by (le, service_instance_id) (rate(pathfind_full_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", "legendFormat": "P95 Full Pathfind [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile(0.5, sum by (le, service_instance_id) (rate(xrpld_pathfind_full_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", + "expr": "histogram_quantile(0.5, sum by (le, service_instance_id) (rate(pathfind_full_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])))", "legendFormat": "P50 Full Pathfind [{{service_instance_id}}]" } ], @@ -306,7 +306,7 @@ "datasource": { "type": "prometheus" }, - "expr": "rate(xrpld_warn_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])", + "expr": "rate(warn_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])", "legendFormat": "Warnings / Sec [{{service_instance_id}}]" } ], @@ -354,7 +354,7 @@ "datasource": { "type": "prometheus" }, - "expr": "rate(xrpld_drop_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])", + "expr": "rate(drop_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])", "legendFormat": "Drops / Sec [{{service_instance_id}}]" } ], @@ -402,7 +402,7 @@ "datasource": { "type": "prometheus" }, - "expr": "sum by (method, service_instance_id) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", method=~\"$grpc_method\", span_name=~\"grpc\\\\..*\"}[5m]))", + "expr": "sum by (method, service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", method=~\"$grpc_method\", span_name=~\"grpc\\\\..*\"}[5m]))", "legendFormat": "{{method}} [{{service_instance_id}}]" } ], @@ -441,7 +441,7 @@ "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile(0.95, sum by (le, method, service_instance_id) (rate(traces_span_metrics_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", method=~\"$grpc_method\", span_name=~\"grpc\\\\..*\"}[5m])))", + "expr": "histogram_quantile(0.95, sum by (le, method, service_instance_id) (rate(span_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", method=~\"$grpc_method\", span_name=~\"grpc\\\\..*\"}[5m])))", "legendFormat": "{{method}} [{{service_instance_id}}]" } ], @@ -480,7 +480,7 @@ "datasource": { "type": "prometheus" }, - "expr": "sum by (grpc_status, service_instance_id) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=~\"grpc\\\\..*\", grpc_status!=\"\"}[5m]))", + "expr": "sum by (grpc_status, service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=~\"grpc\\\\..*\", grpc_status!=\"\"}[5m]))", "legendFormat": "{{grpc_status}} [{{service_instance_id}}]" } ], @@ -519,14 +519,14 @@ "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile(0.95, sum by (le, service_instance_id) (rate(traces_span_metrics_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"pathfind.compute\"}[5m])))", + "expr": "histogram_quantile(0.95, sum by (le, service_instance_id) (rate(span_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"pathfind.compute\"}[5m])))", "legendFormat": "P95 Compute [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile(0.50, sum by (le, service_instance_id) (rate(traces_span_metrics_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"pathfind.compute\"}[5m])))", + "expr": "histogram_quantile(0.50, sum by (le, service_instance_id) (rate(span_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"pathfind.compute\"}[5m])))", "legendFormat": "P50 Compute [{{service_instance_id}}]" } ], @@ -565,14 +565,14 @@ "datasource": { "type": "prometheus" }, - "expr": "sum by (service_instance_id) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"pathfind.request\"}[5m]))", + "expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"pathfind.request\"}[5m]))", "legendFormat": "Requests / Sec [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "sum by (service_instance_id) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"pathfind.discover\"}[5m]))", + "expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"pathfind.discover\"}[5m]))", "legendFormat": "Discoveries / Sec [{{service_instance_id}}]" } ], @@ -660,7 +660,7 @@ "label": "Node", "description": "Filter by rippled node (service.instance.id)", "type": "query", - "query": "label_values(xrpld_rpc_requests_total, service_instance_id)", + "query": "label_values(rpc_requests_total, service_instance_id)", "datasource": { "type": "prometheus", "uid": "prometheus" @@ -680,7 +680,7 @@ "label": "gRPC Method", "description": "Filter by gRPC method (GetLedger, GetLedgerData, GetLedgerDiff, GetLedgerEntry)", "type": "query", - "query": "label_values(traces_span_metrics_calls_total{span_name=~\"grpc\\\\..*\"}, method)", + "query": "label_values(span_calls_total{span_name=~\"grpc\\\\..*\"}, method)", "datasource": { "type": "prometheus", "uid": "prometheus" diff --git a/docker/telemetry/grafana/dashboards/rpc-performance.json b/docker/telemetry/grafana/dashboards/rpc-performance.json index b766c9704f..fd33966d4e 100644 --- a/docker/telemetry/grafana/dashboards/rpc-performance.json +++ b/docker/telemetry/grafana/dashboards/rpc-performance.json @@ -29,7 +29,7 @@ "datasource": { "type": "prometheus" }, - "expr": "sum by (command, service_instance_id) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", command=~\"$command\", span_name=~\"rpc.command.*\"}[5m]))", + "expr": "sum by (command, service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", command=~\"$command\", span_name=~\"rpc.command.*\"}[5m]))", "legendFormat": "{{command}} [{{service_instance_id}}]" } ], @@ -68,7 +68,7 @@ "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile(0.95, sum by (le, command, service_instance_id) (rate(traces_span_metrics_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", command=~\"$command\", span_name=~\"rpc.command.*\"}[5m])))", + "expr": "histogram_quantile(0.95, sum by (le, command, service_instance_id) (rate(span_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", command=~\"$command\", span_name=~\"rpc.command.*\"}[5m])))", "legendFormat": "P95 {{command}} [{{service_instance_id}}]" } ], @@ -107,7 +107,7 @@ "datasource": { "type": "prometheus" }, - "expr": "sum by (command, service_instance_id) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", command=~\"$command\", span_name=~\"rpc.command.*\", status_code=\"STATUS_CODE_ERROR\"}[5m])) / sum by (command, service_instance_id) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", command=~\"$command\", span_name=~\"rpc.command.*\"}[5m])) * 100", + "expr": "sum by (command, service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", command=~\"$command\", span_name=~\"rpc.command.*\", status_code=\"STATUS_CODE_ERROR\"}[5m])) / sum by (command, service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", command=~\"$command\", span_name=~\"rpc.command.*\"}[5m])) * 100", "legendFormat": "{{command}} [{{service_instance_id}}]" } ], @@ -159,7 +159,7 @@ "datasource": { "type": "prometheus" }, - "expr": "sum(increase(traces_span_metrics_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", command=~\"$command\", span_name=~\"rpc.command.*\"}[5m])) by (le)", + "expr": "sum(increase(span_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", command=~\"$command\", span_name=~\"rpc.command.*\"}[5m])) by (le)", "legendFormat": "{{le}}", "format": "heatmap" } @@ -191,14 +191,14 @@ "datasource": { "type": "prometheus" }, - "expr": "sum by (service_instance_id) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", command=~\"$command\", span_name=\"rpc.http_request\"}[5m]))", + "expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", command=~\"$command\", span_name=\"rpc.http_request\"}[5m]))", "legendFormat": "rpc.http_request / Sec [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "sum by (service_instance_id) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", command=~\"$command\", span_name=\"rpc.process\"}[5m]))", + "expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", command=~\"$command\", span_name=\"rpc.process\"}[5m]))", "legendFormat": "rpc.process / Sec [{{service_instance_id}}]" } ], @@ -237,14 +237,14 @@ "datasource": { "type": "prometheus" }, - "expr": "sum by (service_instance_id) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", command=~\"$command\", span_name=~\"rpc.command.*\", status_code=\"STATUS_CODE_UNSET\"}[5m]))", + "expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", command=~\"$command\", span_name=~\"rpc.command.*\", status_code=\"STATUS_CODE_UNSET\"}[5m]))", "legendFormat": "Success [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "sum by (service_instance_id) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", command=~\"$command\", span_name=~\"rpc.command.*\", status_code=\"STATUS_CODE_ERROR\"}[5m]))", + "expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", command=~\"$command\", span_name=~\"rpc.command.*\", status_code=\"STATUS_CODE_ERROR\"}[5m]))", "legendFormat": "Error [{{service_instance_id}}]" } ], @@ -283,7 +283,7 @@ "datasource": { "type": "prometheus" }, - "expr": "topk(10, sum by (command, service_instance_id) (increase(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", command=~\"$command\", span_name=~\"rpc.command.*\"}[5m])))", + "expr": "topk(10, sum by (command, service_instance_id) (increase(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", command=~\"$command\", span_name=~\"rpc.command.*\"}[5m])))", "legendFormat": "{{command}} [{{service_instance_id}}]" } ], @@ -315,7 +315,7 @@ "datasource": { "type": "prometheus" }, - "expr": "sum by (service_instance_id) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", command=~\"$command\", span_name=\"rpc.ws_message\"}[5m]))", + "expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", command=~\"$command\", span_name=\"rpc.ws_message\"}[5m]))", "legendFormat": "WS Messages / Sec [{{service_instance_id}}]" } ], @@ -352,7 +352,7 @@ "datasource": { "type": "prometheus" }, - "expr": "sum by (load_type) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=~\"rpc.command.*\", load_type!=\"\"}[5m]))", + "expr": "sum by (load_type) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=~\"rpc.command.*\", load_type!=\"\"}[5m]))", "legendFormat": "{{load_type}}" } ], @@ -391,14 +391,14 @@ "datasource": { "type": "prometheus" }, - "expr": "sum by (service_instance_id) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"rpc.process\", is_batch=\"true\"}[5m]))", + "expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"rpc.process\", is_batch=\"true\"}[5m]))", "legendFormat": "Batch [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "sum by (service_instance_id) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"rpc.process\", is_batch=\"false\"}[5m]))", + "expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"rpc.process\", is_batch=\"false\"}[5m]))", "legendFormat": "Single [{{service_instance_id}}]" } ], @@ -486,7 +486,7 @@ "label": "Node", "description": "Filter by rippled node (service.instance.id — e.g. Node-1)", "type": "query", - "query": "label_values(traces_span_metrics_calls_total, service_instance_id)", + "query": "label_values(span_calls_total, service_instance_id)", "datasource": { "type": "prometheus", "uid": "prometheus" @@ -506,7 +506,7 @@ "label": "RPC Command", "description": "Filter by RPC command name (e.g., server_info, submit)", "type": "query", - "query": "label_values(traces_span_metrics_calls_total{span_name=~\"rpc.command.*\"}, command)", + "query": "label_values(span_calls_total{span_name=~\"rpc.command.*\"}, command)", "datasource": { "type": "prometheus", "uid": "prometheus" diff --git a/docker/telemetry/grafana/dashboards/transaction-overview.json b/docker/telemetry/grafana/dashboards/transaction-overview.json index 42e56b25c7..c4409d07de 100644 --- a/docker/telemetry/grafana/dashboards/transaction-overview.json +++ b/docker/telemetry/grafana/dashboards/transaction-overview.json @@ -29,14 +29,14 @@ "datasource": { "type": "prometheus" }, - "expr": "sum by (service_instance_id) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"tx.process\", tx_type=~\"$tx_type\"}[5m]))", + "expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"tx.process\", tx_type=~\"$tx_type\"}[5m]))", "legendFormat": "tx.process / Sec [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "sum by (service_instance_id) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"tx.receive\", tx_type=~\"$tx_type\"}[5m]))", + "expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"tx.receive\", tx_type=~\"$tx_type\"}[5m]))", "legendFormat": "tx.receive / Sec [{{service_instance_id}}]" } ], @@ -80,14 +80,14 @@ "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile(0.95, sum by (le, tx_type, service_instance_id) (rate(traces_span_metrics_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"tx.process\", tx_type=~\"$tx_type\"}[5m])))", + "expr": "histogram_quantile(0.95, sum by (le, tx_type, service_instance_id) (rate(span_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"tx.process\", tx_type=~\"$tx_type\"}[5m])))", "legendFormat": "P95 {{tx_type}} [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile(0.50, sum by (le, tx_type, service_instance_id) (rate(traces_span_metrics_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"tx.process\", tx_type=~\"$tx_type\"}[5m])))", + "expr": "histogram_quantile(0.50, sum by (le, tx_type, service_instance_id) (rate(span_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"tx.process\", tx_type=~\"$tx_type\"}[5m])))", "legendFormat": "P50 {{tx_type}} [{{service_instance_id}}]" } ], @@ -126,7 +126,7 @@ "datasource": { "type": "prometheus" }, - "expr": "sum by (local, service_instance_id) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", local=~\"$tx_origin\", span_name=\"tx.process\", tx_type=~\"$tx_type\"}[5m]))", + "expr": "sum by (local, service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", local=~\"$tx_origin\", span_name=\"tx.process\", tx_type=~\"$tx_type\"}[5m]))", "legendFormat": "Local = {{local}} [{{service_instance_id}}]" } ] @@ -152,7 +152,7 @@ "datasource": { "type": "prometheus" }, - "expr": "sum by (suppressed, service_instance_id) (rate(traces_span_metrics_calls_total{span_name=\"tx.receive\", tx_type=~\"$tx_type\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]))", + "expr": "sum by (suppressed, service_instance_id) (rate(span_calls_total{span_name=\"tx.receive\", tx_type=~\"$tx_type\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]))", "legendFormat": "Suppressed={{suppressed}} [{{service_instance_id}}]" } ], @@ -195,7 +195,7 @@ "datasource": { "type": "prometheus" }, - "expr": "sum(increase(traces_span_metrics_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"tx.process\", tx_type=~\"$tx_type\"}[5m])) by (le)", + "expr": "sum(increase(span_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"tx.process\", tx_type=~\"$tx_type\"}[5m])) by (le)", "legendFormat": "{{le}}", "format": "heatmap" } @@ -227,14 +227,14 @@ "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile(0.95, sum by (le, service_instance_id) (rate(traces_span_metrics_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"tx.apply\"}[5m])))", + "expr": "histogram_quantile(0.95, sum by (le, service_instance_id) (rate(span_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"tx.apply\"}[5m])))", "legendFormat": "P95 tx.apply [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile(0.50, sum by (le, service_instance_id) (rate(traces_span_metrics_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"tx.apply\"}[5m])))", + "expr": "histogram_quantile(0.50, sum by (le, service_instance_id) (rate(span_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"tx.apply\"}[5m])))", "legendFormat": "P50 tx.apply [{{service_instance_id}}]" } ], @@ -273,7 +273,7 @@ "datasource": { "type": "prometheus" }, - "expr": "sum by (service_instance_id) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"tx.receive\", tx_type=~\"$tx_type\"}[5m]))", + "expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"tx.receive\", tx_type=~\"$tx_type\"}[5m]))", "legendFormat": "tx.receive / Sec [{{service_instance_id}}]" } ], @@ -312,7 +312,7 @@ "datasource": { "type": "prometheus" }, - "expr": "sum by (service_instance_id) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"tx.apply\", status_code=\"STATUS_CODE_ERROR\"}[5m]))", + "expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"tx.apply\", status_code=\"STATUS_CODE_ERROR\"}[5m]))", "legendFormat": "Failed / Sec [{{service_instance_id}}]" } ], @@ -365,7 +365,7 @@ "datasource": { "type": "prometheus" }, - "expr": "sum by (tx_type) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"tx.process\", tx_type=~\"$tx_type\"}[5m]))", + "expr": "sum by (tx_type) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"tx.process\", tx_type=~\"$tx_type\"}[5m]))", "legendFormat": "{{tx_type}}" } ], @@ -409,7 +409,7 @@ "datasource": { "type": "prometheus" }, - "expr": "sum by (tx_type, ter_result) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"tx.process\", tx_type=~\"$tx_type\", ter_result=~\"$ter_result\", ter_result!=\"tesSUCCESS\"}[5m]))", + "expr": "sum by (tx_type, ter_result) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"tx.process\", tx_type=~\"$tx_type\", ter_result=~\"$ter_result\", ter_result!=\"tesSUCCESS\"}[5m]))", "legendFormat": "{{tx_type}}: {{ter_result}}" } ], @@ -452,7 +452,7 @@ "datasource": { "type": "prometheus" }, - "expr": "sum by (txq_status) (increase(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"txq.accept_tx\", txq_status=~\"$txq_status\"}[5m]))", + "expr": "sum by (txq_status) (increase(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"txq.accept_tx\", txq_status=~\"$txq_status\"}[5m]))", "legendFormat": "{{txq_status}}" } ], @@ -489,7 +489,7 @@ "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile(0.95, sum by (le, tx_type, service_instance_id) (rate(traces_span_metrics_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"tx.transactor\", tx_type=~\"$tx_type\"}[5m])))", + "expr": "histogram_quantile(0.95, sum by (le, tx_type, service_instance_id) (rate(span_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"tx.transactor\", tx_type=~\"$tx_type\"}[5m])))", "legendFormat": "P95 {{tx_type}} [{{service_instance_id}}]" } ], @@ -528,7 +528,7 @@ "datasource": { "type": "prometheus" }, - "expr": "sum by (tx_type, service_instance_id) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"txq.enqueue\"}[5m]))", + "expr": "sum by (tx_type, service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"txq.enqueue\"}[5m]))", "legendFormat": "{{tx_type}} [{{service_instance_id}}]" } ], @@ -567,7 +567,7 @@ "datasource": { "type": "prometheus" }, - "expr": "sum by (service_instance_id) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"txq.apply_direct\"}[5m])) / clamp_min(sum by (service_instance_id) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"txq.apply_direct\"}[5m])) + sum by (service_instance_id) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"txq.enqueue\"}[5m])), 1)", + "expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"txq.apply_direct\"}[5m])) / clamp_min(sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"txq.apply_direct\"}[5m])) + sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"txq.enqueue\"}[5m])), 1)", "legendFormat": "Direct-Apply Fraction [{{service_instance_id}}]" } ], @@ -606,14 +606,14 @@ "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile(0.95, sum by (le, service_instance_id) (rate(traces_span_metrics_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"txq.accept\"}[5m])))", + "expr": "histogram_quantile(0.95, sum by (le, service_instance_id) (rate(span_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"txq.accept\"}[5m])))", "legendFormat": "P95 Drain [{{service_instance_id}}]" }, { "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile(0.50, sum by (le, service_instance_id) (rate(traces_span_metrics_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"txq.accept\"}[5m])))", + "expr": "histogram_quantile(0.50, sum by (le, service_instance_id) (rate(span_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"txq.accept\"}[5m])))", "legendFormat": "P50 Drain [{{service_instance_id}}]" } ], @@ -652,7 +652,7 @@ "datasource": { "type": "prometheus" }, - "expr": "sum by (service_instance_id) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"txq.cleanup\"}[5m]))", + "expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"txq.cleanup\"}[5m]))", "legendFormat": "Cleanups / Sec [{{service_instance_id}}]" } ], @@ -696,7 +696,7 @@ "datasource": { "type": "prometheus" }, - "expr": "sum by (stage, service_instance_id) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=~\"tx.preflight|tx.preclaim|tx.transactor\", stage=~\"$stage\"}[5m]))", + "expr": "sum by (stage, service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=~\"tx.preflight|tx.preclaim|tx.transactor\", stage=~\"$stage\"}[5m]))", "legendFormat": "{{stage}} [{{service_instance_id}}]" } ], @@ -740,7 +740,7 @@ "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile(0.95, sum by (le, stage, service_instance_id) (rate(traces_span_metrics_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=~\"tx.preflight|tx.preclaim|tx.transactor\", stage=~\"$stage\"}[5m])))", + "expr": "histogram_quantile(0.95, sum by (le, stage, service_instance_id) (rate(span_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=~\"tx.preflight|tx.preclaim|tx.transactor\", stage=~\"$stage\"}[5m])))", "legendFormat": "P95 {{stage}} [{{service_instance_id}}]" } ], @@ -784,7 +784,7 @@ "datasource": { "type": "prometheus" }, - "expr": "sum by (stage, service_instance_id) (rate(traces_span_metrics_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=~\"tx.preflight|tx.preclaim|tx.transactor\", stage=~\"$stage\", ter_result!~\"tesSUCCESS|\"}[5m]))", + "expr": "sum by (stage, service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=~\"tx.preflight|tx.preclaim|tx.transactor\", stage=~\"$stage\", ter_result!~\"tesSUCCESS|\"}[5m]))", "legendFormat": "{{stage}} [{{service_instance_id}}]" } ], @@ -828,7 +828,7 @@ "datasource": { "type": "prometheus" }, - "expr": "histogram_quantile(0.95, sum by (le, tx_type, stage, service_instance_id) (rate(traces_span_metrics_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=~\"tx.preflight|tx.preclaim|tx.transactor\", tx_type=~\"$tx_type\", stage=~\"$stage\"}[5m])))", + "expr": "histogram_quantile(0.95, sum by (le, tx_type, stage, service_instance_id) (rate(span_duration_milliseconds_bucket{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=~\"tx.preflight|tx.preclaim|tx.transactor\", tx_type=~\"$tx_type\", stage=~\"$stage\"}[5m])))", "legendFormat": "P95 {{tx_type}} {{stage}} [{{service_instance_id}}]" } ], @@ -916,7 +916,7 @@ "label": "Node", "description": "Filter by rippled node (service.instance.id — e.g. Node-1)", "type": "query", - "query": "label_values(traces_span_metrics_calls_total, service_instance_id)", + "query": "label_values(span_calls_total, service_instance_id)", "datasource": { "type": "prometheus", "uid": "prometheus" @@ -936,7 +936,7 @@ "label": "TX Origin", "description": "Filter by transaction origin (true = local submit, false = peer relay)", "type": "query", - "query": "label_values(traces_span_metrics_calls_total{span_name=\"tx.process\"}, local)", + "query": "label_values(span_calls_total{span_name=\"tx.process\"}, local)", "datasource": { "type": "prometheus", "uid": "prometheus" @@ -957,7 +957,7 @@ "datasource": { "type": "prometheus" }, - "query": "label_values(traces_span_metrics_calls_total{span_name=\"tx.process\", tx_type!=\"\"}, tx_type)", + "query": "label_values(span_calls_total{span_name=\"tx.process\", tx_type!=\"\"}, tx_type)", "refresh": 2, "includeAll": true, "multi": true, @@ -975,7 +975,7 @@ "datasource": { "type": "prometheus" }, - "query": "label_values(traces_span_metrics_calls_total{span_name=\"tx.process\", ter_result!=\"\"}, ter_result)", + "query": "label_values(span_calls_total{span_name=\"tx.process\", ter_result!=\"\"}, ter_result)", "refresh": 2, "includeAll": true, "multi": true, @@ -993,7 +993,7 @@ "datasource": { "type": "prometheus" }, - "query": "label_values(traces_span_metrics_calls_total{span_name=\"txq.accept_tx\", txq_status!=\"\"}, txq_status)", + "query": "label_values(span_calls_total{span_name=\"txq.accept_tx\", txq_status!=\"\"}, txq_status)", "refresh": 2, "includeAll": true, "multi": true, @@ -1011,7 +1011,7 @@ "datasource": { "type": "prometheus" }, - "query": "label_values(traces_span_metrics_calls_total{span_name=~\"tx.preflight|tx.preclaim|tx.transactor\", stage!=\"\"}, stage)", + "query": "label_values(span_calls_total{span_name=~\"tx.preflight|tx.preclaim|tx.transactor\", stage!=\"\"}, stage)", "refresh": 2, "includeAll": true, "multi": true, diff --git a/docker/telemetry/grafana/dashboards/validate_dashboards.py b/docker/telemetry/grafana/dashboards/validate_dashboards.py new file mode 100755 index 0000000000..e921d02c4f --- /dev/null +++ b/docker/telemetry/grafana/dashboards/validate_dashboards.py @@ -0,0 +1,127 @@ +#!/usr/bin/env python3 +"""Dashboard lint: cumulative metrics must be rate()-wrapped; tier filters present.""" + +import json, re, sys + +# Prometheus gauges that hold a CUMULATIVE total -> must be rate()/increase()-wrapped. +CUMULATIVE_PREFIXES = ( + "total_bytes_", + "total_messages_", + "transactions_messages_", + "transactions_duplicate_messages_", + "proposals_", + "validations_", + "overlay_peer_disconnects", + "squelch_", + "overhead_", + "validator_lists_", + "set_get_", + "set_share_", + "have_transactions_", + "requested_transactions_", + "proof_path_", + "replay_delta_", + "ledger_data_", + "ledger_get_", + "ledger_share_", + "ledger_transaction_", + "ledger_account_state_", + "getobject_", + "jq_trans_overflow_total", +) +# nodestore_state{metric=...} cumulative sub-series (raw only inside these metric= labels). +NODESTORE_CUMULATIVE = ( + "node_reads_total", + "node_reads_hit", + "node_writes", + "node_read_bytes", + "node_written_bytes", + "node_reads_duration_us", +) +# state_accounting_*_duration are cumulative µs. +STATE_DURATION = re.compile(r"state_accounting_\w+_duration") +REQUIRED_FILTERS = ( + "$node", + "$deployment_environment", + "$xrpl_network_type", + "$service_name", +) + + +def iter_panels(dash): + for p in dash.get("panels", []): + yield p + for sub in p.get("panels", []) or []: + yield sub + + +def expr_is_wrapped(expr): + return ( + "rate(" in expr + or "increase(" in expr + or "irate(" in expr + or "histogram_quantile(" in expr + ) + + +def check(path, forbid_5m): + errs = [] + try: + dash = json.load(open(path)) + except Exception as e: + return [f"{path}: INVALID JSON: {e}"] + for p in iter_panels(dash): + title = p.get("title", "") + for tg in p.get("targets", []) or []: + expr = (tg.get("expr") or "").strip() + if not expr: + continue + # tier filters + if "{" in expr: + for f in REQUIRED_FILTERS: + if f not in expr: + errs.append(f"{path} [{title}]: missing {f} in expr") + # cumulative metric plotted raw? + hits = [m for m in CUMULATIVE_PREFIXES if m in expr] + nod = [m for m in NODESTORE_CUMULATIVE if f'"{m}"' in expr] + statedur = STATE_DURATION.search(expr) + if (hits or nod or statedur) and not expr_is_wrapped(expr): + who = hits or nod or [statedur.group(0)] + errs.append( + f"{path} [{title}]: cumulative metric {who} plotted RAW (needs rate/increase)" + ) + if "deriv(" in expr or "idelta(" in expr: + errs.append( + f"{path} [{title}]: uses deriv()/idelta() on a cumulative series" + ) + if forbid_5m and ("[5m]" in expr or "[1h]" in expr): + # Only a fixed [5m]/[1h] inside a top-level rate()/increase()/irate() on a + # plain counter is a violation. Legitimate fixed-window exprs -- histogram + # buckets (rate(..._bucket[5m])), avg_over_time windows, and subqueries + # ([5m:]) -- are intentional and must not be flagged. + if not ("_bucket" in expr or "avg_over_time" in expr or ":]" in expr): + errs.append( + f"{path} [{title}]: hardcoded range window; use [$__rate_interval]" + ) + # unit check + unit = p.get("fieldConfig", {}).get("defaults", {}).get("unit", "") + if unit == "mps": + errs.append(f"{path} [{title}]: invalid Grafana unit 'mps' (use 'cps')") + return errs + + +def main(): + args = [a for a in sys.argv[1:] if not a.startswith("--")] + forbid_5m = "--no-5m" in sys.argv + all_errs = [] + for path in args: + all_errs += check(path, forbid_5m) + if all_errs: + print("\n".join(all_errs)) + print(f"\nFAIL: {len(all_errs)} violation(s)") + sys.exit(1) + print(f"OK: {len(args)} dashboard(s) passed") + + +if __name__ == "__main__": + main() diff --git a/docker/telemetry/otel-collector-config.yaml b/docker/telemetry/otel-collector-config.yaml index 5ea95c595b..1c8e214c0e 100644 --- a/docker/telemetry/otel-collector-config.yaml +++ b/docker/telemetry/otel-collector-config.yaml @@ -111,6 +111,7 @@ processors: connectors: spanmetrics: + namespace: "span" # Resource attributes that define the resource-metrics grouping key. # All resource attributes are propagated onto the output metrics # regardless; promotion to Prometheus labels is handled by the diff --git a/docs/telemetry-runbook.md b/docs/telemetry-runbook.md index ee3fb3afdf..a3cf79119a 100644 --- a/docs/telemetry-runbook.md +++ b/docs/telemetry-runbook.md @@ -223,14 +223,14 @@ PromQL on the span-derived metrics (dashboard: _Transaction Overview_): ``` # Per-stage throughput — the funnel preflight >= preclaim >= apply -sum by (stage) (rate(traces_span_metrics_calls_total{span_name=~"tx.preflight|tx.preclaim|tx.transactor"}[5m])) +sum by (stage) (rate(span_calls_total{span_name=~"tx.preflight|tx.preclaim|tx.transactor"}[5m])) # Per-stage p95 latency -histogram_quantile(0.95, sum by (le, stage) (rate(traces_span_metrics_duration_milliseconds_bucket{span_name=~"tx.preflight|tx.preclaim|tx.transactor"}[5m]))) +histogram_quantile(0.95, sum by (le, stage) (rate(span_duration_milliseconds_bucket{span_name=~"tx.preflight|tx.preclaim|tx.transactor"}[5m]))) # Per-stage failure rate (ter_result != tesSUCCESS; a failing ter completes the # span normally, so filter on the attribute, not status_code which only flags exceptions) -sum by (stage) (rate(traces_span_metrics_calls_total{span_name=~"tx.preflight|tx.preclaim|tx.transactor", ter_result!~"tesSUCCESS|"}[5m])) +sum by (stage) (rate(span_calls_total{span_name=~"tx.preflight|tx.preclaim|tx.transactor", ter_result!~"tesSUCCESS|"}[5m])) ``` > **Alerting**: a rising `tx.preflight` / `tx.preclaim` failure rate points to @@ -431,12 +431,12 @@ The OTel Collector's spanmetrics connector automatically derives RED (Rate, Erro ### Generated Metric Names -| Prometheus Metric | Type | Description | -| -------------------------------------------------- | --------- | ---------------------------- | -| `traces_span_metrics_calls_total` | Counter | Total span invocations | -| `traces_span_metrics_duration_milliseconds_bucket` | Histogram | Latency distribution buckets | -| `traces_span_metrics_duration_milliseconds_count` | Histogram | Latency observation count | -| `traces_span_metrics_duration_milliseconds_sum` | Histogram | Cumulative latency | +| Prometheus Metric | Type | Description | +| ----------------------------------- | --------- | ---------------------------- | +| `span_calls_total` | Counter | Total span invocations | +| `span_duration_milliseconds_bucket` | Histogram | Latency distribution buckets | +| `span_duration_milliseconds_count` | Histogram | Latency observation count | +| `span_duration_milliseconds_sum` | Histogram | Cumulative latency | ### Metric Labels @@ -496,38 +496,38 @@ The `OTelCollector` implementation exports metrics via OTLP/HTTP to the same OTe #### Gauges -| Prometheus Metric | Source | Description | -| ------------------------------------------- | ------------------------- | -------------------------------------------------------------------------- | -| `xrpld_LedgerMaster_Validated_Ledger_Age` | LedgerMaster.h:373 | Age of validated ledger (seconds) | -| `xrpld_LedgerMaster_Published_Ledger_Age` | LedgerMaster.h:374 | Age of published ledger (seconds) | -| `xrpld_State_Accounting_{Mode}_duration` | NetworkOPs.cpp:774 | Time in each operating mode (Disconnected/Connected/Syncing/Tracking/Full) | -| `xrpld_State_Accounting_{Mode}_transitions` | NetworkOPs.cpp:780 | Transition count per mode | -| `xrpld_Peer_Finder_Active_Inbound_Peers` | PeerfinderManager.cpp:214 | Active inbound peer connections | -| `xrpld_Peer_Finder_Active_Outbound_Peers` | PeerfinderManager.cpp:215 | Active outbound peer connections | -| `xrpld_Overlay_Peer_Disconnects` | OverlayImpl.h:557 | Peer disconnect count | -| `xrpld_job_count` | JobQueue.cpp:26 | Current job queue depth | -| `xrpld_{category}_Bytes_In/Out` | OverlayImpl.h:535 | Overlay traffic bytes per category (57 categories) | -| `xrpld_{category}_Messages_In/Out` | OverlayImpl.h:535 | Overlay traffic messages per category | +| Prometheus Metric | Source | Description | +| ------------------------------------- | ------------------------- | -------------------------------------------------------------------------- | +| `ledgermaster_validated_ledger_age` | LedgerMaster.h:373 | Age of validated ledger (seconds) | +| `ledgermaster_published_ledger_age` | LedgerMaster.h:374 | Age of published ledger (seconds) | +| `state_accounting_{Mode}_duration` | NetworkOPs.cpp:774 | Time in each operating mode (Disconnected/Connected/Syncing/Tracking/Full) | +| `state_accounting_{Mode}_transitions` | NetworkOPs.cpp:780 | Transition count per mode | +| `peer_finder_active_inbound_peers` | PeerfinderManager.cpp:214 | Active inbound peer connections | +| `peer_finder_active_outbound_peers` | PeerfinderManager.cpp:215 | Active outbound peer connections | +| `overlay_peer_disconnects` | OverlayImpl.h:557 | Peer disconnect count | +| `job_count` | JobQueue.cpp:26 | Current job queue depth | +| `{category}_bytes_in/out` | OverlayImpl.h:535 | Overlay traffic bytes per category (57 categories) | +| `{category}_messages_in/out` | OverlayImpl.h:535 | Overlay traffic messages per category | #### Counters -| Prometheus Metric | Source | Description | -| ------------------------------- | --------------------- | ------------------------------ | -| `xrpld_rpc_requests` | ServerHandler.cpp:108 | Total RPC request count | -| `xrpld_ledger_fetches` | InboundLedgers.cpp:44 | Ledger fetch request count | -| `xrpld_ledger_history_mismatch` | LedgerHistory.cpp:16 | Ledger hash mismatch count | -| `xrpld_warn` | Logic.h:33 | Resource manager warning count | -| `xrpld_drop` | Logic.h:34 | Resource manager drop count | +| Prometheus Metric | Source | Description | +| ------------------------- | --------------------- | ------------------------------ | +| `rpc_requests` | ServerHandler.cpp:108 | Total RPC request count | +| `ledger_fetches` | InboundLedgers.cpp:44 | Ledger fetch request count | +| `ledger_history_mismatch` | LedgerHistory.cpp:16 | Ledger hash mismatch count | +| `warn` | Logic.h:33 | Resource manager warning count | +| `drop` | Logic.h:34 | Resource manager drop count | #### Histograms -| Prometheus Metric | Source | Description | -| --------------------- | --------------------- | ------------------------------ | -| `xrpld_rpc_time` | ServerHandler.cpp:110 | RPC response time (ms) | -| `xrpld_rpc_size` | ServerHandler.cpp:109 | RPC response size (bytes) | -| `xrpld_ios_latency` | Application.cpp:438 | I/O service loop latency (ms) | -| `xrpld_pathfind_fast` | PathRequests.h:23 | Fast pathfinding duration (ms) | -| `xrpld_pathfind_full` | PathRequests.h:24 | Full pathfinding duration (ms) | +| Prometheus Metric | Source | Description | +| ----------------- | --------------------- | ------------------------------ | +| `rpc_time` | ServerHandler.cpp:110 | RPC response time (ms) | +| `rpc_size` | ServerHandler.cpp:109 | RPC response size (bytes) | +| `ios_latency` | Application.cpp:438 | I/O service loop latency (ms) | +| `pathfind_fast` | PathRequests.h:23 | Fast pathfinding duration (ms) | +| `pathfind_full` | PathRequests.h:24 | Full pathfinding duration (ms) | ## Deployment Tiers @@ -619,44 +619,44 @@ Ten dashboards are pre-provisioned in `docker/telemetry/grafana/dashboards/`: ### RPC Performance (`rpc-performance`) -| Panel | Type | PromQL | Labels Used | -| --------------------------- | ---------- | ----------------------------------------------------------------------------------------------------------------------------------------- | ------------------------ | -| RPC Request Rate by Command | timeseries | `sum by (command) (rate(traces_span_metrics_calls_total{span_name=~"rpc.command.*"}[5m]))` | `command` | -| RPC Latency p95 by Command | timeseries | `histogram_quantile(0.95, sum by (le, command) (rate(traces_span_metrics_duration_milliseconds_bucket{span_name=~"rpc.command.*"}[5m])))` | `command` | -| RPC Error Rate | bargauge | Error spans / total spans × 100, grouped by `command` | `command`, `status_code` | -| RPC Latency Heatmap | heatmap | `sum(increase(traces_span_metrics_duration_milliseconds_bucket{span_name=~"rpc.command.*"}[5m])) by (le)` | `le` (bucket boundaries) | -| Overall RPC Throughput | timeseries | `rpc.request` + `rpc.process` rate | — | -| RPC Success vs Error | timeseries | by `status_code` (UNSET vs ERROR) | `status_code` | -| Top Commands by Volume | bargauge | `topk(10, ...)` by `command` | `command` | -| WebSocket Message Rate | stat | `rpc.ws_message` rate | — | +| Panel | Type | PromQL | Labels Used | +| --------------------------- | ---------- | -------------------------------------------------------------------------------------------------------------------------- | ------------------------ | +| RPC Request Rate by Command | timeseries | `sum by (command) (rate(span_calls_total{span_name=~"rpc.command.*"}[5m]))` | `command` | +| RPC Latency p95 by Command | timeseries | `histogram_quantile(0.95, sum by (le, command) (rate(span_duration_milliseconds_bucket{span_name=~"rpc.command.*"}[5m])))` | `command` | +| RPC Error Rate | bargauge | Error spans / total spans × 100, grouped by `command` | `command`, `status_code` | +| RPC Latency Heatmap | heatmap | `sum(increase(span_duration_milliseconds_bucket{span_name=~"rpc.command.*"}[5m])) by (le)` | `le` (bucket boundaries) | +| Overall RPC Throughput | timeseries | `rpc.request` + `rpc.process` rate | — | +| RPC Success vs Error | timeseries | by `status_code` (UNSET vs ERROR) | `status_code` | +| Top Commands by Volume | bargauge | `topk(10, ...)` by `command` | `command` | +| WebSocket Message Rate | stat | `rpc.ws_message` rate | — | ### Transaction Overview (`transaction-overview`) -| Panel | Type | PromQL | Labels Used | -| --------------------------------- | ---------- | ------------------------------------------------------------------------------------ | ------------- | -| Transaction Processing Rate | timeseries | `rate(traces_span_metrics_calls_total{span_name="tx.process"}[5m])` and `tx.receive` | `span_name` | -| Transaction Processing Latency | timeseries | `histogram_quantile(0.95 / 0.50, ... {span_name="tx.process"})` | — | -| Transaction Path Distribution | piechart | `sum by (local) (rate(traces_span_metrics_calls_total{span_name="tx.process"}[5m]))` | `local` | -| Transaction Receive vs Suppressed | timeseries | `rate(traces_span_metrics_calls_total{span_name="tx.receive"}[5m])` | — | -| TX Processing Duration Heatmap | heatmap | `tx.process` histogram buckets | `le` | -| TX Apply Duration per Ledger | timeseries | p95/p50 of `tx.apply` | — | -| Peer TX Receive Rate | timeseries | `tx.receive` rate | — | -| TX Apply Failed Rate | stat | `tx.apply` with `STATUS_CODE_ERROR` | `status_code` | +| Panel | Type | PromQL | Labels Used | +| --------------------------------- | ---------- | --------------------------------------------------------------------- | ------------- | +| Transaction Processing Rate | timeseries | `rate(span_calls_total{span_name="tx.process"}[5m])` and `tx.receive` | `span_name` | +| Transaction Processing Latency | timeseries | `histogram_quantile(0.95 / 0.50, ... {span_name="tx.process"})` | — | +| Transaction Path Distribution | piechart | `sum by (local) (rate(span_calls_total{span_name="tx.process"}[5m]))` | `local` | +| Transaction Receive vs Suppressed | timeseries | `rate(span_calls_total{span_name="tx.receive"}[5m])` | — | +| TX Processing Duration Heatmap | heatmap | `tx.process` histogram buckets | `le` | +| TX Apply Duration per Ledger | timeseries | p95/p50 of `tx.apply` | — | +| Peer TX Receive Rate | timeseries | `tx.receive` rate | — | +| TX Apply Failed Rate | stat | `tx.apply` with `STATUS_CODE_ERROR` | `status_code` | ### Consensus Health (`consensus-health`) -| Panel | Type | PromQL | Labels Used | -| ----------------------------- | ---------- | ---------------------------------------------------------------------------------- | ---------------- | -| Consensus Round Duration | timeseries | `histogram_quantile(0.95 / 0.50, ... {span_name="consensus.accept"})` | — | -| Consensus Proposals Sent Rate | timeseries | `rate(traces_span_metrics_calls_total{span_name="consensus.proposal.send"}[5m])` | — | -| Ledger Close Duration | timeseries | `histogram_quantile(0.95, ... {span_name="consensus.ledger_close"})` | — | -| Validation Send Rate | stat | `rate(traces_span_metrics_calls_total{span_name="consensus.validation.send"}[5m])` | — | -| Ledger Apply Duration | timeseries | `histogram_quantile(0.95 / 0.50, ... {span_name="consensus.accept.apply"})` | — | -| Close Time Agreement | timeseries | `rate(traces_span_metrics_calls_total{span_name="consensus.accept.apply"}[5m])` | — | -| Consensus Mode Over Time | timeseries | `consensus.ledger_close` by `consensus_mode` | `consensus_mode` | -| Accept vs Close Rate | timeseries | `consensus.accept` vs `consensus.ledger_close` rate | — | -| Validation vs Close Rate | timeseries | `consensus.validation.send` vs `consensus.ledger_close` | — | -| Accept Duration Heatmap | heatmap | `consensus.accept` histogram buckets | `le` | +| Panel | Type | PromQL | Labels Used | +| ----------------------------- | ---------- | --------------------------------------------------------------------------- | ---------------- | +| Consensus Round Duration | timeseries | `histogram_quantile(0.95 / 0.50, ... {span_name="consensus.accept"})` | — | +| Consensus Proposals Sent Rate | timeseries | `rate(span_calls_total{span_name="consensus.proposal.send"}[5m])` | — | +| Ledger Close Duration | timeseries | `histogram_quantile(0.95, ... {span_name="consensus.ledger_close"})` | — | +| Validation Send Rate | stat | `rate(span_calls_total{span_name="consensus.validation.send"}[5m])` | — | +| Ledger Apply Duration | timeseries | `histogram_quantile(0.95 / 0.50, ... {span_name="consensus.accept.apply"})` | — | +| Close Time Agreement | timeseries | `rate(span_calls_total{span_name="consensus.accept.apply"}[5m])` | — | +| Consensus Mode Over Time | timeseries | `consensus.ledger_close` by `consensus_mode` | `consensus_mode` | +| Accept vs Close Rate | timeseries | `consensus.accept` vs `consensus.ledger_close` rate | — | +| Validation vs Close Rate | timeseries | `consensus.validation.send` vs `consensus.ledger_close` | — | +| Accept Duration Heatmap | heatmap | `consensus.accept` histogram buckets | `le` | ### Ledger Operations (`ledger-operations`) @@ -684,52 +684,52 @@ Requires `trace_peer=1` in the `[telemetry]` config section. ### Node Health -- System Metrics (`node-health`) -| Panel | Type | PromQL | Labels Used | -| -------------------------------------- | ---------- | --------------------------------------------------------------- | ----------- | -| Validated Ledger Age | stat | `xrpld_LedgerMaster_Validated_Ledger_Age` | — | -| Published Ledger Age | stat | `xrpld_LedgerMaster_Published_Ledger_Age` | — | -| Operating Mode Duration | timeseries | `xrpld_State_Accounting_*_duration` | — | -| Operating Mode Transitions | timeseries | `xrpld_State_Accounting_*_transitions` | — | -| I/O Latency | timeseries | `histogram_quantile(0.95, xrpld_ios_latency_bucket)` | — | -| Job Queue Depth | timeseries | `xrpld_job_count` | — | -| Ledger Fetch Rate | stat | `rate(xrpld_ledger_fetches[5m])` | — | -| Ledger History Mismatches | stat | `rate(xrpld_ledger_history_mismatch[5m])` | — | -| Key Jobs Execution Time | timeseries | `xrpld_acceptLedger{quantile="$quantile"}` (+ 10 more key jobs) | `quantile` | -| Key Jobs Dequeue Wait Time | timeseries | `xrpld_acceptLedger_q{quantile="$quantile"}` (+ 10 more) | `quantile` | -| FullBelowCache Size | timeseries | `xrpld_Node_family_full_below_cache_size` | — | -| FullBelowCache Hit Rate | gauge | `xrpld_Node_family_full_below_cache_hit_rate` | — | -| Ledger Publish Gap | stat | `Published_Ledger_Age - Validated_Ledger_Age` | — | -| State Duration Rate (Full vs Tracking) | timeseries | `rate(xrpld_State_Accounting_Full_duration[5m]) / 1000000` | — | -| All Jobs Execution Time (Detail) | timeseries | `{__name__=~"xrpld_", quantile="$quantile"}` | `quantile` | -| All Jobs Dequeue Wait (Detail) | timeseries | `{__name__=~"xrpld__q", quantile="$quantile"}` | `quantile` | +| Panel | Type | PromQL | Labels Used | +| -------------------------------------- | ---------- | --------------------------------------------------------- | ----------- | +| Validated Ledger Age | stat | `ledgermaster_validated_ledger_age` | — | +| Published Ledger Age | stat | `ledgermaster_published_ledger_age` | — | +| Operating Mode Duration | timeseries | `state_accounting_*_duration` | — | +| Operating Mode Transitions | timeseries | `state_accounting_*_transitions` | — | +| I/O Latency | timeseries | `histogram_quantile(0.95, ios_latency_bucket)` | — | +| Job Queue Depth | timeseries | `job_count` | — | +| Ledger Fetch Rate | stat | `rate(ledger_fetches[5m])` | — | +| Ledger History Mismatches | stat | `rate(ledger_history_mismatch[5m])` | — | +| Key Jobs Execution Time | timeseries | `acceptledger{quantile="$quantile"}` (+ 10 more key jobs) | `quantile` | +| Key Jobs Dequeue Wait Time | timeseries | `acceptledger_q{quantile="$quantile"}` (+ 10 more) | `quantile` | +| FullBelowCache Size | timeseries | `node_family_full_below_cache_size` | — | +| FullBelowCache Hit Rate | gauge | `node_family_full_below_cache_hit_rate` | — | +| Ledger Publish Gap | stat | `Published_Ledger_Age - Validated_Ledger_Age` | — | +| State Duration Rate (Full vs Tracking) | timeseries | `rate(state_accounting_full_duration[5m]) / 1000000` | — | +| All Jobs Execution Time (Detail) | timeseries | `{__name__=~"", quantile="$quantile"}` | `quantile` | +| All Jobs Dequeue Wait (Detail) | timeseries | `{__name__=~"_q", quantile="$quantile"}` | `quantile` | ### Network Traffic -- System Metrics (`network-traffic`) -| Panel | Type | PromQL | Labels Used | -| ------------------------------------ | ---------- | ------------------------------------------ | ----------- | -| Active Peers | timeseries | `xrpld_Peer_Finder_Active_*_Peers` | — | -| Peer Disconnects | timeseries | `xrpld_Overlay_Peer_Disconnects` | — | -| Total Network Bytes | timeseries | `rate(xrpld_total_Bytes_In/Out[5m])` | — | -| Total Network Messages | timeseries | `xrpld_total_Messages_In/Out` | — | -| Transaction Traffic | timeseries | `xrpld_transactions_Messages_In/Out` | — | -| Proposal Traffic | timeseries | `xrpld_proposals_Messages_In/Out` | — | -| Validation Traffic | timeseries | `xrpld_validations_Messages_In/Out` | — | -| Traffic by Category | bargauge | `topk(10, xrpld_*_Bytes_In)` | — | -| Duplicate Traffic (Wasted Bandwidth) | timeseries | `rate(xrpld_*_duplicate_Bytes_In/Out[5m])` | — | -| All Traffic Categories (Detail) | timeseries | `topk(15, rate(xrpld_*_Bytes_In[5m]))` | — | +| Panel | Type | PromQL | Labels Used | +| ------------------------------------ | ---------- | ------------------------------------ | ----------- | +| Active Peers | timeseries | `peer_finder_active_*_peers` | — | +| Peer Disconnects | timeseries | `overlay_peer_disconnects` | — | +| Total Network Bytes | timeseries | `rate(total_bytes_in/out[5m])` | — | +| Total Network Messages | timeseries | `total_messages_in/out` | — | +| Transaction Traffic | timeseries | `transactions_messages_in/out` | — | +| Proposal Traffic | timeseries | `proposals_messages_in/out` | — | +| Validation Traffic | timeseries | `validations_messages_in/out` | — | +| Traffic by Category | bargauge | `topk(10, *_bytes_in)` | — | +| Duplicate Traffic (Wasted Bandwidth) | timeseries | `rate(*_duplicate_bytes_in/out[5m])` | — | +| All Traffic Categories (Detail) | timeseries | `topk(15, rate(*_bytes_in[5m]))` | — | ### RPC & Pathfinding -- System Metrics (`rpc-pathfinding`) -| Panel | Type | PromQL | Labels Used | -| ------------------------- | ---------- | ------------------------------------------------------ | ----------- | -| RPC Request Rate | stat | `rate(xrpld_rpc_requests[5m])` | — | -| RPC Response Time | timeseries | `histogram_quantile(0.95, xrpld_rpc_time_bucket)` | — | -| RPC Response Size | timeseries | `histogram_quantile(0.95, xrpld_rpc_size_bucket)` | — | -| RPC Response Time Heatmap | heatmap | `xrpld_rpc_time_bucket` | — | -| Pathfinding Fast Duration | timeseries | `histogram_quantile(0.95, xrpld_pathfind_fast_bucket)` | — | -| Pathfinding Full Duration | timeseries | `histogram_quantile(0.95, xrpld_pathfind_full_bucket)` | — | -| Resource Warnings Rate | stat | `rate(xrpld_warn[5m])` | — | -| Resource Drops Rate | stat | `rate(xrpld_drop[5m])` | — | +| Panel | Type | PromQL | Labels Used | +| ------------------------- | ---------- | ------------------------------------------------ | ----------- | +| RPC Request Rate | stat | `rate(rpc_requests[5m])` | — | +| RPC Response Time | timeseries | `histogram_quantile(0.95, rpc_time_bucket)` | — | +| RPC Response Size | timeseries | `histogram_quantile(0.95, rpc_size_bucket)` | — | +| RPC Response Time Heatmap | heatmap | `rpc_time_bucket` | — | +| Pathfinding Fast Duration | timeseries | `histogram_quantile(0.95, pathfind_fast_bucket)` | — | +| Pathfinding Full Duration | timeseries | `histogram_quantile(0.95, pathfind_full_bucket)` | — | +| Resource Warnings Rate | stat | `rate(warn[5m])` | — | +| Resource Drops Rate | stat | `rate(drop[5m])` | — | ### Span → Metric → Dashboard Summary @@ -831,7 +831,7 @@ count_over_time({job="xrpld"} |= "trace_id=" [5m]) 2. Verify `server=otel` in the `[insight]` config section 3. Verify the endpoint in `[insight]` points to the OTLP/HTTP port (default: `http://localhost:4318/v1/metrics`) 4. Check that the `otlp` receiver is in the metrics pipeline receivers in `otel-collector-config.yaml` -5. Query Prometheus directly: `curl 'http://localhost:9090/api/v1/query?query=xrpld_job_count'` +5. Query Prometheus directly: `curl 'http://localhost:9090/api/v1/query?query=job_count'` ### High memory usage