mirror of
https://github.com/XRPLF/rippled.git
synced 2026-07-28 17:40:25 +00:00
fix(telemetry): phase-6 statsd dashboards, config, and doc corrections
- Fix invalid PromQL in statsd dashboards (rate/topk/nested-__name__ label selectors); add missing $node filter on a binary-op operand in statsd-node-health "Ledger Publish Gap". - Wrap cumulative counters in rate(); correct Bytes_In metric. - Pin prometheus image; add [insight] server=statsd stanza (prefix rippled matches the rippled_* StatsD metric names the dashboards query); bind statsd port to loopback; fix Jaeger->Grafana comment. - Correct dotted attribute keys to underscore form in 09-data-collection-reference (peer_id, tx_hash, consensus_mode, ...). - Reconcile span count (11) and proposers attribute in the integration-test task list. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -239,36 +239,36 @@ Every span can carry key-value attributes that provide context for filtering and
|
||||
|
||||
#### Transaction Attributes
|
||||
|
||||
| Attribute | Type | Set On | Description |
|
||||
| ------------------- | ------- | ---------------------------------------------- | --------------------------------------------------------------------- |
|
||||
| `xrpl.tx.hash` | string | `tx.process`, `tx.receive` | Transaction hash (hex-encoded) |
|
||||
| `local` | boolean | `tx.process` | `true` if locally submitted, `false` if peer-relayed |
|
||||
| `path` | string | `tx.process` | Submission path: `"sync"` or `"async"` |
|
||||
| `suppressed` | boolean | `tx.receive` | `true` if transaction was suppressed (duplicate) |
|
||||
| `tx_status` | string | `tx.receive` | Transaction status (e.g., `"known_bad"`) |
|
||||
| `xrpl.peer.id` | int64 | `tx.receive` | Peer identifier (also set on peer spans) |
|
||||
| `xrpl.peer.version` | string | `tx.receive` | Peer protocol version string |
|
||||
| `stage` | string | `tx.preflight`, `tx.preclaim`, `tx.transactor` | Apply-pipeline stage: `preflight`, `preclaim`, or `apply` |
|
||||
| `tx_type` | string | `tx.preflight`, `tx.preclaim`, `tx.transactor` | Transaction type name (e.g., `Payment`) |
|
||||
| `ter_result` | string | `tx.preflight`, `tx.preclaim`, `tx.transactor` | Engine result token for that stage (e.g., `tesSUCCESS`, `terPRE_SEQ`) |
|
||||
| `applied` | boolean | `tx.transactor` | `true` if the transaction was applied to the ledger |
|
||||
| Attribute | Type | Set On | Description |
|
||||
| -------------- | ------- | ---------------------------------------------- | --------------------------------------------------------------------- |
|
||||
| `tx_hash` | string | `tx.process`, `tx.receive` | Transaction hash (hex-encoded) |
|
||||
| `local` | boolean | `tx.process` | `true` if locally submitted, `false` if peer-relayed |
|
||||
| `path` | string | `tx.process` | Submission path: `"sync"` or `"async"` |
|
||||
| `suppressed` | boolean | `tx.receive` | `true` if transaction was suppressed (duplicate) |
|
||||
| `tx_status` | string | `tx.receive` | Transaction status (e.g., `"known_bad"`) |
|
||||
| `peer_id` | int64 | `tx.receive` | Peer identifier (also set on peer spans) |
|
||||
| `peer_version` | string | `tx.receive` | Peer protocol version string |
|
||||
| `stage` | string | `tx.preflight`, `tx.preclaim`, `tx.transactor` | Apply-pipeline stage: `preflight`, `preclaim`, or `apply` |
|
||||
| `tx_type` | string | `tx.preflight`, `tx.preclaim`, `tx.transactor` | Transaction type name (e.g., `Payment`) |
|
||||
| `ter_result` | string | `tx.preflight`, `tx.preclaim`, `tx.transactor` | Engine result token for that stage (e.g., `tesSUCCESS`, `terPRE_SEQ`) |
|
||||
| `applied` | boolean | `tx.transactor` | `true` if the transaction was applied to the ledger |
|
||||
|
||||
**Tempo query**: `{span.xrpl.tx.hash="<hash>"}` to trace a specific transaction across nodes.
|
||||
**Tempo query**: `{span.tx_hash="<hash>"}` to trace a specific transaction across nodes.
|
||||
|
||||
**Prometheus label**: `xrpl_tx_local` (used as SpanMetrics dimension).
|
||||
|
||||
#### PathFind Attributes
|
||||
|
||||
| Attribute | Type | Set On | Description |
|
||||
| ---------------------------- | ------- | --------------------- | ----------------------------------------------- |
|
||||
| `source_account` | string | `pathfind.request` | Source account address |
|
||||
| `dest_account` | string | `pathfind.request` | Destination account address |
|
||||
| `fast` | boolean | `pathfind.compute` | Whether this is a fast (non-full) pathfind |
|
||||
| `search_level` | int64 | `pathfind.compute` | Search depth level |
|
||||
| `num_complete_paths` | int64 | `pathfind.compute` | Number of complete paths found |
|
||||
| `num_paths` | int64 | `pathfind.compute` | Total number of paths explored |
|
||||
| `num_requests` | int64 | `pathfind.update_all` | Number of active path requests being recomputed |
|
||||
| `xrpl.pathfind.ledger_index` | int64 | `pathfind.update_all` | Ledger index used for recomputation |
|
||||
| Attribute | Type | Set On | Description |
|
||||
| ----------------------- | ------- | --------------------- | ----------------------------------------------- |
|
||||
| `source_account` | string | `pathfind.request` | Source account address |
|
||||
| `dest_account` | string | `pathfind.request` | Destination account address |
|
||||
| `fast` | boolean | `pathfind.compute` | Whether this is a fast (non-full) pathfind |
|
||||
| `search_level` | int64 | `pathfind.compute` | Search depth level |
|
||||
| `num_complete_paths` | int64 | `pathfind.compute` | Number of complete paths found |
|
||||
| `num_paths` | int64 | `pathfind.compute` | Total number of paths explored |
|
||||
| `num_requests` | int64 | `pathfind.update_all` | Number of active path requests being recomputed |
|
||||
| `pathfind_ledger_index` | int64 | `pathfind.update_all` | Ledger index used for recomputation |
|
||||
|
||||
**Tempo query**: `{span.source_account="rHb9..."}` to find pathfind requests from a specific account.
|
||||
|
||||
@@ -276,13 +276,13 @@ Every span can carry key-value attributes that provide context for filtering and
|
||||
|
||||
| Attribute | Type | Set On | Description |
|
||||
| -------------------- | ------- | ------------------------------ | ---------------------------------------------------------- |
|
||||
| `xrpl.tx.hash` | string | `txq.enqueue`, `txq.accept.tx` | Transaction hash in the queue |
|
||||
| `tx_hash` | string | `txq.enqueue`, `txq.accept.tx` | Transaction hash in the queue |
|
||||
| `txq_status` | string | `txq.enqueue` | Queue result: `"queued"`, `"applied_direct"`, `"rejected"` |
|
||||
| `fee_level_paid` | int64 | `txq.enqueue` | Fee level paid by the transaction |
|
||||
| `required_fee_level` | int64 | `txq.enqueue` | Minimum fee level required for queue admission |
|
||||
| `queue_size` | int64 | `txq.accept` | Queue depth at start of accept |
|
||||
| `ledger_changed` | boolean | `txq.accept` | Whether the open ledger changed since last accept |
|
||||
| `xrpl.ledger.seq` | int64 | `txq.cleanup` | Ledger sequence for cleanup |
|
||||
| `ledger_seq` | int64 | `txq.cleanup` | Ledger sequence for cleanup |
|
||||
| `expired_count` | int64 | `txq.cleanup` | Number of expired transactions removed |
|
||||
| `ter_code` | string | `txq.accept.tx` | Transaction engine result code |
|
||||
| `retries_remaining` | int64 | `txq.accept.tx` | Remaining retry attempts for this transaction |
|
||||
@@ -304,10 +304,10 @@ Every span can carry key-value attributes that provide context for filtering and
|
||||
|
||||
| Attribute | Type | Set On | Description |
|
||||
| --------------------------- | ------- | ---------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------- |
|
||||
| `xrpl.consensus.ledger_id` | string | `consensus.round` | Previous ledger hash (used for deterministic trace ID) |
|
||||
| `xrpl.ledger.seq` | int64 | `consensus.round`, `consensus.ledger_close`, `consensus.accept`, `consensus.validation.send`, `consensus.accept.apply` | Ledger sequence number |
|
||||
| `xrpl.consensus.mode` | string | `consensus.round`, `consensus.proposal.send`, `consensus.ledger_close` | Node mode via `toDisplayString()`: `"Proposing"`, `"Observing"`, etc. |
|
||||
| `xrpl.consensus.round` | int64 | `consensus.proposal.send` | Consensus round number |
|
||||
| `consensus_ledger_id` | string | `consensus.round` | Previous ledger hash (used for deterministic trace ID) |
|
||||
| `ledger_seq` | int64 | `consensus.round`, `consensus.ledger_close`, `consensus.accept`, `consensus.validation.send`, `consensus.accept.apply` | Ledger sequence number |
|
||||
| `consensus_mode` | string | `consensus.round`, `consensus.proposal.send`, `consensus.ledger_close` | Node mode via `toDisplayString()`: `"Proposing"`, `"Observing"`, etc. |
|
||||
| `consensus_round` | int64 | `consensus.proposal.send` | Consensus round number |
|
||||
| `proposers` | int64 | `consensus.proposal.send`, `consensus.accept` | Number of proposers in the round |
|
||||
| `round_time_ms` | int64 | `consensus.accept`, `consensus.accept.apply` | Total consensus round duration in milliseconds |
|
||||
| `proposing` | boolean | `consensus.validation.send` | Whether this node was a proposer |
|
||||
@@ -332,15 +332,15 @@ Every span can carry key-value attributes that provide context for filtering and
|
||||
| `quorum` | int64 | `consensus.check` | Required quorum for validation |
|
||||
| `validation_count` | int64 | `consensus.check` | Number of validations received |
|
||||
| `trace_strategy` | string | `consensus.round` | Trace sampling strategy used for this round |
|
||||
| `xrpl.consensus.round_id` | string | `consensus.round` | Deterministic round identifier |
|
||||
| `xrpl.consensus.mode.old` | string | `consensus.mode_change` | Previous consensus mode |
|
||||
| `xrpl.consensus.mode.new` | string | `consensus.mode_change` | New consensus mode |
|
||||
| `xrpl.tx.id` | string | `consensus.update_positions` | Disputed transaction ID |
|
||||
| `consensus_round_id` | string | `consensus.round` | Deterministic round identifier |
|
||||
| `mode_old` | string | `consensus.mode_change` | Previous consensus mode |
|
||||
| `mode_new` | string | `consensus.mode_change` | New consensus mode |
|
||||
| `tx_id` | string | `consensus.update_positions` | Disputed transaction ID |
|
||||
| `dispute_our_vote` | boolean | `consensus.update_positions` | Our vote on the disputed transaction |
|
||||
| `dispute_yays` | int64 | `consensus.update_positions` | Number of proposers voting to include |
|
||||
| `dispute_nays` | int64 | `consensus.update_positions` | Number of proposers voting to exclude |
|
||||
|
||||
**Tempo query**: `{span.xrpl.consensus.mode="Proposing"}` to find rounds where node was proposing.
|
||||
**Tempo query**: `{span.consensus_mode="Proposing"}` to find rounds where node was proposing.
|
||||
|
||||
**Prometheus label**: `xrpl_consensus_mode` (used as SpanMetrics dimension).
|
||||
|
||||
@@ -348,7 +348,7 @@ Every span can carry key-value attributes that provide context for filtering and
|
||||
|
||||
| Attribute | Type | Set On | Description |
|
||||
| --------------------- | ------- | ------------------------------------------------------------- | ------------------------------------------------ |
|
||||
| `xrpl.ledger.seq` | int64 | `ledger.build`, `ledger.validate`, `ledger.store`, `tx.apply` | Ledger sequence number |
|
||||
| `ledger_seq` | int64 | `ledger.build`, `ledger.validate`, `ledger.store`, `tx.apply` | Ledger sequence number |
|
||||
| `close_time` | int64 | `ledger.build` | Ledger close time (epoch seconds) |
|
||||
| `close_time_correct` | boolean | `ledger.build` | Whether close time was agreed upon by validators |
|
||||
| `close_resolution_ms` | int64 | `ledger.build` | Close time rounding granularity in milliseconds |
|
||||
@@ -356,15 +356,15 @@ Every span can carry key-value attributes that provide context for filtering and
|
||||
| `tx_failed` | int64 | `ledger.build`, `tx.apply` | Failed transactions in the ledger |
|
||||
| `validations` | int64 | `ledger.validate` | Number of validations received for this ledger |
|
||||
|
||||
**Tempo query**: `{span.xrpl.ledger.seq=12345}` to find all spans for a specific ledger.
|
||||
**Tempo query**: `{span.ledger_seq=12345}` to find all spans for a specific ledger.
|
||||
|
||||
#### Peer Attributes
|
||||
|
||||
| Attribute | Type | Set On | Description |
|
||||
| -------------------- | ------- | ---------------------------------------------------------------- | ---------------------------------------------------- |
|
||||
| `xrpl.peer.id` | int64 | `tx.receive`, `peer.proposal.receive`, `peer.validation.receive` | Peer identifier |
|
||||
| `peer_id` | int64 | `tx.receive`, `peer.proposal.receive`, `peer.validation.receive` | Peer identifier |
|
||||
| `proposal_trusted` | boolean | `peer.proposal.receive` | Whether the proposal came from a trusted validator |
|
||||
| `xrpl.ledger.hash` | string | `peer.validation.receive` | Ledger hash the validation refers to |
|
||||
| `ledger_hash` | string | `peer.validation.receive` | Ledger hash the validation refers to |
|
||||
| `validation_full` | boolean | `peer.validation.receive` | Whether this is a full (not partial) validation |
|
||||
| `validation_trusted` | boolean | `peer.validation.receive` | Whether the validation came from a trusted validator |
|
||||
|
||||
@@ -389,15 +389,15 @@ The OTel Collector's SpanMetrics connector automatically generates RED (Rate, Er
|
||||
|
||||
**Additional dimension labels** (configured in `otel-collector-config.yaml`):
|
||||
|
||||
| Span Attribute | Prometheus Label | Applies To |
|
||||
| --------------------- | ------------------------------ | ---------------------------------------------- |
|
||||
| `command` | `xrpl_rpc_command` | `rpc.command.*` |
|
||||
| `rpc_status` | `xrpl_rpc_status` | `rpc.command.*` |
|
||||
| `xrpl.consensus.mode` | `xrpl_consensus_mode` | `consensus.ledger_close` |
|
||||
| `local` | `xrpl_tx_local` | `tx.process` |
|
||||
| `proposal_trusted` | `xrpl_peer_proposal_trusted` | `peer.proposal.receive` |
|
||||
| `validation_trusted` | `xrpl_peer_validation_trusted` | `peer.validation.receive` |
|
||||
| `stage` | `stage` | `tx.preflight`, `tx.preclaim`, `tx.transactor` |
|
||||
| Span Attribute | Prometheus Label | Applies To |
|
||||
| -------------------- | ------------------------------ | ---------------------------------------------- |
|
||||
| `command` | `xrpl_rpc_command` | `rpc.command.*` |
|
||||
| `rpc_status` | `xrpl_rpc_status` | `rpc.command.*` |
|
||||
| `consensus_mode` | `xrpl_consensus_mode` | `consensus.ledger_close` |
|
||||
| `local` | `xrpl_tx_local` | `tx.process` |
|
||||
| `proposal_trusted` | `xrpl_peer_proposal_trusted` | `peer.proposal.receive` |
|
||||
| `validation_trusted` | `xrpl_peer_validation_trusted` | `peer.validation.receive` |
|
||||
| `stage` | `stage` | `tx.preflight`, `tx.preclaim`, `tx.transactor` |
|
||||
|
||||
The `stage` dimension (3 values: `preflight`, `preclaim`, `apply`) turns the
|
||||
apply-pipeline spans into per-stage RED metrics with no native instruments — the
|
||||
@@ -614,11 +614,11 @@ Prometheus label. See [telemetry-runbook.md](../docs/telemetry-runbook.md)
|
||||
| Specific RPC command | `{resource.service.name="xrpld" && name="rpc.command.server_info"}` |
|
||||
| Slow RPC calls | `{resource.service.name="xrpld" && name=~"rpc.command.*"} \| duration > 100ms` |
|
||||
| Failed RPC calls | `{span.rpc_status="error"}` |
|
||||
| Specific transaction | `{span.xrpl.tx.hash="<hex_hash>"}` |
|
||||
| Specific transaction | `{span.tx_hash="<hex_hash>"}` |
|
||||
| Local transactions only | `{span.local=true}` |
|
||||
| Consensus rounds | `{resource.service.name="xrpld" && name="consensus.accept"}` |
|
||||
| Rounds by mode | `{span.xrpl.consensus.mode="proposing"}` |
|
||||
| Specific ledger | `{span.xrpl.ledger.seq=12345}` |
|
||||
| Rounds by mode | `{span.consensus_mode="proposing"}` |
|
||||
| Specific ledger | `{span.ledger_seq=12345}` |
|
||||
| Peer proposals (trusted) | `{span.proposal_trusted=true}` |
|
||||
|
||||
### Trace Structure
|
||||
|
||||
@@ -107,13 +107,13 @@ Tempo/Prometheus.
|
||||
- `consensus.validation.send` (Adaptor::validate)
|
||||
- Verify attributes:
|
||||
- `xrpl.consensus.mode` on `consensus.ledger_close`
|
||||
- `xrpl.consensus.proposers` on `consensus.accept`
|
||||
- `proposers` on `consensus.accept`
|
||||
- `xrpl.consensus.ledger.seq` on `consensus.validation.send`
|
||||
|
||||
**Verification**:
|
||||
|
||||
- [ ] Tempo shows `consensus.ledger_close` traces with `xrpl.consensus.mode`
|
||||
- [ ] Tempo shows `consensus.accept` traces with `xrpl.consensus.proposers`
|
||||
- [ ] Tempo shows `consensus.accept` traces with `proposers`
|
||||
- [ ] Tempo shows `consensus.proposal.send` traces
|
||||
- [ ] Tempo shows `consensus.validation.send` traces
|
||||
|
||||
@@ -147,7 +147,7 @@ Tempo/Prometheus.
|
||||
- Prerequisites section
|
||||
- Single-node standalone test (quick verification)
|
||||
- 6-node consensus test (full verification)
|
||||
- Expected span catalog (all 12 span names with attributes)
|
||||
- Expected span catalog (all 11 span names with attributes)
|
||||
- Verification queries (Tempo API, Prometheus API)
|
||||
- Troubleshooting guide
|
||||
|
||||
@@ -156,7 +156,7 @@ Tempo/Prometheus.
|
||||
**Verification**:
|
||||
|
||||
- [ ] Document covers both single-node and multi-node testing
|
||||
- [ ] All 12 span names documented with source file and attributes
|
||||
- [ ] All 11 span names documented with source file and attributes
|
||||
- [ ] Troubleshooting section covers common failure modes
|
||||
|
||||
---
|
||||
|
||||
@@ -26,7 +26,10 @@ services:
|
||||
ports:
|
||||
- "4317:4317" # OTLP gRPC
|
||||
- "4318:4318" # OTLP HTTP
|
||||
- "8125:8125/udp" # StatsD UDP (beast::insight metrics)
|
||||
# StatsD UDP (beast::insight metrics). Bound to loopback only: the
|
||||
# receiver has no auth, and xrpld runs on the host, so it reaches the
|
||||
# collector via 127.0.0.1. Do not expose on 0.0.0.0 in shared setups.
|
||||
- "127.0.0.1:8125:8125/udp"
|
||||
- "8889:8889" # Prometheus metrics (spanmetrics + statsd)
|
||||
- "13133:13133" # Health check
|
||||
volumes:
|
||||
@@ -53,7 +56,8 @@ services:
|
||||
- xrpld-telemetry
|
||||
|
||||
prometheus:
|
||||
image: prom/prometheus:latest
|
||||
# Pinned to the v2.53 LTS line for reproducible, config-stable runs.
|
||||
image: prom/prometheus:v2.53.3
|
||||
ports:
|
||||
- "9090:9090"
|
||||
volumes:
|
||||
|
||||
@@ -463,7 +463,7 @@
|
||||
"datasource": {
|
||||
"type": "prometheus"
|
||||
},
|
||||
"expr": "topk{exported_instance=~\"$node\"}(20, {__name__=~\"rippled_.*_Bytes_In\", __name__!~\"rippled_total_.*\"})",
|
||||
"expr": "topk(20, {__name__=~\"rippled_.*_Bytes_In\", __name__!~\"rippled_total_.*\", exported_instance=~\"$node\"})",
|
||||
"legendFormat": "{{__name__}}"
|
||||
}
|
||||
],
|
||||
|
||||
@@ -115,14 +115,14 @@
|
||||
"datasource": {
|
||||
"type": "prometheus"
|
||||
},
|
||||
"expr": "rate{exported_instance=~\"$node\"}(rippled_total_Bytes_In[5m])",
|
||||
"expr": "rate(rippled_total_Bytes_In{exported_instance=~\"$node\"}[5m])",
|
||||
"legendFormat": "Bytes In"
|
||||
},
|
||||
{
|
||||
"datasource": {
|
||||
"type": "prometheus"
|
||||
},
|
||||
"expr": "rate{exported_instance=~\"$node\"}(rippled_total_Bytes_Out[5m])",
|
||||
"expr": "rate(rippled_total_Bytes_Out{exported_instance=~\"$node\"}[5m])",
|
||||
"legendFormat": "Bytes Out"
|
||||
}
|
||||
],
|
||||
@@ -161,14 +161,14 @@
|
||||
"datasource": {
|
||||
"type": "prometheus"
|
||||
},
|
||||
"expr": "rippled_total_Messages_In{exported_instance=~\"$node\"}",
|
||||
"expr": "rate(rippled_total_Messages_In{exported_instance=~\"$node\"}[5m])",
|
||||
"legendFormat": "Messages In"
|
||||
},
|
||||
{
|
||||
"datasource": {
|
||||
"type": "prometheus"
|
||||
},
|
||||
"expr": "rippled_total_Messages_Out{exported_instance=~\"$node\"}",
|
||||
"expr": "rate(rippled_total_Messages_Out{exported_instance=~\"$node\"}[5m])",
|
||||
"legendFormat": "Messages Out"
|
||||
}
|
||||
],
|
||||
@@ -380,7 +380,7 @@
|
||||
"datasource": {
|
||||
"type": "prometheus"
|
||||
},
|
||||
"expr": "topk{exported_instance=~\"$node\"}(10, {__name__=~\"rippled_.*_Bytes_In\", __name__!~\"rippled_total_.*\"})",
|
||||
"expr": "topk(10, {__name__=~\"rippled_.*_Bytes_In\", __name__!~\"rippled_total_.*\", exported_instance=~\"$node\"})",
|
||||
"legendFormat": "{{__name__}}"
|
||||
}
|
||||
],
|
||||
@@ -677,42 +677,42 @@
|
||||
"datasource": {
|
||||
"type": "prometheus"
|
||||
},
|
||||
"expr": "rate{exported_instance=~\"$node\"}(rippled_transactions_duplicate_Bytes_In[5m])",
|
||||
"expr": "rate(rippled_transactions_duplicate_Bytes_In{exported_instance=~\"$node\"}[5m])",
|
||||
"legendFormat": "TX Duplicate In"
|
||||
},
|
||||
{
|
||||
"datasource": {
|
||||
"type": "prometheus"
|
||||
},
|
||||
"expr": "rate{exported_instance=~\"$node\"}(rippled_transactions_duplicate_Bytes_Out[5m])",
|
||||
"expr": "rate(rippled_transactions_duplicate_Bytes_Out{exported_instance=~\"$node\"}[5m])",
|
||||
"legendFormat": "TX Duplicate Out"
|
||||
},
|
||||
{
|
||||
"datasource": {
|
||||
"type": "prometheus"
|
||||
},
|
||||
"expr": "rate{exported_instance=~\"$node\"}(rippled_proposals_duplicate_Bytes_In[5m])",
|
||||
"expr": "rate(rippled_proposals_duplicate_Bytes_In{exported_instance=~\"$node\"}[5m])",
|
||||
"legendFormat": "Proposals Duplicate In"
|
||||
},
|
||||
{
|
||||
"datasource": {
|
||||
"type": "prometheus"
|
||||
},
|
||||
"expr": "rate{exported_instance=~\"$node\"}(rippled_proposals_duplicate_Bytes_Out[5m])",
|
||||
"expr": "rate(rippled_proposals_duplicate_Bytes_Out{exported_instance=~\"$node\"}[5m])",
|
||||
"legendFormat": "Proposals Duplicate Out"
|
||||
},
|
||||
{
|
||||
"datasource": {
|
||||
"type": "prometheus"
|
||||
},
|
||||
"expr": "rate{exported_instance=~\"$node\"}(rippled_validations_duplicate_Bytes_In[5m])",
|
||||
"expr": "rate(rippled_validations_duplicate_Bytes_In{exported_instance=~\"$node\"}[5m])",
|
||||
"legendFormat": "Validations Duplicate In"
|
||||
},
|
||||
{
|
||||
"datasource": {
|
||||
"type": "prometheus"
|
||||
},
|
||||
"expr": "rate{exported_instance=~\"$node\"}(rippled_validations_duplicate_Bytes_Out[5m])",
|
||||
"expr": "rate(rippled_validations_duplicate_Bytes_Out{exported_instance=~\"$node\"}[5m])",
|
||||
"legendFormat": "Validations Duplicate Out"
|
||||
}
|
||||
],
|
||||
@@ -751,7 +751,7 @@
|
||||
"datasource": {
|
||||
"type": "prometheus"
|
||||
},
|
||||
"expr": "topk{exported_instance=~\"$node\"}(15, rate({__name__=~\"rippled_.*_Bytes_In\", __name__!~\"rippled_total_.*\"}[5m]))",
|
||||
"expr": "topk(15, rate({__name__=~\"rippled_.*_Bytes_In\", __name__!~\"rippled_total_.*\", exported_instance=~\"$node\"}[5m]))",
|
||||
"legendFormat": "{{__name__}}"
|
||||
}
|
||||
],
|
||||
|
||||
@@ -345,7 +345,7 @@
|
||||
"datasource": {
|
||||
"type": "prometheus"
|
||||
},
|
||||
"expr": "rate{exported_instance=~\"$node\"}(rippled_ledger_fetches_total[5m])",
|
||||
"expr": "rate(rippled_ledger_fetches_total{exported_instance=~\"$node\"}[5m])",
|
||||
"legendFormat": "Fetches / Sec"
|
||||
}
|
||||
],
|
||||
@@ -377,7 +377,7 @@
|
||||
"datasource": {
|
||||
"type": "prometheus"
|
||||
},
|
||||
"expr": "rate{exported_instance=~\"$node\"}(rippled_ledger_history_mismatch_total[5m])",
|
||||
"expr": "rate(rippled_ledger_history_mismatch_total{exported_instance=~\"$node\"}[5m])",
|
||||
"legendFormat": "Mismatches / Sec"
|
||||
}
|
||||
],
|
||||
@@ -728,7 +728,7 @@
|
||||
"datasource": {
|
||||
"type": "prometheus"
|
||||
},
|
||||
"expr": "rippled_LedgerMaster_Published_Ledger_Age{exported_instance=~\"$node\"} - rippled_LedgerMaster_Validated_Ledger_Age",
|
||||
"expr": "rippled_LedgerMaster_Published_Ledger_Age{exported_instance=~\"$node\"} - rippled_LedgerMaster_Validated_Ledger_Age{exported_instance=~\"$node\"}",
|
||||
"legendFormat": "Publish Gap"
|
||||
}
|
||||
],
|
||||
@@ -776,14 +776,14 @@
|
||||
"datasource": {
|
||||
"type": "prometheus"
|
||||
},
|
||||
"expr": "rate{exported_instance=~\"$node\"}(rippled_State_Accounting_Full_duration[5m]) / 1000000",
|
||||
"expr": "rate(rippled_State_Accounting_Full_duration{exported_instance=~\"$node\"}[5m]) / 1000000",
|
||||
"legendFormat": "Full Mode Rate"
|
||||
},
|
||||
{
|
||||
"datasource": {
|
||||
"type": "prometheus"
|
||||
},
|
||||
"expr": "rate{exported_instance=~\"$node\"}(rippled_State_Accounting_Tracking_duration[5m]) / 1000000",
|
||||
"expr": "rate(rippled_State_Accounting_Tracking_duration{exported_instance=~\"$node\"}[5m]) / 1000000",
|
||||
"legendFormat": "Tracking Mode Rate"
|
||||
}
|
||||
],
|
||||
@@ -822,7 +822,7 @@
|
||||
"datasource": {
|
||||
"type": "prometheus"
|
||||
},
|
||||
"expr": "{__name__{exported_instance=~\"$node\"}=~\"rippled_(makeFetchPack|publishAcqLedger|untrustedValidation|manifest|localTransaction|ledgerReplayRequest|ledgerRequest|untrustedProposal|ledgerReplayTask|ledgerData|clientCommand|clientSubscribe|clientFeeChange|clientConsensus|clientAccountHistory|clientRPC|clientWebsocket|RPC|updatePaths|transaction|batch|advanceLedger|publishNewLedger|fetchTxnData|writeAhead|trustedValidation|writeObjects|acceptLedger|trustedProposal|sweep|clusterReport|heartbeat|administration|handleHaveTransactions|doTransactions)\", quantile=\"$quantile\"}",
|
||||
"expr": "{__name__=~\"rippled_(makeFetchPack|publishAcqLedger|untrustedValidation|manifest|localTransaction|ledgerReplayRequest|ledgerRequest|untrustedProposal|ledgerReplayTask|ledgerData|clientCommand|clientSubscribe|clientFeeChange|clientConsensus|clientAccountHistory|clientRPC|clientWebsocket|RPC|updatePaths|transaction|batch|advanceLedger|publishNewLedger|fetchTxnData|writeAhead|trustedValidation|writeObjects|acceptLedger|trustedProposal|sweep|clusterReport|heartbeat|administration|handleHaveTransactions|doTransactions)\", exported_instance=~\"$node\", quantile=\"$quantile\"}",
|
||||
"legendFormat": "{{__name__}} [{{quantile}}]"
|
||||
}
|
||||
],
|
||||
@@ -861,7 +861,7 @@
|
||||
"datasource": {
|
||||
"type": "prometheus"
|
||||
},
|
||||
"expr": "{__name__{exported_instance=~\"$node\"}=~\"rippled_(makeFetchPack_q|publishAcqLedger_q|untrustedValidation_q|manifest_q|localTransaction_q|ledgerReplayRequest_q|ledgerRequest_q|untrustedProposal_q|ledgerReplayTask_q|ledgerData_q|clientCommand_q|clientSubscribe_q|clientFeeChange_q|clientConsensus_q|clientAccountHistory_q|clientRPC_q|clientWebsocket_q|RPC_q|updatePaths_q|transaction_q|batch_q|advanceLedger_q|publishNewLedger_q|fetchTxnData_q|writeAhead_q|trustedValidation_q|writeObjects_q|acceptLedger_q|trustedProposal_q|sweep_q|clusterReport_q|heartbeat_q|administration_q|handleHaveTransactions_q|doTransactions_q)\", quantile=\"$quantile\"}",
|
||||
"expr": "{__name__=~\"rippled_(makeFetchPack_q|publishAcqLedger_q|untrustedValidation_q|manifest_q|localTransaction_q|ledgerReplayRequest_q|ledgerRequest_q|untrustedProposal_q|ledgerReplayTask_q|ledgerData_q|clientCommand_q|clientSubscribe_q|clientFeeChange_q|clientConsensus_q|clientAccountHistory_q|clientRPC_q|clientWebsocket_q|RPC_q|updatePaths_q|transaction_q|batch_q|advanceLedger_q|publishNewLedger_q|fetchTxnData_q|writeAhead_q|trustedValidation_q|writeObjects_q|acceptLedger_q|trustedProposal_q|sweep_q|clusterReport_q|heartbeat_q|administration_q|handleHaveTransactions_q|doTransactions_q)\", exported_instance=~\"$node\", quantile=\"$quantile\"}",
|
||||
"legendFormat": "{{__name__}} [{{quantile}}]"
|
||||
}
|
||||
],
|
||||
|
||||
@@ -30,7 +30,7 @@
|
||||
"datasource": {
|
||||
"type": "prometheus"
|
||||
},
|
||||
"expr": "rate{exported_instance=~\"$node\"}(rippled_rpc_requests_total[5m])",
|
||||
"expr": "rate(rippled_rpc_requests_total{exported_instance=~\"$node\"}[5m])",
|
||||
"legendFormat": "Requests / Sec"
|
||||
}
|
||||
],
|
||||
@@ -306,7 +306,7 @@
|
||||
"datasource": {
|
||||
"type": "prometheus"
|
||||
},
|
||||
"expr": "rate{exported_instance=~\"$node\"}(rippled_warn_total[5m])",
|
||||
"expr": "rate(rippled_warn_total{exported_instance=~\"$node\"}[5m])",
|
||||
"legendFormat": "Warnings / Sec"
|
||||
}
|
||||
],
|
||||
@@ -354,7 +354,7 @@
|
||||
"datasource": {
|
||||
"type": "prometheus"
|
||||
},
|
||||
"expr": "rate{exported_instance=~\"$node\"}(rippled_drop_total[5m])",
|
||||
"expr": "rate(rippled_drop_total{exported_instance=~\"$node\"}[5m])",
|
||||
"legendFormat": "Drops / Sec"
|
||||
}
|
||||
],
|
||||
|
||||
@@ -7,7 +7,7 @@
|
||||
# ./xrpld --conf docker/telemetry/xrpld-telemetry.cfg -a --start
|
||||
# 3. Send RPC commands to exercise tracing:
|
||||
# curl -s http://localhost:5005 -d '{"method":"server_info"}'
|
||||
# 4. View traces in Jaeger UI: http://localhost:16686
|
||||
# 4. View traces in Grafana Explore: http://localhost:3000/explore
|
||||
|
||||
[server]
|
||||
port_rpc_admin_local
|
||||
@@ -57,3 +57,11 @@ trace_transactions=1
|
||||
trace_consensus=1
|
||||
trace_peer=1
|
||||
trace_ledger=1
|
||||
|
||||
# --- StatsD metrics (beast::insight) ---
|
||||
# Sends UDP metrics to the collector's StatsD receiver so the StatsD
|
||||
# dashboards have data. The collector binds this port on the host loopback.
|
||||
[insight]
|
||||
server=statsd
|
||||
address=127.0.0.1:8125
|
||||
prefix=rippled
|
||||
|
||||
Reference in New Issue
Block a user