mirror of
https://github.com/XRPLF/rippled.git
synced 2026-07-11 09:10:21 +00:00
Compare commits
2 Commits
pratik/ote
...
perf/alloc
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
997d066b65 | ||
|
|
162093680b |
@@ -227,7 +227,6 @@ words:
|
||||
- onlatest
|
||||
- ostr
|
||||
- otelc
|
||||
- otelcol
|
||||
- pargs
|
||||
- partitioner
|
||||
- paychan
|
||||
|
||||
@@ -545,24 +545,16 @@ prefix=xrpld
|
||||
|
||||
### 2.3 Histograms (Event timers)
|
||||
|
||||
| Prometheus Metric | Source File | Unit | Description |
|
||||
| ----------------- | ----------------- | ---- | ------------------------------ |
|
||||
| `rpc_time` | ServerHandler.cpp | ms | RPC response time distribution |
|
||||
| `rpc_size` | ServerHandler.cpp | ms\* | RPC response size (see note) |
|
||||
| `ios_latency` | Application.cpp | ms | I/O service loop latency |
|
||||
| `pathfind_fast` | PathRequests.h | ms | Fast pathfinding duration |
|
||||
| `pathfind_full` | PathRequests.h | ms | Full pathfinding duration |
|
||||
| Prometheus Metric | Source File | Unit | Description |
|
||||
| ----------------- | ----------------- | ----- | ------------------------------ |
|
||||
| `rpc_time` | ServerHandler.cpp | ms | RPC response time distribution |
|
||||
| `rpc_size` | ServerHandler.cpp | bytes | RPC response size distribution |
|
||||
| `ios_latency` | Application.cpp | ms | I/O service loop latency |
|
||||
| `pathfind_fast` | PathRequests.h | ms | Fast pathfinding duration |
|
||||
| `pathfind_full` | PathRequests.h | ms | Full pathfinding duration |
|
||||
|
||||
Quantiles collected: 0th, 50th, 90th, 95th, 99th, 100th percentile.
|
||||
|
||||
\* **`rpc_size` instrument mismatch (known issue):** response size in bytes is
|
||||
recorded through the millisecond-scaled event histogram (`makeEvent`), so it is
|
||||
exported as `rpc_size_milliseconds_bucket` with time-scaled boundaries that top
|
||||
out at 5000. Byte values above ~5 KB saturate in the last bucket, so the
|
||||
percentiles are not true byte sizes. The _RPC & Pathfinding_ panel is flagged
|
||||
accordingly. A dedicated byte-unit histogram is needed to fix this; tracked
|
||||
separately.
|
||||
|
||||
**Grafana dashboards**: _Node Health_ (`ios_latency`), _RPC & Pathfinding_ (`rpc_time`, `rpc_size`, `pathfind_*`)
|
||||
|
||||
### 2.4 Overlay Traffic Metrics
|
||||
@@ -1181,21 +1173,21 @@ State value encoding: 0=disconnected, 1=connected, 2=syncing, 3=tracking, 4=full
|
||||
|
||||
#### Synchronous Counters (Phase 7+)
|
||||
|
||||
| Prometheus Metric | Type | Description | Increment Site |
|
||||
| --------------------------- | ------- | ---------------------------- | ---------------- |
|
||||
| `ledgers_closed_total` | Counter | Ledgers closed by consensus | RCLConsensus.cpp |
|
||||
| `validations_sent_total` | Counter | Validations sent | RCLConsensus.cpp |
|
||||
| `validations_checked_total` | Counter | Network validations observed | LedgerMaster.cpp |
|
||||
| `state_changes_total` | Counter | Operating mode transitions | NetworkOPs.cpp |
|
||||
| Prometheus Metric | Type | Description | Increment Site |
|
||||
| --------------------------- | ------- | ------------------------------- | ---------------- |
|
||||
| `ledgers_closed_total` | Counter | Ledgers closed by consensus | RCLConsensus.cpp |
|
||||
| `validations_sent_total` | Counter | Validations sent | RCLConsensus.cpp |
|
||||
| `validations_checked_total` | Counter | Network validations observed | LedgerMaster.cpp |
|
||||
| `state_changes_total` | Counter | Operating mode transitions | NetworkOPs.cpp |
|
||||
| `jq_trans_overflow_total` | Counter | Job queue transaction overflows | JobQueue.cpp |
|
||||
|
||||
Lifetime tallies exported as monotonic **ObservableCounters** (not synchronous
|
||||
counters), observed from an existing cumulative source each collection cycle:
|
||||
Lifetime validation agreement/miss tallies are exported as monotonic **ObservableCounters**
|
||||
(not synchronous counters) observed from `ValidationTracker`'s gross lifetime totals:
|
||||
|
||||
| Prometheus Metric | Type | Description | Source |
|
||||
| ----------------------------- | ----------------- | ------------------------------------------ | ---------------------------------------------------- |
|
||||
| `validation_agreements_total` | ObservableCounter | Lifetime validations that initially agreed | ValidationTracker.cpp |
|
||||
| `validation_missed_total` | ObservableCounter | Lifetime validations that initially missed | ValidationTracker.cpp |
|
||||
| `jq_trans_overflow_total` | ObservableCounter | Job queue transaction overflows | Overlay::getJqTransOverflow (PeerImp.cpp increments) |
|
||||
| Prometheus Metric | Type | Description | Source |
|
||||
| ----------------------------- | ----------------- | ------------------------------------------ | --------------------- |
|
||||
| `validation_agreements_total` | ObservableCounter | Lifetime validations that initially agreed | ValidationTracker.cpp |
|
||||
| `validation_missed_total` | ObservableCounter | Lifetime validations that initially missed | ValidationTracker.cpp |
|
||||
|
||||
> **Counting semantics (initial-classification only):** each reconciled ledger increments exactly
|
||||
> one of these two counters, at first classification. A later late-repair (miss → agreement) does
|
||||
|
||||
@@ -242,7 +242,7 @@ elseif(use_gold AND is_gcc)
|
||||
disabling would be to figure out all the settings
|
||||
required to make gold play nicely with jemalloc.
|
||||
#]=========================================================]
|
||||
if(("${LD_VERSION}" MATCHES "GNU gold") AND (NOT jemalloc))
|
||||
if(("${LD_VERSION}" MATCHES "GNU gold") AND (NOT jemalloc) AND (NOT tcmalloc))
|
||||
target_link_libraries(
|
||||
common
|
||||
INTERFACE
|
||||
|
||||
@@ -54,6 +54,11 @@ if(jemalloc)
|
||||
target_link_libraries(opts INTERFACE jemalloc::jemalloc)
|
||||
endif()
|
||||
|
||||
if(tcmalloc)
|
||||
find_package(gperftools REQUIRED)
|
||||
target_link_libraries(opts INTERFACE gperftools::gperftools)
|
||||
endif()
|
||||
|
||||
#[===================================================================[
|
||||
xrpld transitive library deps via an interface library
|
||||
#]===================================================================]
|
||||
|
||||
@@ -125,6 +125,10 @@ else()
|
||||
endif()
|
||||
|
||||
option(jemalloc "Enables jemalloc for heap profiling" OFF)
|
||||
option(tcmalloc "Link xrpld against tcmalloc (gperftools) as the global allocator" OFF)
|
||||
if(jemalloc AND tcmalloc)
|
||||
message(FATAL_ERROR "jemalloc and tcmalloc are mutually exclusive")
|
||||
endif()
|
||||
option(werr "treat warnings as errors" OFF)
|
||||
option(
|
||||
local_protobuf
|
||||
|
||||
10
conanfile.py
10
conanfile.py
@@ -3,6 +3,7 @@ import re
|
||||
from conan.tools.cmake import CMake, CMakeToolchain, cmake_layout
|
||||
|
||||
from conan import ConanFile
|
||||
from conan.errors import ConanInvalidConfiguration
|
||||
|
||||
|
||||
class Xrpl(ConanFile):
|
||||
@@ -21,6 +22,7 @@ class Xrpl(ConanFile):
|
||||
"rocksdb": [True, False],
|
||||
"shared": [True, False],
|
||||
"static": [True, False],
|
||||
"tcmalloc": [True, False],
|
||||
"telemetry": [True, False],
|
||||
"tests": [True, False],
|
||||
"unity": [True, False],
|
||||
@@ -53,6 +55,7 @@ class Xrpl(ConanFile):
|
||||
"rocksdb": True,
|
||||
"shared": False,
|
||||
"static": True,
|
||||
"tcmalloc": False,
|
||||
"telemetry": True,
|
||||
"tests": False,
|
||||
"unity": False,
|
||||
@@ -125,6 +128,10 @@ class Xrpl(ConanFile):
|
||||
self.version = match.group(1)
|
||||
|
||||
def configure(self):
|
||||
if self.options.tcmalloc and self.options.jemalloc:
|
||||
raise ConanInvalidConfiguration(
|
||||
"jemalloc and tcmalloc cannot both be enabled"
|
||||
)
|
||||
if self.settings.compiler == "apple-clang":
|
||||
self.options["boost"].visibility = "global"
|
||||
if self.settings.compiler in ["clang", "gcc"]:
|
||||
@@ -135,6 +142,8 @@ class Xrpl(ConanFile):
|
||||
self.requires("date/3.0.4", transitive_headers=True)
|
||||
if self.options.jemalloc:
|
||||
self.requires("jemalloc/5.3.1")
|
||||
if self.options.tcmalloc:
|
||||
self.requires("gperftools/2.16")
|
||||
self.requires("lz4/1.10.0", force=True)
|
||||
self.requires("mpt-crypto/0.4.0-rc2", transitive_headers=True)
|
||||
self.requires("protobuf/6.33.5", force=True)
|
||||
@@ -171,6 +180,7 @@ class Xrpl(ConanFile):
|
||||
tc.variables["assert"] = self.options.assertions
|
||||
tc.variables["coverage"] = self.options.coverage
|
||||
tc.variables["jemalloc"] = self.options.jemalloc
|
||||
tc.variables["tcmalloc"] = self.options.tcmalloc
|
||||
tc.variables["rocksdb"] = self.options.rocksdb
|
||||
tc.variables["BUILD_SHARED_LIBS"] = self.options.shared
|
||||
tc.variables["static"] = self.options.static
|
||||
|
||||
@@ -1,27 +0,0 @@
|
||||
# Grafana Alloy -> Grafana Cloud credentials for docker/telemetry/alloy/config.alloy.
|
||||
# Copy to `.env.grafanacloud-alloy`, fill in, and source before starting Alloy.
|
||||
# `.env.grafanacloud-alloy` is gitignored; never commit real tokens or stack ids.
|
||||
#
|
||||
# Prometheus values: Grafana Cloud -> Connections -> "Prometheus".
|
||||
# OTLP values: Grafana Cloud -> Connections -> "OpenTelemetry (OTLP)".
|
||||
|
||||
# --- Host / systemd metrics (prometheus.remote_write) ---
|
||||
# remote_write push URL, e.g. https://prometheus-prod-XX-<region>.grafana.net/api/prom/push
|
||||
GRAFANACLOUD_PROM_URL=
|
||||
# Numeric instance/stack id (Basic-auth username).
|
||||
GRAFANACLOUD_PROM_USER=
|
||||
# Access-policy token with metrics:write (Basic-auth password).
|
||||
GRAFANACLOUD_PROM_KEY=
|
||||
|
||||
# --- xrpld OTLP (otelcol.exporter.otlphttp) ---
|
||||
# OTLP/HTTP gateway URL including the /otlp path,
|
||||
# e.g. https://otlp-gateway-prod-<region>.grafana.net/otlp
|
||||
GRAFANACLOUD_OTLP_URL=
|
||||
# Numeric instance/stack id (Basic-auth username).
|
||||
GRAFANACLOUD_OTLP_USER=
|
||||
# Access-policy token with metrics:write + traces:write (Basic-auth password).
|
||||
GRAFANACLOUD_OTLP_KEY=
|
||||
|
||||
# --- Per-node label ---
|
||||
# host label applied to this node's scraped metrics (e.g. the node's hostname).
|
||||
XRPLD_HOST_LABEL=
|
||||
1
docker/telemetry/.gitignore
vendored
1
docker/telemetry/.gitignore
vendored
@@ -7,4 +7,3 @@ data/
|
||||
# Keep examples
|
||||
!.env.alerting.example
|
||||
!.env.grafanacloud.example
|
||||
!.env.grafanacloud-alloy.example
|
||||
|
||||
@@ -1,268 +0,0 @@
|
||||
// Grafana Alloy collector config for an xrpld node.
|
||||
//
|
||||
// WHY THIS FILE EXISTS
|
||||
// --------------------
|
||||
// Some deployments feed telemetry through Grafana Alloy instead of the
|
||||
// reference OpenTelemetry Collector (otel-collector-config.grafanacloud.yaml).
|
||||
// xrpld sends OTLP (traces + native beast::insight metrics) to Alloy, which
|
||||
// forwards to the Grafana Cloud OTLP gateway. That path carries traces and
|
||||
// native metrics, but on its own produces NO span-derived RED metrics
|
||||
// (span_calls_total / span_duration_milliseconds_*): those are NOT emitted by
|
||||
// xrpld -- they are derived by a spanmetrics connector from the spans. Without
|
||||
// the connector a node is missing from every span-based Grafana dashboard.
|
||||
//
|
||||
// This config therefore mirrors the two stages the reference collector has:
|
||||
// 1. resource tagging (service.name, deployment.environment, xrpl.network.type)
|
||||
// 2. spanmetrics (derives the RED metrics the dashboards query)
|
||||
// See docker/telemetry/otel-collector-config.grafanacloud.yaml for the
|
||||
// authoritative collector equivalent; keep the dimension list in sync with it.
|
||||
//
|
||||
// PIPELINE
|
||||
//
|
||||
// HOST / SYSTEMD METRICS:
|
||||
// unix exporter + custom scrape --> relabel(host) --> prometheus.remote_write
|
||||
//
|
||||
// xrpld OTLP:
|
||||
// receiver.otlp ─▶ processor.transform.tier ─┬─▶ connector.spanmetrics ─┐
|
||||
// │ │
|
||||
// └─▶ processor.batch ◀───────┘
|
||||
// │ traces + metrics
|
||||
// ▼
|
||||
// exporter.otlphttp (GC OTLP gateway)
|
||||
//
|
||||
// The Grafana Cloud OTLP gateway converts OTLP resource attributes to
|
||||
// Prometheus labels server-side, so no otelcol.exporter.prometheus is needed.
|
||||
//
|
||||
// CONFIGURATION -- NO SECRETS OR TENANT IDS ARE HARD-CODED IN THIS FILE.
|
||||
// All credentials, endpoints, usernames and the per-node host label are read
|
||||
// from the environment (same pattern as .env.grafanacloud.example). Copy
|
||||
// .env.grafanacloud-alloy.example to .env.grafanacloud-alloy, fill it in, and
|
||||
// source it before starting Alloy. Required variables:
|
||||
// GRAFANACLOUD_PROM_URL Prometheus remote_write push URL
|
||||
// GRAFANACLOUD_PROM_USER remote_write basic-auth username (numeric stack id)
|
||||
// GRAFANACLOUD_PROM_KEY remote_write basic-auth password (access token)
|
||||
// GRAFANACLOUD_OTLP_URL OTLP/HTTP gateway URL, including the /otlp path
|
||||
// GRAFANACLOUD_OTLP_USER OTLP basic-auth username (numeric stack id)
|
||||
// GRAFANACLOUD_OTLP_KEY OTLP basic-auth password (access token)
|
||||
// XRPLD_HOST_LABEL host label for this node's scraped metrics
|
||||
//
|
||||
// PER-DEPLOYMENT EDITS: the deployment.environment and xrpl.network.type tier
|
||||
// values in otelcol.processor.transform are literals (OTTL cannot read env
|
||||
// vars) -- edit them to match this node's tier and network.
|
||||
|
||||
logging {
|
||||
level = "info"
|
||||
}
|
||||
|
||||
// ===========================================================================
|
||||
// HOST / SYSTEMD METRICS (Prometheus remote_write path)
|
||||
// ===========================================================================
|
||||
|
||||
prometheus.remote_write "grafanacloud" {
|
||||
endpoint {
|
||||
url = sys.env("GRAFANACLOUD_PROM_URL")
|
||||
|
||||
basic_auth {
|
||||
username = sys.env("GRAFANACLOUD_PROM_USER")
|
||||
password = sys.env("GRAFANACLOUD_PROM_KEY")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
prometheus.exporter.unix "default" {
|
||||
enable_collectors = ["systemd", "textfile"]
|
||||
systemd {
|
||||
unit_include = "(rippled|xrpld).*"
|
||||
}
|
||||
textfile {
|
||||
directory = "/usr/local/bin/"
|
||||
}
|
||||
}
|
||||
|
||||
prometheus.scrape "system_metrics" {
|
||||
targets = prometheus.exporter.unix.default.targets
|
||||
scrape_interval = "15s"
|
||||
forward_to = [prometheus.relabel.hostname.receiver]
|
||||
}
|
||||
|
||||
prometheus.scrape "custom_rippled_metrics" {
|
||||
targets = [{
|
||||
__address__ = "localhost:9256",
|
||||
}]
|
||||
scrape_interval = "15s"
|
||||
forward_to = [prometheus.relabel.hostname.receiver]
|
||||
}
|
||||
|
||||
prometheus.relabel "hostname" {
|
||||
forward_to = [prometheus.remote_write.grafanacloud.receiver]
|
||||
|
||||
rule {
|
||||
action = "replace"
|
||||
source_labels = ["instance"]
|
||||
replacement = sys.env("XRPLD_HOST_LABEL")
|
||||
target_label = "host"
|
||||
}
|
||||
}
|
||||
|
||||
// ===========================================================================
|
||||
// xrpld OTLP -> Grafana Cloud (traces + native metrics + derived spanmetrics)
|
||||
// ===========================================================================
|
||||
|
||||
// Receiver: xrpld sends OTLP/HTTP on 4318 and gRPC on 4317 (localhost only).
|
||||
otelcol.receiver.otlp "xrpld" {
|
||||
grpc {
|
||||
endpoint = "127.0.0.1:4317"
|
||||
}
|
||||
http {
|
||||
endpoint = "127.0.0.1:4318"
|
||||
}
|
||||
|
||||
output {
|
||||
// Both signals pass through resource tagging first so they leave with
|
||||
// identical resource identity.
|
||||
traces = [otelcol.processor.transform.tier.input]
|
||||
metrics = [otelcol.processor.transform.tier.input]
|
||||
}
|
||||
}
|
||||
|
||||
// Resource tagging (reference: resource/tier + resource/stripsdk).
|
||||
// * service.name -> "xrpld". Also corrects the service_name="true"
|
||||
// bool-binding bug on stale binaries: even an un-rebuilt node exports a
|
||||
// correct service.name once it passes through here.
|
||||
// * deployment.environment -> set from XRPLD_DEPLOYMENT_ENV (the collector
|
||||
// IS the environment, so it is authoritative -> upsert).
|
||||
// * xrpl.network.type -> set only when absent (don't overwrite the node's
|
||||
// own value). OTTL `where ... == nil` gives insert (not upsert) semantics.
|
||||
// * telemetry.sdk.* -> deleted (SDK noise).
|
||||
otelcol.processor.transform "tier" {
|
||||
error_mode = "ignore"
|
||||
|
||||
// NOTE: statements are OTTL (raw strings) -- River sys.env() does NOT expand
|
||||
// inside them, and OTTL has no env() converter. The tier values below are
|
||||
// plain deployment labels (not secrets), so they are literals here.
|
||||
// PER-DEPLOYMENT EDIT: set deployment.environment and xrpl.network.type to
|
||||
// match this node's tier and network.
|
||||
trace_statements {
|
||||
context = "resource"
|
||||
statements = [
|
||||
`set(attributes["service.name"], "xrpld")`,
|
||||
`set(attributes["deployment.environment"], "prod")`,
|
||||
`set(attributes["xrpl.network.type"], "mainnet") where attributes["xrpl.network.type"] == nil`,
|
||||
`delete_key(attributes, "telemetry.sdk.language")`,
|
||||
`delete_key(attributes, "telemetry.sdk.name")`,
|
||||
`delete_key(attributes, "telemetry.sdk.version")`,
|
||||
]
|
||||
}
|
||||
|
||||
metric_statements {
|
||||
context = "resource"
|
||||
statements = [
|
||||
`set(attributes["service.name"], "xrpld")`,
|
||||
`set(attributes["deployment.environment"], "prod")`,
|
||||
`set(attributes["xrpl.network.type"], "mainnet") where attributes["xrpl.network.type"] == nil`,
|
||||
`delete_key(attributes, "telemetry.sdk.language")`,
|
||||
`delete_key(attributes, "telemetry.sdk.name")`,
|
||||
`delete_key(attributes, "telemetry.sdk.version")`,
|
||||
]
|
||||
}
|
||||
|
||||
output {
|
||||
// Traces fan out: to the batch/gateway path AND into the spanmetrics
|
||||
// connector so the RED metrics are derived from the same tagged spans.
|
||||
traces = [
|
||||
otelcol.processor.batch.xrpld.input,
|
||||
otelcol.connector.spanmetrics.xrpld.input,
|
||||
]
|
||||
// Native metrics go straight to the batch/gateway path.
|
||||
metrics = [otelcol.processor.batch.xrpld.input]
|
||||
}
|
||||
}
|
||||
|
||||
// Spanmetrics connector (reference: connectors.spanmetrics, namespace "span").
|
||||
// Derives span_calls_total and span_duration_milliseconds_* that every span
|
||||
// dashboard queries. The dimension list and buckets are copied from the
|
||||
// reference otel-collector-config.grafanacloud.yaml -- keep them in sync (a
|
||||
// panel that groups by a dimension missing here renders empty).
|
||||
otelcol.connector.spanmetrics "xrpld" {
|
||||
namespace = "span"
|
||||
|
||||
histogram {
|
||||
unit = "ms"
|
||||
explicit {
|
||||
buckets = ["1ms", "5ms", "10ms", "25ms", "50ms", "100ms", "250ms", "500ms", "1s", "5s"]
|
||||
}
|
||||
}
|
||||
|
||||
// RPC / transaction dimensions.
|
||||
dimension { name = "command" }
|
||||
dimension { name = "rpc_status" }
|
||||
dimension { name = "tx_type" }
|
||||
dimension { name = "ter_result" }
|
||||
dimension { name = "stage" }
|
||||
dimension { name = "txq_status" }
|
||||
dimension { name = "load_type" }
|
||||
dimension { name = "is_batch" }
|
||||
|
||||
// Consensus dimensions.
|
||||
dimension { name = "consensus_mode" }
|
||||
dimension { name = "close_time_correct" }
|
||||
dimension { name = "consensus_state" }
|
||||
dimension { name = "mode_new" }
|
||||
dimension { name = "consensus_stalled" }
|
||||
dimension { name = "consensus_phase" }
|
||||
dimension { name = "consensus_result" }
|
||||
|
||||
// Overlay / peer dimensions.
|
||||
dimension { name = "local" }
|
||||
dimension { name = "suppressed" }
|
||||
dimension { name = "proposal_trusted" }
|
||||
dimension { name = "validation_trusted" }
|
||||
|
||||
// gRPC surface dimensions.
|
||||
dimension { name = "method" }
|
||||
dimension { name = "grpc_role" }
|
||||
dimension { name = "grpc_status" }
|
||||
|
||||
// ledger.acquire dimensions.
|
||||
dimension { name = "outcome" }
|
||||
dimension { name = "acquire_reason" }
|
||||
|
||||
output {
|
||||
// Derived span metrics rejoin the metric stream at the batch processor.
|
||||
metrics = [otelcol.processor.batch.xrpld.input]
|
||||
}
|
||||
}
|
||||
|
||||
// Batch traces + metrics (native and span-derived) before export.
|
||||
otelcol.processor.batch "xrpld" {
|
||||
timeout = "1s"
|
||||
send_batch_size = 1024
|
||||
|
||||
output {
|
||||
traces = [otelcol.exporter.otlphttp.grafanacloud.input]
|
||||
metrics = [otelcol.exporter.otlphttp.grafanacloud.input]
|
||||
}
|
||||
}
|
||||
|
||||
// Grafana Cloud OTLP gateway auth + exporter. The gateway converts OTLP
|
||||
// resource attributes to Prometheus labels server-side.
|
||||
otelcol.auth.basic "grafanacloud" {
|
||||
username = sys.env("GRAFANACLOUD_OTLP_USER")
|
||||
password = sys.env("GRAFANACLOUD_OTLP_KEY")
|
||||
}
|
||||
|
||||
otelcol.exporter.otlphttp "grafanacloud" {
|
||||
client {
|
||||
endpoint = sys.env("GRAFANACLOUD_OTLP_URL")
|
||||
auth = otelcol.auth.basic.grafanacloud.handler
|
||||
}
|
||||
retry_on_failure {
|
||||
enabled = true
|
||||
max_elapsed_time = "5m"
|
||||
}
|
||||
sending_queue {
|
||||
enabled = true
|
||||
num_consumers = 4
|
||||
queue_size = 1000
|
||||
}
|
||||
}
|
||||
@@ -30,7 +30,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", consensus_mode=~\"$consensus_mode\", span_name=\"consensus.validation.send\"}[$__rate_interval]))",
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", consensus_mode=~\"$consensus_mode\", span_name=\"consensus.validation.send\"}[5m]))",
|
||||
"legendFormat": "Validations / Sec [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -64,7 +64,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (consensus_mode, service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", consensus_mode=~\"$consensus_mode\", span_name=\"consensus.ledger_close\"}[$__rate_interval]))",
|
||||
"expr": "sum by (consensus_mode, service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", consensus_mode=~\"$consensus_mode\", span_name=\"consensus.ledger_close\"}[5m]))",
|
||||
"legendFormat": "{{consensus_mode}} [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -105,7 +105,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", consensus_mode=~\"$consensus_mode\", span_name=\"consensus.proposal.send\"}[$__rate_interval]))",
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", consensus_mode=~\"$consensus_mode\", span_name=\"consensus.proposal.send\"}[5m]))",
|
||||
"legendFormat": "Proposals / Sec [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -146,7 +146,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.accept\"}[$__rate_interval]))",
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.accept\"}[5m]))",
|
||||
"legendFormat": "Accepts / Sec [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -154,7 +154,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.ledger_close\"}[$__rate_interval]))",
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.ledger_close\"}[5m]))",
|
||||
"legendFormat": "Closes / Sec [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -195,7 +195,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.validation.send\"}[$__rate_interval]))",
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.validation.send\"}[5m]))",
|
||||
"legendFormat": "Validations / Sec [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -203,7 +203,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.ledger_close\"}[$__rate_interval]))",
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.ledger_close\"}[5m]))",
|
||||
"legendFormat": "Closes / Sec [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -853,7 +853,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (consensus_state) (increase(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.accept\", consensus_state!=\"\"}[$__rate_interval]))",
|
||||
"expr": "sum by (consensus_state) (increase(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.accept\", consensus_state!=\"\"}[5m]))",
|
||||
"legendFormat": "{{consensus_state}}"
|
||||
}
|
||||
],
|
||||
@@ -917,7 +917,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.accept\", consensus_state=\"moved_on\"}[$__rate_interval]))",
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.accept\", consensus_state=\"moved_on\"}[5m]))",
|
||||
"legendFormat": "moved_on [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -925,7 +925,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.accept\", consensus_state=\"expired\"}[$__rate_interval]))",
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.accept\", consensus_state=\"expired\"}[5m]))",
|
||||
"legendFormat": "expired [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -966,7 +966,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.check\", consensus_stalled=\"true\"}[$__rate_interval]))",
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.check\", consensus_stalled=\"true\"}[5m]))",
|
||||
"legendFormat": "Stalled [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -974,7 +974,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.check\", consensus_stalled=\"false\"}[$__rate_interval]))",
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.check\", consensus_stalled=\"false\"}[5m]))",
|
||||
"legendFormat": "Not Stalled [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -1015,7 +1015,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (mode_new, service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.mode_change\"}[$__rate_interval]))",
|
||||
"expr": "sum by (mode_new, service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"consensus.mode_change\"}[5m]))",
|
||||
"legendFormat": "{{mode_new}} [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -1056,7 +1056,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (reason, service_instance_id) (rate(ledger_history_mismatch_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]))",
|
||||
"expr": "sum by (reason, service_instance_id) (rate(ledger_history_mismatch_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m]))",
|
||||
"legendFormat": "{{reason}} [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -1171,7 +1171,7 @@
|
||||
"label": "Node",
|
||||
"description": "Filter by rippled node (service.instance.id — e.g. Node-1)",
|
||||
"type": "query",
|
||||
"query": "label_values(target_info, service_instance_id)",
|
||||
"query": "label_values(span_calls_total, service_instance_id)",
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
|
||||
@@ -320,12 +320,12 @@
|
||||
}
|
||||
},
|
||||
{
|
||||
"title": "Queue Abandonment Rate (Expired)",
|
||||
"description": "###### What this is:\n*Transactions dropped from the queue because their last-ledger deadline passed before they could be included.*\n\n###### How it's computed:\n*Per-second rate of the cumulative expired-transaction counter over the dashboard's rate interval.*\n\n###### Reading it:\n*Near zero is healthy; a rising rate means submitters under-bid the escalating fee and their transactions timed out.*\n\n###### Healthy range:\n*Near 0 expirations per second.*\n\n###### Watch for:\n*Sustained expiry rate, a demand-frustration signal often coinciding with fee spikes or spam that crowds out honest traffic.*\n\n###### Source:\n[MetricsRegistry.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/telemetry/MetricsRegistry.cpp)\n\n###### Function:\n`incrementTxqExpired (caller TxQ.cpp)`",
|
||||
"title": "Load Factor Attribution (Stacked Components)",
|
||||
"description": "###### What this is:\n*The stacked contribution of each load-factor component (fee escalation, queue, local, net, cluster) to the effective transaction cost.*\n\n###### How it's computed:\n*Instantaneous gauge readings of each component stacked to show relative share.*\n\n###### Reading it:\n*The tallest band identifies which component is driving the fee, detail the aggregate load factor hides.*\n\n###### Healthy range:\n*Thin bands near 1.0 total under normal traffic.*\n\n###### Watch for:\n*One band dominating during a fee spike, revealing the specific driver of elevated cost.*\n\n###### Source:\n[MetricsRegistry.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/telemetry/MetricsRegistry.cpp)\n\n###### Function:\n`registerLoadFactorGauge`",
|
||||
"type": "timeseries",
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
"w": 12,
|
||||
"w": 24,
|
||||
"x": 0,
|
||||
"y": 24
|
||||
},
|
||||
@@ -341,7 +341,84 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (service_instance_id) (rate(txq_expired_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]))",
|
||||
"expr": "load_factor_metrics{metric=\"load_factor_fee_escalation\",service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Fee Escalation [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "load_factor_metrics{metric=\"load_factor_fee_queue\",service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Fee Queue [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "load_factor_metrics{metric=\"load_factor_local\",service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Local [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "load_factor_metrics{metric=\"load_factor_net\",service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Net [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "load_factor_metrics{metric=\"load_factor_cluster\",service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Cluster [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "none",
|
||||
"custom": {
|
||||
"axisLabel": "Load Factor Multiplier",
|
||||
"spanNulls": true,
|
||||
"insertNulls": false,
|
||||
"showPoints": "auto",
|
||||
"pointSize": 3,
|
||||
"stacking": {
|
||||
"mode": "normal",
|
||||
"group": "A"
|
||||
},
|
||||
"fillOpacity": 30
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"title": "Queue Abandonment Rate (Expired)",
|
||||
"description": "###### What this is:\n*Transactions dropped from the queue because their last-ledger deadline passed before they could be included.*\n\n###### How it's computed:\n*Per-second rate of the cumulative expired-transaction counter over a 5-minute window.*\n\n###### Reading it:\n*Near zero is healthy; a rising rate means submitters under-bid the escalating fee and their transactions timed out.*\n\n###### Healthy range:\n*Near 0 expirations per second.*\n\n###### Watch for:\n*Sustained expiry rate, a demand-frustration signal often coinciding with fee spikes or spam that crowds out honest traffic.*\n\n###### Source:\n[MetricsRegistry.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/telemetry/MetricsRegistry.cpp)\n\n###### Function:\n`incrementTxqExpired (caller TxQ.cpp)`",
|
||||
"type": "timeseries",
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
"w": 12,
|
||||
"x": 0,
|
||||
"y": 32
|
||||
},
|
||||
"options": {
|
||||
"tooltip": {
|
||||
"mode": "multi",
|
||||
"sort": "desc"
|
||||
}
|
||||
},
|
||||
"targets": [
|
||||
{
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (service_instance_id) (rate(txq_expired_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m]))",
|
||||
"legendFormat": "Expired / Sec [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -361,13 +438,13 @@
|
||||
},
|
||||
{
|
||||
"title": "Queue Admission Rejections (Dropped)",
|
||||
"description": "###### What this is:\n*Transactions refused entry to the queue, broken down by reason such as queue_full.*\n\n###### How it's computed:\n*Per-second rate of the cumulative dropped-transaction counter over the dashboard's rate interval, split by reason.*\n\n###### Reading it:\n*Near zero is healthy; queue_full rejections mean the queue is at capacity and applying backpressure.*\n\n###### Healthy range:\n*Near 0 rejections per second.*\n\n###### Watch for:\n*A burst of queue_full drops, distinct from expiry, indicating the node is being flooded faster than it can drain.*\n\n###### Source:\n[MetricsRegistry.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/telemetry/MetricsRegistry.cpp)\n\n###### Function:\n`incrementTxqDropped (caller TxQ.cpp)`",
|
||||
"description": "###### What this is:\n*Transactions refused entry to the queue, broken down by reason such as queue_full.*\n\n###### How it's computed:\n*Per-second rate of the cumulative dropped-transaction counter over a 5-minute window, split by reason.*\n\n###### Reading it:\n*Near zero is healthy; queue_full rejections mean the queue is at capacity and applying backpressure.*\n\n###### Healthy range:\n*Near 0 rejections per second.*\n\n###### Watch for:\n*A burst of queue_full drops, distinct from expiry, indicating the node is being flooded faster than it can drain.*\n\n###### Source:\n[MetricsRegistry.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/telemetry/MetricsRegistry.cpp)\n\n###### Function:\n`incrementTxqDropped (caller TxQ.cpp)`",
|
||||
"type": "timeseries",
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
"w": 12,
|
||||
"x": 12,
|
||||
"y": 24
|
||||
"y": 32
|
||||
},
|
||||
"options": {
|
||||
"tooltip": {
|
||||
@@ -381,7 +458,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (reason, service_instance_id) (rate(txq_dropped_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]))",
|
||||
"expr": "sum by (reason, service_instance_id) (rate(txq_dropped_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m]))",
|
||||
"legendFormat": "{{reason}} [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -482,7 +559,7 @@
|
||||
"label": "Node",
|
||||
"description": "Filter by rippled node (service.instance.id)",
|
||||
"type": "query",
|
||||
"query": "label_values(target_info, service_instance_id)",
|
||||
"query": "label_values(service_instance_id)",
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
|
||||
@@ -95,7 +95,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (service_instance_id) (rate(job_queued_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", job_type=~\"$job_type\"}[$__rate_interval]))",
|
||||
"expr": "sum by (service_instance_id) (rate(job_queued_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", job_type=~\"$job_type\"}[5m]))",
|
||||
"legendFormat": "Queued/s [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -103,7 +103,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (service_instance_id) (rate(job_started_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", job_type=~\"$job_type\"}[$__rate_interval]))",
|
||||
"expr": "sum by (service_instance_id) (rate(job_started_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", job_type=~\"$job_type\"}[5m]))",
|
||||
"legendFormat": "Started/s [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -111,7 +111,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (service_instance_id) (rate(job_finished_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", job_type=~\"$job_type\"}[$__rate_interval]))",
|
||||
"expr": "sum by (service_instance_id) (rate(job_finished_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", job_type=~\"$job_type\"}[5m]))",
|
||||
"legendFormat": "Finished/s [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -163,7 +163,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "topk(10, rate(job_queued_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", job_type=~\"$job_type\"}[$__rate_interval]))",
|
||||
"expr": "topk(10, rate(job_queued_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", job_type=~\"$job_type\"}[5m]))",
|
||||
"legendFormat": "{{job_type}} [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -215,7 +215,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "topk(10, rate(job_finished_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", job_type=~\"$job_type\"}[$__rate_interval]))",
|
||||
"expr": "topk(10, rate(job_finished_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", job_type=~\"$job_type\"}[5m]))",
|
||||
"legendFormat": "{{job_type}} [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -404,7 +404,7 @@
|
||||
},
|
||||
{
|
||||
"title": "Transaction Overflow Rate",
|
||||
"description": "###### What this is:\n*Rate at which transaction jobs are shed when the queue's transaction limit is exceeded.*\n\n###### How it's computed:\n*Per-second rate of the overflow counter over the dashboard's rate interval, scaled to per minute. The counter is observed from the overlay's cumulative overflow tally.*\n\n###### Reading it:\n*Near zero is healthy; a rising rate means the job queue is shedding transaction work under load.*\n\n###### Healthy range:\n*0 overflows per minute.*\n\n###### Watch for:\n*Any sustained non-zero rate — the node is dropping transaction jobs because the queue is saturated.*\n\n###### Source:\n[MetricsRegistry.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/telemetry/MetricsRegistry.cpp)\n\n###### Function:\n`registerParityCounters (observed from Overlay::getJqTransOverflow)`",
|
||||
"description": "###### What this is:\n*Rate at which transaction jobs would be shed when the queue's transaction limit is exceeded.*\n\n###### How it's computed:\n*Per-second rate of the overflow counter over 5 minutes, scaled to per minute.*\n\n###### Reading it:\n*This counter is not currently wired up in the code, so the panel reads empty or zero regardless of load.*\n\n###### Healthy range:\n*Not applicable until wired; currently always empty or zero.*\n\n###### Watch for:\n*n/a until wired — this counter does not rise under load today.*\n\n###### Source:\n[MetricsRegistry.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/telemetry/MetricsRegistry.cpp)\n\n###### Function:\n`(job-queue transaction overflow counter)`",
|
||||
"type": "timeseries",
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
@@ -424,7 +424,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(jq_trans_overflow_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]) * 60",
|
||||
"expr": "rate(jq_trans_overflow_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m]) * 60",
|
||||
"legendFormat": "Overflows/min [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -548,7 +548,7 @@
|
||||
"label": "Node",
|
||||
"description": "Filter by rippled node (service.instance.id)",
|
||||
"type": "query",
|
||||
"query": "label_values(target_info, service_instance_id)",
|
||||
"query": "label_values(service_instance_id)",
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
|
||||
@@ -11,7 +11,7 @@
|
||||
"panels": [
|
||||
{
|
||||
"title": "Ledger Data Exchange (Bytes In)",
|
||||
"description": "###### What this is:\n*Inbound bytes for ledger-data message categories, split into aggregate get/share plus the transaction-set, transaction-node, and account-state-node sub-types the node receives from peers.*\n\n###### How it's computed:\n*Per-category inbound byte rate per node.*\n\n###### Reading it:\n*Normally low and flat once synced. Account-state-node traffic dominates during state sync; transaction-set-candidate traffic dominates during consensus catch-up.*\n\n###### Healthy range:\n*workload-dependent; low and steady on a synced node.*\n\n###### Watch for:\n*Sustained high account-state or tx-node inbound bytes on a node that should be caught up (repeated re-sync, missing history), or a single peer driving all traffic.*\n\n###### Source:\n[OverlayImpl.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/overlay/detail/OverlayImpl.cpp)\n\n###### Function:\n`OverlayImpl ctor (TrafficGauges)`",
|
||||
"description": "###### What this is:\n*Inbound bytes for ledger-data message categories, split into aggregate get/share plus the transaction-set, transaction-node, and account-state-node sub-types the node receives from peers.*\n\n###### How it's computed:\n*Per-category inbound byte counters, shown at their latest cumulative value per node.*\n\n###### Reading it:\n*Normally low and flat once synced. Account-state-node traffic dominates during state sync; transaction-set-candidate traffic dominates during consensus catch-up.*\n\n###### Healthy range:\n*workload-dependent; low and steady on a synced node.*\n\n###### Watch for:\n*Sustained high account-state or tx-node inbound bytes on a node that should be caught up (repeated re-sync, missing history), or a single peer driving all traffic.*\n\n###### Source:\n[OverlayImpl.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/overlay/detail/OverlayImpl.cpp)\n\n###### Function:\n`OverlayImpl ctor (TrafficGauges)`",
|
||||
"type": "timeseries",
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
@@ -31,7 +31,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(ledger_data_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "ledger_data_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Ledger Data Get [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -39,7 +39,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(ledger_data_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "ledger_data_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Ledger Data Share [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -47,7 +47,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(ledger_data_transaction_set_candidate_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "ledger_data_transaction_set_candidate_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "TX Set Candidate Get [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -55,7 +55,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(ledger_data_transaction_set_candidate_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "ledger_data_transaction_set_candidate_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "TX Set Candidate Share [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -63,7 +63,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(ledger_data_transaction_node_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "ledger_data_transaction_node_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "TX Node Get [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -71,7 +71,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(ledger_data_transaction_node_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "ledger_data_transaction_node_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "TX Node Share [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -79,7 +79,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(ledger_data_account_state_node_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "ledger_data_account_state_node_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Account State Node Get [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -87,13 +87,13 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(ledger_data_account_state_node_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "ledger_data_account_state_node_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Account State Node Share [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "Bps",
|
||||
"unit": "decbytes",
|
||||
"custom": {
|
||||
"axisLabel": "Bytes In",
|
||||
"spanNulls": true,
|
||||
@@ -107,7 +107,7 @@
|
||||
},
|
||||
{
|
||||
"title": "Ledger Share/Get Traffic (Bytes)",
|
||||
"description": "###### What this is:\n*Inbound bytes for the older ledger share/get message categories and their tx-set, tx-node, and account-state sub-types.*\n\n###### How it's computed:\n*Per-category inbound byte rate per node.*\n\n###### Reading it:\n*Usually small; these legacy categories carry ledger-fetch traffic for peers using the older protocol.*\n\n###### Healthy range:\n*workload-dependent; low on a synced node.*\n\n###### Watch for:\n*Large sustained volumes indicating heavy fetch load or a peer repeatedly requesting the same data.*\n\n###### Source:\n[OverlayImpl.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/overlay/detail/OverlayImpl.cpp)\n\n###### Function:\n`OverlayImpl ctor (TrafficGauges)`",
|
||||
"description": "###### What this is:\n*Inbound bytes for the older ledger share/get message categories and their tx-set, tx-node, and account-state sub-types.*\n\n###### How it's computed:\n*Per-category inbound byte counters at their latest value per node.*\n\n###### Reading it:\n*Usually small; these legacy categories carry ledger-fetch traffic for peers using the older protocol.*\n\n###### Healthy range:\n*workload-dependent; low on a synced node.*\n\n###### Watch for:\n*Large sustained volumes indicating heavy fetch load or a peer repeatedly requesting the same data.*\n\n###### Source:\n[OverlayImpl.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/overlay/detail/OverlayImpl.cpp)\n\n###### Function:\n`OverlayImpl ctor (TrafficGauges)`",
|
||||
"type": "timeseries",
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
@@ -127,7 +127,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(ledger_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "ledger_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Ledger Share In [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -135,7 +135,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(ledger_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "ledger_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Ledger Get In [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -143,7 +143,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(ledger_transaction_set_candidate_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "ledger_transaction_set_candidate_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "TX Set Candidate Share [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -151,7 +151,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(ledger_transaction_set_candidate_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "ledger_transaction_set_candidate_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "TX Set Candidate Get [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -159,7 +159,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(ledger_transaction_node_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "ledger_transaction_node_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "TX Node Share [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -167,7 +167,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(ledger_transaction_node_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "ledger_transaction_node_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "TX Node Get [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -175,7 +175,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(ledger_account_state_node_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "ledger_account_state_node_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Account State Share [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -183,13 +183,13 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(ledger_account_state_node_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "ledger_account_state_node_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Account State Get [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "Bps",
|
||||
"unit": "decbytes",
|
||||
"custom": {
|
||||
"axisLabel": "Bytes In",
|
||||
"spanNulls": true,
|
||||
@@ -203,7 +203,7 @@
|
||||
},
|
||||
{
|
||||
"title": "GetObject Traffic by Type (Bytes In)",
|
||||
"description": "###### What this is:\n*Inbound bytes for object-fetch traffic broken down by object type: ledger headers, individual transactions, transaction-tree nodes, and state-tree nodes.*\n\n###### How it's computed:\n*Per-type inbound byte rate per node.*\n\n###### Reading it:\n*Small during steady state; grows when the node fetches missing tree nodes.*\n\n###### Healthy range:\n*workload-dependent; low when synced.*\n\n###### Watch for:\n*A large share on state/tx nodes for long periods (persistent gap-filling), meaning the node keeps catching up.*\n\n###### Source:\n[OverlayImpl.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/overlay/detail/OverlayImpl.cpp)\n\n###### Function:\n`OverlayImpl ctor (TrafficGauges)`",
|
||||
"description": "###### What this is:\n*Inbound bytes for object-fetch traffic broken down by object type: ledger headers, individual transactions, transaction-tree nodes, and state-tree nodes.*\n\n###### How it's computed:\n*Per-type inbound byte counters at their latest value per node.*\n\n###### Reading it:\n*Small during steady state; grows when the node fetches missing tree nodes.*\n\n###### Healthy range:\n*workload-dependent; low when synced.*\n\n###### Watch for:\n*A large share on state/tx nodes for long periods (persistent gap-filling), meaning the node keeps catching up.*\n\n###### Source:\n[OverlayImpl.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/overlay/detail/OverlayImpl.cpp)\n\n###### Function:\n`OverlayImpl ctor (TrafficGauges)`",
|
||||
"type": "timeseries",
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
@@ -223,7 +223,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(getobject_ledger_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "getobject_ledger_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Ledger Get [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -231,7 +231,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(getobject_ledger_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "getobject_ledger_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Ledger Share [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -239,7 +239,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(getobject_transaction_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "getobject_transaction_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Transaction Get [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -247,7 +247,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(getobject_transaction_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "getobject_transaction_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Transaction Share [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -255,7 +255,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(getobject_transaction_node_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "getobject_transaction_node_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "TX Node Get [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -263,7 +263,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(getobject_transaction_node_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "getobject_transaction_node_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "TX Node Share [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -271,7 +271,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(getobject_account_state_node_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "getobject_account_state_node_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Account State Get [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -279,13 +279,13 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(getobject_account_state_node_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "getobject_account_state_node_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Account State Share [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "Bps",
|
||||
"unit": "decbytes",
|
||||
"custom": {
|
||||
"axisLabel": "Bytes In",
|
||||
"spanNulls": true,
|
||||
@@ -299,7 +299,7 @@
|
||||
},
|
||||
{
|
||||
"title": "GetObject Aggregate & Special Types (Bytes In)",
|
||||
"description": "###### What this is:\n*Aggregate object-fetch inbound bytes plus special buckets: content-addressed storage fetches, bulk fetch-pack downloads used during catch-up, and bulk transaction fetches.*\n\n###### How it's computed:\n*Per-category inbound byte rate per node.*\n\n###### Reading it:\n*Fetch-pack rises sharply while catching up a range of ledgers; near zero when fully synced.*\n\n###### Healthy range:\n*workload-dependent; low when synced.*\n\n###### Watch for:\n*Continuous fetch-pack traffic (node never fully catches up) or unexpectedly high content-store volume.*\n\n###### Source:\n[OverlayImpl.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/overlay/detail/OverlayImpl.cpp)\n\n###### Function:\n`OverlayImpl ctor (TrafficGauges)`",
|
||||
"description": "###### What this is:\n*Aggregate object-fetch inbound bytes plus special buckets: content-addressed storage fetches, bulk fetch-pack downloads used during catch-up, and bulk transaction fetches.*\n\n###### How it's computed:\n*Per-category inbound byte counters at their latest value per node.*\n\n###### Reading it:\n*Fetch-pack rises sharply while catching up a range of ledgers; near zero when fully synced.*\n\n###### Healthy range:\n*workload-dependent; low when synced.*\n\n###### Watch for:\n*Continuous fetch-pack traffic (node never fully catches up) or unexpectedly high content-store volume.*\n\n###### Source:\n[OverlayImpl.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/overlay/detail/OverlayImpl.cpp)\n\n###### Function:\n`OverlayImpl ctor (TrafficGauges)`",
|
||||
"type": "timeseries",
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
@@ -319,7 +319,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(getobject_cas_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "getobject_cas_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "CAS Get [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -327,7 +327,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(getobject_cas_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "getobject_cas_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "CAS Share [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -335,7 +335,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(getobject_fetch_pack_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "getobject_fetch_pack_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Fetch Pack Share [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -343,7 +343,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(getobject_fetch_pack_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "getobject_fetch_pack_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Fetch Pack Get [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -351,7 +351,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(getobject_transactions_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "getobject_transactions_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Transactions Get [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -359,7 +359,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(getobject_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "getobject_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Aggregate Get [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -367,13 +367,13 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(getobject_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "getobject_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Aggregate Share [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "Bps",
|
||||
"unit": "decbytes",
|
||||
"custom": {
|
||||
"axisLabel": "Bytes In",
|
||||
"spanNulls": true,
|
||||
@@ -387,7 +387,7 @@
|
||||
},
|
||||
{
|
||||
"title": "GetObject Messages by Type",
|
||||
"description": "###### What this is:\n*Count of individual object-fetch request/response messages per object type.*\n\n###### How it's computed:\n*Per-type inbound message rate per node.*\n\n###### Reading it:\n*Many messages with few bytes means small piecemeal fetches; few messages with many bytes means large batch transfers.*\n\n###### Healthy range:\n*workload-dependent.*\n\n###### Watch for:\n*High message counts with tiny payloads sustained over time (inefficient per-node fetching).*\n\n###### Source:\n[OverlayImpl.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/overlay/detail/OverlayImpl.cpp)\n\n###### Function:\n`OverlayImpl ctor (TrafficGauges)`",
|
||||
"description": "###### What this is:\n*Count of individual object-fetch request/response messages per object type.*\n\n###### How it's computed:\n*Per-type inbound message counters at their latest value per node.*\n\n###### Reading it:\n*Many messages with few bytes means small piecemeal fetches; few messages with many bytes means large batch transfers.*\n\n###### Healthy range:\n*workload-dependent.*\n\n###### Watch for:\n*High message counts with tiny payloads sustained over time (inefficient per-node fetching).*\n\n###### Source:\n[OverlayImpl.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/overlay/detail/OverlayImpl.cpp)\n\n###### Function:\n`OverlayImpl ctor (TrafficGauges)`",
|
||||
"type": "timeseries",
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
@@ -407,7 +407,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(getobject_ledger_get_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "getobject_ledger_get_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Ledger Get [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -415,7 +415,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(getobject_transaction_get_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "getobject_transaction_get_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Transaction Get [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -423,7 +423,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(getobject_transaction_node_get_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "getobject_transaction_node_get_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "TX Node Get [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -431,7 +431,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(getobject_account_state_node_get_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "getobject_account_state_node_get_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Account State Get [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -439,7 +439,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(getobject_cas_get_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "getobject_cas_get_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "CAS Get [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -447,7 +447,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(getobject_fetch_pack_get_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "getobject_fetch_pack_get_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Fetch Pack Get [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -455,13 +455,13 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(getobject_transactions_get_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "getobject_transactions_get_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Transactions Get [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "cps",
|
||||
"unit": "short",
|
||||
"custom": {
|
||||
"axisLabel": "Messages In",
|
||||
"spanNulls": true,
|
||||
@@ -613,7 +613,7 @@
|
||||
"label": "Node",
|
||||
"description": "Filter by rippled node (service.instance.id)",
|
||||
"type": "query",
|
||||
"query": "label_values(target_info, service_instance_id)",
|
||||
"query": "label_values(ledger_data_get_bytes_in, service_instance_id)",
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
|
||||
@@ -30,7 +30,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"ledger.build\"}[$__rate_interval]))",
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"ledger.build\"}[5m]))",
|
||||
"legendFormat": "Builds / Sec [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -103,7 +103,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"ledger.validate\"}[$__rate_interval]))",
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"ledger.validate\"}[5m]))",
|
||||
"legendFormat": "Validations / Sec [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -213,7 +213,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"tx.apply\"}[$__rate_interval]))",
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"tx.apply\"}[5m]))",
|
||||
"legendFormat": "tx.apply / Sec [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -253,7 +253,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"ledger.store\"}[$__rate_interval]))",
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"ledger.store\"}[5m]))",
|
||||
"legendFormat": "Stores / Sec [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -395,7 +395,7 @@
|
||||
"label": "Node",
|
||||
"description": "Filter by rippled node (service.instance.id — e.g. Node-1)",
|
||||
"type": "query",
|
||||
"query": "label_values(target_info, service_instance_id)",
|
||||
"query": "label_values(span_calls_total, service_instance_id)",
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
|
||||
@@ -59,7 +59,7 @@
|
||||
},
|
||||
{
|
||||
"title": "Peer Disconnects",
|
||||
"description": "###### What this is:\n*Peer connections that dropped, as a per-interval increase.*\n\n###### How it's computed:\n*Disconnect events per rate-interval, per node.*\n\n###### Reading it:\n*A flat or slowly rising line is normal; the slope matters more than the absolute value.*\n\n###### Healthy range:\n*workload-dependent; slow, steady growth.*\n\n###### Watch for:\n*Sharp step-ups in the slope (network instability, resource exhaustion, or many peers dropping the node at once).*\n\n###### Source:\n[OverlayImpl.h](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/overlay/detail/OverlayImpl.h)\n\n###### Function:\n`OverlayImpl::Stats ctor`",
|
||||
"description": "###### What this is:\n*Cumulative count of peer connections that have dropped.*\n\n###### How it's computed:\n*Running total of disconnect events per node.*\n\n###### Reading it:\n*A flat or slowly rising line is normal; the slope matters more than the absolute value.*\n\n###### Healthy range:\n*workload-dependent; slow, steady growth.*\n\n###### Watch for:\n*Sharp step-ups in the slope (network instability, resource exhaustion, or many peers dropping the node at once).*\n\n###### Source:\n[OverlayImpl.h](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/overlay/detail/OverlayImpl.h)\n\n###### Function:\n`OverlayImpl::Stats ctor`",
|
||||
"type": "timeseries",
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
@@ -79,7 +79,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "increase(overlay_peer_disconnects{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "overlay_peer_disconnects{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Disconnects [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -87,7 +87,7 @@
|
||||
"defaults": {
|
||||
"unit": "none",
|
||||
"custom": {
|
||||
"axisLabel": "Disconnects per interval",
|
||||
"axisLabel": "Disconnects",
|
||||
"spanNulls": true,
|
||||
"insertNulls": false,
|
||||
"showPoints": "auto",
|
||||
@@ -99,7 +99,7 @@
|
||||
},
|
||||
{
|
||||
"title": "Total Network Bytes",
|
||||
"description": "###### What this is:\n*Total bytes received and sent across all peer connections.*\n\n###### How it's computed:\n*Per-second in/out byte rate per node.*\n\n###### Reading it:\n*Overall bandwidth footprint; in and out usually track network activity together.*\n\n###### Healthy range:\n*workload-dependent.*\n\n###### Watch for:\n*Sudden sustained jumps not matched by ledger or transaction activity (relay storms or a noisy peer).*\n\n###### Source:\n[OverlayImpl.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/overlay/detail/OverlayImpl.cpp)\n\n###### Function:\n`OverlayImpl ctor (TrafficGauges)`",
|
||||
"description": "###### What this is:\n*Total bytes received and sent across all peer connections.*\n\n###### How it's computed:\n*Current cumulative in/out byte totals per node.*\n\n###### Reading it:\n*Overall bandwidth footprint; in and out usually track network activity together.*\n\n###### Healthy range:\n*workload-dependent.*\n\n###### Watch for:\n*Sudden sustained jumps not matched by ledger or transaction activity (relay storms or a noisy peer).*\n\n###### Source:\n[OverlayImpl.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/overlay/detail/OverlayImpl.cpp)\n\n###### Function:\n`OverlayImpl ctor (TrafficGauges)`",
|
||||
"type": "timeseries",
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
@@ -119,7 +119,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(total_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "total_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Bytes In [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -127,13 +127,13 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(total_bytes_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "total_bytes_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Bytes Out [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "Bps",
|
||||
"unit": "decbytes",
|
||||
"custom": {
|
||||
"axisLabel": "Bytes",
|
||||
"spanNulls": true,
|
||||
@@ -147,7 +147,7 @@
|
||||
},
|
||||
{
|
||||
"title": "Total Network Messages",
|
||||
"description": "###### What this is:\n*Total messages received and sent across all peer connections.*\n\n###### How it's computed:\n*Per-second in/out message rate per node.*\n\n###### Reading it:\n*Overall message throughput of the overlay; complements the byte totals.*\n\n###### Healthy range:\n*workload-dependent.*\n\n###### Watch for:\n*Message count climbing far faster than bytes (many tiny messages, possible flooding).*\n\n###### Source:\n[OverlayImpl.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/overlay/detail/OverlayImpl.cpp)\n\n###### Function:\n`OverlayImpl ctor (TrafficGauges)`",
|
||||
"description": "###### What this is:\n*Total messages received and sent across all peer connections.*\n\n###### How it's computed:\n*Current cumulative in/out message totals per node.*\n\n###### Reading it:\n*Overall message throughput of the overlay; complements the byte totals.*\n\n###### Healthy range:\n*workload-dependent.*\n\n###### Watch for:\n*Message count climbing far faster than bytes (many tiny messages, possible flooding).*\n\n###### Source:\n[OverlayImpl.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/overlay/detail/OverlayImpl.cpp)\n\n###### Function:\n`OverlayImpl ctor (TrafficGauges)`",
|
||||
"type": "timeseries",
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
@@ -167,7 +167,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(total_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "total_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Messages In [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -175,13 +175,13 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(total_messages_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "total_messages_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Messages Out [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "cps",
|
||||
"unit": "short",
|
||||
"custom": {
|
||||
"axisLabel": "Messages",
|
||||
"spanNulls": true,
|
||||
@@ -195,7 +195,7 @@
|
||||
},
|
||||
{
|
||||
"title": "Transaction Traffic",
|
||||
"description": "###### What this is:\n*Transaction relay messages in and out, plus duplicate transaction messages received.*\n\n###### How it's computed:\n*Per-second message rate for the transaction and transaction-duplicate categories.*\n\n###### Reading it:\n*In/out rise with network transaction volume; duplicates are transactions the node already had.*\n\n###### Healthy range:\n*workload-dependent; duplicates a modest fraction of inbound.*\n\n###### Watch for:\n*Duplicate inbound approaching or exceeding unique inbound (redundant relay), or a sharp spike suggesting transaction flooding.*\n\n###### Source:\n[OverlayImpl.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/overlay/detail/OverlayImpl.cpp)\n\n###### Function:\n`OverlayImpl ctor (TrafficGauges)`",
|
||||
"description": "###### What this is:\n*Transaction relay messages in and out, plus duplicate transaction messages received.*\n\n###### How it's computed:\n*Current message counts for the transaction and transaction-duplicate categories.*\n\n###### Reading it:\n*In/out rise with network transaction volume; duplicates are transactions the node already had.*\n\n###### Healthy range:\n*workload-dependent; duplicates a modest fraction of inbound.*\n\n###### Watch for:\n*Duplicate inbound approaching or exceeding unique inbound (redundant relay), or a sharp spike suggesting transaction flooding.*\n\n###### Source:\n[OverlayImpl.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/overlay/detail/OverlayImpl.cpp)\n\n###### Function:\n`OverlayImpl ctor (TrafficGauges)`",
|
||||
"type": "timeseries",
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
@@ -215,7 +215,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(transactions_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "transactions_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "TX Messages In [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -223,7 +223,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(transactions_messages_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "transactions_messages_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "TX Messages Out [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -231,13 +231,13 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(transactions_duplicate_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "transactions_duplicate_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "TX Duplicate In [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "cps",
|
||||
"unit": "short",
|
||||
"custom": {
|
||||
"axisLabel": "Messages",
|
||||
"spanNulls": true,
|
||||
@@ -251,7 +251,7 @@
|
||||
},
|
||||
{
|
||||
"title": "Proposal Traffic",
|
||||
"description": "###### What this is:\n*Consensus proposal messages in/out, plus untrusted and duplicate proposal messages received.*\n\n###### How it's computed:\n*Per-second message rate for the proposal, proposal-untrusted, and proposal-duplicate categories.*\n\n###### Reading it:\n*Trusted in/out track consensus rounds; untrusted come from validators not on this node's trusted list.*\n\n###### Healthy range:\n*workload-dependent; untrusted and duplicates low relative to trusted.*\n\n###### Watch for:\n*High untrusted (trusted-list misconfiguration) or high duplicates (inefficient relay or proposal spam).*\n\n###### Source:\n[OverlayImpl.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/overlay/detail/OverlayImpl.cpp)\n\n###### Function:\n`OverlayImpl ctor (TrafficGauges)`",
|
||||
"description": "###### What this is:\n*Consensus proposal messages in/out, plus untrusted and duplicate proposal messages received.*\n\n###### How it's computed:\n*Current message counts for the proposal, proposal-untrusted, and proposal-duplicate categories.*\n\n###### Reading it:\n*Trusted in/out track consensus rounds; untrusted come from validators not on this node's trusted list.*\n\n###### Healthy range:\n*workload-dependent; untrusted and duplicates low relative to trusted.*\n\n###### Watch for:\n*High untrusted (trusted-list misconfiguration) or high duplicates (inefficient relay or proposal spam).*\n\n###### Source:\n[OverlayImpl.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/overlay/detail/OverlayImpl.cpp)\n\n###### Function:\n`OverlayImpl ctor (TrafficGauges)`",
|
||||
"type": "timeseries",
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
@@ -271,7 +271,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(proposals_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "proposals_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Proposals In [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -279,7 +279,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(proposals_messages_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "proposals_messages_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Proposals Out [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -287,7 +287,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(proposals_untrusted_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "proposals_untrusted_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Untrusted In [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -295,13 +295,13 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(proposals_duplicate_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "proposals_duplicate_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Duplicate In [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "cps",
|
||||
"unit": "short",
|
||||
"custom": {
|
||||
"axisLabel": "Messages",
|
||||
"spanNulls": true,
|
||||
@@ -315,7 +315,7 @@
|
||||
},
|
||||
{
|
||||
"title": "Validation Traffic",
|
||||
"description": "###### What this is:\n*Validation messages in/out, plus untrusted and duplicate validation messages received.*\n\n###### How it's computed:\n*Per-second message rate for the validation, validation-untrusted, and validation-duplicate categories.*\n\n###### Reading it:\n*Trusted validations should arrive steadily each ledger; untrusted come from non-trusted validators.*\n\n###### Healthy range:\n*workload-dependent; untrusted and duplicates low relative to trusted.*\n\n###### Watch for:\n*Rising untrusted or duplicate validations (trusted-list health issues or validation spam).*\n\n###### Source:\n[OverlayImpl.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/overlay/detail/OverlayImpl.cpp)\n\n###### Function:\n`OverlayImpl ctor (TrafficGauges)`",
|
||||
"description": "###### What this is:\n*Validation messages in/out, plus untrusted and duplicate validation messages received.*\n\n###### How it's computed:\n*Current message counts for the validation, validation-untrusted, and validation-duplicate categories.*\n\n###### Reading it:\n*Trusted validations should arrive steadily each ledger; untrusted come from non-trusted validators.*\n\n###### Healthy range:\n*workload-dependent; untrusted and duplicates low relative to trusted.*\n\n###### Watch for:\n*Rising untrusted or duplicate validations (trusted-list health issues or validation spam).*\n\n###### Source:\n[OverlayImpl.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/overlay/detail/OverlayImpl.cpp)\n\n###### Function:\n`OverlayImpl ctor (TrafficGauges)`",
|
||||
"type": "timeseries",
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
@@ -335,7 +335,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(validations_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "validations_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Validations In [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -343,7 +343,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(validations_messages_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "validations_messages_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Validations Out [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -351,7 +351,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(validations_untrusted_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "validations_untrusted_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Untrusted In [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -359,13 +359,13 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(validations_duplicate_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "validations_duplicate_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Duplicate In [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "cps",
|
||||
"unit": "short",
|
||||
"custom": {
|
||||
"axisLabel": "Messages",
|
||||
"spanNulls": true,
|
||||
@@ -379,7 +379,7 @@
|
||||
},
|
||||
{
|
||||
"title": "Overlay Traffic by Category (Bytes In)",
|
||||
"description": "###### What this is:\n*Top overlay traffic categories ranked by inbound bytes, excluding the all-traffic total.*\n\n###### How it's computed:\n*Top categories by inbound byte rate per node.*\n\n###### Reading it:\n*Shows which message types dominate receive bandwidth right now.*\n\n###### Healthy range:\n*workload-dependent; transactions, proposals, and validations typically lead on a synced node.*\n\n###### Watch for:\n*A fetch or ledger-data category topping the list (sync activity) or an unexpected category dominating.*\n\n###### Source:\n[OverlayImpl.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/overlay/detail/OverlayImpl.cpp)\n\n###### Function:\n`OverlayImpl ctor (TrafficGauges)`",
|
||||
"description": "###### What this is:\n*Top overlay traffic categories ranked by inbound bytes, excluding the all-traffic total.*\n\n###### How it's computed:\n*Top categories by latest inbound byte value per node.*\n\n###### Reading it:\n*Shows which message types dominate receive bandwidth right now.*\n\n###### Healthy range:\n*workload-dependent; transactions, proposals, and validations typically lead on a synced node.*\n\n###### Watch for:\n*A fetch or ledger-data category topping the list (sync activity) or an unexpected category dominating.*\n\n###### Source:\n[OverlayImpl.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/overlay/detail/OverlayImpl.cpp)\n\n###### Function:\n`OverlayImpl ctor (TrafficGauges)`",
|
||||
"type": "bargauge",
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
@@ -399,13 +399,13 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "topk(10, rate({service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", __name__=~\".*_bytes_in\", __name__!~\"total_.*\"}[$__rate_interval]))",
|
||||
"expr": "topk(10, {service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", __name__=~\".*_bytes_in\", __name__!~\"total_.*\"})",
|
||||
"legendFormat": "{{__name__}} [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "Bps"
|
||||
"unit": "decbytes"
|
||||
},
|
||||
"overrides": [
|
||||
{
|
||||
@@ -697,7 +697,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(transactions_duplicate_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "rate(transactions_duplicate_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])",
|
||||
"legendFormat": "TX Duplicate In"
|
||||
},
|
||||
{
|
||||
@@ -705,7 +705,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(transactions_duplicate_bytes_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "rate(transactions_duplicate_bytes_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])",
|
||||
"legendFormat": "TX Duplicate Out"
|
||||
},
|
||||
{
|
||||
@@ -713,7 +713,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(proposals_duplicate_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "rate(proposals_duplicate_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])",
|
||||
"legendFormat": "Proposals Duplicate In"
|
||||
},
|
||||
{
|
||||
@@ -721,7 +721,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(proposals_duplicate_bytes_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "rate(proposals_duplicate_bytes_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])",
|
||||
"legendFormat": "Proposals Duplicate Out"
|
||||
},
|
||||
{
|
||||
@@ -729,7 +729,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(validations_duplicate_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "rate(validations_duplicate_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])",
|
||||
"legendFormat": "Validations Duplicate In"
|
||||
},
|
||||
{
|
||||
@@ -737,7 +737,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(validations_duplicate_bytes_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "rate(validations_duplicate_bytes_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])",
|
||||
"legendFormat": "Validations Duplicate Out"
|
||||
}
|
||||
],
|
||||
@@ -777,7 +777,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "topk(15, rate({__name__=~\".*_bytes_in\", __name__!~\"total_.*\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]))",
|
||||
"expr": "topk(15, rate({__name__=~\".*_bytes_in\", __name__!~\"total_.*\", service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m]))",
|
||||
"legendFormat": "{{__name__}}"
|
||||
}
|
||||
],
|
||||
@@ -878,7 +878,7 @@
|
||||
"label": "Node",
|
||||
"description": "Filter by rippled node (service.instance.id)",
|
||||
"type": "query",
|
||||
"query": "label_values(target_info, service_instance_id)",
|
||||
"query": "label_values(peer_finder_active_inbound_peers, service_instance_id)",
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
|
||||
@@ -110,8 +110,8 @@
|
||||
}
|
||||
},
|
||||
{
|
||||
"title": "Operating Mode (Time Share)",
|
||||
"description": "###### What this is:\n*Fraction of recent wall-clock time the node spent in each operating mode (Disconnected, Connected, Syncing, Tracking, Full).*\n\n###### How it's computed:\n*Per-second rate of each per-mode duration counter divided by the sum of all five mode rates, giving each mode's time share.*\n\n###### Reading it:\n*The Full share should sit at or near 1.0 and dominate; other shares should be near 0.*\n\n###### Healthy range:\n*Full share close to 1.0.*\n\n###### Watch for:\n*Share accumulating in Syncing, Connected, or Disconnected means the node is not staying fully synced.*\n\n###### Source:\n[NetworkOPs.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/app/misc/NetworkOPs.cpp)\n\n###### Function:\n`NetworkOPsImp::Stats`",
|
||||
"title": "Operating Mode Duration",
|
||||
"description": "###### What this is:\n*Cumulative time the node has spent in each operating mode (Disconnected, Connected, Syncing, Tracking, Full).*\n\n###### How it's computed:\n*Current value of each per-mode duration counter, plotted as lines.*\n\n###### Reading it:\n*The Full line should climb steadily and dominate; other lines should be flat.*\n\n###### Healthy range:\n*Almost all accumulated time in Full mode.*\n\n###### Watch for:\n*Time accumulating in Syncing, Connected, or Disconnected means the node is not staying fully synced.*\n\n###### Source:\n[NetworkOPs.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/app/misc/NetworkOPs.cpp)\n\n###### Function:\n`NetworkOPsImp::Stats`",
|
||||
"type": "timeseries",
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
@@ -131,7 +131,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(state_accounting_full_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]) / (rate(state_accounting_disconnected_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]) + rate(state_accounting_connected_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]) + rate(state_accounting_syncing_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]) + rate(state_accounting_tracking_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]) + rate(state_accounting_full_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]))",
|
||||
"expr": "state_accounting_full_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Full [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -139,7 +139,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(state_accounting_tracking_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]) / (rate(state_accounting_disconnected_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]) + rate(state_accounting_connected_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]) + rate(state_accounting_syncing_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]) + rate(state_accounting_tracking_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]) + rate(state_accounting_full_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]))",
|
||||
"expr": "state_accounting_tracking_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Tracking [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -147,7 +147,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(state_accounting_syncing_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]) / (rate(state_accounting_disconnected_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]) + rate(state_accounting_connected_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]) + rate(state_accounting_syncing_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]) + rate(state_accounting_tracking_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]) + rate(state_accounting_full_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]))",
|
||||
"expr": "state_accounting_syncing_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Syncing [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -155,7 +155,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(state_accounting_connected_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]) / (rate(state_accounting_disconnected_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]) + rate(state_accounting_connected_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]) + rate(state_accounting_syncing_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]) + rate(state_accounting_tracking_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]) + rate(state_accounting_full_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]))",
|
||||
"expr": "state_accounting_connected_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Connected [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -163,17 +163,15 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(state_accounting_disconnected_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]) / (rate(state_accounting_disconnected_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]) + rate(state_accounting_connected_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]) + rate(state_accounting_syncing_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]) + rate(state_accounting_tracking_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]) + rate(state_accounting_full_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]))",
|
||||
"expr": "state_accounting_disconnected_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Disconnected [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "percentunit",
|
||||
"min": 0,
|
||||
"max": 1,
|
||||
"unit": "µs",
|
||||
"custom": {
|
||||
"axisLabel": "Time Share",
|
||||
"axisLabel": "Duration",
|
||||
"spanNulls": true,
|
||||
"insertNulls": false,
|
||||
"showPoints": "auto",
|
||||
@@ -357,7 +355,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(ledger_fetches_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "rate(ledger_fetches_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])",
|
||||
"legendFormat": "Fetches / Sec [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -391,7 +389,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(ledger_history_mismatch_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "rate(ledger_history_mismatch_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])",
|
||||
"legendFormat": "Mismatches / Sec [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -449,7 +447,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(nodestore_state{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", metric=\"node_reads_total\"}[$__rate_interval])",
|
||||
"expr": "nodestore_state{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", metric=\"node_reads_total\"}",
|
||||
"legendFormat": "Reads Total [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -457,7 +455,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(nodestore_state{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", metric=\"node_reads_hit\"}[$__rate_interval])",
|
||||
"expr": "nodestore_state{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", metric=\"node_reads_hit\"}",
|
||||
"legendFormat": "Reads Hit (cache) [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -465,23 +463,15 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(nodestore_state{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", metric=\"node_writes\"}[$__rate_interval])",
|
||||
"expr": "nodestore_state{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", metric=\"node_writes\"}",
|
||||
"legendFormat": "Writes Total [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(nodestore_state{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", metric=\"node_reads_hit\"}[$__rate_interval]) / rate(nodestore_state{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", metric=\"node_reads_total\"}[$__rate_interval])",
|
||||
"legendFormat": "Cache Hit Ratio [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "ops",
|
||||
"unit": "short",
|
||||
"custom": {
|
||||
"axisLabel": "Operations / Sec",
|
||||
"axisLabel": "Operations",
|
||||
"drawStyle": "line",
|
||||
"lineWidth": 2,
|
||||
"fillOpacity": 10
|
||||
@@ -490,24 +480,7 @@
|
||||
"mode": "palette-classic"
|
||||
}
|
||||
},
|
||||
"overrides": [
|
||||
{
|
||||
"matcher": {
|
||||
"id": "byRegexp",
|
||||
"options": "/Cache Hit Ratio/"
|
||||
},
|
||||
"properties": [
|
||||
{
|
||||
"id": "unit",
|
||||
"value": "percentunit"
|
||||
},
|
||||
{
|
||||
"id": "custom.axisPlacement",
|
||||
"value": "right"
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
"overrides": []
|
||||
}
|
||||
},
|
||||
{
|
||||
@@ -598,7 +571,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(nodestore_state{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", metric=\"node_read_bytes\"}[$__rate_interval])",
|
||||
"expr": "nodestore_state{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", metric=\"node_read_bytes\"}",
|
||||
"legendFormat": "Bytes Read [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -606,15 +579,15 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(nodestore_state{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", metric=\"node_written_bytes\"}[$__rate_interval])",
|
||||
"expr": "nodestore_state{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", metric=\"node_written_bytes\"}",
|
||||
"legendFormat": "Bytes Written [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "Bps",
|
||||
"unit": "decbytes",
|
||||
"custom": {
|
||||
"axisLabel": "Throughput",
|
||||
"axisLabel": "Bytes",
|
||||
"drawStyle": "line",
|
||||
"lineWidth": 2,
|
||||
"fillOpacity": 10
|
||||
@@ -672,15 +645,15 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(nodestore_state{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", metric=\"node_reads_duration_us\"}[$__rate_interval]) / 1e6",
|
||||
"legendFormat": "Read Busy Ratio [{{service_instance_id}}]"
|
||||
"expr": "nodestore_state{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", metric=\"node_reads_duration_us\"}",
|
||||
"legendFormat": "Read Duration (µs) [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "short",
|
||||
"custom": {
|
||||
"axisLabel": "Threads / Bundle",
|
||||
"axisLabel": "Count / µs",
|
||||
"drawStyle": "line",
|
||||
"lineWidth": 2,
|
||||
"fillOpacity": 10
|
||||
@@ -689,24 +662,7 @@
|
||||
"mode": "palette-classic"
|
||||
}
|
||||
},
|
||||
"overrides": [
|
||||
{
|
||||
"matcher": {
|
||||
"id": "byRegexp",
|
||||
"options": "/Read Busy Ratio/"
|
||||
},
|
||||
"properties": [
|
||||
{
|
||||
"id": "unit",
|
||||
"value": "percentunit"
|
||||
},
|
||||
{
|
||||
"id": "custom.axisPlacement",
|
||||
"value": "right"
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
"overrides": []
|
||||
}
|
||||
},
|
||||
{
|
||||
@@ -1141,9 +1097,8 @@
|
||||
],
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "short",
|
||||
"unit": "none",
|
||||
"custom": {
|
||||
"axisLabel": "Proposers",
|
||||
"drawStyle": "line",
|
||||
"lineWidth": 2,
|
||||
"fillOpacity": 10
|
||||
@@ -1152,24 +1107,7 @@
|
||||
"mode": "palette-classic"
|
||||
}
|
||||
},
|
||||
"overrides": [
|
||||
{
|
||||
"matcher": {
|
||||
"id": "byRegexp",
|
||||
"options": "/Converge Time ms/"
|
||||
},
|
||||
"properties": [
|
||||
{
|
||||
"id": "unit",
|
||||
"value": "ms"
|
||||
},
|
||||
{
|
||||
"id": "custom.axisPlacement",
|
||||
"value": "right"
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
"overrides": []
|
||||
}
|
||||
},
|
||||
{
|
||||
@@ -1577,8 +1515,8 @@
|
||||
},
|
||||
{
|
||||
"title": "Ledger Age",
|
||||
"description": "###### What this is:\n*Seconds since the last validated ledger closed, plotted over time.*\n\n###### How it's computed:\n*Current value of the ledger-age economy gauge, sampled each interval.*\n\n###### Reading it:\n*Lower is better; it should stay near the ledger close interval. Mirrors the Validated Ledger Age panel.*\n\n###### Healthy range:\n*Under about 10 seconds.*\n\n###### Watch for:\n*Growth beyond the expected close interval, meaning the node is not keeping up with validated ledgers.*\n\n###### Source:\n[MetricsRegistry.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/telemetry/MetricsRegistry.cpp)\n\n###### Function:\n`registerLedgerEconomyGauge`",
|
||||
"type": "timeseries",
|
||||
"description": "###### What this is:\n*Seconds since the last validated ledger closed.*\n\n###### How it's computed:\n*Current value of the ledger-age economy gauge.*\n\n###### Reading it:\n*Lower is better; it should stay near the ledger close interval. Mirrors the Validated Ledger Age panel.*\n\n###### Healthy range:\n*Under about 8 seconds.*\n\n###### Watch for:\n*Growth beyond the expected close interval, meaning the node is not keeping up with validated ledgers.*\n\n###### Source:\n[MetricsRegistry.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/telemetry/MetricsRegistry.cpp)\n\n###### Function:\n`registerLedgerEconomyGauge`",
|
||||
"type": "stat",
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
"w": 6,
|
||||
@@ -1612,7 +1550,7 @@
|
||||
},
|
||||
{
|
||||
"color": "yellow",
|
||||
"value": 10
|
||||
"value": 8
|
||||
},
|
||||
{
|
||||
"color": "red",
|
||||
@@ -1620,19 +1558,7 @@
|
||||
}
|
||||
]
|
||||
},
|
||||
"custom": {
|
||||
"axisLabel": "Ledger Age (s)",
|
||||
"drawStyle": "line",
|
||||
"lineWidth": 2,
|
||||
"fillOpacity": 10,
|
||||
"spanNulls": true,
|
||||
"insertNulls": false,
|
||||
"showPoints": "auto",
|
||||
"pointSize": 3,
|
||||
"thresholdsStyle": {
|
||||
"mode": "line"
|
||||
}
|
||||
}
|
||||
"custom": {}
|
||||
},
|
||||
"overrides": []
|
||||
}
|
||||
@@ -2097,7 +2023,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(state_accounting_full_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]) / 1000000",
|
||||
"expr": "rate(state_accounting_full_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m]) / 1000000",
|
||||
"legendFormat": "Full Mode Rate"
|
||||
},
|
||||
{
|
||||
@@ -2105,7 +2031,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(state_accounting_tracking_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]) / 1000000",
|
||||
"expr": "rate(state_accounting_tracking_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m]) / 1000000",
|
||||
"legendFormat": "Tracking Mode Rate"
|
||||
}
|
||||
],
|
||||
@@ -2265,7 +2191,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (outcome, service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"ledger.acquire\"}[$__rate_interval]))",
|
||||
"expr": "sum by (outcome, service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"ledger.acquire\"}[5m]))",
|
||||
"legendFormat": "{{outcome}} [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -2282,155 +2208,6 @@
|
||||
},
|
||||
"overrides": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"title": "Operating Mode (State Timeline)",
|
||||
"description": "###### What this is:\n*The node's operating mode over time as a colored timeline (Disconnected, Connected, Syncing, Tracking, Full).*\n\n###### How it's computed:\n*Current value of the server-state gauge mapped to a mode name; equal consecutive samples are merged into a single band.*\n\n###### Reading it:\n*A solid green Full band across the window is the goal; other colors mark periods the node was not fully synced.*\n\n###### Healthy range:\n*Continuously Full (green).*\n\n###### Watch for:\n*Bands of Syncing, Connected, or Disconnected, which pinpoint exactly when the node dropped out of Full.*\n\n###### Source:\n[MetricsRegistry.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/telemetry/MetricsRegistry.cpp)\n\n###### Function:\n`registerServerInfoGauge`",
|
||||
"type": "state-timeline",
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
"w": 18,
|
||||
"x": 0,
|
||||
"y": 134
|
||||
},
|
||||
"options": {
|
||||
"mergeValues": true,
|
||||
"showValue": "auto",
|
||||
"alignValue": "left",
|
||||
"rowHeight": 0.9,
|
||||
"tooltip": {
|
||||
"mode": "single",
|
||||
"sort": "none"
|
||||
}
|
||||
},
|
||||
"targets": [
|
||||
{
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "server_info{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", metric=\"server_state\"}",
|
||||
"legendFormat": "State [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "none",
|
||||
"mappings": [
|
||||
{
|
||||
"type": "value",
|
||||
"options": {
|
||||
"0": {
|
||||
"text": "Disconnected",
|
||||
"color": "red"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "value",
|
||||
"options": {
|
||||
"1": {
|
||||
"text": "Connected",
|
||||
"color": "orange"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "value",
|
||||
"options": {
|
||||
"2": {
|
||||
"text": "Syncing",
|
||||
"color": "yellow"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "value",
|
||||
"options": {
|
||||
"3": {
|
||||
"text": "Tracking",
|
||||
"color": "blue"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "value",
|
||||
"options": {
|
||||
"4": {
|
||||
"text": "Full",
|
||||
"color": "green"
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"custom": {
|
||||
"fillOpacity": 80,
|
||||
"lineWidth": 0
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"title": "% Time in Full",
|
||||
"description": "###### What this is:\n*The share of recent wall-clock time the node spent in Full mode.*\n\n###### How it's computed:\n*Per-second rate of the Full-mode duration counter divided by the sum of the per-second rates of all five mode duration counters.*\n\n###### Reading it:\n*Higher is better; 1.0 means the node was fully synced for the entire window.*\n\n###### Healthy range:\n*At or above 0.99.*\n\n###### Watch for:\n*Values dropping below 0.9, meaning the node spent meaningful time outside Full mode.*\n\n###### Source:\n[NetworkOPs.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/app/misc/NetworkOPs.cpp)\n\n###### Function:\n`NetworkOPsImp::Stats`",
|
||||
"type": "stat",
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
"w": 6,
|
||||
"x": 18,
|
||||
"y": 134
|
||||
},
|
||||
"options": {
|
||||
"reduceOptions": {
|
||||
"calcs": ["lastNotNull"],
|
||||
"fields": "",
|
||||
"values": false
|
||||
},
|
||||
"orientation": "auto",
|
||||
"textMode": "auto",
|
||||
"colorMode": "value",
|
||||
"tooltip": {
|
||||
"mode": "single",
|
||||
"sort": "none"
|
||||
}
|
||||
},
|
||||
"targets": [
|
||||
{
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(state_accounting_full_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]) / (rate(state_accounting_disconnected_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]) + rate(state_accounting_connected_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]) + rate(state_accounting_syncing_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]) + rate(state_accounting_tracking_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]) + rate(state_accounting_full_duration{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]))",
|
||||
"legendFormat": "% Time in Full [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "percentunit",
|
||||
"min": 0,
|
||||
"max": 1,
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{
|
||||
"color": "red",
|
||||
"value": null
|
||||
},
|
||||
{
|
||||
"color": "yellow",
|
||||
"value": 0.9
|
||||
},
|
||||
{
|
||||
"color": "green",
|
||||
"value": 0.99
|
||||
}
|
||||
]
|
||||
},
|
||||
"custom": {}
|
||||
},
|
||||
"overrides": []
|
||||
}
|
||||
}
|
||||
],
|
||||
"schemaVersion": 39,
|
||||
@@ -2515,7 +2292,7 @@
|
||||
"label": "Node",
|
||||
"description": "Filter by rippled node (service.instance.id)",
|
||||
"type": "query",
|
||||
"query": "label_values(target_info, service_instance_id)",
|
||||
"query": "label_values(ledgermaster_validated_ledger_age, service_instance_id)",
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
|
||||
@@ -11,7 +11,7 @@
|
||||
"panels": [
|
||||
{
|
||||
"title": "Squelch Traffic (Messages)",
|
||||
"description": "###### What this is:\n*Squelch relay-control messages in/out, plus messages suppressed by squelch and squelch directives that were ignored. Squelch reduces redundant message forwarding between peers.*\n\n###### How it's computed:\n*Per-second message rate for the squelch, squelch-suppressed, and squelch-ignored categories, in and out.*\n\n###### Reading it:\n*High suppressed counts mean squelch is saving bandwidth; ignored should stay low.*\n\n###### Healthy range:\n*workload-dependent; suppressed far above ignored.*\n\n###### Watch for:\n*High ignored counts (peers not honoring squelch) or squelch traffic itself dominating.*\n\n###### Source:\n[OverlayImpl.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/overlay/detail/OverlayImpl.cpp)\n\n###### Function:\n`OverlayImpl ctor (TrafficGauges)`",
|
||||
"description": "###### What this is:\n*Squelch relay-control messages in/out, plus messages suppressed by squelch and squelch directives that were ignored. Squelch reduces redundant message forwarding between peers.*\n\n###### How it's computed:\n*Current message counts for the squelch, squelch-suppressed, and squelch-ignored categories, in and out.*\n\n###### Reading it:\n*High suppressed counts mean squelch is saving bandwidth; ignored should stay low.*\n\n###### Healthy range:\n*workload-dependent; suppressed far above ignored.*\n\n###### Watch for:\n*High ignored counts (peers not honoring squelch) or squelch traffic itself dominating.*\n\n###### Source:\n[OverlayImpl.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/overlay/detail/OverlayImpl.cpp)\n\n###### Function:\n`OverlayImpl ctor (TrafficGauges)`",
|
||||
"type": "timeseries",
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
@@ -31,7 +31,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(squelch_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "squelch_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Squelch In [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -39,7 +39,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(squelch_messages_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "squelch_messages_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Squelch Out [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -47,7 +47,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(squelch_suppressed_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "squelch_suppressed_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Suppressed In [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -55,7 +55,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(squelch_suppressed_messages_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "squelch_suppressed_messages_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Suppressed Out [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -63,7 +63,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(squelch_ignored_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "squelch_ignored_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Ignored In [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -71,13 +71,13 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(squelch_ignored_messages_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "squelch_ignored_messages_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Ignored Out [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "cps",
|
||||
"unit": "short",
|
||||
"custom": {
|
||||
"axisLabel": "Messages",
|
||||
"spanNulls": true,
|
||||
@@ -91,7 +91,7 @@
|
||||
},
|
||||
{
|
||||
"title": "Overhead Traffic Breakdown (Bytes)",
|
||||
"description": "###### What this is:\n*Overlay protocol overhead bytes split into base overhead, intra-cluster overhead, and validator-manifest distribution overhead.*\n\n###### How it's computed:\n*Per-second in/out byte rate for the overhead, overhead-cluster, and overhead-manifest categories.*\n\n###### Reading it:\n*Base overhead is routine; cluster and manifest rise around cluster syncs and manifest changes.*\n\n###### Healthy range:\n*workload-dependent; low and stable.*\n\n###### Watch for:\n*Sustained high cluster or manifest overhead (frequent cluster state churn or manifest reissue).*\n\n###### Source:\n[OverlayImpl.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/overlay/detail/OverlayImpl.cpp)\n\n###### Function:\n`OverlayImpl ctor (TrafficGauges)`",
|
||||
"description": "###### What this is:\n*Overlay protocol overhead bytes split into base overhead, intra-cluster overhead, and validator-manifest distribution overhead.*\n\n###### How it's computed:\n*Current in/out byte counts for the overhead, overhead-cluster, and overhead-manifest categories.*\n\n###### Reading it:\n*Base overhead is routine; cluster and manifest rise around cluster syncs and manifest changes.*\n\n###### Healthy range:\n*workload-dependent; low and stable.*\n\n###### Watch for:\n*Sustained high cluster or manifest overhead (frequent cluster state churn or manifest reissue).*\n\n###### Source:\n[OverlayImpl.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/overlay/detail/OverlayImpl.cpp)\n\n###### Function:\n`OverlayImpl ctor (TrafficGauges)`",
|
||||
"type": "timeseries",
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
@@ -111,7 +111,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(overhead_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "overhead_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Base Overhead In [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -119,7 +119,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(overhead_bytes_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "overhead_bytes_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Base Overhead Out [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -127,7 +127,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(overhead_cluster_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "overhead_cluster_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Cluster In [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -135,7 +135,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(overhead_cluster_bytes_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "overhead_cluster_bytes_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Cluster Out [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -143,7 +143,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(overhead_manifest_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "overhead_manifest_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Manifest In [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -151,13 +151,13 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(overhead_manifest_bytes_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "overhead_manifest_bytes_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Manifest Out [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "Bps",
|
||||
"unit": "decbytes",
|
||||
"custom": {
|
||||
"axisLabel": "Bytes",
|
||||
"spanNulls": true,
|
||||
@@ -171,7 +171,7 @@
|
||||
},
|
||||
{
|
||||
"title": "Validator List Traffic",
|
||||
"description": "###### What this is:\n*Bytes and messages exchanged distributing validator lists (trusted-list configuration) between peers.*\n\n###### How it's computed:\n*Per-second in/out byte and message rate for the validator-lists category.*\n\n###### Reading it:\n*Bursts when lists update or new peers connect; quiet otherwise.*\n\n###### Healthy range:\n*workload-dependent; occasional bursts.*\n\n###### Watch for:\n*Continuous high volume (repeated list re-fetching or churn).*\n\n###### Source:\n[OverlayImpl.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/overlay/detail/OverlayImpl.cpp)\n\n###### Function:\n`OverlayImpl ctor (TrafficGauges)`",
|
||||
"description": "###### What this is:\n*Bytes and messages exchanged distributing validator lists (trusted-list configuration) between peers.*\n\n###### How it's computed:\n*Current in/out byte and message counts for the validator-lists category.*\n\n###### Reading it:\n*Bursts when lists update or new peers connect; quiet otherwise.*\n\n###### Healthy range:\n*workload-dependent; occasional bursts.*\n\n###### Watch for:\n*Continuous high volume (repeated list re-fetching or churn).*\n\n###### Source:\n[OverlayImpl.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/overlay/detail/OverlayImpl.cpp)\n\n###### Function:\n`OverlayImpl ctor (TrafficGauges)`",
|
||||
"type": "timeseries",
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
@@ -191,7 +191,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(validator_lists_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "validator_lists_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Bytes In [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -199,7 +199,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(validator_lists_bytes_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "validator_lists_bytes_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Bytes Out [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -207,7 +207,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(validator_lists_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "validator_lists_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Messages In [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -215,13 +215,13 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(validator_lists_messages_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "validator_lists_messages_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Messages Out [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "cps",
|
||||
"unit": "short",
|
||||
"custom": {
|
||||
"axisLabel": "Count",
|
||||
"spanNulls": true,
|
||||
@@ -243,7 +243,7 @@
|
||||
},
|
||||
{
|
||||
"id": "unit",
|
||||
"value": "binBps"
|
||||
"value": "decbytes"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -252,7 +252,7 @@
|
||||
},
|
||||
{
|
||||
"title": "Set Get/Share Traffic (Bytes)",
|
||||
"description": "###### What this is:\n*Transaction-set fetch (get) and share bytes exchanged during ledger close.*\n\n###### How it's computed:\n*Per-second in/out byte rate for the set-get and set-share categories.*\n\n###### Reading it:\n*Some exchange each ledger is normal as peers reconcile transaction sets.*\n\n###### Healthy range:\n*workload-dependent.*\n\n###### Watch for:\n*High set-get (peers frequently missing transaction sets: possible sync delays).*\n\n###### Source:\n[OverlayImpl.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/overlay/detail/OverlayImpl.cpp)\n\n###### Function:\n`OverlayImpl ctor (TrafficGauges)`",
|
||||
"description": "###### What this is:\n*Transaction-set fetch (get) and share bytes exchanged during ledger close.*\n\n###### How it's computed:\n*Current in/out byte counts for the set-get and set-share categories.*\n\n###### Reading it:\n*Some exchange each ledger is normal as peers reconcile transaction sets.*\n\n###### Healthy range:\n*workload-dependent.*\n\n###### Watch for:\n*High set-get (peers frequently missing transaction sets: possible sync delays).*\n\n###### Source:\n[OverlayImpl.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/overlay/detail/OverlayImpl.cpp)\n\n###### Function:\n`OverlayImpl ctor (TrafficGauges)`",
|
||||
"type": "timeseries",
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
@@ -272,7 +272,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(set_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "set_get_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Set Get In [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -280,7 +280,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(set_get_bytes_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "set_get_bytes_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Set Get Out [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -288,7 +288,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(set_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "set_share_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Set Share In [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -296,13 +296,13 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(set_share_bytes_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "set_share_bytes_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Set Share Out [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "Bps",
|
||||
"unit": "decbytes",
|
||||
"custom": {
|
||||
"axisLabel": "Bytes",
|
||||
"spanNulls": true,
|
||||
@@ -316,7 +316,7 @@
|
||||
},
|
||||
{
|
||||
"title": "Have/Requested Transactions (Messages)",
|
||||
"description": "###### What this is:\n*Transaction-availability messages: advertisements that a peer has certain transactions, and explicit requests for transaction data.*\n\n###### How it's computed:\n*Per-second in/out message rate for the have-transactions and requested-transactions categories.*\n\n###### Reading it:\n*Compare requested versus have to gauge how well transactions are propagating.*\n\n###### Healthy range:\n*workload-dependent.*\n\n###### Watch for:\n*Requested far exceeding have (peers behind on transaction propagation).*\n\n###### Source:\n[OverlayImpl.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/overlay/detail/OverlayImpl.cpp)\n\n###### Function:\n`OverlayImpl ctor (TrafficGauges)`",
|
||||
"description": "###### What this is:\n*Transaction-availability messages: advertisements that a peer has certain transactions, and explicit requests for transaction data.*\n\n###### How it's computed:\n*Current in/out message counts for the have-transactions and requested-transactions categories.*\n\n###### Reading it:\n*Compare requested versus have to gauge how well transactions are propagating.*\n\n###### Healthy range:\n*workload-dependent.*\n\n###### Watch for:\n*Requested far exceeding have (peers behind on transaction propagation).*\n\n###### Source:\n[OverlayImpl.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/overlay/detail/OverlayImpl.cpp)\n\n###### Function:\n`OverlayImpl ctor (TrafficGauges)`",
|
||||
"type": "timeseries",
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
@@ -336,7 +336,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(have_transactions_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "have_transactions_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Have TX In [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -344,7 +344,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(have_transactions_messages_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "have_transactions_messages_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Have TX Out [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -352,7 +352,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(requested_transactions_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "requested_transactions_messages_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Requested TX In [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -360,13 +360,13 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(requested_transactions_messages_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "requested_transactions_messages_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Requested TX Out [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "cps",
|
||||
"unit": "short",
|
||||
"custom": {
|
||||
"axisLabel": "Messages",
|
||||
"spanNulls": true,
|
||||
@@ -461,7 +461,7 @@
|
||||
},
|
||||
{
|
||||
"title": "Proof Path Traffic",
|
||||
"description": "###### What this is:\n*Proof-path request/response bytes used to verify individual ledger entries without downloading the whole ledger.*\n\n###### How it's computed:\n*Per-second in/out byte rate for the proof-path request and response categories.*\n\n###### Reading it:\n*Rises when peers verify specific state, often during catch-up.*\n\n###### Healthy range:\n*workload-dependent.*\n\n###### Watch for:\n*High sustained request volume (heavy state-verification load).*\n\n###### Source:\n[OverlayImpl.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/overlay/detail/OverlayImpl.cpp)\n\n###### Function:\n`OverlayImpl ctor (TrafficGauges)`",
|
||||
"description": "###### What this is:\n*Proof-path request/response bytes used to verify individual ledger entries without downloading the whole ledger.*\n\n###### How it's computed:\n*Current in/out byte counts for the proof-path request and response categories.*\n\n###### Reading it:\n*Rises when peers verify specific state, often during catch-up.*\n\n###### Healthy range:\n*workload-dependent.*\n\n###### Watch for:\n*High sustained request volume (heavy state-verification load).*\n\n###### Source:\n[OverlayImpl.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/overlay/detail/OverlayImpl.cpp)\n\n###### Function:\n`OverlayImpl ctor (TrafficGauges)`",
|
||||
"type": "timeseries",
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
@@ -481,7 +481,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(proof_path_request_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "proof_path_request_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Request Bytes In [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -489,7 +489,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(proof_path_request_bytes_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "proof_path_request_bytes_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Request Bytes Out [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -497,7 +497,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(proof_path_response_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "proof_path_response_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Response Bytes In [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -505,13 +505,13 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(proof_path_response_bytes_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "proof_path_response_bytes_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Response Bytes Out [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "Bps",
|
||||
"unit": "decbytes",
|
||||
"custom": {
|
||||
"axisLabel": "Bytes",
|
||||
"spanNulls": true,
|
||||
@@ -525,7 +525,7 @@
|
||||
},
|
||||
{
|
||||
"title": "Replay Delta Traffic",
|
||||
"description": "###### What this is:\n*Replay-delta request/response bytes used to efficiently replay ledger state changes during catch-up.*\n\n###### How it's computed:\n*Per-second in/out byte rate for the replay-delta request and response categories.*\n\n###### Reading it:\n*Active during catch-up and replay; quiet when synced.*\n\n###### Healthy range:\n*workload-dependent; low when synced.*\n\n###### Watch for:\n*Continuous replay traffic (node repeatedly replaying rather than staying current).*\n\n###### Source:\n[OverlayImpl.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/overlay/detail/OverlayImpl.cpp)\n\n###### Function:\n`OverlayImpl ctor (TrafficGauges)`",
|
||||
"description": "###### What this is:\n*Replay-delta request/response bytes used to efficiently replay ledger state changes during catch-up.*\n\n###### How it's computed:\n*Current in/out byte counts for the replay-delta request and response categories.*\n\n###### Reading it:\n*Active during catch-up and replay; quiet when synced.*\n\n###### Healthy range:\n*workload-dependent; low when synced.*\n\n###### Watch for:\n*Continuous replay traffic (node repeatedly replaying rather than staying current).*\n\n###### Source:\n[OverlayImpl.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/overlay/detail/OverlayImpl.cpp)\n\n###### Function:\n`OverlayImpl ctor (TrafficGauges)`",
|
||||
"type": "timeseries",
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
@@ -545,7 +545,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(replay_delta_request_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "replay_delta_request_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Request Bytes In [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -553,7 +553,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(replay_delta_request_bytes_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "replay_delta_request_bytes_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Request Bytes Out [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -561,7 +561,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(replay_delta_response_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "replay_delta_response_bytes_in{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Response Bytes In [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -569,13 +569,13 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(replay_delta_response_bytes_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "replay_delta_response_bytes_out{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}",
|
||||
"legendFormat": "Response Bytes Out [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "Bps",
|
||||
"unit": "decbytes",
|
||||
"custom": {
|
||||
"axisLabel": "Bytes",
|
||||
"spanNulls": true,
|
||||
@@ -670,7 +670,7 @@
|
||||
"label": "Node",
|
||||
"description": "Filter by rippled node (service.instance.id)",
|
||||
"type": "query",
|
||||
"query": "label_values(target_info, service_instance_id)",
|
||||
"query": "label_values(squelch_messages_in, service_instance_id)",
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
|
||||
@@ -31,7 +31,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"peer.proposal.receive\"}[$__rate_interval]))",
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"peer.proposal.receive\"}[5m]))",
|
||||
"legendFormat": "Proposals Received / Sec [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -71,7 +71,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"peer.validation.receive\"}[$__rate_interval]))",
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"peer.validation.receive\"}[5m]))",
|
||||
"legendFormat": "Validations Received / Sec [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -111,7 +111,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (proposal_trusted, service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", proposal_trusted=~\"$proposal_trusted\", span_name=\"peer.proposal.receive\"}[$__rate_interval]))",
|
||||
"expr": "sum by (proposal_trusted, service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", proposal_trusted=~\"$proposal_trusted\", span_name=\"peer.proposal.receive\"}[5m]))",
|
||||
"legendFormat": "Trusted = {{proposal_trusted}} [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -144,7 +144,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (validation_trusted, service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", validation_trusted=~\"$validation_trusted\", span_name=\"peer.validation.receive\"}[$__rate_interval]))",
|
||||
"expr": "sum by (validation_trusted, service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", validation_trusted=~\"$validation_trusted\", span_name=\"peer.validation.receive\"}[5m]))",
|
||||
"legendFormat": "Trusted = {{validation_trusted}} [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -334,7 +334,7 @@
|
||||
"label": "Node",
|
||||
"description": "Filter by rippled node (service.instance.id — e.g. Node-1)",
|
||||
"type": "query",
|
||||
"query": "label_values(target_info, service_instance_id)",
|
||||
"query": "label_values(span_calls_total, service_instance_id)",
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
|
||||
@@ -451,7 +451,7 @@
|
||||
"label": "Node",
|
||||
"description": "Filter by rippled node (service.instance.id)",
|
||||
"type": "query",
|
||||
"query": "label_values(target_info, service_instance_id)",
|
||||
"query": "label_values(peer_quality, service_instance_id)",
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
|
||||
@@ -31,7 +31,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(rpc_requests_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "rate(rpc_requests_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])",
|
||||
"legendFormat": "Requests / Sec [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -84,7 +84,7 @@
|
||||
},
|
||||
{
|
||||
"title": "RPC Response Size",
|
||||
"description": "⚠ Instrument mismatch — values unreliable. Response size is recorded through the millisecond-scaled event histogram (rpc_size_milliseconds_bucket), so byte values saturate at the top time bucket (5000) and the percentiles are not true byte sizes. A dedicated byte-unit histogram is needed to fix this; tracked separately. Treat this panel as indicative only until then.\n\n###### What this is:\n*The 95th-percentile size of RPC response payloads in bytes.*\n\n###### How it's computed:\n*95th-percentile of response payload sizes over the dashboard rate interval, per node.*\n\n###### Reading it:\n*Smaller is cheaper; large responses cost bandwidth and memory.*\n\n###### Healthy range:\n*Workload-dependent; small for status queries, large for bulk data queries.*\n\n###### Watch for:\n*Growth in large responses, consistent with expensive queries or API misuse.*\n\n###### Source:\n[ServerHandler.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/rpc/detail/ServerHandler.cpp)\n\n###### Function:\n`ServerHandler ctor`",
|
||||
"description": "###### What this is:\n*The 95th-percentile size of RPC response payloads in bytes.*\n\n###### How it's computed:\n*95th-percentile of response payload sizes over 5 minutes, per node.*\n\n###### Reading it:\n*Smaller is cheaper; large responses cost bandwidth and memory.*\n\n###### Healthy range:\n*Workload-dependent; small for status queries, large for bulk data queries.*\n\n###### Watch for:\n*Growth in large responses, consistent with expensive queries or API misuse.*\n\n###### Source:\n[ServerHandler.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/rpc/detail/ServerHandler.cpp)\n\n###### Function:\n`ServerHandler ctor`",
|
||||
"type": "timeseries",
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
@@ -280,7 +280,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(warn_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "rate(warn_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])",
|
||||
"legendFormat": "Warnings / Sec [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -329,7 +329,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(drop_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval])",
|
||||
"expr": "rate(drop_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m])",
|
||||
"legendFormat": "Drops / Sec [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -378,7 +378,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (method, service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", method=~\"$grpc_method\", span_name=~\"grpc\\\\..*\"}[$__rate_interval]))",
|
||||
"expr": "sum by (method, service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", method=~\"$grpc_method\", span_name=~\"grpc\\\\..*\"}[5m]))",
|
||||
"legendFormat": "{{method}} [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -458,7 +458,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (grpc_status, service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=~\"grpc\\\\..*\", grpc_status!=\"\"}[$__rate_interval]))",
|
||||
"expr": "sum by (grpc_status, service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=~\"grpc\\\\..*\", grpc_status!=\"\"}[5m]))",
|
||||
"legendFormat": "{{grpc_status}} [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -538,7 +538,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"pathfind.request\"}[$__rate_interval]))",
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"pathfind.request\"}[5m]))",
|
||||
"legendFormat": "Requests / Sec [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -546,7 +546,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"pathfind.discover\"}[$__rate_interval]))",
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"pathfind.discover\"}[5m]))",
|
||||
"legendFormat": "Discoveries / Sec [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -647,7 +647,7 @@
|
||||
"label": "Node",
|
||||
"description": "Filter by rippled node (service.instance.id)",
|
||||
"type": "query",
|
||||
"query": "label_values(target_info, service_instance_id)",
|
||||
"query": "label_values(rpc_requests_total, service_instance_id)",
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
|
||||
@@ -43,7 +43,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (command, service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", command=~\"$command\", span_name=~\"rpc.command.*\"}[$__rate_interval]))",
|
||||
"expr": "sum by (command, service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", command=~\"$command\", span_name=~\"rpc.command.*\"}[5m]))",
|
||||
"legendFormat": "{{command}} [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -125,7 +125,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (command, service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", command=~\"$command\", span_name=~\"rpc.command.*\", status_code=\"STATUS_CODE_ERROR\"}[$__rate_interval])) / sum by (command, service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", command=~\"$command\", span_name=~\"rpc.command.*\"}[$__rate_interval])) * 100",
|
||||
"expr": "sum by (command, service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", command=~\"$command\", span_name=~\"rpc.command.*\", status_code=\"STATUS_CODE_ERROR\"}[5m])) / sum by (command, service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", command=~\"$command\", span_name=~\"rpc.command.*\"}[5m])) * 100",
|
||||
"legendFormat": "{{command}} [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -213,7 +213,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", command=~\"$command\", span_name=\"rpc.http_request\"}[$__rate_interval]))",
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", command=~\"$command\", span_name=\"rpc.http_request\"}[5m]))",
|
||||
"legendFormat": "rpc.http_request / Sec [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -221,7 +221,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", command=~\"$command\", span_name=\"rpc.process\"}[$__rate_interval]))",
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", command=~\"$command\", span_name=\"rpc.process\"}[5m]))",
|
||||
"legendFormat": "rpc.process / Sec [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -262,7 +262,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", command=~\"$command\", span_name=~\"rpc.command.*\", status_code=\"STATUS_CODE_OK\"}[$__rate_interval]))",
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", command=~\"$command\", span_name=~\"rpc.command.*\", status_code=\"STATUS_CODE_OK\"}[5m]))",
|
||||
"legendFormat": "Success [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -270,7 +270,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", command=~\"$command\", span_name=~\"rpc.command.*\", status_code=\"STATUS_CODE_ERROR\"}[$__rate_interval]))",
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", command=~\"$command\", span_name=~\"rpc.command.*\", status_code=\"STATUS_CODE_ERROR\"}[5m]))",
|
||||
"legendFormat": "Error [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -311,7 +311,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "topk(10, sum by (command, service_instance_id) (increase(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", command=~\"$command\", span_name=~\"rpc.command.*\"}[$__rate_interval])))",
|
||||
"expr": "topk(10, sum by (command, service_instance_id) (increase(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", command=~\"$command\", span_name=~\"rpc.command.*\"}[5m])))",
|
||||
"legendFormat": "{{command}} [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -345,7 +345,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", command=~\"$command\", span_name=\"rpc.ws_message\"}[$__rate_interval]))",
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", command=~\"$command\", span_name=\"rpc.ws_message\"}[5m]))",
|
||||
"legendFormat": "WS Messages / Sec [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -384,7 +384,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (load_type) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=~\"rpc.command.*\", load_type!=\"\"}[$__rate_interval]))",
|
||||
"expr": "sum by (load_type) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=~\"rpc.command.*\", load_type!=\"\"}[5m]))",
|
||||
"legendFormat": "{{load_type}}"
|
||||
}
|
||||
],
|
||||
@@ -425,7 +425,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"rpc.process\", is_batch=\"true\"}[$__rate_interval]))",
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"rpc.process\", is_batch=\"true\"}[5m]))",
|
||||
"legendFormat": "Batch [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -433,7 +433,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"rpc.process\", is_batch=\"false\"}[$__rate_interval]))",
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"rpc.process\", is_batch=\"false\"}[5m]))",
|
||||
"legendFormat": "Single [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -487,7 +487,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (service_instance_id) (rate(rpc_method_started_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", method=~\"$method\"}[$__rate_interval]))",
|
||||
"expr": "sum by (service_instance_id) (rate(rpc_method_started_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", method=~\"$method\"}[5m]))",
|
||||
"legendFormat": "Started/s [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -495,7 +495,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (service_instance_id) (rate(rpc_method_finished_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", method=~\"$method\"}[$__rate_interval]))",
|
||||
"expr": "sum by (service_instance_id) (rate(rpc_method_finished_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", method=~\"$method\"}[5m]))",
|
||||
"legendFormat": "Finished/s [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -503,7 +503,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (service_instance_id) (rate(rpc_method_errored_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", method=~\"$method\"}[$__rate_interval]))",
|
||||
"expr": "sum by (service_instance_id) (rate(rpc_method_errored_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", method=~\"$method\"}[5m]))",
|
||||
"legendFormat": "Errored/s [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -555,7 +555,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "topk(10, rate(rpc_method_started_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", method=~\"$method\"}[$__rate_interval]))",
|
||||
"expr": "topk(10, rate(rpc_method_started_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", method=~\"$method\"}[5m]))",
|
||||
"legendFormat": "{{method}} [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -607,7 +607,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "topk(10, rate(rpc_method_errored_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", method=~\"$method\"}[$__rate_interval]))",
|
||||
"expr": "topk(10, rate(rpc_method_errored_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", method=~\"$method\"}[5m]))",
|
||||
"legendFormat": "{{method}} [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -766,7 +766,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "topk(10, rate(rpc_method_errored_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", method=~\"$method\"}[$__rate_interval]) / (rate(rpc_method_started_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", method=~\"$method\"}[$__rate_interval]) > 0))",
|
||||
"expr": "topk(10, rate(rpc_method_errored_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", method=~\"$method\"}[5m]) / (rate(rpc_method_started_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", method=~\"$method\"}[5m]) > 0))",
|
||||
"legendFormat": "{{method}} [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -948,7 +948,7 @@
|
||||
"label": "Node",
|
||||
"description": "Filter by rippled node (service.instance.id — e.g. Node-1)",
|
||||
"type": "query",
|
||||
"query": "label_values(target_info, service_instance_id)",
|
||||
"query": "label_values(span_calls_total, service_instance_id)",
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
|
||||
@@ -10,7 +10,7 @@
|
||||
"panels": [
|
||||
{
|
||||
"title": "Transaction Apply Failed Rate",
|
||||
"description": "###### What this is:\n*Transactions that fail during the apply (transactor) stage, per second — i.e. the transactor ran but the result was not tesSUCCESS.*\n\n###### How it's computed:\n*Per-second rate of tx.transactor spans at the apply stage whose ter_result is not tesSUCCESS, over the dashboard rate interval, per node.*\n\n###### Reading it:\n*A steady low background is normal (tefPAST_SEQ, tecUNFUNDED, etc.); the transactor legitimately rejects many transactions.*\n\n###### Healthy range:\n*A low, stable rate; workload-dependent.*\n\n###### Watch for:\n*A sharp rise above baseline, which points to a submission pattern that is systematically failing at apply.*\n\n###### Source:\n[Transactor.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/app/tx/detail/Transactor.cpp)\n\n###### Function:\n`Transactor::operator()`",
|
||||
"description": "###### What this is:\n*Transactions flagged with an error status during application into a ledger, per second.*\n\n###### How it's computed:\n*Per-second rate of transaction-apply operations that carry an error status, over 5 minutes, per node.*\n\n###### Reading it:\n*This view stays near zero: the count of failed transactions is recorded separately as a per-ledger attribute rather than as an apply-phase error status, so apply operations rarely report an error here.*\n\n###### Healthy range:\n*Effectively zero under normal operation, since apply-phase errors are seldom emitted.*\n\n###### Watch for:\n*Any sustained non-zero rate, which would flag an unusual apply-phase error rather than ordinary transaction failures.*\n\n###### Source:\n[BuildLedger.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/app/ledger/detail/BuildLedger.cpp)\n\n###### Function:\n`applyTransactions`",
|
||||
"type": "stat",
|
||||
"gridPos": {
|
||||
"h": 4,
|
||||
@@ -31,7 +31,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"tx.transactor\", stage=\"apply\", ter_result!~\"tesSUCCESS|\"}[$__rate_interval]))",
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"tx.apply\", status_code=\"STATUS_CODE_ERROR\"}[5m]))",
|
||||
"legendFormat": "Failed / Sec [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -46,11 +46,11 @@
|
||||
},
|
||||
{
|
||||
"color": "yellow",
|
||||
"value": 1
|
||||
"value": 0.1
|
||||
},
|
||||
{
|
||||
"color": "red",
|
||||
"value": 5
|
||||
"value": 1
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -134,7 +134,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (tx_type, service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"tx.process\", tx_type=~\"$tx_type\"}[$__rate_interval]))",
|
||||
"expr": "sum by (tx_type, service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"tx.process\", tx_type=~\"$tx_type\"}[5m]))",
|
||||
"legendFormat": "{{tx_type}} [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -181,7 +181,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (tx_type, ter_result, service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"tx.process\", tx_type=~\"$tx_type\", ter_result=~\"$ter_result\", ter_result!=\"tesSUCCESS\"}[$__rate_interval]))",
|
||||
"expr": "sum by (tx_type, ter_result, service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"tx.process\", tx_type=~\"$tx_type\", ter_result=~\"$ter_result\", ter_result!=\"tesSUCCESS\"}[5m]))",
|
||||
"legendFormat": "{{tx_type}} [{{ter_result}}, {{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -244,7 +244,7 @@
|
||||
},
|
||||
{
|
||||
"title": "Transaction Processing Rate",
|
||||
"description": "###### What this is:\n*Transactions entering the processing pipeline versus raw transactions arriving from peers.*\n\n###### How it's computed:\n*Per-second rates of processed transactions and received peer transactions over 5 minutes, per node.*\n\n###### Reading it:\n*Received sits above processed; the difference is deduplicated relay traffic.*\n\n###### Healthy range:\n*Workload-dependent; both scale with network volume.*\n\n###### Watch for:\n*A receive rate spiking well above processing, consistent with relay flooding.*\n\n###### Source:\n[NetworkOPs.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/app/misc/NetworkOPs.cpp) \u00b7 [PeerImp.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/overlay/detail/PeerImp.cpp)\n\n###### Function:\n`NetworkOPsImp::processTransaction ; PeerImp::handleTransaction`",
|
||||
"description": "###### What this is:\n*Transactions entering the processing pipeline versus raw transactions arriving from peers.*\n\n###### How it's computed:\n*Per-second rates of processed transactions and received peer transactions over 5 minutes, per node.*\n\n###### Reading it:\n*Received sits above processed; the difference is deduplicated relay traffic.*\n\n###### Healthy range:\n*Workload-dependent; both scale with network volume.*\n\n###### Watch for:\n*A receive rate spiking well above processing, consistent with relay flooding.*\n\n###### Source:\n[NetworkOPs.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/app/misc/NetworkOPs.cpp) · [PeerImp.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/overlay/detail/PeerImp.cpp)\n\n###### Function:\n`NetworkOPsImp::processTransaction ; PeerImp::handleTransaction`",
|
||||
"type": "timeseries",
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
@@ -265,7 +265,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"tx.process\", tx_type=~\"$tx_type\"}[$__rate_interval]))",
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"tx.process\", tx_type=~\"$tx_type\"}[5m]))",
|
||||
"legendFormat": "tx.process / Sec [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -273,7 +273,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"tx.receive\", tx_type=~\"$tx_type\"}[$__rate_interval]))",
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"tx.receive\", tx_type=~\"$tx_type\"}[5m]))",
|
||||
"legendFormat": "tx.receive / Sec [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -315,7 +315,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (local, service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", local=~\"$tx_origin\", span_name=\"tx.process\", tx_type=~\"$tx_type\"}[$__rate_interval]))",
|
||||
"expr": "sum by (local, service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", local=~\"$tx_origin\", span_name=\"tx.process\", tx_type=~\"$tx_type\"}[5m]))",
|
||||
"legendFormat": "Local [{{local}}, {{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -360,6 +360,48 @@
|
||||
},
|
||||
"id": 8
|
||||
},
|
||||
{
|
||||
"title": "Peer Transaction Receive Rate",
|
||||
"description": "###### What this is:\n*How many transaction messages arrive from network peers per second.*\n\n###### How it's computed:\n*Per-second rate of received peer transactions over 5 minutes, per node.*\n\n###### Reading it:\n*Reflects network-wide transaction volume reaching this node.*\n\n###### Healthy range:\n*Workload-dependent; tracks overall network activity.*\n\n###### Watch for:\n*A sharp spike above baseline, consistent with network-wide volume surges or peer flooding.*\n\n###### Source:\n[PeerImp.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/overlay/detail/PeerImp.cpp)\n\n###### Function:\n`PeerImp::handleTransaction`",
|
||||
"type": "timeseries",
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
"w": 12,
|
||||
"x": 0,
|
||||
"y": 44
|
||||
},
|
||||
"options": {
|
||||
"tooltip": {
|
||||
"mode": "multi",
|
||||
"sort": "desc",
|
||||
"maxHeight": 500
|
||||
}
|
||||
},
|
||||
"targets": [
|
||||
{
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"tx.receive\", tx_type=~\"$tx_type\"}[5m]))",
|
||||
"legendFormat": "tx.receive / Sec [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "ops",
|
||||
"custom": {
|
||||
"axisLabel": "Transactions / Sec",
|
||||
"spanNulls": true,
|
||||
"insertNulls": false,
|
||||
"showPoints": "auto",
|
||||
"pointSize": 3
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
},
|
||||
"id": 9
|
||||
},
|
||||
{
|
||||
"title": "Transactor Duration by Type (p95)",
|
||||
"description": "###### What this is:\n*The 95th-percentile execution time of the core transactor step, by transaction type.*\n\n###### How it's computed:\n*95th-percentile of transactor durations over 5 minutes, grouped by type and node.*\n\n###### Reading it:\n*Lower is better; reveals which types are most expensive to execute.*\n\n###### Healthy range:\n*Sub-millisecond to a few ms for most types; workload-dependent.*\n\n###### Watch for:\n*A type whose execution time grows, indicating expensive logic or ledger contention.*\n\n###### Source:\n[Transactor.cpp](https://github.com/XRPLF/rippled/blob/develop/src/libxrpl/tx/Transactor.cpp)\n\n###### Function:\n`Transactor::operator()`",
|
||||
@@ -408,9 +450,9 @@
|
||||
"id": 10
|
||||
},
|
||||
{
|
||||
"title": "TxQ Accept: Applied Ratio per Node (State Timeline)",
|
||||
"description": "###### What this is:\n*The applied fraction of TxQ accepts per node over time: the share of queued transactions that were included in a ledger versus removed on failure.*\n\n###### How it's computed:\n*Per node, applied accepts divided by applied-plus-failed accepts over the window.*\n\n###### Reading it:\n*Green (>=90% applied) is a healthy drain; yellow is degraded; red means accepts are mostly failing.*\n\n###### Healthy range:\n*At or near 100% applied when the queue is draining healthily; workload-dependent.*\n\n###### Watch for:\n*A node dropping into yellow or red, which signals queue pressure, under-bidding, or fee escalation on that node.*\n\n###### Source:\n[TxQ.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/app/misc/detail/TxQ.cpp)\n\n###### Function:\n`TxQ::accept`",
|
||||
"type": "state-timeline",
|
||||
"title": "Tx Apply Pipeline Rate by Stage",
|
||||
"description": "###### What this is:\n*How many transactions pass through each apply-pipeline stage (preflight, preclaim, apply) per second.*\n\n###### How it's computed:\n*Per-second rate over 5 minutes, grouped by pipeline stage and node.*\n\n###### Reading it:\n*Stages should step down slightly as invalid transactions are filtered out.*\n\n###### Healthy range:\n*Workload-dependent; preflight highest, apply lowest.*\n\n###### Watch for:\n*A large drop between stages, showing where most transactions are being rejected.*\n\n###### Source:\n[applySteps.cpp](https://github.com/XRPLF/rippled/blob/develop/src/libxrpl/tx/applySteps.cpp) · [Transactor.cpp](https://github.com/XRPLF/rippled/blob/develop/src/libxrpl/tx/Transactor.cpp)\n\n###### Function:\n`makeStageSpan ; Transactor::operator()`",
|
||||
"type": "timeseries",
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
"w": 24,
|
||||
@@ -418,22 +460,15 @@
|
||||
"y": 60
|
||||
},
|
||||
"options": {
|
||||
"mergeValues": true,
|
||||
"showValue": "auto",
|
||||
"alignValue": "center",
|
||||
"rowHeight": 0.85,
|
||||
"tooltip": {
|
||||
"mode": "multi",
|
||||
"sort": "desc",
|
||||
"maxHeight": 500
|
||||
},
|
||||
"legend": {
|
||||
"displayMode": "list",
|
||||
"placement": "bottom",
|
||||
"showLegend": true
|
||||
},
|
||||
"tooltip": {
|
||||
"mode": "single"
|
||||
"displayMode": "table",
|
||||
"placement": "right",
|
||||
"calcs": ["mean", "max"]
|
||||
}
|
||||
},
|
||||
"targets": [
|
||||
@@ -442,30 +477,19 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (service_instance_id) (increase(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"txq.accept_tx\", txq_status=\"applied\"}[$__rate_interval]))\n/\nsum by (service_instance_id) (increase(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"txq.accept_tx\", txq_status=~\"applied|failed\"}[$__rate_interval]))",
|
||||
"interval": "15s",
|
||||
"legendFormat": "{{service_instance_id}}"
|
||||
"expr": "label_replace(label_replace(label_replace(sum by (stage, service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=~\"tx.preflight|tx.preclaim|tx.transactor\", stage=~\"$stage\"}[5m])), \"stage\", \"Preflight\", \"stage\", \"preflight\"), \"stage\", \"Preclaim\", \"stage\", \"preclaim\"), \"stage\", \"Apply\", \"stage\", \"apply\")",
|
||||
"legendFormat": "{{stage}} [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "percentunit",
|
||||
"min": 0,
|
||||
"max": 1,
|
||||
"color": {
|
||||
"mode": "thresholds"
|
||||
},
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{ "color": "red", "value": 0 },
|
||||
{ "color": "yellow", "value": 0.7 },
|
||||
{ "color": "green", "value": 0.9 }
|
||||
]
|
||||
},
|
||||
"unit": "ops",
|
||||
"custom": {
|
||||
"fillOpacity": 80,
|
||||
"lineWidth": 0
|
||||
"axisLabel": "Spans / Sec",
|
||||
"spanNulls": true,
|
||||
"insertNulls": false,
|
||||
"showPoints": "auto",
|
||||
"pointSize": 3
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
@@ -474,7 +498,7 @@
|
||||
},
|
||||
{
|
||||
"title": "Tx Apply Pipeline Latency by Stage (p95)",
|
||||
"description": "###### What this is:\n*The 95th-percentile duration of each apply-pipeline stage.*\n\n###### How it's computed:\n*95th-percentile of per-stage durations over 5 minutes, grouped by stage and node.*\n\n###### Reading it:\n*Lower is better; isolates which stage dominates processing time.*\n\n###### Healthy range:\n*Each stage typically sub-millisecond to a few ms; workload-dependent.*\n\n###### Watch for:\n*One stage's latency rising, pinpointing the bottleneck in transaction handling.*\n\n###### Source:\n[applySteps.cpp](https://github.com/XRPLF/rippled/blob/develop/src/libxrpl/tx/applySteps.cpp) \u00b7 [Transactor.cpp](https://github.com/XRPLF/rippled/blob/develop/src/libxrpl/tx/Transactor.cpp)\n\n###### Function:\n`makeStageSpan ; Transactor::operator()`",
|
||||
"description": "###### What this is:\n*The 95th-percentile duration of each apply-pipeline stage.*\n\n###### How it's computed:\n*95th-percentile of per-stage durations over 5 minutes, grouped by stage and node.*\n\n###### Reading it:\n*Lower is better; isolates which stage dominates processing time.*\n\n###### Healthy range:\n*Each stage typically sub-millisecond to a few ms; workload-dependent.*\n\n###### Watch for:\n*One stage's latency rising, pinpointing the bottleneck in transaction handling.*\n\n###### Source:\n[applySteps.cpp](https://github.com/XRPLF/rippled/blob/develop/src/libxrpl/tx/applySteps.cpp) · [Transactor.cpp](https://github.com/XRPLF/rippled/blob/develop/src/libxrpl/tx/Transactor.cpp)\n\n###### Function:\n`makeStageSpan ; Transactor::operator()`",
|
||||
"type": "timeseries",
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
@@ -521,7 +545,7 @@
|
||||
},
|
||||
{
|
||||
"title": "Tx Apply Pipeline Failure Rate by Stage",
|
||||
"description": "###### What this is:\n*How many transactions fail at each apply-pipeline stage per second.*\n\n###### How it's computed:\n*Per-second rate of non-success outcomes over 5 minutes, grouped by stage and node.*\n\n###### Reading it:\n*Shows whether rejections concentrate in preflight, preclaim, or apply.*\n\n###### Healthy range:\n*Workload-dependent; a modest background of expected rejections is normal.*\n\n###### Watch for:\n*A failure spike concentrated in one stage, consistent with malformed or spam submissions.*\n\n###### Source:\n[applySteps.cpp](https://github.com/XRPLF/rippled/blob/develop/src/libxrpl/tx/applySteps.cpp) \u00b7 [Transactor.cpp](https://github.com/XRPLF/rippled/blob/develop/src/libxrpl/tx/Transactor.cpp)\n\n###### Function:\n`makeStageSpan ; Transactor::operator()`",
|
||||
"description": "###### What this is:\n*How many transactions fail at each apply-pipeline stage per second.*\n\n###### How it's computed:\n*Per-second rate of non-success outcomes over 5 minutes, grouped by stage and node.*\n\n###### Reading it:\n*Shows whether rejections concentrate in preflight, preclaim, or apply.*\n\n###### Healthy range:\n*Workload-dependent; a modest background of expected rejections is normal.*\n\n###### Watch for:\n*A failure spike concentrated in one stage, consistent with malformed or spam submissions.*\n\n###### Source:\n[applySteps.cpp](https://github.com/XRPLF/rippled/blob/develop/src/libxrpl/tx/applySteps.cpp) · [Transactor.cpp](https://github.com/XRPLF/rippled/blob/develop/src/libxrpl/tx/Transactor.cpp)\n\n###### Function:\n`makeStageSpan ; Transactor::operator()`",
|
||||
"type": "timeseries",
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
@@ -547,7 +571,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "label_replace(label_replace(label_replace(sum by (stage, service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=~\"tx.preflight|tx.preclaim|tx.transactor\", stage=~\"$stage\", ter_result!~\"tesSUCCESS|\"}[$__rate_interval])), \"stage\", \"Preflight\", \"stage\", \"preflight\"), \"stage\", \"Preclaim\", \"stage\", \"preclaim\"), \"stage\", \"Apply\", \"stage\", \"apply\")",
|
||||
"expr": "label_replace(label_replace(label_replace(sum by (stage, service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=~\"tx.preflight|tx.preclaim|tx.transactor\", stage=~\"$stage\", ter_result!~\"tesSUCCESS|\"}[5m])), \"stage\", \"Preflight\", \"stage\", \"preflight\"), \"stage\", \"Preclaim\", \"stage\", \"preclaim\"), \"stage\", \"Apply\", \"stage\", \"apply\")",
|
||||
"legendFormat": "{{stage}} [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -568,7 +592,7 @@
|
||||
},
|
||||
{
|
||||
"title": "Tx Apply Pipeline Latency by Type and Stage (p95)",
|
||||
"description": "###### What this is:\n*The 95th-percentile stage duration broken down by both transaction type and pipeline stage.*\n\n###### How it's computed:\n*95th-percentile durations over 5 minutes, grouped by type and stage; higher cardinality than the by-stage view.*\n\n###### Reading it:\n*Lower is better; shows which stage dominates each type's latency.*\n\n###### Healthy range:\n*Workload-dependent; most type/stage pairs sub-millisecond to a few ms.*\n\n###### Watch for:\n*A specific type/stage combination rising sharply, pinpointing a costly path.*\n\n###### Source:\n[applySteps.cpp](https://github.com/XRPLF/rippled/blob/develop/src/libxrpl/tx/applySteps.cpp) \u00b7 [Transactor.cpp](https://github.com/XRPLF/rippled/blob/develop/src/libxrpl/tx/Transactor.cpp)\n\n###### Function:\n`makeStageSpan ; Transactor::operator()`",
|
||||
"description": "###### What this is:\n*The 95th-percentile stage duration broken down by both transaction type and pipeline stage.*\n\n###### How it's computed:\n*95th-percentile durations over 5 minutes, grouped by type and stage; higher cardinality than the by-stage view.*\n\n###### Reading it:\n*Lower is better; shows which stage dominates each type's latency.*\n\n###### Healthy range:\n*Workload-dependent; most type/stage pairs sub-millisecond to a few ms.*\n\n###### Watch for:\n*A specific type/stage combination rising sharply, pinpointing a costly path.*\n\n###### Source:\n[applySteps.cpp](https://github.com/XRPLF/rippled/blob/develop/src/libxrpl/tx/applySteps.cpp) · [Transactor.cpp](https://github.com/XRPLF/rippled/blob/develop/src/libxrpl/tx/Transactor.cpp)\n\n###### Function:\n`makeStageSpan ; Transactor::operator()`",
|
||||
"type": "timeseries",
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
@@ -678,7 +702,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (tx_type, service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"txq.enqueue\"}[$__rate_interval]))",
|
||||
"expr": "sum by (tx_type, service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"txq.enqueue\"}[5m]))",
|
||||
"legendFormat": "{{tx_type}} [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -697,6 +721,45 @@
|
||||
},
|
||||
"id": 16
|
||||
},
|
||||
{
|
||||
"title": "TxQ Accept Status",
|
||||
"description": "###### What this is:\n*The outcome of queued transactions when the queue drains into a ledger: applied, failed, or retried.*\n\n###### How it's computed:\n*Counts of queue-accept outcomes over the window, split by status and shown as proportions.*\n\n###### Reading it:\n*A high applied share is healthy; retries indicate the transaction stayed queued.*\n\n###### Healthy range:\n*Workload-dependent; applied should dominate when not congested.*\n\n###### Watch for:\n*A rising failed or retried share, indicating queue pressure or under-bidding.*\n\n###### Source:\n[TxQ.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/app/misc/detail/TxQ.cpp)\n\n###### Function:\n`TxQ::accept`",
|
||||
"type": "piechart",
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
"w": 12,
|
||||
"x": 12,
|
||||
"y": 100
|
||||
},
|
||||
"options": {
|
||||
"legend": {
|
||||
"displayMode": "table",
|
||||
"placement": "right",
|
||||
"values": ["value", "percent"]
|
||||
},
|
||||
"tooltip": {
|
||||
"mode": "multi",
|
||||
"maxHeight": 500
|
||||
}
|
||||
},
|
||||
"targets": [
|
||||
{
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (txq_status, service_instance_id) (increase(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"txq.accept_tx\", txq_status=~\"$txq_status\"}[5m]))",
|
||||
"legendFormat": "{{txq_status}} [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"unit": "short"
|
||||
},
|
||||
"overrides": []
|
||||
},
|
||||
"id": 17
|
||||
},
|
||||
{
|
||||
"title": "Queue Bypass Ratio (Direct Apply vs Enqueue)",
|
||||
"description": "###### What this is:\n*The fraction of transactions that applied straight to the open ledger instead of being queued.*\n\n###### How it's computed:\n*Direct-apply rate divided by direct-apply plus enqueue rate over 5 minutes, per node.*\n\n###### Reading it:\n*Higher is better; near 1.0 means the network is not congested.*\n\n###### Healthy range:\n*Close to 1.0 when not congested; falls as demand exceeds capacity.*\n\n###### Watch for:\n*A falling ratio, the cleanest single signal the network has entered sustained fee escalation.*\n\n###### Source:\n[TxQ.cpp](https://github.com/XRPLF/rippled/blob/develop/src/xrpld/app/misc/detail/TxQ.cpp)\n\n###### Function:\n`TxQ::tryDirectApply ; TxQ::apply`",
|
||||
@@ -720,7 +783,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"txq.apply_direct\"}[$__rate_interval])) / clamp_min(sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"txq.apply_direct\"}[$__rate_interval])) + sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"txq.enqueue\"}[$__rate_interval])), 1)",
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"txq.apply_direct\"}[5m])) / clamp_min(sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"txq.apply_direct\"}[5m])) + sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"txq.enqueue\"}[5m])), 1)",
|
||||
"legendFormat": "Direct-Apply Fraction [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -804,7 +867,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"txq.cleanup\"}[$__rate_interval]))",
|
||||
"expr": "sum by (service_instance_id) (rate(span_calls_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\", span_name=\"txq.cleanup\"}[5m]))",
|
||||
"legendFormat": "Cleanups / Sec [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -904,9 +967,9 @@
|
||||
{
|
||||
"name": "node",
|
||||
"label": "Node",
|
||||
"description": "Filter by rippled node (service.instance.id \u2014 e.g. Node-1)",
|
||||
"description": "Filter by rippled node (service.instance.id — e.g. Node-1)",
|
||||
"type": "query",
|
||||
"query": "label_values(target_info, service_instance_id)",
|
||||
"query": "label_values(span_calls_total, service_instance_id)",
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
@@ -979,6 +1042,25 @@
|
||||
"sort": 1,
|
||||
"label": "Result Code"
|
||||
},
|
||||
{
|
||||
"name": "txq_status",
|
||||
"type": "query",
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"query": "label_values(span_calls_total{span_name=\"txq.accept_tx\", txq_status!=\"\"}, txq_status)",
|
||||
"refresh": 2,
|
||||
"includeAll": true,
|
||||
"multi": true,
|
||||
"allValue": ".*",
|
||||
"current": {
|
||||
"text": "All",
|
||||
"value": "$__all"
|
||||
},
|
||||
"sort": 1,
|
||||
"label": "Queue Status"
|
||||
},
|
||||
{
|
||||
"name": "stage",
|
||||
"type": "query",
|
||||
|
||||
@@ -1,127 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Dashboard lint: cumulative metrics must be rate()-wrapped; tier filters present."""
|
||||
|
||||
import json, re, sys
|
||||
|
||||
# Prometheus gauges that hold a CUMULATIVE total -> must be rate()/increase()-wrapped.
|
||||
CUMULATIVE_PREFIXES = (
|
||||
"total_bytes_",
|
||||
"total_messages_",
|
||||
"transactions_messages_",
|
||||
"transactions_duplicate_messages_",
|
||||
"proposals_",
|
||||
"validations_",
|
||||
"overlay_peer_disconnects",
|
||||
"squelch_",
|
||||
"overhead_",
|
||||
"validator_lists_",
|
||||
"set_get_",
|
||||
"set_share_",
|
||||
"have_transactions_",
|
||||
"requested_transactions_",
|
||||
"proof_path_",
|
||||
"replay_delta_",
|
||||
"ledger_data_",
|
||||
"ledger_get_",
|
||||
"ledger_share_",
|
||||
"ledger_transaction_",
|
||||
"ledger_account_state_",
|
||||
"getobject_",
|
||||
"jq_trans_overflow_total",
|
||||
)
|
||||
# nodestore_state{metric=...} cumulative sub-series (raw only inside these metric= labels).
|
||||
NODESTORE_CUMULATIVE = (
|
||||
"node_reads_total",
|
||||
"node_reads_hit",
|
||||
"node_writes",
|
||||
"node_read_bytes",
|
||||
"node_written_bytes",
|
||||
"node_reads_duration_us",
|
||||
)
|
||||
# state_accounting_*_duration are cumulative µs.
|
||||
STATE_DURATION = re.compile(r"state_accounting_\w+_duration")
|
||||
REQUIRED_FILTERS = (
|
||||
"$node",
|
||||
"$deployment_environment",
|
||||
"$xrpl_network_type",
|
||||
"$service_name",
|
||||
)
|
||||
|
||||
|
||||
def iter_panels(dash):
|
||||
for p in dash.get("panels", []):
|
||||
yield p
|
||||
for sub in p.get("panels", []) or []:
|
||||
yield sub
|
||||
|
||||
|
||||
def expr_is_wrapped(expr):
|
||||
return (
|
||||
"rate(" in expr
|
||||
or "increase(" in expr
|
||||
or "irate(" in expr
|
||||
or "histogram_quantile(" in expr
|
||||
)
|
||||
|
||||
|
||||
def check(path, forbid_5m):
|
||||
errs = []
|
||||
try:
|
||||
dash = json.load(open(path))
|
||||
except Exception as e:
|
||||
return [f"{path}: INVALID JSON: {e}"]
|
||||
for p in iter_panels(dash):
|
||||
title = p.get("title", "<untitled>")
|
||||
for tg in p.get("targets", []) or []:
|
||||
expr = (tg.get("expr") or "").strip()
|
||||
if not expr:
|
||||
continue
|
||||
# tier filters
|
||||
if "{" in expr:
|
||||
for f in REQUIRED_FILTERS:
|
||||
if f not in expr:
|
||||
errs.append(f"{path} [{title}]: missing {f} in expr")
|
||||
# cumulative metric plotted raw?
|
||||
hits = [m for m in CUMULATIVE_PREFIXES if m in expr]
|
||||
nod = [m for m in NODESTORE_CUMULATIVE if f'"{m}"' in expr]
|
||||
statedur = STATE_DURATION.search(expr)
|
||||
if (hits or nod or statedur) and not expr_is_wrapped(expr):
|
||||
who = hits or nod or [statedur.group(0)]
|
||||
errs.append(
|
||||
f"{path} [{title}]: cumulative metric {who} plotted RAW (needs rate/increase)"
|
||||
)
|
||||
if "deriv(" in expr or "idelta(" in expr:
|
||||
errs.append(
|
||||
f"{path} [{title}]: uses deriv()/idelta() on a cumulative series"
|
||||
)
|
||||
if forbid_5m and ("[5m]" in expr or "[1h]" in expr):
|
||||
# Only a fixed [5m]/[1h] inside a top-level rate()/increase()/irate() on a
|
||||
# plain counter is a violation. Legitimate fixed-window exprs -- histogram
|
||||
# buckets (rate(..._bucket[5m])), avg_over_time windows, and subqueries
|
||||
# ([5m:]) -- are intentional and must not be flagged.
|
||||
if not ("_bucket" in expr or "avg_over_time" in expr or ":]" in expr):
|
||||
errs.append(
|
||||
f"{path} [{title}]: hardcoded range window; use [$__rate_interval]"
|
||||
)
|
||||
# unit check
|
||||
unit = p.get("fieldConfig", {}).get("defaults", {}).get("unit", "")
|
||||
if unit == "mps":
|
||||
errs.append(f"{path} [{title}]: invalid Grafana unit 'mps' (use 'cps')")
|
||||
return errs
|
||||
|
||||
|
||||
def main():
|
||||
args = [a for a in sys.argv[1:] if not a.startswith("--")]
|
||||
forbid_5m = "--no-5m" in sys.argv
|
||||
all_errs = []
|
||||
for path in args:
|
||||
all_errs += check(path, forbid_5m)
|
||||
if all_errs:
|
||||
print("\n".join(all_errs))
|
||||
print(f"\nFAIL: {len(all_errs)} violation(s)")
|
||||
sys.exit(1)
|
||||
print(f"OK: {len(args)} dashboard(s) passed")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -295,7 +295,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(validations_sent_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]) * 60",
|
||||
"expr": "rate(validations_sent_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m]) * 60",
|
||||
"legendFormat": "Sent/min [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -345,7 +345,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(validations_checked_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]) * 60",
|
||||
"expr": "rate(validations_checked_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m]) * 60",
|
||||
"legendFormat": "Checked/min [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -561,7 +561,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(validation_agreements_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]) * 60",
|
||||
"expr": "rate(validation_agreements_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m]) * 60",
|
||||
"legendFormat": "Agreements/min [{{service_instance_id}}]"
|
||||
},
|
||||
{
|
||||
@@ -569,7 +569,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(validation_missed_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]) * 60",
|
||||
"expr": "rate(validation_missed_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m]) * 60",
|
||||
"legendFormat": "Missed/min [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -757,7 +757,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "3600 * sum by (service_instance_id) (rate(state_changes_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]))",
|
||||
"expr": "3600 * sum by (service_instance_id) (rate(state_changes_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[1h]))",
|
||||
"legendFormat": "Changes/hr [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -807,7 +807,7 @@
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
},
|
||||
"expr": "rate(ledgers_closed_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[$__rate_interval]) * 60",
|
||||
"expr": "rate(ledgers_closed_total{service_instance_id=~\"$node\", deployment_environment=~\"$deployment_environment\", xrpl_network_type=~\"$xrpl_network_type\", service_name=~\"$service_name\"}[5m]) * 60",
|
||||
"legendFormat": "Closed/min [{{service_instance_id}}]"
|
||||
}
|
||||
],
|
||||
@@ -1018,7 +1018,7 @@
|
||||
"label": "Node",
|
||||
"description": "Filter by rippled node (service.instance.id)",
|
||||
"type": "query",
|
||||
"query": "label_values(target_info, service_instance_id)",
|
||||
"query": "label_values(validation_agreement, service_instance_id)",
|
||||
"datasource": {
|
||||
"type": "prometheus",
|
||||
"uid": "${DS_PROMETHEUS}"
|
||||
|
||||
@@ -748,8 +748,8 @@ The `OTelCollector` implementation exports metrics via OTLP/HTTP to the same OTe
|
||||
| `peer_finder_active_outbound_peers` | PeerfinderManager.cpp:215 | Active outbound peer connections |
|
||||
| `overlay_peer_disconnects` | OverlayImpl.h:557 | Peer disconnect count |
|
||||
| `job_count` | JobQueue.cpp:26 | Current job queue depth |
|
||||
| `{category}_bytes_in/out` | OverlayImpl.h:535 | Overlay traffic bytes per category (57 categories) |
|
||||
| `{category}_messages_in/out` | OverlayImpl.h:535 | Overlay traffic messages per category |
|
||||
| `{category}_bytes_in/Out` | OverlayImpl.h:535 | Overlay traffic bytes per category (57 categories) |
|
||||
| `{category}_messages_in/Out` | OverlayImpl.h:535 | Overlay traffic messages per category |
|
||||
|
||||
#### OTel MetricsRegistry Gauges
|
||||
|
||||
@@ -900,17 +900,16 @@ Ten dashboards are pre-provisioned in `docker/telemetry/grafana/dashboards/`:
|
||||
|
||||
### Transaction Overview (`transaction-overview`)
|
||||
|
||||
| Panel | Type | PromQL | Labels Used |
|
||||
| ---------------------------------- | -------------- | -------------------------------------------------------------------------------------------- | ----------------------------------- |
|
||||
| Transaction Processing Rate | timeseries | `rate(span_calls_total{span_name="tx.process"}[5m])` and `tx.receive` | `span_name` |
|
||||
| Transaction Processing Latency | timeseries | `histogram_quantile(0.95 / 0.50, ... {span_name="tx.process"})` | — |
|
||||
| Transaction Path Distribution | piechart | `sum by (local) (rate(span_calls_total{span_name="tx.process"}[5m]))` | `local` |
|
||||
| Transaction Receive vs Suppressed | timeseries | `rate(span_calls_total{span_name="tx.receive"}[5m])` | — |
|
||||
| TX Processing Duration Heatmap | heatmap | `tx.process` histogram buckets | `le` |
|
||||
| TX Apply Duration per Ledger | timeseries | p95/p50 of `tx.apply` | — |
|
||||
| Peer TX Receive Rate | timeseries | `tx.receive` rate | — |
|
||||
| TX Apply Failed Rate | stat | `rate(span_calls_total{span_name="tx.transactor",stage="apply",ter_result!~"tesSUCCESS\|"})` | `stage`, `ter_result` |
|
||||
| TxQ Accept: Applied Ratio per Node | state-timeline | applied / (applied+failed) of `span_calls_total{span_name="txq.accept_tx"}` per node | `txq_status`, `service_instance_id` |
|
||||
| Panel | Type | PromQL | Labels Used |
|
||||
| --------------------------------- | ---------- | --------------------------------------------------------------------- | ------------- |
|
||||
| Transaction Processing Rate | timeseries | `rate(span_calls_total{span_name="tx.process"}[5m])` and `tx.receive` | `span_name` |
|
||||
| Transaction Processing Latency | timeseries | `histogram_quantile(0.95 / 0.50, ... {span_name="tx.process"})` | — |
|
||||
| Transaction Path Distribution | piechart | `sum by (local) (rate(span_calls_total{span_name="tx.process"}[5m]))` | `local` |
|
||||
| Transaction Receive vs Suppressed | timeseries | `rate(span_calls_total{span_name="tx.receive"}[5m])` | — |
|
||||
| TX Processing Duration Heatmap | heatmap | `tx.process` histogram buckets | `le` |
|
||||
| TX Apply Duration per Ledger | timeseries | p95/p50 of `tx.apply` | — |
|
||||
| Peer TX Receive Rate | timeseries | `tx.receive` rate | — |
|
||||
| TX Apply Failed Rate | stat | `tx.apply` with `STATUS_CODE_ERROR` | `status_code` |
|
||||
|
||||
### Consensus Health (`consensus-health`)
|
||||
|
||||
@@ -953,47 +952,47 @@ Requires `trace_peer=1` in the `[telemetry]` config section.
|
||||
|
||||
### Node Health -- System Metrics (`node-health`)
|
||||
|
||||
| Panel | Type | PromQL | Labels Used |
|
||||
| -------------------------------------- | ---------- | ---------------------------------------------------------- | ---------------- |
|
||||
| Validated Ledger Age | stat | `ledgermaster_validated_ledger_age` | — |
|
||||
| Published Ledger Age | stat | `ledgermaster_published_ledger_age` | — |
|
||||
| Operating Mode (Time Share) | timeseries | `rate(state_accounting_X_duration) / sum(rate(all modes))` | — |
|
||||
| Operating Mode Transitions | timeseries | `state_accounting_*_transitions` | — |
|
||||
| I/O Latency | timeseries | `histogram_quantile(0.95, ios_latency_bucket)` | — |
|
||||
| Job Queue Depth | timeseries | `job_count` | — |
|
||||
| Ledger Fetch Rate | stat | `rate(ledger_fetches[5m])` | — |
|
||||
| Ledger History Mismatches | stat | `rate(ledger_history_mismatch[5m])` | — |
|
||||
| Key Jobs Execution Time | timeseries | `acceptledger{quantile="$quantile"}` (+ 10 more key jobs) | `quantile` |
|
||||
| Key Jobs Dequeue Wait Time | timeseries | `acceptledger_q{quantile="$quantile"}` (+ 10 more) | `quantile` |
|
||||
| FullBelowCache Size | timeseries | `node_family_full_below_cache_size` | — |
|
||||
| FullBelowCache Hit Rate | gauge | `node_family_full_below_cache_hit_rate` | — |
|
||||
| Ledger Publish Gap | stat | `Published_Ledger_Age - Validated_Ledger_Age` | — |
|
||||
| State Duration Rate (Full vs Tracking) | timeseries | `rate(state_accounting_full_duration[5m]) / 1000000` | — |
|
||||
| All Jobs Execution Time (Detail) | timeseries | `{__name__=~"<all_jobs>", quantile="$quantile"}` | `quantile` |
|
||||
| All Jobs Dequeue Wait (Detail) | timeseries | `{__name__=~"<all_jobs>_q", quantile="$quantile"}` | `quantile` |
|
||||
| Server State | stat | `server_info{metric="server_state"}` | `metric` |
|
||||
| Uptime | stat | `server_info{metric="uptime"}` | `metric` |
|
||||
| Peer Count | stat | `server_info{metric="peers"}` | `metric` |
|
||||
| Validated Ledger Seq | stat | `server_info{metric="validated_ledger_seq"}` | `metric` |
|
||||
| Build Version | stat | `build_info` | `version` |
|
||||
| Complete Ledger Ranges | table | `complete_ledgers` | `bound`, `index` |
|
||||
| Database Sizes | timeseries | `db_metrics{metric=~"db_kb_.*"}` | `metric` |
|
||||
| Historical Fetch Rate | stat | `db_metrics{metric="historical_perminute"}` | `metric` |
|
||||
| Panel | Type | PromQL | Labels Used |
|
||||
| -------------------------------------- | ---------- | --------------------------------------------------------- | ---------------- |
|
||||
| Validated Ledger Age | stat | `ledgermaster_validated_ledger_age` | — |
|
||||
| Published Ledger Age | stat | `ledgermaster_published_ledger_age` | — |
|
||||
| Operating Mode Duration | timeseries | `state_accounting_*_duration` | — |
|
||||
| Operating Mode Transitions | timeseries | `state_accounting_*_transitions` | — |
|
||||
| I/O Latency | timeseries | `histogram_quantile(0.95, ios_latency_bucket)` | — |
|
||||
| Job Queue Depth | timeseries | `job_count` | — |
|
||||
| Ledger Fetch Rate | stat | `rate(ledger_fetches[5m])` | — |
|
||||
| Ledger History Mismatches | stat | `rate(ledger_history_mismatch[5m])` | — |
|
||||
| Key Jobs Execution Time | timeseries | `acceptledger{quantile="$quantile"}` (+ 10 more key jobs) | `quantile` |
|
||||
| Key Jobs Dequeue Wait Time | timeseries | `acceptledger_q{quantile="$quantile"}` (+ 10 more) | `quantile` |
|
||||
| FullBelowCache Size | timeseries | `node_family_full_below_cache_size` | — |
|
||||
| FullBelowCache Hit Rate | gauge | `node_family_full_below_cache_hit_rate` | — |
|
||||
| Ledger Publish Gap | stat | `Published_Ledger_Age - Validated_Ledger_Age` | — |
|
||||
| State Duration Rate (Full vs Tracking) | timeseries | `rate(state_accounting_full_duration[5m]) / 1000000` | — |
|
||||
| All Jobs Execution Time (Detail) | timeseries | `{__name__=~"<all_jobs>", quantile="$quantile"}` | `quantile` |
|
||||
| All Jobs Dequeue Wait (Detail) | timeseries | `{__name__=~"<all_jobs>_q", quantile="$quantile"}` | `quantile` |
|
||||
| Server State | stat | `server_info{metric="server_state"}` | `metric` |
|
||||
| Uptime | stat | `server_info{metric="uptime"}` | `metric` |
|
||||
| Peer Count | stat | `server_info{metric="peers"}` | `metric` |
|
||||
| Validated Ledger Seq | stat | `server_info{metric="validated_ledger_seq"}` | `metric` |
|
||||
| Build Version | stat | `build_info` | `version` |
|
||||
| Complete Ledger Ranges | table | `complete_ledgers` | `bound`, `index` |
|
||||
| Database Sizes | timeseries | `db_metrics{metric=~"db_kb_.*"}` | `metric` |
|
||||
| Historical Fetch Rate | stat | `db_metrics{metric="historical_perminute"}` | `metric` |
|
||||
|
||||
### Network Traffic -- System Metrics (`network-traffic`)
|
||||
|
||||
| Panel | Type | PromQL | Labels Used |
|
||||
| ------------------------------------ | ---------- | ------------------------------------------------------ | ----------- |
|
||||
| Active Peers | timeseries | `peer_finder_active_*_peers` | — |
|
||||
| Peer Disconnects | timeseries | `increase(overlay_peer_disconnects[$__rate_interval])` | — |
|
||||
| Total Network Bytes | timeseries | `rate(total_bytes_in/out[$__rate_interval])` | — |
|
||||
| Total Network Messages | timeseries | `rate(total_messages_in/out[$__rate_interval])` | — |
|
||||
| Transaction Traffic | timeseries | `rate(transactions_messages_in/out[$__rate_interval])` | — |
|
||||
| Proposal Traffic | timeseries | `rate(proposals_messages_in/out[$__rate_interval])` | — |
|
||||
| Validation Traffic | timeseries | `rate(validations_messages_in/out[$__rate_interval])` | — |
|
||||
| Traffic by Category | bargauge | `topk(10, rate(*_bytes_in[$__rate_interval]))` | — |
|
||||
| Duplicate Traffic (Wasted Bandwidth) | timeseries | `rate(*_duplicate_bytes_in/out[$__rate_interval])` | — |
|
||||
| All Traffic Categories (Detail) | timeseries | `topk(15, rate(*_bytes_in[$__rate_interval]))` | — |
|
||||
| Panel | Type | PromQL | Labels Used |
|
||||
| ------------------------------------ | ---------- | ------------------------------------ | ----------- |
|
||||
| Active Peers | timeseries | `peer_finder_active_*_Peers` | — |
|
||||
| Peer Disconnects | timeseries | `overlay_peer_disconnects` | — |
|
||||
| Total Network Bytes | timeseries | `rate(total_bytes_in/Out[5m])` | — |
|
||||
| Total Network Messages | timeseries | `total_messages_in/Out` | — |
|
||||
| Transaction Traffic | timeseries | `transactions_messages_in/Out` | — |
|
||||
| Proposal Traffic | timeseries | `proposals_messages_in/Out` | — |
|
||||
| Validation Traffic | timeseries | `validations_messages_in/Out` | — |
|
||||
| Traffic by Category | bargauge | `topk(10, *_bytes_in)` | — |
|
||||
| Duplicate Traffic (Wasted Bandwidth) | timeseries | `rate(*_duplicate_bytes_in/out[5m])` | — |
|
||||
| All Traffic Categories (Detail) | timeseries | `topk(15, rate(*_bytes_in[5m]))` | — |
|
||||
|
||||
### RPC & Pathfinding -- System Metrics (`rpc-pathfinding`)
|
||||
|
||||
|
||||
@@ -1129,7 +1129,8 @@ public:
|
||||
<< "; size after: " << cachedSLEs_.size();
|
||||
}
|
||||
|
||||
mallocTrim("doSweep", journal_);
|
||||
// Disabled for allocator benchmark: no proactive malloc_trim on sweep.
|
||||
// mallocTrim("doSweep", journal_);
|
||||
|
||||
// Set timer to do another sweep later.
|
||||
setSweepTimer();
|
||||
|
||||
@@ -254,30 +254,8 @@ MetricsRegistry::initSyncInstruments()
|
||||
"validations_checked_total", "Total network validations received and checked");
|
||||
stateChangesCounter_ =
|
||||
meter_->CreateUInt64Counter("state_changes_total", "Total operating mode changes");
|
||||
// jq_trans_overflow_total is observed from Overlay's existing cumulative
|
||||
// atomic (Overlay::getJqTransOverflow()) rather than pushed. The overlay
|
||||
// owns the only increment site (PeerImp), so an ObservableCounter reads the
|
||||
// live total each collection cycle without threading a push path through
|
||||
// develop-owned overlay code.
|
||||
jqTransOverflowObservable_ = meter_->CreateInt64ObservableCounter(
|
||||
jqTransOverflowCounter_ = meter_->CreateUInt64Counter(
|
||||
"jq_trans_overflow_total", "Total job queue transaction overflows");
|
||||
jqTransOverflowObservable_->AddCallback(
|
||||
[](opentelemetry::metrics::ObserverResult result, void* state) {
|
||||
auto* self = static_cast<MetricsRegistry*>(state);
|
||||
if (self->callbacksDetached_.load(std::memory_order_acquire))
|
||||
return;
|
||||
try
|
||||
{
|
||||
opentelemetry::nostd::get<opentelemetry::nostd::shared_ptr<
|
||||
opentelemetry::metrics::ObserverResultT<int64_t>>>(result)
|
||||
->Observe(static_cast<int64_t>(self->app_.getOverlay().getJqTransOverflow()));
|
||||
}
|
||||
catch (...) // NOLINT(bugprone-empty-catch)
|
||||
{
|
||||
// Silently skip on error.
|
||||
}
|
||||
},
|
||||
this);
|
||||
ledgerHistoryMismatchCounter_ = meter_->CreateUInt64Counter(
|
||||
"ledger_history_mismatch_total", "Total built-vs-validated ledger mismatches by reason");
|
||||
txqExpiredCounter_ = meter_->CreateUInt64Counter(
|
||||
@@ -1480,6 +1458,15 @@ MetricsRegistry::incrementStateChanges()
|
||||
#endif
|
||||
}
|
||||
|
||||
void
|
||||
MetricsRegistry::incrementJqTransOverflow()
|
||||
{
|
||||
#ifdef XRPL_ENABLE_TELEMETRY
|
||||
if (enabled_ && jqTransOverflowCounter_)
|
||||
jqTransOverflowCounter_->Add(1);
|
||||
#endif
|
||||
}
|
||||
|
||||
void
|
||||
MetricsRegistry::incrementLedgerHistoryMismatch(std::string_view reason)
|
||||
{
|
||||
|
||||
@@ -36,6 +36,7 @@
|
||||
| +-- validations_sent_total
|
||||
| +-- validations_checked_total
|
||||
| +-- state_changes_total
|
||||
| +-- jq_trans_overflow_total
|
||||
| +-- ledger_history_mismatch_total{reason}
|
||||
| +-- txq_expired_total
|
||||
| +-- txq_dropped_total{reason}
|
||||
@@ -60,7 +61,6 @@
|
||||
+-- State tracking (mode value, time in state)
|
||||
+-- Storage detail (NuDB sizes)
|
||||
+-- Validation agreement (1h/24h pct, counts)
|
||||
+-- jq_trans_overflow_total (observed from Overlay)
|
||||
|
||||
Control-flow for async gauges:
|
||||
|
||||
@@ -347,6 +347,13 @@ public:
|
||||
void
|
||||
incrementStateChanges();
|
||||
|
||||
/** Increment the jq_trans_overflow_total counter.
|
||||
Called when the job queue transaction limit overflows (mirrors
|
||||
Overlay::incJqTransOverflow()).
|
||||
*/
|
||||
void
|
||||
incrementJqTransOverflow();
|
||||
|
||||
/** Increment the ledger_history_mismatch_total counter for a reason.
|
||||
Called from LedgerHistory::handleMismatch() once the mismatch has
|
||||
been classified. The reason label turns fork diagnosis from a
|
||||
@@ -510,10 +517,9 @@ private:
|
||||
/// Counter: state_changes_total — incremented on operating mode transitions.
|
||||
opentelemetry::nostd::unique_ptr<opentelemetry::metrics::Counter<uint64_t>>
|
||||
stateChangesCounter_;
|
||||
/// ObservableCounter: jq_trans_overflow_total — observed from
|
||||
/// Overlay::getJqTransOverflow() (cumulative overflow tally owned by the overlay).
|
||||
opentelemetry::nostd::shared_ptr<opentelemetry::metrics::ObservableInstrument>
|
||||
jqTransOverflowObservable_;
|
||||
/// Counter: jq_trans_overflow_total — incremented on job queue transaction overflows.
|
||||
opentelemetry::nostd::unique_ptr<opentelemetry::metrics::Counter<uint64_t>>
|
||||
jqTransOverflowCounter_;
|
||||
/// Counter: ledger_history_mismatch_total{reason} — incremented per classified
|
||||
/// built-vs-validated ledger mismatch.
|
||||
opentelemetry::nostd::unique_ptr<opentelemetry::metrics::Counter<uint64_t>>
|
||||
|
||||
Reference in New Issue
Block a user