mirror of
https://github.com/XRPLF/rippled.git
synced 2026-07-25 16:10:57 +00:00
fix(telemetry): use service_instance_id in Grafana Cloud transform and docs
Update the transform/cloudlabels processor to set service_instance_id instead of exported_instance, matching the local Prometheus path where resource_to_telemetry_conversion promotes service.instance.id with that exact label name. Also update source comments and documentation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -618,7 +618,7 @@ matches its Prometheus label), letting one Grafana stack be sliced by tier:
|
||||
|
||||
| Variable | Source label | Description |
|
||||
| ------------------------- | ------------------------ | ------------------------------------------------------------ |
|
||||
| `$node` | `exported_instance` | Filter by xrpld node instance |
|
||||
| `$node` | `service_instance_id` | Filter by xrpld node instance |
|
||||
| `$service_name` | `service_name` | Filter by service (`service.name`, e.g. `xrpld`) |
|
||||
| `$deployment_environment` | `deployment_environment` | Filter by deployment tier (`local` / `test` / `ci` / `prod`) |
|
||||
| `$xrpl_network_type` | `xrpl_network_type` | Filter by network (`mainnet` / `testnet` / `devnet`) |
|
||||
|
||||
@@ -97,13 +97,12 @@ processors:
|
||||
# local prometheus exporter promotes them (resource_to_telemetry_conversion
|
||||
# is Prometheus-exporter-only). Copy them onto datapoint labels so the
|
||||
# dashboards' $node / $deployment_environment / $xrpl_network_type filters
|
||||
# resolve on Cloud exactly as they do locally. exported_instance mirrors the
|
||||
# instance label the local Prometheus scrape produces from service.instance.id.
|
||||
# resolve on Cloud exactly as they do locally.
|
||||
transform/cloudlabels:
|
||||
metric_statements:
|
||||
- context: datapoint
|
||||
statements:
|
||||
- set(attributes["exported_instance"], resource.attributes["service.instance.id"])
|
||||
- set(attributes["service_instance_id"], resource.attributes["service.instance.id"])
|
||||
- set(attributes["deployment_environment"], resource.attributes["deployment.environment"])
|
||||
- set(attributes["xrpl_network_type"], resource.attributes["xrpl.network.type"])
|
||||
|
||||
@@ -190,7 +189,7 @@ service:
|
||||
# labels via resource_to_telemetry_conversion; Grafana Cloud (OTLP) does
|
||||
# not, so it runs a separate pipeline that copies them onto datapoint
|
||||
# labels via transform/cloudlabels. Splitting avoids the local scrape and
|
||||
# the transform both writing an exported_instance label on the same series.
|
||||
# the transform both writing a service_instance_id label on the same series.
|
||||
metrics/local:
|
||||
receivers: [otlp, spanmetrics]
|
||||
processors: [resource/tier, resource/stripsdk, batch]
|
||||
|
||||
@@ -115,7 +115,7 @@ server=otel
|
||||
endpoint=http://localhost:4318/v1/metrics
|
||||
prefix=xrpld
|
||||
# Sets the OTel service.instance.id resource attribute, which Prometheus
|
||||
# exposes as the `exported_instance` label. Dashboards filter on it via the
|
||||
# exposes as the `service_instance_id` label. Dashboards filter on it via the
|
||||
# $node template variable, so without this every insight-backed panel is
|
||||
# empty. Matches [telemetry] service_instance_id for a single node identity.
|
||||
service_instance_id=xrpld-mainnet
|
||||
|
||||
@@ -113,7 +113,7 @@ server=otel
|
||||
endpoint=http://localhost:4318/v1/metrics
|
||||
prefix=xrpld
|
||||
# Sets the OTel service.instance.id resource attribute, which Prometheus
|
||||
# exposes as the `exported_instance` label. Dashboards filter on it via the
|
||||
# exposes as the `service_instance_id` label. Dashboards filter on it via the
|
||||
# $node template variable, so without this every insight-backed panel is
|
||||
# empty. Matches [telemetry] service_instance_id for a single node identity.
|
||||
service_instance_id=xrpld-devnet
|
||||
|
||||
@@ -58,7 +58,7 @@ public network with all tracing and native metrics enabled:
|
||||
|
||||
Both set `[insight] server=otel` (native metrics → collector → Prometheus, which
|
||||
drives the dashboards) and `service_instance_id`, exposed by Prometheus as the
|
||||
`exported_instance` label that the `$node` dashboard variable filters on. The
|
||||
`service_instance_id` label that the `$node` dashboard variable filters on. The
|
||||
mainnet config logs to `/var/log/xrpld/mainnet/debug.log` — the path
|
||||
the collector's filelog receiver tails for log-trace correlation.
|
||||
|
||||
@@ -1020,7 +1020,7 @@ folder **xrpld**.
|
||||
### Alert catalogue
|
||||
|
||||
All rules evaluate every minute against the Prometheus datasource, over a
|
||||
5-minute window, and group by `exported_instance` so each node alerts on its
|
||||
5-minute window, and group by `service_instance_id` so each node alerts on its
|
||||
own. Alerts fire only after the condition holds for the `for` dwell time.
|
||||
|
||||
| Alert | Severity | Fires when | For |
|
||||
@@ -1097,7 +1097,7 @@ The severity split lives in
|
||||
sends everything to `xrpld-default`, and a child route matching
|
||||
`severity = critical` overrides to `xrpld-critical`. So a critical alert goes
|
||||
to Slack **and** email; a warning goes to Slack only. Both group by
|
||||
`alertname` + `exported_instance`; critical alerts re-page hourly vs the 4h default.
|
||||
`alertname` + `service_instance_id`; critical alerts re-page hourly vs the 4h default.
|
||||
|
||||
#### Configure delivery (no secrets in git)
|
||||
|
||||
|
||||
@@ -1572,7 +1572,7 @@ ApplicationImp::start(bool withTimers)
|
||||
set(endpoint, "metrics_endpoint", section);
|
||||
|
||||
// Pass the service_instance_id so the MeterProvider Resource
|
||||
// carries it, giving Prometheus an exported_instance label.
|
||||
// carries it, giving Prometheus an service_instance_id label.
|
||||
std::string instanceId;
|
||||
set(instanceId, "service_instance_id", section);
|
||||
if (instanceId.empty() && nodeIdentity_)
|
||||
|
||||
@@ -54,7 +54,7 @@ public:
|
||||
|
||||
// Read service_instance_id, same key as the [telemetry]
|
||||
// section uses, so multi-node deployments can distinguish
|
||||
// metric sources via the exported_instance Prometheus label.
|
||||
// metric sources via the service_instance_id Prometheus label.
|
||||
std::string const instanceId = get(params, "service_instance_id");
|
||||
|
||||
// service.name from [insight] (falls back to the value the
|
||||
|
||||
@@ -195,7 +195,7 @@ MetricsRegistry::initExporterAndProvider(std::string const& endpoint, std::strin
|
||||
auto reader =
|
||||
metric_sdk::PeriodicExportingMetricReaderFactory::Create(std::move(exporter), readerOpts);
|
||||
|
||||
// Configure resource attributes so Prometheus exported_instance labels
|
||||
// Configure resource attributes so Prometheus service_instance_id labels
|
||||
// distinguish metrics from different nodes (matches OTelCollector setup).
|
||||
resource::ResourceAttributes attrs;
|
||||
// Use std::string, not a string literal: ResourceAttributes stores an
|
||||
|
||||
@@ -230,7 +230,7 @@ public:
|
||||
(e.g. "http://localhost:4318/v1/metrics").
|
||||
@param instanceId Value for the service.instance.id resource
|
||||
attribute. When non-empty, Prometheus metrics
|
||||
carry an exported_instance label for per-node
|
||||
carry an service_instance_id label for per-node
|
||||
filtering.
|
||||
*/
|
||||
void
|
||||
|
||||
Reference in New Issue
Block a user