fix(telemetry): correct service.name label and make alerting boot without env

- MetricsRegistry: service.name was recorded as boolean true because the
  string literal "xrpld" bound to the OTel AttributeValue variant's
  char-const* -> bool overload. Assign std::string so it selects the
  string alternative; Prometheus now shows service_name=xrpld (and
  exported_job round-trips correctly) on every MetricsRegistry series.
- Grafana alerting contact points: the Slack url / email addresses used
  ${ENV_VAR} references that expand to empty when unset, and Grafana's
  provisioning validator (which lacks ${VAR:-default} support) then
  crashes the whole stack on startup. Use non-empty disabled placeholders
  (unroutable webhook host, .invalid email) so the stack boots with zero
  configuration; delivery stays off until a real destination is supplied.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Pratik Mankawde
2026-07-07 18:28:11 +01:00
parent 8919be22bf
commit 979f4b64e4
2 changed files with 29 additions and 14 deletions

View File

@@ -7,13 +7,20 @@
# xrpld-critical — Slack + email; receives critical-severity alerts.
# The severity split is wired in policies.yaml.
#
# Secrets and personal addresses are NOT hard-coded. Grafana expands
# ${ENV_VAR} references in provisioning files at load time, so the real
# Slack webhook and alert-email addresses come from environment variables
# supplied via docker/telemetry/.env.alerting (gitignored). See
# .env.alerting.example for the variable list and the Alerting section of
# Secrets and personal addresses are NOT hard-coded. To enable delivery,
# replace the disabled placeholder values below with a real Slack webhook
# and alert email — e.g. by copying docker/telemetry/.env.alerting.example
# to .env.alerting (gitignored) and swapping the placeholder lines to
# ${SLACK_WEBHOOK_URL} / ${ALERT_EMAIL_TO}. See the Alerting section of
# docs/telemetry-runbook.md for setup.
# If a variable is unset, that receiver has no live destination.
#
# The defaults are deliberately non-empty, invalid-but-valid-shaped
# placeholders (an unroutable webhook host and a .invalid email). Grafana's
# alerting provisioning validator REQUIRES a non-empty Slack url and email
# addresses, and it does NOT support ${VAR:-default} expansion — an unset
# ${VAR} expands to empty and crashes Grafana on startup. The placeholders
# keep the whole stack booting with zero configuration; alerts simply route
# nowhere until a real destination is supplied.
#
# Email delivery additionally requires SMTP configured on the Grafana
# service (GF_SMTP_* in docker-compose.yml).
@@ -28,10 +35,11 @@ contactPoints:
- uid: xrpld-slack-default
type: slack
settings:
# Incoming-webhook delivery: the channel is fixed by the webhook, so
# `recipient` is only here to satisfy Grafana's Slack validator.
url: ${SLACK_WEBHOOK_URL}
recipient: ${SLACK_CHANNEL}
# Disabled placeholder: an unroutable webhook host. A non-empty url
# selects Slack webhook mode (no recipient/token required) and keeps
# provisioning valid. Replace with a real ${SLACK_WEBHOOK_URL} to
# enable delivery.
url: https://hooks.slack.invalid/disabled
title: "{{ .CommonLabels.alertname }} on {{ .CommonLabels.exported_instance }}"
disableResolveMessage: false
@@ -42,13 +50,16 @@ contactPoints:
- uid: xrpld-slack-critical
type: slack
settings:
url: ${SLACK_WEBHOOK_URL}
recipient: ${SLACK_CHANNEL}
# Disabled placeholder — see xrpld-slack-default above.
url: https://hooks.slack.invalid/disabled
title: "[CRITICAL] {{ .CommonLabels.alertname }} on {{ .CommonLabels.exported_instance }}"
disableResolveMessage: false
- uid: xrpld-email-critical
type: email
settings:
addresses: ${ALERT_EMAIL_TO}
# Disabled placeholder: a .invalid address keeps the required
# `addresses` field non-empty so provisioning validates. Replace
# with a real ${ALERT_EMAIL_TO} (and enable SMTP) to deliver.
addresses: alerts-disabled@xrpld.invalid
singleEmail: true
disableResolveMessage: false

View File

@@ -182,7 +182,11 @@ MetricsRegistry::start(std::string const& endpoint, std::string const& instanceI
// Configure resource attributes so Prometheus exported_instance labels
// distinguish metrics from different nodes (matches OTelCollector setup).
resource::ResourceAttributes attrs;
attrs[opentelemetry::semconv::service::kServiceName] = "xrpld";
// Use std::string, not a string literal: ResourceAttributes stores an
// OTel AttributeValue variant whose char-const* overload binds to bool,
// so "xrpld" would be recorded as the boolean true. std::string selects
// the string alternative and the value round-trips as service.name=xrpld.
attrs[opentelemetry::semconv::service::kServiceName] = std::string("xrpld");
if (!instanceId.empty())
attrs[opentelemetry::semconv::service::kServiceInstanceId] = instanceId;
auto resourceAttrs = resource::Resource::Create(attrs);