Replace the hardcoded /home/pratik/xrpld-logs path with the conventional
/var/log/xrpld in the mainnet config's [debug_logfile], the docker-compose
log volume mount, and the collector-config comment. Removes a personal home
directory from shipped config.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Add an operator-facing Alerting section to the runbook covering the six
provisioned Grafana rules and how to point them at a real receiver.
- Strip the "(Phase 9)" suffix from the MetricsRegistry Gauges heading.
- ALERTING.md: drop the internal dev-phase reference from the intro.
- Phase9_taskList.md: expand Task 9.9 to describe the provisioned alerting
(rules, contact point, policy, ALERTING.md runbook).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add operator-facing alerting on the health-critical metrics phase 9 emits,
completing the "alerting rules" deliverable already listed in the phase-9
task list and Jira story.
Provisioned YAML under grafana/provisioning/alerting/ (auto-loaded by the
Grafana container; no docker-compose change needed):
- rules.yaml: 6 alerts in 3 groups — consensus/ledger health, validator
health, job queue. Each is a query(A, 5m rate/quantile by exported_instance)
-> reduce(B, last) -> threshold(C) on the Prometheus datasource.
- contactpoints.yaml: xrpld-default webhook receiver (local-dev placeholder).
- policies.yaml: flat routing to xrpld-default, grouped by
alertname + exported_instance.
Also adds docker/telemetry/ALERTING.md runbook (per-alert meaning, causes,
threshold tuning, how to point alerts at a real receiver) and the design spec.
Verified live: Grafana 11.5.2 provisioned all 6 rules with no rule in error
health; contact point loaded.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Filter dropdowns were empty on Grafana Cloud: template query-variables
referenced uid ${DS_PROMETHEUS} (a datasource variable that had no value
and was hidden), while panels used the bare default datasource. So panels
rendered but Service/Environment/Network/Node variables resolved against
nothing.
Wire DS_PROMETHEUS (and DS_TEMPO where used) through every panel, target,
annotation, and query-variable, and make the pickers visible. Empty
current value means each instance auto-selects its default datasource of
that type, so one dashboard set works on both the local stack and Grafana
Cloud without per-instance uids.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Grafana Cloud ingests metrics via OTLP with no Prometheus scrape, so the
tier/instance resource attributes never became series labels the way the
local prometheus exporter promotes them. Dashboard $node,
$deployment_environment, and $xrpl_network_type filters therefore matched
nothing on Cloud (only service_name existed).
Split the metrics pipeline: metrics/local keeps the Prometheus exporter
(unchanged label promotion), metrics/cloud adds a transform that copies
service.instance.id -> exported_instance, deployment.environment ->
deployment_environment, and xrpl.network.type -> xrpl_network_type onto
datapoint labels so the same dashboards filter correctly on both backends.
Also strip telemetry.sdk.* on all pipelines. Validated against
otel-collector-contrib 0.121.0.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a resource/stripsdk processor that deletes telemetry.sdk.language,
telemetry.sdk.name, and telemetry.sdk.version (auto-added by the OTel SDK
to every Resource). Wire it into the traces pipeline; the metrics and
logs pipelines pick it up on the branches that introduce them.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Point the workload validation uid list and implementation-phases doc at
the renamed bare dashboard uids.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Rename the 4 xrpld-* dashboard files to bare names and set every
dashboard uid to its bare filename stem; update doc/plan references.
Removes the redundant xrpld-/system- prefixes now that titles are clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Rename the 5 system-* dashboard files to bare domain names (uid ==
filename stem) and update all doc references. The system- prefix was
redundant now that titles no longer carry a recorder-pipeline suffix.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Make dashboard uids bare domain names (uid == filename stem) and update
doc references. Grafana links/bookmarks to the old uids will need
refreshing.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Grafana renders the dashboard-level description as plain text (not
markdown), so the h6 headings/italics showed as literal characters.
Convert to plain 'What this shows:' / 'Use it to:' labeled lines. Panel
descriptions (which DO render markdown) are unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Grafana renders the dashboard-level description as plain text (not
markdown), so the h6 headings/italics showed as literal characters.
Convert to plain 'What this shows:' / 'Use it to:' labeled lines. Panel
descriptions (which DO render markdown) are unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The separate xrpld-rpc-perf-otel dashboard was merged into xrpld-rpc-perf,
so the validation harness must no longer expect it to exist.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Fold the 7 panels from the separate RPC Performance (OTel) dashboard into
RPC Performance as a labelled 'Per-Method Detail' section, and retire the
duplicate dashboard. The two showed the same subject (RPC) from two data
sources; one dashboard with two sections is clearer and loses no panel.
Adds the $method template variable for per-method filtering.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Consensus Health, Ledger Operations, and Transaction Overview had no
top-level description, so Grafana showed no info button. Add a one-line
summary of what each dashboard shows and its data source.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
