Commit Graph

222 Commits

Author SHA1 Message Date
Pratik Mankawde
0ee6da59be fix(telemetry): resolve clang-tidy include and modernize-use-auto CI errors
- Replace unused observer_result.h with nostd/shared_ptr.h and
  nostd/unique_ptr.h in MetricsRegistry.h (misc-include-cleaner)
- Use auto for cast initializations in MetricsRegistry.cpp and
  ValidationTracker test (modernize-use-auto)
- Run rename scripts: rippled -> xrpld in comments and docs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-08 15:36:44 +01:00
Pratik Mankawde
e3188db2b3 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill
# Conflicts:
#	docker/telemetry/docker-compose.yml
#	docker/telemetry/otel-collector-config.yaml
#	docker/telemetry/xrpld-telemetry.cfg
2026-07-07 20:57:52 +01:00
Pratik Mankawde
f99a1c5310 fix(telemetry): consolidate xrpld log mount to a single rootless path
The collector's log ingestion used a container path named after the old
`rippled` binary and defaulted its host mount to a location that either
needed root or diverged from where the telemetry configs actually write.

Consolidate on one accessible log root:
- container mount target renamed /var/log/rippled -> /var/log/xrpld
- filelog glob -> /var/log/xrpld/*/debug.log
- compose mount source defaults to the repo-relative ./data/logs
  (user-owned, no root), overridable via XRPLD_LOG_DIR
- devnet telemetry cfg writes to data/logs/devnet/debug.log so it lands
  one subdir below the mount root and matches the glob
- integration-test.sh sets XRPLD_LOG_DIR to its own workdir
- docs/runbook/task-list updated to match

The default xrpld config (cfg/xrpld-example.cfg) is intentionally left
untouched.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 20:54:53 +01:00
Pratik Mankawde
6a2570e26e Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill
# Conflicts:
#	docker/telemetry/docker-compose.yml
2026-07-07 20:00:36 +01:00
Pratik Mankawde
9e5f1caff2 docs(telemetry): address review feedback on phase-8 log correlation
- Correct log-timestamp examples to real Logs::format() output
  (2024-Jan-15 10:30:45.123456 UTC) in docs, TESTING.md, and reference.
- Update Loki to v3.4.2 and switch the collector exporter to
  otlphttp/loki on the native /otlp endpoint (docs + task list).
- Make the collector log-mount path configurable via XRPLD_LOG_DIR.
- Remove implementation-phase references from shipped config/script
  comments, keeping the functional descriptions.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 19:55:26 +01:00
Pratik Mankawde
f240bb2922 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-07-07 13:09:44 +01:00
Pratik Mankawde
4abc180778 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-07-07 13:09:33 +01:00
Pratik Mankawde
ed5a618aa2 docs(telemetry): fold alerting design spec into Phase9 task list
Move the phase-9 alerting design spec out of docs/superpowers/ (which the
project guidelines say not to create) into an appendix of
OpenTelemetryPlan/Phase9_taskList.md, keeping the phase plan self-contained.
Drops the stale one-off branch/PR-restructure section from the spec.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 13:09:25 +01:00
Pratik Mankawde
134b419895 docs(telemetry): fold external-parity spec into implementation-phases doc
Move the standalone External Dashboard Parity design spec out of
docs/superpowers/ (which the project guidelines say not to create) and into
an appendix of OpenTelemetryPlan/06-implementation-phases.md, so the phase
plan is self-contained. Repoint the 10 "Source" links in the phase 3/4/7
task lists to the new appendix anchor.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 13:08:04 +01:00
Pratik Mankawde
9b3045d782 docs(telemetry): repoint alerting docs from ALERTING.md to runbook
Update the phase-9 task list and the alerting design spec to reference the
Alerting section of docs/telemetry-runbook.md, now that the standalone
ALERTING.md has been folded into the runbook.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 12:58:13 +01:00
Pratik Mankawde
4ef8a02389 feat(telemetry): route alerts to Slack + email with severity tiers
Replace the placeholder webhook contact point with two env-driven tiers:
- xrpld-default (Slack) receives warning-severity alerts
- xrpld-critical (Slack + email) receives critical-severity alerts, via a
  severity=critical child route in the notification policy

Secrets stay out of git: contactpoints.yaml references ${SLACK_WEBHOOK_URL},
${SLACK_CHANNEL}, and ${ALERT_EMAIL_TO}, which Grafana expands from a
gitignored .env.alerting (loaded via an optional env_file on the grafana
service). .env.alerting.example documents the variables; email additionally
needs GF_SMTP_* configured.

Fold the ALERTING.md runbook content into the Alerting section of
docs/telemetry-runbook.md (operator docs live in one place) and delete
ALERTING.md. Move the .env.* ignore rule into docker/telemetry/.gitignore.

Verified live: Grafana 11.5.2 provisioned both contact points, the
severity-routed policy, and all 6 rules with no errors.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 12:57:43 +01:00
Pratik Mankawde
dce39bc71d Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill
# Conflicts:
#	OpenTelemetryPlan/08-appendix.md
#	OpenTelemetryPlan/presentation.md
#	docker/telemetry/xrpld-telemetry.cfg
#	docs/telemetry-runbook.md
2026-07-06 22:06:45 +01:00
Pratik Mankawde
55052917db Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation
# Conflicts:
#	OpenTelemetryPlan/08-appendix.md
2026-07-06 22:02:14 +01:00
Pratik Mankawde
363c2caf94 Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics
# Conflicts:
#	OpenTelemetryPlan/08-appendix.md
#	OpenTelemetryPlan/09-data-collection-reference.md
#	docker/telemetry/docker-compose.yml
#	docker/telemetry/grafana/dashboards/statsd-ledger-data-sync.json
#	docker/telemetry/grafana/dashboards/statsd-network-traffic.json
#	docker/telemetry/grafana/dashboards/statsd-node-health.json
#	docker/telemetry/grafana/dashboards/statsd-rpc-pathfinding.json
2026-07-06 22:01:22 +01:00
Pratik Mankawde
f4981a1907 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd
# Conflicts:
#	OpenTelemetryPlan/08-appendix.md
#	docker/telemetry/grafana/provisioning/datasources/tempo.yaml
#	docker/telemetry/otel-collector-config.yaml
#	docs/telemetry-runbook.md
2026-07-06 21:47:17 +01:00
Pratik Mankawde
b7d0b700c6 fix(telemetry): correct runbook pathfind spans + mTLS validation
- Add PathFinding Spans section to the runbook Span Reference and
  remove the incorrect "not emitted / later-phase" annotations:
  pathfind.request/compute/discover/update_all are emitted since
  Phase 2 (upstream of Phase 5).
- Runbook label names use the bare spanmetrics dimensions
  (command, rpc_status, local, consensus_mode); fix stale docs.
- 05-configuration-reference: note tls_client_cert/key require use_tls=1.
- TelemetryConfig: reject mTLS cert paths when use_tls=0 so telemetry
  is not silently sent in plaintext.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 21:04:07 +01:00
Pratik Mankawde
206d386469 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill
# Conflicts:
#	src/xrpld/app/ledger/InboundLedger.h
#	src/xrpld/perflog/detail/PerfLogImp.cpp
2026-07-06 17:17:21 +01:00
Pratik Mankawde
fe88034ea9 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-07-06 17:15:46 +01:00
Pratik Mankawde
fd518d3e7b Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics
# Conflicts:
#	.github/scripts/levelization/results/ordering.txt
#	include/xrpl/beast/insight/Insight.h
#	src/test/csf/Peer.h
2026-07-06 17:15:27 +01:00
Pratik Mankawde
d5b858ce9d Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd
# Conflicts:
#	.cspell.config.yaml
#	src/xrpld/app/ledger/detail/BuildLedger.cpp
2026-07-06 17:13:00 +01:00
Pratik Mankawde
eec92f52e9 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-07-06 17:11:53 +01:00
Pratik Mankawde
3789ac45e2 Merge branch 'pratik/otel-phase1a-plan-docs' into pratik/otel-phase1b-telemetry-infra
# Conflicts:
#	conan.lock
#	conan/profiles/default
#	conanfile.py
2026-07-06 17:07:23 +01:00
Pratik Mankawde
b5d7e3984f docs(telemetry): add Alerting section, drop phase label, expand alerting task
- Add an operator-facing Alerting section to the runbook covering the six
  provisioned Grafana rules and how to point them at a real receiver.
- Strip the "(Phase 9)" suffix from the MetricsRegistry Gauges heading.
- ALERTING.md: drop the internal dev-phase reference from the intro.
- Phase9_taskList.md: expand Task 9.9 to describe the provisioned alerting
  (rules, contact point, policy, ALERTING.md runbook).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 16:46:14 +01:00
Pratik Mankawde
aa631f7fd4 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-07-06 16:44:27 +01:00
Pratik Mankawde
2b419338fb docs(telemetry): drop dev-phase label from Log-Trace Correlation heading
Operator-facing cleanup: strip the "(Phase 8)" suffix from the Log-Trace
Correlation section heading in the runbook.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 16:44:19 +01:00
Pratik Mankawde
2ab9fb9a54 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-07-06 16:44:10 +01:00
Pratik Mankawde
d4ee847bcc Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-07-06 16:43:52 +01:00
Pratik Mankawde
d84fdbd29f docs(telemetry): drop dev-phase labels from Ledger/Peer span headings
Same operator-facing cleanup as the RPC/transaction/consensus headings: the
runbook documents the running system, not the development phase that added
each span. Strip the "(Phase 6)" suffixes from the Ledger and Peer span
reference headings.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 16:43:40 +01:00
Pratik Mankawde
0f630d1afa Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd 2026-07-06 16:43:27 +01:00
Pratik Mankawde
22940969ff docs(telemetry): drop dev-phase labels and personal name from runbook
The operator runbook is for people running OpenTelemetry against a node, not
a record of the internal development phases. Strip the "(Phase N)" suffixes
from the span-reference headings and replace the personal-name example
service.name value with a generic one.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 16:43:15 +01:00
Pratik Mankawde
7cabf91a0d feat(telemetry): add provisioned Grafana alert rules for phase-9 metrics
Add operator-facing alerting on the health-critical metrics phase 9 emits,
completing the "alerting rules" deliverable already listed in the phase-9
task list and Jira story.

Provisioned YAML under grafana/provisioning/alerting/ (auto-loaded by the
Grafana container; no docker-compose change needed):
- rules.yaml: 6 alerts in 3 groups — consensus/ledger health, validator
  health, job queue. Each is a query(A, 5m rate/quantile by exported_instance)
  -> reduce(B, last) -> threshold(C) on the Prometheus datasource.
- contactpoints.yaml: xrpld-default webhook receiver (local-dev placeholder).
- policies.yaml: flat routing to xrpld-default, grouped by
  alertname + exported_instance.

Also adds docker/telemetry/ALERTING.md runbook (per-alert meaning, causes,
threshold tuning, how to point alerts at a real receiver) and the design spec.

Verified live: Grafana 11.5.2 provisioned all 6 rules with no rule in error
health; contact point loaded.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 16:25:15 +01:00
Pratik Mankawde
d82f76d517 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-07-04 03:01:31 +01:00
Pratik Mankawde
8c02c1595d Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-07-04 03:01:31 +01:00
Pratik Mankawde
629514dcc3 docs(telemetry): fix missed system-node-health rename in parity spec
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-04 03:01:13 +01:00
Pratik Mankawde
2f027551c9 docs(telemetry): drop redundant xrpld- prefix from dashboard uids in runbook
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-04 02:53:32 +01:00
Pratik Mankawde
85ce6467d0 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-07-04 02:47:50 +01:00
Pratik Mankawde
25868f2740 refactor(telemetry): rename xrpld-* dashboards to bare names, bare uids across all dashboards
Rename the 4 xrpld-* dashboard files to bare names and set every
dashboard uid to its bare filename stem; update doc/plan references.
Removes the redundant xrpld-/system- prefixes now that titles are clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-04 02:46:37 +01:00
Pratik Mankawde
2c590a47c5 refactor(telemetry): rename system-* dashboards to bare names, drop redundant prefix
Rename the 5 system-* dashboard files to bare domain names (uid ==
filename stem) and update all doc references. The system- prefix was
redundant now that titles no longer carry a recorder-pipeline suffix.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-04 02:45:39 +01:00
Pratik Mankawde
d5d9d1d8da refactor(telemetry): drop redundant xrpld- prefix from dashboard uids
Make dashboard uids bare domain names (uid == filename stem) and update
doc references. Grafana links/bookmarks to the old uids will need
refreshing.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-04 02:44:54 +01:00
Pratik Mankawde
93f776fda4 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-07-03 20:01:54 +01:00
Pratik Mankawde
84eb08eb8c Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-07-03 20:01:54 +01:00
Pratik Mankawde
fdfa651006 Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics
# Conflicts:
#	OpenTelemetryPlan/09-data-collection-reference.md
2026-07-03 20:01:44 +01:00
Pratik Mankawde
f1f0189c11 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd
# Conflicts:
#	docs/telemetry-runbook.md
2026-07-03 20:00:19 +01:00
Pratik Mankawde
4c46d39955 docs(telemetry): document deployment-tier filtering in runbook
Add a Deployment Tiers section covering the four filtering dimensions
(node, service, network, environment), who owns each attribute, the
collector's upsert-vs-insert rule, per-tier collector values, and how the
tier labels reach metrics. This is the operator-facing home for the
deployment-tier feature.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 19:54:42 +01:00
Ayaz Salikhov
6aed3bb71d chore: Update pre-commit hooks && actions (#7686) 2026-07-01 13:28:14 +00:00
Pratik Mankawde
fa46aa55fc Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-26 16:23:16 +01:00
Pratik Mankawde
dc3d5107fd Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-06-26 14:50:10 +01:00
Pratik Mankawde
3cfec0f1e0 Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-06-26 14:50:05 +01:00
Pratik Mankawde
ce7fe95715 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-26 14:49:43 +01:00
Pratik Mankawde
3463a85943 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-06-26 14:46:10 +01:00