Commit Graph

15328 Commits

Author SHA1 Message Date
Pratik Mankawde
b503fc921e Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-07-07 17:39:16 +01:00
Pratik Mankawde
8ea61b88f2 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-07-07 17:38:49 +01:00
Pratik Mankawde
f80c1c3b4e Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-07-07 17:38:01 +01:00
Pratik Mankawde
04f351f3b7 fix(telemetry): more phase-7 review fixes
- ValidationTracker: remove dead hard-trim Pass 1. The preceding
  eviction loop already erases every reconciled entry older than the
  late-repair cutoff using the same condition, so Pass 1 could never
  match. The "drop any reconciled entry" fallback is retained.
- integration-test.sh: probe the StatsD port with a UDP-aware check
  (ss -ulnp) instead of curl. StatsD is UDP-only on 8125, so the TCP
  curl probe always reported "refused" — a false negative. Guarded by
  a command -v ss check that logs a skip when ss is unavailable.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 16:55:19 +01:00
Pratik Mankawde
ab0d8249fd feat(telemetry): expose direct OTel metrics API via getMeter()
Give developers direct access to the full OpenTelemetry metrics API,
symmetric with getTracer(), so all seven OTel instrument types
(including UpDownCounter, sync Gauge, and the observable variants) are
reachable — not just the four beast::insight models.

- Telemetry: build and own the metrics pipeline (OTLP HTTP metric
  exporter + PeriodicExportingMetricReader + SpanMetrics histogram view)
  alongside the tracer, sharing the same resource attributes and TLS
  config. Register it globally via metrics::Provider::SetMeterProvider
  and expose Telemetry::getMeter(). Metrics enable with [telemetry];
  the metrics endpoint is derived from the trace endpoint.
- beast OTelCollector: no longer owns a pipeline. It fetches the global
  Meter, becoming a thin shim over the shared provider (legacy path
  during beast deprecation). This also resolves the review note that the
  metric exporter ignored [telemetry] use_tls — TLS now comes from the
  shared telemetry pipeline.
- Add a libxrpl unit test covering getMeter() on the enabled (global
  provider) and disabled (noop) paths, exercising an UpDownCounter.

Design: docs/superpowers/specs/2026-07-07-direct-otel-metrics-api-design.md

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 16:52:45 +01:00
Pratik Mankawde
cc4457b5a2 fix(telemetry): address phase-7 review comments
- cmake: keep the opentelemetry-cpp umbrella target for the beast metrics
  link and document why. The reviewer suggested linking individual
  component targets to avoid over-linking, but the OTel Conan package
  under-declares inter-component dependencies (the OTLP client references
  sdk::common symbols without a declared edge), so naming components
  directly reorders the static link into an unresolvable state. Verified
  by building xrpl_tests both ways.
- OTelCollector.h: add usage examples, thread-safety and limitations
  @note blocks to the class doc.
- OTelCollector.cpp: correct the @param name docs on the instrument
  Impl constructors to describe the already-formatName()'d value.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 16:50:38 +01:00
Pratik Mankawde
c6d43b9884 docs(telemetry): repoint Phase11 parity-spec links to appendix
The Phase 11 task list still linked the moved External Dashboard Parity spec
at its old docs/superpowers/ path. Point them at the appendix in
06-implementation-phases.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 13:13:41 +01:00
Pratik Mankawde
6736ce363e Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-07-07 13:13:09 +01:00
Pratik Mankawde
81d3db3a1f docs(telemetry): repoint remaining parity-spec links to appendix
The 09-data-collection-reference and Phase9 task list still linked the moved
External Dashboard Parity spec at its old docs/superpowers/ path. Point them
at the appendix in 06-implementation-phases.md where the content now lives.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 13:12:49 +01:00
Pratik Mankawde
f8ac3e38cc Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation
# Conflicts:
#	.gitignore
#	docker/telemetry/.gitignore
#	docs/superpowers/specs/2026-03-30-external-dashboard-parity-design.md
2026-07-07 13:11:28 +01:00
Pratik Mankawde
f240bb2922 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-07-07 13:09:44 +01:00
Pratik Mankawde
4abc180778 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-07-07 13:09:33 +01:00
Pratik Mankawde
ed5a618aa2 docs(telemetry): fold alerting design spec into Phase9 task list
Move the phase-9 alerting design spec out of docs/superpowers/ (which the
project guidelines say not to create) into an appendix of
OpenTelemetryPlan/Phase9_taskList.md, keeping the phase plan self-contained.
Drops the stale one-off branch/PR-restructure section from the spec.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 13:09:25 +01:00
Pratik Mankawde
134b419895 docs(telemetry): fold external-parity spec into implementation-phases doc
Move the standalone External Dashboard Parity design spec out of
docs/superpowers/ (which the project guidelines say not to create) and into
an appendix of OpenTelemetryPlan/06-implementation-phases.md, so the phase
plan is self-contained. Repoint the 10 "Source" links in the phase 3/4/7
task lists to the new appendix anchor.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 13:08:04 +01:00
Pratik Mankawde
9b3045d782 docs(telemetry): repoint alerting docs from ALERTING.md to runbook
Update the phase-9 task list and the alerting design spec to reference the
Alerting section of docs/telemetry-runbook.md, now that the standalone
ALERTING.md has been folded into the runbook.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 12:58:13 +01:00
Pratik Mankawde
4ef8a02389 feat(telemetry): route alerts to Slack + email with severity tiers
Replace the placeholder webhook contact point with two env-driven tiers:
- xrpld-default (Slack) receives warning-severity alerts
- xrpld-critical (Slack + email) receives critical-severity alerts, via a
  severity=critical child route in the notification policy

Secrets stay out of git: contactpoints.yaml references ${SLACK_WEBHOOK_URL},
${SLACK_CHANNEL}, and ${ALERT_EMAIL_TO}, which Grafana expands from a
gitignored .env.alerting (loaded via an optional env_file on the grafana
service). .env.alerting.example documents the variables; email additionally
needs GF_SMTP_* configured.

Fold the ALERTING.md runbook content into the Alerting section of
docs/telemetry-runbook.md (operator docs live in one place) and delete
ALERTING.md. Move the .env.* ignore rule into docker/telemetry/.gitignore.

Verified live: Grafana 11.5.2 provisioned both contact points, the
severity-routed policy, and all 6 rules with no errors.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 12:57:43 +01:00
Pratik Mankawde
a55b7dad54 Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation
# Conflicts:
#	OpenTelemetryPlan/presentation.md
2026-07-06 22:07:24 +01:00
Pratik Mankawde
dce39bc71d Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill
# Conflicts:
#	OpenTelemetryPlan/08-appendix.md
#	OpenTelemetryPlan/presentation.md
#	docker/telemetry/xrpld-telemetry.cfg
#	docs/telemetry-runbook.md
2026-07-06 22:06:45 +01:00
Pratik Mankawde
55052917db Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation
# Conflicts:
#	OpenTelemetryPlan/08-appendix.md
2026-07-06 22:02:14 +01:00
Pratik Mankawde
363c2caf94 Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics
# Conflicts:
#	OpenTelemetryPlan/08-appendix.md
#	OpenTelemetryPlan/09-data-collection-reference.md
#	docker/telemetry/docker-compose.yml
#	docker/telemetry/grafana/dashboards/statsd-ledger-data-sync.json
#	docker/telemetry/grafana/dashboards/statsd-network-traffic.json
#	docker/telemetry/grafana/dashboards/statsd-node-health.json
#	docker/telemetry/grafana/dashboards/statsd-rpc-pathfinding.json
2026-07-06 22:01:22 +01:00
Pratik Mankawde
f4981a1907 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd
# Conflicts:
#	OpenTelemetryPlan/08-appendix.md
#	docker/telemetry/grafana/provisioning/datasources/tempo.yaml
#	docker/telemetry/otel-collector-config.yaml
#	docs/telemetry-runbook.md
2026-07-06 21:47:17 +01:00
Pratik Mankawde
7cc963f494 fix(telemetry): phase-6 statsd dashboards, config, and doc corrections
- Fix invalid PromQL in statsd dashboards (rate/topk/nested-__name__
  label selectors); add missing $node filter on a binary-op operand
  in statsd-node-health "Ledger Publish Gap".
- Wrap cumulative counters in rate(); correct Bytes_In metric.
- Pin prometheus image; add [insight] server=statsd stanza (prefix
  rippled matches the rippled_* StatsD metric names the dashboards
  query); bind statsd port to loopback; fix Jaeger->Grafana comment.
- Correct dotted attribute keys to underscore form in
  09-data-collection-reference (peer_id, tx_hash, consensus_mode, ...).
- Reconcile span count (11) and proposers attribute in the
  integration-test task list.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 21:36:59 +01:00
Pratik Mankawde
1938b9e377 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-07-06 21:05:06 +01:00
Pratik Mankawde
04d4bba780 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-07-06 21:04:57 +01:00
Pratik Mankawde
5287893d82 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing 2026-07-06 21:04:50 +01:00
Pratik Mankawde
b7d0b700c6 fix(telemetry): correct runbook pathfind spans + mTLS validation
- Add PathFinding Spans section to the runbook Span Reference and
  remove the incorrect "not emitted / later-phase" annotations:
  pathfind.request/compute/discover/update_all are emitted since
  Phase 2 (upstream of Phase 5).
- Runbook label names use the bare spanmetrics dimensions
  (command, rpc_status, local, consensus_mode); fix stale docs.
- 05-configuration-reference: note tls_client_cert/key require use_tls=1.
- TelemetryConfig: reject mTLS cert paths when use_tls=0 so telemetry
  is not silently sent in plaintext.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 21:04:07 +01:00
Pratik Mankawde
89b5210e5b docs(telemetry): clarify redactAccount is obfuscation, not secrecy
Soften the "non-reversible" wording in Redaction.h. XRP account
addresses are a public, enumerable set, so an unsalted hash is
obfuscation (keeps plaintext out of traces/dashboards), not a
secrecy guarantee. Document why a salt is intentionally omitted:
it would break cross-node/restart correlation, the reason for
hashing rather than dropping the attribute.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 20:54:09 +01:00
Pratik Mankawde
ea9c639ee9 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-07-06 20:50:24 +01:00
Pratik Mankawde
7998d01dc9 code review comments.
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-07-06 20:49:51 +01:00
Pratik Mankawde
e02b7968ad Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-07-06 20:41:14 +01:00
Pratik Mankawde
38f6afbf4c code review comments
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-07-06 20:40:12 +01:00
Pratik Mankawde
092afb7aae Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-07-06 20:39:27 +01:00
Pratik Mankawde
7a6ff619e5 code review changes
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-07-06 20:37:08 +01:00
Pratik Mankawde
60a2ac7bfe fix(telemetry): bound RPC error-path span cardinality; balance LCOV markers
Resolve the doCommand error-path span suffix and command attribute against
the handler registry instead of using raw request input. A recognized command
yields its canonical handler name; anything else (unregistered command, or
neither command/method present) collapses to the literal "unknown". This bounds
telemetry cardinality to the finite set of handler names plus "unknown" while
preserving per-command error attribution for known-but-rejected commands
(e.g. a submit that hit rpcTOO_BUSY stays rpc.command.submit). The command
attribute is promoted to a Prometheus label by the spanmetrics connector, so
attacker-controlled input previously enabled unbounded label cardinality.

Also add the missing LCOV_EXCL_STOP in the first SpanGuard::linkedSpan overload
so its coverage exclusion no longer bleeds into the second overload's guards.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 19:17:34 +01:00
Pratik Mankawde
5ee8ee71fd Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration 2026-07-06 18:16:10 +01:00
Pratik Mankawde
3b08c2d90a Merge branch 'pratik/otel-phase1a-plan-docs' into pratik/otel-phase1b-telemetry-infra 2026-07-06 18:15:41 +01:00
Pratik Mankawde
26a85c764e code review comment addressed
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-07-06 18:15:23 +01:00
Pratik Mankawde
0505ca35aa fix(telemetry): make Tempo search filters dynamic
Change the six remaining `type: static` search filters in tempo.yaml to
`type: dynamic` so Grafana queries Tempo for available tag values instead
of requiring manual free-text entry. Aligns service-name, node-id,
network-type, span-name, span-status, and span-duration with the existing
node-version and network-id filters.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 17:41:38 +01:00
Pratik Mankawde
ae37de4b40 formatting update
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-07-06 17:25:06 +01:00
Pratik Mankawde
369cd96878 Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-07-06 17:17:44 +01:00
Pratik Mankawde
206d386469 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill
# Conflicts:
#	src/xrpld/app/ledger/InboundLedger.h
#	src/xrpld/perflog/detail/PerfLogImp.cpp
2026-07-06 17:17:21 +01:00
Pratik Mankawde
fe88034ea9 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-07-06 17:15:46 +01:00
Pratik Mankawde
fd518d3e7b Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics
# Conflicts:
#	.github/scripts/levelization/results/ordering.txt
#	include/xrpl/beast/insight/Insight.h
#	src/test/csf/Peer.h
2026-07-06 17:15:27 +01:00
Pratik Mankawde
d5b858ce9d Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd
# Conflicts:
#	.cspell.config.yaml
#	src/xrpld/app/ledger/detail/BuildLedger.cpp
2026-07-06 17:13:00 +01:00
Pratik Mankawde
eec92f52e9 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-07-06 17:11:53 +01:00
Pratik Mankawde
e969b0c6ed Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing
# Conflicts:
#	src/test/csf/Peer.h
#	src/xrpld/app/consensus/RCLConsensus.h
2026-07-06 17:11:29 +01:00
Pratik Mankawde
33c17e50b3 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing 2026-07-06 17:09:47 +01:00
Pratik Mankawde
e129b0793c Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing
# Conflicts:
#	src/tests/libxrpl/CMakeLists.txt
2026-07-06 17:09:33 +01:00
Pratik Mankawde
45aa2f276a Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration
# Conflicts:
#	.github/scripts/levelization/results/loops.txt
2026-07-06 17:08:46 +01:00
Pratik Mankawde
3789ac45e2 Merge branch 'pratik/otel-phase1a-plan-docs' into pratik/otel-phase1b-telemetry-infra
# Conflicts:
#	conan.lock
#	conan/profiles/default
#	conanfile.py
2026-07-06 17:07:23 +01:00