Commit Graph

280 Commits

Author SHA1 Message Date
Pratik Mankawde
f99a1c5310 fix(telemetry): consolidate xrpld log mount to a single rootless path
The collector's log ingestion used a container path named after the old
`rippled` binary and defaulted its host mount to a location that either
needed root or diverged from where the telemetry configs actually write.

Consolidate on one accessible log root:
- container mount target renamed /var/log/rippled -> /var/log/xrpld
- filelog glob -> /var/log/xrpld/*/debug.log
- compose mount source defaults to the repo-relative ./data/logs
  (user-owned, no root), overridable via XRPLD_LOG_DIR
- devnet telemetry cfg writes to data/logs/devnet/debug.log so it lands
  one subdir below the mount root and matches the glob
- integration-test.sh sets XRPLD_LOG_DIR to its own workdir
- docs/runbook/task-list updated to match

The default xrpld config (cfg/xrpld-example.cfg) is intentionally left
untouched.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 20:54:53 +01:00
Pratik Mankawde
9e5f1caff2 docs(telemetry): address review feedback on phase-8 log correlation
- Correct log-timestamp examples to real Logs::format() output
  (2024-Jan-15 10:30:45.123456 UTC) in docs, TESTING.md, and reference.
- Update Loki to v3.4.2 and switch the collector exporter to
  otlphttp/loki on the native /otlp endpoint (docs + task list).
- Make the collector log-mount path configurable via XRPLD_LOG_DIR.
- Remove implementation-phase references from shipped config/script
  comments, keeping the functional descriptions.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 19:55:26 +01:00
Pratik Mankawde
4abc180778 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-07-07 13:09:33 +01:00
Pratik Mankawde
134b419895 docs(telemetry): fold external-parity spec into implementation-phases doc
Move the standalone External Dashboard Parity design spec out of
docs/superpowers/ (which the project guidelines say not to create) and into
an appendix of OpenTelemetryPlan/06-implementation-phases.md, so the phase
plan is self-contained. Repoint the 10 "Source" links in the phase 3/4/7
task lists to the new appendix anchor.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 13:08:04 +01:00
Pratik Mankawde
55052917db Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation
# Conflicts:
#	OpenTelemetryPlan/08-appendix.md
2026-07-06 22:02:14 +01:00
Pratik Mankawde
363c2caf94 Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics
# Conflicts:
#	OpenTelemetryPlan/08-appendix.md
#	OpenTelemetryPlan/09-data-collection-reference.md
#	docker/telemetry/docker-compose.yml
#	docker/telemetry/grafana/dashboards/statsd-ledger-data-sync.json
#	docker/telemetry/grafana/dashboards/statsd-network-traffic.json
#	docker/telemetry/grafana/dashboards/statsd-node-health.json
#	docker/telemetry/grafana/dashboards/statsd-rpc-pathfinding.json
2026-07-06 22:01:22 +01:00
Pratik Mankawde
f4981a1907 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd
# Conflicts:
#	OpenTelemetryPlan/08-appendix.md
#	docker/telemetry/grafana/provisioning/datasources/tempo.yaml
#	docker/telemetry/otel-collector-config.yaml
#	docs/telemetry-runbook.md
2026-07-06 21:47:17 +01:00
Pratik Mankawde
7cc963f494 fix(telemetry): phase-6 statsd dashboards, config, and doc corrections
- Fix invalid PromQL in statsd dashboards (rate/topk/nested-__name__
  label selectors); add missing $node filter on a binary-op operand
  in statsd-node-health "Ledger Publish Gap".
- Wrap cumulative counters in rate(); correct Bytes_In metric.
- Pin prometheus image; add [insight] server=statsd stanza (prefix
  rippled matches the rippled_* StatsD metric names the dashboards
  query); bind statsd port to loopback; fix Jaeger->Grafana comment.
- Correct dotted attribute keys to underscore form in
  09-data-collection-reference (peer_id, tx_hash, consensus_mode, ...).
- Reconcile span count (11) and proposers attribute in the
  integration-test task list.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 21:36:59 +01:00
Pratik Mankawde
1938b9e377 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-07-06 21:05:06 +01:00
Pratik Mankawde
b7d0b700c6 fix(telemetry): correct runbook pathfind spans + mTLS validation
- Add PathFinding Spans section to the runbook Span Reference and
  remove the incorrect "not emitted / later-phase" annotations:
  pathfind.request/compute/discover/update_all are emitted since
  Phase 2 (upstream of Phase 5).
- Runbook label names use the bare spanmetrics dimensions
  (command, rpc_status, local, consensus_mode); fix stale docs.
- 05-configuration-reference: note tls_client_cert/key require use_tls=1.
- TelemetryConfig: reject mTLS cert paths when use_tls=0 so telemetry
  is not silently sent in plaintext.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 21:04:07 +01:00
Pratik Mankawde
ea9c639ee9 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-07-06 20:50:24 +01:00
Pratik Mankawde
7998d01dc9 code review comments.
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-07-06 20:49:51 +01:00
Pratik Mankawde
e02b7968ad Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-07-06 20:41:14 +01:00
Pratik Mankawde
38f6afbf4c code review comments
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-07-06 20:40:12 +01:00
Pratik Mankawde
092afb7aae Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-07-06 20:39:27 +01:00
Pratik Mankawde
7a6ff619e5 code review changes
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-07-06 20:37:08 +01:00
Pratik Mankawde
3b08c2d90a Merge branch 'pratik/otel-phase1a-plan-docs' into pratik/otel-phase1b-telemetry-infra 2026-07-06 18:15:41 +01:00
Pratik Mankawde
26a85c764e code review comment addressed
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-07-06 18:15:23 +01:00
Pratik Mankawde
85ce6467d0 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-07-04 02:47:50 +01:00
Pratik Mankawde
2c590a47c5 refactor(telemetry): rename system-* dashboards to bare names, drop redundant prefix
Rename the 5 system-* dashboard files to bare domain names (uid ==
filename stem) and update all doc references. The system- prefix was
redundant now that titles no longer carry a recorder-pipeline suffix.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-04 02:45:39 +01:00
Pratik Mankawde
d5d9d1d8da refactor(telemetry): drop redundant xrpld- prefix from dashboard uids
Make dashboard uids bare domain names (uid == filename stem) and update
doc references. Grafana links/bookmarks to the old uids will need
refreshing.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-04 02:44:54 +01:00
Pratik Mankawde
2a90b582ff Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-07-03 22:29:18 +01:00
Pratik Mankawde
6961727ba7 docs(telemetry): drop (System Metrics) suffix from dashboard names in reference
Match the reader-facing dashboard catalog to the actual dashboard titles,
which no longer carry the recorder-pipeline suffix.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 22:28:56 +01:00
Pratik Mankawde
93f776fda4 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-07-03 20:01:54 +01:00
Pratik Mankawde
fdfa651006 Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics
# Conflicts:
#	OpenTelemetryPlan/09-data-collection-reference.md
2026-07-03 20:01:44 +01:00
Pratik Mankawde
f1f0189c11 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd
# Conflicts:
#	docs/telemetry-runbook.md
2026-07-03 20:00:19 +01:00
Pratik Mankawde
1ed08d09d0 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-07-03 19:59:01 +01:00
Pratik Mankawde
8e4ab5bd04 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-07-03 19:59:01 +01:00
Pratik Mankawde
73edc2dd58 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing
# Conflicts:
#	OpenTelemetryPlan/05-configuration-reference.md
2026-07-03 19:58:52 +01:00
Pratik Mankawde
a2e8f3a6b2 Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing 2026-07-03 19:57:45 +01:00
Pratik Mankawde
6e9d71c5eb Merge branch 'pratik/otel-phase1a-plan-docs' into pratik/otel-phase1b-telemetry-infra 2026-07-03 19:57:45 +01:00
Pratik Mankawde
f81919af05 docs(telemetry): document tier template variables and label checks
- 09-data-collection-reference: add $service_name, $deployment_environment,
  $xrpl_network_type to the template-variable table; note they appear on all
  dashboards and point to the runbook's Deployment Tiers section.
- TESTING.md: add a Prometheus check verifying the tier labels
  (deployment_environment, xrpl_network_type, service_name) are present on
  metric series.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 19:56:15 +01:00
Pratik Mankawde
d2c7a00584 docs(telemetry): note service.name on metrics and collector tier tagging
- 05-configuration-reference: service_name now applies to metrics as well
  as traces (service.name resource attribute).
- 07-observability-backends: document that collector enrichment includes
  deployment-tier tagging (deployment.environment + fallback
  xrpl.network.type) for filtering one Grafana stack by tier.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 19:53:54 +01:00
Pratik Mankawde
2668e9f8ca Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-06-26 16:31:37 +01:00
Pratik Mankawde
0ce13ff2e7 Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-06-26 16:31:37 +01:00
Pratik Mankawde
394a64f437 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd 2026-06-26 16:31:37 +01:00
Pratik Mankawde
ea2e6787c6 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-06-26 16:31:37 +01:00
Pratik Mankawde
b3c5f8860c Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing 2026-06-26 16:31:37 +01:00
Pratik Mankawde
f48b212a1c Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing 2026-06-26 16:31:37 +01:00
Pratik Mankawde
0f1bd86ba2 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-06-26 16:31:37 +01:00
Pratik Mankawde
cf629e2a76 Merge branch 'pratik/otel-phase1a-plan-docs' into pratik/otel-phase1b-telemetry-infra 2026-06-26 16:31:36 +01:00
Pratik Mankawde
190b9470a4 docs(telemetry): fix xrpl.network.type value (standalone → unknown)
The network-type label is derived from [network_id] in TelemetryConfig.cpp;
unmapped/unset IDs fall through to "unknown", not "standalone". Align the
design-doc resource-attribute table with the code and cfg example.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 16:31:06 +01:00
Pratik Mankawde
dc3d5107fd Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-06-26 14:50:10 +01:00
Pratik Mankawde
3cfec0f1e0 Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-06-26 14:50:05 +01:00
Pratik Mankawde
ce7fe95715 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-26 14:49:43 +01:00
Pratik Mankawde
3463a85943 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-06-26 14:46:10 +01:00
Pratik Mankawde
bc7cff230c Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-26 14:45:43 +01:00
Pratik Mankawde
54d92ff973 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-26 14:42:55 +01:00
Pratik Mankawde
9a2d682a42 Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing 2026-06-26 14:33:16 +01:00
Pratik Mankawde
0ff947454c removed line numbers and cleanup
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-12 20:33:42 +01:00