Commit Graph

35 Commits

Author SHA1 Message Date
Pratik Mankawde
829cad81af docs(telemetry): update metric names across docs, tests, alert rules, and plan docs
Strip xrpld_ prefix, lowercase beast::insight names, and replace
traces_span_metrics_ with span_ in all remaining tracked files:
alert rules, integration tests, workload validation, TESTING.md,
OpenTelemetryPlan docs, code comments, and config templates.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-08 18:01:53 +01:00
Pratik Mankawde
dcb50cdfb3 Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-07-07 20:58:32 +01:00
Pratik Mankawde
f99a1c5310 fix(telemetry): consolidate xrpld log mount to a single rootless path
The collector's log ingestion used a container path named after the old
`rippled` binary and defaulted its host mount to a location that either
needed root or diverged from where the telemetry configs actually write.

Consolidate on one accessible log root:
- container mount target renamed /var/log/rippled -> /var/log/xrpld
- filelog glob -> /var/log/xrpld/*/debug.log
- compose mount source defaults to the repo-relative ./data/logs
  (user-owned, no root), overridable via XRPLD_LOG_DIR
- devnet telemetry cfg writes to data/logs/devnet/debug.log so it lands
  one subdir below the mount root and matches the glob
- integration-test.sh sets XRPLD_LOG_DIR to its own workdir
- docs/runbook/task-list updated to match

The default xrpld config (cfg/xrpld-example.cfg) is intentionally left
untouched.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 20:54:53 +01:00
Pratik Mankawde
53b1751384 Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-07-07 20:00:54 +01:00
Pratik Mankawde
9e5f1caff2 docs(telemetry): address review feedback on phase-8 log correlation
- Correct log-timestamp examples to real Logs::format() output
  (2024-Jan-15 10:30:45.123456 UTC) in docs, TESTING.md, and reference.
- Update Loki to v3.4.2 and switch the collector exporter to
  otlphttp/loki on the native /otlp endpoint (docs + task list).
- Make the collector log-mount path configurable via XRPLD_LOG_DIR.
- Remove implementation-phase references from shipped config/script
  comments, keeping the functional descriptions.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 19:55:26 +01:00
Pratik Mankawde
4e566949da Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation
# Conflicts:
#	docker/telemetry/TESTING.md
2026-07-03 20:02:37 +01:00
Pratik Mankawde
93f776fda4 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-07-03 20:01:54 +01:00
Pratik Mankawde
f81919af05 docs(telemetry): document tier template variables and label checks
- 09-data-collection-reference: add $service_name, $deployment_environment,
  $xrpl_network_type to the template-variable table; note they appear on all
  dashboards and point to the runbook's Deployment Tiers section.
- TESTING.md: add a Prometheus check verifying the tier labels
  (deployment_environment, xrpl_network_type, service_name) are present on
  metric series.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 19:56:15 +01:00
Pratik Mankawde
6e71ee1888 feat(telemetry): add runtime-selectable Grafana Cloud export
Add an opt-in collector config + compose override that forward traces,
metrics, and logs to a Grafana Cloud OTLP gateway alongside the local
backends. Enabling it is a runtime choice (layer the override compose
file) — no xrpld rebuild, base stack unchanged. Credentials come from a
gitignored .env file. Documented in the telemetry runbook and TESTING.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 17:34:29 +01:00
Pratik Mankawde
81919612f8 Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-06-10 10:22:07 +01:00
Pratik Mankawde
21c177c8f9 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-06-10 10:20:20 +01:00
Pratik Mankawde
55a674f042 docs(telemetry): remove sampling_ratio knob from phase-6 docs and configs
Head sampling is fixed at 1.0 and not configurable. Drop sampling_ratio
from the docker sample/test configs and the data-collection reference;
rewrite the span-metric sampling caveat and volume guidance to point at
collector-side tail sampling.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 10:03:31 +01:00
Pratik Mankawde
010b0d1bd6 Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation
# Conflicts:
#	src/tests/libxrpl/CMakeLists.txt
2026-06-09 16:46:24 +01:00
Pratik Mankawde
3e4b1eb60f Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-06-09 16:43:45 +01:00
Pratik Mankawde
bbabe8d8a7 docs(telemetry): enable peer tracing in docker sample configs
Update the docker/telemetry sample config and testing guide to set
trace_peer=1, matching the new on-by-default behavior and the other
trace categories already enabled in these all-categories sample profiles.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 16:40:21 +01:00
Pratik Mankawde
98fc939851 Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-01 15:01:19 +01:00
Pratik Mankawde
ba7e1f98e4 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-05-29 18:24:43 +01:00
Pratik Mankawde
d7579b2861 formatting changes
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-05-29 18:21:00 +01:00
Pratik Mankawde
088848e7ab formatting updates
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-05-29 18:20:08 +01:00
Pratik Mankawde
ac57a91b77 merge: phase-9 (dashboard UID + line-number cleanup, detach callbacks) into phase-10
# Conflicts:
#	docker/telemetry/TESTING.md
2026-05-14 17:23:55 +01:00
Pratik Mankawde
0e5e802e5e merge: pratik/otel-phase7-native-metrics (dashboard UID + line-number cleanup) into pratik/otel-phase8-log-correlation 2026-05-14 17:07:34 +01:00
Pratik Mankawde
a789f6ccf5 docs(telemetry): fix stale rpc.request refs + drop unparsed exporter key in TESTING.md
Follow-up to the dashboard cleanup on this branch. Caught additional sites
in TESTING.md that still reference the never-emitted `rpc.request` span:

- TraceQL query examples in Step 5 "Verify traces in Tempo" now filter on
  `name="rpc.http_request"` (the real emitted name).
- Expected-spans table replaces `rpc.request` with `rpc.http_request`.
- Query loop under the Prometheus verification section now iterates over
  the full set of emitted RPC entry-point names
  (`rpc.http_request`, `rpc.ws_upgrade`, `rpc.ws_message`, `rpc.process`).

Also drop `exporter=otlp_http` from the sample telemetry config block.
`TelemetryConfig.cpp` does not parse an `exporter` key in any phase through
Phase 8; only OTLP/HTTP is wired up, so the line is either a silently
ignored no-op or misleading documentation.
2026-05-14 16:53:40 +01:00
Pratik Mankawde
44cdc8133e fix(telemetry): phase-6 dashboards — rename UIDs, add $node filter, drop line numbers
Phase-6 introduces ledger-operations, peer-network, and the five StatsD
dashboards. Align them with the rest of the chain:

- Rename dashboard UIDs from `rippled-*` to `xrpld-*` so the provisioned
  UIDs match the post-rename-script documentation (`docs.sh` rewrites
  .md but not .json, so the two drifted). Runbook references
  `xrpld-rpc-perf`, `xrpld-transactions`, etc., now the JSON matches.
- Add the `$node` template variable + `exported_instance=~"$node"` filter
  to every target in the five `statsd-*` dashboards. Mirrors the pattern
  already used by consensus-health, ledger-operations, and peer-network
  per the project rule that every dashboard must support per-node
  filtering.
- Strip `:<line>` (and `:NN-NN` range) suffixes from C++ file references
  in every dashboard panel description and in docker/telemetry/TESTING.md.
  Line numbers drift on every refactor; the filename alone is enough to
  grep.
- Replace stale `rpc.request` entries with the real emitted span names
  (`rpc.http_request`, `rpc.ws_upgrade`, `rpc.ws_message`, `rpc.process`)
  in TESTING.md so operators can copy-paste the filters and hit real
  traces.
- Also drop the `:706` line ref from the `StatsDCollector.cpp` callout
  in `06-implementation-phases.md`.
2026-05-14 16:51:14 +01:00
Pratik Mankawde
815e2b1f5d refactor(telemetry): fix remaining old attr refs in tests, docs, workload
- Update Telemetry.h doc example: xrpl.rpc.command -> command.
- Update SpanGuardFactory.cpp test: use new bare attr names.
- Update TESTING.md: rename attr refs in span table + PromQL example.
- Update expected_spans.json: all attrs match simplified naming.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-13 16:21:18 +01:00
Pratik Mankawde
a8549a7ab2 fix(telemetry): address code review findings for Phase 8 log-trace correlation
- Replace GetSpan() with direct context value check in Logs::format()
  to avoid heap allocation (new DefaultSpan) on the no-span path
- Restore Phase 7 documentation accidentally deleted during merge
- Fix undefined $JAEGER variable → use $TEMPO in integration test
- Remove useless LCOV_EXCL markers around #ifdef block
- Fix indentation inconsistencies in Log.cpp injection block
- Remove incorrect url field from loki.yaml derivedFields
- Update stale code sample in Phase8_taskList.md to match implementation
- Correct "<10ns" performance claims to accurate ~15-20ns (no-span)
  and ~50ns (active-span) measurements across all docs
- Replace Jaeger references with Tempo in TESTING.md (port 16686→3200)
- Improve error handling in check_log_correlation(): track files_scanned,
  detect missing log files, fix silent grep error masking

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-06 14:32:46 +01:00
Pratik Mankawde
70d86d7ebf Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation
# Conflicts:
#	OpenTelemetryPlan/06-implementation-phases.md
#	OpenTelemetryPlan/09-data-collection-reference.md
#	OpenTelemetryPlan/OpenTelemetryPlan.md
#	docker/telemetry/docker-compose.yml
#	docker/telemetry/grafana/dashboards/statsd-network-traffic.json
#	docker/telemetry/otel-collector-config.yaml
#	src/xrpld/overlay/detail/PeerImp.cpp
2026-04-29 20:38:00 +01:00
Pratik Mankawde
7ab6f4d34b fix: address CI rename checks (rippled -> xrpld) in phase-8 docs
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-29 20:09:43 +01:00
Pratik Mankawde
81b47afde7 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation
# Conflicts:
#	OpenTelemetryPlan/06-implementation-phases.md
#	OpenTelemetryPlan/08-appendix.md
#	OpenTelemetryPlan/OpenTelemetryPlan.md
#	docker/telemetry/grafana/dashboards/statsd-network-traffic.json
#	docker/telemetry/grafana/dashboards/statsd-node-health.json
#	docker/telemetry/grafana/dashboards/statsd-rpc-pathfinding.json
2026-04-29 20:07:43 +01:00
Pratik Mankawde
cbbd6ebee2 feat(telemetry): add Phase 6 StatsD metrics, ledger/peer spans, and expanded dashboards
Integrate the existing StatsD metrics pipeline (beast::insight) into
the OpenTelemetry observability stack and add new trace spans for
ledger build/store/validate and peer proposal/validation receive.

Phase 5b — Ledger, peer, and transaction spans:
- Add ledger.build span with close time attributes in BuildLedger.cpp
- Add tx.apply span with tx_count/tx_failed in BuildLedger.cpp
- Add ledger.store and ledger.validate spans in LedgerMaster.cpp
- Add peer.proposal.receive span with trusted attribute in PeerImp.cpp
- Add peer.validation.receive span with ledger_hash, full, trusted
  attributes in PeerImp.cpp
- Add ledger-operations and peer-network Grafana dashboards

Phase 6 — StatsD metrics integration:
- Add StatsD UDP receiver (port 8125) to OTel Collector
- Add 5 StatsD Grafana dashboards: node health, network traffic,
  overlay traffic detail, ledger data sync, RPC pathfinding
- Add 09-data-collection-reference.md cataloging all metrics/spans
- Update existing dashboards with new span panels
- Expand telemetry runbook and integration test script
- Add codecov exclusions for telemetry modules

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-28 15:00:57 +01:00
Pratik Mankawde
de7194011d fix(docs): apply rename scripts to telemetry deployment docs
Run .github/scripts/rename/docs.sh to replace rippled → xrpld
references in TESTING.md, xrpld-telemetry.cfg, and
telemetry-runbook.md, fixing the check-rename CI failure.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-28 15:00:40 +01:00
Pratik Mankawde
f6105ece98 feat(telemetry): add Phase 5 documentation, deployment configs, and integration tests
Add the observability stack deployment infrastructure and integration
test framework for verifying end-to-end trace export.

- Add Grafana dashboards: RPC performance, transaction overview,
  consensus health (pre-provisioned via dashboards.yaml)
- Add Prometheus config for spanmetrics collection from OTel Collector
- Update OTel Collector config with spanmetrics connector and
  prometheus exporter for RED metrics
- Add docker-compose services: prometheus, dashboard provisioning
- Add integration-test.sh with Tempo API-based span verification
  (replaces previous Jaeger-based approach)
- Add TESTING.md with step-by-step deployment and verification guide
- Add telemetry-runbook.md for production operations reference
- Add xrpld-telemetry.cfg sample configuration
- Add toDisplayString() for ConsensusMode (human-readable span values)
- Update Phase 2/3 task lists with known issues sections
- Add Phase 5 integration test task list
- Add TraceContext protobuf fields for future relay propagation
- Wire telemetry lifecycle (setServiceInstanceId/start/stop) in
  Application.cpp

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-28 15:00:40 +01:00
Pratik Mankawde
7e149f7773 refactor(telemetry): remove residual Jaeger references across chain
Fix remaining Jaeger references that accumulated across intermediate
branches in the stacked PR chain. These were in files modified by
multiple phases where the per-branch fixes didn't cover all additions.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 22:35:04 +01:00
Pratik Mankawde
fdec3ce5c4 Phase 8: Log-trace correlation with Loki and filelog receiver
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 22:31:37 +01:00
Pratik Mankawde
87ed778efe refactor(telemetry): migrate integration test and docs from Jaeger to Tempo API
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 22:29:30 +01:00
Pratik Mankawde
f940290866 Phase 5: Documentation, deployment configs, integration test infrastructure
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 22:29:30 +01:00