Commit Graph

15386 Commits

Author SHA1 Message Date
Pratik Mankawde
809d7fcb47 fix(telemetry): raise TX error threshold for CI workload validation
Raise TX error rate threshold from 50% to 95%. Short-lived CI test
environments lack pre-funded accounts for complex transactions
(AMMCreate, EscrowFinish, NFTokenCreateOffer, etc.), causing expected
failures that do not indicate an instrumentation problem.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-08 15:37:21 +01:00
Pratik Mankawde
c835ad528f Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-07-08 15:36:55 +01:00
Pratik Mankawde
0ee6da59be fix(telemetry): resolve clang-tidy include and modernize-use-auto CI errors
- Replace unused observer_result.h with nostd/shared_ptr.h and
  nostd/unique_ptr.h in MetricsRegistry.h (misc-include-cleaner)
- Use auto for cast initializations in MetricsRegistry.cpp and
  ValidationTracker test (modernize-use-auto)
- Run rename scripts: rippled -> xrpld in comments and docs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-08 15:36:44 +01:00
Pratik Mankawde
1730cdb50d Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill
# Conflicts:
#	docker/telemetry/xrpld-telemetry.cfg
2026-07-08 15:26:40 +01:00
Pratik Mankawde
a681eea6c4 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-07-08 15:26:06 +01:00
Pratik Mankawde
0a373f631c fix(telemetry): resolve otel-naming and UBSAN CI failures
- Add job_type to otel-naming builtins (standard Prometheus label for
  job-queue metrics used in node-health dashboard)
- Add OpenTelemetry SDK to UBSAN suppressions (intentional unsigned
  integer overflow in attributemap_hash.h hash computation)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-08 15:25:59 +01:00
Pratik Mankawde
a1a9d45bff Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-07-08 15:22:52 +01:00
Pratik Mankawde
4826038068 fix(telemetry): run rename scripts for CI compliance
Rename rippled → xrpld in statsd config prefix.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-08 15:22:45 +01:00
Pratik Mankawde
45d000cd27 Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-07-08 14:57:54 +01:00
Pratik Mankawde
d24534a340 docs(telemetry): update Phase9 task list to use service_instance_id
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-08 14:57:46 +01:00
Pratik Mankawde
a13535fed7 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-07-08 14:57:41 +01:00
Pratik Mankawde
c75de2d58c Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-07-08 14:57:37 +01:00
Pratik Mankawde
a2a58a5d85 docs(telemetry): update plan docs to use service_instance_id
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-08 14:57:29 +01:00
Pratik Mankawde
e64afb53d0 fix(telemetry): use service_instance_id in Grafana Cloud transform and docs
Update the transform/cloudlabels processor to set service_instance_id
instead of exported_instance, matching the local Prometheus path where
resource_to_telemetry_conversion promotes service.instance.id with that
exact label name. Also update source comments and documentation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-08 14:55:15 +01:00
Pratik Mankawde
f5c3f64fad Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-07-08 14:53:02 +01:00
Pratik Mankawde
21acd060c8 fix(telemetry): use service_instance_id label instead of exported_instance
Merge forward from phase6/7 and apply the same fix to phase9's own
dashboards and alerting rules. exported_instance is a Prometheus scrape
artifact absent in push-based pipelines; service_instance_id is always
present via resource_to_telemetry_conversion.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-08 14:52:55 +01:00
Pratik Mankawde
a3053a34a7 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-07-08 14:51:55 +01:00
Pratik Mankawde
87f370a64d fix(telemetry): use service_instance_id label instead of exported_instance
exported_instance is a Prometheus scrape artifact that only exists when
Prometheus scrapes a target with a conflicting instance label. Users
running push-based pipelines (remote_write, OTLP) never get this label.
service_instance_id is always present via resource_to_telemetry_conversion
regardless of ingestion path.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-08 14:51:48 +01:00
Pratik Mankawde
ff92c3727c Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-07-08 14:51:36 +01:00
Pratik Mankawde
1eb44689d0 fix(telemetry): use service_instance_id label instead of exported_instance
exported_instance is a Prometheus scrape artifact that only exists when
Prometheus scrapes a target with a conflicting instance label. Users
running push-based pipelines (remote_write, OTLP) never get this label.
service_instance_id is always present via resource_to_telemetry_conversion
regardless of ingestion path.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-08 14:51:26 +01:00
Pratik Mankawde
e7f9685e0e fix(telemetry): poll Prometheus for late-populating metrics in validation
The workload validator queried each expected metric once, immediately after
a fixed post-workload propagation wait. Several beast::insight metrics
(ledger-age and peer-finder gauges, overlay-traffic and rpc-request counters)
only populate after the node validates ledgers and sustains peer traffic,
then travel a 1s OTLP export + 15s Prometheus scrape before they are
queryable. On a slower CI runner that pipeline can settle after the wait
ends, so the single query raced and reported "0 series", failing 12 checks
that pass locally with the same config and binary.

Poll each metric on the /api/v1/series endpoint until it appears or a 45s
window (two scrape cycles) elapses. Present metrics still return on the first
query with no added delay; a genuinely-absent metric still fails after the
timeout. Makes the check robust to runner speed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 12:02:42 +01:00
Pratik Mankawde
5997f1bbe8 Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-07-07 21:34:36 +01:00
Pratik Mankawde
f8fbcc2a2e Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill
# Conflicts:
#	docker/telemetry/xrpld-telemetry.cfg
2026-07-07 21:34:13 +01:00
Pratik Mankawde
e4a78684e1 fix(telemetry): make devnet debug_logfile config-dir-relative
xrpld resolves a relative [debug_logfile] against the config file's own
directory (docker/telemetry), not the working directory. The value
`docker/telemetry/data/logs/devnet/debug.log` therefore doubled the
prefix to docker/telemetry/docker/telemetry/... — the collector's mount
never saw the log, so no lines reached Loki.

Use the config-dir-relative form `data/logs/devnet/debug.log`, which
resolves to docker/telemetry/data/logs/devnet/debug.log — exactly the
dir the compose stack bind-mounts as /var/log/xrpld.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 21:31:59 +01:00
Pratik Mankawde
a381027bb1 fix(telemetry): apply single rootless log path to workload stack
Extend the log-mount consolidation to the phase-10 workload-validation
assets, which still used the old `rippled` container path:
- docker-compose.workload.yaml: mount target renamed to /var/log/xrpld,
  source made overridable via XRPLD_LOG_DIR (default /tmp/xrpld-validation)
- otel-collector-config.grafanacloud.yaml: single glob /var/log/xrpld/*/debug.log
- run-full-validation.sh sets XRPLD_LOG_DIR to its workdir

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 21:01:11 +01:00
Pratik Mankawde
dcb50cdfb3 Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-07-07 20:58:32 +01:00
Pratik Mankawde
e3188db2b3 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill
# Conflicts:
#	docker/telemetry/docker-compose.yml
#	docker/telemetry/otel-collector-config.yaml
#	docker/telemetry/xrpld-telemetry.cfg
2026-07-07 20:57:52 +01:00
Pratik Mankawde
f99a1c5310 fix(telemetry): consolidate xrpld log mount to a single rootless path
The collector's log ingestion used a container path named after the old
`rippled` binary and defaulted its host mount to a location that either
needed root or diverged from where the telemetry configs actually write.

Consolidate on one accessible log root:
- container mount target renamed /var/log/rippled -> /var/log/xrpld
- filelog glob -> /var/log/xrpld/*/debug.log
- compose mount source defaults to the repo-relative ./data/logs
  (user-owned, no root), overridable via XRPLD_LOG_DIR
- devnet telemetry cfg writes to data/logs/devnet/debug.log so it lands
  one subdir below the mount root and matches the glob
- integration-test.sh sets XRPLD_LOG_DIR to its own workdir
- docs/runbook/task-list updated to match

The default xrpld config (cfg/xrpld-example.cfg) is intentionally left
untouched.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 20:54:53 +01:00
Pratik Mankawde
15f9c66dcf fix(telemetry): address phase-9 metric gap-fill review comments
- PerfLogImp::rpcEnd(): return after the requestId-not-found UNREACHABLE
  so a stale (now - epoch) duration is no longer recorded to the counter
  and histogram in release builds.
- MetricsRegistry peer-version gauge: compare versions numerically via
  BuildInfo::encodeSoftwareVersion() instead of a lexicographic string
  compare, stripping the non-digit prefix so peer 'rippled-X.Y.Z' lines
  up with our bare 'X.Y.Z'. Fixes every peer counting as higher-version.
- MetricsRegistry::stop(): call Shutdown() before ForceFlush() before
  reset() so the reader thread stops before teardown and no gauge
  callback fires during shutdown.
- MetricsRegistry::start(): extract initExporterAndProvider() and
  initSyncInstruments() helpers to keep each function under the line
  limit; no behavior change.
- time_in_current_state_seconds: read NetworkOPs::getServerStateDurationUs()
  (a lightweight accessor over StateAccounting) and convert microseconds
  to seconds, replacing the hardcoded 0.0.
2026-07-07 20:47:05 +01:00
Pratik Mankawde
53b1751384 Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-07-07 20:00:54 +01:00
Pratik Mankawde
6a2570e26e Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill
# Conflicts:
#	docker/telemetry/docker-compose.yml
2026-07-07 20:00:36 +01:00
Pratik Mankawde
41875b1897 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation
# Conflicts:
#	docker/telemetry/grafana/provisioning/datasources/tempo.yaml
2026-07-07 19:59:04 +01:00
Pratik Mankawde
9a769b9729 Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-07-07 19:58:08 +01:00
Pratik Mankawde
afe653e4b6 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd 2026-07-07 19:57:31 +01:00
Pratik Mankawde
fca98d1003 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-07-07 19:57:22 +01:00
Pratik Mankawde
35148b57f8 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing
# Conflicts:
#	docker/telemetry/grafana/provisioning/datasources/tempo.yaml
2026-07-07 19:57:14 +01:00
Pratik Mankawde
5e77968ca2 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing
# Conflicts:
#	docker/telemetry/grafana/provisioning/datasources/tempo.yaml
2026-07-07 19:56:33 +01:00
Pratik Mankawde
9e5f1caff2 docs(telemetry): address review feedback on phase-8 log correlation
- Correct log-timestamp examples to real Logs::format() output
  (2024-Jan-15 10:30:45.123456 UTC) in docs, TESTING.md, and reference.
- Update Loki to v3.4.2 and switch the collector exporter to
  otlphttp/loki on the native /otlp endpoint (docs + task list).
- Make the collector log-mount path configurable via XRPLD_LOG_DIR.
- Remove implementation-phase references from shipped config/script
  comments, keeping the functional descriptions.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 19:55:26 +01:00
Pratik Mankawde
e7bdf659fb Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing
# Conflicts:
#	docker/telemetry/grafana/provisioning/datasources/tempo.yaml
2026-07-07 19:48:02 +01:00
Pratik Mankawde
ba987bcb39 Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration 2026-07-07 19:46:16 +01:00
Pratik Mankawde
aefc87a572 docs(telemetry): remove Phase references from loki/tempo datasource comments
Reviewer noted implementation-phase references should not appear in
shipped config comments. Drop the phase-8-owned 'Phase 8' tokens from
loki.yaml and tempo.yaml, keeping the functional descriptions.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 19:45:36 +01:00
Pratik Mankawde
89cb633869 fix(consensus): drop DisputedTx accessor churn unrelated to log correlation
The getYays()/getNays() accessors already exist on the phase-7 branch.
A cleanup+restore pair on this branch moved them, producing a no-op
diff against phase-7 that mixed consensus churn into the log-correlation
PR. Restore DisputedTx.h to the phase-7 state so this branch carries no
unrelated consensus change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 19:45:20 +01:00
Pratik Mankawde
40b457669e docs(telemetry): remove Phase reference from tempo.yaml additional-filter comment
Reviewer noted implementation-phase references should not appear in
shipped config comments. Drop the "Phase 3/4" token, keep the description.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 19:44:44 +01:00
Pratik Mankawde
b54e0e50c0 docs(telemetry): remove Phase references from tempo.yaml consensus filter comments
Reviewer noted implementation-phase references should not appear in
shipped config comments. Drop the "Phase 4" tokens, keep descriptions.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 19:44:00 +01:00
Pratik Mankawde
36c08fdc66 docs(telemetry): remove Phase references from tempo.yaml TX filter comments
Reviewer noted implementation-phase references should not appear in
shipped config comments. Drop the "Phase 3" tokens, keep descriptions.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 19:43:59 +01:00
Pratik Mankawde
9879780dd8 docs(telemetry): remove Phase references from tempo.yaml RPC filter comments
Reviewer noted implementation-phase references should not appear in
shipped config comments. Drop the "Phase 2" tokens, keep descriptions.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 19:43:53 +01:00
Pratik Mankawde
b1af0a9511 docs(telemetry): remove Phase reference from tempo.yaml base-filter comment
Reviewer noted implementation-phase references should not appear in
shipped config comments. Drop the "Phase 1b" token, keep the filter
description.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 19:43:40 +01:00
Pratik Mankawde
72e345c1cc Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-07-07 19:18:54 +01:00
Pratik Mankawde
58790ff9fb Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-07-07 19:18:44 +01:00
Pratik Mankawde
c3ac033a78 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-07-07 19:15:35 +01:00