Commit Graph

15242 Commits

Author SHA1 Message Date
Pratik Mankawde
0ee6da59be fix(telemetry): resolve clang-tidy include and modernize-use-auto CI errors
- Replace unused observer_result.h with nostd/shared_ptr.h and
  nostd/unique_ptr.h in MetricsRegistry.h (misc-include-cleaner)
- Use auto for cast initializations in MetricsRegistry.cpp and
  ValidationTracker test (modernize-use-auto)
- Run rename scripts: rippled -> xrpld in comments and docs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-08 15:36:44 +01:00
Pratik Mankawde
1730cdb50d Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill
# Conflicts:
#	docker/telemetry/xrpld-telemetry.cfg
2026-07-08 15:26:40 +01:00
Pratik Mankawde
a681eea6c4 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-07-08 15:26:06 +01:00
Pratik Mankawde
0a373f631c fix(telemetry): resolve otel-naming and UBSAN CI failures
- Add job_type to otel-naming builtins (standard Prometheus label for
  job-queue metrics used in node-health dashboard)
- Add OpenTelemetry SDK to UBSAN suppressions (intentional unsigned
  integer overflow in attributemap_hash.h hash computation)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-08 15:25:59 +01:00
Pratik Mankawde
a1a9d45bff Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-07-08 15:22:52 +01:00
Pratik Mankawde
4826038068 fix(telemetry): run rename scripts for CI compliance
Rename rippled → xrpld in statsd config prefix.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-08 15:22:45 +01:00
Pratik Mankawde
d24534a340 docs(telemetry): update Phase9 task list to use service_instance_id
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-08 14:57:46 +01:00
Pratik Mankawde
a13535fed7 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-07-08 14:57:41 +01:00
Pratik Mankawde
c75de2d58c Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-07-08 14:57:37 +01:00
Pratik Mankawde
a2a58a5d85 docs(telemetry): update plan docs to use service_instance_id
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-08 14:57:29 +01:00
Pratik Mankawde
21acd060c8 fix(telemetry): use service_instance_id label instead of exported_instance
Merge forward from phase6/7 and apply the same fix to phase9's own
dashboards and alerting rules. exported_instance is a Prometheus scrape
artifact absent in push-based pipelines; service_instance_id is always
present via resource_to_telemetry_conversion.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-08 14:52:55 +01:00
Pratik Mankawde
a3053a34a7 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-07-08 14:51:55 +01:00
Pratik Mankawde
87f370a64d fix(telemetry): use service_instance_id label instead of exported_instance
exported_instance is a Prometheus scrape artifact that only exists when
Prometheus scrapes a target with a conflicting instance label. Users
running push-based pipelines (remote_write, OTLP) never get this label.
service_instance_id is always present via resource_to_telemetry_conversion
regardless of ingestion path.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-08 14:51:48 +01:00
Pratik Mankawde
ff92c3727c Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-07-08 14:51:36 +01:00
Pratik Mankawde
1eb44689d0 fix(telemetry): use service_instance_id label instead of exported_instance
exported_instance is a Prometheus scrape artifact that only exists when
Prometheus scrapes a target with a conflicting instance label. Users
running push-based pipelines (remote_write, OTLP) never get this label.
service_instance_id is always present via resource_to_telemetry_conversion
regardless of ingestion path.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-08 14:51:26 +01:00
Pratik Mankawde
f8fbcc2a2e Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill
# Conflicts:
#	docker/telemetry/xrpld-telemetry.cfg
2026-07-07 21:34:13 +01:00
Pratik Mankawde
e4a78684e1 fix(telemetry): make devnet debug_logfile config-dir-relative
xrpld resolves a relative [debug_logfile] against the config file's own
directory (docker/telemetry), not the working directory. The value
`docker/telemetry/data/logs/devnet/debug.log` therefore doubled the
prefix to docker/telemetry/docker/telemetry/... — the collector's mount
never saw the log, so no lines reached Loki.

Use the config-dir-relative form `data/logs/devnet/debug.log`, which
resolves to docker/telemetry/data/logs/devnet/debug.log — exactly the
dir the compose stack bind-mounts as /var/log/xrpld.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 21:31:59 +01:00
Pratik Mankawde
e3188db2b3 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill
# Conflicts:
#	docker/telemetry/docker-compose.yml
#	docker/telemetry/otel-collector-config.yaml
#	docker/telemetry/xrpld-telemetry.cfg
2026-07-07 20:57:52 +01:00
Pratik Mankawde
f99a1c5310 fix(telemetry): consolidate xrpld log mount to a single rootless path
The collector's log ingestion used a container path named after the old
`rippled` binary and defaulted its host mount to a location that either
needed root or diverged from where the telemetry configs actually write.

Consolidate on one accessible log root:
- container mount target renamed /var/log/rippled -> /var/log/xrpld
- filelog glob -> /var/log/xrpld/*/debug.log
- compose mount source defaults to the repo-relative ./data/logs
  (user-owned, no root), overridable via XRPLD_LOG_DIR
- devnet telemetry cfg writes to data/logs/devnet/debug.log so it lands
  one subdir below the mount root and matches the glob
- integration-test.sh sets XRPLD_LOG_DIR to its own workdir
- docs/runbook/task-list updated to match

The default xrpld config (cfg/xrpld-example.cfg) is intentionally left
untouched.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 20:54:53 +01:00
Pratik Mankawde
15f9c66dcf fix(telemetry): address phase-9 metric gap-fill review comments
- PerfLogImp::rpcEnd(): return after the requestId-not-found UNREACHABLE
  so a stale (now - epoch) duration is no longer recorded to the counter
  and histogram in release builds.
- MetricsRegistry peer-version gauge: compare versions numerically via
  BuildInfo::encodeSoftwareVersion() instead of a lexicographic string
  compare, stripping the non-digit prefix so peer 'rippled-X.Y.Z' lines
  up with our bare 'X.Y.Z'. Fixes every peer counting as higher-version.
- MetricsRegistry::stop(): call Shutdown() before ForceFlush() before
  reset() so the reader thread stops before teardown and no gauge
  callback fires during shutdown.
- MetricsRegistry::start(): extract initExporterAndProvider() and
  initSyncInstruments() helpers to keep each function under the line
  limit; no behavior change.
- time_in_current_state_seconds: read NetworkOPs::getServerStateDurationUs()
  (a lightweight accessor over StateAccounting) and convert microseconds
  to seconds, replacing the hardcoded 0.0.
2026-07-07 20:47:05 +01:00
Pratik Mankawde
6a2570e26e Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill
# Conflicts:
#	docker/telemetry/docker-compose.yml
2026-07-07 20:00:36 +01:00
Pratik Mankawde
41875b1897 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation
# Conflicts:
#	docker/telemetry/grafana/provisioning/datasources/tempo.yaml
2026-07-07 19:59:04 +01:00
Pratik Mankawde
9a769b9729 Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-07-07 19:58:08 +01:00
Pratik Mankawde
afe653e4b6 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd 2026-07-07 19:57:31 +01:00
Pratik Mankawde
fca98d1003 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-07-07 19:57:22 +01:00
Pratik Mankawde
35148b57f8 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing
# Conflicts:
#	docker/telemetry/grafana/provisioning/datasources/tempo.yaml
2026-07-07 19:57:14 +01:00
Pratik Mankawde
5e77968ca2 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing
# Conflicts:
#	docker/telemetry/grafana/provisioning/datasources/tempo.yaml
2026-07-07 19:56:33 +01:00
Pratik Mankawde
9e5f1caff2 docs(telemetry): address review feedback on phase-8 log correlation
- Correct log-timestamp examples to real Logs::format() output
  (2024-Jan-15 10:30:45.123456 UTC) in docs, TESTING.md, and reference.
- Update Loki to v3.4.2 and switch the collector exporter to
  otlphttp/loki on the native /otlp endpoint (docs + task list).
- Make the collector log-mount path configurable via XRPLD_LOG_DIR.
- Remove implementation-phase references from shipped config/script
  comments, keeping the functional descriptions.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 19:55:26 +01:00
Pratik Mankawde
e7bdf659fb Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing
# Conflicts:
#	docker/telemetry/grafana/provisioning/datasources/tempo.yaml
2026-07-07 19:48:02 +01:00
Pratik Mankawde
ba987bcb39 Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration 2026-07-07 19:46:16 +01:00
Pratik Mankawde
aefc87a572 docs(telemetry): remove Phase references from loki/tempo datasource comments
Reviewer noted implementation-phase references should not appear in
shipped config comments. Drop the phase-8-owned 'Phase 8' tokens from
loki.yaml and tempo.yaml, keeping the functional descriptions.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 19:45:36 +01:00
Pratik Mankawde
89cb633869 fix(consensus): drop DisputedTx accessor churn unrelated to log correlation
The getYays()/getNays() accessors already exist on the phase-7 branch.
A cleanup+restore pair on this branch moved them, producing a no-op
diff against phase-7 that mixed consensus churn into the log-correlation
PR. Restore DisputedTx.h to the phase-7 state so this branch carries no
unrelated consensus change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 19:45:20 +01:00
Pratik Mankawde
40b457669e docs(telemetry): remove Phase reference from tempo.yaml additional-filter comment
Reviewer noted implementation-phase references should not appear in
shipped config comments. Drop the "Phase 3/4" token, keep the description.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 19:44:44 +01:00
Pratik Mankawde
b54e0e50c0 docs(telemetry): remove Phase references from tempo.yaml consensus filter comments
Reviewer noted implementation-phase references should not appear in
shipped config comments. Drop the "Phase 4" tokens, keep descriptions.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 19:44:00 +01:00
Pratik Mankawde
36c08fdc66 docs(telemetry): remove Phase references from tempo.yaml TX filter comments
Reviewer noted implementation-phase references should not appear in
shipped config comments. Drop the "Phase 3" tokens, keep descriptions.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 19:43:59 +01:00
Pratik Mankawde
9879780dd8 docs(telemetry): remove Phase references from tempo.yaml RPC filter comments
Reviewer noted implementation-phase references should not appear in
shipped config comments. Drop the "Phase 2" tokens, keep descriptions.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 19:43:53 +01:00
Pratik Mankawde
b1af0a9511 docs(telemetry): remove Phase reference from tempo.yaml base-filter comment
Reviewer noted implementation-phase references should not appear in
shipped config comments. Drop the "Phase 1b" token, keep the filter
description.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 19:43:40 +01:00
Pratik Mankawde
58790ff9fb Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-07-07 19:18:44 +01:00
Pratik Mankawde
c3ac033a78 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-07-07 19:15:35 +01:00
Pratik Mankawde
a8ca3d50c3 fix(telemetry): node-health detail panels + OTelCollector hook deadlock
- node-health.json: repoint the two "All Jobs" detail panels from the
  __name__ regex over old xrpld_<job>_milliseconds_bucket names to the
  native xrpld_job_running/queued_duration_us_bucket histograms, grouping
  by the job_type label and legending on {{job_type}}.
- OTelCollector::callHooks: copy the hook list under mutex_ and invoke
  handlers outside the lock. A handler can drop the last reference to an
  OTelHookImpl, whose destructor calls removeHook() and re-acquires the
  non-recursive mutex_ — invoking handlers under the lock could deadlock.
- ~OTelGaugeImpl: document that the SDK ObservableRegistry serializes
  RemoveCallback against the Observe() callback pass with one mutex, so
  callback removal is synchronous and gaugeCallback cannot run on a
  dangling pointer after the destructor.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 19:15:11 +01:00
Pratik Mankawde
b15c549f63 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-07-07 18:55:16 +01:00
Pratik Mankawde
d76e067148 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-07-07 18:55:04 +01:00
Pratik Mankawde
bc140d9e96 fix(telemetry): repoint node-health job panels to native job-duration metrics
The Key Jobs execution/dequeue panels queried StatsD-era per-job metric
names (xrpld_<job>_milliseconds_bucket, xrpld_<job>_q_milliseconds_bucket)
that the native OTel path no longer emits. Phase 9's MetricsRegistry emits
job timings as two label-dimensioned histograms instead:
  xrpld_job_running_duration_us_bucket{job_type="<job>"}
  xrpld_job_queued_duration_us_bucket{job_type="<job>"}

Rewrite the 22 affected panel queries (11 job types x running/queued) to
the label-dimensioned form, preserving the histogram_quantile / rate /
sum-by structure and all template-variable filters.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 18:54:01 +01:00
Pratik Mankawde
979f4b64e4 fix(telemetry): correct service.name label and make alerting boot without env
- MetricsRegistry: service.name was recorded as boolean true because the
  string literal "xrpld" bound to the OTel AttributeValue variant's
  char-const* -> bool overload. Assign std::string so it selects the
  string alternative; Prometheus now shows service_name=xrpld (and
  exported_job round-trips correctly) on every MetricsRegistry series.
- Grafana alerting contact points: the Slack url / email addresses used
  ${ENV_VAR} references that expand to empty when unset, and Grafana's
  provisioning validator (which lacks ${VAR:-default} support) then
  crashes the whole stack on startup. Use non-empty disabled placeholders
  (unroutable webhook host, .invalid email) so the stack boots with zero
  configuration; delivery stays off until a real destination is supplied.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 18:28:11 +01:00
Pratik Mankawde
8919be22bf levelization
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-07-07 17:53:45 +01:00
Pratik Mankawde
8ea61b88f2 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-07-07 17:38:49 +01:00
Pratik Mankawde
f80c1c3b4e Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-07-07 17:38:01 +01:00
Pratik Mankawde
04f351f3b7 fix(telemetry): more phase-7 review fixes
- ValidationTracker: remove dead hard-trim Pass 1. The preceding
  eviction loop already erases every reconciled entry older than the
  late-repair cutoff using the same condition, so Pass 1 could never
  match. The "drop any reconciled entry" fallback is retained.
- integration-test.sh: probe the StatsD port with a UDP-aware check
  (ss -ulnp) instead of curl. StatsD is UDP-only on 8125, so the TCP
  curl probe always reported "refused" — a false negative. Guarded by
  a command -v ss check that logs a skip when ss is unavailable.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 16:55:19 +01:00
Pratik Mankawde
ab0d8249fd feat(telemetry): expose direct OTel metrics API via getMeter()
Give developers direct access to the full OpenTelemetry metrics API,
symmetric with getTracer(), so all seven OTel instrument types
(including UpDownCounter, sync Gauge, and the observable variants) are
reachable — not just the four beast::insight models.

- Telemetry: build and own the metrics pipeline (OTLP HTTP metric
  exporter + PeriodicExportingMetricReader + SpanMetrics histogram view)
  alongside the tracer, sharing the same resource attributes and TLS
  config. Register it globally via metrics::Provider::SetMeterProvider
  and expose Telemetry::getMeter(). Metrics enable with [telemetry];
  the metrics endpoint is derived from the trace endpoint.
- beast OTelCollector: no longer owns a pipeline. It fetches the global
  Meter, becoming a thin shim over the shared provider (legacy path
  during beast deprecation). This also resolves the review note that the
  metric exporter ignored [telemetry] use_tls — TLS now comes from the
  shared telemetry pipeline.
- Add a libxrpl unit test covering getMeter() on the enabled (global
  provider) and disabled (noop) paths, exercising an UpDownCounter.

Design: docs/superpowers/specs/2026-07-07-direct-otel-metrics-api-design.md

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 16:52:45 +01:00
Pratik Mankawde
cc4457b5a2 fix(telemetry): address phase-7 review comments
- cmake: keep the opentelemetry-cpp umbrella target for the beast metrics
  link and document why. The reviewer suggested linking individual
  component targets to avoid over-linking, but the OTel Conan package
  under-declares inter-component dependencies (the OTLP client references
  sdk::common symbols without a declared edge), so naming components
  directly reorders the static link into an unresolvable state. Verified
  by building xrpl_tests both ways.
- OTelCollector.h: add usage examples, thread-safety and limitations
  @note blocks to the class doc.
- OTelCollector.cpp: correct the @param name docs on the instrument
  Impl constructors to describe the already-formatName()'d value.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 16:50:38 +01:00