Commit Graph

10869 Commits

Author SHA1 Message Date
Pratik Mankawde
11d2d9038b Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd 2026-07-08 16:02:58 +01:00
Pratik Mankawde
f0d72f54d0 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-07-08 16:02:58 +01:00
Pratik Mankawde
9819abba3a Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-07-08 16:02:58 +01:00
Pratik Mankawde
12060bc219 Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-07-08 16:02:58 +01:00
Pratik Mankawde
a8ce1408a9 Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration 2026-07-08 16:02:51 +01:00
Pratik Mankawde
c0a3e113d9 Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing 2026-07-08 16:02:51 +01:00
Pratik Mankawde
ce9bdfcc55 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-07-08 16:02:51 +01:00
Pratik Mankawde
02b32ecdd8 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-07-08 16:02:51 +01:00
Pratik Mankawde
61ae9998e1 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing 2026-07-08 16:02:51 +01:00
Pratik Mankawde
22f5b77e0c Merge branch 'pratik/otel-phase1a-plan-docs' into pratik/otel-phase1b-telemetry-infra 2026-07-08 16:02:43 +01:00
Pratik Mankawde
8766012c65 Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-07-08 15:38:42 +01:00
Pratik Mankawde
9b84398c87 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-07-08 15:38:35 +01:00
Pratik Mankawde
c2e5c933a7 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-07-08 15:38:31 +01:00
Pratik Mankawde
a396fe073f Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-07-08 15:38:26 +01:00
Pratik Mankawde
0bfb338666 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd 2026-07-08 15:38:21 +01:00
Pratik Mankawde
c835ad528f Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-07-08 15:36:55 +01:00
Pratik Mankawde
0ee6da59be fix(telemetry): resolve clang-tidy include and modernize-use-auto CI errors
- Replace unused observer_result.h with nostd/shared_ptr.h and
  nostd/unique_ptr.h in MetricsRegistry.h (misc-include-cleaner)
- Use auto for cast initializations in MetricsRegistry.cpp and
  ValidationTracker test (modernize-use-auto)
- Run rename scripts: rippled -> xrpld in comments and docs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-08 15:36:44 +01:00
Pratik Mankawde
b1091a482c Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment
# Conflicts:
#	src/libxrpl/telemetry/Telemetry.cpp
2026-07-08 15:23:03 +01:00
Pratik Mankawde
749535a3ca fix(telemetry): resolve clang-tidy CI errors in consensus tracing
Add missing <string_view> include in Consensus.h and RCLConsensus.h.
Remove unused SHAMap.h include from RCLConsensus.h.
Add braces around single-line if/else bodies in RCLConsensus.cpp.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-08 15:22:24 +01:00
Pratik Mankawde
87c003230c Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-07-08 15:20:07 +01:00
Pratik Mankawde
6938614468 fix(telemetry): remove unused includes (misc-include-cleaner)
Remove unused SpanNames.h from Transactor.cpp (transitively included
via TxApplySpanNames.h). Remove unused <ranges> from
TraceContextValidation.h (std::ranges::any_of lives in <algorithm>).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-08 15:20:01 +01:00
Pratik Mankawde
8108caffe3 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing 2026-07-08 15:19:01 +01:00
Pratik Mankawde
db38ba268f fix(telemetry): use std::ranges algorithms (modernize-use-ranges)
Replace std::transform with std::ranges::transform in Redaction.cpp.
Replace std::all_of with std::ranges::all_of in Redaction test.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-08 15:18:55 +01:00
Pratik Mankawde
01c9f6cf08 Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing 2026-07-08 15:18:11 +01:00
Pratik Mankawde
ba8785dc16 Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration 2026-07-08 15:18:06 +01:00
Pratik Mankawde
c6be7cf83a fix(telemetry): resolve clang-tidy CI errors (include-cleaner, braces)
Add missing <cstdint> include for std::uint32_t in Telemetry.h.
Add braces around single-line if bodies in Telemetry.cpp.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-08 15:17:58 +01:00
Pratik Mankawde
e64afb53d0 fix(telemetry): use service_instance_id in Grafana Cloud transform and docs
Update the transform/cloudlabels processor to set service_instance_id
instead of exported_instance, matching the local Prometheus path where
resource_to_telemetry_conversion promotes service.instance.id with that
exact label name. Also update source comments and documentation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-08 14:55:15 +01:00
Ayaz Salikhov
58af1e6f18 release: Bump version to 3.3.0-b1 (#7755) 2026-07-08 12:40:24 +00:00
Ayaz Salikhov
e372c45836 chore: Enable most performance checks (#7727) 2026-07-07 21:03:56 +00:00
Ayaz Salikhov
2ebc96a4a6 perf: Use std::from/to_chars for JSON double parsing/formating (#7735) 2026-07-07 21:02:39 +00:00
Pratik Mankawde
15f9c66dcf fix(telemetry): address phase-9 metric gap-fill review comments
- PerfLogImp::rpcEnd(): return after the requestId-not-found UNREACHABLE
  so a stale (now - epoch) duration is no longer recorded to the counter
  and histogram in release builds.
- MetricsRegistry peer-version gauge: compare versions numerically via
  BuildInfo::encodeSoftwareVersion() instead of a lexicographic string
  compare, stripping the non-digit prefix so peer 'rippled-X.Y.Z' lines
  up with our bare 'X.Y.Z'. Fixes every peer counting as higher-version.
- MetricsRegistry::stop(): call Shutdown() before ForceFlush() before
  reset() so the reader thread stops before teardown and no gauge
  callback fires during shutdown.
- MetricsRegistry::start(): extract initExporterAndProvider() and
  initSyncInstruments() helpers to keep each function under the line
  limit; no behavior change.
- time_in_current_state_seconds: read NetworkOPs::getServerStateDurationUs()
  (a lightweight accessor over StateAccounting) and convert microseconds
  to seconds, replacing the hardcoded 0.0.
2026-07-07 20:47:05 +01:00
Pratik Mankawde
6a2570e26e Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill
# Conflicts:
#	docker/telemetry/docker-compose.yml
2026-07-07 20:00:36 +01:00
Pratik Mankawde
89cb633869 fix(consensus): drop DisputedTx accessor churn unrelated to log correlation
The getYays()/getNays() accessors already exist on the phase-7 branch.
A cleanup+restore pair on this branch moved them, producing a no-op
diff against phase-7 that mixed consensus churn into the log-correlation
PR. Restore DisputedTx.h to the phase-7 state so this branch carries no
unrelated consensus change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 19:45:20 +01:00
Pratik Mankawde
58790ff9fb Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-07-07 19:18:44 +01:00
Pratik Mankawde
c3ac033a78 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-07-07 19:15:35 +01:00
Pratik Mankawde
a8ca3d50c3 fix(telemetry): node-health detail panels + OTelCollector hook deadlock
- node-health.json: repoint the two "All Jobs" detail panels from the
  __name__ regex over old xrpld_<job>_milliseconds_bucket names to the
  native xrpld_job_running/queued_duration_us_bucket histograms, grouping
  by the job_type label and legending on {{job_type}}.
- OTelCollector::callHooks: copy the hook list under mutex_ and invoke
  handlers outside the lock. A handler can drop the last reference to an
  OTelHookImpl, whose destructor calls removeHook() and re-acquires the
  non-recursive mutex_ — invoking handlers under the lock could deadlock.
- ~OTelGaugeImpl: document that the SDK ObservableRegistry serializes
  RemoveCallback against the Observe() callback pass with one mutex, so
  callback removal is synchronous and gaugeCallback cannot run on a
  dangling pointer after the destructor.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 19:15:11 +01:00
Pratik Mankawde
979f4b64e4 fix(telemetry): correct service.name label and make alerting boot without env
- MetricsRegistry: service.name was recorded as boolean true because the
  string literal "xrpld" bound to the OTel AttributeValue variant's
  char-const* -> bool overload. Assign std::string so it selects the
  string alternative; Prometheus now shows service_name=xrpld (and
  exported_job round-trips correctly) on every MetricsRegistry series.
- Grafana alerting contact points: the Slack url / email addresses used
  ${ENV_VAR} references that expand to empty when unset, and Grafana's
  provisioning validator (which lacks ${VAR:-default} support) then
  crashes the whole stack on startup. Use non-empty disabled placeholders
  (unroutable webhook host, .invalid email) so the stack boots with zero
  configuration; delivery stays off until a real destination is supplied.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 18:28:11 +01:00
Pratik Mankawde
8ea61b88f2 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-07-07 17:38:49 +01:00
Pratik Mankawde
f80c1c3b4e Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-07-07 17:38:01 +01:00
Pratik Mankawde
04f351f3b7 fix(telemetry): more phase-7 review fixes
- ValidationTracker: remove dead hard-trim Pass 1. The preceding
  eviction loop already erases every reconciled entry older than the
  late-repair cutoff using the same condition, so Pass 1 could never
  match. The "drop any reconciled entry" fallback is retained.
- integration-test.sh: probe the StatsD port with a UDP-aware check
  (ss -ulnp) instead of curl. StatsD is UDP-only on 8125, so the TCP
  curl probe always reported "refused" — a false negative. Guarded by
  a command -v ss check that logs a skip when ss is unavailable.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 16:55:19 +01:00
Pratik Mankawde
ab0d8249fd feat(telemetry): expose direct OTel metrics API via getMeter()
Give developers direct access to the full OpenTelemetry metrics API,
symmetric with getTracer(), so all seven OTel instrument types
(including UpDownCounter, sync Gauge, and the observable variants) are
reachable — not just the four beast::insight models.

- Telemetry: build and own the metrics pipeline (OTLP HTTP metric
  exporter + PeriodicExportingMetricReader + SpanMetrics histogram view)
  alongside the tracer, sharing the same resource attributes and TLS
  config. Register it globally via metrics::Provider::SetMeterProvider
  and expose Telemetry::getMeter(). Metrics enable with [telemetry];
  the metrics endpoint is derived from the trace endpoint.
- beast OTelCollector: no longer owns a pipeline. It fetches the global
  Meter, becoming a thin shim over the shared provider (legacy path
  during beast deprecation). This also resolves the review note that the
  metric exporter ignored [telemetry] use_tls — TLS now comes from the
  shared telemetry pipeline.
- Add a libxrpl unit test covering getMeter() on the enabled (global
  provider) and disabled (noop) paths, exercising an UpDownCounter.

Design: docs/superpowers/specs/2026-07-07-direct-otel-metrics-api-design.md

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 16:52:45 +01:00
Pratik Mankawde
cc4457b5a2 fix(telemetry): address phase-7 review comments
- cmake: keep the opentelemetry-cpp umbrella target for the beast metrics
  link and document why. The reviewer suggested linking individual
  component targets to avoid over-linking, but the OTel Conan package
  under-declares inter-component dependencies (the OTLP client references
  sdk::common symbols without a declared edge), so naming components
  directly reorders the static link into an unresolvable state. Verified
  by building xrpl_tests both ways.
- OTelCollector.h: add usage examples, thread-safety and limitations
  @note blocks to the class doc.
- OTelCollector.cpp: correct the @param name docs on the instrument
  Impl constructors to describe the already-formatName()'d value.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 16:50:38 +01:00
Ayaz Salikhov
c5a6de6ef7 chore: Enable most cppcoreguidelines checks (#7660) 2026-07-07 13:12:52 +00:00
Pratik Mankawde
dce39bc71d Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill
# Conflicts:
#	OpenTelemetryPlan/08-appendix.md
#	OpenTelemetryPlan/presentation.md
#	docker/telemetry/xrpld-telemetry.cfg
#	docs/telemetry-runbook.md
2026-07-06 22:06:45 +01:00
Pratik Mankawde
55052917db Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation
# Conflicts:
#	OpenTelemetryPlan/08-appendix.md
2026-07-06 22:02:14 +01:00
Pratik Mankawde
363c2caf94 Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics
# Conflicts:
#	OpenTelemetryPlan/08-appendix.md
#	OpenTelemetryPlan/09-data-collection-reference.md
#	docker/telemetry/docker-compose.yml
#	docker/telemetry/grafana/dashboards/statsd-ledger-data-sync.json
#	docker/telemetry/grafana/dashboards/statsd-network-traffic.json
#	docker/telemetry/grafana/dashboards/statsd-node-health.json
#	docker/telemetry/grafana/dashboards/statsd-rpc-pathfinding.json
2026-07-06 22:01:22 +01:00
Pratik Mankawde
f4981a1907 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd
# Conflicts:
#	OpenTelemetryPlan/08-appendix.md
#	docker/telemetry/grafana/provisioning/datasources/tempo.yaml
#	docker/telemetry/otel-collector-config.yaml
#	docs/telemetry-runbook.md
2026-07-06 21:47:17 +01:00
Pratik Mankawde
1938b9e377 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-07-06 21:05:06 +01:00
Pratik Mankawde
b7d0b700c6 fix(telemetry): correct runbook pathfind spans + mTLS validation
- Add PathFinding Spans section to the runbook Span Reference and
  remove the incorrect "not emitted / later-phase" annotations:
  pathfind.request/compute/discover/update_all are emitted since
  Phase 2 (upstream of Phase 5).
- Runbook label names use the bare spanmetrics dimensions
  (command, rpc_status, local, consensus_mode); fix stale docs.
- 05-configuration-reference: note tls_client_cert/key require use_tls=1.
- TelemetryConfig: reject mTLS cert paths when use_tls=0 so telemetry
  is not silently sent in plaintext.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 21:04:07 +01:00
Pratik Mankawde
ea9c639ee9 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-07-06 20:50:24 +01:00