Commit Graph

16751 Commits

Author SHA1 Message Date
Pratik Mankawde
5945dd5f28 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-09-18 10:03:14 +01:00
Pratik Mankawde
95cc50d43f Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-09-18 10:03:14 +01:00
Pratik Mankawde
4e2b8ce536 merge: bring the plan-doc suppressed cleanup forward from phase-6
The attribute table conflicted because this branch had added the fee and
sequence rows and widened tx_type's set-on list. Kept those, and dropped the
suppressed row the incoming side removed.
2026-09-18 10:02:54 +01:00
Pratik Mankawde
6e4d566ba6 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd 2026-09-18 10:00:51 +01:00
Pratik Mankawde
84ff06ecec Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-09-18 10:00:51 +01:00
Pratik Mankawde
b2b5879e8e Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-09-18 10:00:51 +01:00
Pratik Mankawde
ff44c3c851 docs(telemetry): drop suppressed from the attribute and label references
tx.receive no longer sets it, and it is no longer a spanmetrics dimension, so
the attribute tables, the span-scope list and the Prometheus label list all
named something that is not emitted. The reduce-relay suppressed_peers gauge is
a different thing and stays.
2026-09-18 09:56:57 +01:00
Pratik Mankawde
073ff1a93f docs(telemetry): drop the suppressed row from the attribute reference
tx.receive does not set it. The span starts after the duplicate check, so a
dropped copy produces no span for the attribute to sit on.
2026-09-18 09:55:45 +01:00
Pratik Mankawde
a0a23f0b61 docs(telemetry): match the task list to where the receive span now starts
The steps described setting a suppressed attribute on a dropped duplicate. The
span is now created after the duplicate check, so there is no span on that path.
2026-09-18 09:55:14 +01:00
Pratik Mankawde
41140ec6d1 merge: bring the runbook attribute-row fix forward from phase-8
The table conflicted because both sides edited it: this branch had dropped
ledger_seq from the tx.apply row, and the incoming side updated the tx.receive
row. Kept both.
2026-09-18 09:34:19 +01:00
Pratik Mankawde
673e12e7b3 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-09-18 09:33:24 +01:00
Pratik Mankawde
795c9ad39c Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-09-18 09:33:24 +01:00
Pratik Mankawde
3b60a190de fix(telemetry): drop the remaining suppressed dimensions and diagram values
Alloy and the Grafana Cloud collector still declared a spanmetrics dimension
for an attribute tx.receive no longer sets, which only widens the label set.
The runbook's drop-path diagram still listed two tx_status values that no
longer exist, because the paths that set them now run before the span starts.
2026-09-18 09:32:29 +01:00
Pratik Mankawde
675dffdc71 docs(telemetry): drop suppressed from the tx.receive attribute list
The span is created after the duplicate check, so it never carries the
attribute, and the row now says which transactions it covers.
2026-09-18 09:31:23 +01:00
Pratik Mankawde
8ae1d4c6a5 merge: bring the suppressed-reference removal forward from phase-8
transaction-overview.json conflicted only because this branch reordered every
panel, so the diff could not line the two sides up. The incoming side changed
one panel and nothing else, so this keeps every panel and rewrite this branch
made and re-applies that single change in this branch's own style: the receive
panel drops the suppressed grouping, names its series like the other
single-series panels, and says where the duplicate count now lives.
2026-09-18 09:29:35 +01:00
Pratik Mankawde
f666427570 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-09-18 09:26:33 +01:00
Pratik Mankawde
b972682955 merge: bring the suppressed-reference removal forward from phase-6
Resolved transaction-overview.json by composing both sides: this branch's
renamed span_calls_total metric and its 15s interval, with phase-6's removal of
the suppressed grouping and its new legend.
2026-09-18 09:26:17 +01:00
Pratik Mankawde
e87432fe3c fix(telemetry): drop the suppressed references the span no longer emits
tx.receive is created after the duplicate check, so it never carries a
suppressed attribute. The Tempo tag filter and the dashboard panel still
referenced it, and check_otel_naming rules C and D fail on a reference with no
constant behind it.

The panel keeps its remaining signal as a plain receive rate. How many relayed
copies were dropped is the transactions_duplicate traffic category, which does
not depend on a span.

The spanmetrics dimension goes too; a dimension for an attribute nothing sets
only widens the label set.
2026-09-18 09:24:56 +01:00
Pratik Mankawde
73af31c27f Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-09-17 23:01:58 +01:00
Pratik Mankawde
caa3eb5984 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-09-17 23:01:58 +01:00
Pratik Mankawde
6024954459 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-09-17 23:01:47 +01:00
Pratik Mankawde
67c9303c0c Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-09-17 23:01:47 +01:00
Pratik Mankawde
8c10cd710d Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd 2026-09-17 23:01:47 +01:00
Pratik Mankawde
c0c9478093 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-09-17 23:01:35 +01:00
Pratik Mankawde
fe4a778fed perf(telemetry): trace only the transactions this node will process
A peer relays every transaction it hears, so most inbound copies are ones
handleTransaction() drops. The tx.receive span was created before those checks
ran, so every dropped copy paid for a span, a hex transaction id, and an
open-ledger index read that takes the mutex the apply path needs.

Measured over a four-hour payment run: 83.2M of 101.1M tx.receive spans
described a copy the node dropped. That is 82% of this span and 23% of every
span the node emitted.

Move the span and its attributes below the duplicate check. How many copies
were dropped is already reported as the transactions_duplicate traffic
category, which costs no span. Why a copy was dropped is no longer recorded;
a labelled counter restores it on the branch that carries the metric registry.

The suppressed attribute and the suppressed and rejected_inner_batch status
values go with it. This function was their only user.
2026-09-17 23:00:59 +01:00
Pratik Mankawde
9abbdc0a2f merge: bring the single telemetry switch forward from phase-8
Both sides had already dropped option(telemetry) and describe the same behaviour;
the conflicts were wording. Kept this branch's text, which spells out that
opentelemetry-cpp is not needed at all when telemetry is off and keeps the note
about -DXRPL_ENABLE_TELEMETRY.
2026-09-17 17:02:16 +01:00
Pratik Mankawde
d0baef8c92 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-09-17 17:01:21 +01:00
Pratik Mankawde
c1086e63a2 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-09-17 17:01:20 +01:00
Pratik Mankawde
6220c211fe Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-09-17 17:01:20 +01:00
Pratik Mankawde
aea4431807 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd 2026-09-17 17:01:20 +01:00
Pratik Mankawde
de3e376944 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-09-17 17:01:20 +01:00
Pratik Mankawde
94bdd19608 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing 2026-09-17 17:01:20 +01:00
Pratik Mankawde
177e229aa9 Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing 2026-09-17 17:01:20 +01:00
Pratik Mankawde
4553fbf2a5 merge: bring the single telemetry switch forward from phase-1b
CMakeLists conflicted: this branch had rewritten the description to name
SpanGuard, phase-1b removed the option() call. Kept both — this branch's wording
with phase-1b's structure.
2026-09-17 17:01:07 +01:00
Pratik Mankawde
8cb8f6b6c6 build: drop the second telemetry default and its stale docs
This branch rewrote the CMakeLists telemetry block and the build doc. Both now
describe a CMake option that no longer exists: the Conan option is the switch and
the generated toolchain carries it into CMake. The comment also recorded the
state of the change rather than the behaviour of the code.

The doc's "Building without telemetry" section told readers to pass
-Dtelemetry=OFF to CMake as well, which would override the toolchain rather than
follow it.
2026-09-17 16:54:18 +01:00
Pratik Mankawde
e52f235356 build: make the Conan option the only telemetry switch
CMakeLists declared option(telemetry) with a default of its own, so the setting
had two homes and they were free to disagree. The Conan option already reaches
CMake without it: conanfile.py forwards the option into the generated toolchain,
which sets the variable this file reads, and every build here goes through Conan.

Drop the option() call and keep the if(telemetry) test. docs/build/telemetry.md
told readers a CMake option had to be set as well, so that goes with it.
2026-09-17 16:54:03 +01:00
Pratik Mankawde
327b5bb229 fix(telemetry): keep the telemetry-off build clean
A -DXRPL_ENABLE_TELEMETRY=0 build expands every XRPL_METRIC_* macro to
nothing, so anything whose only use sits inside one looks unused.

- MetricsRegistry::hasPipeline touches no member in that build, so
  clang-tidy asks for it to be static. Making it static would give the two
  builds different signatures, so suppress it the way Recording.h already
  does for the same reason.
- PathRequest.cpp's only use of MetricMacros.h is such a macro, so mark the
  include as kept.
- AppMetricGauges.cpp names MetricsRegistry and beast::Journal in a
  constructor signature that is compiled either way, so their includes move
  outside the telemetry guard.
2026-09-17 13:44:53 +01:00
Pratik Mankawde
ec0bfe521d refactor(telemetry): move the metrics pipeline core into libxrpl
MetricsRegistry did two jobs. It owned the OTel metrics pipeline, and it
registered the observable gauges whose callbacks read live application
services. The second job is what made the whole class xrpld-tier, so the
pipeline's lifecycle -- the recording() gate and the stop() teardown that
closes a use-after-free window -- could not be unit-tested in xrpl_tests.

Split it in two:

- xrpl::telemetry::MetricsRegistry (libxrpl) owns the exporter, provider,
  meter, the 16 synchronous instruments, recording(), stop(), and the
  record*/increment* methods.
- xrpl::telemetry::AppMetricGauges (xrpld) owns the 19 observable gauges
  and their callbacks, holding a reference to the core and to the
  ServiceRegistry.

MetricMacros.h and ValidationTracker move with the core. The macros need
only recording() and meter(), both core members; the core holds a tracker
by value, and a libxrpl header cannot include one from src/.

ApplicationImp owns both objects and sequences them. The core is built in
the member-init list, so every synchronous instrument exists before any
subsystem can record one. The gauges are armed once overlay_ exists, the
last service their callbacks read. Shutdown detaches the gauge callbacks
before the core drops the provider, and each shutdown step is isolated so
a failure in one cannot skip the others.

That detach call is new. detachCallbacks() had no callers, and the flag it
sets is read by the gauge callbacks but can no longer be written by the
core, so the caller now has to make the ordering explicit.

The telemetry module links xrpl.libxrpl.core and xrpl.libxrpl.protocol
PUBLIC: ValidationTracker.h takes a LedgerIndex and MetricMacros.h takes a
ServiceRegistry, both in interfaces a consumer compiles against.

Adds a MetricsRegistry gtest that drives an enabled core with telemetry on
and pins the recording() gate, stop() leaving the registry inert, and
stop() being idempotent. The libxrpl test tree no longer depends on
xrpld.telemetry at all, and the two CMake workarounds that compiled xrpld
sources into xrpl_tests are gone.

Documentation and dashboard source links follow the code to their new
paths, split between the two classes by which one now defines each metric.
2026-09-16 13:45:52 +01:00
Pratik Mankawde
d29e392c0b alert(NodeStateFlapping): fire on a single flap, keep the always-present metric
The rule watched state_accounting_full_transitions > 3 per hour, so a node
that flaps once (one full -> syncing -> full round, e.g. per online-delete
rotation) never tripped it. Lower the threshold to > 0 so a single re-entry
into FULL, past the one-hour uptime gate, alerts.

Keep the state_accounting_full_transitions metric: it is a cumulative gauge
every node always reports, so increase() yields a real series (0 when
healthy) and the rule never evaluates to NoData. A sparse counter would
raise a false DatasourceNoData on a healthy node. Set noDataState: OK so a
scrape gap cannot page either.
2026-09-15 16:57:52 +01:00
Pratik Mankawde
913721b0bc Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-09-15 16:17:44 +01:00
Pratik Mankawde
75661c3a33 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-09-15 16:17:44 +01:00
Pratik Mankawde
e476241b5f Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-09-15 16:17:44 +01:00
Pratik Mankawde
260bc7f2ed Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd 2026-09-15 16:17:44 +01:00
Pratik Mankawde
8bd043ecb7 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-09-15 16:17:44 +01:00
Pratik Mankawde
5b10ef0d56 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-09-15 16:17:43 +01:00
Pratik Mankawde
b14c537df1 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing 2026-09-15 16:17:43 +01:00
Pratik Mankawde
9545ab4d4e Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing 2026-09-15 16:17:43 +01:00
Pratik Mankawde
6a0642817d Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration 2026-09-15 16:17:43 +01:00
Pratik Mankawde
d008b2f641 Merge branch 'pratik/otel-phase1a-plan-docs' into pratik/otel-phase1b-telemetry-infra 2026-09-15 16:17:43 +01:00
Pratik Mankawde
a910a2dc91 Merge remote-tracking branch 'origin/develop' into pratik/otel-phase1a-plan-docs 2026-09-15 16:17:43 +01:00