Commit Graph

16726 Commits

Author SHA1 Message Date
Pratik Mankawde
9abbdc0a2f merge: bring the single telemetry switch forward from phase-8
Both sides had already dropped option(telemetry) and describe the same behaviour;
the conflicts were wording. Kept this branch's text, which spells out that
opentelemetry-cpp is not needed at all when telemetry is off and keeps the note
about -DXRPL_ENABLE_TELEMETRY.
2026-09-17 17:02:16 +01:00
Pratik Mankawde
d0baef8c92 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-09-17 17:01:21 +01:00
Pratik Mankawde
6220c211fe Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-09-17 17:01:20 +01:00
Pratik Mankawde
aea4431807 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd 2026-09-17 17:01:20 +01:00
Pratik Mankawde
c1086e63a2 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-09-17 17:01:20 +01:00
Pratik Mankawde
de3e376944 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-09-17 17:01:20 +01:00
Pratik Mankawde
94bdd19608 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing 2026-09-17 17:01:20 +01:00
Pratik Mankawde
177e229aa9 Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing 2026-09-17 17:01:20 +01:00
Pratik Mankawde
4553fbf2a5 merge: bring the single telemetry switch forward from phase-1b
CMakeLists conflicted: this branch had rewritten the description to name
SpanGuard, phase-1b removed the option() call. Kept both — this branch's wording
with phase-1b's structure.
2026-09-17 17:01:07 +01:00
Pratik Mankawde
8cb8f6b6c6 build: drop the second telemetry default and its stale docs
This branch rewrote the CMakeLists telemetry block and the build doc. Both now
describe a CMake option that no longer exists: the Conan option is the switch and
the generated toolchain carries it into CMake. The comment also recorded the
state of the change rather than the behaviour of the code.

The doc's "Building without telemetry" section told readers to pass
-Dtelemetry=OFF to CMake as well, which would override the toolchain rather than
follow it.
2026-09-17 16:54:18 +01:00
Pratik Mankawde
e52f235356 build: make the Conan option the only telemetry switch
CMakeLists declared option(telemetry) with a default of its own, so the setting
had two homes and they were free to disagree. The Conan option already reaches
CMake without it: conanfile.py forwards the option into the generated toolchain,
which sets the variable this file reads, and every build here goes through Conan.

Drop the option() call and keep the if(telemetry) test. docs/build/telemetry.md
told readers a CMake option had to be set as well, so that goes with it.
2026-09-17 16:54:03 +01:00
Pratik Mankawde
327b5bb229 fix(telemetry): keep the telemetry-off build clean
A -DXRPL_ENABLE_TELEMETRY=0 build expands every XRPL_METRIC_* macro to
nothing, so anything whose only use sits inside one looks unused.

- MetricsRegistry::hasPipeline touches no member in that build, so
  clang-tidy asks for it to be static. Making it static would give the two
  builds different signatures, so suppress it the way Recording.h already
  does for the same reason.
- PathRequest.cpp's only use of MetricMacros.h is such a macro, so mark the
  include as kept.
- AppMetricGauges.cpp names MetricsRegistry and beast::Journal in a
  constructor signature that is compiled either way, so their includes move
  outside the telemetry guard.
2026-09-17 13:44:53 +01:00
Pratik Mankawde
ec0bfe521d refactor(telemetry): move the metrics pipeline core into libxrpl
MetricsRegistry did two jobs. It owned the OTel metrics pipeline, and it
registered the observable gauges whose callbacks read live application
services. The second job is what made the whole class xrpld-tier, so the
pipeline's lifecycle -- the recording() gate and the stop() teardown that
closes a use-after-free window -- could not be unit-tested in xrpl_tests.

Split it in two:

- xrpl::telemetry::MetricsRegistry (libxrpl) owns the exporter, provider,
  meter, the 16 synchronous instruments, recording(), stop(), and the
  record*/increment* methods.
- xrpl::telemetry::AppMetricGauges (xrpld) owns the 19 observable gauges
  and their callbacks, holding a reference to the core and to the
  ServiceRegistry.

MetricMacros.h and ValidationTracker move with the core. The macros need
only recording() and meter(), both core members; the core holds a tracker
by value, and a libxrpl header cannot include one from src/.

ApplicationImp owns both objects and sequences them. The core is built in
the member-init list, so every synchronous instrument exists before any
subsystem can record one. The gauges are armed once overlay_ exists, the
last service their callbacks read. Shutdown detaches the gauge callbacks
before the core drops the provider, and each shutdown step is isolated so
a failure in one cannot skip the others.

That detach call is new. detachCallbacks() had no callers, and the flag it
sets is read by the gauge callbacks but can no longer be written by the
core, so the caller now has to make the ordering explicit.

The telemetry module links xrpl.libxrpl.core and xrpl.libxrpl.protocol
PUBLIC: ValidationTracker.h takes a LedgerIndex and MetricMacros.h takes a
ServiceRegistry, both in interfaces a consumer compiles against.

Adds a MetricsRegistry gtest that drives an enabled core with telemetry on
and pins the recording() gate, stop() leaving the registry inert, and
stop() being idempotent. The libxrpl test tree no longer depends on
xrpld.telemetry at all, and the two CMake workarounds that compiled xrpld
sources into xrpl_tests are gone.

Documentation and dashboard source links follow the code to their new
paths, split between the two classes by which one now defines each metric.
2026-09-16 13:45:52 +01:00
Pratik Mankawde
d29e392c0b alert(NodeStateFlapping): fire on a single flap, keep the always-present metric
The rule watched state_accounting_full_transitions > 3 per hour, so a node
that flaps once (one full -> syncing -> full round, e.g. per online-delete
rotation) never tripped it. Lower the threshold to > 0 so a single re-entry
into FULL, past the one-hour uptime gate, alerts.

Keep the state_accounting_full_transitions metric: it is a cumulative gauge
every node always reports, so increase() yields a real series (0 when
healthy) and the rule never evaluates to NoData. A sparse counter would
raise a false DatasourceNoData on a healthy node. Set noDataState: OK so a
scrape gap cannot page either.
2026-09-15 16:57:52 +01:00
Pratik Mankawde
913721b0bc Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-09-15 16:17:44 +01:00
Pratik Mankawde
e476241b5f Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-09-15 16:17:44 +01:00
Pratik Mankawde
260bc7f2ed Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd 2026-09-15 16:17:44 +01:00
Pratik Mankawde
75661c3a33 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-09-15 16:17:44 +01:00
Pratik Mankawde
8bd043ecb7 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-09-15 16:17:44 +01:00
Pratik Mankawde
5b10ef0d56 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-09-15 16:17:43 +01:00
Pratik Mankawde
b14c537df1 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing 2026-09-15 16:17:43 +01:00
Pratik Mankawde
9545ab4d4e Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing 2026-09-15 16:17:43 +01:00
Pratik Mankawde
6a0642817d Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration 2026-09-15 16:17:43 +01:00
Pratik Mankawde
d008b2f641 Merge branch 'pratik/otel-phase1a-plan-docs' into pratik/otel-phase1b-telemetry-infra 2026-09-15 16:17:43 +01:00
Pratik Mankawde
a910a2dc91 Merge remote-tracking branch 'origin/develop' into pratik/otel-phase1a-plan-docs 2026-09-15 16:17:43 +01:00
Pratik Mankawde
866ab77ece docs(telemetry): describe rotation measurements without naming the host
The runbook provenance paragraph named the internal AWS dev box and a
build hash and dates. State what was measured (one mainnet node, same
host and binary, differing only in store state) without the deployment
detail, which belongs in an internal runbook, not the public repo.
2026-09-15 14:26:17 +01:00
Pratik Mankawde
751c23bd98 merge: bring the optional-access NOLINT fix forward from phase7-native-metrics 2026-09-15 14:22:58 +01:00
Pratik Mankawde
0306120ed3 merge: bring the optional-access NOLINT fix forward from phase8-log-correlation 2026-09-15 14:22:58 +01:00
Pratik Mankawde
42181b8ed2 test(server): silence false-positive optional-access on asserted reads
clang-tidy's bugprone-unchecked-optional-access does not model GTest's
ASSERT_TRUE(x.has_value()), so it flags every deref that follows one.
The reads are guarded; mark them NOLINT, matching the same suppression
in src/tests/libxrpl/consensus/LedgerTrie.cpp. .value() does not help --
the checker treats it as an unchecked access too.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-09-15 14:22:40 +01:00
Bart
e302e4eeed fix: Set the peer limit total when per-direction limits are configured (#8220)
Co-authored-by: Bart <11445373+bthomee@users.noreply.github.com>
Co-authored-by: Vito Tumas <5780819+Tapanito@users.noreply.github.com>
2026-09-15 13:14:12 +00:00
Pratik Mankawde
e92548a185 merge: bring the clang-tidy include-cleaner fixes forward from phase7-native-metrics 2026-09-15 11:38:45 +01:00
Pratik Mankawde
118565a8af merge: bring the clang-tidy include-cleaner fixes forward from phase8-log-correlation 2026-09-15 11:38:45 +01:00
Pratik Mankawde
ea0bd444c8 fix(test): clear clang-tidy include-cleaner and nodiscard findings
OTelCollectorHooks.cpp named nothing from Hook.h, took ResourceMetrics
without a direct include, and left the overriding
GetAggregationTemporality() unmarked. NodeIdentity.h carried <optional>
and <string> it no longer uses. All fail CI under warnings-as-errors.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-09-15 11:38:26 +01:00
Pratik Mankawde
ec28856a20 merge: bring the SecretKey test-compare fix forward from phase7-native-metrics 2026-09-15 11:24:42 +01:00
Pratik Mankawde
59a833571a merge: bring the SecretKey test-compare fix forward from phase8-log-correlation 2026-09-15 11:24:42 +01:00
Pratik Mankawde
8cd99e17e9 fix(test): compare SecretKey by bytes and drop an include CI flags
SecretKey deletes operator==, so EXPECT_EQ on the pair's second member
does not compile; compare the byte ranges instead. Application.cpp no
longer calls logicError(), so clang-tidy's include-cleaner rejects
<xrpl/basics/contract.h> there.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-15 11:24:21 +01:00
Mayukha Vadari
1a4a40ebb8 fix: Update noripple_check to exclude transactions field on error responses (#6303)
Co-authored-by: Timur Yalymov <36795566+tyalymov@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Vito Tumas <5780819+Tapanito@users.noreply.github.com>
Co-authored-by: Ayaz Salikhov <mathbunnyru@users.noreply.github.com>
2026-09-15 00:21:57 +00:00
Pratik Mankawde
26a2bafb23 merge: bring the noopMeter() clang-tidy fixes forward from phase7-native-metrics 2026-09-14 23:48:23 +01:00
Pratik Mankawde
5df8f71c7a merge: bring the noopMeter() clang-tidy fixes forward from phase8-log-correlation 2026-09-14 23:48:23 +01:00
Pratik Mankawde
e5950af95f merge: bring the StatsD test include fix forward from phase6-statsd 2026-09-14 23:48:23 +01:00
Pratik Mankawde
7a4806c00a fix(telemetry): satisfy clang-tidy on the noopMeter() helper
Static constants take the k prefix (readability-identifier-naming), and
SpanGuardScope.cpp no longer names anything from <opentelemetry/metrics/noop.h>
since it calls noopMeter(). Both fail CI under warnings-as-errors.

The helper's docstring also claimed NoopMeterProvider hides the base
two-argument GetMeter; it declares that overload itself, so the only
detail worth sharing is the version.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-14 23:47:55 +01:00
Pratik Mankawde
7b41b00bab test(insight): drop two includes the StatsD collector test never uses
clang-tidy's include-cleaner flags Counter.h and Gauge.h here: the test
names neither type, so the includes fail CI under warnings-as-errors.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-14 23:47:14 +01:00
Pratik Mankawde
6dc961f195 merge: bring the resolveNodeIdentity() decision tests forward from phase8-log-correlation 2026-09-14 23:34:57 +01:00
Pratik Mankawde
b8dbba917a merge: bring the resolveNodeIdentity() decision tests forward from phase7-native-metrics 2026-09-14 23:34:41 +01:00
Pratik Mankawde
6c218f9d19 fix(telemetry): gate record calls so stop() cannot hit a dead pipeline
stop() destroys the MeterProvider, and with it every View's
AggregationConfig. The SDK's SyncMetricStorage keeps a raw pointer to
that config, and the call-site statics keep the storage alive, so a
histogram record with a first-seen attribute set during the shutdown
drain would dereference freed memory. Application::run() stops the
registry before the job queue and server handler, so that window is
real.

phase_ is now atomic and stop() stores Stopped before tearing down.
Every XRPL_METRIC_* macro and every record*/increment* method checks
recording() (enabled and not stopped) instead of isEnabled(). meter_ is
never written after construction, so record threads read it without a
lock.

Also: an empty [telemetry] service_instance_id now falls back to the
node key on both the trace and the metrics side, so one node reports one
identity; disablePipeline() uses telemetry::noopMeter(); comments
corrected.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-14 23:31:33 +01:00
Pratik Mankawde
e55f48caf8 test(server): cover every resolveNodeIdentity() decision branch
Lift the seed parsing and the stored-vs-mint choice into two libxrpl
helpers, parseNodeIdentitySeed() and selectNodeIdentity(), so xrpl_tests
can drive each branch without an xrpld Config. resolveNodeIdentity() now
marshals Config and the cmdline into them; behaviour is unchanged.

Also pin that storeNodeIdentity() appends (row count, not SQLite row
order), fix the test header that described getNodeIdentity()'s property
as the store's, and route NullTelemetry::getMeter() through noopMeter().

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-14 23:31:27 +01:00
Pratik Mankawde
f678241126 merge: bring phase-8 forward
Merges pratik/otel-phase8-log-correlation into pratik/otel-phase9-metric-gap-fill.

Auto-merged. TESTING.md picked up phase-8's Test 4 rewrite alongside this
branch's Step 2 standalone-store fix. OTelCollector.cpp gained the weak_ptr
gauges_ list; the StatsD test gained its onCollectionReady() calls.
2026-09-14 21:11:01 +01:00
Pratik Mankawde
0c2b002206 merge: bring phase-7's gauges weak-ref refactor forward
Merges pratik/otel-phase7-native-metrics into pratik/otel-phase8-log-correlation.

Conflict was one TESTING.md hunk under "Nodes not reaching proposing state":
this branch renamed the node directories to Node-N in integration-test.sh,
phase-7 kept nodeN and expanded the [peer_private] explanation. Resolution
keeps this branch's Node-1 path (its own script uses that naming) and
phase-7's fuller prose citing peerfinder/Config.cpp.

Non-conflicting phase-7 changes come through: OTelCollector's gauges_ list
becomes weak_ptr, matching the earlier hooks_ change; the phase-6 revert of
the StatsD-test onCollectionReady() calls resolved against phase-7's version
that keeps them.
2026-09-14 21:09:56 +01:00
Pratik Mankawde
28773e903b merge: bring the phase-6 revert forward, keeping phase-7's collection lifecycle
Merges pratik/otel-phase6-statsd into pratik/otel-phase7-native-metrics.
Phase-6 dropped the three onCollectionReady() calls that had been added to
its StatsD test, because that method is only declared here on phase-7.
This branch's own copy of the file was unchanged from the merge base, so
the default merge would have silently deleted the calls from here too —
where they are needed, because this branch gates polling behind
onCollectionReady() in OTelCollectorImp::onTimer.

Resolution keeps both sides: phase-6's two new include lines
(Counter.h, Gauge.h) and phase-7's three onCollectionReady() calls plus
their doxygen and inline explanations. The merged file is exactly
phase-7's tip plus those two includes.

TESTING.md auto-merged cleanly; both sides added text under Test 1 in
different regions.
2026-09-14 21:07:59 +01:00
Pratik Mankawde
f04f03b6b2 fix(telemetry): drop the optional dereference the identity change left behind
nodeIdentity_ is no longer a std::optional, so setNodeId() must read it
directly. The merge could not flag this: phase-8 changed the member's type and
this line lives only on phase-9, so neither side of the merge touched the same
file region.
2026-09-14 20:42:08 +01:00