Commit Graph

724 Commits

Author SHA1 Message Date
Pratik Mankawde
8bca6a8351 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-09-22 15:38:47 +01:00
Pratik Mankawde
f68375213e Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-09-22 15:38:36 +01:00
Pratik Mankawde
a570216c94 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-09-22 15:38:21 +01:00
Pratik Mankawde
7b275f6c0d Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing 2026-09-22 15:38:17 +01:00
Pratik Mankawde
dad8838c31 Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing 2026-09-22 15:38:12 +01:00
Pratik Mankawde
3b2192a48b Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration 2026-09-22 15:38:06 +01:00
Pratik Mankawde
359ae64170 Merge branch 'pratik/otel-phase1a-plan-docs' into pratik/otel-phase1b-telemetry-infra 2026-09-22 15:38:01 +01:00
dependabot[bot]
0229c294a9 ci: [DEPENDABOT] bump codecov/codecov-action from 7.0.0 to 7.1.1 in the github-actions group across 1 directory (#8251)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-21 12:09:57 +00:00
Pratik Mankawde
5e398cd6b7 merge: bring develop forward into phase-9 metric gap fill
Resolved five files.

PerfLogImp.cpp: PerfLog now takes its method names as string_view, so the
methods map stores the map key rather than a char const*. Kept this branch's
scoped lock so the OTel recording still runs after methodsMutex is released,
and kept both the NetworkOPs and metric-macro includes.

PeerImp.h: took develop's earlier `protected:`, which also covers
processLedgerRequest, and kept recordGetObjectMetrics inside it. Dropped the
comment naming one test, since four test files now rely on that access.

TMGetObjectByHash_test.cpp: kept this branch's suite. It already runs develop's
three bound cases (limit+1, limit, limit-1) as testReplyLimit, plus the charge
and hit/miss cases. develop's extracted PeerTest.h stays, used by its four new
overlay tests.

ordering.txt: regenerated. Both perflog edges are real, to xrpl.server and to
xrpl.telemetry.

.cspell.config.yaml: kept both added words.
2026-09-21 12:51:31 +01:00
Pratik Mankawde
05ce773671 Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-09-21 12:45:59 +01:00
Pratik Mankawde
e5d28ccbc4 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-09-21 12:45:54 +01:00
Pratik Mankawde
870367d71e merge: bring develop forward into phase-3 tx tracing
Resolved src/libxrpl/tx/applySteps.cpp. calculateBaseFee now returns
std::expected<XRPAmount, TER>, so invokePreclaim rejects the transaction when
the fee cannot be computed. Kept that inside this branch's preclaimTer wrapper,
so the returned error also reaches the span's ter_result attribute.
2026-09-21 12:45:45 +01:00
Pratik Mankawde
c030880a36 Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing 2026-09-21 12:44:43 +01:00
Pratik Mankawde
d16f2c4181 Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration 2026-09-21 12:43:48 +01:00
Pratik Mankawde
f48a2b4871 Merge branch 'pratik/otel-phase1a-plan-docs' into pratik/otel-phase1b-telemetry-infra 2026-09-21 12:43:44 +01:00
Bart
f6b51f0b8b ci: Say when a package publish is a dry run (#8247)
Co-authored-by: Bart <11445373+bthomee@users.noreply.github.com>
2026-09-18 18:31:07 +00:00
Denis Angell
07b36871c7 ci: Publish packages through packages-upload.xrplf.org (#8241) 2026-09-17 18:05:51 +00:00
Ayaz Salikhov
d6022fbc4d build: Fix test installation on debian:11 due to EOL 2026-09-17 13:33:06 +02:00
Ayaz Salikhov
295b74da1c build: Add assert-enabled builds and packages 2026-09-17 13:33:05 +02:00
Bart
04108a030c refactor: Build the RPC dispatch and command-line tables at compile time (#8006)
Co-authored-by: Bart <11445373+bthomee@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-17 10:01:28 +00:00
Pratik Mankawde
ec0bfe521d refactor(telemetry): move the metrics pipeline core into libxrpl
MetricsRegistry did two jobs. It owned the OTel metrics pipeline, and it
registered the observable gauges whose callbacks read live application
services. The second job is what made the whole class xrpld-tier, so the
pipeline's lifecycle -- the recording() gate and the stop() teardown that
closes a use-after-free window -- could not be unit-tested in xrpl_tests.

Split it in two:

- xrpl::telemetry::MetricsRegistry (libxrpl) owns the exporter, provider,
  meter, the 16 synchronous instruments, recording(), stop(), and the
  record*/increment* methods.
- xrpl::telemetry::AppMetricGauges (xrpld) owns the 19 observable gauges
  and their callbacks, holding a reference to the core and to the
  ServiceRegistry.

MetricMacros.h and ValidationTracker move with the core. The macros need
only recording() and meter(), both core members; the core holds a tracker
by value, and a libxrpl header cannot include one from src/.

ApplicationImp owns both objects and sequences them. The core is built in
the member-init list, so every synchronous instrument exists before any
subsystem can record one. The gauges are armed once overlay_ exists, the
last service their callbacks read. Shutdown detaches the gauge callbacks
before the core drops the provider, and each shutdown step is isolated so
a failure in one cannot skip the others.

That detach call is new. detachCallbacks() had no callers, and the flag it
sets is read by the gauge callbacks but can no longer be written by the
core, so the caller now has to make the ordering explicit.

The telemetry module links xrpl.libxrpl.core and xrpl.libxrpl.protocol
PUBLIC: ValidationTracker.h takes a LedgerIndex and MetricMacros.h takes a
ServiceRegistry, both in interfaces a consumer compiles against.

Adds a MetricsRegistry gtest that drives an enabled core with telemetry on
and pins the recording() gate, stop() leaving the registry inert, and
stop() being idempotent. The libxrpl test tree no longer depends on
xrpld.telemetry at all, and the two CMake workarounds that compiled xrpld
sources into xrpl_tests are gone.

Documentation and dashboard source links follow the code to their new
paths, split between the two classes by which one now defines each metric.
2026-09-16 13:45:52 +01:00
Pratik Mankawde
e476241b5f Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-09-15 16:17:44 +01:00
Pratik Mankawde
8bd043ecb7 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-09-15 16:17:44 +01:00
Pratik Mankawde
9545ab4d4e Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing 2026-09-15 16:17:43 +01:00
Pratik Mankawde
5b10ef0d56 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-09-15 16:17:43 +01:00
Pratik Mankawde
b14c537df1 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing 2026-09-15 16:17:43 +01:00
Pratik Mankawde
6a0642817d Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration 2026-09-15 16:17:43 +01:00
Pratik Mankawde
d008b2f641 Merge branch 'pratik/otel-phase1a-plan-docs' into pratik/otel-phase1b-telemetry-infra 2026-09-15 16:17:43 +01:00
Pratik Mankawde
b10fe32657 merge: bring the node-identity change forward from phase-8
Three conflicts, all from both branches editing the same passage:

- Main.cpp: kept phase-9's wording. The metrics registry only exists here, so
  "unwinding destroys little: the metrics registry, whose destructor joins its
  export thread" is the true statement on this branch.
- TESTING.md: kept both paragraphs. They document different things (the
  private [network_id], and the log path plus log_level).
- 05-configuration-reference.md: composed both. The identity is now resolved
  before construction and never empty, so every producer stamps the node key
  from the start; the only divergence left is a wallet that already holds a
  different key, which corrects the tracer alone. Rewrote the earlier
  "three producers" blockquote too: its "no fallback", "first boot ... left
  off" and "Known issue" claims are what this change removes.

One silent break the merge could not flag: makeMetricsRegistryOptions() took
the std::optional<std::string> node key that used to be a constructor
parameter, and that parameter is now the resolved keypair. It takes the base58
string directly, and the constructor derives it from nodeIdentity_, which is
declared before both telemetry_ and metricsRegistry_.
2026-09-14 20:39:18 +01:00
Pratik Mankawde
bec9e1c8a9 fix(telemetry): resolve the node identity before the Application is built
resolveNodePublicKey() returned std::nullopt in three real cases: a first boot
with no wallet database, a standalone run (its wallet is a private temporary
database), and --newnodeid. Telemetry's resources are built during
ApplicationImp's member-init list and are immutable, so on those runs the node
reported an empty service.instance.id and no xrpl.node.id for the whole run,
while setup() minted a key moments later and patched only the tracer.

Replace it with resolveNodeIdentity(), which always returns a keypair: derived
from a configured seed, else read from an existing wallet database, else
minted. Main.cpp passes that pair to makeApplication(), ApplicationImp stores
it in nodeIdentity_ -- now declared before telemetry_ and no longer an optional,
because it is always set -- and builds the telemetry resource from it.

setup() calls getNodeIdentity(), which now persists rather than mints: it
stores the resolved pair when the wallet holds no identity, adopts the stored
one when it does, and clears first for --newnodeid. The write stays in setup()
because that is where the database exists; a standalone run has no persistent
wallet to write to, which is why the pair has to be decided before
construction rather than read back afterwards. Wallet gains storeNodeIdentity()
for that write, and getNodeIdentity(session) now uses it instead of repeating
the insert.

The three-argument makeApplication() mints a keypair, so jtx::Env and any other
test Application behave as a standalone run always did.

Also fold the three hand-rolled "meter from a NoopMeterProvider" copies into
telemetry::noopMeter(): the base-pointer call and the kMeterVersion argument are
both easy to get wrong alone, and the meter identity has to match the one the
histogram views select on.

The new gtest covers the wallet half: store-then-read, store not replacing an
existing identity, clear-then-store, and that the mint path persists. It adds
the tests.libxrpl > xrpl.rdb levelization edge, regenerated here.
2026-09-14 20:34:31 +01:00
Sergey Kuznetsov
9403736199 ci: Exclude Rust unit tests from code coverage (#8203) 2026-09-10 11:38:25 +00:00
Ayaz Salikhov
1381483c7a build: Fix test installation on debian:11 due to EOL (#8200) 2026-09-08 22:49:16 +00:00
Pratik Mankawde
bae31955db merge: bring the CI fixes forward from phase8-log-correlation
One conflicted file, docs/telemetry-runbook.md, with three spots:

- Build section: both sides added different text at one point. Kept both,
  incoming sentence first, then this branch's "Run against a live network".
- Disabling section, first spot: this branch's wording names the config
  section and says no rebuild is needed, so it already covers the incoming
  sentence.
- Disabling section, second spot: kept this branch's paragraph and folded in
  the one point it lacked, that both flags have to be passed.
2026-09-08 17:22:04 +01:00
Pratik Mankawde
ff1a905ede merge: bring the CI fixes forward from phase6-statsd
Three conflicts, all resolved by keeping this branch's rewrite and
re-applying the incoming change onto it:

- 09-data-collection-reference.md: phase-7 rewrote both attribute tables,
  so the incoming table would have reverted them. Kept phase-7's and
  re-applied the two "XRPL epoch" spellings.
- integration-test.sh: phase-7 moved these checks from StatsD to OTel and
  no longer defines check_statsd_metric, so only this side compiles.
- TelemetryConfig.cpp: the incoming side carried networkTypeFromId(), which
  this branch already has. Kept one definition and took the incoming
  doc wording, which the auto-merged body below it already matches.
2026-09-08 17:17:50 +01:00
Pratik Mankawde
dc2a8f121e Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-09-08 16:03:00 +01:00
Pratik Mankawde
0fd7ce9f8b Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing 2026-09-08 16:02:54 +01:00
Pratik Mankawde
96c11264b0 Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing 2026-09-08 16:02:50 +01:00
Pratik Mankawde
31ace1a357 Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration 2026-09-08 16:02:42 +01:00
Pratik Mankawde
9e72c1a8bf Merge branch 'pratik/otel-phase1a-plan-docs' into pratik/otel-phase1b-telemetry-infra 2026-09-08 16:02:36 +01:00
dependabot[bot]
3e4bdf2782 ci: [DEPENDABOT] bump actions/deploy-pages from 5.0.0 to 5.0.1 in the github-actions group across 1 directory (#8180)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-07 16:07:56 +00:00
Pratik Mankawde
efdd9758f2 merge: bring the review fixes forward from otel-phase8-log-correlation 2026-09-07 15:39:59 +01:00
Pratik Mankawde
4ad84cae14 merge: bring the review fixes forward from otel-phase6-statsd 2026-09-07 15:23:22 +01:00
Pratik Mankawde
3d9ea4b9da merge: bring the review fixes forward from phase3-tx-tracing
One conflict, in 06-implementation-phases.md: this branch had rewritten the
phase-4 task table with a Status column, a descoping note and a Spans Produced
section, while upstream corrected the class name in the old plain table. This
branch's section is kept and the name correction re-applied to its 4.1 row.
2026-09-07 15:04:39 +01:00
Pratik Mankawde
08df97f431 merge: bring the review fixes forward from phase2-rpc-tracing
One conflict, in SpanGuard.h: this branch added struct TraceBytes and upstream
added enum SpanRole at the same position after TraceCategory. Unrelated
declarations, so both are kept.
2026-09-07 15:03:44 +01:00
Pratik Mankawde
caa704de10 merge: bring the review fixes forward from phase1c-rpc-integration
Two conflicts, both resolved by composing the sides rather than taking one.

cfg/xrpld-example.cfg: this branch had moved the batch-processor keys under
their own heading while upstream edited them in place, so a merge-both would
have documented them twice. Upstream's range sentences are applied to the
relocated block and the head-sampling note keeps its position.

02-design-decisions.md: the summary table changed on both sides for different
reasons. Upstream renamed ledger_index to current_ledger_seq and ledger_seq;
this branch had corrected the PathFinding row to the keys it actually emits.
Both are kept.
2026-09-07 15:00:53 +01:00
Pratik Mankawde
623e3d8ad3 feat(telemetry): add batch-size and discovered-path histograms
Review feedback asked for a Histogram rather than a span attribute at these two
places. Both, not either: the attribute answers how big one sampled request was,
which an aggregate cannot, and the histogram answers the distribution across all
requests, which an unsampled trace never reveals. Both attributes stay.

The metrics land here rather than with the attributes because neither
HistogramBuckets.h nor the metric macro exists on the branch that added them.

Both use kObjectCountBuckets. The argument is the floor, not the ceiling: the SDK
default edges start 0,5,10,25, so an ordinary batch of one to five falls in a
single bucket and every quantile becomes an interpolation on one edge. The object
ladder puts five edges over the mass of both distributions. Path count is bounded
at 352 by kMaxPaths times kMaxAutoSrcCur and cannot saturate. Batch size can, at
roughly 333k, but no measured traffic goes near it, so the ladder is not widened
for a range nothing occupies; the runbook records the overflow query and a test
asserts it stays readable.
2026-09-07 13:51:44 +01:00
Pratik Mankawde
f8e0a19b9f fix(telemetry): correct the childSpan doc examples and make Rule D tests real
Review feedback on the RPC integration PR.

The childSpan examples could not work as written. childSpan() takes its parent
from the ambient context and uses impl_ only as a liveness gate, so an unscoped
SpanGuard parent produced two siblings rather than a parent and child. The parent
is now a ScopedSpanGuard, the child no longer reuses the parent's name, and the
examples pass a full dotted constant because childSpan() takes the name verbatim.

Five of the ten Rule D tests could not fail. Four passed an empty L1 key set,
which makes the rule skip validation altogether; the fifth asserted an empty
result against an escaped-quote selector that extracted no labels at all. Each
now passes a nonempty L1 set and carries a known-bad label in the same
expression, so it asserts both that the intended labels are accepted and that
Rule D ran. Verified by disabling the rule: the old tests stay green, the new
ones all fail.

Span kind is not fixed here. categoryToSpanKind and the span factories belong to
the telemetry library, so the role parameter is routed to that branch, and the
two call sites here follow once it exists.
2026-09-07 13:25:22 +01:00
Ayaz Salikhov
3e54e7d00b fix: Don't use github.workspace as it slow downs gcovr (#8173) 2026-09-03 19:20:38 +00:00
Ayaz Salikhov
986065c16f build: Verify glibc version was determined in debian package (#8170) 2026-09-03 16:54:54 +00:00
Pratik Mankawde
54d6e80e2e merge: bring develop forward from phase8-log-correlation
Two conflicts, both additions at the same spot:
NetworkOPs.cpp include block kept develop's rpc/detail/SyntheticFields.h
alongside this branch's telemetry/MetricsRegistry.h; .cspell.config.yaml
kept both new words.
2026-09-03 17:39:21 +01:00