Commit Graph

852 Commits

Author SHA1 Message Date
Pratik Mankawde
b6072f2677 Merge branch 'pratik/otel-phase10-workload-validation' into pratik/otel-sync-diagnostics 2026-09-23 18:46:25 +01:00
Pratik Mankawde
8ec0044bca Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-09-23 18:44:57 +01:00
Pratik Mankawde
26db3eff82 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-09-23 18:44:45 +01:00
Pratik Mankawde
e8b5322c5b Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-09-23 18:42:00 +01:00
Pratik Mankawde
6bf8cbf265 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-09-23 18:39:12 +01:00
Pratik Mankawde
334d2346b0 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing 2026-09-23 18:39:12 +01:00
Pratik Mankawde
7aaa985c0c Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing 2026-09-23 18:39:12 +01:00
Pratik Mankawde
5f0d3a4186 Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration 2026-09-23 18:39:12 +01:00
Pratik Mankawde
1db1ae9b54 Merge branch 'pratik/otel-phase1a-plan-docs' into pratik/otel-phase1b-telemetry-infra 2026-09-23 18:39:12 +01:00
Bart
d8870162eb test: Share peer test doubles across the overlay and app suites (#8246)
Co-authored-by: Bart <11445373+bthomee@users.noreply.github.com>
2026-09-22 20:03:55 +00:00
Pratik Mankawde
6b1ced940e Merge branch 'pratik/otel-phase10-workload-validation' into pratik/otel-sync-diagnostics 2026-09-22 15:43:12 +01:00
Pratik Mankawde
406e38275a Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-09-22 15:42:55 +01:00
Pratik Mankawde
8bca6a8351 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-09-22 15:38:47 +01:00
Pratik Mankawde
f68375213e Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-09-22 15:38:36 +01:00
Pratik Mankawde
a570216c94 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-09-22 15:38:21 +01:00
Pratik Mankawde
7b275f6c0d Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing 2026-09-22 15:38:17 +01:00
Pratik Mankawde
dad8838c31 Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing 2026-09-22 15:38:12 +01:00
Pratik Mankawde
3b2192a48b Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration 2026-09-22 15:38:06 +01:00
Pratik Mankawde
359ae64170 Merge branch 'pratik/otel-phase1a-plan-docs' into pratik/otel-phase1b-telemetry-infra 2026-09-22 15:38:01 +01:00
dependabot[bot]
0229c294a9 ci: [DEPENDABOT] bump codecov/codecov-action from 7.0.0 to 7.1.1 in the github-actions group across 1 directory (#8251)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-21 12:09:57 +00:00
Pratik Mankawde
143538581c Merge branch 'pratik/otel-phase10-workload-validation' into pratik/otel-sync-diagnostics 2026-09-21 12:52:31 +01:00
Pratik Mankawde
34b71571a0 Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-09-21 12:52:23 +01:00
Pratik Mankawde
5e398cd6b7 merge: bring develop forward into phase-9 metric gap fill
Resolved five files.

PerfLogImp.cpp: PerfLog now takes its method names as string_view, so the
methods map stores the map key rather than a char const*. Kept this branch's
scoped lock so the OTel recording still runs after methodsMutex is released,
and kept both the NetworkOPs and metric-macro includes.

PeerImp.h: took develop's earlier `protected:`, which also covers
processLedgerRequest, and kept recordGetObjectMetrics inside it. Dropped the
comment naming one test, since four test files now rely on that access.

TMGetObjectByHash_test.cpp: kept this branch's suite. It already runs develop's
three bound cases (limit+1, limit, limit-1) as testReplyLimit, plus the charge
and hit/miss cases. develop's extracted PeerTest.h stays, used by its four new
overlay tests.

ordering.txt: regenerated. Both perflog edges are real, to xrpl.server and to
xrpl.telemetry.

.cspell.config.yaml: kept both added words.
2026-09-21 12:51:31 +01:00
Pratik Mankawde
05ce773671 Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-09-21 12:45:59 +01:00
Pratik Mankawde
e5d28ccbc4 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-09-21 12:45:54 +01:00
Pratik Mankawde
870367d71e merge: bring develop forward into phase-3 tx tracing
Resolved src/libxrpl/tx/applySteps.cpp. calculateBaseFee now returns
std::expected<XRPAmount, TER>, so invokePreclaim rejects the transaction when
the fee cannot be computed. Kept that inside this branch's preclaimTer wrapper,
so the returned error also reaches the span's ter_result attribute.
2026-09-21 12:45:45 +01:00
Pratik Mankawde
c030880a36 Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing 2026-09-21 12:44:43 +01:00
Pratik Mankawde
d16f2c4181 Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration 2026-09-21 12:43:48 +01:00
Pratik Mankawde
f48a2b4871 Merge branch 'pratik/otel-phase1a-plan-docs' into pratik/otel-phase1b-telemetry-infra 2026-09-21 12:43:44 +01:00
Bart
f6b51f0b8b ci: Say when a package publish is a dry run (#8247)
Co-authored-by: Bart <11445373+bthomee@users.noreply.github.com>
2026-09-18 18:31:07 +00:00
Denis Angell
07b36871c7 ci: Publish packages through packages-upload.xrplf.org (#8241) 2026-09-17 18:05:51 +00:00
Pratik Mankawde
37bee9a61c Merge branch 'pratik/otel-phase10-workload-validation' into pratik/otel-sync-diagnostics 2026-09-17 17:02:47 +01:00
Pratik Mankawde
3c05bdfdd8 ci: call the telemetry validation from on-pr.yml
The validation owned its own push trigger and paths filter, which made it the one
build-and-test workflow outside the on-pr.yml plus reusable-*.yml pair the rest of
CI uses. Rename it to reusable-telemetry-validation.yml, take workflow_call, and
let on-pr.yml decide when it runs.

on-pr.yml gains a second changed-files list for the telemetry paths. Its existing
`go` flag is true for nearly every pull request, so gating on `go` alone would run
a self-hosted build and a 30-minute cluster on all of them.

The configure step no longer repeats the telemetry option. A command-line define
is written to the cache before the toolchain runs and wins over it, so it could
turn telemetry on with the package never fetched. build-xrpld reads the value the
build actually configured out of CMakeCache.txt and publishes it as a job output;
validate-telemetry runs only when that says telemetry is on, and every value CMake
does not accept as a boolean fails the job rather than skipping silently.
2026-09-17 16:55:56 +01:00
Ayaz Salikhov
d6022fbc4d build: Fix test installation on debian:11 due to EOL 2026-09-17 13:33:06 +02:00
Ayaz Salikhov
295b74da1c build: Add assert-enabled builds and packages 2026-09-17 13:33:05 +02:00
Bart
04108a030c refactor: Build the RPC dispatch and command-line tables at compile time (#8006)
Co-authored-by: Bart <11445373+bthomee@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-17 10:01:28 +00:00
Pratik Mankawde
f7037c8dc3 Merge branch 'pratik/otel-phase10-workload-validation' into pratik/otel-sync-diagnostics
Brings the MetricsRegistry split onto this branch. The pipeline half is now
xrpl::telemetry::MetricsRegistry in libxrpl; the observable gauges are
xrpl::telemetry::AppMetricGauges in xrpld.

This branch had added its own instrumentation to the pre-split class, so the
merge had to route each addition to the correct half:

- The thirteen gauges added here -- amendment block, cache hit-rate detail,
  clock skew, job-queue saturation, ledger quorum publish, peer ledger supply,
  rotation state, slot census, stall events, sync acquire, sync state, UNL
  quorum, and the cache lock-hold observer -- all land on AppMetricGauges,
  reading the core's meter and validation tracker through it.
- The pipeline additions stay in libxrpl: the consensus round-duration and
  rotation-phase histogram views, the malloc-trim and dns/dial latency bucket
  ladders, the job-stall counter, and the switch from literal metric names to
  the MetricNames.h constants.

Git detected the pre-split MetricsRegistry.cpp and .h as renames of the gauge
files, so both sides' pipeline changes initially landed in the gauge half. They
were moved back, and the result was audited by inventory: every method
definition, instrument creation, view registration, and emitted string from
either side is present, with identical multiplicity.

MetricNames.h moves to include/xrpl/telemetry/ alongside the core. It has no
includes of its own and its two sibling name headers already live there, so
keeping it under src/ would leave an xrpld path in libxrpl's dependency
surface. Nineteen files follow it.

incrementStateChanges() stays removed. The labelled state_changes_total{from,to}
counter this branch introduced replaces it, and the test asserting the method is
absent is kept -- an unlabelled instrument alongside the labelled one would give
Prometheus two conflicting versions of one metric name.

Two tests that drove startAsyncGauges() against a mock ServiceRegistry are
dropped: xrpl_tests links only xrpl.libxrpl and cannot reach the gauge class.

Levelization regenerated. Both xrpld.telemetry loops become bidirectional
rather than one-way; neither is new.
2026-09-16 18:05:48 +01:00
Pratik Mankawde
5f03e41f83 Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-09-16 13:46:34 +01:00
Pratik Mankawde
ec0bfe521d refactor(telemetry): move the metrics pipeline core into libxrpl
MetricsRegistry did two jobs. It owned the OTel metrics pipeline, and it
registered the observable gauges whose callbacks read live application
services. The second job is what made the whole class xrpld-tier, so the
pipeline's lifecycle -- the recording() gate and the stop() teardown that
closes a use-after-free window -- could not be unit-tested in xrpl_tests.

Split it in two:

- xrpl::telemetry::MetricsRegistry (libxrpl) owns the exporter, provider,
  meter, the 16 synchronous instruments, recording(), stop(), and the
  record*/increment* methods.
- xrpl::telemetry::AppMetricGauges (xrpld) owns the 19 observable gauges
  and their callbacks, holding a reference to the core and to the
  ServiceRegistry.

MetricMacros.h and ValidationTracker move with the core. The macros need
only recording() and meter(), both core members; the core holds a tracker
by value, and a libxrpl header cannot include one from src/.

ApplicationImp owns both objects and sequences them. The core is built in
the member-init list, so every synchronous instrument exists before any
subsystem can record one. The gauges are armed once overlay_ exists, the
last service their callbacks read. Shutdown detaches the gauge callbacks
before the core drops the provider, and each shutdown step is isolated so
a failure in one cannot skip the others.

That detach call is new. detachCallbacks() had no callers, and the flag it
sets is read by the gauge callbacks but can no longer be written by the
core, so the caller now has to make the ordering explicit.

The telemetry module links xrpl.libxrpl.core and xrpl.libxrpl.protocol
PUBLIC: ValidationTracker.h takes a LedgerIndex and MetricMacros.h takes a
ServiceRegistry, both in interfaces a consumer compiles against.

Adds a MetricsRegistry gtest that drives an enabled core with telemetry on
and pins the recording() gate, stop() leaving the registry inert, and
stop() being idempotent. The libxrpl test tree no longer depends on
xrpld.telemetry at all, and the two CMake workarounds that compiled xrpld
sources into xrpl_tests are gone.

Documentation and dashboard source links follow the code to their new
paths, split between the two classes by which one now defines each metric.
2026-09-16 13:45:52 +01:00
Pratik Mankawde
e476241b5f Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-09-15 16:17:44 +01:00
Pratik Mankawde
d86036f88a Merge branch 'pratik/otel-phase10-workload-validation' into pratik/otel-sync-diagnostics 2026-09-15 16:17:44 +01:00
Pratik Mankawde
5277ef4224 Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-09-15 16:17:44 +01:00
Pratik Mankawde
8bd043ecb7 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-09-15 16:17:44 +01:00
Pratik Mankawde
9545ab4d4e Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing 2026-09-15 16:17:43 +01:00
Pratik Mankawde
5b10ef0d56 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-09-15 16:17:43 +01:00
Pratik Mankawde
b14c537df1 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing 2026-09-15 16:17:43 +01:00
Pratik Mankawde
6a0642817d Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration 2026-09-15 16:17:43 +01:00
Pratik Mankawde
d008b2f641 Merge branch 'pratik/otel-phase1a-plan-docs' into pratik/otel-phase1b-telemetry-infra 2026-09-15 16:17:43 +01:00
Pratik Mankawde
1880c9a498 merge: bring phase10-workload-validation forward into sync-diagnostics
Resolutions:
- MetricsRegistry.cpp: keep both <exception> and <limits>; drop
  incrementStateChanges(), which this branch removed on purpose (the
  labelled state_changes_total call site in NetworkOPsImp::setMode
  replaces it, and a compile-time test guards that).
- tests/MetricsRegistry.cpp: constructor-built pipeline wording from
  phase-10, this branch's test list and gauge paragraphs kept; the two
  lifecycle tests now call startAsyncGauges() and pass kTestOptions.
- tests/MetricMacros.cpp: comments name the recording() gate.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-14 23:45:32 +01:00
Pratik Mankawde
6601697a29 Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-09-14 20:40:14 +01:00