Commit Graph

17325 Commits

Author SHA1 Message Date
Pratik Mankawde
f7037c8dc3 Merge branch 'pratik/otel-phase10-workload-validation' into pratik/otel-sync-diagnostics
Brings the MetricsRegistry split onto this branch. The pipeline half is now
xrpl::telemetry::MetricsRegistry in libxrpl; the observable gauges are
xrpl::telemetry::AppMetricGauges in xrpld.

This branch had added its own instrumentation to the pre-split class, so the
merge had to route each addition to the correct half:

- The thirteen gauges added here -- amendment block, cache hit-rate detail,
  clock skew, job-queue saturation, ledger quorum publish, peer ledger supply,
  rotation state, slot census, stall events, sync acquire, sync state, UNL
  quorum, and the cache lock-hold observer -- all land on AppMetricGauges,
  reading the core's meter and validation tracker through it.
- The pipeline additions stay in libxrpl: the consensus round-duration and
  rotation-phase histogram views, the malloc-trim and dns/dial latency bucket
  ladders, the job-stall counter, and the switch from literal metric names to
  the MetricNames.h constants.

Git detected the pre-split MetricsRegistry.cpp and .h as renames of the gauge
files, so both sides' pipeline changes initially landed in the gauge half. They
were moved back, and the result was audited by inventory: every method
definition, instrument creation, view registration, and emitted string from
either side is present, with identical multiplicity.

MetricNames.h moves to include/xrpl/telemetry/ alongside the core. It has no
includes of its own and its two sibling name headers already live there, so
keeping it under src/ would leave an xrpld path in libxrpl's dependency
surface. Nineteen files follow it.

incrementStateChanges() stays removed. The labelled state_changes_total{from,to}
counter this branch introduced replaces it, and the test asserting the method is
absent is kept -- an unlabelled instrument alongside the labelled one would give
Prometheus two conflicting versions of one metric name.

Two tests that drove startAsyncGauges() against a mock ServiceRegistry are
dropped: xrpl_tests links only xrpl.libxrpl and cannot reach the gauge class.

Levelization regenerated. Both xrpld.telemetry loops become bidirectional
rather than one-way; neither is new.
2026-09-16 18:05:48 +01:00
Pratik Mankawde
5f03e41f83 Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-09-16 13:46:34 +01:00
Pratik Mankawde
ec0bfe521d refactor(telemetry): move the metrics pipeline core into libxrpl
MetricsRegistry did two jobs. It owned the OTel metrics pipeline, and it
registered the observable gauges whose callbacks read live application
services. The second job is what made the whole class xrpld-tier, so the
pipeline's lifecycle -- the recording() gate and the stop() teardown that
closes a use-after-free window -- could not be unit-tested in xrpl_tests.

Split it in two:

- xrpl::telemetry::MetricsRegistry (libxrpl) owns the exporter, provider,
  meter, the 16 synchronous instruments, recording(), stop(), and the
  record*/increment* methods.
- xrpl::telemetry::AppMetricGauges (xrpld) owns the 19 observable gauges
  and their callbacks, holding a reference to the core and to the
  ServiceRegistry.

MetricMacros.h and ValidationTracker move with the core. The macros need
only recording() and meter(), both core members; the core holds a tracker
by value, and a libxrpl header cannot include one from src/.

ApplicationImp owns both objects and sequences them. The core is built in
the member-init list, so every synchronous instrument exists before any
subsystem can record one. The gauges are armed once overlay_ exists, the
last service their callbacks read. Shutdown detaches the gauge callbacks
before the core drops the provider, and each shutdown step is isolated so
a failure in one cannot skip the others.

That detach call is new. detachCallbacks() had no callers, and the flag it
sets is read by the gauge callbacks but can no longer be written by the
core, so the caller now has to make the ordering explicit.

The telemetry module links xrpl.libxrpl.core and xrpl.libxrpl.protocol
PUBLIC: ValidationTracker.h takes a LedgerIndex and MetricMacros.h takes a
ServiceRegistry, both in interfaces a consumer compiles against.

Adds a MetricsRegistry gtest that drives an enabled core with telemetry on
and pins the recording() gate, stop() leaving the registry inert, and
stop() being idempotent. The libxrpl test tree no longer depends on
xrpld.telemetry at all, and the two CMake workarounds that compiled xrpld
sources into xrpl_tests are gone.

Documentation and dashboard source links follow the code to their new
paths, split between the two classes by which one now defines each metric.
2026-09-16 13:45:52 +01:00
Pratik Mankawde
d29e392c0b alert(NodeStateFlapping): fire on a single flap, keep the always-present metric
The rule watched state_accounting_full_transitions > 3 per hour, so a node
that flaps once (one full -> syncing -> full round, e.g. per online-delete
rotation) never tripped it. Lower the threshold to > 0 so a single re-entry
into FULL, past the one-hour uptime gate, alerts.

Keep the state_accounting_full_transitions metric: it is a cumulative gauge
every node always reports, so increase() yields a real series (0 when
healthy) and the rule never evaluates to NoData. A sparse counter would
raise a false DatasourceNoData on a healthy node. Set noDataState: OK so a
scrape gap cannot page either.
2026-09-15 16:57:52 +01:00
Pratik Mankawde
d86036f88a Merge branch 'pratik/otel-phase10-workload-validation' into pratik/otel-sync-diagnostics 2026-09-15 16:17:44 +01:00
Pratik Mankawde
5277ef4224 Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-09-15 16:17:44 +01:00
Pratik Mankawde
8bd043ecb7 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-09-15 16:17:44 +01:00
Pratik Mankawde
913721b0bc Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-09-15 16:17:44 +01:00
Pratik Mankawde
e476241b5f Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-09-15 16:17:44 +01:00
Pratik Mankawde
260bc7f2ed Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd 2026-09-15 16:17:44 +01:00
Pratik Mankawde
75661c3a33 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-09-15 16:17:44 +01:00
Pratik Mankawde
5b10ef0d56 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-09-15 16:17:43 +01:00
Pratik Mankawde
b14c537df1 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing 2026-09-15 16:17:43 +01:00
Pratik Mankawde
9545ab4d4e Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing 2026-09-15 16:17:43 +01:00
Pratik Mankawde
6a0642817d Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration 2026-09-15 16:17:43 +01:00
Pratik Mankawde
d008b2f641 Merge branch 'pratik/otel-phase1a-plan-docs' into pratik/otel-phase1b-telemetry-infra 2026-09-15 16:17:43 +01:00
Pratik Mankawde
a910a2dc91 Merge remote-tracking branch 'origin/develop' into pratik/otel-phase1a-plan-docs 2026-09-15 16:17:43 +01:00
Pratik Mankawde
11b03b460d docs(telemetry): drop the devnet host name from the p99 example
The consensus-round p99 note named an internal devnet host. State 'a
devnet node' instead; the measurement is the point, not the host.
2026-09-15 14:44:07 +01:00
Pratik Mankawde
0e125d08d7 Merge branch 'pratik/otel-phase10-workload-validation' into pratik/otel-sync-diagnostics 2026-09-15 14:27:16 +01:00
Pratik Mankawde
8efc8e90b5 Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-09-15 14:27:05 +01:00
Pratik Mankawde
b1345fff8d docs(telemetry): describe the rotation stall without internal host names
The reference doc, span-harness notes and histogram-bucket comments
named the internal AWS dev box and dates while explaining why the
rotation phases are timed. Reword to the general mechanism (a
multi-second freeze at the copy-walk to freshen boundary on a populated
node); the specific hosts, dates and trace ids stay in the task notes.
2026-09-15 14:26:25 +01:00
Pratik Mankawde
50eff17dd4 docs(telemetry): drop the host name from the sampling-clock comment
The comment measured date +%s%N cost 'on a dev box'; say 'on one Linux
host' instead. The number is the point, not where it was taken.
2026-09-15 14:26:23 +01:00
Pratik Mankawde
866ab77ece docs(telemetry): describe rotation measurements without naming the host
The runbook provenance paragraph named the internal AWS dev box and a
build hash and dates. State what was measured (one mainnet node, same
host and binary, differing only in store state) without the deployment
detail, which belongs in an internal runbook, not the public repo.
2026-09-15 14:26:17 +01:00
Pratik Mankawde
b71121e545 merge: bring the optional-access NOLINT fix forward from phase10-workload-validation 2026-09-15 14:22:59 +01:00
Pratik Mankawde
a305b105bf merge: bring the optional-access NOLINT fix forward from phase9-metric-gap-fill 2026-09-15 14:22:59 +01:00
Pratik Mankawde
751c23bd98 merge: bring the optional-access NOLINT fix forward from phase7-native-metrics 2026-09-15 14:22:58 +01:00
Pratik Mankawde
0306120ed3 merge: bring the optional-access NOLINT fix forward from phase8-log-correlation 2026-09-15 14:22:58 +01:00
Pratik Mankawde
42181b8ed2 test(server): silence false-positive optional-access on asserted reads
clang-tidy's bugprone-unchecked-optional-access does not model GTest's
ASSERT_TRUE(x.has_value()), so it flags every deref that follows one.
The reads are guarded; mark them NOLINT, matching the same suppression
in src/tests/libxrpl/consensus/LedgerTrie.cpp. .value() does not help --
the checker treats it as an unchecked access too.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-09-15 14:22:40 +01:00
Bart
e302e4eeed fix: Set the peer limit total when per-direction limits are configured (#8220)
Co-authored-by: Bart <11445373+bthomee@users.noreply.github.com>
Co-authored-by: Vito Tumas <5780819+Tapanito@users.noreply.github.com>
2026-09-15 13:14:12 +00:00
Pratik Mankawde
13f807b023 merge: bring the clang-tidy include-cleaner fixes forward from phase10-workload-validation 2026-09-15 11:38:46 +01:00
Pratik Mankawde
d87ca870cc merge: bring the clang-tidy include-cleaner fixes forward from phase9-metric-gap-fill 2026-09-15 11:38:46 +01:00
Pratik Mankawde
e92548a185 merge: bring the clang-tidy include-cleaner fixes forward from phase7-native-metrics 2026-09-15 11:38:45 +01:00
Pratik Mankawde
118565a8af merge: bring the clang-tidy include-cleaner fixes forward from phase8-log-correlation 2026-09-15 11:38:45 +01:00
Pratik Mankawde
ea0bd444c8 fix(test): clear clang-tidy include-cleaner and nodiscard findings
OTelCollectorHooks.cpp named nothing from Hook.h, took ResourceMetrics
without a direct include, and left the overriding
GetAggregationTemporality() unmarked. NodeIdentity.h carried <optional>
and <string> it no longer uses. All fail CI under warnings-as-errors.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-09-15 11:38:26 +01:00
Pratik Mankawde
f88af116b4 merge: bring the SecretKey test-compare fix forward from phase10-workload-validation 2026-09-15 11:24:42 +01:00
Pratik Mankawde
904d4da363 merge: bring the SecretKey test-compare fix forward from phase9-metric-gap-fill 2026-09-15 11:24:42 +01:00
Pratik Mankawde
ec28856a20 merge: bring the SecretKey test-compare fix forward from phase7-native-metrics 2026-09-15 11:24:42 +01:00
Pratik Mankawde
59a833571a merge: bring the SecretKey test-compare fix forward from phase8-log-correlation 2026-09-15 11:24:42 +01:00
Pratik Mankawde
8cd99e17e9 fix(test): compare SecretKey by bytes and drop an include CI flags
SecretKey deletes operator==, so EXPECT_EQ on the pair's second member
does not compile; compare the byte ranges instead. Application.cpp no
longer calls logicError(), so clang-tidy's include-cleaner rejects
<xrpl/basics/contract.h> there.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-15 11:24:21 +01:00
Mayukha Vadari
1a4a40ebb8 fix: Update noripple_check to exclude transactions field on error responses (#6303)
Co-authored-by: Timur Yalymov <36795566+tyalymov@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Vito Tumas <5780819+Tapanito@users.noreply.github.com>
Co-authored-by: Ayaz Salikhov <mathbunnyru@users.noreply.github.com>
2026-09-15 00:21:57 +00:00
Pratik Mankawde
545ec26eed merge: bring the noopMeter() clang-tidy fixes forward from phase10-workload-validation 2026-09-14 23:48:42 +01:00
Pratik Mankawde
fe7d973e62 merge: bring the noopMeter() clang-tidy fixes forward from phase9-metric-gap-fill 2026-09-14 23:48:24 +01:00
Pratik Mankawde
26a2bafb23 merge: bring the noopMeter() clang-tidy fixes forward from phase7-native-metrics 2026-09-14 23:48:23 +01:00
Pratik Mankawde
5df8f71c7a merge: bring the noopMeter() clang-tidy fixes forward from phase8-log-correlation 2026-09-14 23:48:23 +01:00
Pratik Mankawde
e5950af95f merge: bring the StatsD test include fix forward from phase6-statsd 2026-09-14 23:48:23 +01:00
Pratik Mankawde
7a4806c00a fix(telemetry): satisfy clang-tidy on the noopMeter() helper
Static constants take the k prefix (readability-identifier-naming), and
SpanGuardScope.cpp no longer names anything from <opentelemetry/metrics/noop.h>
since it calls noopMeter(). Both fail CI under warnings-as-errors.

The helper's docstring also claimed NoopMeterProvider hides the base
two-argument GetMeter; it declares that overload itself, so the only
detail worth sharing is the version.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-14 23:47:55 +01:00
Pratik Mankawde
7b41b00bab test(insight): drop two includes the StatsD collector test never uses
clang-tidy's include-cleaner flags Counter.h and Gauge.h here: the test
names neither type, so the includes fail CI under warnings-as-errors.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-14 23:47:14 +01:00
Pratik Mankawde
1880c9a498 merge: bring phase10-workload-validation forward into sync-diagnostics
Resolutions:
- MetricsRegistry.cpp: keep both <exception> and <limits>; drop
  incrementStateChanges(), which this branch removed on purpose (the
  labelled state_changes_total call site in NetworkOPsImp::setMode
  replaces it, and a compile-time test guards that).
- tests/MetricsRegistry.cpp: constructor-built pipeline wording from
  phase-10, this branch's test list and gauge paragraphs kept; the two
  lifecycle tests now call startAsyncGauges() and pass kTestOptions.
- tests/MetricMacros.cpp: comments name the recording() gate.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-14 23:45:32 +01:00
Pratik Mankawde
379bd5a02b merge: bring the stop() recording gate and identity tests forward from phase9-metric-gap-fill 2026-09-14 23:35:11 +01:00
Pratik Mankawde
6dc961f195 merge: bring the resolveNodeIdentity() decision tests forward from phase8-log-correlation 2026-09-14 23:34:57 +01:00