Commit Graph

11080 Commits

Author SHA1 Message Date
Pratik Mankawde
4cd657fde8 Merge branch 'pratik/otel-phase10-workload-validation' into pratik/otel-sync-diagnostics
Resolves the nodestore latency-accumulator conflict by keeping this
branch's API surface and applying the incoming nanosecond fix to it.

This branch had renamed storeDurationStats() to recordStoreDuration() and
given it noexcept, an explicit relaxed fetch_add, and call-frequency and
thread-safety notes. The incoming change fixed the accumulators to hold
nanoseconds so sub-microsecond backend calls stop rounding to zero. Both
are kept: the name, signature and documentation from here, the nanosecond
accumulation from there.

The incoming duplicate getFetchDurationUs()/getStoreDurationUs() pair is
dropped in favour of this branch's noexcept versions, which now read the
nanosecond members and convert on read. recordStoreDuration()'s `if (us >
0)` guard is gone: it discarded every sub-microsecond store, which is the
same rounding the incoming change removes one level up.

Also updates a MetricMacros comment that explained the zero write mean in
terms of that removed guard; the injected totals it asserts on are
unchanged.
2026-08-07 16:54:24 +01:00
Pratik Mankawde
c42874467c fix(nodestore): accumulate read and write latency in nanoseconds
The fetch and store duration counters converted each sample to
microseconds before adding it, so any backend call finishing in under a
microsecond contributed zero. A warm nudb read answers in a few hundred
nanoseconds, so on fast hardware every read floored and the totals stayed
at zero no matter how many reads happened -- the same loss of resolution
the microsecond report was introduced to avoid, one decade lower.

Both accumulators now hold nanoseconds, the clock's own resolution, and
convert once in getFetchDurationUs() and getStoreDurationUs(). The public
accessors, the node_reads_duration_us and node_writes_duration_us JSON
fields, and the metrics that read them all keep microseconds, so nothing
downstream changes unit. storeDurationStats() takes the raw duration
instead of a pre-converted integer so no caller can round early, and
updateFetchMetrics() scales its microsecond input to match.

FetchReport::elapsed stays microseconds: it carries one fetch, not a
total, and that is the unit it declares. The reported sum is therefore the
accumulated total minus a sub-microsecond remainder per fetch, so the two
tests that asserted exact equality between them now assert that bound.
Both had assertions that depended on how fast the host reads; the bound
holds on any hardware.
2026-08-07 16:51:05 +01:00
Pratik Mankawde
e8d8a70438 Merge branch 'pratik/otel-phase10-workload-validation' into pratik/otel-sync-diagnostics 2026-08-07 14:36:20 +01:00
Pratik Mankawde
dd9c8cbd86 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-08-07 14:36:20 +01:00
Pratik Mankawde
91fd4d3532 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-08-07 14:36:20 +01:00
Pratik Mankawde
b0ae15ad8c Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-08-07 14:36:20 +01:00
Pratik Mankawde
63173d2703 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd 2026-08-07 14:36:19 +01:00
Pratik Mankawde
8e3b0735a2 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-08-07 14:35:28 +01:00
Pratik Mankawde
2e21758a49 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-08-07 14:35:28 +01:00
Pratik Mankawde
05335fa3e3 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing 2026-08-07 14:35:28 +01:00
Pratik Mankawde
b47d4f94fd Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing 2026-08-07 14:35:28 +01:00
Pratik Mankawde
379f4ff60b Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration 2026-08-07 14:34:47 +01:00
Pratik Mankawde
71d1338313 Merge branch 'pratik/otel-phase1a-plan-docs' into pratik/otel-phase1b-telemetry-infra 2026-08-07 14:34:39 +01:00
Pratik Mankawde
e2f4b3c8e6 fix: repair build against renamed namespaces from develop
develop renamed CamelCase namespaces to snake_case with no compatibility
aliases, so lines this branch added still referenced the old names. The
merge was textually clean because develop never touched those lines, which
left the breakage invisible to git and visible only at compile time.

  PeerFinder:: -> peer_finder::   Overlay.h, OverlayImpl.h, MetricMacros.cpp
  Tuning::     -> tuning::        PeerImp.cpp (the other call sites in this
                                  file already used the lowercase form)

Two further merge artifacts:

  json_value.h  develop added `using value_type = Value const` to
                ValueConstIterator, and this branch had added
                `using value_type = Value` for the same reason, so the merge
                kept both and they redefined the alias with different types.
                Keep develop's const-qualified type -- correct for an
                iterator whose reference is `Value const&` -- and keep only
                this branch's iterator_category addition.

  InboundLedger.h  <string> is no longer used directly by the header, but
                   five translation units that include it use std::string
                   without including <string> themselves. Mark it
                   `IWYU pragma: keep` rather than remove it, matching
                   suite_list.h.
2026-08-06 12:00:06 +01:00
Pratik Mankawde
1388366475 Merge branch 'pratik/otel-phase10-workload-validation' into pratik/otel-sync-diagnostics 2026-08-06 11:56:36 +01:00
Pratik Mankawde
761c4e2872 fix(telemetry): rename OTel resource alias to avoid xrpl::resource clash
develop renamed xrpl::Resource to xrpl::resource. MetricsRegistry.cpp
declared `namespace resource = opentelemetry::sdk::resource` at file
scope, but both use sites are inside namespace xrpl::telemetry, where
inner-scope lookup finds the enclosing xrpl::resource first and never
reaches the alias. That namespace has no ResourceAttributes or Resource,
so the build failed on all four platforms:

  error: no type named 'ResourceAttributes' in namespace 'xrpl::resource'
  error: no member named 'Create' in namespace 'xrpl::resource';
         did you mean 'creat'?

Rename the alias to otel_resource. Moving it inside xrpl::telemetry would
also compile, but only by shadowing xrpl::resource -- a reader at the use
site still could not tell which namespace `resource::` meant, and a later
using-directive would reintroduce the ambiguity. A distinct name removes
the collision by construction and matches the metric_sdk / otlp_http
aliases already in this file.
2026-08-06 11:56:21 +01:00
Gregory Tsipenyuk
8d7524f03b fix: Use consistent endianness serializing MPT STIssue sequence (#7429)
Co-authored-by: Ed Hennis <ed@ripple.com>
Co-authored-by: David Fuelling <sappenin@gmail.com>
2026-08-05 17:54:44 +00:00
Pratik Mankawde
e6b02acd01 Merge branch 'pratik/otel-phase10-workload-validation' into pratik/otel-sync-diagnostics
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-08-05 16:51:22 +01:00
Pratik Mankawde
0a58627869 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-08-05 16:46:32 +01:00
Pratik Mankawde
0a74688026 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-08-05 16:46:23 +01:00
Pratik Mankawde
31a3a7913e Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-08-05 16:46:09 +01:00
Pratik Mankawde
dee6848d06 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd 2026-08-05 16:42:16 +01:00
Pratik Mankawde
7240e3ecbd Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-08-05 16:41:50 +01:00
Pratik Mankawde
a56e63421a Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-08-05 16:38:48 +01:00
Pratik Mankawde
f4dde26eb1 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing 2026-08-05 16:37:41 +01:00
Pratik Mankawde
6b2d9dc19f clang-tidy fix
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-08-05 16:30:10 +01:00
Pratik Mankawde
fa06982028 fixed namespace issues
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-08-05 16:26:20 +01:00
Pratik Mankawde
46ac6df81f Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill
09-data-collection-reference.md: keep phase-9's structure, which relocated
the §2a call-site-metrics content into §5b (3860c93db2). The incoming §2a
block is superseded; its snake_case namespace fix is already applied to
phase-9's copy of the same text.
2026-08-05 16:01:01 +01:00
Pratik Mankawde
e57b85985a Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-08-05 15:59:44 +01:00
Pratik Mankawde
505a00ba61 Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-08-05 15:59:44 +01:00
Pratik Mankawde
e55a6ad78d Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd 2026-08-05 15:59:44 +01:00
Pratik Mankawde
68ba778e9c Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-08-05 15:59:44 +01:00
Pratik Mankawde
e767225964 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-08-05 15:59:44 +01:00
Pratik Mankawde
7186be73e8 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing 2026-08-05 15:59:43 +01:00
Pratik Mankawde
bf2f81e02f fixed clang-tidy issue
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-08-05 15:56:42 +01:00
Pratik Mankawde
5ed32063f5 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill
Resolve conflicts by keeping phase-9's getobject_* metric instrumentation
and applying develop's snake_case namespace rename (#7933) to it:
Resource:: -> resource::, Tuning:: -> tuning::, BuildInfo:: -> build_info::.
2026-08-05 15:54:33 +01:00
Pratik Mankawde
781ec3d4a5 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-08-05 15:49:28 +01:00
Pratik Mankawde
84145a5469 Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-08-05 15:49:20 +01:00
Pratik Mankawde
9b3a16ae11 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-08-05 15:47:59 +01:00
Pratik Mankawde
50f146b25d Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd 2026-08-05 15:47:59 +01:00
Pratik Mankawde
312b87d840 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-08-05 15:47:58 +01:00
Pratik Mankawde
5fb1457518 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing 2026-08-05 15:47:43 +01:00
Pratik Mankawde
7505ac623e Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing 2026-08-05 15:43:03 +01:00
Pratik Mankawde
c432c1f4c5 Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-08-05 15:29:12 +01:00
Pratik Mankawde
ad9b63f11d fixes
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-08-05 15:24:23 +01:00
Pratik Mankawde
5e1f96da4f Merge branch 'pratik/otel-phase1a-plan-docs' into pratik/otel-phase1b-telemetry-infra
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-08-05 14:29:58 +01:00
Pratik Mankawde
7d4c0b00ee clang-tidy fix
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-08-05 14:22:48 +01:00
Ayaz Salikhov
41d6bb5f73 build: Fix build on macOS 15 and Nix environment (#7953) 2026-08-04 23:27:23 +00:00
Vito Tumas
c3ee602002 test: Split Loan_test.cpp into topical suites (#7864)
Co-authored-by: Ayaz Salikhov <mathbunnyru@users.noreply.github.com>
2026-08-04 15:43:59 +00:00
Pratik Mankawde
3860c93db2 refactor(telemetry): route dashboards, runbook and collector work to phase-9
These changes were developed on the phase-10 branch but belong to content this
branch and its upstreams introduced. Carrying them on phase-10 made its PR diff
report churn in files phase-10 does not own, and left each PR claiming a scope
that did not match its contents.

Moved here from phase-10 (identical content, no functional change):

- Dashboards: all 14 existing boards plus the new log-derived-insights board.
- Docs: telemetry-runbook.md (minus the workload/benchmark sections, which
  describe phase-10 tooling) and the new telemetry-glossary.md.
- Grafana Cloud + Alloy export path: collector config, compose override, the
  two .env examples and alloy/config.alloy.
- Local stack: otel-collector-config.yaml gains sub-millisecond and
  second-scale spanmetrics buckets, pins unit=ms, and promotes
  close_time_correct; integration-test.sh and TESTING.md follow.
- Node configs: exported_instance -> service_instance_id in comments; the
  mainnet sample now logs at warning to bound log volume.
- Metrics code: Telemetry.cpp builds the metrics pipeline in the constructor
  via initMetrics() so the global MeterProvider is published before any
  subsystem creates a beast::insight instrument, and the histogram view keeps
  each instrument's own name instead of collapsing them under one series.
  MetricsRegistry gains a last_close_time gauge and skips negative job-queue
  durations. OTelCollector drops an unused accessor.
- Naming CI: xrpl_work_item joins EXTERNAL_INFRA_LABELS and Rule E accepts the
  dotted perf-iac resource-attribute form. This must travel with the
  dashboards and runbook that reference those labels, or the rules fail.
- Doxygen input glob no longer recurses dot-directories.

Sections describing phase-10 tooling stay on phase-10 and keep their
"Future Enhancement" / "Planned, not yet implemented" markers here; phase-10
removes those markers when it lands the tooling.
2026-08-04 16:10:04 +01:00