Commit Graph

10777 Commits

Author SHA1 Message Date
Pratik Mankawde
0e02a67e81 Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-07-22 13:46:45 +01:00
Pratik Mankawde
f59e34192a Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-07-22 13:43:24 +01:00
Pratik Mankawde
4858f65be4 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-07-22 13:43:24 +01:00
Pratik Mankawde
091d3b2bb2 Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-07-22 13:43:24 +01:00
Pratik Mankawde
476963f1db Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd 2026-07-22 13:43:24 +01:00
Pratik Mankawde
23a0d70d49 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-07-22 13:43:24 +01:00
Pratik Mankawde
d64b744540 docs(telemetry): InboundLedger acquireSpan_ is thread-free, not detached
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 13:43:12 +01:00
Pratik Mankawde
b0b174b94d docs(telemetry): RCLConsensus roundSpan_ is thread-free, not detached
The roundSpan_ / roundSpanContext_ comments still described the old
'detached, Scope stripped' model. roundSpan_ is now a thread-free SpanGuard
(no Scope); children link via the captured roundSpanContext_.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 13:43:07 +01:00
Pratik Mankawde
5150159ca2 refactor(telemetry): ledger.acquire uses thread-free SpanGuard (no detach)
acquireSpan_ is emplaced on the acquiring thread and reset() on a JtLedgerData
worker. SpanGuard is now thread-free (owns no thread-local Scope), so it can be
created here and destroyed on the worker with no scope to strip — dropping the
.detached() call the old scoped SpanGuard required.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 13:30:50 +01:00
Pratik Mankawde
dd96d8ad34 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-07-22 13:28:44 +01:00
Pratik Mankawde
dad2dc2acb Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-07-22 13:28:24 +01:00
Pratik Mankawde
cef7a8b16c Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics
# Conflicts:
#	OpenTelemetryPlan/09-data-collection-reference.md
2026-07-22 13:28:15 +01:00
Pratik Mankawde
30398ce2fe Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd 2026-07-22 13:13:24 +01:00
Pratik Mankawde
4998384247 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-07-22 13:13:24 +01:00
Pratik Mankawde
b769542283 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-07-22 13:13:24 +01:00
Pratik Mankawde
ba1b693177 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing 2026-07-22 13:13:24 +01:00
Pratik Mankawde
beb5b2dddc fix(telemetry): scope ledger.build so tx.apply nests under it
ledger.build was left as unscoped SpanGuard after the type split, so tx.apply
(created synchronously during applyTxs on the same JtAccept worker) no longer
nested under it. buildLedgerImpl runs synchronously with no yield, so
ScopedSpanGuard is safe and restores the ledger.build -> tx.apply edge.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 13:11:53 +01:00
Pratik Mankawde
b1f1e30450 fix(telemetry): scope pathfind.request/compute so children nest
pathfind.request (PathFind.cpp) and pathfind.compute (PathRequest.cpp doUpdate)
were left as unscoped SpanGuard after the type split, so pathfind.compute and
pathfind.discover no longer nested under them. Both handlers run synchronously
with no coroutine yield, so ScopedSpanGuard is safe and restores the
request -> compute -> discover sub-tree. rpc.command.* stays unscoped (callMethod
wraps doRipplePathFind which holds across a yield), so pathfind.request parents
to rpc.process.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 13:10:21 +01:00
Pratik Mankawde
f52673b6c1 feat(telemetry): correct ledger-close-time derivation on dashboards
The "ledger close time" was mis-derived and partly un-queryable:

- Build vs Close Duration derived close time from the consensus.ledger_close
  span, which only wraps the onClose() prologue (~0.8ms live) — not the close.
  Repoint the close series to consensus.round (full round, live P95 ~4.8s).
- The network close-time value (close_time) lived only as a span attribute,
  un-queryable in Prometheus and unfit as a spanmetrics label (monotonic
  timestamp -> unbounded cardinality). Expose it as last_close_time on the
  existing server_info observable gauge (native gauge, no new instrument).
- Add a "Ledger Close Interval & Age" panel to ledger-operations and
  node-health: interval = 1/rate(ledgers_closed_total) (counter-based,
  scrape-independent); age = time() - (last_close_time + epoch_offset).

A gauge delta is deliberately NOT used for the interval — a timestamp gauge's
delta aliases to the scrape period, not the close cadence (verified live).
Guardrail comments in both collector configs record why close_time must never
become a spanmetrics dimension. Docs (09-reference) and the operator runbook
updated to match.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 12:23:21 +01:00
Pratik Mankawde
f78bc433b0 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-07-22 10:58:55 +01:00
Pratik Mankawde
eec3062c9c Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-07-22 10:58:50 +01:00
Pratik Mankawde
9725f5b48f fix(telemetry): phase-6 peer.proposal/validation.receive use ScopedSpanGuard::freshRoot
These peer-level entry spans (introduced on phase-6) used the removed
SpanGuard::rootSpan(). They are scoped on the peer thread and end there, so
use ScopedSpanGuard::freshRoot() — a scoped fresh trace root that does not
inherit any span left active on the peer thread.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 10:56:47 +01:00
Pratik Mankawde
6c21cc4af5 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd
# Conflicts:
#	src/xrpld/overlay/detail/PeerImp.cpp
2026-07-22 10:53:58 +01:00
Pratik Mankawde
b2d6f06a9f Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-07-21 22:14:36 +01:00
Pratik Mankawde
7fc145ebd2 refactor(telemetry): consensus + peer-receive spans use spanContext()/freshRoot, drop detach
With unscoped SpanGuard + spanContext() (own-span capture), the consensus
round/establish/accept sites just capture and drop the detach dance;
openSpan + peer proposal/validation receive are thread-free handoffs (no
detach); rootSpan->freshRoot.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 22:09:36 +01:00
Pratik Mankawde
cc7585c0a8 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing 2026-07-21 21:57:35 +01:00
Pratik Mankawde
30ee25a28b Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-07-21 21:57:35 +01:00
Pratik Mankawde
64a3c93188 test(telemetry): SpanGuardFactory uses spanContext() (captureContext renamed)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 21:57:22 +01:00
Pratik Mankawde
e2eba09f12 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-07-21 21:55:25 +01:00
Pratik Mankawde
81805b514f Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing
# Conflicts:
#	src/libxrpl/telemetry/SpanGuard.cpp
2026-07-21 21:55:21 +01:00
Pratik Mankawde
3d2542dc6e Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing 2026-07-21 21:54:05 +01:00
Pratik Mankawde
c9fb16e67b Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration
# Conflicts:
#	include/xrpl/telemetry/SpanGuard.h
2026-07-21 21:53:54 +01:00
Pratik Mankawde
0e7ca13cf3 refactor(telemetry): split context capture into spanContext() (own span) + threadLocalContext() (current)
OTel distinguishes a span's own context from the thread's
current/ambient context; SpanGuard is unscoped so spanContext()
(own span, default) is what cross-thread childSpan parenting needs,
and threadLocalContext() (static) snapshots RuntimeContext::GetCurrent()
for propagation. Renames captureContext.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 21:48:40 +01:00
Pratik Mankawde
cba6c69df4 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-07-21 21:07:24 +01:00
Pratik Mankawde
b4db8d0797 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing 2026-07-21 21:07:24 +01:00
Pratik Mankawde
8a5c806346 Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing 2026-07-21 21:07:24 +01:00
Pratik Mankawde
ec5cd97de6 Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration 2026-07-21 21:07:24 +01:00
Pratik Mankawde
97e6586f71 fix(telemetry): captureContext() captures the guard's own span, not ambient
SpanGuard is unscoped (never pushed onto the thread-local context stack), so
RuntimeContext::GetCurrent() returned an unrelated ambient span. Build the
captured context from impl_->span so captureContext() is correct for every
caller regardless of scoping — childSpan(name, ctx) then parents explicitly to
this span across threads. Matches getTraceBytes()'s own-context behavior.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 21:07:20 +01:00
Pratik Mankawde
fbb79d73fb Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing
# Conflicts:
#	src/libxrpl/telemetry/SpanGuard.cpp
2026-07-21 20:43:10 +01:00
Pratik Mankawde
0b771c2f41 Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation
Brings the call-site metric macros feature (MetricMacros.h + meter() accessor +
2 proof-of-concept call sites + docs) forward from phase9.
2026-07-21 20:20:52 +01:00
Pratik Mankawde
bc432a3244 docs(telemetry): correct incrementLedgersClosed comment (no callers, macro path used) 2026-07-21 20:16:15 +01:00
Pratik Mankawde
deecae0f01 refactor(telemetry): tx spans thread-free (no detach); txq spans scoped for nesting
SpanGuard is now thread-free (holds no Scope), so the tx.receive (PeerImp)
and tx.process (NetworkOPs) handoff sites no longer need .detached() before
being stored and ended on a worker thread — just construct the guard. The
stale "Scope leak" comments are replaced accordingly.

Make the six txq.* spans ScopedSpanGuard so their sub-spans nest via the
ambient context: txq.apply_direct/batch_clear under txq.enqueue, and
txq.accept_tx under txq.accept. All six are verified synchronous, ended at
scope, and never moved/handed off, so scoping is safe. applyDirect's span
is pure RAII (no method calls), so it is declared const.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 20:15:31 +01:00
Pratik Mankawde
d63d5bd45e fix(telemetry): hashSpan standalone spans are true roots via DeterministicIdGenerator
The single-arg hashSpan fabricated a synthetic DefaultSpan parent (random
span_id) to carry the deterministic trace_id. That left every standalone
hash span with a phantom parent_span_id pointing at a span that is never
exported, so Tempo reported "root span not yet received".

Inject the deterministic trace_id through the DeterministicIdGenerator
instead: start the span on the SDK root branch (Context{kIsRootSpanKey,true})
with a PendingTraceId pinning hashData[0:16]. The SDK's no-valid-parent
branch calls GenerateTraceId(), which returns the pinned id, yielding a
TRUE root (empty parent_span_id) whose trace_id == hashData[0:16].

The deterministic bytes are unchanged (still raw hashData[0:16]). The
cross-node overload (real remote parent, remote=true) is untouched and must
keep producing a child of the sender's span. Drop the now-unused
<xrpl/basics/random.h> include; add <array> and DeterministicIdGenerator.h.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 20:10:10 +01:00
Pratik Mankawde
14d3d5baa3 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing
# Conflicts:
#	include/xrpl/telemetry/SpanGuard.h
#	src/libxrpl/telemetry/SpanGuard.cpp
2026-07-21 20:01:54 +01:00
Pratik Mankawde
9b991c9297 perflog: add rpc_in_flight_requests UpDownCounter via XRPL_METRIC_UPDOWN_ADD (proof of concept) 2026-07-21 19:38:40 +01:00
Pratik Mankawde
e4d02904ad test(telemetry): rewrite SpanGuard/ScopedSpanGuard + deterministic-root tests; fix RipplePathFind
Rewrite SpanGuardScope.cpp for the unscoped SpanGuard / scoped
ScopedSpanGuard split and the DeterministicIdGenerator: install the
generator in the TestTelemetry mock (4-arg TracerProviderFactory), drop the
obsolete detached()/detachInPlace() tests, and add freshRoot, scoped-ambient,
scope-handoff, ends-once, deterministic-root, and cross-thread death tests
with exact assertions.

Fix the last detached() caller: RipplePathFind now uses SpanGuard::freshRoot
(SpanGuard is thread-free, so it is held across the coroutine yield and ended
on resume with no scope to strip).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 19:38:10 +01:00
Pratik Mankawde
0992593149 consensus: convert ledgers_closed_total to XRPL_METRIC_COUNTER_INC (proof of concept) 2026-07-21 19:31:58 +01:00
Pratik Mankawde
ae950e1dce telemetry: add missing UpDownCounter no-op stubs to MetricMacros telemetry-off branch 2026-07-21 19:26:55 +01:00
Pratik Mankawde
3170589880 telemetry: add XRPL_METRIC_* call-site macros for lazy counter/histogram registration
Add MetricMacros.h exposing 13 call-site metric macros covering all 7 OTel
instrument kinds (Counter INC/ADD, UpDownCounter, Histogram, sync Gauge, and
Observable Counter/UpDownCounter/Gauge), each with a no-op branch when
XRPL_ENABLE_TELEMETRY is undefined. Sync macros lazily create-once via
std::call_once and cache the instrument at the call site; observable macros
register a callback eagerly. Sync Gauge is ABI-v2-gated with an actionable
static_assert under this repo's ABI v1 build.

Add GTest coverage (8 tests) driving the macros against a bare SDK
MeterProvider via a duck-typed FakeApp/FakeMetricsRegistry, asserting exact
create-once and disabled-gating behavior. No CMakeLists change needed: the
telemetry test-module glob already discovers the new source.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 18:55:43 +01:00
Pratik Mankawde
c7b3271edb Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing
# Conflicts:
#	src/xrpld/rpc/detail/ServerHandler.cpp
2026-07-21 18:50:49 +01:00