Pratik Mankawde
f0df68ae2e
docs(telemetry): add protocol span-flow diagrams to runbook
...
Add a "Protocol Span Flow" section to the telemetry operator runbook: 8
Mermaid diagrams that map every OTel span onto the real xrpld control flow
and XRPL protocol order (verified against code and docs/consensus.md), for
use as the canonical key when linking the span hierarchy.
- Master overview, client/peer ingress, shared apply pipeline, consensus
round, accept/build/finalize, and pathfinding/ledger-acquire side flows.
- Every node/branch is labelled with the span that represents that state or
transition (or explicit "(no span)"); drops/abandons are marked terminal.
- Shows real loops, retries, recovery, and drop branches: multi-round
consensus settling (avalanche threshold rounds + MovedOn/Expired retry with
wrong-ledger recovery), 3-pass tx apply retry, TxQ cross-ledger retry,
quorum-gated async validation with abandoned-ledger, ingress backpressure
drops, and cross-node context propagation.
- Adds a divergence table noting where OTel span parenting does not match the
real protocol flow (deterministic/hash trace-id roots, JtAccept/JtUpdatePf
job hand-offs, sequential peer->consensus receive stages).
Replace the stale planning-era diagram in OpenTelemetryPlan/08-appendix.md
(which named spans that were never built) with a pointer to the runbook.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-23 19:42:13 +01:00
Pratik Mankawde
5eed34d1a2
Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation
...
Carries the telemetry.md doc refresh (coro-aware SpanGuard model) forward.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-23 14:26:08 +01:00
Pratik Mankawde
dc9e38b454
Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill
...
Carries the telemetry.md doc refresh (coro-aware SpanGuard model) forward.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-23 14:25:57 +01:00
Pratik Mankawde
29df9c4ed4
Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation
...
Carries the telemetry.md doc refresh (coro-aware SpanGuard model) forward.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-23 14:25:56 +01:00
Pratik Mankawde
6656780e45
Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics
...
Carries the telemetry.md doc refresh (coro-aware SpanGuard model) forward.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-23 14:25:56 +01:00
Pratik Mankawde
ccba9fc1f0
Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd
...
Carries the telemetry.md doc refresh (coro-aware SpanGuard model) forward.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-23 14:25:56 +01:00
Pratik Mankawde
b6ba1a75ff
Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment
...
Carries the telemetry.md doc refresh (coro-aware SpanGuard model) forward.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-23 14:25:55 +01:00
Pratik Mankawde
9ee75bba94
Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing
...
Carries the telemetry.md doc refresh (coro-aware SpanGuard model) forward.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-23 14:25:55 +01:00
Pratik Mankawde
435252d851
Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing
...
Carries the telemetry.md doc refresh (coro-aware SpanGuard model) forward.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-23 14:25:55 +01:00
Pratik Mankawde
5579098068
Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing
...
Carries the telemetry.md doc refresh (coro-aware SpanGuard model) forward.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-23 14:25:54 +01:00
Pratik Mankawde
17866bcc8e
Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration
...
Carries the telemetry.md doc refresh (coro-aware SpanGuard model) forward.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-23 14:25:54 +01:00
Pratik Mankawde
7a19cb6da6
docs(telemetry): update SpanGuard model — coro-aware scopes, ScopedActivation, drop removed rootSpan/detached
...
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-23 14:24:43 +01:00
Pratik Mankawde
11e5454a0f
Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation
...
Carries the review fixes forward to phase-10: no-alloc GetCurrent hot path,
activateIfLive helper deduplicating the four worker-body activation sites,
non-copyable CoroAwareContextStorage, and the clarified drop-counter doc.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-23 14:10:02 +01:00
Pratik Mankawde
18eab81628
refactor(telemetry): use activateIfLive helper at ledger.acquire site
...
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-23 14:09:18 +01:00
Pratik Mankawde
d66e0c7fc2
Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill
...
Carries activation-site dedup (activateIfLive) + no-alloc GetCurrent forward.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-23 14:08:03 +01:00
Pratik Mankawde
0187250108
Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation
...
Carries activation-site dedup (activateIfLive) + no-alloc GetCurrent forward.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-23 14:08:02 +01:00
Pratik Mankawde
b2f5f65b2f
Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics
...
Carries activation-site dedup (activateIfLive) + no-alloc GetCurrent forward.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-23 14:08:01 +01:00
Pratik Mankawde
4975a38089
Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd
...
Carries activation-site dedup (activateIfLive) + no-alloc GetCurrent forward.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-23 14:07:58 +01:00
Pratik Mankawde
f2c0534544
Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment
...
Carries activation-site dedup (activateIfLive) + no-alloc GetCurrent forward.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-23 14:07:57 +01:00
Pratik Mankawde
88c83e6f34
Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing
...
Carries the tx activation-site dedup (activateIfLive) forward.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-23 14:07:54 +01:00
Pratik Mankawde
a2c50245af
refactor(telemetry): use activateIfLive helper at consensus accept site
...
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-23 14:07:11 +01:00
Pratik Mankawde
f547c80f4c
fix(telemetry): mirror second-scale histogram buckets into Alloy config
...
The Alloy spanmetrics connector (perf-iac nodes run Alloy instead of the
collector-contrib image) carried the same old [.. 1s, 5s] bucket list, whose
buckets are documented as "copied from" the collector config. Sync it to the
15-bucket layout (adds 2s, 3s, 4s, 10s, 30s) so Alloy-fed nodes get the same
accurate second-scale consensus and ledger.acquire percentiles as
collector-fed nodes. unit was already "ms". Kept strictly ascending.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-23 14:05:37 +01:00
Pratik Mankawde
9f5a095079
refactor(telemetry): use activateIfLive helper at tx activation sites
...
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-23 14:02:15 +01:00
Pratik Mankawde
34dfc4edf0
Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing
...
Brings review fixes: no-alloc GetCurrent hot path, activateIfLive helper,
non-copyable storage, accurate drop-counter doc.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-23 13:59:05 +01:00
Pratik Mankawde
8fddbe84a3
Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing
...
Brings review fixes: no-alloc GetCurrent hot path, activateIfLive helper,
non-copyable storage, accurate drop-counter doc.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-23 13:59:02 +01:00
Pratik Mankawde
842f994dbb
Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing
...
Brings review fixes: no-alloc GetCurrent hot path, activateIfLive helper,
non-copyable storage, accurate drop-counter doc.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-23 13:59:01 +01:00
Pratik Mankawde
be78519581
Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration
...
Brings review fixes: no-alloc GetCurrent hot path, activateIfLive helper,
non-copyable storage, accurate drop-counter doc.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-23 13:58:58 +01:00
Pratik Mankawde
c209829c44
refactor(telemetry): no-alloc GetCurrent hot path, activateIfLive helper, non-copyable storage, accurate drop-counter doc
...
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-23 13:50:16 +01:00
Pratik Mankawde
48a5aad91d
Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation
...
Brings the coroutine-aware OTel context storage refactor forward to phase-10:
CoroAwareContextStorage + install, ScopedSpanGuard same-store assertion,
non-owning ScopedActivation, scoped rpc.command spans, tx/consensus/ledger
worker-body activation, coro-store-swap tests, and the M1/M2 hardening.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-22 23:16:18 +01:00
Pratik Mankawde
24db0b246c
Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill
...
Brings M1 (dropped-deterministic-id counter) + M2 (childSpan doc) forward.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-22 23:15:04 +01:00
Pratik Mankawde
75317175d9
Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation
...
Brings M1 (dropped-deterministic-id counter) + M2 (childSpan doc) forward.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-22 23:15:03 +01:00
Pratik Mankawde
691a4e8b4f
Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics
...
Brings M1 (dropped-deterministic-id counter) + M2 (childSpan doc) forward.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-22 23:15:02 +01:00
Pratik Mankawde
479300484f
Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd
...
Brings M1 (dropped-deterministic-id counter) + M2 (childSpan doc) forward.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-22 23:15:00 +01:00
Pratik Mankawde
de97f27c3d
Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment
...
Brings M1 (dropped-deterministic-id counter) + M2 (childSpan doc) forward.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-22 23:15:00 +01:00
Pratik Mankawde
c4315b1551
Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing
...
Brings M1 (dropped-deterministic-id counter) + M2 (childSpan doc) forward.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-22 23:14:45 +01:00
Pratik Mankawde
94459bea19
Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing
...
Brings M1 (dropped-deterministic-id counter) + M2 (childSpan doc) forward.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-22 23:14:41 +01:00
Pratik Mankawde
45cf7cccd0
Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing
...
Brings M1 (dropped-deterministic-id counter) + M2 (childSpan doc) forward.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-22 23:14:39 +01:00
Pratik Mankawde
fe0bfe5147
Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration
...
Brings M1 (release-visible dropped-deterministic-id counter) + M2 (childSpan doc) forward.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-22 23:14:22 +01:00
Pratik Mankawde
8c86e01ace
feat(telemetry): release-visible counter for dropped deterministic trace_id; clarify childSpan(name) doc
...
M1: ~PendingTraceId now increments a process-wide atomic counter when a
pinned deterministic trace_id is destroyed unconsumed, so the silent drop
stays observable in release builds where the existing XRPL_ASSERT is a
no-op. Exposed via unconsumedDeterministicIdDrops() for a future metric.
The deterministic trace_id bytes, the GenerateTraceId() consume logic, and
the unconditional reset() are all unchanged.
M2: tighten the Doxygen on SpanGuard::childSpan(std::string_view) to state
it parents to the current ambient context of this store (meaningful only
when a scope/ScopedSpanGuard/ScopedActivation is active) and to point
callers at childSpan(name, ctx) for explicit cross-store parenting.
Doc-only; no behavior change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-22 23:10:49 +01:00
Pratik Mankawde
28a8ab144b
feat(telemetry): correlate ledger.acquire outcome log via span activation
...
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-22 22:58:17 +01:00
Pratik Mankawde
68bf633955
Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill
...
Brings coroutine-aware context storage + tx/consensus worker-body activation.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-22 22:48:16 +01:00
Pratik Mankawde
31f6eee983
Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation
...
Brings coroutine-aware context storage + tx/consensus worker-body activation.
The coro-aware storage keeps Log.cpp's ambient-context trace-id stamping correct
across coroutine yields, so RPC/tx/consensus log lines stay trace-correlated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-22 22:47:45 +01:00
Pratik Mankawde
3d495f9622
Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics
...
Brings coroutine-aware context storage + tx/consensus worker-body activation.
Resolved: Telemetry.cpp keeps both meterProvider_ (phase-7) and contextStorage_
(coro-aware); doc-09 keeps phase-7 structure and applies the pathfind.request →
rpc.command.<name> correction to phase-7's own PathFind section.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-22 22:46:54 +01:00
Pratik Mankawde
e0c4272f5b
docs(otel): coro-aware context storage; RPC/tx/consensus log correlation retained
...
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-22 22:40:13 +01:00
Pratik Mankawde
64f7672ac0
Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd
...
Brings coroutine-aware context storage + tx/consensus worker-body activation forward.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-22 22:29:45 +01:00
Pratik Mankawde
0441d50793
Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment
...
Brings coroutine-aware context storage + tx/consensus worker-body activation
forward. Preserves phase-5's mTLS exporter config in Telemetry.cpp.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-22 22:29:11 +01:00
Pratik Mankawde
737bd7211e
feat(telemetry): correlate consensus doAccept logs via non-owning span activation
...
Activate the accept span as ambient context at the top of doAccept using the
non-owning ScopedActivation, so doAccept's log lines (and any spans created in
the body) carry the accept span's trace_id. doAccept runs to completion on the
JtAccept worker with no coroutine yield, so the activation is thread-local and
safe; the acceptSpan still owns and ends the span. The accept.apply child span
keeps its explicit childSpan(acceptApply, acceptSpanContext_) parentage --
activation only adds log correlation and does not alter parent linkage.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-22 22:21:08 +01:00
Pratik Mankawde
7de0229450
Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing
...
Brings coroutine-aware context storage + tx worker-body activation forward.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-22 22:15:38 +01:00
Pratik Mankawde
93c9ec672c
feat(telemetry): correlate tx apply + receive logs via non-owning span activation
...
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-22 22:05:44 +01:00
Pratik Mankawde
6706f3cc5f
Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing
...
Brings coroutine-aware context storage, scoped rpc.command, coro-store-swap
tests, and the scoped pathfind.request forward.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-22 21:58:57 +01:00