Commit Graph

91 Commits

Author SHA1 Message Date
Pratik Mankawde
5705c4dc02 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-11 17:49:55 +01:00
Pratik Mankawde
448aaa4718 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment
Bring phase-4 forward into phase 5 (docs & deployment). Phase 5 owns the
operator runbook (docs/telemetry-runbook.md).

Conflict resolution:
- 05-configuration-reference.md: took phase-4's code-block-free prose for the
  config-parser section; the mTLS options (tls_client_cert/tls_client_key)
  remain documented in the §5.1.2 options table.

Phase-5-owned naming fix:
- docs/telemetry-runbook.md: converted 20 dotted xrpl.<domain>. attribute keys
  in the Span Reference tables to the underscore convention (tx_hash, peer_id,
  ledger_seq, consensus_mode/round/round_id/ledger_id, tx_id). Span NAMES stay
  dotted (rule 5). `trusted` left as-is — verified it matches the code constant
  ConsensusSpanNames.h (the proposal_trusted/validation_trusted split is a
  separate, not-yet-applied code change).

End-to-end reconciliation: every span-attribute token in the runbook (L5) now
matches an L1 *SpanNames.h constant. The xrpl_<domain>_<field> tokens are the
spanmetrics-derived Prometheus labels (documented mapping), not span attrs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 17:43:57 +01:00
Pratik Mankawde
294c276e29 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing
Bring phase-3 forward into phase 4 (consensus tracing). Phase 4 introduces
ConsensusSpanNames.h (and already did the trusted→proposal/validation rename
and underscore-attr conversion in ffc197b914).

Conflict resolution (reviewed by a code-review agent before commit):
- 02-design-decisions.md §2.4 Consensus Attributes: merge-both — kept phase-3's
  underscore table form AND folded in phase-4's richer "Phase 4a" attribute set
  (round_id, ledger_id, trace_strategy, converge/establish/disputes counts,
  agree/disagree counts, threshold_percent, consensus_result, mode_old/new),
  each mapped to its authoritative ConsensusSpanNames.h constant. Dropped the
  planned-but-unimplemented proposers_agreed/proposers_total (no code constant;
  agree_count/disagree_count serve that role).
- SpanGuardFactory.cpp: kept phase-4's explanatory comment about why a libxrpl
  test uses literal keys, plus the converted command/rpc_status keys.

Naming check passes (100 keys across 8 *SpanNames.h headers, including consensus).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 17:04:06 +01:00
Pratik Mankawde
7cb08307a7 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing
Bring phase-2 forward into phase 3 (transaction tracing). Phase 3 introduces
TxSpanNames.h, TxQSpanNames.h, and TxApplySpanNames.h.

Conflict resolution:
- TxQ.cpp: kept phase-3's txq_span-based instrumentation (phase-2 had none).
  Dropped the orphaned `NumberSO{... fixUniversalNumber}` line — develop's
  #5962 (Retire fixUniversalNumber) removed that symbol repo-wide; the
  conflict block had carried one stale copy that would not compile.
- 05/08/OpenTelemetryPlan.md: dropped the deleted 04-code-samples / POC_taskList
  references (carried from phase-2), kept phase-3's new secure-OTel.md doc rows,
  section, and Mermaid node/edge/style. Config code block -> prose; merged the
  secure-OTel hardening pointer with the authoritative-config prose.
- Phase3_taskList.md: removed the "dotted keys for readability" note that came
  from phase-2 — phase 3 already uses the underscore keys.

Reviewed by code-review agents: telemetry instrumentation intact, naming check
green (47 keys across 7 *SpanNames.h headers), no conflict markers.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 16:45:34 +01:00
Pratik Mankawde
4d044e6254 ci: Harden OTel naming check — unconditional Rule F, test exemption, Rule H
Three robustness fixes to check_otel_naming.py, all on phase-1c where the
script lives:

- Rule F now runs UNCONDITIONALLY. It is a purely syntactic check on the
  call-sites and does not need the L1 key set, so code that calls
  SpanGuard::span/setAttribute directly without ever defining a *SpanNames.h
  is still caught (previously it was silently skipped when no header existed).
- Exempt test files from Rule F (tests pass arbitrary literal keys to exercise
  the API). The call-site matcher now requires a SpanGuard/`.`/`->` receiver,
  so std::span and bare declarations no longer false-positive.
- Add Rule H (warning, non-fatal): a namespace-qualified constant used at a
  telemetry call-site but not defined in any *SpanNames.h is flagged, catching
  constants defined in-place instead of in the proper header. Bare locals and
  std:: names are not warned to avoid noise.

SpanGuard.h / Telemetry.h @code examples updated to reference constants that
exist on this branch. README documents the new behavior.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 16:10:08 +01:00
Pratik Mankawde
134a24d5bc ci: Add OpenTelemetry span-attribute naming check (phase 1c)
Add check_otel_naming.py and wire it into on-pr.yml so every PR validates
that span-attribute names stay consistent across the code, collector, Tempo,
dashboards, and docs.

- The valid key set is derived dynamically from the *SpanNames.h constants and
  the resource attributes the code registers in Telemetry.cpp — no hardcoded
  allowlist to drift.
- Each rule is presence-gated: it runs only when the file it needs is in the
  tree, so the check is correct whether telemetry changes land in one PR or
  several (the collector/Tempo/dashboard/runbook layers arrive in later phases).
- Rule A flags dotted span-attribute keys; Rule F flags string-literal
  attribute keys and span-name arguments (values may be runtime data).
- stdlib-only, mirroring the levelization check (bare `python`, no pip step).
- Telemetry.h / SpanGuard.h @code examples now use *SpanNames.h constants so
  the strict literal check passes.
- CONTRIBUTING.md documents the check and how to run it locally.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 15:26:38 +01:00
Pratik Mankawde
53ebd7a8c2 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-06-10 17:47:27 +01:00
Pratik Mankawde
4a0994209e Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-06-10 16:26:23 +01:00
Pratik Mankawde
f37589b1f5 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing 2026-06-10 16:05:29 +01:00
Pratik Mankawde
8908036b11 Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration 2026-06-10 14:55:29 +01:00
Pratik Mankawde
e205d0ef8e handle gTlDiscardCurrentSpan change
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-10 14:00:52 +01:00
Pratik Mankawde
c128625857 Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration 2026-06-10 10:43:54 +01:00
Pratik Mankawde
e11bf35691 fix: Update BasicConfig.h include path after upstream reorg
BasicConfig.h moved from xrpl/basics/ to xrpl/config/ on develop
(PR #7095 / reorg). Phase 1b's telemetry headers still referenced the
old path, breaking a fresh compile with "BasicConfig.h: No such file or
directory". Point both telemetry includes at the new location.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 10:42:52 +01:00
Pratik Mankawde
c35874be6d Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd 2026-06-10 10:31:15 +01:00
Pratik Mankawde
c2a39bc21f Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-06-10 10:31:15 +01:00
Pratik Mankawde
0ff2bed63a Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-06-10 10:31:15 +01:00
Pratik Mankawde
331d9d55b1 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing 2026-06-10 10:31:05 +01:00
Pratik Mankawde
385c3cd91c Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration 2026-06-10 10:29:32 +01:00
Pratik Mankawde
7a4baab7cb minor clangtidy fix
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-10 10:29:19 +01:00
Pratik Mankawde
5e6817df61 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd 2026-06-09 19:10:50 +01:00
Pratik Mankawde
b9de9d0abe Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-09 19:10:37 +01:00
Pratik Mankawde
f12c896ac7 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-06-09 19:05:52 +01:00
Pratik Mankawde
8fe3f06999 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-09 19:05:40 +01:00
Pratik Mankawde
142e8c5b36 Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-09 18:58:11 +01:00
Pratik Mankawde
fa71280795 removed head sampling ratio from config
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-09 18:45:52 +01:00
Pratik Mankawde
5e448e32f9 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd 2026-06-09 14:54:47 +01:00
Pratik Mankawde
cf71dc0a01 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-06-09 14:53:27 +01:00
Pratik Mankawde
9512930b0f Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-06-09 14:52:29 +01:00
Pratik Mankawde
be67ad25e7 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing
# Conflicts:
#	OpenTelemetryPlan/05-configuration-reference.md
2026-06-09 14:52:12 +01:00
Pratik Mankawde
a119efc478 Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration 2026-06-09 13:44:53 +01:00
Pratik Mankawde
a946ce7458 feat(telemetry): enable peer tracing by default
Flip the tracePeer Setup default and the trace_peer config parser
default from off to on, and update the example config and build doc to
match. Peer spans record only peer_id (a node-local numeric connection
id) plus trust/ledger metadata — no IP addresses or public keys — so
the privacy concern behind disabling it does not apply. The high-volume
characteristic is retained in the docs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 13:43:32 +01:00
Pratik Mankawde
4a3ba35c1d Added mTls config for Node-Collector verification
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-08 16:26:09 +01:00
Pratik Mankawde
5cedc5d44b Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-06-08 16:08:27 +01:00
Pratik Mankawde
f12fa8d3a9 Use TraceContextValidation
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-08 16:08:11 +01:00
Pratik Mankawde
994e425804 more clang-tid fixes!
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-01 18:07:23 +01:00
Pratik Mankawde
f51b113f4b Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd 2026-06-01 14:46:22 +01:00
Pratik Mankawde
3cd3d5e80e clang-tidy fixes
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-01 14:45:29 +01:00
Pratik Mankawde
8d730b8b9a Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-05-29 18:16:35 +01:00
Pratik Mankawde
2f96c6547c Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-05-29 16:51:31 +01:00
Pratik Mankawde
c187a62353 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-05-29 16:47:15 +01:00
Pratik Mankawde
f66a53cfc9 Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-05-29 14:51:12 +01:00
Pratik Mankawde
68a69d9064 updated as per latest clang-tidy
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-05-29 14:50:24 +01:00
Pratik Mankawde
309d3e6449 fixed interround consensus linking and added some state attrs to spans.
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-05-28 20:15:41 +01:00
Pratik Mankawde
c070177800 removed direct opentelemetry code from consensus code.
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-05-28 18:02:51 +01:00
Pratik Mankawde
d6b101069e refactor(telemetry): remove consensus tracing from phase-3
Phase-3 (PR #6425) is scoped to transaction tracing only; consensus
tracing belongs to phase-4 (PR #6426). The previous commit on this
branch removed the namespace/attribute scaffolding c6c019ed8b leaked
into phase-3, but phase-3 still carried the consensus span construction
and trace-context propagation introduced in earlier commits
(61cb1faf8f, 93bed03d8d). Move that out too so phase-3 creates and
propagates no consensus spans of any kind.

Removed:
  - src/xrpld/telemetry/ConsensusReceiveTracing.h (deleted; phase-4
    owns it).
  - PeerImp.cpp: remove the std::make_shared<SpanGuard>(
    proposalReceiveSpan(...))/validationReceiveSpan(...) constructions
    in onMessage(TMProposeSet)/onMessage(TMValidation), drop the
    sp = std::move(span) lambda captures, and drop the
    #include <xrpld/telemetry/ConsensusReceiveTracing.h>.
  - RCLConsensus.cpp: drop the two telemetry::injectToProtobuf() blocks
    that injected the active trace context into TMProposeSet (in
    Adaptor::propose, after addSuppression) and TMValidation (in
    Adaptor::validate, around the broadcast call). Drop the now-unused
    #include of TraceContextPropagator.h and the
    XRPL_ENABLE_TELEMETRY-gated include of
    opentelemetry/context/runtime_context.h.
  - TraceContextPropagator.h: update file-level @see comment to drop
    the ConsensusReceiveTracing.h reference and to scope the
    "wired into the P2P message flow via PropagationHelpers.h"
    sentence to TMTransaction only.
  - PropagationHelpers.h: replace the
    @see ConsensusReceiveTracing.h cross-reference with
    @see TxTracing.h.

Inert consensus metadata (TraceCategory::Consensus enum value,
seg::consensus constant, isCategoryEnabled/categoryToSpanKind switch
arms, the SpanGuard.h doc-comment example) is intentionally preserved
on phase-3: nothing references it after this commit, but phase-4
needs it and removing it would widen the phase-3 -> phase-4 merge
surface for no benefit.

Verified via git grep: no remaining phase-3 references to
proposalReceiveSpan, validationReceiveSpan, ConsensusReceiveTracing,
consensus_span::, consensus.proposal, or consensus.validation.

Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-05-28 17:01:46 +01:00
Pratik Mankawde
c9521b97fe refactor(telemetry): pull consensus-tracing scope-leak out of phase-3
Commit c6c019ed8b ("addressed code review comments") bundled tx-tracing
review fixes with consensus-tracing scaffolding that belongs on
pratik/otel-phase4-consensus-tracing (PR #6426). This commit lifts the
consensus-only parts back out of phase-3 so PR #6425 stays scoped to
transaction tracing. Phase-4 already carries the same content (via prior
phase-3 -> phase-4 merges) plus its own evolution on top, so nothing is
moved across — only removed here.

Removed:
  - src/xrpld/app/consensus/ConsensusSpanNames.h (deleted; phase-4 owns
    it).
  - PeerImp.cpp: revert onMessage(TMProposeSet)/onMessage(TMValidation)
    consensus-attr setAttribute calls to the hardcoded
    "xrpl.consensus.{trusted,round,ledger.seq}" strings used before
    c6c019ed8b. Drop the now-unused
    #include <xrpld/app/consensus/ConsensusSpanNames.h>.
  - RCLConsensus::Adaptor::validate(): remove the
    TODO(observability/secure-OTel) block on validation trace_context.
  - TraceContextPropagator.h: remove the TODO(observability/secure-OTel)
    block on injectToProtobuf().

Tx-tracing parts of c6c019ed8b are intentionally untouched.

No phase-3 caller of telemetry::consensus_span:: remains; verified via
git grep. No test on phase-3 references the removed header.

Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-05-28 16:51:23 +01:00
Pratik Mankawde
7ac5343119 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-05-28 16:09:41 +01:00
Pratik Mankawde
c6c019ed8b addressed code review comments
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-05-28 15:55:25 +01:00
Pratik Mankawde
4bd1176df5 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-05-28 11:38:05 +01:00
Pratik Mankawde
9498b2865f fix(telemetry): address PR #6424 review comments
- Drop xrpl.node.amendment_blocked / xrpl.node.server_state from telemetry
  surface (constants in SpanNames.h, two filters in tempo.yaml). Operators
  read the same data via server_info / server_state RPC; OTel SDK 1.18.0
  cannot refresh resource attrs at runtime so resource-level emission was
  not viable either.

- Namespace all pathfind span attributes under pathfind_* (underscore form
  per Phase 1c rule 5). Renames in PathFindSpanNames.h and call sites in
  PathRequest.cpp, PathRequestManager.cpp, plus the rule-5 retention
  xrpl.pathfind.ledger_index -> pathfind_ledger_index.

- Wire pathfind_source_account / pathfind_dest_account on pathfind.request
  in doPathFind / doRipplePathFind handlers (only when present + string).

- Collapse per-asset pathfind.discover / pathfind.rank spans into one
  pathfind.discover hoisted around the per-source-asset loop in
  PathRequest::findPaths. Span count goes from 2N to 1 per RPC call;
  per-asset breakdown traded for bounded storage and cardinality. Trade-off
  documented inline.

- Fix pathfind_num_paths semantics: now sums getBestPaths().size() across
  the loop (paths actually returned) instead of the maxPaths input cap.

- PathRequestManager::updateAll: move span creation after the locked
  requests_ snapshot, early-return when no active subscriptions exist
  (avoids empty span on every ledger close), set pathfind_num_requests
  = requests.size().

- Update Phase2_taskList.md and 02-design-decisions.md to match.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 11:27:29 +01:00