Commit Graph

82 Commits

Author SHA1 Message Date
Pratik Mankawde
54d92ff973 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-26 14:42:55 +01:00
Pratik Mankawde
9a2d682a42 Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing 2026-06-26 14:33:16 +01:00
Pratik Mankawde
0ff947454c removed line numbers and cleanup
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-12 20:33:42 +01:00
Pratik Mankawde
b9f0a3ae19 Merge branch 'pratik/otel-phase1a-plan-docs' into pratik/otel-phase1b-telemetry-infra
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-12 20:21:52 +01:00
Pratik Mankawde
46dbc92b5f code review comments addressed
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-12 20:18:12 +01:00
Pratik Mankawde
a3c629251a docs(telemetry): fix stale txq.accept_tx span name
The TxQSpanNames.h span-hierarchy comment and the Phase 3 task-list attribute
table showed txq.accept.tx, but the constant emits txq.accept_tx
(op::acceptTx = "accept_tx"). Correct both.
2026-06-11 23:20:47 +01:00
Pratik Mankawde
84089b4b4d Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing 2026-06-11 23:16:41 +01:00
Pratik Mankawde
c8e5670aed Merge branch 'pratik/otel-phase1a-plan-docs' into pratik/otel-phase1b-telemetry-infra 2026-06-11 23:16:41 +01:00
Pratik Mankawde
c3ccde3e39 docs(telemetry): document the span-attribute naming rules
State the rules so they stay consistent across code, collector, Tempo,
dashboards, and docs:

- Per-span-unique field -> bare name (the span name carries the domain).
- Same concept on more than one span -> ONE shared key, reused verbatim and
  distinguished by span name, never tagged with the emitting workflow
  (e.g. ledger_hash, full_validation, proposal_trusted/validation_trusted).
  Defined once in the base SpanNames.h and re-exported by each domain header.
- Collision qualifier <domain>_<field> only to separate DIFFERENT concepts that
  share a word, or the OTel-reserved status key (rpc_status, consensus_state).
- Dotted xrpl.<...> is reserved for resource attributes (xrpl.network.*).

Updates CONTRIBUTING.md (permanent home) and OpenTelemetryPlan §2.3.3.
2026-06-11 23:01:55 +01:00
Pratik Mankawde
7cb08307a7 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing
Bring phase-2 forward into phase 3 (transaction tracing). Phase 3 introduces
TxSpanNames.h, TxQSpanNames.h, and TxApplySpanNames.h.

Conflict resolution:
- TxQ.cpp: kept phase-3's txq_span-based instrumentation (phase-2 had none).
  Dropped the orphaned `NumberSO{... fixUniversalNumber}` line — develop's
  #5962 (Retire fixUniversalNumber) removed that symbol repo-wide; the
  conflict block had carried one stale copy that would not compile.
- 05/08/OpenTelemetryPlan.md: dropped the deleted 04-code-samples / POC_taskList
  references (carried from phase-2), kept phase-3's new secure-OTel.md doc rows,
  section, and Mermaid node/edge/style. Config code block -> prose; merged the
  secure-OTel hardening pointer with the authoritative-config prose.
- Phase3_taskList.md: removed the "dotted keys for readability" note that came
  from phase-2 — phase 3 already uses the underscore keys.

Reviewed by code-review agents: telemetry instrumentation intact, naming check
green (47 keys across 7 *SpanNames.h headers), no conflict markers.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 16:45:34 +01:00
Pratik Mankawde
9e6c5b5778 docs(telemetry): note dotted attr keys in phase 3-5 task lists are illustrative
The Phase3/4/5 task lists show attribute keys in the older dotted
`xrpl.<domain>.<field>` form because that mirrors how a fully qualified
attribute reads in a Tempo trace view, which keeps the planning prose readable.
Add a note to each clarifying that the implemented keys follow the underscore
convention in CONTRIBUTING.md and that the *SpanNames.h constants (enforced by
the CI naming check) are the single source of truth — so the dotted form here
is illustrative, not the literal key.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 16:21:28 +01:00
Pratik Mankawde
afe0818c33 Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing
Bring the naming convention, code-sample cleanup, and CI naming check into
phase 2 (RPC tracing). Phase 2 introduces PathFindSpanNames.h.

Conflict resolution:
- 04-code-samples.md, POC_taskList.md: deletion wins.
- 02-design-decisions.md: took the convention-applied tables, but kept phase-2's
  accurate PathFinding summary row (pathfind_fast/search_level/num_paths/...,
  matching the implemented PathFindSpanNames.h).
- 05/08: took the code-block-free prose; kept phase-2's Phase2-5_taskList.md
  index rows (dropping only the deleted POC row). Fixed stale setup_Telemetry/
  make_Telemetry doc references to the code-correct setupTelemetry/makeTelemetry.
- Telemetry.h auto-merged to the constant-based @code examples.

check_otel_naming.py change: exempt test files from Rule F (tests pass
arbitrary literal keys to exercise the API). The check passes clean on the
merged tree (24 keys across 4 *SpanNames.h headers, including PathFind).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 15:47:30 +01:00
Pratik Mankawde
d5efd657ae Merge branch 'pratik/otel-phase1a-plan-docs' into pratik/otel-phase1b-telemetry-infra
Bring the phase-1a code-sample cleanup into phase-1b.

Phase 1a deleted the two code-sample docs (04-code-samples.md,
POC_taskList.md) and stripped C++/config code blocks from the remaining
plan docs, replacing them with prose summaries and pointers to the real
source/config files.

Conflict resolution:
- 04-code-samples.md, POC_taskList.md: deletion wins (phase-1b's SpanGuard
  rewrites of these files are intentionally dropped).
- 03/05/OpenTelemetryPlan.md: took phase-1a's code-block-free prose, but
  kept phase-1b's accurate descriptions of the real telemetry code
  (DiscardFlag.h, FilteringSpanProcessor, SpanGuard factory methods, and
  the corrected file-count/file-list tables).
- presentation.md left untouched.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 14:40:31 +01:00
Pratik Mankawde
d6450631bf removed code blocks from plan docs
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-11 14:29:01 +01:00
Pratik Mankawde
e9cb9421ef Merge branch 'pratik/otel-phase1a-plan-docs' into pratik/otel-phase1b-telemetry-infra
Bring the span attribute naming convention (phase 1a) into phase 1b.

Conflict resolution kept phase-1b's SpanGuard-based workflow and applied
the underscore naming convention to all non-code-sample text:
- Converted prose, tables, Mermaid labels, and TraceQL/PromQL query
  references across the plan docs to the underscore form.
- Converted the two @code attribute-key examples in Telemetry.h
  (command, tx_type).
- Left the code-sample files (04-code-samples.md, POC_taskList.md) and
  03-implementation-strategy.md code blocks at the phase-1b version; the
  code-sample docs are slated for removal on phase-1a.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 14:10:52 +01:00
Pratik Mankawde
c0272f1314 docs: Add span attribute naming convention (OTel phase 1a)
Establish the single, authoritative naming convention for OpenTelemetry
span attribute keys so the code, collector, Tempo, dashboards, and docs
stay in sync.

- CONTRIBUTING.md: new "Telemetry span attribute naming" section under
  the Style guide as the permanent, canonical home for the rules.
- OpenTelemetryPlan/02-design-decisions.md: new section 2.3.3 stating the
  decided convention as design, and section 2.4 attribute schema realigned
  to the underscore form (exact key spelling defers to the *SpanNames.h
  constants).
- Sweep the remaining plan docs: convert dotted xrpl.<domain>.<field> span
  attribute keys to the underscore form; leave span names and the
  OTel-standard service.*/http.* and xrpl.network.* resource keys dotted.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 13:29:53 +01:00
Pratik Mankawde
8fe3f06999 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-09 19:05:40 +01:00
Pratik Mankawde
d3c09fd3f4 Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-09 19:04:17 +01:00
Pratik Mankawde
142e8c5b36 Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-09 18:58:11 +01:00
Pratik Mankawde
bb8f7f0e9a Merge branch 'pratik/otel-phase1a-plan-docs' into pratik/otel-phase1b-telemetry-infra 2026-06-09 18:25:26 +01:00
Pratik Mankawde
5598b0eac7 docs(telemetry): fix head sampling at 1.0, remove configurable ratio
Document that head sampling is intentionally fixed at 100% and no longer
exposes a sampling_ratio config knob. A per-node ratio let nodes make
divergent keep/drop decisions for the same distributed trace, producing
broken/partial traces; pinning at 1.0 with a ParentBased sampler keeps
decisions coherent across the network. Volume reduction is delegated to
collector-side tail sampling.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 18:22:52 +01:00
Pratik Mankawde
be67ad25e7 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing
# Conflicts:
#	OpenTelemetryPlan/05-configuration-reference.md
2026-06-09 14:52:12 +01:00
Pratik Mankawde
3ee8f900ec Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing
# Conflicts:
#	src/tests/libxrpl/CMakeLists.txt
2026-06-09 14:50:20 +01:00
Pratik Mankawde
a119efc478 Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration 2026-06-09 13:44:53 +01:00
Pratik Mankawde
57a54ad0fe Merge branch 'pratik/otel-phase1a-plan-docs' into pratik/otel-phase1b-telemetry-infra 2026-06-09 13:34:58 +01:00
Pratik Mankawde
fe13359024 docs(telemetry): enable peer tracing by default in plan docs
Flip trace_peer default false->true across the Phase-1a plan docs and
correct the rationale: peer spans record only peer_id (numeric local
connection id) plus trust/ledger metadata, never IP addresses or public
keys, so the 'includes addresses' caveat was inaccurate. The high-volume
note is retained.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 13:34:20 +01:00
Pratik Mankawde
3df7e9cba6 code review changes and wire unused attributes
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-05 11:42:33 +01:00
Pratik Mankawde
8dd5ac55e8 docs(telemetry): document Task 3.11 TX/TxQ span attribute gap fill
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-03 16:15:33 +01:00
Pratik Mankawde
507828edde Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing 2026-06-03 16:14:57 +01:00
Pratik Mankawde
aca6623f14 docs(telemetry): document Task 2.10 RPC/PathFind span attribute gap fill
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-03 16:14:49 +01:00
Pratik Mankawde
b46ee12a19 formatting fixes
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-03 14:07:21 +01:00
Pratik Mankawde
046e2e2b85 minor doc update
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-01 11:48:47 +01:00
Pratik Mankawde
9f81e770eb Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-01 11:36:19 +01:00
Pratik Mankawde
e321f294e5 clang issues
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-05-29 19:22:07 +01:00
Pratik Mankawde
c187a62353 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-05-29 16:47:15 +01:00
Pratik Mankawde
c848e51e13 Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-05-29 16:44:07 +01:00
Pratik Mankawde
8f9057729c Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-05-29 16:14:21 +01:00
Pratik Mankawde
f031befc6e compilation fixes and levelization fixes
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-05-29 16:04:19 +01:00
Pratik Mankawde
53e8c4d54e fix(docs): apply rename scripts to secure-OTel doc references
Two stray "rippled" tokens introduced by 43258e8d ("docs(telemetry):
add secure-OTel pipeline analysis…") were caught by check-rename in
CI. Re-run docs.sh to convert them to xrpld so the rename check
passes on PR #6425 (and downstream PR #6426 once merged up).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 18:27:58 +01:00
Pratik Mankawde
954223958f renames
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-05-28 16:07:34 +01:00
Pratik Mankawde
c6c019ed8b addressed code review comments
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-05-28 15:55:25 +01:00
Pratik Mankawde
43258e8dc0 docs(telemetry): add secure-OTel pipeline analysis and link into plan
Document the threat model and chosen hardening approach for the OTel
pipeline: mTLS to the collector as primary defense (across-network
deployment), NetworkPolicy as defense-in-depth, and source-side
validation plus per-peer rate limiting for protocol::TraceContext on
peer messages. Skips Basic Auth (wrong shape for multi-operator
fleet) and HTTP-gateway header stripping (rippled is P2P).

Wires the new doc into the master plan ToC, mermaid diagram, and
body section, plus cross-refs from the privacy section in
02-design-decisions.md and the collector config in
05-configuration-reference.md so readers reach it from natural
in-context entry points. Adds a backlink at the top of secure-OTel.md
to the master plan.

Adds 'exfiltration' and 'htpasswd' to cspell dictionary.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 12:33:16 +01:00
Pratik Mankawde
4bd1176df5 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-05-28 11:38:05 +01:00
Pratik Mankawde
9498b2865f fix(telemetry): address PR #6424 review comments
- Drop xrpl.node.amendment_blocked / xrpl.node.server_state from telemetry
  surface (constants in SpanNames.h, two filters in tempo.yaml). Operators
  read the same data via server_info / server_state RPC; OTel SDK 1.18.0
  cannot refresh resource attrs at runtime so resource-level emission was
  not viable either.

- Namespace all pathfind span attributes under pathfind_* (underscore form
  per Phase 1c rule 5). Renames in PathFindSpanNames.h and call sites in
  PathRequest.cpp, PathRequestManager.cpp, plus the rule-5 retention
  xrpl.pathfind.ledger_index -> pathfind_ledger_index.

- Wire pathfind_source_account / pathfind_dest_account on pathfind.request
  in doPathFind / doRipplePathFind handlers (only when present + string).

- Collapse per-asset pathfind.discover / pathfind.rank spans into one
  pathfind.discover hoisted around the per-source-asset loop in
  PathRequest::findPaths. Span count goes from 2N to 1 per RPC call;
  per-asset breakdown traded for bounded storage and cardinality. Trade-off
  documented inline.

- Fix pathfind_num_paths semantics: now sums getBestPaths().size() across
  the loop (paths actually returned) instead of the maxPaths input cap.

- PathRequestManager::updateAll: move span creation after the locked
  requests_ snapshot, early-return when no active subscriptions exist
  (avoids empty span on every ledger close), set pathfind_num_requests
  = requests.size().

- Update Phase2_taskList.md and 02-design-decisions.md to match.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 11:27:29 +01:00
Pratik Mankawde
45e1c15d24 merge: pratik/otel-phase2-rpc-tracing (phase-1a docs fixes) into pratik/otel-phase3-tx-tracing
# Conflicts:
#	OpenTelemetryPlan/05-configuration-reference.md
2026-05-14 16:13:35 +01:00
Pratik Mankawde
865ab65a07 merge: pratik/otel-phase1c-rpc-integration (phase-1a docs fixes) into pratik/otel-phase2-rpc-tracing 2026-05-14 16:11:04 +01:00
Pratik Mankawde
009c63e7db merge: pratik/otel-phase1b-telemetry-infra (phase-1a docs fixes) into pratik/otel-phase1c-rpc-integration 2026-05-14 16:11:01 +01:00
Pratik Mankawde
5d70a5fffd merge: pratik/otel-phase1a-plan-docs (phase-1a docs fixes) into pratik/otel-phase1b-telemetry-infra 2026-05-14 16:10:59 +01:00
Pratik Mankawde
f3a095ab65 docs(telemetry): align Phase 1a plan docs with Phase 1b implementation
Phase-1a plan documents advertised OTLP/gRPC on port 4317 as the default
exporter, four unparsed [telemetry] config keys, and "Phase 4a Complete"
status with exit-criteria checkboxes marked done. Every downstream branch
through Phase 5 ships only OTLP/HTTP on port 4318 via OtlpHttpExporterFactory,
never parses the advertised keys, and the Phase 4 work is not yet delivered.

Fixes:
- 02-design-decisions.md: flip §2.1.1 SDK dependency recommendations to
  OTLP/HTTP (shipped) with OTLP/gRPC marked Future. Update §2.2 architecture
  diagram and text from OTLP/gRPC:4317 to OTLP/HTTP:4318. Rewrite §2.2.1 as
  "OTLP/HTTP (Shipped)" and §2.2.2 as "OTLP/gRPC (Future Work — Planned
  Upgrade)" with a concrete checklist (Conan dep, config parsing, factory
  branch, runbook/dashboard updates) for landing the gRPC transport later.
- 05-configuration-reference.md: drop the fabricated exporter/otlp_grpc key
  and the :4317 default from the sample config block and the options-summary
  table. Move trace_pathfind, trace_txq, trace_validator, trace_amendment
  into a new "Planned (not yet implemented)" table citing the phase that will
  add each one. Keep the example config minimal so copy-paste does not produce
  a silently-ignored stanza.
- 06-implementation-phases.md: reset Phase 4 Exit Criteria checkboxes from
  [x] to [ ] (Phase 4 is not shipped at Phase-1a time). Rename "Phase 4a
  Complete" to "Phase 4a Plan" and describe the work as future. Replace the
  broken forward link to Phase4_taskList.md (introduced in the Phase 2 PR)
  with a sentence pointing readers to where that spec will land. Renumber
  the final section 6.12 to 6.11 so it sits directly after 6.10; section 6.11
  ("Effort Summary") was intentionally removed in earlier edits.
2026-05-14 16:09:48 +01:00
Pratik Mankawde
5c14b57462 docs(telemetry): update Phase3 task list for simplified tx/txq attr naming
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-13 16:31:22 +01:00