Commit Graph

14606 Commits

Author SHA1 Message Date
Pratik Mankawde
a7c7e2767e Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd 2026-06-11 23:21:14 +01:00
Pratik Mankawde
8eac1a0e20 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-06-11 23:21:14 +01:00
Pratik Mankawde
f76f0c00ac Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-06-11 23:21:14 +01:00
Pratik Mankawde
a3c629251a docs(telemetry): fix stale txq.accept_tx span name
The TxQSpanNames.h span-hierarchy comment and the Phase 3 task-list attribute
table showed txq.accept.tx, but the constant emits txq.accept_tx
(op::acceptTx = "accept_tx"). Correct both.
2026-06-11 23:20:47 +01:00
Pratik Mankawde
a0b5f0ecd7 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd
# Conflicts:
#	src/xrpld/overlay/detail/PeerImp.cpp
2026-06-11 23:18:14 +01:00
Pratik Mankawde
e1045a84aa Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-06-11 23:16:54 +01:00
Pratik Mankawde
25447e508e Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-06-11 23:16:54 +01:00
Pratik Mankawde
1ad67e8aa5 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing 2026-06-11 23:16:53 +01:00
Pratik Mankawde
84089b4b4d Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing 2026-06-11 23:16:41 +01:00
Pratik Mankawde
e5b46e6cf8 Merge branch 'pratik/otel-phase1b-telemetry-infra' into pratik/otel-phase1c-rpc-integration 2026-06-11 23:16:41 +01:00
Pratik Mankawde
c8e5670aed Merge branch 'pratik/otel-phase1a-plan-docs' into pratik/otel-phase1b-telemetry-infra 2026-06-11 23:16:41 +01:00
Pratik Mankawde
43a70551b9 docs(telemetry): fix stale txq.accept_tx span name in header diagram
The span-hierarchy comment showed txq.accept.tx, but the constant emits
txq.accept_tx (op::acceptTx = "accept_tx"). Correct the diagram.
2026-06-11 23:15:28 +01:00
Pratik Mankawde
ae114c5341 refactor(telemetry): drop dotted ledger hash; share peer validation attrs
The peer.validation.receive span recorded the ledger hash under the dotted
xrpl.ledger.hash form — the only dotted span attribute, and inconsistent with
the bare ledger_hash the consensus validation spans use for the same value.

Make the base SpanNames.h ledgerHash bare (ledger_hash) and add the shared
fullValidation key, so the peer and consensus validation spans share one key
per concept (told apart by span name, not an emitter prefix). PeerSpanNames now
re-exports both from the base; the peer validation_full key folds into the
shared full_validation. The peer trust attrs (proposal_trusted /
validation_trusted) already follow the convention and are unchanged.
2026-06-11 23:11:23 +01:00
Pratik Mankawde
19a6c2a306 refactor(telemetry): consistent consensus span-attribute names
Make consensus span-attribute keys consistent with the peer spans that record
the same concepts, using ONE shared name per concept (told apart by span name),
not an emitter prefix:

- Add the shared ledger_hash and full_validation keys to the base SpanNames.h
  (a ledger-object property and an is-full-validation flag, both shared with the
  peer validation spans); consensus re-exports them via `using` instead of
  defining local copies.
- trusted (bare) -> proposal_trusted on consensus.proposal.receive and
  validation_trusted on consensus.validation.receive — the same message-type
  qualification the peer.{proposal,validation}.receive spans use.

Also collapse a same-emitter duplicate: establishCounter_ was recorded under
both establish_counter and establish_count on two consensus spans; both now use
establish_count (the establishCounter span-name constant is removed; the
establishCounter_ member variable is unchanged).
2026-06-11 23:08:39 +01:00
Pratik Mankawde
172dff77d5 docs(telemetry): split bare trusted in runbook by message type
The consensus.proposal.receive and consensus.validation.receive spans record
the trust flag under proposal_trusted / validation_trusted (shared with the
peer spans of the same message type), not a bare `trusted`. Update the span
reference table to match.
2026-06-11 23:02:50 +01:00
Pratik Mankawde
c3ccde3e39 docs(telemetry): document the span-attribute naming rules
State the rules so they stay consistent across code, collector, Tempo,
dashboards, and docs:

- Per-span-unique field -> bare name (the span name carries the domain).
- Same concept on more than one span -> ONE shared key, reused verbatim and
  distinguished by span name, never tagged with the emitting workflow
  (e.g. ledger_hash, full_validation, proposal_trusted/validation_trusted).
  Defined once in the base SpanNames.h and re-exported by each domain header.
- Collision qualifier <domain>_<field> only to separate DIFFERENT concepts that
  share a word, or the OTel-reserved status key (rpc_status, consensus_state).
- Dotted xrpl.<...> is reserved for resource attributes (xrpl.network.*).

Updates CONTRIBUTING.md (permanent home) and OpenTelemetryPlan §2.3.3.
2026-06-11 23:01:55 +01:00
Pratik Mankawde
282aec4367 ci: Fix OTel naming check blind spot for dotted span attrs
Rule A silently missed a dotted span attribute (xrpl.ledger.hash) because of
two interacting bugs:

1. attr_keys_from_header resolved each constant via a flat global symbol table
   keyed by bare name, so a later header defining a same-named constant (e.g.
   consensus attr::ledgerHash = "ledger_hash") clobbered the base header's
   attr::ledgerHash = "xrpl.ledger.hash", erasing the real dotted key from L1.
   Now each constant is resolved against its own header (the global table only
   seeds seg::/join() cross-file references); using-re-exports still resolve
   globally.

2. derive_dotted_resource_keys allowlisted any dotted key declared in the base
   SpanNames.h. Now it allowlists only the keys actually passed to
   Resource::Create() in Telemetry.cpp (semconv service.* + the attr:: constants
   set there, e.g. xrpl.network.*). A dotted key declared in a header but never
   set as a resource attr is a Rule-A violation.

Adds 4 regression tests (collision, using-re-export, allowlist scope, brace
matching). No allowlist exception is added — the check now catches the
violation so the offending code can be fixed.
2026-06-11 22:41:08 +01:00
Pratik Mankawde
e97878c5d7 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd
Forward-merge the OTel naming-check Rule C/D fixes (TraceQL span. prefix
stripping, L6 native-metric labels, tempo datasource span-filter enforcement).
2026-06-11 19:34:31 +01:00
Pratik Mankawde
bf7fcf58f0 Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing
Forward-merge the OTel naming-check Rule C/D fixes (TraceQL span. prefix
stripping, L6 native-metric labels, tempo datasource span-filter enforcement).
2026-06-11 19:32:00 +01:00
Pratik Mankawde
a8a88410f4 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing
Forward-merge the OTel naming-check Rule C/D fixes (TraceQL span. prefix
stripping, L6 native-metric labels, tempo datasource span-filter enforcement).
2026-06-11 19:32:00 +01:00
Pratik Mankawde
5e7b71d600 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing
Forward-merge the OTel naming-check Rule C/D fixes (TraceQL span. prefix
stripping, L6 native-metric labels, tempo datasource span-filter enforcement).
2026-06-11 19:32:00 +01:00
Pratik Mankawde
68173a8354 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment
Forward-merge the OTel naming-check Rule C/D fixes (TraceQL span. prefix
stripping, L6 native-metric labels, tempo datasource span-filter enforcement).
2026-06-11 19:32:00 +01:00
Pratik Mankawde
2644179a42 ci: Rule C — read Grafana tempo datasource, enforce span-scope filter tags
Rule C was reading docker/telemetry/tempo.yaml (the Tempo server config), which
has no filter tags, so it always SKIPped — L3 was silently unenforced. The
trace-search filter tags actually live in the Grafana datasource provisioning
file (docker/telemetry/grafana/provisioning/datasources/tempo.yaml) as
search.filters[].{tag,scope}. Point Rule C there (server file as fallback),
pair each tag with its scope, validate only span-scope tags against L1 (resource/
intrinsic tags like service.*/name/status/duration are exempt), and strip the
TraceQL span. prefix.

On phase-9 this turns "SKIP: C" into "OK: C: 24 tempo span-filter tags all in
L1" — L3 is now genuinely guarded. Adds a RuleCTempo test class (4 cases:
span-tag-not-in-L1 flagged, span-tags-pass, resource/intrinsic ignored, skip
when datasource absent). 83 tests total.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 19:28:24 +01:00
Pratik Mankawde
6c62bfd2ad ci: Rule D — strip TraceQL scope prefix, recognize native-metric labels (L6)
Phase 9 surfaced two Rule D gaps (false positives, not data errors):
- TraceQL `span.<attr>` / `resource.<attr>` references: the bare attribute is
  in L1, but the scope-prefixed form was flagged. Now strip the
  span./resource./event./link/instrumentation_scope. prefix before the L1
  lookup.
- Native OTel metric labels (e.g. `job_type`, `reason`) emitted by
  MetricsRegistry are valid dashboard labels but are not span attributes. Add
  an L6 source: parse `Add(.., {{"label", ...}})` instrument calls and accept
  those label keys alongside L1 and builtins.

Verified against phase-9's real dashboards: 6 prior false positives -> 0.
79 tests (7 new for span-prefix stripping and metric-label extraction).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 19:16:56 +01:00
Pratik Mankawde
5d116ff2ec Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd
Forward-merge the phase-2/3 BasicConfig.h include-path fix to keep the chain
consistent.
2026-06-11 19:01:37 +01:00
Pratik Mankawde
4dff30b6c6 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing
Forward-merge the phase-2/3 BasicConfig.h include-path fix to keep the chain
consistent. (phases 4+ already had the correct include; no file change here.)
2026-06-11 19:01:14 +01:00
Pratik Mankawde
2709c7ebc5 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment
Forward-merge the phase-2/3 BasicConfig.h include-path fix to keep the chain
consistent. (phases 4+ already had the correct include; no file change here.)
2026-06-11 19:01:14 +01:00
Pratik Mankawde
686c8a8ffb Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing
Forward-merge the BasicConfig.h include-path fix (basics -> config) so the
phase-3 build-test job compiles. phase-3 had the same broken include as phase-2.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 19:00:42 +01:00
Pratik Mankawde
be9cc0df5b fix(telemetry): correct BasicConfig.h include path in phase-2 test
src/tests/libxrpl/telemetry/TelemetryConfig.cpp included
<xrpl/basics/BasicConfig.h>, but the header lives at
<xrpl/config/BasicConfig.h> (the phase-1b layout). This broke the build-test
job on phase-2 (and phase-3) across all platforms with a "file not found"
fatal error. The fix already exists on phase-4+; backport it to the branch
that introduced the test so each PR builds on its own.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 18:59:28 +01:00
Pratik Mankawde
c28e432cd7 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd
Forward-merge the Rule D __name__ builtin fix (and prior naming-check work).
2026-06-11 18:34:48 +01:00
Pratik Mankawde
3272f07058 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing
Forward-merge the Rule D __name__ builtin fix (and prior naming-check work).
2026-06-11 18:34:48 +01:00
Pratik Mankawde
b1ecb718ba Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment
Forward-merge the Rule D __name__ builtin fix (and prior naming-check work).
2026-06-11 18:34:48 +01:00
Pratik Mankawde
d058e5ac3c Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing
Forward-merge the Rule D __name__ builtin fix (and prior naming-check work).
2026-06-11 18:34:48 +01:00
Pratik Mankawde
ae80391da6 Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing
Forward-merge the Rule D __name__ builtin fix (and prior naming-check work).
2026-06-11 18:34:34 +01:00
Pratik Mankawde
6ec60ff52c ci: Add __name__ to OTel naming check Rule D builtins
Rule D (dashboard PromQL labels must exist in L1) flagged `__name__` once the
phase-7 system-*.json dashboards started using `sum by (le, __name__)`.
`__name__` is the Prometheus reserved label for the metric name itself — a
builtin, not a span attribute. Add it to the builtin allowlist and cover it
with a test. (Earlier dashboards only used `__name__` inside `{__name__=~...}`
matchers, which the label regex did not extract, so this surfaced only now.)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 18:34:19 +01:00
Pratik Mankawde
8b2695b015 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd 2026-06-11 18:23:49 +01:00
Pratik Mankawde
65e64205ef Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-06-11 18:23:29 +01:00
Pratik Mankawde
1b6945317e Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-06-11 18:23:20 +01:00
Pratik Mankawde
8add336c1a Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing 2026-06-11 18:23:02 +01:00
Pratik Mankawde
d27d67dfe4 Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing
# Conflicts:
#	.github/scripts/levelization/results/loops.txt
2026-06-11 18:22:54 +01:00
Pratik Mankawde
59030e5d61 fixed a rule in otel naming check file. added tests for it.
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-11 18:21:42 +01:00
Pratik Mankawde
6c4c3e1049 layering
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-11 18:01:18 +01:00
Pratik Mankawde
5705c4dc02 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-11 17:49:55 +01:00
Pratik Mankawde
448aaa4718 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment
Bring phase-4 forward into phase 5 (docs & deployment). Phase 5 owns the
operator runbook (docs/telemetry-runbook.md).

Conflict resolution:
- 05-configuration-reference.md: took phase-4's code-block-free prose for the
  config-parser section; the mTLS options (tls_client_cert/tls_client_key)
  remain documented in the §5.1.2 options table.

Phase-5-owned naming fix:
- docs/telemetry-runbook.md: converted 20 dotted xrpl.<domain>. attribute keys
  in the Span Reference tables to the underscore convention (tx_hash, peer_id,
  ledger_seq, consensus_mode/round/round_id/ledger_id, tx_id). Span NAMES stay
  dotted (rule 5). `trusted` left as-is — verified it matches the code constant
  ConsensusSpanNames.h (the proposal_trusted/validation_trusted split is a
  separate, not-yet-applied code change).

End-to-end reconciliation: every span-attribute token in the runbook (L5) now
matches an L1 *SpanNames.h constant. The xrpl_<domain>_<field> tokens are the
spanmetrics-derived Prometheus labels (documented mapping), not span attrs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 17:43:57 +01:00
Pratik Mankawde
180c905a8a docs(telemetry): remove incorrect "rule 5" justification in Phase 4 task list
Three entries in Phase4_taskList.md annotated dotted attribute keys
(`xrpl.consensus.round`, `round_id`, `ledger_id`) as "(kept — rule 5)". That
is wrong: rule 5 keeps SPAN NAMES dotted, not attribute keys — the code emits
the underscore form (consensus_round, consensus_round_id, consensus_ledger_id)
per ConsensusSpanNames.h. Remove the false justification; the dotted form in
this task list remains as the illustrative-readability form documented in the
note at the top of the file.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 17:20:50 +01:00
Pratik Mankawde
85103e4550 levelization fix
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-11 17:16:36 +01:00
Pratik Mankawde
087fcf54b1 docs(telemetry): fix dotted attribute keys in 06 Spans Produced tables
The two "Spans Produced" tables in 06-implementation-phases.md (added by the
Phase 4 consensus work) listed span ATTRIBUTE keys in the dotted
xrpl.consensus.* / ledger.seq / mode.old form. Convert them to the underscore
convention, matching the authoritative ConsensusSpanNames.h constants
(consensus_round, ledger_seq, consensus_mode, consensus_round_id,
consensus_ledger_id, consensus_result, mode_old/new, agree_count/disagree_count,
etc.). Span NAMES in column 1 stay dotted (rule 5). Every key verified to exist
as a constant in ConsensusSpanNames.h / SpanNames.h.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 17:12:23 +01:00
Pratik Mankawde
294c276e29 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing
Bring phase-3 forward into phase 4 (consensus tracing). Phase 4 introduces
ConsensusSpanNames.h (and already did the trusted→proposal/validation rename
and underscore-attr conversion in ffc197b914).

Conflict resolution (reviewed by a code-review agent before commit):
- 02-design-decisions.md §2.4 Consensus Attributes: merge-both — kept phase-3's
  underscore table form AND folded in phase-4's richer "Phase 4a" attribute set
  (round_id, ledger_id, trace_strategy, converge/establish/disputes counts,
  agree/disagree counts, threshold_percent, consensus_result, mode_old/new),
  each mapped to its authoritative ConsensusSpanNames.h constant. Dropped the
  planned-but-unimplemented proposers_agreed/proposers_total (no code constant;
  agree_count/disagree_count serve that role).
- SpanGuardFactory.cpp: kept phase-4's explanatory comment about why a libxrpl
  test uses literal keys, plus the converted command/rpc_status keys.

Naming check passes (100 keys across 8 *SpanNames.h headers, including consensus).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 17:04:06 +01:00
Pratik Mankawde
7cb08307a7 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing
Bring phase-2 forward into phase 3 (transaction tracing). Phase 3 introduces
TxSpanNames.h, TxQSpanNames.h, and TxApplySpanNames.h.

Conflict resolution:
- TxQ.cpp: kept phase-3's txq_span-based instrumentation (phase-2 had none).
  Dropped the orphaned `NumberSO{... fixUniversalNumber}` line — develop's
  #5962 (Retire fixUniversalNumber) removed that symbol repo-wide; the
  conflict block had carried one stale copy that would not compile.
- 05/08/OpenTelemetryPlan.md: dropped the deleted 04-code-samples / POC_taskList
  references (carried from phase-2), kept phase-3's new secure-OTel.md doc rows,
  section, and Mermaid node/edge/style. Config code block -> prose; merged the
  secure-OTel hardening pointer with the authoritative-config prose.
- Phase3_taskList.md: removed the "dotted keys for readability" note that came
  from phase-2 — phase 3 already uses the underscore keys.

Reviewed by code-review agents: telemetry instrumentation intact, naming check
green (47 keys across 7 *SpanNames.h headers), no conflict markers.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 16:45:34 +01:00
Pratik Mankawde
b1e6d90af1 ordering changes
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-11 16:29:06 +01:00