Commit Graph

376 Commits

Author SHA1 Message Date
Pratik Mankawde
0ff947454c removed line numbers and cleanup
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-12 20:33:42 +01:00
Pratik Mankawde
b9f0a3ae19 Merge branch 'pratik/otel-phase1a-plan-docs' into pratik/otel-phase1b-telemetry-infra
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-12 20:21:52 +01:00
Pratik Mankawde
46dbc92b5f code review comments addressed
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-12 20:18:12 +01:00
Pratik Mankawde
8eac1a0e20 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-06-11 23:21:14 +01:00
Pratik Mankawde
f76f0c00ac Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-06-11 23:21:14 +01:00
Pratik Mankawde
03b92bfca7 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-06-11 23:21:14 +01:00
Pratik Mankawde
c30123796e Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-06-11 23:21:14 +01:00
Pratik Mankawde
88d2ff61cf Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-06-11 23:21:14 +01:00
Pratik Mankawde
a7c7e2767e Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd 2026-06-11 23:21:14 +01:00
Pratik Mankawde
5a6a00c458 Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-06-11 23:21:14 +01:00
Pratik Mankawde
a3c629251a docs(telemetry): fix stale txq.accept_tx span name
The TxQSpanNames.h span-hierarchy comment and the Phase 3 task-list attribute
table showed txq.accept.tx, but the constant emits txq.accept_tx
(op::acceptTx = "accept_tx"). Correct both.
2026-06-11 23:20:47 +01:00
Pratik Mankawde
f29f32816b docs(telemetry): fix stale txq.accept_tx span name in 09 doc 2026-06-11 23:20:09 +01:00
Pratik Mankawde
c8c540e388 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-06-11 23:18:37 +01:00
Pratik Mankawde
22bb4a74a9 Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-06-11 23:18:37 +01:00
Pratik Mankawde
663ef989eb Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-06-11 23:18:36 +01:00
Pratik Mankawde
6baad644b7 Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-06-11 23:18:24 +01:00
Pratik Mankawde
a0b5f0ecd7 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd
# Conflicts:
#	src/xrpld/overlay/detail/PeerImp.cpp
2026-06-11 23:18:14 +01:00
Pratik Mankawde
e1045a84aa Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment 2026-06-11 23:16:54 +01:00
Pratik Mankawde
25447e508e Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing 2026-06-11 23:16:54 +01:00
Pratik Mankawde
1ad67e8aa5 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing 2026-06-11 23:16:53 +01:00
Pratik Mankawde
84089b4b4d Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing 2026-06-11 23:16:41 +01:00
Pratik Mankawde
c8e5670aed Merge branch 'pratik/otel-phase1a-plan-docs' into pratik/otel-phase1b-telemetry-infra 2026-06-11 23:16:41 +01:00
Pratik Mankawde
725b4fb5cb docs(telemetry): drop the dotted ledger-hash exception in 09 doc
The peer.validation.receive span now records the ledger hash and full-validation
flag under the shared bare keys ledger_hash / full_validation (same keys the
consensus validation spans use), so there is no longer a dotted span attribute.
Remove the "only dotted span attribute" note and update the peer attribute table.
2026-06-11 23:12:42 +01:00
Pratik Mankawde
c3ccde3e39 docs(telemetry): document the span-attribute naming rules
State the rules so they stay consistent across code, collector, Tempo,
dashboards, and docs:

- Per-span-unique field -> bare name (the span name carries the domain).
- Same concept on more than one span -> ONE shared key, reused verbatim and
  distinguished by span name, never tagged with the emitting workflow
  (e.g. ledger_hash, full_validation, proposal_trusted/validation_trusted).
  Defined once in the base SpanNames.h and re-exported by each domain header.
- Collision qualifier <domain>_<field> only to separate DIFFERENT concepts that
  share a word, or the OTel-reserved status key (rpc_status, consensus_state).
- Dotted xrpl.<...> is reserved for resource attributes (xrpl.network.*).

Updates CONTRIBUTING.md (permanent home) and OpenTelemetryPlan §2.3.3.
2026-06-11 23:01:55 +01:00
Pratik Mankawde
8172c0c811 Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation
Forward-merge the OTel naming-check Rule C/D fixes and the phase 2-9 chain.

Conflict resolutions:
- 02-design-decisions.md TxQ Attributes: take the convention-correct markdown
  table (txq_* keys) over the stale planned-attrs code block.
- SpanGuardFactory.cpp: keep the explanatory comment for the literal attribute
  keys in the libxrpl test.
2026-06-11 19:38:09 +01:00
Pratik Mankawde
25227141b3 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill
Bring phase-8 (log correlation) and the naming-check + BasicConfig fixes forward
into phase 9 (native-metrics gap fill). Phase 9 adds MetricsRegistry and
LedgerSpanNames.h.

Conflict resolution:
- presentation.md: took phase-9's restructured deck — sampling now lives in
  Slide 8 (Head vs Tail) and the Phase 10/11/12 future-phase slides; phase-8's
  older standalone sampling section is superseded (no content lost).
- ordering.txt/loops.txt: regenerated via generate.py.

Naming check: A/B/E/F/G green. Rule D reports 6 findings that are all check
gaps, not data errors (verified every dashboard label traces to code):
  - 3 are TraceQL `span.<attr>` references (bare attr is in L1); Rule D should
    strip the span./resource. scope prefix.
  - 2 (`reason`, `job_type`) are native OTel metric labels emitted by
    MetricsRegistry.cpp — a new source of truth the check does not yet model.
The Rule D enhancement lands on phase-1c and merges forward.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 19:11:54 +01:00
Pratik Mankawde
3236daf08e Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation
Bring native-metrics (phase 7) and the naming-check work forward into phase 8
(log correlation).

Conflict resolution — tempo.yaml datasource provisioning (search.filters):
- Took phase-7's complete 30-filter list as the base (it is the comprehensive
  set), preserved phase-8's tracesToLogs Loki-correlation block, and added
  phase-8's two unique filters (tx-type, ledger-hash). Verified the result is
  the full union (32 filters) with no filter dropped and no duplicate ids.
- Fixed a pre-existing dotted tag carried in from phase-7: the
  consensus-ledger-id filter used `xrpl.consensus.ledger_id`; the code emits
  the underscore key `consensus_ledger_id` (ConsensusSpanNames.h), so the
  dotted form was a dead filter. Now corrected.

Every span-scope filter tag verified to exist in the L1 *SpanNames.h key set.
Naming check green; 72 tests pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 18:48:20 +01:00
Pratik Mankawde
2efc932165 Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics
Bring the hardened OTel naming check (Rule E fix + 71 tests) and the phase 1-6
convention work forward into phase 7 (native metrics).

Conflict resolution:
- 05-configuration-reference.md: kept phase-7's native-OTLP metrics story
  (server=otel, /v1/metrics) over phase-6's superseded StatsD scrape job, in
  prose form (no code block).
- ordering.txt/loops.txt: regenerated via generate.py (not hand-edited).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 18:32:21 +01:00
Pratik Mankawde
5705c4dc02 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-11 17:49:55 +01:00
Pratik Mankawde
448aaa4718 Merge branch 'pratik/otel-phase4-consensus-tracing' into pratik/otel-phase5-docs-deployment
Bring phase-4 forward into phase 5 (docs & deployment). Phase 5 owns the
operator runbook (docs/telemetry-runbook.md).

Conflict resolution:
- 05-configuration-reference.md: took phase-4's code-block-free prose for the
  config-parser section; the mTLS options (tls_client_cert/tls_client_key)
  remain documented in the §5.1.2 options table.

Phase-5-owned naming fix:
- docs/telemetry-runbook.md: converted 20 dotted xrpl.<domain>. attribute keys
  in the Span Reference tables to the underscore convention (tx_hash, peer_id,
  ledger_seq, consensus_mode/round/round_id/ledger_id, tx_id). Span NAMES stay
  dotted (rule 5). `trusted` left as-is — verified it matches the code constant
  ConsensusSpanNames.h (the proposal_trusted/validation_trusted split is a
  separate, not-yet-applied code change).

End-to-end reconciliation: every span-attribute token in the runbook (L5) now
matches an L1 *SpanNames.h constant. The xrpl_<domain>_<field> tokens are the
spanmetrics-derived Prometheus labels (documented mapping), not span attrs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 17:43:57 +01:00
Pratik Mankawde
180c905a8a docs(telemetry): remove incorrect "rule 5" justification in Phase 4 task list
Three entries in Phase4_taskList.md annotated dotted attribute keys
(`xrpl.consensus.round`, `round_id`, `ledger_id`) as "(kept — rule 5)". That
is wrong: rule 5 keeps SPAN NAMES dotted, not attribute keys — the code emits
the underscore form (consensus_round, consensus_round_id, consensus_ledger_id)
per ConsensusSpanNames.h. Remove the false justification; the dotted form in
this task list remains as the illustrative-readability form documented in the
note at the top of the file.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 17:20:50 +01:00
Pratik Mankawde
087fcf54b1 docs(telemetry): fix dotted attribute keys in 06 Spans Produced tables
The two "Spans Produced" tables in 06-implementation-phases.md (added by the
Phase 4 consensus work) listed span ATTRIBUTE keys in the dotted
xrpl.consensus.* / ledger.seq / mode.old form. Convert them to the underscore
convention, matching the authoritative ConsensusSpanNames.h constants
(consensus_round, ledger_seq, consensus_mode, consensus_round_id,
consensus_ledger_id, consensus_result, mode_old/new, agree_count/disagree_count,
etc.). Span NAMES in column 1 stay dotted (rule 5). Every key verified to exist
as a constant in ConsensusSpanNames.h / SpanNames.h.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 17:12:23 +01:00
Pratik Mankawde
294c276e29 Merge branch 'pratik/otel-phase3-tx-tracing' into pratik/otel-phase4-consensus-tracing
Bring phase-3 forward into phase 4 (consensus tracing). Phase 4 introduces
ConsensusSpanNames.h (and already did the trusted→proposal/validation rename
and underscore-attr conversion in ffc197b914).

Conflict resolution (reviewed by a code-review agent before commit):
- 02-design-decisions.md §2.4 Consensus Attributes: merge-both — kept phase-3's
  underscore table form AND folded in phase-4's richer "Phase 4a" attribute set
  (round_id, ledger_id, trace_strategy, converge/establish/disputes counts,
  agree/disagree counts, threshold_percent, consensus_result, mode_old/new),
  each mapped to its authoritative ConsensusSpanNames.h constant. Dropped the
  planned-but-unimplemented proposers_agreed/proposers_total (no code constant;
  agree_count/disagree_count serve that role).
- SpanGuardFactory.cpp: kept phase-4's explanatory comment about why a libxrpl
  test uses literal keys, plus the converted command/rpc_status keys.

Naming check passes (100 keys across 8 *SpanNames.h headers, including consensus).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 17:04:06 +01:00
Pratik Mankawde
7cb08307a7 Merge branch 'pratik/otel-phase2-rpc-tracing' into pratik/otel-phase3-tx-tracing
Bring phase-2 forward into phase 3 (transaction tracing). Phase 3 introduces
TxSpanNames.h, TxQSpanNames.h, and TxApplySpanNames.h.

Conflict resolution:
- TxQ.cpp: kept phase-3's txq_span-based instrumentation (phase-2 had none).
  Dropped the orphaned `NumberSO{... fixUniversalNumber}` line — develop's
  #5962 (Retire fixUniversalNumber) removed that symbol repo-wide; the
  conflict block had carried one stale copy that would not compile.
- 05/08/OpenTelemetryPlan.md: dropped the deleted 04-code-samples / POC_taskList
  references (carried from phase-2), kept phase-3's new secure-OTel.md doc rows,
  section, and Mermaid node/edge/style. Config code block -> prose; merged the
  secure-OTel hardening pointer with the authoritative-config prose.
- Phase3_taskList.md: removed the "dotted keys for readability" note that came
  from phase-2 — phase 3 already uses the underscore keys.

Reviewed by code-review agents: telemetry instrumentation intact, naming check
green (47 keys across 7 *SpanNames.h headers), no conflict markers.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 16:45:34 +01:00
Pratik Mankawde
9e6c5b5778 docs(telemetry): note dotted attr keys in phase 3-5 task lists are illustrative
The Phase3/4/5 task lists show attribute keys in the older dotted
`xrpl.<domain>.<field>` form because that mirrors how a fully qualified
attribute reads in a Tempo trace view, which keeps the planning prose readable.
Add a note to each clarifying that the implemented keys follow the underscore
convention in CONTRIBUTING.md and that the *SpanNames.h constants (enforced by
the CI naming check) are the single source of truth — so the dotted form here
is illustrative, not the literal key.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 16:21:28 +01:00
Pratik Mankawde
afe0818c33 Merge branch 'pratik/otel-phase1c-rpc-integration' into pratik/otel-phase2-rpc-tracing
Bring the naming convention, code-sample cleanup, and CI naming check into
phase 2 (RPC tracing). Phase 2 introduces PathFindSpanNames.h.

Conflict resolution:
- 04-code-samples.md, POC_taskList.md: deletion wins.
- 02-design-decisions.md: took the convention-applied tables, but kept phase-2's
  accurate PathFinding summary row (pathfind_fast/search_level/num_paths/...,
  matching the implemented PathFindSpanNames.h).
- 05/08: took the code-block-free prose; kept phase-2's Phase2-5_taskList.md
  index rows (dropping only the deleted POC row). Fixed stale setup_Telemetry/
  make_Telemetry doc references to the code-correct setupTelemetry/makeTelemetry.
- Telemetry.h auto-merged to the constant-based @code examples.

check_otel_naming.py change: exempt test files from Rule F (tests pass
arbitrary literal keys to exercise the API). The check passes clean on the
merged tree (24 keys across 4 *SpanNames.h headers, including PathFind).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 15:47:30 +01:00
Pratik Mankawde
d5efd657ae Merge branch 'pratik/otel-phase1a-plan-docs' into pratik/otel-phase1b-telemetry-infra
Bring the phase-1a code-sample cleanup into phase-1b.

Phase 1a deleted the two code-sample docs (04-code-samples.md,
POC_taskList.md) and stripped C++/config code blocks from the remaining
plan docs, replacing them with prose summaries and pointers to the real
source/config files.

Conflict resolution:
- 04-code-samples.md, POC_taskList.md: deletion wins (phase-1b's SpanGuard
  rewrites of these files are intentionally dropped).
- 03/05/OpenTelemetryPlan.md: took phase-1a's code-block-free prose, but
  kept phase-1b's accurate descriptions of the real telemetry code
  (DiscardFlag.h, FilteringSpanProcessor, SpanGuard factory methods, and
  the corrected file-count/file-list tables).
- presentation.md left untouched.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 14:40:31 +01:00
Pratik Mankawde
d6450631bf removed code blocks from plan docs
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-11 14:29:01 +01:00
Pratik Mankawde
e9cb9421ef Merge branch 'pratik/otel-phase1a-plan-docs' into pratik/otel-phase1b-telemetry-infra
Bring the span attribute naming convention (phase 1a) into phase 1b.

Conflict resolution kept phase-1b's SpanGuard-based workflow and applied
the underscore naming convention to all non-code-sample text:
- Converted prose, tables, Mermaid labels, and TraceQL/PromQL query
  references across the plan docs to the underscore form.
- Converted the two @code attribute-key examples in Telemetry.h
  (command, tx_type).
- Left the code-sample files (04-code-samples.md, POC_taskList.md) and
  03-implementation-strategy.md code blocks at the phase-1b version; the
  code-sample docs are slated for removal on phase-1a.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 14:10:52 +01:00
Pratik Mankawde
c0272f1314 docs: Add span attribute naming convention (OTel phase 1a)
Establish the single, authoritative naming convention for OpenTelemetry
span attribute keys so the code, collector, Tempo, dashboards, and docs
stay in sync.

- CONTRIBUTING.md: new "Telemetry span attribute naming" section under
  the Style guide as the permanent, canonical home for the rules.
- OpenTelemetryPlan/02-design-decisions.md: new section 2.3.3 stating the
  decided convention as design, and section 2.4 attribute schema realigned
  to the underscore form (exact key spelling defers to the *SpanNames.h
  constants).
- Sweep the remaining plan docs: convert dotted xrpl.<domain>.<field> span
  attribute keys to the underscore form; leave span names and the
  OTel-standard service.*/http.* and xrpl.network.* resource keys dotted.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 13:29:53 +01:00
Pratik Mankawde
81919612f8 Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-06-10 10:22:07 +01:00
Pratik Mankawde
81beeb6747 Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-10 10:21:12 +01:00
Pratik Mankawde
21c177c8f9 Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation 2026-06-10 10:20:20 +01:00
Pratik Mankawde
a7c3cd8a79 Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-06-10 10:20:03 +01:00
Pratik Mankawde
55a674f042 docs(telemetry): remove sampling_ratio knob from phase-6 docs and configs
Head sampling is fixed at 1.0 and not configurable. Drop sampling_ratio
from the docker sample/test configs and the data-collection reference;
rewrite the span-metric sampling caveat and volume guidance to point at
collector-side tail sampling.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 10:03:31 +01:00
Pratik Mankawde
0b36443382 Merge branch 'pratik/otel-phase9-metric-gap-fill' into pratik/otel-phase10-workload-validation 2026-06-09 19:12:25 +01:00
Pratik Mankawde
91159c318b Merge branch 'pratik/otel-phase8-log-correlation' into pratik/otel-phase9-metric-gap-fill 2026-06-09 19:12:17 +01:00
Pratik Mankawde
72b449c3ce Merge branch 'pratik/otel-phase7-native-metrics' into pratik/otel-phase8-log-correlation
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-06-09 19:12:06 +01:00
Pratik Mankawde
ffd972f8be Merge branch 'pratik/otel-phase6-statsd' into pratik/otel-phase7-native-metrics 2026-06-09 19:10:58 +01:00
Pratik Mankawde
5e6817df61 Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd 2026-06-09 19:10:50 +01:00