Merge branch 'pratik/otel-phase5-docs-deployment' into pratik/otel-phase6-statsd

Resolve runbook conflict: keep both phase 6 ledger/peer span tables
AND new insights/sample queries section from the enrichment work.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Pratik Mankawde
2026-06-03 16:22:20 +01:00
17 changed files with 373 additions and 53 deletions

View File

@@ -154,21 +154,53 @@ Node health (`amendment_blocked`, `server_state`) is not part of the telemetry s
---
## Task 2.10: RPC and PathFind Span Attribute Gap Fill
**Status**: COMPLETE
**Objective**: Wire up workflow-identifying attributes that enable filtering and grouping traces by request characteristics without drilling into child spans.
**Attributes added**:
| Span | Attribute | Type | Source |
| ------------------- | ---------------------------- | ------ | --------------------------------- |
| `rpc.http_request` | `request_payload_size` | int64 | `request.body().size()` |
| `rpc.process` | `is_batch` | bool | `method == "batch"` check |
| `rpc.process` | `batch_size` | int64 | `params.size()` (only when batch) |
| `rpc.ws_message` | `command` | string | `jv[command]` or `jv[method]` |
| `rpc.command.*` | `load_type` | string | `context.loadType.label()` |
| `pathfind.compute` | `pathfind_dest_amount` | string | `saDstAmount_.getFullText()` |
| `pathfind.compute` | `pathfind_dest_currency` | string | `to_string(saDstAmount_.asset())` |
| `pathfind.discover` | `pathfind_num_source_assets` | int64 | `sourceAssets.size()` |
**New attr keys**: `RpcSpanNames.h` (`isBatch`, `batchSize`, `loadType`), `PathFindSpanNames.h` (`destAmount`, `destCurrency`, `numSourceAssets`).
**Modified files**:
- `src/xrpld/rpc/detail/RpcSpanNames.h`
- `src/xrpld/rpc/detail/PathFindSpanNames.h`
- `src/xrpld/rpc/detail/ServerHandler.cpp`
- `src/xrpld/rpc/detail/RPCHandler.cpp`
- `src/xrpld/rpc/detail/PathRequest.cpp`
---
## Summary
| Task | Description | Status | Notes |
| ---- | ------------------------------------------- | ------------------- | ------------------------------------------------ |
| 2.1 | W3C Trace Context header extraction | Deferred → Phase 3 | No consumer in Phase 2; needs cross-node tracing |
| 2.2 | Per-category span creation | Complete (Phase 1c) | Superseded by TraceCategory enum + SpanGuard |
| 2.3 | Add shouldTraceLedger() interface method | Complete (Phase 1c) | Delivered in Phase 1c base branch |
| 2.4 | Unit tests for core telemetry | Complete | TelemetryConfig + SpanGuardFactory tests |
| 2.5 | Enhanced RPC span attributes (HTTP-level) | Deferred | Low value; span duration covers timing natively |
| 2.6 | Build verification and performance baseline | Complete | Verified in CI on Phase 1c |
| 2.7 | Grafana Tempo search filters | Complete | rpc-command, rpc-status, rpc-role filters |
| 2.8 | RPC span attribute enrichment (node health) | Dropped | Available via `server_info`/`server_state` RPC |
| 2.9 | PathFind RPC instrumentation | Complete | request, compute, update_all, discover |
| Task | Description | Status | Notes |
| ---- | ------------------------------------------- | ------------------- | --------------------------------------------------------- |
| 2.1 | W3C Trace Context header extraction | Deferred → Phase 3 | No consumer in Phase 2; needs cross-node tracing |
| 2.2 | Per-category span creation | Complete (Phase 1c) | Superseded by TraceCategory enum + SpanGuard |
| 2.3 | Add shouldTraceLedger() interface method | Complete (Phase 1c) | Delivered in Phase 1c base branch |
| 2.4 | Unit tests for core telemetry | Complete | TelemetryConfig + SpanGuardFactory tests |
| 2.5 | Enhanced RPC span attributes (HTTP-level) | Deferred | Low value; span duration covers timing natively |
| 2.6 | Build verification and performance baseline | Complete | Verified in CI on Phase 1c |
| 2.7 | Grafana Tempo search filters | Complete | rpc-command, rpc-status, rpc-role filters |
| 2.8 | RPC span attribute enrichment (node health) | Dropped | Available via `server_info`/`server_state` RPC |
| 2.9 | PathFind RPC instrumentation | Complete | request, compute, update_all, discover |
| 2.10 | RPC/PathFind span attribute gap fill | Complete | Batch detection, payload size, load cost, pathfind params |
**Delivered in this branch**: Tasks 2.4, 2.7, 2.9.
**Delivered in this branch**: Tasks 2.4, 2.7, 2.9, 2.10.
**Deferred with rationale**: Tasks 2.1 (→Phase 3), 2.5 (low priority).
**Dropped**: Task 2.8 (node health not duplicated on traces).
**Superseded**: Task 2.2 (Phase 1c SpanGuard factory covers this).

View File

@@ -466,6 +466,38 @@ This gives the best of both worlds: guaranteed cross-node correlation via determ
---
## Task 3.11: TX and TxQ Span Attribute Gap Fill
**Status**: COMPLETE
**Objective**: Add workflow-identifying attributes to transaction spans so operators can filter by transaction type and see outcomes without off-chain correlation.
**Attributes added**:
| Span | Attribute | Type | Source |
| --------------- | ---------------- | ------ | ------------------------------------------------------------------- |
| `tx.process` | `tx_type` | string | `TxFormats::getInstance().findByType(stx->getTxnType())->getName()` |
| `tx.process` | `fee` | int64 | `stx->getFieldAmount(sfFee).xrp().drops()` |
| `tx.process` | `sequence` | int64 | `stx->getSeqProxy().value()` |
| `tx.process` | `ter_result` | string | `transToken(e.result)` (set after batch application) |
| `tx.process` | `applied` | bool | `e.applied` (set after batch application) |
| `tx.receive` | `tx_type` | string | `TxFormats::getInstance().findByType(stx->getTxnType())->getName()` |
| `txq.enqueue` | `tx_type` | string | same pattern as above |
| `txq.accept.tx` | `txq_status` | string | `applied` / `failed` / `retried` |
| `txq.accept` | `ledger_changed` | bool | set at end of accept loop |
**New attr keys**: `TxSpanNames.h` (`txType`, `fee`, `sequence`, `terResult`, `applied`), `TxQSpanNames.h` (`txType`).
**Modified files**:
- `src/xrpld/app/misc/TxSpanNames.h`
- `src/xrpld/app/misc/detail/TxQSpanNames.h`
- `src/xrpld/app/misc/NetworkOPs.cpp`
- `src/xrpld/overlay/detail/PeerImp.cpp`
- `src/xrpld/app/misc/detail/TxQ.cpp`
---
## Summary
| Task | Description | New Files | Modified Files | Depends On |
@@ -480,6 +512,7 @@ This gives the best of both worlds: guaranteed cross-node correlation via determ
| 3.8 | TX span peer version attribute | 0 | 1 | 3.3 |
| 3.9 | Deterministic transaction trace ID | 0-1 | 3 | 3.2, 3.3 |
| 3.10 | TxQ instrumentation (6 spans) | 1 | 1 | 3.4 |
| 3.11 | TX/TxQ span attribute gap fill | 0 | 5 | 3.3, 3.10 |
**Parallel work**: Tasks 3.1 and 3.4 can start in parallel. Task 3.2 depends on 3.1. Tasks 3.3 and 3.5 depend on 3.2. Task 3.6 depends on 3.3 and 3.5. Task 3.8 depends on 3.3 (span must exist). Task 3.9 depends on 3.2 and 3.3. Task 3.10 depends on 3.4 (tx.process span must exist).

View File

@@ -255,14 +255,38 @@ Phase 7's `ValidationTracker` builds metric-level aggregation (1h/24h agreement
**Exit Criteria**:
- [ ] `consensus.validation.send` spans carry `xrpl.validation.ledger_hash` and `xrpl.validation.full`
- [x] `consensus.validation.send` spans carry `ledger_hash` and `full_validation`
- [ ] `peer.validation.receive` spans carry `xrpl.peer.validation.ledger_hash` and `xrpl.peer.validation.full`
- [ ] `consensus.accept` spans carry `validation_quorum` and `proposers_validated`
- [ ] Ledger hash attributes match between send and receive for the same ledger
- [x] Ledger hash attributes match between send and receive for the same ledger
- [ ] No impact on consensus performance
---
## Task 4.9: Consensus Span Attribute Gap Fill
**Status**: COMPLETE
**Objective**: Add workflow-critical attributes to consensus spans that enable operators to understand consensus outcomes, identify bow-out proposals, and correlate validations to specific ledgers.
**Attributes added**:
| Span | Attribute | Type | Source |
| --------------------------- | ----------------- | ------ | ------------------------------------- |
| `consensus.proposal.send` | `is_bow_out` | bool | `proposal.isBowOut()` |
| `consensus.accept` | `consensus_state` | string | `result.state` (yes/moved_on/expired) |
| `consensus.accept` | `disputes_count` | int64 | `result.disputes.size()` |
| `consensus.validation.send` | `ledger_hash` | string | `ledger.ledger->header().hash` |
**New attr keys**: `ConsensusSpanNames.h` (`isBowOut`, `ledgerHash`).
**Modified files**:
- `src/xrpld/consensus/ConsensusSpanNames.h`
- `src/xrpld/app/consensus/RCLConsensus.cpp`
---
## Summary
| Task | Description | Status | New Files | Modified Files | Depends On |
@@ -275,6 +299,7 @@ Phase 7's `ValidationTracker` builds metric-level aggregation (1h/24h agreement
| 4.6 | Transaction-consensus correlation | ✅ Done | 0 | 1 | 4.2, Phase 3 |
| 4.7 | Build verification and testing | ✅ Done | 0 | 0 | 4.1-4.6 |
| 4.8 | Validation span enrichment (ext. dashboard) | ❌ Not done | 0 | 2 | 4.4 |
| 4.9 | Consensus span attribute gap fill | ✅ Done | 0 | 2 | 4.1-4.5 |
**Parallel work**: Tasks 4.2, 4.3, and 4.4 can run in parallel after 4.1 is complete. Task 4.5 depends on all three. Task 4.6 depends on 4.2 and Phase 3. Task 4.8 depends on 4.4 (validation spans must exist).
@@ -282,11 +307,11 @@ Phase 7's `ValidationTracker` builds metric-level aggregation (1h/24h agreement
| Span Name | Method | Key Attributes |
| --------------------------- | ---------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `consensus.proposal.send` | `Adaptor::propose` | `xrpl.consensus.round` |
| `consensus.proposal.send` | `Adaptor::propose` | `xrpl.consensus.round`, `is_bow_out` |
| `consensus.ledger_close` | `Adaptor::onClose` | `xrpl.ledger.seq`, `xrpl.consensus.mode` |
| `consensus.accept` | `Adaptor::onAccept` | `proposers`, `round_time_ms` |
| `consensus.accept` | `Adaptor::onAccept` | `proposers`, `round_time_ms`, `quorum`, `disputes_count`, `consensus_state` |
| `consensus.accept.apply` | `Adaptor::doAccept` | `close_time`, `close_time_correct`, `close_resolution_ms`, `consensus_state`, `proposing`, `round_time_ms`, `xrpl.ledger.seq`, `parent_close_time`, `close_time_self`, `close_time_vote_bins`, `resolution_direction` |
| `consensus.validation.send` | `Adaptor::onAccept` (via validate) | `proposing` |
| `consensus.validation.send` | `Adaptor::onAccept` (via validate) | `proposing`, `ledger_hash`, `ledger_seq`, `full_validation`, `validation_sign_time` |
#### Close Time Attributes (consensus.accept.apply)