ARCHIVE/rippled
1
0
Fork 0
mirror of https://github.com/XRPLF/rippled.git synced 2026-08-20 13:50:57 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
6bbb4cfcf57a32515a7a9c5fbdb0ddda0093bb7c
rippled/OpenTelemetryPlan/08-appendix.md
Pratik Mankawde 3860c93db2 refactor(telemetry): route dashboards, runbook and collector work to phase-9 
These changes were developed on the phase-10 branch but belong to content this
branch and its upstreams introduced. Carrying them on phase-10 made its PR diff
report churn in files phase-10 does not own, and left each PR claiming a scope
that did not match its contents.

Moved here from phase-10 (identical content, no functional change):

- Dashboards: all 14 existing boards plus the new log-derived-insights board.
- Docs: telemetry-runbook.md (minus the workload/benchmark sections, which
  describe phase-10 tooling) and the new telemetry-glossary.md.
- Grafana Cloud + Alloy export path: collector config, compose override, the
  two .env examples and alloy/config.alloy.
- Local stack: otel-collector-config.yaml gains sub-millisecond and
  second-scale spanmetrics buckets, pins unit=ms, and promotes
  close_time_correct; integration-test.sh and TESTING.md follow.
- Node configs: exported_instance -> service_instance_id in comments; the
  mainnet sample now logs at warning to bound log volume.
- Metrics code: Telemetry.cpp builds the metrics pipeline in the constructor
  via initMetrics() so the global MeterProvider is published before any
  subsystem creates a beast::insight instrument, and the histogram view keeps
  each instrument's own name instead of collapsing them under one series.
  MetricsRegistry gains a last_close_time gauge and skips negative job-queue
  durations. OTelCollector drops an unused accessor.
- Naming CI: xrpl_work_item joins EXTERNAL_INFRA_LABELS and Rule E accepts the
  dotted perf-iac resource-attribute form. This must travel with the
  dashboards and runbook that reference those labels, or the rules fail.
- Doxygen input glob no longer recurses dot-directories.

Sections describing phase-10 tooling stay on phase-10 and keep their
"Future Enhancement" / "Planned, not yet implemented" markers here; phase-10
removes those markers when it lands the tooling.
2026-08-04 16:10:04 +01:00

13 KiB
Raw Blame History

Appendix

Parent Document: OpenTelemetryPlan.md Related: Observability Backends

8.1 Glossary

OTLP = OpenTelemetry Protocol | TxQ = Transaction Queue

Term Definition
Span A unit of work with start/end time, name, and attributes
Trace A collection of spans representing a complete request flow
Trace ID 128-bit unique identifier for a trace
Span ID 64-bit unique identifier for a span within a trace
Context Carrier for trace/span IDs across boundaries
Propagator Component that injects/extracts context
Sampler Decides which traces to record
Exporter Sends spans to backend
Collector Receives, processes, and forwards telemetry
OTLP OpenTelemetry Protocol (wire format)
W3C Trace Context Standard HTTP headers for trace propagation
Baggage Key-value pairs propagated across service boundaries
Resource Entity producing telemetry (service, host, etc.)
Instrumentation Code that creates telemetry data

xrpld-Specific Terms

Term Definition
Overlay P2P network layer managing peer connections
Consensus XRP Ledger consensus algorithm (RCL)
Proposal Validator's suggested transaction set for a ledger
Validation Validator's signature on a closed ledger
HashRouter Component for transaction deduplication
JobQueue Thread pool for asynchronous task execution
PerfLog Existing performance logging system in xrpld
Beast Insight Existing metrics framework in xrpld
PathFinding Payment path computation engine for cross-currency payments
TxQ Transaction queue managing fee-based prioritization
LoadManager Dynamic fee escalation based on network load
SHAMap SHA-256 hash-based map (Merkle trie variant) for ledger state

Phase 9–11 Terms

Term Definition
MetricsRegistry Centralized class for OTel async gauge registrations (Phase 9)
ObservableGauge OTel Metrics SDK async instrument polled via callback at fixed intervals
PeriodicMetricReader OTel SDK component that invokes gauge callbacks at configurable intervals
CountedObject xrpld template that tracks live instance counts via atomic counters
TxQ Transaction queue managing fee escalation and ordering
Load Factor Combined multiplier affecting transaction cost (local, cluster, network)
OTel Collector Receiver Custom Go plugin that polls xrpld RPC and emits OTel metrics (Phase 11)

8.2 Span Hierarchy Visualization

The authoritative span-flow diagrams — a master overview plus per-stage flowcharts (ingress, the shared apply pipeline, the consensus round, ledger finalize, and the pathfinding / ledger-acquire side flows) — live in the operator runbook. They map every span onto the real xrpld control flow and XRPL protocol order (verified against code and docs/consensus.md, with file:line evidence), label every node and branch with the span that represents that state or transition, and call out where the OpenTelemetry span parent links diverge from that flow.

See: docs/telemetry-runbook.md § Protocol Span Flow.

The full span inventory (names, attributes, parents as instrumented) is in 09-data-collection-reference.md §1.

8.3 References

OTLP = OpenTelemetry Protocol

OpenTelemetry Resources

  1. OpenTelemetry C++ SDK
  2. OpenTelemetry Specification
  3. OpenTelemetry Collector
  4. OTLP Protocol Specification

Standards

  1. W3C Trace Context
  2. W3C Baggage
  3. Protocol Buffers

xrpld Resources

  1. xrpld Source Code
  2. XRP Ledger Documentation
  3. xrpld Overlay README
  4. xrpld RPC README
  5. xrpld Consensus README

8.4 Version History

Version Date Author Changes
1.0 2026-02-12 - Initial implementation plan
1.1 2026-02-13 - Refactored into modular documents
1.2 2026-03-09 - Added Phases 9–11 (future enhancement plans)
1.3 2026-03-24 - Review fixes: accuracy corrections, cross-document consistency

8.5 Document Index

Plan Documents

Document Description
OpenTelemetryPlan.md Master overview and executive summary
00-tracing-fundamentals.md Distributed tracing concepts and OTel primer
01-architecture-analysis.md xrpld architecture and trace points
02-design-decisions.md SDK selection, exporters, span conventions
03-implementation-strategy.md Directory structure, performance analysis
05-configuration-reference.md xrpld config, CMake, Collector configs
06-implementation-phases.md Timeline, tasks, risks, success metrics
07-observability-backends.md Backend selection and architecture
08-appendix.md Glossary, references, version history
secure-OTel.md Threat model and hardening (mTLS, peer validation)
09-data-collection-reference.md Span/metric/dashboard inventory

Task Lists

Document Description
Phase2_taskList.md RPC layer trace instrumentation
Phase3_taskList.md Peer overlay & consensus tracing
Phase4_taskList.md Transaction lifecycle tracing
Phase5_taskList.md Ledger processing & advanced tracing
Phase5_IntegrationTest_taskList.md Observability stack integration tests
Phase7_taskList.md Native OTel metrics migration
Phase8_taskList.md Log-trace correlation
Phase9_taskList.md Internal metric instrumentation gap fill (future)
Phase10_taskList.md Synthetic workload generation & validation (future)
Phase11_taskList.md Third-party data collection pipelines (future)

Note

: Phases 1 and 6 do not have separate task list files. Phase 1 tasks are documented in 06-implementation-phases.md §6.2. Phase 6 tasks are documented in 06-implementation-phases.md §6.7.

8.6 Phase 9–11 Cross-Reference Guide

This guide maps Phase 9–11 content to its location across the documentation.

Phase 9: Internal Metric Instrumentation Gap Fill

Content Location
Plan & architecture 06-implementation-phases.md §6.8.2
Task list (10 tasks) Phase9_taskList.md
Future metric definitions (~50) 09-data-collection-reference.md §5b
New class: MetricsRegistry src/xrpld/telemetry/MetricsRegistry.h/.cpp (planned)
New dashboards fee-market, job-queue (planned)

Metric categories: NodeStore I/O, Cache Hit Rates, TxQ, PerfLog Per-RPC, PerfLog Per-Job, Counted Objects, Fee Escalation & Load Factors.

Phase 10: Synthetic Workload Generation & Telemetry Validation

Content Location
Plan & architecture 06-implementation-phases.md §6.8.3
Task list (7 tasks) Phase10_taskList.md
Validation inventory 09-data-collection-reference.md §5c
Test harness docker/telemetry/docker-compose.workload.yaml (planned)
CI workflow .github/workflows/telemetry-validation.yml (planned)

Validates: 16 spans, 22 attributes, 300+ metrics, 10 dashboards, log-trace correlation.

Phase 11: Third-Party Data Collection Pipelines

Content Location
Plan & architecture 06-implementation-phases.md §6.8.4
Task list (11 tasks) Phase11_taskList.md
External metric definitions (~30) 09-data-collection-reference.md §5d
Custom OTel Collector receiver docker/telemetry/otel-rippled-receiver/ (planned)
Prometheus alerting rules (11) 09-data-collection-reference.md §5d
New dashboards (4) Validator Health, Network Topology, Fee Market (External), DEX & AMM

Consumer categories: Exchanges, Payment Processors, DeFi/AMM, NFT Marketplaces, Analytics Providers, Wallets, Compliance, Academic Researchers, Institutional Custody, CBDC Bridge Operators.

Previous: Observability Backends | Back to: Overview