Commit Graph

269 Commits

Author SHA1 Message Date
Pratik Mankawde
360ecbdb44 feat(telemetry): add Phase 5 documentation, deployment configs, and integration tests
Add the observability stack deployment infrastructure and integration
test framework for verifying end-to-end trace export.

- Add Grafana dashboards: RPC performance, transaction overview,
  consensus health (pre-provisioned via dashboards.yaml)
- Add Prometheus config for spanmetrics collection from OTel Collector
- Update OTel Collector config with spanmetrics connector and
  prometheus exporter for RED metrics
- Add docker-compose services: prometheus, dashboard provisioning
- Add integration-test.sh with Tempo API-based span verification
  (replaces previous Jaeger-based approach)
- Add TESTING.md with step-by-step deployment and verification guide
- Add telemetry-runbook.md for production operations reference
- Add xrpld-telemetry.cfg sample configuration
- Add toDisplayString() for ConsensusMode (human-readable span values)
- Update Phase 2/3 task lists with known issues sections
- Add Phase 5 integration test task list
- Add TraceContext protobuf fields for future relay propagation
- Wire telemetry lifecycle (setServiceInstanceId/start/stop) in
  Application.cpp

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-24 21:43:17 +01:00
Pratik Mankawde
ccd3ceac84 feat(telemetry): add Phase 4 consensus tracing with SpanGuard API
Instrument the consensus subsystem with OpenTelemetry spans covering
the full round lifecycle: round start, establish phase, proposal send,
ledger close, position updates, consensus check, accept, validation
send, and mode changes.

Key design choices adapted from the original Phase 4 implementation
to the new SpanGuard factory pattern introduced in Phase 3:

- Add SpanGuard::hashSpan() for category-gated hash-derived trace IDs
  (consensus round spans share trace_id across validators via ledger hash)
- Add SpanGuard::addEvent() overload with key-value attribute pairs
  (used for dispute.resolve events during position updates)
- Add ConsensusSpanNames.h with compile-time span name constants
  following the colocated *SpanNames.h pattern from Phase 3
- Add consensusTraceStrategy config option ("deterministic"/"attribute")
  for cross-node trace correlation strategy selection
- Use SpanGuard::linkedSpan() for follows-from relationships between
  consecutive rounds and cross-thread validation spans
- Use SpanGuard::captureContext() for thread-safe context propagation
  from consensus thread to jtACCEPT worker thread

Spans produced: consensus.round, consensus.proposal.send,
consensus.ledger_close, consensus.establish, consensus.update_positions,
consensus.check, consensus.accept, consensus.accept.apply,
consensus.validation.send, consensus.mode_change

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-24 21:35:50 +01:00
Pratik Mankawde
5fe4a60aa5 feat(telemetry): add hash-derived trace IDs for transaction spans
Derive trace_id from txHash[0:16] so all nodes handling the same
transaction produce spans under the same trace. Protobuf span_id
propagation provides parent-child relay ordering when available.

- Add SpanGuard::txSpan() factory methods (hash-derived trace ID)
- Add TxTracing.h helpers: txReceiveSpan(), txProcessSpan()
- Update PeerImp and NetworkOPs to use the new helpers

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-21 17:31:16 +01:00
Pratik Mankawde
c7ef0eb647 feat(telemetry): Phase 3 transaction tracing with protobuf context propagation
- TraceContext protobuf message for cross-node trace propagation
  (added to TMTransaction, TMProposeSet, TMValidation at field 1001)
- TraceContextPropagator.h: inline extractFromProtobuf/injectToProtobuf
- PeerImp::handleTransaction: tx.receive span with peer.id, peer.version,
  tx.hash, tx.suppressed, tx.status attributes
- NetworkOPsImp::processTransaction: tx.process span with tx.hash,
  tx.local, tx.path attributes
- Tempo search filters for tx.hash, tx.local, tx.status
- Unit tests for TraceContextPropagator (round-trip, edge cases)
- Levelization: xrpld.app/overlay > xrpld.telemetry dependencies

Translated from macro API (XRPL_TRACE_TX/SET_ATTR) to SpanGuard factory
pattern introduced in Phase 1c.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-21 15:35:43 +01:00
Pratik Mankawde
c59ea77bdc fix(telemetry): address Phase 2 code review findings
- Move node health attribute strings to compile-time constants in
  SpanNames.h (attr::nodeAmendmentBlocked, attr::nodeServerState)
- Add Tempo search filters for node health attributes
- Remove unnecessary .c_str() on strOperatingMode() return
- Add samplingRatio clamping test (values > 1.0 and < 0.0)
- Fix Task 2.3 status: delivered in Phase 1c, not Phase 2

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-20 16:49:40 +01:00
Pratik Mankawde
8671798864 refactor(telemetry): extract span name constants into modular headers
Centralise scattered string literals into compile-time constants using
StaticStr<N> and join() for dot-separated composition. Shared primitives
live in SpanNames.h; RPC-specific names in RpcSpanNames.h. Future modules
(consensus, peer, ledger) add their own *SpanNames.h without bloating
the central header.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-20 16:48:12 +01:00
Pratik Mankawde
7c8a3906a1 refactor(telemetry): replace per-category factory methods with TraceCategory enum
Replace rpcSpan(), txSpan(), consensusSpan(), peerSpan(), ledgerSpan()
with a single span(TraceCategory, prefix, name) factory method. Adding
a new traceable subsystem now requires only a new enum value and one
switch case — no new methods or header changes.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-17 17:40:37 +01:00
Pratik Mankawde
ad233846fa fix(telemetry): address Phase 1b code review findings
- SpanContext::isValid(): add inline no-op when XRPL_ENABLE_TELEMETRY
  is not defined, preventing a linker error if called in that path
- linkedSpan(): set kIsRootSpanKey on the StartSpanOptions parent
  context so linked spans start a genuinely independent sub-tree
  instead of silently becoming children of the current active span
- Telemetry::instance_: use std::atomic with acquire/release ordering
  to avoid a data race between start()/stop() and factory methods
  called from worker threads

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-17 17:15:36 +01:00
Pratik Mankawde
714ac49f47 fix(telemetry): address Phase 1b code review findings
Redesign SpanGuard with pimpl idiom to hide all OpenTelemetry types
from public headers. Add global Telemetry accessor so SpanGuard factory
methods work without explicit Telemetry references. Add child/linked
span creation and cross-thread context propagation. Update plan docs
to reflect macro removal in favor of SpanGuard factory pattern.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-17 16:31:50 +01:00
Pratik Mankawde
0de5a77605 feat(telemetry): add FilteringSpanProcessor and SpanGuard::discard()
Add span discard mechanism that drops unwanted spans before they enter
the batch export queue, saving both network bandwidth and storage.

FilteringSpanProcessor is a custom SpanProcessor decorator that wraps
BatchSpanProcessor. SpanGuard::discard() sets a thread-local flag
(tl_discardCurrentSpan) before calling Span::End(). The OTel SDK calls
OnEnd() synchronously on the same thread, where the flag is checked and
cleared to drop the span.

New file: DiscardFlag.h — zero-dependency header for the thread-local
flag, avoiding transitive include bloat from Telemetry.h.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-16 17:43:58 +01:00
Pratik Mankawde
4830715d57 fix(telemetry): address review findings and PR #6437 comments
Critical fixes:
- Restore accidentally removed mallocTrim call and MallocTrim.h include
- Add missing shouldTraceLedger() to interface and all implementations
- Derive networkId/networkType from config_->NETWORK_ID (0=mainnet,
  1=testnet, 2=devnet) instead of leaving defaults unpopulated
- Clamp sampling_ratio to [0.0, 1.0] in config parser

PR comment fixes:
- Rename rippled -> xrpld in service name defaults, getTracer() calls,
  Docker network, comments, and docs/build/telemetry.md
- Remove exporter config option (only otlp_http supported)
- Add trace_ledger and service_name to example config
- Clarify head-based sampling semantics in config comments
- Add filter descriptions for span intrinsic filters in Grafana datasource
- Add inline comments to Docker Compose services

Docker/config improvements:
- Remove deprecated version: "3.8" from docker-compose.yml
- Pin images: collector 0.121.0, grafana 11.5.2
- Add health_check extension to otel-collector-config.yaml
- Comment out Tempo metrics_generator remote_write (no Prometheus service)
- Add Prometheus datasource caveat in Grafana datasource config

Other:
- Revert unrelated formatting changes in ServiceRegistry.h
- Change Conan telemetry default to False (matches CMake OFF)
- Add CLAUDE.md-required docs (ASCII diagrams, usage examples,
  @note thread-safety) to Telemetry.h and SpanGuard.h

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-16 17:07:01 +01:00
Pratik Mankawde
ce82e067bd Phase 1b: Telemetry core infrastructure - CMake, Conan, SpanGuard, config
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-16 15:37:58 +01:00
Sergey Kuznetsov
d52d735543 chore: Move codegen venv setup into build stage (#6617)
Co-authored-by: JCW <a1q123456@users.noreply.github.com>
Co-authored-by: Bart <bthomee@users.noreply.github.com>
2026-04-15 18:50:49 +00:00
Alex Kremer
6a0ce46755 chore: Enable most clang-tidy bugprone checks (#6929)
Co-authored-by: Ayaz Salikhov <mathbunnyru@users.noreply.github.com>
2026-04-14 20:24:21 +00:00
Bart
2f029a2120 refactor: Improve exception handling (#6540) (#6735)
Co-authored-by: Bart <11445373+bthomee@users.noreply.github.com>
2026-04-14 17:14:24 +00:00
Zhiyuan Wang
61fbde3a71 refactor: Remove unused notTooManyOffers function from NFTokenUtils (#6737) 2026-04-13 23:18:10 +00:00
yinyiqian1
d52dd29d20 fix: Address AI reviewer comments for Permission Delegation (#6675) 2026-04-08 20:22:19 +00:00
Mayukha Vadari
7793b5f10b refactor: Combine AMMHelpers and AMMUtils (#6733) 2026-04-08 17:38:33 +00:00
Gregory Tsipenyuk
dfcad69155 feat: Add MPT support to DEX (#5285) 2026-04-08 16:17:37 +00:00
Bart
c00ed673a8 refactor: Rename non-functional uses of ripple(d) to xrpl(d) (#6676)
Co-authored-by: Bart <11445373+bthomee@users.noreply.github.com>
2026-04-07 13:00:17 +00:00
Mayukha Vadari
f239256d87 refactor: Move more helper files into libxrpl/ledger/helpers (#6731)
Co-authored-by: xrplf-ai-reviewer[bot] <266832837+xrplf-ai-reviewer[bot]@users.noreply.github.com>
2026-04-06 22:36:32 +00:00
Alex Kremer
29e49abd3c chore: Enable clang-tidy coreguidelines checks (#6698)
Co-authored-by: Ayaz Salikhov <mathbunnyru@users.noreply.github.com>
2026-04-01 15:46:14 +00:00
Alex Kremer
2502befb42 chore: Enable clang-tidy misc checks (#6655) 2026-03-31 17:29:45 +00:00
Bart
7f53351920 chore: Remove unnecessary clang-format off/on directives (#6682)
Co-authored-by: Bart <11445373+bthomee@users.noreply.github.com>
2026-03-31 15:38:04 +00:00
Alex Kremer
ab8c168e3b chore: Enable remaining clang-tidy performance checks (#6648)
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-30 17:08:47 +00:00
Jingchen
3a477e4d01 refactor: Address PR comments after the modularisation PRs (#6389)
Signed-off-by: JCW <a1q123456@users.noreply.github.com>
Co-authored-by: Bart <bthomee@users.noreply.github.com>
2026-03-30 15:22:38 +00:00
Pratik Mankawde
3d9c545f59 fix: Guard Coro::resume() against completed coroutines (#6608)
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 18:52:18 +00:00
Vito Tumas
9b944ee8c2 refactor: Split LoanInvariant into LoanBrokerInvariant and LoanInvariant (#6674) 2026-03-27 18:35:42 +00:00
Jingchen
addc1e8e25 refactor: Make function naming in ServiceRegistry consistent (#6390)
Signed-off-by: JCW <a1q123456@users.noreply.github.com>
Co-authored-by: Ed Hennis <ed@ripple.com>
2026-03-26 14:11:16 +00:00
Vito Tumas
b9d07730f3 feat: Add placeholder amendment for assorted bug fixes (#6652) 2026-03-25 23:54:33 +00:00
Jingchen
8f182e825a refactor: Modularise ledger (#6536)
Signed-off-by: JCW <a1q123456@users.noreply.github.com>
Co-authored-by: Bart <bthomee@users.noreply.github.com>
Co-authored-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-03-25 16:32:45 +00:00
Alex Kremer
0eedefbf45 refactor: Enable more clang-tidy readability checks (#6595)
Co-authored-by: Sergey Kuznetsov <kuzzz99@gmail.com>
2026-03-24 15:42:12 +00:00
Mayukha Vadari
8b986e4ab0 refactor: Improve imports to only call the needed helpers (#6624) 2026-03-24 10:20:32 +00:00
Mayukha Vadari
e0dbe90370 refactor: Move ledger entry helper functions from View.h/View.cpp to dedicated helper files (#6453)
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-23 15:39:58 +00:00
Jingchen
b1e5ba0518 feat: Add code generator for transactions and ledger entries (#6443)
Signed-off-by: JCW <a1q123456@users.noreply.github.com>
Co-authored-by: Bart <bthomee@users.noreply.github.com>
2026-03-18 21:11:51 +00:00
Mayukha Vadari
d360e7c5b6 refactor: Rename transactor files/classes to match the tx name (#6580) 2026-03-18 19:52:07 +00:00
Alex Kremer
57e4cbbcd9 refactor: Add simple clang-tidy readability checks (#6556)
This change enables the following clang-tidy checks:
-  readability-avoid-nested-conditional-operator,
-  readability-avoid-return-with-void-value,
-  readability-braces-around-statements,
-  readability-const-return-type,
-  readability-container-contains,
-  readability-container-size-empty,
-  readability-else-after-return,
-  readability-make-member-function-const,
-  readability-redundant-casting,
-  readability-redundant-inline-specifier,
-  readability-redundant-member-init,
-  readability-redundant-string-init,
-  readability-reference-to-constructed-temporary,
-  readability-static-definition
2026-03-18 16:41:49 +00:00
Mayukha Vadari
b92a9a3053 fix: Make assorted NFT fixes (#6566)
This change:
* Removes a set of unnecessary brackets in the initialization of an `std::uint32_t`.
* Fixes a couple of incorrect flags (same value, just wrong variables - so no amendment needed).
2026-03-18 14:47:59 +00:00
Mayukha Vadari
dcaef828b4 refactor: Replace !=/== tesSuccess with using isTesSuccess (#6409)
This change replaces all instances of `<variable> != tesSUCCESS` with `!isTesSuccess(<variable>)` and `<variable> == tesSUCCESS` with `isTesSuccess(<variable>)`.
2026-03-18 14:15:10 +00:00
Copilot
9e14707e77 fix: Peer crawler port field type inconsistency (#6318)
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: mvadari <8029314+mvadari@users.noreply.github.com>
Co-authored-by: Mayukha Vadari <mvadari@gmail.com>
Co-authored-by: Mayukha Vadari <mvadari@ripple.com>
Co-authored-by: Bart <bthomee@users.noreply.github.com>
2026-03-17 22:03:56 +00:00
tequ
b129b71c33 refactor: Use ReadView instead of ApplyView in authorizedDepositPreauth() (#6560) 2026-03-17 15:56:51 -04:00
tequ
013c2d6a56 refactor: Add const qualifier to SLE in verifyDepositPreauth parameter (#6555) 2026-03-17 19:33:18 +00:00
Alex Kremer
72f4cb097f refactor: Enable remaining clang-tidy cppcoreguidelines checks (#6538) 2026-03-17 19:09:05 +00:00
Mayukha Vadari
78b2d70a11 refactor: Assorted small DID fixes (#6552)
This change:
* Makes `addSLE` in `DIDSet` a static function, instead of a free function.
* Renames `Attestation` to `Data` everywhere (an artifact of a previous name for the field).
* Actually runs a set of tests that were not included in the `run` function of `DID_test`.
2026-03-17 14:44:07 +00:00
Mayukha Vadari
252c6768df refactor: Clean up getFeePayer, mSourceBalance, and mPriorBalance (#6478)
This change:
* Introduces a new helper function on `STTx`, `getFeePayer`.
* Removes the usage of `mSourceBalance` and replaces it with SLE balance lookups.
* Renames `mPriorBalance` to `preFeeBalance_`

This simplifies some of the code in the transactors and makes it a lot more readable.
2026-03-17 14:12:16 +00:00
Pratik Mankawde
5ae97fa8ae refactor: Add no-ASAN macro for Throw statements (#6373)
Throwing exceptions from code sometime confuses ASAN, as it cannot keep track of stack frames. This change therefore adds a macro to skip instrumentation around the `Throw` function.
2026-03-17 13:10:32 +00:00
Bart
ffea3977f0 refactor: Rename system name from 'ripple' to 'xrpld' (#6347)
Per [XLS-0095](https://xls.xrpl.org/xls/XLS-0095-rename-rippled-to-xrpld.html), we are taking steps to rename ripple(d) to xrpl(d). This change modifies the system name from `rippled` to `xrpld`.

The system name is used in limited places:
* When no explicit config file is passed via the `--config` flag, then the system name is used to construct the path where the config file and database may be stored, via the `$XDG_CONFIG_HOME` and `$XDG_DATA_HOME` directories, respectively.
* It is used in the metadata and user-agent as part of RPC calls.
* It is newly used in the full version string.
2026-03-16 21:51:31 +00:00
Alex Kremer
47a235b7be chore: Enable clang-tidy switch-missing-default-case check (#6461) 2026-03-16 17:19:37 -04:00
Pratik Mankawde
983816248a fix: Switch to boost::coroutine2 (#6372)
ASAN wasn't able to keep track of `boost::coroutine` context switches, and would lead to many false positives being detected. By switching to `boost::coroutine2` and `ucontext`, ASAN is able to know about the context switches advertised by the `boost::fiber` class, which in turn leads to more cleaner ASAN analysis.
2026-03-16 15:34:15 +00:00
Pratik Mankawde
b585dc78bb fix: Fix memory leaks in HTTPClient (#6370)
The `HTTPClient` class initializes a global SSL context via `initializeSSLContext()`. However, it had no way to release it, which caused memory leaks flagged by the LeakSanitizer. Multiple LSAN suppressions in the sanitizers' suppressions file were masking these leaks. Our test code also manually called `initializeSSLContext()` in each test without guaranteed cleanup on failure paths.

This change fixes these memory leaks by adding a `cleanupSSLContext()` method to properly release the global SSL context, and removes the corresponding LSAN suppressions. The change further refactors the `HTTPClient` tests to use a Google Test fixture (`HTTPClientTest`) that manages the SSL context lifecycle via RAII (SetUp/TearDown), making it impossible for tests to leak the context.
2026-03-16 14:12:48 +00:00