Compare commits

...

252 Commits

Author SHA1 Message Date
Pratik Mankawde
ea2224dc62 comments update
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-05-06 14:44:47 +01:00
Pratik Mankawde
6bc1492817 build sanitizers on gcc-15
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-05-06 14:43:03 +01:00
Pratik Mankawde
ad8f30f06a variable naming correction
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-05-06 12:23:59 +01:00
Pratik Mankawde
e7c7ab9a2f Merge branch 'develop' into pratik/test-tsan-and-gcc14-asan
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-05-06 11:57:42 +01:00
Pratik Mankawde
d67bec3bd6 Merge branch 'develop' into pratik/test-tsan-and-gcc14-asan
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-04-24 12:52:31 +01:00
Pratik Mankawde
1866a07add fix: Use BOOST_USE_TSAN instead of BOOST_USE_ASAN for TSAN builds
The cleanup in 959c7bcae2 merged separate ASAN/TSAN if-blocks into one,
but incorrectly defined BOOST_USE_ASAN for both. TSAN builds don't link
ASan's runtime, so __sanitizer_start/finish_switch_fiber are unresolved.

Restore the original separate blocks from d2079574c7.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-24 15:53:46 +00:00
Pratik Mankawde
959c7bcae2 minor cleanup
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-03-24 13:48:04 +00:00
Pratik Mankawde
fbac9b90e3 Merge branch 'develop' into pratik/test-tsan-and-gcc14-asan 2026-03-24 12:22:19 +00:00
Pratik Mankawde
c7d41dd486 Merge branch 'pratik/test-tsan-and-gcc14-asan' of github.com:XRPLF/rippled into pratik/test-tsan-and-gcc14-asan 2026-03-24 12:22:03 +00:00
Pratik Mankawde
02c9830184 fix: Add Doxygen docs and remove temporary CI filter
- Add Doxygen documentation to new methods, members, constants, and
  macros introduced by the TSAN/ASAN PR (Database::startReadThreads,
  BasicApp::DeferStart, ResourceManager::start, XRPL_SANITIZER_ACTIVE,
  coroStackSize, yieldStackSize)
- Add @note thread-safety tags where atomics were introduced
- Update Database constructor docs to reflect deferred thread startup
- Remove temporary CI filter that restricted builds to sanitizer-only
  variants (must run full matrix before merge)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-23 10:46:16 +00:00
Pratik Mankawde
390622f7c9 Merge branch 'develop' into pratik/test-tsan-and-gcc14-asan 2026-03-23 10:28:23 +00:00
Pratik Mankawde
6094b101b8 fix: Keep Clang sanitizers combined, only separate GCC
Clang doesn't have the 2GB shadow memory collision that GCC has,
so keep ASAN+UBSAN and TSAN+UBSAN combined for Clang-20. Only GCC-13
needs separate ASAN, TSAN, and UBSAN jobs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-20 16:32:35 +00:00
Pratik Mankawde
9bb2505c42 Merge branch 'develop' into pratik/test-tsan-and-gcc14-asan 2026-03-20 15:40:10 +00:00
Pratik Mankawde
2d9b5a39e8 fix: Separate ASAN+UBSAN and TSAN+UBSAN into independent CI jobs
Combined ASAN+UBSAN instrumentation inflates data sections past 2GB,
colliding with ASAN's fixed shadow memory layout at 0x7fff8000+.
Split into independent ASAN, TSAN, and UBSAN jobs per compiler to
keep each binary within the shadow memory limit.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-20 15:39:06 +00:00
Pratik Mankawde
661ce8f79b fix: Use -mcmodel=medium for GCC ASAN in CMake (match Conan profile)
The previous commit (f13d17165a) switched the Conan profile from
-mcmodel=large to -mcmodel=medium for GCC ASAN builds, but missed
updating cmake/XrplSanitizers.cmake. The main xrpld binary was still
compiled with -mcmodel=large, inflating it to ~2.1GB with 64-bit
absolute addresses, which collided with ASAN's fixed shadow memory
layout at 0x7fff8000+.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-20 11:08:07 +00:00
Pratik Mankawde
2fef71f235 Merge branch 'develop' into pratik/test-tsan-and-gcc14-asan 2026-03-19 15:22:43 +00:00
Pratik Mankawde
f13d17165a fix: Switch GCC ASAN to -mcmodel=medium and reduce TSAN build parallelism
- Change -mcmodel=large to -mcmodel=medium for GCC ASAN builds. The large
  model inflates code size with 64-bit absolute addresses, pushing the
  binary past the 2GB limit where GCC's pre-compiled CRT startup code
  (crtstuff.c) can't reach data sections with 32-bit relocations.
  Medium model keeps code compact (CRT-compatible) while allowing data
  beyond 2GB.

- Add nproc_subtract=20 for TSAN builds to reduce build parallelism.
  TSAN instrumentation significantly increases per-compilation-unit memory
  usage, causing OOM on CI runners with high parallelism.

- Plumb nproc_subtract from the CI matrix through to the build workflow
  so individual configurations can control their parallelism.

- Temporarily restrict CI matrix to only the two previously-failing
  configs (gcc-13 asan-ubsan, clang-20 tsan-ubsan) to validate fixes
  without burning CI resources on passing configs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-19 15:22:23 +00:00
Pratik Mankawde
f2cc697cb4 fix: Suppress Ledger::setValidated data race in TSAN
Add suppression for pre-existing data race on LedgerHeader::validated
(plain bool mutable) between LedgerMaster worker thread and test code.
The flag is monotonic (false→true only), so the race is benign but
technically UB. Proper fix (atomic<bool>) deferred to a follow-up.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-19 14:03:07 +00:00
Pratik Mankawde
7d975cae2f fix: Add LedgerReplay deadlock suppression and fix GCC ASAN linker error
- Suppress lock-order-inversion in LedgerReplayTask/LedgerDeltaAcquire
  (28 TSAN warnings from pre-existing lock ordering issue).
- Add tools.build:cflags to Conan sanitizer profiles so that C
  dependencies (ed25519) are compiled with -mcmodel=large and sanitizer
  flags, fixing the GOTPCREL relocation overflow on GCC ASAN builds.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-19 14:00:42 +00:00
Pratik Mankawde
2117959900 fix: Resolve CI levelization and rename check failures
- Revert yield_to.h to use inline preprocessor blocks instead of
  including sanitizers.h, avoiding a levelization loop between
  xrpl.basics and xrpl.beast.
- Restructure XRPL_SANITIZER_ACTIVE fallback in sanitizers.h to avoid
  the #ifndef/#define pattern that the rename check misdetects as an
  include guard.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-19 12:59:54 +00:00
Pratik Mankawde
9dd2ca1e4a fix: Address code review findings for TSAN/ASAN PR
- Fix BasicApp::startIOThreads() member corruption: replace
  while(numberOfThreads_--) with a for loop to preserve the member value.
- Fix Coro.ipp non-sanitizer stack size: increase default from 1MB to 2MB
  to match yield_to.h and prevent stack overflows in deep call chains.
- Remove stale "TSAN deactivated" comment and dead activate_tsan variable
  from CI matrix generator.
- Clarify Application.cpp setup() comment to distinguish io_context threads
  from subsystem-specific threads.
- Use explicit load(std::memory_order_relaxed) at all SHAMap::ledgerSeq_
  read sites for consistency with the atomic store.
- Extract sanitizer detection into XRPL_SANITIZER_ACTIVE macro in
  sanitizers.h, replacing duplicated preprocessor blocks in Coro.ipp
  and yield_to.h.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-19 12:51:51 +00:00
Pratik Mankawde
9ec3f091f2 fix: Resolve TSAN-detected data races and suppress lock-order-inversion
- Make Journal::Sink::thresh_ atomic to fix data race between
  concurrent log threshold reads (hot path) and writes (RPC/startup)
- Make SHAMap::ledgerSeq_ atomic to fix data race between concurrent
  node fetch reads and ledger acquisition writes
- Suppress known lock-order-inversion between getMasterMutex() and
  RCLConsensus::mutex_ (requires larger lock-ordering redesign)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-19 12:29:08 +00:00
Pratik Mankawde
d2079574c7 fix: Build Boost.Context with ucontext backend for TSAN builds
TSAN cannot instrument fcontext (assembly) context switches, causing
undefined symbol errors for fiber_activation_record::current(). Add
Conan build options and CMake defines to use the ucontext backend
for TSAN, matching the existing ASAN configuration.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-19 11:18:14 +00:00
Pratik Mankawde
018fe98696 fix: Add missing ByteUtilities.h include and use coroStackSize constant
- Include ByteUtilities.h in Coro.ipp so megabytes() is declared
- Use coroStackSize constant in coroutine constructor instead of
  hardcoded 1536*1024, enabling sanitizer-aware stack sizing
- Initialize local variable in PropertyStream destructor to satisfy
  clang-tidy cppcoreguidelines-init-variables

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-18 16:46:18 +00:00
Pratik Mankawde
cc51d5430b Merge branch 'develop' into pratik/test-tsan-and-gcc14-asan
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-03-18 15:51:58 +00:00
Pratik Mankawde
094c0358d1 Merge remote-tracking branch 'origin/develop' into pratik/test-tsan-and-gcc14-asan 2026-03-10 21:33:44 +00:00
Pratik Mankawde
40535024dc Merge remote-tracking branch 'origin/develop' into pratik/test-tsan-and-gcc14-asan
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-03-10 18:23:44 +00:00
Pratik Mankawde
745219f77d Merge branch 'develop' into pratik/test-tsan-and-gcc14-asan
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-02-20 15:42:18 +00:00
Pratik Mankawde
acc8b1da3e remove parallel test run
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-02-20 14:35:35 +00:00
Pratik Mankawde
008b98edf7 increased timeout and 2 threads for tests
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-02-19 21:26:24 +00:00
Pratik Mankawde
0870d346e1 reduce stack size and remove multi-threaded test execution
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-02-19 17:59:43 +00:00
Pratik Mankawde
5274ce697a only run tsan
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-02-19 17:54:57 +00:00
Pratik Mankawde
9a62a6d429 compilation fix
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-02-19 15:36:03 +00:00
Pratik Mankawde
ad112399bb conditionally change stack size
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-02-19 15:13:07 +00:00
Pratik Mankawde
c14afa1088 lock ordering issue
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-02-19 15:06:27 +00:00
Pratik Mankawde
eef0d84d1f more fixes
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-02-17 16:44:51 +00:00
Pratik Mankawde
4a4a8e40b3 increase timeout
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-02-17 10:45:57 +00:00
Pratik Mankawde
36d1a48435 testing more fixes
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-02-16 19:46:17 +00:00
Pratik Mankawde
e55c4969f3 minor fixes
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-02-16 17:12:57 +00:00
Pratik Mankawde
cb40ed6f40 tring to fix the thread race in BasicApp
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-02-16 16:49:29 +00:00
Pratik Mankawde
e4da8e87a7 Merge branch 'develop' into pratik/test-tsan-and-gcc14-asan 2026-02-16 16:32:47 +00:00
Pratik Mankawde
5b7156e859 added supp commands
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-01-16 14:20:52 +00:00
Pratik Mankawde
261fef7370 activate thread sanitizer
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-01-16 14:18:36 +00:00
Pratik Mankawde
06b999d9e9 Merge branch 'develop' into pratik/test-tsan-and-gcc14-asan
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-01-16 14:18:07 +00:00
Pratik Mankawde
9049100193 testing gcc-14 and tsan
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-01-07 16:52:52 +00:00
Pratik Mankawde
75226af325 renaming and doc update
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-01-07 16:43:59 +00:00
Pratik Mankawde
95a6bbb097 Merge remote-tracking branch 'origin/develop' into pratik/Add-sanitizers-to-CI-builds 2026-01-07 16:06:38 +00:00
Pratik Mankawde
1223bb497c add . in version string
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-01-07 16:03:53 +00:00
Pratik Mankawde
9646537c62 code review changes
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-01-07 15:18:14 +00:00
Pratik Mankawde
8b89ae8c52 Merge remote-tracking branch 'origin/develop' into pratik/Add-sanitizers-to-CI-builds
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2026-01-07 11:51:27 +00:00
Pratik Mankawde
bc3553d53c include guard and build failure issue
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2025-12-19 18:25:34 +00:00
Pratik Mankawde
0ab3e11b21 Merge branch 'pratik/Add-sanitizers-to-CI-builds' of github.com:XRPLF/rippled into pratik/Add-sanitizers-to-CI-builds 2025-12-19 18:00:24 +00:00
Pratik Mankawde
5e41e060a8 Code review changes
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2025-12-19 17:59:12 +00:00
Pratik Mankawde
04417e18d5 Update cmake/XrplCompiler.cmake
Co-authored-by: Ayaz Salikhov <mathbunnyru@users.noreply.github.com>
2025-12-19 17:57:07 +00:00
Pratik Mankawde
738aa9482e code review changes
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2025-12-19 17:31:14 +00:00
Pratik Mankawde
6c1ad00e28 fixed SANITIZERS bug
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2025-12-19 16:21:47 +00:00
Pratik Mankawde
5862ba952d Update cmake/XrplSanitizers.cmake
Co-authored-by: Ayaz Salikhov <mathbunnyru@users.noreply.github.com>
2025-12-19 16:19:12 +00:00
Pratik Mankawde
46d9937ffa code review changes
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2025-12-18 16:58:49 +00:00
Pratik Mankawde
1c10f730a1 moved env. detection to compilationenv.cmake
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2025-12-18 16:07:32 +00:00
Pratik Mankawde
453297442f Merge branch 'develop' into pratik/Add-sanitizers-to-CI-builds 2025-12-18 15:47:19 +00:00
Pratik Mankawde
6479ab29c6 only build sanitizers with clang-20
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2025-12-18 15:46:32 +00:00
Pratik Mankawde
09f75f3026 removed pic and pie setup.
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2025-12-18 15:43:51 +00:00
Pratik Mankawde
755833508c minor comments update
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2025-12-16 12:45:11 +00:00
Pratik Mankawde
e4e3900876 suppress issue in invariants_test
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2025-12-16 12:36:41 +00:00
Pratik Mankawde
870e079a8e Merge remote-tracking branch 'origin/develop' into pratik/Add-sanitizers-to-CI-builds 2025-12-16 12:00:58 +00:00
Pratik Mankawde
32f7baca44 code review changes
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2025-12-16 11:57:04 +00:00
Pratik Mankawde
93b0554a36 Apply suggestions from code review
Co-authored-by: Ayaz Salikhov <mathbunnyru@users.noreply.github.com>
2025-12-16 11:14:39 +00:00
Pratik Mankawde
e0f9de0ea0 Update .github/actions/build-deps/action.yml
update description

Co-authored-by: Ayaz Salikhov <mathbunnyru@users.noreply.github.com>
2025-12-16 11:12:08 +00:00
Pratik Mankawde
3ea3e41b18 updated lockfile
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2025-12-15 17:56:07 +00:00
Pratik Mankawde
531b0d051a Merge branch 'develop' into pratik/Add-sanitizers-to-CI-builds 2025-12-15 17:35:47 +00:00
Pratik Mankawde
3cd4e5cb5a code review comments
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2025-12-15 17:34:17 +00:00
Pratik Mankawde
c8e3ba5d90 updated conan.lock file
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2025-12-12 16:32:33 +00:00
Pratik Mankawde
49c037493d updated docs
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2025-12-12 15:23:58 +00:00
Pratik Mankawde
a4f112d479 Moved to entirely cmake workflow
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2025-12-12 15:23:35 +00:00
Pratik Mankawde
8d7398419a reverted abseil to original version 20250127
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2025-12-12 14:22:16 +00:00
Pratik Mankawde
06e31f417c reverting to abseil/20240722.0
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2025-12-12 12:40:35 +00:00
Pratik Mankawde
4d64e83992 Merge branch 'develop' into pratik/Add-sanitizers-to-CI-builds
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2025-12-11 14:45:24 +00:00
Pratik Mankawde
3d6dd17325 Merge remote-tracking branch 'origin/develop' into pratik/Add-sanitizers-to-CI-builds 2025-12-10 19:06:08 +00:00
Pratik Mankawde
405a0f4309 updating abseil version 2025-12-10 14:05:46 +00:00
Pratik Mankawde
5e89e2dfe7 removed no-pic
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2025-12-09 17:50:01 +00:00
Pratik Mankawde
63ba065440 cflags
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2025-12-09 17:40:24 +00:00
Pratik Mankawde
b1f924bfeb testing with profile_dir
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2025-12-09 17:32:25 +00:00
Pratik Mankawde
033986728c removed cmake prefix
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2025-12-09 17:21:44 +00:00
Pratik Mankawde
2b1479fa99 trying ABSL_ENABLE_CONSTANT_INIT_V2 flag
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2025-12-09 17:11:55 +00:00
Pratik Mankawde
9ec5ef96f8 trying ABSL_ENABLE_CONSTANT_INIT_V2 2025-12-09 16:11:30 +00:00
Pratik Mankawde
b28489934d using DABSL_ENABLE_CONSTANT_INIT_V2 flag
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2025-12-09 14:24:37 +00:00
Pratik Mankawde
3aeb9b3b86 try to silent gcc error on Abseil
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2025-12-09 14:02:19 +00:00
Pratik Mankawde
f827e1b2a7 Merge remote-tracking branch 'origin/develop' into pratik/Add-sanitizers-to-CI-builds 2025-12-09 13:49:25 +00:00
Pratik Mankawde
ca9fd5e319 removed -g 2025-12-05 15:32:12 +00:00
Pratik Mankawde
5bb30cff15 Merge remote-tracking branch 'origin/develop' into pratik/Add-sanitizers-to-CI-builds 2025-12-05 13:23:02 +00:00
Pratik Mankawde
8b761b9ce6 minor change 2025-12-04 18:38:29 +00:00
Pratik Mankawde
1f8fbaedca asan errors, may not be related but testing.
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
2025-12-04 15:36:39 +00:00
Pratik Mankawde
1cdd43f259 Merge branch 'develop' into pratik/Add-sanitizers-to-CI-builds 2025-12-04 14:57:54 +00:00
Pratik Mankawde
982d26610d Merge remote-tracking branch 'origin/develop' into pratik/Add-sanitizers-to-CI-builds 2025-12-04 14:25:39 +00:00
Pratik Mankawde
1f2f05d320 code review changes 2025-12-03 17:40:42 +00:00
Pratik Mankawde
7f09fbe807 removed extra signed-integer-overflow
Signed-off-by: Pratik Mankawde <pratikmankawde@gmail.com>
2025-11-28 17:34:39 +00:00
Pratik Mankawde
734f8a9810 minor cleanup
Signed-off-by: Pratik Mankawde <pratikmankawde@gmail.com>
2025-11-28 16:51:39 +00:00
Pratik Mankawde
935e45bcca Merge branch 'develop' into pratik/Add-sanitizers-to-CI-builds 2025-11-28 16:36:25 +00:00
Pratik Mankawde
0b612b5c13 changed job name
Signed-off-by: Pratik Mankawde <pratikmankawde@gmail.com>
2025-11-28 16:20:19 +00:00
Pratik Mankawde
7617072ebc sanitizer variable now has separate address,undefined and thread values
Signed-off-by: Pratik Mankawde <pratikmankawde@gmail.com>
2025-11-28 16:01:01 +00:00
Pratik Mankawde
4a61ff3bef minor change
Signed-off-by: Pratik Mankawde <pratikmankawde@gmail.com>
2025-11-28 14:45:56 +00:00
Pratik Mankawde
7097566d9d code review comments
Signed-off-by: Pratik Mankawde <pratikmankawde@gmail.com>
2025-11-28 13:15:34 +00:00
Pratik Mankawde
afedb276db quote issue again
Signed-off-by: Pratik Mankawde <pratikmankawde@gmail.com>
2025-11-27 15:59:04 +00:00
Pratik Mankawde
941775fec4 fix path issue
Signed-off-by: Pratik Mankawde <pratikmankawde@gmail.com>
2025-11-27 15:28:10 +00:00
Pratik Mankawde
4058ca01ac minor cleanup
Signed-off-by: Pratik Mankawde <pratikmankawde@gmail.com>
2025-11-27 13:47:50 +00:00
Pratik Mankawde
9c5df200b8 cleanup
Signed-off-by: Pratik Mankawde <pratikmankawde@gmail.com>
2025-11-27 13:46:11 +00:00
Pratik Mankawde
96e8ee4bdd added an extra line
Signed-off-by: Pratik Mankawde <pratikmankawde@gmail.com>
2025-11-27 13:33:28 +00:00
Pratik Mankawde
c182d13ab5 Merge branch 'develop' into pratik/Add-sanitizers-to-CI-builds
Signed-off-by: Pratik Mankawde <pratikmankawde@gmail.com>
2025-11-27 13:21:51 +00:00
Pratik Mankawde
d942fd9f9c updated docs
Signed-off-by: Pratik Mankawde <pratikmankawde@gmail.com>
2025-11-26 15:19:13 +00:00
Pratik Mankawde
064d51b79c more code changes
Signed-off-by: Pratik Mankawde <pratikmankawde@gmail.com>
2025-11-26 15:13:12 +00:00
Pratik Mankawde
6202148623 Merge branch 'develop' into pratik/Add-sanitizers-to-CI-builds 2025-11-26 14:07:07 +00:00
Pratik Mankawde
954fe68e91 added more info to package id configs
Signed-off-by: Pratik Mankawde <pratikmankawde@gmail.com>
2025-11-26 14:03:58 +00:00
Pratik Mankawde
1d38504d85 code review fixes 1
Signed-off-by: Pratik Mankawde <pratikmankawde@gmail.com>
2025-11-26 13:57:44 +00:00
Pratik Mankawde
9cdabe98e2 link to sanitizer_readme.md
Signed-off-by: Pratik Mankawde <pratikmankawde@gmail.com>
2025-11-21 18:10:18 +00:00
Pratik Mankawde
49f8daf82f updated link
Signed-off-by: Pratik Mankawde <pratikmankawde@gmail.com>
2025-11-21 18:09:14 +00:00
Pratik Mankawde
dd9e554d4a added readme.
Signed-off-by: Pratik Mankawde <pratikmankawde@gmail.com>
2025-11-21 18:06:01 +00:00
Pratik Mankawde
facb6c63e3 cleanup and additions to asan, ubsan suppressions
Signed-off-by: Pratik Mankawde <pratikmankawde@gmail.com>
2025-11-21 16:24:38 +00:00
Pratik Mankawde
32d129ee5e Merge branch 'develop' into pratik/Add-sanitizers-to-CI-builds 2025-11-21 12:18:31 +00:00
Pratik Mankawde
6e32392f93 added no-PIC
Signed-off-by: Pratik Mankawde <pratikmankawde@gmail.com>
2025-11-21 12:17:53 +00:00
Pratik Mankawde
39ace64833 updated blacklist file
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-19 18:40:11 +00:00
Pratik Mankawde
4430cb64b8 trying a diff. sanitizer-blacklist syntax
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-19 18:27:26 +00:00
Pratik Mankawde
d4b033958e removed static linking.
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-19 18:05:39 +00:00
Pratik Mankawde
1a08b97fb6 Merge branch 'develop' into pratik/Add-sanitizers-to-CI-builds 2025-11-19 17:51:07 +00:00
Pratik Mankawde
6407788b0f added halt on error=0 and -fno-pie
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-19 17:44:39 +00:00
Pratik Mankawde
702d94134e minor correction
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-19 17:22:06 +00:00
Pratik Mankawde
7d3fc5c396 added static linking flags
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-19 17:13:31 +00:00
Pratik Mankawde
810f5e94db statically link sanitizers
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-19 17:08:33 +00:00
Pratik Mankawde
d026d0640f build conan deps with sanitizers to fix tsan crashes
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-19 15:34:24 +00:00
Pratik Mankawde
5befd5d249 added tsan option second_deadlock_stack=1 and more tsan supps
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-19 13:23:48 +00:00
Pratik Mankawde
4f496db291 few more additions
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-19 12:20:48 +00:00
Pratik Mankawde
10553fc437 added few more suppressions in tsan.supp
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-19 11:48:24 +00:00
Pratik Mankawde
087e52ea36 suppress crashes in tsan intrumentation code
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-19 11:02:55 +00:00
Pratik Mankawde
cb982e7a8a added pie flag and signal suppressions
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-19 10:42:23 +00:00
Pratik Mankawde
2610e653d6 only for clang
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-18 17:58:59 +00:00
Pratik Mankawde
b6c313bb94 only use ignorelist in gcc
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-18 17:41:21 +00:00
Pratik Mankawde
a3e42a0d11 alternate approach
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-18 17:19:14 +00:00
Pratik Mankawde
92a6c986b3 try blacklist file
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-18 17:10:29 +00:00
Pratik Mankawde
dd0408ac53 added blacklist file
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-18 17:00:38 +00:00
Pratik Mankawde
0fcfcd059e more aggressive suppresions
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-18 16:37:10 +00:00
Pratik Mankawde
4c0f7a337a few more additions
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-18 15:59:44 +00:00
Pratik Mankawde
640f9ff5e2 Added more files to ubsan.supp
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-18 15:18:08 +00:00
Pratik Mankawde
42cab6c826 updated ubsan file
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-18 15:02:21 +00:00
Pratik Mankawde
9267756944 fixed generate.py file " issue
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-18 14:14:49 +00:00
Pratik Mankawde
3b1e82b412 revert back to clang-20
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-18 13:45:18 +00:00
Pratik Mankawde
7d7c659822 target clang-21
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-18 13:42:59 +00:00
Pratik Mankawde
a58df41a4d Merge branch 'develop' into pratik/Add-sanitizers-to-CI-builds
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-18 13:42:12 +00:00
Pratik Mankawde
d01b1da80e cleaned up linker flags
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-18 11:55:24 +00:00
Pratik Mankawde
216ecf67e1 updated tsan and ubsan
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-17 19:18:15 +00:00
Pratik Mankawde
d8c8900fbb updated supp files
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-17 18:43:39 +00:00
Pratik Mankawde
e2c75f5d60 Merge branch 'develop' into pratik/Add-sanitizers-to-CI-builds 2025-11-17 17:59:34 +00:00
Pratik Mankawde
2428c5c196 added more suppressions
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-17 17:58:08 +00:00
Pratik Mankawde
d8ff72b342 updated asan and lsan
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-17 17:11:36 +00:00
Pratik Mankawde
4fc5c00c24 added lsan and updated tsan
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-17 16:53:51 +00:00
Pratik Mankawde
5a9014912c trying without begin-regex
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-17 12:12:27 +00:00
Pratik Mankawde
8006a1e967 removed leak type
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-16 12:30:23 +00:00
Pratik Mankawde
6610f469df trying diff. asan supp options
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-16 11:23:38 +00:00
Pratik Mankawde
9f026929cf adding asio to tsan.supp
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-14 19:43:19 +00:00
Pratik Mankawde
a9d134af85 more chnages
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-14 19:13:26 +00:00
Pratik Mankawde
18b9b9da19 updated supp files
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-14 18:50:08 +00:00
Pratik Mankawde
6a89a544fe tsan race suppression
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-14 18:10:18 +00:00
Pratik Mankawde
427771775e added afew suppressors
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-14 17:43:15 +00:00
Pratik Mankawde
67946f4993 more cleanup
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-14 16:51:54 +00:00
Pratik Mankawde
8d8a0cb970 Merge branch 'develop' into pratik/Add-sanitizers-to-CI-builds 2025-11-14 16:25:11 +00:00
Pratik Mankawde
89f9ede1f5 cleanup
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-14 16:24:04 +00:00
Pratik Mankawde
aac1076d83 minor cleanup
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-14 13:01:30 +00:00
Pratik Mankawde
c0da02814b Merge branch 'develop' into pratik/Add-sanitizers-to-CI-builds 2025-11-14 12:53:30 +00:00
Pratik Mankawde
013ff18fce cleanup
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-14 12:51:47 +00:00
Pratik Mankawde
e1403d56ef skip mold, gold, lld
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-14 12:20:07 +00:00
Pratik Mankawde
58b248d9a4 pass lld flag to cmake
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-14 11:23:59 +00:00
Pratik Mankawde
f1561c5b48 use lld
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-14 10:57:11 +00:00
Pratik Mankawde
73ab466029 use gold+large
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-14 10:25:39 +00:00
Pratik Mankawde
4cf4802971 for tsan only use medium
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-13 16:54:41 +00:00
Pratik Mankawde
8d6d2ec455 trying with bfd
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-13 16:31:41 +00:00
Pratik Mankawde
4248a1fbb6 only set mcmodel=large for gcc
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-13 16:18:34 +00:00
Pratik Mankawde
7edba0a856 trying with mcmodel=large
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-13 16:01:36 +00:00
Pratik Mankawde
b38ef53c44 trying lld.
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-13 15:03:23 +00:00
Pratik Mankawde
baee65bd28 fixing minor spacing issue
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-13 13:01:32 +00:00
Pratik Mankawde
2e53c5ab80 changed linker for gcc
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-13 12:28:48 +00:00
Pratik Mankawde
09a6e46c39 pass sanitizer flags to linker as well
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-12 17:32:39 +00:00
Pratik Mankawde
a93052e765 added -fPIC flag
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-12 16:58:01 +00:00
Pratik Mankawde
1cc00cddd8 trying with " and dyn linking
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-12 16:19:34 +00:00
Pratik Mankawde
5f2a351e3f reverted mcmodel to medium
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-12 16:08:36 +00:00
Pratik Mankawde
5e89bce8d9 use mcmodel=large
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-12 14:48:10 +00:00
Pratik Mankawde
9b793cd429 linker flag correction
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-12 14:16:58 +00:00
Pratik Mankawde
b650852fa7 static liking for gcc
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-12 14:09:12 +00:00
Pratik Mankawde
1d76de83f6 minor correction
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-12 13:12:01 +00:00
Pratik Mankawde
ea4d062e68 added ubsan file
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-12 12:15:45 +00:00
Pratik Mankawde
dcd0553050 sanitizer builds only for gcc15 and clang20. no linking check for sanitizer builds
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-12 11:46:20 +00:00
Pratik Mankawde
6355eba88e Merge branch 'develop' into pratik/Add-sanitizers-to-CI-builds 2025-11-12 10:37:48 +00:00
Pratik Mankawde
2e5afa7556 updated asan suppressions file
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-12 10:36:02 +00:00
Pratik Mankawde
7d05090d68 setting variables in run
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-11 19:21:33 +00:00
Pratik Mankawde
916f00039c try 8
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-11 19:15:50 +00:00
Pratik Mankawde
e687ab8653 try 7
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-11 19:07:22 +00:00
Pratik Mankawde
b182b6abf7 try 6
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-11 19:02:08 +00:00
Pratik Mankawde
509d388f97 try 6
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-11 18:58:21 +00:00
Pratik Mankawde
fd712770e1 try 5
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-11 18:53:42 +00:00
Pratik Mankawde
695f4a2cc9 try 4
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-11 18:50:04 +00:00
Pratik Mankawde
5ea3b4327d try 3
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-11 18:46:13 +00:00
Pratik Mankawde
a9444d3a42 try 2
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-11 18:42:14 +00:00
Pratik Mankawde
20326a785b putting commented code back
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-11 18:39:00 +00:00
Pratik Mankawde
5ef2ced584 testing by printing asan_options
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-11 18:38:21 +00:00
Pratik Mankawde
a011d29fcb trying again with minor change
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-11 18:36:51 +00:00
Pratik Mankawde
9abec17536 commenting out windows and mac for now.
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-11 18:32:58 +00:00
Pratik Mankawde
2bc089a962 Merge branch 'develop' into pratik/Add-sanitizers-to-CI-builds
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-11 18:30:47 +00:00
Pratik Mankawde
14f605b2f9 minor
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-11 18:29:52 +00:00
Pratik Mankawde
f2365543fb added variable in an early action
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-11 18:19:04 +00:00
Pratik Mankawde
d9c26bd7a9 another comb.
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-11 18:17:37 +00:00
Pratik Mankawde
937dc8740e another combi
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-11 17:53:16 +00:00
Pratik Mankawde
9b92aafe6d fixed path to atsan
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-11 17:42:23 +00:00
Pratik Mankawde
124b6ca4bd using GITHUB_WORKSPACE now
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-11 17:27:51 +00:00
Pratik Mankawde
28885a4638 path revert
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-11 17:13:45 +00:00
Pratik Mankawde
5478e4fee6 move working_dir up
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-11 16:57:32 +00:00
Pratik Mankawde
72b3a03538 Merge branch 'develop' into pratik/Add-sanitizers-to-CI-builds 2025-11-11 16:51:12 +00:00
Pratik Mankawde
18b65fd129 linked model flags specific to amd64
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-11 16:42:57 +00:00
Pratik Mankawde
ebf917ab15 no-var-tracking is gcc specific
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-11 15:33:17 +00:00
Pratik Mankawde
2178fb919f Merge branch 'develop' into pratik/Add-sanitizers-to-CI-builds
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-11 15:08:00 +00:00
Pratik Mankawde
e156ed40ba minor formatting
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-11 15:06:47 +00:00
Pratik Mankawde
db5aa2d277 added no-var-tracking
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-11 15:03:20 +00:00
Pratik Mankawde
e347da9fac added mcmodel flags for linker
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-11 14:56:07 +00:00
Pratik Mankawde
6b8d5b57e1 updated suppression file paths
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-11 14:36:27 +00:00
Pratik Mankawde
0029210926 minor fix
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-11 14:16:39 +00:00
Pratik Mankawde
832c32d15c formatting changes
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-11 13:59:08 +00:00
Pratik Mankawde
39adc5a82b fixes forgcc compilation errors.
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-11 13:57:31 +00:00
Pratik Mankawde
1f88697f54 trying suppression files
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-11 12:42:36 +00:00
Pratik Mankawde
4a0fc0f686 formatting changes
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-11 11:22:51 +00:00
Pratik Mankawde
f5b473ccd1 Merge branch 'develop' into pratik/Add-sanitizers-to-CI-builds
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-11 11:21:25 +00:00
Pratik Mankawde
71212677c0 don't build deps with sanitizers
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-11 11:02:16 +00:00
Pratik Mankawde
abfaac5c64 trying direct input placement
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-10 17:16:55 +00:00
Pratik Mankawde
cb5a76589e tryring escaped " in config file
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-10 17:02:50 +00:00
Pratik Mankawde
d077141d75 name to small case
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-10 16:46:41 +00:00
Pratik Mankawde
1eb3cc4ec7 fixed pre-commit check
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-10 16:44:37 +00:00
Pratik Mankawde
34b10b87e6 merged develop
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-10 16:13:08 +00:00
Pratik Mankawde
c0ecb9a0cd Merge branch 'develop' into pratik/Add-sanitizers-to-CI-builds
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-10 16:02:33 +00:00
Pratik Mankawde
c15d6399fb minor revert in clang-format
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-10 15:58:17 +00:00
Pratik Mankawde
0b1bd42cc0 Add sanitizers to CI builds
- Added Address and Thread sanitizers for Debian Bookworm builds
- Updated build-deps action to support sanitizer flags
- Modified strategy matrix generation to include sanitizer configurations
- Updated Conan profiles for sanitizer support
- Added InsertNewlineAtEOF setting to clang-format
2025-11-10 15:54:44 +00:00
Pratik Mankawde
30f7ef7676 Merge branch 'develop' into pratik/Add-sanitizers-to-CI-builds 2025-11-06 10:11:55 +00:00
Pratik Mankawde
e7da05e44c added setting EOF-ExtraLine in clang-format
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-06 10:10:18 +00:00
Pratik Mankawde
1cdd0bf63e end of line
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-06 10:03:17 +00:00
Pratik Mankawde
48a3e5ea31 special case only for gcc
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-06 10:00:46 +00:00
Pratik Mankawde
10a422dd69 seperate clang and gcc
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-05 15:31:09 +00:00
Pratik Mankawde
b150feaab6 fixes sanitizer arg not available in build-test
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-05 15:14:29 +00:00
Pratik Mankawde
ada523071f passing sanitizers to on-pr and on-trigger
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-05 14:30:14 +00:00
Pratik Mankawde
ff0284e984 removed sanitizers from script
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-05 13:33:40 +00:00
Pratik Mankawde
0be98ac610 made sanitizers optional
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-05 13:29:58 +00:00
Pratik Mankawde
83eb93f5d7 added back type
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-05 13:22:31 +00:00
Pratik Mankawde
db74cebc8b added build with sanitizers ins.
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-05 12:49:08 +00:00
Pratik Mankawde
3b32210ef4 formatting changes
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-05 12:26:33 +00:00
Pratik Mankawde
21b9f6d1af Merge branch 'develop' into pratik/Add-sanitizers-to-CI-builds 2025-11-05 12:25:47 +00:00
Pratik Mankawde
977a087bb3 code review changes
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-05 12:25:29 +00:00
Pratik Mankawde
e95299dac5 code review changes
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-05 09:46:56 +00:00
Pratik Mankawde
7ae3a85f21 formatting correction
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-04 16:12:22 +00:00
Pratik Mankawde
c38b5aa2d4 Added flow for the sanitizers build
Signed-off-by: Pratik Mankawde <pmankawde@ripple.com>
2025-11-04 15:59:15 +00:00
24 changed files with 369 additions and 167 deletions

View File

@@ -235,16 +235,14 @@ def generate_strategy_matrix(all: bool, config: Config) -> list:
# Add the configuration to the list, with the most unique fields first,
# so that they are easier to identify in the GitHub Actions UI, as long
# names get truncated.
# Add Address and UB sanitizers as separate configurations for specific
# bookworm distros. Thread sanitizer is currently disabled (see below).
# GCC-Asan xrpld-embedded tests are failing because of https://github.com/google/sanitizers/issues/856
if os[
"distro_version"
] == "bookworm" and f"{os['compiler_name']}-{os['compiler_version']}" in [
"gcc-15",
"clang-20",
]:
# Add ASAN configuration.
# Add sanitizer jobs for specific bookworm distros.
# GCC sanitizers run independently because combining ASAN+UBSAN inflates
# GCC data sections past ASAN's 2GB shadow memory limit
# (see https://github.com/google/sanitizers/issues/856).
# Clang doesn't have this issue, so TSAN+UBSAN stay combined but ASAN and UBSAN are separated.
compiler_id = f"{os['compiler_name']}-{os['compiler_version']}"
if os["distro_version"] == "bookworm" and compiler_id == "gcc-15":
# Add separate ASAN, TSAN, and UBSAN configurations for GCC.
configurations.append(
{
"config_name": config_name + "-asan",
@@ -257,7 +255,21 @@ def generate_strategy_matrix(all: bool, config: Config) -> list:
"sanitizers": "address",
}
)
# Add UBSAN configuration.
# TSAN instrumentation significantly increases memory usage during
# compilation, so reduce build parallelism to avoid OOM on CI runners.
configurations.append(
{
"config_name": config_name + "-tsan",
"cmake_args": cmake_args,
"cmake_target": cmake_target,
"build_only": build_only,
"build_type": build_type,
"os": os,
"architecture": architecture,
"sanitizers": "thread",
"nproc_subtract": 20,
}
)
configurations.append(
{
"config_name": config_name + "-ubsan",
@@ -270,21 +282,47 @@ def generate_strategy_matrix(all: bool, config: Config) -> list:
"sanitizers": "undefinedbehavior",
}
)
# TSAN is deactivated due to seg faults with latest compilers.
activate_tsan = False
if activate_tsan:
configurations.append(
{
"config_name": config_name + "-tsan-ubsan",
"cmake_args": cmake_args,
"cmake_target": cmake_target,
"build_only": build_only,
"build_type": build_type,
"os": os,
"architecture": architecture,
"sanitizers": "thread,undefinedbehavior",
}
)
elif os["distro_version"] == "bookworm" and compiler_id == "clang-20":
# Add combined ASAN+UBSAN and TSAN+UBSAN configurations for Clang.
configurations.append(
{
"config_name": config_name + "-asan",
"cmake_args": cmake_args,
"cmake_target": cmake_target,
"build_only": build_only,
"build_type": build_type,
"os": os,
"architecture": architecture,
"sanitizers": "address",
}
)
configurations.append(
{
"config_name": config_name + "-ubsan",
"cmake_args": cmake_args,
"cmake_target": cmake_target,
"build_only": build_only,
"build_type": build_type,
"os": os,
"architecture": architecture,
"sanitizers": "undefinedbehavior",
}
)
# TSAN instrumentation significantly increases memory usage during
# compilation, so reduce build parallelism to avoid OOM on CI runners.
configurations.append(
{
"config_name": config_name + "-tsan-ubsan",
"cmake_args": cmake_args,
"cmake_target": cmake_target,
"build_only": build_only,
"build_type": build_type,
"os": os,
"architecture": architecture,
"sanitizers": "thread,undefinedbehavior",
"nproc_subtract": 20,
}
)
else:
configurations.append(
{

View File

@@ -58,5 +58,6 @@ jobs:
image: ${{ contains(matrix.architecture.platform, 'linux') && format('ghcr.io/xrplf/ci/{0}-{1}:{2}-{3}-sha-{4}', matrix.os.distro_name, matrix.os.distro_version, matrix.os.compiler_name, matrix.os.compiler_version, matrix.os.image_sha) || '' }}
config_name: ${{ matrix.config_name }}
sanitizers: ${{ matrix.sanitizers }}
nproc_subtract: ${{ matrix.nproc_subtract && matrix.nproc_subtract || 2 }}
secrets:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}

View File

@@ -61,3 +61,14 @@ if(enable_asan)
INTERFACE BOOST_USE_ASAN BOOST_USE_UCONTEXT
)
endif()
# TSAN also requires the ucontext backend so that Boost.Context uses
# POSIX ucontext (not fcontext assembly) for fiber switching. This
# allows TSAN to properly track context switches and avoids false positives.
# These defines must match what Boost was compiled with (see conan/profiles/sanitizers).
if(enable_tsan)
target_compile_definitions(
xrpl_boost
INTERFACE BOOST_USE_TSAN BOOST_USE_UCONTEXT
)
endif()

View File

@@ -103,6 +103,7 @@ words:
- dxrpl
- enabled
- endmacro
- eventfd
- exceptioned
- Falco
- fcontext

View File

@@ -1,13 +1,31 @@
#pragma once
// Helper to disable ASan/HwASan for specific functions
/*
ASAN flags some false positives with sudden jumps in control flow, like
exceptions, or when encountering coroutine stack switches. This macro can be used to disable ASAN
intrumentation for specific functions.
*/
// Helper to disable ASan/HwASan for specific functions.
// ASAN flags some false positives with sudden jumps in control flow, like
// exceptions, or when encountering coroutine stack switches. This macro can
// be used to disable ASAN instrumentation for specific functions.
#if defined(__GNUC__) || defined(__clang__)
#define XRPL_NO_SANITIZE_ADDRESS __attribute__((no_sanitize("address", "hwaddress")))
#else
#define XRPL_NO_SANITIZE_ADDRESS
#endif
/** Detect whether a memory sanitizer (TSAN or ASAN) is active at compile time.
*
* Evaluates to 1 when the translation unit is compiled with ThreadSanitizer
* or AddressSanitizer instrumentation, 0 otherwise.
*
* GCC defines __SANITIZE_THREAD__ / __SANITIZE_ADDRESS__ directly.
* Clang uses __has_feature(thread_sanitizer) / __has_feature(address_sanitizer).
*/
#if defined(__SANITIZE_THREAD__) || defined(__SANITIZE_ADDRESS__)
#define XRPL_SANITIZER_ACTIVE 1
#elif defined(__has_feature)
#if __has_feature(thread_sanitizer) || __has_feature(address_sanitizer)
#define XRPL_SANITIZER_ACTIVE 1
#else
#define XRPL_SANITIZER_ACTIVE 0
#endif
#else
#define XRPL_SANITIZER_ACTIVE 0
#endif

View File

@@ -307,7 +307,7 @@ public:
reference
front() noexcept
{
return element_from(head_.next_);
return elementFrom(head_.next_);
}
/** Obtain a const reference to the first element.

View File

@@ -11,12 +11,34 @@
#include <boost/thread/csbl/memory/allocator_arg.hpp>
#include <condition_variable>
#include <cstddef>
#include <mutex>
#include <thread>
#include <vector>
namespace beast::test {
/** Stack size for yield_to coroutines.
*
* Sanitizers significantly increase stack frame sizes
* (TSAN ~3-5x, ASAN ~2-3x), requiring larger coroutine stacks.
*
* @note This duplicates the detection logic from xrpl/basics/sanitizers.h
* because xrpl.beast cannot depend on xrpl.basics (levelization
* constraint).
*/
#if defined(__SANITIZE_THREAD__) || defined(__SANITIZE_ADDRESS__)
inline constexpr std::size_t yieldStackSize = 2 * 1024 * 1024;
#elif defined(__has_feature)
#if __has_feature(thread_sanitizer) || __has_feature(address_sanitizer)
inline constexpr std::size_t yieldStackSize = 2 * 1024 * 1024;
#else
inline constexpr std::size_t yieldStackSize = 1.5 * 1024 * 1024;
#endif
#else
inline constexpr std::size_t yieldStackSize = 1.5 * 1024 * 1024;
#endif
/** Mix-in to support tests using asio coroutines.
Derive from this class and use yield_to to launch test
@@ -111,7 +133,7 @@ EnableYieldTo::spawn(F0&& f, FN&&... fn)
boost::asio::spawn(
ios_,
boost::allocator_arg,
boost::context::fixedsize_stack(2 * 1024 * 1024),
boost::context::fixedsize_stack(yieldStackSize),
[&](yield_context yield) {
f(yield);
std::scoped_lock const lock{m_};

View File

@@ -2,6 +2,7 @@
#include <xrpl/beast/utility/instrumentation.h>
#include <atomic>
#include <sstream>
namespace beast {
@@ -57,7 +58,7 @@ public:
class Sink
{
protected:
explicit Sink(Sink const& sink) = default;
explicit Sink(Sink const& sink);
Sink(Severity thresh, bool console);
public:
@@ -107,7 +108,9 @@ public:
writeAlways(Severity level, std::string const& text) = 0;
private:
Severity thresh_;
/// Minimum severity level. Atomic for safe concurrent reads/writes
/// (e.g. RPC threshold changes vs. hot-path log checks).
std::atomic<Severity> thresh_;
bool console_;
};

View File

@@ -1,12 +1,20 @@
#pragma once
#include <xrpl/basics/ByteUtilities.h>
#include <xrpl/basics/sanitizers.h>
#include <cstddef>
#include <utility>
namespace xrpl {
/// Coroutine stack size (1.5 MB). Increased from 1 MB because
/// ASAN-instrumented deep call stacks exceeded the original limit.
constexpr std::size_t kCORO_STACK_SIZE = 1536 * 1024;
/** Stack size for JobQueue coroutines.
*
* Sanitizers significantly increase stack frame sizes
* (TSAN ~3-5x, ASAN ~2-3x), requiring larger coroutine stacks.
*/
inline constexpr std::size_t kCORO_STACK_SIZE =
XRPL_SANITIZER_ACTIVE ? megabytes(2) : megabytes(1.5);
template <class F>
JobQueue::Coro::Coro(CoroCreateT, JobQueue& jq, JobType type, std::string name, F&& f)

View File

@@ -32,13 +32,28 @@ public:
/** Construct the node store.
Construction configures the database but does not start read threads.
Call startReadThreads() after construction to begin asynchronous reads.
@param scheduler The scheduler to use for performing asynchronous tasks.
@param readThreads The number of asynchronous read threads to create.
@param config The configuration settings
@param readThreads The desired number of asynchronous read threads.
@param config The configuration settings.
@param journal Destination for logging output.
*/
Database(Scheduler& scheduler, int readThreads, Section const& config, beast::Journal j);
/** Start the asynchronous read threads.
Must be called after construction to start read threads. It is safe
to destroy the Database without calling this; in that case no
asynchronous reads will be serviced.
@note Not thread-safe. Must be called exactly once, before any
concurrent access to the database.
*/
void
startReadThreads();
/** Destroy the node store.
All pending operations are completed, pending writes flushed,
and files closed before this returns.
@@ -251,6 +266,9 @@ private:
std::atomic<int> readThreads_ = 0;
std::atomic<int> runningThreads_ = 0;
/// The number of read threads to create when startReadThreads() is called.
int const desiredReadThreads_;
virtual std::shared_ptr<NodeObject>
fetchNodeObject(
uint256 const& hash,

View File

@@ -21,6 +21,15 @@ protected:
public:
~Manager() override = 0;
/** Start the manager's background thread.
Must be called after construction to begin periodic charge decay
and inactive-consumer sweeps.
@note Not thread-safe. Must be called exactly once, before any
concurrent access to the manager.
*/
virtual void
start() = 0;
/** Create a new endpoint keyed by inbound IP address or the forwarded
* IP if proxied. */
virtual Consumer

View File

@@ -14,6 +14,7 @@
#include <xrpl/shamap/SHAMapMissingNode.h>
#include <xrpl/shamap/SHAMapTreeNode.h>
#include <atomic>
#include <set>
#include <stack>
#include <vector>
@@ -82,8 +83,11 @@ private:
/** ID to distinguish this map for all others we're sharing nodes with. */
std::uint32_t cowid_ = 1;
/** The sequence of the ledger that this map references, if any. */
std::uint32_t ledgerSeq_ = 0;
/** The sequence of the ledger that this map references, if any.
* Atomic because it is written during ledger acquisition and read
* concurrently by node fetch and sync operations.
*/
std::atomic<std::uint32_t> ledgerSeq_ = 0;
intr_ptr::SharedPtr<SHAMapTreeNode> root_;
mutable SHAMapState state_;
@@ -537,7 +541,7 @@ SHAMap::setFull()
inline void
SHAMap::setLedgerSeq(std::uint32_t lseq)
{
ledgerSeq_ = lseq;
ledgerSeq_.store(lseq, std::memory_order_relaxed);
}
inline void

View File

@@ -1,102 +1,64 @@
# The idea is to empty this file gradually by fixing the underlying issues and removing suppresions.
# TSAN suppression file for rippled.
# Only suppress issues in third-party libraries and TSAN's own instrumentation.
# Races in rippled's own code should be fixed, not suppressed.
# Suppress race in Boost ASIO scheduler detected by GCC-15
# This is a false positive in Boost's internal pipe() synchronization
# Boost ASIO / Boost Context false positives
# These are internal to Boost's reactor, fiber context switching, and memory management.
race:boost/asio/
race:boost/context/
race:boost/asio/executor.hpp
race:boost::asio
# Suppress tsan related issues in xrpld code.
race:src/libxrpl/basics/make_SSLContext.cpp
race:src/libxrpl/basics/Number.cpp
race:src/libxrpl/json/json_value.cpp
race:src/libxrpl/json/to_string.cpp
race:src/libxrpl/ledger/OpenView.cpp
race:src/libxrpl/net/HTTPClient.cpp
race:src/libxrpl/nodestore/backend/NuDBFactory.cpp
race:src/libxrpl/protocol/InnerObjectFormats.cpp
race:src/libxrpl/protocol/STParsedJSON.cpp
race:src/libxrpl/resource/ResourceManager.cpp
race:src/test/app/Flow_test.cpp
race:src/test/app/LedgerReplay_test.cpp
race:src/test/app/NFToken_test.cpp
race:src/test/app/Offer_test.cpp
race:src/test/app/ValidatorSite_test.cpp
race:src/test/consensus/NegativeUNL_test.cpp
race:src/test/jtx/impl/Env.cpp
race:src/test/jtx/impl/JSONRPCClient.cpp
race:src/test/jtx/impl/pay.cpp
race:src/test/jtx/impl/token.cpp
race:src/test/rpc/Book_test.cpp
race:src/xrpld/app/ledger/detail/InboundTransactions.cpp
race:src/xrpld/app/main/Application.cpp
race:src/xrpld/app/main/BasicApp.cpp
race:src/xrpld/app/main/GRPCServer.cpp
race:src/xrpld/app/misc/detail/AmendmentTable.cpp
race:src/xrpld/app/misc/FeeVoteImpl.cpp
race:src/xrpld/app/rdb/detail/Wallet.cpp
race:src/xrpld/overlay/detail/OverlayImpl.cpp
race:src/xrpld/peerfinder/detail/PeerfinderManager.cpp
race:src/xrpld/peerfinder/detail/SourceStrings.cpp
race:src/xrpld/rpc/detail/ServerHandler.cpp
race:xrpl/server/detail/Door.h
race:xrpl/server/detail/Spawn.h
race:xrpl/server/detail/ServerImpl.h
race:xrpl/nodestore/detail/DatabaseNodeImp.h
race:src/libxrpl/beast/utility/beast_Journal.cpp
race:src/test/beast/LexicalCast_test.cpp
race:ServerHandler
# More suppressions in external library code.
race:crtstuff.c
race:pipe
# Deadlock / lock-order-inversion suppressions
# Note: GCC's TSAN may not fully support all deadlock suppression patterns
deadlock:src/libxrpl/beast/utility/beast_Journal.cpp
deadlock:src/libxrpl/beast/utility/beast_PropertyStream.cpp
deadlock:src/test/beast/beast_PropertyStream_test.cpp
deadlock:src/xrpld/core/detail/Workers.cpp
deadlock:src/xrpld/app/misc/detail/Manifest.cpp
deadlock:src/xrpld/app/misc/detail/ValidatorList.cpp
deadlock:src/xrpld/app/misc/detail/ValidatorSite.cpp
signal:src/libxrpl/beast/utility/beast_Journal.cpp
signal:src/xrpld/core/detail/Workers.cpp
signal:src/xrpld/core/JobQueue.cpp
signal:Workers::Worker
# Aggressive suppressing of deadlock tsan errors
deadlock:pthread_create
deadlock:pthread_rwlock_rdlock
deadlock:boost::asio
# Suppress SEGV crashes in TSAN itself during stringbuf operations
# This appears to be a GCC-15 TSAN instrumentation issue with basic_stringbuf::str()
# Commonly triggered in beast::Journal::ScopedStream destructor
signal:std::__cxx11::basic_stringbuf
signal:basic_stringbuf
signal:basic_ostringstream
called_from_lib:libclang_rt
race:ostreambuf_iterator
race:basic_ostream
# Suppress SEGV in Boost ASIO memory allocation with GCC-15 TSAN
signal:boost::asio::aligned_new
signal:boost::asio::detail::memory
# Suppress SEGV in execute_native_thread_routine
signal:execute_native_thread_routine
# Suppress data race in Boost Context fiber management
# This is a false positive in Boost's exception state management during fiber context switching
race:__cxxabiv1::manage_exception_state
race:boost::context::fiber::resume
race:boost::asio::detail::spawned_fiber_thread
race:boost::asio::detail::spawned_fiber_thread::suspend_with
race:boost::asio::detail::spawned_fiber_thread::destroy
# Suppress data race in __tsan_memcpy called from Boost fiber operations
race:__cxxabiv1::manage_exception_state
race:__tsan_memcpy
# TSAN's own syscall interceptors (false positives from fd tracking)
race:crtstuff.c
race:pipe
race:epoll_ctl
race:epoll_create
race:closedir
race:close
race:socket
race:accept
race:eventfd
# C++ standard library internals
race:ostreambuf_iterator
race:basic_ostream
called_from_lib:libclang_rt
# Deadlock false positives in Boost and threading primitives
deadlock:pthread_create
deadlock:pthread_rwlock_rdlock
deadlock:boost::asio
# Known lock-order-inversion between getMasterMutex() and RCLConsensus::mutex_.
# NetworkOPsImp::processHeartbeatTimer acquires master → consensus, while
# RCLConsensus::startRound (via acceptLedger) acquires consensus → master.
# Requires a larger lock-ordering redesign to fix properly.
deadlock:RCLConsensus
# Lock-order-inversion in LedgerReplay subsystem.
# LedgerDeltaAcquire::notify() calls LedgerReplayTask::deltaReady() while
# holding LedgerDeltaAcquire's mutex, which then acquires LedgerReplayTask's
# mutex. The reverse order also occurs. Pre-existing design issue.
deadlock:LedgerReplayTask
deadlock:LedgerDeltaAcquire
# Data race on LedgerHeader::validated (bool mutable).
# This is a monotonic flag (false→true, never back) set by LedgerMaster
# worker threads and read by test/RPC code. Benign but technically UB.
# Proper fix: make validated std::atomic<bool>.
race:Ledger::setValidated
# Signal/crash suppressions for GCC TSAN instrumentation issues
signal:std::__cxx11::basic_stringbuf
signal:basic_stringbuf
signal:basic_ostringstream
signal:boost::asio::aligned_new
signal:boost::asio::detail::memory
signal:execute_native_thread_routine

View File

@@ -68,6 +68,11 @@ Journal::getNullSink()
//------------------------------------------------------------------------------
Journal::Sink::Sink(Sink const& sink)
: thresh_(sink.thresh_.load(std::memory_order_relaxed)), console_(sink.console_)
{
}
Journal::Sink::Sink(Severity thresh, bool console) : thresh_(thresh), console_(console)
{
}
@@ -77,7 +82,7 @@ Journal::Sink::~Sink() = default;
bool
Journal::Sink::active(Severity level) const
{
return level >= thresh_;
return level >= thresh_.load(std::memory_order_relaxed);
}
bool
@@ -95,13 +100,13 @@ Journal::Sink::console(bool output)
severities::Severity
Journal::Sink::threshold() const
{
return thresh_;
return thresh_.load(std::memory_order_relaxed);
}
void
Journal::Sink::threshold(Severity thresh)
{
thresh_ = thresh;
thresh_.store(thresh, std::memory_order_relaxed);
}
//------------------------------------------------------------------------------

View File

@@ -158,9 +158,18 @@ PropertyStream::Source::Source(std::string name) : name_(std::move(name)), item_
PropertyStream::Source::~Source()
{
std::scoped_lock const _(lock_);
if (parent_ != nullptr)
parent_->remove(*this);
// Read parent_ under lock, then release before calling remove()
// to maintain consistent lock ordering (parent before child),
// matching the order used in find_one_deep().
Source* parent = nullptr;
{
std::lock_guard const _(lock_);
parent = parent_;
}
if (parent != nullptr)
{
parent->remove(*this);
}
removeAll();
}
@@ -196,10 +205,12 @@ void
PropertyStream::Source::removeAll()
{
std::scoped_lock const _(lock_);
for (auto iter = children_.begin(); iter != children_.end();)
while (!children_.empty())
{
std::scoped_lock const cl((*iter)->lock_);
remove(*(*iter));
Source& child = children_.front().source();
std::scoped_lock const _cl(child.lock_);
children_.erase(children_.iteratorTo(child.item_));
child.parent_ = nullptr;
}
}

View File

@@ -40,7 +40,7 @@ Database::Database(
, scheduler_(scheduler)
, earliestLedgerSeq_(get<std::uint32_t>(config, "earliest_seq", kXRP_LEDGER_EARLIEST_SEQ))
, requestBundle_(get<int>(config, "rq_bundle", 4))
, readThreads_(std::max(1, readThreads))
, desiredReadThreads_(std::max(1, readThreads))
{
XRPL_ASSERT(readThreads, "xrpl::NodeStore::Database::Database : nonzero threads input");
@@ -49,8 +49,14 @@ Database::Database(
if (requestBundle_ < 1 || requestBundle_ > 64)
Throw<std::runtime_error>("Invalid rq_bundle");
}
for (int i = readThreads_.load(); i != 0; --i)
void
Database::startReadThreads()
{
readThreads_ = desiredReadThreads_;
for (int i = desiredReadThreads_; i != 0; --i)
{
std::thread t(
[this](int i) {

View File

@@ -90,8 +90,10 @@ ManagerImp::makeDatabase(
{
auto backend{makeBackend(config, burstSize, scheduler, journal)};
backend->open();
return std::make_unique<DatabaseNodeImp>(
auto db = std::make_unique<DatabaseNodeImp>(
scheduler, readThreads, std::move(backend), config, journal);
db->startReadThreads();
return db;
}
void

View File

@@ -38,6 +38,11 @@ private:
public:
ManagerImp(beast::insight::Collector::ptr const& collector, beast::Journal journal)
: journal_(journal), logic_(collector, stopwatch(), journal)
{
}
void
start() override
{
thread_ = std::thread{&ManagerImp::run, this};
}
@@ -49,12 +54,15 @@ public:
~ManagerImp() override
{
if (thread_.joinable())
{
std::scoped_lock const lock(mutex_);
stop_ = true;
cond_.notify_one();
{
std::scoped_lock const lock(mutex_);
stop_ = true;
cond_.notify_one();
}
thread_.join();
}
thread_.join();
}
Consumer

View File

@@ -77,7 +77,7 @@ SHAMap::SHAMap(SHAMap const& other, bool isMutable)
: f_(other.f_)
, journal_(other.f_.journal())
, cowid_(other.cowid_ + 1)
, ledgerSeq_(other.ledgerSeq_)
, ledgerSeq_(other.ledgerSeq_.load(std::memory_order_relaxed))
, root_(other.root_)
, state_(isMutable ? SHAMapState::Modifying : SHAMapState::Immutable)
, type_(other.type_)
@@ -169,7 +169,8 @@ intr_ptr::SharedPtr<SHAMapTreeNode>
SHAMap::fetchNodeFromDB(SHAMapHash const& hash) const
{
XRPL_ASSERT(backed_, "xrpl::SHAMap::fetchNodeFromDB : is backed");
auto obj = f_.db().fetchNodeObject(hash.asUint256(), ledgerSeq_);
auto obj =
f_.db().fetchNodeObject(hash.asUint256(), ledgerSeq_.load(std::memory_order_relaxed));
return finishFetch(hash, obj);
}
@@ -185,7 +186,8 @@ SHAMap::finishFetch(SHAMapHash const& hash, std::shared_ptr<NodeObject> const& o
if (full_)
{
full_ = false;
f_.missingNodeAcquireBySeq(ledgerSeq_, hash.asUint256());
f_.missingNodeAcquireBySeq(
ledgerSeq_.load(std::memory_order_relaxed), hash.asUint256());
}
return {};
}
@@ -218,7 +220,12 @@ SHAMap::checkFilter(SHAMapHash const& hash, SHAMapSyncFilter* filter) const
auto node = SHAMapTreeNode::makeFromPrefix(makeSlice(*nodeData), hash);
if (node)
{
filter->gotNode(true, hash, ledgerSeq_, std::move(*nodeData), node->getType());
filter->gotNode(
true,
hash,
ledgerSeq_.load(std::memory_order_relaxed),
std::move(*nodeData),
node->getType());
if (backed_)
canonicalize(hash, node);
}
@@ -399,7 +406,7 @@ SHAMap::descendAsync(
{
f_.db().asyncFetch(
hash.asUint256(),
ledgerSeq_,
ledgerSeq_.load(std::memory_order_relaxed),
[this, hash, cb{std::move(callback)}](std::shared_ptr<NodeObject> const& object) {
auto node = finishFetch(hash, object);
cb(node, hash);
@@ -947,7 +954,11 @@ SHAMap::writeNode(NodeObjectType t, intr_ptr::SharedPtr<SHAMapTreeNode> node) co
Serializer s;
node->serializeWithPrefix(s);
f_.db().store(t, std::move(s.modData()), node->getHash().asUint256(), ledgerSeq_);
f_.db().store(
t,
std::move(s.modData()),
node->getHash().asUint256(),
ledgerSeq_.load(std::memory_order_relaxed));
return node;
}

View File

@@ -537,7 +537,11 @@ SHAMap::addRootNode(SHAMapHash const& hash, Slice const& rootNode, SHAMapSyncFil
Serializer s;
root_->serializeWithPrefix(s);
filter->gotNode(
false, root_->getHash(), ledgerSeq_, std::move(s.modData()), root_->getType());
false,
root_->getHash(),
ledgerSeq_.load(std::memory_order_relaxed),
std::move(s.modData()),
root_->getType());
}
return SHAMapAddNode::useful();
@@ -643,7 +647,11 @@ SHAMap::addKnownNode(SHAMapNodeID const& node, Slice const& rawNode, SHAMapSyncF
Serializer s;
newNode->serializeWithPrefix(s);
filter->gotNode(
false, childHash, ledgerSeq_, std::move(s.modData()), newNode->getType());
false,
childHash,
ledgerSeq_.load(std::memory_order_relaxed),
std::move(s.modData()),
newNode->getType());
}
return SHAMapAddNode::useful();

View File

@@ -307,7 +307,7 @@ public:
std::unique_ptr<Config> config,
std::unique_ptr<Logs> logs,
std::unique_ptr<TimeKeeper> timeKeeper)
: BasicApp(numberOfThreads(*config))
: BasicApp(numberOfThreads(*config), DeferStart{})
, config_(std::move(config))
, logs_(std::move(logs))
, timeKeeper_(std::move(timeKeeper))
@@ -1145,6 +1145,14 @@ private:
bool
ApplicationImp::setup(boost::program_options::variables_map const& cmdline)
{
// NOTE: The main io_context threads are NOT started until start().
// However, subsystem-specific threads (resource manager, nodestore
// read threads) are started here because they use their own
// threading and do not contend with io_context setup work.
resourceManager_->start();
nodeStore_->startReadThreads();
// We want to intercept CTRL-C and the standard termination signal SIGTERM
// and terminate the process. This handler will NEVER be invoked twice.
//
@@ -1485,6 +1493,11 @@ ApplicationImp::start(bool withTimers)
{
JLOG(journal_.info()) << "Application starting. Version is " << BuildInfo::getVersionString();
// Start IO threads now that all setup is complete.
// This must happen before starting subsystems that post work
// to the io_context (resolver, overlay, etc.).
startIOThreads();
if (withTimers)
{
setSweepTimer();

View File

@@ -7,15 +7,26 @@
#include <cstddef>
#include <string>
BasicApp::BasicApp(std::size_t numberOfThreads)
BasicApp::BasicApp(std::size_t numberOfThreads) : numberOfThreads_(numberOfThreads)
{
work_.emplace(boost::asio::make_work_guard(io_context_));
threads_.reserve(numberOfThreads);
startIOThreads();
}
for (std::size_t i = 0; i < numberOfThreads; ++i)
BasicApp::BasicApp(std::size_t numberOfThreads, DeferStart) : numberOfThreads_(numberOfThreads)
{
work_.emplace(boost::asio::make_work_guard(io_context_));
}
void
BasicApp::startIOThreads()
{
threads_.reserve(numberOfThreads_);
for (std::size_t i = 0; i < numberOfThreads_; ++i)
{
threads_.emplace_back([this, i]() {
beast::setCurrentThreadName("io svc #" + std::to_string(i));
threads_.emplace_back([this, n = i]() {
beast::setCurrentThreadName("io svc #" + std::to_string(n));
this->io_context_.run();
});
}

View File

@@ -6,7 +6,12 @@
#include <thread>
#include <vector>
// This is so that the io_context can outlive all the children
/** Manages an io_context and its worker threads.
* Ensures the io_context outlives all derived classes by joining worker
* threads in the destructor. Supports immediate or deferred thread startup.
* @note Thread-safe after construction completes. The deferred-start
* constructor and startIOThreads() must be called from a single thread.
*/
class BasicApp
{
private:
@@ -14,6 +19,9 @@ private:
std::vector<std::thread> threads_;
boost::asio::io_context io_context_;
/// Number of IO worker threads to create.
std::size_t numberOfThreads_;
public:
BasicApp(std::size_t numberOfThreads);
~BasicApp();
@@ -29,4 +37,26 @@ public:
{
return threads_.size();
}
protected:
/** Tag type for deferred thread startup.
* Pass to the protected constructor to construct without starting IO
* threads. The derived class must call startIOThreads() once its own
* construction is complete.
*/
struct DeferStart
{
};
/** Construct without starting IO threads.
* @param numberOfThreads Desired number of IO worker threads.
* @param Tag to select deferred startup.
*/
BasicApp(std::size_t numberOfThreads, DeferStart);
/** Start the IO worker threads.
* @note Must be called exactly once after the deferred-start constructor.
*/
void
startIOThreads();
};

View File

@@ -15,7 +15,9 @@
#include <xrpl/beast/utility/instrumentation.h>
#include <xrpl/ledger/Ledger.h>
#include <xrpl/nodestore/Database.h>
#include <xrpl/nodestore/Manager.h>
#include <xrpl/nodestore/Scheduler.h>
#include <xrpl/nodestore/detail/DatabaseNodeImp.h>
#include <xrpl/nodestore/detail/DatabaseRotatingImp.h>
#include <xrpl/protocol/Protocol.h>
#include <xrpl/server/NetworkOPs.h>